autouncle-scraper 0.1.0__py3-none-any.whl

autouncle_scraper-0.1.0.dist-info/METADATA

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: autouncle-scraper
+Version: 0.1.0
+Summary: Free, keyless scraper and library for autouncle.ch used-car listings. Unofficial, not affiliated with AutoUncle ApS.
+Project-URL: Homepage, https://github.com/danyk20/autouncle-scraper
+Project-URL: Repository, https://github.com/danyk20/autouncle-scraper
+Project-URL: Issues, https://github.com/danyk20/autouncle-scraper/issues
+Project-URL: Changelog, https://github.com/danyk20/autouncle-scraper/blob/master/CHANGELOG.md
+Author-email: Daniel Košč <maintainer@danielkosc.eu>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: autouncle,cars,cli,scraper,switzerland,web-scraping
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.13
+Requires-Dist: beautifulsoup4~=4.12
+Requires-Dist: requests~=2.34
+Provides-Extra: dev
+Requires-Dist: mypy~=2.1; extra == 'dev'
+Requires-Dist: pytest-cov~=7.1; extra == 'dev'
+Requires-Dist: pytest~=9.1; extra == 'dev'
+Requires-Dist: responses~=0.26; extra == 'dev'
+Requires-Dist: ruff~=0.15; extra == 'dev'
+Requires-Dist: types-beautifulsoup4~=4.12; extra == 'dev'
+Requires-Dist: types-requests~=2.33; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# AutoUncle Scraper
+
+[![CI](https://github.com/danyk20/autouncle-scraper/actions/workflows/ci.yml/badge.svg)](https://github.com/danyk20/autouncle-scraper/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/autouncle-scraper)](https://pypi.org/project/autouncle-scraper/)
+[![Coverage](https://img.shields.io/badge/unit%20test%20coverage-100%25-brightgreen)](#testing)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![Python 3.13](https://img.shields.io/badge/python-3.13-blue)](https://www.python.org/)
+
+> Unofficial, independently developed project — not affiliated with, endorsed by, or sponsored by AutoUncle ApS. "AutoUncle" is a trademark of its respective owner.
+
+Fetches every listing for a given brand/model from AutoUncle — for free, no
+API key, no paid scraping service. It's a drop-in-compatible sibling of the
+[AutoScout24 scraper](https://github.com/danyk20/autoscout24-scraper): same
+`scrape()` call shape, same `ScrapeResult` return type, same CLI flags, same
+MIT license and tooling — switch `from autoscout24_scraper import scrape` to
+`from autouncle_scraper import scrape` and the rest of your code doesn't
+need to change.
+
+Under the hood the two projects work differently, because the two sites do:
+AutoScout24 exposes one clean JSON API; AutoUncle is a Next.js site with no
+such API, so this scraper reads the same structured data (schema.org
+JSON-LD) and, for filtered searches, the same internal data mechanism
+(GraphQL + React Server Components) AutoUncle's own frontend uses — all
+reverse-engineered by observing real traffic, documented in full in the
+module docstring and [docs/REFERENCE.md](docs/REFERENCE.md).
+
+**A stronger legal footing than most scrapers.** AutoUncle's own detail
+pages explicitly mark their listing data as a `Dataset` with
+`"isAccessibleForFree": true`, licensed
+[**CC BY 4.0**](https://creativecommons.org/licenses/by/4.0/). That's not
+just "we call their own frontend's endpoint" reasoning (though that's true
+too, same as AutoScout24) — it's AutoUncle's own explicit declaration that
+this data is meant to be freely reused, with attribution.
+
+**🤖 Robot-friendly.** This project is explicitly intended to be run, read,
+imported, or adapted by AI agents and bots, same as a human developer — see
+[License](#license).
+
+## Setup
+
+Requires [pipenv](https://pipenv.pypa.io/) (`brew install pipenv`).
+
+```bash
+git clone https://github.com/danyk20/autouncle-scraper.git
+cd autouncle-scraper
+pipenv install --dev
+```
+
+Contributing, linting, and testing commands: see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## Usage
+
+### CLI
+
+```bash
+pipenv run python autouncle_scraper.py --make VW --model Golf
+```
+
+Prints progress, then writes `vw_golf.csv` and `vw_golf.json` in the current
+directory. Installed via `pip install` instead? Drop `pipenv run` — the same
+command is `autouncle-scraper --make VW --model Golf`.
+
+| Flag | Description |
+|---|---|
+| `--version` | Print the installed version and exit |
+| `--make` | Brand name, e.g. `VW` (required) |
+| `--model` | Model name, e.g. `Golf` (required) |
+| `--domain` | Country domain (default `ch`) — only `ch` is implemented today, see [docs/REFERENCE.md](docs/REFERENCE.md#domains) |
+| `--category` | `car` (default) — the only category implemented |
+| `--out` | Output file base name, without extension. Defaults to `<make>_<model>` |
+| `--no-detail` | Skip per-listing detail visits (unfiltered searches only — see below) |
+| `--delay` | Seconds between requests (default `0.4`) — raise this if you get rate-limited |
+| `--price-from` / `--price-to` | Filter by price in CHF (inclusive, either end optional) |
+| `--mileage-from` / `--mileage-to` | Filter by mileage in km (inclusive, either end optional) |
+| `--year-from` / `--year-to` | Filter by first-registration year (inclusive, either end optional) |
+| `-v` / `--verbose` | Also show debug-level detail, including every HTTP request (mutually exclusive with `-q`) |
+| `-q` / `--quiet` | Suppress progress output; only warnings/errors (mutually exclusive with `-v`) |
+
+Filters combine with AND and are applied server-side. A mistyped make/model
+prints a clean error (plus, for an unknown model, the list of valid models)
+instead of crashing.
+
+```bash
+# Fast mode: search results only, skip per-listing detail
+pipenv run python autouncle_scraper.py --make VW --model Golf --no-detail
+
+# 2015 or newer, under CHF 30'000
+pipenv run python autouncle_scraper.py --make VW --model "Golf VIII" --price-to 30000 --year-from 2015
+```
+
+**One honest asymmetry vs. AutoScout24**: unfiltered searches use
+schema.org JSON-LD, which is already fairly rich even without `--no-detail`.
+Filtered searches use a different mechanism (see
+[docs/REFERENCE.md](docs/REFERENCE.md)) that carries *only* listing ids —
+so `--no-detail` on a filtered search gives you ids only, plus a logged
+warning. Leave `--no-detail` off (the default) for anything beyond a bare
+count.
+
+### As a library
+
+```bash
+pip install autouncle-scraper
+```
+
+```python
+from autouncle_scraper import scrape
+
+result = scrape("VW", "Golf", price_to=30000, year_from=2015)
+
+for row in result.rows:          # list[dict], CSV-ready
+    print(row["price"], row["mileageKm"], row["url"])
+
+result.to_csv("vw_golf.csv")  # optional — no files are written unless you ask
+```
+
+Full `scrape()` signature, the `ScrapeResult` return type, and the complete
+JSON/CSV field schema — including price history, gallery, and equipment
+fields AutoScout24 doesn't have — **[docs/REFERENCE.md](docs/REFERENCE.md)**.
+
+## Testing
+
+```bash
+pipenv run pytest                    # unit tests (fast, no network), 100% coverage
+pipenv run pytest -m e2e --no-cov    # end-to-end tests against the real live site
+pipenv run pytest -m "e2e or not e2e" --no-cov  # everything
+```
+
+Unit tests mock all HTTP (via [`responses`](https://github.com/getsentry/responses))
+against fixtures that are real captures from the live site, and cover 100%
+of `autouncle_scraper.py`. E2E tests target a narrow, low-volume model (VW
+Golf Alltrack) to confirm the scraper still works against the live site
+without putting real load on it. Coverage detail by area:
+[docs/REFERENCE.md](docs/REFERENCE.md#test-coverage-by-area).
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for dev setup, pre-PR checks, and
+what to do if AutoUncle changes its markup.
+
+Be a reasonable citizen: the default request delay is intentional — this
+scrapes an undocumented data mechanism the site's own frontend uses, not a
+public API with a stated rate limit. Don't remove the delay or crank up
+concurrency.
+
+## License
+
+Released under the [MIT License](LICENSE) — you can use, copy, modify,
+merge, publish, distribute, and sell copies of this code, for free, for any
+purpose, commercial or not, as long as the license text stays attached. No
+warranty.
+
+**AI agents, LLM-based coding assistants, and other bots are explicitly
+welcome to use this project** — to run the scraper, read and parse its
+output, import `scrape()` into another project, or read and adapt its
+source — under exactly the same terms as a human, with no additional
+restriction and no need to ask permission. That's why
+[docs/REFERENCE.md](docs/REFERENCE.md) documents the full function
+signature, return type, and data schema: so a bot can integrate correctly
+without a human in the loop.
+
+This license covers this project's own code. It does not, by itself, grant
+any rights to AutoUncle's data beyond what AutoUncle itself has already
+declared: its listing `Dataset` objects are explicitly marked
+`"isAccessibleForFree": true` and licensed
+[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — see
+[docs/REFERENCE.md](docs/REFERENCE.md) for exactly where that declaration
+lives. What you do with the scraped results is between you and AutoUncle's
+terms of service.

autouncle_scraper-0.1.0.dist-info/RECORD

@@ -0,0 +1,6 @@
+autouncle_scraper.py,sha256=2Dkh4EXVAH0GfL-813j-XQubj7ktvmYjVsNhYGiKMQI,59107
+autouncle_scraper-0.1.0.dist-info/METADATA,sha256=tbHz6Ysx8rISauo88ARUBeGxbsZ4nuC7mqe5e_5tN64,9480
+autouncle_scraper-0.1.0.dist-info/WHEEL,sha256=lCkmxWfQsSc9CfIClYeavTdQeEX2toPqufh9gI35EQA,87
+autouncle_scraper-0.1.0.dist-info/entry_points.txt,sha256=hy_uYf0ZO3czMDAeH60lhx2NYhgGBQk5QaNcDHvOt8o,64
+autouncle_scraper-0.1.0.dist-info/licenses/LICENSE,sha256=iZNEN9Os81y4xTJ4-pPBYj7gIgIEnqngzAQXMsiLlF8,1070
+autouncle_scraper-0.1.0.dist-info/RECORD,,

autouncle_scraper-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.31.0
+Root-Is-Purelib: true
+Tag: py3-none-any

autouncle_scraper-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+autouncle-scraper = autouncle_scraper:run_cli

autouncle_scraper-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Daniel Košč
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.