domain-com-au-scraper 0.1.0__tar.gz

domain_com_au_scraper-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Buğrahan Saka
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

domain_com_au_scraper-0.1.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md LICENSE
+recursive-include tests *.py *.html
+recursive-include examples *.py

domain_com_au_scraper-0.1.0/PKG-INFO

@@ -0,0 +1,94 @@
+Metadata-Version: 2.4
+Name: domain-com-au-scraper
+Version: 0.1.0
+Summary: Scrape Domain.com.au property listings for sale, for rent and sold (price, beds, address, agency, suburb medians) with Python.
+Author: Buğrahan Saka
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/bugrahansss/domain-com-au-scraper
+Project-URL: Source, https://github.com/bugrahansss/domain-com-au-scraper
+Project-URL: Issues, https://github.com/bugrahansss/domain-com-au-scraper/issues
+Keywords: domain.com.au,domain,real estate,property,australia,web scraping,scraper,scrapedo
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25
+Requires-Dist: beautifulsoup4>=4.9
+Dynamic: license-file
+
+# Domain.com.au Scraper
+
+[![PyPI](https://img.shields.io/pypi/v/domain-com-au-scraper)](https://pypi.org/project/domain-com-au-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/domain-com-au-scraper)](https://pypi.org/project/domain-com-au-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+Scrape [Domain.com.au](https://www.domain.com.au), one of Australia's two big property portals, into structured data. Search any suburb **for sale, for rent or sold** and get **price text and parsed price range, sale status and date, beds, baths, parking, land size, property type, address with coordinates, agency and agents, auction and inspection times**. Open a listing for its **description, features, agents' phone numbers, listing dates and Domain's suburb medians** (median sale and rent price, days on market, auction clearance). JSON or CSV.
+
+```bash
+pip install domain-com-au-scraper
+export SCRAPEDO_TOKEN=your_token
+domain-scraper "Bondi Beach NSW 2026" --mode sold --pages 3 -f csv -o bondi_sold.csv
+domain-scraper "Sydney NSW 2000" --details --limit 20 -o sydney.json
+```
+
+## Why plain requests fail
+
+```python
+import requests
+
+r = requests.get("https://www.domain.com.au/sale/sydney-nsw-2000/")
+print(r.status_code)   # 403
+print(r.text[:60])     # <HTML><HEAD><TITLE>Access Denied</TITLE>
+```
+
+Domain sits behind Akamai, which answers a script with `403 Access Denied` before any HTML. This package sends each request through [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=domain-com-au-scraper) with `super=true` (residential proxies), which returns the real page. Each page costs 10 credits.
+
+## Where the data is
+
+Domain is a Next.js site, so every page carries its data in the `__NEXT_DATA__` script. No HTML parsing:
+
+* **Search pages** (`/sale/<suburb>-<state>-<postcode>/`, `/rent/...`, `/sold-listings/...`, `?page=N`): `props.pageProps.componentProps.listingsMap` holds about 20 listings per page, plus `totalListings` and `totalPages`. New-development "projects" sit in the same map and are skipped. Domain stops at page 50.
+* **Listing pages** (`/<address-slug>-<listingId>`): the same object holds the description, structured features, the price guide with agents, `createdOn`/`modifiedOn`, the gallery and `suburbInsights`.
+
+In a live test: Sydney NSW 2000 had 934 listings for sale over 46 pages; Bondi Beach had 16,626 sold listings, 20 per page, each with its sale status and date ("Sold at auction 10 Nov 2026"). Many prices are text ("Price Withheld", "Contact Agent"); `price_min`/`price_max` are filled only when the text has a number.
+
+## Python
+
+```python
+from domain_scraper import DomainScraper
+
+dm = DomainScraper(token="YOUR_SCRAPEDO_TOKEN")   # or set SCRAPEDO_TOKEN
+
+for l in dm.search("Bondi Beach NSW 2026", mode="sold", pages=2):
+    print(l.status, l.price_text, l.beds, l.address, l.agency)
+
+detail = dm.get_listing("https://www.domain.com.au/<address-slug>-<listingId>")
+print(detail.description[:200], detail.features, detail.suburb_median_price)
+print(dm.credits_used)
+```
+
+## Fields
+
+**Search result:** `listing_id`, `url`, `price_text`, `price_min`, `price_max`, `status`, `property_type`, `beds`, `baths`, `parking`, `land_size`, `land_unit`, `street`, `suburb`, `state`, `postcode`, `latitude`, `longitude`, `agency`, `agency_id`, `agents`, `auction`, `inspection_open`, `inspection_close`, `images`, `promo_type`, `address`.
+
+**Listing page:** `listing_id`, `url`, `title` (price text), `headline`, `listing_type`, `status`, `method`, `property_type`, `beds`, `baths`, `parking`, `land_area`, `address`, `unit_number`, `street_number`, `street`, `suburb`, `state`, `postcode`, `latitude`, `longitude`, `description`, `features`, `agency`, `agents` (name, phone, mobile, profile URL), `inspection`, `created_on`, `modified_on`, `images`, `suburb_median_price`, `suburb_median_rent`, `suburb_days_on_market`, `suburb_auction_clearance`, `suburb_sold_this_year`.
+
+## Use it responsibly
+
+Listings are public, but agents are real people: use their contact details only for legitimate purposes, respect the Australian Privacy Act and Spam Act, and Domain's terms. Don't republish listing photos or descriptions. This is not legal advice.
+
+## Tests
+
+```bash
+PYTHONPATH=src python -m unittest discover -s tests
+```
+
+The fixtures are fictional ("Example Realty", "Jane Doe", example.com).
+
+## License
+
+MIT

domain_com_au_scraper-0.1.0/README.md

@@ -0,0 +1,72 @@
+# Domain.com.au Scraper
+
+[![PyPI](https://img.shields.io/pypi/v/domain-com-au-scraper)](https://pypi.org/project/domain-com-au-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/domain-com-au-scraper)](https://pypi.org/project/domain-com-au-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+Scrape [Domain.com.au](https://www.domain.com.au), one of Australia's two big property portals, into structured data. Search any suburb **for sale, for rent or sold** and get **price text and parsed price range, sale status and date, beds, baths, parking, land size, property type, address with coordinates, agency and agents, auction and inspection times**. Open a listing for its **description, features, agents' phone numbers, listing dates and Domain's suburb medians** (median sale and rent price, days on market, auction clearance). JSON or CSV.
+
+```bash
+pip install domain-com-au-scraper
+export SCRAPEDO_TOKEN=your_token
+domain-scraper "Bondi Beach NSW 2026" --mode sold --pages 3 -f csv -o bondi_sold.csv
+domain-scraper "Sydney NSW 2000" --details --limit 20 -o sydney.json
+```
+
+## Why plain requests fail
+
+```python
+import requests
+
+r = requests.get("https://www.domain.com.au/sale/sydney-nsw-2000/")
+print(r.status_code)   # 403
+print(r.text[:60])     # <HTML><HEAD><TITLE>Access Denied</TITLE>
+```
+
+Domain sits behind Akamai, which answers a script with `403 Access Denied` before any HTML. This package sends each request through [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=domain-com-au-scraper) with `super=true` (residential proxies), which returns the real page. Each page costs 10 credits.
+
+## Where the data is
+
+Domain is a Next.js site, so every page carries its data in the `__NEXT_DATA__` script. No HTML parsing:
+
+* **Search pages** (`/sale/<suburb>-<state>-<postcode>/`, `/rent/...`, `/sold-listings/...`, `?page=N`): `props.pageProps.componentProps.listingsMap` holds about 20 listings per page, plus `totalListings` and `totalPages`. New-development "projects" sit in the same map and are skipped. Domain stops at page 50.
+* **Listing pages** (`/<address-slug>-<listingId>`): the same object holds the description, structured features, the price guide with agents, `createdOn`/`modifiedOn`, the gallery and `suburbInsights`.
+
+In a live test: Sydney NSW 2000 had 934 listings for sale over 46 pages; Bondi Beach had 16,626 sold listings, 20 per page, each with its sale status and date ("Sold at auction 10 Nov 2026"). Many prices are text ("Price Withheld", "Contact Agent"); `price_min`/`price_max` are filled only when the text has a number.
+
+## Python
+
+```python
+from domain_scraper import DomainScraper
+
+dm = DomainScraper(token="YOUR_SCRAPEDO_TOKEN")   # or set SCRAPEDO_TOKEN
+
+for l in dm.search("Bondi Beach NSW 2026", mode="sold", pages=2):
+    print(l.status, l.price_text, l.beds, l.address, l.agency)
+
+detail = dm.get_listing("https://www.domain.com.au/<address-slug>-<listingId>")
+print(detail.description[:200], detail.features, detail.suburb_median_price)
+print(dm.credits_used)
+```
+
+## Fields
+
+**Search result:** `listing_id`, `url`, `price_text`, `price_min`, `price_max`, `status`, `property_type`, `beds`, `baths`, `parking`, `land_size`, `land_unit`, `street`, `suburb`, `state`, `postcode`, `latitude`, `longitude`, `agency`, `agency_id`, `agents`, `auction`, `inspection_open`, `inspection_close`, `images`, `promo_type`, `address`.
+
+**Listing page:** `listing_id`, `url`, `title` (price text), `headline`, `listing_type`, `status`, `method`, `property_type`, `beds`, `baths`, `parking`, `land_area`, `address`, `unit_number`, `street_number`, `street`, `suburb`, `state`, `postcode`, `latitude`, `longitude`, `description`, `features`, `agency`, `agents` (name, phone, mobile, profile URL), `inspection`, `created_on`, `modified_on`, `images`, `suburb_median_price`, `suburb_median_rent`, `suburb_days_on_market`, `suburb_auction_clearance`, `suburb_sold_this_year`.
+
+## Use it responsibly
+
+Listings are public, but agents are real people: use their contact details only for legitimate purposes, respect the Australian Privacy Act and Spam Act, and Domain's terms. Don't republish listing photos or descriptions. This is not legal advice.
+
+## Tests
+
+```bash
+PYTHONPATH=src python -m unittest discover -s tests
+```
+
+The fixtures are fictional ("Example Realty", "Jane Doe", example.com).
+
+## License
+
+MIT

domain_com_au_scraper-0.1.0/examples/sold_prices.py

@@ -0,0 +1,21 @@
+"""Recent sales in a suburb to CSV: price, sale type and date, beds, address.
+
+    export SCRAPEDO_TOKEN=your_token
+    python examples/sold_prices.py "Bondi Beach NSW 2026" 3
+"""
+
+import csv
+import sys
+
+from domain_scraper import DomainScraper
+
+suburb = sys.argv[1] if len(sys.argv) > 1 else "Bondi Beach NSW 2026"
+pages = int(sys.argv[2]) if len(sys.argv) > 2 else 2
+
+dm = DomainScraper()
+with open("sold.csv", "w", newline="", encoding="utf-8") as f:
+    writer = csv.writer(f)
+    writer.writerow(["status", "price_text", "price_min", "beds", "baths", "property_type", "address", "agency", "url"])
+    for l in dm.search(suburb, mode="sold", pages=pages):
+        writer.writerow([l.status, l.price_text, l.price_min, l.beds, l.baths, l.property_type, l.address, l.agency, l.url])
+print(f"done, {dm.credits_used} credits")

domain_com_au_scraper-0.1.0/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "domain-com-au-scraper"
+version = "0.1.0"
+description = "Scrape Domain.com.au property listings for sale, for rent and sold (price, beds, address, agency, suburb medians) with Python."
+readme = "README.md"
+requires-python = ">=3.8"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Buğrahan Saka" }]
+keywords = ["domain.com.au", "domain", "real estate", "property", "australia", "web scraping", "scraper", "scrapedo"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Topic :: Internet :: WWW/HTTP :: Indexing/Search",
+]
+dependencies = ["requests>=2.25", "beautifulsoup4>=4.9"]
+
+[project.urls]
+Homepage = "https://github.com/bugrahansss/domain-com-au-scraper"
+Source = "https://github.com/bugrahansss/domain-com-au-scraper"
+Issues = "https://github.com/bugrahansss/domain-com-au-scraper/issues"
+
+[project.scripts]
+domain-scraper = "domain_scraper.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

domain_com_au_scraper-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

domain_com_au_scraper-0.1.0/src/domain_com_au_scraper.egg-info/PKG-INFO

@@ -0,0 +1,94 @@
+Metadata-Version: 2.4
+Name: domain-com-au-scraper
+Version: 0.1.0
+Summary: Scrape Domain.com.au property listings for sale, for rent and sold (price, beds, address, agency, suburb medians) with Python.
+Author: Buğrahan Saka
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/bugrahansss/domain-com-au-scraper
+Project-URL: Source, https://github.com/bugrahansss/domain-com-au-scraper
+Project-URL: Issues, https://github.com/bugrahansss/domain-com-au-scraper/issues
+Keywords: domain.com.au,domain,real estate,property,australia,web scraping,scraper,scrapedo
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25
+Requires-Dist: beautifulsoup4>=4.9
+Dynamic: license-file
+
+# Domain.com.au Scraper
+
+[![PyPI](https://img.shields.io/pypi/v/domain-com-au-scraper)](https://pypi.org/project/domain-com-au-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/domain-com-au-scraper)](https://pypi.org/project/domain-com-au-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+Scrape [Domain.com.au](https://www.domain.com.au), one of Australia's two big property portals, into structured data. Search any suburb **for sale, for rent or sold** and get **price text and parsed price range, sale status and date, beds, baths, parking, land size, property type, address with coordinates, agency and agents, auction and inspection times**. Open a listing for its **description, features, agents' phone numbers, listing dates and Domain's suburb medians** (median sale and rent price, days on market, auction clearance). JSON or CSV.
+
+```bash
+pip install domain-com-au-scraper
+export SCRAPEDO_TOKEN=your_token
+domain-scraper "Bondi Beach NSW 2026" --mode sold --pages 3 -f csv -o bondi_sold.csv
+domain-scraper "Sydney NSW 2000" --details --limit 20 -o sydney.json
+```
+
+## Why plain requests fail
+
+```python
+import requests
+
+r = requests.get("https://www.domain.com.au/sale/sydney-nsw-2000/")
+print(r.status_code)   # 403
+print(r.text[:60])     # <HTML><HEAD><TITLE>Access Denied</TITLE>
+```
+
+Domain sits behind Akamai, which answers a script with `403 Access Denied` before any HTML. This package sends each request through [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=domain-com-au-scraper) with `super=true` (residential proxies), which returns the real page. Each page costs 10 credits.
+
+## Where the data is
+
+Domain is a Next.js site, so every page carries its data in the `__NEXT_DATA__` script. No HTML parsing:
+
+* **Search pages** (`/sale/<suburb>-<state>-<postcode>/`, `/rent/...`, `/sold-listings/...`, `?page=N`): `props.pageProps.componentProps.listingsMap` holds about 20 listings per page, plus `totalListings` and `totalPages`. New-development "projects" sit in the same map and are skipped. Domain stops at page 50.
+* **Listing pages** (`/<address-slug>-<listingId>`): the same object holds the description, structured features, the price guide with agents, `createdOn`/`modifiedOn`, the gallery and `suburbInsights`.
+
+In a live test: Sydney NSW 2000 had 934 listings for sale over 46 pages; Bondi Beach had 16,626 sold listings, 20 per page, each with its sale status and date ("Sold at auction 10 Nov 2026"). Many prices are text ("Price Withheld", "Contact Agent"); `price_min`/`price_max` are filled only when the text has a number.
+
+## Python
+
+```python
+from domain_scraper import DomainScraper
+
+dm = DomainScraper(token="YOUR_SCRAPEDO_TOKEN")   # or set SCRAPEDO_TOKEN
+
+for l in dm.search("Bondi Beach NSW 2026", mode="sold", pages=2):
+    print(l.status, l.price_text, l.beds, l.address, l.agency)
+
+detail = dm.get_listing("https://www.domain.com.au/<address-slug>-<listingId>")
+print(detail.description[:200], detail.features, detail.suburb_median_price)
+print(dm.credits_used)
+```
+
+## Fields
+
+**Search result:** `listing_id`, `url`, `price_text`, `price_min`, `price_max`, `status`, `property_type`, `beds`, `baths`, `parking`, `land_size`, `land_unit`, `street`, `suburb`, `state`, `postcode`, `latitude`, `longitude`, `agency`, `agency_id`, `agents`, `auction`, `inspection_open`, `inspection_close`, `images`, `promo_type`, `address`.
+
+**Listing page:** `listing_id`, `url`, `title` (price text), `headline`, `listing_type`, `status`, `method`, `property_type`, `beds`, `baths`, `parking`, `land_area`, `address`, `unit_number`, `street_number`, `street`, `suburb`, `state`, `postcode`, `latitude`, `longitude`, `description`, `features`, `agency`, `agents` (name, phone, mobile, profile URL), `inspection`, `created_on`, `modified_on`, `images`, `suburb_median_price`, `suburb_median_rent`, `suburb_days_on_market`, `suburb_auction_clearance`, `suburb_sold_this_year`.
+
+## Use it responsibly
+
+Listings are public, but agents are real people: use their contact details only for legitimate purposes, respect the Australian Privacy Act and Spam Act, and Domain's terms. Don't republish listing photos or descriptions. This is not legal advice.
+
+## Tests
+
+```bash
+PYTHONPATH=src python -m unittest discover -s tests
+```
+
+The fixtures are fictional ("Example Realty", "Jane Doe", example.com).
+
+## License
+
+MIT

domain_com_au_scraper-0.1.0/src/domain_com_au_scraper.egg-info/SOURCES.txt

@@ -0,0 +1,19 @@
+LICENSE
+MANIFEST.in
+README.md
+pyproject.toml
+examples/sold_prices.py
+src/domain_com_au_scraper.egg-info/PKG-INFO
+src/domain_com_au_scraper.egg-info/SOURCES.txt
+src/domain_com_au_scraper.egg-info/dependency_links.txt
+src/domain_com_au_scraper.egg-info/entry_points.txt
+src/domain_com_au_scraper.egg-info/requires.txt
+src/domain_com_au_scraper.egg-info/top_level.txt
+src/domain_scraper/__init__.py
+src/domain_scraper/cli.py
+src/domain_scraper/client.py
+src/domain_scraper/parser.py
+tests/test_domain.py
+tests/fixtures/blocked.html
+tests/fixtures/listing.html
+tests/fixtures/search_bondi.html

domain_com_au_scraper-0.1.0/src/domain_com_au_scraper.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

domain_com_au_scraper-0.1.0/src/domain_com_au_scraper.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+domain-scraper = domain_scraper.cli:main

domain_com_au_scraper-0.1.0/src/domain_com_au_scraper.egg-info/requires.txt

@@ -0,0 +1,2 @@
+requests>=2.25
+beautifulsoup4>=4.9

domain_com_au_scraper-0.1.0/src/domain_com_au_scraper.egg-info/top_level.txt

@@ -0,0 +1 @@
+domain_scraper

domain_com_au_scraper-0.1.0/src/domain_scraper/__init__.py

@@ -0,0 +1,23 @@
+"""Scrape Domain.com.au listings (for sale, for rent, sold) with Python."""
+
+from .client import DomainScraper, ScrapeError
+from .parser import (
+    DETAIL_FIELDS,
+    LISTING_FIELDS,
+    Agent,
+    Listing,
+    ListingDetail,
+    SearchPage,
+    listing_url,
+    next_data,
+    parse_listing,
+    parse_price,
+    parse_search,
+    search_url,
+    suburb_slug,
+)
+
+__all__ = ["DETAIL_FIELDS", "LISTING_FIELDS", "Agent", "DomainScraper", "Listing", "ListingDetail", "ScrapeError",
+           "SearchPage", "listing_url", "next_data", "parse_listing", "parse_price", "parse_search", "search_url",
+           "suburb_slug"]
+__version__ = "0.1.0"

domain_com_au_scraper-0.1.0/src/domain_scraper/cli.py

@@ -0,0 +1,80 @@
+"""Command line entry point.
+
+    domain-scraper "Bondi Beach NSW 2026" --mode sold --pages 3 -f csv -o bondi_sold.csv
+    domain-scraper "Sydney NSW 2000" --details --limit 20 -o sydney.json
+    domain-scraper https://www.domain.com.au/2702-91-liverpool-street-sydney-nsw-2000-2020997448
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import sys
+from typing import Any, List, Optional
+
+from . import __version__
+from .client import TOKEN_ENV, DomainScraper, ScrapeError
+from .parser import DETAIL_FIELDS, LISTING_FIELDS, is_listing_url
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(prog="domain-scraper", description="Scrape Domain.com.au listings: for sale, for rent and sold.")
+    p.add_argument("targets", nargs="+", help='Suburbs ("Bondi Beach NSW 2026" or bondi-beach-nsw-2026) or listing URLs')
+    p.add_argument("--mode", choices=("sale", "rent", "sold"), default="sale", help="Search type (default: sale)")
+    p.add_argument("--pages", type=int, default=1, help="Result pages per suburb, about 20 listings each (max 50)")
+    p.add_argument("--details", action="store_true", help="Open every listing page (10 credits each)")
+    p.add_argument("--limit", type=int, default=0, help="Stop after this many listings")
+    p.add_argument("-t", "--token", help=f"Scrape.do API token (default: ${TOKEN_ENV})")
+    p.add_argument("-f", "--format", choices=("json", "csv"), default="json")
+    p.add_argument("-o", "--output", help="Write to this file instead of stdout")
+    p.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
+    return p
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    args = build_parser().parse_args(argv)
+    try:
+        dm = DomainScraper(token=args.token)
+    except ValueError as exc:
+        print(exc, file=sys.stderr)
+        return 2
+    items: List[Any] = []
+    failures = 0
+    for target in args.targets:
+        if args.limit and len(items) >= args.limit:
+            break
+        try:
+            if is_listing_url(target):
+                items.append(dm.get_listing(target))
+                continue
+            found = 0
+            for listing in dm.search(target, args.mode, args.pages):
+                if args.limit and len(items) >= args.limit:
+                    break
+                items.append(dm.get_listing(listing.url) if args.details else listing)
+                found += 1
+            print(f"{target}: {found} listings", file=sys.stderr)
+        except (ScrapeError, ValueError) as exc:
+            failures += 1
+            print(f"fail  {target}: {exc}", file=sys.stderr)
+
+    stream = open(args.output, "w", encoding="utf-8", newline="") if args.output else sys.stdout
+    try:
+        if args.format == "csv":
+            fields = DETAIL_FIELDS if items and hasattr(items[0], "headline") else LISTING_FIELDS + ["address"]
+            writer = csv.DictWriter(stream, fieldnames=fields, extrasaction="ignore")
+            writer.writeheader()
+            writer.writerows(i.to_row() for i in items)
+        else:
+            json.dump([i.to_dict() for i in items], stream, indent=2, ensure_ascii=False)
+            stream.write("\n")
+    finally:
+        if args.output:
+            stream.close()
+            print(f"Saved {len(items)} items to {args.output} ({dm.credits_used} credits)", file=sys.stderr)
+    return 1 if failures and not items else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

domain_com_au_scraper-0.1.0/src/domain_scraper/client.py

@@ -0,0 +1,106 @@
+"""Fetch Domain.com.au pages through the Scrape.do API.
+
+Domain sits behind Akamai: a plain ``requests.get`` from a script gets
+``403 Access Denied`` before any HTML. Sending the request through Scrape.do
+with ``super=true`` (residential proxies) returns the real page with its
+``__NEXT_DATA__``. Each page costs 10 Scrape.do credits.
+"""
+
+from __future__ import annotations
+
+import os
+import time
+from typing import Iterator, List, Optional
+
+import requests
+
+from .parser import (
+    ListingDetail,
+    Listing,
+    listing_url,
+    parse_listing,
+    parse_search,
+    search_url,
+)
+
+API_URL = "https://api.scrape.do/"
+TOKEN_ENV = "SCRAPEDO_TOKEN"
+MAX_PAGES = 50  # Domain stops at page 50
+
+
+class ScrapeError(RuntimeError):
+    def __init__(self, message: str, status_code: Optional[int] = None):
+        super().__init__(message)
+        self.status_code = status_code
+
+
+class DomainScraper:
+    """Search Domain.com.au by suburb (for sale, for rent, sold) and read listing pages.
+
+    >>> dm = DomainScraper(token="YOUR_SCRAPEDO_TOKEN")
+    >>> for listing in dm.search("Bondi Beach NSW 2026", mode="sold", pages=2):
+    ...     print(listing.price_text, listing.status, listing.address)
+    >>> detail = dm.get_listing(listing.url)
+    """
+
+    def __init__(self, token: Optional[str] = None, timeout: float = 90.0, retries: int = 2,
+                 geo_code: str = "", session: Optional[requests.Session] = None):
+        self.token = token or os.environ.get(TOKEN_ENV, "")
+        if not self.token:
+            raise ValueError(f"A Scrape.do API token is required. Pass token=... or set {TOKEN_ENV}.")
+        self.timeout = timeout
+        self.retries = max(0, retries)
+        self.geo_code = geo_code
+        self.session = session or requests.Session()
+        self.requests_made = 0
+
+    def fetch(self, url: str) -> str:
+        params = {"token": self.token, "url": url, "super": "true"}
+        if self.geo_code:
+            params["geoCode"] = self.geo_code
+        last: Optional[ScrapeError] = None
+        for attempt in range(self.retries + 1):
+            try:
+                response = self.session.get(API_URL, params=params, timeout=self.timeout)
+            except requests.RequestException as exc:
+                last = ScrapeError(f"Request failed: {exc}")
+            else:
+                if response.status_code == 200 and "__NEXT_DATA__" in response.text:
+                    self.requests_made += 1
+                    return response.text
+                if response.status_code == 200:
+                    self.requests_made += 1
+                    last = ScrapeError(f"Blocked page for {url} (no __NEXT_DATA__)", 200)
+                else:
+                    last = ScrapeError(f"Request for {url} failed with status {response.status_code}",
+                                       response.status_code)
+                    if response.status_code in (400, 401, 404, 410):
+                        break
+            if attempt < self.retries:
+                time.sleep(2 * (attempt + 1))
+        assert last is not None
+        raise last
+
+    def search(self, location: str, mode: str = "sale", pages: int = 1) -> Iterator[Listing]:
+        """Listings for a suburb (``"Bondi Beach NSW 2026"``), ``mode`` = ``sale``, ``rent`` or ``sold``."""
+        seen = set()
+        last_page = None
+        for page in range(1, min(max(1, pages), MAX_PAGES) + 1):
+            if last_page is not None and page > last_page:
+                break
+            result = parse_search(self.fetch(search_url(location, mode, page)))
+            last_page = result.total_pages or page
+            fresh = [l for l in result.listings if l.listing_id not in seen]
+            if not fresh:
+                break
+            for listing in fresh:
+                seen.add(listing.listing_id)
+                yield listing
+
+    def get_listing(self, url: str) -> ListingDetail:
+        url = listing_url(url)
+        return parse_listing(self.fetch(url), url=url)
+
+    @property
+    def credits_used(self) -> int:
+        return self.requests_made * 10

domain_com_au_scraper-0.1.0/src/domain_scraper/parser.py

@@ -0,0 +1,333 @@
+"""Parse Domain.com.au pages.
+
+Domain is a Next.js site: every search page and every listing page carries its
+data in the ``__NEXT_DATA__`` script, so there is no HTML to pick apart.
+
+* A **search page** (``/sale/<suburb-state-postcode>/``, ``/rent/...``,
+  ``/sold-listings/...``) has ``componentProps.listingsMap``: about 20 listings
+  per page with price text, beds, baths, parking, land size, property type,
+  address with latitude/longitude, sale tags ("Sold by private treaty 26 Oct
+  2026"), auction and inspection times, agency and agents. ``totalListings``
+  and ``totalPages`` give the size of the result (Domain stops at 50 pages).
+  Projects (new developments) sit in the same map and are skipped.
+* A **listing page** (``/<address-slug>-<listingId>``) adds the description,
+  structured features, the price guide with agents' phone numbers, the dates
+  the listing was created and modified, the gallery, and Domain's suburb
+  insights (median sale and rent prices, days on market, auction clearance).
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from dataclasses import asdict, dataclass, field
+from typing import Any, Dict, List, Optional, Tuple
+
+BASE_URL = "https://www.domain.com.au"
+NEXT_RE = re.compile(r'<script id="__NEXT_DATA__"[^>]*>(.*?)</script>', re.S)
+LISTING_URL_RE = re.compile(r"^(?:https?://(?:www\.)?domain\.com\.au)?/([a-z0-9-]+-(\d{6,}))/?(?:[?#].*)?$")
+SLUG_RE = re.compile(r"^[a-z0-9-]+-(?:nsw|vic|qld|wa|sa|tas|act|nt)-\d{4}$")
+MODES = {"sale": "sale", "buy": "sale", "rent": "rent", "sold": "sold-listings", "sold-listings": "sold-listings"}
+
+
+def next_data(html: str) -> Dict[str, Any]:
+    """The ``__NEXT_DATA__`` JSON of a Domain page."""
+    match = NEXT_RE.search(html or "")
+    if not match:
+        raise ValueError("No __NEXT_DATA__ on this page (blocked or not a Domain page)")
+    return json.loads(match.group(1))
+
+
+def _props(html: str) -> Dict[str, Any]:
+    return next_data(html).get("props", {}).get("pageProps", {}).get("componentProps", {}) or {}
+
+
+def suburb_slug(location: str) -> str:
+    """``Bondi Beach NSW 2026`` or ``bondi-beach-nsw-2026`` -> ``bondi-beach-nsw-2026``."""
+    slug = re.sub(r"[^a-z0-9]+", "-", (location or "").strip().lower()).strip("-")
+    if not SLUG_RE.match(slug):
+        raise ValueError(f"Use a suburb with state and postcode, e.g. 'Bondi Beach NSW 2026' (got {location!r})")
+    return slug
+
+
+def search_url(location: str, mode: str = "sale", page: int = 1) -> str:
+    """Search URL for a suburb: ``mode`` is ``sale``, ``rent`` or ``sold``."""
+    path = MODES.get((mode or "sale").lower())
+    if not path:
+        raise ValueError("mode must be 'sale', 'rent' or 'sold'")
+    url = f"{BASE_URL}/{path}/{suburb_slug(location)}/"
+    return url + (f"?page={page}" if page > 1 else "")
+
+
+def listing_url(value: str) -> str:
+    match = LISTING_URL_RE.match((value or "").strip())
+    if not match:
+        raise ValueError(f"Not a Domain listing URL: {value}")
+    return f"{BASE_URL}/{match.group(1)}"
+
+
+def is_listing_url(value: str) -> bool:
+    return bool(LISTING_URL_RE.match((value or "").strip()))
+
+
+def parse_price(text: str) -> Tuple[Optional[int], Optional[int]]:
+    """Numbers in a price text: ``$675,000 to $725,000`` -> (675000, 725000); ``$1,100,000`` -> (1100000, 1100000)."""
+    nums = []
+    for raw, suffix in re.findall(r"\$\s?([\d,]+(?:\.\d+)?)\s*([mk])?\b", text or "", re.I):
+        try:
+            value = float(raw.replace(",", ""))
+        except ValueError:
+            continue
+        value *= {"m": 1_000_000, "k": 1_000}.get(suffix.lower(), 1) if suffix else 1
+        nums.append(int(value))
+    if not nums:
+        return None, None
+    return min(nums), max(nums)
+
+
+@dataclass
+class Listing:
+    """One listing from a search page (and, after ``get_listing``, its page)."""
+
+    listing_id: int = 0
+    url: str = ""
+    price_text: str = ""
+    price_min: Optional[int] = None
+    price_max: Optional[int] = None
+    status: str = ""
+    property_type: str = ""
+    beds: Optional[int] = None
+    baths: Optional[int] = None
+    parking: Optional[int] = None
+    land_size: Optional[float] = None
+    land_unit: str = ""
+    street: str = ""
+    suburb: str = ""
+    state: str = ""
+    postcode: str = ""
+    latitude: Optional[float] = None
+    longitude: Optional[float] = None
+    agency: str = ""
+    agency_id: Optional[int] = None
+    agents: List[str] = field(default_factory=list)
+    auction: str = ""
+    inspection_open: str = ""
+    inspection_close: str = ""
+    images: List[str] = field(default_factory=list)
+    promo_type: str = ""
+
+    @property
+    def address(self) -> str:
+        parts = [self.street, self.suburb, f"{self.state} {self.postcode}".strip()]
+        return ", ".join(p for p in parts if p)
+
+    def to_dict(self) -> Dict[str, Any]:
+        out = asdict(self)
+        out["address"] = self.address
+        return out
+
+    def to_row(self) -> Dict[str, Any]:
+        row = self.to_dict()
+        row["agents"] = " | ".join(self.agents)
+        row["images"] = len(self.images)
+        return row
+
+
+@dataclass
+class SearchPage:
+    listings: List[Listing]
+    total_listings: int = 0
+    total_pages: int = 0
+    current_page: int = 1
+
+
+def _int(value: Any) -> Optional[int]:
+    try:
+        return int(value) if value not in (None, "") else None
+    except (TypeError, ValueError):
+        return None
+
+
+def _listing_id(url: str) -> int:
+    match = re.search(r"-(\d{6,})/?$", url or "")
+    return int(match.group(1)) if match else 0
+
+
+def listing_from_model(model: Dict[str, Any]) -> Listing:
+    features = model.get("features") or {}
+    address = model.get("address") or {}
+    branding = model.get("branding") or {}
+    inspection = model.get("inspection") or {}
+    tags = model.get("tags") or {}
+    url = model.get("url") or ""
+    listing = Listing(
+        listing_id=_listing_id(url),
+        url=BASE_URL + url if url.startswith("/") else url,
+        price_text=(model.get("price") or "").strip(),
+        status=(tags.get("tagText") or "").strip(),
+        property_type=features.get("propertyTypeFormatted") or features.get("propertyType") or "",
+        beds=_int(features.get("beds")),
+        baths=_int(features.get("baths")),
+        parking=_int(features.get("parking")),
+        land_size=features.get("landSize") or None,
+        land_unit=features.get("landUnit") or "" if features.get("landSize") else "",
+        street=address.get("street") or "",
+        suburb=address.get("suburb") or "",
+        state=address.get("state") or "",
+        postcode=str(address.get("postcode") or ""),
+        latitude=address.get("lat"),
+        longitude=address.get("lng"),
+        agency=branding.get("brandName") or "",
+        agency_id=_int(branding.get("agencyId")),
+        agents=[a.get("agentName", "") for a in branding.get("agents") or [] if a.get("agentName")],
+        auction=model.get("auction") or "",
+        inspection_open=inspection.get("openTime") or "",
+        inspection_close=inspection.get("closeTime") or "",
+        images=list(model.get("images") or []),
+        promo_type=model.get("promoType") or "",
+    )
+    listing.price_min, listing.price_max = parse_price(listing.price_text)
+    return listing
+
+
+def parse_search(html: str) -> SearchPage:
+    """Listings on a search page, in the order Domain shows them (projects skipped)."""
+    props = _props(html)
+    listings_map = props.get("listingsMap") or {}
+    order = [str(i) for i in props.get("listingSearchResultIds") or []] or list(listings_map)
+    listings = []
+    for key in order:
+        item = listings_map.get(key) or listings_map.get(_int(key))
+        if not item or item.get("listingType") != "listing":
+            continue
+        listings.append(listing_from_model(item.get("listingModel") or {}))
+    return SearchPage(
+        listings=listings,
+        total_listings=_int(props.get("totalListings")) or 0,
+        total_pages=_int(props.get("totalPages")) or 0,
+        current_page=_int(props.get("currentPage")) or 1,
+    )
+
+
+@dataclass
+class Agent:
+    name: str = ""
+    phone: str = ""
+    mobile: str = ""
+    profile_url: str = ""
+
+
+@dataclass
+class ListingDetail:
+    """Everything on a listing page."""
+
+    listing_id: int = 0
+    url: str = ""
+    title: str = ""
+    headline: str = ""
+    listing_type: str = ""
+    status: str = ""
+    method: str = ""
+    property_type: str = ""
+    beds: Optional[int] = None
+    baths: Optional[int] = None
+    parking: Optional[int] = None
+    land_area: Optional[float] = None
+    address: str = ""
+    unit_number: str = ""
+    street_number: str = ""
+    street: str = ""
+    suburb: str = ""
+    state: str = ""
+    postcode: str = ""
+    latitude: Optional[float] = None
+    longitude: Optional[float] = None
+    description: str = ""
+    features: List[str] = field(default_factory=list)
+    agency: str = ""
+    agents: List[Agent] = field(default_factory=list)
+    inspection: str = ""
+    created_on: str = ""
+    modified_on: str = ""
+    images: List[str] = field(default_factory=list)
+    suburb_median_price: Optional[int] = None
+    suburb_median_rent: Optional[int] = None
+    suburb_days_on_market: Optional[int] = None
+    suburb_auction_clearance: Optional[int] = None
+    suburb_sold_this_year: Optional[int] = None
+
+    def to_dict(self) -> Dict[str, Any]:
+        return asdict(self)
+
+    def to_row(self) -> Dict[str, Any]:
+        row = self.to_dict()
+        row["features"] = " | ".join(self.features)
+        row["agents"] = " | ".join(a.name for a in self.agents)
+        row["images"] = len(self.images)
+        return row
+
+
+def parse_listing(html: str, url: str = "") -> ListingDetail:
+    """Parse a listing page."""
+    cp = _props(html)
+    if not cp.get("listingId"):
+        raise ValueError("No listing on this page")
+    summary = cp.get("listingSummary") or {}
+    insights = cp.get("suburbInsights") or {}
+    mapdata = cp.get("map") or {}
+    inspection = cp.get("inspection") or {}
+    stats = {s.get("key"): s.get("value") for s in summary.get("stats") or [] if isinstance(s, dict)}
+    description = cp.get("description") or ""
+    if isinstance(description, list):
+        description = "\n".join(p for p in description if p)
+    images = []
+    for slide in (cp.get("gallery") or {}).get("slides") or []:
+        original = (slide.get("images") or {}).get("original") or {}
+        src = original.get("url") if isinstance(original, dict) else original
+        if src and slide.get("mediaType", "image") == "image":
+            images.append(src)
+    features = [f.get("name") for f in cp.get("structuredFeatures") or [] if f.get("name")] or list(cp.get("features") or [])
+    agents = []
+    for a in (cp.get("priceGuide") or {}).get("agents") or cp.get("agents") or []:
+        agents.append(Agent(name=a.get("name", ""), phone=a.get("phone") or "", mobile=a.get("mobile") or "",
+                            profile_url=a.get("agentProfileUrl") or ""))
+    return ListingDetail(
+        listing_id=_int(cp.get("listingId")) or 0,
+        url=url or cp.get("listingUrl") or "",
+        title=(summary.get("title") or "").strip(),
+        headline=(cp.get("headline") or "").strip(),
+        listing_type={"buy": "sale"}.get(summary.get("mode") or "", summary.get("mode") or ""),
+        status=summary.get("status") or "",
+        method=summary.get("method") or "",
+        property_type=summary.get("propertyType") or cp.get("propertyType") or "",
+        beds=_int(summary.get("beds")),
+        baths=_int(summary.get("baths")),
+        parking=_int(summary.get("parking")),
+        land_area=stats.get("landArea"),
+        address=summary.get("address") or cp.get("address") or "",
+        unit_number=str(cp.get("unitNumber") or ""),
+        street_number=str(cp.get("streetNumber") or ""),
+        street=cp.get("street") or "",
+        suburb=cp.get("suburb") or "",
+        state=(cp.get("stateAbbreviation") or "").upper(),
+        postcode=str(cp.get("postcode") or ""),
+        latitude=mapdata.get("latitude"),
+        longitude=mapdata.get("longitude"),
+        description=description.strip(),
+        features=features,
+        agency=(cp.get("priceGuide") or {}).get("agencyName") or cp.get("agencyName") or "",
+        agents=agents,
+        inspection=(inspection.get("inspectionText") or "").strip(),
+        created_on=cp.get("createdOn") or "",
+        modified_on=cp.get("modifiedOn") or "",
+        images=images,
+        suburb_median_price=_int(insights.get("medianPrice")),
+        suburb_median_rent=_int(insights.get("medianRentPrice")),
+        suburb_days_on_market=_int(insights.get("avgDaysOnMarket")),
+        suburb_auction_clearance=_int(insights.get("auctionClearance")),
+        suburb_sold_this_year=_int(insights.get("nrSoldThisYear")),
+    )
+
+
+LISTING_FIELDS = list(Listing().to_dict().keys())
+DETAIL_FIELDS = list(ListingDetail().to_dict().keys())

domain_com_au_scraper-0.1.0/tests/fixtures/blocked.html

@@ -0,0 +1 @@
+<HTML><HEAD><TITLE>Access Denied</TITLE></HEAD><BODY>Access Denied</BODY></HTML>

domain_com_au_scraper-0.1.0/tests/fixtures/listing.html

@@ -0,0 +1 @@
+<!DOCTYPE html><html><head><title>Domain</title></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props": {"pageProps": {"componentProps": {"listingId": 2099000001, "listingUrl": "https://www.domain.com.au/1-example-street-bondi-beach-nsw-2026-2099000001", "headline": "Bright apartment by the beach", "listingSummary": {"title": "$1,250,000 - $1,350,000", "mode": "buy", "status": "live", "method": "auction", "propertyType": "Apartment / Unit / Flat", "beds": 2, "baths": 1, "parking": 1, "address": "1/1 Example Street, Bondi Beach NSW 2026", "stats": [{"key": "landArea", "value": 85}]}, "unitNumber": "1", "streetNumber": "1", "street": "Example Street", "suburb": "Bondi Beach", "stateAbbreviation": "nsw", "postcode": "2026", "map": {"latitude": -33.89, "longitude": 151.27}, "description": ["First paragraph.", "", "Second paragraph."], "structuredFeatures": [{"name": "Balcony"}, {"name": "Air conditioning"}], "priceGuide": {"agencyName": "Example Realty", "agents": [{"name": "Jane Doe", "phone": "02 5550 0000", "mobile": "0400 000 000", "agentProfileUrl": "https://www.domain.com.au/real-estate-agent/jane-doe-1"}]}, "inspection": {"inspectionText": "Sat 21 Nov, 10:00am"}, "createdOn": "2026-10-01T09:00:00", "modifiedOn": "2026-11-10T09:00:00", "gallery": {"slides": [{"mediaType": "image", "images": {"original": {"url": "https://example.com/a.jpg"}}}, {"mediaType": "video", "images": {"original": {"url": "https://example.com/v.jpg"}}}]}, "suburbInsights": {"medianPrice": 1500000, "medianRentPrice": 900, "avgDaysOnMarket": 30, "auctionClearance": 70, "nrSoldThisYear": 200}}}}}</script></body></html>

domain_com_au_scraper-0.1.0/tests/fixtures/search_bondi.html

@@ -0,0 +1 @@
+<!DOCTYPE html><html><head><title>Domain</title></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props": {"pageProps": {"componentProps": {"listingsMap": {"2099000001": {"id": 2099000001, "listingType": "listing", "listingModel": {"url": "/1-example-street-bondi-beach-nsw-2026-2099000001", "price": "$1,250,000 - $1,350,000", "images": ["https://example.com/a.jpg", "https://example.com/b.jpg"], "features": {"beds": 2, "baths": 1, "parking": 1, "propertyType": "ApartmentUnitFlat", "propertyTypeFormatted": "Apartment / Unit / Flat", "landSize": 0, "landUnit": "m\u00b2"}, "inspection": {"openTime": "2026-11-21T10:00:00", "closeTime": "2026-11-21T10:30:00"}, "tags": {"tagText": "Sold at auction 10 Nov 2026", "tagClassName": "is-sold"}, "address": {"street": "1/1 Example Street", "suburb": "Bondi Beach", "state": "NSW", "postcode": "2026", "lat": -33.89, "lng": 151.27}, "branding": {"agencyId": 999, "brandName": "Example Realty", "agents": [{"agentName": "Jane Doe"}]}, "auction": "2026-11-10T10:00:00", "promoType": "premium"}}, "2099000002": {"id": 2099000002, "listingType": "listing", "listingModel": {"url": "/7-sample-road-bondi-beach-nsw-2026-2099000002", "price": "Price Withheld", "images": [], "features": {"beds": 4, "baths": 3, "parking": 2, "propertyTypeFormatted": "House", "landSize": 450, "landUnit": "m\u00b2"}, "inspection": {"openTime": null, "closeTime": null}, "tags": {"tagText": null}, "address": {"street": "7 Sample Road", "suburb": "Bondi Beach", "state": "NSW", "postcode": "2026", "lat": -33.88, "lng": 151.26}}}, "1": {"id": 1, "listingType": "project", "listingModel": {"url": "/project/1/x/", "projectName": "Example Tower"}}}, "listingSearchResultIds": [2099000002, 1, 2099000001], "totalListings": 3, "totalPages": 1, "currentPage": 1}}}}</script></body></html>

domain_com_au_scraper-0.1.0/tests/test_domain.py

@@ -0,0 +1,140 @@
+import csv
+import io
+import json
+import sys
+import unittest
+from pathlib import Path
+from unittest import mock
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
+
+from domain_scraper import DomainScraper, ScrapeError, listing_url, parse_listing, parse_price, parse_search, search_url, suburb_slug  # noqa: E402
+from domain_scraper import cli  # noqa: E402
+
+FIX = Path(__file__).parent / "fixtures"
+SEARCH = (FIX / "search_bondi.html").read_text(encoding="utf-8")
+LISTING = (FIX / "listing.html").read_text(encoding="utf-8")
+BLOCKED = (FIX / "blocked.html").read_text(encoding="utf-8")
+URL1 = "https://www.domain.com.au/1-example-street-bondi-beach-nsw-2026-2099000001"
+
+
+class R:
+    def __init__(self, status, text=""):
+        self.status_code, self.text = status, text
+
+
+class Session:
+    def __init__(self, responses):
+        self.responses, self.calls = list(responses), []
+
+    def get(self, url, params=None, timeout=None):
+        self.calls.append(dict(params))
+        return self.responses.pop(0)
+
+
+class UrlTests(unittest.TestCase):
+    def test_slug_and_urls(self):
+        self.assertEqual(suburb_slug("Bondi Beach NSW 2026"), "bondi-beach-nsw-2026")
+        self.assertEqual(search_url("bondi-beach-nsw-2026", "sold", 2), "https://www.domain.com.au/sold-listings/bondi-beach-nsw-2026/?page=2")
+        self.assertEqual(search_url("Sydney NSW 2000", "rent"), "https://www.domain.com.au/rent/sydney-nsw-2000/")
+        with self.assertRaises(ValueError):
+            suburb_slug("Bondi")
+        with self.assertRaises(ValueError):
+            search_url("Sydney NSW 2000", "lease")
+        self.assertEqual(listing_url(URL1 + "/"), URL1)
+        with self.assertRaises(ValueError):
+            listing_url("https://example.com/x")
+
+    def test_price(self):
+        self.assertEqual(parse_price("$675,000 to $725,000"), (675000, 725000))
+        self.assertEqual(parse_price("$1,100,000"), (1100000, 1100000))
+        self.assertEqual(parse_price("$1700 per week"), (1700, 1700))
+        self.assertEqual(parse_price("Auction guide $1.2m"), (1200000, 1200000))
+        self.assertEqual(parse_price("Price Withheld"), (None, None))
+
+
+class ParseTests(unittest.TestCase):
+    def test_search(self):
+        page = parse_search(SEARCH)
+        self.assertEqual((page.total_listings, page.total_pages), (3, 1))
+        self.assertEqual([l.listing_id for l in page.listings], [2099000002, 2099000001])
+        l = page.listings[1]
+        self.assertEqual((l.price_min, l.price_max), (1250000, 1350000))
+        self.assertEqual(l.status, "Sold at auction 10 Nov 2026")
+        self.assertEqual((l.beds, l.baths, l.parking), (2, 1, 1))
+        self.assertEqual(l.address, "1/1 Example Street, Bondi Beach, NSW 2026")
+        self.assertEqual((l.agency, l.agents), ("Example Realty", ["Jane Doe"]))
+        self.assertEqual(l.url, URL1)
+        self.assertEqual(page.listings[0].land_size, 450)
+        self.assertIsNone(page.listings[0].price_min)
+
+    def test_listing(self):
+        d = parse_listing(LISTING)
+        self.assertEqual(d.listing_id, 2099000001)
+        self.assertEqual((d.title, d.listing_type, d.method), ("$1,250,000 - $1,350,000", "sale", "auction"))
+        self.assertEqual(d.description, "First paragraph.\nSecond paragraph.")
+        self.assertEqual(d.features, ["Balcony", "Air conditioning"])
+        self.assertEqual(d.agents[0].mobile, "0400 000 000")
+        self.assertEqual((d.state, d.land_area, d.images), ("NSW", 85, ["https://example.com/a.jpg"]))
+        self.assertEqual((d.suburb_median_price, d.suburb_auction_clearance), (1500000, 70))
+        with self.assertRaises(ValueError):
+            parse_listing(BLOCKED)
+        with self.assertRaises(ValueError):
+            parse_listing(SEARCH)
+
+
+class ClientTests(unittest.TestCase):
+    def test_token_required(self):
+        with mock.patch.dict("os.environ", {}, clear=True):
+            with self.assertRaises(ValueError):
+                DomainScraper()
+
+    def test_search_super(self):
+        s = Session([R(200, SEARCH)])
+        dm = DomainScraper(token="t", session=s)
+        out = list(dm.search("Bondi Beach NSW 2026", "sold", pages=5))
+        self.assertEqual(len(out), 2)
+        self.assertEqual(len(s.calls), 1)  # total_pages = 1
+        self.assertEqual(s.calls[0]["super"], "true")
+        self.assertEqual(dm.credits_used, 10)
+
+    @mock.patch("domain_scraper.client.time.sleep")
+    def test_blocked_retried(self, _):
+        s = Session([R(200, BLOCKED), R(403, "x"), R(200, LISTING)])
+        d = DomainScraper(token="t", session=s).get_listing(URL1)
+        self.assertEqual(d.agency, "Example Realty")
+        self.assertEqual(len(s.calls), 3)
+
+    def test_401_stops(self):
+        s = Session([R(401)])
+        with self.assertRaises(ScrapeError):
+            DomainScraper(token="t", session=s, retries=3).get_listing(URL1)
+        self.assertEqual(len(s.calls), 1)
+
+
+class CliTests(unittest.TestCase):
+    def run_cli(self, argv, responses):
+        s = Session(responses)
+        out, err = io.StringIO(), io.StringIO()
+        with mock.patch("domain_scraper.client.requests.Session", return_value=s), \
+                mock.patch.object(sys, "stdout", out), mock.patch.object(sys, "stderr", err):
+            code = cli.main(argv + ["-t", "t"])
+        return code, out.getvalue()
+
+    def test_csv(self):
+        code, out = self.run_cli(["Bondi Beach NSW 2026", "--mode", "sold", "-f", "csv"], [R(200, SEARCH)])
+        rows = list(csv.DictReader(io.StringIO(out)))
+        self.assertEqual(code, 0)
+        self.assertEqual(rows[1]["agency"], "Example Realty")
+
+    def test_details_limit(self):
+        code, out = self.run_cli(["Bondi Beach NSW 2026", "--details", "--limit", "1"], [R(200, SEARCH), R(200, LISTING)])
+        self.assertEqual(len(json.loads(out)), 1)
+
+    def test_listing_url(self):
+        code, out = self.run_cli([URL1], [R(200, LISTING)])
+        self.assertEqual(json.loads(out)[0]["suburb"], "Bondi Beach")
+
+
+if __name__ == "__main__":
+    unittest.main()