blinkit-scraper 0.1.0__tar.gz

blinkit_scraper-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Buğrahan Saka
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

blinkit_scraper-0.1.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md LICENSE
+recursive-include tests *.py *.html *.json
+recursive-include examples *.py

blinkit_scraper-0.1.0/PKG-INFO

@@ -0,0 +1,135 @@
+Metadata-Version: 2.4
+Name: blinkit-scraper
+Version: 0.1.0
+Summary: Scrape Blinkit categories and product pages (price, MRP, stock per dark store, FSSAI licence, shelf life) with Python.
+Author: Buğrahan Saka
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/bugrahansss/blinkit-scraper
+Project-URL: Source, https://github.com/bugrahansss/blinkit-scraper
+Project-URL: Issues, https://github.com/bugrahansss/blinkit-scraper/issues
+Project-URL: Scrape.do, https://scrape.do/?utm_source=pypi&utm_medium=package&utm_campaign=blinkit-scraper
+Keywords: blinkit,web scraping,quick commerce,grocery,scraper,cloudflare,india
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25
+Dynamic: license-file
+
+# blinkit-scraper
+
+[![PyPI](https://img.shields.io/pypi/v/blinkit-scraper)](https://pypi.org/project/blinkit-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/blinkit-scraper)](https://pypi.org/project/blinkit-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+A Python scraper for [Blinkit](https://blinkit.com), India's 10-minute grocery app. Two jobs:
+
+- **Category listings:** any category page (`/cn/chips-crisps/cid/1237/940`) → name, brand, pack size, price, MRP, discount, **units in stock**, rating, rating count, variants and URL, **for the dark store that serves your location**. Real pagination: 30 products on the first page, 15 per page after that, out-of-stock products kept.
+- **Product pages:** price, MRP and stock plus the catalog sheet: **FSSAI licence, shelf life, country of origin, ingredients, manufacturer, return policy** and every other attribute Blinkit lists.
+
+Output as Python dicts, JSON or CSV. One dependency (`requests`), no browser.
+
+```bash
+pip install blinkit-scraper
+```
+
+## Why a plain request fails
+
+Blinkit sits behind Cloudflare. A script gets an error page, not products:
+
+| Request | Result |
+| --- | --- |
+| `requests`, default or Chrome user agent | HTTP 403, "blinkit \| Error Page" |
+| Through Scrape.do (`super=true&geoCode=in`) | 200 in ~3 s, full page with the data, 5 of 5 tries |
+
+Pages are fetched through the [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=blinkit-scraper) API with residential proxies in India. Blinkit requests cost 10 credits each; the free plan includes 1,000 credits a month.
+
+## Quick start
+
+```python
+from blinkit_scraper import BlinkitScraper
+
+scraper = BlinkitScraper(token="YOUR_SCRAPEDO_TOKEN", city="mumbai")  # or lat=..., lon=...
+
+# 3 pages = 30 + 15 + 15 = 60 products, 3 requests
+rows = scraper.get_category("https://blinkit.com/cn/chips-crisps/cid/1237/940", max_pages=3)
+for row in rows[:3]:
+    print(row["name"], row["pack_size"], row["price_inr"], row["mrp_inr"], row["inventory"], row["in_stock"])
+
+product = scraper.get_product(rows[0]["url"])
+print(product["fssai_license"], product["shelf_life"], product["country_of_origin"])
+```
+
+### Command line
+
+```bash
+export SCRAPEDO_TOKEN=your_token
+
+# 3 pages of a category for a Mumbai address, CSV
+blinkit-scraper category "https://blinkit.com/cn/chips-crisps/cid/1237/940" --pages 3 --city mumbai -f csv -o blinkit-chips.csv
+
+# product sheets for the first 10 products of that file
+blinkit-scraper product -i blinkit-chips.csv --limit 10 --city mumbai -f csv -o blinkit-products.csv
+
+# any address: pass its coordinates
+blinkit-scraper category "https://blinkit.com/cn/chips-crisps/cid/1237/940" --lat 12.9352 --lon 77.6245
+```
+
+City presets: `bengaluru`, `chennai`, `delhi`, `gurugram` (Blinkit's default), `hyderabad`, `kolkata`, `mumbai`, `pune`.
+
+### Output
+
+Listing row (from the fictional test page in [`tests/fixtures`](tests/fixtures/category_page.html)):
+
+```json
+{"product_id": 1001, "name": "Example Salted Chips", "brand": "Northwind Snacks", "pack_size": "60 g",
+ "price_inr": 43, "mrp_inr": 70, "discount_pct": 38, "in_stock": true, "inventory": 5,
+ "rating": 4.31, "rating_count": 120000, "variants": 2, "merchant_id": 90001,
+ "image": "https://cdn.example.com/p/1001.png", "url": "https://blinkit.com/prn/example-salted-chips/prid/1001"}
+```
+
+Product columns: `product_id, name, brand, pack_size, price_inr, mrp_inr, discount_pct, in_stock, inventory, shelf_life, fssai_license, country_of_origin, category_ids, merchant_id, url` (plus `ingredients`, `manufacturer`, `return_policy`, `images` and the full `attributes` dict in Python/JSON).
+
+## How it works
+
+**The data is one JSON object.** Blinkit renders every page on the server and puts the data in `window.grofers.PRELOADED_STATE = {...}`. Category pages keep the product cards in `ui.plpContainer.feedData.snippets`, product pages keep the catalog sheet in `ui.pdp.bffPdp.bffData.tracking.le_meta.custom_data.seo`. The scraper reads the object with `json.JSONDecoder.raw_decode`. No CSS selectors.
+
+**Each card carries the stock count.** `inventory` is the number of units the dark store has (Blinkit stops counting at 50). `rating_count` arrives as text in Indian units ("7 lac" = 700,000) and is converted.
+
+**Location picks the store.** Prices, assortment and stock belong to one dark store, chosen from the delivery address. Without one, Blinkit uses Gurugram. In our test the chips category listed 673 products in Gurugram (store 31719) and 524 in Mumbai (store 41649); a banana chips pack had 5 units in one and 1 in the other. Pages read the location from the `gr_1_lat` / `gr_1_lon` cookies, the listing API from `lat` / `lon` headers; the scraper sends both.
+
+**Pagination is a POST API.** The page holds 30 products and a `next_url` for `/v1/layout/listing_widgets`. That endpoint returns 15 products per call (a larger `limit` is ignored) and the next `next_url`. It only answers through residential proxies, so every request uses `super=true&geoCode=in`. Reading a whole category of 673 products takes 44 requests.
+
+## Limits
+
+- **Credits.** 10 Scrape.do credits per request: page 1 of a category, each further page of 15, each product page.
+- **Stock cap.** `inventory` tops out at 50.
+- **Search.** `/s/` search pages are disallowed in Blinkit's robots.txt and not supported; use category pages.
+- **Errors.** Non-200 responses raise `ScrapeError` with the status code; `from_target` tells a Blinkit status from a Scrape.do one (a wrong token is a Scrape.do 401). A 200 without Blinkit's data is retried. Server errors, 429s and empty pages are retried twice; client errors are not.
+
+## Responsible use
+
+The scraper reads category (`/cn/`) and product (`/prn/`) pages, which Blinkit's robots.txt allows, plus the listing API those pages call. It never touches search, cart, checkout or account paths. Product data is not personal data. Read Blinkit's terms for your use case and keep request rates modest. This is not legal advice.
+
+## Development
+
+```bash
+git clone https://github.com/bugrahansss/blinkit-scraper
+cd blinkit-scraper
+pip install -e .
+python -m unittest discover -s tests -v
+```
+
+The tests run offline against fictional fixture pages.
+
+## License
+
+MIT
+
+---
+
+Built and tested with [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=blinkit-scraper), November 2026.

blinkit_scraper-0.1.0/README.md

@@ -0,0 +1,113 @@
+# blinkit-scraper
+
+[![PyPI](https://img.shields.io/pypi/v/blinkit-scraper)](https://pypi.org/project/blinkit-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/blinkit-scraper)](https://pypi.org/project/blinkit-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+A Python scraper for [Blinkit](https://blinkit.com), India's 10-minute grocery app. Two jobs:
+
+- **Category listings:** any category page (`/cn/chips-crisps/cid/1237/940`) → name, brand, pack size, price, MRP, discount, **units in stock**, rating, rating count, variants and URL, **for the dark store that serves your location**. Real pagination: 30 products on the first page, 15 per page after that, out-of-stock products kept.
+- **Product pages:** price, MRP and stock plus the catalog sheet: **FSSAI licence, shelf life, country of origin, ingredients, manufacturer, return policy** and every other attribute Blinkit lists.
+
+Output as Python dicts, JSON or CSV. One dependency (`requests`), no browser.
+
+```bash
+pip install blinkit-scraper
+```
+
+## Why a plain request fails
+
+Blinkit sits behind Cloudflare. A script gets an error page, not products:
+
+| Request | Result |
+| --- | --- |
+| `requests`, default or Chrome user agent | HTTP 403, "blinkit \| Error Page" |
+| Through Scrape.do (`super=true&geoCode=in`) | 200 in ~3 s, full page with the data, 5 of 5 tries |
+
+Pages are fetched through the [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=blinkit-scraper) API with residential proxies in India. Blinkit requests cost 10 credits each; the free plan includes 1,000 credits a month.
+
+## Quick start
+
+```python
+from blinkit_scraper import BlinkitScraper
+
+scraper = BlinkitScraper(token="YOUR_SCRAPEDO_TOKEN", city="mumbai")  # or lat=..., lon=...
+
+# 3 pages = 30 + 15 + 15 = 60 products, 3 requests
+rows = scraper.get_category("https://blinkit.com/cn/chips-crisps/cid/1237/940", max_pages=3)
+for row in rows[:3]:
+    print(row["name"], row["pack_size"], row["price_inr"], row["mrp_inr"], row["inventory"], row["in_stock"])
+
+product = scraper.get_product(rows[0]["url"])
+print(product["fssai_license"], product["shelf_life"], product["country_of_origin"])
+```
+
+### Command line
+
+```bash
+export SCRAPEDO_TOKEN=your_token
+
+# 3 pages of a category for a Mumbai address, CSV
+blinkit-scraper category "https://blinkit.com/cn/chips-crisps/cid/1237/940" --pages 3 --city mumbai -f csv -o blinkit-chips.csv
+
+# product sheets for the first 10 products of that file
+blinkit-scraper product -i blinkit-chips.csv --limit 10 --city mumbai -f csv -o blinkit-products.csv
+
+# any address: pass its coordinates
+blinkit-scraper category "https://blinkit.com/cn/chips-crisps/cid/1237/940" --lat 12.9352 --lon 77.6245
+```
+
+City presets: `bengaluru`, `chennai`, `delhi`, `gurugram` (Blinkit's default), `hyderabad`, `kolkata`, `mumbai`, `pune`.
+
+### Output
+
+Listing row (from the fictional test page in [`tests/fixtures`](tests/fixtures/category_page.html)):
+
+```json
+{"product_id": 1001, "name": "Example Salted Chips", "brand": "Northwind Snacks", "pack_size": "60 g",
+ "price_inr": 43, "mrp_inr": 70, "discount_pct": 38, "in_stock": true, "inventory": 5,
+ "rating": 4.31, "rating_count": 120000, "variants": 2, "merchant_id": 90001,
+ "image": "https://cdn.example.com/p/1001.png", "url": "https://blinkit.com/prn/example-salted-chips/prid/1001"}
+```
+
+Product columns: `product_id, name, brand, pack_size, price_inr, mrp_inr, discount_pct, in_stock, inventory, shelf_life, fssai_license, country_of_origin, category_ids, merchant_id, url` (plus `ingredients`, `manufacturer`, `return_policy`, `images` and the full `attributes` dict in Python/JSON).
+
+## How it works
+
+**The data is one JSON object.** Blinkit renders every page on the server and puts the data in `window.grofers.PRELOADED_STATE = {...}`. Category pages keep the product cards in `ui.plpContainer.feedData.snippets`, product pages keep the catalog sheet in `ui.pdp.bffPdp.bffData.tracking.le_meta.custom_data.seo`. The scraper reads the object with `json.JSONDecoder.raw_decode`. No CSS selectors.
+
+**Each card carries the stock count.** `inventory` is the number of units the dark store has (Blinkit stops counting at 50). `rating_count` arrives as text in Indian units ("7 lac" = 700,000) and is converted.
+
+**Location picks the store.** Prices, assortment and stock belong to one dark store, chosen from the delivery address. Without one, Blinkit uses Gurugram. In our test the chips category listed 673 products in Gurugram (store 31719) and 524 in Mumbai (store 41649); a banana chips pack had 5 units in one and 1 in the other. Pages read the location from the `gr_1_lat` / `gr_1_lon` cookies, the listing API from `lat` / `lon` headers; the scraper sends both.
+
+**Pagination is a POST API.** The page holds 30 products and a `next_url` for `/v1/layout/listing_widgets`. That endpoint returns 15 products per call (a larger `limit` is ignored) and the next `next_url`. It only answers through residential proxies, so every request uses `super=true&geoCode=in`. Reading a whole category of 673 products takes 44 requests.
+
+## Limits
+
+- **Credits.** 10 Scrape.do credits per request: page 1 of a category, each further page of 15, each product page.
+- **Stock cap.** `inventory` tops out at 50.
+- **Search.** `/s/` search pages are disallowed in Blinkit's robots.txt and not supported; use category pages.
+- **Errors.** Non-200 responses raise `ScrapeError` with the status code; `from_target` tells a Blinkit status from a Scrape.do one (a wrong token is a Scrape.do 401). A 200 without Blinkit's data is retried. Server errors, 429s and empty pages are retried twice; client errors are not.
+
+## Responsible use
+
+The scraper reads category (`/cn/`) and product (`/prn/`) pages, which Blinkit's robots.txt allows, plus the listing API those pages call. It never touches search, cart, checkout or account paths. Product data is not personal data. Read Blinkit's terms for your use case and keep request rates modest. This is not legal advice.
+
+## Development
+
+```bash
+git clone https://github.com/bugrahansss/blinkit-scraper
+cd blinkit-scraper
+pip install -e .
+python -m unittest discover -s tests -v
+```
+
+The tests run offline against fictional fixture pages.
+
+## License
+
+MIT
+
+---
+
+Built and tested with [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=blinkit-scraper), November 2026.

blinkit_scraper-0.1.0/examples/compare_cities.py

@@ -0,0 +1,17 @@
+"""Same category, two cities: which products and how much stock each dark store has."""
+from blinkit_scraper import BlinkitScraper
+
+URL = "https://blinkit.com/cn/chips-crisps/cid/1237/940"
+
+stores = {}
+for city in ("gurugram", "mumbai"):
+    scraper = BlinkitScraper(city=city)  # token from SCRAPEDO_TOKEN
+    info = scraper.get_category(URL, max_pages=2, with_info=True)
+    stores[city] = {row["product_id"]: row for row in info["products"]}
+    print(f"{city}: store {info['merchant_id']}, {info['total']} products listed, {scraper.credits_used} credits")
+
+both = stores["gurugram"].keys() & stores["mumbai"].keys()
+print(f"{len(both)} products on page 1-2 in both cities")
+for pid in list(both)[:10]:
+    g, m = stores["gurugram"][pid], stores["mumbai"][pid]
+    print(f"{g['name'][:40]:40} {g['price_inr']:>5} {m['price_inr']:>5}   stock {g['inventory']:>3} / {m['inventory']:>3}")

blinkit_scraper-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "blinkit-scraper"
+version = "0.1.0"
+description = "Scrape Blinkit categories and product pages (price, MRP, stock per dark store, FSSAI licence, shelf life) with Python."
+readme = "README.md"
+requires-python = ">=3.8"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Buğrahan Saka" }]
+keywords = ["blinkit", "web scraping", "quick commerce", "grocery", "scraper", "cloudflare", "india"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Topic :: Internet :: WWW/HTTP :: Indexing/Search",
+]
+dependencies = ["requests>=2.25"]
+
+[project.urls]
+Homepage = "https://github.com/bugrahansss/blinkit-scraper"
+Source = "https://github.com/bugrahansss/blinkit-scraper"
+Issues = "https://github.com/bugrahansss/blinkit-scraper/issues"
+"Scrape.do" = "https://scrape.do/?utm_source=pypi&utm_medium=package&utm_campaign=blinkit-scraper"
+
+[project.scripts]
+blinkit-scraper = "blinkit_scraper.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

blinkit_scraper-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

blinkit_scraper-0.1.0/src/blinkit_scraper/__init__.py

@@ -0,0 +1,33 @@
+"""Scrape Blinkit category listings and product pages (price, MRP, stock per dark store) with Python."""
+
+from .client import CITIES, BlinkitScraper, ScrapeError
+from .parser import (
+    LISTING_FIELDS,
+    PRODUCT_FIELDS,
+    extract_state,
+    parse_card,
+    parse_cards,
+    parse_category,
+    parse_count,
+    parse_listing_api,
+    parse_product,
+    product_url,
+)
+
+__all__ = [
+    "CITIES",
+    "LISTING_FIELDS",
+    "PRODUCT_FIELDS",
+    "BlinkitScraper",
+    "ScrapeError",
+    "extract_state",
+    "parse_card",
+    "parse_cards",
+    "parse_category",
+    "parse_count",
+    "parse_listing_api",
+    "parse_product",
+    "product_url",
+]
+
+__version__ = "0.1.0"

blinkit_scraper-0.1.0/src/blinkit_scraper/cli.py

@@ -0,0 +1,117 @@
+"""Command line entry point: ``blinkit-scraper category URL`` / ``blinkit-scraper product URL ...``."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import sys
+from typing import List, Optional
+
+from . import __version__
+from .client import CITIES, TOKEN_ENV, BlinkitScraper, ScrapeError
+from .parser import LISTING_FIELDS, PRODUCT_FIELDS
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="blinkit-scraper",
+        description="Scrape Blinkit categories and product pages (price, MRP, stock per dark store) into JSON or CSV.",
+    )
+    parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    cat = sub.add_parser("category", help="Products of a category page (30 on page 1, 15 per further page)")
+    cat.add_argument("url", help="A Blinkit category URL, e.g. https://blinkit.com/cn/chips-crisps/cid/1237/940")
+    cat.add_argument("--pages", type=int, default=1, help="Pages to read (default 1)")
+
+    prod = sub.add_parser("product", help="Product pages: price, stock, FSSAI licence, shelf life, attributes")
+    prod.add_argument("urls", nargs="*", help="Blinkit product URLs or product ids")
+    prod.add_argument("-i", "--input", help="Text file with one URL per line, or a CSV with a 'url' column")
+    prod.add_argument("--limit", type=int, default=0, help="Stop after this many products (0 = all)")
+
+    for p in (cat, prod):
+        p.add_argument("-t", "--token", help=f"Scrape.do API token (default: ${TOKEN_ENV})")
+        p.add_argument("--city", choices=sorted(CITIES), help="Delivery location preset (default: Gurugram)")
+        p.add_argument("--lat", type=float, help="Delivery latitude (with --lon)")
+        p.add_argument("--lon", type=float, help="Delivery longitude (with --lat)")
+        p.add_argument("-f", "--format", choices=("json", "csv"), default="json")
+        p.add_argument("-o", "--output", help="Write to this file instead of stdout")
+    return parser
+
+
+def _read_urls(args: argparse.Namespace) -> List[str]:
+    urls = list(args.urls)
+    if args.input:
+        with open(args.input, encoding="utf-8", newline="") as handle:
+            if args.input.endswith(".csv"):
+                urls.extend(row["url"] for row in csv.DictReader(handle) if row.get("url"))
+            else:
+                urls.extend(line.strip() for line in handle if line.strip() and not line.startswith("#"))
+    return urls[: args.limit] if args.limit else urls
+
+
+def _write(rows: List[dict], fields: List[str], fmt: str, stream) -> None:
+    if fmt == "csv":
+        writer = csv.DictWriter(stream, fieldnames=fields, extrasaction="ignore")
+        writer.writeheader()
+        writer.writerows(rows)
+    else:
+        json.dump(rows, stream, indent=2, ensure_ascii=False)
+        stream.write("\n")
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    args = build_parser().parse_args(argv)
+    if (args.lat is None) != (args.lon is None):
+        print("Pass both --lat and --lon.", file=sys.stderr)
+        return 2
+    try:
+        scraper = BlinkitScraper(token=args.token, lat=args.lat, lon=args.lon, city=args.city)
+    except ValueError as exc:
+        print(exc, file=sys.stderr)
+        return 2
+
+    failures = 0
+    if args.command == "category":
+        fields = LISTING_FIELDS
+        try:
+            info = scraper.get_category(args.url, max_pages=max(1, args.pages), with_info=True)
+        except (ScrapeError, ValueError) as exc:
+            print(f"fail  {args.url}: {exc}", file=sys.stderr)
+            return 1
+        rows = info["products"]
+        print(f"{len(rows)} of {info['total']} products, store {info['merchant_id']} ({info['locality']}), "
+              f"{scraper.credits_used} credits", file=sys.stderr)
+    else:
+        fields = PRODUCT_FIELDS
+        urls = _read_urls(args)
+        if not urls:
+            print("No URLs given. Pass product URLs or --input FILE.", file=sys.stderr)
+            return 2
+        rows = []
+        for url in urls:
+            try:
+                product = scraper.get_product(url)
+            except (ScrapeError, ValueError) as exc:
+                failures += 1
+                print(f"fail  {url}: {exc}", file=sys.stderr)
+                continue
+            if not product:
+                failures += 1
+                print(f"fail  {url}: no product data", file=sys.stderr)
+                continue
+            rows.append(product)
+            print(f"ok    {url}", file=sys.stderr)
+
+    if args.output:
+        with open(args.output, "w", encoding="utf-8", newline="" if args.format == "csv" else None) as handle:
+            _write(rows, fields, args.format, handle)
+        print(f"Saved {len(rows)} rows to {args.output}", file=sys.stderr)
+    else:
+        _write(rows, fields, args.format, sys.stdout)
+    return 1 if failures and not rows else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())