swiggy-scraper 0.1.0__tar.gz

swiggy_scraper-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Buğrahan Saka
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

swiggy_scraper-0.1.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md LICENSE
+recursive-include tests *.py *.json
+recursive-include examples *.py

swiggy_scraper-0.1.0/PKG-INFO

@@ -0,0 +1,130 @@
+Metadata-Version: 2.4
+Name: swiggy-scraper
+Version: 0.1.0
+Summary: Scrape Swiggy dishes, restaurants and full menus (prices, variants, ratings, delivery time, FSSAI licence) with Python.
+Author: Buğrahan Saka
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/bugrahansss/swiggy-scraper
+Project-URL: Source, https://github.com/bugrahansss/swiggy-scraper
+Project-URL: Issues, https://github.com/bugrahansss/swiggy-scraper/issues
+Project-URL: Scrape.do, https://scrape.do/?utm_source=pypi&utm_medium=package&utm_campaign=swiggy-scraper
+Keywords: swiggy,web scraping,food delivery,restaurant menu,scraper,aws waf,india
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25
+Dynamic: license-file
+
+# swiggy-scraper
+
+[![PyPI](https://img.shields.io/pypi/v/swiggy-scraper)](https://pypi.org/project/swiggy-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/swiggy-scraper)](https://pypi.org/project/swiggy-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+A Python scraper for [Swiggy](https://www.swiggy.com), India's food delivery app. Three jobs:
+
+- **Dish search:** search a dish ("biryani") and get every matching dish from the restaurants that deliver to your location: price, veg/non-veg, bestseller flag, rating, rating count, every variant price, and the restaurant with its delivery time. One request, ~50 restaurants.
+- **Restaurant search and nearby list:** name, area, cuisines, rating, rating count, cost for two, delivery time, distance, open/serviceable, and whether the listing is a paid (promoted) placement.
+- **Full menus:** every dish of a restaurant with price, variants, veg flag, stock and rating, plus the restaurant's **FSSAI licence** and full address.
+
+Output as Python dicts, JSON or CSV. One dependency (`requests`), no browser.
+
+```bash
+pip install swiggy-scraper
+```
+
+## Why a plain request fails
+
+Swiggy sits behind AWS WAF on CloudFront. A script gets no data:
+
+| Request | Result |
+| --- | --- |
+| `requests` with a Chrome user agent | HTTP 202, empty body, `x-amzn-waf-action: challenge` |
+| `requests` with its default user agent | HTTP 403, an error page |
+| Through Scrape.do (`geoCode=in`) | 200 in ~3 s, the JSON, every try |
+
+Requests go through the [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=swiggy-scraper) API from India. Each request costs 1 credit; the free plan includes 1,000 credits a month.
+
+## Quick start
+
+```python
+from swiggy_scraper import SwiggyScraper
+
+scraper = SwiggyScraper(token="YOUR_SCRAPEDO_TOKEN", city="mumbai")  # or lat=..., lng=...
+
+dishes = scraper.search_dishes("biryani")          # 1 request
+for d in sorted(dishes, key=lambda d: d["price_inr"])[:3]:
+    print(d["price_inr"], d["name"], d["restaurant_name"], d["delivery_time_min"])
+
+menu = scraper.get_menu("https://www.swiggy.com/city/bangalore/starbucks-coffee-ashok-nagar-rest124178")
+print(menu["restaurant"]["fssai_license"], len(menu["items"]))
+```
+
+### Command line
+
+```bash
+export SCRAPEDO_TOKEN=your_token
+
+# every biryani on Swiggy for a Mumbai address
+swiggy-scraper dishes biryani --city mumbai -f csv -o swiggy-biryani.csv
+
+# pizza places that deliver to central Bengaluru
+swiggy-scraper restaurants pizza -f csv -o swiggy-pizza.csv
+
+# restaurants on the home page for any address
+swiggy-scraper nearby --lat 12.9352 --lng 77.6245 -f csv -o swiggy-nearby.csv
+
+# full menus of those restaurants
+swiggy-scraper menu -i swiggy-nearby.csv --limit 5 -f csv -o swiggy-menus.csv
+```
+
+City presets: `bengaluru` (default), `chennai`, `delhi`, `gurugram`, `hyderabad`, `kolkata`, `mumbai`, `pune`.
+
+### Output
+
+Dish row (from the fictional test data in [`tests/fixtures`](tests/fixtures/menu.json)):
+
+```json
+{"item_id": "9001", "name": "Paneer Dum Biryani", "category": "Biryani", "price_inr": 249, "is_veg": true,
+ "in_stock": true, "bestseller": true, "rating": 4.4, "rating_count": 210,
+ "variant_prices_inr": "Half 249; Full 389", "restaurant_id": "5001", "restaurant_name": "Example Biryani House"}
+```
+
+Restaurant columns: `restaurant_id, name, area, locality, cuisines, rating, rating_count, cost_for_two_inr, delivery_time_min, distance_km, serviceable, is_open, promoted, veg_only, url` (menus add `fssai_license` and `address`).
+
+## How it works
+
+**The page is a shell.** A restaurant page (`/city/bangalore/…-rest124178`) is 78 KB of HTML with no menu in it. The browser fills it from JSON endpoints under `/dapi`, and this package calls those directly:
+
+| Data | Endpoint |
+| --- | --- |
+| Dish / restaurant search | `/dapi/restaurants/search/v3?lat&lng&str=biryani&selectedPLTab=DISH` (or `RESTAURANT`) |
+| Home page restaurants | `/dapi/restaurants/list/v5?lat&lng` |
+| Full menu | `/dapi/menu/pl?page-type=REGULAR_MENU&complete-menu=true&lat&lng&restaurantId=…` |
+
+Every response is a list of widget cards with an `@type`; the parsers keep the restaurant, dish and licence cards and skip banners, filters and SEO blocks.
+
+**Money is in paise.** `"price": 42000` is Rs 420. Some dishes have no `price`, only `defaultPrice`. Variant prices come either as absolute prices (`variantsV2.pricingModels`) or as a surcharge on the base price (`variants`); both are turned into rupees.
+
+**The menu repeats itself.** The "Recommended" section lists dishes that also appear in their own sections: Starbucks in Bengaluru has 168 dish cards and 148 dishes. Each dish is kept once.
+
+**Location decides the restaurants.** `lat`/`lng` decide which restaurants deliver, how far they are and how long delivery takes. The same "biryani" search returned 50 restaurants in Bengaluru and 50 in Mumbai with none in common. A menu still loads from a far-away location, with `serviceable: false`.
+
+**Some results are ads.** Search mixes in paid placements; they carry `promoted: true` so you can drop them or keep them.
+
+## Responsible use
+
+The package calls the JSON endpoints Swiggy's own web pages use for search, the home page and menus. Swiggy's robots.txt allows these paths and disallows account, cart, checkout, order tracking and payment pages, which this package never touches. It collects restaurant and menu data, not personal data. Read Swiggy's terms for your use case and keep request rates modest.
+
+## License
+
+MIT. Not affiliated with Swiggy.
+
+---
+
+Built with [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=swiggy-scraper). [Get 1000 free credits](https://dashboard.scrape.do/sign-up?utm_source=github&utm_medium=readme&utm_campaign=swiggy-scraper).

swiggy_scraper-0.1.0/README.md

@@ -0,0 +1,108 @@
+# swiggy-scraper
+
+[![PyPI](https://img.shields.io/pypi/v/swiggy-scraper)](https://pypi.org/project/swiggy-scraper/)
+[![Python](https://img.shields.io/pypi/pyversions/swiggy-scraper)](https://pypi.org/project/swiggy-scraper/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+A Python scraper for [Swiggy](https://www.swiggy.com), India's food delivery app. Three jobs:
+
+- **Dish search:** search a dish ("biryani") and get every matching dish from the restaurants that deliver to your location: price, veg/non-veg, bestseller flag, rating, rating count, every variant price, and the restaurant with its delivery time. One request, ~50 restaurants.
+- **Restaurant search and nearby list:** name, area, cuisines, rating, rating count, cost for two, delivery time, distance, open/serviceable, and whether the listing is a paid (promoted) placement.
+- **Full menus:** every dish of a restaurant with price, variants, veg flag, stock and rating, plus the restaurant's **FSSAI licence** and full address.
+
+Output as Python dicts, JSON or CSV. One dependency (`requests`), no browser.
+
+```bash
+pip install swiggy-scraper
+```
+
+## Why a plain request fails
+
+Swiggy sits behind AWS WAF on CloudFront. A script gets no data:
+
+| Request | Result |
+| --- | --- |
+| `requests` with a Chrome user agent | HTTP 202, empty body, `x-amzn-waf-action: challenge` |
+| `requests` with its default user agent | HTTP 403, an error page |
+| Through Scrape.do (`geoCode=in`) | 200 in ~3 s, the JSON, every try |
+
+Requests go through the [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=swiggy-scraper) API from India. Each request costs 1 credit; the free plan includes 1,000 credits a month.
+
+## Quick start
+
+```python
+from swiggy_scraper import SwiggyScraper
+
+scraper = SwiggyScraper(token="YOUR_SCRAPEDO_TOKEN", city="mumbai")  # or lat=..., lng=...
+
+dishes = scraper.search_dishes("biryani")          # 1 request
+for d in sorted(dishes, key=lambda d: d["price_inr"])[:3]:
+    print(d["price_inr"], d["name"], d["restaurant_name"], d["delivery_time_min"])
+
+menu = scraper.get_menu("https://www.swiggy.com/city/bangalore/starbucks-coffee-ashok-nagar-rest124178")
+print(menu["restaurant"]["fssai_license"], len(menu["items"]))
+```
+
+### Command line
+
+```bash
+export SCRAPEDO_TOKEN=your_token
+
+# every biryani on Swiggy for a Mumbai address
+swiggy-scraper dishes biryani --city mumbai -f csv -o swiggy-biryani.csv
+
+# pizza places that deliver to central Bengaluru
+swiggy-scraper restaurants pizza -f csv -o swiggy-pizza.csv
+
+# restaurants on the home page for any address
+swiggy-scraper nearby --lat 12.9352 --lng 77.6245 -f csv -o swiggy-nearby.csv
+
+# full menus of those restaurants
+swiggy-scraper menu -i swiggy-nearby.csv --limit 5 -f csv -o swiggy-menus.csv
+```
+
+City presets: `bengaluru` (default), `chennai`, `delhi`, `gurugram`, `hyderabad`, `kolkata`, `mumbai`, `pune`.
+
+### Output
+
+Dish row (from the fictional test data in [`tests/fixtures`](tests/fixtures/menu.json)):
+
+```json
+{"item_id": "9001", "name": "Paneer Dum Biryani", "category": "Biryani", "price_inr": 249, "is_veg": true,
+ "in_stock": true, "bestseller": true, "rating": 4.4, "rating_count": 210,
+ "variant_prices_inr": "Half 249; Full 389", "restaurant_id": "5001", "restaurant_name": "Example Biryani House"}
+```
+
+Restaurant columns: `restaurant_id, name, area, locality, cuisines, rating, rating_count, cost_for_two_inr, delivery_time_min, distance_km, serviceable, is_open, promoted, veg_only, url` (menus add `fssai_license` and `address`).
+
+## How it works
+
+**The page is a shell.** A restaurant page (`/city/bangalore/…-rest124178`) is 78 KB of HTML with no menu in it. The browser fills it from JSON endpoints under `/dapi`, and this package calls those directly:
+
+| Data | Endpoint |
+| --- | --- |
+| Dish / restaurant search | `/dapi/restaurants/search/v3?lat&lng&str=biryani&selectedPLTab=DISH` (or `RESTAURANT`) |
+| Home page restaurants | `/dapi/restaurants/list/v5?lat&lng` |
+| Full menu | `/dapi/menu/pl?page-type=REGULAR_MENU&complete-menu=true&lat&lng&restaurantId=…` |
+
+Every response is a list of widget cards with an `@type`; the parsers keep the restaurant, dish and licence cards and skip banners, filters and SEO blocks.
+
+**Money is in paise.** `"price": 42000` is Rs 420. Some dishes have no `price`, only `defaultPrice`. Variant prices come either as absolute prices (`variantsV2.pricingModels`) or as a surcharge on the base price (`variants`); both are turned into rupees.
+
+**The menu repeats itself.** The "Recommended" section lists dishes that also appear in their own sections: Starbucks in Bengaluru has 168 dish cards and 148 dishes. Each dish is kept once.
+
+**Location decides the restaurants.** `lat`/`lng` decide which restaurants deliver, how far they are and how long delivery takes. The same "biryani" search returned 50 restaurants in Bengaluru and 50 in Mumbai with none in common. A menu still loads from a far-away location, with `serviceable: false`.
+
+**Some results are ads.** Search mixes in paid placements; they carry `promoted: true` so you can drop them or keep them.
+
+## Responsible use
+
+The package calls the JSON endpoints Swiggy's own web pages use for search, the home page and menus. Swiggy's robots.txt allows these paths and disallows account, cart, checkout, order tracking and payment pages, which this package never touches. It collects restaurant and menu data, not personal data. Read Swiggy's terms for your use case and keep request rates modest.
+
+## License
+
+MIT. Not affiliated with Swiggy.
+
+---
+
+Built with [Scrape.do](https://scrape.do/?utm_source=github&utm_medium=readme&utm_campaign=swiggy-scraper). [Get 1000 free credits](https://dashboard.scrape.do/sign-up?utm_source=github&utm_medium=readme&utm_campaign=swiggy-scraper).

swiggy_scraper-0.1.0/examples/cheapest_dish.py

@@ -0,0 +1,19 @@
+"""Cheapest delivered biryani in two cities, from one search request each.
+
+    export SCRAPEDO_TOKEN=your_token
+    python examples/cheapest_dish.py biryani
+"""
+import sys
+
+from swiggy_scraper import SwiggyScraper
+
+query = sys.argv[1] if len(sys.argv) > 1 else "biryani"
+for city in ("bengaluru", "mumbai"):
+    scraper = SwiggyScraper(city=city)
+    # search also returns sides (raita, pappadam): keep dishes that carry the query in their name
+    dishes = [d for d in scraper.search_dishes(query) if d["in_stock"] and d["price_inr"] and query.lower() in d["name"].lower()]
+    restaurants = {d["restaurant_id"] for d in dishes}
+    cheapest = sorted(dishes, key=lambda d: d["price_inr"])[:3]
+    print(f"\n{city}: {len(dishes)} dishes from {len(restaurants)} restaurants")
+    for d in cheapest:
+        print(f"  Rs {d['price_inr']:>6}  {d['name']}  ({d['restaurant_name']}, {d['delivery_time_min']} min)")

swiggy_scraper-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "swiggy-scraper"
+version = "0.1.0"
+description = "Scrape Swiggy dishes, restaurants and full menus (prices, variants, ratings, delivery time, FSSAI licence) with Python."
+readme = "README.md"
+requires-python = ">=3.8"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Buğrahan Saka" }]
+keywords = ["swiggy", "web scraping", "food delivery", "restaurant menu", "scraper", "aws waf", "india"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Topic :: Internet :: WWW/HTTP :: Indexing/Search",
+]
+dependencies = ["requests>=2.25"]
+
+[project.urls]
+Homepage = "https://github.com/bugrahansss/swiggy-scraper"
+Source = "https://github.com/bugrahansss/swiggy-scraper"
+Issues = "https://github.com/bugrahansss/swiggy-scraper/issues"
+"Scrape.do" = "https://scrape.do/?utm_source=pypi&utm_medium=package&utm_campaign=swiggy-scraper"
+
+[project.scripts]
+swiggy-scraper = "swiggy_scraper.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

swiggy_scraper-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

swiggy_scraper-0.1.0/src/swiggy_scraper/__init__.py

@@ -0,0 +1,36 @@
+"""Scrape Swiggy dishes, restaurants and full menus with Python, through the Scrape.do API."""
+
+from .client import CITIES, ScrapeError, SwiggyScraper
+from .parser import (
+    DISH_FIELDS,
+    RESTAURANT_FIELDS,
+    parse_count,
+    parse_item,
+    parse_listing,
+    parse_menu,
+    parse_restaurant,
+    parse_search_dishes,
+    parse_search_restaurants,
+    restaurant_id_from_url,
+    rupees,
+)
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "CITIES",
+    "DISH_FIELDS",
+    "RESTAURANT_FIELDS",
+    "ScrapeError",
+    "SwiggyScraper",
+    "parse_count",
+    "parse_item",
+    "parse_listing",
+    "parse_menu",
+    "parse_restaurant",
+    "parse_search_dishes",
+    "parse_search_restaurants",
+    "restaurant_id_from_url",
+    "rupees",
+    "__version__",
+]

swiggy_scraper-0.1.0/src/swiggy_scraper/cli.py

@@ -0,0 +1,118 @@
+"""Command line entry point: ``swiggy-scraper dishes|restaurants|nearby|menu``."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import sys
+from typing import List, Optional
+
+from . import __version__
+from .client import CITIES, TOKEN_ENV, ScrapeError, SwiggyScraper
+from .parser import DISH_FIELDS, RESTAURANT_FIELDS
+
+MENU_FIELDS = DISH_FIELDS
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="swiggy-scraper",
+        description="Scrape Swiggy dishes, restaurants and full menus (prices, ratings, delivery time) into JSON or CSV.",
+    )
+    parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    dishes = sub.add_parser("dishes", help="Search dishes: every matching dish with price and restaurant")
+    dishes.add_argument("query", help='What to search, e.g. "biryani"')
+    rest = sub.add_parser("restaurants", help="Search restaurants that deliver to the location")
+    rest.add_argument("query", help='What to search, e.g. "pizza"')
+    sub.add_parser("nearby", help="Restaurants on the Swiggy home page for the location")
+    menu = sub.add_parser("menu", help="Full menus: every dish with price, variants, veg, stock, rating")
+    menu.add_argument("urls", nargs="*", help="Swiggy restaurant URLs or ids")
+    menu.add_argument("-i", "--input", help="Text file with one URL per line, or a CSV with a 'url' column")
+    menu.add_argument("--limit", type=int, default=0, help="Stop after this many restaurants (0 = all)")
+
+    for p in (dishes, rest, sub.choices["nearby"], menu):
+        p.add_argument("-t", "--token", help=f"Scrape.do API token (default: ${TOKEN_ENV})")
+        p.add_argument("--city", choices=sorted(CITIES), help="Delivery location preset (default: bengaluru)")
+        p.add_argument("--lat", type=float, help="Delivery latitude (with --lng)")
+        p.add_argument("--lng", type=float, help="Delivery longitude (with --lat)")
+        p.add_argument("-f", "--format", choices=("json", "csv"), default="json")
+        p.add_argument("-o", "--output", help="Write to this file instead of stdout")
+    return parser
+
+
+def _read_urls(args: argparse.Namespace) -> List[str]:
+    urls = list(args.urls)
+    if args.input:
+        with open(args.input, encoding="utf-8", newline="") as handle:
+            if args.input.endswith(".csv"):
+                urls.extend(row["url"] for row in csv.DictReader(handle) if row.get("url"))
+            else:
+                urls.extend(line.strip() for line in handle if line.strip() and not line.startswith("#"))
+    return urls[: args.limit] if args.limit else urls
+
+
+def _write(rows: List[dict], fields: List[str], fmt: str, stream) -> None:
+    if fmt == "csv":
+        writer = csv.DictWriter(stream, fieldnames=fields, extrasaction="ignore")
+        writer.writeheader()
+        writer.writerows(rows)
+    else:
+        json.dump(rows, stream, indent=2, ensure_ascii=False)
+        stream.write("\n")
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    args = build_parser().parse_args(argv)
+    if (args.lat is None) != (args.lng is None):
+        print("Pass both --lat and --lng.", file=sys.stderr)
+        return 2
+    try:
+        scraper = SwiggyScraper(token=args.token, lat=args.lat, lng=args.lng, city=args.city)
+    except ValueError as exc:
+        print(exc, file=sys.stderr)
+        return 2
+
+    failures = 0
+    rows: List[dict] = []
+    try:
+        if args.command == "dishes":
+            fields, rows = DISH_FIELDS + ["restaurant_rating", "delivery_time_min", "restaurant_promoted"], scraper.search_dishes(args.query)
+        elif args.command == "restaurants":
+            fields, rows = RESTAURANT_FIELDS, scraper.search_restaurants(args.query)
+        elif args.command == "nearby":
+            fields, rows = RESTAURANT_FIELDS, scraper.get_restaurants()
+        else:
+            fields = MENU_FIELDS
+            urls = _read_urls(args)
+            if not urls:
+                print("No URLs given. Pass restaurant URLs or --input FILE.", file=sys.stderr)
+                return 2
+            for url in urls:
+                try:
+                    menu = scraper.get_menu(url)
+                except (ScrapeError, ValueError) as exc:
+                    failures += 1
+                    print(f"fail  {url}: {exc}", file=sys.stderr)
+                    continue
+                rows.extend(menu["items"])
+                r = menu["restaurant"]
+                print(f"ok    {r['name']} ({r['area']}): {len(menu['items'])} dishes, FSSAI {r['fssai_license']}", file=sys.stderr)
+    except (ScrapeError, ValueError) as exc:
+        print(f"fail  {exc}", file=sys.stderr)
+        return 1
+
+    print(f"{len(rows)} rows, {scraper.credits_used} credits", file=sys.stderr)
+    if args.output:
+        with open(args.output, "w", encoding="utf-8", newline="" if args.format == "csv" else None) as handle:
+            _write(rows, fields, args.format, handle)
+        print(f"Saved {len(rows)} rows to {args.output}", file=sys.stderr)
+    else:
+        _write(rows, fields, args.format, sys.stdout)
+    return 1 if failures and not rows else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

swiggy_scraper-0.1.0/src/swiggy_scraper/client.py

@@ -0,0 +1,164 @@
+"""Fetch Swiggy's JSON APIs through the Scrape.do API.
+
+Swiggy sits behind AWS WAF. A script with a browser user agent gets HTTP 202
+with an empty body and ``x-amzn-waf-action: challenge``; with the default
+``requests`` user agent it gets a 403 page. Through Scrape.do a plain request
+from India (``geoCode=in``) returns the JSON, for 1 credit.
+
+Everything depends on the delivery location (``lat``/``lng`` in the query):
+which restaurants deliver, how far they are and how long delivery takes.
+Without a location this package uses central Bengaluru.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import time
+from typing import Any, Dict, List, Optional
+from urllib.parse import urlencode
+
+import requests
+
+from .parser import (
+    SWIGGY, parse_listing, parse_menu, parse_search_dishes, parse_search_restaurants, restaurant_id_from_url,
+)
+
+API_URL = "https://api.scrape.do/"
+TOKEN_ENV = "SCRAPEDO_TOKEN"
+
+CITIES = {
+    "bengaluru": (12.9715987, 77.5945627),
+    "mumbai": (19.0760, 72.8777),
+    "delhi": (28.6315, 77.2167),
+    "gurugram": (28.4595, 77.0266),
+    "hyderabad": (17.3850, 78.4867),
+    "pune": (18.5204, 73.8567),
+    "chennai": (13.0827, 80.2707),
+    "kolkata": (22.5726, 88.3639),
+}
+DEFAULT_CITY = "bengaluru"
+
+
+class ScrapeError(RuntimeError):
+    """A request that did not return usable data. ``from_target`` is True when the status came from Swiggy."""
+
+    def __init__(self, message: str, status_code: Optional[int] = None, from_target: bool = False) -> None:
+        super().__init__(message)
+        self.status_code = status_code
+        self.from_target = from_target
+
+
+class SwiggyScraper:
+    """Swiggy search, restaurant lists and full menus, for one delivery location.
+
+    >>> scraper = SwiggyScraper(token="...", city="mumbai")
+    >>> dishes = scraper.search_dishes("biryani")
+    >>> menu = scraper.get_menu("https://www.swiggy.com/city/bangalore/starbucks-coffee-ashok-nagar-rest124178")
+    """
+
+    def __init__(
+        self,
+        token: Optional[str] = None,
+        lat: Optional[float] = None,
+        lng: Optional[float] = None,
+        city: Optional[str] = None,
+        timeout: float = 60,
+        retries: int = 2,
+        session: Optional[requests.Session] = None,
+    ) -> None:
+        self.token = (token or os.getenv(TOKEN_ENV) or "").strip()
+        if not self.token:
+            raise ValueError(f"A Scrape.do API token is required (pass token= or set {TOKEN_ENV}).")
+        if lat is None or lng is None:
+            key = (city or DEFAULT_CITY).strip().lower()
+            if key in ("bangalore", "bengaluru"):
+                key = "bengaluru"
+            if key not in CITIES:
+                raise ValueError(f"Unknown city {city!r}. Known: {', '.join(sorted(CITIES))}. Or pass lat and lng.")
+            lat, lng = CITIES[key]
+        self.lat = float(lat)
+        self.lng = float(lng)
+        self.timeout = timeout
+        self.retries = max(0, retries)
+        self.session = session or requests.Session()
+        self.credits_used = 0
+
+    # ------------------------------------------------------------ transport
+
+    def fetch_json(self, path: str, **query: Any) -> Dict[str, Any]:
+        """GET a Swiggy ``/dapi`` endpoint through Scrape.do and return its ``data`` object."""
+        url = f"{SWIGGY}{path}?{urlencode(query)}"
+        params = {"token": self.token, "url": url, "geoCode": "in"}
+        last: Optional[ScrapeError] = None
+        for attempt in range(self.retries + 1):
+            if attempt:
+                time.sleep(1.5 * attempt)
+            try:
+                response = self.session.get(API_URL, params=params, timeout=self.timeout)
+            except requests.RequestException as exc:
+                last = ScrapeError(f"Request failed: {str(exc).replace(self.token, '***')}")
+                continue
+            from_target = "scrape.do-initial-status-code" in {k.lower() for k in response.headers}
+            try:
+                self.credits_used += int(response.headers.get("scrape.do-request-cost") or 0)
+            except ValueError:
+                pass
+            if response.status_code == 401 and not from_target:
+                raise ScrapeError("Scrape.do rejected the token (HTTP 401).", 401, False)
+            if response.status_code == 200:
+                try:
+                    body = response.json()
+                except ValueError:
+                    body = None
+                if isinstance(body, dict) and body.get("statusCode") == 0 and isinstance(body.get("data"), dict):
+                    return body["data"]
+                message = (body or {}).get("statusMessage") if isinstance(body, dict) else None
+                last = ScrapeError(f"Swiggy returned no data for {url}" + (f": {message}" if message else ""), 200, True)
+                if isinstance(body, dict) and body.get("statusCode") not in (None, 0):
+                    break
+                continue
+            last = ScrapeError(f"HTTP {response.status_code} for {url}", response.status_code, from_target)
+            if response.status_code not in (429, 500, 502, 503, 504):
+                break
+        raise last or ScrapeError(f"Request failed: {url}")
+
+    def _search(self, query: str, tab: str) -> Dict[str, Any]:
+        if not str(query or "").strip():
+            raise ValueError("Search query is empty.")
+        return self.fetch_json(
+            "/dapi/restaurants/search/v3", lat=self.lat, lng=self.lng, str=query.strip(),
+            trackingId="undefined", submitAction="ENTER", selectedPLTab=tab,
+        )
+
+    # ------------------------------------------------------------- scraping
+
+    def search_dishes(self, query: str) -> List[Dict[str, Any]]:
+        """Dishes matching ``query`` from restaurants that deliver here: price, veg, rating, variants,
+        restaurant. One request (about 50 restaurants, every matching dish of each)."""
+        return parse_search_dishes(self._search(query, "DISH"))
+
+    def search_restaurants(self, query: str) -> List[Dict[str, Any]]:
+        """Restaurants matching ``query`` that deliver here. One request, ~100 restaurants."""
+        return parse_search_restaurants(self._search(query, "RESTAURANT"))
+
+    def get_restaurants(self) -> List[Dict[str, Any]]:
+        """The restaurants on Swiggy's home page for this location (top brands and the main grid)."""
+        return parse_listing(self.fetch_json(
+            "/dapi/restaurants/list/v5", lat=self.lat, lng=self.lng,
+            **{"is-seo-homepage-enabled": "true", "page_type": "DESKTOP_WEB_LISTING"},
+        ))
+
+    def get_menu(self, url_or_id: Any) -> Dict[str, Any]:
+        """Full menu of a restaurant: ``{"restaurant": {...}, "items": [...]}``. The restaurant row includes
+        the FSSAI licence, address, and whether it delivers to this location."""
+        restaurant_id = restaurant_id_from_url(str(url_or_id))
+        if not restaurant_id:
+            raise ValueError("Expected a Swiggy restaurant URL (…-rest<ID> or /restaurants/…-<ID>) or a restaurant id.")
+        menu = parse_menu(self.fetch_json(
+            "/dapi/menu/pl", **{"page-type": "REGULAR_MENU", "complete-menu": "true"},
+            lat=self.lat, lng=self.lng, restaurantId=restaurant_id,
+        ))
+        if not menu["restaurant"].get("restaurant_id"):
+            raise ScrapeError(f"No restaurant {restaurant_id} on Swiggy.", 200, True)
+        return menu