scrapy-crawio 1.1.0__tar.gz

scrapy_crawio-1.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Crawio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

scrapy_crawio-1.1.0/PKG-INFO

@@ -0,0 +1,125 @@
+Metadata-Version: 2.4
+Name: scrapy-crawio
+Version: 1.1.0
+Summary: Scrapy downloader middleware for Crawio: route requests through the Crawio API, with proxies and anti-bot handling included.
+Author-email: Crawio <support@crawio.com>
+License-Expression: MIT
+Project-URL: Homepage, https://crawio.com
+Project-URL: Documentation, https://crawio.com/docs
+Keywords: scrapy,scraping,proxy,crawler,anti-bot,crawio
+Classifier: Development Status :: 4 - Beta
+Classifier: Framework :: Scrapy
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Internet :: WWW/HTTP
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: scrapy>=2.6
+Requires-Dist: requests>=2.25
+Dynamic: license-file
+
+# scrapy-crawio
+
+Scrapy downloader middleware for [Crawio](https://crawio.com).
+
+Your spiders make normal requests; this middleware routes them through the Crawio API, which
+fetches each one from a healthy exit — using the cheapest route that works, up to a full browser
+when the site's anti-bot needs one — and returns the site's own response. No spider changes, no
+per-request proxy juggling. Only successful requests are billed.
+
+## Install
+
+```bash
+pip install scrapy-crawio
+```
+
+## Configure (`settings.py`)
+
+```python
+DOWNLOADER_MIDDLEWARES = {
+    "scrapy_crawio.CrawioMiddleware": 1000,
+}
+
+CRAWIO_API_KEY = "rk-..."                   # required — dashboard -> API Key
+CRAWIO_DOMAINS = ["example.com"]            # hosts to route; omit to route every request
+CRAWIO_API_URL = "https://api.crawio.com"   # optional
+CRAWIO_TIMEOUT = 120                        # seconds, optional
+CRAWIO_SESSION = "job-1"                    # optional: default sticky session id
+
+# Recommended: let Scrapy retry the transient answers.
+RETRY_ENABLED = True
+RETRY_HTTP_CODES = [429, 502, 503, 504]
+```
+
+## Use
+
+Write spiders exactly as normal — the matched hosts go through Crawio:
+
+```python
+import scrapy
+
+
+class BooksSpider(scrapy.Spider):
+    name = "books"
+
+    def start_requests(self):
+        yield scrapy.Request("https://example.com/catalogue/page-1.html", callback=self.parse)
+
+    def parse(self, response):
+        self.logger.info("fetched in %s ms", response.headers.get("X-Crawio-Ms", b"").decode())
+        for href in response.css("h3 a::attr(href)").getall():
+            yield response.follow(href, callback=self.parse_item)
+```
+
+A page or JSON arrives as a `TextResponse`. A file (PDF, image, archive) arrives as a plain
+`scrapy.http.Response` with its original bytes in `response.body`.
+
+## Response headers
+
+Every response through Crawio carries these, alongside the site's own headers:
+
+| Header | Meaning |
+|---|---|
+| `X-Crawio-Request-Id` | The request's reference (`req_...`). Quote it when you contact support. |
+| `X-Crawio-Cost` | Credits this request used: `1`, or `0` when it was not billed (blocks, errors, timeouts). |
+| `X-Crawio-Ms` | Crawio's own time for the fetch, in milliseconds. |
+
+Error responses (429, 502, 503, 504) carry `X-Crawio-Request-Id` and `X-Crawio-Cost` too, and a 429
+carries `Retry-After`.
+
+## Per-request overrides
+
+```python
+scrapy.Request(url, meta={"crawio_skip": True})       # download this one directly, not through us
+scrapy.Request(url, meta={"crawio_session": "job-2"}) # pin to a sticky session
+```
+
+A sticky session keeps the same exit and the same cookies across requests: sign in once, reuse the
+id, and the following requests stay signed in.
+
+## Retries and billing
+
+Every request carries an `Idempotency-Key`, so a retry reattaches to the same job instead of
+starting — and billing — a second one. The key is stamped on the `Request`, so Scrapy's own
+`RetryMiddleware` reuses it.
+
+## What Crawio answers
+
+| code | meaning |
+|------|---------|
+| 200 | success — the site's response is in the body |
+| 401 | invalid API key; the request is dropped, not retried |
+| 403 | account suspended; fix the account, do not retry |
+| 410 | a strict sticky session lost its exit — sign in again on a new session |
+| 429 | a plan limit: the body names which (`rate_limited`, `concurrency_limit`, `daily_quota_exceeded`, `monthly_quota_exceeded`, `credits_used_up`) and `Retry-After` says when, where the wait is known |
+| 502 | the site blocked the request behind its anti-bot |
+| 503 | no capacity right now; retry |
+| 504 | the fetch timed out |
+
+Everything except 401 comes back as a real `Response`, so `RetryMiddleware` decides what to do with
+it and the body tells you why.
+
+## License
+
+MIT

scrapy_crawio-1.1.0/README.md

@@ -0,0 +1,104 @@
+# scrapy-crawio
+
+Scrapy downloader middleware for [Crawio](https://crawio.com).
+
+Your spiders make normal requests; this middleware routes them through the Crawio API, which
+fetches each one from a healthy exit — using the cheapest route that works, up to a full browser
+when the site's anti-bot needs one — and returns the site's own response. No spider changes, no
+per-request proxy juggling. Only successful requests are billed.
+
+## Install
+
+```bash
+pip install scrapy-crawio
+```
+
+## Configure (`settings.py`)
+
+```python
+DOWNLOADER_MIDDLEWARES = {
+    "scrapy_crawio.CrawioMiddleware": 1000,
+}
+
+CRAWIO_API_KEY = "rk-..."                   # required — dashboard -> API Key
+CRAWIO_DOMAINS = ["example.com"]            # hosts to route; omit to route every request
+CRAWIO_API_URL = "https://api.crawio.com"   # optional
+CRAWIO_TIMEOUT = 120                        # seconds, optional
+CRAWIO_SESSION = "job-1"                    # optional: default sticky session id
+
+# Recommended: let Scrapy retry the transient answers.
+RETRY_ENABLED = True
+RETRY_HTTP_CODES = [429, 502, 503, 504]
+```
+
+## Use
+
+Write spiders exactly as normal — the matched hosts go through Crawio:
+
+```python
+import scrapy
+
+
+class BooksSpider(scrapy.Spider):
+    name = "books"
+
+    def start_requests(self):
+        yield scrapy.Request("https://example.com/catalogue/page-1.html", callback=self.parse)
+
+    def parse(self, response):
+        self.logger.info("fetched in %s ms", response.headers.get("X-Crawio-Ms", b"").decode())
+        for href in response.css("h3 a::attr(href)").getall():
+            yield response.follow(href, callback=self.parse_item)
+```
+
+A page or JSON arrives as a `TextResponse`. A file (PDF, image, archive) arrives as a plain
+`scrapy.http.Response` with its original bytes in `response.body`.
+
+## Response headers
+
+Every response through Crawio carries these, alongside the site's own headers:
+
+| Header | Meaning |
+|---|---|
+| `X-Crawio-Request-Id` | The request's reference (`req_...`). Quote it when you contact support. |
+| `X-Crawio-Cost` | Credits this request used: `1`, or `0` when it was not billed (blocks, errors, timeouts). |
+| `X-Crawio-Ms` | Crawio's own time for the fetch, in milliseconds. |
+
+Error responses (429, 502, 503, 504) carry `X-Crawio-Request-Id` and `X-Crawio-Cost` too, and a 429
+carries `Retry-After`.
+
+## Per-request overrides
+
+```python
+scrapy.Request(url, meta={"crawio_skip": True})       # download this one directly, not through us
+scrapy.Request(url, meta={"crawio_session": "job-2"}) # pin to a sticky session
+```
+
+A sticky session keeps the same exit and the same cookies across requests: sign in once, reuse the
+id, and the following requests stay signed in.
+
+## Retries and billing
+
+Every request carries an `Idempotency-Key`, so a retry reattaches to the same job instead of
+starting — and billing — a second one. The key is stamped on the `Request`, so Scrapy's own
+`RetryMiddleware` reuses it.
+
+## What Crawio answers
+
+| code | meaning |
+|------|---------|
+| 200 | success — the site's response is in the body |
+| 401 | invalid API key; the request is dropped, not retried |
+| 403 | account suspended; fix the account, do not retry |
+| 410 | a strict sticky session lost its exit — sign in again on a new session |
+| 429 | a plan limit: the body names which (`rate_limited`, `concurrency_limit`, `daily_quota_exceeded`, `monthly_quota_exceeded`, `credits_used_up`) and `Retry-After` says when, where the wait is known |
+| 502 | the site blocked the request behind its anti-bot |
+| 503 | no capacity right now; retry |
+| 504 | the fetch timed out |
+
+Everything except 401 comes back as a real `Response`, so `RetryMiddleware` decides what to do with
+it and the body tells you why.
+
+## License
+
+MIT

scrapy_crawio-1.1.0/pyproject.toml

@@ -0,0 +1,35 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "scrapy-crawio"
+dynamic = ["version"]
+description = "Scrapy downloader middleware for Crawio: route requests through the Crawio API, with proxies and anti-bot handling included."
+readme = "README.md"
+requires-python = ">=3.9"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Crawio", email = "support@crawio.com" }]
+keywords = ["scrapy", "scraping", "proxy", "crawler", "anti-bot", "crawio"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Framework :: Scrapy",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Topic :: Internet :: WWW/HTTP",
+]
+dependencies = [
+    "scrapy>=2.6",
+    "requests>=2.25",
+]
+
+[project.urls]
+Homepage = "https://crawio.com"
+Documentation = "https://crawio.com/docs"
+
+[tool.setuptools.dynamic]
+version = { attr = "scrapy_crawio.__version__" }
+
+[tool.setuptools.packages.find]
+where = ["src"]

scrapy_crawio-1.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

scrapy_crawio-1.1.0/src/scrapy_crawio/__init__.py

@@ -0,0 +1,5 @@
+"""Scrapy downloader middleware for Crawio — https://crawio.com"""
+from .middleware import CrawioMiddleware
+
+__all__ = ["CrawioMiddleware"]
+__version__ = "1.1.0"

scrapy_crawio-1.1.0/src/scrapy_crawio/middleware.py

@@ -0,0 +1,161 @@
+"""
+Scrapy downloader middleware for Crawio (https://crawio.com).
+
+A matched request is POSTed to Crawio's /scrape endpoint with your API key. Crawio fetches it
+through a healthy exit, handles the site's anti-bot, and returns the site's own response, which is
+handed back to Scrapy as if it had been downloaded directly. Spiders need no changes.
+
+settings.py:
+    DOWNLOADER_MIDDLEWARES = {"scrapy_crawio.CrawioMiddleware": 1000}
+    CRAWIO_API_KEY = "rk-..."                   # required
+    CRAWIO_DOMAINS = ["example.com"]            # hosts to route; omit to route every request
+    CRAWIO_API_URL = "https://api.crawio.com"   # optional
+    CRAWIO_TIMEOUT = 120                        # seconds, optional
+    CRAWIO_SESSION = "job-1"                    # optional default sticky session id
+
+Per-request, through Request.meta:
+    meta={"crawio_skip": True}          # download this one directly, not through Crawio
+    meta={"crawio_session": "job-2"}    # pin to a sticky session: sign in once, reuse the id and
+                                        # the cookies are kept for you across requests
+
+Every proxied response carries three headers of Crawio's own:
+    X-Crawio-Request-Id   the request's reference (req_...): quote it to support
+    X-Crawio-Cost         credits this request used, 1 or 0 (blocks, errors and timeouts are 0)
+    X-Crawio-Ms           Crawio's own time for the fetch, in milliseconds
+Error responses (429, 502, 503, 504...) carry the request id and cost too.
+
+Only successful requests are billed. Blocks, errors and timeouts are free.
+"""
+import base64
+import logging
+import uuid
+
+import requests
+from scrapy.exceptions import IgnoreRequest, NotConfigured
+from scrapy.http import Response, TextResponse
+from twisted.internet.threads import deferToThread
+
+log = logging.getLogger(__name__)
+
+DEFAULT_API_URL = "https://api.crawio.com"
+
+# Response headers that would misdescribe the already-decoded body.
+_DROP = {"content-encoding", "content-length", "transfer-encoding", "connection"}
+# Crawio's own response headers, passed through on every response (the API sends them since 2026-09-18)
+_CRAWIO_HEADERS = ("X-Crawio-Request-Id", "X-Crawio-Cost")
+
+
+class CrawioMiddleware:
+    def __init__(self, api_key, url=DEFAULT_API_URL, domains=(), timeout=120, session=None):
+        self.url = (url or DEFAULT_API_URL).rstrip("/")
+        self.api_key = api_key
+        self.domains = tuple(d.lower().lstrip(".") for d in domains if d)
+        self.timeout = float(timeout)
+        self.session = session   # default sticky session id (per-request meta overrides)
+
+    @classmethod
+    def from_crawler(cls, crawler):
+        s = crawler.settings
+        key = s.get("CRAWIO_API_KEY")
+        if not key:
+            raise NotConfigured("CRAWIO_API_KEY is required (dashboard -> API Key)")
+        return cls(
+            api_key=key,
+            url=s.get("CRAWIO_API_URL", DEFAULT_API_URL),
+            # No default host list: an empty CRAWIO_DOMAINS routes every request through Crawio,
+            # which is what installing the middleware asks for. Narrow it when only some hosts
+            # need us.
+            domains=s.getlist("CRAWIO_DOMAINS", []),
+            timeout=s.getfloat("CRAWIO_TIMEOUT", 120),
+            session=s.get("CRAWIO_SESSION") or None,
+        )
+
+    def _host(self, url: str) -> str:
+        return (url.split("/")[2].split("@")[-1].split(":")[0] if "://" in url else "").lower()
+
+    def _should_handle(self, request) -> bool:
+        if request.meta.get("crawio_skip"):
+            return False
+        if not self.domains:
+            return True
+        host = self._host(request.url)
+        return any(host == d or host.endswith("." + d) for d in self.domains)
+
+    def process_request(self, request, spider):
+        if not self._should_handle(request):
+            return None  # download normally
+        return deferToThread(self._fetch, request, spider)  # blocking POST off the reactor
+
+    def _fetch(self, request, spider):
+        payload = {
+            "method": request.method,
+            "url": request.url,
+            "headers": dict(request.headers.to_unicode_dict()),
+            "body": request.body.decode("utf-8", "replace") if request.body else None,
+            # Crawio's own budget sits just under our socket timeout, so a slow fetch comes back
+            # as a documented 504 rather than a dead connection.
+            "timeout": max(10, self.timeout - 10),
+        }
+        headers = {"X-API-Key": self.api_key}
+        session_id = request.meta.get("crawio_session") or self.session
+        if session_id:
+            headers["X-Session-Id"] = str(session_id)   # sticky session, cookies kept for you
+        # A retry (Scrapy's RetryMiddleware, or a severed connection) reattaches to the SAME job
+        # instead of starting — and billing — a second one. The key is stamped on the Request so
+        # every retry of it carries the same value.
+        idem = request.meta.setdefault("crawio_idempotency_key", uuid.uuid4().hex)
+        headers["Idempotency-Key"] = idem
+
+        try:
+            resp = requests.post(f"{self.url}/scrape", json=payload,
+                                 headers=headers, timeout=self.timeout)
+        except requests.RequestException as e:
+            raise IgnoreRequest(f"crawio unreachable: {e}")
+
+        if resp.status_code == 401:
+            raise IgnoreRequest("crawio rejected the API key (401): check CRAWIO_API_KEY")
+
+        if resp.status_code != 200:
+            # 403 suspended, 410 session expired, 429 a plan limit, 502 the site blocked us,
+            # 503 no capacity, 504 timeout. Surfaced as a real Response so Scrapy's own
+            # RetryMiddleware decides, and the body says which. A 429 carries Retry-After.
+            out = {"Content-Type": "application/json"}
+            for name in ("Retry-After",) + _CRAWIO_HEADERS:
+                value = resp.headers.get(name)
+                if value:
+                    out[name] = value
+            spider.logger.warning("crawio %s for %s (request %s): %s", resp.status_code, request.url,
+                                  out.get("X-Crawio-Request-Id", "-"), resp.text[:200])
+            return TextResponse(url=request.url, status=resp.status_code,
+                                request=request, body=resp.content, headers=out)
+
+        data = resp.json()
+        headers = {k: v for k, v in (data.get("headers") or {}).items()
+                   if k.lower() not in _DROP}
+        if data.get("ms") is not None:
+            headers["X-Crawio-Ms"] = str(data["ms"])
+        # the request reference and its cost: from the JSON body, else from Crawio's own headers
+        rid = data.get("request_id") or resp.headers.get("X-Crawio-Request-Id")
+        if rid:
+            headers["X-Crawio-Request-Id"] = str(rid)
+        cost = data.get("cost") if data.get("cost") is not None else resp.headers.get("X-Crawio-Cost")
+        if cost is not None:
+            headers["X-Crawio-Cost"] = str(cost)
+        url = data.get("finalUrl") or data.get("final_url") or request.url
+        if data.get("body_encoding") == "base64":
+            # a file (PDF, image, archive): Crawio sends its bytes base64 encoded
+            return Response(
+                url=url,
+                status=data.get("status", 200),
+                headers=headers,
+                body=base64.b64decode(data.get("body") or ""),
+                request=request,
+            )
+        return TextResponse(
+            url=url,
+            status=data.get("status", 200),
+            headers=headers,
+            body=(data.get("body") or "").encode("utf-8"),
+            request=request,
+            encoding="utf-8",
+        )

scrapy_crawio-1.1.0/src/scrapy_crawio.egg-info/PKG-INFO

@@ -0,0 +1,125 @@
+Metadata-Version: 2.4
+Name: scrapy-crawio
+Version: 1.1.0
+Summary: Scrapy downloader middleware for Crawio: route requests through the Crawio API, with proxies and anti-bot handling included.
+Author-email: Crawio <support@crawio.com>
+License-Expression: MIT
+Project-URL: Homepage, https://crawio.com
+Project-URL: Documentation, https://crawio.com/docs
+Keywords: scrapy,scraping,proxy,crawler,anti-bot,crawio
+Classifier: Development Status :: 4 - Beta
+Classifier: Framework :: Scrapy
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Internet :: WWW/HTTP
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: scrapy>=2.6
+Requires-Dist: requests>=2.25
+Dynamic: license-file
+
+# scrapy-crawio
+
+Scrapy downloader middleware for [Crawio](https://crawio.com).
+
+Your spiders make normal requests; this middleware routes them through the Crawio API, which
+fetches each one from a healthy exit — using the cheapest route that works, up to a full browser
+when the site's anti-bot needs one — and returns the site's own response. No spider changes, no
+per-request proxy juggling. Only successful requests are billed.
+
+## Install
+
+```bash
+pip install scrapy-crawio
+```
+
+## Configure (`settings.py`)
+
+```python
+DOWNLOADER_MIDDLEWARES = {
+    "scrapy_crawio.CrawioMiddleware": 1000,
+}
+
+CRAWIO_API_KEY = "rk-..."                   # required — dashboard -> API Key
+CRAWIO_DOMAINS = ["example.com"]            # hosts to route; omit to route every request
+CRAWIO_API_URL = "https://api.crawio.com"   # optional
+CRAWIO_TIMEOUT = 120                        # seconds, optional
+CRAWIO_SESSION = "job-1"                    # optional: default sticky session id
+
+# Recommended: let Scrapy retry the transient answers.
+RETRY_ENABLED = True
+RETRY_HTTP_CODES = [429, 502, 503, 504]
+```
+
+## Use
+
+Write spiders exactly as normal — the matched hosts go through Crawio:
+
+```python
+import scrapy
+
+
+class BooksSpider(scrapy.Spider):
+    name = "books"
+
+    def start_requests(self):
+        yield scrapy.Request("https://example.com/catalogue/page-1.html", callback=self.parse)
+
+    def parse(self, response):
+        self.logger.info("fetched in %s ms", response.headers.get("X-Crawio-Ms", b"").decode())
+        for href in response.css("h3 a::attr(href)").getall():
+            yield response.follow(href, callback=self.parse_item)
+```
+
+A page or JSON arrives as a `TextResponse`. A file (PDF, image, archive) arrives as a plain
+`scrapy.http.Response` with its original bytes in `response.body`.
+
+## Response headers
+
+Every response through Crawio carries these, alongside the site's own headers:
+
+| Header | Meaning |
+|---|---|
+| `X-Crawio-Request-Id` | The request's reference (`req_...`). Quote it when you contact support. |
+| `X-Crawio-Cost` | Credits this request used: `1`, or `0` when it was not billed (blocks, errors, timeouts). |
+| `X-Crawio-Ms` | Crawio's own time for the fetch, in milliseconds. |
+
+Error responses (429, 502, 503, 504) carry `X-Crawio-Request-Id` and `X-Crawio-Cost` too, and a 429
+carries `Retry-After`.
+
+## Per-request overrides
+
+```python
+scrapy.Request(url, meta={"crawio_skip": True})       # download this one directly, not through us
+scrapy.Request(url, meta={"crawio_session": "job-2"}) # pin to a sticky session
+```
+
+A sticky session keeps the same exit and the same cookies across requests: sign in once, reuse the
+id, and the following requests stay signed in.
+
+## Retries and billing
+
+Every request carries an `Idempotency-Key`, so a retry reattaches to the same job instead of
+starting — and billing — a second one. The key is stamped on the `Request`, so Scrapy's own
+`RetryMiddleware` reuses it.
+
+## What Crawio answers
+
+| code | meaning |
+|------|---------|
+| 200 | success — the site's response is in the body |
+| 401 | invalid API key; the request is dropped, not retried |
+| 403 | account suspended; fix the account, do not retry |
+| 410 | a strict sticky session lost its exit — sign in again on a new session |
+| 429 | a plan limit: the body names which (`rate_limited`, `concurrency_limit`, `daily_quota_exceeded`, `monthly_quota_exceeded`, `credits_used_up`) and `Retry-After` says when, where the wait is known |
+| 502 | the site blocked the request behind its anti-bot |
+| 503 | no capacity right now; retry |
+| 504 | the fetch timed out |
+
+Everything except 401 comes back as a real `Response`, so `RetryMiddleware` decides what to do with
+it and the body tells you why.
+
+## License
+
+MIT

scrapy_crawio-1.1.0/src/scrapy_crawio.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+LICENSE
+README.md
+pyproject.toml
+src/scrapy_crawio/__init__.py
+src/scrapy_crawio/middleware.py
+src/scrapy_crawio.egg-info/PKG-INFO
+src/scrapy_crawio.egg-info/SOURCES.txt
+src/scrapy_crawio.egg-info/dependency_links.txt
+src/scrapy_crawio.egg-info/requires.txt
+src/scrapy_crawio.egg-info/top_level.txt

scrapy_crawio-1.1.0/src/scrapy_crawio.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

scrapy_crawio-1.1.0/src/scrapy_crawio.egg-info/requires.txt

@@ -0,0 +1,2 @@
+scrapy>=2.6
+requests>=2.25

scrapy_crawio-1.1.0/src/scrapy_crawio.egg-info/top_level.txt

@@ -0,0 +1 @@
+scrapy_crawio