proxy-scraper-cli 1.10.0__tar.gz → 1.11.0__tar.gz

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.10.0
+Version: 1.11.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -205,7 +205,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon (no captcha, no 403 in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -219,6 +219,8 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | head
 ```
 
+**Over time:** every day the first run's `proxies.json` is kept for good as a gzipped asset of the release of that year, like [snapshots-2026](../../releases/tag/snapshots-2026) – `proxies-2026-09-28.json.gz` and so on, for anyone who wants to look at free proxies over weeks and months.
+
 Every file is also on GitHub Pages, which sits behind a CDN, isn't rate limited like raw.githubusercontent and sends CORS headers, so it works straight from the browser: `https://maximilianfeix.github.io/proxy-scraper/socks5.txt`. [jsDelivr](https://cdn.jsdelivr.net/gh/maximilianfeix/proxy-scraper@proxy-list/) works too, but can lag behind by a few hours.
 
 Or let the tool start from it: `proxy-scraper --recheck live` downloads the list and checks it again from **your** network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With `--serve` you have a rotating proxy in under a minute.
@@ -440,7 +442,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 <a id="proxy-server"></a>
 

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/README.md

@@ -164,7 +164,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon (no captcha, no 403 in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -178,6 +178,8 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | head
 ```
 
+**Over time:** every day the first run's `proxies.json` is kept for good as a gzipped asset of the release of that year, like [snapshots-2026](../../releases/tag/snapshots-2026) – `proxies-2026-09-28.json.gz` and so on, for anyone who wants to look at free proxies over weeks and months.
+
 Every file is also on GitHub Pages, which sits behind a CDN, isn't rate limited like raw.githubusercontent and sends CORS headers, so it works straight from the browser: `https://maximilianfeix.github.io/proxy-scraper/socks5.txt`. [jsDelivr](https://cdn.jsdelivr.net/gh/maximilianfeix/proxy-scraper@proxy-list/) works too, but can lag behind by a few hours.
 
 Or let the tool start from it: `proxy-scraper --recheck live` downloads the list and checks it again from **your** network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With `--serve` you have a rotating proxy in under a minute.
@@ -399,7 +401,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 <a id="proxy-server"></a>
 

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxy_scraper_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.10.0
+Version: 1.11.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -205,7 +205,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon (no captcha, no 403 in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -219,6 +219,8 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | head
 ```
 
+**Over time:** every day the first run's `proxies.json` is kept for good as a gzipped asset of the release of that year, like [snapshots-2026](../../releases/tag/snapshots-2026) – `proxies-2026-09-28.json.gz` and so on, for anyone who wants to look at free proxies over weeks and months.
+
 Every file is also on GitHub Pages, which sits behind a CDN, isn't rate limited like raw.githubusercontent and sends CORS headers, so it works straight from the browser: `https://maximilianfeix.github.io/proxy-scraper/socks5.txt`. [jsDelivr](https://cdn.jsdelivr.net/gh/maximilianfeix/proxy-scraper@proxy-list/) works too, but can lag behind by a few hours.
 
 Or let the tool start from it: `proxy-scraper --recheck live` downloads the list and checks it again from **your** network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With `--serve` you have a rotating proxy in under a minute.
@@ -440,7 +442,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 <a id="proxy-server"></a>
 

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxy_scraper_cli.egg-info/SOURCES.txt

@@ -43,6 +43,7 @@ proxyscraper/report.py
 proxyscraper/sites.py
 proxyscraper/sources.json
 proxyscraper/sources.py
+proxyscraper/speed.py
 proxyscraper/targets.py
 proxyscraper/server/__init__.py
 proxyscraper/server/core.py
@@ -104,6 +105,7 @@ tests/test_server_closed_writer.py
 tests/test_server_fixes.py
 tests/test_sites.py
 tests/test_sources_json.py
+tests/test_speed.py
 tests/test_tamper.py
 tests/test_targets.py
 tests/test_ui.py

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/__init__.py

@@ -1,7 +1,7 @@
 """proxy-scraper: collects public HTTP/SOCKS4/SOCKS5 proxies from hundreds of sources, checks them
 in parallel with its own protocol handshakes and learns which sources deliver good proxies."""
 
-__version__ = "1.10.0"
+__version__ = "1.11.0"
 
 
 def __getattr__(name):

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/agent.py

@@ -179,17 +179,18 @@ def normalize_countries(countries: Iterable[str]) -> List[str]:
 def select(rows: Iterable[dict], protocol: str = "any", countries: Iterable[str] = (), https_only: bool = False,
            elite_only: bool = False, exclude_datacenter: bool = False, exclude_blocklisted: bool = False,
            stable_only: bool = False, max_latency_ms: int = 0, run_hours: int = 1, min_uptime: int = 0,
-           works_on: Iterable[str] = ()) -> List[dict]:
+           works_on: Iterable[str] = (), min_speed_kbps: int = 0) -> List[dict]:
     """The rows that pass every filter, fastest first – the same filters as on the website."""
     return sorted(matching(rows, protocol, countries, https_only, elite_only, exclude_datacenter,
-                           exclude_blocklisted, stable_only, max_latency_ms, run_hours, min_uptime, works_on),
+                           exclude_blocklisted, stable_only, max_latency_ms, run_hours, min_uptime, works_on,
+                           min_speed_kbps),
                   key=lambda r: r.get("latency") or 0)
 
 
 def matching(rows: Iterable[dict], protocol: str = "any", countries: Iterable[str] = (), https_only: bool = False,
              elite_only: bool = False, exclude_datacenter: bool = False, exclude_blocklisted: bool = False,
              stable_only: bool = False, max_latency_ms: int = 0, run_hours: int = 1,
-             min_uptime: int = 0, works_on: Iterable[str] = ()) -> Iterator[dict]:
+             min_uptime: int = 0, works_on: Iterable[str] = (), min_speed_kbps: int = 0) -> Iterator[dict]:
     """The rows that pass every filter, in list order – checks the filters before the first row is asked for."""
     protocol = _check_protocol(protocol)
     wanted = set(normalize_countries(countries))
@@ -205,7 +206,8 @@ def matching(rows: Iterable[dict], protocol: str = "any", countries: Iterable[st
            and (not stable_only or (r.get("streak") or 0) >= stable_runs)
            and (not max_latency_ms or (r.get("latency") or 0) <= max_latency_ms)
            and (not min_uptime or (r.get("uptime_7d") or 0) >= min_uptime)
-           and all((r.get("sites") or {}).get(s) for s in sites))
+           and all((r.get("sites") or {}).get(s) for s in sites)
+           and (not min_speed_kbps or (r.get("speed_kbps") or 0) >= min_speed_kbps))
 
 
 def describe(item: Union[dict, CheckResult], run_hours: Optional[int] = None) -> dict:
@@ -229,6 +231,7 @@ def describe(item: Union[dict, CheckResult], run_hours: Optional[int] = None) ->
         "up_for_hours": streak * run_hours if streak and run_hours else None,
         "uptime_7d_percent": item.get("uptime_7d"),
         "works_on": [name for name, ok in (item.get("sites") or {}).items() if ok],
+        "speed_kbps": item.get("speed_kbps"),
     }
 
 

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/api.py

@@ -166,7 +166,8 @@ class LiveProxy(CheckResult):
     uptime_7d: Optional[int] = None   # the same over the week
     first_seen: str = ""              # ISO time of the first run it was listed in
     up_for_hours: int = 0             # listed without a gap for this long
-    sites: Dict[str, bool] = field(default_factory=dict)  # "google", "reddit", "amazon" -> got through last run?
+    sites: Dict[str, bool] = field(default_factory=dict)  # "google", "reddit", … -> got through in the last run?
+    speed_kbps: Optional[int] = None  # download speed in KiB/s in the last run (HTTPS-capable ones only)
 
 
 def _live_fetch(url: str, timeout: float = 20, headers=None):
@@ -184,19 +185,22 @@ def _live_proxy(row: dict, run_hours: int) -> LiveProxy:
         org=row.get("org") or "", hosting=row.get("hosting"), blocklisted=row.get("blocklisted"),
         uptime_24h=row.get("uptime_24h"), uptime_7d=row.get("uptime_7d"), first_seen=row.get("first_seen") or "",
         up_for_hours=streak * run_hours,
-        sites={k: v for k, v in (row.get("sites") or {}).items() if isinstance(v, bool)})
+        sites={k: v for k, v in (row.get("sites") or {}).items() if isinstance(v, bool)},
+        speed_kbps=row.get("speed_kbps") if type(row.get("speed_kbps")) is int else None)
 
 
 async def live_proxies_async(*, types: Iterable[str] = PROXY_TYPES, countries: Iterable[str] = (), https: bool = False,
                              anonymity: str = "", max_latency: int = 0, no_datacenter: bool = False,
                              no_blocklisted: bool = False, min_uptime: int = 0, works_on: Iterable[str] = (),
-                             limit: int = 0) -> List[LiveProxy]:
+                             min_speed: int = 0, limit: int = 0) -> List[LiveProxy]:
     """The proxies from the hourly list that pass the filters, fastest first. Nothing is checked here: they
     worked from GitHub's servers in the last run (at most an hour ago). find_proxies checks from your network.
 
     Same filters as find_proxies, plus
     min_uptime  only proxies listed in at least this share (percent) of the week's runs, e.g. 90
-    works_on    only proxies that got through to these sites in the last run: "google", "reddit", "amazon"
+    works_on    only proxies that got through to these sites in the last run: google, reddit, amazon, instagram,
+                tiktok
+    min_speed   only proxies that downloaded at least this many KiB/s in the last run
     limit       at most this many (0 = all)
     """
     from .agent import AgentError, LiveSource
@@ -226,7 +230,7 @@ async def live_proxies_async(*, types: Iterable[str] = PROXY_TYPES, countries: I
                     if r.get("ptype") in types and isinstance(r.get("proxy"), str)),
                    key=lambda p: p.latency)
     found = [p for p in found if opts.filters.accepts(p) and (not min_uptime or (p.uptime_7d or 0) >= min_uptime)
-             and all(p.sites.get(s) for s in works_on)]
+             and all(p.sites.get(s) for s in works_on) and (not min_speed or (p.speed_kbps or 0) >= min_speed)]
     return found[:limit] if limit else found
 
 

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/mcp_server.py

@@ -17,6 +17,7 @@ from mcp.types import ToolAnnotations
 from pydantic import BaseModel, Field
 
 from . import __version__, agent
+from .sites import SITES
 
 REPO = "https://github.com/maximilianfeix/proxy-scraper"
 
@@ -33,6 +34,7 @@ Free proxies are run by strangers: never send passwords, cookies, API keys or pe
 and expect some of them to stop working at any time."""
 
 Protocol = Literal["any", "http", "socks4", "socks5"]
+SiteName = Literal[tuple(s.name for s in SITES)]  # the sites the hourly check tries (sites.py)
 Countries = Annotated[List[str], Field(description="Two-letter country codes of the exit IP, e.g. ['DE', 'NL']. "
                                                    "Empty = any country.")]
 
@@ -51,7 +53,10 @@ class Proxy(BaseModel):
     blocklisted: Optional[bool] = Field(description="Exit IP on the SpamCop blocklist – expect captchas")
     up_for_hours: Optional[int] = Field(description="How long it has worked without a gap (live list only)")
     works_on: List[str] = Field(default_factory=list, description="Big sites it got through to in the last hourly "
-                                                                  "check: google, reddit, amazon (live list only)")
+                                                                  f"check: {', '.join(s.name for s in SITES)} "
+                                                                  "(live list only)")
+    speed_kbps: Optional[int] = Field(default=None, description="Download speed in KiB/s in the last hourly check "
+                                                                "(HTTPS-capable proxies, live list only)")
     uptime_7d_percent: Optional[int] = Field(default=None, description="Share of the last 7 days' hourly checks it "
                                                                        "passed (live list only)")
 
@@ -121,9 +126,12 @@ def build_server(live: Optional[agent.LiveSource] = None, fetcher: Optional[agen
         min_uptime_percent: Annotated[int, Field(ge=0, le=100, description="Only proxies that passed at least this "
                                                                            "share of the last 7 days' hourly checks, "
                                                                            "e.g. 90 for the most reliable")] = 0,
-        works_on: Annotated[List[Literal["google", "reddit", "amazon"]],
+        works_on: Annotated[List[SiteName],
                             Field(description="Only proxies that got through to all of these sites in the last "
                                               "hourly check (most free proxies get a captcha or a 403 there)")] = [],  # noqa: B006
+        min_speed_kbps: Annotated[int, Field(ge=0, description="Only proxies that downloaded at least this many KiB/s "
+                                                                "in the last hourly check, e.g. 300 for loading "
+                                                                "real pages")] = 0,
         limit: Annotated[int, Field(ge=1, le=100, description="How many to return (fastest first)")] = 10,
     ) -> ProxyList:
         """Get working free proxies right now, fastest first.
@@ -135,7 +143,7 @@ def build_server(live: Optional[agent.LiveSource] = None, fetcher: Optional[agen
             data = await live.get()
             rows = agent.select(data.rows, protocol, countries, https_only, elite_only, exclude_datacenter,
                                 exclude_blocklisted, stable_only, max_latency_ms, data.run_hours, min_uptime_percent,
-                                works_on)
+                                works_on, min_speed_kbps)
         except agent.AgentError as e:
             _fail(e)
         return ProxyList(proxies=[Proxy(**agent.describe(r, data.run_hours)) for r in rows[:limit]],

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/proxypages.py

@@ -14,9 +14,10 @@ from pathlib import Path
 from typing import Dict, List, Optional, Sequence, Tuple
 
 from .pages import COUNTRIES, CSS, SITE_URL, _short_name
+from .sites import SITES
 
 INDEX_MIN_UPTIME = 50  # percent of the week's runs
-SITE_NAMES = {"google": "Google", "reddit": "Reddit", "amazon": "Amazon"}
+SITE_NAMES = {s.name: s.title for s in SITES}
 
 EXTRA_CSS = """
 .wrap > * { min-width: 0; }
@@ -97,8 +98,10 @@ def _summary(entries: List[dict]) -> dict:
     https = [r.get("https") for r in entries]
     rank = {"elite": 3, "anonymous": 2, "transparent": 1}
     sites = {k for r in entries for k, ok in (r.get("sites") or {}).items() if ok}
+    speeds = [r["speed_kbps"] for r in entries if r.get("speed_kbps")]
     return {
         "uptime": max(uptimes) if uptimes else None,
+        "speed": max(speeds) if speeds else None,
         "latency": min(r["latency"] for r in entries) if all("latency" in r for r in entries) else None,
         "https": True if True in https else False if False in https else None,
         "anonymity": max((r.get("anonymity") or "" for r in entries), key=lambda a: rank.get(a, 0)) or None,
@@ -128,6 +131,7 @@ def _render(proxy: str, entries: List[dict], updated: datetime, bits: Optional[i
             ("Country", _country(cc)),
             ("Provider", f"{info['org'] or 'unknown'}" + (f" (AS{info['asn']})" if info["asn"] else "")),
             ("Latency", f"{info['latency']:,} ms" if info["latency"] is not None else "unknown"),
+            ("Download speed", f"{info['speed']:,} KiB/s" if info["speed"] else "not measured"),
             ("HTTPS", _yes_no(info["https"], "yes, verified TLS", "no")),
             ("Anonymity", info["anonymity"] or "unknown"),
         ]

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/publish.py

@@ -74,6 +74,9 @@ def stats_for(rows: List[dict], now: datetime) -> dict:
         "run_hours": RUN_HOURS,
         "countries": dict(Counter(r["country"] for r in rows if r.get("country")).most_common(15)),
         "median_latency": round(statistics.median(r["latency"] for r in rows)) if rows else 0,
+        # download speed of the HTTPS-capable ones (speed.py), None when it wasn't measured
+        "median_speed_kbps": round(statistics.median(speeds)) if (speeds := [r["speed_kbps"] for r in rows
+                                                                               if r.get("speed_kbps")]) else None,
     }
 
 
@@ -244,7 +247,8 @@ def load_streaks(path: Optional[Path]) -> Dict[str, int]:
 
 
 def publish(run_dir: Path, out: Path, minimum: int = 20, now: Optional[datetime] = None,
-            history: Optional[Path] = None, streaks: Optional[Path] = None, seen: Optional[Path] = None) -> int:
+            history: Optional[Path] = None, streaks: Optional[Path] = None, seen: Optional[Path] = None,
+            feed: Optional[Path] = None) -> int:
     rows = load_rows(run_dir)
     if len(rows) < minimum:
         print(f"Only {len(rows)} hits (< {minimum}) – the old list stays online.")
@@ -270,7 +274,7 @@ def publish(run_dir: Path, out: Path, minimum: int = 20, now: Optional[datetime]
     with (run_dir / "proxies.csv").open(newline="", encoding="utf-8") as src, \
             (out / "proxies.csv").open("w", newline="", encoding="utf-8") as dst:
         reader = csv.DictReader(src)
-        extra = ("first_seen", "uptime_24h", "uptime_7d", "works_with")
+        extra = ("first_seen", "uptime_24h", "uptime_7d", "works_with", "speed_kbps")
         base = list(reader.fieldnames or ["proxy"])
         writer = csv.DictWriter(dst, fieldnames=[*base, *(k for k in extra if k not in base)])
         writer.writeheader()
@@ -309,7 +313,7 @@ def publish(run_dir: Path, out: Path, minimum: int = 20, now: Optional[datetime]
     write_pages(rows, out, now, extra=proxy_pages)
     write_json(out / "history.json", runs)
     # the weekly report: lifetimes, sites, countries – a page, markdown and a short post (report.py)
-    write_report(weekly_facts(rows, runs, {url: e["bits"] for url, e in listed.items()}, now), out)
+    write_report(weekly_facts(rows, runs, {url: e["bits"] for url, e in listed.items()}, now), out, previous_feed=feed)
     for theme in THEMES:  # charts for the README, which embeds them from GitHub Pages
         (out / f"chart-{theme}.svg").write_text(trend_svg(runs, now, theme), encoding="utf-8")
         (out / f"countries-{theme}.svg").write_text(countries_svg(stats["countries"], theme), encoding="utf-8")
@@ -335,8 +339,10 @@ def main(argv: Optional[List[str]] = None) -> int:
     p.add_argument("--streaks", type=Path, help="streaks.json from last time (how long each proxy has been listed)")
     p.add_argument("--seen", type=Path,
                    help="seen.json from last time (which runs each proxy was listed in, for the uptime)")
+    p.add_argument("--feed", type=Path, help="report/feed.xml from last time (the weekly entries it keeps)")
     args = p.parse_args(argv)
-    return publish(args.run_dir, args.out_dir, args.min, history=args.history, streaks=args.streaks, seen=args.seen)
+    return publish(args.run_dir, args.out_dir, args.min, history=args.history, streaks=args.streaks, seen=args.seen,
+                   feed=args.feed)
 
 
 if __name__ == "__main__":

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/report.py

@@ -5,6 +5,7 @@ get through to the big sites, where they are. Rebuilt with every run from the sa
 
 from __future__ import annotations
 
+import xml.etree.ElementTree as ET
 from collections import Counter
 from datetime import datetime, timedelta
 from html import escape
@@ -13,11 +14,12 @@ from typing import Dict, List, Optional, Sequence
 
 from .charts import FONT, THEMES, WIDTH
 from .pages import COUNTRIES, CSS, SITE_URL, _short_name
+from .sites import SITES
 
 DAYS = 7
 RUN_HOURS = 1
 REPORT_URL = f"{SITE_URL}report/"
-SITE_NAMES = {"google": "Google", "reddit": "Reddit", "amazon": "Amazon"}
+SITE_NAMES = {s.name: s.title for s in SITES}
 # checks in a row a proxy was on the list for -> label (the runs are roughly hourly, but not always)
 BUCKETS = [(1, 1, "1 check"), (2, 5, "2–5 checks"), (6, 23, "6–23 checks"), (24, 71, "24–71 checks"),
            (72, 10 ** 6, "72 checks +")]
@@ -102,6 +104,7 @@ def weekly_facts(rows: List[dict], runs: Sequence[dict], seen: Dict[str, int], n
         if any(r.get("hosting") is not None for r in rows) else None,
         "countries": Counter(r["country"] for r in rows if r.get("country")).most_common(5),
         "latency_now": window[-1][1].get("median_latency") if window else None,
+        "speeds": sorted(r["speed_kbps"] for r in rows if r.get("speed_kbps")),
         "sites": sites,
     }
 
@@ -141,6 +144,11 @@ def _lines(f: dict) -> List[str]:
     line = (f"Between {f['low']:,} and {f['peak']:,} proxies worked at any one time. Right now: {f['total']:,}, "
             f"{f['https']} % of them tunnel HTTPS")
     out.append(line + (f", {f['datacenter']} % exit from a datacenter." if f["datacenter"] is not None else "."))
+    speeds = f.get("speeds") or []
+    if speeds:
+        median, top = speeds[len(speeds) // 2], speeds[9 * len(speeds) // 10]
+        out.append(f"Downloading 100 KB through them: half manage {median:,} KiB/s or more, "
+                   f"the fastest tenth over {top:,} KiB/s.")
     if f["countries"]:
         out.append("Most of them sit in " + ", ".join(f"{_country(cc)} ({n:,})" for cc, n in f["countries"]) + ".")
     return out
@@ -215,6 +223,7 @@ def page(f: dict) -> str:
 <title>The week in free proxies ({escape(_period(f))}) | proxy-scraper</title>
 <meta name="description" content="{escape(_lines(f)[0])}">
 <link rel="canonical" href="{REPORT_URL}">
+<link rel="alternate" type="application/atom+xml" title="The week in free proxies" href="{REPORT_URL}feed.xml">
 <meta property="og:title" content="The week in free proxies">
 <meta property="og:description" content="{escape(_lines(f)[0])}">
 <meta property="og:image" content="{SITE_URL}og.png">
@@ -249,6 +258,7 @@ picture img {{ width: 100%; height: auto; border-radius: 16px; }}
     <img src="{root}chart-dark.svg" alt="Working proxies over the week, by protocol">
   </picture>
   <footer><span>Want to share it? <a href="report.md">Markdown</a> · <a href="post.txt">a short post</a> ·
+  <a href="feed.xml">RSS/Atom feed</a> ·
   <a href="https://github.com/maximilianfeix/proxy-scraper">the code</a></span></footer>
 </div>
 </body>
@@ -256,11 +266,63 @@ picture img {{ width: 100%; height: auto; border-radius: 16px; }}
 """
 
 
-def write_report(f: dict, out: Path) -> None:
+FEED_WEEKS = 12
+_ATOM = "{http://www.w3.org/2005/Atom}"
+
+
+def feed(f: dict, previous: Optional[str]) -> str:
+    """Atom feed with one entry per finished ISO week. The first run of a week adds the entry for the week that just
+    ended; every later run that week hands back the feed unchanged – feed readers and Slack/Discord bridges see one
+    new entry a week, not an update every hour. `previous`: the feed from the last run (None or garbage: start over)."""
+    now = f["now"]
+    monday = (now - timedelta(days=now.weekday())).replace(hour=0, minute=0, second=0, microsecond=0)
+    year, week, _ = (monday - timedelta(days=1)).isocalendar()
+    entry_id = f"{REPORT_URL}{year}-W{week:02d}"
+    kept: List[str] = []
+    if previous:
+        try:
+            root = ET.fromstring(previous)
+            old = root.findall(f"{_ATOM}entry")
+        except ET.ParseError:
+            old = []
+        if any((e.findtext(f"{_ATOM}id") or "") == entry_id for e in old):
+            return previous  # this week's entry is out already – leave it exactly as it is
+        ET.register_namespace("", _ATOM[1:-1])
+        kept = [ET.tostring(e, encoding="unicode") for e in old][:FEED_WEEKS - 1]
+    stamp = monday.strftime("%Y-%m-%dT%H:%M:%SZ")
+    text = "\n\n".join(_lines(f))
+    new = f"""<entry>
+    <title>The week in free proxies: {escape(_period(f))}</title>
+    <link rel="alternate" href="{REPORT_URL}"/>
+    <id>{entry_id}</id>
+    <updated>{stamp}</updated>
+    <content type="text">{escape(text)}</content>
+  </entry>"""
+    body = "\n  ".join([new, *kept])
+    return f"""<?xml version="1.0" encoding="utf-8"?>
+<feed xmlns="http://www.w3.org/2005/Atom">
+  <title>The week in free proxies</title>
+  <subtitle>How long free proxies last and which sites let them through – from hourly checks by proxy-scraper</subtitle>
+  <link rel="self" href="{REPORT_URL}feed.xml"/>
+  <link rel="alternate" href="{REPORT_URL}"/>
+  <id>{REPORT_URL}</id>
+  <updated>{stamp}</updated>
+  <author><name>proxy-scraper</name></author>
+  {body}
+</feed>
+"""
+
+
+def write_report(f: dict, out: Path, previous_feed: Optional[Path] = None) -> None:
     folder = out / "report"
     folder.mkdir(parents=True, exist_ok=True)
     (folder / "index.html").write_text(page(f), encoding="utf-8")
     (folder / "report.md").write_text(markdown(f), encoding="utf-8")
     (folder / "post.txt").write_text(post(f) + "\n", encoding="utf-8")
+    try:
+        previous = previous_feed.read_text(encoding="utf-8") if previous_feed else None
+    except OSError:
+        previous = None
+    (folder / "feed.xml").write_text(feed(f, previous), encoding="utf-8")
     for theme in THEMES:
         (folder / f"lifetimes-{theme}.svg").write_text(lifetimes_svg(f, theme), encoding="utf-8")

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/site/index.html

@@ -582,7 +582,7 @@
       <button class="chip" type="button" id="f-bl" aria-pressed="false" title="Exit IP not on the SpamCop blocklist – fewer captchas">Not blocklisted</button>
       <select class="field" id="f-country" aria-label="Country"><option value="">All countries</option></select>
       <select class="field" id="f-site" aria-label="Gets through to" title="Checked every hour: most free proxies get a captcha or a 403 on these sites">
-        <option value="">Any site</option><option value="google">Gets through to Google</option><option value="reddit">Gets through to Reddit</option><option value="amazon">Gets through to Amazon</option></select>
+        <option value="">Any site</option><option value="google">Gets through to Google</option><option value="reddit">Gets through to Reddit</option><option value="amazon">Gets through to Amazon</option><option value="instagram">Gets through to Instagram</option><option value="tiktok">Gets through to TikTok</option></select>
       <label class="range">Up to <input type="range" id="f-lat" min="200" max="8000" step="100" value="8000" aria-label="Maximum latency"><output id="f-lat-v" for="f-lat">any</output></label>
       <input class="field" type="search" id="f-q" placeholder="Search IP, country or provider    /" aria-label="Search" aria-keyshortcuts="/">
     </div>
@@ -595,10 +595,10 @@
             <th scope="col" data-key="proxy"><button class="sort" type="button">Proxy</button></th>
             <th scope="col" data-key="country"><button class="sort" type="button">Country</button></th>
             <th scope="col" data-key="latency" aria-sort="ascending"><button class="sort" type="button">Latency</button></th>
+            <th scope="col" data-key="speed_kbps" class="hide-sm"><button class="sort" type="button" title="Download speed in the last run – 100 KB through the proxy">Speed</button></th>
             <th scope="col" data-key="https" class="hide-sm"><button class="sort" type="button">HTTPS</button></th>
             <th scope="col" data-key="uptime_7d" class="hide-sm"><button class="sort" type="button" title="Share of this week's runs it was on the list">Uptime</button></th>
             <th scope="col" class="hide-sm"><span class="th-plain" title="Big sites it got through to in the last hourly check">Gets through</span></th>
-            <th scope="col" data-key="anonymity" class="hide-sm"><button class="sort" type="button">Anonymity</button></th>
             <th scope="col" data-key="org" class="hide-sm"><button class="sort" type="button">Provider</button></th>
           </tr></thead>
           <tbody id="rows"><tr><td colspan="9" class="empty">Loading the list</td></tr></tbody>
@@ -716,6 +716,7 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
           <a href="history.json"><span class="mono">history.json</span><span>the runs of the last 30 days, for charts</span></a>
           <a href="streaks.json"><span class="mono">streaks.json</span><span>how many runs in a row each proxy worked</span></a>
           <a href="seen.json"><span class="mono">seen.json</span><span>which of the last 168 runs each proxy was listed in</span></a>
+          <a href="https://github.com/maximilianfeix/proxy-scraper/releases?q=snapshots&amp;expanded=true"><span class="mono">snapshots</span><span>one proxies.json a day, kept for good</span></a>
         </div>
       </div>
       <div class="cmd"><pre>curl -s https://maximilianfeix.github.io/proxy-scraper/proxies.json | jq '.[0]'
@@ -799,11 +800,13 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
   };
   const share = (r) => r.uptime_7d == null ? uptime(r.streak)  // lists from before the uptime existed
     : `<span class="uptime${(r.streak || 0) >= STABLE ? " stable-badge" : ""}" title="On the list in ${r.uptime_7d} % of this week's runs and ${r.uptime_24h} % of today's – up without a gap for ${upFor(r.streak)}"><span class="meter" style="--v:${r.uptime_7d}%"></span>${r.uptime_7d} %</span>`;
-  const SITE_NAMES = { google: "Google", reddit: "Reddit", amazon: "Amazon" };
+  const SITE_NAMES = { google: "Google", reddit: "Reddit", amazon: "Amazon", instagram: "Instagram", tiktok: "TikTok" };
   const through = (r) => {
     const ok = Object.keys(SITE_NAMES).filter(k => (r.sites || {})[k] === true);
     return ok.length ? `<span class="sites">${ok.map(k => `<span class="site">${SITE_NAMES[k]}</span>`).join("")}</span>` : '<span class="quiet">–</span>';
   };
+  const speed = (k) => !k ? '<span class="quiet">–</span>'
+    : `<span class="${k >= 500 ? "fast" : k >= 100 ? "mid" : "slow"}">${k >= 1024 ? (k / 1024).toFixed(1) + " MB/s" : fmt(k) + " KB/s"}</span>`;
   const latency = (ms) => `<span class="${ms < 1000 ? "fast" : ms < 3000 ? "mid" : "slow"}">${fmt(ms)} ms</span>`;
   const state = { rows: [], shown: PAGE, sort: "latency", dir: 1, types: new Set(["http", "socks4", "socks5"]),
                   https: false, elite: false, noDc: false, noBl: false, stable: false, country: "", site: "", maxLat: 8000, q: "" };
@@ -952,7 +955,9 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
       && (state.maxLat >= 8000 || r.latency <= state.maxLat)  // all the way right = no limit
       && (!q || `${r.ptype} ${r.proxy} ${r.country} ${r.country ? countryName(r.country) : ""} ${r.exit_ip} ${r.org || ""} AS${r.asn || ""}`.toLowerCase().includes(q)));
     const k = state.sort, d = state.dir;
-    return rows.sort((a, b) => (a[k] > b[k] ? 1 : a[k] < b[k] ? -1 : 0) * d || a.latency - b.latency);
+    const missing = (v) => v == null || v === "";  // not measured: always last, whichever way it's sorted
+    return rows.sort((a, b) => (missing(a[k]) - missing(b[k]))
+      || (a[k] > b[k] ? 1 : a[k] < b[k] ? -1 : 0) * d || a.latency - b.latency);
   }
   function render() {
     const rows = filtered();
@@ -961,10 +966,10 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
       <td><button class="addr-btn" type="button" data-url="${esc(r.url)}" aria-label="Copy ${esc(r.url)}">${esc(r.proxy)}<svg aria-hidden="true"><use href="#i-copy"/></svg></button><a class="detail" href="proxy/${esc(r.proxy.replace(":", "-"))}/" aria-label="Details and week of ${esc(r.proxy)}" title="Details and this week's checks"><svg aria-hidden="true"><use href="#i-open"/></svg></a></td>
       <td title="${esc(r.country ? countryName(r.country) : "")}">${flag(r.country)} ${esc(r.country || "–")}</td>
       <td>${latency(r.latency)}</td>
+      <td class="hide-sm">${speed(r.speed_kbps)}</td>
       <td class="hide-sm">${r.https ? "Yes" : '<span class="quiet">No</span>'}</td>
       <td class="hide-sm">${share(r)}</td>
       <td class="hide-sm">${through(r)}</td>
-      <td class="hide-sm">${esc(r.anonymity || "–")}</td>
       <td class="hide-sm provider" title="${esc(r.org ? `AS${r.asn} ${r.org}, exit ${r.exit_ip}` : r.exit_ip)}">${r.hosting ? '<span class="tag" title="Datacenter">DC</span>' : ""}${r.blocklisted ? '<span class="tag bl" title="Exit IP on the SpamCop blocklist">BL</span>' : ""}${esc(short(r.org) || r.exit_ip)}</td></tr>`).join("")
       || '<tr><td colspan="9" class="empty">No proxy matches these filters. Try a higher latency limit or fewer filters.</td></tr>';
     $("#count").textContent = rows.length ? `Showing ${fmt(Math.min(state.shown, rows.length))} of ${fmt(rows.length)} matching proxies` : "";
@@ -1003,7 +1008,7 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
     if (q.has("types")) state.types = new Set(q.get("types").split(",").filter(t => ALL_TYPES.includes(t)));
     Object.entries(FLAGS).forEach(([key, name]) => { state[key] = q.get(name) === "1"; });
     state.country = /^[A-Z]{2}$/.test(q.get("country") || "") ? q.get("country") : "";
-    state.site = ["google", "reddit", "amazon"].includes(q.get("site")) ? q.get("site") : "";
+    state.site = Object.keys(SITE_NAMES).includes(q.get("site")) ? q.get("site") : "";
     const lat = parseInt(q.get("latency"), 10);
     state.maxLat = lat >= 200 && lat < 8000 ? Math.round(lat / 100) * 100 : 8000;
     state.q = (q.get("q") || "").slice(0, 100);

{proxy_scraper_cli-1.10.0 → proxy_scraper_cli-1.11.0}/proxyscraper/sites.py

@@ -36,6 +36,7 @@ class Site:
     blocked_redirect: str = ""          # a redirect to a URL containing this is a block too (a captcha page)
     ok_redirect: str = ""               # ... and one containing this is fine (a consent screen)
     min_body: int = 0                   # an "ok" answer must come with at least this much page, else it's a stub
+    title: str = ""                     # for people: "Google"
 
     def verdict(self, status: int, location: str = "", body: Optional[int] = None) -> Optional[bool]:
         """True = got through, False = blocked, None = the answer says neither. body: bytes of page read after
@@ -54,11 +55,18 @@ class Site:
 
 SITES = (
     Site("google", "www.google.com", "/search?q=weather", ok=(200,), blocked=(429,),
-         blocked_redirect="/sorry/", ok_redirect="consent.google."),
-    Site("reddit", "www.reddit.com", "/", ok=(200,), blocked=(403, 429)),
+         blocked_redirect="/sorry/", ok_redirect="consent.google.", title="Google"),
+    Site("reddit", "www.reddit.com", "/", ok=(200,), blocked=(403, 429), title="Reddit"),
     # Amazon answers bots with a 2-4 KB stub or captcha page, and since 2026-09 with status 200 on the home page
     # too – even without a proxy. The search tells them apart: real results are well over 100 KB.
-    Site("amazon", "www.amazon.com", "/s?k=usb+cable", ok=(200,), blocked=(202, 503), min_body=32_000),
+    Site("amazon", "www.amazon.com", "/s?k=usb+cable", ok=(200,), blocked=(202, 503), min_body=32_000,
+         title="Amazon"),
+    # measured on 60 live proxies (2026-09-27): most get Instagram's login wall, a few the real profile page;
+    # TikTok gives some a 403 or a small stub. ChatGPT, eBay and Indeed block everyone (the last two even without
+    # a proxy) and LinkedIn and Twitch let everyone through – those say nothing, so they're not in here.
+    Site("instagram", "www.instagram.com", "/instagram/", ok=(200,), blocked=(429,), blocked_redirect="accounts/login",
+         min_body=30_000, title="Instagram"),
+    Site("tiktok", "www.tiktok.com", "/@tiktok", ok=(200,), blocked=(403, 429), min_body=30_000, title="TikTok"),
 )
 SITE = {s.name: s for s in SITES}