proxy-scraper-cli 1.9.0__tar.gz → 1.11.0__tar.gz

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.9.0
+Version: 1.11.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -35,6 +35,7 @@ Provides-Extra: mcp
 Requires-Dist: mcp<3,>=2.2; python_version >= "3.10" and extra == "mcp"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
+Requires-Dist: hypothesis; extra == "dev"
 Requires-Dist: ruff==0.16.8; extra == "dev"
 Dynamic: license-file
 
@@ -138,11 +139,12 @@ python3 proxy_scraper.py
 
 Every [release](https://github.com/maximilianfeix/proxy-scraper/releases/latest) also ships a wheel you can install with `pip install <file>.whl`, and a multi-arch image (amd64/arm64) on `ghcr.io`. In the container the wizard never shows up, it runs straight away. For the proxy server use `--serve --serve-host 0.0.0.0` with `-p 127.0.0.1:8899:8899`, so the port is only open on your own machine.
 
-Tab completion for bash, zsh and fish:
+Tab completion for bash, zsh, fish and PowerShell:
 
 ```bash
 eval "$(proxy-scraper --completion zsh)"     # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
 proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
+proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
 Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
@@ -203,7 +205,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon (no captcha, no 403 in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -217,6 +219,8 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | head
 ```
 
+**Over time:** every day the first run's `proxies.json` is kept for good as a gzipped asset of the release of that year, like [snapshots-2026](../../releases/tag/snapshots-2026) – `proxies-2026-09-28.json.gz` and so on, for anyone who wants to look at free proxies over weeks and months.
+
 Every file is also on GitHub Pages, which sits behind a CDN, isn't rate limited like raw.githubusercontent and sends CORS headers, so it works straight from the browser: `https://maximilianfeix.github.io/proxy-scraper/socks5.txt`. [jsDelivr](https://cdn.jsdelivr.net/gh/maximilianfeix/proxy-scraper@proxy-list/) works too, but can lag behind by a few hours.
 
 Or let the tool start from it: `proxy-scraper --recheck live` downloads the list and checks it again from **your** network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With `--serve` you have a rotating proxy in under a minute.
@@ -438,7 +442,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 <a id="proxy-server"></a>
 
@@ -656,14 +660,14 @@ if __name__ == "__main__":
 
 In a Scrapy project, put `RotatingProxy` in `middlewares.py` and add it to `DOWNLOADER_MIDDLEWARES` in `settings.py`. For long crawls, reload the pool now and then – the list changes every hour.
 
-**proxychains, Clash / Mihomo** – ready-made configs with `--export`
+**proxychains, Clash / Mihomo, sing-box** – ready-made configs with `--export`
 
 ```bash
-proxy-scraper --want 30 -y --export proxychains,clash
+proxy-scraper --want 30 -y --export proxychains,clash,singbox
 proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.org
 ```
 
-`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
+`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. `singbox.json` does the same for sing-box, SOCKS4 included, and opens a local proxy: `sing-box run -c results/latest/singbox.json`, then use `127.0.0.1:2080` as HTTP or SOCKS5 proxy. All three leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
 
 **In a pipe** – `-o -` prints the hits to stdout, the interface moves to stderr
 
@@ -679,6 +683,22 @@ export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
 pip download requests   # git, pip, npm & co. now go through the pool
 ```
 
+**In a GitHub workflow** – the action picks working proxies for the next steps
+
+```yaml
+- id: proxies
+  uses: maximilianfeix/proxy-scraper@v1
+  with:
+    types: socks5
+    https: true
+    min-uptime: 90        # on the list for 90 %+ of the week
+    works-on: google      # optional: google, reddit, amazon
+    recheck: true         # optional: check them again from the runner
+- run: curl -x "${{ steps.proxies.outputs.proxy }}" https://api.ipify.org
+```
+
+`proxy` is the fastest one, `file` a text file with all of them (up to `limit`, default 20), `count` how many. Without a match the step fails, unless `fail-if-empty: false`.
+
 **Without installing anything** – straight from the live list
 
 ```bash
@@ -722,16 +742,16 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `-t`, `--timeout S` | timeout per proxy (default: 8 s) |
 | `--discover` | search GitHub for new sources right now |
 | `--no-cache` | download every list again (unchanged ones are normally skipped via ETag) |
-| `--list-sources [N]` | show the source ranking |
+| `--list-sources [N]` | show the source ranking (add `--json` for scripts: url, status, hit rate, checks, last change) |
 | `--serve-host ADDR` | where the proxy server listens (default `127.0.0.1`; `0.0.0.0` for Docker, with a warning) |
 | `--serve-password SECRET` | clients must send this password in the proxy login; better set `PROXY_SCRAPER_SERVE_PASSWORD` |
 | `--rotate STRATEGY` · `--sticky SEC` | how the proxy server picks proxies, see [above](#proxy-server) |
 | `--serve-refill HOURS` | while serving, check fresh proxies every HOURS and add the hits to the pool |
 | `--serve [PORT]` | afterwards serve as a rotating proxy on `127.0.0.1:PORT` (default: 8899) |
 | `-o FILE` | also write all hits to this file; `-o -` prints them to stdout |
-| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `curl` or `all` |
+| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `singbox`, `curl` or `all` |
 | `-V`, `--version` | print the version |
-| `--completion SHELL` | print the tab completion script for bash, zsh or fish |
+| `--completion SHELL` | print the tab completion script for bash, zsh, fish or PowerShell |
 
 Everything else: `proxy-scraper --help`
 
@@ -813,10 +833,12 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 ## Roadmap
 
-What's next is tracked in the milestone [**v1.8**](../../milestone/8) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+
+- [ ] [Measure throughput, not just latency](../../issues/140)
+- [ ] [Daily snapshots as a dataset](../../issues/143)
 
-- [ ] [Protocol detection on the same connection](../../issues/44)
-- [ ] [sing-box export](../../issues/113), [PowerShell completion](../../issues/114), [shareable filters on the website](../../issues/116)
+Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 
 Shipped in [v1.7](../../milestone/7): an MCP server so AI agents get working proxies and can load pages through them, 28 new sources and a GitHub search that runs daily and keeps what it found, and a cleaner website.
 
@@ -861,7 +883,7 @@ proxyscraper/
 ├── geodb.py            DB-IP country database (monthly, binary search)
 ├── targets.py          target sites for --target
 ├── output.py           result files
-├── exporters.py        proxychains, Clash and curl formats (--export)
+├── exporters.py        proxychains, Clash, sing-box and curl formats (--export)
 ├── server/             rotating proxy server (--serve): pool · http · upstream · socks · status · core
 ├── api.py              find_proxies() / check_proxies() for Python
 ├── agent.py            MCP tools without the SDK: live list, filters, fetch through proxies

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/README.md

@@ -98,11 +98,12 @@ python3 proxy_scraper.py
 
 Every [release](https://github.com/maximilianfeix/proxy-scraper/releases/latest) also ships a wheel you can install with `pip install <file>.whl`, and a multi-arch image (amd64/arm64) on `ghcr.io`. In the container the wizard never shows up, it runs straight away. For the proxy server use `--serve --serve-host 0.0.0.0` with `-p 127.0.0.1:8899:8899`, so the port is only open on your own machine.
 
-Tab completion for bash, zsh and fish:
+Tab completion for bash, zsh, fish and PowerShell:
 
 ```bash
 eval "$(proxy-scraper --completion zsh)"     # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
 proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
+proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
 Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
@@ -163,7 +164,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon (no captcha, no 403 in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -177,6 +178,8 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | head
 ```
 
+**Over time:** every day the first run's `proxies.json` is kept for good as a gzipped asset of the release of that year, like [snapshots-2026](../../releases/tag/snapshots-2026) – `proxies-2026-09-28.json.gz` and so on, for anyone who wants to look at free proxies over weeks and months.
+
 Every file is also on GitHub Pages, which sits behind a CDN, isn't rate limited like raw.githubusercontent and sends CORS headers, so it works straight from the browser: `https://maximilianfeix.github.io/proxy-scraper/socks5.txt`. [jsDelivr](https://cdn.jsdelivr.net/gh/maximilianfeix/proxy-scraper@proxy-list/) works too, but can lag behind by a few hours.
 
 Or let the tool start from it: `proxy-scraper --recheck live` downloads the list and checks it again from **your** network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With `--serve` you have a rotating proxy in under a minute.
@@ -398,7 +401,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 <a id="proxy-server"></a>
 
@@ -616,14 +619,14 @@ if __name__ == "__main__":
 
 In a Scrapy project, put `RotatingProxy` in `middlewares.py` and add it to `DOWNLOADER_MIDDLEWARES` in `settings.py`. For long crawls, reload the pool now and then – the list changes every hour.
 
-**proxychains, Clash / Mihomo** – ready-made configs with `--export`
+**proxychains, Clash / Mihomo, sing-box** – ready-made configs with `--export`
 
 ```bash
-proxy-scraper --want 30 -y --export proxychains,clash
+proxy-scraper --want 30 -y --export proxychains,clash,singbox
 proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.org
 ```
 
-`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
+`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. `singbox.json` does the same for sing-box, SOCKS4 included, and opens a local proxy: `sing-box run -c results/latest/singbox.json`, then use `127.0.0.1:2080` as HTTP or SOCKS5 proxy. All three leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
 
 **In a pipe** – `-o -` prints the hits to stdout, the interface moves to stderr
 
@@ -639,6 +642,22 @@ export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
 pip download requests   # git, pip, npm & co. now go through the pool
 ```
 
+**In a GitHub workflow** – the action picks working proxies for the next steps
+
+```yaml
+- id: proxies
+  uses: maximilianfeix/proxy-scraper@v1
+  with:
+    types: socks5
+    https: true
+    min-uptime: 90        # on the list for 90 %+ of the week
+    works-on: google      # optional: google, reddit, amazon
+    recheck: true         # optional: check them again from the runner
+- run: curl -x "${{ steps.proxies.outputs.proxy }}" https://api.ipify.org
+```
+
+`proxy` is the fastest one, `file` a text file with all of them (up to `limit`, default 20), `count` how many. Without a match the step fails, unless `fail-if-empty: false`.
+
 **Without installing anything** – straight from the live list
 
 ```bash
@@ -682,16 +701,16 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `-t`, `--timeout S` | timeout per proxy (default: 8 s) |
 | `--discover` | search GitHub for new sources right now |
 | `--no-cache` | download every list again (unchanged ones are normally skipped via ETag) |
-| `--list-sources [N]` | show the source ranking |
+| `--list-sources [N]` | show the source ranking (add `--json` for scripts: url, status, hit rate, checks, last change) |
 | `--serve-host ADDR` | where the proxy server listens (default `127.0.0.1`; `0.0.0.0` for Docker, with a warning) |
 | `--serve-password SECRET` | clients must send this password in the proxy login; better set `PROXY_SCRAPER_SERVE_PASSWORD` |
 | `--rotate STRATEGY` · `--sticky SEC` | how the proxy server picks proxies, see [above](#proxy-server) |
 | `--serve-refill HOURS` | while serving, check fresh proxies every HOURS and add the hits to the pool |
 | `--serve [PORT]` | afterwards serve as a rotating proxy on `127.0.0.1:PORT` (default: 8899) |
 | `-o FILE` | also write all hits to this file; `-o -` prints them to stdout |
-| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `curl` or `all` |
+| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `singbox`, `curl` or `all` |
 | `-V`, `--version` | print the version |
-| `--completion SHELL` | print the tab completion script for bash, zsh or fish |
+| `--completion SHELL` | print the tab completion script for bash, zsh, fish or PowerShell |
 
 Everything else: `proxy-scraper --help`
 
@@ -773,10 +792,12 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 ## Roadmap
 
-What's next is tracked in the milestone [**v1.8**](../../milestone/8) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+
+- [ ] [Measure throughput, not just latency](../../issues/140)
+- [ ] [Daily snapshots as a dataset](../../issues/143)
 
-- [ ] [Protocol detection on the same connection](../../issues/44)
-- [ ] [sing-box export](../../issues/113), [PowerShell completion](../../issues/114), [shareable filters on the website](../../issues/116)
+Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 
 Shipped in [v1.7](../../milestone/7): an MCP server so AI agents get working proxies and can load pages through them, 28 new sources and a GitHub search that runs daily and keeps what it found, and a cleaner website.
 
@@ -821,7 +842,7 @@ proxyscraper/
 ├── geodb.py            DB-IP country database (monthly, binary search)
 ├── targets.py          target sites for --target
 ├── output.py           result files
-├── exporters.py        proxychains, Clash and curl formats (--export)
+├── exporters.py        proxychains, Clash, sing-box and curl formats (--export)
 ├── server/             rotating proxy server (--serve): pool · http · upstream · socks · status · core
 ├── api.py              find_proxies() / check_proxies() for Python
 ├── agent.py            MCP tools without the SDK: live list, filters, fetch through proxies

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/proxy_scraper_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.9.0
+Version: 1.11.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -35,6 +35,7 @@ Provides-Extra: mcp
 Requires-Dist: mcp<3,>=2.2; python_version >= "3.10" and extra == "mcp"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
+Requires-Dist: hypothesis; extra == "dev"
 Requires-Dist: ruff==0.16.8; extra == "dev"
 Dynamic: license-file
 
@@ -138,11 +139,12 @@ python3 proxy_scraper.py
 
 Every [release](https://github.com/maximilianfeix/proxy-scraper/releases/latest) also ships a wheel you can install with `pip install <file>.whl`, and a multi-arch image (amd64/arm64) on `ghcr.io`. In the container the wizard never shows up, it runs straight away. For the proxy server use `--serve --serve-host 0.0.0.0` with `-p 127.0.0.1:8899:8899`, so the port is only open on your own machine.
 
-Tab completion for bash, zsh and fish:
+Tab completion for bash, zsh, fish and PowerShell:
 
 ```bash
 eval "$(proxy-scraper --completion zsh)"     # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
 proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
+proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
 Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
@@ -203,7 +205,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon (no captcha, no 403 in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -217,6 +219,8 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | head
 ```
 
+**Over time:** every day the first run's `proxies.json` is kept for good as a gzipped asset of the release of that year, like [snapshots-2026](../../releases/tag/snapshots-2026) – `proxies-2026-09-28.json.gz` and so on, for anyone who wants to look at free proxies over weeks and months.
+
 Every file is also on GitHub Pages, which sits behind a CDN, isn't rate limited like raw.githubusercontent and sends CORS headers, so it works straight from the browser: `https://maximilianfeix.github.io/proxy-scraper/socks5.txt`. [jsDelivr](https://cdn.jsdelivr.net/gh/maximilianfeix/proxy-scraper@proxy-list/) works too, but can lag behind by a few hours.
 
 Or let the tool start from it: `proxy-scraper --recheck live` downloads the list and checks it again from **your** network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With `--serve` you have a rotating proxy in under a minute.
@@ -438,7 +442,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 <a id="proxy-server"></a>
 
@@ -656,14 +660,14 @@ if __name__ == "__main__":
 
 In a Scrapy project, put `RotatingProxy` in `middlewares.py` and add it to `DOWNLOADER_MIDDLEWARES` in `settings.py`. For long crawls, reload the pool now and then – the list changes every hour.
 
-**proxychains, Clash / Mihomo** – ready-made configs with `--export`
+**proxychains, Clash / Mihomo, sing-box** – ready-made configs with `--export`
 
 ```bash
-proxy-scraper --want 30 -y --export proxychains,clash
+proxy-scraper --want 30 -y --export proxychains,clash,singbox
 proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.org
 ```
 
-`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
+`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. `singbox.json` does the same for sing-box, SOCKS4 included, and opens a local proxy: `sing-box run -c results/latest/singbox.json`, then use `127.0.0.1:2080` as HTTP or SOCKS5 proxy. All three leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
 
 **In a pipe** – `-o -` prints the hits to stdout, the interface moves to stderr
 
@@ -679,6 +683,22 @@ export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
 pip download requests   # git, pip, npm & co. now go through the pool
 ```
 
+**In a GitHub workflow** – the action picks working proxies for the next steps
+
+```yaml
+- id: proxies
+  uses: maximilianfeix/proxy-scraper@v1
+  with:
+    types: socks5
+    https: true
+    min-uptime: 90        # on the list for 90 %+ of the week
+    works-on: google      # optional: google, reddit, amazon
+    recheck: true         # optional: check them again from the runner
+- run: curl -x "${{ steps.proxies.outputs.proxy }}" https://api.ipify.org
+```
+
+`proxy` is the fastest one, `file` a text file with all of them (up to `limit`, default 20), `count` how many. Without a match the step fails, unless `fail-if-empty: false`.
+
 **Without installing anything** – straight from the live list
 
 ```bash
@@ -722,16 +742,16 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `-t`, `--timeout S` | timeout per proxy (default: 8 s) |
 | `--discover` | search GitHub for new sources right now |
 | `--no-cache` | download every list again (unchanged ones are normally skipped via ETag) |
-| `--list-sources [N]` | show the source ranking |
+| `--list-sources [N]` | show the source ranking (add `--json` for scripts: url, status, hit rate, checks, last change) |
 | `--serve-host ADDR` | where the proxy server listens (default `127.0.0.1`; `0.0.0.0` for Docker, with a warning) |
 | `--serve-password SECRET` | clients must send this password in the proxy login; better set `PROXY_SCRAPER_SERVE_PASSWORD` |
 | `--rotate STRATEGY` · `--sticky SEC` | how the proxy server picks proxies, see [above](#proxy-server) |
 | `--serve-refill HOURS` | while serving, check fresh proxies every HOURS and add the hits to the pool |
 | `--serve [PORT]` | afterwards serve as a rotating proxy on `127.0.0.1:PORT` (default: 8899) |
 | `-o FILE` | also write all hits to this file; `-o -` prints them to stdout |
-| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `curl` or `all` |
+| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `singbox`, `curl` or `all` |
 | `-V`, `--version` | print the version |
-| `--completion SHELL` | print the tab completion script for bash, zsh or fish |
+| `--completion SHELL` | print the tab completion script for bash, zsh, fish or PowerShell |
 
 Everything else: `proxy-scraper --help`
 
@@ -813,10 +833,12 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 ## Roadmap
 
-What's next is tracked in the milestone [**v1.8**](../../milestone/8) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+
+- [ ] [Measure throughput, not just latency](../../issues/140)
+- [ ] [Daily snapshots as a dataset](../../issues/143)
 
-- [ ] [Protocol detection on the same connection](../../issues/44)
-- [ ] [sing-box export](../../issues/113), [PowerShell completion](../../issues/114), [shareable filters on the website](../../issues/116)
+Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 
 Shipped in [v1.7](../../milestone/7): an MCP server so AI agents get working proxies and can load pages through them, 28 new sources and a GitHub search that runs daily and keeps what it found, and a cleaner website.
 
@@ -861,7 +883,7 @@ proxyscraper/
 ├── geodb.py            DB-IP country database (monthly, binary search)
 ├── targets.py          target sites for --target
 ├── output.py           result files
-├── exporters.py        proxychains, Clash and curl formats (--export)
+├── exporters.py        proxychains, Clash, sing-box and curl formats (--export)
 ├── server/             rotating proxy server (--serve): pool · http · upstream · socks · status · core
 ├── api.py              find_proxies() / check_proxies() for Python
 ├── agent.py            MCP tools without the SDK: live list, filters, fetch through proxies

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/proxy_scraper_cli.egg-info/SOURCES.txt

@@ -9,6 +9,7 @@ proxy_scraper_cli.egg-info/requires.txt
 proxy_scraper_cli.egg-info/top_level.txt
 proxyscraper/__init__.py
 proxyscraper/__main__.py
+proxyscraper/action.py
 proxyscraper/agent.py
 proxyscraper/api.py
 proxyscraper/app.py
@@ -42,6 +43,7 @@ proxyscraper/report.py
 proxyscraper/sites.py
 proxyscraper/sources.json
 proxyscraper/sources.py
+proxyscraper/speed.py
 proxyscraper/targets.py
 proxyscraper/server/__init__.py
 proxyscraper/server/core.py
@@ -62,6 +64,7 @@ proxyscraper/ui/report.py
 proxyscraper/ui/serve.py
 proxyscraper/ui/widgets.py
 proxyscraper/ui/wizard.py
+tests/test_action.py
 tests/test_agent.py
 tests/test_api.py
 tests/test_app.py
@@ -86,6 +89,7 @@ tests/test_output_stdout.py
 tests/test_packaging.py
 tests/test_pages.py
 tests/test_parsing.py
+tests/test_parsing_fuzz.py
 tests/test_paths.py
 tests/test_pipeline.py
 tests/test_proxy_pages.py
@@ -97,8 +101,11 @@ tests/test_serve_password.py
 tests/test_serve_refill.py
 tests/test_serve_v2.py
 tests/test_server.py
+tests/test_server_closed_writer.py
 tests/test_server_fixes.py
 tests/test_sites.py
+tests/test_sources_json.py
+tests/test_speed.py
 tests/test_tamper.py
 tests/test_targets.py
 tests/test_ui.py

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/proxy_scraper_cli.egg-info/requires.txt

@@ -3,6 +3,7 @@ certifi
 
 [dev]
 pytest
+hypothesis
 ruff==0.16.8
 
 [fast]

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/proxyscraper/__init__.py

@@ -1,7 +1,7 @@
 """proxy-scraper: collects public HTTP/SOCKS4/SOCKS5 proxies from hundreds of sources, checks them
 in parallel with its own protocol handshakes and learns which sources deliver good proxies."""
 
-__version__ = "1.9.0"
+__version__ = "1.11.0"
 
 
 def __getattr__(name):

proxy_scraper_cli-1.11.0/proxyscraper/action.py

@@ -0,0 +1,96 @@
+"""The GitHub Action (action.yml): working proxies from the hourly list for the next steps of a workflow.
+
+Inputs come in as PS_* environment variables, results go to $GITHUB_OUTPUT:
+  proxy   the fastest one, ready to use (socks5://1.2.3.4:1080)
+  file    a text file with all of them, one per line, fastest first
+  count   how many
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+import tempfile
+from pathlib import Path
+from typing import List
+
+from .api import check_proxies, live_proxies
+
+
+class InputError(ValueError):
+    pass
+
+
+def _flag(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name, "").strip().lower()
+    if not value:
+        return default
+    if value in ("true", "yes", "1"):
+        return True
+    if value in ("false", "no", "0"):
+        return False
+    raise InputError(f"{name[3:].lower().replace('_', '-')} must be true or false, not {value!r}")
+
+
+def _number(name: str, default: int) -> int:
+    value = os.environ.get(name, "").strip()
+    if not value:
+        return default
+    if not value.isdigit():
+        raise InputError(f"{name[3:].lower().replace('_', '-')} must be a whole number, not {value!r}")
+    return int(value)
+
+
+def _list(name: str) -> List[str]:
+    return [part.strip().lower() for part in os.environ.get(name, "").split(",") if part.strip()]
+
+
+def _output(**values: str) -> None:
+    path = os.environ.get("GITHUB_OUTPUT")
+    lines = "".join(f"{key}={value}\n" for key, value in values.items())
+    if path:
+        with open(path, "a", encoding="utf-8") as fh:
+            fh.write(lines)
+    else:
+        sys.stdout.write(lines)
+
+
+def main() -> int:
+    try:
+        limit = _number("PS_LIMIT", 20)
+        recheck = _flag("PS_RECHECK")
+        fail_if_empty = _flag("PS_FAIL_IF_EMPTY", True)
+        filters = {"types": _list("PS_TYPES") or ["http", "socks4", "socks5"],
+                   "countries": os.environ.get("PS_COUNTRIES", "").strip(),
+                   "https": _flag("PS_HTTPS"), "min_uptime": _number("PS_MIN_UPTIME", 0),
+                   "works_on": _list("PS_WORKS_ON")}
+        # a recheck starts from all of them – the limit applies to what still works from here
+        found = live_proxies(**filters, limit=0 if recheck else limit)
+    except (InputError, ValueError) as e:
+        print(f"::error::{e}")
+        return 1
+    except ConnectionError as e:
+        print(f"::error::The live list couldn't be loaded: {e}")
+        return 1
+    urls = [p.url for p in found]
+    if recheck and urls:
+        print(f"Checking {len(urls)} proxies again from this runner …")
+        urls = [r.url for r in sorted(check_proxies(urls, https=filters["https"]), key=lambda r: r.latency)]
+    if limit:
+        urls = urls[:limit]
+    # a file of its own per step: a job may use the action twice (socks5 in one step, http in the next)
+    fd, name = tempfile.mkstemp(prefix="proxies-", suffix=".txt", dir=os.environ.get("RUNNER_TEMP") or None)
+    with os.fdopen(fd, "w", encoding="utf-8") as fh:
+        fh.write("".join(f"{u}\n" for u in urls))
+    file = Path(name)
+    _output(proxy=urls[0] if urls else "", file=str(file), count=str(len(urls)))
+    print(f"{len(urls)} working {'proxy' if len(urls) == 1 else 'proxies'}" + (f", fastest: {urls[0]}" if urls else ""))
+    if not urls and fail_if_empty:
+        print("::error::No proxy matches these filters right now – loosen them (min-uptime, works-on, countries) "
+              "or set fail-if-empty: false")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.11.0}/proxyscraper/agent.py

@@ -179,17 +179,18 @@ def normalize_countries(countries: Iterable[str]) -> List[str]:
 def select(rows: Iterable[dict], protocol: str = "any", countries: Iterable[str] = (), https_only: bool = False,
            elite_only: bool = False, exclude_datacenter: bool = False, exclude_blocklisted: bool = False,
            stable_only: bool = False, max_latency_ms: int = 0, run_hours: int = 1, min_uptime: int = 0,
-           works_on: Iterable[str] = ()) -> List[dict]:
+           works_on: Iterable[str] = (), min_speed_kbps: int = 0) -> List[dict]:
     """The rows that pass every filter, fastest first – the same filters as on the website."""
     return sorted(matching(rows, protocol, countries, https_only, elite_only, exclude_datacenter,
-                           exclude_blocklisted, stable_only, max_latency_ms, run_hours, min_uptime, works_on),
+                           exclude_blocklisted, stable_only, max_latency_ms, run_hours, min_uptime, works_on,
+                           min_speed_kbps),
                   key=lambda r: r.get("latency") or 0)
 
 
 def matching(rows: Iterable[dict], protocol: str = "any", countries: Iterable[str] = (), https_only: bool = False,
              elite_only: bool = False, exclude_datacenter: bool = False, exclude_blocklisted: bool = False,
              stable_only: bool = False, max_latency_ms: int = 0, run_hours: int = 1,
-             min_uptime: int = 0, works_on: Iterable[str] = ()) -> Iterator[dict]:
+             min_uptime: int = 0, works_on: Iterable[str] = (), min_speed_kbps: int = 0) -> Iterator[dict]:
     """The rows that pass every filter, in list order – checks the filters before the first row is asked for."""
     protocol = _check_protocol(protocol)
     wanted = set(normalize_countries(countries))
@@ -205,7 +206,8 @@ def matching(rows: Iterable[dict], protocol: str = "any", countries: Iterable[st
            and (not stable_only or (r.get("streak") or 0) >= stable_runs)
            and (not max_latency_ms or (r.get("latency") or 0) <= max_latency_ms)
            and (not min_uptime or (r.get("uptime_7d") or 0) >= min_uptime)
-           and all((r.get("sites") or {}).get(s) for s in sites))
+           and all((r.get("sites") or {}).get(s) for s in sites)
+           and (not min_speed_kbps or (r.get("speed_kbps") or 0) >= min_speed_kbps))
 
 
 def describe(item: Union[dict, CheckResult], run_hours: Optional[int] = None) -> dict:
@@ -229,6 +231,7 @@ def describe(item: Union[dict, CheckResult], run_hours: Optional[int] = None) ->
         "up_for_hours": streak * run_hours if streak and run_hours else None,
         "uptime_7d_percent": item.get("uptime_7d"),
         "works_on": [name for name, ok in (item.get("sites") or {}).items() if ok],
+        "speed_kbps": item.get("speed_kbps"),
     }