proxy-scraper-cli 1.9.0__tar.gz → 1.10.0__tar.gz

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.9.0
+Version: 1.10.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -35,6 +35,7 @@ Provides-Extra: mcp
 Requires-Dist: mcp<3,>=2.2; python_version >= "3.10" and extra == "mcp"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
+Requires-Dist: hypothesis; extra == "dev"
 Requires-Dist: ruff==0.16.8; extra == "dev"
 Dynamic: license-file
 
@@ -138,11 +139,12 @@ python3 proxy_scraper.py
 
 Every [release](https://github.com/maximilianfeix/proxy-scraper/releases/latest) also ships a wheel you can install with `pip install <file>.whl`, and a multi-arch image (amd64/arm64) on `ghcr.io`. In the container the wizard never shows up, it runs straight away. For the proxy server use `--serve --serve-host 0.0.0.0` with `-p 127.0.0.1:8899:8899`, so the port is only open on your own machine.
 
-Tab completion for bash, zsh and fish:
+Tab completion for bash, zsh, fish and PowerShell:
 
 ```bash
 eval "$(proxy-scraper --completion zsh)"     # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
 proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
+proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
 Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
@@ -656,14 +658,14 @@ if __name__ == "__main__":
 
 In a Scrapy project, put `RotatingProxy` in `middlewares.py` and add it to `DOWNLOADER_MIDDLEWARES` in `settings.py`. For long crawls, reload the pool now and then – the list changes every hour.
 
-**proxychains, Clash / Mihomo** – ready-made configs with `--export`
+**proxychains, Clash / Mihomo, sing-box** – ready-made configs with `--export`
 
 ```bash
-proxy-scraper --want 30 -y --export proxychains,clash
+proxy-scraper --want 30 -y --export proxychains,clash,singbox
 proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.org
 ```
 
-`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
+`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. `singbox.json` does the same for sing-box, SOCKS4 included, and opens a local proxy: `sing-box run -c results/latest/singbox.json`, then use `127.0.0.1:2080` as HTTP or SOCKS5 proxy. All three leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
 
 **In a pipe** – `-o -` prints the hits to stdout, the interface moves to stderr
 
@@ -679,6 +681,22 @@ export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
 pip download requests   # git, pip, npm & co. now go through the pool
 ```
 
+**In a GitHub workflow** – the action picks working proxies for the next steps
+
+```yaml
+- id: proxies
+  uses: maximilianfeix/proxy-scraper@v1
+  with:
+    types: socks5
+    https: true
+    min-uptime: 90        # on the list for 90 %+ of the week
+    works-on: google      # optional: google, reddit, amazon
+    recheck: true         # optional: check them again from the runner
+- run: curl -x "${{ steps.proxies.outputs.proxy }}" https://api.ipify.org
+```
+
+`proxy` is the fastest one, `file` a text file with all of them (up to `limit`, default 20), `count` how many. Without a match the step fails, unless `fail-if-empty: false`.
+
 **Without installing anything** – straight from the live list
 
 ```bash
@@ -722,16 +740,16 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `-t`, `--timeout S` | timeout per proxy (default: 8 s) |
 | `--discover` | search GitHub for new sources right now |
 | `--no-cache` | download every list again (unchanged ones are normally skipped via ETag) |
-| `--list-sources [N]` | show the source ranking |
+| `--list-sources [N]` | show the source ranking (add `--json` for scripts: url, status, hit rate, checks, last change) |
 | `--serve-host ADDR` | where the proxy server listens (default `127.0.0.1`; `0.0.0.0` for Docker, with a warning) |
 | `--serve-password SECRET` | clients must send this password in the proxy login; better set `PROXY_SCRAPER_SERVE_PASSWORD` |
 | `--rotate STRATEGY` · `--sticky SEC` | how the proxy server picks proxies, see [above](#proxy-server) |
 | `--serve-refill HOURS` | while serving, check fresh proxies every HOURS and add the hits to the pool |
 | `--serve [PORT]` | afterwards serve as a rotating proxy on `127.0.0.1:PORT` (default: 8899) |
 | `-o FILE` | also write all hits to this file; `-o -` prints them to stdout |
-| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `curl` or `all` |
+| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `singbox`, `curl` or `all` |
 | `-V`, `--version` | print the version |
-| `--completion SHELL` | print the tab completion script for bash, zsh or fish |
+| `--completion SHELL` | print the tab completion script for bash, zsh, fish or PowerShell |
 
 Everything else: `proxy-scraper --help`
 
@@ -813,10 +831,12 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 ## Roadmap
 
-What's next is tracked in the milestone [**v1.8**](../../milestone/8) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+
+- [ ] [Measure throughput, not just latency](../../issues/140)
+- [ ] [Daily snapshots as a dataset](../../issues/143)
 
-- [ ] [Protocol detection on the same connection](../../issues/44)
-- [ ] [sing-box export](../../issues/113), [PowerShell completion](../../issues/114), [shareable filters on the website](../../issues/116)
+Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 
 Shipped in [v1.7](../../milestone/7): an MCP server so AI agents get working proxies and can load pages through them, 28 new sources and a GitHub search that runs daily and keeps what it found, and a cleaner website.
 
@@ -861,7 +881,7 @@ proxyscraper/
 ├── geodb.py            DB-IP country database (monthly, binary search)
 ├── targets.py          target sites for --target
 ├── output.py           result files
-├── exporters.py        proxychains, Clash and curl formats (--export)
+├── exporters.py        proxychains, Clash, sing-box and curl formats (--export)
 ├── server/             rotating proxy server (--serve): pool · http · upstream · socks · status · core
 ├── api.py              find_proxies() / check_proxies() for Python
 ├── agent.py            MCP tools without the SDK: live list, filters, fetch through proxies

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/README.md

@@ -98,11 +98,12 @@ python3 proxy_scraper.py
 
 Every [release](https://github.com/maximilianfeix/proxy-scraper/releases/latest) also ships a wheel you can install with `pip install <file>.whl`, and a multi-arch image (amd64/arm64) on `ghcr.io`. In the container the wizard never shows up, it runs straight away. For the proxy server use `--serve --serve-host 0.0.0.0` with `-p 127.0.0.1:8899:8899`, so the port is only open on your own machine.
 
-Tab completion for bash, zsh and fish:
+Tab completion for bash, zsh, fish and PowerShell:
 
 ```bash
 eval "$(proxy-scraper --completion zsh)"     # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
 proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
+proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
 Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
@@ -616,14 +617,14 @@ if __name__ == "__main__":
 
 In a Scrapy project, put `RotatingProxy` in `middlewares.py` and add it to `DOWNLOADER_MIDDLEWARES` in `settings.py`. For long crawls, reload the pool now and then – the list changes every hour.
 
-**proxychains, Clash / Mihomo** – ready-made configs with `--export`
+**proxychains, Clash / Mihomo, sing-box** – ready-made configs with `--export`
 
 ```bash
-proxy-scraper --want 30 -y --export proxychains,clash
+proxy-scraper --want 30 -y --export proxychains,clash,singbox
 proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.org
 ```
 
-`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
+`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. `singbox.json` does the same for sing-box, SOCKS4 included, and opens a local proxy: `sing-box run -c results/latest/singbox.json`, then use `127.0.0.1:2080` as HTTP or SOCKS5 proxy. All three leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
 
 **In a pipe** – `-o -` prints the hits to stdout, the interface moves to stderr
 
@@ -639,6 +640,22 @@ export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
 pip download requests   # git, pip, npm & co. now go through the pool
 ```
 
+**In a GitHub workflow** – the action picks working proxies for the next steps
+
+```yaml
+- id: proxies
+  uses: maximilianfeix/proxy-scraper@v1
+  with:
+    types: socks5
+    https: true
+    min-uptime: 90        # on the list for 90 %+ of the week
+    works-on: google      # optional: google, reddit, amazon
+    recheck: true         # optional: check them again from the runner
+- run: curl -x "${{ steps.proxies.outputs.proxy }}" https://api.ipify.org
+```
+
+`proxy` is the fastest one, `file` a text file with all of them (up to `limit`, default 20), `count` how many. Without a match the step fails, unless `fail-if-empty: false`.
+
 **Without installing anything** – straight from the live list
 
 ```bash
@@ -682,16 +699,16 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `-t`, `--timeout S` | timeout per proxy (default: 8 s) |
 | `--discover` | search GitHub for new sources right now |
 | `--no-cache` | download every list again (unchanged ones are normally skipped via ETag) |
-| `--list-sources [N]` | show the source ranking |
+| `--list-sources [N]` | show the source ranking (add `--json` for scripts: url, status, hit rate, checks, last change) |
 | `--serve-host ADDR` | where the proxy server listens (default `127.0.0.1`; `0.0.0.0` for Docker, with a warning) |
 | `--serve-password SECRET` | clients must send this password in the proxy login; better set `PROXY_SCRAPER_SERVE_PASSWORD` |
 | `--rotate STRATEGY` · `--sticky SEC` | how the proxy server picks proxies, see [above](#proxy-server) |
 | `--serve-refill HOURS` | while serving, check fresh proxies every HOURS and add the hits to the pool |
 | `--serve [PORT]` | afterwards serve as a rotating proxy on `127.0.0.1:PORT` (default: 8899) |
 | `-o FILE` | also write all hits to this file; `-o -` prints them to stdout |
-| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `curl` or `all` |
+| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `singbox`, `curl` or `all` |
 | `-V`, `--version` | print the version |
-| `--completion SHELL` | print the tab completion script for bash, zsh or fish |
+| `--completion SHELL` | print the tab completion script for bash, zsh, fish or PowerShell |
 
 Everything else: `proxy-scraper --help`
 
@@ -773,10 +790,12 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 ## Roadmap
 
-What's next is tracked in the milestone [**v1.8**](../../milestone/8) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+
+- [ ] [Measure throughput, not just latency](../../issues/140)
+- [ ] [Daily snapshots as a dataset](../../issues/143)
 
-- [ ] [Protocol detection on the same connection](../../issues/44)
-- [ ] [sing-box export](../../issues/113), [PowerShell completion](../../issues/114), [shareable filters on the website](../../issues/116)
+Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 
 Shipped in [v1.7](../../milestone/7): an MCP server so AI agents get working proxies and can load pages through them, 28 new sources and a GitHub search that runs daily and keeps what it found, and a cleaner website.
 
@@ -821,7 +840,7 @@ proxyscraper/
 ├── geodb.py            DB-IP country database (monthly, binary search)
 ├── targets.py          target sites for --target
 ├── output.py           result files
-├── exporters.py        proxychains, Clash and curl formats (--export)
+├── exporters.py        proxychains, Clash, sing-box and curl formats (--export)
 ├── server/             rotating proxy server (--serve): pool · http · upstream · socks · status · core
 ├── api.py              find_proxies() / check_proxies() for Python
 ├── agent.py            MCP tools without the SDK: live list, filters, fetch through proxies

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/proxy_scraper_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.9.0
+Version: 1.10.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -35,6 +35,7 @@ Provides-Extra: mcp
 Requires-Dist: mcp<3,>=2.2; python_version >= "3.10" and extra == "mcp"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
+Requires-Dist: hypothesis; extra == "dev"
 Requires-Dist: ruff==0.16.8; extra == "dev"
 Dynamic: license-file
 
@@ -138,11 +139,12 @@ python3 proxy_scraper.py
 
 Every [release](https://github.com/maximilianfeix/proxy-scraper/releases/latest) also ships a wheel you can install with `pip install <file>.whl`, and a multi-arch image (amd64/arm64) on `ghcr.io`. In the container the wizard never shows up, it runs straight away. For the proxy server use `--serve --serve-host 0.0.0.0` with `-p 127.0.0.1:8899:8899`, so the port is only open on your own machine.
 
-Tab completion for bash, zsh and fish:
+Tab completion for bash, zsh, fish and PowerShell:
 
 ```bash
 eval "$(proxy-scraper --completion zsh)"     # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
 proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
+proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
 Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
@@ -656,14 +658,14 @@ if __name__ == "__main__":
 
 In a Scrapy project, put `RotatingProxy` in `middlewares.py` and add it to `DOWNLOADER_MIDDLEWARES` in `settings.py`. For long crawls, reload the pool now and then – the list changes every hour.
 
-**proxychains, Clash / Mihomo** – ready-made configs with `--export`
+**proxychains, Clash / Mihomo, sing-box** – ready-made configs with `--export`
 
 ```bash
-proxy-scraper --want 30 -y --export proxychains,clash
+proxy-scraper --want 30 -y --export proxychains,clash,singbox
 proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.org
 ```
 
-`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
+`clash.yaml` has all HTTP and SOCKS5 proxies plus a `url-test` group that always picks the fastest. `singbox.json` does the same for sing-box, SOCKS4 included, and opens a local proxy: `sing-box run -c results/latest/singbox.json`, then use `127.0.0.1:2080` as HTTP or SOCKS5 proxy. All three leave out HTTP proxies that can't tunnel (`CONNECT`), because these tools tunnel everything.
 
 **In a pipe** – `-o -` prints the hits to stdout, the interface moves to stderr
 
@@ -679,6 +681,22 @@ export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
 pip download requests   # git, pip, npm & co. now go through the pool
 ```
 
+**In a GitHub workflow** – the action picks working proxies for the next steps
+
+```yaml
+- id: proxies
+  uses: maximilianfeix/proxy-scraper@v1
+  with:
+    types: socks5
+    https: true
+    min-uptime: 90        # on the list for 90 %+ of the week
+    works-on: google      # optional: google, reddit, amazon
+    recheck: true         # optional: check them again from the runner
+- run: curl -x "${{ steps.proxies.outputs.proxy }}" https://api.ipify.org
+```
+
+`proxy` is the fastest one, `file` a text file with all of them (up to `limit`, default 20), `count` how many. Without a match the step fails, unless `fail-if-empty: false`.
+
 **Without installing anything** – straight from the live list
 
 ```bash
@@ -722,16 +740,16 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `-t`, `--timeout S` | timeout per proxy (default: 8 s) |
 | `--discover` | search GitHub for new sources right now |
 | `--no-cache` | download every list again (unchanged ones are normally skipped via ETag) |
-| `--list-sources [N]` | show the source ranking |
+| `--list-sources [N]` | show the source ranking (add `--json` for scripts: url, status, hit rate, checks, last change) |
 | `--serve-host ADDR` | where the proxy server listens (default `127.0.0.1`; `0.0.0.0` for Docker, with a warning) |
 | `--serve-password SECRET` | clients must send this password in the proxy login; better set `PROXY_SCRAPER_SERVE_PASSWORD` |
 | `--rotate STRATEGY` · `--sticky SEC` | how the proxy server picks proxies, see [above](#proxy-server) |
 | `--serve-refill HOURS` | while serving, check fresh proxies every HOURS and add the hits to the pool |
 | `--serve [PORT]` | afterwards serve as a rotating proxy on `127.0.0.1:PORT` (default: 8899) |
 | `-o FILE` | also write all hits to this file; `-o -` prints them to stdout |
-| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `curl` or `all` |
+| `--export FORMATS` | extra files for other tools: `proxychains`, `clash`, `singbox`, `curl` or `all` |
 | `-V`, `--version` | print the version |
-| `--completion SHELL` | print the tab completion script for bash, zsh or fish |
+| `--completion SHELL` | print the tab completion script for bash, zsh, fish or PowerShell |
 
 Everything else: `proxy-scraper --help`
 
@@ -813,10 +831,12 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 ## Roadmap
 
-What's next is tracked in the milestone [**v1.8**](../../milestone/8) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
+
+- [ ] [Measure throughput, not just latency](../../issues/140)
+- [ ] [Daily snapshots as a dataset](../../issues/143)
 
-- [ ] [Protocol detection on the same connection](../../issues/44)
-- [ ] [sing-box export](../../issues/113), [PowerShell completion](../../issues/114), [shareable filters on the website](../../issues/116)
+Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 
 Shipped in [v1.7](../../milestone/7): an MCP server so AI agents get working proxies and can load pages through them, 28 new sources and a GitHub search that runs daily and keeps what it found, and a cleaner website.
 
@@ -861,7 +881,7 @@ proxyscraper/
 ├── geodb.py            DB-IP country database (monthly, binary search)
 ├── targets.py          target sites for --target
 ├── output.py           result files
-├── exporters.py        proxychains, Clash and curl formats (--export)
+├── exporters.py        proxychains, Clash, sing-box and curl formats (--export)
 ├── server/             rotating proxy server (--serve): pool · http · upstream · socks · status · core
 ├── api.py              find_proxies() / check_proxies() for Python
 ├── agent.py            MCP tools without the SDK: live list, filters, fetch through proxies

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/proxy_scraper_cli.egg-info/SOURCES.txt

@@ -9,6 +9,7 @@ proxy_scraper_cli.egg-info/requires.txt
 proxy_scraper_cli.egg-info/top_level.txt
 proxyscraper/__init__.py
 proxyscraper/__main__.py
+proxyscraper/action.py
 proxyscraper/agent.py
 proxyscraper/api.py
 proxyscraper/app.py
@@ -62,6 +63,7 @@ proxyscraper/ui/report.py
 proxyscraper/ui/serve.py
 proxyscraper/ui/widgets.py
 proxyscraper/ui/wizard.py
+tests/test_action.py
 tests/test_agent.py
 tests/test_api.py
 tests/test_app.py
@@ -86,6 +88,7 @@ tests/test_output_stdout.py
 tests/test_packaging.py
 tests/test_pages.py
 tests/test_parsing.py
+tests/test_parsing_fuzz.py
 tests/test_paths.py
 tests/test_pipeline.py
 tests/test_proxy_pages.py
@@ -97,8 +100,10 @@ tests/test_serve_password.py
 tests/test_serve_refill.py
 tests/test_serve_v2.py
 tests/test_server.py
+tests/test_server_closed_writer.py
 tests/test_server_fixes.py
 tests/test_sites.py
+tests/test_sources_json.py
 tests/test_tamper.py
 tests/test_targets.py
 tests/test_ui.py

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/proxy_scraper_cli.egg-info/requires.txt

@@ -3,6 +3,7 @@ certifi
 
 [dev]
 pytest
+hypothesis
 ruff==0.16.8
 
 [fast]

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/proxyscraper/__init__.py

@@ -1,7 +1,7 @@
 """proxy-scraper: collects public HTTP/SOCKS4/SOCKS5 proxies from hundreds of sources, checks them
 in parallel with its own protocol handshakes and learns which sources deliver good proxies."""
 
-__version__ = "1.9.0"
+__version__ = "1.10.0"
 
 
 def __getattr__(name):

proxy_scraper_cli-1.10.0/proxyscraper/action.py

@@ -0,0 +1,96 @@
+"""The GitHub Action (action.yml): working proxies from the hourly list for the next steps of a workflow.
+
+Inputs come in as PS_* environment variables, results go to $GITHUB_OUTPUT:
+  proxy   the fastest one, ready to use (socks5://1.2.3.4:1080)
+  file    a text file with all of them, one per line, fastest first
+  count   how many
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+import tempfile
+from pathlib import Path
+from typing import List
+
+from .api import check_proxies, live_proxies
+
+
+class InputError(ValueError):
+    pass
+
+
+def _flag(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name, "").strip().lower()
+    if not value:
+        return default
+    if value in ("true", "yes", "1"):
+        return True
+    if value in ("false", "no", "0"):
+        return False
+    raise InputError(f"{name[3:].lower().replace('_', '-')} must be true or false, not {value!r}")
+
+
+def _number(name: str, default: int) -> int:
+    value = os.environ.get(name, "").strip()
+    if not value:
+        return default
+    if not value.isdigit():
+        raise InputError(f"{name[3:].lower().replace('_', '-')} must be a whole number, not {value!r}")
+    return int(value)
+
+
+def _list(name: str) -> List[str]:
+    return [part.strip().lower() for part in os.environ.get(name, "").split(",") if part.strip()]
+
+
+def _output(**values: str) -> None:
+    path = os.environ.get("GITHUB_OUTPUT")
+    lines = "".join(f"{key}={value}\n" for key, value in values.items())
+    if path:
+        with open(path, "a", encoding="utf-8") as fh:
+            fh.write(lines)
+    else:
+        sys.stdout.write(lines)
+
+
+def main() -> int:
+    try:
+        limit = _number("PS_LIMIT", 20)
+        recheck = _flag("PS_RECHECK")
+        fail_if_empty = _flag("PS_FAIL_IF_EMPTY", True)
+        filters = {"types": _list("PS_TYPES") or ["http", "socks4", "socks5"],
+                   "countries": os.environ.get("PS_COUNTRIES", "").strip(),
+                   "https": _flag("PS_HTTPS"), "min_uptime": _number("PS_MIN_UPTIME", 0),
+                   "works_on": _list("PS_WORKS_ON")}
+        # a recheck starts from all of them – the limit applies to what still works from here
+        found = live_proxies(**filters, limit=0 if recheck else limit)
+    except (InputError, ValueError) as e:
+        print(f"::error::{e}")
+        return 1
+    except ConnectionError as e:
+        print(f"::error::The live list couldn't be loaded: {e}")
+        return 1
+    urls = [p.url for p in found]
+    if recheck and urls:
+        print(f"Checking {len(urls)} proxies again from this runner …")
+        urls = [r.url for r in sorted(check_proxies(urls, https=filters["https"]), key=lambda r: r.latency)]
+    if limit:
+        urls = urls[:limit]
+    # a file of its own per step: a job may use the action twice (socks5 in one step, http in the next)
+    fd, name = tempfile.mkstemp(prefix="proxies-", suffix=".txt", dir=os.environ.get("RUNNER_TEMP") or None)
+    with os.fdopen(fd, "w", encoding="utf-8") as fh:
+        fh.write("".join(f"{u}\n" for u in urls))
+    file = Path(name)
+    _output(proxy=urls[0] if urls else "", file=str(file), count=str(len(urls)))
+    print(f"{len(urls)} working {'proxy' if len(urls) == 1 else 'proxies'}" + (f", fastest: {urls[0]}" if urls else ""))
+    if not urls and fail_if_empty:
+        print("::error::No proxy matches these filters right now – loosen them (min-uptime, works-on, countries) "
+              "or set fail-if-empty: false")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/proxyscraper/cli.py

@@ -4,9 +4,11 @@ from __future__ import annotations
 
 import argparse
 import asyncio
+import json
 import os
 import sys
 from collections import Counter
+from datetime import datetime, timezone
 from typing import List, Optional
 
 from rich.console import Console
@@ -38,13 +40,22 @@ from .ui.keys import is_interactive
 from .ui.wizard import run_wizard
 
 
-def list_sources(limit: int) -> int:
-    """Ranking of every known source by learned hit rate."""
+def list_sources(limit: int, as_json: bool = False) -> int:
+    """Ranking of every known source by learned hit rate – as a table, or as JSON on stdout for scripts."""
     quality = srcs.SourceStats()
     curated, _meta = srcs.load_source_file()
     urls = set(curated) | set(srcs.load_discovered()) | set(quality.records)
     ranking = quality.ranking(urls)
     rows = [(u, r, quality.skip_reason(u) or "active") for u, r in ranking if r.runs][:limit]
+    if as_json:
+        def stamp(t: float) -> Optional[str]:
+            return datetime.fromtimestamp(t, timezone.utc).isoformat(timespec="seconds") if t else None
+        sys.stdout.write(json.dumps([
+            {"url": u, "status": status, "hit_rate": round(r.score, 4), "checked": round(r.checked),
+             "working": round(r.working), "proxies": r.count, "runs": r.runs,
+             "first_seen": stamp(r.first_seen), "last_change": stamp(r.last_change)}
+            for u, r, status in rows], indent=1) + "\n")
+        return 0
     reasons = Counter(quality.skip_reason(u) or "active" for u, _ in ranking)
     widgets.console.print(banner())
     render_source_ranking(rows, len(urls), reasons)
@@ -181,7 +192,7 @@ def parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
                         "(the interface moves to stderr then)")
     o.add_argument("--export", type=export_list, metavar="FORMATS",
                    help=f"extra formats in the results folder: {', '.join(EXPORTERS)} or all "
-                        "(e.g. --export proxychains,clash)")
+                        "(e.g. --export proxychains,singbox)")
 
     s = p.add_argument_group("Sources")
     s.add_argument("--discover", action="store_true",
@@ -194,7 +205,11 @@ def parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
                    help="reload every list completely (otherwise unchanged ones are skipped via ETag)")
     s.add_argument("--list-sources", nargs="?", const=50, type=positive_int, metavar="N",
                    help="show the source ranking by hit rate (default: top 50) and exit")
-    return p.parse_args(argv)
+    s.add_argument("--json", action="store_true", help="with --list-sources: print the ranking as JSON to stdout")
+    args = p.parse_args(argv)
+    if args.json and args.list_sources is None:
+        p.error("--json only works together with --list-sources")
+    return args
 
 
 def last_options() -> Optional[RunOptions]:
@@ -244,7 +259,7 @@ def run(argv: Optional[List[str]] = None) -> int:
         from .mcp_entry import main as mcp_main
         return mcp_main()
     if args.list_sources is not None:
-        return list_sources(args.list_sources)
+        return list_sources(args.list_sources, args.json)
     opts = RunOptions.from_args(args)
     if opts.output == STDOUT:
         widgets.console = Console(highlight=False, stderr=True)  # stdout only carries the hits

{proxy_scraper_cli-1.9.0 → proxy_scraper_cli-1.10.0}/proxyscraper/completion.py

@@ -1,4 +1,4 @@
-"""Tab completion for bash, zsh and fish – generated straight from the argparse parser.
+"""Tab completion for bash, zsh, fish and PowerShell – generated straight from the argparse parser.
 
 That way the script can never go stale: every new option shows up automatically, with its choices
 (--types, --rotate, …) and the start of its help text as the description (zsh, fish).
@@ -12,7 +12,7 @@ import sys
 from dataclasses import dataclass
 from typing import List, Optional, Tuple
 
-SHELLS = ("bash", "zsh", "fish")
+SHELLS = ("bash", "zsh", "fish", "powershell")
 PROG = "proxy-scraper"
 FILE_OPTIONS = {"--recheck": ("live",), "--output": ()}  # the value is a file path (or one of these words)
 OWN_HELP = {"help": "show help", "version": "show the version"}  # shorter than argparse's own texts
@@ -181,8 +181,66 @@ def fish(opts: List[Option]) -> str:
     return "\n".join(lines) + "\n"
 
 
+def _ps_quote(text: str) -> str:
+    return "'" + text.replace("'", "''") + "'"
+
+
+def powershell(opts: List[Option]) -> str:
+    """A native argument completer: options with their help as tooltip, the choices after an option (also for
+    several values in a row, --types http socks5), and PowerShell's own path completion for file options."""
+    rows = []
+    for o in opts:
+        # an optional value without choices (--serve, --list-sources): options may follow right away, like in bash
+        expects_value = o.takes_value and not (o.optional_value and not o.is_file and not o.choices)
+        fields = [f"Flags = @({', '.join(_ps_quote(f) for f in o.flags)})", f"Help = {_ps_quote(o.help)}",
+                  f"Choices = @({', '.join(_ps_quote(c) for c in (*o.choices, *o.extra))})",
+                  f"Value = ${str(expects_value).lower()}", f"Multi = ${str(o.multi).lower()}",
+                  f"File = ${str(o.is_file).lower()}"]
+        rows.append("        @{ " + "; ".join(fields) + " }")
+    table = ",\n".join(rows)
+    return f"""# PowerShell completion for {PROG}
+# enable: {PROG} --completion powershell | Out-String | Invoke-Expression   (add that line to $PROFILE)
+Register-ArgumentCompleter -Native -CommandName '{PROG}' -ScriptBlock {{
+    param($wordToComplete, $commandAst, $cursorPosition)
+    $options = @(
+{table}
+    )
+    $words = @($commandAst.CommandElements | Where-Object {{ $_.Extent.EndOffset -le $cursorPosition }} |
+               ForEach-Object {{ $_.ToString() }})
+    if ($wordToComplete -and $words.Count -gt 0 -and $words[-1] -eq $wordToComplete) {{
+        $words = @($words | Select-Object -SkipLast 1)
+    }}
+    # the last option before the cursor, and whether the cursor is right after it
+    $last = $null; $distance = 0
+    for ($i = $words.Count - 1; $i -ge 1; $i--) {{
+        if ($words[$i].StartsWith('-')) {{ $last = $options | Where-Object {{ $_.Flags -contains $words[$i] }} |
+                                          Select-Object -First 1; break }}
+        $distance++
+    }}
+    if ($last -and $last.Value -and -not $wordToComplete.StartsWith('-') -and ($distance -eq 0 -or $last.Multi)) {{
+        if ($last.Choices.Count -gt 0) {{
+            $last.Choices | Where-Object {{ $_ -like "$wordToComplete*" }} | ForEach-Object {{
+                [System.Management.Automation.CompletionResult]::new($_, $_, 'ParameterValue', $_)
+            }}
+            if ($last.File) {{  # a file works too (--recheck live or a path)
+                [System.Management.Automation.CompletionCompleters]::CompleteFilename($wordToComplete)
+            }}
+        }}
+        return  # nothing to suggest (a number, a path): PowerShell falls back to its own completion
+    }}
+    foreach ($option in $options) {{
+        foreach ($flag in $option.Flags) {{
+            if ($flag -like "$wordToComplete*") {{
+                [System.Management.Automation.CompletionResult]::new($flag, $flag, 'ParameterName', $option.Help)
+            }}
+        }}
+    }}
+}}
+"""
+
+
 def script(parser: argparse.ArgumentParser, shell: str) -> str:
-    return {"bash": bash, "zsh": zsh, "fish": fish}[shell](options(parser))
+    return {"bash": bash, "zsh": zsh, "fish": fish, "powershell": powershell}[shell](options(parser))
 
 
 class CompletionAction(argparse.Action):