proxy-scraper-cli 1.23.0__tar.gz → 1.24.0__tar.gz

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.23.0
+Version: 1.24.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -75,16 +75,23 @@ Most free proxy lists are 95 % dead, and a good part of the rest are honeypots o
 
 **In numbers:** in one day it checked 3.6 million free proxies – 1.1 % worked, and more than half of the ones that answered failed a second request. [What's actually out there →](docs/free-proxies-in-numbers.md)
 
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
-  <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
-  <img src="docs/checks-dark.svg" alt="The five checks: 700+ lists, a real handshake, two sites with one IP, nothing injected, the details" width="100%">
-</picture>
+**Try it – no install, about five seconds:**
+
+```bash
+uvx proxy-scraper-cli --pick 5 --types socks5     # 5 checked proxies
+
+# or just the plain list
+curl -s https://maximilianfeix.github.io/proxy-scraper/socks5.txt | head
+```
+
+`uvx` comes with [uv](https://docs.astral.sh/uv/); `pipx run proxy-scraper-cli --pick 5` does the same.
 
 <div align="center">
-<img src="docs/demo.svg" alt="Animated demo: setup wizard, collecting, live dashboard and final report" width="880">
+<img src="docs/demo.svg" alt="Animated demo: the live dashboard during a check, the report, the setup wizard and collecting" width="880">
 </div>
 
+> **Who runs free proxies – and is it safe to use them?** Mostly nobody on purpose: misconfigured servers with an open port, a few deliberately open ones, infected machines, and traps that log or rewrite your traffic. That's why every proxy here has to pass [five checks](#how-it-works), and why they're for public data and testing – never for logins, payments or anything personal. [More on where they come from →](#who-runs-free-proxies)
+
 <details>
 <summary><b>Table of contents</b></summary>
 
@@ -222,7 +229,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok · Discord (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) · [discord.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/discord.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -460,7 +467,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`, `"discord"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 Or let the library do the retrying: `ProxyRotator` loads a URL through the list and moves on to the next proxy when one fails – HTTPS only through proxies with verified TLS.
 
@@ -583,6 +590,13 @@ The live list can also come to you: [`bot/`](bot/) is a Discord bot that posts e
 
 ## How it works
 
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
+  <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
+  <img src="docs/checks-dark.svg" alt="The five checks: 700+ lists, a real handshake, two sites with one IP, nothing injected, the details" width="100%">
+</picture>
+
+
 ```mermaid
 flowchart LR
     A[sources.json<br/>meta lists<br/>GitHub discovery] --> B[Fetch & parse<br/>in parallel on all cores]
@@ -777,6 +791,14 @@ Username wishes – `country-us` sends everything out through US proxies, `sessi
 proxy-scraper --recheck live --want 20 -y -o - | grep '^socks5://' > socks.txt
 ```
 
+**Check a list you already have** – `--recheck -` reads it from stdin: plain `ip:port` lines, `type://ip:port`, logins, JSON, a CSV export or a table pasted from a website (HTML or Markdown), in the order it came in. Only what works from your network comes out:
+
+```bash
+curl -s https://example.com/proxies.txt | proxy-scraper --recheck - -o - > working.txt
+```
+
+Proxies without a type are tried as HTTP and as SOCKS5 – or as SOCKS4 with `--types socks4` alone. A password with a comma, quote, `|` or bracket can't be told apart from the text around it in a paste; put such proxies in a file, one per line, and use `--recheck file.txt`.
+
 **Any tool, through the rotating server**
 
 ```bash
@@ -849,7 +871,7 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `--no-blocklisted` | skip proxies whose exit IP is on the SpamCop blocklist – those often get captchas |
 | `--no-dnsbl` | skip the blocklist lookup |
 | `--target URL` | only proxies that reach this site (repeatable) |
-| `--recheck [FILE\|live]` | only check proxies from a file, the last run, or the public live list |
+| `--recheck [FILE\|-\|live]` | only check proxies from a file, stdin (`-`), the last run, or the public live list |
 | `--fast` | skip the HTTPS test (confirmation and anonymity still run) |
 | `--no-geo` | skip the country lookup |
 | `-c`, `--concurrency N` | simultaneous checks (default: 2000) |
@@ -896,6 +918,25 @@ The repo does part of the work itself:
 
 ## FAQ
 
+<a id="who-runs-free-proxies"></a>
+
+<details open>
+<summary><b>Who runs free proxies – and should you use them?</b></summary>
+<br>
+
+Almost nobody runs a free proxy for you. What ends up on public lists is usually one of these:
+
+- **Misconfigured servers** – a Squid, a router or an app with a proxy port that was meant to stay internal.
+- **Deliberately open proxies** – a few volunteers and research projects, and services that give some away to sell the rest.
+- **Infected machines** – computers and IoT devices whose owners don't know they relay traffic. Many residential exits are like this.
+- **Traps** – honeypots that log what goes through them, and proxies that rewrite pages to inject ads or scripts.
+
+proxy-scraper can't tell an infected home router from an open office proxy, but it filters what it can measure: a proxy has to answer two independent requests with the same exit IP (honeypots that only answer scanners drop out), deliver a known page byte for byte (injectors drop out), and HTTPS counts only with a certificate that verifies end to end. Each hit says whether its exit is in a datacenter.
+
+Good uses: public data, checking how a site looks from another country, testing your own blocking and rate limits, research. Never send passwords, cookies, payment or personal data through a free proxy, and keep HTTPS certificate checks on. Use them only where you're allowed to.
+
+</details>
+
 <details>
 <summary><b>Almost nothing gets through.</b></summary>
 <br>
@@ -950,6 +991,8 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
 
+Shipped in v1.24: `--recheck -` for lists you pipe in, which proxies get through to Discord, and a README you can try in five seconds.
+
 Shipped in v1.22 and v1.23: a live dashboard for the rotating server, `--export pac` and an hourly `proxy.pac`, "best first" that counts how likely a proxy is still up, the lists in a repository of their own ([free-proxy-list](https://github.com/maximilianfeix/free-proxy-list)), daily snapshots on Hugging Face, `brew install`, a Chinese README, Python 3.14, and a recipe for Burp Suite, ZAP and mitmproxy.
 
 Shipped in v1.10 to v1.21: download speed per proxy and "best first" everywhere, daily snapshots, a rotating proxy URL for requests, httpx and Playwright, `--pick` for working proxies without a scan, a proxy pool API compatible with jhao104/proxy_pool, your own lists with `--source`, a GitHub Action, a Telegram bot and an Atom feed for the weekly report.
@@ -1020,6 +1063,14 @@ proxyscraper/
 
 Questions, ideas and things you built with it go to [Discussions](https://github.com/maximilianfeix/proxy-scraper/discussions) – bugs to the [issues](../../issues). If proxy-scraper saves you time, a ⭐ helps others find it.
 
+Thanks to everyone who helped build it – new sources, countries, Python 3.14, examples and more:
+
+<a href="https://github.com/maximilianfeix/proxy-scraper/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=maximilianfeix/proxy-scraper" alt="Contributors to proxy-scraper">
+</a>
+
+Want to join? [Beginner-friendly issues](../../issues?q=is%3Aopen+label%3A%22good+first+issue%22) say where to start.
+
 <a href="https://star-history.com/#maximilianfeix/proxy-scraper&Date">
   <picture>
     <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=maximilianfeix/proxy-scraper&type=Date&theme=dark">

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/README.md

@@ -33,16 +33,23 @@ Most free proxy lists are 95 % dead, and a good part of the rest are honeypots o
 
 **In numbers:** in one day it checked 3.6 million free proxies – 1.1 % worked, and more than half of the ones that answered failed a second request. [What's actually out there →](docs/free-proxies-in-numbers.md)
 
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
-  <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
-  <img src="docs/checks-dark.svg" alt="The five checks: 700+ lists, a real handshake, two sites with one IP, nothing injected, the details" width="100%">
-</picture>
+**Try it – no install, about five seconds:**
+
+```bash
+uvx proxy-scraper-cli --pick 5 --types socks5     # 5 checked proxies
+
+# or just the plain list
+curl -s https://maximilianfeix.github.io/proxy-scraper/socks5.txt | head
+```
+
+`uvx` comes with [uv](https://docs.astral.sh/uv/); `pipx run proxy-scraper-cli --pick 5` does the same.
 
 <div align="center">
-<img src="docs/demo.svg" alt="Animated demo: setup wizard, collecting, live dashboard and final report" width="880">
+<img src="docs/demo.svg" alt="Animated demo: the live dashboard during a check, the report, the setup wizard and collecting" width="880">
 </div>
 
+> **Who runs free proxies – and is it safe to use them?** Mostly nobody on purpose: misconfigured servers with an open port, a few deliberately open ones, infected machines, and traps that log or rewrite your traffic. That's why every proxy here has to pass [five checks](#how-it-works), and why they're for public data and testing – never for logins, payments or anything personal. [More on where they come from →](#who-runs-free-proxies)
+
 <details>
 <summary><b>Table of contents</b></summary>
 
@@ -180,7 +187,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok · Discord (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) · [discord.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/discord.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -418,7 +425,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`, `"discord"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 Or let the library do the retrying: `ProxyRotator` loads a URL through the list and moves on to the next proxy when one fails – HTTPS only through proxies with verified TLS.
 
@@ -541,6 +548,13 @@ The live list can also come to you: [`bot/`](bot/) is a Discord bot that posts e
 
 ## How it works
 
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
+  <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
+  <img src="docs/checks-dark.svg" alt="The five checks: 700+ lists, a real handshake, two sites with one IP, nothing injected, the details" width="100%">
+</picture>
+
+
 ```mermaid
 flowchart LR
     A[sources.json<br/>meta lists<br/>GitHub discovery] --> B[Fetch & parse<br/>in parallel on all cores]
@@ -735,6 +749,14 @@ Username wishes – `country-us` sends everything out through US proxies, `sessi
 proxy-scraper --recheck live --want 20 -y -o - | grep '^socks5://' > socks.txt
 ```
 
+**Check a list you already have** – `--recheck -` reads it from stdin: plain `ip:port` lines, `type://ip:port`, logins, JSON, a CSV export or a table pasted from a website (HTML or Markdown), in the order it came in. Only what works from your network comes out:
+
+```bash
+curl -s https://example.com/proxies.txt | proxy-scraper --recheck - -o - > working.txt
+```
+
+Proxies without a type are tried as HTTP and as SOCKS5 – or as SOCKS4 with `--types socks4` alone. A password with a comma, quote, `|` or bracket can't be told apart from the text around it in a paste; put such proxies in a file, one per line, and use `--recheck file.txt`.
+
 **Any tool, through the rotating server**
 
 ```bash
@@ -807,7 +829,7 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `--no-blocklisted` | skip proxies whose exit IP is on the SpamCop blocklist – those often get captchas |
 | `--no-dnsbl` | skip the blocklist lookup |
 | `--target URL` | only proxies that reach this site (repeatable) |
-| `--recheck [FILE\|live]` | only check proxies from a file, the last run, or the public live list |
+| `--recheck [FILE\|-\|live]` | only check proxies from a file, stdin (`-`), the last run, or the public live list |
 | `--fast` | skip the HTTPS test (confirmation and anonymity still run) |
 | `--no-geo` | skip the country lookup |
 | `-c`, `--concurrency N` | simultaneous checks (default: 2000) |
@@ -854,6 +876,25 @@ The repo does part of the work itself:
 
 ## FAQ
 
+<a id="who-runs-free-proxies"></a>
+
+<details open>
+<summary><b>Who runs free proxies – and should you use them?</b></summary>
+<br>
+
+Almost nobody runs a free proxy for you. What ends up on public lists is usually one of these:
+
+- **Misconfigured servers** – a Squid, a router or an app with a proxy port that was meant to stay internal.
+- **Deliberately open proxies** – a few volunteers and research projects, and services that give some away to sell the rest.
+- **Infected machines** – computers and IoT devices whose owners don't know they relay traffic. Many residential exits are like this.
+- **Traps** – honeypots that log what goes through them, and proxies that rewrite pages to inject ads or scripts.
+
+proxy-scraper can't tell an infected home router from an open office proxy, but it filters what it can measure: a proxy has to answer two independent requests with the same exit IP (honeypots that only answer scanners drop out), deliver a known page byte for byte (injectors drop out), and HTTPS counts only with a certificate that verifies end to end. Each hit says whether its exit is in a datacenter.
+
+Good uses: public data, checking how a site looks from another country, testing your own blocking and rate limits, research. Never send passwords, cookies, payment or personal data through a free proxy, and keep HTTPS certificate checks on. Use them only where you're allowed to.
+
+</details>
+
 <details>
 <summary><b>Almost nothing gets through.</b></summary>
 <br>
@@ -908,6 +949,8 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
 
+Shipped in v1.24: `--recheck -` for lists you pipe in, which proxies get through to Discord, and a README you can try in five seconds.
+
 Shipped in v1.22 and v1.23: a live dashboard for the rotating server, `--export pac` and an hourly `proxy.pac`, "best first" that counts how likely a proxy is still up, the lists in a repository of their own ([free-proxy-list](https://github.com/maximilianfeix/free-proxy-list)), daily snapshots on Hugging Face, `brew install`, a Chinese README, Python 3.14, and a recipe for Burp Suite, ZAP and mitmproxy.
 
 Shipped in v1.10 to v1.21: download speed per proxy and "best first" everywhere, daily snapshots, a rotating proxy URL for requests, httpx and Playwright, `--pick` for working proxies without a scan, a proxy pool API compatible with jhao104/proxy_pool, your own lists with `--source`, a GitHub Action, a Telegram bot and an Atom feed for the weekly report.
@@ -978,6 +1021,14 @@ proxyscraper/
 
 Questions, ideas and things you built with it go to [Discussions](https://github.com/maximilianfeix/proxy-scraper/discussions) – bugs to the [issues](../../issues). If proxy-scraper saves you time, a ⭐ helps others find it.
 
+Thanks to everyone who helped build it – new sources, countries, Python 3.14, examples and more:
+
+<a href="https://github.com/maximilianfeix/proxy-scraper/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=maximilianfeix/proxy-scraper" alt="Contributors to proxy-scraper">
+</a>
+
+Want to join? [Beginner-friendly issues](../../issues?q=is%3Aopen+label%3A%22good+first+issue%22) say where to start.
+
 <a href="https://star-history.com/#maximilianfeix/proxy-scraper&Date">
   <picture>
     <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=maximilianfeix/proxy-scraper&type=Date&theme=dark">

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/proxy_scraper_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.23.0
+Version: 1.24.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -75,16 +75,23 @@ Most free proxy lists are 95 % dead, and a good part of the rest are honeypots o
 
 **In numbers:** in one day it checked 3.6 million free proxies – 1.1 % worked, and more than half of the ones that answered failed a second request. [What's actually out there →](docs/free-proxies-in-numbers.md)
 
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
-  <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
-  <img src="docs/checks-dark.svg" alt="The five checks: 700+ lists, a real handshake, two sites with one IP, nothing injected, the details" width="100%">
-</picture>
+**Try it – no install, about five seconds:**
+
+```bash
+uvx proxy-scraper-cli --pick 5 --types socks5     # 5 checked proxies
+
+# or just the plain list
+curl -s https://maximilianfeix.github.io/proxy-scraper/socks5.txt | head
+```
+
+`uvx` comes with [uv](https://docs.astral.sh/uv/); `pipx run proxy-scraper-cli --pick 5` does the same.
 
 <div align="center">
-<img src="docs/demo.svg" alt="Animated demo: setup wizard, collecting, live dashboard and final report" width="880">
+<img src="docs/demo.svg" alt="Animated demo: the live dashboard during a check, the report, the setup wizard and collecting" width="880">
 </div>
 
+> **Who runs free proxies – and is it safe to use them?** Mostly nobody on purpose: misconfigured servers with an open port, a few deliberately open ones, infected machines, and traps that log or rewrite your traffic. That's why every proxy here has to pass [five checks](#how-it-works), and why they're for public data and testing – never for logins, payments or anything personal. [More on where they come from →](#who-runs-free-proxies)
+
 <details>
 <summary><b>Table of contents</b></summary>
 
@@ -222,7 +229,7 @@ Every protocol and country also has its own page with a plain download, e.g. [SO
 | HTTP · SOCKS4 · SOCKS5 | `1.2.3.4:8080` | [http.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/http.txt) · [socks4.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks4.txt) · [socks5.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt) |
 | HTTPS-capable only | `type://ip:port` | [https.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt) |
 | Elite only | `type://ip:port` | [elite.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/elite.txt) |
-| Gets through to Google · Reddit · Amazon · Instagram · TikTok (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) |
+| Gets through to Google · Reddit · Amazon · Instagram · TikTok · Discord (no captcha, no block in the last run) | `type://ip:port` | [google.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/google.txt) · [reddit.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/reddit.txt) · [amazon.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/amazon.txt) · [instagram.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/instagram.txt) · [tiktok.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/tiktok.txt) · [discord.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/works-with/discord.txt) |
 | Stable, on the list in 90 %+ of this week's runs | `type://ip:port` | [stable.txt](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/stable.txt) |
 | With all details | latency, country, HTTPS, anonymity, exit IP, uptime, sites | [proxies.json](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.json) · [proxies.csv](https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/proxies.csv) |
 
@@ -460,7 +467,7 @@ pool = itertools.cycle(p.url for p in proxies)
 r = requests.get("https://api.ipify.org", proxies={"https": next(pool)}, timeout=15)  # pip install "requests[socks]"
 ```
 
-Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
+Same filters as `find_proxies`, plus `min_uptime`, `works_on` (`["google"]`, `"reddit"`, `"amazon"`, `"instagram"`, `"tiktok"`, `"discord"`) and `limit`. Every result also has `uptime_24h`, `uptime_7d`, `first_seen`, `up_for_hours` and `sites`.
 
 Or let the library do the retrying: `ProxyRotator` loads a URL through the list and moves on to the next proxy when one fails – HTTPS only through proxies with verified TLS.
 
@@ -583,6 +590,13 @@ The live list can also come to you: [`bot/`](bot/) is a Discord bot that posts e
 
 ## How it works
 
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
+  <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
+  <img src="docs/checks-dark.svg" alt="The five checks: 700+ lists, a real handshake, two sites with one IP, nothing injected, the details" width="100%">
+</picture>
+
+
 ```mermaid
 flowchart LR
     A[sources.json<br/>meta lists<br/>GitHub discovery] --> B[Fetch & parse<br/>in parallel on all cores]
@@ -777,6 +791,14 @@ Username wishes – `country-us` sends everything out through US proxies, `sessi
 proxy-scraper --recheck live --want 20 -y -o - | grep '^socks5://' > socks.txt
 ```
 
+**Check a list you already have** – `--recheck -` reads it from stdin: plain `ip:port` lines, `type://ip:port`, logins, JSON, a CSV export or a table pasted from a website (HTML or Markdown), in the order it came in. Only what works from your network comes out:
+
+```bash
+curl -s https://example.com/proxies.txt | proxy-scraper --recheck - -o - > working.txt
+```
+
+Proxies without a type are tried as HTTP and as SOCKS5 – or as SOCKS4 with `--types socks4` alone. A password with a comma, quote, `|` or bracket can't be told apart from the text around it in a paste; put such proxies in a file, one per line, and use `--recheck file.txt`.
+
 **Any tool, through the rotating server**
 
 ```bash
@@ -849,7 +871,7 @@ curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.org
 | `--no-blocklisted` | skip proxies whose exit IP is on the SpamCop blocklist – those often get captchas |
 | `--no-dnsbl` | skip the blocklist lookup |
 | `--target URL` | only proxies that reach this site (repeatable) |
-| `--recheck [FILE\|live]` | only check proxies from a file, the last run, or the public live list |
+| `--recheck [FILE\|-\|live]` | only check proxies from a file, stdin (`-`), the last run, or the public live list |
 | `--fast` | skip the HTTPS test (confirmation and anonymity still run) |
 | `--no-geo` | skip the country lookup |
 | `-c`, `--concurrency N` | simultaneous checks (default: 2000) |
@@ -896,6 +918,25 @@ The repo does part of the work itself:
 
 ## FAQ
 
+<a id="who-runs-free-proxies"></a>
+
+<details open>
+<summary><b>Who runs free proxies – and should you use them?</b></summary>
+<br>
+
+Almost nobody runs a free proxy for you. What ends up on public lists is usually one of these:
+
+- **Misconfigured servers** – a Squid, a router or an app with a proxy port that was meant to stay internal.
+- **Deliberately open proxies** – a few volunteers and research projects, and services that give some away to sell the rest.
+- **Infected machines** – computers and IoT devices whose owners don't know they relay traffic. Many residential exits are like this.
+- **Traps** – honeypots that log what goes through them, and proxies that rewrite pages to inject ads or scripts.
+
+proxy-scraper can't tell an infected home router from an open office proxy, but it filters what it can measure: a proxy has to answer two independent requests with the same exit IP (honeypots that only answer scanners drop out), deliver a known page byte for byte (injectors drop out), and HTTPS counts only with a certificate that verifies end to end. Each hit says whether its exit is in a datacenter.
+
+Good uses: public data, checking how a site looks from another country, testing your own blocking and rate limits, research. Never send passwords, cookies, payment or personal data through a free proxy, and keep HTTPS certificate checks on. Use them only where you're allowed to.
+
+</details>
+
 <details>
 <summary><b>Almost nothing gets through.</b></summary>
 <br>
@@ -950,6 +991,8 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
 
+Shipped in v1.24: `--recheck -` for lists you pipe in, which proxies get through to Discord, and a README you can try in five seconds.
+
 Shipped in v1.22 and v1.23: a live dashboard for the rotating server, `--export pac` and an hourly `proxy.pac`, "best first" that counts how likely a proxy is still up, the lists in a repository of their own ([free-proxy-list](https://github.com/maximilianfeix/free-proxy-list)), daily snapshots on Hugging Face, `brew install`, a Chinese README, Python 3.14, and a recipe for Burp Suite, ZAP and mitmproxy.
 
 Shipped in v1.10 to v1.21: download speed per proxy and "best first" everywhere, daily snapshots, a rotating proxy URL for requests, httpx and Playwright, `--pick` for working proxies without a scan, a proxy pool API compatible with jhao104/proxy_pool, your own lists with `--source`, a GitHub Action, a Telegram bot and an Atom feed for the weekly report.
@@ -1020,6 +1063,14 @@ proxyscraper/
 
 Questions, ideas and things you built with it go to [Discussions](https://github.com/maximilianfeix/proxy-scraper/discussions) – bugs to the [issues](../../issues). If proxy-scraper saves you time, a ⭐ helps others find it.
 
+Thanks to everyone who helped build it – new sources, countries, Python 3.14, examples and more:
+
+<a href="https://github.com/maximilianfeix/proxy-scraper/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=maximilianfeix/proxy-scraper" alt="Contributors to proxy-scraper">
+</a>
+
+Want to join? [Beginner-friendly issues](../../issues?q=is%3Aopen+label%3A%22good+first+issue%22) say where to start.
+
 <a href="https://star-history.com/#maximilianfeix/proxy-scraper&Date">
   <picture>
     <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=maximilianfeix/proxy-scraper&type=Date&theme=dark">

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/proxy_scraper_cli.egg-info/SOURCES.txt

@@ -105,6 +105,7 @@ tests/test_preferences.py
 tests/test_proxy_pages.py
 tests/test_proxy_sources.py
 tests/test_publish.py
+tests/test_recheck_stdin.py
 tests/test_report.py
 tests/test_robustness.py
 tests/test_rotator.py

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/proxyscraper/__init__.py

@@ -1,7 +1,7 @@
 """proxy-scraper: collects public HTTP/SOCKS4/SOCKS5 proxies from hundreds of sources, checks them
 in parallel with its own protocol handshakes and learns which sources deliver good proxies."""
 
-__version__ = "1.23.0"
+__version__ = "1.24.0"
 
 
 def __getattr__(name):

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/proxyscraper/api.py

@@ -215,7 +215,7 @@ async def live_proxies_async(*, types: Iterable[str] = PROXY_TYPES, countries: I
     Same filters as find_proxies, plus
     min_uptime  only proxies listed in at least this share (percent) of the week's runs, e.g. 90
     works_on    only proxies that got through to these sites in the last run: google, reddit, amazon, instagram,
-                tiktok
+                tiktok, discord
     min_speed   only proxies that downloaded at least this many KiB/s in the last run
     limit       at most this many (0 = all)
     """

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/proxyscraper/app.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 import asyncio
 import contextlib
 import ipaddress
+import re
 import socket
 import sys
 import tempfile
@@ -12,7 +13,7 @@ import time
 from collections import Counter
 from dataclasses import replace
 from pathlib import Path
-from typing import List, Optional, Tuple
+from typing import Dict, List, Optional, Tuple
 
 from rich.live import Live
 from rich.text import Text
@@ -42,7 +43,17 @@ from .netio import INSECURE_HOSTS, http_get
 from .options import STDOUT, RunOptions
 from .output import ResultWriter, latest_results
 from .pages import REPO_URL
-from .parsing import PROXY_TYPES, parse_keys, split_key
+from .parsing import (
+    JSON_IP_PORT_RE,
+    JSON_PORT_IP_RE,
+    PROXY_RE,
+    PROXY_TYPES,
+    TYPE_ALIASES,
+    parse_blob,
+    parse_keys,
+    parse_proxy_line,
+    split_key,
+)
 from .paths import is_checkout
 from .pipeline import (
     CheckRun,
@@ -114,9 +125,90 @@ async def confirm_target() -> Optional[str]:
     return ip if await probe_confirm_target(ip, timeout=8) else None
 
 
+STDIN = "-"  # --recheck -: the candidates come from a pipe
+
+
+def read_stdin() -> bytes:
+    """What was piped in, as bytes – a stray non-UTF-8 byte in a pasted table mustn't sink the whole list.
+    A terminal on stdin means nothing was piped: reading it would just wait."""
+    if sys.stdin is None or sys.stdin.isatty():
+        raise OSError(0, "nothing was piped in (try: cat proxies.txt | proxy-scraper --recheck -)")
+    buffer = getattr(sys.stdin, "buffer", None)
+    data = buffer.read() if buffer is not None else sys.stdin.read().encode("utf-8", "replace")
+    return data[3:] if data.startswith(b"\xef\xbb\xbf") else data  # the BOM Notepad and PowerShell write
+
+
+# a typed proxy anywhere in a word – at its start, or after "url":" in compact JSON; it ends at a quote, comma or
+# bracket, which a password could in theory contain but practically never does
+_TYPED_PROXY = re.compile(
+    rb"(?<![A-Za-z0-9])(?:" + b"|".join(s.encode() for s in TYPE_ALIASES if TYPE_ALIASES[s] != "auto")
+    + rb")://[^\s\"'`,;()\[\]{}<>]+", re.IGNORECASE)
+# what separates an untyped proxy from the text around it: spaces, but also the quotes, commas, pipes and
+# brackets of compact JSON, a CSV row or a Markdown table – otherwise '"proxy":"user:pass@ip:port' would read
+# as a login of '"proxy":"user'
+_WORD_SPLIT = re.compile(rb"[\s\"'`,;|()\[\]{}<>]+")
+# what a proxy user name is made of; anything else in front of "@" ("proxy=user", "US|user") is text glued on,
+# and the address is then checked without a login rather than with a made-up one
+_LOGIN_USER = re.compile(r"[A-Za-z0-9._~+%-]+")
+
+
+# IP and port in neighbouring columns of a CSV export or a Markdown table: "8.8.4.4,8080" / "| 8.8.4.4 | 8080 |"
+_COLUMN_PAIR = re.compile(rb"(?<![\d.])(\d{1,3}(?:\.\d{1,3}){3})\s*[,;|]\s*(\d{2,5})(?![\d.])")
+
+
+def _address_positions(data: bytes) -> Dict[str, int]:
+    """Where each ip:port first appears, in any layout the extraction reads: ip:port, "ip port", a table cell
+    pair, JSON in either order – so a list keeps its order whatever it looks like."""
+    hits = [(m.start(), m.group(1), m.group(2))
+            for regex in (PROXY_RE, JSON_IP_PORT_RE, _COLUMN_PAIR) for m in regex.finditer(data)]
+    hits += [(m.start(), m.group(2), m.group(1)) for m in JSON_PORT_IP_RE.finditer(data)]
+    first: Dict[str, int] = {}
+    for at, ip, port in sorted(hits):
+        first.setdefault(f"{'.'.join(str(int(p)) for p in ip.split(b'.'))}:{int(port)}", at)
+    return first
+
+
+def stdin_keys(data: bytes, types) -> List[str]:
+    """Every proxy in the piped text, in the order it came in (lists often come best first, and --limit takes
+    the first jobs).
+
+    Every type://… in the text is read as a proxy on its own – at the start of a line, after other text,
+    several on a line, inside JSON – which also keeps a login whose password has '@' or '/'. Untyped proxies –
+    a bare word like user:pass@ip:port (read whole, so the login stays), or ip/port inside JSON or a table –
+    are tried as HTTP and right after as SOCKS5, or as the types asked for when neither of those is (a list of
+    SOCKS4 proxies piped in with --types socks4). Not the "auto" mode of sources: its "plain lines only count
+    as the majority" rule is for scraped lists, and whoever pipes a list in means all of it."""
+    wanted = tuple(types)
+    bare_types = tuple(t for t in ("http", "socks5") if t in wanted) or wanted
+
+    def address(key: str) -> str:
+        return split_key(key)[1].rpartition("@")[2]
+
+    typed = {k for k in (parse_proxy_line(m.group(0).decode("utf-8", "replace"), None)
+                         for m in _TYPED_PROXY.finditer(data)) if k}
+    words = [w.decode("utf-8", "replace") for w in _WORD_SPLIT.split(data) if w and b"://" not in w]
+    words = [w for w in words if "@" not in w or _LOGIN_USER.fullmatch(w.rpartition("@")[0].partition(":")[0])]
+    whole_words = {k for w in words for t in bare_types for k in [parse_proxy_line(w, t)] if k}
+    # full: sources pick the "ip port"/table regex from their first 8 KB, a paste may be a.txt then b.html
+    found = {k for t in bare_types for k in parse_blob(data, t, wanted, full=True).split("\n") if k}
+    columns = [f"{m.group(1).decode()}:{m.group(2).decode()}" for m in _COLUMN_PAIR.finditer(data)]
+    found |= {k for pair in columns for t in bare_types for k in [parse_proxy_line(pair, t)] if k}
+    # an address read with its login (or with a type of its own) owns it: the regex's login-less copy, or a
+    # bare copy of a typed proxy of an unwanted type, mustn't come back. Claimed before the type filter.
+    claimed = {address(k) for k in typed} | {address(k) for k in whole_words if "@" in split_key(k)[1]}
+    keys = ({k for k in typed | whole_words if split_key(k)[0] in wanted}
+            | {k for k in found if address(k) not in claimed})
+
+    first = _address_positions(data)
+    rank = {t: i for i, t in enumerate(PROXY_TYPES)}  # http before socks4 before socks5 for the same address
+    return sorted(keys, key=lambda k: (first.get(address(k), len(data)), rank.get(split_key(k)[0], 9), k))
+
+
 def load_recheck_jobs(target: str, types, history: ProxyHistory) -> List[str]:
-    """--recheck: a file, otherwise the last run plus history."""
-    if target:
+    """--recheck: a file, '-' for stdin, otherwise the last run plus history."""
+    if target == STDIN:
+        return stdin_keys(read_stdin(), types)
+    elif target:
         lines = Path(target).expanduser().read_text(encoding="utf-8").splitlines()
         # lines without type:// in files like http.txt take the type from the file name
         default = next((t for t in PROXY_TYPES if t in Path(target).name.lower()), None)
@@ -311,9 +403,10 @@ class Run:
                 jobs = load_recheck_jobs(opts.recheck, opts.types, self.history)
             except (OSError, UnicodeDecodeError) as e:
                 reason = e.strerror if isinstance(e, OSError) and e.strerror else "not a text file"
-                note(f"Can't read {opts.recheck}: {reason}.", BAD, "✘")
+                note(f"Can't read {'stdin' if opts.recheck == STDIN else opts.recheck}: {reason}.", BAD, "✘")
                 return []
-            info("Recheck", f"{fmt(len(jobs))} proxies from {opts.recheck or 'the last run + history'}")
+            where = "stdin" if opts.recheck == STDIN else opts.recheck or "the last run + history"
+            info("Recheck", f"{fmt(len(jobs))} proxies from {where}")
         else:
             jobs = await self._scrape_jobs()
 

{proxy_scraper_cli-1.23.0 → proxy_scraper_cli-1.24.0}/proxyscraper/cli.py

@@ -183,7 +183,7 @@ def parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
     g.add_argument("--no-dnsbl", action="store_true",
                    help="don't look up whether exit IPs are on the SpamCop blocklist")
     g.add_argument("--recheck", nargs="?", const="", metavar="FILE",
-                   help="only check proxies from FILE – without FILE: last run + history; "
+                   help="only check proxies from FILE ('-' reads them from stdin) – without FILE: last run + history; "
                         "'live': the live list from GitHub (seconds instead of minutes, e.g. with --serve)")
 
     f = p.add_argument_group("Filters (for the result files)")