proxy-scraper-cli 1.21.0__tar.gz → 1.22.0__tar.gz

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.21.0
+Version: 1.22.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -69,6 +69,8 @@ Dynamic: license-file
 
 Most free proxy lists are 95 % dead, and a good part of the rest are honeypots or proxies that inject scripts into your pages. **proxy-scraper** collects public HTTP, SOCKS4 and SOCKS5 proxies from 700+ sources and keeps only the ones that pass every check. It learns with every run which sources are worth it, and it can turn the result into a single rotating proxy.
 
+**In numbers:** in one day it checked 3.6 million free proxies – 1.1 % worked, and more than half of the ones that answered failed a second request. [What's actually out there →](docs/free-proxies-in-numbers.md)
+
 <picture>
   <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
   <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
@@ -148,7 +150,7 @@ proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
 proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
-Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
+Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project. After the second run that finds proxies it asks once for a GitHub star; `PROXY_SCRAPER_NO_STAR_HINT=1` turns that off (it never shows in CI).
 
 </details>
 
@@ -918,8 +920,9 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
 
-- [ ] [Measure throughput, not just latency](../../issues/140)
-- [ ] [Daily snapshots as a dataset](../../issues/143)
+- [ ] [Mirror the daily snapshots to a Hugging Face dataset](../../issues/188)
+
+Shipped in v1.10 to v1.21: download speed per proxy and "best first" everywhere, daily snapshots, a rotating proxy URL for requests, httpx and Playwright, `--pick` for working proxies without a scan, a proxy pool API compatible with jhao104/proxy_pool, your own lists with `--source`, a GitHub Action, a Telegram bot and an Atom feed for the weekly report.
 
 Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/README.md

@@ -28,6 +28,8 @@
 
 Most free proxy lists are 95 % dead, and a good part of the rest are honeypots or proxies that inject scripts into your pages. **proxy-scraper** collects public HTTP, SOCKS4 and SOCKS5 proxies from 700+ sources and keeps only the ones that pass every check. It learns with every run which sources are worth it, and it can turn the result into a single rotating proxy.
 
+**In numbers:** in one day it checked 3.6 million free proxies – 1.1 % worked, and more than half of the ones that answered failed a second request. [What's actually out there →](docs/free-proxies-in-numbers.md)
+
 <picture>
   <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
   <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
@@ -107,7 +109,7 @@ proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
 proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
-Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
+Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project. After the second run that finds proxies it asks once for a GitHub star; `PROXY_SCRAPER_NO_STAR_HINT=1` turns that off (it never shows in CI).
 
 </details>
 
@@ -877,8 +879,9 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
 
-- [ ] [Measure throughput, not just latency](../../issues/140)
-- [ ] [Daily snapshots as a dataset](../../issues/143)
+- [ ] [Mirror the daily snapshots to a Hugging Face dataset](../../issues/188)
+
+Shipped in v1.10 to v1.21: download speed per proxy and "best first" everywhere, daily snapshots, a rotating proxy URL for requests, httpx and Playwright, `--pick` for working proxies without a scan, a proxy pool API compatible with jhao104/proxy_pool, your own lists with `--source`, a GitHub Action, a Telegram bot and an Atom feed for the weekly report.
 
 Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/proxy_scraper_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-scraper-cli
-Version: 1.21.0
+Version: 1.22.0
 Summary: Finds free HTTP, SOCKS4 and SOCKS5 proxies that actually work: 700+ sources, real checks, honeypot filtering and a rotating local proxy server.
 Author: Maximilian Feix
 License-Expression: MIT
@@ -69,6 +69,8 @@ Dynamic: license-file
 
 Most free proxy lists are 95 % dead, and a good part of the rest are honeypots or proxies that inject scripts into your pages. **proxy-scraper** collects public HTTP, SOCKS4 and SOCKS5 proxies from 700+ sources and keeps only the ones that pass every check. It learns with every run which sources are worth it, and it can turn the result into a single rotating proxy.
 
+**In numbers:** in one day it checked 3.6 million free proxies – 1.1 % worked, and more than half of the ones that answered failed a second request. [What's actually out there →](docs/free-proxies-in-numbers.md)
+
 <picture>
   <source media="(prefers-color-scheme: dark)" srcset="docs/checks-dark.svg">
   <source media="(prefers-color-scheme: light)" srcset="docs/checks-light.svg">
@@ -148,7 +150,7 @@ proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fish
 proxy-scraper --completion powershell | Out-String | Invoke-Expression   # in $PROFILE
 ```
 
-Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project.
+Installed, the learned state lives in your user data folder (`~/Library/Application Support/proxy-scraper`, `%LOCALAPPDATA%\proxy-scraper` or `~/.local/share/proxy-scraper`; override with `PROXY_SCRAPER_HOME`) and results go to `./results`. Run from a clone, both stay inside the project. After the second run that finds proxies it asks once for a GitHub star; `PROXY_SCRAPER_NO_STAR_HINT=1` turns that off (it never shows in CI).
 
 </details>
 
@@ -918,8 +920,9 @@ Only for things that don't matter. Public proxies are run by strangers who can r
 
 What's next is in the [open issues](../../issues) – ideas and wishes are welcome as an [issue](../../issues/new/choose). During Hacktoberfest there are [beginner-friendly issues](../../issues?q=is%3Aopen+label%3Ahacktoberfest) with pointers on where to start.
 
-- [ ] [Measure throughput, not just latency](../../issues/140)
-- [ ] [Daily snapshots as a dataset](../../issues/143)
+- [ ] [Mirror the daily snapshots to a Hugging Face dataset](../../issues/188)
+
+Shipped in v1.10 to v1.21: download speed per proxy and "best first" everywhere, daily snapshots, a rotating proxy URL for requests, httpx and Playwright, `--pick` for working proxies without a scan, a proxy pool API compatible with jhao104/proxy_pool, your own lists with `--source`, a GitHub Action, a Telegram bot and an Atom feed for the weekly report.
 
 Shipped in v1.8 and v1.9: uptime per proxy and `stable.txt`, which proxies get through to Google, Reddit and Amazon, `live_proxies()` in Python, a page per proxy with its week of checks, a weekly report, live charts in this README, shareable filters on the website, sources mined from other scrapers' lists, untyped lists tried as HTTP and SOCKS5, `--export singbox`, PowerShell completion and recipes for httpx, aiohttp and Scrapy.
 

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/proxy_scraper_cli.egg-info/SOURCES.txt

@@ -97,6 +97,7 @@ tests/test_parsing_fuzz.py
 tests/test_paths.py
 tests/test_pick.py
 tests/test_pipeline.py
+tests/test_preferences.py
 tests/test_proxy_pages.py
 tests/test_proxy_sources.py
 tests/test_publish.py

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/proxyscraper/__init__.py

@@ -1,7 +1,7 @@
 """proxy-scraper: collects public HTTP/SOCKS4/SOCKS5 proxies from hundreds of sources, checks them
 in parallel with its own protocol handshakes and learns which sources deliver good proxies."""
 
-__version__ = "1.21.0"
+__version__ = "1.22.0"
 
 
 def __getattr__(name):

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/proxyscraper/app.py

@@ -41,6 +41,7 @@ from .judges import JudgeProbe, JudgeWatch, rank_judges
 from .netio import INSECURE_HOSTS, http_get
 from .options import STDOUT, RunOptions
 from .output import ResultWriter, latest_results
+from .pages import REPO_URL
 from .parsing import PROXY_TYPES, parse_keys, split_key
 from .paths import is_checkout
 from .pipeline import (
@@ -53,6 +54,7 @@ from .pipeline import (
     run_checks,
     scrape,
 )
+from .preferences import star_hint_due
 from .publish import RAW_BASE
 from .server import ProxyPool, RotatingServer
 from .targets import Target, parse_target
@@ -591,6 +593,9 @@ class Run:
             note(f"{fmt(stats.blocklisted)} of {fmt(len(run.results))} hits ({share}) "
                  "exit from an IP on the SpamCop blocklist – sites that use it show captchas or block them. "
                  "Only the others: --no-blocklisted", MUTED, "ℹ")
+        if kept and opts.output != STDOUT and widgets.console.is_terminal and star_hint_due():
+            note("Found what you needed? A star on GitHub is how others find this: "
+                 f"[link={REPO_URL}]{REPO_URL}[/link] [{MUTED}](shown once)[/]", ACCENT, "★")
         if run.reached_goal:
             note(f"Goal of {fmt(opts.want)} hits reached – stopped early.", GOOD, "✔")
         elif run.interrupted:

proxy_scraper_cli-1.22.0/proxyscraper/preferences.py

@@ -0,0 +1,59 @@
+"""Remembers the last choice from the setup wizard (as command line arguments), and whether the
+one-time star hint was shown."""
+
+from __future__ import annotations
+
+import json
+import os
+from typing import List, Mapping, Optional
+
+from .paths import DATA_DIR, atomic_write
+
+PREFERENCES_FILE = DATA_DIR / "preferences.json"
+STAR_HINT_AFTER = 2  # runs with hits before the hint – the first run is for trying it out
+
+
+def _load() -> dict:
+    try:
+        data = json.loads(PREFERENCES_FILE.read_text(encoding="utf-8"))
+    except (OSError, ValueError):
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+def _save(data: dict) -> None:
+    atomic_write(PREFERENCES_FILE, json.dumps(data, indent=1))
+
+
+def load_last_argv() -> Optional[List[str]]:
+    argv = _load().get("last")
+    if isinstance(argv, list) and all(isinstance(a, str) for a in argv):
+        return argv
+    return None
+
+
+def save_last_argv(argv: List[str]) -> None:
+    data = _load()
+    data["last"] = argv
+    _save(data)
+
+
+def star_hint_due(env: Mapping[str, str] = os.environ) -> bool:
+    """Counts a run that found proxies; True exactly once, on run STAR_HINT_AFTER.
+    Never in CI or with PROXY_SCRAPER_NO_STAR_HINT set."""
+    if env.get("PROXY_SCRAPER_NO_STAR_HINT") or env.get("CI"):
+        return False
+    data = _load()
+    if data.get("star_hint_shown"):
+        return False
+    runs = data.get("runs_with_hits")
+    runs = (runs if isinstance(runs, int) else 0) + 1
+    data["runs_with_hits"] = runs
+    due = runs >= STAR_HINT_AFTER
+    if due:
+        data["star_hint_shown"] = True
+    try:
+        _save(data)
+    except OSError:
+        return False  # can't remember it – better never than every time
+    return due

{proxy_scraper_cli-1.21.0 → proxy_scraper_cli-1.22.0}/proxyscraper/site/index.html

@@ -288,19 +288,26 @@
   .stable-badge { color: var(--signal-ink); font-weight: 500; }
   .sites { display: inline-flex; gap: 4px; }
   .site { font-size: 12px; padding: 2px 7px; border-radius: 99px; border: 1px solid var(--line); color: var(--text); white-space: nowrap; }
+  .site.more { color: var(--muted); }
   .th-plain { display: block; padding: 16px 18px; }
   .uptime { display: inline-flex; align-items: center; gap: 8px; font-variant-numeric: tabular-nums; }
   .meter { width: 32px; height: 4px; border-radius: 2px; background: var(--line); overflow: hidden; }
   .meter::after { content: ""; display: block; height: 100%; width: var(--v); background: var(--signal-ink); border-radius: inherit; }
-  .provider { max-width: 240px; overflow: hidden; text-overflow: ellipsis; color: var(--muted); }
+  .provider { max-width: 160px; overflow: hidden; text-overflow: ellipsis; color: var(--muted); }
   .icon-btn { width: 34px; height: 34px; border-radius: 10px; border: 0; background: transparent; color: var(--muted); cursor: pointer; display: inline-grid; place-items: center;
               transition: background-color .25s, color .25s; }
   .icon-btn:hover { background: var(--line); color: var(--text); }
   .icon-btn svg { width: 16px; height: 16px; }
   .empty { padding: 64px 24px; text-align: center; color: var(--muted); }
+  /* The table fits the page from phone to desktop instead of hiding columns behind a sideways scroll */
+  @media (max-width: 1300px) { td { padding: 13px 12px; } .sort, .th-plain { padding: 16px 12px; } }
+  @media (max-width: 1100px) { th:nth-child(9), td:nth-child(9) { display: none; } .addr-btn svg { display: none; } }
+  @media (max-width: 960px) { th:nth-child(6), td:nth-child(6), th:nth-child(8), td:nth-child(8) { display: none; } }
+  @media (max-width: 800px) { th:nth-child(5), td:nth-child(5) { display: none; } }
   .list-foot { display: flex; justify-content: space-between; align-items: center; gap: 16px; flex-wrap: wrap; margin-top: 20px; }
   .list-foot .count { color: var(--muted); font-size: 15px; }
   .list-foot .actions { display: flex; gap: 10px; flex-wrap: wrap; }
+  .list-foot .fmt { height: 40px; }
 
   /* Route: how a proxy gets on the list */
   .route { height: 460vh; position: relative; }
@@ -474,12 +481,14 @@
     .route-svg .gate-label { font-size: 19px; }
     .filters { position: static; }
     .t-type .name, th .name { display: none; }
-    .addr-btn { font-size: 13.5px; }
+    table { font-size: 13.5px; }
+    .addr-btn { font-size: 12.5px; gap: 0; }
     .addr-btn svg { display: none; }
-    .sort { padding: 14px 9px; }
-    th:first-child .sort { padding-left: 16px; }
-    td { padding-inline: 9px; }
-    td:first-child { padding-left: 16px; }
+    .detail { margin-left: 0; width: 22px; }
+    .sort { padding: 14px 6px; }
+    th:first-child .sort { padding-left: 14px; }
+    td { padding-inline: 6px; }
+    td:first-child { padding-left: 14px; }
     .steps { min-height: 17em; }
     .gh .label, .gh .star-count { display: none; }
     .gh { padding: 0; width: 36px; justify-content: center; }
@@ -490,6 +499,7 @@
     .hide-sm { display: none; }
     .btn { height: 48px; padding: 0 20px; }
   }
+  @media (max-width: 380px) { .addr-btn { font-size: 11.5px; } td { padding-inline: 4px; } }
   @media (prefers-reduced-motion: reduce) {
     html.js .transcript.pending p { opacity: 1; transform: none; }
     *, *::before, *::after { animation: none !important; transition: none !important; }
@@ -640,6 +650,12 @@
         <button class="btn small" id="best" type="button" hidden title="The ones that answer quickly and download fast">Best first</button>
         <button class="btn small" id="more" type="button" hidden>Show more</button>
         <button class="btn small" id="share" type="button" title="The filters are in the address, so the link opens this exact view">Share view</button>
+        <select class="field fmt" id="fmt" aria-label="Format for copy and download" title="Format for copy and download">
+          <option value="url">type://ip:port</option>
+          <option value="plain">ip:port</option>
+          <option value="json">JSON</option>
+          <option value="csv">CSV</option>
+        </select>
         <button class="btn small" id="copy-all" type="button">Copy filtered</button>
         <button class="btn small primary" id="download" type="button">Download filtered</button>
       </div>
@@ -867,7 +883,10 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
   const SITE_NAMES = { google: "Google", reddit: "Reddit", amazon: "Amazon", instagram: "Instagram", tiktok: "TikTok" };
   const through = (r) => {
     const ok = Object.keys(SITE_NAMES).filter(k => (r.sites || {})[k] === true);
-    return ok.length ? `<span class="sites">${ok.map(k => `<span class="site">${SITE_NAMES[k]}</span>`).join("")}</span>` : '<span class="quiet">–</span>';
+    if (!ok.length) return '<span class="quiet">–</span>';
+    // two chips and a count keep the column one line high; the title lists them all
+    const names = ok.map(k => SITE_NAMES[k]), more = names.length - 2;
+    return `<span class="sites" title="Gets through to ${names.join(", ")}">${names.slice(0, 2).map(n => `<span class="site">${n}</span>`).join("")}${more > 0 ? `<span class="site more">+${more}</span>` : ""}</span>`;
   };
   const speed = (k) => !k ? '<span class="quiet">–</span>'
     : `<span class="${k >= 500 ? "fast" : k >= 100 ? "mid" : "slow"}">${k >= 1024 ? (k / 1024).toFixed(1) + " MB/s" : fmt(k) + " KB/s"}</span>`;
@@ -905,9 +924,23 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
   const toastEl = $("#toast");
   function toast(msg) { toastEl.textContent = msg; toastEl.classList.add("show"); clearTimeout(toastEl._t); toastEl._t = setTimeout(() => toastEl.classList.remove("show"), 1800); }
   async function copy(text, msg) { try { await navigator.clipboard.writeText(text); toast(msg); } catch (e) { toast("Copying isn't allowed here. Select the text instead."); } }
-  function download(rows, name) {
-    const blob = new Blob([rows.map(r => r.url).join("\n") + "\n"], { type: "text/plain" });
-    const a = Object.assign(document.createElement("a"), { href: URL.createObjectURL(blob), download: name });
+  // Copy and download in the format the viewer picked – plain ip:port is what many tools want
+  const CSV_COLS = ["proxy", "ptype", "country", "latency", "speed_kbps", "https", "anonymity", "uptime_7d", "org", "hosting", "blocklisted"];
+  const csvCell = (v) => v == null ? "" : /[",\n]/.test(String(v)) ? `"${String(v).replace(/"/g, '""')}"` : String(v);
+  const FORMATS = {
+    url: { ext: "txt", type: "text/plain", text: (rows) => rows.map(r => r.url).join("\n") + "\n" },
+    plain: { ext: "txt", type: "text/plain", text: (rows) => rows.map(r => r.proxy).join("\n") + "\n" },
+    json: { ext: "json", type: "application/json", text: (rows) => JSON.stringify(rows.map(({ rank, ...r }) => r), null, 1) + "\n" },
+    csv: { ext: "csv", type: "text/csv", text: (rows) => [CSV_COLS.join(","), ...rows.map(r => CSV_COLS.map(c => csvCell(r[c])).join(","))].join("\n") + "\n" },
+  };
+  const fmtSelect = $("#fmt");
+  try { const f = localStorage.getItem("format"); if (FORMATS[f]) fmtSelect.value = f; } catch (e) {}
+  fmtSelect.onchange = () => { try { localStorage.setItem("format", fmtSelect.value); } catch (e) {} };
+  const format = () => FORMATS[fmtSelect.value] || FORMATS.url;
+  function download(rows, base) {
+    const f = format();
+    const blob = new Blob([f.text(rows)], { type: f.type });
+    const a = Object.assign(document.createElement("a"), { href: URL.createObjectURL(blob), download: `${base}.${f.ext}` });
     document.body.append(a); a.click(); a.remove(); setTimeout(() => URL.revokeObjectURL(a.href), 1000);
   }
   function ago(iso) {
@@ -1124,9 +1157,9 @@ proxy-scraper --want 50 --https-only</pre><button class="icon-btn" type="button"
     th.querySelector(".sort").onclick = () => { const k = th.dataset.key; state.dir = state.sort === k ? -state.dir : 1; state.sort = k; morph(render); };
   });
   $("#rows").onclick = (e) => { const b = e.target.closest(".addr-btn"); if (b) copy(b.dataset.url, `Copied ${b.dataset.url}`); };
-  $("#copy-all").onclick = () => { const rows = filtered(); copy(rows.map(r => r.url).join("\n") + "\n", `Copied ${fmt(rows.length)} proxies`); };
-  $("#download").onclick = () => download(filtered(), "proxies.txt");
-  $("#dl-all").onclick = () => state.rows.length ? download(state.rows, "all.txt") : toast("The list is still loading");
+  $("#copy-all").onclick = () => { const rows = filtered(); copy(format().text(rows), `Copied ${fmt(rows.length)} proxies`); };
+  $("#download").onclick = () => download(filtered(), "proxies");
+  $("#dl-all").onclick = () => state.rows.length ? download(state.rows, "all") : toast("The list is still loading");
   // the transcript plays once when it comes into view
   const transcript = $("#transcript");
   if (calm || !("IntersectionObserver" in window)) transcript.classList.remove("pending");

proxy_scraper_cli-1.22.0/tests/test_preferences.py

@@ -0,0 +1,37 @@
+import json
+
+import pytest
+
+from proxyscraper import preferences
+
+
+@pytest.fixture(autouse=True)
+def prefs_file(tmp_path, monkeypatch):
+    path = tmp_path / "preferences.json"
+    monkeypatch.setattr(preferences, "PREFERENCES_FILE", path)
+    return path
+
+
+def test_star_hint_once_on_the_second_run_with_hits():
+    assert [preferences.star_hint_due(env={}) for _ in range(4)] == [False, True, False, False]
+
+
+@pytest.mark.parametrize("env", [{"CI": "true"}, {"PROXY_SCRAPER_NO_STAR_HINT": "1"}])
+def test_star_hint_never_in_ci_or_when_turned_off(env, prefs_file):
+    assert not any(preferences.star_hint_due(env=env) for _ in range(3))
+    assert not prefs_file.exists()
+
+
+def test_star_hint_and_last_argv_share_the_file(prefs_file):
+    preferences.save_last_argv(["--want", "20"])
+    preferences.star_hint_due(env={})
+    preferences.save_last_argv(["--want", "30"])
+    assert preferences.load_last_argv() == ["--want", "30"]
+    assert json.loads(prefs_file.read_text())["runs_with_hits"] == 1
+
+
+def test_broken_file_starts_over(prefs_file):
+    prefs_file.write_text("[1, 2")
+    assert preferences.load_last_argv() is None
+    assert preferences.star_hint_due(env={}) is False
+    assert preferences.star_hint_due(env={}) is True

proxy_scraper_cli-1.21.0/proxyscraper/preferences.py

@@ -1,24 +0,0 @@
-"""Remembers the last choice from the setup wizard (as command line arguments)."""
-
-from __future__ import annotations
-
-import json
-from typing import List, Optional
-
-from .paths import DATA_DIR, atomic_write
-
-PREFERENCES_FILE = DATA_DIR / "preferences.json"
-
-
-def load_last_argv() -> Optional[List[str]]:
-    try:
-        argv = json.loads(PREFERENCES_FILE.read_text(encoding="utf-8")).get("last")
-    except (OSError, ValueError, AttributeError):
-        return None
-    if isinstance(argv, list) and all(isinstance(a, str) for a in argv):
-        return argv
-    return None
-
-
-def save_last_argv(argv: List[str]) -> None:
-    atomic_write(PREFERENCES_FILE, json.dumps({"last": argv}, indent=1))