voidaccess 1.5.0__tar.gz → 1.6.2__tar.gz

{voidaccess-1.5.0 → voidaccess-1.6.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: voidaccess
-Version: 1.5.0
+Version: 1.6.2
 Summary: Dark web OSINT CLI — automated threat intelligence from query to report
 Author: VoidAccess
 License-Expression: MIT
@@ -65,6 +65,17 @@ Commercial threat intelligence platforms often charge prohibitive annual fees fo
 
 ---
 
+## What's New in v1.6.0
+
+- **Optional clearnet ScrapingAnt integration** — paste sites and RSS feeds can now be routed through ScrapingAnt. Affects clearnet scraping only; Tor, `.onion`, GitHub, and GitLab traffic are never affected.
+- **Three independent products** — Web Scraping API, Residential Proxy transport, and Datacenter Proxy transport are documented separately so the API key is not conflated with the proxy credentials.
+- **One-shot CLI flags** — `voidaccess investigate --use-scraping-api` and `voidaccess investigate --use-proxies` select the REST API or proxy transport for a single run.
+- **`SCRAPINGANT_PROXY_TYPE`** — `residential` (default) or `datacenter`; this controls the proxy pool used by the proxy transports and is not a separate hostname.
+- **New CLI surfaces** — `voidaccess configure proxy` now prompts for the ScrapingAnt keys and transport toggles in a clearly separated flow, plus `--enable-proxy / --disable-proxy` for non-interactive proxy transport toggling and `--show` for masked state inspection.
+- **55 → 74 proxy-config tests** — covers both transports, the `apply_env()` independent-toggle guarantee, the transport-selection logic, and the masked `--show` output.
+
+---
+
 ## What's New in v1.5.0
 
 - 37 new entity types across crypto, credentials, messaging, and network/forensic indicators.
@@ -128,6 +139,7 @@ The Docker stack includes PostgreSQL, Tor, FastAPI, and Next.js.
 | `voidaccess actor <handle> --note "text"` | Append an analyst note to an actor profile |
 | `voidaccess timeline <handle>` | Shortcut for `voidaccess actor <handle> --timeline` |
 | `voidaccess configure` | Setup wizard |
+| `voidaccess configure proxy` | ScrapingAnt key, username, type, and routing toggles. Flags: `--enable / --disable` (API gate), `--enable-proxy / --disable-proxy` (proxy gate), `--show` (masked state) |
 
 Export examples:
 
@@ -299,6 +311,35 @@ All enrichment sources that require a key degrade gracefully when the key is abs
 | `GITLAB_TOKEN` | Raises GitLab scraping from 15 to 60 req/min | Free | [gitlab.com/profile/personal_access_tokens](https://gitlab.com/-/profile/personal_access_tokens) |
 | `BLOCKCYPHER_TOKEN` | BTC/ETH wallet balance and transaction graph | Yes | [blockcypher.com](https://www.blockcypher.com) |
 | `ETHERSCAN_API_KEY` | ETH wallet lookups | Yes | [etherscan.io/apis](https://etherscan.io/apis) |
+| `SCRAPINGANT_API_KEY` + `VOIDACCESS_USE_PROXIES=true` | Optional clearnet proxy for paste sites + RSS feeds (see below) | Yes (free tier) | [scrapingant.com](https://scrapingant.com/?ref=mzliyzh) |
+
+### Optional: Clearnet Scraping Proxy (ScrapingAnt)
+
+When a third-party clearnet site rate-limits or blocks VoidAccess's outbound IP, paste-site and RSS-feed fetches can be routed through ScrapingAnt. This integration is optional and affects only clearnet scraping.
+
+| Product | What it is | Credentials | Persistent config | One-shot CLI flag | Notes |
+|---|---|---|---|---|---|
+| Web Scraping API | REST transport for paste sites and RSS feeds | `SCRAPINGANT_API_KEY` | `VOIDACCESS_USE_PROXIES=true` | `--use-scraping-api` | POSTs the target URL to `https://api.scrapingant.com/v2/general` and is billed in request credits. |
+| Residential Proxy transport | HTTP CONNECT proxy for paste sites and RSS feeds | `SCRAPINGANT_PROXY_USERNAME` + `SCRAPINGANT_PROXY_PASSWORD` | `VOIDACCESS_USE_PROXY=true` | `--use-proxies` | Uses `residential.scrapingant.com:8080` for HTTP proxying or `:443` for HTTPS tunneling and is billed by traffic volume. |
+| Datacenter Proxy transport | Same proxy flow as Residential, but a different proxy pool | `SCRAPINGANT_PROXY_USERNAME` + `SCRAPINGANT_PROXY_PASSWORD` | `SCRAPINGANT_PROXY_TYPE=datacenter` plus `VOIDACCESS_USE_PROXY=true` | `--use-proxies` | Configuration exists, but live verification is still open; do not assume access is provisioned on every account. |
+
+The proxy transports use `SCRAPINGANT_PROXY_TYPE=residential` or `SCRAPINGANT_PROXY_TYPE=datacenter` to choose the pool. The REST API transport does not use the proxy username/password pair.
+
+The integration is strictly scoped:
+
+- Tor and `.onion` traffic are completely unaffected in every configuration.
+- GitHub and GitLab scraping are permanently excluded because their requests carry `GITHUB_TOKEN` and `GITLAB_TOKEN`.
+- If both the REST API and proxy transport are enabled, the proxy transport wins for that request and VoidAccess logs the choice once.
+- There is no chained mode. Each request uses exactly one transport, and failures fall back to direct fetches.
+
+How to turn it on:
+
+- `voidaccess configure proxy` for the persistent config path.
+- `voidaccess investigate "query" --use-scraping-api` for a one-shot REST API run.
+- `voidaccess investigate "query" --use-proxies` for a one-shot proxy run.
+- `setup.sh` and the Docker / web settings path provide the same persistent storage surfaces as the rest of the project.
+
+Referral signup: [https://scrapingant.com/?ref=mzliyzh](https://scrapingant.com/?ref=mzliyzh)
 
 ---
 
@@ -495,3 +536,7 @@ To report a security vulnerability, see [docs/SECURITY.md](docs/SECURITY.md).
 ## License
 
 MIT License. See [LICENSE](LICENSE) for details.
+
+
+
+