osintkit 0.6.0__tar.gz → 0.8.0__tar.gz

{osintkit-0.6.0 → osintkit-0.8.0}/CHANGELOG.md

@@ -6,6 +6,21 @@ All notable changes are recorded here. The format follows
 
 ## Unreleased
 
+## 0.8.0 — 2026-11-02
+
+### Added
+
+- `watch`: interval re-runs that stay silent unless a diff shows change, with optional desktop
+  notification and bounded cycles
+
+## 0.7.0 — 2026-11-02
+
+### Added
+
+- `xposed_analytics`: dated, typed, verified breach membership and pastes, keyless
+- post-synthesis self-check listing any claimed breach, date, host, or account URL absent from
+  the supplied data
+
 ## 0.6.0 — 2026-11-02
 
 ### Added

{osintkit-0.6.0 → osintkit-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: osintkit
-Version: 0.6.0
+Version: 0.8.0
 Summary: Public-data OSINT aggregator for emails, usernames, domains, and companies
 Project-URL: Homepage, https://github.com/sjdsjadjajdjsa/osintkit
 Project-URL: Issues, https://github.com/sjdsjadjajdjsa/osintkit/issues
@@ -93,8 +93,10 @@ Any OpenAI-compatible endpoint works. `init --provider ollama` needs no key for
 `--provider`, `--base-url`, and `--model` cover everything else. The preset table, key variables,
 and precedence order are in [docs/REFERENCE.md](docs/REFERENCE.md#providers).
 
-Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts GitHub from 60 to
-5000 requests/hour. Everything else needs no credentials.
+Optional keys: a Have I Been Pwned subscription (`--hibp-key` on init, or `HIBP_API_KEY`) adds
+per-account breach and paste membership; `GITHUB_TOKEN` lifts GitHub from 60 to 5000
+requests/hour. Both stay local — the key is yours, and every user of the tool brings their own.
+Everything else needs no credentials.
 
 ## Commands
 
@@ -104,6 +106,7 @@ Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts G
 | `batch` | A CSV of targets, run concurrently, reported in order |
 | `diff` | What changed between two archived runs |
 | `password` | Is a password in Pwned Passwords (free, no key, hash-prefix only) |
+| `watch` | Re-check on an interval, print only changes, optionally notify |
 | `history` | Past runs, replayable by id |
 | `doctor` | Key, endpoint, and directory checks (`--deep` verifies auth) |
 | `collectors` | Every collector and what it touches |
@@ -144,6 +147,8 @@ the model fail.
   undercut.
 - **Cached everything.** 24h default, shorter for volatile sources; repeat prompts cost nothing.
 - **Validated output.** Strict JSON schema, one repair round-trip, then a clean failure.
+- **Self-checked claims.** Every breach, date, host, and account URL is compared back to the
+  collected data; anything unsupported is called out.
 
 ## Collectors
 
@@ -168,6 +173,18 @@ Passwords are checked separately with `osintkit password`: Pwned Passwords cover
 millions of leaked passwords, needs no key, and only the first five SHA-1 characters leave the
 machine.
 
+## Monitoring
+
+```bash
+osintkit watch --domain example.com --every 1d
+osintkit watch --email you@example.com --every 6h --no-notify
+```
+
+The first run prints a baseline. Later runs print nothing unless `diff` sees movement — a new
+breach, a new host, a source failing, or a risk change — and optionally raise a desktop
+notification through `notify-send`. `--cycles` bounds the loop, `ctrl-c` stops it, and every run
+is archived so you can inspect it afterwards. Pair with cron or a systemd timer for unattended use.
+
 ## Safety and legality
 
 - Public data only. No logins, no scraping behind auth, no paid brokers.

{osintkit-0.6.0 → osintkit-0.8.0}/README.md

@@ -61,8 +61,10 @@ Any OpenAI-compatible endpoint works. `init --provider ollama` needs no key for
 `--provider`, `--base-url`, and `--model` cover everything else. The preset table, key variables,
 and precedence order are in [docs/REFERENCE.md](docs/REFERENCE.md#providers).
 
-Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts GitHub from 60 to
-5000 requests/hour. Everything else needs no credentials.
+Optional keys: a Have I Been Pwned subscription (`--hibp-key` on init, or `HIBP_API_KEY`) adds
+per-account breach and paste membership; `GITHUB_TOKEN` lifts GitHub from 60 to 5000
+requests/hour. Both stay local — the key is yours, and every user of the tool brings their own.
+Everything else needs no credentials.
 
 ## Commands
 
@@ -72,6 +74,7 @@ Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts G
 | `batch` | A CSV of targets, run concurrently, reported in order |
 | `diff` | What changed between two archived runs |
 | `password` | Is a password in Pwned Passwords (free, no key, hash-prefix only) |
+| `watch` | Re-check on an interval, print only changes, optionally notify |
 | `history` | Past runs, replayable by id |
 | `doctor` | Key, endpoint, and directory checks (`--deep` verifies auth) |
 | `collectors` | Every collector and what it touches |
@@ -112,6 +115,8 @@ the model fail.
   undercut.
 - **Cached everything.** 24h default, shorter for volatile sources; repeat prompts cost nothing.
 - **Validated output.** Strict JSON schema, one repair round-trip, then a clean failure.
+- **Self-checked claims.** Every breach, date, host, and account URL is compared back to the
+  collected data; anything unsupported is called out.
 
 ## Collectors
 
@@ -136,6 +141,18 @@ Passwords are checked separately with `osintkit password`: Pwned Passwords cover
 millions of leaked passwords, needs no key, and only the first five SHA-1 characters leave the
 machine.
 
+## Monitoring
+
+```bash
+osintkit watch --domain example.com --every 1d
+osintkit watch --email you@example.com --every 6h --no-notify
+```
+
+The first run prints a baseline. Later runs print nothing unless `diff` sees movement — a new
+breach, a new host, a source failing, or a risk change — and optionally raise a desktop
+notification through `notify-send`. `--cycles` bounds the loop, `ctrl-c` stops it, and every run
+is archived so you can inspect it afterwards. Pair with cron or a systemd timer for unattended use.
+
 ## Safety and legality
 
 - Public data only. No logins, no scraping behind auth, no paid brokers.

{osintkit-0.6.0 → osintkit-0.8.0}/docs/REFERENCE.md

@@ -8,7 +8,7 @@ Everything the README leaves out: every flag, every collector, storage, and fail
 | --- | --- | --- | --- |
 | `infostealer` | email / username / domain | Hudson Rock stealer logs | no |
 | `hibp_breaches`, `hibp_pastes` | email | Have I Been Pwned | yes |
-| `xposed_or_not`, `leakcheck_public` | email | free public breach indexes | no |
+| `xposed_or_not`, `xposed_analytics`, `leakcheck_public` | email | free public breach indexes | no |
 | `gravatar` | email | Gravatar | no |
 | `crt_sh_subdomains`, `crt_sh_certificates` | domain | crt.sh | no |
 | `crt_sh_organization` | company | crt.sh organisation search, with apex-domain identity fallback | no |
@@ -62,6 +62,20 @@ Explicit `--base-url` and `--model` beat both the provider and the prefix. Tempe
 Per-command: `--json`, `--markdown`, `-o/--out`, `--raw`, `--offline`, `--dry-run`, `--only`, `--skip`,
 `--ports`, `--fail-fast`, `-c/--concurrency`, `--limit`, `--show`, `--days`, `--clear`, `--deep`.
 
+## Watching
+
+`watch` repeats a lookup on an interval (`--every`, minimum 60s), prints a baseline, and then
+only reports diffs: added breaches or hosts, risk direction, source failure and recovery.
+`--notify` (default) calls `notify-send` when available; `--full` prints the whole report on a
+change; `--cycles N` bounds the loop. Every cycle is archived and logged like a normal run.
+
+## Self-check
+
+After synthesis every breach name, date, subdomain, and account URL is compared with the supplied
+data. Anything the model asserted that no source returned is listed under a "not found in source
+data" note, so drift is visible instead of plausible. Checks only run for source families that
+were actually collected.
+
 ## Info-stealer logs
 
 `infostealer` queries Hudson Rock's keyless endpoint for infections that saved credentials on a
@@ -88,7 +102,7 @@ pwned. `--json` for machines. Nothing about the password beyond the prefix leave
 
 `osintkit init` writes `~/.config/osintkit/config.toml` (or `$XDG_CONFIG_HOME`) with mode 600 in
 a 700 directory, after checking the key with a one-token request. Keys recognised:
-`provider`, `model`, `base_url`, `api_key`, `resolvers`.
+`provider`, `model`, `base_url`, `api_key`, `hibp_api_key`, `resolvers`.
 
 `osintkit config` prints them masked, `--reveal` shows the key, `--unset KEY` removes one, and
 `--path` prints the location. Deleting the file is the same as `--unset` everything.
@@ -96,6 +110,7 @@ a 700 directory, after checking the key with a one-token request. Keys recognise
 Precedence for each value: command-line flag, then the provider-specific environment variable,
 then a key prefix, then the config file, then the built-in default. Stored values apply only when
 the stored provider matches the one in use, so a one-off key export never gets the stored key.
+`HIBP_API_KEY` in the environment beats a stored `hibp_api_key`.
 
 ## Offline mode
 

{osintkit-0.6.0 → osintkit-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "osintkit"
-version = "0.6.0"
+version = "0.8.0"
 description = "Public-data OSINT aggregator for emails, usernames, domains, and companies"
 readme = "README.md"
 requires-python = ">=3.12"

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/__init__.py

@@ -1,3 +1,3 @@
 """osintkit: a public-data OSINT aggregator."""
 
-__version__ = "0.6.0"
+__version__ = "0.8.0"

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/cli.py

@@ -18,7 +18,7 @@ from rich.console import Console
 from rich.panel import Panel
 from rich.table import Table
 
-from . import __version__, diff, doctor, passwords, pipeline, render, retention, userconfig
+from . import __version__, diff, doctor, passwords, pipeline, render, retention, userconfig, watch
 from .collectors import REGISTRY
 from .config import (
     DEFAULT_RESOLVERS,
@@ -112,6 +112,7 @@ def root(
         base_url=base_url or (stored.get("base_url") if config_applies else None) or preset[0],
         model=model or (stored.get("model") if config_applies else None) or preset[1],
         api_key=api_key,
+        hibp_api_key_value=stored.get("hibp_api_key"),
         temperature=temperature,
         max_tokens=max_tokens,
         timeout=timeout,
@@ -561,6 +562,9 @@ def init(
     provider: str | None = typer.Option(None, "--provider", help="Provider to store."),
     api_key: str | None = typer.Option(None, "--api-key", help="Key to store."),
     model: str | None = typer.Option(None, "--model", help="Default model to store."),
+    hibp_key: str | None = typer.Option(
+        None, "--hibp-key", help="Optional Have I Been Pwned key for breach and paste checks."
+    ),
     base_url: str | None = typer.Option(None, "--base-url", help="Custom endpoint to store."),
     force: bool = typer.Option(False, "--force", help="Replace an existing config."),
     verify: bool = typer.Option(True, "--verify/--no-verify", help="Check the key afterwards."),
@@ -603,11 +607,19 @@ def init(
             raise typer.Exit(2)
         data["api_key"] = key
 
+    stored_hibp = stored.get("hibp_api_key")
+    if hibp_key:
+        data["hibp_api_key"] = hibp_key
+    elif stored_hibp and not force:
+        data["hibp_api_key"] = stored_hibp
+
     written = userconfig.save(data)
     out.print(f"[green]stored[/green] {written} (mode 600)")
     out.print(f"provider: {chosen}")
     if data.get("api_key"):
         out.print(f"key: {userconfig.mask(str(data['api_key']))}")
+    if data.get("hibp_api_key"):
+        out.print(f"HIBP key: {userconfig.mask(str(data['hibp_api_key']))}")
 
     if verify:
         probe = Settings(
@@ -627,6 +639,57 @@ def init(
     out.print("[dim]try: osintkit lookup --domain example.com[/dim]")
 
 
+@app.command("watch")
+def watch_cmd(
+    target: str | None = typer.Argument(None, help="Email, username, domain, or company."),
+    email: str | None = typer.Option(None, "--email", help="Email target."),
+    username: str | None = typer.Option(None, "--username", help="Username target."),
+    domain: str | None = typer.Option(None, "--domain", help="Domain target."),
+    company: str | None = typer.Option(None, "--company", help="Company target."),
+    every: str = typer.Option("1d", "--every", help="Interval: 30m, 6h, 1d."),
+    cycles: int = typer.Option(0, "--cycles", min=0, help="Stop after N runs; 0 means forever."),
+    full: bool = typer.Option(False, "--full", help="Print the whole report on each change."),
+    notify_changes: bool = typer.Option(True, "--notify/--no-notify", help="Desktop notification."),
+    offline: bool = typer.Option(False, "--offline", help="Local report only; no model, no key."),
+    only: list[str] | None = typer.Option(None, "--only", help="Run only these collectors."),
+    skip: list[str] | None = typer.Option(None, "--skip", help="Skip these collectors."),
+) -> None:
+    """Re-check a target on a schedule and report only changes."""
+    out = _console()
+    try:
+        interval = watch.parse_duration(every)
+        raw, declared = _pick_target(target, email, username, domain, company)
+    except (ValueError, TargetParseError) as exc:
+        out.print(f"[red]{exc}[/red]")
+        raise typer.Exit(2) from exc
+
+    settings = _settings(offline=offline, output_json=False, output_markdown=False, raw=False)
+
+    async def run_once():
+        outcome = await pipeline.execute(
+            raw, settings, RunLogger(pipeline.new_run_id()), out, declared_type=declared
+        )
+        return outcome.report
+
+    def on_report(report, lines, baseline):
+        watch.render_event(report, lines, baseline, out, full=full)
+        if lines and notify_changes:
+            body = "; ".join(lines)[:180]
+            watch.notify(f"change for {report.target}", body)
+
+    out.print(
+        f"[dim]watching {render.escape(raw)} every {every}"
+        + (f", {cycles} runs" if cycles else "")
+        + " · ctrl-c stops[/dim]"
+    )
+    try:
+        changes = asyncio.run(watch.run(raw, run_once, on_report, every=interval, cycles=cycles))
+    except KeyboardInterrupt:
+        out.print("[yellow]stopped[/yellow]")
+        return
+    out.print(f"[dim]{changes} change event(s)[/dim]")
+
+
 @app.command()
 def password(
     value: str | None = typer.Argument(None, help="Password to check; prompt if omitted."),
@@ -698,7 +761,7 @@ def config_cmd(
     table.add_column("value")
     for name, value in stored.items():
         shown: object = value
-        if name == "api_key" and isinstance(value, str) and not reveal:
+        if name in {"api_key", "hibp_api_key"} and isinstance(value, str) and not reveal:
             shown = userconfig.mask(value)
         table.add_row(name, render.escape(str(shown)))
     out.print(table)

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/collectors/__init__.py

@@ -13,6 +13,7 @@ from . import (  # noqa: F401  (import order matters: side-effect registration)
     ports,
     subdomains,
     whois_lookup,
+    xposed_analytics,
 )
 from .base import REGISTRY, Collector, Context, collectors_for, run_collector
 

osintkit-0.8.0/src/osintkit/collectors/xposed_analytics.py

@@ -0,0 +1,73 @@
+"""XposedOrNot breach analytics: dated, typed membership plus pastes."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from .base import Context, collector
+
+URL = "https://api.xposedornot.com/v1/breach-analytics"
+MAX_LISTED = 80
+
+
+def _types(value: Any) -> list[str]:
+    return [part.strip() for part in str(value or "").split(";") if part.strip()]
+
+
+def _breach(row: dict[str, Any]) -> dict[str, Any] | None:
+    name = str(row.get("breach") or "").strip()
+    if not name:
+        return None
+    date = str(row.get("xposed_date") or "").strip() or None
+    return {
+        "name": name,
+        "date": date,
+        "data_types": _types(row.get("xposed_data")),
+        "domain": row.get("domain"),
+        "pwn_count": row.get("xposed_records"),
+        "verified": str(row.get("verified") or "").lower() == "yes",
+        "password_risk": row.get("password_risk"),
+        "source": "xposedornot-analytics",
+    }
+
+
+def _paste(row: dict[str, Any]) -> dict[str, Any] | None:
+    paste_id = str(row.get("paste_id") or row.get("id") or "").strip()
+    if not paste_id and not row.get("title"):
+        return None
+    return {
+        "id": paste_id or None,
+        "title": row.get("title"),
+        "date": row.get("date") or row.get("xposed_date"),
+        "source": row.get("source") or "xposedornot-analytics",
+        "origin": "xposedornot",
+    }
+
+
+@collector("xposed_analytics", ("email",), "Dated breach and paste membership", order=14, ttl=21600)
+async def xposed_analytics(ctx: Context) -> dict:
+    response = await ctx.http.get(
+        URL,
+        namespace="xposed-analytics",
+        params={"email": ctx.target},
+        empty_on=(404, 422, 429),
+        timeout=ctx.http_timeout or 60,
+    )
+    if response.status_code != 200:
+        return {"breaches": [], "pastes": [], "note": f"HTTP {response.status_code}"}
+    try:
+        body = response.json()
+    except ValueError:
+        return {"breaches": [], "pastes": [], "note": "non-JSON"}
+
+    raw = ((body.get("ExposedBreaches") or {}).get("breaches_details")) or []
+    breaches = [entry for entry in (_breach(row) for row in raw if isinstance(row, dict)) if entry]
+    raw_pastes = ((body.get("ExposedPastes") or {}).get("pastes_details")) or []
+    pastes = [entry for entry in (_paste(row) for row in raw_pastes if isinstance(row, dict)) if entry]
+    breaches.sort(key=lambda row: str(row.get("date") or ""), reverse=True)
+    return {
+        "found": len(breaches),
+        "breaches": breaches[:MAX_LISTED],
+        "pastes": pastes[:25],
+        "source": "xposedornot-analytics",
+    }

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/config.py

@@ -109,6 +109,7 @@ class Settings:
     base_url: str = DEFAULT_BASE_URL
     model: str = DEFAULT_MODEL
     api_key: str | None = None
+    hibp_api_key_value: str | None = None
     temperature: float = DEFAULT_TEMPERATURE
     max_tokens: int = 4000
     timeout: float = DEFAULT_TIMEOUT
@@ -127,7 +128,7 @@ class Settings:
 
     @property
     def hibp_api_key(self) -> str | None:
-        return os.environ.get("HIBP_API_KEY") or None
+        return os.environ.get("HIBP_API_KEY") or self.hibp_api_key_value or None
 
     @property
     def github_token(self) -> str | None:

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/doctor.py

@@ -124,10 +124,10 @@ def collect_checks(settings: Settings) -> list[Check]:
         ),
     ]
     for name, variable, why in (
-        ("Have I Been Pwned", "HIBP_API_KEY", "breach collectors will be skipped"),
+        ("Have I Been Pwned", "HIBP_API_KEY", "breach and paste collectors will be skipped"),
         ("GitHub", "GITHUB_TOKEN", "GitHub is limited to 60 requests/hour"),
     ):
-        present = bool(os.environ.get(variable))
+        present = bool(os.environ.get(variable)) or (name.startswith("Have I") and bool(settings.hibp_api_key))
         checks.append(Check(name, "ok" if present else "warn", "set" if present else why))
     if key:
         checks.append(Check("api key shape", "ok" if len(key) >= 20 else "warn", f"{len(key)} chars"))

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/normalise.py

@@ -7,7 +7,7 @@ from typing import Any
 
 from .models import CollectorResult
 
-BREACH_SOURCES = frozenset({"hibp_breaches", "xposed_or_not", "leakcheck_public"})
+BREACH_SOURCES = frozenset({"hibp_breaches", "xposed_or_not", "xposed_analytics", "leakcheck_public"})
 KEY_RE = re.compile(r"[^a-z0-9]+")
 CREDENTIAL_RE = re.compile(r"pass|hash|credential|secret|token|api[ _-]?key", re.I)
 SEVERITY = ("low", "medium", "high")

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/pipeline.py

@@ -14,7 +14,7 @@ from pathlib import Path
 
 from rich.console import Console
 
-from . import catalogue, collectors, normalise, offline, render, retention, synthesis
+from . import catalogue, collectors, normalise, offline, render, retention, synthesis, verify
 from .config import Settings
 from .errors import MissingAPIKey, OsintKitError, SynthesisError, TargetParseError
 from .http import HttpClient
@@ -248,6 +248,9 @@ async def execute(
                     report.analysis.risk_level = normalise.apply_floor(
                         report.analysis.risk_level, signals.get("risk_floor", "low")
                     )
+                    unsupported = verify.claims(report.analysis, prompt_results)
+                    if unsupported:
+                        report.notes.append("not found in source data: " + "; ".join(unsupported))
                     if not report.analysis.summary.strip():
                         fallback = offline.build(prompt_results, report.notes, target, narrated=True)
                         report.analysis.summary = fallback.summary

{osintkit-0.6.0 → osintkit-0.8.0}/src/osintkit/userconfig.py

@@ -7,8 +7,8 @@ from pathlib import Path
 from typing import Any
 
 FILENAME = "config.toml"
-KEYS = ("provider", "model", "base_url", "api_key", "resolvers")
-SECRET_KEYS = frozenset({"api_key"})
+KEYS = ("provider", "model", "base_url", "api_key", "hibp_api_key", "resolvers")
+SECRET_KEYS = frozenset({"api_key", "hibp_api_key"})
 
 
 def config_home() -> Path:

osintkit-0.8.0/src/osintkit/verify.py

@@ -0,0 +1,77 @@
+"""Post-synthesis check that claims exist in the collected data."""
+
+from __future__ import annotations
+
+from .models import Analysis, CollectorResult
+
+MAX_ISSUES = 5
+
+
+def _breach_rows(results: list[CollectorResult]) -> list[dict]:
+    rows: list[dict] = []
+    for result in results:
+        if result.collector == "breach_index" and isinstance(result.data, list):
+            rows.extend(row for row in result.data if isinstance(row, dict))
+    return rows
+
+
+def _names(results: list[CollectorResult]) -> set[str]:
+    return {str(row.get("name") or "").strip().lower() for row in _breach_rows(results)}
+
+
+def _dates(results: list[CollectorResult]) -> set[str]:
+    return {str(row.get("date") or "")[:10] for row in _breach_rows(results) if row.get("date")}
+
+
+def _subdomains(results: list[CollectorResult]) -> set[str]:
+    names: set[str] = set()
+    for result in results:
+        data = result.data
+        if result.collector in {"crt_sh_subdomains"} and isinstance(data, list):
+            names.update(str(item).lower() for item in data)
+        if result.collector == "passive_subdomains" and isinstance(data, dict):
+            for key in ("live_hosts", "common_prefix_hits"):
+                names.update(str(row.get("host")).lower() for row in data.get(key) or [] if row.get("host"))
+    return names
+
+
+def _urls(results: list[CollectorResult]) -> set[str]:
+    urls: set[str] = set()
+    for result in results:
+        data = result.data
+        if result.collector == "username_presence" and isinstance(data, dict):
+            urls.update(str(row.get("url")) for row in data.get("found") or [] if row.get("url"))
+        if result.collector == "github_profile" and isinstance(data, dict):
+            url = (data.get("profile") or {}).get("url")
+            if url:
+                urls.add(str(url))
+    return urls
+
+
+def claims(analysis: Analysis, results: list[CollectorResult]) -> list[str]:
+    """Return short descriptions of claims that the source data does not support."""
+    issues: list[str] = []
+    names = _names(results)
+    dates = _dates(results)
+    hosts = _subdomains(results)
+    urls = _urls(results)
+    checked = {result.collector for result in results}
+    has_breaches = "breach_index" in checked
+    has_hosts = bool(checked & {"crt_sh_subdomains", "passive_subdomains"})
+    has_accounts = bool(checked & {"username_presence", "github_profile"})
+
+    for breach in analysis.breaches:
+        if has_breaches and breach.name.strip().lower() not in names:
+            issues.append(f"breach {breach.name!r} not in the supplied data")
+        if has_breaches and breach.date and breach.date[:10] not in dates:
+            issues.append(f"date {breach.date} for {breach.name!r} not in the supplied data")
+
+    for host in analysis.subdomains:
+        if has_hosts and host.strip().lower() not in hosts:
+            issues.append(f"subdomain {host!r} not in the supplied data")
+
+    for account in analysis.associated_accounts:
+        if has_accounts and account.url and account.url.rstrip("/") not in {url.rstrip("/") for url in urls}:
+            issues.append(f"account URL {account.url!r} not in the supplied data")
+
+    return issues[:MAX_ISSUES]

osintkit-0.8.0/src/osintkit/watch.py

@@ -0,0 +1,97 @@
+"""Repeat a lookup on an interval and speak only when something changed."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+import shutil
+import subprocess
+from collections.abc import Awaitable, Callable
+
+from . import diff, render
+from .models import Report
+
+UNITS = {"s": 1, "m": 60, "h": 3600, "d": 86400}
+DURATION_RE = re.compile(r"^(\d+)([smhd])$")
+MIN_INTERVAL = 60
+
+
+def parse_duration(text: str) -> int:
+    match = DURATION_RE.match(text.strip().lower())
+    if not match:
+        raise ValueError(f"{text!r} is not a duration like 30m, 6h, or 1d")
+    value = int(match.group(1)) * UNITS[match.group(2)]
+    if value < MIN_INTERVAL:
+        raise ValueError(f"interval must be at least {MIN_INTERVAL}s")
+    return value
+
+
+def describe(change: dict) -> list[str]:
+    lines: list[str] = []
+    if change["risk_rose"]:
+        lines.append(f"risk rose {change['risk']['old']} -> {change['risk']['new']}")
+    if change["risk_fell"]:
+        lines.append(f"risk fell {change['risk']['old']} -> {change['risk']['new']}")
+    for name in change["breaches"]["added"]:
+        lines.append(f"new breach: {name}")
+    for name in change["subdomains"]["added"]:
+        lines.append(f"new host: {name}")
+    for name in change["collectors"]["started_failing"]:
+        lines.append(f"source failing: {name}")
+    for name in change["collectors"]["started_working"]:
+        lines.append(f"source recovered: {name}")
+    return lines
+
+
+def notify(title: str, body: str) -> bool:
+    binary = shutil.which("notify-send")
+    if not binary:
+        return False
+    try:
+        subprocess.run([binary, "osintkit", title, body], check=False, timeout=10)
+        return True
+    except (OSError, subprocess.SubprocessError):
+        return False
+
+
+async def run(
+    target: str,
+    run_once: Callable[[], Awaitable[Report]],
+    on_report: Callable[[Report, list[str], bool], None],
+    *,
+    every: int,
+    cycles: int = 0,
+    sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
+) -> int:
+    """Run `run_once` on an interval. Returns the number of change events."""
+    previous: Report | None = None
+    changes = 0
+    done = 0
+    while True:
+        report = await run_once()
+        done += 1
+        lines: list[str] = []
+        if previous is not None:
+            comparison = diff.compare(previous, report)
+            if not diff.is_empty(comparison):
+                changes += 1
+                lines = describe(comparison)
+        if previous is None or lines:
+            on_report(report, lines, previous is None)
+        previous = report
+        if cycles and done >= cycles:
+            return changes
+        await sleep(every)
+
+
+def render_event(report: Report, lines: list[str], baseline: bool, console, *, full: bool) -> None:
+    stamp = render.escape(report.generated_at)
+    if baseline:
+        console.print(f"[dim]{stamp} baseline for {render.escape(report.target)}[/dim]")
+        render.render_report(report, console)
+        return
+    console.print(f"[yellow]change[/yellow] {stamp} {render.escape(report.target)}")
+    for line in lines:
+        console.print(f"  - {render.escape(line)}")
+    if full:
+        render.render_report(report, console)

{osintkit-0.6.0 → osintkit-0.8.0}/tests/test_cli.py

@@ -643,3 +643,58 @@ def test_password_command_hit_and_miss(monkeypatch):
         miss = runner.invoke(app, ["password", "hunter3", "--json"])
     assert miss.exit_code == 0
     assert jsonish(miss.output)["pwned"] is False
+
+
+def test_init_stores_optional_hibp_key(xdg):
+    from osintkit import userconfig
+
+    result = runner.invoke(
+        app,
+        ["init", "--provider", "venice", "--api-key", "k" * 30, "--hibp-key", "h" * 32, "--no-verify"],
+    )
+    assert result.exit_code == 0
+    assert userconfig.load()["hibp_api_key"] == "h" * 32
+    shown = runner.invoke(app, ["config"])
+    assert "hhhhhh…" in shown.output
+    assert "h" * 32 not in shown.output
+
+
+def test_stored_hibp_key_reaches_settings(xdg, monkeypatch):
+    from osintkit import userconfig
+
+    userconfig.save({"provider": "venice", "api_key": "k" * 30, "hibp_api_key": "stored-hibp"})
+    captured = {}
+
+    async def spy(target, settings, *args, **kwargs):
+        captured["hibp"] = settings.hibp_api_key
+        raise RuntimeError("stop")
+
+    monkeypatch.setattr(pipeline, "execute", spy)
+    runner.invoke(app, ["lookup", "--domain", "a.com"])
+    assert captured["hibp"] == "stored-hibp"
+
+
+def test_env_hibp_key_beats_stored(xdg, monkeypatch):
+    from osintkit import userconfig
+
+    userconfig.save({"provider": "venice", "api_key": "k" * 30, "hibp_api_key": "stored-hibp"})
+    monkeypatch.setenv("HIBP_API_KEY", "env-hibp")
+    captured = {}
+
+    async def spy(target, settings, *args, **kwargs):
+        captured["hibp"] = settings.hibp_api_key
+        raise RuntimeError("stop")
+
+    monkeypatch.setattr(pipeline, "execute", spy)
+    runner.invoke(app, ["lookup", "--domain", "a.com"])
+    assert captured["hibp"] == "env-hibp"
+
+
+def test_watch_runs_a_single_cycle(monkeypatch):
+    monkeypatch.setattr(pipeline, "LLMClient", None)
+    result = runner.invoke(
+        app, ["watch", "--domain", "example.com", "--every", "60s", "--cycles", "1", "--offline"]
+    )
+    assert result.exit_code == 0
+    assert "baseline" in result.output
+    assert "0 change event" in result.output

osintkit-0.8.0/tests/test_verify.py

@@ -0,0 +1,54 @@
+from osintkit import verify
+from osintkit.models import Analysis, CollectorResult
+
+
+def result(name, data):
+    return CollectorResult(collector=name, target="t", target_type="email", data=data)
+
+
+def analysis(**kwargs):
+    base = {"summary": "s", "risk_level": "low"}
+    base.update(kwargs)
+    return Analysis.model_validate(base)
+
+
+def test_supported_claims_pass():
+    results = [result("breach_index", [{"name": "Acme", "date": "2020-01-01"}])]
+    check = analysis(breaches=[{"name": "Acme", "date": "2020-01-01"}])
+    assert verify.claims(check, results) == []
+
+
+def test_unknown_breach_is_flagged():
+    results = [result("breach_index", [{"name": "Acme"}])]
+    check = analysis(breaches=[{"name": "Invented"}])
+    assert any("Invented" in issue for issue in verify.claims(check, results))
+
+
+def test_wrong_date_is_flagged():
+    results = [result("breach_index", [{"name": "Acme", "date": "2020-01-01"}])]
+    check = analysis(breaches=[{"name": "Acme", "date": "2024-09-09"}])
+    assert any("2024-09-09" in issue for issue in verify.claims(check, results))
+
+
+def test_unknown_subdomain_and_url_are_flagged():
+    results = [
+        result("crt_sh_subdomains", ["api.example.com"]),
+        result("username_presence", {"found": [{"url": "https://github.com/other"}]}),
+    ]
+    check = analysis(
+        subdomains=["api.example.com", "made-up.example.com"],
+        associated_accounts=[{"platform": "GitHub", "url": "https://github.com/x"}],
+    )
+    issues = verify.claims(check, results)
+    assert any("made-up" in issue for issue in issues)
+    assert any("github.com/x" in issue for issue in issues)
+
+
+def test_no_source_data_means_no_noise():
+    assert verify.claims(analysis(breaches=[{"name": "Anything"}]), []) == []
+
+
+def test_issue_list_is_capped():
+    results = [result("breach_index", [])]
+    check = analysis(breaches=[{"name": f"B{index}"} for index in range(20)])
+    assert len(verify.claims(check, results)) == verify.MAX_ISSUES

osintkit-0.8.0/tests/test_watch.py

@@ -0,0 +1,81 @@
+import pytest
+
+from osintkit import watch
+from osintkit.models import Analysis, Report
+
+
+def report(risk="low", breaches=()):
+    return Report(
+        target="example.com",
+        target_type="domain",
+        analysis=Analysis(summary="s", risk_level=risk, breaches=[{"name": b} for b in breaches]),
+    )
+
+
+def test_parse_duration():
+    assert watch.parse_duration("30m") == 1800
+    assert watch.parse_duration("2h") == 7200
+    assert watch.parse_duration("1d") == 86400
+
+
+@pytest.mark.parametrize("text", ["10s", "soon", "5x", ""])
+def test_parse_duration_rejects(text):
+    with pytest.raises(ValueError):
+        watch.parse_duration(text)
+
+
+def test_describe_lists_changes():
+    change = {
+        "risk": {"old": "low", "new": "high"},
+        "risk_rose": True,
+        "risk_fell": False,
+        "breaches": {"added": ["Acme"], "removed": []},
+        "subdomains": {"added": ["x.example.com"], "removed": []},
+        "collectors": {"started_failing": ["crt_sh_subdomains"], "started_working": []},
+    }
+    lines = watch.describe(change)
+    assert "risk rose low -> high" in lines
+    assert "new breach: Acme" in lines
+    assert "new host: x.example.com" in lines
+    assert "source failing: crt_sh_subdomains" in lines
+
+
+async def test_run_reports_baseline_then_only_changes():
+    sequence = [report(), report(), report(risk="high"), report(risk="high")]
+    events = []
+    slept = []
+
+    async def run_once():
+        return sequence.pop(0)
+
+    async def sleeper(seconds):
+        slept.append(seconds)
+
+    changes = await watch.run(
+        "example.com",
+        run_once,
+        lambda rep, lines, baseline: events.append((baseline, list(lines))),
+        every=60,
+        cycles=4,
+        sleep=sleeper,
+    )
+    assert changes == 1
+    assert events[0] == (True, [])
+    assert events[1][0] is False
+    assert any("risk rose" in line for line in events[1][1])
+
+
+async def test_run_respects_cycles():
+    events = []
+
+    async def run_once():
+        return report()
+
+    async def sleeper(seconds):
+        return None
+
+    changes = await watch.run(
+        "example.com", run_once, lambda *a: events.append(a), every=60, cycles=3, sleep=sleeper
+    )
+    assert changes == 0
+    assert len(events) == 1

osintkit-0.8.0/tests/test_xposed_analytics.py

@@ -0,0 +1,61 @@
+import httpx
+import respx
+
+from osintkit.collectors import xposed_analytics as xa
+
+URL = "https://api.xposedornot.com/v1/breach-analytics"
+
+
+@respx.mock
+async def test_breaches_are_normalised_and_sorted(make_context):
+    ctx = make_context("a@b.co", "email")
+    respx.get(URL).mock(
+        return_value=httpx.Response(
+            200,
+            json={
+                "ExposedBreaches": {
+                    "breaches_details": [
+                        {"breach": "Old", "xposed_date": "2019", "xposed_data": "Emails;Passwords"},
+                        {"breach": "New", "xposed_date": "2026", "xposed_records": 10, "verified": "Yes"},
+                    ]
+                }
+            },
+        )
+    )
+    result = await xa.xposed_analytics(ctx)
+    assert [row["name"] for row in result["breaches"]] == ["New", "Old"]
+    assert result["breaches"][1]["data_types"] == ["Emails", "Passwords"]
+    assert result["breaches"][0]["verified"] is True
+
+
+@respx.mock
+async def test_pastes_are_normalised(make_context):
+    ctx = make_context("a@b.co", "email")
+    respx.get(URL).mock(
+        return_value=httpx.Response(
+            200,
+            json={"ExposedPastes": {"pastes_details": [{"paste_id": "abc", "title": "t"}]}},
+        )
+    )
+    result = await xa.xposed_analytics(ctx)
+    assert result["pastes"][0]["id"] == "abc"
+
+
+@respx.mock
+async def test_error_status_is_a_note_not_a_crash(make_context):
+    ctx = make_context("a@b.co", "email")
+    respx.get(URL).mock(return_value=httpx.Response(429))
+    result = await xa.xposed_analytics(ctx)
+    assert result["breaches"] == []
+    assert "429" in result["note"]
+
+
+@respx.mock
+async def test_junk_rows_are_dropped(make_context):
+    ctx = make_context("a@b.co", "email")
+    respx.get(URL).mock(
+        return_value=httpx.Response(
+            200, json={"ExposedBreaches": {"breaches_details": [None, {}, {"breach": ""}]}}
+        )
+    )
+    assert (await xa.xposed_analytics(ctx))["breaches"] == []