osintkit 0.1.2__tar.gz → 0.3.0__tar.gz

{osintkit-0.1.2 → osintkit-0.3.0}/CHANGELOG.md

@@ -6,6 +6,28 @@ All notable changes are recorded here. The format follows
 
 ## Unreleased
 
+## 0.3.0 — 2026-11-02
+
+### Added
+
+- providers for DeepSeek, Mistral, Groq, Together, Anthropic, and local Ollama
+- provider selection from its own environment variable (`DEEPSEEK_API_KEY`, `MISTRAL_API_KEY`, ...)
+- keyless providers (Ollama) work without any key, including in `doctor`
+
+### Changed
+
+- `doctor` reports the provider and where the key came from
+
+## 0.2.0 — 2026-11-02
+
+### Added
+
+- `--offline` mode: a full report built from local rules, no model and no key
+
+### Fixed
+
+- the ports collector was registered as a sync helper, so `--ports` and the registry entry broke
+
 ## 0.1.2 — 2026-11-02
 
 ### Fixed

{osintkit-0.1.2 → osintkit-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: osintkit
-Version: 0.1.2
+Version: 0.3.0
 Summary: Public-data OSINT aggregator for emails, usernames, domains, and companies
 Project-URL: Homepage, https://github.com/sjdsjadjajdjsa/osintkit
 Project-URL: Issues, https://github.com/sjdsjadjajdjsa/osintkit/issues
@@ -74,16 +74,20 @@ checksums, and `pip install -e .` works from a clone. Python 3.12+ only.
 ## Quick start
 
 ```bash
-export API_KEY=...                 # model key, any OpenAI-compatible endpoint
+export API_KEY=...                 # or DEEPSEEK_API_KEY, MISTRAL_API_KEY, ...
 osintkit doctor --deep             # prove the key and endpoint work
 osintkit lookup alice@example.com
 ```
 
-No key? `osintkit lookup example.com --raw` still collects everything; `--dry-run` shows the
-exact synthesis prompt.
+No key? `osintkit lookup example.com --offline` builds the whole report locally from rule checks,
+no model involved. `--raw` gives the collected JSON and `--dry-run` prints the prompt a model
+would receive.
 
-Any OpenAI-compatible endpoint works. `--provider venice|openrouter|openai` picks a preset, and
-a recognisable key prefix picks one for you; `--base-url` and `--model` always win.
+Any OpenAI-compatible endpoint works: export the provider's own variable (`DEEPSEEK_API_KEY`,
+`MISTRAL_API_KEY`, `GROQ_API_KEY`, `VENICE_API_KEY`, `OPENROUTER_API_KEY`, `OPENAI_API_KEY`,
+`TOGETHER_API_KEY`, `ANTHROPIC_API_KEY`) and that provider is used. `--provider` also accepts
+`ollama` for a local server with no key. `--base-url` and `--model` always win, and the full
+preset table is in [docs/REFERENCE.md](docs/REFERENCE.md#providers).
 
 Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts GitHub from 60 to
 5000 requests/hour. Everything else needs no credentials.
@@ -102,7 +106,7 @@ Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts G
 | `schema` | The JSON contract the model must satisfy |
 
 Output: default colour report, `--json`, `--markdown` (they combine), `--raw` for collected
-data, `--dry-run` for the prompt. Exit codes: `0` ok, `1` synthesis or partial failure, `2` bad
+data, `--offline` for a local report with no model, `--dry-run` for the prompt. Exit codes: `0` ok, `1` synthesis or partial failure, `2` bad
 input.
 
 ## How it works
@@ -115,6 +119,7 @@ target -> type detection -> collectors (parallel, cached, retried)
                                  |
                                  v
            LLM briefing -> pydantic validation -> one repair retry
+                                 |            (--offline replaces this with local rules)
                                  |
                                  v
               colour report . JSON . Markdown . archive

{osintkit-0.1.2 → osintkit-0.3.0}/README.md

@@ -42,16 +42,20 @@ checksums, and `pip install -e .` works from a clone. Python 3.12+ only.
 ## Quick start
 
 ```bash
-export API_KEY=...                 # model key, any OpenAI-compatible endpoint
+export API_KEY=...                 # or DEEPSEEK_API_KEY, MISTRAL_API_KEY, ...
 osintkit doctor --deep             # prove the key and endpoint work
 osintkit lookup alice@example.com
 ```
 
-No key? `osintkit lookup example.com --raw` still collects everything; `--dry-run` shows the
-exact synthesis prompt.
+No key? `osintkit lookup example.com --offline` builds the whole report locally from rule checks,
+no model involved. `--raw` gives the collected JSON and `--dry-run` prints the prompt a model
+would receive.
 
-Any OpenAI-compatible endpoint works. `--provider venice|openrouter|openai` picks a preset, and
-a recognisable key prefix picks one for you; `--base-url` and `--model` always win.
+Any OpenAI-compatible endpoint works: export the provider's own variable (`DEEPSEEK_API_KEY`,
+`MISTRAL_API_KEY`, `GROQ_API_KEY`, `VENICE_API_KEY`, `OPENROUTER_API_KEY`, `OPENAI_API_KEY`,
+`TOGETHER_API_KEY`, `ANTHROPIC_API_KEY`) and that provider is used. `--provider` also accepts
+`ollama` for a local server with no key. `--base-url` and `--model` always win, and the full
+preset table is in [docs/REFERENCE.md](docs/REFERENCE.md#providers).
 
 Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts GitHub from 60 to
 5000 requests/hour. Everything else needs no credentials.
@@ -70,7 +74,7 @@ Optional keys: `HIBP_API_KEY` turns on Have I Been Pwned, `GITHUB_TOKEN` lifts G
 | `schema` | The JSON contract the model must satisfy |
 
 Output: default colour report, `--json`, `--markdown` (they combine), `--raw` for collected
-data, `--dry-run` for the prompt. Exit codes: `0` ok, `1` synthesis or partial failure, `2` bad
+data, `--offline` for a local report with no model, `--dry-run` for the prompt. Exit codes: `0` ok, `1` synthesis or partial failure, `2` bad
 input.
 
 ## How it works
@@ -83,6 +87,7 @@ target -> type detection -> collectors (parallel, cached, retried)
                                  |
                                  v
            LLM briefing -> pydantic validation -> one repair retry
+                                 |            (--offline replaces this with local rules)
                                  |
                                  v
               colour report . JSON . Markdown . archive

{osintkit-0.1.2 → osintkit-0.3.0}/docs/REFERENCE.md

@@ -34,11 +34,20 @@ osintkit --provider openrouter ...
 osintkit --base-url https://gateway.internal/v1 --model house-model ...
 ```
 
-| `--provider` | Endpoint | Default model | Auto-selected by |
+| `--provider` | Endpoint | Default model | Key variable |
 | --- | --- | --- | --- |
-| `venice` (default) | `api.venice.ai/api/v1` | `deepseek-v4-1-flash` | anything else |
-| `openrouter` | `openrouter.ai/api/v1` | `deepseek/deepseek-v4.1-flash-1017` | `sk-or-` key |
-| `openai` | `api.openai.com/v1` | `gpt-4.1-mini` | `sk-proj-` key |
+| `venice` (default) | `api.venice.ai/api/v1` | `deepseek-v4-1-flash` | `API_KEY`, `VENICE_API_KEY` |
+| `openrouter` | `openrouter.ai/api/v1` | `deepseek/deepseek-v4.1-flash-1017` | `OPENROUTER_API_KEY`, `sk-or-` keys |
+| `openai` | `api.openai.com/v1` | `gpt-4.1-mini` | `OPENAI_API_KEY`, `sk-proj-` keys |
+| `deepseek` | `api.deepseek.com/v1` | `deepseek-chat` | `DEEPSEEK_API_KEY` |
+| `mistral` | `api.mistral.ai/v1` | `mistral-small-latest` | `MISTRAL_API_KEY` |
+| `groq` | `api.groq.com/openai/v1` | `llama-3.3-70b-versatile` | `GROQ_API_KEY`, `gsk_` keys |
+| `together` | `api.together.xyz/v1` | Llama 3.3 70B | `TOGETHER_API_KEY` |
+| `anthropic` | `api.anthropic.com/v1` | `claude-sonnet-4-5` | `ANTHROPIC_API_KEY`, `sk-ant-` keys |
+| `ollama` | `localhost:11434/v1` | `llama3.2` | none |
+
+Set a provider-specific variable and that provider is chosen; `API_KEY` alone falls back to the
+default. Recognisable key prefixes override that inference. `--provider` beats both.
 
 Explicit `--base-url` and `--model` beat both the provider and the prefix. Temperature is 0.3,
 `--max-tokens` defaults to 2400.
@@ -49,9 +58,16 @@ Explicit `--base-url` and `--model` beat both the provider and the prefix. Tempe
 `--max-connections`, `--max-attempts`, `--no-cache`, `--cache-ttl`, `--resolver` (repeatable),
 `-v/--verbose`, `--version`.
 
-Per-command: `--json`, `--markdown`, `-o/--out`, `--raw`, `--dry-run`, `--only`, `--skip`,
+Per-command: `--json`, `--markdown`, `-o/--out`, `--raw`, `--offline`, `--dry-run`, `--only`, `--skip`,
 `--ports`, `--fail-fast`, `-c/--concurrency`, `--limit`, `--show`, `--days`, `--clear`, `--deep`.
 
+## Offline mode
+
+`--offline` skips the model: breaches, accounts, hosts, and policy gaps are rendered from the
+collected data, risk comes from the same deterministic floor (plus open risky ports and a
+wildcard with no DMARC), and recommendations come from rule checks. Summary text says what was
+and was not collected. Useful without a key, in CI, and when a provider is down.
+
 ## Output contract
 
 `osintkit schema` prints the JSON Schema. Synthesis must return `summary`, `breaches[]`,

{osintkit-0.1.2 → osintkit-0.3.0}/examples/README.md

@@ -36,3 +36,10 @@ Narrow a port scan:
 ```bash
 osintkit lookup --domain example.com --only tcp_ports --ports 22,443
 ```
+
+No key at all:
+
+```bash
+osintkit lookup --domain example.com --offline
+osintkit batch --file targets.csv --offline --json
+```

{osintkit-0.1.2 → osintkit-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "osintkit"
-version = "0.1.2"
+version = "0.3.0"
 description = "Public-data OSINT aggregator for emails, usernames, domains, and companies"
 readme = "README.md"
 requires-python = ">=3.12"

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/__init__.py

@@ -1,3 +1,3 @@
 """osintkit: a public-data OSINT aggregator."""
 
-__version__ = "0.1.2"
+__version__ = "0.3.0"

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/cli.py

@@ -21,10 +21,12 @@ from rich.table import Table
 from . import __version__, diff, doctor, pipeline, render, retention
 from .collectors import REGISTRY
 from .config import (
+    GENERIC_KEY_ENVS,
     PROVIDERS,
     Settings,
     cache_clear,
     cache_stats,
+    provider_from_env,
     provider_from_key,
 )
 from .errors import OsintKitError, TargetParseError
@@ -32,7 +34,7 @@ from .logging_store import RunLogger, find_report_for_run, read_report, read_run
 from .targets import VALID_TYPES
 
 app = typer.Typer(
-    add_completion=False,
+    add_completion=True,
     no_args_is_help=True,
     help="Public-data OSINT aggregator for emails, usernames, domains, and companies.",
     rich_markup_mode="rich",
@@ -61,7 +63,7 @@ def _version_callback(value: bool) -> None:
 @app.callback()
 def root(
     provider: str | None = typer.Option(
-        None, "--provider", help="Endpoint preset: venice, openrouter, or openai."
+        None, "--provider", help="Endpoint preset; see docs/REFERENCE.md#providers."
     ),
     base_url: str | None = typer.Option(
         None, "--base-url", help="OpenAI-compatible base URL (overrides the provider)."
@@ -88,10 +90,17 @@ def root(
 ) -> None:
     if provider and provider not in PROVIDERS:
         raise typer.BadParameter(f"unknown provider {provider!r}; choose from {', '.join(PROVIDERS)}")
-    effective_key = api_key or os.environ.get("API_KEY") or os.environ.get("OPENAI_API_KEY")
-    chosen = provider or provider_from_key(effective_key) or "venice"
+    generic_env = next((os.environ[name] for name in GENERIC_KEY_ENVS if os.environ.get(name)), None)
+    chosen = (
+        provider
+        or provider_from_key(api_key)
+        or provider_from_env()
+        or provider_from_key(generic_env)
+        or "venice"
+    )
     preset = PROVIDERS[chosen]
     settings = Settings(
+        provider=chosen,
         base_url=base_url or preset[0],
         model=model or preset[1],
         api_key=api_key,
@@ -235,13 +244,20 @@ def lookup(
     out_path: Path | None = typer.Option(None, "--out", "-o", help="Markdown destination."),
     dry_run: bool = typer.Option(False, "--dry-run", help="Print the prompt, skip the model."),
     raw: bool = typer.Option(False, "--raw", help="Dump collected data as JSON, skip the model."),
+    offline: bool = typer.Option(False, "--offline", help="Report from collected data only; no model, no key."),
     only: list[str] | None = typer.Option(None, "--only", help="Run only these collectors."),
     skip: list[str] | None = typer.Option(None, "--skip", help="Skip these collectors."),
     ports: list[str] | None = typer.Option(None, "--ports", help="Ports to probe, e.g. 22,443."),
 ) -> None:
     """Look up one target."""
     out = _console()
-    settings = _settings(output_json=json_out, output_markdown=markdown, dry_run=dry_run, raw=raw)
+    settings = _settings(
+        output_json=json_out,
+        output_markdown=markdown,
+        dry_run=dry_run,
+        raw=raw,
+        offline=offline,
+    )
     try:
         settings.extra["ports"] = _parse_ports(ports)
     except typer.BadParameter as exc:
@@ -317,6 +333,7 @@ def batch(
     out_dir: Path | None = typer.Option(None, "--out", "-o", help="Markdown output directory."),
     dry_run: bool = typer.Option(False, "--dry-run", help="Print prompts, skip the model."),
     raw: bool = typer.Option(False, "--raw", help="JSON array of collected data, skip the model."),
+    offline: bool = typer.Option(False, "--offline", help="Report from collected data only; no model, no key."),
     fail_fast: bool = typer.Option(False, "--fail-fast", help="Stop reporting at the first error."),
     concurrency: int = typer.Option(3, "--concurrency", "-c", min=1, max=10, help="Targets to run at once."),
     only: list[str] | None = typer.Option(None, "--only", help="Run only these collectors."),
@@ -325,7 +342,13 @@ def batch(
 ) -> None:
     """Run every target in a CSV."""
     out = _console()
-    settings = _settings(output_json=json_out, output_markdown=markdown, dry_run=dry_run, raw=raw)
+    settings = _settings(
+        output_json=json_out,
+        output_markdown=markdown,
+        dry_run=dry_run,
+        raw=raw,
+        offline=offline,
+    )
     try:
         settings.extra["ports"] = _parse_ports(ports)
     except typer.BadParameter as exc:

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/config.py

@@ -23,8 +23,31 @@ PROVIDERS: dict[str, tuple[str, str]] = {
     "venice": (DEFAULT_BASE_URL, DEFAULT_MODEL),
     "openrouter": ("https://openrouter.ai/api/v1", "deepseek/deepseek-v4.1-flash-1017"),
     "openai": ("https://api.openai.com/v1", "gpt-4.1-mini"),
+    "deepseek": ("https://api.deepseek.com/v1", "deepseek-chat"),
+    "mistral": ("https://api.mistral.ai/v1", "mistral-small-latest"),
+    "groq": ("https://api.groq.com/openai/v1", "llama-3.3-70b-versatile"),
+    "together": ("https://api.together.xyz/v1", "meta-llama/Llama-3.3-70B-Instruct-Turbo"),
+    "anthropic": ("https://api.anthropic.com/v1", "claude-sonnet-4-5"),
+    "ollama": ("http://localhost:11434/v1", "llama3.2"),
 }
-KEY_PREFIXES: dict[str, str] = {"sk-or-": "openrouter", "sk-proj-": "openai"}
+KEYLESS_PROVIDERS = frozenset({"ollama"})
+KEY_PREFIXES: dict[str, str] = {
+    "sk-or-": "openrouter",
+    "sk-proj-": "openai",
+    "gsk_": "groq",
+    "sk-ant-": "anthropic",
+}
+PROVIDER_ENV: dict[str, str] = {
+    "VENICE_API_KEY": "venice",
+    "OPENROUTER_API_KEY": "openrouter",
+    "OPENAI_API_KEY": "openai",
+    "DEEPSEEK_API_KEY": "deepseek",
+    "MISTRAL_API_KEY": "mistral",
+    "GROQ_API_KEY": "groq",
+    "TOGETHER_API_KEY": "together",
+    "ANTHROPIC_API_KEY": "anthropic",
+}
+GENERIC_KEY_ENVS = ("API_KEY",)
 
 
 def provider_from_key(api_key: str | None) -> str | None:
@@ -37,6 +60,14 @@ def provider_from_key(api_key: str | None) -> str | None:
     return None
 
 
+def provider_from_env() -> str | None:
+    """Infer a provider from whichever provider-specific variable is set."""
+    for variable, provider in PROVIDER_ENV.items():
+        if os.environ.get(variable):
+            return provider
+    return None
+
+
 def app_home() -> Path:
     override = os.environ.get(APP_HOME_ENV)
     if override:
@@ -67,6 +98,7 @@ class Settings:
 
     target: str = ""
     target_type: str = ""
+    provider: str = "venice"
     base_url: str = DEFAULT_BASE_URL
     model: str = DEFAULT_MODEL
     api_key: str | None = None
@@ -79,6 +111,7 @@ class Settings:
     cache_ttl: int = DEFAULT_CACHE_TTL
     resolvers: list[str] = field(default_factory=lambda: list(DEFAULT_RESOLVERS))
     dry_run: bool = False
+    offline: bool = False
     raw: bool = False
     output_json: bool = False
     output_markdown: bool = False
@@ -94,7 +127,19 @@ class Settings:
         return os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN") or None
 
     def resolved_api_key(self) -> str | None:
-        return self.api_key or os.environ.get("API_KEY") or os.environ.get("OPENAI_API_KEY")
+        if self.api_key:
+            return self.api_key
+        for variable, provider in PROVIDER_ENV.items():
+            if provider == self.provider and os.environ.get(variable):
+                return os.environ[variable]
+        for variable in GENERIC_KEY_ENVS:
+            if os.environ.get(variable):
+                return os.environ[variable]
+        return None
+
+    @property
+    def needs_key(self) -> bool:
+        return self.provider not in KEYLESS_PROVIDERS
 
 
 def _cache_key(namespace: str, key: str) -> Path:

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/doctor.py

@@ -7,7 +7,7 @@ import tempfile
 from dataclasses import dataclass, replace
 from pathlib import Path
 
-from .config import Settings, cache_dir, ensure_dirs, log_dir
+from .config import PROVIDER_ENV, Settings, cache_dir, ensure_dirs, log_dir
 from .http import HttpClient
 
 
@@ -30,10 +30,10 @@ def _writable(path: Path) -> bool:
 
 async def _reach_endpoint(settings: Settings, deep: bool = False) -> Check:
     key = settings.resolved_api_key()
-    if not key:
+    if settings.needs_key and not key:
         return Check("model endpoint", "warn", "no API key; synthesis will fail, dry runs still work")
     base = settings.base_url.rstrip("/")
-    headers = {"Authorization": f"Bearer {key}"}
+    headers = {"Authorization": f"Bearer {key}"} if key else {}
     fresh = replace(settings, use_cache=False)
 
     try:
@@ -81,14 +81,30 @@ async def _reach_endpoint(settings: Settings, deep: bool = False) -> Check:
     return Check("model endpoint", "ok", f"{base} accepted a completion for {settings.model}")
 
 
+def _key_source(settings: Settings) -> str | None:
+    if settings.api_key:
+        return "--api-key"
+    for variable, provider in PROVIDER_ENV.items():
+        if provider == settings.provider and os.environ.get(variable):
+            return variable
+    for variable in ("API_KEY",):
+        if os.environ.get(variable):
+            return variable
+    return None
+
+
 def collect_checks(settings: Settings) -> list[Check]:
     key = settings.resolved_api_key()
+    source = _key_source(settings)
+    if not settings.needs_key:
+        key_check = Check("api key", "ok", f"not needed for {settings.provider}")
+    elif key:
+        key_check = Check("api key", "ok", f"from {source}")
+    else:
+        key_check = Check("api key", "warn", f"no key for {settings.provider}; set one or use --offline")
     checks = [
-        Check(
-            "api key",
-            "ok" if key else "warn",
-            "set" if key else "API_KEY is not set",
-        ),
+        Check("provider", "ok", settings.provider),
+        key_check,
         Check(
             "cache dir",
             "ok" if _writable(cache_dir()) else "fail",

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/llm.py

@@ -38,7 +38,7 @@ class LLMClient:
         response_format: Mapping[str, Any] | None = None,
     ) -> str:
         api_key = self.settings.resolved_api_key()
-        if not api_key:
+        if not api_key and self.settings.needs_key:
             raise MissingAPIKey("No API key. Set API_KEY in the environment or pass --api-key.")
 
         payload: dict[str, Any] = {
@@ -57,7 +57,7 @@ class LLMClient:
                 namespace="llm",
                 payload=payload,
                 headers={
-                    "Authorization": f"Bearer {api_key}",
+                    **({"Authorization": f"Bearer {api_key}"} if api_key else {}),
                     "Content-Type": "application/json",
                 },
             )

osintkit-0.3.0/src/osintkit/offline.py

@@ -0,0 +1,205 @@
+"""Build a report from collected data alone, with no model call."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from .models import Analysis, AssociatedAccount, Breach, CollectorResult
+from .normalise import apply_floor, signals_for
+
+RISKY_PORTS = {
+    21: "FTP is plaintext",
+    23: "Telnet is plaintext",
+    445: "SMB should not face the internet",
+    3306: "MySQL is exposed",
+    5432: "PostgreSQL is exposed",
+    6379: "Redis is often unauthenticated",
+    9200: "Elasticsearch is often unauthenticated",
+    27017: "MongoDB is often unauthenticated",
+}
+MAX_SUBDOMAINS = 25
+
+
+def _data(results: list[CollectorResult], name: str) -> Any:
+    for result in results:
+        if result.collector == name and result.has_data:
+            return result.data
+    return None
+
+
+def _found_accounts(results: list[CollectorResult]) -> list[AssociatedAccount]:
+    accounts: list[AssociatedAccount] = []
+    presence = _data(results, "username_presence")
+    if isinstance(presence, dict):
+        for hit in presence.get("found") or []:
+            accounts.append(
+                AssociatedAccount(
+                    platform=str(hit.get("site")),
+                    url=hit.get("url"),
+                    handle=str(presence.get("handle") or ""),
+                    note="profile URL answers 200; ownership unconfirmed",
+                )
+            )
+    profile = _data(results, "github_profile")
+    if isinstance(profile, dict):
+        user = profile.get("profile") or {}
+        if user.get("login"):
+            accounts.append(
+                AssociatedAccount(
+                    platform="GitHub",
+                    url=user.get("url"),
+                    handle=user.get("login"),
+                    note="returned by the GitHub API",
+                )
+            )
+    return accounts
+
+
+def _subdomains(results: list[CollectorResult]) -> list[str]:
+    names: list[str] = []
+    passive = _data(results, "passive_subdomains")
+    if isinstance(passive, dict):
+        names.extend(str(row.get("host")) for row in passive.get("live_hosts") or [])
+        names.extend(str(row.get("host")) for row in passive.get("common_prefix_hits") or [])
+    certs = _data(results, "crt_sh_subdomains")
+    if isinstance(certs, list):
+        names.extend(str(name) for name in certs)
+    seen: list[str] = []
+    for name in names:
+        if name not in seen:
+            seen.append(name)
+    return seen[:MAX_SUBDOMAINS]
+
+
+def _infrastructure(results: list[CollectorResult], report_notes: list[str]) -> list[str]:
+    notes: list[str] = []
+    mail = _data(results, "dns_mail_posture")
+    if isinstance(mail, dict):
+        if mail.get("spf_present"):
+            notes.append(f"SPF present, all-qualifier {mail.get('spf_all') or 'unknown'}")
+        else:
+            notes.append("No SPF record: sender enforcement is missing")
+        if mail.get("dmarc_present"):
+            notes.append(f"DMARC present with policy {mail.get('dmarc_policy') or 'unknown'}")
+        else:
+            notes.append("No DMARC record: spoofing is not instructed")
+        selectors = mail.get("dkim_selectors_found") or []
+        if selectors:
+            notes.append(f"{len(selectors)} DKIM selectors answer")
+
+    wildcard = _data(results, "dns_wildcard")
+    if isinstance(wildcard, dict) and wildcard.get("wildcard"):
+        notes.append("Wildcard DNS is enabled, so unknown labels resolve")
+
+    whois = _data(results, "whois_registration")
+    if isinstance(whois, dict) and whois.get("registrar"):
+        parts = [f"registrar {whois['registrar']}"]
+        if whois.get("creation_date"):
+            parts.append(f"registered {whois['creation_date']}")
+        if whois.get("expiration_date"):
+            parts.append(f"expires {whois['expiration_date']}")
+        notes.append(", ".join(parts))
+
+    ports = _data(results, "tcp_ports")
+    if isinstance(ports, dict):
+        for entry in ports.get("open") or []:
+            port = entry.get("port")
+            if port in RISKY_PORTS:
+                notes.append(f"port {port} open: {RISKY_PORTS[port]}")
+        if not ports.get("open"):
+            notes.append("No probed port accepted a connection")
+
+    dns = _data(results, "dns_records")
+    if isinstance(dns, dict) and dns.get("MX") == ["0 ."]:
+        notes.append("Null MX: the domain accepts no mail")
+    notes.extend(report_notes)
+    return notes
+
+
+def _recommendations(
+    creds: int, mail: dict | None, wildcard: bool, ports: list[int], breach_count: int
+) -> list[str]:
+    recs: list[str] = []
+    if creds:
+        recs.append("Rotate passwords for the affected services and enable MFA where offered")
+    elif breach_count:
+        recs.append("Confirm whether the breach set carries credentials before rotating")
+    if mail is not None:
+        if not mail.get("dmarc_present"):
+            recs.append("Publish a DMARC record, starting at p=none with reporting")
+        elif mail.get("dmarc_policy") in {"none", "quarantine"}:
+            recs.append(f"Move DMARC from p={mail.get('dmarc_policy')} to p=reject once reporting is clean")
+        if not mail.get("spf_present"):
+            recs.append("Publish an SPF record ending in -all")
+        elif mail.get("spf_all") != "-all":
+            recs.append("Tighten the SPF all-qualifier to -all")
+    if wildcard:
+        recs.append("Inventory wildcard-backed hostnames and remove ones no longer served")
+    if ports:
+        recs.append(f"Close or firewall public ports {', '.join(str(p) for p in ports)}")
+    if not recs:
+        recs.append("Re-run with a model key for a narrated briefing, or widen the collector set")
+    return recs
+
+
+def build(results: list[CollectorResult], report_notes: list[str], target: str) -> Analysis:
+    breaches_raw = next((r.data for r in results if r.collector == "breach_index" and r.has_data), [])
+    signals = signals_for(breaches_raw)
+    accounts = _found_accounts(results)
+    subdomains = _subdomains(results)
+    mail = _data(results, "dns_mail_posture")
+    wildcard = bool((_data(results, "dns_wildcard") or {}).get("wildcard"))
+    ports_data = _data(results, "tcp_ports") or {}
+    risky = [entry.get("port") for entry in ports_data.get("open") or [] if entry.get("port") in RISKY_PORTS]
+    infrastructure = _infrastructure(results, report_notes)
+
+    collected = [r.collector for r in results if r.has_data]
+    skipped = [r.collector for r in results if r.skipped]
+    failed = [r.collector for r in results if not r.ok and not r.skipped]
+
+    summary_parts = [
+        f"Collected public data for {target} from {len(collected)} of "
+        f"{len(results)} sources, with no model call."
+    ]
+    if signals["breach_count"]:
+        summary_parts.append(
+            f"{signals['breach_count']} breach records mention the target, "
+            f"{signals['credential_bearing']} of them credential-bearing."
+        )
+    if accounts:
+        summary_parts.append(f"{len(accounts)} public profiles answered.")
+    if subdomains:
+        summary_parts.append(f"{len(subdomains)} hostnames were discovered.")
+    if failed:
+        summary_parts.append(f"Failed sources: {', '.join(failed)}.")
+    if skipped:
+        summary_parts.append(f"Skipped sources: {', '.join(skipped)}.")
+    summary_parts.append("Risk is computed from counts and policy checks, not narrated.")
+
+    risk = apply_floor("low", signals["risk_floor"])
+    if risky or (wildcard and isinstance(mail, dict) and not mail.get("dmarc_present")):
+        risk = apply_floor(risk, "medium")
+
+    return Analysis(
+        summary=" ".join(summary_parts),
+        breaches=[
+            Breach(
+                name=str(row.get("name")),
+                date=row.get("date"),
+                data_types=list(row.get("data_types") or []),
+                source=row.get("source"),
+            )
+            for row in breaches_raw[:15]
+        ],
+        associated_accounts=accounts,
+        infrastructure_notes=infrastructure,
+        subdomains=subdomains,
+        risk_level=risk,
+        recommendations=_recommendations(
+            signals["credential_bearing"],
+            mail if isinstance(mail, dict) else None,
+            wildcard,
+            risky,
+            signals["breach_count"],
+        ),
+    )

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/pipeline.py

@@ -14,7 +14,7 @@ from pathlib import Path
 
 from rich.console import Console
 
-from . import collectors, normalise, render, retention, synthesis
+from . import collectors, normalise, offline, render, retention, synthesis
 from .config import Settings
 from .errors import MissingAPIKey, OsintKitError, SynthesisError, TargetParseError
 from .http import HttpClient
@@ -182,6 +182,20 @@ async def execute(
             logger.event("run_end", raw=True, report_path=str(archived) if archived else None)
             return Outcome(report=report, extra={"bundle": bundle}, archived=archived)
 
+        if settings.offline:
+            report.model = None
+            report.analysis = offline.build(prompt_results, report.notes, target)
+            report.notes.append("offline mode: risk computed locally, no model call")
+            report.elapsed_ms = int((time.perf_counter() - started) * 1000)
+            archived = archive_report(report.model_dump(mode="json"))
+            logger.event(
+                "run_end",
+                offline=True,
+                report_path=str(archived) if archived else None,
+                risk_level=report.analysis.risk_level,
+            )
+            return Outcome(report=report, archived=archived)
+
         if settings.dry_run:
             report.notes.append("dry run: model not called")
             report.elapsed_ms = int((time.perf_counter() - started) * 1000)
@@ -198,8 +212,10 @@ async def execute(
             return Outcome(report=report, extra={"prompt": synthesis_prompt}, archived=archived)
 
         try:
-            if not settings.resolved_api_key():
-                raise MissingAPIKey("No API key. Set API_KEY or pass --api-key (dry run works without one).")
+            if settings.needs_key and not settings.resolved_api_key():
+                raise MissingAPIKey(
+                    "No API key. Set API_KEY, pass --api-key, or use --offline for a local report."
+                )
             async with LLMClient(settings, http) as client:
                 with _status(progress, settings, "[cyan]synthesising briefing…[/cyan]"):
                     report.analysis = await synthesis.synthesize(

{osintkit-0.1.2 → osintkit-0.3.0}/tests/test_cli.py

@@ -447,3 +447,64 @@ def test_collectors_json():
 
 def fake_name():
     return "fake_source"
+
+
+def test_offline_mode_needs_no_key(monkeypatch):
+    monkeypatch.delenv("API_KEY")
+    monkeypatch.setattr(pipeline, "LLMClient", None)
+    result = runner.invoke(app, ["lookup", "--domain", "example.com", "--offline"])
+    assert result.exit_code == 0
+    assert "no model call" in result.output
+
+
+def test_offline_json_is_a_normal_report(monkeypatch):
+    monkeypatch.delenv("API_KEY")
+    result = runner.invoke(app, ["lookup", "--domain", "example.com", "--offline", "--json"])
+    payload = jsonish(result.output)
+    assert payload["analysis"]["risk_level"] == "low"
+    assert payload["target"] == "example.com"
+
+
+def test_offline_batch(monkeypatch, tmp_path):
+    monkeypatch.delenv("API_KEY")
+    csv_file = tmp_path / "t.csv"
+    csv_file.write_text("a.com\n")
+    result = runner.invoke(app, ["batch", "--file", str(csv_file), "--offline", "--json"])
+    assert result.exit_code == 0
+    assert jsonish(result.output)[0]["analysis"]["risk_level"] == "low"
+
+
+def test_missing_key_mentions_offline(monkeypatch):
+    monkeypatch.delenv("API_KEY")
+    monkeypatch.setattr(pipeline, "LLMClient", None)
+    result = runner.invoke(app, ["lookup", "--domain", "example.com"])
+    assert "--offline" in " ".join(result.output.split())
+
+
+def test_deepseek_env_selects_provider(monkeypatch):
+    monkeypatch.setenv("DEEPSEEK_API_KEY", "d" * 30)
+    monkeypatch.setattr(pipeline, "LLMClient", FakeLLM)
+    result = runner.invoke(app, ["lookup", "--domain", "a.com", "--json"])
+    assert jsonish(result.output)["base_url"] == "https://api.deepseek.com/v1"
+
+
+def test_mistral_env_selects_provider(monkeypatch):
+    monkeypatch.setenv("MISTRAL_API_KEY", "m" * 30)
+    monkeypatch.setattr(pipeline, "LLMClient", FakeLLM)
+    result = runner.invoke(app, ["lookup", "--domain", "a.com", "--json"])
+    assert jsonish(result.output)["base_url"] == "https://api.mistral.ai/v1"
+
+
+def test_explicit_provider_beats_env(monkeypatch):
+    monkeypatch.setenv("DEEPSEEK_API_KEY", "d" * 30)
+    monkeypatch.setattr(pipeline, "LLMClient", FakeLLM)
+    result = runner.invoke(app, ["--provider", "mistral", "lookup", "--domain", "a.com", "--json"])
+    assert jsonish(result.output)["base_url"] == "https://api.mistral.ai/v1"
+
+
+def test_ollama_needs_no_key(monkeypatch):
+    monkeypatch.delenv("API_KEY")
+    monkeypatch.setattr(pipeline, "LLMClient", FakeLLM)
+    result = runner.invoke(app, ["--provider", "ollama", "lookup", "--domain", "a.com", "--json"])
+    assert result.exit_code == 0
+    assert jsonish(result.output)["base_url"] == "http://localhost:11434/v1"

{osintkit-0.1.2 → osintkit-0.3.0}/tests/test_collectors.py

@@ -477,3 +477,10 @@ async def test_ports_override_limits_the_scan(make_context, monkeypatch):
 def test_selected_ports_keeps_order_and_marks_unknown():
     assert ports.selected_ports(["443", "9999"]) == {443: "https", 9999: "unknown"}
     assert ports.selected_ports(None)[22] == "ssh"
+
+
+def test_every_registered_collector_is_coroutine():
+    import inspect
+
+    for spec in base.REGISTRY:
+        assert inspect.iscoroutinefunction(spec.func), f"{spec.name} is not async"

{osintkit-0.1.2 → osintkit-0.3.0}/tests/test_doctor.py

@@ -47,6 +47,19 @@ async def test_no_key_skips_endpoint_call():
     assert "no API key" in checks[-1].detail
 
 
+async def test_keyless_provider_needs_no_key():
+    checks = await doctor.diagnose(Settings(provider="ollama", base_url="http://localhost:1/v1"))
+    key = next(c for c in checks if c.name == "api key")
+    assert key.status == "ok"
+    assert "not needed" in key.detail
+
+
+def test_key_source_names_the_variable(monkeypatch):
+    monkeypatch.setenv("DEEPSEEK_API_KEY", "d" * 25)
+    check = next(c for c in doctor.collect_checks(Settings(provider="deepseek")) if c.name == "api key")
+    assert check.detail == "from DEEPSEEK_API_KEY"
+
+
 @respx.mock
 async def test_deep_probe_accepts_completion():
     respx.get("https://api.test/v1/models").mock(return_value=httpx.Response(200, json={}))
@@ -81,3 +94,11 @@ async def test_shallow_probe_does_not_call_the_model():
     respx.get("https://api.test/v1/models").mock(return_value=httpx.Response(200, json={}))
     checks = await doctor.diagnose(Settings(base_url="https://api.test/v1", api_key="k" * 25))
     assert "not exercised" in checks[-1].detail
+
+
+@respx.mock
+async def test_keyless_provider_probes_without_auth():
+    route = respx.get("http://localhost:1/v1/models").mock(return_value=httpx.Response(200, json={}))
+    checks = await doctor.diagnose(Settings(provider="ollama", base_url="http://localhost:1/v1"))
+    assert checks[-1].status == "ok"
+    assert "authorization" not in route.calls[0].request.headers

{osintkit-0.1.2 → osintkit-0.3.0}/tests/test_llm.py

@@ -96,3 +96,16 @@ async def test_other_errors_stay_source_errors(settings):
     async with client(settings) as llm:
         with pytest.raises(SourceUnavailable):
             await llm.chat([{"role": "user", "content": "x"}])
+
+
+@respx.mock
+async def test_keyless_provider_sends_no_auth_header(settings):
+    settings.provider = "ollama"
+    settings.api_key = None
+    settings.base_url = "http://localhost:1/v1"
+    route = respx.post("http://localhost:1/v1/chat/completions").mock(
+        return_value=httpx.Response(200, json={"choices": [{"message": {"content": "ok"}}]})
+    )
+    async with LLMClient(settings) as llm:
+        assert await llm.chat([{"role": "user", "content": "x"}]) == "ok"
+    assert "authorization" not in route.calls[0].request.headers

osintkit-0.3.0/tests/test_offline.py

@@ -0,0 +1,99 @@
+from osintkit import offline
+from osintkit.models import CollectorResult
+
+
+def result(name, data, **kwargs):
+    return CollectorResult(collector=name, target="example.com", target_type="domain", data=data, **kwargs)
+
+
+def test_breaches_drive_risk_and_rotation():
+    results = [
+        result("breach_index", [{"name": "Acme", "data_types": ["Passwords"], "source": "x"}]),
+        result("breach_index", []),
+    ]
+    analysis = offline.build(results, [], "a@b.co")
+    assert analysis.risk_level == "medium"
+    assert analysis.breaches[0].name == "Acme"
+    assert "Rotate" in analysis.recommendations[0]
+
+
+def test_missing_dmarc_is_recommended():
+    results = [result("dns_mail_posture", {"spf_present": True, "spf_all": "-all", "dmarc_present": False})]
+    analysis = offline.build(results, [], "example.com")
+    assert any("DMARC" in note for note in analysis.infrastructure_notes)
+    assert any("DMARC" in rec for rec in analysis.recommendations)
+
+
+def test_loose_spf_is_recommended():
+    results = [
+        result(
+            "dns_mail_posture",
+            {"spf_present": True, "spf_all": "~all", "dmarc_present": True, "dmarc_policy": "reject"},
+        )
+    ]
+    analysis = offline.build(results, [], "example.com")
+    assert any("-all" in rec for rec in analysis.recommendations)
+
+
+def test_risky_port_raises_risk_and_is_noted():
+    results = [result("tcp_ports", {"open": [{"port": 6379, "service": "redis"}]})]
+    analysis = offline.build(results, [], "example.com")
+    assert analysis.risk_level == "medium"
+    assert any("6379" in note for note in analysis.infrastructure_notes)
+    assert any("firewall" in rec for rec in analysis.recommendations)
+
+
+def test_accounts_come_from_presence_and_profile():
+    results = [
+        result(
+            "username_presence",
+            {"handle": "alice", "found": [{"site": "GitHub", "url": "https://github.com/alice"}]},
+        ),
+        result("github_profile", {"profile": {"login": "alice", "url": "https://github.com/alice"}}),
+    ]
+    analysis = offline.build(results, [], "alice")
+    assert [account.platform for account in analysis.associated_accounts] == ["GitHub", "GitHub"]
+    assert "unconfirmed" in analysis.associated_accounts[0].note
+
+
+def test_subdomains_are_deduped_and_capped():
+    hosts = [{"host": f"h{index}.example.com"} for index in range(40)]
+    results = [
+        result("passive_subdomains", {"live_hosts": hosts}),
+        result("crt_sh_subdomains", ["h0.example.com"]),
+    ]
+    analysis = offline.build(results, [], "example.com")
+    assert len(analysis.subdomains) == offline.MAX_SUBDOMAINS
+    assert analysis.subdomains[0] == "h0.example.com"
+
+
+def test_wildcard_without_dmarc_is_medium():
+    results = [
+        result("dns_wildcard", {"wildcard": True}),
+        result(
+            "dns_mail_posture",
+            {"dmarc_present": True, "spf_present": True, "spf_all": "-all", "dmarc_policy": "reject"},
+        ),
+    ]
+    analysis = offline.build(results, [], "example.com")
+    assert any("Wildcard" in note for note in analysis.infrastructure_notes)
+    assert any("wildcard" in rec for rec in analysis.recommendations)
+
+
+def test_gaps_are_reported_in_summary():
+    results = [
+        CollectorResult(collector="hibp_breaches", target="t", target_type="email", ok=False, skipped=True),
+        CollectorResult(collector="dns_records", target="t", target_type="email", ok=False, error="boom"),
+        result("dns_records", {"A": ["1.2.3.4"]}),
+    ]
+    analysis = offline.build(results, ["carried note"], "t")
+    assert "Failed sources: dns_records" in analysis.summary
+    assert "Skipped sources: hibp_breaches" in analysis.summary
+    assert "carried note" in analysis.infrastructure_notes
+
+
+def test_empty_collection_still_yields_a_report():
+    analysis = offline.build([], [], "example.com")
+    assert analysis.risk_level == "low"
+    assert analysis.breaches == []
+    assert analysis.recommendations

{osintkit-0.1.2 → osintkit-0.3.0}/tests/test_pipeline.py

@@ -185,3 +185,12 @@ async def test_no_breaches_keep_the_model_choice(settings, monkeypatch, console)
     monkeypatch.setattr(pipeline, "LLMClient", FakeLLM)
     outcome = await pipeline.execute("a@b.co", settings, RunLogger("floor2"), console)
     assert outcome.report.analysis.risk_level == "low"
+
+
+async def test_offline_builds_analysis_without_model(settings, fake_collector, monkeypatch, console):
+    monkeypatch.setattr(pipeline, "LLMClient", None)
+    settings.offline = True
+    outcome = await pipeline.execute("example.com", settings, RunLogger("off"), console)
+    assert outcome.report.analysis is not None
+    assert outcome.report.model is None
+    assert any("offline" in note for note in outcome.report.notes)

{osintkit-0.1.2 → osintkit-0.3.0}/src/osintkit/collectors/ports.py

@@ -59,7 +59,6 @@ async def _probe(host: str, port: int, service: str, semaphore: asyncio.Semaphor
         return {"port": port, "service": service, "banner": banner or None}
 
 
-@collector("tcp_ports", ("domain",), "Open TCP ports on common services", order=80, ttl=3600)
 def selected_ports(override: list[str] | None) -> dict[int, str]:
     if not override:
         return dict(PORTS)
@@ -70,6 +69,7 @@ def selected_ports(override: list[str] | None) -> dict[int, str]:
     return dict(sorted(chosen.items()))
 
 
+@collector("tcp_ports", ("domain",), "Open TCP ports on common services", order=80, ttl=3600)
 async def tcp_ports(ctx: Context) -> dict:
     ports = selected_ports(ctx.settings.extra.get("ports"))
     host = ctx.target.lower()