reconkit 0.1.0__tar.gz

reconkit-0.1.0/.gitignore

@@ -0,0 +1,30 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+build/
+dist/
+*.egg-info/
+.eggs/
+
+# Virtual environments
+.venv/
+venv/
+ENV/
+
+# Test / tooling caches
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# OS / editor
+.DS_Store
+Thumbs.db
+.vscode/

reconkit-0.1.0/CHANGELOG.md

@@ -0,0 +1,21 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+Format: loosely based on *Keep a Changelog*. Versioning: SemVer where practical.
+
+## [0.1.0] - 2026-02-09
+
+### Added
+- Noir Stack public primitives: `EvidenceArtifact`, `EvidenceBundle`, `ProvenanceRecord`, `VerificationResult`.
+- Domain-specific errors that describe trust boundaries and provenance constraints.
+- Audit-grade CLI vocabulary: `evidence`, `boundary`, `verify`.
+- Structured JSON outputs include: `decision_basis`, `verification_status`, `provenance_confidence`.
+- `robots.txt` compliance (default) and rate limiting for web acquisition.
+
+### Changed
+- Public-facing surface and CLI terminology refactored to evidence/verification framing.
+
+### Notes
+- This release is intentionally conservative.
+- The system is capable of refusing to speculate. This is a feature.

reconkit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Noir Stack LLC
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

reconkit-0.1.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: reconkit
+Version: 0.1.0
+Summary: A small, safe-by-default toolkit for scraping, OSINT, and file triage.
+Author: reconkit
+License: MIT
+License-File: LICENSE
+Keywords: forensics,osint,scraping,triage
+Requires-Python: >=3.10
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: dnspython>=2.6
+Requires-Dist: exifread>=3.0
+Requires-Dist: filetype>=1.2
+Requires-Dist: httpx>=0.27
+Requires-Dist: ipwhois>=1.2
+Requires-Dist: lxml>=5.1
+Requires-Dist: python-whois>=0.9
+Requires-Dist: rich>=13.7
+Requires-Dist: typer>=0.12
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.4; extra == 'dev'
+Requires-Dist: twine>=5.1; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# reconkit (Judicium surface)
+
+`reconkit` is a **conservative, safe-by-default** toolkit that exposes a Noir Stack / Hexarch / Judicium-flavored public API.
+
+It is designed to produce **audit-grade structured outputs** that explicitly state:
+- what was observed (evidence)
+- how it was observed (provenance)
+- what is and is not being asserted (verification limits)
+
+This project is intended for lawful use on systems/data you own or have permission to assess.
+
+## Noir Stack primitives
+
+Public primitives:
+- `EvidenceArtifact`: immutable bytes + provenance
+- `EvidenceBundle`: a set of artifacts for joint assessment
+- `ProvenanceRecord`: acquisition/derivation metadata + trust boundary
+- `VerificationResult`: structured decision output
+
+All structured outputs include:
+- `decision_basis`
+- `verification_status`
+- `provenance_confidence`
+
+Because this is Judicium, not divination.
+
+## Guarantees
+
+What the system will reliably do:
+- Preserve evidence payload bytes as acquired/derived, and compute stable SHA-256 for artifacts.
+- Emit structured, JSON-serializable outputs suitable for logging and later review.
+- Respect declared acquisition constraints (e.g., `max_bytes`) and treat them as policy boundaries.
+- Respect `robots.txt` by default for web acquisition.
+
+## Assumptions
+
+What the system assumes unless you override it:
+- `trust_boundary` is declared explicitly for acquisitions. The tool will not invent one.
+- Web acquisition is transport-level observation, not identity/authenticity proof.
+- Public-internet boundary data (DNS/WHOIS/RDAP) is administrative metadata.
+
+## Non-goals
+
+What the system explicitly does not do:
+- No anti-bot bypassing, fingerprint evasion, proxy-rotation orchestration, or credentialed scraping.
+- No claims of attribution, intent, or identity. (You can do that in your report; the tool will not do it for you.)
+- No “automatic truth.” The system is already busy being correct.
+
+## Install
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## CLI quickstart (Noir terminology)
+
+All commands emit audit-grade JSON.
+
+```bash
+# Evidence acquisition / derivation
+reconkit evidence acquire-web "https://example.com" --trust-boundary public-internet
+reconkit evidence derive-links "https://example.com"
+reconkit evidence derive-text "https://example.com"
+
+# Boundary observations (OSINT-ish, but we say the quiet part out loud)
+reconkit boundary dns example.com
+reconkit boundary whois example.com
+reconkit boundary ip 8.8.8.8
+
+# Verification (informational)
+reconkit verify file-hash .\somefile.bin --algo sha256
+reconkit verify file-type .\somefile.bin
+reconkit verify strings .\somefile.bin --min-len 6
+reconkit verify exif .\photo.jpg
+```
+
+## Notes
+
+- If acquisition is blocked by `robots.txt`, that is not a “bug.” It is a boundary condition.
+- If you need features that reduce accountability, you are in the wrong repository. Convenient.

reconkit-0.1.0/README.md

@@ -0,0 +1,80 @@
+# reconkit (Judicium surface)
+
+`reconkit` is a **conservative, safe-by-default** toolkit that exposes a Noir Stack / Hexarch / Judicium-flavored public API.
+
+It is designed to produce **audit-grade structured outputs** that explicitly state:
+- what was observed (evidence)
+- how it was observed (provenance)
+- what is and is not being asserted (verification limits)
+
+This project is intended for lawful use on systems/data you own or have permission to assess.
+
+## Noir Stack primitives
+
+Public primitives:
+- `EvidenceArtifact`: immutable bytes + provenance
+- `EvidenceBundle`: a set of artifacts for joint assessment
+- `ProvenanceRecord`: acquisition/derivation metadata + trust boundary
+- `VerificationResult`: structured decision output
+
+All structured outputs include:
+- `decision_basis`
+- `verification_status`
+- `provenance_confidence`
+
+Because this is Judicium, not divination.
+
+## Guarantees
+
+What the system will reliably do:
+- Preserve evidence payload bytes as acquired/derived, and compute stable SHA-256 for artifacts.
+- Emit structured, JSON-serializable outputs suitable for logging and later review.
+- Respect declared acquisition constraints (e.g., `max_bytes`) and treat them as policy boundaries.
+- Respect `robots.txt` by default for web acquisition.
+
+## Assumptions
+
+What the system assumes unless you override it:
+- `trust_boundary` is declared explicitly for acquisitions. The tool will not invent one.
+- Web acquisition is transport-level observation, not identity/authenticity proof.
+- Public-internet boundary data (DNS/WHOIS/RDAP) is administrative metadata.
+
+## Non-goals
+
+What the system explicitly does not do:
+- No anti-bot bypassing, fingerprint evasion, proxy-rotation orchestration, or credentialed scraping.
+- No claims of attribution, intent, or identity. (You can do that in your report; the tool will not do it for you.)
+- No “automatic truth.” The system is already busy being correct.
+
+## Install
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## CLI quickstart (Noir terminology)
+
+All commands emit audit-grade JSON.
+
+```bash
+# Evidence acquisition / derivation
+reconkit evidence acquire-web "https://example.com" --trust-boundary public-internet
+reconkit evidence derive-links "https://example.com"
+reconkit evidence derive-text "https://example.com"
+
+# Boundary observations (OSINT-ish, but we say the quiet part out loud)
+reconkit boundary dns example.com
+reconkit boundary whois example.com
+reconkit boundary ip 8.8.8.8
+
+# Verification (informational)
+reconkit verify file-hash .\somefile.bin --algo sha256
+reconkit verify file-type .\somefile.bin
+reconkit verify strings .\somefile.bin --min-len 6
+reconkit verify exif .\photo.jpg
+```
+
+## Notes
+
+- If acquisition is blocked by `robots.txt`, that is not a “bug.” It is a boundary condition.
+- If you need features that reduce accountability, you are in the wrong repository. Convenient.

reconkit-0.1.0/pyproject.toml

@@ -0,0 +1,56 @@
+[build-system]
+requires = ["hatchling>=1.25"]
+build-backend = "hatchling.build"
+
+[project]
+name = "reconkit"
+version = "0.1.0"
+description = "A small, safe-by-default toolkit for scraping, OSINT, and file triage."
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "reconkit" }]
+keywords = ["osint", "forensics", "scraping", "triage"]
+dependencies = [
+  "httpx>=0.27",
+  "typer>=0.12",
+  "rich>=13.7",
+  "beautifulsoup4>=4.12",
+  "lxml>=5.1",
+  "dnspython>=2.6",
+  "python-whois>=0.9",
+  "ipwhois>=1.2",
+  "exifread>=3.0",
+  "filetype>=1.2",
+]
+
+[project.optional-dependencies]
+dev = [
+  "build>=1.2",
+  "twine>=5.1",
+  "pytest>=8.0",
+  "pytest-asyncio>=0.23",
+  "ruff>=0.4",
+  "mypy>=1.10",
+]
+
+[project.scripts]
+reconkit = "reconkit.cli:app"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/reconkit"]
+
+[tool.pytest.ini_options]
+addopts = "-q"
+asyncio_mode = "auto"
+
+[tool.ruff]
+line-length = 100
+
+[tool.mypy]
+python_version = "3.10"
+warn_return_any = true
+warn_unused_ignores = true
+no_implicit_optional = true
+strict_optional = true
+ignore_missing_imports = true

reconkit-0.1.0/src/reconkit/__init__.py

@@ -0,0 +1,22 @@
+"""reconkit.
+
+Public API: Noir Stack / Hexarch / Judicium surface lives under `reconkit.noir`.
+Internal modules remain available but are not the supported public interface.
+"""
+
+__version__ = "0.1.0"
+
+from reconkit.noir.primitives import (  # noqa: E402
+    EvidenceArtifact,
+    EvidenceBundle,
+    ProvenanceRecord,
+    VerificationResult,
+)
+
+__all__ = [
+    "__version__",
+    "EvidenceArtifact",
+    "EvidenceBundle",
+    "ProvenanceRecord",
+    "VerificationResult",
+]

reconkit-0.1.0/src/reconkit/cli.py

@@ -0,0 +1,218 @@
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Optional
+
+import typer
+from rich.console import Console
+
+from reconkit.noir import public_api
+from reconkit.noir.primitives import VerificationResult, json_dumps_audit
+
+app = typer.Typer(add_completion=False, no_args_is_help=True)
+console = Console()
+
+evidence_app = typer.Typer(no_args_is_help=True)
+boundary_app = typer.Typer(no_args_is_help=True)
+verify_app = typer.Typer(no_args_is_help=True)
+app.add_typer(evidence_app, name="evidence")
+app.add_typer(boundary_app, name="boundary")
+app.add_typer(verify_app, name="verify")
+
+
+def _emit(result: VerificationResult) -> None:
+    # Audit-grade requirement: stdout must be machine-parseable JSON.
+    # Rich is excellent at presentation and equally excellent at line-wrapping.
+    # We only need one of those properties.
+    typer.echo(json_dumps_audit(result.to_json_dict()))
+
+
+@evidence_app.command("acquire-web")
+def evidence_acquire_web(
+    url: str,
+    trust_boundary: str = typer.Option(
+        "public-internet",
+        help="Declared trust boundary for acquisition (audited, not inferred).",
+    ),
+    max_bytes: int = typer.Option(2_000_000, help="Max download size in bytes"),
+    timeout_s: float = typer.Option(20.0, help="Timeout in seconds"),
+    rps: float = typer.Option(1.0, help="Max requests per second"),
+    obey_robots: bool = typer.Option(True, help="Respect robots.txt"),
+    include_payload: bool = typer.Option(False, help="Include payload in output (base64)."),
+):
+    """Acquire a web response as evidence.
+
+    Conservative note: this is acquisition, not authentication. The system is many things.
+    Omniscient is not currently one of them.
+    """
+
+    artifact = public_api.acquire_web_artifact(
+        url,
+        trust_boundary=trust_boundary,
+        max_bytes=max_bytes,
+        timeout_s=timeout_s,
+        rps=rps,
+        obey_robots=obey_robots,
+    )
+    _emit(
+        VerificationResult(
+            verification_status="informational",
+            decision_basis="http-get acquisition within declared trust boundary",
+            provenance_confidence="medium",
+            subject={"url": url, "trust_boundary": trust_boundary},
+            findings={"artifact": artifact.to_json_dict(include_payload=include_payload)},
+            limitations=[
+                "Transport acquisition does not assert publisher identity or content authenticity.",
+            ],
+        )
+    )
+
+
+@evidence_app.command("derive-links")
+def evidence_derive_links(
+    url: str,
+    timeout_s: float = typer.Option(20.0, help="Timeout in seconds"),
+    rps: float = typer.Option(1.0, help="Max requests per second"),
+    obey_robots: bool = typer.Option(True, help="Respect robots.txt"),
+    include_payload: bool = typer.Option(False, help="Include payload in output (base64)."),
+):
+    """Acquire a web response and derive a link-list artifact."""
+
+    web = public_api.acquire_web_artifact(url, timeout_s=timeout_s, rps=rps, obey_robots=obey_robots)
+    derived = public_api.derive_links(web, base_url=url)
+    _emit(
+        VerificationResult(
+            verification_status="informational",
+            decision_basis="derived link extraction from acquired HTML",
+            provenance_confidence="medium",
+            subject={"url": url},
+            findings={
+                "source_artifact": web.to_json_dict(include_payload=False),
+                "derived_artifact": derived.to_json_dict(include_payload=include_payload),
+            },
+            limitations=[
+                "Link extraction is heuristic and may miss dynamically generated URLs.",
+            ],
+        )
+    )
+
+
+@evidence_app.command("derive-text")
+def evidence_derive_text(
+    url: str,
+    timeout_s: float = typer.Option(20.0, help="Timeout in seconds"),
+    rps: float = typer.Option(1.0, help="Max requests per second"),
+    obey_robots: bool = typer.Option(True, help="Respect robots.txt"),
+    include_payload: bool = typer.Option(False, help="Include payload in output (base64)."),
+):
+    """Acquire a web response and derive a readable-text artifact."""
+
+    web = public_api.acquire_web_artifact(url, timeout_s=timeout_s, rps=rps, obey_robots=obey_robots)
+    derived = public_api.derive_readable_text(web, base_url=url)
+    _emit(
+        VerificationResult(
+            verification_status="informational",
+            decision_basis="derived readable text from acquired HTML",
+            provenance_confidence="medium",
+            subject={"url": url},
+            findings={
+                "source_artifact": web.to_json_dict(include_payload=False),
+                "derived_artifact": derived.to_json_dict(include_payload=include_payload),
+            },
+            limitations=[
+                "Readability extraction removes scripts/styles and may omit critical context.",
+            ],
+        )
+    )
+
+
+@boundary_app.command("dns")
+def boundary_dns(domain: str, include_payload: bool = typer.Option(False, help="Include payload in output (base64).")):
+    """Observe DNS answers across the public-internet boundary."""
+
+    artifact = public_api.boundary_dns(domain)
+    _emit(
+        VerificationResult(
+            verification_status="informational",
+            decision_basis="DNS resolution observation",
+            provenance_confidence="medium",
+            subject={"domain": domain, "trust_boundary": "public-internet"},
+            findings={"artifact": artifact.to_json_dict(include_payload=include_payload)},
+            limitations=[
+                "DNS is an administrative naming system; answers are not identity proofs.",
+            ],
+        )
+    )
+
+
+@boundary_app.command("whois")
+def boundary_whois(domain: str, include_payload: bool = typer.Option(False, help="Include payload in output (base64).")):
+    """Observe WHOIS/RDAP administrative metadata across the public-internet boundary."""
+
+    artifact = public_api.boundary_whois(domain)
+    _emit(
+        VerificationResult(
+            verification_status="informational",
+            decision_basis="WHOIS observation",
+            provenance_confidence="low",
+            subject={"domain": domain, "trust_boundary": "public-internet"},
+            findings={"artifact": artifact.to_json_dict(include_payload=include_payload)},
+            limitations=[
+                "WHOIS fields vary by registrar/TLD and may be redacted or stale.",
+            ],
+        )
+    )
+
+
+@boundary_app.command("ip")
+def boundary_ip(ip: str, include_payload: bool = typer.Option(False, help="Include payload in output (base64).")):
+    """Observe RDAP/ASN administrative metadata across the public-internet boundary."""
+
+    artifact = public_api.boundary_ip(ip)
+    _emit(
+        VerificationResult(
+            verification_status="informational",
+            decision_basis="RDAP/ASN observation",
+            provenance_confidence="medium",
+            subject={"ip": ip, "trust_boundary": "public-internet"},
+            findings={"artifact": artifact.to_json_dict(include_payload=include_payload)},
+            limitations=[
+                "Administrative allocations do not imply actor attribution.",
+            ],
+        )
+    )
+
+
+@verify_app.command("file-hash")
+def verify_file_hash(
+    path: Path,
+    algo: str = typer.Option("sha256", help="md5|sha1|sha256"),
+):
+    """Verify (informational): compute a cryptographic digest over file bytes."""
+
+    _emit(public_api.verify_file_hash(path, algo=algo))
+
+
+@verify_app.command("file-type")
+def verify_file_type(path: Path):
+    """Verify (informational): identify file type by signature."""
+
+    _emit(public_api.verify_file_type(path))
+
+
+@verify_app.command("strings")
+def verify_strings(
+    path: Path,
+    min_len: int = typer.Option(6, "--min-len"),
+    max_strings: Optional[int] = typer.Option(None, "--max"),
+):
+    """Verify (informational): extract printable strings as a triage aid."""
+
+    _emit(public_api.verify_strings(path, min_len=min_len, max_strings=max_strings))
+
+
+@verify_app.command("exif")
+def verify_exif(path: Path):
+    """Verify (informational): extract EXIF tags."""
+
+    _emit(public_api.verify_exif(path))

reconkit-0.1.0/src/reconkit/core/rate_limit.py

@@ -0,0 +1,24 @@
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass
+
+
+@dataclass
+class RateLimiter:
+    """Simple, process-local rate limiter.
+
+    This intentionally stays basic (no jitter, no token bucket) to be easy to understand and
+    replicate.
+    """
+
+    requests_per_second: float = 1.0
+    _next_allowed_at: float = 0.0
+
+    def wait(self) -> None:
+        if self.requests_per_second <= 0:
+            return
+        now = time.monotonic()
+        if now < self._next_allowed_at:
+            time.sleep(self._next_allowed_at - now)
+        self._next_allowed_at = time.monotonic() + (1.0 / self.requests_per_second)

reconkit-0.1.0/src/reconkit/forensics/exif.py

@@ -0,0 +1,16 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import exifread
+
+
+def extract_exif(path: Path) -> dict[str, str]:
+    with path.open("rb") as f:
+        tags = exifread.process_file(f, details=False)
+
+    # normalize to plain strings
+    out: dict[str, str] = {}
+    for k, v in tags.items():
+        out[str(k)] = str(v)
+    return out

reconkit-0.1.0/src/reconkit/forensics/hashing.py

@@ -0,0 +1,22 @@
+from __future__ import annotations
+
+import hashlib
+from pathlib import Path
+
+
+_SUPPORTED = {"md5", "sha1", "sha256"}
+
+
+def hash_file(path: Path, *, algo: str = "sha256", chunk_size: int = 1024 * 1024) -> str:
+    algo = algo.lower()
+    if algo not in _SUPPORTED:
+        raise ValueError(f"Unsupported algo: {algo}. Choose one of: {sorted(_SUPPORTED)}")
+
+    h = hashlib.new(algo)
+    with path.open("rb") as f:
+        while True:
+            chunk = f.read(chunk_size)
+            if not chunk:
+                break
+            h.update(chunk)
+    return h.hexdigest()

reconkit-0.1.0/src/reconkit/forensics/identify.py

@@ -0,0 +1,12 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import filetype
+
+
+def identify_file(path: Path) -> dict[str, str | None]:
+    kind = filetype.guess(str(path))
+    if kind is None:
+        return {"path": str(path), "ext": None, "mime": None}
+    return {"path": str(path), "ext": kind.extension, "mime": kind.mime}