clearfront 2.7.0__tar.gz → 2.7.2__tar.gz

{clearfront-2.7.0 → clearfront-2.7.2}/PKG-INFO

@@ -1,13 +1,13 @@
 Metadata-Version: 2.4
 Name: clearfront
-Version: 2.7.0
-Summary: AI-powered OSINT agent, MCP server, and CLI. Interactive REPL + 30 tools. Anthropic Claude or local Ollama. For authorized security research use only.
+Version: 2.7.2
+Summary: Clearfront OSINT: AI-powered OSINT agent, MCP server, and CLI that maps your digital footprint across 3,400+ public data sources. Interactive REPL + 30 tools. Anthropic Claude or local Ollama. For authorized security research use only.
 Author: Scott Martin Anderson
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/scottmartinanderson/clearfront
 Project-URL: Repository, https://github.com/scottmartinanderson/clearfront
 Project-URL: Bug Tracker, https://github.com/scottmartinanderson/clearfront/issues
-Keywords: osint,security,mcp,cli,ai,agent,anthropic,claude,investigation,recon,cybersecurity
+Keywords: clearfront,clearfront-osint,osint,open-source-intelligence,digital-footprint,security,mcp,cli,ai,agent,anthropic,claude,investigation,recon,cybersecurity,privacy
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Information Technology
@@ -69,10 +69,12 @@ Requires-Dist: uvicorn[standard]>=0.51.0; extra == "all"
 Requires-Dist: sse-starlette>=3.4.5; extra == "all"
 Dynamic: license-file
 
+<!-- mcp-name: sh.clearfront/clearfront -->
+
 <div align="center">
   <h1>Clearfront</h1>
   <p><strong>Open-source AI intelligence on your digital footprint.</strong></p>
-  <p>Give Clearfront an email, username, domain, IP, or name, and its AI security analyst scans public data sources in one sweep, then returns a calibrated report and an interactive evidence graph. Check your own exposure, or investigate an authorized target. It runs locally with your own API keys and sends nothing to us.</p>
+  <p>Clearfront, also known as Clearfront OSINT, is a free and open-source OSINT agent. Give it an email, username, domain, IP, or name, and its AI security analyst scans 3,400+ public data sources in one sweep, then returns a calibrated report and an interactive evidence graph. Check your own exposure, or investigate an authorized target. It runs locally with your own API keys and sends nothing to us.</p>
   <p>Interactive REPL · CLI · local web console · MCP server. Powered by Anthropic Claude, a local Ollama model, or any OpenAI-compatible endpoint.</p>
 </div>
 
@@ -81,7 +83,7 @@ Dynamic: license-file
 [![Python](https://img.shields.io/badge/python-3.10%2B-blue?style=flat-square)](https://www.python.org/)
 [![License MIT](https://img.shields.io/badge/license-MIT-green?style=flat-square)](LICENSE)
 [![MCP](https://img.shields.io/badge/protocol-MCP-blueviolet?style=flat-square)](https://modelcontextprotocol.io/)
-[![Version](https://img.shields.io/badge/version-2.7.0-informational?style=flat-square)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-2.7.2-informational?style=flat-square)](CHANGELOG.md)
 
 **[clearfront.sh](https://clearfront.sh)** · [Disclaimer](./DISCLAIMER.md)
 
@@ -226,7 +228,7 @@ clearfront web
 
 Browser-based AI chat with streaming tool output, inline result cards, and a light/dark theme toggle. Supports fully local inference via Ollama or any OpenAI-compatible endpoint (no Anthropic API key required when using a local backend).
 
-The console runs entirely on your machine and binds to `127.0.0.1` by default. Choose your backend and paste your own key in Settings; your keys and the targets you investigate never touch our servers. Screenshots are in [`media/screenshots/`](media/screenshots/).
+The console runs entirely on your machine and binds to `127.0.0.1` by default. Choose your backend and paste your own key in Settings; your keys and the targets you investigate never touch our servers. Screenshots are in [`media/`](media/).
 
 ### MCP Server
 
@@ -252,6 +254,24 @@ claude mcp list
 }
 ```
 
+### Agent Skill
+
+[`skills/clearfront-osint`](skills/clearfront-osint/SKILL.md) teaches an agent
+how to drive Clearfront: which approach fits a given request, the direct
+subcommands, how to read the confidence ratings, and the authorized-use rules.
+It follows the [Agent Skills](https://agentskills.io) open standard, so it works
+in Claude Code, Cursor, Gemini CLI, Copilot, Goose, OpenCode and the other
+skills-compatible clients.
+
+```bash
+npx skills add scottmartinanderson/clearfront
+```
+
+Install Clearfront itself as well (`pip install clearfront`); the skill is the
+instructions, not the tool. Use the skill when you want the agent to decide how
+to run a sweep, and the MCP server above when you want it calling the 30 tools
+directly.
+
 ## Docker
 
 ```bash

{clearfront-2.7.0 → clearfront-2.7.2}/README.md

@@ -1,7 +1,9 @@
+<!-- mcp-name: sh.clearfront/clearfront -->
+
 <div align="center">
   <h1>Clearfront</h1>
   <p><strong>Open-source AI intelligence on your digital footprint.</strong></p>
-  <p>Give Clearfront an email, username, domain, IP, or name, and its AI security analyst scans public data sources in one sweep, then returns a calibrated report and an interactive evidence graph. Check your own exposure, or investigate an authorized target. It runs locally with your own API keys and sends nothing to us.</p>
+  <p>Clearfront, also known as Clearfront OSINT, is a free and open-source OSINT agent. Give it an email, username, domain, IP, or name, and its AI security analyst scans 3,400+ public data sources in one sweep, then returns a calibrated report and an interactive evidence graph. Check your own exposure, or investigate an authorized target. It runs locally with your own API keys and sends nothing to us.</p>
   <p>Interactive REPL · CLI · local web console · MCP server. Powered by Anthropic Claude, a local Ollama model, or any OpenAI-compatible endpoint.</p>
 </div>
 
@@ -10,7 +12,7 @@
 [![Python](https://img.shields.io/badge/python-3.10%2B-blue?style=flat-square)](https://www.python.org/)
 [![License MIT](https://img.shields.io/badge/license-MIT-green?style=flat-square)](LICENSE)
 [![MCP](https://img.shields.io/badge/protocol-MCP-blueviolet?style=flat-square)](https://modelcontextprotocol.io/)
-[![Version](https://img.shields.io/badge/version-2.7.0-informational?style=flat-square)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-2.7.2-informational?style=flat-square)](CHANGELOG.md)
 
 **[clearfront.sh](https://clearfront.sh)** · [Disclaimer](./DISCLAIMER.md)
 
@@ -155,7 +157,7 @@ clearfront web
 
 Browser-based AI chat with streaming tool output, inline result cards, and a light/dark theme toggle. Supports fully local inference via Ollama or any OpenAI-compatible endpoint (no Anthropic API key required when using a local backend).
 
-The console runs entirely on your machine and binds to `127.0.0.1` by default. Choose your backend and paste your own key in Settings; your keys and the targets you investigate never touch our servers. Screenshots are in [`media/screenshots/`](media/screenshots/).
+The console runs entirely on your machine and binds to `127.0.0.1` by default. Choose your backend and paste your own key in Settings; your keys and the targets you investigate never touch our servers. Screenshots are in [`media/`](media/).
 
 ### MCP Server
 
@@ -181,6 +183,24 @@ claude mcp list
 }
 ```
 
+### Agent Skill
+
+[`skills/clearfront-osint`](skills/clearfront-osint/SKILL.md) teaches an agent
+how to drive Clearfront: which approach fits a given request, the direct
+subcommands, how to read the confidence ratings, and the authorized-use rules.
+It follows the [Agent Skills](https://agentskills.io) open standard, so it works
+in Claude Code, Cursor, Gemini CLI, Copilot, Goose, OpenCode and the other
+skills-compatible clients.
+
+```bash
+npx skills add scottmartinanderson/clearfront
+```
+
+Install Clearfront itself as well (`pip install clearfront`); the skill is the
+instructions, not the tool. Use the skill when you want the agent to decide how
+to run a sweep, and the MCP server above when you want it calling the 30 tools
+directly.
+
 ## Docker
 
 ```bash

clearfront-2.7.2/clearfront/boards.py

@@ -0,0 +1,250 @@
+# clearfront/boards.py
+"""
+Persistent investigation boards for Clearfront.
+
+A *board* is a saved evidence-graph investigation: a title, an optional
+description and colour tag, and the D3 node-link graph the web console renders.
+Boards are stored as individual JSON files in ``~/.clearfront/boards/``, one
+file per board keyed by a filesystem-safe slug id.
+
+This is the local-first store, holding only what the analyst already sees in
+their own browser; nothing is uploaded anywhere. A future cloud-hosted store can
+implement the same board shape behind the same web routes.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+logger = logging.getLogger(__name__)
+
+BOARDS_DIR = Path.home() / ".clearfront" / "boards"
+MAX_BOARDS = 200
+
+# Colour tags the UI offers; a board's colour is validated to this set so a
+# malformed value can never reach the stored file or the client.
+BOARD_COLORS = (
+    "blue", "purple", "pink", "red", "amber",
+    "green", "cyan", "indigo", "teal", "grey",
+)
+DEFAULT_COLOR = "blue"
+
+# A board id is only ever a lowercase slug. This regex also blocks path
+# traversal (no dots, no slashes) when an id from a URL is turned into a path.
+_SAFE_ID_RE = re.compile(r"^[a-z0-9][a-z0-9-]{0,63}$")
+_SLUG_STRIP_RE = re.compile(r"[^a-z0-9]+")
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _now_iso() -> str:
+    return datetime.now(timezone.utc).isoformat(timespec="seconds")
+
+
+def _ensure_dir() -> None:
+    BOARDS_DIR.mkdir(parents=True, exist_ok=True)
+
+
+def _slugify(title: str) -> str:
+    """Turn a human title into a short filesystem-safe slug stem (no id suffix)."""
+    return _SLUG_STRIP_RE.sub("-", (title or "").lower()).strip("-")[:40].strip("-")
+
+
+def _new_id(title: str) -> str:
+    """Generate a unique board id: title slug + short random suffix."""
+    stem = _slugify(title) or "board"
+    return f"{stem}-{uuid.uuid4().hex[:6]}"
+
+
+def _board_path(board_id: str) -> Path | None:
+    """Return the file path for a validated id, or None if the id is unsafe."""
+    if not board_id or not _SAFE_ID_RE.match(board_id):
+        return None
+    return BOARDS_DIR / f"{board_id}.json"
+
+
+def _clean_graph(graph: Any) -> dict[str, list]:
+    """Keep only the node/link fields we persist; drop force-layout state.
+
+    Link endpoints may be node objects after the client's force layout runs, so
+    they are normalised back to plain ids. ``label``/``kind`` are preserved when
+    present (forward compatibility with typed relationships).
+    """
+    if not isinstance(graph, dict):
+        return {"nodes": [], "links": []}
+
+    nodes: list[dict] = []
+    for n in graph.get("nodes") or []:
+        if not isinstance(n, dict) or n.get("id") is None:
+            continue
+        nodes.append({
+            "id": n.get("id"),
+            "label": n.get("label", ""),
+            "type": n.get("type", ""),
+            "value": n.get("value", ""),
+            "source": n.get("source", ""),
+            "confidence": n.get("confidence", ""),
+            "severity": n.get("severity", ""),
+        })
+
+    links: list[dict] = []
+    for l in graph.get("links") or []:
+        if not isinstance(l, dict):
+            continue
+        src, tgt = l.get("source"), l.get("target")
+        if isinstance(src, dict):
+            src = src.get("id")
+        if isinstance(tgt, dict):
+            tgt = tgt.get("id")
+        if src is None or tgt is None:
+            continue
+        link: dict[str, Any] = {"source": src, "target": tgt}
+        if l.get("label"):
+            link["label"] = l["label"]
+        if l.get("kind"):
+            link["kind"] = l["kind"]
+        links.append(link)
+
+    return {"nodes": nodes, "links": links}
+
+
+def _summary(board: dict[str, Any]) -> dict[str, Any]:
+    """A light board descriptor for the list view (no full graph payload)."""
+    graph = board.get("graph") or {}
+    return {
+        "id": board.get("id", ""),
+        "title": board.get("title", ""),
+        "description": board.get("description", ""),
+        "color": board.get("color", DEFAULT_COLOR),
+        "created_at": board.get("created_at", ""),
+        "updated_at": board.get("updated_at", ""),
+        "node_count": len(graph.get("nodes") or []),
+        "link_count": len(graph.get("links") or []),
+    }
+
+
+# ---------------------------------------------------------------------------
+# Public API
+# ---------------------------------------------------------------------------
+
+
+def list_boards() -> list[dict[str, Any]]:
+    """Return board summaries, newest-updated first (no graph payloads)."""
+    if not BOARDS_DIR.exists():
+        return []
+    boards: list[dict[str, Any]] = []
+    for f in BOARDS_DIR.glob("*.json"):
+        try:
+            boards.append(_summary(json.loads(f.read_text(encoding="utf-8"))))
+        except Exception:
+            logger.debug("Failed to read board file: %s", f, exc_info=True)
+    boards.sort(key=lambda b: b.get("updated_at", ""), reverse=True)
+    return boards
+
+
+def load_board(board_id: str) -> dict[str, Any] | None:
+    """Return the full board (including graph), or None if missing/invalid id."""
+    path = _board_path(board_id)
+    if path is None or not path.exists():
+        return None
+    try:
+        return json.loads(path.read_text(encoding="utf-8"))
+    except Exception:
+        logger.debug("Failed to read board file: %s", path, exc_info=True)
+        return None
+
+
+def save_board(
+    *,
+    board_id: str = "",
+    title: str = "",
+    description: str = "",
+    color: str = DEFAULT_COLOR,
+    graph: Any = None,
+    notes: str = "",
+) -> dict[str, Any] | None:
+    """Create or update a board. Returns the saved board, or None on bad input.
+
+    A blank/unknown ``board_id`` creates a new board; a valid existing id updates
+    it in place, preserving the original ``created_at``. Requires a non-empty
+    ``title``.
+    """
+    title = (title or "").strip()
+    if not title:
+        return None
+    color = color if color in BOARD_COLORS else DEFAULT_COLOR
+
+    existing = load_board(board_id) if board_id else None
+    now = _now_iso()
+    if existing is not None:
+        bid = existing["id"]
+        created = existing.get("created_at", now)
+    else:
+        bid = _new_id(title)
+        p = _board_path(bid)
+        while p is not None and p.exists():   # astronomically unlikely, but cheap
+            bid = _new_id(title)
+            p = _board_path(bid)
+        created = now
+
+    path = _board_path(bid)
+    if path is None:
+        return None
+
+    board = {
+        "id": bid,
+        "title": title[:200],
+        "description": (description or "").strip()[:2000],
+        "color": color,
+        "created_at": created,
+        "updated_at": now,
+        "notes": (notes or "")[:20000],
+        "graph": _clean_graph(graph),
+    }
+
+    _ensure_dir()
+    path.write_text(json.dumps(board, indent=2), encoding="utf-8")
+    _prune()
+    return board
+
+
+def delete_board(board_id: str) -> bool:
+    """Delete a board by id. Returns True if a file was removed."""
+    path = _board_path(board_id)
+    if path is None or not path.exists():
+        return False
+    try:
+        path.unlink()
+        return True
+    except Exception:
+        logger.debug("Failed to delete board file: %s", path, exc_info=True)
+        return False
+
+
+def _prune() -> None:
+    """Keep at most MAX_BOARDS files, deleting the oldest-updated first."""
+    files = list(BOARDS_DIR.glob("*.json"))
+    if len(files) <= MAX_BOARDS:
+        return
+
+    def _updated(f: Path) -> str:
+        try:
+            return json.loads(f.read_text(encoding="utf-8")).get("updated_at", "")
+        except Exception:
+            return ""
+
+    files.sort(key=_updated)   # oldest first
+    for f in files[: len(files) - MAX_BOARDS]:
+        try:
+            f.unlink()
+        except Exception:
+            logger.debug("Failed to prune board file: %s", f, exc_info=True)

{clearfront-2.7.0 → clearfront-2.7.2}/clearfront/mcp_server.py

@@ -1,9 +1,12 @@
 # clearfront/mcp_server.py
 """
-Clearfront MCP Server, v2.7.0
+Clearfront OSINT MCP Server, v2.7.1
 
-Exposes all 30 OSINT tool capabilities plus multi-target investigation
-to MCP-compliant AI clients over standard I/O. Tools include:
+Clearfront (Clearfront OSINT) is an open-source AI OSINT agent that maps a
+digital footprint across 3,400+ public data sources and returns a calibrated
+intelligence report. This server exposes all 30 OSINT tool capabilities plus
+multi-target investigation to MCP-compliant AI clients over standard I/O.
+Tools include:
 search_email, search_username, search_maigret, search_breach, search_whois,
 search_ip, search_domain, generate_dorks, search_paste, search_phone,
 search_shodan, search_virustotal, search_censys, search_ip2location,