devguard-scanner 0.3.0__tar.gz → 0.4.1__tar.gz

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/.gitignore

@@ -18,3 +18,8 @@ htmlcov/
 dist/
 build/
 *.egg-info/
+
+# Personal notes and local deploy state (secrets)
+.local/
+.vercel/
+.env*

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/CHANGELOG.md

@@ -3,6 +3,54 @@
 All notable changes are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/) and
 the project uses [semantic versioning](https://semver.org).
 
+## 0.4.1
+
+- **`devguard check`**: is this repository flagged as malicious by the community? Reads the GitHub remote and
+  HEAD of a checkout from `.git` as data (git is never run; credentials in remote URLs are dropped), or takes a
+  URL / `OWNER/REPO`. Exit code `1` when flagged.
+- **`devguard flag`**: report a repository. The server scans that commit itself and records the flag only for a
+  `block` verdict, with the evidence and the number of distinct reporters, so clean repositories cannot be
+  flagged. API: `POST /v1/flags`, `GET /v1/flags/github/{owner}/{repo}`.
+- **Shared scan results (opt-in, asked once)**: after `devguard scan` of a GitHub checkout, the verdict, counts,
+  and finding titles (never code) go to `POST /v1/usage` and are kept for public repositories only.
+  `--share` / `--no-share`, `DEVGUARD_SHARE`, `[community] share`. A flagged repository is reported right after the
+  scan.
+- Every request record has `from: "api"` or `"cli"`; the dashboard and `/v1/stats` show API usage only.
+- New rules for info-stealers that search the computer rather than known credential paths (ruleset 2026.10.1):
+  `DG-EXFIL-FILE-SWEEP-001` (critical: the home directory or disk is searched for keys, `.env` files, wallets, or
+  documents, and they are uploaded with `fetch`, `axios`, `FormData`, `requests.post(files=…)`, `curl -F`, …),
+  `DG-SECRET-SWEEP-001` (high: the same search without a visible upload, including `find ~ -name "*.env"`,
+  `grep -r "PRIVATE KEY" ~`, `ls -R /home`, `Get-ChildItem -Recurse`, and `dir /s`), and `DG-SECRET-HOME-ENV-001`
+  (high: a `.env` file read from the home directory). Before, such code only produced weak indicators.
+- Content rules accept `within: "chars:N"`: every part within N characters, which also holds in minified bundles.
+- **Dashboard** at `/` of the API (and <https://devguard-scanner.vercel.app>): scan a repository from the browser,
+  usage over 24 h / 7 / 30 / 90 days (requests, scans, passed, failed, errors, repositories, verdicts, scans per
+  day), and the recently flagged repositories with a detail page each. Plain HTML/CSS/JS served by the API, with a
+  strict CSP; it needs the MongoDB request log.
+- `GET /v1/stats` and `GET /v1/stats/repos/{owner}/{repo}`: the data behind the dashboard (aggregates only; client
+  IPs are never returned). `/v1/stats` is cached for 60 s and not itself logged.
+- New `scans` collection next to `api_requests`: one document per repository and commit (latest verdict, counts,
+  the 20 most severe findings without excerpts, first/last seen, scan count). It never expires.
+- **AI triage of flagged scans** with the server's own key `DEVGUARD_OPENAI_KEY`: one strict OpenAI function call
+  per flagged repository and commit (`result` malware/suspicious/clean/unsure, `risk_level`, `reason`,
+  `reason_detail`, techniques), plus the GitHub owner and the scanned commit's author, date, and message. Runs in
+  the background, capped by `DEVGUARD_API_TRIAGE_MAX_PER_DAY` (200). Callers' own AI keys are unchanged.
+- Leaner request log: the JSON report is stored only for `block`/`review` verdicts
+  (`DEVGUARD_API_MONGODB_STORE_REPORT=flagged|all|none`; `true`/`false` still work), and the `report_md_head`
+  field is gone (it duplicated the summary and rendered Markdown for every request).
+- `sandbox/`: one Dockerfile on top of the published image, portable paths, reports and config mounted where the
+  CLI actually writes and reads them, and the proxy variant no longer resolves names directly.
+
+## 0.4.0
+
+- `devguard config init` writes the user config with an AI profile: numbered choices on a terminal (Enter takes
+  the default), or the defaults and flags with `-y` or without a terminal (`--provider anthropic|openai|local`,
+  `--model`, `--key-env`, `--base-url`, `--path`, `--force`). `devguard config path` prints the file location.
+- With a single AI profile, `--ai` uses it without `default_profile`. Without any profile, the error points to
+  `devguard config init`.
+- Container images are published on every push to `main` (`:main`, `:sha-<commit>`), and the hosted API on Vercel
+  is deployed from CI with that exact image digest.
+
 ## 0.3.0
 
 - The default drop folder for local scans (`DEVGUARD_API_LOCAL_ROOT`, and the folder `compose.yaml` mounts) is

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: devguard-scanner
-Version: 0.3.0
+Version: 0.4.1
 Summary: Scan untrusted code repositories for malware before you open, install, or run them. Files are analyzed as data and never executed.
 Project-URL: Homepage, https://github.com/yilmazhasan/devguard-scanner
 Project-URL: Documentation, https://github.com/yilmazhasan/devguard-scanner/tree/main/docs
@@ -77,6 +77,16 @@ Report (md): ~/devguard-scanner/reports/interview-task-1a2b3c4d/latest/report.md
 
 Exit code `1` means findings at or above `--fail-on` (default `high`), so it works as a CI gate.
 
+## Check whether others flagged it
+
+```bash
+devguard check                    # this checkout, by its GitHub remote; or: devguard check owner/repo
+devguard flag --note "..."        # report it; the server re-scans and accepts only a `block` verdict
+```
+
+Sharing your scan verdicts for public repositories is opt-in (asked once). See
+[community features](docs/getting-started.md#community-check-flag-and-share).
+
 ## Scan a GitHub repository without cloning it
 
 ```bash

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/README.md

@@ -39,6 +39,16 @@ Report (md): ~/devguard-scanner/reports/interview-task-1a2b3c4d/latest/report.md
 
 Exit code `1` means findings at or above `--fail-on` (default `high`), so it works as a CI gate.
 
+## Check whether others flagged it
+
+```bash
+devguard check                    # this checkout, by its GitHub remote; or: devguard check owner/repo
+devguard flag --note "..."        # report it; the server re-scans and accepts only a `block` verdict
+```
+
+Sharing your scan verdicts for public repositories is opt-in (asked once). See
+[community features](docs/getting-started.md#community-check-flag-and-share).
+
 ## Scan a GitHub repository without cloning it
 
 ```bash

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/docs/README.md

@@ -14,4 +14,5 @@
 | [Rules](rules.md) | Rule catalogue, writing a detection rule |
 | [Threat reference](threats.md) | The `TC-xx` threat identifiers that rules reference |
 | [Dependencies](dependencies.md) | Why each runtime dependency exists |
+| [Developer guide](development.md) | A tour of the codebase: what is where, how a scan flows, where to start, how to debug |
 | [Releasing](releasing.md) | Publishing to PyPI and GHCR, deploying the hosted API (maintainers) |

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/pyproject.toml

@@ -55,6 +55,7 @@ dev = [
     "mypy>=1.11",
     "types-PyYAML",
     "httpx>=0.27",  # FastAPI TestClient
+    "mongomock>=4.3",
 ]
 
 [build-system]

devguard_scanner-0.4.1/sandbox/README.md

@@ -0,0 +1,34 @@
+# CLI sandbox
+
+Run the `devguard` CLI in a locked-down container: read-only root filesystem, no capabilities, your repositories
+mounted read-only, and only the reports directory writable. Both variants build `sandbox/Dockerfile`, which is the
+published image plus `vi`, `less`, `file`, and `tree` for looking at a suspicious repository from the inside.
+
+| Variant | Network |
+|---|---|
+| `docker-with-network/` | full internet (GitHub URL scans, any AI provider) |
+| `docker-with-proxy/` | none, except HTTPS to `api.openai.com` / `api.anthropic.com` through an allowlist proxy (Docker Engine >= 26) |
+
+```bash
+mkdir -p ~/devguard-scanner-data ~/devguard-scanner-reports ~/.config/devguard-scanner
+cp -r ~/Downloads/interview-task ~/devguard-scanner-data/     # what you want to scan
+cd sandbox/docker-with-proxy                                  # or docker-with-network
+export OPENAI_API_KEY=sk-...                                  # optional, for --ai
+
+docker compose run --rm devguard-scanner scan /data/interview-task
+docker compose run --rm --entrypoint bash devguard-scanner    # a shell, to inspect /data by hand
+```
+
+Reports land in `~/devguard-scanner-reports/<repo>-<hash>/latest/` on the host. Your
+`~/.config/devguard-scanner/config.toml` (from `devguard config init`) is mounted read-only.
+
+| Variable | Default |
+|---|---|
+| `DEVGUARD_DROP_FOLDER` | `~/devguard-scanner-data` (mounted read-only at `/data`) |
+| `DEVGUARD_REPORTS` | `~/devguard-scanner-reports` |
+| `DEVGUARD_CONFIG` | `~/.config/devguard-scanner` |
+| `DEVGUARD_UID` / `DEVGUARD_GID` | `1000` (set to `$(id -u)` / `$(id -g)` if yours differ) |
+
+A container limits what a malicious repository can reach, but DevGuard never executes the code it scans, so the
+sandbox mainly protects you from the scanner's own dependencies and from your own curiosity (`bash`, `npm install`).
+For genuinely hostile code, add a VM. To scan without installing anything, use the [hosted API](../docs/api.md).

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/src/devguard/__init__.py

@@ -1,5 +1,5 @@
 """DevGuard: pre-open security scanner for untrusted code workspaces."""
 
-__version__ = "0.3.0"
-RULESET_VERSION = "2026.10.0"
+__version__ = "0.4.1"
+RULESET_VERSION = "2026.10.1"
 SCHEMA_VERSION = "1"

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/src/devguard/ai/registry.py

@@ -101,8 +101,15 @@ def load_provider(
 ) -> LoadedProvider:
     """Load a profile's provider. `secrets` supplies already-resolved values (an API caller's own key)."""
     name = profile_name or cfg.default_profile
+    if not name and len(cfg.profiles) == 1:
+        name = next(iter(cfg.profiles))
     if not name:
-        raise ProviderError("no AI profile selected: pass --ai-profile or set ai.default_profile in your user config")
+        if not cfg.profiles:
+            raise ProviderError("no AI profile configured: run `devguard config init` to create one")
+        raise ProviderError(
+            f"several AI profiles ({', '.join(sorted(cfg.profiles))}) and no default: pass --ai-profile NAME "
+            "or set ai.default_profile in your config"
+        )
     profile = cfg.profiles.get(name)
     if profile is None:
         raise ProviderError(f"unknown AI profile {name!r}; configured: {', '.join(sorted(cfg.profiles)) or 'none'}")

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/src/devguard/analyzers/content.py

@@ -5,7 +5,8 @@ Rule match expressions:
     {all: [expr, ...]}      every child must match (optionally within N lines, see `within`)
     {any: [expr, ...]}      at least one child matches
     {not: expr}             child must not match
-`within: file` (default) or `within: "lines:N"` (top-level `all` only).
+`within: file` (default), `within: "lines:N"`, or `within: "chars:N"` (top-level `all` only). `chars` measures
+distance in characters, so it also holds in minified files, where a whole bundle sits on a few lines.
 """
 
 from __future__ import annotations
@@ -61,8 +62,8 @@ class YamlRule(BaseModel):
     @field_validator("within")
     @classmethod
     def _within_format(cls, v: str) -> str:
-        if v != "file" and not re.fullmatch(r"lines:\d+", v):
-            raise ValueError("within must be 'file' or 'lines:N'")
+        if v != "file" and not re.fullmatch(r"(lines|chars):\d+", v):
+            raise ValueError("within must be 'file', 'lines:N', or 'chars:N'")
         return v
 
 
@@ -120,6 +121,7 @@ class CompiledRule:
     spec: YamlRule
     root: _Node
     window: int | None
+    window_chars: bool = False
 
     def applies(self, rel: str, lang: str | None) -> bool:
         if self.spec.languages and "any" not in self.spec.languages and lang not in self.spec.languages:
@@ -141,7 +143,8 @@ class CompiledRule:
                 per_child.append(res)
         if not per_child:
             return []
-        child_lines = [[(line_of(s[0]), s) for s in spans] for spans in per_child]
+        position = (lambda off: off) if self.window_chars else line_of
+        child_lines = [[(position(s[0]), s) for s in spans] for spans in per_child]
         for anchor_line, anchor_span in child_lines[0]:
             picked = [anchor_span]
             for others in child_lines[1:]:
@@ -174,8 +177,8 @@ class ContentRuleEngine:
         compiled: list[CompiledRule] = []
         for pack in packs:
             for r in pack.rules:
-                window = int(r.within.split(":")[1]) if r.within.startswith("lines:") else None
-                compiled.append(CompiledRule(r, _compile(r.match), window))
+                window = int(r.within.split(":")[1]) if r.within != "file" else None
+                compiled.append(CompiledRule(r, _compile(r.match), window, r.within.startswith("chars:")))
                 register(
                     RuleMeta(
                         id=r.id,

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/src/devguard/api/app.py

@@ -17,8 +17,10 @@ from pathlib import Path
 from typing import Annotated, Any, Literal
 
 from fastapi import FastAPI, Header, Query, Request
+from fastapi.encoders import jsonable_encoder
 from fastapi.exceptions import RequestValidationError
-from fastapi.responses import JSONResponse, RedirectResponse, Response
+from fastapi.responses import FileResponse, JSONResponse, RedirectResponse, Response
+from fastapi.staticfiles import StaticFiles
 from starlette.exceptions import HTTPException as StarletteHTTPException
 
 from devguard import RULESET_VERSION, SCHEMA_VERSION, __version__
@@ -30,6 +32,7 @@ from devguard.api.schemas import (
     AIOptions,
     ErrorResponse,
     FailOn,
+    FlagRequestBody,
     Gate,
     LocalListing,
     LocalScanRequestBody,
@@ -39,17 +42,30 @@ from devguard.api.schemas import (
     ScanRequestBody,
     ScanResponse,
     Summary,
+    UsageRequestBody,
     VersionInfo,
 )
 from devguard.api.settings import APISettings, load_settings
-from devguard.api.traffic import MongoSink, NullSink, TrafficSink, client_ip, markdown_head
+from devguard.api.traffic import (
+    FLAGGED,
+    MongoSink,
+    NullSink,
+    TrafficSink,
+    cli_scan_upsert,
+    client_ip,
+    flag_upsert,
+    reporter_key,
+    top_findings,
+)
+from devguard.api.triage import Triager, openai_complete, triage_job
 from devguard.api.worker import AI_PROVIDERS
 from devguard.core.config import parse_size
 from devguard.core.models import ScanIssue, ScanResult, Severity
 from devguard.core.scoring import VERDICT_TEXT
 from devguard.core.service import registered_rules
-from devguard.remote.errors import RemoteError
+from devguard.remote.errors import NotFound, RemoteError
 from devguard.remote.fetch import MIB
+from devguard.remote.github import GitHubClient
 from devguard.remote.url import RepoSpec, is_full_sha, parse_repo
 from devguard.report.writer import render
 from devguard.rules.catalog import CATALOG, rule_details, rule_summary
@@ -59,6 +75,13 @@ log = logging.getLogger("devguard.api")
 Format = Literal["json", "md", "sarif"]
 MEDIA_TYPES = {"json": "application/json", "md": "text/markdown; charset=utf-8", "sarif": "application/sarif+json"}
 _DOCS_PATHS = ("/docs", "/redoc", "/openapi.json")
+CLIENT_HEADER = "X-DevGuard-Client"  # "cli/<version>" from the devguard CLI (devguard.community)
+_UNLOGGED = ("/v1/stats",)  # the dashboard polls it; logging it would only measure the dashboard
+STATIC = Path(__file__).parent / "static"
+_PAGE_CSP = (
+    "default-src 'none'; script-src 'self'; style-src 'self'; connect-src 'self'; img-src 'self' data:; "
+    "base-uri 'none'; form-action 'none'; frame-ancestors 'none'"
+)
 
 DESCRIPTION = """
 Scan a **public GitHub repository** for malware **before** you open it in your IDE, install its dependencies,
@@ -135,6 +158,8 @@ def create_app(
     settings: APISettings | None = None,
     executor: Executor | None = None,
     traffic: TrafficSink | None = None,
+    triager: Triager | None = None,
+    github: Callable[[], GitHubClient] | None = None,
 ) -> FastAPI:
     settings = settings or load_settings()
     if executor is None:
@@ -154,8 +179,21 @@ def create_app(
         )
     sink: TrafficSink = traffic
     recording = not isinstance(sink, NullSink)
+    if github is None:
+        token = settings.github_token.get_secret_value() if settings.github_token else None
+        github = lambda: GitHubClient(token, timeout=10.0)  # noqa: E731 (metadata only: triage, public checks)
+    if triager is None and isinstance(sink, MongoSink) and settings.openai_key and settings.triage_max_per_day > 0:
+        triager = Triager(
+            sink.database,
+            openai_complete(settings.openai_key.get_secret_value(), settings.triage_model, settings.triage_timeout_s),
+            github,
+            settings.triage_model,
+            settings.triage_max_per_day,
+        )
     gate = ConcurrencyGate(settings.concurrency)
     limiter = RateLimiter(settings.rate_limit_per_min)
+    usage_limiter = RateLimiter(settings.usage_rate_limit_per_min)
+    public_repos: dict[str, tuple[float, bool]] = {}  # owner/repo → (checked at, public)
     registered_rules()  # load analyzers and rules once, before the first request
 
     app = FastAPI(
@@ -168,11 +206,16 @@ def create_app(
                 "name": "local",
                 "description": "Self-hosted only: scan a directory you copied into the server's drop folder.",
             },
+            {
+                "name": "community",
+                "description": "Flags of malicious repositories (verified by a server scan), checks, and scan "
+                "results shared by the CLI (`devguard flag`, `devguard check`).",
+            },
             {"name": "rules", "description": "The rule catalogue (`devguard rules` / `devguard explain`)."},
             {"name": "meta", "description": "Version, limits, health."},
         ],
         license_info={"name": "Apache-2.0", "identifier": "Apache-2.0"},
-        lifespan=_lifespan(sink),
+        lifespan=_lifespan(sink, triager),
     )
     app.state.settings = settings
 
@@ -183,10 +226,15 @@ def create_app(
                 return forwarded[-1]  # the entry added by the proxy closest to us
         return request.client.host if request.client else "unknown"
 
+    def origin(request: Request) -> str:
+        return "cli" if request.headers.get(CLIENT_HEADER, "").startswith("cli/") else "api"
+
     def record_traffic(request: Request, status: int, started: float, request_id: str) -> None:
         doc: dict[str, Any] = {
             "ts": datetime.now(UTC),
             "request_id": request_id,
+            "from": origin(request),
+            "client_version": request.headers.get(CLIENT_HEADER, "")[:40] or None,
             "method": request.method,
             "path": request.url.path[:512],
             "query": {k: v[:255] for k, v in request.query_params.items() if k in ("ref", "format", "fail_on")},
@@ -227,12 +275,14 @@ def create_app(
                 response = await call_next(request)
             status = response.status_code
         finally:
-            if recording and request.url.path.startswith("/v1/"):
+            if recording and request.url.path.startswith("/v1/") and not request.url.path.startswith(_UNLOGGED):
                 record_traffic(request, status, started, request_id)
         response.headers["X-Request-Id"] = request_id
         response.headers["X-Content-Type-Options"] = "nosniff"
         response.headers["Referrer-Policy"] = "no-referrer"
-        if not request.url.path.startswith(_DOCS_PATHS):
+        if request.url.path == "/" or request.url.path.startswith("/static/"):
+            response.headers["Content-Security-Policy"] = _PAGE_CSP
+        elif not request.url.path.startswith(_DOCS_PATHS):
             response.headers["Content-Security-Policy"] = "default-src 'none'; frame-ancestors 'none'"
         return response
 
@@ -350,14 +400,23 @@ def create_app(
         fitted, truncated = _fit(result, fmt, summary, settings.max_response_bytes)
         body = _body(fitted, fmt, summary, truncated)
         if recording:
+            store = settings.mongodb_store_report
             _note(
                 request,
                 format=fmt,
                 summary=summary.model_dump(mode="json"),
                 truncated=truncated,
-                report_md_head=markdown_head(body if fmt == "md" else render(fitted, "md")),
-                report=fitted.model_dump(mode="json") if settings.mongodb_store_report else None,
+                top_findings=top_findings(result.findings),
+                report=fitted.model_dump(mode="json")
+                if store == "all" or (store == "flagged" and result.verdict in FLAGGED)
+                else None,
             )
+        if triager is not None and result.source is not None and result.verdict in FLAGGED and origin(request) != "cli":
+            client = {
+                "ip": client_ip(client_key(request), settings.traffic_ip),
+                "country": request.headers.get("x-vercel-ip-country"),
+            }
+            triager.submit(triage_job(result, summary.model_dump(mode="json"), client))
         headers = {
             "Cache-Control": cache_control,
             "Vary": "Accept",
@@ -484,23 +543,200 @@ def create_app(
             limits=settings.public_limits(),
         )
 
+    # ------------------------------------------------------------------ community: flags, checks, shared CLI scans
+    def community_db() -> MongoSink:
+        if not isinstance(sink, MongoSink):
+            raise APIError(503, "community_disabled", "flags and checks need the request log (MongoDB) on this server")
+        return sink
+
+    def is_public(spec: RepoSpec) -> bool | None:
+        """Whether GitHub serves the repository without credentials (cached for an hour). None if unknown."""
+        key = spec.full_name.lower()
+        cached = public_repos.get(key)
+        if cached and time.monotonic() - cached[0] < 3600:
+            return cached[1]
+        try:
+            public = github().repo(spec.owner, spec.repo).get("private") is False
+        except NotFound:
+            public = False
+        except RemoteError:
+            return None
+        if len(public_repos) > 5000:
+            public_repos.clear()
+        public_repos[key] = (time.monotonic(), public)
+        return public
+
+    def check_document(db: MongoSink, spec: RepoSpec, commit: str | None) -> dict[str, Any]:
+        flags = db.flags_for(spec.owner, spec.repo)
+        return {
+            "repo": spec.full_name,
+            "flagged": bool(flags),
+            "commit_flagged": any(f["commit"] == commit for f in flags) if commit else None,
+            "flags": flags,
+            "latest_scan": db.repository(spec.owner, spec.repo),
+        }
+
+    @app.post(
+        "/v1/flags",
+        tags=["community"],
+        summary="Flag a repository as malicious (the server scans it first)",
+        responses={**_ERROR_RESPONSES},
+    )
+    def create_flag(body: FlagRequestBody, request: Request) -> JSONResponse:
+        db = community_db()
+        spec = _parse(body.repo, body.ref)
+        result = run_scan(request, spec, ScanOptions())
+        summary = _summary(result, "high").model_dump(mode="json")
+        _note(request, summary=summary, top_findings=top_findings(result.findings))
+        if result.verdict != "block" or result.source is None:
+            _note(request, flag={"accepted": False})
+            return JSONResponse(
+                {
+                    "accepted": False,
+                    "verdict": result.verdict,
+                    "message": "The server's own scan of this commit did not find strong indicators of malware "
+                    f"(verdict: {result.verdict}), so no flag was recorded. Only `block` verdicts can be flagged; "
+                    "this keeps clean repositories from being flagged by anyone.",
+                },
+                headers={"Cache-Control": "no-store"},
+            )
+        ip = client_key(request)
+        secret = settings.mongodb_uri.get_secret_value() if settings.mongodb_uri else "devguard"
+        report = {
+            "at": datetime.now(UTC),
+            "from": origin(request),
+            "ip": client_ip(ip, settings.traffic_ip),
+            "country": request.headers.get("x-vercel-ip-country"),
+            "note": body.note,
+        }
+        summary.update(devguard_version=__version__, ruleset_version=RULESET_VERSION)
+        try:
+            flag = db.flag(*flag_upsert(result, summary, reporter_key(ip, secret), report))
+        except Exception as exc:
+            log.warning("flag not stored: %s", type(exc).__name__)
+            raise APIError(
+                503, "community_unavailable", "the flag could not be stored; try again shortly", 30
+            ) from None
+        _note(request, flag={"accepted": True, "reporters": flag["reporters"]})
+        return JSONResponse(
+            jsonable_encoder({"accepted": True, "verdict": result.verdict, "flag": flag}),
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @app.get(
+        "/v1/flags/github/{owner}/{repo}",
+        tags=["community"],
+        summary="Check whether a repository was flagged as malicious",
+        responses={503: {"model": ErrorResponse}},
+    )
+    def check_flags(
+        owner: str,
+        repo: str,
+        commit: Annotated[str | None, Query(pattern=r"^[0-9a-fA-F]{40}$", description="Your HEAD commit.")] = None,
+    ) -> JSONResponse:
+        db = community_db()
+        spec = _parse(f"{owner}/{repo}", None)
+        try:
+            doc = check_document(db, spec, commit.lower() if commit else None)
+        except Exception as exc:
+            log.warning("check unavailable: %s", type(exc).__name__)
+            raise APIError(503, "community_unavailable", "checks are temporarily unavailable", 30) from None
+        return JSONResponse(jsonable_encoder(doc), headers={"Cache-Control": "public, max-age=30, s-maxage=60"})
+
+    @app.post(
+        "/v1/usage",
+        tags=["community"],
+        summary="Share a CLI scan result (usage statistics; public repositories only)",
+        responses={429: {"model": ErrorResponse}, 503: {"model": ErrorResponse}},
+    )
+    def share_usage(body: UsageRequestBody, request: Request) -> JSONResponse:
+        db = community_db()
+        spec = _parse(body.repo, None)
+        wait = usage_limiter.check(client_key(request))
+        if wait is not None:
+            raise APIError(429, "rate_limited", "too many shared results from this address; try again later", wait)
+        public = is_public(spec)
+        stored = public is True
+        _note(
+            request,
+            repo={"owner": spec.owner, "repo": spec.repo, "full_name": spec.full_name, "ref": body.commit}
+            if stored
+            else None,  # never keep the name of a private or unknown repository
+            usage={"verdict": body.verdict, "findings": body.findings, "public": public},
+        )
+        if stored:
+            db.upsert_later(
+                "cli_scans", *cli_scan_upsert(body.model_dump(mode="json"), spec.full_name, datetime.now(UTC))
+            )
+        try:
+            check = check_document(db, spec, body.commit.lower() if body.commit else None) if stored else None
+        except Exception:
+            check = None
+        return JSONResponse(jsonable_encoder({"stored": stored, "check": check}), headers={"Cache-Control": "no-store"})
+
+    stats_cache: dict[int, tuple[float, dict[str, Any]]] = {}
+
+    @app.get(
+        "/v1/stats",
+        tags=["meta"],
+        summary="Usage statistics and recently flagged repositories",
+        responses={503: {"model": ErrorResponse}},
+    )
+    def stats(days: Annotated[int, Query(ge=1, le=365, description="Window in days.")] = 30) -> JSONResponse:
+        if not settings.stats_public or not isinstance(sink, MongoSink):
+            return JSONResponse({"enabled": False}, headers={"Cache-Control": "public, max-age=60"})
+        cached = stats_cache.get(days)
+        if cached is None or time.monotonic() - cached[0] > 60:
+            try:
+                cached = (time.monotonic(), jsonable_encoder(sink.stats(days)))
+            except Exception as exc:  # the database is optional; never let it break the API
+                log.warning("stats unavailable: %s", type(exc).__name__)
+                raise APIError(503, "stats_unavailable", "statistics are temporarily unavailable", 30) from None
+            stats_cache[days] = cached
+        return JSONResponse(cached[1], headers={"Cache-Control": "public, max-age=30, s-maxage=60"})
+
+    @app.get(
+        "/v1/stats/repos/{owner}/{repo}",
+        tags=["meta"],
+        summary="The latest scan of one repository, with its triage (dashboard detail view)",
+        responses={404: {"model": ErrorResponse}, 503: {"model": ErrorResponse}},
+    )
+    def stats_repo(owner: str, repo: str) -> JSONResponse:
+        spec = _parse(f"{owner}/{repo}", None)
+        if not settings.stats_public or not isinstance(sink, MongoSink):
+            raise APIError(404, "stats_disabled", "statistics are not enabled on this server")
+        try:
+            doc = sink.repository(spec.owner, spec.repo)
+        except Exception as exc:
+            log.warning("stats unavailable: %s", type(exc).__name__)
+            raise APIError(503, "stats_unavailable", "statistics are temporarily unavailable", 30) from None
+        if doc is None:
+            raise APIError(404, "not_found", f"{spec.full_name} has not been scanned here")
+        return JSONResponse(jsonable_encoder(doc), headers={"Cache-Control": "public, max-age=30, s-maxage=60"})
+
     @app.get("/healthz", tags=["meta"], summary="Liveness")
     def healthz() -> dict[str, str]:
         return {"status": "ok"}
 
     @app.get("/", include_in_schema=False)
-    def root() -> RedirectResponse:
-        return RedirectResponse("/docs")
+    def root() -> Response:
+        if not settings.stats_public:
+            return RedirectResponse("/docs")
+        return FileResponse(STATIC / "index.html", headers={"Cache-Control": "public, max-age=300"})
+
+    app.mount("/static", StaticFiles(directory=STATIC), name="static")
 
     return app
 
 
 # ---------------------------------------------------------------------- helpers
-def _lifespan(sink: TrafficSink) -> Callable[[FastAPI], Any]:
+def _lifespan(sink: TrafficSink, triager: Triager | None) -> Callable[[FastAPI], Any]:
     @asynccontextmanager
     async def lifespan(_: FastAPI) -> AsyncIterator[None]:
         yield
         sink.close(5.0)  # flush pending traffic records on shutdown (Vercel sends SIGTERM before scale-in)
+        if triager is not None:
+            triager.close(2.0)  # unfinished triage is retried by the next flagged scan of the same commit
 
     return lifespan
 

{devguard_scanner-0.3.0 → devguard_scanner-0.4.1}/src/devguard/api/schemas.py

@@ -129,3 +129,44 @@ class RuleDetails(RuleSummary):
     benign_context: str
     next_step: str
     origin: str
+
+
+# ---------------------------------------------------------------------- community: flags, checks, shared CLI scans
+_SHA = r"^[0-9a-fA-F]{40}$"
+
+
+class FlagRequestBody(BaseModel):
+    model_config = ConfigDict(
+        extra="forbid",
+        json_schema_extra={"examples": [{"repo": "owner/repo", "ref": "1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b"}]},
+    )
+
+    repo: str = Field(max_length=512, description="https://github.com/OWNER/REPO or OWNER/REPO.")
+    ref: str | None = Field(None, max_length=255, description="The commit (or branch/tag) you scanned.")
+    note: str | None = Field(None, max_length=500, description="What you saw. Kept private; never shown publicly.")
+
+
+class UsageFinding(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    rule_id: str = Field(max_length=64)
+    severity: str = Field(max_length=16)
+    title: str = Field(max_length=200)
+    path: str = Field(max_length=300)
+    line: int | None = Field(None, ge=0)
+
+
+class UsageRequestBody(BaseModel):
+    """A scan result shared by the CLI. Client-reported, so it never flags anything by itself."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    repo: str = Field(max_length=512)
+    commit: str | None = Field(None, pattern=_SHA)
+    verdict: Verdict
+    findings: int = Field(ge=0, le=100_000)
+    severity_counts: dict[str, int] = Field(default_factory=dict, max_length=8)
+    automatic_triggers: int = Field(0, ge=0, le=10_000)
+    top_findings: list[UsageFinding] = Field(default_factory=list, max_length=20)
+    devguard_version: str = Field(max_length=32)
+    ruleset_version: str = Field(max_length=32)