srclawyer 0.5.0 → 0.6.0

package/README.md

@@ -2,9 +2,9 @@
 
 Static analysis engine that derives privacy policy documentation directly from source code, and keeps it in sync as the code changes.
 
-> This is a Phase 1 scaffold: the CLI (`init`, `scan`), the Layer 1 static analysis engine — SDK/framework pattern detection, HTML/browser-API detection, and OpenAPI/GraphQL schema parsing — LLM-assisted reasoning for ambiguous findings (Layer 2, bring-your-own-key or managed), and markdown privacy policy generation. CI/CD integration is not yet implemented.
+> **Beta.** The CLI (`init`, `scan`), the Layer 1 static analysis engine, and markdown privacy policy generation all work today. JavaScript/TypeScript (Express, Next.js) is the most tested path; Python (Flask) is newer, with FastAPI and Django explicitly detected and flagged as unsupported rather than silently skipped — see [Supported languages and frameworks](#supported-languages-and-frameworks) and [Known gaps](#known-gaps) below. LLM-assisted reasoning for ambiguous findings (Layer 2) is available now under the bring-your-own-key tier, documented below as one optional step in `init` — a managed (hosted) tier is planned but not yet available.
 >
-> Scanning is manual-only (`srclawyer scan`, run on demand) — this is the free/solo tier's permanent behavior, not a placeholder. CI/CD integration (PR-triggered scans, a weekly backstop scan) is scoped for the paid/team tier only; see `PRODUCT_SPEC.md` Section 8.5.
+> Scanning is manual-only (`srclawyer scan`, run on demand) — this is the free/solo tier's permanent behavior, not a placeholder. Automatic CI/CD updating (a scan that runs itself on every PR, or on a schedule) isn't available yet; it's scoped for a future paid/team tier. See `PRODUCT_SPEC.md` Section 8.5 for the plan.
 
 ## Usage
 
@@ -63,18 +63,34 @@ If none of this resolves it, `npx srclawyer scan` (see [Usage](#usage) above) si
 
 ## What `scan` does today
 
-- **Known SDK integrations** (import/require detection): Stripe, SendGrid, Twilio, Google Analytics, Mixpanel, Auth0, Firebase, Cognito — each mapped to the data categories it typically processes (`src/rules/processorProfiles.ts`).
-- **Request-body field collection** in Express (`req.body.x`, destructured `req.body`) and Next.js (both `pages/api` and the app router's `await request.json()`), classified against a field-name lexicon (`src/rules/dataCategoryLexicon.ts`).
+- **Known SDK integrations** (import/require detection): Stripe, SendGrid, Twilio, Google Analytics, Mixpanel, Auth0, Firebase, Cognito for JS/TS; Stripe, SendGrid, Twilio, Mixpanel, Auth0, Firebase for Python — each mapped to the data categories it typically processes (`src/rules/processorProfiles.ts`).
+- **Request-body field collection** in Express (`req.body.x`, destructured `req.body`) and Next.js (both `pages/api` and the app router's `await request.json()`), and in **Flask** (`request.form`/`.json`/`.args`, both subscript and `.get(...)` forms), classified against a field-name lexicon (`src/rules/dataCategoryLexicon.ts`). In both languages, a value assigned to a local variable earlier in the *same function* (`const body = req.body` / `data = request.get_json()`) is tracked one mechanical hop, so a later `body.email` / `data['email']` is still caught — not just the direct, unaliased form.
 - **OpenAPI/Swagger schemas** (`components.schemas.*.properties`) and **GraphQL SDL** type/input fields.
 - **HTML forms and inline scripts** (`src/rules/htmlRules.ts`): native `<input>` elements classified by `type` (password/email/tel are unambiguous native signals) or by `id`/`name` against the lexicon; inline `<script>` blocks are extracted and run through the same SDK/framework/web-API rules as any other file.
 - **Browser platform APIs** (`src/rules/webApiRules.ts`): `getUserMedia` (split into microphone and camera access), geolocation, and `localStorage`/`sessionStorage` writes classified by key name.
 - Fields that can't be confidently classified (generic names like `data`, `payload`) are surfaced with `requiresReview: true` rather than silently guessed.
-- **A field-name allowlist** (`src/rules/safeFieldNames.ts`) suppresses findings entirely for exact-match structural fields (`id`, `createdAt`, `status`, pagination params, etc.) so "needs review" stays a signal, not noise from routine boilerplate. Anything not on the allowlist keeps the existing classify-or-flag behavior above.
-- **Unsupported language/framework detection** (`src/engine/languageDetection.ts`) runs at the start of every scan: it checks for Django (`manage.py`), Rails (`Gemfile` + `app/controllers`), gRPC (`.proto` files), and falls back to a file-count ratio for Python/Ruby/Go/Java/PHP/C# vs JS/TS/HTML. When it fires, a bordered warning appears at the top of the CLI output *and* at the top of the generated `PRIVACY_POLICY.md` (not just a log line easy to miss), the warning is included in `--json` output, and the process exits with code 1 so CI can't silently treat a scan of an unsupported codebase as a pass.
+- **A field-name allowlist** (`src/rules/safeFieldNames.ts`) suppresses findings entirely for exact-match structural and pagination/sort fields (`id`, `createdAt`, `status`, `per_page`, `cursor`, `sort_by`, etc.) so "needs review" stays a signal, not noise from routine boilerplate. Anything not on the allowlist keeps the existing classify-or-flag behavior above.
+- **Unsupported language/framework detection** (`src/engine/languageDetection.ts`) runs at the start of every scan: it checks for Django (`manage.py`), FastAPI (`from fastapi import` / `import fastapi`), Rails (`Gemfile` + `app/controllers`), gRPC (`.proto` files), and falls back to a file-count ratio for Ruby/Go/Java/PHP/C# vs JS/TS/HTML/Python. When it fires, a bordered warning appears at the top of the CLI output *and* at the top of the generated `PRIVACY_POLICY.md` (not just a log line easy to miss), the warning is included in `--json` output, and the process exits with code 1 so CI can't silently treat a scan of an unsupported codebase as a pass.
 - **Generates a markdown privacy policy** (`src/policy/generatePolicy.ts`) in the same command: findings are grouped by data category, each clause cites the file:line that justified it, third-party processors get their own section, and anything flagged `requiresReview` is excluded from the body and listed separately under "Items Requiring Manual Review" instead of being guessed. The mandatory disclaimer (`src/policy/disclaimer.ts`) is always appended, unconditionally.
 
 Every finding traces back to a file and line (`src/engine/types.ts#Finding`).
 
+### Supported languages and frameworks
+
+| Language | Frameworks | Status |
+|---|---|---|
+| JavaScript / TypeScript | Express, Next.js (pages + app router) | Supported |
+| Python | Flask | Supported (beta — see "Known gaps" below) |
+| Python | FastAPI, Django | Explicitly detected and flagged as unsupported, never silently skipped |
+| Ruby, Go, Java, PHP, C#, gRPC | — | Explicitly detected and flagged as unsupported |
+
+### Known gaps
+
+- **Cross-function aliasing isn't tracked, in either language.** The same-function alias tracking described above follows exactly one assignment hop within the function it occurs in. If that value is passed on — as a function parameter, through a closure, via a module-level variable — a field read from it afterward is not detected. Example: a Flask handler reads `data = request.get_json()` and calls `User.from_dict(data)`; a field read as `data['password']` *inside* `from_dict` is not caught, only reads inside the original handler function are.
+- **Data sent to a third party via a raw HTTP call isn't attributed to that processor.** SDK-integration detection (above) is anchored to known package imports. A call like Python's `requests.post("https://api.some-provider.com/...", json={...})` or JS's `fetch("https://api.some-provider.com/...")` — bypassing a dedicated SDK entirely — is invisible to this detection today, in both languages, regardless of how sensitive the data it sends is.
+- **Python support is based on one real-world corpus repository so far** (a Flask app; see `test-fixtures/golden-corpus/`), not the broader validation JS detection has had. Expect rougher edges on Flask codebases with unusual patterns.
+- **OpenAPI/Swagger scanning reports every schema under `components.schemas`, including response-only ones** — a schema that's only ever returned by the server (never submitted by a client) gets identical treatment to a real request-input schema, since nothing currently checks whether a schema is actually reachable from an operation's `requestBody` or `parameters`. On a large, schema-heavy spec this can mean a long "Items Requiring Manual Review" list that includes fields the product never collects, only ever returns. Separately, a schema defined **inline** inside a `requestBody` (not as a named, `$ref`'d `components.schemas` entry) isn't read at all today, in either direction — only named component schemas are scanned.
+
 ## Network calls
 
 Detection itself is fully local and offline — no code leaves your machine for pattern matching. Once detection completes, `scan` makes one batched network call to grade the confidence of what it found (field names and derived yes/no signals only, never source code), and, for scans that use Zod schemas as request validators, one additional call sending a small redacted code fragment. This runs on every scan regardless of tier and is unrelated to the separate, optional LLM-assisted (Layer 2) reasoning described above. See `PRODUCT_SPEC.md`'s "Static Analysis Depth" section for the precise wording.
@@ -91,6 +107,12 @@ Detection itself is fully local and offline — no code leaves your machine for
 
 `npm test` never touches the network. A separate, real-world accuracy check lives in `test-fixtures/golden-corpus/` — see its own README for what it measures and why it's not part of the default test run.
 
+## Legal
+
+SrcLawyer generates a *starting draft*, not a substitute for legal review — every generated policy carries that disclaimer (`src/policy/disclaimer.ts`), and `scan` repeats a short reminder of it at the end of its own output.
+
+A dedicated privacy policy and terms of service **for SrcLawyer itself** (the tool, not the policy it writes for your project) don't exist yet. Until they do, this README is the authoritative reference for what this tool sends over the network and to whom — see [Network calls](#network-calls) above — and for how it's licensed — see [License](#license) below.
+
 ## License
 
 [Business Source License 1.1](LICENSE) (BUSL-1.1) — source-available, not OSI-approved open source.

package/dist/cloud/protectedLogicClient.d.ts

@@ -3,6 +3,9 @@ export interface ProtectedLogicClientOptions {
     baseUrl: string;
     apiKey: string;
     batchSize?: number;
+    /** Max total JSON-serialized bytes of items per batch, in addition to batchSize -- whichever limit
+     *  is hit first ends the batch. */
+    batchMaxBytes?: number;
     timeoutMs?: number;
 }
 export interface ScoreFieldsItem {

package/dist/cloud/protectedLogicClient.js

@@ -3,42 +3,83 @@
  * these calls are bound by). Mirrors resolveAmbiguousFindings.ts's established pattern deliberately:
  * batch, try/catch per batch, never throw out of this module — a failure here degrades the scan
  * (safe local fallback + a scan-level warning), it never crashes it.
+ *
+ * Batching is by total request-body bytes as well as item count: confirmed via wrangler tail that the
+ * Worker's own CPU time limit is hit intermittently on large detect-zod-schema batches (large
+ * code fragments mean more server-side parsing work per batch, not just more JSON to transfer). Count
+ * alone (the original design) doesn't bound that — 25 large fragments can still be a very large batch.
  */
 const DEFAULT_BATCH_SIZE = 25;
+const DEFAULT_BATCH_MAX_BYTES = 50_000;
 const DEFAULT_TIMEOUT_MS = 10_000;
-function chunk(items, size) {
+const RETRY_BACKOFF_MS = 500;
+/**
+ * Greedily fills batches up to maxCount items AND maxBytes of JSON-serialized item size, whichever
+ * comes first. A single item already at or over maxBytes (shouldn't happen for zod candidates — see
+ * zodPreFilter.ts's own per-fragment cap, enforced upstream — but score-fields items have no such cap)
+ * still gets its own one-item batch rather than being dropped, so this never silently discards data;
+ * only the per-fragment cap upstream does that, and it always warns when it does.
+ */
+function chunkByCountAndBytes(items, maxCount, maxBytes) {
     const batches = [];
-    for (let i = 0; i < items.length; i += size)
-        batches.push(items.slice(i, i + size));
+    let current = [];
+    let currentBytes = 0;
+    for (const item of items) {
+        const itemBytes = Buffer.byteLength(JSON.stringify(item), "utf8");
+        const wouldExceed = current.length > 0 && (current.length >= maxCount || currentBytes + itemBytes > maxBytes);
+        if (wouldExceed) {
+            batches.push(current);
+            current = [];
+            currentBytes = 0;
+        }
+        current.push(item);
+        currentBytes += itemBytes;
+    }
+    if (current.length > 0)
+        batches.push(current);
     return batches;
 }
+function sleep(ms) {
+    return new Promise((resolve) => setTimeout(resolve, ms));
+}
 async function postJson(url, apiKey, body, timeoutMs) {
-    const controller = new AbortController();
-    const timeout = setTimeout(() => controller.abort(), timeoutMs);
-    try {
-        const response = await fetch(url, {
-            method: "POST",
-            headers: { "content-type": "application/json", "x-api-key": apiKey },
-            body: JSON.stringify(body),
-            signal: controller.signal,
-        });
-        if (!response.ok) {
-            throw new Error(`protected-logic request failed (${response.status})`);
+    const MAX_ATTEMPTS = 2; // one retry, on a 5xx only -- a 4xx (bad request, auth) won't succeed on retry
+    for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
+        const controller = new AbortController();
+        const timeout = setTimeout(() => controller.abort(), timeoutMs);
+        let status;
+        try {
+            const response = await fetch(url, {
+                method: "POST",
+                headers: { "content-type": "application/json", "x-api-key": apiKey },
+                body: JSON.stringify(body),
+                signal: controller.signal,
+            });
+            if (response.ok)
+                return (await response.json());
+            status = response.status;
         }
-        return (await response.json());
-    }
-    finally {
-        clearTimeout(timeout);
+        finally {
+            clearTimeout(timeout);
+        }
+        const isLastAttempt = attempt === MAX_ATTEMPTS;
+        if (status < 500 || isLastAttempt) {
+            throw new Error(`protected-logic request failed (${status})`);
+        }
+        await sleep(RETRY_BACKOFF_MS);
     }
+    // Unreachable: the loop above always either returns or throws.
+    throw new Error("protected-logic request failed (unreachable)");
 }
 export async function scoreFields(items, options) {
     const results = new Map();
     if (items.length === 0)
         return { results, error: null };
     const batchSize = options.batchSize ?? DEFAULT_BATCH_SIZE;
+    const batchMaxBytes = options.batchMaxBytes ?? DEFAULT_BATCH_MAX_BYTES;
     const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
     let error = null;
-    for (const batch of chunk(items, batchSize)) {
+    for (const batch of chunkByCountAndBytes(items, batchSize, batchMaxBytes)) {
         try {
             const response = await postJson(`${options.baseUrl}/v1/score-fields`, options.apiKey, { fields: batch }, timeoutMs);
             for (const result of response.results)
@@ -55,9 +96,10 @@ export async function detectZodSchemas(items, options) {
     if (items.length === 0)
         return { results, error: null };
     const batchSize = options.batchSize ?? DEFAULT_BATCH_SIZE;
+    const batchMaxBytes = options.batchMaxBytes ?? DEFAULT_BATCH_MAX_BYTES;
     const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
     let error = null;
-    for (const batch of chunk(items, batchSize)) {
+    for (const batch of chunkByCountAndBytes(items, batchSize, batchMaxBytes)) {
         try {
             const response = await postJson(`${options.baseUrl}/v1/detect-zod-schema`, options.apiKey, { candidates: batch }, timeoutMs);
             for (const result of response.results)

package/dist/cloud/resolveProtectedLogic.d.ts

@@ -11,4 +11,4 @@ export interface ResolveProtectedLogicResult {
  * same way resolveAmbiguousFindings.ts is layered on top of it for Layer 2 — not called from inside
  * scan() itself, so scan()'s own purity/auditability claim stays true.
  */
-export declare function resolveProtectedLogic(findings: Finding[], pendingZodCandidates: ZodCandidate[], options: ProtectedLogicClientOptions): Promise<ResolveProtectedLogicResult>;
+export declare function resolveProtectedLogic(findings: Finding[], pendingZodCandidates: ZodCandidate[], options: ProtectedLogicClientOptions, oversizedZodFiles?: string[]): Promise<ResolveProtectedLogicResult>;

package/dist/cloud/resolveProtectedLogic.js

@@ -1,4 +1,5 @@
 import { classifyFieldName, isAmbiguousFieldName } from "../rules/dataCategoryLexicon.js";
+import { isSafeFieldName } from "../rules/safeFieldNames.js";
 import { countConfidentSiblings } from "../rules/confidenceFactors.js";
 import { detectZodSchemas, scoreFields } from "./protectedLogicClient.js";
 /**
@@ -7,9 +8,20 @@ import { detectZodSchemas, scoreFields } from "./protectedLogicClient.js";
  * same way resolveAmbiguousFindings.ts is layered on top of it for Layer 2 — not called from inside
  * scan() itself, so scan()'s own purity/auditability claim stays true.
  */
-export async function resolveProtectedLogic(findings, pendingZodCandidates, options) {
+export async function resolveProtectedLogic(findings, pendingZodCandidates, options, oversizedZodFiles = []) {
     let workingFindings = findings;
     const warnings = [];
+    if (oversizedZodFiles.length > 0) {
+        // Kept to a single summarized line -- this list is meant to be rare now that the hard ceiling is
+        // only for pathological cases (see MAX_FRAGMENT_BYTES), but a long list here would still be an
+        // unreadable wall of text. The full, untruncated list is always in ScanResult.oversizedZodFiles
+        // (surfaced in --json output), so nothing here is actually lost, just not all printed inline.
+        const PREVIEW_COUNT = 3;
+        const preview = oversizedZodFiles.slice(0, PREVIEW_COUNT).join(", ");
+        const remaining = oversizedZodFiles.length - PREVIEW_COUNT;
+        const suffix = remaining > 0 ? `, and ${remaining} more (full list in --json output)` : "";
+        warnings.push(`Zod-schema detection skipped for ${oversizedZodFiles.length} file(s) because the extracted fragment was too large to send safely: ${preview}${suffix}.`);
+    }
     if (pendingZodCandidates.length > 0) {
         const byId = new Map(pendingZodCandidates.map((c) => [c.id, c]));
         const { results, error } = await detectZodSchemas(pendingZodCandidates.map((c) => ({ id: c.id, codeFragment: c.codeFragment })), options);
@@ -22,6 +34,8 @@ export async function resolveProtectedLogic(findings, pendingZodCandidates, opti
                 continue;
             const allNames = result.fields.map((f) => f.name);
             for (const field of result.fields) {
+                if (isSafeFieldName(field.name))
+                    continue;
                 const sourceLine = candidate.fragmentLineToSourceLine[field.line - 1] ?? field.line;
                 const lexiconCategory = classifyFieldName(field.name);
                 const ambiguousName = isAmbiguousFieldName(field.name);

package/dist/commands/init.js

@@ -2,6 +2,7 @@ import { existsSync, readFileSync } from "node:fs";
 import { join } from "node:path";
 import { createInterface } from "node:readline/promises";
 import { CONFIG_FILENAME, DEFAULT_CONFIG, configPath, hasUnmigratedLegacyConfig, legacyConfigMessage, writeConfig, } from "../config/config.js";
+import { NETWORK_CALLS_NOTICE } from "../policy/legalNotices.js";
 function guessIndustry(root) {
     const pkgPath = join(root, "package.json");
     if (!existsSync(pkgPath))
@@ -36,12 +37,19 @@ export async function runInit(root) {
     const guessedIndustry = guessIndustry(root);
     const rl = createInterface({ input: process.stdin, output: process.stdout });
     const lines = rl[Symbol.asyncIterator]();
+    // Printed once, here, not re-shown on every scan -- informational only, never blocks: nothing below
+    // waits on an acknowledgment, so this is safe in non-interactive contexts (CI, scripted installs).
+    console.log(`${NETWORK_CALLS_NOTICE}\n`);
     console.log("SrcLawyer setup — answered once, reused on every scan.\n");
     const entityLocation = await ask(lines, "Where is your legal entity located? (country)", DEFAULT_CONFIG.entityLocation ?? "");
     const targetMarketsRaw = await ask(lines, "Which regions/markets do you serve? (comma-separated, e.g. US, EU)", "");
     const industry = await ask(lines, "What industry/vertical is this product in?", guessedIndustry ?? "");
     const childrensDataRaw = await ask(lines, "Does this product knowingly collect data from children under 13/16?", "no");
-    const tierRaw = await ask(lines, "Tier: free (static analysis only) / byok (bring your own Anthropic key) / managed (not yet available)", "free");
+    // "managed" is deliberately not offered here -- it's a real SrcLawyerTier value and config.ts/
+    // resolveProviderForTier.ts both still support it fully (a hand-edited config with tier: managed
+    // still gets a clear, accurate error at scan time), but there's no advertised path to it from this
+    // prompt, so a tester reading the choices here can't select a dead end during setup.
+    const tierRaw = await ask(lines, "Tier: free (static analysis only) / byok (bring your own Anthropic key)", "free");
     const tier = normalizeTier(tierRaw);
     let apiKeyEnvVar = null;
     if (tier === "byok") {

package/dist/commands/scan.js

@@ -8,6 +8,8 @@ import { resolveProviderForTier } from "../llm/resolveProviderForTier.js";
 import { resolveAmbiguousFindings } from "../llm/resolveAmbiguousFindings.js";
 import { resolveProtectedLogic } from "../cloud/resolveProtectedLogic.js";
 import { resolveProtectedLogicEndpoint } from "../cloud/config.js";
+import { MAX_FRAGMENT_BYTES } from "../rules/zodPreFilter.js";
+import { SCAN_DISCLAIMER_REMINDER, byokNetworkNotice } from "../policy/legalNotices.js";
 export async function runScan(root, options) {
     const config = loadConfig(root);
     let legacyConfigWarning = null;
@@ -22,15 +24,18 @@ export async function runScan(root, options) {
     }
     const result = await scan(root);
     let findings = result.findings;
+    if (result.maxZodFragmentBytes > 0) {
+        console.error(`Largest Zod-schema fragment this scan: ${result.maxZodFragmentBytes} bytes (hard ceiling: ${MAX_FRAGMENT_BYTES} bytes).`);
+    }
     // Not tier-gated, not behind --no-llm: confidence scoring and Zod-schema detection are Layer 1
     // capabilities now, needed on every real scan regardless of which LLM tier (if any) is configured.
     // A failure here degrades honestly (findings keep their safe local fallback) rather than blocking
     // the scan — see resolveProtectedLogic's own per-call error handling.
-    const hasScorableFindings = findings.some((f) => f.confidenceFactors) || result.pendingZodCandidates.length > 0;
+    const hasScorableFindings = findings.some((f) => f.confidenceFactors) || result.pendingZodCandidates.length > 0 || result.oversizedZodFiles.length > 0;
     let protectedLogicWarning = null;
     if (hasScorableFindings) {
         const endpoint = resolveProtectedLogicEndpoint();
-        const resolved = await resolveProtectedLogic(findings, result.pendingZodCandidates, endpoint);
+        const resolved = await resolveProtectedLogic(findings, result.pendingZodCandidates, endpoint, result.oversizedZodFiles);
         findings = resolved.findings;
         protectedLogicWarning = resolved.warning;
         if (protectedLogicWarning)
@@ -46,6 +51,9 @@ export async function runScan(root, options) {
         else if (provider) {
             const ambiguous = findings.filter((f) => f.requiresReview);
             if (ambiguous.length > 0) {
+                if (config.tier === "byok" && config.llm) {
+                    console.log(byokNetworkNotice(config.llm.apiKeyEnvVar));
+                }
                 console.log(`Sending ${ambiguous.length} ambiguous item(s) to ${provider.name}...`);
                 const resolved = await resolveAmbiguousFindings(ambiguous, provider, {
                     collectsChildrensData: config.collectsChildrensData,
@@ -58,7 +66,7 @@ export async function runScan(root, options) {
     const ambiguousCount = findings.filter((f) => f.requiresReview).length;
     const finalResult = { ...result, findings, ambiguousCount };
     const policyPath = resolve(root, options.out);
-    writeFileSync(policyPath, generatePolicyMarkdown(finalResult, config, llmError));
+    writeFileSync(policyPath, generatePolicyMarkdown(finalResult, config, llmError, protectedLogicWarning));
     if (result.unsupportedStackWarning || llmError || protectedLogicWarning || legacyConfigWarning) {
         process.exitCode = 1;
     }
@@ -69,5 +77,8 @@ export async function runScan(root, options) {
     console.log(formatText(finalResult));
     if (protectedLogicWarning)
         console.log(`\n${protectedLogicWarning}`);
+    if (legacyConfigWarning)
+        console.log(`\n${legacyConfigWarning}`);
     console.log(`\nPrivacy policy written to ${join(options.out)}`);
+    console.log(`\n${SCAN_DISCLAIMER_REMINDER}`);
 }

package/dist/engine/discoverFiles.d.ts

@@ -1,5 +1,6 @@
 export interface DiscoveredFiles {
     codeFiles: string[];
+    pythonFiles: string[];
     htmlFiles: string[];
     schemaFiles: string[];
     manifestFiles: string[];

package/dist/engine/discoverFiles.js

@@ -1,5 +1,6 @@
 import fg from "fast-glob";
 const CODE_PATTERNS = ["**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "**/*.mjs", "**/*.cjs"];
+const PYTHON_PATTERNS = ["**/*.py"];
 const HTML_PATTERNS = ["**/*.html", "**/*.htm"];
 const SCHEMA_PATTERNS = ["**/*.graphql", "**/*.gql", "**/openapi.yaml", "**/openapi.yml", "**/openapi.json", "**/swagger.yaml", "**/swagger.yml", "**/swagger.json"];
 const MANIFEST_PATTERNS = ["**/package.json", "**/requirements.txt"];
@@ -14,12 +15,25 @@ const IGNORE = [
     "**/*.spec.*",
     "**/*.d.ts",
 ];
+// Separate from IGNORE: node_modules has no Python analogue, and excluding a venv/site-packages
+// directory matters even more here, since it's a full copy of every installed pip package's own
+// source -- the Python equivalent of node_modules, and just as irrelevant/misleading to scan.
+const PYTHON_IGNORE = [
+    ...IGNORE,
+    "**/venv/**",
+    "**/.venv/**",
+    "**/__pycache__/**",
+    "**/site-packages/**",
+    "**/test_*.py",
+    "**/*_test.py",
+];
 export async function discoverFiles(root) {
-    const [codeFiles, htmlFiles, schemaFiles, manifestFiles] = await Promise.all([
+    const [codeFiles, pythonFiles, htmlFiles, schemaFiles, manifestFiles] = await Promise.all([
         fg(CODE_PATTERNS, { cwd: root, absolute: true, ignore: IGNORE, dot: false }),
+        fg(PYTHON_PATTERNS, { cwd: root, absolute: true, ignore: PYTHON_IGNORE, dot: false }),
         fg(HTML_PATTERNS, { cwd: root, absolute: true, ignore: IGNORE, dot: false }),
         fg(SCHEMA_PATTERNS, { cwd: root, absolute: true, ignore: ["**/node_modules/**"], dot: false }),
         fg(MANIFEST_PATTERNS, { cwd: root, absolute: true, ignore: ["**/node_modules/**"], dot: false }),
     ]);
-    return { codeFiles, htmlFiles, schemaFiles, manifestFiles };
+    return { codeFiles, pythonFiles, htmlFiles, schemaFiles, manifestFiles };
 }

package/dist/engine/languageDetection.js

@@ -1,22 +1,25 @@
-import { existsSync } from "node:fs";
+import { existsSync, readFileSync } from "node:fs";
 import { join } from "node:path";
 import fg from "fast-glob";
-const IGNORE = ["**/node_modules/**", "**/dist/**", "**/build/**", "**/.git/**", "**/vendor/**"];
-const SUPPORTED_EXTENSIONS = ["ts", "tsx", "js", "jsx", "mjs", "cjs", "html", "htm"];
+const IGNORE = ["**/node_modules/**", "**/dist/**", "**/build/**", "**/.git/**", "**/vendor/**", "**/venv/**", "**/.venv/**", "**/__pycache__/**"];
+// Python moved from UNSUPPORTED_LANGUAGES to here once Flask detection shipped (Stage A) -- it's a
+// supported *language* now, but only one framework within it is actually understood. A Python project
+// using a different framework still needs an honest gap warning; see detectUnsupportedPythonFramework.
+const SUPPORTED_EXTENSIONS = ["ts", "tsx", "js", "jsx", "mjs", "cjs", "html", "htm", "py"];
 const UNSUPPORTED_LANGUAGES = [
-    { label: "Python", extensions: ["py"] },
     { label: "Ruby", extensions: ["rb"] },
     { label: "Go", extensions: ["go"] },
     { label: "Java", extensions: ["java"] },
     { label: "PHP", extensions: ["php"] },
     { label: "C#", extensions: ["cs"] },
 ];
+const FASTAPI_IMPORT = /\bfrom\s+fastapi\b|\bimport\s+fastapi\b/;
 function bannerize(message) {
-    return `UNSUPPORTED CODEBASE — ${message} SrcLawyer only analyzes JavaScript/TypeScript and HTML today; the findings below (if any) are incomplete and MUST NOT be treated as a full compliance scan.`;
+    return `UNSUPPORTED CODEBASE — ${message} SrcLawyer only analyzes JavaScript/TypeScript, HTML, and Flask-based Python today; the findings below (if any) are incomplete and MUST NOT be treated as a full compliance scan.`;
 }
 async function detectFrameworkSignature(root) {
     if (existsSync(join(root, "manage.py"))) {
-        return bannerize("Django detected (manage.py found at the project root).");
+        return bannerize("Django detected (manage.py found at the project root) — Django support isn't built yet.");
     }
     const hasGemfile = existsSync(join(root, "Gemfile"));
     const hasRailsLayout = existsSync(join(root, "config", "routes.rb")) || existsSync(join(root, "app", "controllers"));
@@ -29,6 +32,27 @@ async function detectFrameworkSignature(root) {
     }
     return null;
 }
+/**
+ * Presence-based, like detectFrameworkSignature's manage.py check -- not an absence heuristic ("we
+ * found nothing so something must be wrong"), which can't tell a genuinely clean Flask file apart
+ * from an unsupported framework. Scans actual file content (not just a marker file) since FastAPI,
+ * unlike Django, has no single root-level marker file to check for instead.
+ */
+async function detectUnsupportedPythonFramework(root, pythonFiles) {
+    for (const filePath of pythonFiles) {
+        let content;
+        try {
+            content = readFileSync(filePath, "utf8");
+        }
+        catch {
+            continue;
+        }
+        if (FASTAPI_IMPORT.test(content)) {
+            return bannerize("FastAPI detected (a `fastapi` import was found) — FastAPI support isn't built yet.");
+        }
+    }
+    return null;
+}
 async function detectDominantUnsupportedLanguage(root) {
     const [supportedFiles, ...unsupportedGroups] = await Promise.all([
         fg(SUPPORTED_EXTENSIONS.map((ext) => `**/*.${ext}`), { cwd: root, ignore: IGNORE, dot: false }),
@@ -38,11 +62,17 @@ async function detectDominantUnsupportedLanguage(root) {
     const dominant = counts.filter((c) => c.count > 0).sort((a, b) => b.count - a.count)[0];
     if (!dominant || dominant.count <= supportedFiles.length)
         return null;
-    return bannerize(`${dominant.label} detected as the primary language (${dominant.count} file(s), vs ${supportedFiles.length} supported JS/TS/HTML file(s)).`);
+    return bannerize(`${dominant.label} detected as the primary language (${dominant.count} file(s), vs ${supportedFiles.length} supported file(s)).`);
 }
 export async function detectUnsupportedStack(root) {
     const frameworkWarning = await detectFrameworkSignature(root);
     if (frameworkWarning)
         return frameworkWarning;
+    const pythonFiles = await fg("**/*.py", { cwd: root, absolute: true, ignore: IGNORE, dot: false });
+    if (pythonFiles.length > 0) {
+        const pythonFrameworkWarning = await detectUnsupportedPythonFramework(root, pythonFiles);
+        if (pythonFrameworkWarning)
+            return pythonFrameworkWarning;
+    }
     return detectDominantUnsupportedLanguage(root);
 }

package/dist/engine/pythonAstUtils.d.ts

@@ -0,0 +1,2 @@
+import { type Tree } from "web-tree-sitter";
+export declare function parsePythonSource(source: string): Promise<Tree | null>;

package/dist/engine/pythonAstUtils.js

@@ -0,0 +1,44 @@
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+import { Parser, Language } from "web-tree-sitter";
+// This file compiles to dist/engine/pythonAstUtils.js; the grammar is copied to dist/grammars/ by
+// scripts/copyGrammars.js as part of `npm run build` (see grammars/README.md for why it's vendored
+// rather than an npm dependency).
+const HERE = dirname(fileURLToPath(import.meta.url));
+const GRAMMAR_PATH = join(HERE, "..", "grammars", "tree-sitter-python.wasm");
+// Initialized at most once per process, reused across every Python file in a scan -- mirrors
+// tsProject.ts's lazy-memoized pattern for the same reason (real setup cost, no benefit to repeating
+// it per file). Parser.init()/Language.load() are both async; parser.parse() itself is synchronous
+// once a Parser is set up, and this module's scan-time usage is sequential (not concurrent), so one
+// shared Parser instance is safe to reuse.
+let parserPromise = null;
+function initParser() {
+    if (!parserPromise) {
+        parserPromise = (async () => {
+            try {
+                await Parser.init();
+                const language = await Language.load(GRAMMAR_PATH);
+                const parser = new Parser();
+                parser.setLanguage(language);
+                return parser;
+            }
+            catch {
+                // Honest null, matching astUtils.ts's parseSource -- a missing/incompatible grammar build
+                // degrades this file's detection to nothing rather than crashing the whole scan.
+                return null;
+            }
+        })();
+    }
+    return parserPromise;
+}
+export async function parsePythonSource(source) {
+    const parser = await initParser();
+    if (!parser)
+        return null;
+    try {
+        return parser.parse(source);
+    }
+    catch {
+        return null;
+    }
+}

package/dist/engine/scanEngine.js

@@ -4,10 +4,13 @@ import { discoverProjects } from "./discoverProjects.js";
 import { discoverFiles } from "./discoverFiles.js";
 import { detectUnsupportedStack } from "./languageDetection.js";
 import { parseSource } from "./astUtils.js";
+import { parsePythonSource } from "./pythonAstUtils.js";
 import { runSdkRules } from "../rules/sdkRules.js";
 import { runFrameworkRules } from "../rules/frameworkRules.js";
 import { runWebApiRules } from "../rules/webApiRules.js";
 import { runTypedRequestBodyRules } from "../rules/typedRequestBodyRules.js";
+import { runPythonSdkRules } from "../rules/pythonSdkRules.js";
+import { runPythonFrameworkRules } from "../rules/pythonFrameworkRules.js";
 import { findZodCandidates } from "../rules/zodPreFilter.js";
 import { buildTsProject } from "./tsProject.js";
 import { runHtmlInputRules, extractInlineScripts } from "../rules/htmlRules.js";
@@ -34,9 +37,11 @@ function makeLazyTsProject(files) {
         return handle;
     };
 }
-function runCodeRules(filePath, source, root, getTsProject, zodCandidates) {
+function runCodeRules(filePath, source, root, getTsProject, zodCandidates, oversizedZodFiles) {
     const ast = parseSource(source, filePath);
-    zodCandidates.push(...findZodCandidates(source, filePath, root));
+    const zodResult = findZodCandidates(source, filePath, root);
+    zodCandidates.push(...zodResult.candidates);
+    oversizedZodFiles.push(...zodResult.oversizedFiles);
     if (!ast)
         return [];
     return [
@@ -46,6 +51,12 @@ function runCodeRules(filePath, source, root, getTsProject, zodCandidates) {
         ...runTypedRequestBodyRules(ast, filePath, source, root, getTsProject),
     ];
 }
+async function runPythonCodeRules(filePath, source, root) {
+    const tree = await parsePythonSource(source);
+    if (!tree)
+        return [];
+    return [...runPythonSdkRules(tree, filePath, source, root), ...runPythonFrameworkRules(tree, filePath, source, root)];
+}
 /**
  * Pure, local, offline — no network access anywhere in this function or anything it calls. Zod
  * schemas and confidence scoring can no longer be fully resolved here (see pendingZodCandidates and
@@ -59,10 +70,12 @@ export async function scan(root) {
     const [projects, unsupportedStackWarning] = await Promise.all([discoverProjects(root), detectUnsupportedStack(root)]);
     const findings = [];
     const pendingZodCandidates = [];
+    const oversizedZodFiles = [];
     let filesScanned = 0;
     const projectRootList = [...new Set(projects.map((p) => p.root))];
     const fileSets = await Promise.all(projectRootList.map((projectRoot) => discoverFiles(projectRoot)));
     const allCodeFiles = new Set();
+    const allPythonFiles = new Set();
     const allHtmlFiles = new Set();
     const allSchemaFiles = new Set();
     const rootForFile = new Map();
@@ -74,6 +87,7 @@ export async function scan(root) {
             allCodeFiles.add(f);
             rootForFile.set(f, projectRoot);
         });
+        set.pythonFiles.forEach((f) => allPythonFiles.add(f));
         set.htmlFiles.forEach((f) => allHtmlFiles.add(f));
         set.schemaFiles.forEach((f) => allSchemaFiles.add(f));
     });
@@ -90,7 +104,18 @@ export async function scan(root) {
             continue;
         }
         const getTsProject = tsProjectByRoot.get(rootForFile.get(filePath) ?? "") ?? noTsProject;
-        findings.push(...runCodeRules(filePath, source, root, getTsProject, pendingZodCandidates));
+        findings.push(...runCodeRules(filePath, source, root, getTsProject, pendingZodCandidates, oversizedZodFiles));
+    }
+    for (const filePath of allPythonFiles) {
+        filesScanned += 1;
+        let source;
+        try {
+            source = readFileSync(filePath, "utf8");
+        }
+        catch {
+            continue;
+        }
+        findings.push(...(await runPythonCodeRules(filePath, source, root)));
     }
     for (const filePath of allHtmlFiles) {
         filesScanned += 1;
@@ -103,7 +128,7 @@ export async function scan(root) {
         }
         findings.push(...runHtmlInputRules(html, filePath, root));
         for (const script of extractInlineScripts(html)) {
-            findings.push(...runCodeRules(filePath, script.code, root, noTsProject, pendingZodCandidates));
+            findings.push(...runCodeRules(filePath, script.code, root, noTsProject, pendingZodCandidates, oversizedZodFiles));
         }
     }
     for (const filePath of allSchemaFiles) {
@@ -126,5 +151,17 @@ export async function scan(root) {
         }
     }
     const ambiguousCount = findings.filter((f) => f.requiresReview).length;
-    return { projects, findings, ambiguousCount, filesScanned, unsupportedStackWarning, pendingZodCandidates };
+    const maxZodFragmentBytes = pendingZodCandidates.length
+        ? Math.max(...pendingZodCandidates.map((c) => Buffer.byteLength(c.codeFragment, "utf8")))
+        : 0;
+    return {
+        projects,
+        findings,
+        ambiguousCount,
+        filesScanned,
+        unsupportedStackWarning,
+        pendingZodCandidates,
+        oversizedZodFiles,
+        maxZodFragmentBytes,
+    };
 }

package/dist/engine/types.d.ts

@@ -68,4 +68,13 @@ export interface ScanResult {
         codeFragment: string;
         fragmentLineToSourceLine: number[];
     }>;
+    /** Relative paths of files whose extracted Zod fragment was too large to send safely and was
+     *  dropped before ever reaching pendingZodCandidates -- surfaced in resolveProtectedLogic()'s
+     *  warning so a schema going unchecked is never silent, just like a failed network call is. */
+    oversizedZodFiles: string[];
+    /** The largest pendingZodCandidates fragment's byte size this scan, or 0 if none. Logged by the
+     *  caller so real-world fragment sizes keep being observed against MAX_FRAGMENT_BYTES
+     *  (zodPreFilter.ts) and the batching cap (protectedLogicClient.ts) -- both are provisional,
+     *  unvalidated against the Worker's real limit until its plan upgrade is confirmed. */
+    maxZodFragmentBytes: number;
 }