srclawyer 0.5.0 → 0.6.1

package/README.md

@@ -2,9 +2,9 @@
 
 Static analysis engine that derives privacy policy documentation directly from source code, and keeps it in sync as the code changes.
 
-> This is a Phase 1 scaffold: the CLI (`init`, `scan`), the Layer 1 static analysis engine — SDK/framework pattern detection, HTML/browser-API detection, and OpenAPI/GraphQL schema parsing — LLM-assisted reasoning for ambiguous findings (Layer 2, bring-your-own-key or managed), and markdown privacy policy generation. CI/CD integration is not yet implemented.
+> **Beta.** The CLI (`init`, `scan`), the Layer 1 static analysis engine, and markdown privacy policy generation all work today. JavaScript/TypeScript (Express, Next.js) is the most tested path; Python (Flask) is newer, with FastAPI and Django explicitly detected and flagged as unsupported rather than silently skipped — see [Supported languages and frameworks](#supported-languages-and-frameworks) and [Known gaps](#known-gaps) below. LLM-assisted reasoning for ambiguous findings (Layer 2) is available now under the bring-your-own-key tier, documented below as one optional step in `init` — a managed (hosted) tier is planned but not yet available.
 >
-> Scanning is manual-only (`srclawyer scan`, run on demand) — this is the free/solo tier's permanent behavior, not a placeholder. CI/CD integration (PR-triggered scans, a weekly backstop scan) is scoped for the paid/team tier only; see `PRODUCT_SPEC.md` Section 8.5.
+> Scanning is manual-only (`srclawyer scan`, run on demand) — this is the free/solo tier's permanent behavior, not a placeholder. Automatic CI/CD updating (a scan that runs itself on every PR, or on a schedule) isn't available yet; it's scoped for a future paid/team tier. See `PRODUCT_SPEC.md` Section 8.5 for the plan.
 
 ## Usage
 
@@ -63,18 +63,34 @@ If none of this resolves it, `npx srclawyer scan` (see [Usage](#usage) above) si
 
 ## What `scan` does today
 
-- **Known SDK integrations** (import/require detection): Stripe, SendGrid, Twilio, Google Analytics, Mixpanel, Auth0, Firebase, Cognito — each mapped to the data categories it typically processes (`src/rules/processorProfiles.ts`).
-- **Request-body field collection** in Express (`req.body.x`, destructured `req.body`) and Next.js (both `pages/api` and the app router's `await request.json()`), classified against a field-name lexicon (`src/rules/dataCategoryLexicon.ts`).
+- **Known SDK integrations** (import/require detection): Stripe, SendGrid, Twilio, Google Analytics, Mixpanel, Auth0, Firebase, Cognito for JS/TS; Stripe, SendGrid, Twilio, Mixpanel, Auth0, Firebase for Python — each mapped to the data categories it typically processes (`src/rules/processorProfiles.ts`).
+- **Request-body field collection** in Express (`req.body.x`, destructured `req.body`) and Next.js (both `pages/api` and the app router's `await request.json()`), and in **Flask** (`request.form`/`.json`/`.args`, both subscript and `.get(...)` forms), classified against a field-name lexicon (`src/rules/dataCategoryLexicon.ts`). In both languages, a value assigned to a local variable earlier in the *same function* (`const body = req.body` / `data = request.get_json()`) is tracked one mechanical hop, so a later `body.email` / `data['email']` is still caught — not just the direct, unaliased form.
 - **OpenAPI/Swagger schemas** (`components.schemas.*.properties`) and **GraphQL SDL** type/input fields.
 - **HTML forms and inline scripts** (`src/rules/htmlRules.ts`): native `<input>` elements classified by `type` (password/email/tel are unambiguous native signals) or by `id`/`name` against the lexicon; inline `<script>` blocks are extracted and run through the same SDK/framework/web-API rules as any other file.
 - **Browser platform APIs** (`src/rules/webApiRules.ts`): `getUserMedia` (split into microphone and camera access), geolocation, and `localStorage`/`sessionStorage` writes classified by key name.
 - Fields that can't be confidently classified (generic names like `data`, `payload`) are surfaced with `requiresReview: true` rather than silently guessed.
-- **A field-name allowlist** (`src/rules/safeFieldNames.ts`) suppresses findings entirely for exact-match structural fields (`id`, `createdAt`, `status`, pagination params, etc.) so "needs review" stays a signal, not noise from routine boilerplate. Anything not on the allowlist keeps the existing classify-or-flag behavior above.
-- **Unsupported language/framework detection** (`src/engine/languageDetection.ts`) runs at the start of every scan: it checks for Django (`manage.py`), Rails (`Gemfile` + `app/controllers`), gRPC (`.proto` files), and falls back to a file-count ratio for Python/Ruby/Go/Java/PHP/C# vs JS/TS/HTML. When it fires, a bordered warning appears at the top of the CLI output *and* at the top of the generated `PRIVACY_POLICY.md` (not just a log line easy to miss), the warning is included in `--json` output, and the process exits with code 1 so CI can't silently treat a scan of an unsupported codebase as a pass.
+- **A field-name allowlist** (`src/rules/safeFieldNames.ts`) suppresses findings entirely for exact-match structural and pagination/sort fields (`id`, `createdAt`, `status`, `per_page`, `cursor`, `sort_by`, etc.) so "needs review" stays a signal, not noise from routine boilerplate. Anything not on the allowlist keeps the existing classify-or-flag behavior above.
+- **Unsupported language/framework detection** (`src/engine/languageDetection.ts`) runs at the start of every scan: it checks for Django (`manage.py`), FastAPI (`from fastapi import` / `import fastapi`), Rails (`Gemfile` + `app/controllers`), gRPC (`.proto` files), and falls back to a file-count ratio for Ruby/Go/Java/PHP/C# vs JS/TS/HTML/Python. When it fires, a bordered warning appears at the top of the CLI output *and* at the top of the generated `PRIVACY_POLICY.md` (not just a log line easy to miss), the warning is included in `--json` output, and the process exits with code 1 so CI can't silently treat a scan of an unsupported codebase as a pass.
 - **Generates a markdown privacy policy** (`src/policy/generatePolicy.ts`) in the same command: findings are grouped by data category, each clause cites the file:line that justified it, third-party processors get their own section, and anything flagged `requiresReview` is excluded from the body and listed separately under "Items Requiring Manual Review" instead of being guessed. The mandatory disclaimer (`src/policy/disclaimer.ts`) is always appended, unconditionally.
 
 Every finding traces back to a file and line (`src/engine/types.ts#Finding`).
 
+### Supported languages and frameworks
+
+| Language | Frameworks | Status |
+|---|---|---|
+| JavaScript / TypeScript | Express, Next.js (pages + app router) | Supported |
+| Python | Flask | Supported (beta — see "Known gaps" below) |
+| Python | FastAPI, Django | Explicitly detected and flagged as unsupported, never silently skipped |
+| Ruby, Go, Java, PHP, C#, gRPC | — | Explicitly detected and flagged as unsupported |
+
+### Known gaps
+
+- **Cross-function aliasing isn't tracked, in either language.** The same-function alias tracking described above follows exactly one assignment hop within the function it occurs in. If that value is passed on — as a function parameter, through a closure, via a module-level variable — a field read from it afterward is not detected. Example: a Flask handler reads `data = request.get_json()` and calls `User.from_dict(data)`; a field read as `data['password']` *inside* `from_dict` is not caught, only reads inside the original handler function are.
+- **Data sent to a third party via a raw HTTP call isn't attributed to that processor.** SDK-integration detection (above) is anchored to known package imports. A call like Python's `requests.post("https://api.some-provider.com/...", json={...})` or JS's `fetch("https://api.some-provider.com/...")` — bypassing a dedicated SDK entirely — is invisible to this detection today, in both languages, regardless of how sensitive the data it sends is.
+- **Python support is based on one real-world corpus repository so far** (a Flask app; see `test-fixtures/golden-corpus/`), not the broader validation JS detection has had. Expect rougher edges on Flask codebases with unusual patterns.
+- **OpenAPI/Swagger scanning reports every schema under `components.schemas`, including response-only ones** — a schema that's only ever returned by the server (never submitted by a client) gets identical treatment to a real request-input schema, since nothing currently checks whether a schema is actually reachable from an operation's `requestBody` or `parameters`. On a large, schema-heavy spec this can mean a long "Items Requiring Manual Review" list that includes fields the product never collects, only ever returns. Separately, a schema defined **inline** inside a `requestBody` (not as a named, `$ref`'d `components.schemas` entry) isn't read at all today, in either direction — only named component schemas are scanned.
+
 ## Network calls
 
 Detection itself is fully local and offline — no code leaves your machine for pattern matching. Once detection completes, `scan` makes one batched network call to grade the confidence of what it found (field names and derived yes/no signals only, never source code), and, for scans that use Zod schemas as request validators, one additional call sending a small redacted code fragment. This runs on every scan regardless of tier and is unrelated to the separate, optional LLM-assisted (Layer 2) reasoning described above. See `PRODUCT_SPEC.md`'s "Static Analysis Depth" section for the precise wording.
@@ -91,6 +107,12 @@ Detection itself is fully local and offline — no code leaves your machine for
 
 `npm test` never touches the network. A separate, real-world accuracy check lives in `test-fixtures/golden-corpus/` — see its own README for what it measures and why it's not part of the default test run.
 
+## Legal
+
+SrcLawyer generates a *starting draft*, not a substitute for legal review — every generated policy carries that disclaimer (`src/policy/disclaimer.ts`), and `scan` repeats a short reminder of it at the end of its own output.
+
+A dedicated privacy policy and terms of service **for SrcLawyer itself** (the tool, not the policy it writes for your project) don't exist yet. Until they do, this README is the authoritative reference for what this tool sends over the network and to whom — see [Network calls](#network-calls) above — and for how it's licensed — see [License](#license) below.
+
 ## License
 
 [Business Source License 1.1](LICENSE) (BUSL-1.1) — source-available, not OSI-approved open source.

package/dist/cli.js

@@ -16,9 +16,51 @@ program
     .version(version);
 program
     .command("init")
-    .description("One-time setup: jurisdiction, markets, industry")
-    .action(async () => {
-    await runInit(process.cwd());
+    .description("One-time setup: jurisdiction, markets, industry, and optional policy details (company info, data practices, regional rights)")
+    .option("--policy-details", "ask the optional policy-detail questions (company info, data practices, regional rights)")
+    .option("--no-policy-details", "skip all optional policy-detail questions, leaving them unanswered")
+    .option("--collects-childrens-data", "declare that this product knowingly collects data from children under 13/16")
+    .option("--no-collects-childrens-data", "declare that this product does not knowingly collect data from children under 13/16")
+    .option("--company-name <name>", "company or product name")
+    .option("--site-url <url>", "site or app URL")
+    .option("--privacy-contact-email <email>", "privacy contact email address")
+    .option("--eu-uk-users", "declare that this product has users in the EU/UK")
+    .option("--no-eu-uk-users", "declare that this product has no users in the EU/UK")
+    .option("--california-users", "declare that this product has users in California")
+    .option("--no-california-users", "declare that this product has no users in California")
+    .option("--sells-or-shares-data", "declare that personal data is sold or shared with third parties")
+    .option("--no-sells-or-shares-data", "declare that personal data is not sold or shared with third parties")
+    .option("--uses-data-for-advertising", "declare that personal data is used for advertising")
+    .option("--no-uses-data-for-advertising", "declare that personal data is not used for advertising")
+    .option("--data-retention-policy <text>", "how long personal data is retained")
+    .option("--account-deletion-instructions <text>", "how a user can delete their account or request data deletion")
+    .option("--minimum-age <number>", "minimum age required to use this product")
+    .option("--sends-data-to-ai-services", "declare that user data is sent to third-party AI services")
+    .option("--no-sends-data-to-ai-services", "declare that user data is not sent to third-party AI services")
+    .action(async (opts) => {
+    let minimumAge;
+    if (opts.minimumAge !== undefined) {
+        const parsed = Number(opts.minimumAge);
+        if (!Number.isInteger(parsed) || parsed < 0) {
+            program.error(`error: --minimum-age must be a whole, non-negative number (got "${opts.minimumAge}")`);
+        }
+        minimumAge = parsed;
+    }
+    await runInit(process.cwd(), {
+        policyDetails: opts.policyDetails,
+        collectsChildrensData: opts.collectsChildrensData,
+        companyName: opts.companyName,
+        siteUrl: opts.siteUrl,
+        privacyContactEmail: opts.privacyContactEmail,
+        euUkUsers: opts.euUkUsers,
+        californiaUsers: opts.californiaUsers,
+        sellsOrSharesData: opts.sellsOrSharesData,
+        usesDataForAdvertising: opts.usesDataForAdvertising,
+        dataRetentionPolicy: opts.dataRetentionPolicy,
+        accountDeletionInstructions: opts.accountDeletionInstructions,
+        minimumAge,
+        sendsDataToAiServices: opts.sendsDataToAiServices,
+    });
 });
 program
     .command("scan")

package/dist/cloud/protectedLogicClient.d.ts

@@ -3,6 +3,9 @@ export interface ProtectedLogicClientOptions {
     baseUrl: string;
     apiKey: string;
     batchSize?: number;
+    /** Max total JSON-serialized bytes of items per batch, in addition to batchSize -- whichever limit
+     *  is hit first ends the batch. */
+    batchMaxBytes?: number;
     timeoutMs?: number;
 }
 export interface ScoreFieldsItem {

package/dist/cloud/protectedLogicClient.js

@@ -3,42 +3,83 @@
  * these calls are bound by). Mirrors resolveAmbiguousFindings.ts's established pattern deliberately:
  * batch, try/catch per batch, never throw out of this module — a failure here degrades the scan
  * (safe local fallback + a scan-level warning), it never crashes it.
+ *
+ * Batching is by total request-body bytes as well as item count: confirmed via wrangler tail that the
+ * Worker's own CPU time limit is hit intermittently on large detect-zod-schema batches (large
+ * code fragments mean more server-side parsing work per batch, not just more JSON to transfer). Count
+ * alone (the original design) doesn't bound that — 25 large fragments can still be a very large batch.
  */
 const DEFAULT_BATCH_SIZE = 25;
+const DEFAULT_BATCH_MAX_BYTES = 50_000;
 const DEFAULT_TIMEOUT_MS = 10_000;
-function chunk(items, size) {
+const RETRY_BACKOFF_MS = 500;
+/**
+ * Greedily fills batches up to maxCount items AND maxBytes of JSON-serialized item size, whichever
+ * comes first. A single item already at or over maxBytes (shouldn't happen for zod candidates — see
+ * zodPreFilter.ts's own per-fragment cap, enforced upstream — but score-fields items have no such cap)
+ * still gets its own one-item batch rather than being dropped, so this never silently discards data;
+ * only the per-fragment cap upstream does that, and it always warns when it does.
+ */
+function chunkByCountAndBytes(items, maxCount, maxBytes) {
     const batches = [];
-    for (let i = 0; i < items.length; i += size)
-        batches.push(items.slice(i, i + size));
+    let current = [];
+    let currentBytes = 0;
+    for (const item of items) {
+        const itemBytes = Buffer.byteLength(JSON.stringify(item), "utf8");
+        const wouldExceed = current.length > 0 && (current.length >= maxCount || currentBytes + itemBytes > maxBytes);
+        if (wouldExceed) {
+            batches.push(current);
+            current = [];
+            currentBytes = 0;
+        }
+        current.push(item);
+        currentBytes += itemBytes;
+    }
+    if (current.length > 0)
+        batches.push(current);
     return batches;
 }
+function sleep(ms) {
+    return new Promise((resolve) => setTimeout(resolve, ms));
+}
 async function postJson(url, apiKey, body, timeoutMs) {
-    const controller = new AbortController();
-    const timeout = setTimeout(() => controller.abort(), timeoutMs);
-    try {
-        const response = await fetch(url, {
-            method: "POST",
-            headers: { "content-type": "application/json", "x-api-key": apiKey },
-            body: JSON.stringify(body),
-            signal: controller.signal,
-        });
-        if (!response.ok) {
-            throw new Error(`protected-logic request failed (${response.status})`);
+    const MAX_ATTEMPTS = 2; // one retry, on a 5xx only -- a 4xx (bad request, auth) won't succeed on retry
+    for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
+        const controller = new AbortController();
+        const timeout = setTimeout(() => controller.abort(), timeoutMs);
+        let status;
+        try {
+            const response = await fetch(url, {
+                method: "POST",
+                headers: { "content-type": "application/json", "x-api-key": apiKey },
+                body: JSON.stringify(body),
+                signal: controller.signal,
+            });
+            if (response.ok)
+                return (await response.json());
+            status = response.status;
         }
-        return (await response.json());
-    }
-    finally {
-        clearTimeout(timeout);
+        finally {
+            clearTimeout(timeout);
+        }
+        const isLastAttempt = attempt === MAX_ATTEMPTS;
+        if (status < 500 || isLastAttempt) {
+            throw new Error(`protected-logic request failed (${status})`);
+        }
+        await sleep(RETRY_BACKOFF_MS);
     }
+    // Unreachable: the loop above always either returns or throws.
+    throw new Error("protected-logic request failed (unreachable)");
 }
 export async function scoreFields(items, options) {
     const results = new Map();
     if (items.length === 0)
         return { results, error: null };
     const batchSize = options.batchSize ?? DEFAULT_BATCH_SIZE;
+    const batchMaxBytes = options.batchMaxBytes ?? DEFAULT_BATCH_MAX_BYTES;
     const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
     let error = null;
-    for (const batch of chunk(items, batchSize)) {
+    for (const batch of chunkByCountAndBytes(items, batchSize, batchMaxBytes)) {
         try {
             const response = await postJson(`${options.baseUrl}/v1/score-fields`, options.apiKey, { fields: batch }, timeoutMs);
             for (const result of response.results)
@@ -55,9 +96,10 @@ export async function detectZodSchemas(items, options) {
     if (items.length === 0)
         return { results, error: null };
     const batchSize = options.batchSize ?? DEFAULT_BATCH_SIZE;
+    const batchMaxBytes = options.batchMaxBytes ?? DEFAULT_BATCH_MAX_BYTES;
     const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
     let error = null;
-    for (const batch of chunk(items, batchSize)) {
+    for (const batch of chunkByCountAndBytes(items, batchSize, batchMaxBytes)) {
         try {
             const response = await postJson(`${options.baseUrl}/v1/detect-zod-schema`, options.apiKey, { candidates: batch }, timeoutMs);
             for (const result of response.results)

package/dist/cloud/resolveProtectedLogic.d.ts

@@ -11,4 +11,4 @@ export interface ResolveProtectedLogicResult {
  * same way resolveAmbiguousFindings.ts is layered on top of it for Layer 2 — not called from inside
  * scan() itself, so scan()'s own purity/auditability claim stays true.
  */
-export declare function resolveProtectedLogic(findings: Finding[], pendingZodCandidates: ZodCandidate[], options: ProtectedLogicClientOptions): Promise<ResolveProtectedLogicResult>;
+export declare function resolveProtectedLogic(findings: Finding[], pendingZodCandidates: ZodCandidate[], options: ProtectedLogicClientOptions, oversizedZodFiles?: string[]): Promise<ResolveProtectedLogicResult>;

package/dist/cloud/resolveProtectedLogic.js

@@ -1,4 +1,5 @@
 import { classifyFieldName, isAmbiguousFieldName } from "../rules/dataCategoryLexicon.js";
+import { isSafeFieldName } from "../rules/safeFieldNames.js";
 import { countConfidentSiblings } from "../rules/confidenceFactors.js";
 import { detectZodSchemas, scoreFields } from "./protectedLogicClient.js";
 /**
@@ -7,9 +8,20 @@ import { detectZodSchemas, scoreFields } from "./protectedLogicClient.js";
  * same way resolveAmbiguousFindings.ts is layered on top of it for Layer 2 — not called from inside
  * scan() itself, so scan()'s own purity/auditability claim stays true.
  */
-export async function resolveProtectedLogic(findings, pendingZodCandidates, options) {
+export async function resolveProtectedLogic(findings, pendingZodCandidates, options, oversizedZodFiles = []) {
     let workingFindings = findings;
     const warnings = [];
+    if (oversizedZodFiles.length > 0) {
+        // Kept to a single summarized line -- this list is meant to be rare now that the hard ceiling is
+        // only for pathological cases (see MAX_FRAGMENT_BYTES), but a long list here would still be an
+        // unreadable wall of text. The full, untruncated list is always in ScanResult.oversizedZodFiles
+        // (surfaced in --json output), so nothing here is actually lost, just not all printed inline.
+        const PREVIEW_COUNT = 3;
+        const preview = oversizedZodFiles.slice(0, PREVIEW_COUNT).join(", ");
+        const remaining = oversizedZodFiles.length - PREVIEW_COUNT;
+        const suffix = remaining > 0 ? `, and ${remaining} more (full list in --json output)` : "";
+        warnings.push(`Zod-schema detection skipped for ${oversizedZodFiles.length} file(s) because the extracted fragment was too large to send safely: ${preview}${suffix}.`);
+    }
     if (pendingZodCandidates.length > 0) {
         const byId = new Map(pendingZodCandidates.map((c) => [c.id, c]));
         const { results, error } = await detectZodSchemas(pendingZodCandidates.map((c) => ({ id: c.id, codeFragment: c.codeFragment })), options);
@@ -22,6 +34,8 @@ export async function resolveProtectedLogic(findings, pendingZodCandidates, opti
                 continue;
             const allNames = result.fields.map((f) => f.name);
             for (const field of result.fields) {
+                if (isSafeFieldName(field.name))
+                    continue;
                 const sourceLine = candidate.fragmentLineToSourceLine[field.line - 1] ?? field.line;
                 const lexiconCategory = classifyFieldName(field.name);
                 const ambiguousName = isAmbiguousFieldName(field.name);

package/dist/commands/init.d.ts

@@ -1 +1,16 @@
-export declare function runInit(root: string): Promise<void>;
+export interface InitFlags {
+    policyDetails?: boolean;
+    collectsChildrensData?: boolean;
+    companyName?: string;
+    siteUrl?: string;
+    privacyContactEmail?: string;
+    euUkUsers?: boolean;
+    californiaUsers?: boolean;
+    sellsOrSharesData?: boolean;
+    usesDataForAdvertising?: boolean;
+    dataRetentionPolicy?: string;
+    accountDeletionInstructions?: string;
+    minimumAge?: number;
+    sendsDataToAiServices?: boolean;
+}
+export declare function runInit(root: string, flags?: InitFlags): Promise<void>;

package/dist/commands/init.js

@@ -2,6 +2,7 @@ import { existsSync, readFileSync } from "node:fs";
 import { join } from "node:path";
 import { createInterface } from "node:readline/promises";
 import { CONFIG_FILENAME, DEFAULT_CONFIG, configPath, hasUnmigratedLegacyConfig, legacyConfigMessage, writeConfig, } from "../config/config.js";
+import { NETWORK_CALLS_NOTICE } from "../policy/legalNotices.js";
 function guessIndustry(root) {
     const pkgPath = join(root, "package.json");
     if (!existsSync(pkgPath))
@@ -23,7 +24,62 @@ async function ask(lines, question, fallback) {
     const answer = done ? "" : value.trim();
     return answer.length > 0 ? answer : fallback;
 }
-export async function runInit(root) {
+async function askOptional(lines, question) {
+    process.stdout.write(`${question} (optional, press Enter to skip): `);
+    const { value, done } = await lines.next();
+    const answer = done ? "" : value.trim();
+    return answer.length > 0 ? answer : null;
+}
+// suggested, when not null, is shown as the bracketed default and adopted on a blank answer -- used
+// to prefill EU/UK and California questions from targetMarkets without ever forcing an answer. When
+// suggested is null (no signal), blank means genuinely unanswered, same as every other optional field.
+async function askYesNoOptional(lines, question, suggested = null) {
+    const suffix = suggested === null ? " (y/n, optional, press Enter to skip)" : ` [${suggested ? "y" : "n"}]`;
+    process.stdout.write(`${question}${suffix}: `);
+    const { value, done } = await lines.next();
+    const answer = done ? "" : value.trim().toLowerCase();
+    if (answer.length === 0)
+        return suggested;
+    return /^y(es)?$/.test(answer);
+}
+async function askOptionalAge(lines, question) {
+    const raw = await askOptional(lines, question);
+    if (raw === null)
+        return null;
+    const parsed = Number(raw);
+    if (!Number.isInteger(parsed) || parsed < 0) {
+        console.log(`  (Could not read "${raw}" as a whole, non-negative number -- skipping. You can set this later by editing ${CONFIG_FILENAME}.)`);
+        return null;
+    }
+    return parsed;
+}
+// Generic resolution order for every new optional field: an explicit flag always wins outright;
+// otherwise the field is only asked if the policy-details gate resolved to yes; otherwise it's null.
+async function resolveField(flagValue, gateYes, askFn) {
+    if (flagValue !== undefined)
+        return flagValue;
+    if (!gateYes)
+        return null;
+    return askFn();
+}
+const EU_UK_TOKENS = new Set(["eu", "uk", "gb", "eea", "european union", "united kingdom"]);
+const CALIFORNIA_TOKENS = new Set(["ca", "california", "us-ca"]);
+function targetMarketsSuggestEuUk(markets) {
+    return markets.some((m) => EU_UK_TOKENS.has(m.trim().toLowerCase()));
+}
+function targetMarketsSuggestCalifornia(markets) {
+    return markets.some((m) => CALIFORNIA_TOKENS.has(m.trim().toLowerCase()));
+}
+// Only warns in the one direction that's a real contradiction: target markets signal a region, but
+// the final answer explicitly denies it. The reverse (no signal, but the user says yes) isn't a
+// contradiction -- targetMarkets is free text and was never meant to be an exhaustive jurisdiction
+// list. Never blocks or overrides -- the explicit answer always stands, this only warns.
+function warnIfContradictsTargetMarkets(label, finalValue, suggestedFromMarkets) {
+    if (suggestedFromMarkets && finalValue === false) {
+        console.log(`\nWarning: your target markets suggest ${label} users, but you indicated none -- the generated policy will omit ${label}-specific disclosures. Re-run init or edit ${CONFIG_FILENAME} if this is incorrect.\n`);
+    }
+}
+export async function runInit(root, flags = {}) {
     const existingPath = configPath(root);
     if (existsSync(existingPath)) {
         console.log(`${CONFIG_FILENAME} already exists. Delete it first if you want to redo setup.`);
@@ -36,36 +92,72 @@ export async function runInit(root) {
     const guessedIndustry = guessIndustry(root);
     const rl = createInterface({ input: process.stdin, output: process.stdout });
     const lines = rl[Symbol.asyncIterator]();
+    // Printed once, here, not re-shown on every scan -- informational only, never blocks: nothing below
+    // waits on an acknowledgment, so this is safe in non-interactive contexts (CI, scripted installs).
+    console.log(`${NETWORK_CALLS_NOTICE}\n`);
     console.log("SrcLawyer setup — answered once, reused on every scan.\n");
     const entityLocation = await ask(lines, "Where is your legal entity located? (country)", DEFAULT_CONFIG.entityLocation ?? "");
     const targetMarketsRaw = await ask(lines, "Which regions/markets do you serve? (comma-separated, e.g. US, EU)", "");
+    const targetMarkets = targetMarketsRaw.split(",").map((s) => s.trim()).filter(Boolean);
     const industry = await ask(lines, "What industry/vertical is this product in?", guessedIndustry ?? "");
-    const childrensDataRaw = await ask(lines, "Does this product knowingly collect data from children under 13/16?", "no");
-    const tierRaw = await ask(lines, "Tier: free (static analysis only) / byok (bring your own Anthropic key) / managed (not yet available)", "free");
-    const tier = normalizeTier(tierRaw);
+    const gateYes = flags.policyDetails !== undefined
+        ? flags.policyDetails
+        : /^y(es)?$/i.test((await ask(lines, "Add privacy policy details now? (company info, data practices, regional rights, etc.)", "y")).trim());
+    // Deliberately requires an explicit y/n -- a bare Enter, empty stdin, or a declined gate must leave
+    // this null (unanswered, rendered as a placeholder), never silently "no". Unlike askOptional's other
+    // callers, there is no safe default here: "no" is itself a real, consequential legal declaration, not
+    // a neutral fallback. Layer 2's high-stakes override (scan.ts) coerces null to false at its one call
+    // site, so leaving this unanswered never weakens that safety check -- see childrensDataOverride.test.ts.
+    const collectsChildrensData = await resolveField(flags.collectsChildrensData, gateYes, () => askYesNoOptional(lines, "Does this product knowingly collect data from children under 13/16?"));
+    const minimumAge = await resolveField(flags.minimumAge, gateYes, () => askOptionalAge(lines, "Minimum age required to use this product"));
+    const companyName = await resolveField(flags.companyName, gateYes, () => askOptional(lines, "Company or product name"));
+    const siteUrl = await resolveField(flags.siteUrl, gateYes, () => askOptional(lines, "Site or app URL"));
+    const privacyContactEmail = await resolveField(flags.privacyContactEmail, gateYes, () => askOptional(lines, "Privacy contact email"));
+    const euUkSuggested = targetMarketsSuggestEuUk(targetMarkets);
+    const hasEuUkUsers = await resolveField(flags.euUkUsers, gateYes, () => askYesNoOptional(lines, "Do you have users in the EU/UK?", euUkSuggested ? true : null));
+    warnIfContradictsTargetMarkets("EU/UK", hasEuUkUsers, euUkSuggested);
+    const californiaSuggested = targetMarketsSuggestCalifornia(targetMarkets);
+    const hasCaliforniaUsers = await resolveField(flags.californiaUsers, gateYes, () => askYesNoOptional(lines, "Do you have users in California?", californiaSuggested ? true : null));
+    warnIfContradictsTargetMarkets("California", hasCaliforniaUsers, californiaSuggested);
+    const sellsOrSharesData = await resolveField(flags.sellsOrSharesData, gateYes, () => askYesNoOptional(lines, "Does this product sell or share personal data with third parties?"));
+    const usesDataForAdvertising = await resolveField(flags.usesDataForAdvertising, gateYes, () => askYesNoOptional(lines, "Does this product use personal data for advertising?"));
+    const dataRetentionPolicy = await resolveField(flags.dataRetentionPolicy, gateYes, () => askOptional(lines, "How long is personal data retained?"));
+    const accountDeletionInstructions = await resolveField(flags.accountDeletionInstructions, gateYes, () => askOptional(lines, "How can a user delete their account or request data deletion?"));
+    // Deliberately about the SCANNED PRODUCT's own behavior (does it send its end-users' data to AI
+    // services as part of its own functionality) -- not about SrcLawyer's own, separately-documented
+    // BYOK network calls described in NETWORK_CALLS_NOTICE above.
+    const sendsDataToAiServices = await resolveField(flags.sendsDataToAiServices, gateYes, () => askYesNoOptional(lines, "Does this product send user data to third-party AI services (e.g. OpenAI, Anthropic) as part of its own functionality?"));
+    // "managed" is deliberately not offered here -- it's a real SrcLawyerTier value and config.ts/
+    // resolveProviderForTier.ts both still support it fully (a hand-edited config with tier: managed
+    // still gets a clear, accurate error at scan time), but there's no advertised path to it from this
+    // prompt, so a tester reading the choices here can't select a dead end during setup.
+    const enableByokRaw = await ask(lines, "Enable optional AI-assisted review with your own Anthropic key? (advanced)", "n");
+    const tier = /^y(es)?$/i.test(enableByokRaw.trim()) ? "byok" : "free";
     let apiKeyEnvVar = null;
     if (tier === "byok") {
         apiKeyEnvVar = await ask(lines, "Which environment variable holds your Anthropic API key?", "ANTHROPIC_API_KEY");
         console.log(`\nNote: only the name "${apiKeyEnvVar}" is written to ${CONFIG_FILENAME} — the key itself is never written to disk. It's read from your environment at scan time.`);
     }
-    else if (tier === "managed") {
-        console.log("\nHeads up: the managed tier's hosted backend isn't available yet. Layer 1 (static analysis) will still run fine; AI-assisted resolution will report a clear error until it ships.");
-    }
     rl.close();
     const config = {
         entityLocation: entityLocation || null,
-        targetMarkets: targetMarketsRaw.split(",").map((s) => s.trim()).filter(Boolean),
+        targetMarkets,
         industry: industry || null,
-        collectsChildrensData: /^y(es)?$/i.test(childrensDataRaw.trim()),
+        collectsChildrensData,
         tier,
         ...(tier === "byok" ? { llm: { provider: "anthropic", apiKeyEnvVar } } : {}),
+        companyName,
+        siteUrl,
+        privacyContactEmail,
+        hasEuUkUsers,
+        hasCaliforniaUsers,
+        sellsOrSharesData,
+        usesDataForAdvertising,
+        dataRetentionPolicy,
+        accountDeletionInstructions,
+        minimumAge,
+        sendsDataToAiServices,
     };
     writeConfig(root, config);
     console.log(`\nWrote ${CONFIG_FILENAME}. Run "srclawyer scan" to analyze this codebase.`);
 }
-function normalizeTier(raw) {
-    const normalized = raw.trim().toLowerCase();
-    if (normalized === "byok" || normalized === "managed")
-        return normalized;
-    return "free";
-}

package/dist/commands/scan.js

@@ -8,6 +8,8 @@ import { resolveProviderForTier } from "../llm/resolveProviderForTier.js";
 import { resolveAmbiguousFindings } from "../llm/resolveAmbiguousFindings.js";
 import { resolveProtectedLogic } from "../cloud/resolveProtectedLogic.js";
 import { resolveProtectedLogicEndpoint } from "../cloud/config.js";
+import { MAX_FRAGMENT_BYTES } from "../rules/zodPreFilter.js";
+import { SCAN_DISCLAIMER_REMINDER, byokNetworkNotice } from "../policy/legalNotices.js";
 export async function runScan(root, options) {
     const config = loadConfig(root);
     let legacyConfigWarning = null;
@@ -22,15 +24,18 @@ export async function runScan(root, options) {
     }
     const result = await scan(root);
     let findings = result.findings;
+    if (result.maxZodFragmentBytes > 0) {
+        console.error(`Largest Zod-schema fragment this scan: ${result.maxZodFragmentBytes} bytes (hard ceiling: ${MAX_FRAGMENT_BYTES} bytes).`);
+    }
     // Not tier-gated, not behind --no-llm: confidence scoring and Zod-schema detection are Layer 1
     // capabilities now, needed on every real scan regardless of which LLM tier (if any) is configured.
     // A failure here degrades honestly (findings keep their safe local fallback) rather than blocking
     // the scan — see resolveProtectedLogic's own per-call error handling.
-    const hasScorableFindings = findings.some((f) => f.confidenceFactors) || result.pendingZodCandidates.length > 0;
+    const hasScorableFindings = findings.some((f) => f.confidenceFactors) || result.pendingZodCandidates.length > 0 || result.oversizedZodFiles.length > 0;
     let protectedLogicWarning = null;
     if (hasScorableFindings) {
         const endpoint = resolveProtectedLogicEndpoint();
-        const resolved = await resolveProtectedLogic(findings, result.pendingZodCandidates, endpoint);
+        const resolved = await resolveProtectedLogic(findings, result.pendingZodCandidates, endpoint, result.oversizedZodFiles);
         findings = resolved.findings;
         protectedLogicWarning = resolved.warning;
         if (protectedLogicWarning)
@@ -46,9 +51,16 @@ export async function runScan(root, options) {
         else if (provider) {
             const ambiguous = findings.filter((f) => f.requiresReview);
             if (ambiguous.length > 0) {
+                if (config.tier === "byok" && config.llm) {
+                    console.log(byokNetworkNotice(config.llm.apiKeyEnvVar));
+                }
                 console.log(`Sending ${ambiguous.length} ambiguous item(s) to ${provider.name}...`);
                 const resolved = await resolveAmbiguousFindings(ambiguous, provider, {
-                    collectsChildrensData: config.collectsChildrensData,
+                    // Preserves today's exact override behavior bit-for-bit: a missing/legacy key already
+                    // behaved as false before collectsChildrensData became nullable. Only the generated
+                    // document (generatePolicy.ts) distinguishes "explicitly no" from "never answered" -- this
+                    // safety-critical override is unchanged.
+                    collectsChildrensData: config.collectsChildrensData ?? false,
                 });
                 const resolvedById = new Map(resolved.map((f) => [f.id, f]));
                 findings = findings.map((f) => resolvedById.get(f.id) ?? f);
@@ -58,7 +70,7 @@ export async function runScan(root, options) {
     const ambiguousCount = findings.filter((f) => f.requiresReview).length;
     const finalResult = { ...result, findings, ambiguousCount };
     const policyPath = resolve(root, options.out);
-    writeFileSync(policyPath, generatePolicyMarkdown(finalResult, config, llmError));
+    writeFileSync(policyPath, generatePolicyMarkdown(finalResult, config, llmError, protectedLogicWarning));
     if (result.unsupportedStackWarning || llmError || protectedLogicWarning || legacyConfigWarning) {
         process.exitCode = 1;
     }
@@ -69,5 +81,8 @@ export async function runScan(root, options) {
     console.log(formatText(finalResult));
     if (protectedLogicWarning)
         console.log(`\n${protectedLogicWarning}`);
+    if (legacyConfigWarning)
+        console.log(`\n${legacyConfigWarning}`);
     console.log(`\nPrivacy policy written to ${join(options.out)}`);
+    console.log(`\n${SCAN_DISCLAIMER_REMINDER}`);
 }

package/dist/config/config.d.ts

@@ -10,9 +10,20 @@ export interface SrcLawyerConfig {
     entityLocation: string | null;
     targetMarkets: string[];
     industry: string | null;
-    collectsChildrensData: boolean;
+    collectsChildrensData: boolean | null;
     tier?: SrcLawyerTier;
     llm?: SrcLawyerLlmConfig;
+    companyName?: string | null;
+    siteUrl?: string | null;
+    privacyContactEmail?: string | null;
+    hasEuUkUsers?: boolean | null;
+    hasCaliforniaUsers?: boolean | null;
+    sellsOrSharesData?: boolean | null;
+    usesDataForAdvertising?: boolean | null;
+    dataRetentionPolicy?: string | null;
+    accountDeletionInstructions?: string | null;
+    minimumAge?: number | null;
+    sendsDataToAiServices?: boolean | null;
 }
 export declare const DEFAULT_CONFIG: SrcLawyerConfig;
 export declare function configPath(root: string): string;

package/dist/config/config.js

@@ -8,8 +8,19 @@ export const DEFAULT_CONFIG = {
     entityLocation: null,
     targetMarkets: [],
     industry: null,
-    collectsChildrensData: false,
+    collectsChildrensData: null,
     tier: "free",
+    companyName: null,
+    siteUrl: null,
+    privacyContactEmail: null,
+    hasEuUkUsers: null,
+    hasCaliforniaUsers: null,
+    sellsOrSharesData: null,
+    usesDataForAdvertising: null,
+    dataRetentionPolicy: null,
+    accountDeletionInstructions: null,
+    minimumAge: null,
+    sendsDataToAiServices: null,
 };
 export function configPath(root) {
     return join(root, CONFIG_FILENAME);

package/dist/engine/discoverFiles.d.ts

@@ -1,5 +1,6 @@
 export interface DiscoveredFiles {
     codeFiles: string[];
+    pythonFiles: string[];
     htmlFiles: string[];
     schemaFiles: string[];
     manifestFiles: string[];

package/dist/engine/discoverFiles.js

@@ -1,5 +1,6 @@
 import fg from "fast-glob";
 const CODE_PATTERNS = ["**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "**/*.mjs", "**/*.cjs"];
+const PYTHON_PATTERNS = ["**/*.py"];
 const HTML_PATTERNS = ["**/*.html", "**/*.htm"];
 const SCHEMA_PATTERNS = ["**/*.graphql", "**/*.gql", "**/openapi.yaml", "**/openapi.yml", "**/openapi.json", "**/swagger.yaml", "**/swagger.yml", "**/swagger.json"];
 const MANIFEST_PATTERNS = ["**/package.json", "**/requirements.txt"];
@@ -14,12 +15,25 @@ const IGNORE = [
     "**/*.spec.*",
     "**/*.d.ts",
 ];
+// Separate from IGNORE: node_modules has no Python analogue, and excluding a venv/site-packages
+// directory matters even more here, since it's a full copy of every installed pip package's own
+// source -- the Python equivalent of node_modules, and just as irrelevant/misleading to scan.
+const PYTHON_IGNORE = [
+    ...IGNORE,
+    "**/venv/**",
+    "**/.venv/**",
+    "**/__pycache__/**",
+    "**/site-packages/**",
+    "**/test_*.py",
+    "**/*_test.py",
+];
 export async function discoverFiles(root) {
-    const [codeFiles, htmlFiles, schemaFiles, manifestFiles] = await Promise.all([
+    const [codeFiles, pythonFiles, htmlFiles, schemaFiles, manifestFiles] = await Promise.all([
         fg(CODE_PATTERNS, { cwd: root, absolute: true, ignore: IGNORE, dot: false }),
+        fg(PYTHON_PATTERNS, { cwd: root, absolute: true, ignore: PYTHON_IGNORE, dot: false }),
         fg(HTML_PATTERNS, { cwd: root, absolute: true, ignore: IGNORE, dot: false }),
         fg(SCHEMA_PATTERNS, { cwd: root, absolute: true, ignore: ["**/node_modules/**"], dot: false }),
         fg(MANIFEST_PATTERNS, { cwd: root, absolute: true, ignore: ["**/node_modules/**"], dot: false }),
     ]);
-    return { codeFiles, htmlFiles, schemaFiles, manifestFiles };
+    return { codeFiles, pythonFiles, htmlFiles, schemaFiles, manifestFiles };
 }