forgeos 0.1.0-alpha.68 → 0.1.0-alpha.69

package/docs/repository-agent-benchmark.md

@@ -0,0 +1,179 @@
+# Repository agent benchmark
+
+Forge can prepare a reproducible experiment with and without repository maps, then
+evaluate evidence supplied by an external runner. Planning and evaluation are pure:
+they do not invoke Codex, read source files, execute application checks, install dependencies,
+or grant permission to spend credits. A static retrieval fixture passing is useful evidence
+about a packet, but does not demonstrate improved model productivity.
+
+## Prepare a paired plan
+
+The CLI validates cache ownership, graph integrity and current source freshness before
+planning. Save its complete JSON output; reporting accepts either that `{plan, ok,
+exitCode}` envelope or the bare API plan. A model config contains only `provider`,
+`name` and scalar `settings`, with no credentials:
+
+```text
+forge repository benchmark-plan --cases tasks.json --model-config model.json --repetitions 2 --json
+forge repository benchmark-report --plan plan.json --observations observations.json --json
+```
+
+The model name is selected explicitly for the experiment, never inferred from the app.
+Planning/reporting are read-only. `--write` is rejected. A valid report with no observations
+has zero comparable pairs and pending results; CLI exit zero confirms valid input/report
+generation only. Planning inputs are capped at 128 KiB, plan files at 128 MiB and observation
+files at 16 MiB. Symlinks and sensitive input paths are refused.
+
+Use `createRepositoryAgentBenchmarkPlan(snapshot, reviewedCases, options)` from
+`src/forge/repository-analysis/benchmark.ts`. The caller must first read and validate a
+current snapshot and verify source freshness through the repository context API. The
+planner validates the complete snapshot structure and fingerprint; as a pure function
+it cannot establish that files on disk have not changed since analysis.
+
+```ts
+const plan = createRepositoryAgentBenchmarkPlan(snapshot, reviewedCases, {
+  model: {
+    provider: "codex",
+    name: "the-explicit-model-selected-for-the-experiment",
+    settings: { reasoningEffort: "high" },
+  },
+  repetitions: 2,
+  maxChars: 12000,
+});
+```
+
+`reviewedCases` uses the same strict contract as `repository quality`: unique case ID,
+task query, optional repository-relative scope, expected/forbidden files and test names.
+Keep failed retrieval cases in the experiment; removing them after seeing a failure
+biases the comparison. Expected files are context relevance expectations, not a requirement
+to edit those files and not sufficient acceptance criteria for application behavior.
+Use a concrete task query with enough behavior and acceptance detail for independent review.
+
+The JSON-serializable plan contains:
+
+- `binding`: exact source root, snapshot ID, manifest/scenario hashes, and a digest of
+  the analyzed input file hashes and metadata.
+- `model`: explicit provider, model name and scalar settings. Both arms share all settings.
+- `cases`: reviewed context expectations, retained for the evaluator and reviewer.
+- `runs`: two isolated runs per case and repetition. Both share prompt, scope, suggested
+  check declarations and model settings. `with-maps` additionally receives the bounded
+  `repositoryContext` packet. This is the only difference in `workerInput`.
+- `planId`, `taskDigest`, `executionDigest`, and `runId`: deterministic content bindings.
+  Editing a plan invalidates its identity; foreign or altered observations are rejected.
+- `execution: "not-executed"` and `authorization: "external-runner-required"`.
+
+Repetitions are bounded to 1–10 and cases to 1–100. Run order alternates by repetition
+to reduce ordering bias. Model settings include no implicit SDK defaults: declare any
+settings that can affect output, and require the runner to use them consistently.
+
+## External runner contract
+
+First obtain the normal authorization for model costs, repository changes and actual
+application checks. The presence of a check command in a manifest or plan is a suggestion,
+not authorization to execute it. The planner never starts a runner.
+
+For each planned run, the external runner should:
+
+1. Prepare a fresh isolated checkout from the exact analyzed source baseline. Keep
+   originals unchanged. Verify input hashes, manifest/scenario, dependency state and
+   relevant environment before execution. Bind the observation to the original `binding`;
+   isolated checkout paths are runner artifacts, not replacement source bindings.
+2. Create a fresh worker session. Never reuse sessions between arms, cases or repetitions,
+   and prevent shared conversation or generated-artifact leakage between them.
+3. Pass **only that run's `workerInput`** to the worker. Do not pass the whole plan or
+   hidden expected/forbidden context files to the worker. Withhold other map tools from
+   the control arm; provide the same ordinary repository tools and permissions to both.
+   Ordinary agents may still read source and discover the same relationships themselves.
+4. Record SDK/provider counters for actual input/output/cached input tokens, wall-clock
+   latency, and instrumented source-read events. Use evidence references to durable
+   traces or logs. Do not infer tokens from character count, reads from filenames in the
+   response, or zero latency/read counts from missing telemetry.
+5. Preserve the resulting diff/artifact and its SHA-256 digest. Run authorized declared
+   application checks in each isolated checkout and record observed outcomes. A map
+   query succeeding is not an application check. Missing or skipped checks remain unknown
+   or `not-run` and prevent a comparable quality-qualified pair.
+6. Have a distinct reviewer inspect the concrete task, actual resulting artifact and
+   check evidence. Keep the reviewer blind to the arm and metrics where practical.
+   The reviewer must not be any worker in this experiment. Record their verdict against
+   that artifact digest; a static retrieval pass cannot substitute for this review.
+7. Supply one observation per completed, failed or cancelled run. Keep missing runs
+   absent rather than inventing outcomes. Preserve all original evidence artifacts.
+
+The source binding describes the baseline before the worker's edits. Never stamp a
+newer snapshot onto an older result. If the source baseline changes, create a new plan.
+Identical baseline hashes are necessary but insufficient for environment equivalence;
+the runner must record and control dependency versions, network state, tools, limits
+and permissions. The evaluator cannot authenticate those conditions.
+
+An observation has this shape (digest placeholders must be replaced by real values):
+
+```json
+{
+  "planId": "copy-exact-planId",
+  "runId": "copy-exact-runId",
+  "binding": { "root": "copy-source-root", "snapshotId": "copy-snapshotId", "inputDigest": "copy-inputDigest", "manifestHash": "copy-manifestHash", "scenarioHash": "copy-scenarioHash" },
+  "taskDigest": "copy-exact-taskDigest",
+  "executionDigest": "copy-exact-executionDigest",
+  "model": { "provider": "codex", "name": "the-actual-explicit-model-used", "settings": { "reasoningEffort": "high" } },
+  "workerId": "actual-isolated-session-id",
+  "status": "completed",
+  "artifactDigest": "sha256:<actual-64-hex-digest>",
+  "artifactEvidenceRef": "artifact:actual-diff",
+  "metrics": {
+    "inputTokens": 1200,
+    "outputTokens": 400,
+    "latencyMs": 5000,
+    "sourceReads": [{ "path": "src/example.ts", "chars": 800, "evidenceRef": "trace:actual-read-event" }],
+    "evidenceRefs": ["trace:actual-provider-and-runner-counters"],
+    "provenance": "executor-observed"
+  },
+  "checks": [{ "id": "actual-declared-check-id", "status": "passed", "evidenceRef": "check:actual-exit-and-output" }],
+  "review": {
+    "reviewerId": "actual-independent-review-session-id",
+    "artifactDigest": "sha256:<same-actual-artifact-digest>",
+    "outcome": "accepted",
+    "evidenceRef": "review:actual-review-report"
+  }
+}
+```
+
+All metric fields except `evidenceRefs` and `provenance` are optional. Omit unavailable
+counters. The observation's actual reported model/settings must match the declared model,
+including settings. A substituted model or reasoning setting invalidates the comparison.
+`sourceReads: []` asserts the runner observed zero reads; absence means unknown.
+Repeated read events count separately, while `uniqueSourceFiles` deduplicates paths.
+Read character totals remain unknown unless every observed event includes a character count.
+Token counters are nonnegative safe integers; cached input cannot exceed observed input.
+Check statuses are `passed`, `failed`, `not-run`, or `unknown`; executed statuses require
+evidence. Review outcomes are `accepted`, `changes-requested`, or `inconclusive`.
+
+## Evaluate and interpret
+
+```ts
+const report = evaluateRepositoryAgentBenchmark(plan, observations);
+```
+
+The evaluator rejects mutated plans, foreign source roots/snapshots/input digests,
+different tasks/execution settings, duplicate runs, undeclared checks, unsupported
+fields, invalid counters, artifact review mismatches and reused worker identities.
+Reviewers must be independent of all workers. It does not execute or authenticate
+anything: the report labels its provenance `external-runner-reported` and preserves
+the references supplied with each observation for independent verification.
+
+A pair is comparable only when both arms completed, both actual artifacts received
+accepted independent reviews, and all suggested checks declared for that task passed
+with evidence. The report keeps other pairs as `pending`, `execution-incomplete`,
+`review-required`, or `checks-required`. An empty declared check set does not invent
+application coverage: independent task review is still required.
+
+For each comparable pair, differences are **with maps minus without maps**. Negative
+token, latency or source-read differences indicate lower observed values for that pair;
+they say nothing about quality unless its review and checks qualified it. Aggregate
+metrics contain an observed-pair count and mean difference. Missing metrics remain
+`null`, including when a quality-qualified pair has no telemetry. Zero is reserved for
+an explicitly observed zero or an actual equal-value difference.
+
+Treat the report as a reproducible protocol and evidence ledger. Small fixture samples
+do not establish a general benefit, statistical significance, real user acceptance or
+production readiness. Report failures, review disagreements, omitted telemetry, environment
+limitations and the exact tasks/model/settings alongside any measured improvement.

package/docs/repository-analysis.md

@@ -0,0 +1,139 @@
+# Forge maps in existing repositories
+
+Optional [runtime observation](repository-runtime-observation.md) supplements static maps
+with explicitly executed, source-bound generated artifacts. Review `runtime-plan`, opt in
+with `runtime-observe --execute`, and read current evidence with `runtime-context`.
+Static analysis remains free of implicit project execution.
+
+Large snapshot storage uses canonical-order v2 partitions and on-demand graph
+hydration, with eager compatibility for v1. Published analysis performs conservative
+cache maintenance. Use `forge repository cache-gc --json` to preview it, and `--write`
+to apply after retention (24 hours by default). Checks/models remain unexecuted.
+See [context quality and cache contracts](repository-context-quality.md) and
+[paired agent benchmark](repository-agent-benchmark.md) for the detailed guarantees.
+
+Forge repository analysis produces static maps and compact agent context without migrating
+the application to Forge's runtime. Vue/Nuxt, Java/Spring, TypeScript/JavaScript and
+Docker/Compose can coexist in one repository manifest. Analysis is available without Git;
+managed Agent Fabric workflows require a Git repository with an initial commit.
+
+## Prepare a manifest with Codex
+
+Ask the Codex agent to prepare the project for Forge maps. Discovery returns a draft;
+the agent checks component roots and frameworks against project instructions and source,
+then saves the manifest within the authorized edit scope. Discovery does not overwrite
+an existing service manifest.
+
+From the target project, using an absolute shared runtime path:
+
+```text
+node <forge-runtime>/bin/forge.mjs manifest discover --json
+node <forge-runtime>/bin/forge.mjs manifest validate forge.manifest.json --json
+node <forge-runtime>/bin/forge.mjs repository analyze --write --json
+node <forge-runtime>/bin/forge.mjs repository context --query routes --json
+node <forge-runtime>/bin/forge.mjs repository context --query infrastructure --json
+node <forge-runtime>/bin/forge.mjs repository context --query coverage --json
+node <forge-runtime>/bin/forge.mjs repository quality --cases <absolute-reviewed-cases.json> --json
+```
+
+Discovery without `--write` and validation are read-only. Analysis with `--write` explicitly produces local artifacts;
+no application generation, runtime dependency, hook installation, Docker startup or Java
+build is required. The runtime does not need to reside in the target repository. Use
+`--root` for an explicit target, including directories without package.json or Git.
+
+Repository manifests use protocol 2.0 and a repository discriminator. Service manifests
+using protocol 1.0 remain separate executable contracts and retain compatibility.
+
+```json
+{
+  "forgeProtocol": "2.0",
+  "kind": "repository",
+  "components": [
+    { "id": "web", "root": "frontend", "adapters": ["vue", "typescript"] },
+    { "id": "api", "root": "backend", "adapters": ["java", "spring", "maven"] },
+    { "id": "infra", "root": ".", "adapters": ["docker"], "files": ["compose.yaml", "backend/Dockerfile"] }
+  ],
+  "checks": [{ "id": "web-tests", "component": "web", "argv": ["npm", "test"] }]
+}
+```
+
+The manifest describes analysis configuration, not every function. Roots and files are
+relative to the target project. Checks are suggestions and never run during discovery,
+analysis or queries. `.forge/fabric.json` continues to configure managed execution;
+repository manifest checks do not override explicit workflow acceptance obligations.
+
+## Understand evidence and coverage
+
+The graph records nodes, relations, locations, source hashes, adapter versions and
+assurance (`declared`, `syntactic`, `resolved`, `inferred`). The coverage report describes
+ignored and unsupported files, diagnostics, unresolved behavior and adapter limitations.
+
+Vue maps expose components, script symbols and visible bindings. Java maps expose syntax,
+Spring endpoints and build declarations. Docker maps expose declared builds and Compose
+topology. None of these constitute observed runtime behavior: a port is not a health check,
+`depends_on` is not proof of traffic, and static test association is not executed coverage.
+Dynamic configuration, Java reflection and Vue autoimports can remain unresolved.
+Static Nuxt aliases, local autoimports and supported HTTP clients have additional
+resolution paths. Ambiguous candidates remain visible. Test associations are case-level;
+sharing a test file does not establish that all cases exercise an imported symbol.
+See [repository context quality](repository-context-quality.md) for the declaration
+schema, retrieval rules, cache limits and reviewed task benchmark.
+
+Local TypeScript/JavaScript config inheritance, project references, baseUrl and path aliases
+are read statically inside the selected repository. Package-provided executable configuration
+and external inheritance remain unresolved. Vue conditional template branches are included;
+Docker instructions continued across physical lines are joined before extraction.
+
+Real `.env` files and credential files are excluded. Unknown environment interpolation
+remains unknown. Analysis does not run scripts, executable config, Gradle/Maven wrappers,
+Docker or arbitrary external adapters. Source snippets are not emitted by default.
+
+## Query current snapshots
+
+Queries include overview, locate, symbol, references, routes, dependencies, impact, tests,
+infrastructure and coverage. Use exact node IDs to disambiguate repeated names. Compact
+symbol handles are local to a snapshot and require its snapshotId; they cannot be carried
+to a new snapshot. Sources changed after analysis make the result stale. Reanalyze before
+requesting definitions or applying map evidence to a task.
+
+Repository context accepts `--limit` from 1 to 100 and `--max-chars` from 2048 to 50000.
+The same limits apply to MCP context tools. MCP analysis returns bounded coverage samples
+with omitted counts; saved snapshots retain the complete analysis report. Cached graphs are
+validated for structure, identity, containment and source provenance before queries.
+
+Artifacts normally reside in `.forge/repository/`, outside generated application code.
+No `.gitignore` is edited automatically. A host-local cache can keep outputs outside the
+project. Omit `--write` when only an ephemeral analysis result is authorized. Parsing
+reuse is based on content hashes; graph identity also includes manifest and scenario.
+
+The CAIR repository provider serves read queries. Runtime-specific or mutating CAIR
+actions are unavailable for this provider. An impact map describes potential dependencies
+and does not authorize skipping required tests.
+
+## Use with Agent Fabric
+
+Install the personal Forge Agent Fabric skill once. Its `scripts/repository.mjs` helper
+uses the installed shared runtime with an explicit target directory and does not require
+Git. The original `scripts/fabric.mjs` helper retains the Git requirement for execution.
+No per-project runtime dependency, hook or MCP installation is required for maps.
+
+When a managed SDK attempt contains a valid repository manifest, Fabric analyzes its
+prepared clone after composing upstream artifacts and preparing dependencies. A bounded
+packet contains scoped nodes and edges, coverage limitations, clone snapshot identity and
+suggested checks with argv/cwd, reasons and requirement status. Task neighbors are
+read-only unless already inside write scope. Static quality counters remain partial;
+`ready` means the packet was prepared. The run status records source project and clone identities separately,
+with phase `prepared-input`; it does not claim the same map describes subsequent worker edits.
+Reviewers receive a fresh snapshot of the combined upstream changes, not the map from the
+original checkout. Matching parse facts can be reused from the source cache while graph
+identity and connections are rebuilt for the clone. Analysis writes no cache into managed clones.
+
+A missing manifest leaves existing workflows unchanged. Invalid configuration or
+unavailable analysis is reported as unavailable; the worker must inspect current code
+directly. It never receives a purported fallback graph from another checkout. Maps do not
+alter write scopes, independent review, explicit command verification or publication gates.
+Publication still means applying scoped local changes, not Git push, npm or deployment.
+
+MCP queries can use projects registered with Fabric; select the project explicitly.
+Each root and snapshot remain distinct, including separate Windows/WSL copies. Cross-project
+queries do not assert a simultaneous global snapshot or automatically synchronize checkouts.

package/docs/repository-context-quality.md

@@ -0,0 +1,179 @@
+# Repository context quality for Agent Fabric
+
+Forge can map an existing repository without converting it into a Forge application. The Codex chat remains the interface; a repository manifesto declares the analysis boundary and optional checks. Analysis reads source and static configuration. It does not evaluate Nuxt/Vite configuration, start Docker, install packages, call remote APIs or execute the checks.
+
+## Implementation plan and boundaries
+
+1. Capture a managed baseline and editable task scope. Carry readonly descriptors/lockfiles with the captured input. Preserve source changes outside that scope.
+2. Apply upstream artifacts in an isolated clone and prepare its authorized environment. Regenerate repository context from that prepared clone, rather than reusing the source snapshot. Bind the clone identity, source digests, manifest and scenario to the snapshot.
+3. Resolve static alias and Nuxt declarations inside each component. Give explicit manifesto declarations precedence where automatic static discovery cannot resolve dynamic configuration. Keep unresolved or ambiguous references visible; avoid inventing imports from names alone.
+4. Link local HTTP client calls to routes only with supporting evidence: known client declaration, literal method/path and compatible API component. Preserve unresolved calls when hosts, path construction or multiple candidates make the connection uncertain.
+5. Select graph context relevant to the task's files/nodes/components. Preserve snapshot binding, paths, locations, relation evidence and gaps when truncating. The snapshot retains source hashes; the compact packet refers back to it. A compact packet must say what was omitted.
+6. Select relevant declared checks and explain why. Include their original argv, explicit working directory, cost/category and requirement status. The selector does not authorize or execute them.
+7. Report preparation, freshness, coverage and relation assurance separately. Verify the final clone packet with fixtures for Nuxt, HTTP clients, partial graphs and check relevance. A paid agent run and a real application test require their own evidence.
+
+Changes to the manifesto are optional extensions of protocol `2.0`, kind `repository`. Existing components, checks and the executable service protocol `1.0` retain their previous contracts. Missing extensions do not mean a framework feature is unsupported; they mean no extra declaration was supplied.
+
+## Static analysis declarations
+
+```json
+{
+  "forgeProtocol": "2.0",
+  "kind": "repository",
+  "components": [
+    {
+      "id": "web",
+      "root": "web",
+      "adapters": ["vue", "nuxt"],
+      "analysis": {
+        "aliases": { "@": "src", "~": "src" },
+        "nuxt": {
+          "rootDir": ".",
+          "srcDir": "src",
+          "components": ["src/components"],
+          "imports": ["src/composables"]
+        }
+      }
+    },
+    { "id": "api", "root": "api", "adapters": ["java", "spring"] }
+  ],
+  "httpClients": [
+    {
+      "id": "apiClient",
+      "component": "web",
+      "files": ["src/services/**"],
+      "basePath": "/api",
+      "apiComponent": "api"
+    }
+  ],
+  "checks": [
+    {
+      "id": "web-typecheck",
+      "component": "web",
+      "argv": ["npm", "--prefix", "web", "run", "typecheck"],
+      "files": ["src/**"],
+      "category": "typecheck",
+      "cost": "low",
+      "requires": ["node"]
+    },
+    {
+      "id": "api-integration",
+      "component": "api",
+      "cwd": "api",
+      "argv": ["./mvnw", "verify"],
+      "category": "integration",
+      "cost": "high",
+      "requires": ["java", "docker", "network"]
+    }
+  ]
+}
+```
+
+All `analysis` paths, alias targets and HTTP client `files` are relative to `component.root`. In particular, `srcDir` is relative to the component root, **not** to `rootDir`. Component roots remain repository-relative. These declarations provide a static interpretation; they do not promise complete Nuxt runtime equivalence.
+
+Aliases have at most 100 entries. Names are bounded to 128 characters, reject traversal segments and prototype keys, and map to contained relative paths. Nuxt declarations accept only `rootDir`, `srcDir`, `components` and `imports`; each directory list has at most 100 entries. Unknown fields, scripts, absolute paths, Windows streams, backslashes, control characters and `..` segments are rejected. Consumers must preserve physical containment and refuse symlink traversal when reading resolved files.
+
+An HTTP client `id` identifies the local variable/function used by the source adapter, such as `apiClient`, `$fetch` or `axios`. IDs are unique within their component and have at most 80 characters. `files` scopes that declaration to component-relative files/globs. `apiComponent` names an existing component and scopes possible route targets; it is an explicit declaration, not proof that a call reaches that service at runtime. `basePath` contains only a literal URL path beginning with `/`, at most 2048 characters. Origins, credentials, query strings, fragments, percent escapes and traversal are rejected. The manifesto never contains headers, environment values, credentials or executable client configuration.
+
+The runtime validator also checks component references and duplicate identities. JSON Schema validates structure and bounds; reference/identity checks remain the runtime validator's responsibility. Manifest files retain the existing 256 KiB reading limit.
+
+## Check relevance contract
+
+`selectRepositoryChecks(snapshot, task)` is a pure selection API. `task` accepts optional repository-relative `scope`, exact `nodeIds`, component IDs and caller-reported `capabilities`. It returns `checks`, `diagnostics` and `execution: "not-executed"`. The caller should pass a freshly validated snapshot; the pure selector itself does not read files or verify freshness.
+
+Checks retain required `id`, `component` and `argv`. Optional fields are:
+
+| Field | Meaning |
+| --- | --- |
+| `cwd` | Repository-relative working directory. Omitted means `.`; old argv with `--prefix web` keeps its original interpretation. |
+| `files` | Component-relative files/globs limiting relevance. Empty/omitted means component-wide. This is not permission to edit those files. |
+| `category` | `typecheck`, `test`, `lint`, `build`, `integration`, `security` or `other`. |
+| `cost` | Declared `low`, `medium` or `high`; low-cost declarations sort first. No cost is measured or presumed free. |
+| `requires` | Up to 32 unique symbolic capability names, such as `java`, `docker` or `network`. No values or environment assignments. |
+
+A component is relevant when explicitly selected, touched by task scopes/nodes, or directly adjacent to a selected node through a complete declared/resolved edge. Inferred or unresolved edges do not automatically expand check selection. File-limited checks must overlap a relevant file or scope, unless the user explicitly selected the whole component. Directory/glob scopes conservatively include future/unindexed files. This may suggest extra checks; it does not prove those checks cover the task.
+
+Every suggestion contains reasons such as `task-component:web`, `task-node-component:web`, `related-node-component:api`, `task-scope-overlaps-component`, `check-files-match-task` or `check-files-overlap-scope`. Unknown node/component references produce diagnostics. Unsafe task scopes or invalid manifests return no suggestions. Empty task selectors return no suggestions.
+
+Requirements report `satisfied` when there are no requirements or the caller explicitly reports all of them; `missing` when the caller's capability list omits required entries; and `unknown` when capabilities were not provided. The selector never probes executables, reads environment variables or infers network authorization. Even `satisfied` means only the declaration was satisfied by caller-provided information, not that the command ran successfully.
+
+The consumer must show argv as an array, without shell concatenation, and retain `execution: "not-executed"`. A future execution tool must separately bind the checkout, validate physical cwd containment, apply environment/network/cost authorization and record exit status/output against the input digest. A check named `test` is not evidence until it actually executes successfully on the relevant bytes.
+
+## Context quality contract
+
+`ready` describes whether a usable packet was prepared. `prepared-input` describes the clone phase from which that packet was generated. Neither describes semantic completeness or successful application verification.
+
+A packet should expose these independent dimensions:
+
+| Dimension | Required interpretation |
+| --- | --- |
+| Preparation | Absent/invalid manifest, unavailable analyzer, successful packet or bounded failure. Errors must remain visible. |
+| Input phase | Source, composed upstream input or prepared input, with clone identity and snapshot binding. |
+| Freshness | Source/config/manifest inventory matches the analyzed snapshot, or stale/mismatch status. |
+| Coverage | Found/analyzed/ignored/unsupported/error counts and visible limitations. |
+| Relation assurance | Declared, syntactic, resolved or inferred, together with complete/partial/unresolved resolution. |
+| Budget | Packet truncation and omitted context. A small packet is not the entire graph. |
+| Verification | Suggested checks versus independently observed execution results. |
+
+Avoid reducing these dimensions to one percentage or a generic “complete” flag. For example, a prepared-input packet can be fresh and usable while Java reflection, Nuxt runtime injection or dynamic HTTP URLs remain unresolved. Explicit manifesto hints improve navigation but remain declared evidence.
+
+## Validation and remaining limits
+
+### Case-level test evidence
+
+Test suites and executable cases are separate nodes. Importing a module establishes a `test-file-depends-on` edge between files; it does not associate every case in that file with every imported function. `test-references` means a reference inside that case's callback. `test-exercises` means a statically resolved call, including named import aliases and bounded same-file helper chains (three levels, at most 128 visited nodes). Both carry `observedCoverage: false`. A conventional test filename supplies navigation only. Test queries follow precise associations and exclude unrelated sibling cases.
+
+### Static Nuxt and HTTP resolution
+
+Literal Nuxt `rootDir`, `srcDir`, aliases, component directories and import directories are read through the syntax tree. Executable expressions remain unknown. Component and composable candidates come only from scanned local files, and multiple candidates remain ambiguous. External UI packages, module-generated imports, layers and runtime injection can remain unresolved. Manifest analysis paths provide contained, explicit overrides. TypeScript compiler aliases and explicit relative imports keep their precedence.
+
+HTTP analysis recognizes immutable local constants, bounded templates, Axios and `$fetch` factories, Nuxt-provided clients and simple returned-call wrappers across imports. A template can additionally retain a separate path pattern when each unknown placeholder occupies a whole segment and comes from a numeric parameter or an encoded scalar parameter. The URL remains dynamic; only a unique compatible endpoint can become a partial candidate link. Dynamic hosts/query strings, embedded or unbounded string expressions and ambiguous routes remain unresolved. Calls belong to the nearest containing function or arrow, including calls initializing local response variables. An absolute external origin needs an explicit API-component declaration before it can become a local service association. Unknown runtime base URLs and document-relative URLs remain unresolved. Manifest client mappings carry declared evidence with `runtimeRoutingVerified: false`; they cannot establish what a deployed reverse proxy or environment actually does.
+
+### Task retrieval and write scope
+
+`selectRepositoryContext` uses deterministic ranking for exact symbols, IDs and paths, normalized English/Portuguese task terms and declared write scope. Exact targets suppress expansion through generic instruction verbs. The selector reserves space for actual consumers and precise tests, then follows at most two relevant graph hops, including function-to-HTTP-call-to-endpoint paths. Shared test files and imports are not bridges to unrelated cases. Java call sites are grouped for broad locate queries; expanding a group requires its snapshot binding.
+
+Each selected node includes location, assurance, selected metadata, selection reasons and `access: write-scope | read-only`. Neighbor inclusion never changes the scope. Relations and groups retain their endpoints when the packet is truncated. Node/edge counts, character budgets and omitted counts are explicit. Fabric uses a 10,000-character selection budget and a 16,000-character packet budget, includes relevant check argv/cwd as informational data, and records static quality in run metadata. Preparation failures report `unavailable`, including failures after successful analysis.
+
+### Partitioned cache and prepared clones
+
+Analyzer version `1.2.1` invalidates incompatible parse facts. Small snapshots retain legacy JSON storage. Large graphs use a bounded index plus content-addressed, hash-checked partitions; source facts are partitioned by file. Snapshot storage v2 partitions preserve canonical ID order, enabling sequential validation and scans. V1 component partitions remain readable through the eager compatibility path. Publication uses the existing analysis lock and an atomic header rename, so an incomplete write does not publish an incomplete graph. Readers reject corrupt, oversized, missing or unsafe partitions.
+
+The v2 reader builds compact ID/position and relation indexes, then hydrates nodes/edges on demand through array-compatible read-only collections. A two-partition LRU limits retained decoded partitions within that cache. Full integrity, structure, root ownership and canonical fingerprint validation still reads every current partition before exposing the graph; source freshness is checked separately as before. Streaming canonical JSON hashing avoids building a second complete graph and serialization string. Relation sorting uses compact summaries, and budget-impossible neighbors are rejected before hydration. `repositoryStorageMetrics(snapshot)` reports actual partition reads/bytes and peak **serialized resident payload**, not decoded heap. Candidate lists, broad query results and callers retaining nodes can consume additional memory; this is not a promise of constant memory for arbitrary queries or a database with a full-text index. Returned node objects retain the previous mutable data contract, while partition-backed arrays reject mutations.
+
+Successful `repository analyze --write` performs conservative cache maintenance automatically. Identical published headers remain stable apart from the creation time of a new in-memory result; unchanged cache generations keep their original `createdAt`. The first warm run may update reuse statistics. Library callers can disable maintenance with `cacheCleanup: false`. A maintenance failure preserves the successful publication and exposes `REPOSITORY_CACHE_CLEANUP_DEFERRED`; it never reclaims a foreign lock.
+
+`repository cache-gc --json` previews cleanup; adding `--write` applies it. The collector preserves all current snapshot/fact references, validates them before mutation, rejects symlinked inventories and journals first-observed orphan identities. Default retention is 24 hours (`--grace-hours 1..720`), with a one-hour minimum. It deletes only chunks continuously observed orphaned through that window. Any header replacement restarts the window conservatively; frequent changing inputs may delay reclamation. Readers that remain alive beyond the configured grace need external coordination or longer retention. Unknown regular files are retained. This command manages generated cache chunks only and never deletes repository source.
+
+Managed clones may reuse read-only source parse facts only when path, component, adapter, analyzer version and source digest match. The graph is rebuilt for the actual clone, current manifest and scenario. A changed source file is reparsed. Facts do not supply source-root graph identities or relax the clone freshness check, and preparation writes no analysis cache in the clone. This is an optimization within the host user's trust boundary, not a sandbox against a malicious host user.
+
+### Reviewed task benchmark
+
+Create a reviewed JSON array of 1–100 tasks. Names and file paths are bounded and duplicate identities/expectations are rejected. Include required files/tests and explicitly forbidden associations:
+
+```json
+[
+  {
+    "id": "seo-homologation",
+    "query": "Corrigir validação SEO em homologação",
+    "scope": ["web/utils/seoEnvironment.ts"],
+    "expectedFiles": ["web/utils/seoEnvironment.ts", "web/nuxt.config.ts", "tests/seo.test.ts"],
+    "expectedTests": ["unsafe SEO in homologation"],
+    "forbiddenTests": ["unrelated checkout price"]
+  }
+]
+```
+
+After explicitly creating the current analysis cache, run from the target repository:
+
+```text
+node <forge-runtime>/bin/forge.mjs repository quality --cases <absolute-reviewed-cases.json> --json
+```
+
+The command reads current snapshots, rejects stale source evidence, and never executes checks or agents. Exit zero means every supplied expectation passed; one means a benchmark/validation failure. Reports show missing/forbidden results, context characters, truncation, recall and precision against the declared file set. That precision is meaningful as total retrieval precision only when the reviewed set is exhaustive; legitimate undeclared neighbors lower it. A passing fixture is not proof that all repository relationships are resolved. Synthetic regression tasks cover SEO, checkout, scheduling, Vue consumers and frontend-to-Java HTTP links.
+
+The paired agent experiment is implemented as a planner and evidence evaluator: `repository benchmark-plan --cases tasks.json --model-config model.json --repetitions 2 --json`, followed by `repository benchmark-report --plan plan.json --observations observations.json --json`. The planner binds identical prompts, scopes, models/settings and declared checks to source, manifest, scenario and snapshot identity; only the map packet differs between arms. The report accepts observed token/latency/source-read metrics, artifact-bound independent review and authorized check results. Missing evidence remains unknown and metric differences require comparable completed pairs. Neither command executes a model or an application check, and exit zero on a valid pending report is not an experiment success. See [the runner protocol](repository-agent-benchmark.md) for JSON contracts, counterbalancing and the external authorization boundary. No productivity results are claimed without actual runs.
+
+Fixtures cover legacy/new manifesto validation, JSON Schema structure, traversal/unknown fields, declaration bounds, scoped HTTP identities, check selection by files/nodes/components, missing/unknown requirements and unchanged argv/cwd interpretation. Integration must additionally prove that the Fabric packet is rebuilt after upstream composition and environment preparation, and that source/manifest changes invalidate reused context.
+
+Static repository analysis is not a Nuxt build, Java compiler, Docker scenario execution, network trace or production test. Same-user clones isolate ordinary edits but are not a security sandbox against an adversarial process with access to the host. Source repository publication and paid Codex workers remain governed by the existing Fabric workflow. EasyGrow validation must run only in a dedicated copy; its original repositories remain read-only.

package/docs/repository-runtime-observation.md

@@ -0,0 +1,82 @@
+# Repository runtime observation
+
+## Goal and scope
+
+Forge can analyze an existing Vue/Nuxt, Java/Spring or Docker repository without converting it into a Forge application. Static maps remain useful before installing dependencies or starting the application. The optional runtime layer supplements those maps with artifacts produced by explicit commands in an independent temporary source copy.
+
+The runtime layer answers questions such as which components Nuxt registered after module preparation, which Spring beans and mappings a selected profile exposed, which containers and networks an exported Docker inspection described, and which HTTP paths a recorded execution requested. It does not claim comprehensive execution coverage, production correctness, authentication of external services, or that a generated Nuxt declaration proves a rendered component.
+
+## Implementation plan
+
+1. Extend protocol 2.0 repository manifests with optional `runtime.observations`. Each observation identifies a component, a sequence of executable argv arrays, contained working directories, per-command timeouts and a list of supported artifact formats. Reject unknown fields, duplicate identities, excessive inputs and unsafe paths. Commands are reviewed configuration, never inferred execution authority from static checks.
+2. Implement pure, bounded collectors for Nuxt declarations, Spring Actuator mappings/beans, Docker inspection, HTTP traces and normalized Forge observations. Collect useful structural fields. Omit environment values, authorization, headers, request/response bodies, credentials, origins and query parameters. Malformed or oversized artifacts must not produce confident evidence.
+3. Build a read-only plan API. Show exactly which commands and artifacts are requested, the input binding and limits before execution. Planning must not create an owner, start an application, install dependencies, run a model or write a cache.
+4. Execute only with explicit runtime authorization. Copy eligible current sources into a unique temporary directory, excluding Git metadata, secrets, dependency directories and Forge caches. Execute finite commands with `shell:false`, bounded output and time, a restricted environment and cleanup of the process tree owned by the invocation. Source copy isolation protects the original checkout from ordinary relative writes; it is not an operating-system, network or container sandbox.
+5. Collect only fresh, contained regular artifacts from the invocation. Bind hashes, command statuses, observation identities, component, source inventory, manifest, scenario, environment label and timestamps into a digest-protected report. A failed command cannot contribute successful facts. Recheck source freshness before publication and fail closed on change.
+6. Save reports only with explicit `--write`. Read them without executing anything, verify structure, digest, source binding and expiration, and return a bounded context packet. File hashes establish integrity, not the authenticity or semantic correctness of a project-generated observation.
+7. Expose the same operations through CLI, portable skill and MCP. `runtime-plan` is read-only, `runtime-observe` requires `--execute`/`execute:true`, and `runtime-context` is read-only. Existing static context, quality checks and model benchmarks continue to report their own limited evidence.
+8. Let Agent Fabric consume compatible saved runtime evidence as an additional packet. Do not start runtime commands automatically while preparing a worker. Do not silently reuse an observation when composed upstream changes, manifest, scenario or full copied-input digest differ. Runtime context does not widen write scope or satisfy review/publication gates.
+9. Test actual child-process execution, fresh artifacts, failures, timeout/cancellation, process cleanup, secret omission, stale input and report corruption. Test formats against representative generated exports and distinguish fixtures from observations of a deployed system. Run repository tests, typecheck, generator/check, framework verification and package smoke before publication.
+
+## Manifest example
+
+```json
+{
+  "forgeProtocol": "2.0",
+  "kind": "repository",
+  "components": [{ "id": "store", "root": "frontend-store", "adapters": ["typescript", "vue", "nuxt"] }],
+  "scenario": { "id": "local-observation", "mode": "development" },
+  "runtime": {
+    "observations": [{
+      "id": "store-generated-registry",
+      "component": "store",
+      "commands": [
+        { "argv": ["npm", "ci", "--ignore-scripts"], "cwd": "frontend-store", "timeoutMs": 120000 },
+        { "argv": ["node", "node_modules/nuxt/bin/nuxt.mjs", "prepare"], "cwd": "frontend-store", "timeoutMs": 120000 }
+      ],
+      "artifacts": [
+        { "path": "frontend-store/.nuxt/components.d.ts", "format": "nuxt-components" },
+        { "path": "frontend-store/.nuxt/imports.d.ts", "format": "nuxt-imports" }
+      ]
+    }]
+  }
+}
+```
+
+Command `cwd` is repository-relative and defaults to the observation's component root. Artifact paths are always repository-relative. Use the package manager and lockfile actually present in the project. Dependency installation is an explicit command, runs only in the copy and can require network access. Installation scripts are disabled in this example; `nuxt prepare` then runs the reviewed project configuration. On Windows, prefer a native executable or `node` plus the absolute npm CLI JavaScript path when `npm` resolves only to a `.cmd` wrapper: `shell:false` intentionally does not evaluate a command shell. Do not put tokens, passwords or environment values into argv.
+
+Collectors do not export Spring/Docker data themselves. Declare a reviewed project script that starts the local test application if needed, exports only the intended artifacts, closes its resources and exits. A Maven/Gradle wrapper can run a project-specific integration/export task; Spring Actuator must be configured by the project for that scenario. For Docker, use a unique Compose project and an export script that tears down only its own resources in `finally`. Forge never shuts down unrelated containers or a developer's existing services.
+
+Windows commands run in an owned Job Object. POSIX commands run in an owned process group; a program that deliberately starts another session can escape that group. Exporters must remain finite and close any detached children, servers and container resources they create. This layer does not promise containment of untrusted code.
+
+For HTTP, export a HAR or the supported normalized request list from a test execution. Only method, sanitized path and response status enter agent context. Forge does not replay the traffic, inspect authorization headers, or infer the backend process from an unverified hostname.
+
+## Usage from the Codex chat
+
+```sh
+forge repository runtime-plan --root /path/to/repo --environment-id local-test --json
+forge repository runtime-observe --root /path/to/repo --environment-id local-test --execute --write --json
+forge repository analyze --root /path/to/repo --write --json
+forge repository runtime-context --root /path/to/repo --environment-id local-test --query checkout --json
+```
+
+Review the proposed manifesto and execution commands in the chat. Ask Codex to run the observation when the commands, selected scenario and external effects are authorized. No per-repository hook or MCP installation is needed for the CLI/portable skill. A configured shared Forge MCP can expose the same operations for its selected project. Updating a runtime package does not automatically activate hooks or alter global Codex configuration.
+
+Run static analysis again after changing the manifest. Runtime context requires a current saved static snapshot and an unexpired report. Runtime commands do not update static caches implicitly. Do not change the original EasyGrow checkout to test this flow: use the dedicated pilot copy or a new independent test copy.
+
+## Evidence and freshness
+
+- **Nuxt declaration:** confirms a generated component/import registry for the preparation scenario; it does not prove the application rendered the component.
+- **Nuxt import paths:** relative declarations can omit the source extension. They remain artifact-relative references; bare package imports and virtual aliases such as `vue` and `#app` are omitted.
+- **Spring exported mappings/beans:** describes the exporter result for that invocation and profile; it does not prove every conditional request path executed.
+- **Docker inspection:** describes the exported service/container state; it does not prove application health, correct credentials or production connectivity.
+- **HTTP trace:** records the supplied test execution's request path/status. Origins and queries are omitted, so destination identity and parameter-dependent behavior remain limitations.
+- **Normalized Forge artifact:** describes selected project-exported facts. Extra arbitrary details are excluded; this is not a general telemetry ingestion API.
+
+The environment identifier is a human-selected label, not a fingerprint of every installed dependency, database or remote service. Reports expire and input changes invalidate them, but an external service can change sooner; collect again after changing the environment. Maps and reports remain partially complete and are aids for investigation, not a replacement for source inspection or application tests.
+
+Reports expire after 15 minutes. The CLI/MCP plan shows a root-neutral digest of the complete eligible source copy, including dependency lockfiles and preparation scripts outside the static component globs. The static snapshot ID alone does not bind every preparation input. Runtime copy limits are 50,000 entries, 16 MiB per file and 256 MiB total; dependency and generated directories are excluded. Artifact input is limited to 2 MiB, reports to 4 MiB and command execution to five minutes total. Update the scenario when changing profiles or topology, and recollect after environment changes.
+
+## Publication checklist
+
+Keep alpha.69 unpublished until all validation gates and package-content checks pass. Include this document, schema and runtime modules in the package. Preserve existing alpha.69 changes and regenerate derived artifacts through the compiler. Record the current turn's logs and independent-copy pilot results separately from earlier static-map validation. Do not initiate npm authentication or publish as part of preparation.