agent-bios 0.1.0

package/claude/guides/llm-capability-boundary.md

@@ -0,0 +1,255 @@
+---
+guide_id: llm-capability-boundary
+language: en
+status: active
+use_when:
+  - designing structured outputs
+  - changing runtime-owned artifacts
+  - defining submit tools or accepted output channels
+  - separating LLM semantic judgment from deterministic execution
+  - designing validators, grounding checks, or capability-surface constraints
+  - designing tool use, retrieval, side effects, or artifact persistence
+core_rules:
+  - instructions describe intended work, semantic criteria, and completion criteria
+  - capability surface enforces constraints through context, tools, permissions, routes, paths, output channels, validators, gates, and approvals
+  - assign each field or operation one primary authority, then add layered checks where needed
+  - LLM handles semantic judgment, rationale, tradeoffs, and evidence reduction
+  - runtime/tools handle deterministic execution, artifact creation, merge, serialization, validation, persistence, and tests
+  - machine-consumed artifacts are created through submit tools or equivalent constrained channels
+  - provider strict schema is an execution aid, not the source of artifact truth
+  - tool inputs, tool results, retrieved content, and rendered views are untrusted until validated or sanitized
+field_assignment:
+  short_closed_values: provider_closed_selection_plus_runtime_enum_validation
+  runtime_known_ids: runtime_owned_unless_selection_is_semantic
+  long_refs_and_source_snippets: runtime_allowed_set_validation
+  evidence_anchors: grounding_blocked_when_source_truth_is_decidable
+  source_provenance: runtime_owned_snapshot_hash_scope_and_trust_tier
+  open_rationale: free_generation_with_shape_checks
+  side_effects: capability_surface_plus_policy_gate
+  artifact_envelope_and_serialization: runtime_only
+verification_focus:
+  - accepted output channel is explicit
+  - LLM cannot create canonical machine artifacts through free prose
+  - runtime-owned fields and unknown fields fail loudly
+  - provider schema limits are known and probed before relying on them
+  - long refs are validated by runtime allowed sets instead of provider enums
+  - grounding gates are used only for decidable truths
+  - source trust, permission, staleness, and poisoning risk are checked separately from quote grounding
+  - side-effect tools are classified by risk and gated by permission or approval
+  - artifact writes are atomic, idempotent where possible, and auditable
+  - schema, validator, allowed-set, prompt, and tests share one authority or have drift-catching tests
+---
+
+# LLM And Capability Boundary Guide
+
+Use this guide when an LLM produces, selects, validates, summarizes, or routes
+machine-consumed artifacts, code, ontology, pipeline outputs, review findings,
+structured documents, or runtime decisions.
+
+The core lesson is simple: instructions describe the intended work; the
+capability surface makes the valid execution path available, bounded, accepted,
+and observable.
+
+Scoped extensions:
+
+- For enforcement mechanics — submit tools, runtime-owned fields, output
+  channel locks, provider schema use, allowed-set validation, grounding and
+  provenance, deterministic projection, security and side effects,
+  persistence and retry, and schema evolution — read and use
+  `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/llm-capability-boundary-patterns.md`.
+- For worked case studies applying this boundary, read and use
+  `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/llm-capability-boundary-examples.md`.
+
+## Core Model
+
+- Instructions describe intended work, semantic criteria, tradeoffs, and
+  completion criteria.
+- The LLM performs semantic work: intent clarification, meaning assignment,
+  materiality and causality judgment, rationale writing, option comparison,
+  and evidence reduction.
+- Runtime/tools perform authoritative mechanical work: parsing, counting,
+  calculation, API calls, deterministic merge, serialization, validation,
+  persistence, tests, and diff comparison.
+- The capability surface provides structural constraints: accessible context,
+  available tools, permissions, execution routes, artifact paths, accepted output
+  channels, validators, retry policy, approval gates, and failure behavior.
+- Canonical artifacts are created by runtime/tools, not by raw LLM prose, when
+  downstream systems consume them.
+
+Prefer this framing:
+
+> The LLM may propose semantic content. The runtime decides what becomes
+> artifact truth.
+
+Use stronger prompts to clarify meaning. Use capability design to constrain
+behavior that affects correctness, reproducibility, artifact truth, privacy,
+security, or side effects.
+
+## Capability Surface
+
+The capability surface is the set of actions the LLM can actually take and the
+outputs the runtime will actually accept. Shape it before relying on the LLM to
+follow a rule.
+
+Important levers:
+
+- Accessible context: files, refs, rows, artifacts, source snippets, projections,
+  and retrieved evidence the unit may inspect.
+- Available tools: read tools, submit tools, validation tools, search tools, API
+  tools, renderers, and their input schemas.
+- Permissions: read-only, workspace-write, denied paths, network access,
+  sandbox mode, route-specific side effects, and user approval.
+- Execution routes: tool-capable executor, text-only executor, structured output
+  route, direct-call route, deterministic runtime route, or human-review route.
+- Artifact paths: exact output paths, canonical truth locations, temp paths,
+  and denied write locations.
+- Accepted output channels: submit tool call, structured JSON payload, runtime
+  projection, generated YAML, markdown view, or final prose.
+- Validators and gates: schema validation, unknown-field rejection,
+  runtime-owned-field rejection, enum validation, allowed-set validation,
+  grounding checks, provenance checks, citation checks, static checks, E2E
+  checks, and semantic quality gates.
+- Retry/fail policy: retry transient generation failures; fail clearly when the
+  available route cannot enforce the required contract.
+- Observability: prompt packet snapshot, model/provider version, schema hash,
+  source snapshot, validator decision, retry reason, and artifact lineage.
+
+## Boundary Decision Table
+
+| Need | Primary authority | Preferred mechanism |
+|---|---|---|
+| Clarify user intent or product meaning | LLM | Prose reasoning and decision framing |
+| Choose tradeoffs or materiality | LLM | Bounded semantic judgment with evidence |
+| Produce open rationale or explanation | LLM | Free generation with shape constraints when needed |
+| Inspect files or fresh facts | Tools/runtime | Search, parse, read, API call, source snapshot |
+| Create canonical machine artifact | Runtime | Submit payload plus runtime serialization |
+| Assign ids, paths, metadata, timestamps | Runtime | Runtime-owned fields |
+| Merge artifacts by explicit rule | Runtime | Deterministic projection or merge |
+| Validate syntax, schema, refs, counts | Runtime | Parser, schema, allowed-set, tests |
+| Check source-span truth | Runtime | Grounding gate when decidable |
+| Judge source trust or completeness | Runtime plus policy | Provenance, permission, staleness, trust tier |
+| Prevent forbidden action | Capability surface | Make it unavailable, invalid, or unaccepted |
+| Perform side effect | Capability surface plus policy | Risk class, permission, approval, audit log |
+| Structured output required | Capability surface plus runtime | Submit tool or equivalent constrained channel |
+
+## Structured Output Field Assignment
+
+For each artifact field, assign one primary authority. Add layered checks for
+cross-field invariants, security policy, privacy policy, and artifact-level
+consistency.
+
+| Field kind | Primary mechanism |
+|---|---|
+| Short closed values | Provider closed selection plus runtime enum validation |
+| Runtime-known ids | Runtime-owned; provider enum only when LLM must select |
+| Long refs and source snippets | Runtime closed validation; provider enum excluded |
+| Evidence anchors | Grounding-blocked when source truth is decidable |
+| Source provenance | Runtime-owned snapshot, hash, scope, trust tier, staleness |
+| Open materiality or causal rationale | Free generation with structured shape checks |
+| Side-effect decision | Capability surface plus policy and approval gate |
+| Artifact envelope and serialization | Runtime only |
+
+The goal is not to force every field into provider strict schema. The goal is
+to use the weakest mechanism that is strong enough for that field.
+
+## Generate-And-Validate
+
+Generate-and-validate means the LLM generates a payload through one accepted
+output channel, and runtime validates after generation.
+
+It is useful when:
+
+- fields are open and expressive
+- iteration speed matters
+- shape correctness is enough for the current step
+- validator failures can be retried safely
+
+It guarantees:
+
+- expected top-level shape
+- absence of unknown fields
+- absence of runtime-owned fields
+- parseable structured payload
+- canonical artifact written by runtime
+
+It does not guarantee meaning by itself. A well-formed payload can still contain
+an unsupported claim, wrong ref, stale source, unauthorized source, unsafe link,
+or semantically wrong category unless a validator or review gate catches it.
+
+## Construct-And-Verify
+
+Construct-and-verify means the runtime owns artifact construction, provides
+closed choices where possible, and verifies grounded claims.
+
+It is useful when:
+
+- options can be enumerated before the LLM call
+- refs or anchors can be checked against source truth
+- invalid values must be impossible or fail-loud
+- artifact authority or downstream impact is high
+- side effects require permission, preview, or approval
+
+It costs more:
+
+- runtime must enumerate option space
+- validators become product-critical code
+- false constraints can exclude the right answer
+- nuanced judgments can be discretized into the nearest bucket
+- dependencies become more sequential
+
+Use it per field or operation, not as a blanket replacement for LLM judgment.
+
+## Design Procedure
+
+Use this procedure when designing a new LLM-assisted artifact or revising an
+existing one.
+
+1. Identify the canonical artifact and downstream consumers.
+2. Split fields into semantic fields, deterministic fields, provenance fields,
+   and side-effect operations.
+3. Assign each field or operation one primary authority.
+4. Decide the accepted output channel.
+5. Shape accessible context and available tools around the task.
+6. Make deterministic fields runtime-owned.
+7. Decide which provider schema constraints are actually supported for the
+   selected model and route.
+8. Derive schemas and validators from one canonical source.
+9. Add fail-loud checks for unknown fields, runtime-owned fields, unsupported
+   refs, and denied actions.
+10. Add grounding checks only where truth is decidable.
+11. Add provenance checks for source trust, permission, freshness, and integrity.
+12. Decide retry/fail policy by failure kind and side-effect class.
+13. Make artifact persistence atomic and auditable.
+14. Add focused tests for invalid values, unsupported refs, route rejection,
+   grounding failure, policy failure, schema drift, and artifact persistence.
+15. Report which checks prove shape, which prove wiring, which prove source
+   grounding, and which only estimate semantic quality.
+
+## Verification Checklist
+
+- The LLM cannot create the canonical machine artifact through free prose.
+- The accepted output channel is explicit.
+- Structured output uses a submit tool or equivalent constrained channel.
+- Runtime-owned fields are rejected if submitted by the LLM.
+- Unknown fields fail loudly.
+- Short closed values are provider enums where supported and runtime enums
+  everywhere.
+- Provider schema limits and refusal/incomplete behavior are tested for the
+  selected route.
+- Long refs are not provider enums when they are quote-heavy, private,
+  tenant-scoped, or source-derived.
+- Runtime allowed-set checks reject unsupported refs.
+- Grounding gates are used only for decidable truths.
+- Source trust, permission, freshness, and integrity are checked separately from
+  source-span grounding.
+- Tool inputs, tool results, retrieved content, LLM output, and rendered views
+  are treated as untrusted at each boundary.
+- Side-effect tools have risk classes, permissions, and approval gates.
+- Human views are derived from machine artifacts when possible and sanitized
+  before rendering.
+- Artifact writes are atomic or have a clear recovery path.
+- Retries are safe, idempotent, or explicitly blocked for the failure class.
+- Schema, validator, allowed-set, prompt, and tests share one authority or have
+  drift-catching tests.
+- Observability captures prompt packet, model/provider version, schema hash,
+  source snapshot, validator decision, retry reason, and artifact lineage.

package/claude/guides/mock-realization-boundary.md

@@ -0,0 +1,275 @@
+---
+guide_id: mock-realization-boundary
+language: en
+status: active
+use_when:
+  - adding mocks, fakes, stubs, fixtures, or simulated providers
+  - deciding whether a mock-backed path counts as completion
+  - separating verification harnesses from production semantic paths
+  - centralizing mock payloads for later deletion or replacement
+  - reporting verification results that include mock-backed checks
+core_rules:
+  - mocks are verification realizations, not production semantic paths
+  - use mocks to verify wiring, schemas, artifact contracts, deterministic projections, failure handling, and harness stability
+  - use real semantic paths to verify product behavior, materiality judgment, causal reasoning, and semantic quality
+  - keep mock behavior behind explicit realization switches, fixture modules, or mock executors
+  - centralize mock payloads in a small deletion boundary so mock support can be removed or replaced together
+  - share validators and artifact contracts between mock and production paths
+  - report mock-backed verification separately from production-path verification
+completion_policy:
+  mock_backed_paths_support_verification: true
+  mock_backed_paths_count_as_product_completion: false
+  semantic_quality_from_mock: not_applicable
+verification_focus:
+  - mock paths exercise the same artifact contracts as production paths
+  - mock payloads are centralized and easy to delete
+  - mock execution is selected only through explicit realization config
+  - verification reports distinguish mock, fixture, and live paths
+  - production semantic gates run on real semantic paths
+---
+
+# Mock Realization Boundary Guide
+
+This guide is a scoped extension of the global Coding Guidelines. Use it when
+adding, reviewing, or cleaning up mocks, fakes, stubs, fixtures, simulated
+providers, or mock executors.
+
+The central rule is that a mock is a verification realization. It can prove that
+the harness, contract, validator, projection, or failure path works. It does not
+prove that the product semantic path works.
+
+## Core Distinction
+
+| Question | Mock path | Production semantic path |
+|---|---|---|
+| What does it prove? | Wiring, schemas, persistence, projections, failures | Real behavior, semantic judgment, user-visible quality |
+| Who owns semantics? | Fixture author | Real model, real service, real user input, or real runtime authority |
+| Completion credit | Verification support only | Product completion evidence |
+| Quality gate status | `not_applicable` or harness-only | Passed/failed against real semantic criteria |
+| Cleanup strategy | Central deletion boundary | Maintained product path |
+
+Use the same artifact contracts and validators whenever possible. The mock path
+should exercise the product contract, not create a parallel contract.
+
+## Appropriate Mock Uses
+
+Use mocks for:
+
+- wiring checks
+- schema and parser checks
+- artifact persistence checks
+- deterministic projection checks
+- retry and failure handling checks
+- timeout and cancellation checks
+- E2E harness stability
+- fixture-based regression tests
+- external dependency isolation when the dependency is not under test
+
+Use real paths for:
+
+- product behavior
+- materiality judgment
+- causal reasoning
+- semantic quality gates
+- user-visible recommendations
+- provider integration confidence
+- release evidence for high-risk flows
+
+## Structural Pattern
+
+1. Define an explicit realization selector such as `mock`, `fixture`, `live`, or
+   `direct`.
+2. Keep mock payloads in a fixture module or fixture directory.
+3. Keep mock executors as thin shells that route fixture payloads through the
+   same validators and artifact writers used by production.
+4. Mark semantic-quality evaluation as `not_applicable` for mock realization.
+5. Report mock-backed verification separately from live or production-path
+   verification.
+6. Keep the deletion boundary small enough that mock support can be removed or
+   replaced in one focused change.
+
+## Realization Selector
+
+Use explicit realization config so a mock cannot silently become the product
+path.
+
+Illustrative implementation:
+
+```ts
+type Realization = "mock" | "fixture" | "live";
+
+function parseRealization(value: string | undefined): Realization {
+  if (value === "mock" || value === "fixture" || value === "live") return value;
+  return "live";
+}
+
+function selectProvider(realization: Realization) {
+  if (realization === "mock") return createMockProvider();
+  if (realization === "fixture") return createFixtureProvider();
+  return createLiveProvider();
+}
+```
+
+## Central Fixture Boundary
+
+Centralize mock payloads. Runtime code that needs deterministic mock artifacts
+imports from this boundary instead of growing inline mock-specific payloads.
+
+Illustrative implementation:
+
+```ts
+export const reviewMockFixtures = {
+  findingLedger() {
+    return {
+      schema_version: 1,
+      findings: [
+        {
+          finding_id: "finding-001",
+          target: "fixture-target",
+          claim: "fixture finding",
+          severity: "low",
+        },
+      ],
+    };
+  },
+};
+```
+
+## Thin Mock Executor
+
+Keep the mock executor as an execution shell. It should not become the place
+where product behavior is reimplemented.
+
+Illustrative implementation:
+
+```ts
+async function runMockUnit(unit: Unit, ctx: RuntimeContext) {
+  const payload = reviewMockFixtures[unit.artifactKind]();
+  const artifact = validateArtifact(payload, unit.artifactKind);
+  await writeArtifact(ctx.outputPath, artifact);
+  return {
+    realization: "mock",
+    artifact_path: ctx.outputPath,
+    semantic_quality: "not_applicable",
+  };
+}
+```
+
+## Shared Validators
+
+The mock path should pass through the same validators as production. This makes
+mock tests useful for contract drift without letting mocks become product
+completion evidence.
+
+Illustrative implementation:
+
+```ts
+function writeArtifact(path: string, payload: unknown) {
+  const validated = validateCurrentArtifactContract(payload);
+  return writeYaml(path, validated);
+}
+
+await writeArtifact(outputPath, reviewMockFixtures.findingLedger());
+await writeArtifact(outputPath, liveModelArtifact);
+```
+
+## Semantic Quality Gate
+
+Mock-backed runs should make semantic quality explicitly not applicable. They
+can still verify the harness and artifact collection.
+
+Illustrative implementation:
+
+```ts
+function evaluateSemanticQuality(args: {
+  realization: Realization;
+  artifact: unknown;
+}) {
+  if (args.realization === "mock") {
+    return {
+      status: "not_applicable",
+      applicability: "real_semantic_path_only",
+      reason: "mock verifies harness and contracts, not product semantics",
+    };
+  }
+  return evaluateLiveSemanticQuality(args.artifact);
+}
+```
+
+## Verification Reporting
+
+Separate mock-backed checks from production-path checks in final reports.
+
+Preferred reporting shape:
+
+```ts
+type VerificationReport = {
+  mock_checks: CheckResult[];
+  fixture_checks: CheckResult[];
+  production_path_checks: CheckResult[];
+  unverified_risks: string[];
+};
+```
+
+Useful report language:
+
+- "Mock run passed artifact-contract checks."
+- "Mock run did not evaluate product semantic quality."
+- "Live semantic path remains unverified."
+- "Production path passed semantic quality gate."
+
+## Deletion Boundary
+
+Mock support should be easy to remove or replace. A good deletion boundary has:
+
+- one fixture module or fixture directory
+- one realization selector
+- one mock executor shell if needed
+- shared validators outside the mock boundary
+- tests that name the realization explicitly
+- no product runtime dependency on mock-only payloads
+
+When cleanup is needed, remove or replace the fixture module, realization route,
+and mock-only tests together.
+
+## Product Completion Rule
+
+Mock-backed paths can support verification but do not count as product
+completion.
+
+Examples:
+
+- A mock E2E can prove the pipeline writes all expected artifacts.
+- A mock E2E cannot prove the recommendation is semantically correct.
+- A fixture can prove a validator rejects malformed payloads.
+- A fixture cannot prove a live provider follows the intended reasoning path.
+- A mock provider can prove retry handling.
+- A mock provider cannot prove the real provider integration is production-ready.
+
+## Design Procedure
+
+Use this procedure before adding or extending mock behavior.
+
+1. Name what the mock is meant to verify.
+2. Name what the mock does not verify.
+3. Choose an explicit realization selector.
+4. Put mock payloads in one fixture boundary.
+5. Route mock outputs through shared validators and artifact writers.
+6. Keep mock executors thin.
+7. Mark semantic quality as not applicable for mock runs.
+8. Add reporting that separates mock checks from production-path checks.
+9. Add at least one real semantic-path check when product behavior is part of
+   the completion claim.
+10. Keep deletion or replacement possible in one focused change.
+
+## Verification Checklist
+
+- Mock use is explicitly selected.
+- Mock payloads are centralized.
+- Mock behavior is not embedded across production logic.
+- Mock output uses the current production artifact contract.
+- Shared validators run on mock output.
+- Mock-backed verification is reported separately.
+- Semantic quality is not claimed from mock-backed output.
+- Product completion has real semantic-path evidence.
+- Mock support has a clear deletion boundary.