agent-bios 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DEPENDENCIES.md +89 -0
- package/LICENSE +21 -0
- package/README.md +86 -0
- package/claude/CLAUDE.md +138 -0
- package/claude/guides/cli-multi-model-workflow.md +194 -0
- package/claude/guides/coding-staged-workflow.md +70 -0
- package/claude/guides/implementation-map.md +34 -0
- package/claude/guides/llm-capability-boundary-examples.md +123 -0
- package/claude/guides/llm-capability-boundary-patterns.md +339 -0
- package/claude/guides/llm-capability-boundary.md +255 -0
- package/claude/guides/mock-realization-boundary.md +275 -0
- package/claude/guides/svg-visualization-guide.md +321 -0
- package/codex/AGENTS.md +139 -0
- package/codex/agents/frontier.toml +8 -0
- package/codex/agents/reviewer.toml +9 -0
- package/codex/agents/sweep.toml +9 -0
- package/codex/agents/workhorse.toml +8 -0
- package/codex/guides/cli-multi-model-workflow.md +194 -0
- package/codex/guides/coding-staged-workflow.md +70 -0
- package/codex/guides/implementation-map.md +34 -0
- package/codex/guides/llm-capability-boundary-examples.md +123 -0
- package/codex/guides/llm-capability-boundary-patterns.md +339 -0
- package/codex/guides/llm-capability-boundary.md +255 -0
- package/codex/guides/mock-realization-boundary.md +275 -0
- package/codex/guides/svg-visualization-guide.md +321 -0
- package/config/agent-launch.toml +94 -0
- package/package.json +54 -0
- package/scripts/agent-launch.py +1742 -0
- package/scripts/check-parity.sh +1703 -0
- package/scripts/codex-helm.sh +370 -0
- package/scripts/codex-run.sh +176 -0
- package/scripts/install.sh +310 -0
- package/scripts/provision-venv.sh +28 -0
- package/scripts/session-cost.py +106 -0
- package/shell/agent-launch.zsh +38 -0
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
---
|
|
2
|
+
guide_id: coding-staged-workflow
|
|
3
|
+
language: en
|
|
4
|
+
status: active
|
|
5
|
+
use_when:
|
|
6
|
+
- meaningful development work needing execution depth, review loops, and stop conditions
|
|
7
|
+
- architecture changes, new features, cross-module or ontology changes, review-driven fixes
|
|
8
|
+
- the user asks to design ("설계") before implementation
|
|
9
|
+
- judging materiality of review findings and deciding when to stop or redesign
|
|
10
|
+
- choosing the per-domain verification mix (code, ontology, config/data, spreadsheets, docs)
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Coding Guidelines: Staged Workflow
|
|
14
|
+
|
|
15
|
+
This guide is a scoped extension of the global Coding Guidelines. Use it for meaningful development work to choose execution depth, review loops, and stop conditions.
|
|
16
|
+
|
|
17
|
+
It operates inside the existing global rules for requested scope, concept economy, LLM/tools/code boundary, verification discipline, and documentation hygiene.
|
|
18
|
+
|
|
19
|
+
For trivial edits, use the lightweight inspect-edit-verify path from the global Coding Guidelines.
|
|
20
|
+
|
|
21
|
+
When the user asks to "설계" or design, stay in design mode. Focus on high-level design and implementation-process design, then present the plan, tradeoffs, review gates, and implementation trigger. Move to implementation after the user asks to implement or approves the plan.
|
|
22
|
+
|
|
23
|
+
## When To Use
|
|
24
|
+
|
|
25
|
+
- Use this workflow for architecture changes, new features, cross-module behavior changes, ontology changes, review-driven fixes, or work that affects user-visible behavior, authority, lifecycle, validation, failure handling, or roadmap commitments.
|
|
26
|
+
- Use the lightweight path for small text edits, narrow config changes, or single-file adjustments whose completion criteria and verification are obvious.
|
|
27
|
+
- Increase workflow depth when new evidence shows broader risk than the initial request suggested.
|
|
28
|
+
|
|
29
|
+
## Stages
|
|
30
|
+
|
|
31
|
+
1. High-level design: define the goal, scope, architecture direction, affected concepts, tradeoffs, and completion criteria.
|
|
32
|
+
2. Implementation-process design: turn the design into an ordered work plan with dependencies, verification points, review gates, and redesign triggers.
|
|
33
|
+
3. Implementation: make the smallest viable functional changes that satisfy the approved design and process plan.
|
|
34
|
+
|
|
35
|
+
## Review Loop
|
|
36
|
+
|
|
37
|
+
- At each stage, run review loops as appropriate: self review, subagent review when available, and structured multi-lens review when the repository or domain supports one (concrete tool: Environment Binding below).
|
|
38
|
+
- Iterate until material issues reach zero: review, identify material issues, fix them, and review again.
|
|
39
|
+
- Use the severity contract for materiality — the canonical definition is the ladder below; external review tools map their levels onto it: blocker, high, and medium are material; low and info are non-material.
|
|
40
|
+
- Treat blocker as primary happy-path or core-contract failure.
|
|
41
|
+
- Treat high as supported user, environment, data, or execution path failure.
|
|
42
|
+
- Treat medium as meaningful weakening of trust, auditability, reproducibility, completeness, or decision quality.
|
|
43
|
+
- Treat low and info as non-blocking unless requested or promoted by new evidence.
|
|
44
|
+
|
|
45
|
+
## Verification Menus
|
|
46
|
+
|
|
47
|
+
Per-domain menus for the global Verification Discipline loop; pick the narrowest reliable mix that proves the changed behavior, meaning, or contract.
|
|
48
|
+
|
|
49
|
+
- Code: a layered mix of unit tests, integration tests for E2E segments, targeted E2E for changed flows, and full E2E for release or high-risk changes.
|
|
50
|
+
- Ontology: static graph checks, concept economy gates, changed-path integration checks, and competency-question E2E checks.
|
|
51
|
+
- Config or data: real parsers, schema checks, fixture validation, and sample transformations.
|
|
52
|
+
- Spreadsheets: static workbook checks, fixture-based output checks, cross-sheet flow checks, visual/layout checks, and real Microsoft Excel engine recalculation for formula-dependent results.
|
|
53
|
+
- Docs: links, terminology, current behavior alignment, and references to isolated historical notes.
|
|
54
|
+
|
|
55
|
+
## Stop Conditions
|
|
56
|
+
|
|
57
|
+
- If the issue boundary expands compared with the previous review, stop and ask the user to choose redesign/rework or continuing the current iteration.
|
|
58
|
+
- Consider the boundary expanded when review reveals a broader affected purpose, failure condition, impact area, concept boundary, architecture boundary, or severity class.
|
|
59
|
+
- Before calling the work done, report the current stage, review results, remaining material issues, verification results, and any stop reason.
|
|
60
|
+
|
|
61
|
+
## Environment Binding (edit per environment)
|
|
62
|
+
|
|
63
|
+
The only section of this guide that names concrete tools. Dated; expires ~8 weeks after the date or when the bound tool changes.
|
|
64
|
+
|
|
65
|
+
Binding (2026-07):
|
|
66
|
+
|
|
67
|
+
| Slot | Binding | Notes |
|
|
68
|
+
|---|---|---|
|
|
69
|
+
| Structured multi-lens review | onto-mcp review (private MCP) | consumes/emits the severity contract defined in Review Loop |
|
|
70
|
+
| Subagent review | host CLI's native review mechanism | e.g. Claude Code `/code-review` or Agent-tool reviewers |
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
---
|
|
2
|
+
guide_id: implementation-map
|
|
3
|
+
language: en
|
|
4
|
+
status: active
|
|
5
|
+
use_when:
|
|
6
|
+
- creating or updating IMPLEMENTATION_MAP.html in a repo with implementation code
|
|
7
|
+
- building the single SVG service blueprint inside it
|
|
8
|
+
- deciding what belongs in the current-state dashboard vs isolated history notes
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Implementation Map Guide
|
|
12
|
+
|
|
13
|
+
Scoped extension of the **Implementation Map** section of the global instructions. Use this when creating or updating `IMPLEMENTATION_MAP.html` for a repo with implementation code.
|
|
14
|
+
|
|
15
|
+
## Purpose
|
|
16
|
+
|
|
17
|
+
`IMPLEMENTATION_MAP.html` is a **current-state dashboard** — it answers "where is this work now, what decides next, and what is at risk," not "what happened." It is not a changelog, handoff log, or accumulated project diary.
|
|
18
|
+
|
|
19
|
+
## Build / rebuild rules
|
|
20
|
+
|
|
21
|
+
- Rebuild it around the current task, current architecture, current risks, current decisions, and current verification status.
|
|
22
|
+
- Compress completed history into the smallest useful summary; keep detailed past progress, abandoned alternatives, and long completed-task lists in isolated notes (`docs/`, `design/`, `archive/`).
|
|
23
|
+
- The first viewport must show current goal, phase, health, next decision, and main risk.
|
|
24
|
+
- Make it a self-contained HTML view with compact visual sections for status, architecture, roadmap, decisions, risks, verification, and change impact.
|
|
25
|
+
- Update it before committing, when writing a handoff, or after meaningful architecture, roadmap, risk, decision, or verification changes.
|
|
26
|
+
|
|
27
|
+
## The SVG service blueprint
|
|
28
|
+
|
|
29
|
+
Include exactly **one** self-contained SVG service blueprint that visualizes the whole service or implemented system at the right level of abstraction. Build it using `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/svg-visualization-guide.md`.
|
|
30
|
+
|
|
31
|
+
- Keep the blueprint focused on a single judgment question; use compact nodes rather than exhaustive file or task lists.
|
|
32
|
+
- Use stable lanes, a legend, fixed role colors, short labels, and explicit arrows to separate: inputs, runtime/tools, LLM work, canonical artifacts, views, gates, quality checks, postponed work, and downstream/future work.
|
|
33
|
+
- Distinguish time flow from authority flow, and distinguish canonical artifacts from JSON/YAML/Markdown/HTML projections.
|
|
34
|
+
- Validate SVG syntax and layout hygiene when practical; ensure text does not overlap, and keep hot-path work visually separate from postponed or excluded work.
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
---
|
|
2
|
+
guide_id: llm-capability-boundary-examples
|
|
3
|
+
parent: llm-capability-boundary
|
|
4
|
+
language: en
|
|
5
|
+
status: active
|
|
6
|
+
use_when:
|
|
7
|
+
- looking for worked precedents of boundary and field-authority design
|
|
8
|
+
- designing cases similar to sidecar submit, structured-output hybrid, MCP schema projection, or projection enrichment
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# LLM And Capability Boundary: Worked Examples
|
|
12
|
+
|
|
13
|
+
This guide is a scoped extension of
|
|
14
|
+
`${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/llm-capability-boundary.md`.
|
|
15
|
+
Each example records a problem, the structural path taken, and the learning.
|
|
16
|
+
|
|
17
|
+
## Lens Sidecar To Finding Ledger
|
|
18
|
+
|
|
19
|
+
Problem:
|
|
20
|
+
|
|
21
|
+
- Lens markdown was heterogeneous.
|
|
22
|
+
- Finding-ledger LLM had to reread and normalize noisy markdown.
|
|
23
|
+
- Runtime could not strongly guarantee ids, refs, artifact paths, or validation
|
|
24
|
+
scaffolds.
|
|
25
|
+
|
|
26
|
+
Structural path:
|
|
27
|
+
|
|
28
|
+
- Add `submit_lens_findings` as a batched submit tool.
|
|
29
|
+
- LLM submits semantic finding fields once.
|
|
30
|
+
- Runtime writes `round1/{lens}.findings.yaml`.
|
|
31
|
+
- Runtime owns `session_id`, `lens_id`, `candidate_id`, `source_ref`,
|
|
32
|
+
`human_output_ref`, validation, and YAML serialization.
|
|
33
|
+
- Optional markdown is rendered from the sidecar.
|
|
34
|
+
- When every lens output is a sidecar, runtime writes `finding-ledger.yaml`
|
|
35
|
+
deterministically.
|
|
36
|
+
|
|
37
|
+
Learning:
|
|
38
|
+
|
|
39
|
+
- Machine artifacts should not depend on LLM prose formatting.
|
|
40
|
+
- Batched submit reduces partial-output and per-call overhead.
|
|
41
|
+
- Prompt packets can become audit packets when runtime owns the artifact.
|
|
42
|
+
- A tool-capable route is required; text-only fallback should fail clearly for
|
|
43
|
+
this contract.
|
|
44
|
+
|
|
45
|
+
## Structured Output Hybrid
|
|
46
|
+
|
|
47
|
+
Problem:
|
|
48
|
+
|
|
49
|
+
- Provider strict schema can enforce short enums.
|
|
50
|
+
- Long evidence refs can contain quotes or source text that make provider enum
|
|
51
|
+
schemas brittle.
|
|
52
|
+
- Pure post-hoc validation catches shape but not meaning.
|
|
53
|
+
|
|
54
|
+
Structural path:
|
|
55
|
+
|
|
56
|
+
- Use provider strict schema for short closed fields.
|
|
57
|
+
- Keep `evidence_refs` as string arrays in provider schema.
|
|
58
|
+
- Compute runtime allowed refs from prompt packet projections.
|
|
59
|
+
- Reject unsupported refs at submit time.
|
|
60
|
+
- Use a route where artifacts are created only by runtime submit handling.
|
|
61
|
+
|
|
62
|
+
Learning:
|
|
63
|
+
|
|
64
|
+
- Strict schema is real enforcement only where provider support and schema shape
|
|
65
|
+
are suitable.
|
|
66
|
+
- Long refs need runtime allowed-set validation.
|
|
67
|
+
- Each field needs one primary authority plus enough layered checks.
|
|
68
|
+
|
|
69
|
+
## MCP-Projectable Schema Boundary
|
|
70
|
+
|
|
71
|
+
Problem:
|
|
72
|
+
|
|
73
|
+
- MCP/Claude tool surfaces need simple, directly valid object schemas.
|
|
74
|
+
- Internal artifact schemas may benefit from richer JSON Schema composition.
|
|
75
|
+
- Treating every repository schema as a tool schema can over-constrain internal
|
|
76
|
+
design, while exposing composed schemas can break tool hosts.
|
|
77
|
+
|
|
78
|
+
Structural path:
|
|
79
|
+
|
|
80
|
+
- Expose pattern-valid canonical tool names: use `namespace_verb` snake_case
|
|
81
|
+
such as `sheets_read`, matching `^[a-zA-Z0-9_-]{1,64}$`; reuse the catalog
|
|
82
|
+
name for dispatch, audit, and allowed-tool config, and enforce it in catalog
|
|
83
|
+
or seed validation.
|
|
84
|
+
- Treat MCP tool `input_schema` values and schemas intended for MCP/Claude tool
|
|
85
|
+
projection as MCP-projectable schemas.
|
|
86
|
+
- For MCP-projectable schemas, prefer direct object schemas with explicit
|
|
87
|
+
fields and avoid `oneOf`, `anyOf`, and `allOf`.
|
|
88
|
+
- Put variant behavior behind operation enums, deterministic dispatch, runtime
|
|
89
|
+
validation, or explicit projection adapters.
|
|
90
|
+
- Internal-only schemas may use composition when it materially reduces
|
|
91
|
+
complexity, but project them into compatible direct object schemas before
|
|
92
|
+
they reach an MCP/Claude tool surface.
|
|
93
|
+
|
|
94
|
+
Learning:
|
|
95
|
+
|
|
96
|
+
- The compatibility rule belongs at the tool projection boundary, not as a
|
|
97
|
+
universal ban on every internal schema.
|
|
98
|
+
- Tools/code should own projection and validation so the LLM cannot accidentally
|
|
99
|
+
expose an incompatible schema shape.
|
|
100
|
+
|
|
101
|
+
## Issue Stance Matrix Projection
|
|
102
|
+
|
|
103
|
+
Problem:
|
|
104
|
+
|
|
105
|
+
- A compact projection lacked action and dependency context needed by later LLM
|
|
106
|
+
units.
|
|
107
|
+
- Asking later units to reread raw artifacts would increase latency and drift.
|
|
108
|
+
|
|
109
|
+
Structural path:
|
|
110
|
+
|
|
111
|
+
- Enrich the runtime projection with action, dependency, threshold, singleton,
|
|
112
|
+
shared-cause, and bounded-source-ref fields from authoritative upstream
|
|
113
|
+
artifacts.
|
|
114
|
+
- Keep matrix merge deterministic.
|
|
115
|
+
- Validate refs against allowed source variants.
|
|
116
|
+
- Track projection coverage and fallback when omitted context may matter.
|
|
117
|
+
|
|
118
|
+
Learning:
|
|
119
|
+
|
|
120
|
+
- Projection-first enrichment is often better than expanding LLM context.
|
|
121
|
+
- Add semantic context to deterministic projections when later judgment depends
|
|
122
|
+
on it.
|
|
123
|
+
- Projection quality needs coverage checks, not only schema checks.
|
|
@@ -0,0 +1,339 @@
|
|
|
1
|
+
---
|
|
2
|
+
guide_id: llm-capability-boundary-patterns
|
|
3
|
+
parent: llm-capability-boundary
|
|
4
|
+
language: en
|
|
5
|
+
status: active
|
|
6
|
+
use_when:
|
|
7
|
+
- implementing submit tools, runtime-owned fields, or output channel locks
|
|
8
|
+
- choosing provider strict schema vs runtime allowed-set validation
|
|
9
|
+
- implementing grounding, provenance, projection, or evidence-index mechanics
|
|
10
|
+
- handling security, side effects, persistence, idempotency, or retry policy
|
|
11
|
+
- managing schema single source of truth and migration
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# LLM And Capability Boundary: Enforcement Patterns
|
|
15
|
+
|
|
16
|
+
This guide is a scoped extension of
|
|
17
|
+
`${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/llm-capability-boundary.md`.
|
|
18
|
+
Use it when implementing the enforcement mechanics that the boundary doctrine
|
|
19
|
+
assigns to runtime/tools and the capability surface.
|
|
20
|
+
|
|
21
|
+
## Structural Enforcement Patterns
|
|
22
|
+
|
|
23
|
+
### Bounded Submit Tool
|
|
24
|
+
|
|
25
|
+
Use a submit tool when an LLM must provide semantic content for a
|
|
26
|
+
machine-consumed artifact.
|
|
27
|
+
|
|
28
|
+
Pattern:
|
|
29
|
+
|
|
30
|
+
1. Runtime creates a submit tool with a narrow schema.
|
|
31
|
+
2. LLM submits only bounded semantic fields.
|
|
32
|
+
3. Runtime rejects unknown fields and runtime-owned fields.
|
|
33
|
+
4. Runtime validates enums, refs, grounding, and policy constraints.
|
|
34
|
+
5. Runtime writes the canonical artifact.
|
|
35
|
+
6. Downstream consumers read only the runtime artifact.
|
|
36
|
+
|
|
37
|
+
This is stronger than asking the LLM to "write valid YAML." The LLM can still
|
|
38
|
+
make semantic judgments, but shape, path, ids, metadata, and serialization are
|
|
39
|
+
owned by runtime/tools.
|
|
40
|
+
|
|
41
|
+
Illustrative implementation:
|
|
42
|
+
|
|
43
|
+
```ts
|
|
44
|
+
type FindingSubmitPayload = {
|
|
45
|
+
findings: Array<{
|
|
46
|
+
target: string;
|
|
47
|
+
claim: string;
|
|
48
|
+
evidence_refs: string[];
|
|
49
|
+
rationale?: string;
|
|
50
|
+
}>;
|
|
51
|
+
};
|
|
52
|
+
|
|
53
|
+
function submitFindings(payload: FindingSubmitPayload, ctx: RuntimeContext) {
|
|
54
|
+
rejectUnknownFields(payload, ["findings"]);
|
|
55
|
+
rejectRuntimeOwnedFieldsDeep(payload, [
|
|
56
|
+
"schema_version",
|
|
57
|
+
"session_id",
|
|
58
|
+
"lens_id",
|
|
59
|
+
"candidate_id",
|
|
60
|
+
"source_ref",
|
|
61
|
+
"output_path",
|
|
62
|
+
]);
|
|
63
|
+
|
|
64
|
+
validateEvidenceRefs(payload.findings, ctx.allowedEvidenceRefs);
|
|
65
|
+
|
|
66
|
+
const artifact = {
|
|
67
|
+
schema_version: 1,
|
|
68
|
+
session_id: ctx.sessionId,
|
|
69
|
+
lens_id: ctx.lensId,
|
|
70
|
+
findings: payload.findings.map((finding, index) => ({
|
|
71
|
+
...finding,
|
|
72
|
+
candidate_id: stableCandidateId(ctx, finding, index),
|
|
73
|
+
source_ref: `${ctx.outputPath}#candidate-${index + 1}`,
|
|
74
|
+
})),
|
|
75
|
+
};
|
|
76
|
+
|
|
77
|
+
atomicWriteYaml(ctx.outputPath, validateFindingArtifact(artifact));
|
|
78
|
+
}
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Runtime-Owned Deterministic Fields
|
|
82
|
+
|
|
83
|
+
Keep deterministic fields outside LLM authority when runtime/tools can derive
|
|
84
|
+
them.
|
|
85
|
+
|
|
86
|
+
Common runtime-owned fields:
|
|
87
|
+
|
|
88
|
+
- `schema_version`
|
|
89
|
+
- `session_id`
|
|
90
|
+
- `lens_id`
|
|
91
|
+
- `issue_id` when unit identity already determines it
|
|
92
|
+
- `candidate_id`, `finding_id`, `cause_id` when stable runtime assignment is
|
|
93
|
+
available
|
|
94
|
+
- `source_ref` when it can be derived from artifact path and local id
|
|
95
|
+
- `output_path`
|
|
96
|
+
- validation scaffolds
|
|
97
|
+
- artifact envelope and serialization
|
|
98
|
+
- source snapshot id, source hash, trust tier, permission scope, and staleness
|
|
99
|
+
metadata
|
|
100
|
+
|
|
101
|
+
The LLM may select a known id only when selection is the semantic task. If the
|
|
102
|
+
runtime already knows the id, the LLM should not submit it.
|
|
103
|
+
|
|
104
|
+
Stable ordering alone can break under retry, batching, dedupe, or parallelism.
|
|
105
|
+
Use normalized hashes, idempotency keys, persisted sequence tables, or prior-run
|
|
106
|
+
mappings when ids must remain stable across runs.
|
|
107
|
+
|
|
108
|
+
### Accepted Output Channel Lock
|
|
109
|
+
|
|
110
|
+
When structured output matters, make the submit path the only accepted path.
|
|
111
|
+
|
|
112
|
+
Examples:
|
|
113
|
+
|
|
114
|
+
- Canonical artifact writes happen only through runtime submit handling.
|
|
115
|
+
- Text output can be captured for diagnostics, but does not become artifact
|
|
116
|
+
truth.
|
|
117
|
+
- A text-only executor is rejected when the contract requires a tool-capable
|
|
118
|
+
structured-output path.
|
|
119
|
+
- Runtime-owned canonical paths are isolated from LLM-written scratch paths.
|
|
120
|
+
|
|
121
|
+
This turns "please use the right format" into "only this channel is accepted."
|
|
122
|
+
A read-only filesystem route is one implementation. The deeper rule is that the
|
|
123
|
+
canonical artifact truth is writable only through runtime-controlled paths.
|
|
124
|
+
|
|
125
|
+
### Provider Strict Schema For Short Closed Values
|
|
126
|
+
|
|
127
|
+
Provider strict schema is useful for short, stable, closed vocabularies. It is
|
|
128
|
+
not the artifact authority.
|
|
129
|
+
|
|
130
|
+
Good strict-schema candidates:
|
|
131
|
+
|
|
132
|
+
- `severity`
|
|
133
|
+
- `stance`
|
|
134
|
+
- `issue_role`
|
|
135
|
+
- `judgment_state`
|
|
136
|
+
- `impact_kind`
|
|
137
|
+
- `timing_class`
|
|
138
|
+
- `closure_class`
|
|
139
|
+
- short bounded `issue_id` values when the LLM must select one
|
|
140
|
+
- confidence or relation enums
|
|
141
|
+
|
|
142
|
+
Keep runtime enum validation too. Provider support depends on model, route,
|
|
143
|
+
schema subset, schema size, and refusal/incomplete behavior. Probe the route
|
|
144
|
+
before relying on strict schema, and fail or downgrade deliberately when support
|
|
145
|
+
is unavailable.
|
|
146
|
+
|
|
147
|
+
Keep sensitive data, long source text, private refs, and user-specific secrets
|
|
148
|
+
out of schema names, enum values, const values, and regex patterns. Schema text
|
|
149
|
+
itself is data.
|
|
150
|
+
|
|
151
|
+
### Runtime Allowed-Set Validation For Long Refs
|
|
152
|
+
|
|
153
|
+
Long refs, source-derived refs, quoted snippets, and path-heavy strings are
|
|
154
|
+
better handled as strings in provider schema plus runtime allowed-set
|
|
155
|
+
validation.
|
|
156
|
+
|
|
157
|
+
Good runtime allowed-set candidates:
|
|
158
|
+
|
|
159
|
+
- `evidence_refs`
|
|
160
|
+
- source refs containing quotes
|
|
161
|
+
- refs that include line text
|
|
162
|
+
- generated artifact anchors
|
|
163
|
+
- source snippets
|
|
164
|
+
- long path-like values
|
|
165
|
+
- user- or tenant-scoped ids
|
|
166
|
+
|
|
167
|
+
This keeps provider schemas robust while preserving fail-loud validation. The
|
|
168
|
+
LLM can emit a string, but runtime rejects strings outside the computed allowed
|
|
169
|
+
set.
|
|
170
|
+
|
|
171
|
+
### Grounding And Provenance
|
|
172
|
+
|
|
173
|
+
Use grounding as a hard gate only when source truth is decidable.
|
|
174
|
+
|
|
175
|
+
Good grounding-blocked candidates:
|
|
176
|
+
|
|
177
|
+
- evidence anchor resolves to a known source span
|
|
178
|
+
- quoted source text exists in the cited file
|
|
179
|
+
- ref belongs to a known artifact and anchor set
|
|
180
|
+
- count, id, or relation coverage can be deterministically checked
|
|
181
|
+
|
|
182
|
+
Keep warning-style audits for free prose when false positives are likely. A
|
|
183
|
+
free-text synthesis citation audit may be useful, but it should not become a
|
|
184
|
+
hard gate until the verifier is reliable.
|
|
185
|
+
|
|
186
|
+
Grounding is not provenance. A quote can match a source span while the source is
|
|
187
|
+
stale, unauthorized, poisoned, incomplete, or low-trust. Track provenance
|
|
188
|
+
separately:
|
|
189
|
+
|
|
190
|
+
- `source_snapshot_id`
|
|
191
|
+
- source hash or version
|
|
192
|
+
- ingest time
|
|
193
|
+
- permission scope
|
|
194
|
+
- trust tier
|
|
195
|
+
- retrieval policy
|
|
196
|
+
- staleness policy
|
|
197
|
+
- poisoning or integrity checks where relevant
|
|
198
|
+
|
|
199
|
+
Use robust quote checks in production: normalize whitespace and Unicode, use
|
|
200
|
+
stable offsets or line anchors, disambiguate duplicate spans, and record source
|
|
201
|
+
snapshot ids.
|
|
202
|
+
|
|
203
|
+
### Deterministic Projection
|
|
204
|
+
|
|
205
|
+
When an artifact is a direct projection from upstream artifacts, make it
|
|
206
|
+
runtime-owned.
|
|
207
|
+
|
|
208
|
+
Examples:
|
|
209
|
+
|
|
210
|
+
- Finding ledger from lens sidecars.
|
|
211
|
+
- Issue stance matrix from individual stance responses.
|
|
212
|
+
- Synthesis ledger from issue synthesis responses.
|
|
213
|
+
- Review record counts and classification summaries from canonical issue
|
|
214
|
+
artifacts.
|
|
215
|
+
|
|
216
|
+
Use the LLM to define projection rules when semantic design is needed. Use
|
|
217
|
+
runtime/tools to apply the rules.
|
|
218
|
+
|
|
219
|
+
Projection-first context is often better than asking downstream LLM units to
|
|
220
|
+
reread large raw artifacts. Add compact semantic fields from authoritative
|
|
221
|
+
upstream artifacts, such as `proposed_action`, `issue_statement`,
|
|
222
|
+
`domain_threshold_used`, `singleton_reason`, `shared_cause`, dependencies, and
|
|
223
|
+
bounded source refs.
|
|
224
|
+
|
|
225
|
+
Projection can also hide important evidence. Track coverage, omitted evidence,
|
|
226
|
+
and fallback triggers when the projection may be insufficient.
|
|
227
|
+
|
|
228
|
+
### Human View From Machine Artifact
|
|
229
|
+
|
|
230
|
+
For machine artifacts that also need a human-readable view, generate the human
|
|
231
|
+
view from the machine artifact when possible.
|
|
232
|
+
|
|
233
|
+
Pattern:
|
|
234
|
+
|
|
235
|
+
1. LLM submits semantic payload.
|
|
236
|
+
2. Runtime writes validated machine sidecar.
|
|
237
|
+
3. Runtime renders markdown or HTML from the sidecar.
|
|
238
|
+
4. Machine consumers read the sidecar.
|
|
239
|
+
5. Humans read the rendered view.
|
|
240
|
+
|
|
241
|
+
This avoids asking the LLM to keep two outputs consistent. Rendered views must
|
|
242
|
+
still be treated as untrusted output: escape HTML, sanitize links, strip unsafe
|
|
243
|
+
markup, and avoid executing model- or source-generated content.
|
|
244
|
+
|
|
245
|
+
### Evidence Index
|
|
246
|
+
|
|
247
|
+
Use an evidence index when repeated semantic review needs exact, re-checkable
|
|
248
|
+
evidence.
|
|
249
|
+
|
|
250
|
+
Preferred shape:
|
|
251
|
+
|
|
252
|
+
- one claim per row
|
|
253
|
+
- one file path per row
|
|
254
|
+
- numeric line, byte offset, or stable anchor per row
|
|
255
|
+
- split multi-target claims into multiple rows
|
|
256
|
+
- convert prose locators into exact refs using runtime/tools
|
|
257
|
+
- include source snapshot, permission scope, and trust tier when retrieval is
|
|
258
|
+
involved
|
|
259
|
+
|
|
260
|
+
The LLM uses the evidence index for semantic judgment. Runtime/tools use it for
|
|
261
|
+
deterministic re-verification.
|
|
262
|
+
|
|
263
|
+
## Security And Side Effects
|
|
264
|
+
|
|
265
|
+
Treat prompt text, retrieved content, tool results, LLM output, rendered views,
|
|
266
|
+
and external API responses as untrusted until validated for the next boundary.
|
|
267
|
+
|
|
268
|
+
Required rules:
|
|
269
|
+
|
|
270
|
+
- Keep source documents and tool results as data rather than authority.
|
|
271
|
+
- Validate and sanitize LLM output before passing it to code, shells, SQL,
|
|
272
|
+
browsers, renderers, APIs, or downstream agents.
|
|
273
|
+
- Use least privilege for tools and routes.
|
|
274
|
+
- Classify side effects: read-only, reversible write, external write, external
|
|
275
|
+
send, financial/legal action, destructive action.
|
|
276
|
+
- Require preview, diff, approval, or downstream authorization for high-impact
|
|
277
|
+
actions.
|
|
278
|
+
- Log tool calls, arguments, policy decisions, and results for audit.
|
|
279
|
+
- Rate-limit and timeout tools that can loop, scan, spend, mutate, or call the
|
|
280
|
+
network.
|
|
281
|
+
|
|
282
|
+
The LLM can recommend an action. The capability surface decides whether the
|
|
283
|
+
action is available, permitted, confirmed, and accepted.
|
|
284
|
+
|
|
285
|
+
## Persistence, Idempotency, And Retry
|
|
286
|
+
|
|
287
|
+
Artifact writes should be atomic and auditable.
|
|
288
|
+
|
|
289
|
+
Preferred persistence pattern:
|
|
290
|
+
|
|
291
|
+
1. Build artifact in memory from accepted payload and runtime-owned fields.
|
|
292
|
+
2. Validate schema, refs, policy, and grounding.
|
|
293
|
+
3. Write to a temp path.
|
|
294
|
+
4. Verify persisted bytes or checksum.
|
|
295
|
+
5. Atomically rename or register as canonical.
|
|
296
|
+
6. Record artifact lineage and validator result.
|
|
297
|
+
|
|
298
|
+
Retry policy must distinguish:
|
|
299
|
+
|
|
300
|
+
- transient provider failure
|
|
301
|
+
- invalid structured payload
|
|
302
|
+
- unsupported ref
|
|
303
|
+
- grounding failure
|
|
304
|
+
- permission or policy failure
|
|
305
|
+
- partial persistence failure
|
|
306
|
+
- side-effect uncertainty
|
|
307
|
+
|
|
308
|
+
Retries are safe for pure generation and validation. They are not automatically
|
|
309
|
+
safe for external side effects. Use idempotency keys, locks, duplicate detection,
|
|
310
|
+
or compensation plans where needed.
|
|
311
|
+
|
|
312
|
+
## Single Source Of Truth And Schema Evolution
|
|
313
|
+
|
|
314
|
+
Hybrid enforcement creates drift risk. A single constraint can appear in prompt
|
|
315
|
+
text, submit schema, provider schema, runtime validator, allowed-set builder,
|
|
316
|
+
artifact validator, and tests.
|
|
317
|
+
|
|
318
|
+
For each stage, define one canonical source and derive the others:
|
|
319
|
+
|
|
320
|
+
- submit tool schema
|
|
321
|
+
- provider schema
|
|
322
|
+
- runtime validator
|
|
323
|
+
- allowed-set validator
|
|
324
|
+
- artifact validator
|
|
325
|
+
- prompt contract
|
|
326
|
+
- tests
|
|
327
|
+
- migration sample artifacts
|
|
328
|
+
|
|
329
|
+
When this is not possible yet, mark the authoritative source explicitly and add
|
|
330
|
+
tests that catch schema/validator drift.
|
|
331
|
+
|
|
332
|
+
Versioned artifacts need a migration policy:
|
|
333
|
+
|
|
334
|
+
- what requires a schema version bump
|
|
335
|
+
- backward and forward compatibility expectations
|
|
336
|
+
- migration scripts or readers for old artifacts
|
|
337
|
+
- consumer contract tests
|
|
338
|
+
- deprecation window
|
|
339
|
+
- sample artifact updates
|