runward 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/NOTICE.md +13 -0
- package/README.md +128 -0
- package/dist/cli.js +76 -0
- package/dist/commands/check.js +50 -0
- package/dist/commands/doctor.js +75 -0
- package/dist/commands/init.js +90 -0
- package/dist/commands/status.js +49 -0
- package/dist/commands/update.js +48 -0
- package/dist/lib/constants.js +2 -0
- package/dist/lib/mission.js +96 -0
- package/dist/lib/paths.js +28 -0
- package/dist/lib/styles.js +60 -0
- package/dist/lib/tools.js +59 -0
- package/dist/lib/write.js +30 -0
- package/package.json +29 -0
- package/templates/mission/adr/ADR-0000-template.md +38 -0
- package/templates/mission/architecture.md +60 -0
- package/templates/mission/decision-matrix.md +37 -0
- package/templates/mission/evaluation-rubric.md +60 -0
- package/templates/mission/floor.md +47 -0
- package/templates/mission/framing.md +58 -0
- package/templates/mission/mission-contract.md +58 -0
- package/templates/mission/observability-schema.md +53 -0
- package/templates/mission/port-contract.md +63 -0
- package/templates/mission/reference-stack.md +70 -0
- package/templates/mission/runbook.md +70 -0
- package/templates/mission/shared-bricks.md +74 -0
- package/templates/mission/threat-model.md +53 -0
- package/templates/rules/async-job-guardrails.md +95 -0
- package/templates/rules/async-post-turn-pipeline.md +96 -0
- package/templates/rules/async-scheduled-maintenance.md +95 -0
- package/templates/rules/cache-three-tier-architecture.md +44 -0
- package/templates/rules/checklist-day-zero-project.md +97 -0
- package/templates/rules/checklist-pre-production-observability.md +99 -0
- package/templates/rules/checklist-pre-production-performance.md +109 -0
- package/templates/rules/checklist-pre-production-resilience.md +99 -0
- package/templates/rules/checklist-pre-production-security.md +86 -0
- package/templates/rules/config-secrets-boundary.md +57 -0
- package/templates/rules/config-typing-zod.md +78 -0
- package/templates/rules/contracts-governance.md +64 -0
- package/templates/rules/data-memory-consolidation.md +81 -0
- package/templates/rules/data-memory-invalidation.md +90 -0
- package/templates/rules/data-memory-scoring.md +154 -0
- package/templates/rules/data-migrations-forward-only.md +55 -0
- package/templates/rules/data-orphan-cleanup.md +142 -0
- package/templates/rules/data-ttl-types.md +100 -0
- package/templates/rules/eval-loop.md +52 -0
- package/templates/rules/frontier-deterministic-boundary.md +91 -0
- package/templates/rules/hexa-adapter-pattern.md +85 -0
- package/templates/rules/hexa-architecture.md +84 -0
- package/templates/rules/hexa-move-deterministic-out.md +93 -0
- package/templates/rules/hexa-recommended-stack.md +33 -0
- package/templates/rules/hexa-typescript-native.md +80 -0
- package/templates/rules/observability-alert-configuration.md +160 -0
- package/templates/rules/observability-llm-metrics.md +150 -0
- package/templates/rules/observability-startup-provider-log.md +128 -0
- package/templates/rules/observability-structured-json-logs.md +138 -0
- package/templates/rules/patterns-memory-router-tiered.md +139 -0
- package/templates/rules/patterns-prompt-compiler.md +126 -0
- package/templates/rules/patterns-request-id-propagation.md +137 -0
- package/templates/rules/process-adr-and-journal.md +50 -0
- package/templates/rules/provider-llm-auto-detection.md +54 -0
- package/templates/rules/provider-no-crash-missing-env.md +67 -0
- package/templates/rules/quality-codebase-metrics.md +115 -0
- package/templates/rules/quality-zod-input-validation.md +131 -0
- package/templates/rules/resilience-fail-open.md +65 -0
- package/templates/rules/resilience-multi-provider-fallback.md +100 -0
- package/templates/rules/resilience-retry-backoff.md +115 -0
- package/templates/rules/resilience-retryable-errors.md +107 -0
- package/templates/rules/routing-confidence-upgrade.md +89 -0
- package/templates/rules/routing-model-cost-ratios.md +68 -0
- package/templates/rules/routing-smart-complexity.md +131 -0
- package/templates/rules/scaling-db-connection-pooling.md +121 -0
- package/templates/rules/scaling-distributed-rate-limiting.md +141 -0
- package/templates/rules/scaling-state-externalization.md +84 -0
- package/templates/rules/security-prompt-injection.md +63 -0
- package/templates/rules/state-event-sourcing.md +66 -0
- package/templates/rules/tools-registry-pattern.md +153 -0
- package/templates/rules/tools-scope-atomicity.md +103 -0
- package/templates/targets/AGENTS.md +32 -0
- package/templates/workflows/architect.md +50 -0
- package/templates/workflows/brownfield.md +52 -0
- package/templates/workflows/decision-loop.md +52 -0
- package/templates/workflows/floor.md +60 -0
- package/templates/workflows/frame.md +69 -0
- package/templates/workflows/govern.md +55 -0
- package/templates/workflows/handover.md +55 -0
- package/templates/workflows/iterate.md +52 -0
- package/templates/workflows/method.md +53 -0
- package/templates/workflows/review.md +59 -0
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Tool Registry Pattern
|
|
3
|
+
impact: HIGH
|
|
4
|
+
impactDescription: Replaces giant switch statements with maintainable, extensible tool management
|
|
5
|
+
tags: [tools, architecture, patterns, maintainability]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Tool Registry Pattern
|
|
9
|
+
|
|
10
|
+
Replace switch statements with a registry pattern for tool execution. Easier to test, extend, and maintain.
|
|
11
|
+
|
|
12
|
+
**Incorrect:**
|
|
13
|
+
|
|
14
|
+
```typescript
|
|
15
|
+
// BAD: Giant switch statement
|
|
16
|
+
async function executeTool(name: string, input: unknown): Promise<ToolResult> {
|
|
17
|
+
switch (name) {
|
|
18
|
+
case 'get_project':
|
|
19
|
+
return await getProject(input as GetProjectInput);
|
|
20
|
+
case 'analyze_market':
|
|
21
|
+
return await analyzeMarket(input as AnalyzeMarketInput);
|
|
22
|
+
case 'search_documents':
|
|
23
|
+
return await searchDocuments(input as SearchDocumentsInput);
|
|
24
|
+
// ... 50 more cases
|
|
25
|
+
default:
|
|
26
|
+
throw new Error(`Unknown tool: ${name}`);
|
|
27
|
+
}
|
|
28
|
+
}
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**Correct:**
|
|
32
|
+
|
|
33
|
+
```typescript
|
|
34
|
+
// Tool interface
|
|
35
|
+
interface ITool {
|
|
36
|
+
name: string;
|
|
37
|
+
description: string;
|
|
38
|
+
schema: z.ZodSchema;
|
|
39
|
+
execute(input: unknown, ctx: ToolContext): Promise<ToolResult>;
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
// Tool context for shared dependencies
|
|
43
|
+
interface ToolContext {
|
|
44
|
+
userId: string;
|
|
45
|
+
tenantId: string;
|
|
46
|
+
db: Database;
|
|
47
|
+
logger: Logger;
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
// Registry class
|
|
51
|
+
class ToolRegistry {
|
|
52
|
+
private tools = new Map<string, ITool>();
|
|
53
|
+
|
|
54
|
+
register(tool: ITool): void {
|
|
55
|
+
if (this.tools.has(tool.name)) {
|
|
56
|
+
throw new Error(`Tool already registered: ${tool.name}`);
|
|
57
|
+
}
|
|
58
|
+
this.tools.set(tool.name, tool);
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
get(name: string): ITool | undefined {
|
|
62
|
+
return this.tools.get(name);
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
async execute(name: string, input: unknown, ctx: ToolContext): Promise<ToolResult> {
|
|
66
|
+
const tool = this.tools.get(name);
|
|
67
|
+
if (!tool) {
|
|
68
|
+
throw new ToolNotFoundError(name);
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
// Validate input with Zod
|
|
72
|
+
const validated = tool.schema.parse(input);
|
|
73
|
+
|
|
74
|
+
// Execute with context
|
|
75
|
+
return tool.execute(validated, ctx);
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
// For LLM: get all tool definitions
|
|
79
|
+
getDefinitions(): ToolDefinition[] {
|
|
80
|
+
return Array.from(this.tools.values()).map(t => ({
|
|
81
|
+
name: t.name,
|
|
82
|
+
description: t.description,
|
|
83
|
+
parameters: zodToJsonSchema(t.schema),
|
|
84
|
+
}));
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
// Example tool implementation
|
|
89
|
+
const getProjectTool: ITool = {
|
|
90
|
+
name: 'get_project',
|
|
91
|
+
description: 'Retrieves project details by ID',
|
|
92
|
+
schema: z.object({
|
|
93
|
+
projectId: z.string().uuid(),
|
|
94
|
+
}),
|
|
95
|
+
async execute(input, ctx) {
|
|
96
|
+
const project = await ctx.db.projects.findUnique({
|
|
97
|
+
where: { id: input.projectId, tenantId: ctx.tenantId },
|
|
98
|
+
});
|
|
99
|
+
return { success: true, data: project };
|
|
100
|
+
},
|
|
101
|
+
};
|
|
102
|
+
|
|
103
|
+
// Registry setup
|
|
104
|
+
const registry = new ToolRegistry();
|
|
105
|
+
registry.register(getProjectTool);
|
|
106
|
+
registry.register(analyzeMarketTool);
|
|
107
|
+
registry.register(searchDocumentsTool);
|
|
108
|
+
|
|
109
|
+
// Usage
|
|
110
|
+
const result = await registry.execute('get_project', { projectId: '...' }, ctx);
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
**Benefits:**
|
|
114
|
+
|
|
115
|
+
- Each tool is independently testable
|
|
116
|
+
- Easy to add new tools (just register)
|
|
117
|
+
- Automatic schema validation
|
|
118
|
+
- Centralized logging and metrics
|
|
119
|
+
- No giant switch to maintain
|
|
120
|
+
|
|
121
|
+
**Advanced: Tool middleware:**
|
|
122
|
+
|
|
123
|
+
```typescript
|
|
124
|
+
class ToolRegistry {
|
|
125
|
+
private middleware: ToolMiddleware[] = [];
|
|
126
|
+
|
|
127
|
+
use(middleware: ToolMiddleware): void {
|
|
128
|
+
this.middleware.push(middleware);
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
async execute(name: string, input: unknown, ctx: ToolContext): Promise<ToolResult> {
|
|
132
|
+
// Apply middleware chain
|
|
133
|
+
let handler = (input: unknown) => this.executeCore(name, input, ctx);
|
|
134
|
+
|
|
135
|
+
for (const mw of this.middleware.reverse()) {
|
|
136
|
+
const next = handler;
|
|
137
|
+
handler = (input) => mw(name, input, ctx, next);
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
return handler(input);
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
// Logging middleware
|
|
145
|
+
const loggingMiddleware: ToolMiddleware = async (name, input, ctx, next) => {
|
|
146
|
+
const start = Date.now();
|
|
147
|
+
const result = await next(input);
|
|
148
|
+
ctx.logger.info('Tool executed', { name, duration: Date.now() - start });
|
|
149
|
+
return result;
|
|
150
|
+
};
|
|
151
|
+
|
|
152
|
+
registry.use(loggingMiddleware);
|
|
153
|
+
```
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Tool Scope and Atomicity
|
|
3
|
+
impact: HIGH
|
|
4
|
+
impactDescription: Reduces token usage and improves LLM tool selection accuracy
|
|
5
|
+
tags: [tools, llm, architecture, cost-optimization]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Tool Scope and Atomicity
|
|
9
|
+
|
|
10
|
+
Each tool should do ONE atomic action. Avoid catch-all tools that confuse the LLM.
|
|
11
|
+
|
|
12
|
+
**The Problem:**
|
|
13
|
+
|
|
14
|
+
Tools that do everything:
|
|
15
|
+
- Use more tokens in descriptions
|
|
16
|
+
- Confuse the LLM about when to use them
|
|
17
|
+
- Make debugging harder
|
|
18
|
+
- Can't be cached or optimized individually
|
|
19
|
+
|
|
20
|
+
**Incorrect:**
|
|
21
|
+
|
|
22
|
+
```typescript
|
|
23
|
+
// BAD: Catch-all tool
|
|
24
|
+
const analyzeProjectTool = {
|
|
25
|
+
name: 'analyze_project',
|
|
26
|
+
description: 'Analyzes project overview, market fit, traction, team, financials, and generates recommendations',
|
|
27
|
+
parameters: {
|
|
28
|
+
projectId: { type: 'string' },
|
|
29
|
+
analysisType: { type: 'string', enum: ['overview', 'market', 'traction', 'team', 'finance', 'all'] },
|
|
30
|
+
},
|
|
31
|
+
};
|
|
32
|
+
|
|
33
|
+
// BAD: Vague tool
|
|
34
|
+
const getInsightsTool = {
|
|
35
|
+
name: 'get_insights',
|
|
36
|
+
description: 'Gets insights about anything', // Too vague
|
|
37
|
+
};
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
**Correct:**
|
|
41
|
+
|
|
42
|
+
```typescript
|
|
43
|
+
// GOOD: Atomic tools - one action each
|
|
44
|
+
const tools = [
|
|
45
|
+
{
|
|
46
|
+
name: 'get_project_overview',
|
|
47
|
+
description: 'Retrieves basic project information: name, stage, sector, founding date. Use for initial context.',
|
|
48
|
+
parameters: { projectId: { type: 'string', required: true } },
|
|
49
|
+
},
|
|
50
|
+
{
|
|
51
|
+
name: 'analyze_market_fit',
|
|
52
|
+
description: 'Evaluates product-market fit based on user feedback and metrics. Use when assessing market validation.',
|
|
53
|
+
parameters: { projectId: { type: 'string', required: true } },
|
|
54
|
+
},
|
|
55
|
+
{
|
|
56
|
+
name: 'assess_traction',
|
|
57
|
+
description: 'Calculates traction metrics: MRR growth, user growth, engagement. Use for performance assessment.',
|
|
58
|
+
parameters: { projectId: { type: 'string', required: true } },
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
name: 'get_team_assessment',
|
|
62
|
+
description: 'Returns team composition and experience analysis. Use when evaluating team strength.',
|
|
63
|
+
parameters: { projectId: { type: 'string', required: true } },
|
|
64
|
+
},
|
|
65
|
+
];
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
**Principles:**
|
|
69
|
+
|
|
70
|
+
1. **One tool = One action** - Atomic operations
|
|
71
|
+
2. **2-3 examples max** - Don't bloat descriptions
|
|
72
|
+
3. **Explicit "when to use"** - Guide LLM selection
|
|
73
|
+
4. **Group by domain** - Organize related tools together
|
|
74
|
+
|
|
75
|
+
**Tool Documentation Template:**
|
|
76
|
+
|
|
77
|
+
```typescript
|
|
78
|
+
interface ToolDefinition {
|
|
79
|
+
name: string; // verb_noun format: get_project, analyze_market
|
|
80
|
+
description: string; // What it does + when to use it
|
|
81
|
+
parameters: {
|
|
82
|
+
// Only required parameters, minimal
|
|
83
|
+
};
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
// Example
|
|
87
|
+
{
|
|
88
|
+
name: 'search_documents',
|
|
89
|
+
description: 'Searches project documents by keyword. Use when user asks about specific topics or needs to find information in uploaded files.',
|
|
90
|
+
parameters: {
|
|
91
|
+
projectId: { type: 'string', required: true },
|
|
92
|
+
query: { type: 'string', required: true },
|
|
93
|
+
limit: { type: 'number', default: 10 },
|
|
94
|
+
},
|
|
95
|
+
}
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
**Signs of tool bloat:**
|
|
99
|
+
|
|
100
|
+
- Tool description > 100 words
|
|
101
|
+
- More than 5 parameters
|
|
102
|
+
- "or" in the description (does multiple things)
|
|
103
|
+
- Enum with > 5 options
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Agent charter — Runward mission
|
|
2
|
+
|
|
3
|
+
This project is delivered with the Runward method: floor first, evolution on evidence, governance from day zero. Any agent working here follows this charter. It is vendor-neutral: the method lives in `runward/workflows/`, the mission state in `runward/`.
|
|
4
|
+
|
|
5
|
+
## Non-negotiable boundaries
|
|
6
|
+
|
|
7
|
+
1. **The architecture constrains the model, not the other way around.** The model is a replaceable adapter behind a stable port. No domain logic in prompts, no prompt fragments in the domain. This principle is the opening posture, not the whole frame: the five architecture gestures the workflows apply (named in `runward/workflows/method.md`) and the decision matrix (`runward/decision-matrix.md`) carry the complete method.
|
|
8
|
+
2. **Boundaries before the stack.** Ports and contracts are decided before languages, frameworks or topology. Contracts are versioned, additive, tolerant-reader. One single language in the core; polyglot only as a sidecar behind the tool protocol, on a proven trigger.
|
|
9
|
+
3. **Complexity is deferred until a trigger commands it.** Multi-agent, long-term memory, service extraction, a bigger model: each requires an objective trigger recorded in an ADR. No trigger, no change.
|
|
10
|
+
4. **Deterministic frontier.** Anything that can be computed deterministically is. The model never invents figures that the system can compute.
|
|
11
|
+
5. **Security on actions, not display.** Untrusted input never gains write or exfiltration capability in the same session (lethal trifecta, 2-of-3 rule). Reads may fail open; actions fail closed.
|
|
12
|
+
|
|
13
|
+
## How to work
|
|
14
|
+
|
|
15
|
+
- Apply the craft rules in `runward/rules/` while building: they cover memory scoring, tiered retrieval, event sourcing, request-id propagation, resilience, cost routing, secrets and prompt-injection defenses. When a rule and a habit conflict, the rule wins; deviating from a rule requires an ADR.
|
|
16
|
+
- Consult `runward/decision-matrix.md` before adding any capability: 22 arbitrations, each with a sober default and an explicit trigger. No trigger, no change.
|
|
17
|
+
- Before any structural decision, run `runward/workflows/decision-loop.md`: verify in the real code, check the sourced state of the art, challenge the source, take a durable position, lock it in an ADR — only then edit.
|
|
18
|
+
- One ADR per structural decision, in `runward/adr/`, with a dated re-evaluation trigger. Use the template `runward/adr/ADR-0000-template.md`.
|
|
19
|
+
- Current phase and gates: see `runward/framing.md` (Definition of Ready) and each workflow's Definition of Done. Do not skip a gate on assertion; pass it on evidence.
|
|
20
|
+
- Show before you build: for any deliverable meant for humans, produce a reviewable preview first.
|
|
21
|
+
- Never mark a phase done if its Definition of Done is not demonstrably met.
|
|
22
|
+
|
|
23
|
+
## Mission state
|
|
24
|
+
|
|
25
|
+
| File | What it holds |
|
|
26
|
+
|---|---|
|
|
27
|
+
| `runward/framing.md` | Problem, value, observable success criterion, floor vs target |
|
|
28
|
+
| `runward/architecture.md` | Boundaries, ports, integration protocol |
|
|
29
|
+
| `runward/floor.md` | The floor's scope and its measured proof |
|
|
30
|
+
| `runward/adr/` | Decision journal |
|
|
31
|
+
| `runward/governance/` | Threat model, evaluation rubric, observability schema |
|
|
32
|
+
| `runward/runbook.md` | Recovery runbook for the receiving team |
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Architect — Boundaries Before Stack
|
|
2
|
+
|
|
3
|
+
## When to use
|
|
4
|
+
|
|
5
|
+
Use this workflow once framing is decided and structure must follow: "how do we architect this?", "what do we build on?", "which contracts?", "monolith or microservices?", "which backend language?" (answer: ports first). This is phase 2 of `method`. It turns a decided perimeter into a decided architecture: boundaries fixed, contracts named, language and topology deliberately left open.
|
|
6
|
+
|
|
7
|
+
## Inputs
|
|
8
|
+
|
|
9
|
+
- The framing note: floor/target split, success criterion, hard constraints, presumed boundaries.
|
|
10
|
+
- The `mission/architecture.md`, `mission/port-contract.md`, and `mission/adr/ADR-0000-template.md` templates.
|
|
11
|
+
|
|
12
|
+
## Outputs
|
|
13
|
+
|
|
14
|
+
- A light architecture note.
|
|
15
|
+
- The port list with contracts and the integration protocol.
|
|
16
|
+
- One ADR per structuring decision.
|
|
17
|
+
|
|
18
|
+
## Procedure
|
|
19
|
+
|
|
20
|
+
**Refuse stack questions, again.** Contracts first, technology later. The phase rests on the method's founding inversion — the LLM Boundary Principle: the architecture constrains the model, never the reverse. The model and the infrastructure are adapter decisions, taken behind stable contracts. That is why boundaries come before stack. Each stack choice is taken adapter by adapter once boundaries are known, justified by a local technical reason — never by habit.
|
|
21
|
+
|
|
22
|
+
**Fix the two boundaries that make the stack secondary.**
|
|
23
|
+
|
|
24
|
+
1. **Domain ports.** The domain expresses its needs as contracts: generate a model completion, persist state, execute an action, read a source. Each contract is honored by an adapter written in any language, as long as it honors the contract. The model port is a port like any other: the reasoning engine is bound only by its contract, not by its brand. The daily dividend is a domain testable without the model; substitutability is the insurance you keep in reserve. This is the founding inversion made concrete: the model is a replaceable adapter behind a stable port.
|
|
25
|
+
2. **The cross-process integration protocol.** When a capability lives in another process or language, expose it through a standardized tool protocol. The system consumes that process as a tool provider and publishes its own capabilities the same way. A service is just an adapter that moved into its own process; the domain does not change.
|
|
26
|
+
|
|
27
|
+
**Govern every contract.** A port is more than a typed schema. Version it; make changes additive by default; read as a tolerant reader that ignores unknown fields and accepts missing optional ones. For a genuinely breaking change, expand then contract: introduce the new, migrate consumers, retire the old — never in one move. Track provenance: who produces, who consumes, under which version, so impact is measurable before a change. Fix meaning per bounded context: without pinned semantics the model will invent false mappings; the semantic contract is to meaning what the typed schema is to malformed data. On legacy, the adapter becomes an anticorruption layer that translates the old dialect into the domain language without contaminating the new — and that boundary is never free: name the translation cost and budget it.
|
|
28
|
+
|
|
29
|
+
**Name the default topology and its triggers.** A modular hexagonal monolith by default: one deployable, pure domain plus adapters. A single orchestrator directing specialists: it composes, it carries no business logic. A tool registry plus a middleware chain as the single transversal surface (logging, access, cost, approval, traces) — the chain stays thin, the registry stays an index, never a brain. One core language for server and interface, a thin model abstraction (a direct SDK, not a heavy chain framework); polyglot only via a sidecar or service, justified by a mature library or proven performance need. Name each default's evolution trigger here; cross it only in `iterate`.
|
|
30
|
+
|
|
31
|
+
**Lock structuring choices in ADRs.** Starting topology, core language, legacy integration strategy, bounded-context boundaries: each goes through `decision-loop` — reality-check against reference implementations, sourced state of the art, challenge, durable position, written lock. A decision that is not locked does not enter the architecture note.
|
|
32
|
+
|
|
33
|
+
**Write the architecture note.** Short, readable, decided: the ports, the default topology and its triggers, what stays open (language, provider), and the target named without being built. A few pages, not a detailed design dossier.
|
|
34
|
+
|
|
35
|
+
## Definition of Done
|
|
36
|
+
|
|
37
|
+
- Architecture note produced, boundaries first, stack open.
|
|
38
|
+
- Every port named with contract and initial version; integration protocol stated.
|
|
39
|
+
- One ADR per structuring choice, locked before the note mentions it.
|
|
40
|
+
- Note reviewed via `review` before circulation.
|
|
41
|
+
- Deliverable form matched: the presented architecture note carries an expected delivery form (a readable PDF or HTML); ADRs and contract specs stay as repository markdown.
|
|
42
|
+
|
|
43
|
+
## Anti-patterns
|
|
44
|
+
|
|
45
|
+
- Deciding language, framework, or provider at this phase.
|
|
46
|
+
- Building microservices or multi-agent into the starting topology instead of naming them in the target.
|
|
47
|
+
- Confusing typed contracts with behavior: the behavioral boundary is validated in `govern`, never guaranteed by types.
|
|
48
|
+
- Writing the note before the ADRs are locked.
|
|
49
|
+
- Pretending the boundary is free on legacy — the integration cost concentrates exactly there.
|
|
50
|
+
- Letting the middleware chain accumulate orchestration or business logic.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Brownfield — Enter a Mission That Already Has a Past
|
|
2
|
+
|
|
3
|
+
## When to use
|
|
4
|
+
|
|
5
|
+
Use this workflow whenever the mission does not start on green grass: "resume the mission in progress", "we inherited a half-built project", "this system ignores our principles, audit it", "rebuild a clean one from the old one", "where were we?". The default chain in `method` starts from framing on greenfield; most real missions start elsewhere. Establish the entry point with the sponsor, reconstruct what is missing, then re-enter the chain at the right phase — never by instinct.
|
|
6
|
+
|
|
7
|
+
## Inputs
|
|
8
|
+
|
|
9
|
+
- Access to the existing system: code, deployments, data, and whatever artifacts exist.
|
|
10
|
+
- The sponsor's intent: resume, join, rebuild, or derive.
|
|
11
|
+
- The framework's templates for retrofitted artifacts (`mission/framing.md`, `mission/architecture.md`, `mission/adr/`).
|
|
12
|
+
|
|
13
|
+
## Outputs
|
|
14
|
+
|
|
15
|
+
- The entry mode identified and agreed.
|
|
16
|
+
- The missing upstream artifacts reconstructed.
|
|
17
|
+
- Re-entry into the `method` chain at the correct phase.
|
|
18
|
+
|
|
19
|
+
## Procedure
|
|
20
|
+
|
|
21
|
+
**Enforce the entry rule.** Characterize before you touch; never rewrite in one block. On an existing system the first move is not to propose a target — it is to understand what is there, what works, and why it is the way it is. Value is reconquered tier by tier, exactly as on greenfield; a mass rewrite destroys the value already in place and postpones all proof.
|
|
22
|
+
|
|
23
|
+
**Identify the entry mode.** Four modes; when they overlap, calibrate on the most demanding one.
|
|
24
|
+
|
|
25
|
+
- **M1 — Resume your own mission.** Artifacts exist: framing note, ADRs, journal, code. Opening move: reread them, rebuild the state, locate the phase reached, spot what has aged, and pick the chain back up where it stopped. The state of a mission is its persisted artifacts, never anyone's memory.
|
|
26
|
+
- **M2 — Join a project in flight.** The project exists, usually without (or with partial) upstream artifacts. Opening move: inventory the existing system, then produce the missing upstream deliverables — a light retro-framing, an architecture note reconstituted by observation, and retroactive ADRs that record the structuring choices already made and the signal under which to reopen them. With the map drawn, enter the chain at the current tier.
|
|
27
|
+
- **M3 — Audit then rebuild.** The system was not built on these principles. Opening move: a principled audit, a gap analysis section by section, then a rebuild plan. The rebuild is never one block; it follows the brownfield procedure below.
|
|
28
|
+
- **M4 — Derive a new system from an old one.** The existing system is an input and a legacy constraint, not a base to copy. Opening move: frame the new system using the old one as a source of requirements and constraints, then enter the chain as greenfield from `frame`.
|
|
29
|
+
|
|
30
|
+
**Apply the brownfield procedure (M3 and M4).** On legacy, the boundary is not given and is never free — integration cost concentrates exactly there. Three moves, in order, never skipped:
|
|
31
|
+
|
|
32
|
+
1. **Characterize before touching.** Capture the real behavior of the existing system, quirks included, with characterization tests before any modification. Find the seams — the points where behavior can be altered without rewriting. Until behavior is captured, do not rebuild: you would silently break what worked.
|
|
33
|
+
2. **Install an anticorruption layer.** An adapter at the boundary translates the legacy dialect into the clean domain language and stops the old model from contaminating the new. The new domain stays pure behind that boundary.
|
|
34
|
+
3. **Replace progressively, strangler-style.** Build the new around the old, unplug it feature by feature on proof, until the old retires naturally. Never a single-shot cutover. Each replacement is guarded, measured, and reversible behind the boundary.
|
|
35
|
+
|
|
36
|
+
**Lock and re-enter.** Every structuring choice discovered or taken on the existing system — what is kept, what is rebuilt, where the anticorruption boundary runs — is locked as an ADR via `decision-loop`, retroactive ADRs included. Then re-enter the `method` chain at the right phase: usually `architect` for M2 and M3, `frame` for M4, the current phase for M1. Governance, iteration, and handover proceed unchanged.
|
|
37
|
+
|
|
38
|
+
## Definition of Done
|
|
39
|
+
|
|
40
|
+
- Entry mode chosen explicitly with the sponsor.
|
|
41
|
+
- Missing upstream artifacts produced (retro-framing, reconstituted architecture note, retroactive ADRs).
|
|
42
|
+
- For rebuilds: behavior characterized, anticorruption layer in place, replacement plan staged feature by feature.
|
|
43
|
+
- Chain re-entered at a named phase, with the same gates as greenfield.
|
|
44
|
+
|
|
45
|
+
## Anti-patterns
|
|
46
|
+
|
|
47
|
+
- Rewriting in one block — the mass cutover is forbidden.
|
|
48
|
+
- Modifying behavior that has not been captured.
|
|
49
|
+
- Confusing audit with rebuild: the audit establishes the gap, factual and sourced; the rebuild is a separate, guarded, progressive decision.
|
|
50
|
+
- Resuming a mission from memory instead of its artifacts.
|
|
51
|
+
- Pretending the boundary is free on legacy — name the translation cost and budget it.
|
|
52
|
+
- Copying the old system instead of treating it as requirements and constraints.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Decision Loop — Lock a Position Before Touching the Document
|
|
2
|
+
|
|
3
|
+
## When to use
|
|
4
|
+
|
|
5
|
+
Use this workflow whenever a durable technical position must be decided before editing an architecture document or committing a design: "should we change this?", "does this critique hold?", "what is the right practice here?", "add this idea to the doc". It pairs with `review`, which provides the expert eyes; this workflow provides the method. The rule that dominates everything: challenge and ask before any modification. Never edit on instinct, never edit before the lock.
|
|
6
|
+
|
|
7
|
+
## Inputs
|
|
8
|
+
|
|
9
|
+
- The critique, requirement, transcript, or doubt that triggered the loop.
|
|
10
|
+
- The document or design under question.
|
|
11
|
+
- Access to the real code that implements (or will implement) the system, or to public reference implementations.
|
|
12
|
+
|
|
13
|
+
## Outputs
|
|
14
|
+
|
|
15
|
+
- A locked design note in ADR form.
|
|
16
|
+
- A validated edit plan — and only then, edits.
|
|
17
|
+
|
|
18
|
+
## Procedure
|
|
19
|
+
|
|
20
|
+
**Hold the governing principle.** The document standardizes best practice, not the current state of the code that implements it. Check the code for reality, confront the state of the art for reference, then write the best defensible version — without ever exposing the code's limitations in the document. If the code is not yet up to standard, the code catches up; the document never stoops to the code.
|
|
21
|
+
|
|
22
|
+
**Run the loop, in order, for every critique, doubt, or proposed addition.**
|
|
23
|
+
|
|
24
|
+
1. **Reality-check the code.** Read what the code already does today in the repositories that carry the system. Identify what is implemented, partial, or absent. On greenfield, reality-check against public reference implementations and established architectures instead. These findings calibrate the position; they are never quoted in the document.
|
|
25
|
+
2. **Source the state of the art.** Hunt for evidence: research papers, leaders' documentation, standards, audits. Search, then read the sources. Never lock a position on intuition — anchor it on nameable sources. If an angle stays blurry, dig before locking.
|
|
26
|
+
3. **Challenge the source.** Confront the critique or requirement with what you just learned. Keep what has real merit; discard noise, passing fads, and overclaims. Distinguish the timeless pattern from the fashion of the season.
|
|
27
|
+
4. **Take a durable position.** Decide: what is the standard, defensible stance that would hold in front of a regulated-sector architecture committee? Formulate the general rule, not just the one-off answer.
|
|
28
|
+
5. **Lock it in writing.** Freeze the decision in a design note before any edit to the document. Not locked, not validated — not edited.
|
|
29
|
+
6. **Write the lock as an ADR.** Three sections: **Context** (the problem, what the code already does, the sourced state of the art), **Decision** (the position taken and the alternative discarded), **Consequences** (effects, plus the precise spec of the edits: which sections, which additions). Precise vocabulary, no hedging, no subjective escape hatches. The edit spec is validated before anything is written. Use `mission/adr/ADR-0000-template.md`.
|
|
30
|
+
7. **Integrate, in strict order.** First the blocking fixes — propagation of already-locked decisions for consistency, no new design. Then the important design questions, each by rerunning this full loop. Then a whole-document consistency pass — the entire document, not section by section. Minors last.
|
|
31
|
+
|
|
32
|
+
**Apply the golden rules throughout.** Ask before modifying, always. Never write before the lock. Standardize best practice; never expose the code's internal constraints in the document. Source every state-of-the-art claim. Distinguish the durable pattern from the passing fad. Watch for overlapping decisions: when two designs touch, reconcile them in one transition, never as two parallel mechanisms. End every cycle with whole-document consistency, never only local.
|
|
33
|
+
|
|
34
|
+
**Articulate with the review panel.** In practice: `review` spots a finding through its six-criterion grid and grades it blocking / important / minor. If the finding is a genuine design question, this loop runs (steps 1–6). Then integration happens (step 7), and the panel repasses for whole-document consistency. Both workflows share one golden rule: the reviewer criticizes and the loop decides — nobody rewrites without validation.
|
|
35
|
+
|
|
36
|
+
**Name the artifacts predictably.** In the mission's decision folder (for example `decisions/` or `adr/`): a dated triage of retained critiques; one dated, locked design note (ADR) per decision; a dated record of panel verdicts; a dated consolidated edit plan.
|
|
37
|
+
|
|
38
|
+
## Definition of Done
|
|
39
|
+
|
|
40
|
+
- Every triggered question ran the full loop in order — no step skipped.
|
|
41
|
+
- One locked ADR per decision, with sourced context and a discarded alternative.
|
|
42
|
+
- The edit spec validated before any edit.
|
|
43
|
+
- Integration done in order: blocking, important (with loop), whole-document consistency, minors.
|
|
44
|
+
|
|
45
|
+
## Anti-patterns
|
|
46
|
+
|
|
47
|
+
- Editing the document first and justifying afterward.
|
|
48
|
+
- Locking a position on intuition, without nameable sources.
|
|
49
|
+
- Quoting the code's current limitations in the document.
|
|
50
|
+
- Adopting a passing fad as if it were a durable pattern.
|
|
51
|
+
- Running two parallel mechanisms where two decisions should have been reconciled into one.
|
|
52
|
+
- Skipping the whole-document consistency pass after local edits.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Floor — Ship the Smallest System That Proves Value
|
|
2
|
+
|
|
3
|
+
## When to use
|
|
4
|
+
|
|
5
|
+
Use this workflow once the architecture is fixed and building must start: "what do we code first?", "we need an MVP", "should we add memory / multi-agent now?" (answer: not without a trigger). This is phase 3 of `method`. It delivers the floor: the smallest system that proves value on real traffic — a running system, not a demo.
|
|
6
|
+
|
|
7
|
+
## Inputs
|
|
8
|
+
|
|
9
|
+
- The architecture note, port list, and integration protocol from `architect`.
|
|
10
|
+
- The observable success criterion from `frame`.
|
|
11
|
+
- Real model access, real inputs, and a hook into the existing infrastructure.
|
|
12
|
+
|
|
13
|
+
## Outputs
|
|
14
|
+
|
|
15
|
+
- A running floor wired to real traffic.
|
|
16
|
+
- A floor note (use `mission/floor.md`): scope, measured proof, gaps, next tier.
|
|
17
|
+
|
|
18
|
+
## Procedure
|
|
19
|
+
|
|
20
|
+
**Enforce the entry rule.** The floor plugs into real traffic and existing infrastructure. Anything not required to prove value is deferred and attached to an explicit trigger. A component without a trigger does not enter the floor.
|
|
21
|
+
|
|
22
|
+
**Start from the reference floor, not a blank page.** The starting point is the reference floor (`floor-ts/` in the Runward repository): a clonable hexagonal scaffold that already carries the pure domain, the ports, the adapters, and the middleware chain, with a principle-to-code table mapping each principle to where it lives in the code. Clone it, implement the project's concrete adapters behind the ports fixed in `architect`, and leave intact the structure that keeps the domain testable without the model. Do not reinvent the skeleton; populate it.
|
|
23
|
+
|
|
24
|
+
**Build exactly these six pieces, and nothing more.**
|
|
25
|
+
|
|
26
|
+
1. **An entry point matched to actual use.** A minimal interface when a human operates the system (submit, review, approve, decide); an API or tool protocol when another system drives it. It is a primary adapter — plain, replaceable, never touching the domain, and never a demo showcase.
|
|
27
|
+
2. **A single orchestrator.** It runs the loop: plans, delegates, synthesizes. It composes; it carries no business logic.
|
|
28
|
+
3. **A model port behind a direct SDK.** Thin abstraction, one balanced tier, no heavy chain framework. The model port is the heart of the floor, not a deferred item: ship a real, provider-agnostic adapter that activates as soon as a key is configured, with a deterministic fallback when none is. The app must run for real by filling in configuration — never stay in simulation by default. The adapter names no provider; provider quirks (headers, required fields) resolve at assembly time through a profile, never inside the domain.
|
|
29
|
+
4. **Persistence.** An immutable interaction log as the source of truth, attached to a business entity. The agent is a reducer with no hidden state: each turn acts on explicit context and produces explicit state.
|
|
30
|
+
5. **Guardrails in code.** Everything that can be deterministic — classification, validation, access control on mutations — leaves the model for testable code. The model only reasons.
|
|
31
|
+
6. **Baseline observability.** Structured logs (one line per event, typed context), persisted lifecycle events, and a request ID propagated from the entry point to every tool and model call. This is what makes a trajectory replayable and auditable later.
|
|
32
|
+
|
|
33
|
+
**Resist everything else.** Until an objective trigger fires, the floor has no elaborate memory (default: explicit per-turn context, minimal retrieval), no multi-agent (default: one orchestrator, specialists in-process), no externalized state (default: in-memory, single instance), and no crossed process boundary (a sidecar only when a mature library in another ecosystem demands it). These deferrals are not omissions: each is named with its trigger and handled in `iterate` when the signal arrives.
|
|
34
|
+
|
|
35
|
+
**Never defer two things.** The working model adapter, and baseline governance — observability, guardrails, and a simple per-run cost ceiling that stops and synthesizes on overrun. These are floor components, not later tiers. Deferral applies to memory, multi-agent, external state, and distribution — never to the fact that the app runs and is governed.
|
|
36
|
+
|
|
37
|
+
**Prove the floor before declaring it done.**
|
|
38
|
+
|
|
39
|
+
1. Wire it to real traffic or a representative sample — not to cases picked to impress.
|
|
40
|
+
2. Measure against the success criterion fixed at framing. Without this measurement you cannot decide the next tier.
|
|
41
|
+
3. Verify observability holds: a full trajectory reconstructs after the fact from its request ID. Without that, the system is ungovernable.
|
|
42
|
+
|
|
43
|
+
Lock any structuring decision met during the build through `decision-loop`. Write the floor note and pass it through `review`.
|
|
44
|
+
|
|
45
|
+
## Definition of Done
|
|
46
|
+
|
|
47
|
+
- The floor runs on real traffic through the existing infrastructure.
|
|
48
|
+
- Value measured against the observable success criterion.
|
|
49
|
+
- A complete trajectory replays from a single request ID.
|
|
50
|
+
- Every deferral named with its trigger.
|
|
51
|
+
- Floor note produced and reviewed.
|
|
52
|
+
|
|
53
|
+
## Anti-patterns
|
|
54
|
+
|
|
55
|
+
- Shipping a demo and calling it a floor — a system that never touches real traffic proves nothing.
|
|
56
|
+
- Leaving the model port in simulation mode by default.
|
|
57
|
+
- Letting the model perform work that deterministic code can do.
|
|
58
|
+
- Retrofitting observability after the fact — it always costs more.
|
|
59
|
+
- Adding memory, agents, or distribution "while we're at it".
|
|
60
|
+
- Freezing the target: its richness is earned tier by tier, never built in one block.
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
# Frame — Decide the Perimeter Before Writing Code
|
|
2
|
+
|
|
3
|
+
## When to use
|
|
4
|
+
|
|
5
|
+
Use this workflow at the start of any agentic-system mission, or whenever the perimeter is fuzzy: "we'd like an agent that…", "what do we build first?", "do we need multi-agent / long-term memory?", "which language, which stack?" (answer: not yet). This is phase 1 of `method`. It turns a vague intent into a decided perimeter: what gets built first, what is explicitly deferred, and under which conditions complexity will be added.
|
|
6
|
+
|
|
7
|
+
## Inputs
|
|
8
|
+
|
|
9
|
+
- A sponsor and a real business problem.
|
|
10
|
+
- Access to the people who run the process today.
|
|
11
|
+
- The Definition of Ready checklist below.
|
|
12
|
+
|
|
13
|
+
## Outputs
|
|
14
|
+
|
|
15
|
+
- A framing note (use the `mission/framing.md` template).
|
|
16
|
+
- The floor/target split with named deferrals.
|
|
17
|
+
- Presumed architecture boundaries — ports and integration protocol, stack left open.
|
|
18
|
+
|
|
19
|
+
## Procedure
|
|
20
|
+
|
|
21
|
+
**Refuse stack questions.** If the conversation opens with framework, language, or provider, push it back: those are adapter decisions, reversible, taken later behind stable contracts. Framing decides the problem and the floor, not the technology.
|
|
22
|
+
|
|
23
|
+
**Check the Definition of Ready.** Eight conditions in three families.
|
|
24
|
+
|
|
25
|
+
- **Mandate**: a real problem with an identified sponsor; an observable success criterion; acceptance of the floor-first principle.
|
|
26
|
+
- **Access**: to the real process and its people; to usable data or a path to it; to the technical infrastructure.
|
|
27
|
+
- **Constraints**: hard constraints known (sovereignty, regulation, legacy integration); a human available to decide and approve sensitive actions.
|
|
28
|
+
|
|
29
|
+
Treat these as a floor, not a barrier: a missing condition becomes a named risk, owned by the sponsor, and often the first object of discovery — it is not a compliance turnstile.
|
|
30
|
+
|
|
31
|
+
**Establish entry mode and stopping tier first.** Present both as explicit choices to the sponsor: where the work starts (greenfield, or an existing system — in which case route through `brownfield` first) and how far it goes (framing, floor, full chain). Detail lives in `method`.
|
|
32
|
+
|
|
33
|
+
**Run discovery in five question families, in order.**
|
|
34
|
+
|
|
35
|
+
1. **The real process.** How does it actually happen today — frictions, manual rework, waiting? Hunt the observed process, not the idealized one.
|
|
36
|
+
2. **The value.** Where is value created if this works? Time saved, errors avoided, quality raised, new service? Who benefits, how often?
|
|
37
|
+
3. **The success criterion.** What observable, measurable fact would say "it works" on real traffic? Reject impressions.
|
|
38
|
+
4. **The data.** What comes in, what goes out, from where, to where? Sensitive data, expected traceability.
|
|
39
|
+
5. **The hard constraints.** Sovereignty, regulated sector, legacy integration, confidentiality, mandatory human approval. These bound the solution space; tool preferences do not.
|
|
40
|
+
|
|
41
|
+
**Split floor from target.** The floor is the smallest system that proves value on real traffic — typically a single orchestrator, a model port, persistence, a few guardrails, baseline observability, nothing more without a trigger. The target is the full architecture you are heading toward — elaborate memory, multi-agent, externalized state, distribution, continuous evaluation. Name the target to set direction; do not build it up front. Every deferral is written down with the trigger that will reactivate it.
|
|
42
|
+
|
|
43
|
+
**Apply the upstream decision matrix.** For each structuring choice, record a lean default and the objective trigger that commands a switch. Apply the default; switch only on signal.
|
|
44
|
+
|
|
45
|
+
- **Topology**: default single agent; trigger — genuinely independent, parallelizable subtasks.
|
|
46
|
+
- **Memory**: default explicit per-turn context; trigger — a measured need for cross-session continuity.
|
|
47
|
+
- **State**: default in-memory, single instance; trigger — replication or cross-process sharing.
|
|
48
|
+
- **Process boundary**: default one process; trigger — a capability whose lifecycle, ecosystem, or scale truly differs.
|
|
49
|
+
- **Model tier**: default one balanced tier; trigger up — measured quality demands it; trigger down — deterministic tasks routed cheaper.
|
|
50
|
+
- **Determinism**: default — everything that can be deterministic (classification, validation, guardrails) leaves the model for testable code; the model only reasons.
|
|
51
|
+
- **Untrusted input and privilege**: default — retrieved content is data, never instruction; least privilege; human approval on sensitive actions. In regulated sectors this is a framing constraint, not a late add-on.
|
|
52
|
+
|
|
53
|
+
Lock any truly structuring decision through `decision-loop` before committing it to the note.
|
|
54
|
+
|
|
55
|
+
## Definition of Done
|
|
56
|
+
|
|
57
|
+
- Framing note produced: problem, value, observable success criterion, hard constraints — one page.
|
|
58
|
+
- Floor perimeter listed, every deferral named with its trigger.
|
|
59
|
+
- Presumed boundaries stated; language and topology explicitly left open.
|
|
60
|
+
- Note reviewed via `review` before circulation.
|
|
61
|
+
- Deliverable form matched: the presented note carries an expected delivery form (a readable PDF or HTML); engineering artifacts stay as repository markdown.
|
|
62
|
+
|
|
63
|
+
## Anti-patterns
|
|
64
|
+
|
|
65
|
+
- Deciding the stack during framing.
|
|
66
|
+
- Accepting a fuzzy success criterion — without it, the floor can never be proven.
|
|
67
|
+
- Padding the floor with untriggered features.
|
|
68
|
+
- Discovering a sovereignty or compliance constraint after the floor is built.
|
|
69
|
+
- Declaring the phase done without its artifact, silently.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Govern — Trust Through Instrumentation, From Day Zero
|
|
2
|
+
|
|
3
|
+
## When to use
|
|
4
|
+
|
|
5
|
+
Use this workflow whenever a system must be made reliable, auditable, or safe: "how do we secure this?", "we need observability", "how do you test an agent?", "do we add human approval?", "how do we stop prompt injection?". This is phase 5 of `method` and it is transversal: it starts at day zero of phase 3, wired onto the floor and maintained at every tier — never after the incident. Trust is built by instrumentation; retrofitting always costs more.
|
|
6
|
+
|
|
7
|
+
## Inputs
|
|
8
|
+
|
|
9
|
+
- A floor (or later tier) with its middleware chain and tool registry.
|
|
10
|
+
- The `mission/threat-model.md`, `mission/evaluation-rubric.md`, and `mission/observability-schema.md` templates.
|
|
11
|
+
|
|
12
|
+
## Outputs
|
|
13
|
+
|
|
14
|
+
- A single instrumented middleware chain.
|
|
15
|
+
- A threat model, an evaluation rubric with hold-out, an observability schema.
|
|
16
|
+
- Cost ceilings and approval points live in production.
|
|
17
|
+
|
|
18
|
+
## Procedure
|
|
19
|
+
|
|
20
|
+
**Route all transversal concerns through one middleware chain** on the tool registry — never scatter them.
|
|
21
|
+
|
|
22
|
+
- **Structured logging**: one line per event, typed context, plus persisted lifecycle events and per-model-call metrics (tokens, tier, duration, status).
|
|
23
|
+
- **Propagated request ID**: from entry to every tool and model call, and down to sub-agents through parent/child lineage — the thread that reconstructs a full trajectory.
|
|
24
|
+
- **Provenance**: per inference, keep a fingerprint of what was actually injected, so you can replay what the model saw even after working memory is gone.
|
|
25
|
+
- **Cost control**: the model boundary (deterministic work pays no model call), tier routing, caching, and explicit ceilings per root task and per time window — from day zero. On overrun the orchestrator stops and returns a synthesis instead of running open-ended.
|
|
26
|
+
- **Approval point**: tools with impact declare in their contract that they require approval. The guard is enforced by infrastructure, never by the model's discipline.
|
|
27
|
+
|
|
28
|
+
The chain carries transversal concerns only — no orchestration, no business logic. Keep the channel thin and the registry an index. One trace stream feeds provenance, observability, and evaluation separately.
|
|
29
|
+
|
|
30
|
+
**Keep the interactive turn thin — and govern the background like the foreground.** The turn does only what the answer needs; everything else (fact extraction, memory consolidation, episode summaries, context pre-warming) leaves the turn for an event-driven post-turn pipeline: each step isolated with its own bounded retry, one step's failure never failing the others, the provenance fingerprint deposited at emission. Periodic upkeep — score decay, pruning derived links past retention, cold-archiving old journal segments, trend detection, reminders, enforced erasure — belongs to a scheduled cron that updates and cools but never destroys the truth. Every background job carries four non-negotiable guardrails: bounded retry per step; idempotency (keys plus atomic deduplication, including under concurrency); bounded concurrency (a global cap and a per-user partition); and job observability — queue lag and failure rate are first-class metrics. The craft rules `async-post-turn-pipeline`, `async-scheduled-maintenance` and `async-job-guardrails` carry the detail.
|
|
31
|
+
|
|
32
|
+
**Make resilience the default.** Qualify every failure first, because the response depends on the type: transient — bounded exponential-backoff retry; validation — one retry with the diagnostic fed back; business — diagnosis or human escalation; model provider down — automatic fallback behind the same port; non-critical service down — a signaled degraded mode. Then apply the rule: **fail-open on reads, fail-closed on actions.** Cache, observability, enrichment degrade silently; a sensitive action (mutation, write, external push) fails closed and explicit rather than executing in doubt. Degrade reading, never acting. Add feature detection at startup: a minimal mode that runs on the strict necessary, a full mode when all dependencies are present, and an interface that says what is active instead of failing silently.
|
|
33
|
+
|
|
34
|
+
**Constrain security by architecture, not detection.** Prompt injection is the first-rank threat, intrinsic to any memory or retrieval; detection is unreliable, so defend structurally: retrieved content is data, never instruction; least privilege on tools (registry filtered by role before the model sees it); ownership guards before mutations; schema validation on outputs; an immutable log that keeps every injected action traceable. Apply the **lethal trifecta** rule: while untrusted content is in the context window, allow at most **two of three** — private data access, untrusted content, external communication; if all three are needed, the action runs under human supervision. The window opens at ingestion and closes only when the content is purged. For high-privilege agents, add dedicated patterns: pre-approved action sets, a plan frozen before exposure to tool outputs, quarantine of untrusted content, or a privileged planner split from a read-only model. Approval summaries are deterministic and faithful to the tool's real arguments — never a model paraphrase. An awaiting-approval agent suspends: state serialized durably, resources freed, rehydrated when the decision arrives.
|
|
35
|
+
|
|
36
|
+
**Test and evaluate — both.** Build the test pyramid: unit tests without network (pure domain, mocked model adapter); consumer-driven contract tests that catch drift between schema and real data; integration through the injection container with mock adapters; behavioral evaluations at the top. Then run the continuous evaluation loop: sample the trace stream off the hot path; score hybrid — deterministic checks wherever a guarantee exists, an anchored judge model (pinned version or replayed anchor set) only for the irreducibly behavioral, abstention first. The loop is valid only under a **hold-out the optimizer never sees**; self-tuning stays inside a pre-approved, audited envelope or goes through human validation — never autonomous self-rewriting. The hard floor — safety, security, authorization, audit — stays deterministic; never assemble it from soft judgments. Promote a new model via shadow deployment: same port, real traffic, silent; measure divergence with the same evaluation; roll out in stages with instant rollback.
|
|
37
|
+
|
|
38
|
+
## Definition of Done
|
|
39
|
+
|
|
40
|
+
- All transversal concerns pass through the single middleware chain; a trajectory replays from one request ID.
|
|
41
|
+
- Cost ceilings enforced with stop-and-synthesize behavior.
|
|
42
|
+
- Threat model written; two-of-three trifecta rule enforced while untrusted content is in the context window.
|
|
43
|
+
- Test pyramid in place; evaluation loop running with anchored judge and hold-out.
|
|
44
|
+
- Background work guarded: pipeline steps isolated and idempotent, the maintenance cron never deletes truth, queue lag and failure rate alerting in place.
|
|
45
|
+
|
|
46
|
+
## Anti-patterns
|
|
47
|
+
|
|
48
|
+
- Postponing instrumentation until after the first incident.
|
|
49
|
+
- Treating the model as a trust boundary — once untrusted content is ingested, the action space must narrow.
|
|
50
|
+
- Letting the model write its own approval summaries.
|
|
51
|
+
- Closing the evaluation loop without a hold-out: it drifts toward flattering the judge.
|
|
52
|
+
- Handing the hard floor to a judge model.
|
|
53
|
+
- Failing open on a sensitive action.
|
|
54
|
+
- Running consolidation, extraction or summarization inside the interactive turn — the user pays latency for work the answer does not need.
|
|
55
|
+
- A cleanup cron that deletes from the immutable journal.
|