agentboot 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -7
- package/agentboot.config.json +4 -1
- package/package.json +2 -2
- package/scripts/cli.ts +42 -14
- package/scripts/compile.ts +30 -7
- package/scripts/dev-sync.ts +1 -1
- package/scripts/lib/config.ts +17 -1
- package/scripts/validate.ts +12 -7
- package/.github/ISSUE_TEMPLATE/persona-request.md +0 -62
- package/.github/ISSUE_TEMPLATE/quality-feedback.md +0 -67
- package/.github/workflows/cla.yml +0 -25
- package/.github/workflows/validate.yml +0 -49
- package/.idea/agentboot.iml +0 -9
- package/.idea/misc.xml +0 -6
- package/.idea/modules.xml +0 -8
- package/.idea/vcs.xml +0 -6
- package/CLAUDE.md +0 -230
- package/CONTRIBUTING.md +0 -168
- package/PERSONAS.md +0 -156
- package/core/instructions/baseline.instructions.md +0 -133
- package/core/instructions/security.instructions.md +0 -186
- package/core/personas/code-reviewer/SKILL.md +0 -175
- package/core/personas/security-reviewer/SKILL.md +0 -233
- package/core/personas/test-data-expert/SKILL.md +0 -234
- package/core/personas/test-generator/SKILL.md +0 -262
- package/core/traits/audit-trail.md +0 -182
- package/core/traits/confidence-signaling.md +0 -172
- package/core/traits/critical-thinking.md +0 -129
- package/core/traits/schema-awareness.md +0 -132
- package/core/traits/source-citation.md +0 -174
- package/core/traits/structured-output.md +0 -199
- package/docs/ci-cd-automation.md +0 -548
- package/docs/claude-code-reference/README.md +0 -21
- package/docs/claude-code-reference/agentboot-coverage.md +0 -484
- package/docs/claude-code-reference/feature-inventory.md +0 -906
- package/docs/cli-commands-audit.md +0 -112
- package/docs/cli-design.md +0 -924
- package/docs/concepts.md +0 -1117
- package/docs/config-schema-audit.md +0 -121
- package/docs/configuration.md +0 -645
- package/docs/delivery-methods.md +0 -758
- package/docs/developer-onboarding.md +0 -342
- package/docs/extending.md +0 -448
- package/docs/getting-started.md +0 -298
- package/docs/knowledge-layer.md +0 -464
- package/docs/marketplace.md +0 -822
- package/docs/org-connection.md +0 -570
- package/docs/plans/architecture.md +0 -2429
- package/docs/plans/design.md +0 -2018
- package/docs/plans/prd.md +0 -1862
- package/docs/plans/stack-rank.md +0 -261
- package/docs/plans/technical-spec.md +0 -2755
- package/docs/privacy-and-safety.md +0 -807
- package/docs/prompt-optimization.md +0 -1071
- package/docs/test-plan.md +0 -972
- package/docs/third-party-ecosystem.md +0 -496
- package/domains/compliance-template/README.md +0 -173
- package/domains/compliance-template/traits/compliance-aware.md +0 -228
- package/examples/enterprise/agentboot.config.json +0 -184
- package/examples/minimal/agentboot.config.json +0 -46
- package/tests/REGRESSION-PLAN.md +0 -705
- package/tests/TEST-PLAN.md +0 -111
- package/tests/cli.test.ts +0 -705
- package/tests/pipeline.test.ts +0 -608
- package/tests/validate.test.ts +0 -278
- package/tsconfig.json +0 -62
|
@@ -1,2755 +0,0 @@
|
|
|
1
|
-
# AgentBoot Technical Specification
|
|
2
|
-
|
|
3
|
-
Version: 0.1.0-draft
|
|
4
|
-
Status: Implementation Blueprint
|
|
5
|
-
License: Apache-2.0
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## 1. Summary
|
|
10
|
-
|
|
11
|
-
AgentBoot is a build tool that compiles agentic personas into distributable artifacts for engineering teams. It operates on a hub-and-spoke model: a central personas repository (hub) produces compiled persona definitions that are synced to target repositories (spokes). The pipeline is validate, compile, sync.
|
|
12
|
-
|
|
13
|
-
The system manages four core primitives: **personas** (agent definitions with structured prompts), **traits** (reusable behavioral building blocks with weight-based composition), **instructions** (always-on guardrails), and **rules** (path-scoped gotchas). These primitives are composed at build time into platform-specific output formats: a native format using `@import`-based CLAUDE.md with full agent frontmatter, and a cross-platform format using standalone inlined SKILL.md files compatible with the agentskills.io specification.
|
|
14
|
-
|
|
15
|
-
The CLI (`agentboot`) is the primary interface for all operations: interactive onboarding, persona compilation, distribution, linting, testing, cost estimation, telemetry aggregation, marketplace publishing, and diagnostics. It is distributed as a compiled binary via native package managers (brew, apt, choco) with an npm/npx fallback.
|
|
16
|
-
|
|
17
|
-
AgentBoot enforces a four-level scope hierarchy (Org, Group, Team, Repo) where more specific scopes layer on top of general ones. Optional behaviors follow team-wins-on-conflict semantics; mandatory behaviors follow org-wins inheritance. The sync system writes a manifest tracking every managed file, enabling clean uninstall and non-destructive upgrades.
|
|
18
|
-
|
|
19
|
-
The optimization subsystem provides static prompt analysis (lint), behavioral testing against known inputs, snapshot regression detection, LLM-as-judge evaluation, token budget enforcement, cost projection, and structured telemetry collection. All developer-facing optimization tools run locally first; CI gates run on PR submission.
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## 2. Technology Stack
|
|
24
|
-
|
|
25
|
-
### Language and Runtime
|
|
26
|
-
|
|
27
|
-
| Component | Technology | Notes |
|
|
28
|
-
|-----------|-----------|-------|
|
|
29
|
-
| Source language | TypeScript (ES modules) | `"type": "module"` in package.json |
|
|
30
|
-
| Runtime | Node.js 18+ | `"engines": {"node": ">=18"}` |
|
|
31
|
-
| Script executor | tsx 4.16+ | Used for `scripts/*.ts` execution |
|
|
32
|
-
| Type checking | TypeScript 5.5+ | `tsc --noEmit` for type validation |
|
|
33
|
-
| Package manager | npm | Lockfile committed |
|
|
34
|
-
|
|
35
|
-
### Dependencies (Production)
|
|
36
|
-
|
|
37
|
-
| Package | Version | Purpose |
|
|
38
|
-
|---------|---------|---------|
|
|
39
|
-
| chalk | ^5.3.0 | Terminal colorized output |
|
|
40
|
-
| glob | ^11.0.0 | File pattern matching |
|
|
41
|
-
| zod | ^3.23.8 | Schema validation for configs and frontmatter |
|
|
42
|
-
|
|
43
|
-
### Dependencies (Development)
|
|
44
|
-
|
|
45
|
-
| Package | Version | Purpose |
|
|
46
|
-
|---------|---------|---------|
|
|
47
|
-
| @types/node | ^22.0.0 | Node.js type definitions |
|
|
48
|
-
| tsx | ^4.16.0 | TypeScript execution without pre-compilation |
|
|
49
|
-
| typescript | ^5.5.0 | Type checking and compilation |
|
|
50
|
-
| vitest | ^2.0.0 | Test framework |
|
|
51
|
-
|
|
52
|
-
### Planned Dependencies (Not Yet Installed)
|
|
53
|
-
|
|
54
|
-
| Package | Purpose | Required For |
|
|
55
|
-
|---------|---------|-------------|
|
|
56
|
-
| inquirer or @inquirer/prompts | Interactive setup wizard prompts | `agentboot setup` |
|
|
57
|
-
| commander or yargs | CLI argument parsing and subcommand routing | CLI binary |
|
|
58
|
-
| js-yaml | YAML parsing for test files | `agentboot test` |
|
|
59
|
-
| tiktoken or gpt-tokenizer | Token counting without API calls | `agentboot lint`, `agentboot cost-estimate` |
|
|
60
|
-
| better-sqlite3 | SQLite for knowledge layer Stage 2 | `agentboot build --index` |
|
|
61
|
-
| semver | Version comparison for upgrades | `agentboot upgrade` |
|
|
62
|
-
|
|
63
|
-
### Terminal UI Stack
|
|
64
|
-
|
|
65
|
-
| Component | Library | Purpose |
|
|
66
|
-
|-----------|---------|---------|
|
|
67
|
-
| Colored output | chalk | Severity indicators, status messages |
|
|
68
|
-
| Interactive prompts | inquirer (recommended) | Setup wizard question flow, confirmations |
|
|
69
|
-
| Spinners | ora (recommended) | Progress indicators for build/sync |
|
|
70
|
-
| Tables | cli-table3 (recommended) | Status dashboard, metrics display |
|
|
71
|
-
|
|
72
|
-
### Build and Distribution
|
|
73
|
-
|
|
74
|
-
| Target | Method | Binary Size | Runtime Dependency |
|
|
75
|
-
|--------|--------|-------------|-------------------|
|
|
76
|
-
| npm/npx | `npm publish` | N/A (source) | Node.js 18+ |
|
|
77
|
-
| brew (macOS) | `agentboot/homebrew-tap` formula pointing to GitHub Releases | ~50MB (bundled) | None |
|
|
78
|
-
| apt (Debian/Ubuntu) | `.deb` package from GitHub Releases | ~50MB (bundled) | None |
|
|
79
|
-
| dnf (RHEL/Fedora) | `.rpm` package from GitHub Releases | ~50MB (bundled) | None |
|
|
80
|
-
| choco (Windows) | Chocolatey package | ~50MB (bundled) | None |
|
|
81
|
-
| winget (Windows) | Windows Package Manager manifest | ~50MB (bundled) | None |
|
|
82
|
-
|
|
83
|
-
**Build strategy (V1):** TypeScript compiled via `bun build --compile` or `pkg` to produce a single executable with Node.js runtime embedded. The binary is uploaded to GitHub Releases. Package manager formulae download and install the pre-built binary.
|
|
84
|
-
|
|
85
|
-
**Future migration path:** If binary size (~50MB) or startup time becomes a problem, migrate the CLI to Go (cobra for CLI, bubbletea for TUI). The core logic (validate, compile, sync) is I/O-bound, not CPU-bound, so language performance is not a bottleneck.
|
|
86
|
-
|
|
87
|
-
### Test Framework
|
|
88
|
-
|
|
89
|
-
| Component | Tool |
|
|
90
|
-
|-----------|------|
|
|
91
|
-
| Test runner | vitest 2.0+ |
|
|
92
|
-
| Assertions | vitest built-in matchers |
|
|
93
|
-
| Custom matchers | Frontmatter validation, token counting |
|
|
94
|
-
| Fixtures | `tests/fixtures/` directory |
|
|
95
|
-
| Coverage | vitest c8 provider |
|
|
96
|
-
| CI | GitHub Actions |
|
|
97
|
-
|
|
98
|
-
---
|
|
99
|
-
|
|
100
|
-
## 3. CLI Specification
|
|
101
|
-
|
|
102
|
-
The CLI binary is registered at `./dist/scripts/cli.js` in package.json under `"bin": {"agentboot": ...}`. All commands support `--non-interactive` mode (auto-detected when stdout is not a TTY). All commands support `--help` and `--version` flags.
|
|
103
|
-
|
|
104
|
-
### Global Flags
|
|
105
|
-
|
|
106
|
-
| Flag | Type | Default | Description |
|
|
107
|
-
|------|------|---------|-------------|
|
|
108
|
-
| `--help` | boolean | false | Show help text |
|
|
109
|
-
| `--version` | boolean | false | Print version and exit |
|
|
110
|
-
| `--non-interactive` | boolean | auto-detect | Disable interactive prompts |
|
|
111
|
-
| `--verbose` | boolean | false | Show detailed output |
|
|
112
|
-
| `--quiet` | boolean | false | Suppress non-error output |
|
|
113
|
-
| `--format` | string | `"text"` | Output format: `text`, `json` |
|
|
114
|
-
| `--config` | string | `"./agentboot.config.json"` | Path to config file |
|
|
115
|
-
|
|
116
|
-
### Exit Code Convention
|
|
117
|
-
|
|
118
|
-
| Code | Meaning |
|
|
119
|
-
|------|---------|
|
|
120
|
-
| 0 | Success |
|
|
121
|
-
| 1 | Error (invalid input, build failure, test failure) |
|
|
122
|
-
| 2 | Warning (with `--strict` flag, warnings become errors) |
|
|
123
|
-
|
|
124
|
-
---
|
|
125
|
-
|
|
126
|
-
### 3.1 `agentboot setup`
|
|
127
|
-
|
|
128
|
-
Interactive onboarding wizard that determines role, tooling, and organizational context, then executes the appropriate setup.
|
|
129
|
-
|
|
130
|
-
**Signature:**
|
|
131
|
-
```
|
|
132
|
-
agentboot setup [--role <role>] [--tool <tool>] [--org <org>] [--compliance <type>] [--skip-detect]
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
**Flags:**
|
|
136
|
-
|
|
137
|
-
| Flag | Type | Default | Description |
|
|
138
|
-
|------|------|---------|-------------|
|
|
139
|
-
| `--role` | string | (prompt) | `developer`, `platform`, `it-security`, `exploring` |
|
|
140
|
-
| `--tool` | string | (prompt) | `claude-code`, `copilot`, `cursor`, `mixed`, `none` |
|
|
141
|
-
| `--org` | string | (auto-detect) | Organization identifier |
|
|
142
|
-
| `--compliance` | string[] | (prompt) | `hipaa`, `soc2`, `pci-dss`, `gdpr`, `none` |
|
|
143
|
-
| `--skip-detect` | boolean | false | Skip auto-detection of existing infrastructure |
|
|
144
|
-
| `--mdm` | string | (prompt) | MDM platform: `jamf`, `intune`, `jumpcloud`, `kandji`, `other`, `none` |
|
|
145
|
-
|
|
146
|
-
**Input:**
|
|
147
|
-
- Interactive TTY prompts (questions Q1-Q8 per the decision tree in cli-design.md)
|
|
148
|
-
- Auto-detection reads: git remote, `.claude/` directory, managed settings paths, `claude --version`
|
|
149
|
-
|
|
150
|
-
**Output (Quick Start -- solo developer):**
|
|
151
|
-
- `.claude/agents/{name}/CLAUDE.md` for each default persona
|
|
152
|
-
- `.claude/skills/{name}/SKILL.md` for each default skill
|
|
153
|
-
- `.claude/traits/{name}.md` for each default trait
|
|
154
|
-
- `.claude/CLAUDE.md` with `@import` references
|
|
155
|
-
|
|
156
|
-
**Output (Standard Setup -- platform team):**
|
|
157
|
-
- `agentboot.config.json` with org structure
|
|
158
|
-
- `repos.json` (empty array)
|
|
159
|
-
- `core/personas/` with 4 default persona directories
|
|
160
|
-
- `core/traits/` with default trait files
|
|
161
|
-
- `core/instructions/` with baseline always-on instructions
|
|
162
|
-
|
|
163
|
-
**Output (Enterprise Setup -- IT/MDM):**
|
|
164
|
-
- `agentboot.config.json` (enterprise template)
|
|
165
|
-
- `domains/compliance/` with applicable domain layer
|
|
166
|
-
- `dist/managed/managed-settings.json`
|
|
167
|
-
- `dist/managed/managed-mcp.json`
|
|
168
|
-
- `dist/managed/CLAUDE.md`
|
|
169
|
-
- Marketplace template directory
|
|
170
|
-
|
|
171
|
-
**Exit Codes:**
|
|
172
|
-
- 0: Setup completed successfully
|
|
173
|
-
- 1: Setup failed (filesystem error, invalid input)
|
|
174
|
-
|
|
175
|
-
**Error Handling:**
|
|
176
|
-
- If git remote detection fails, prompt for org name manually
|
|
177
|
-
- If `.claude/` already exists, warn and offer to merge or skip
|
|
178
|
-
- If managed settings are detected, inform user they are already governed
|
|
179
|
-
- All filesystem writes are atomic: create temp files then rename
|
|
180
|
-
|
|
181
|
-
**Example:**
|
|
182
|
-
```bash
|
|
183
|
-
# Non-interactive quick start
|
|
184
|
-
agentboot setup --role exploring
|
|
185
|
-
|
|
186
|
-
# Non-interactive platform team setup
|
|
187
|
-
agentboot setup --role platform --tool claude-code --org acme --compliance soc2
|
|
188
|
-
|
|
189
|
-
# Non-interactive enterprise with MDM
|
|
190
|
-
agentboot setup --role it-security --mdm jamf --compliance hipaa,soc2
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
---
|
|
194
|
-
|
|
195
|
-
### 3.2 `agentboot connect`
|
|
196
|
-
|
|
197
|
-
Developer self-service command to connect to an organization's existing AgentBoot marketplace.
|
|
198
|
-
|
|
199
|
-
**Signature:**
|
|
200
|
-
```
|
|
201
|
-
agentboot connect <org-or-marketplace> [--marketplace <repo>] [--url <url>]
|
|
202
|
-
```
|
|
203
|
-
|
|
204
|
-
**Arguments:**
|
|
205
|
-
|
|
206
|
-
| Argument | Type | Required | Description |
|
|
207
|
-
|----------|------|----------|-------------|
|
|
208
|
-
| `<org-or-marketplace>` | string | Yes (unless `--marketplace` or `--url`) | Org identifier for auto-detection |
|
|
209
|
-
|
|
210
|
-
**Flags:**
|
|
211
|
-
|
|
212
|
-
| Flag | Type | Default | Description |
|
|
213
|
-
|------|------|---------|-------------|
|
|
214
|
-
| `--marketplace` | string | (auto-detect) | Explicit marketplace repository slug (e.g., `acme/personas`) |
|
|
215
|
-
| `--url` | string | none | Full URL for self-hosted or non-GitHub marketplaces |
|
|
216
|
-
|
|
217
|
-
**Input:**
|
|
218
|
-
- Organization name or marketplace identifier
|
|
219
|
-
- Reads current git remote for context
|
|
220
|
-
|
|
221
|
-
**Output:**
|
|
222
|
-
- Adds marketplace to the local configuration
|
|
223
|
-
- Installs the organization plugin
|
|
224
|
-
- Prints verification status
|
|
225
|
-
|
|
226
|
-
**Algorithm:**
|
|
227
|
-
1. Resolve marketplace location:
|
|
228
|
-
- If `--marketplace` or `--url` provided, use directly
|
|
229
|
-
- Else, attempt to resolve `<org>` by checking `https://github.com/<org>/<org>-personas`
|
|
230
|
-
- Fall back to prompting for the marketplace URL
|
|
231
|
-
2. Run `claude plugin install <marketplace>` (or equivalent API call)
|
|
232
|
-
3. Verify installation by checking plugin presence in config
|
|
233
|
-
4. Print success message with available personas
|
|
234
|
-
|
|
235
|
-
**Exit Codes:**
|
|
236
|
-
- 0: Connected successfully
|
|
237
|
-
- 1: Connection failed (marketplace not found, network error, plugin install failed)
|
|
238
|
-
|
|
239
|
-
**Error Handling:**
|
|
240
|
-
- If marketplace repository does not exist, print: `Marketplace not found at <url>. Check the URL or ask your platform team.`
|
|
241
|
-
- If plugin install fails, print the underlying error and suggest `agentboot doctor`
|
|
242
|
-
- If already connected, print: `Already connected to <org>. Plugin <name> is active.`
|
|
243
|
-
|
|
244
|
-
**Example:**
|
|
245
|
-
```bash
|
|
246
|
-
agentboot connect acme
|
|
247
|
-
agentboot connect --marketplace acme/personas
|
|
248
|
-
agentboot connect --url https://gitlab.internal/platform/personas
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
---
|
|
252
|
-
|
|
253
|
-
### 3.3 `agentboot build`
|
|
254
|
-
|
|
255
|
-
Compile personas with trait composition, producing platform-specific output artifacts.
|
|
256
|
-
|
|
257
|
-
**Signature:**
|
|
258
|
-
```
|
|
259
|
-
agentboot build [--format <format>] [--validate-only] [--persona <name>] [--index] [--embeddings]
|
|
260
|
-
```
|
|
261
|
-
|
|
262
|
-
**Flags:**
|
|
263
|
-
|
|
264
|
-
| Flag | Type | Default | Description |
|
|
265
|
-
|------|------|---------|-------------|
|
|
266
|
-
| `--format` | string | `"all"` | Output format: `claude-code`, `copilot`, `cross-platform`, `plugin`, `all` |
|
|
267
|
-
| `--validate-only` | boolean | false | Dry run: check for errors without writing files |
|
|
268
|
-
| `--persona` | string | all | Build a specific persona only |
|
|
269
|
-
| `--index` | boolean | false | Generate SQLite knowledge index (Stage 2) |
|
|
270
|
-
| `--embeddings` | boolean | false | Generate vector embeddings (Stage 3) |
|
|
271
|
-
| `--fix` | boolean | false | Auto-generate missing `persona.config.json` files |
|
|
272
|
-
|
|
273
|
-
**Input:**
|
|
274
|
-
- `agentboot.config.json` (root config)
|
|
275
|
-
- `core/personas/{name}/SKILL.md` (persona definitions)
|
|
276
|
-
- `core/personas/{name}/persona.config.json` (trait composition config)
|
|
277
|
-
- `core/traits/*.md` (trait files)
|
|
278
|
-
- `core/instructions/*.md` (always-on instructions)
|
|
279
|
-
- Domain layer directories (from `extend.domains`)
|
|
280
|
-
|
|
281
|
-
**Output:**
|
|
282
|
-
- `dist/core/` -- org-level compiled personas
|
|
283
|
-
- `dist/groups/{group}/` -- group-level overrides
|
|
284
|
-
- `dist/teams/{group}/{team}/` -- team-level overrides
|
|
285
|
-
- `PERSONAS.md` -- human-readable registry (if `output.personas_registry` is true)
|
|
286
|
-
|
|
287
|
-
**Output structure per persona (claude-code format):**
|
|
288
|
-
```
|
|
289
|
-
dist/core/{persona-name}/
|
|
290
|
-
.claude/
|
|
291
|
-
agents/{persona-name}/CLAUDE.md # Full agent frontmatter
|
|
292
|
-
skills/{skill-name}/SKILL.md # With context: fork
|
|
293
|
-
rules/{topic}.md # With paths: frontmatter
|
|
294
|
-
traits/{trait-name}.md # Separate files for @import
|
|
295
|
-
CLAUDE.md # Using @imports
|
|
296
|
-
settings.json # Hook entries
|
|
297
|
-
.mcp.json # MCP server configs
|
|
298
|
-
```
|
|
299
|
-
|
|
300
|
-
**Output structure per persona (cross-platform format):**
|
|
301
|
-
```
|
|
302
|
-
dist/core/{persona-name}/
|
|
303
|
-
SKILL.md # Standalone, traits inlined
|
|
304
|
-
copilot-instructions.md # Copilot-compatible output
|
|
305
|
-
```
|
|
306
|
-
|
|
307
|
-
**Exit Codes:**
|
|
308
|
-
- 0: Build succeeded
|
|
309
|
-
- 1: Build failed (missing traits, invalid config, schema errors)
|
|
310
|
-
- 2: Build succeeded with warnings (missing optional fields)
|
|
311
|
-
|
|
312
|
-
**Error Handling:**
|
|
313
|
-
- Missing `persona.config.json`: error with message `persona.config.json missing for: {name}. Run: agentboot add persona-config {name}`
|
|
314
|
-
- Missing trait reference: error with message `Trait '{trait}' referenced by persona '{persona}' does not exist in core/traits/`
|
|
315
|
-
- Circular trait dependency: error with message `Circular trait dependency detected: {A} -> {B} -> {A}`
|
|
316
|
-
- Invalid SKILL.md frontmatter: error with schema validation details
|
|
317
|
-
|
|
318
|
-
**Example:**
|
|
319
|
-
```bash
|
|
320
|
-
agentboot build
|
|
321
|
-
agentboot build --format claude-code
|
|
322
|
-
agentboot build --validate-only
|
|
323
|
-
agentboot build --persona code-reviewer
|
|
324
|
-
agentboot build --index
|
|
325
|
-
```
|
|
326
|
-
|
|
327
|
-
---
|
|
328
|
-
|
|
329
|
-
### 3.4 `agentboot sync`
|
|
330
|
-
|
|
331
|
-
Distribute compiled output to target repositories.
|
|
332
|
-
|
|
333
|
-
**Signature:**
|
|
334
|
-
```
|
|
335
|
-
agentboot sync [--repo <slug>] [--dry-run] [--mode <mode>]
|
|
336
|
-
```
|
|
337
|
-
|
|
338
|
-
**Flags:**
|
|
339
|
-
|
|
340
|
-
| Flag | Type | Default | Description |
|
|
341
|
-
|------|------|---------|-------------|
|
|
342
|
-
| `--repo` | string | all repos | Sync to a specific repo only |
|
|
343
|
-
| `--dry-run` | boolean | false | Show what would change without writing |
|
|
344
|
-
| `--mode` | string | from config | `local`, `github-api`, `gitlab-api` |
|
|
345
|
-
|
|
346
|
-
**Input:**
|
|
347
|
-
- `dist/` directory (compiled output from `agentboot build`)
|
|
348
|
-
- `repos.json` or inline repo config from `agentboot.config.json`
|
|
349
|
-
- `agentboot.config.json` (for scope hierarchy resolution)
|
|
350
|
-
|
|
351
|
-
**Output (local mode):**
|
|
352
|
-
- Files written to `{repo.path}/{output.dir}/` for each registered repo
|
|
353
|
-
- `.claude/.agentboot-manifest.json` written in each target repo
|
|
354
|
-
- Git branch created and PR opened (if `sync.pr.enabled` is true)
|
|
355
|
-
|
|
356
|
-
**Output (github-api mode):**
|
|
357
|
-
- PRs created in each target repo via GitHub API
|
|
358
|
-
- Branch name: `{sync.pr.branch_prefix}{date-or-version}`
|
|
359
|
-
- PR title: rendered from `sync.pr.title_template`
|
|
360
|
-
|
|
361
|
-
**Exit Codes:**
|
|
362
|
-
- 0: Sync completed (all repos succeeded)
|
|
363
|
-
- 1: Sync failed (at least one repo failed)
|
|
364
|
-
|
|
365
|
-
**Error Handling:**
|
|
366
|
-
- Repo path does not exist (local mode): error `Repo path not found: {path}. Is the repo cloned?`
|
|
367
|
-
- Dirty working tree (local mode with PR): error `Repo {name} has uncommitted changes. Commit or stash first.`
|
|
368
|
-
- GitHub API auth failure: error `GITHUB_TOKEN not set or insufficient permissions for {repo}.`
|
|
369
|
-
- Repo not in repos.json: error `Repo '{slug}' not found in repos.json.`
|
|
370
|
-
|
|
371
|
-
**Example:**
|
|
372
|
-
```bash
|
|
373
|
-
agentboot sync
|
|
374
|
-
agentboot sync --repo acme/api-service
|
|
375
|
-
agentboot sync --dry-run
|
|
376
|
-
agentboot sync --mode github-api
|
|
377
|
-
```
|
|
378
|
-
|
|
379
|
-
---
|
|
380
|
-
|
|
381
|
-
### 3.5 `agentboot export`
|
|
382
|
-
|
|
383
|
-
Generate distributable artifacts in specific formats.
|
|
384
|
-
|
|
385
|
-
**Signature:**
|
|
386
|
-
```
|
|
387
|
-
agentboot export --format <format> [--output <dir>]
|
|
388
|
-
```
|
|
389
|
-
|
|
390
|
-
**Flags:**
|
|
391
|
-
|
|
392
|
-
| Flag | Type | Default | Description |
|
|
393
|
-
|------|------|---------|-------------|
|
|
394
|
-
| `--format` | string | required | `plugin`, `marketplace`, `managed`, `github-action`, `mcp-server` |
|
|
395
|
-
| `--output` | string | `"dist/"` | Output directory |
|
|
396
|
-
|
|
397
|
-
**Input:**
|
|
398
|
-
- Compiled `dist/` directory
|
|
399
|
-
- `agentboot.config.json`
|
|
400
|
-
|
|
401
|
-
**Output by format:**
|
|
402
|
-
|
|
403
|
-
| Format | Output |
|
|
404
|
-
|--------|--------|
|
|
405
|
-
| `plugin` | `.claude-plugin/plugin.json`, `agents/`, `skills/`, `hooks/` |
|
|
406
|
-
| `marketplace` | Full marketplace repo scaffold with `marketplace.json` |
|
|
407
|
-
| `managed` | `managed-settings.json`, `managed-mcp.json`, `CLAUDE.md` for MDM paths |
|
|
408
|
-
| `github-action` | `.github/workflows/agentboot-review.yml` reusable workflow |
|
|
409
|
-
| `mcp-server` | MCP server package with tool definitions |
|
|
410
|
-
|
|
411
|
-
**Exit Codes:**
|
|
412
|
-
- 0: Export succeeded
|
|
413
|
-
- 1: Export failed (missing dist/, invalid config)
|
|
414
|
-
|
|
415
|
-
**Error Handling:**
|
|
416
|
-
- `dist/` does not exist: error `No compiled output found. Run 'agentboot build' first.`
|
|
417
|
-
- Unknown format: error `Unknown export format: '{format}'. Valid: plugin, marketplace, managed, github-action, mcp-server`
|
|
418
|
-
|
|
419
|
-
**Example:**
|
|
420
|
-
```bash
|
|
421
|
-
agentboot export --format plugin
|
|
422
|
-
agentboot export --format managed --output ./deploy/
|
|
423
|
-
```
|
|
424
|
-
|
|
425
|
-
---
|
|
426
|
-
|
|
427
|
-
### 3.6 `agentboot publish`
|
|
428
|
-
|
|
429
|
-
Push compiled plugin to a marketplace repository.
|
|
430
|
-
|
|
431
|
-
**Signature:**
|
|
432
|
-
```
|
|
433
|
-
agentboot publish [--marketplace <path>] [--bump <level>]
|
|
434
|
-
```
|
|
435
|
-
|
|
436
|
-
**Flags:**
|
|
437
|
-
|
|
438
|
-
| Flag | Type | Default | Description |
|
|
439
|
-
|------|------|---------|-------------|
|
|
440
|
-
| `--marketplace` | string | from config | Path to marketplace repository |
|
|
441
|
-
| `--bump` | string | none | Version bump level: `patch`, `minor`, `major` |
|
|
442
|
-
| `--dry-run` | boolean | false | Preview what would be published |
|
|
443
|
-
|
|
444
|
-
**Input:**
|
|
445
|
-
- Plugin output from `agentboot export --format plugin`
|
|
446
|
-
- `marketplace.json` from marketplace repository
|
|
447
|
-
- `package.json` for current version
|
|
448
|
-
|
|
449
|
-
**Output:**
|
|
450
|
-
- Updated `marketplace.json` with new version entry
|
|
451
|
-
- Plugin files copied to marketplace directory
|
|
452
|
-
- Git commit and push (if not `--dry-run`)
|
|
453
|
-
|
|
454
|
-
**Algorithm:**
|
|
455
|
-
1. If `--bump` specified, increment version in `package.json` and `plugin.json`
|
|
456
|
-
2. Copy plugin artifacts to marketplace directory
|
|
457
|
-
3. Update `marketplace.json` with new entry (version, hash, timestamp)
|
|
458
|
-
4. Commit: `chore: publish agentboot plugin v{version}`
|
|
459
|
-
5. Push to remote
|
|
460
|
-
|
|
461
|
-
**Exit Codes:**
|
|
462
|
-
- 0: Published successfully
|
|
463
|
-
- 1: Publish failed (no plugin output, git error, push rejected)
|
|
464
|
-
|
|
465
|
-
**Error Handling:**
|
|
466
|
-
- No plugin output: error `No plugin found. Run 'agentboot export --format plugin' first.`
|
|
467
|
-
- Marketplace repo not clean: error `Marketplace repo has uncommitted changes.`
|
|
468
|
-
- Push rejected: error with git push output
|
|
469
|
-
|
|
470
|
-
**Example:**
|
|
471
|
-
```bash
|
|
472
|
-
agentboot publish
|
|
473
|
-
agentboot publish --bump patch
|
|
474
|
-
agentboot publish --marketplace ./acme-personas --bump minor
|
|
475
|
-
```
|
|
476
|
-
|
|
477
|
-
---
|
|
478
|
-
|
|
479
|
-
### 3.7 `agentboot add`
|
|
480
|
-
|
|
481
|
-
Scaffold new components.
|
|
482
|
-
|
|
483
|
-
**Signature:**
|
|
484
|
-
```
|
|
485
|
-
agentboot add <type> <name> [options]
|
|
486
|
-
```
|
|
487
|
-
|
|
488
|
-
**Subcommands:**
|
|
489
|
-
|
|
490
|
-
| Subcommand | Description |
|
|
491
|
-
|------------|-------------|
|
|
492
|
-
| `persona <name>` | Scaffold a new persona directory |
|
|
493
|
-
| `trait <name>` | Scaffold a new trait file |
|
|
494
|
-
| `domain <name>` | Add a domain layer template |
|
|
495
|
-
| `gotcha <name>` | Add a gotchas rule template |
|
|
496
|
-
| `hook <name>` | Add a compliance hook template |
|
|
497
|
-
| `repo <slug>` | Register a repo for sync |
|
|
498
|
-
|
|
499
|
-
**Output per subcommand:**
|
|
500
|
-
|
|
501
|
-
| Subcommand | Files Created |
|
|
502
|
-
|------------|--------------|
|
|
503
|
-
| `persona` | `core/personas/{name}/SKILL.md`, `core/personas/{name}/persona.config.json` |
|
|
504
|
-
| `trait` | `core/traits/{name}.md` |
|
|
505
|
-
| `domain` | `domains/{name}/` with template structure |
|
|
506
|
-
| `gotcha` | `.claude/rules/{name}.md` with `paths:` frontmatter template |
|
|
507
|
-
| `hook` | `.claude/hooks/{name}.sh` with hook script template |
|
|
508
|
-
| `repo` | Appends entry to `repos.json` |
|
|
509
|
-
|
|
510
|
-
**Exit Codes:**
|
|
511
|
-
- 0: Component created
|
|
512
|
-
- 1: Component already exists or invalid name
|
|
513
|
-
|
|
514
|
-
**Error Handling:**
|
|
515
|
-
- Name already exists: error `Persona '{name}' already exists at core/personas/{name}/`
|
|
516
|
-
- Invalid name (non-lowercase, special chars): error `Name must be lowercase alphanumeric with hyphens: got '{name}'`
|
|
517
|
-
|
|
518
|
-
**Example:**
|
|
519
|
-
```bash
|
|
520
|
-
agentboot add persona api-reviewer
|
|
521
|
-
agentboot add trait cost-awareness
|
|
522
|
-
agentboot add domain healthcare
|
|
523
|
-
agentboot add repo acme/new-service
|
|
524
|
-
```
|
|
525
|
-
|
|
526
|
-
---
|
|
527
|
-
|
|
528
|
-
### 3.8 `agentboot add prompt`
|
|
529
|
-
|
|
530
|
-
Ingest raw prompts: classify, format, and save as proper AgentBoot content.
|
|
531
|
-
|
|
532
|
-
**Signature:**
|
|
533
|
-
```
|
|
534
|
-
agentboot add prompt <text> [--file <path>] [--clipboard] [--url <url>] [--batch] [--dry-run] [--stdin]
|
|
535
|
-
```
|
|
536
|
-
|
|
537
|
-
**Flags:**
|
|
538
|
-
|
|
539
|
-
| Flag | Type | Default | Description |
|
|
540
|
-
|------|------|---------|-------------|
|
|
541
|
-
| `--file` | string | none | Read prompt from file |
|
|
542
|
-
| `--clipboard` | boolean | false | Read prompt from system clipboard |
|
|
543
|
-
| `--url` | string | none | Fetch prompt from URL |
|
|
544
|
-
| `--batch` | boolean | false | Decompose multi-instruction file into individual components |
|
|
545
|
-
| `--dry-run` | boolean | false | Preview classification without writing |
|
|
546
|
-
| `--stdin` | boolean | false | Read from stdin |
|
|
547
|
-
| `--interactive` | boolean | false | Open editor for multi-line input |
|
|
548
|
-
|
|
549
|
-
**Input:**
|
|
550
|
-
- Raw text from one of: argument, file, clipboard, URL, stdin, or editor
|
|
551
|
-
|
|
552
|
-
**Classification Algorithm:**
|
|
553
|
-
1. Send raw text to LLM (Haiku for speed) with classification prompt
|
|
554
|
-
2. LLM analyzes and returns:
|
|
555
|
-
- `type`: `rule`, `gotcha`, `trait`, `persona`, `session-instruction`, `rejected`
|
|
556
|
-
- `name`: suggested kebab-case name
|
|
557
|
-
- `scope`: suggested path patterns (for gotchas)
|
|
558
|
-
- `content`: formatted markdown with appropriate frontmatter
|
|
559
|
-
3. Present classification to user for confirmation
|
|
560
|
-
4. On approval, write to appropriate location
|
|
561
|
-
|
|
562
|
-
**Classification Signals:**
|
|
563
|
-
|
|
564
|
-
| Signal in Prompt | Classified As | Destination |
|
|
565
|
-
|-----------------|---------------|-------------|
|
|
566
|
-
| "Always...", "Never...", "Verify that..." | Rule / Gotcha | `.claude/rules/` |
|
|
567
|
-
| Technology-specific warning with examples | Gotcha (path-scoped) | `.claude/rules/` with `paths:` |
|
|
568
|
-
| Behavioral stance ("be skeptical", "cite sources") | Trait | `core/traits/` |
|
|
569
|
-
| Complete review workflow with output format | Persona | `core/personas/` |
|
|
570
|
-
| Single-use instruction | Session instruction | Not persisted |
|
|
571
|
-
| Vague/motivational | Rejected | User feedback with improvement suggestion |
|
|
572
|
-
|
|
573
|
-
**Output:**
|
|
574
|
-
- One or more files written to the appropriate location
|
|
575
|
-
- Lint results for the generated content
|
|
576
|
-
- Token impact estimate
|
|
577
|
-
|
|
578
|
-
**Exit Codes:**
|
|
579
|
-
- 0: Prompt ingested successfully
|
|
580
|
-
- 1: Failed to classify or write
|
|
581
|
-
- 2: Dry run completed (no files written)
|
|
582
|
-
|
|
583
|
-
**Error Handling:**
|
|
584
|
-
- URL fetch fails: error `Could not fetch content from {url}: {http-error}`
|
|
585
|
-
- Clipboard empty: error `Clipboard is empty.`
|
|
586
|
-
- Classification rejected: print `This prompt is too vague to be actionable. Try: {suggestion}`
|
|
587
|
-
|
|
588
|
-
**Example:**
|
|
589
|
-
```bash
|
|
590
|
-
agentboot add prompt "Always check null safety before DB calls"
|
|
591
|
-
agentboot add prompt --file ~/tips.md --batch
|
|
592
|
-
agentboot add prompt --clipboard --dry-run
|
|
593
|
-
agentboot add prompt --url https://blog.example.com/gotchas
|
|
594
|
-
```
|
|
595
|
-
|
|
596
|
-
---
|
|
597
|
-
|
|
598
|
-
### 3.9 `agentboot discover`
|
|
599
|
-
|
|
600
|
-
Scan repositories and local configuration for existing agentic content.
|
|
601
|
-
|
|
602
|
-
**Signature:**
|
|
603
|
-
```
|
|
604
|
-
agentboot discover [--path <dir>] [--repos <slugs...>] [--github-org <org>] [--local] [--all] [--format <fmt>]
|
|
605
|
-
```
|
|
606
|
-
|
|
607
|
-
**Flags:**
|
|
608
|
-
|
|
609
|
-
| Flag | Type | Default | Description |
|
|
610
|
-
|------|------|---------|-------------|
|
|
611
|
-
| `--path` | string | `.` | Directory containing repos to scan |
|
|
612
|
-
| `--repos` | string[] | none | Specific repo slugs to scan |
|
|
613
|
-
| `--github-org` | string | none | Scan all repos in a GitHub org via API |
|
|
614
|
-
| `--local` | boolean | false | Scan local machine config (`~/.claude/`) |
|
|
615
|
-
| `--all` | boolean | false | Full scan: repos + local + managed settings |
|
|
616
|
-
| `--format` | string | `"text"` | Output: `text`, `json`, `markdown` |
|
|
617
|
-
|
|
618
|
-
**Scan Targets:**
|
|
619
|
-
|
|
620
|
-
| Target | Files Scanned | Method |
|
|
621
|
-
|--------|--------------|--------|
|
|
622
|
-
| `.claude/` | CLAUDE.md, agents, skills, rules, hooks, settings, .mcp.json | Filesystem |
|
|
623
|
-
| `.github/` | copilot-instructions.md, prompts/*.prompt.md | Filesystem |
|
|
624
|
-
| `.cursor/` | .cursorrules, rules/ | Filesystem |
|
|
625
|
-
| Repo root | CLAUDE.md, GEMINI.md, .mcp.json | Filesystem |
|
|
626
|
-
| Subdirectories | Nested CLAUDE.md files | Recursive scan |
|
|
627
|
-
| `~/.claude/` | User agents, skills, rules, CLAUDE.md, settings | Filesystem |
|
|
628
|
-
| Managed paths | managed-settings.json, managed-mcp.json | OS-specific paths |
|
|
629
|
-
| GitHub API | Repo contents via `gh api` | HTTP |
|
|
630
|
-
| package.json | Scripts referencing AI tools | Pattern match |
|
|
631
|
-
| Git history | Recent .claude/ changes | `git log` |
|
|
632
|
-
|
|
633
|
-
**Non-Destructive Guarantee:** Discovery never modifies, moves, or deletes existing files. All actions create new files in the AgentBoot personas repo. Originals stay untouched.
|
|
634
|
-
|
|
635
|
-
**Output (interactive):**
|
|
636
|
-
After scanning, presents 5 action options:
|
|
637
|
-
1. Generate detailed report (Markdown)
|
|
638
|
-
2. Classify and ingest (batch `add prompt` on each file)
|
|
639
|
-
3. Show overlap analysis (duplicate/similar content across repos)
|
|
640
|
-
4. Show migration plan (what becomes traits, gotchas, personas, always-on)
|
|
641
|
-
5. Export as `agentboot.config.json` (infer org structure from repos)
|
|
642
|
-
|
|
643
|
-
**Exit Codes:**
|
|
644
|
-
- 0: Scan completed
|
|
645
|
-
- 1: Scan failed (filesystem permission, API error)
|
|
646
|
-
|
|
647
|
-
**Error Handling:**
|
|
648
|
-
- Path does not exist: error `Directory not found: {path}`
|
|
649
|
-
- GitHub API rate limit: warn and continue with partial results
|
|
650
|
-
- Permission denied on file: skip with warning
|
|
651
|
-
|
|
652
|
-
**Example:**
|
|
653
|
-
```bash
|
|
654
|
-
agentboot discover
|
|
655
|
-
agentboot discover --path ~/work/
|
|
656
|
-
agentboot discover --github-org acme --local
|
|
657
|
-
agentboot discover --all --format json > report.json
|
|
658
|
-
```
|
|
659
|
-
|
|
660
|
-
---
|
|
661
|
-
|
|
662
|
-
### 3.10 `agentboot validate`
|
|
663
|
-
|
|
664
|
-
CI-friendly schema and config validation.
|
|
665
|
-
|
|
666
|
-
**Signature:**
|
|
667
|
-
```
|
|
668
|
-
agentboot validate [--personas] [--traits] [--config] [--strict]
|
|
669
|
-
```
|
|
670
|
-
|
|
671
|
-
**Flags:**
|
|
672
|
-
|
|
673
|
-
| Flag | Type | Default | Description |
|
|
674
|
-
|------|------|---------|-------------|
|
|
675
|
-
| `--personas` | boolean | false | Validate personas only |
|
|
676
|
-
| `--traits` | boolean | false | Validate traits only |
|
|
677
|
-
| `--config` | boolean | false | Validate config only |
|
|
678
|
-
| `--strict` | boolean | false | Treat warnings as errors |
|
|
679
|
-
|
|
680
|
-
When no scope flag is provided, validates everything.
|
|
681
|
-
|
|
682
|
-
**Validation Checks (5 checks from validate.ts):**
|
|
683
|
-
|
|
684
|
-
| Check | ID | What It Verifies |
|
|
685
|
-
|-------|----|-----------------|
|
|
686
|
-
| Persona existence | `persona-exists` | Every persona in `personas.enabled` has a directory under `core/personas/` |
|
|
687
|
-
| Trait references | `trait-refs` | Every trait referenced in `persona.config.json` exists in `core/traits/` or extension dirs |
|
|
688
|
-
| SKILL.md frontmatter | `frontmatter` | Every persona SKILL.md has valid YAML frontmatter with required fields |
|
|
689
|
-
| PERSONAS.md sync | `registry-sync` | PERSONAS.md is up to date with the current set of compiled personas |
|
|
690
|
-
| Secret scanning | `no-secrets` | No API keys, tokens, passwords, or internal URLs in any persona or trait file |
|
|
691
|
-
|
|
692
|
-
**Input:**
|
|
693
|
-
- `agentboot.config.json`
|
|
694
|
-
- All files under `core/personas/`, `core/traits/`, `core/instructions/`
|
|
695
|
-
- Extension directories referenced in config
|
|
696
|
-
|
|
697
|
-
**Output:**
|
|
698
|
-
- Line-by-line pass/fail for each check
|
|
699
|
-
- Error messages with fix suggestions
|
|
700
|
-
|
|
701
|
-
**Exit Codes:**
|
|
702
|
-
- 0: All checks passed
|
|
703
|
-
- 1: At least one error
|
|
704
|
-
- 2: Warnings only (fails when `--strict` is set)
|
|
705
|
-
|
|
706
|
-
**Error Handling:**
|
|
707
|
-
- Missing config file: error `agentboot.config.json not found. Run 'agentboot setup' first.`
|
|
708
|
-
- Each validation failure includes a fix command suggestion
|
|
709
|
-
|
|
710
|
-
**Example:**
|
|
711
|
-
```bash
|
|
712
|
-
agentboot validate
|
|
713
|
-
agentboot validate --strict
|
|
714
|
-
agentboot validate --personas
|
|
715
|
-
```
|
|
716
|
-
|
|
717
|
-
---
|
|
718
|
-
|
|
719
|
-
### 3.11 `agentboot lint`
|
|
720
|
-
|
|
721
|
-
Static prompt analysis covering token budgets, language quality, and security.
|
|
722
|
-
|
|
723
|
-
**Signature:**
|
|
724
|
-
```
|
|
725
|
-
agentboot lint [--fix] [--persona <name>] [--severity <level>] [--ci] [--format <fmt>]
|
|
726
|
-
```
|
|
727
|
-
|
|
728
|
-
**Flags:**
|
|
729
|
-
|
|
730
|
-
| Flag | Type | Default | Description |
|
|
731
|
-
|------|------|---------|-------------|
|
|
732
|
-
| `--fix` | boolean | false | Auto-fix what is possible (trim whitespace, formatting) |
|
|
733
|
-
| `--persona` | string | all | Lint a specific persona only |
|
|
734
|
-
| `--severity` | string | `"warn"` | Minimum severity to report: `info`, `warn`, `error` |
|
|
735
|
-
| `--ci` | boolean | false | CI mode: summary counts only, full detail in log |
|
|
736
|
-
| `--format` | string | `"text"` | Output: `text`, `json` |
|
|
737
|
-
|
|
738
|
-
**Input:**
|
|
739
|
-
- Source files in `core/personas/`, `core/traits/`, `core/instructions/`
|
|
740
|
-
- `agentboot.config.json` (for custom rule overrides and token budget config)
|
|
741
|
-
|
|
742
|
-
**Output:**
|
|
743
|
-
- List of findings with rule ID, severity, file, line, message, and fix suggestion
|
|
744
|
-
- Summary counts by severity
|
|
745
|
-
|
|
746
|
-
See Section 6 (Lint Specification) for the full rule catalog.
|
|
747
|
-
|
|
748
|
-
**Exit Codes:**
|
|
749
|
-
- 0: No errors (warnings allowed)
|
|
750
|
-
- 1: At least one error
|
|
751
|
-
- 2: Warnings present (with `--severity error`, warnings do not cause non-zero exit)
|
|
752
|
-
|
|
753
|
-
**Example:**
|
|
754
|
-
```bash
|
|
755
|
-
agentboot lint
|
|
756
|
-
agentboot lint --fix
|
|
757
|
-
agentboot lint --persona code-reviewer
|
|
758
|
-
agentboot lint --severity error --format json --ci
|
|
759
|
-
```
|
|
760
|
-
|
|
761
|
-
---
|
|
762
|
-
|
|
763
|
-
### 3.12 `agentboot test`
|
|
764
|
-
|
|
765
|
-
Persona testing across multiple layers: deterministic, behavioral, snapshot, eval, and mutation.
|
|
766
|
-
|
|
767
|
-
**Signature:**
|
|
768
|
-
```
|
|
769
|
-
agentboot test [--type <type>] [--persona <name>] [--model <model>] [--max-budget <usd>] [--ci] [--update-snapshots]
|
|
770
|
-
```
|
|
771
|
-
|
|
772
|
-
**Flags:**
|
|
773
|
-
|
|
774
|
-
| Flag | Type | Default | Description |
|
|
775
|
-
|------|------|---------|-------------|
|
|
776
|
-
| `--type` | string | `"all"` | Test type: `deterministic`, `behavioral`, `snapshot`, `eval`, `mutation`, `all` |
|
|
777
|
-
| `--persona` | string | all | Test a specific persona only |
|
|
778
|
-
| `--model` | string | from test file | Override model for behavioral tests |
|
|
779
|
-
| `--max-budget` | number | 5.00 | Maximum USD spend for the test run |
|
|
780
|
-
| `--ci` | boolean | false | CI mode: exit codes + JSON summary only |
|
|
781
|
-
| `--update-snapshots` | boolean | false | Update snapshot baselines |
|
|
782
|
-
| `--skip-if-behavioral-passed` | boolean | false | Skip eval tests if behavioral passed |
|
|
783
|
-
|
|
784
|
-
**Input:**
|
|
785
|
-
- `tests/*.test.yaml` (behavioral test definitions)
|
|
786
|
-
- `tests/eval/*.eval.yaml` (LLM-as-judge definitions)
|
|
787
|
-
- `tests/snapshots/*.json` (snapshot baselines)
|
|
788
|
-
- `tests/fixtures/` (known-buggy code samples)
|
|
789
|
-
|
|
790
|
-
See Section 7 (Test Specification) for the canonical test file format and assertion types.
|
|
791
|
-
|
|
792
|
-
**Exit Codes:**
|
|
793
|
-
- 0: All tests passed
|
|
794
|
-
- 1: At least one test failed
|
|
795
|
-
- 2: Budget exceeded before all tests completed
|
|
796
|
-
|
|
797
|
-
**Example:**
|
|
798
|
-
```bash
|
|
799
|
-
agentboot test
|
|
800
|
-
agentboot test --type deterministic
|
|
801
|
-
agentboot test --type behavioral --persona security-reviewer --max-budget 2.00
|
|
802
|
-
agentboot test --type snapshot --update-snapshots
|
|
803
|
-
agentboot test --type eval --persona code-reviewer
|
|
804
|
-
```
|
|
805
|
-
|
|
806
|
-
---
|
|
807
|
-
|
|
808
|
-
### 3.13 `agentboot search`
|
|
809
|
-
|
|
810
|
-
Search the marketplace for traits, gotchas, personas, and domains.
|
|
811
|
-
|
|
812
|
-
**Signature:**
|
|
813
|
-
```
|
|
814
|
-
agentboot search <query> [--type <type>] [--marketplace <url>]
|
|
815
|
-
```
|
|
816
|
-
|
|
817
|
-
**Flags:**
|
|
818
|
-
|
|
819
|
-
| Flag | Type | Default | Description |
|
|
820
|
-
|------|------|---------|-------------|
|
|
821
|
-
| `--type` | string | `"all"` | Filter: `trait`, `gotcha`, `persona`, `domain`, `all` |
|
|
822
|
-
| `--marketplace` | string | default registry | Marketplace URL to search |
|
|
823
|
-
|
|
824
|
-
**Input:**
|
|
825
|
-
- Query string
|
|
826
|
-
- Marketplace index (fetched from marketplace URL or cached locally)
|
|
827
|
-
|
|
828
|
-
**Output:**
|
|
829
|
-
- List of matching items with name, description, type, author, version, install command
|
|
830
|
-
|
|
831
|
-
**Exit Codes:**
|
|
832
|
-
- 0: Results found (or no results, still success)
|
|
833
|
-
- 1: Marketplace unreachable
|
|
834
|
-
|
|
835
|
-
**Example:**
|
|
836
|
-
```bash
|
|
837
|
-
agentboot search "sql injection"
|
|
838
|
-
agentboot search "react" --type trait
|
|
839
|
-
agentboot search "hipaa" --type domain
|
|
840
|
-
```
|
|
841
|
-
|
|
842
|
-
---
|
|
843
|
-
|
|
844
|
-
### 3.14 `agentboot metrics`
|
|
845
|
-
|
|
846
|
-
Read telemetry data and produce per-persona, per-team, per-period reports.
|
|
847
|
-
|
|
848
|
-
**Signature:**
|
|
849
|
-
```
|
|
850
|
-
agentboot metrics [--persona <name>] [--team <name>] [--period <duration>] [--format <fmt>]
|
|
851
|
-
```
|
|
852
|
-
|
|
853
|
-
**Flags:**
|
|
854
|
-
|
|
855
|
-
| Flag | Type | Default | Description |
|
|
856
|
-
|------|------|---------|-------------|
|
|
857
|
-
| `--persona` | string | all | Filter to one persona |
|
|
858
|
-
| `--team` | string | all | Filter to one team |
|
|
859
|
-
| `--period` | string | `"30d"` | Time window: `7d`, `30d`, `90d`, `all` |
|
|
860
|
-
| `--format` | string | `"text"` | Output: `text`, `json`, `csv` |
|
|
861
|
-
|
|
862
|
-
**Input:**
|
|
863
|
-
- NDJSON telemetry log file(s) (default location: `~/.agentboot/telemetry.ndjson`, configurable via `agentboot.config.json` or `$AGENTBOOT_TELEMETRY_LOG`)
|
|
864
|
-
|
|
865
|
-
**Output:**
|
|
866
|
-
- Aggregated metrics per persona: invocation count, avg tokens, avg cost, avg duration, findings distribution
|
|
867
|
-
- Per-team rollup
|
|
868
|
-
- Cost trends over time
|
|
869
|
-
|
|
870
|
-
See Section 8 (Telemetry Specification) for the canonical event schema.
|
|
871
|
-
|
|
872
|
-
**Exit Codes:**
|
|
873
|
-
- 0: Report generated
|
|
874
|
-
- 1: No telemetry data found
|
|
875
|
-
|
|
876
|
-
**Example:**
|
|
877
|
-
```bash
|
|
878
|
-
agentboot metrics
|
|
879
|
-
agentboot metrics --persona code-reviewer --period 7d
|
|
880
|
-
agentboot metrics --team api --format json
|
|
881
|
-
```
|
|
882
|
-
|
|
883
|
-
---
|
|
884
|
-
|
|
885
|
-
### 3.15 `agentboot cost-estimate`
|
|
886
|
-
|
|
887
|
-
Project per-persona costs across the organization.
|
|
888
|
-
|
|
889
|
-
**Signature:**
|
|
890
|
-
```
|
|
891
|
-
agentboot cost-estimate [--developers <n>] [--invocations <n>] [--format <fmt>]
|
|
892
|
-
```
|
|
893
|
-
|
|
894
|
-
**Flags:**
|
|
895
|
-
|
|
896
|
-
| Flag | Type | Default | Description |
|
|
897
|
-
|------|------|---------|-------------|
|
|
898
|
-
| `--developers` | number | 50 | Number of developers |
|
|
899
|
-
| `--invocations` | number | 10 | Average invocations per developer per day |
|
|
900
|
-
| `--format` | string | `"text"` | Output: `text`, `json` |
|
|
901
|
-
|
|
902
|
-
**Input:**
|
|
903
|
-
- Compiled persona output (for token estimates)
|
|
904
|
-
- Model assignments from `persona.config.json`
|
|
905
|
-
|
|
906
|
-
**Algorithm:**
|
|
907
|
-
1. For each persona, estimate input tokens (persona prompt + traits + instructions + average file context)
|
|
908
|
-
2. For each persona, estimate output tokens (based on persona type: reviewer ~3k, generator ~8k)
|
|
909
|
-
3. Look up model pricing (Haiku: $0.80/$4.00 per M tokens, Sonnet: $3.00/$15.00, Opus: $15.00/$75.00)
|
|
910
|
-
4. Calculate per-invocation cost: `(input_tokens * input_price + output_tokens * output_price) / 1_000_000`
|
|
911
|
-
5. Calculate monthly cost: `per_invocation * developers * invocations * 21 (working days)`
|
|
912
|
-
6. Flag personas where Opus is used but Sonnet might suffice
|
|
913
|
-
|
|
914
|
-
**Output:**
|
|
915
|
-
- Per-persona table: model, estimated input/output tokens, estimated cost per invocation
|
|
916
|
-
- Monthly projection with developer count and invocation rate
|
|
917
|
-
- Optimization suggestions (model downgrade candidates)
|
|
918
|
-
|
|
919
|
-
**Exit Codes:**
|
|
920
|
-
- 0: Estimate generated
|
|
921
|
-
- 1: No compiled personas found
|
|
922
|
-
|
|
923
|
-
**Example:**
|
|
924
|
-
```bash
|
|
925
|
-
agentboot cost-estimate
|
|
926
|
-
agentboot cost-estimate --developers 100 --invocations 5
|
|
927
|
-
agentboot cost-estimate --format json
|
|
928
|
-
```
|
|
929
|
-
|
|
930
|
-
---
|
|
931
|
-
|
|
932
|
-
### 3.16 `agentboot review`
|
|
933
|
-
|
|
934
|
-
Guided human review of persona output samples.
|
|
935
|
-
|
|
936
|
-
**Signature:**
|
|
937
|
-
```
|
|
938
|
-
agentboot review --persona <name> [--sample <n>] [--period <duration>]
|
|
939
|
-
```
|
|
940
|
-
|
|
941
|
-
**Flags:**
|
|
942
|
-
|
|
943
|
-
| Flag | Type | Default | Description |
|
|
944
|
-
|------|------|---------|-------------|
|
|
945
|
-
| `--persona` | string | required | Persona to review |
|
|
946
|
-
| `--sample` | number | 5 | Number of samples to review |
|
|
947
|
-
| `--period` | string | `"7d"` | Time window for sampling |
|
|
948
|
-
|
|
949
|
-
**Input:**
|
|
950
|
-
- Telemetry log (to identify recent invocations)
|
|
951
|
-
- Persona output samples (from session transcripts or CI artifacts)
|
|
952
|
-
|
|
953
|
-
**Output:**
|
|
954
|
-
- Interactive review session with guided questions per sample:
|
|
955
|
-
- Accuracy assessment (Yes / Partially / No)
|
|
956
|
-
- Severity calibration check
|
|
957
|
-
- Missing findings
|
|
958
|
-
- Unnecessary findings
|
|
959
|
-
- Overall quality score and recommendation (Ship as-is / Needs tuning / Needs rewrite)
|
|
960
|
-
|
|
961
|
-
**Exit Codes:**
|
|
962
|
-
- 0: Review completed
|
|
963
|
-
- 1: No samples found
|
|
964
|
-
|
|
965
|
-
**Example:**
|
|
966
|
-
```bash
|
|
967
|
-
agentboot review --persona code-reviewer --sample 5
|
|
968
|
-
agentboot review --persona security-reviewer --period 30d
|
|
969
|
-
```
|
|
970
|
-
|
|
971
|
-
---
|
|
972
|
-
|
|
973
|
-
### 3.17 `agentboot issue`
|
|
974
|
-
|
|
975
|
-
Streamlined bug reporting against AgentBoot core.
|
|
976
|
-
|
|
977
|
-
**Signature:**
|
|
978
|
-
```
|
|
979
|
-
agentboot issue <title>
|
|
980
|
-
```
|
|
981
|
-
|
|
982
|
-
**Arguments:**
|
|
983
|
-
|
|
984
|
-
| Argument | Type | Required | Description |
|
|
985
|
-
|----------|------|----------|-------------|
|
|
986
|
-
| `<title>` | string | Yes | Issue title |
|
|
987
|
-
|
|
988
|
-
**Input:**
|
|
989
|
-
- Environment: AgentBoot version, Node.js version, OS
|
|
990
|
-
- Diagnosis output (if `agentboot doctor --diagnose` was run)
|
|
991
|
-
- `agentboot.config.json` (field names and types, values redacted)
|
|
992
|
-
|
|
993
|
-
**Output:**
|
|
994
|
-
- Opens GitHub issue creation in browser with pre-filled template
|
|
995
|
-
- Auto-attaches: version info, OS, diagnosis output, redacted config structure
|
|
996
|
-
- Never attaches: persona content, trait content, developer prompts, session transcripts
|
|
997
|
-
|
|
998
|
-
**Exit Codes:**
|
|
999
|
-
- 0: Issue page opened in browser
|
|
1000
|
-
- 1: Could not construct issue URL
|
|
1001
|
-
|
|
1002
|
-
**Example:**
|
|
1003
|
-
```bash
|
|
1004
|
-
agentboot issue "Build produces empty output on missing trait"
|
|
1005
|
-
```
|
|
1006
|
-
|
|
1007
|
-
---
|
|
1008
|
-
|
|
1009
|
-
### 3.18 `agentboot doctor`
|
|
1010
|
-
|
|
1011
|
-
Diagnose issues with configuration, personas, sync, and environment.
|
|
1012
|
-
|
|
1013
|
-
**Signature:**
|
|
1014
|
-
```
|
|
1015
|
-
agentboot doctor [--diagnose]
|
|
1016
|
-
```
|
|
1017
|
-
|
|
1018
|
-
**Flags:**
|
|
1019
|
-
|
|
1020
|
-
| Flag | Type | Default | Description |
|
|
1021
|
-
|------|------|---------|-------------|
|
|
1022
|
-
| `--diagnose` | boolean | false | Run layered isolation test to pinpoint bug location |
|
|
1023
|
-
|
|
1024
|
-
**Standard checks:**
|
|
1025
|
-
|
|
1026
|
-
| Category | Check |
|
|
1027
|
-
|----------|-------|
|
|
1028
|
-
| Environment | Claude Code installed, Node.js version, git version, gh CLI version |
|
|
1029
|
-
| Configuration | `agentboot.config.json` exists and validates, `repos.json` exists |
|
|
1030
|
-
| Personas | Each enabled persona has valid SKILL.md, persona.config.json, all traits resolve |
|
|
1031
|
-
| Sync Status | Each registered repo: last sync date, version, staleness |
|
|
1032
|
-
| Plugin | Plugin generated (yes/no), last publish date |
|
|
1033
|
-
| Managed Settings | Generated (yes/no), deployment target |
|
|
1034
|
-
|
|
1035
|
-
**Diagnose mode (5-layer isolation):**
|
|
1036
|
-
|
|
1037
|
-
| Layer | What it tests | If it fails |
|
|
1038
|
-
|-------|--------------|-------------|
|
|
1039
|
-
| 1: Core only | Core persona with zero customization | AgentBoot bug |
|
|
1040
|
-
| 2: Core + org config | Core persona with org config | Config issue |
|
|
1041
|
-
| 3: Core + config + org traits | Custom traits compose | Trait issue |
|
|
1042
|
-
| 4: Core + config + traits + org personas | Custom personas compile and lint | Persona issue |
|
|
1043
|
-
| 5: Full stack | Everything including extensions | Extension issue |
|
|
1044
|
-
|
|
1045
|
-
**Exit Codes:**
|
|
1046
|
-
- 0: All checks passed
|
|
1047
|
-
- 1: Issues found (with fix suggestions)
|
|
1048
|
-
|
|
1049
|
-
**Example:**
|
|
1050
|
-
```bash
|
|
1051
|
-
agentboot doctor
|
|
1052
|
-
agentboot doctor --diagnose
|
|
1053
|
-
```
|
|
1054
|
-
|
|
1055
|
-
---
|
|
1056
|
-
|
|
1057
|
-
### 3.19 `agentboot status`
|
|
1058
|
-
|
|
1059
|
-
Dashboard of what is deployed where.
|
|
1060
|
-
|
|
1061
|
-
**Signature:**
|
|
1062
|
-
```
|
|
1063
|
-
agentboot status [--format <fmt>]
|
|
1064
|
-
```
|
|
1065
|
-
|
|
1066
|
-
**Input:**
|
|
1067
|
-
- `agentboot.config.json`
|
|
1068
|
-
- `repos.json`
|
|
1069
|
-
- `.agentboot-manifest.json` in each registered repo (if accessible)
|
|
1070
|
-
- `package.json` for version
|
|
1071
|
-
|
|
1072
|
-
**Output:**
|
|
1073
|
-
- Org name and version
|
|
1074
|
-
- Enabled personas (count and names)
|
|
1075
|
-
- Enabled traits (count and names)
|
|
1076
|
-
- Registered repos grouped by team, with sync status (version, date, staleness)
|
|
1077
|
-
- Plugin status (name, version, marketplace URL, last published)
|
|
1078
|
-
- Managed settings status (generated, deployment target)
|
|
1079
|
-
|
|
1080
|
-
**Exit Codes:**
|
|
1081
|
-
- 0: Status retrieved
|
|
1082
|
-
|
|
1083
|
-
**Example:**
|
|
1084
|
-
```bash
|
|
1085
|
-
agentboot status
|
|
1086
|
-
agentboot status --format json
|
|
1087
|
-
```
|
|
1088
|
-
|
|
1089
|
-
---
|
|
1090
|
-
|
|
1091
|
-
### 3.20 `agentboot upgrade`
|
|
1092
|
-
|
|
1093
|
-
Update AgentBoot core.
|
|
1094
|
-
|
|
1095
|
-
**Signature:**
|
|
1096
|
-
```
|
|
1097
|
-
agentboot upgrade [--check] [--version <ver>]
|
|
1098
|
-
```
|
|
1099
|
-
|
|
1100
|
-
**Flags:**
|
|
1101
|
-
|
|
1102
|
-
| Flag | Type | Default | Description |
|
|
1103
|
-
|------|------|---------|-------------|
|
|
1104
|
-
| `--check` | boolean | false | Check for updates without applying |
|
|
1105
|
-
| `--version` | string | latest | Upgrade to a specific version |
|
|
1106
|
-
|
|
1107
|
-
**Algorithm:**
|
|
1108
|
-
1. Check current version from `package.json`
|
|
1109
|
-
2. Fetch latest version from npm registry (or GitHub Releases for binary installs)
|
|
1110
|
-
3. If `--check`, print comparison and exit
|
|
1111
|
-
4. Download and install new version
|
|
1112
|
-
5. Run `agentboot doctor` to verify post-upgrade health
|
|
1113
|
-
|
|
1114
|
-
**Exit Codes:**
|
|
1115
|
-
- 0: Upgrade successful (or already up to date)
|
|
1116
|
-
- 1: Upgrade failed
|
|
1117
|
-
|
|
1118
|
-
**Example:**
|
|
1119
|
-
```bash
|
|
1120
|
-
agentboot upgrade
|
|
1121
|
-
agentboot upgrade --check
|
|
1122
|
-
agentboot upgrade --version 2.0.0
|
|
1123
|
-
```
|
|
1124
|
-
|
|
1125
|
-
---
|
|
1126
|
-
|
|
1127
|
-
### 3.21 `agentboot uninstall`
|
|
1128
|
-
|
|
1129
|
-
Clean removal of AgentBoot from repos, plugins, and managed settings.
|
|
1130
|
-
|
|
1131
|
-
**Signature:**
|
|
1132
|
-
```
|
|
1133
|
-
agentboot uninstall [--repo <slug>] [--all-repos] [--plugin] [--managed] [--everything] [--dry-run]
|
|
1134
|
-
```
|
|
1135
|
-
|
|
1136
|
-
**Flags:**
|
|
1137
|
-
|
|
1138
|
-
| Flag | Type | Default | Description |
|
|
1139
|
-
|------|------|---------|-------------|
|
|
1140
|
-
| `--repo` | string | none | Remove from a specific repo |
|
|
1141
|
-
| `--all-repos` | boolean | false | Remove from all synced repos |
|
|
1142
|
-
| `--plugin` | boolean | false | Uninstall the plugin |
|
|
1143
|
-
| `--managed` | boolean | false | Generate IT instructions for managed settings removal |
|
|
1144
|
-
| `--everything` | boolean | false | Full removal |
|
|
1145
|
-
| `--dry-run` | boolean | false | Preview what would be removed |
|
|
1146
|
-
|
|
1147
|
-
**Algorithm:**
|
|
1148
|
-
1. Read `.agentboot-manifest.json` from target repo to identify managed files
|
|
1149
|
-
2. For each managed file:
|
|
1150
|
-
a. Compare file hash against manifest hash
|
|
1151
|
-
b. If hash matches: remove silently
|
|
1152
|
-
c. If hash differs: warn "modified after sync" and prompt for confirmation
|
|
1153
|
-
3. Check for mixed content in CLAUDE.md (both AgentBoot-generated and manual edits)
|
|
1154
|
-
a. If mixed: offer to remove only AgentBoot lines, or keep entire file
|
|
1155
|
-
4. Check for `.agentboot-archive/` with pre-AgentBoot originals
|
|
1156
|
-
a. If present: offer to restore originals
|
|
1157
|
-
5. For `--plugin`: run `claude plugin uninstall <plugin-name>`
|
|
1158
|
-
6. For `--managed`: generate IT instructions for MDM removal (cannot remove directly)
|
|
1159
|
-
|
|
1160
|
-
**Exit Codes:**
|
|
1161
|
-
- 0: Uninstall completed
|
|
1162
|
-
- 1: Uninstall failed
|
|
1163
|
-
|
|
1164
|
-
**Example:**
|
|
1165
|
-
```bash
|
|
1166
|
-
agentboot uninstall --repo acme/api-service --dry-run
|
|
1167
|
-
agentboot uninstall --all-repos
|
|
1168
|
-
agentboot uninstall --everything
|
|
1169
|
-
```
|
|
1170
|
-
|
|
1171
|
-
---
|
|
1172
|
-
|
|
1173
|
-
## 4. Build System Specification
|
|
1174
|
-
|
|
1175
|
-
### 4.1 validate.ts
|
|
1176
|
-
|
|
1177
|
-
The validation script runs 5 pre-build checks. Each check is independent and produces pass/fail with actionable error messages.
|
|
1178
|
-
|
|
1179
|
-
**Check 1: Persona Existence (`persona-exists`)**
|
|
1180
|
-
|
|
1181
|
-
| Property | Value |
|
|
1182
|
-
|----------|-------|
|
|
1183
|
-
| Input | `agentboot.config.json` `personas.enabled[]`, filesystem `core/personas/` |
|
|
1184
|
-
| Algorithm | For each ID in `personas.enabled`, verify `core/personas/{id}/` directory exists and contains `SKILL.md` |
|
|
1185
|
-
| Error message | `Persona '{id}' is enabled in config but directory core/personas/{id}/ does not exist.` |
|
|
1186
|
-
| Fix suggestion | `Run: agentboot add persona {id}` |
|
|
1187
|
-
|
|
1188
|
-
**Check 2: Trait References (`trait-refs`)**
|
|
1189
|
-
|
|
1190
|
-
| Property | Value |
|
|
1191
|
-
|----------|-------|
|
|
1192
|
-
| Input | All `persona.config.json` files, filesystem `core/traits/` |
|
|
1193
|
-
| Algorithm | For each trait referenced in any `persona.config.json`, verify file `core/traits/{trait}.md` exists |
|
|
1194
|
-
| Error message | `Trait '{trait}' referenced by persona '{persona}' does not exist in core/traits/` |
|
|
1195
|
-
| Fix suggestion | `Create core/traits/{trait}.md or remove the reference from core/personas/{persona}/persona.config.json` |
|
|
1196
|
-
|
|
1197
|
-
**Check 3: SKILL.md Frontmatter (`frontmatter`)**
|
|
1198
|
-
|
|
1199
|
-
| Property | Value |
|
|
1200
|
-
|----------|-------|
|
|
1201
|
-
| Input | All `core/personas/{name}/SKILL.md` files |
|
|
1202
|
-
| Algorithm | Parse YAML frontmatter, validate against zod schema requiring: `name` (string), `description` (string) |
|
|
1203
|
-
| Error message | `SKILL.md frontmatter validation failed for '{persona}': {zod error details}` |
|
|
1204
|
-
| Fix suggestion | `Edit core/personas/{persona}/SKILL.md and fix the frontmatter` |
|
|
1205
|
-
|
|
1206
|
-
**Check 4: PERSONAS.md Sync (`registry-sync`)**
|
|
1207
|
-
|
|
1208
|
-
| Property | Value |
|
|
1209
|
-
|----------|-------|
|
|
1210
|
-
| Input | `PERSONAS.md` in repo root, compiled persona list |
|
|
1211
|
-
| Algorithm | Generate expected PERSONAS.md content, compare against actual file (if `output.personas_registry` is true) |
|
|
1212
|
-
| Error message | `PERSONAS.md is out of date. Expected {n} personas, found {m}.` |
|
|
1213
|
-
| Fix suggestion | `Run: agentboot build to regenerate PERSONAS.md` |
|
|
1214
|
-
|
|
1215
|
-
**Check 5: Secret Scanning (`no-secrets`)**
|
|
1216
|
-
|
|
1217
|
-
| Property | Value |
|
|
1218
|
-
|----------|-------|
|
|
1219
|
-
| Input | All `.md` files in `core/`, all `*.json` config files |
|
|
1220
|
-
| Algorithm | Regex scan for patterns: API keys (`sk-`, `ghp_`, `AKIA`, `xox[bpsa]-`), tokens (JWT `eyJ`), passwords (`password\s*[:=]`), internal URLs (`https?://.*\.internal\.`, `https?://10\.`, `https?://192\.168\.`) |
|
|
1221
|
-
| Error message | `Potential secret found in {file}:{line}: matches pattern '{pattern}'` |
|
|
1222
|
-
| Fix suggestion | `Remove the secret from {file}. Use environment variables for sensitive values.` |
|
|
1223
|
-
|
|
1224
|
-
### 4.2 compile.ts
|
|
1225
|
-
|
|
1226
|
-
The compilation script loads config, resolves trait references, and emits output artifacts.
|
|
1227
|
-
|
|
1228
|
-
**Trait Resolution Algorithm:**
|
|
1229
|
-
|
|
1230
|
-
```
|
|
1231
|
-
function resolveTraits(persona: PersonaConfig, config: AgentBootConfig):
|
|
1232
|
-
1. Read persona.config.json for the persona
|
|
1233
|
-
2. For each trait entry in persona.config.json:
|
|
1234
|
-
a. Resolve trait file path: core/traits/{trait-id}.md
|
|
1235
|
-
- If not found, check extension directories
|
|
1236
|
-
- If not found, throw TraitNotFoundError
|
|
1237
|
-
b. Parse trait file content (strip frontmatter if present)
|
|
1238
|
-
c. Map weight string to numeric value:
|
|
1239
|
-
- "HIGH" -> 0.7
|
|
1240
|
-
- "MEDIUM" -> 0.5
|
|
1241
|
-
- "LOW" -> 0.3
|
|
1242
|
-
- Numeric (0.0-1.0) -> use as-is
|
|
1243
|
-
- Boolean true -> 1.0
|
|
1244
|
-
- Boolean false -> 0.0 (excluded)
|
|
1245
|
-
d. Store: { id, content, weight, filePath }
|
|
1246
|
-
3. Sort traits by declaration order (preserve order from persona.config.json)
|
|
1247
|
-
4. Return resolved trait list
|
|
1248
|
-
```
|
|
1249
|
-
|
|
1250
|
-
**Weight Mapping:**
|
|
1251
|
-
|
|
1252
|
-
| Input | Numeric Value | Semantic |
|
|
1253
|
-
|-------|--------------|----------|
|
|
1254
|
-
| `"HIGH"` | 0.7 | Strong emphasis in persona prompt |
|
|
1255
|
-
| `"MEDIUM"` | 0.5 | Standard inclusion |
|
|
1256
|
-
| `"LOW"` | 0.3 | Light inclusion, may be abbreviated |
|
|
1257
|
-
| `true` | 1.0 | Full inclusion |
|
|
1258
|
-
| `false` | 0.0 | Excluded |
|
|
1259
|
-
| `0.0-1.0` | as-is | Direct numeric weight |
|
|
1260
|
-
|
|
1261
|
-
**Weight Application:** Weights currently affect the `emphasis` directive prepended to each trait in the compiled output. A trait at weight 0.7 is prefixed with `<!-- weight: 0.7 — apply this trait with HIGH emphasis -->`. The LLM interprets this as guidance for how strongly to follow the trait's instructions. Future: weights may control token budget allocation per trait.
|
|
1262
|
-
|
|
1263
|
-
**@import Generation (Claude Code native output):**
|
|
1264
|
-
|
|
1265
|
-
For each persona, instead of inlining trait content into the agent CLAUDE.md:
|
|
1266
|
-
|
|
1267
|
-
```
|
|
1268
|
-
// Generated .claude/CLAUDE.md
|
|
1269
|
-
@.claude/traits/critical-thinking.md
|
|
1270
|
-
@.claude/traits/structured-output.md
|
|
1271
|
-
@.claude/traits/source-citation.md
|
|
1272
|
-
|
|
1273
|
-
[always-on instructions from core/instructions/]
|
|
1274
|
-
```
|
|
1275
|
-
|
|
1276
|
-
Traits are written as separate files under `.claude/traits/` so they are loaded once regardless of how many personas reference them.
|
|
1277
|
-
|
|
1278
|
-
**Claude Code Native Output Generation:**
|
|
1279
|
-
|
|
1280
|
-
For each persona, compile.ts generates:
|
|
1281
|
-
|
|
1282
|
-
1. **`.claude/agents/{persona}/CLAUDE.md`** -- Agent definition with YAML frontmatter:
|
|
1283
|
-
```yaml
|
|
1284
|
-
---
|
|
1285
|
-
name: {persona-id}
|
|
1286
|
-
description: {from SKILL.md frontmatter}
|
|
1287
|
-
model: {from persona.config.json, default: inherit}
|
|
1288
|
-
permissionMode: {from persona.config.json, default: default}
|
|
1289
|
-
maxTurns: {from persona.config.json, default: 50}
|
|
1290
|
-
disallowedTools: {from persona.config.json, default: []}
|
|
1291
|
-
tools: {from persona.config.json, default: all}
|
|
1292
|
-
skills: {list of preloaded skills}
|
|
1293
|
-
mcpServers: {from persona.config.json}
|
|
1294
|
-
hooks: {from persona.config.json or domain layer}
|
|
1295
|
-
memory: {from persona.config.json, default: none}
|
|
1296
|
-
---
|
|
1297
|
-
|
|
1298
|
-
[Persona system prompt from SKILL.md body]
|
|
1299
|
-
```
|
|
1300
|
-
|
|
1301
|
-
2. **`.claude/skills/{skill}/SKILL.md`** -- Skill definition:
|
|
1302
|
-
```yaml
|
|
1303
|
-
---
|
|
1304
|
-
name: {skill-name}
|
|
1305
|
-
description: {from SKILL.md}
|
|
1306
|
-
context: fork
|
|
1307
|
-
agent: {persona-name}
|
|
1308
|
-
argument-hint: {if defined}
|
|
1309
|
-
---
|
|
1310
|
-
|
|
1311
|
-
[Skill instructions]
|
|
1312
|
-
```
|
|
1313
|
-
|
|
1314
|
-
3. **`.claude/rules/{topic}.md`** -- Rules with path scoping:
|
|
1315
|
-
```yaml
|
|
1316
|
-
---
|
|
1317
|
-
paths:
|
|
1318
|
-
- "pattern/**"
|
|
1319
|
-
---
|
|
1320
|
-
|
|
1321
|
-
[Rule content]
|
|
1322
|
-
```
|
|
1323
|
-
|
|
1324
|
-
4. **`.claude/traits/{name}.md`** -- Trait files as standalone markdown.
|
|
1325
|
-
|
|
1326
|
-
5. **`.claude/CLAUDE.md`** -- Using `@import` references.
|
|
1327
|
-
|
|
1328
|
-
6. **`.claude/settings.json`** -- Hook entries:
|
|
1329
|
-
```json
|
|
1330
|
-
{
|
|
1331
|
-
"hooks": {
|
|
1332
|
-
"PreToolUse": [...],
|
|
1333
|
-
"PostToolUse": [...],
|
|
1334
|
-
"Stop": [...]
|
|
1335
|
-
}
|
|
1336
|
-
}
|
|
1337
|
-
```
|
|
1338
|
-
|
|
1339
|
-
7. **`.claude/.mcp.json`** -- MCP server configurations from domain layers.
|
|
1340
|
-
|
|
1341
|
-
**Cross-Platform Output Generation:**
|
|
1342
|
-
|
|
1343
|
-
For each persona, compile.ts generates:
|
|
1344
|
-
|
|
1345
|
-
1. **`SKILL.md`** -- Standalone file with traits inlined (no @imports):
|
|
1346
|
-
```yaml
|
|
1347
|
-
---
|
|
1348
|
-
name: {persona-id}
|
|
1349
|
-
description: {from frontmatter}
|
|
1350
|
-
version: {from config}
|
|
1351
|
-
---
|
|
1352
|
-
|
|
1353
|
-
[Persona prompt with traits inlined between injection markers]
|
|
1354
|
-
```
|
|
1355
|
-
|
|
1356
|
-
2. **`copilot-instructions.md`** -- Copilot-compatible format with all instructions flattened.
|
|
1357
|
-
|
|
1358
|
-
**Inline Trait Injection:** For cross-platform output, traits are injected between markers in SKILL.md:
|
|
1359
|
-
```markdown
|
|
1360
|
-
<!-- traits:start -->
|
|
1361
|
-
[trait content, one after another, separated by blank lines]
|
|
1362
|
-
<!-- traits:end -->
|
|
1363
|
-
```
|
|
1364
|
-
|
|
1365
|
-
**PERSONAS.md Generation:**
|
|
1366
|
-
|
|
1367
|
-
compile.ts generates a human-readable registry:
|
|
1368
|
-
|
|
1369
|
-
```markdown
|
|
1370
|
-
# Personas Registry
|
|
1371
|
-
|
|
1372
|
-
Generated by AgentBoot v{version} on {date}.
|
|
1373
|
-
|
|
1374
|
-
## {persona-name}
|
|
1375
|
-
- **Description:** {description}
|
|
1376
|
-
- **Model:** {model}
|
|
1377
|
-
- **Traits:** {trait-list with weights}
|
|
1378
|
-
- **Token estimate:** {estimated_tokens}
|
|
1379
|
-
|
|
1380
|
-
[... for each enabled persona]
|
|
1381
|
-
```
|
|
1382
|
-
|
|
1383
|
-
### 4.3 sync.ts
|
|
1384
|
-
|
|
1385
|
-
The sync script reads compiled output and distributes it to target repos.
|
|
1386
|
-
|
|
1387
|
-
**Repo Targeting Algorithm:**
|
|
1388
|
-
|
|
1389
|
-
```
|
|
1390
|
-
function resolveReposForSync(config, cliRepoFilter):
|
|
1391
|
-
1. Load repo list from config.sync.repos (file path or inline array)
|
|
1392
|
-
2. If cliRepoFilter specified, filter to matching repos
|
|
1393
|
-
3. For each repo:
|
|
1394
|
-
a. Resolve scope: match repo.team to config.groups[*].teams
|
|
1395
|
-
b. Determine output format from repo.platform (default: "claude-code")
|
|
1396
|
-
c. Determine output source directory:
|
|
1397
|
-
- Start with dist/core/
|
|
1398
|
-
- If repo.group exists: overlay dist/groups/{group}/
|
|
1399
|
-
- If repo.team exists: overlay dist/teams/{group}/{team}/
|
|
1400
|
-
4. Return list of { repo, scopeChain, outputFormat, sourceDirs }
|
|
1401
|
-
```
|
|
1402
|
-
|
|
1403
|
-
**Scope Merging Algorithm:**
|
|
1404
|
-
|
|
1405
|
-
```
|
|
1406
|
-
function mergeScopes(core, group, team):
|
|
1407
|
-
result = deepClone(core)
|
|
1408
|
-
|
|
1409
|
-
// Group layer: additive for enabled lists, override for scalar values
|
|
1410
|
-
if group:
|
|
1411
|
-
result.personas.enabled = union(core.personas.enabled, group.personas.enabled)
|
|
1412
|
-
result.traits.enabled = union(core.traits.enabled, group.traits.enabled)
|
|
1413
|
-
// Group extensions layered on top
|
|
1414
|
-
|
|
1415
|
-
// Team layer: team wins on conflicts for optional behaviors
|
|
1416
|
-
if team:
|
|
1417
|
-
result.personas.enabled = union(result.personas.enabled, team.personas.enabled)
|
|
1418
|
-
result.traits.enabled = union(result.traits.enabled, team.traits.enabled)
|
|
1419
|
-
// Team overrides group for scalar config values
|
|
1420
|
-
// Mandatory behaviors are enforced by rules, not persona config
|
|
1421
|
-
|
|
1422
|
-
return result
|
|
1423
|
-
```
|
|
1424
|
-
|
|
1425
|
-
**Mandatory vs. Optional:** Rules are mandatory and follow top-down inheritance (org wins). Personas are optional and follow bottom-up overriding (team wins). Personas do not have a `required` field -- use rules to enforce mandatory persona usage at any scope.
|
|
1426
|
-
|
|
1427
|
-
**Platform Detection:**
|
|
1428
|
-
|
|
1429
|
-
| `repo.platform` Value | Output Written |
|
|
1430
|
-
|-----------------------|---------------|
|
|
1431
|
-
| `"claude-code"` (default) | `.claude/` directory with agents, skills, rules, traits, CLAUDE.md, settings.json, .mcp.json |
|
|
1432
|
-
| `"copilot"` | `.github/copilot-instructions.md` |
|
|
1433
|
-
| `"cross-platform"` | SKILL.md files + copilot-instructions.md |
|
|
1434
|
-
|
|
1435
|
-
**File Writing (local mode):**
|
|
1436
|
-
|
|
1437
|
-
```
|
|
1438
|
-
function syncToRepo(repo, outputFiles):
|
|
1439
|
-
1. Verify repo.path exists and is a git repository
|
|
1440
|
-
2. Read existing .agentboot-manifest.json (if present) for diff comparison
|
|
1441
|
-
3. For each output file:
|
|
1442
|
-
a. Compute file hash (SHA-256)
|
|
1443
|
-
b. Write file to {repo.path}/{output.dir}/{relative-path}
|
|
1444
|
-
4. Write new .agentboot-manifest.json with all managed file paths and hashes
|
|
1445
|
-
5. If sync.pr.enabled:
|
|
1446
|
-
a. Create branch: {sync.pr.branch_prefix}{ISO-date}
|
|
1447
|
-
b. Stage all written files
|
|
1448
|
-
c. Commit: rendered sync.pr.title_template
|
|
1449
|
-
d. Push branch
|
|
1450
|
-
e. Create PR via gh CLI or GitHub API
|
|
1451
|
-
```
|
|
1452
|
-
|
|
1453
|
-
**Manifest Tracking:**
|
|
1454
|
-
|
|
1455
|
-
The manifest (`.claude/.agentboot-manifest.json`) tracks every file AgentBoot manages:
|
|
1456
|
-
|
|
1457
|
-
```json
|
|
1458
|
-
{
|
|
1459
|
-
"managed_by": "agentboot",
|
|
1460
|
-
"version": "1.2.0",
|
|
1461
|
-
"synced_at": "2026-03-19T14:30:00Z",
|
|
1462
|
-
"source_commit": "abc123",
|
|
1463
|
-
"files": [
|
|
1464
|
-
{
|
|
1465
|
-
"path": "agents/code-reviewer/CLAUDE.md",
|
|
1466
|
-
"hash": "sha256:a3f2..."
|
|
1467
|
-
}
|
|
1468
|
-
]
|
|
1469
|
-
}
|
|
1470
|
-
```
|
|
1471
|
-
|
|
1472
|
-
**PR Creation (github-api mode):**
|
|
1473
|
-
|
|
1474
|
-
```
|
|
1475
|
-
function syncViaGitHubApi(repo, outputFiles):
|
|
1476
|
-
1. Require GITHUB_TOKEN env var
|
|
1477
|
-
2. Use GitHub API (via gh or octokit):
|
|
1478
|
-
a. Create branch from default branch HEAD
|
|
1479
|
-
b. For each file, create/update blob via API
|
|
1480
|
-
c. Create tree with all blobs
|
|
1481
|
-
d. Create commit on the new tree
|
|
1482
|
-
e. Update branch ref to new commit
|
|
1483
|
-
f. Create PR with rendered title template
|
|
1484
|
-
3. Return PR URL
|
|
1485
|
-
```
|
|
1486
|
-
|
|
1487
|
-
---
|
|
1488
|
-
|
|
1489
|
-
## 5. Persona Configuration Specification
|
|
1490
|
-
|
|
1491
|
-
### 5.1 `persona.config.json` Schema
|
|
1492
|
-
|
|
1493
|
-
This file does not yet exist in the codebase. The following is the canonical schema definition.
|
|
1494
|
-
|
|
1495
|
-
**Location:** `core/personas/{persona-name}/persona.config.json`
|
|
1496
|
-
|
|
1497
|
-
**JSON Schema:**
|
|
1498
|
-
|
|
1499
|
-
```json
|
|
1500
|
-
{
|
|
1501
|
-
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
1502
|
-
"$id": "https://agentboot.dev/schema/persona-config/v1",
|
|
1503
|
-
"title": "AgentBoot Persona Configuration",
|
|
1504
|
-
"type": "object",
|
|
1505
|
-
"required": ["traits"],
|
|
1506
|
-
"additionalProperties": false,
|
|
1507
|
-
"properties": {
|
|
1508
|
-
"traits": {
|
|
1509
|
-
"type": "object",
|
|
1510
|
-
"description": "Trait composition map. Keys are trait IDs; values are weight configurations.",
|
|
1511
|
-
"additionalProperties": {
|
|
1512
|
-
"oneOf": [
|
|
1513
|
-
{ "type": "boolean" },
|
|
1514
|
-
{ "type": "number", "minimum": 0.0, "maximum": 1.0 },
|
|
1515
|
-
{ "type": "string", "enum": ["HIGH", "MEDIUM", "LOW"] }
|
|
1516
|
-
]
|
|
1517
|
-
}
|
|
1518
|
-
},
|
|
1519
|
-
"model": {
|
|
1520
|
-
"type": "string",
|
|
1521
|
-
"description": "Model to use for this persona.",
|
|
1522
|
-
"enum": ["haiku", "sonnet", "opus", "inherit"],
|
|
1523
|
-
"default": "inherit"
|
|
1524
|
-
},
|
|
1525
|
-
"permissionMode": {
|
|
1526
|
-
"type": "string",
|
|
1527
|
-
"description": "Permission mode for this persona's agent.",
|
|
1528
|
-
"enum": ["default", "acceptEdits", "dontAsk", "plan", "bypassPermissions"],
|
|
1529
|
-
"default": "default"
|
|
1530
|
-
},
|
|
1531
|
-
"maxTurns": {
|
|
1532
|
-
"type": "integer",
|
|
1533
|
-
"description": "Maximum agentic turns for this persona.",
|
|
1534
|
-
"minimum": 1,
|
|
1535
|
-
"maximum": 1000,
|
|
1536
|
-
"default": 50
|
|
1537
|
-
},
|
|
1538
|
-
"disallowedTools": {
|
|
1539
|
-
"type": "array",
|
|
1540
|
-
"description": "Tools this persona cannot use.",
|
|
1541
|
-
"items": { "type": "string" },
|
|
1542
|
-
"default": []
|
|
1543
|
-
},
|
|
1544
|
-
"tools": {
|
|
1545
|
-
"type": "array",
|
|
1546
|
-
"description": "Explicit tool allowlist. If set, only these tools are available.",
|
|
1547
|
-
"items": { "type": "string" }
|
|
1548
|
-
},
|
|
1549
|
-
"effort": {
|
|
1550
|
-
"type": "string",
|
|
1551
|
-
"description": "Extended thinking effort level.",
|
|
1552
|
-
"enum": ["low", "medium", "high", "max"],
|
|
1553
|
-
"default": "medium"
|
|
1554
|
-
},
|
|
1555
|
-
"autonomy": {
|
|
1556
|
-
"type": "string",
|
|
1557
|
-
"description": "Autonomy progression level (V2+).",
|
|
1558
|
-
"enum": ["advisory", "auto-approve", "autonomous"],
|
|
1559
|
-
"default": "advisory"
|
|
1560
|
-
},
|
|
1561
|
-
"skills": {
|
|
1562
|
-
"type": "array",
|
|
1563
|
-
"description": "Skills to preload when this persona is invoked.",
|
|
1564
|
-
"items": { "type": "string" },
|
|
1565
|
-
"default": []
|
|
1566
|
-
},
|
|
1567
|
-
"mcpServers": {
|
|
1568
|
-
"type": "object",
|
|
1569
|
-
"description": "MCP servers scoped to this persona.",
|
|
1570
|
-
"additionalProperties": {
|
|
1571
|
-
"type": "object",
|
|
1572
|
-
"properties": {
|
|
1573
|
-
"type": { "type": "string", "enum": ["stdio", "http", "ws", "sse"] },
|
|
1574
|
-
"command": { "type": "string" },
|
|
1575
|
-
"args": { "type": "array", "items": { "type": "string" } },
|
|
1576
|
-
"url": { "type": "string" },
|
|
1577
|
-
"env": { "type": "object", "additionalProperties": { "type": "string" } }
|
|
1578
|
-
}
|
|
1579
|
-
}
|
|
1580
|
-
},
|
|
1581
|
-
"hooks": {
|
|
1582
|
-
"type": "object",
|
|
1583
|
-
"description": "Hooks scoped to this persona's agent.",
|
|
1584
|
-
"additionalProperties": {
|
|
1585
|
-
"type": "array",
|
|
1586
|
-
"items": {
|
|
1587
|
-
"type": "object",
|
|
1588
|
-
"properties": {
|
|
1589
|
-
"matcher": { "type": "string" },
|
|
1590
|
-
"hooks": {
|
|
1591
|
-
"type": "array",
|
|
1592
|
-
"items": {
|
|
1593
|
-
"type": "object",
|
|
1594
|
-
"required": ["type", "command"],
|
|
1595
|
-
"properties": {
|
|
1596
|
-
"type": { "type": "string", "enum": ["command", "http", "prompt", "agent"] },
|
|
1597
|
-
"command": { "type": "string" },
|
|
1598
|
-
"url": { "type": "string" },
|
|
1599
|
-
"timeout": { "type": "integer" },
|
|
1600
|
-
"async": { "type": "boolean", "default": false }
|
|
1601
|
-
}
|
|
1602
|
-
}
|
|
1603
|
-
}
|
|
1604
|
-
}
|
|
1605
|
-
}
|
|
1606
|
-
}
|
|
1607
|
-
},
|
|
1608
|
-
"memory": {
|
|
1609
|
-
"type": "string",
|
|
1610
|
-
"description": "Persistent memory scope for this persona.",
|
|
1611
|
-
"enum": ["user", "project", "local"],
|
|
1612
|
-
"default": null
|
|
1613
|
-
},
|
|
1614
|
-
"background": {
|
|
1615
|
-
"type": "boolean",
|
|
1616
|
-
"description": "Whether this persona runs as a background agent.",
|
|
1617
|
-
"default": false
|
|
1618
|
-
},
|
|
1619
|
-
"isolation": {
|
|
1620
|
-
"type": "string",
|
|
1621
|
-
"description": "Isolation mode for this persona.",
|
|
1622
|
-
"enum": ["none", "worktree"],
|
|
1623
|
-
"default": "none"
|
|
1624
|
-
},
|
|
1625
|
-
"tokenBudget": {
|
|
1626
|
-
"type": "integer",
|
|
1627
|
-
"description": "Maximum token budget for this persona's full context.",
|
|
1628
|
-
"default": 6000
|
|
1629
|
-
}
|
|
1630
|
-
}
|
|
1631
|
-
}
|
|
1632
|
-
```
|
|
1633
|
-
|
|
1634
|
-
> **Note:** Personas are not marked `required`. Mandatory behavior is enforced through rules, not persona configuration. If a persona must always be active at a given scope, define a rule for that scope instead.
|
|
1635
|
-
|
|
1636
|
-
### 5.2 Example persona.config.json
|
|
1637
|
-
|
|
1638
|
-
```json
|
|
1639
|
-
{
|
|
1640
|
-
"traits": {
|
|
1641
|
-
"critical-thinking": "HIGH",
|
|
1642
|
-
"structured-output": true,
|
|
1643
|
-
"source-citation": "MEDIUM",
|
|
1644
|
-
"confidence-signaling": "LOW"
|
|
1645
|
-
},
|
|
1646
|
-
"model": "sonnet",
|
|
1647
|
-
"permissionMode": "plan",
|
|
1648
|
-
"maxTurns": 30,
|
|
1649
|
-
"disallowedTools": ["Write", "Edit", "Agent"],
|
|
1650
|
-
"effort": "high",
|
|
1651
|
-
"skills": ["review-code"],
|
|
1652
|
-
"memory": "project",
|
|
1653
|
-
"tokenBudget": 6000
|
|
1654
|
-
}
|
|
1655
|
-
```
|
|
1656
|
-
|
|
1657
|
-
### 5.3 Mapping to Claude Code Agent CLAUDE.md Frontmatter
|
|
1658
|
-
|
|
1659
|
-
| persona.config.json field | Agent CLAUDE.md frontmatter field | Transformation |
|
|
1660
|
-
|--------------------------|----------------------------------|----------------|
|
|
1661
|
-
| `model` | `model` | Direct pass-through |
|
|
1662
|
-
| `permissionMode` | `permissionMode` | Direct pass-through |
|
|
1663
|
-
| `maxTurns` | `maxTurns` | Direct pass-through |
|
|
1664
|
-
| `disallowedTools` | `disallowedTools` | Direct pass-through (comma-separated in YAML) |
|
|
1665
|
-
| `tools` | `tools` | Direct pass-through |
|
|
1666
|
-
| `skills` | `skills` | List of skill names |
|
|
1667
|
-
| `mcpServers` | `mcpServers` | Nested YAML map |
|
|
1668
|
-
| `hooks` | `hooks` | Nested YAML structure |
|
|
1669
|
-
| `memory` | `memory` | Direct pass-through |
|
|
1670
|
-
| `background` | `background` | Direct pass-through |
|
|
1671
|
-
| `isolation` | `isolation` | Direct pass-through |
|
|
1672
|
-
| `effort` | not in frontmatter | Emitted as instruction in body: `Use effort level: {effort}` |
|
|
1673
|
-
| `autonomy` | not in frontmatter | V2+ feature, emitted as instruction |
|
|
1674
|
-
| `traits` | not in frontmatter | Resolved and composed into body or @import references |
|
|
1675
|
-
| `tokenBudget` | `estimated_tokens` (informational) | Calculated and emitted as comment |
|
|
1676
|
-
|
|
1677
|
-
### 5.4 Mapping to Cross-Platform SKILL.md Frontmatter
|
|
1678
|
-
|
|
1679
|
-
| persona.config.json field | SKILL.md frontmatter field | Transformation |
|
|
1680
|
-
|--------------------------|---------------------------|----------------|
|
|
1681
|
-
| `model` | not emitted | Cross-platform SKILL.md does not set model |
|
|
1682
|
-
| `traits` | inlined in body | Traits are injected between `<!-- traits:start -->` and `<!-- traits:end -->` |
|
|
1683
|
-
| `permissionMode` | not emitted | Platform-specific |
|
|
1684
|
-
| `maxTurns` | not emitted | Platform-specific |
|
|
1685
|
-
| All other fields | not emitted | Only `name`, `description`, `version` in cross-platform frontmatter |
|
|
1686
|
-
|
|
1687
|
-
---
|
|
1688
|
-
|
|
1689
|
-
## 6. Lint Specification
|
|
1690
|
-
|
|
1691
|
-
### 6.1 Lint Rule Catalog
|
|
1692
|
-
|
|
1693
|
-
#### Token Budget Rules
|
|
1694
|
-
|
|
1695
|
-
**Rule: `prompt-too-long`**
|
|
1696
|
-
|
|
1697
|
-
| Property | Value |
|
|
1698
|
-
|----------|-------|
|
|
1699
|
-
| ID | `prompt-too-long` |
|
|
1700
|
-
| Description | Persona prompt exceeds recommended line count |
|
|
1701
|
-
| Severity | WARN at 500 lines, ERROR at 1000 lines |
|
|
1702
|
-
| What it checks | Line count of SKILL.md body (excluding frontmatter) |
|
|
1703
|
-
| Auto-fixable | No |
|
|
1704
|
-
| Configurable thresholds | `lint.rules.prompt-too-long: { warn: N, error: N }` |
|
|
1705
|
-
|
|
1706
|
-
**Rule: `claude-md-too-long`**
|
|
1707
|
-
|
|
1708
|
-
| Property | Value |
|
|
1709
|
-
|----------|-------|
|
|
1710
|
-
| ID | `claude-md-too-long` |
|
|
1711
|
-
| Description | Generated CLAUDE.md exceeds effective limit for adherence |
|
|
1712
|
-
| Severity | WARN at 200 lines, ERROR at 500 lines |
|
|
1713
|
-
| What it checks | Line count of generated CLAUDE.md (after @import expansion) |
|
|
1714
|
-
| Auto-fixable | No |
|
|
1715
|
-
| Configurable thresholds | `lint.rules.claude-md-too-long: { warn: N, error: N }` |
|
|
1716
|
-
|
|
1717
|
-
**Rule: `trait-too-long`**
|
|
1718
|
-
|
|
1719
|
-
| Property | Value |
|
|
1720
|
-
|----------|-------|
|
|
1721
|
-
| ID | `trait-too-long` |
|
|
1722
|
-
| Description | Individual trait file is too large and should be split |
|
|
1723
|
-
| Severity | WARN at 100 lines |
|
|
1724
|
-
| What it checks | Line count of each trait file |
|
|
1725
|
-
| Auto-fixable | No |
|
|
1726
|
-
| Configurable threshold | `lint.rules.trait-too-long: { warn: N }` |
|
|
1727
|
-
|
|
1728
|
-
**Rule: `total-context-estimate`**
|
|
1729
|
-
|
|
1730
|
-
| Property | Value |
|
|
1731
|
-
|----------|-------|
|
|
1732
|
-
| ID | `total-context-estimate` |
|
|
1733
|
-
| Description | Combined persona + traits + instructions exceed context budget |
|
|
1734
|
-
| Severity | WARN when estimated tokens exceed configured percentage of context window |
|
|
1735
|
-
| What it checks | Token estimate for full persona context (SKILL.md + all traits + always-on instructions + path-scoped rules) |
|
|
1736
|
-
| Auto-fixable | No |
|
|
1737
|
-
| Configurable | `lint.rules.total-context-estimate: { warn: 30 }` (percentage of 200k context window) |
|
|
1738
|
-
|
|
1739
|
-
#### Quality Rules
|
|
1740
|
-
|
|
1741
|
-
**Rule: `vague-instruction`**
|
|
1742
|
-
|
|
1743
|
-
| Property | Value |
|
|
1744
|
-
|----------|-------|
|
|
1745
|
-
| ID | `vague-instruction` |
|
|
1746
|
-
| Description | Instruction uses weak, non-actionable language |
|
|
1747
|
-
| Severity | WARN |
|
|
1748
|
-
| What it checks | Regex patterns: `/\b(be thorough|try to|if possible|as needed|best practices|when appropriate|good code|clean code|high quality)\b/i` |
|
|
1749
|
-
| Auto-fixable | No (requires human judgment to make specific) |
|
|
1750
|
-
| Configurable severity | `lint.rules.vague-instruction: "error"` |
|
|
1751
|
-
|
|
1752
|
-
**Rule: `conflicting-instructions`**
|
|
1753
|
-
|
|
1754
|
-
| Property | Value |
|
|
1755
|
-
|----------|-------|
|
|
1756
|
-
| ID | `conflicting-instructions` |
|
|
1757
|
-
| Description | Two traits or instructions contradict each other |
|
|
1758
|
-
| Severity | ERROR |
|
|
1759
|
-
| What it checks | Known contradiction patterns: "always X" in one file + "never X" in another; "use framework A" + "do not use framework A" |
|
|
1760
|
-
| Algorithm | Build instruction index, detect opposing directives on same topic |
|
|
1761
|
-
| Auto-fixable | No |
|
|
1762
|
-
|
|
1763
|
-
**Rule: `missing-output-format`**
|
|
1764
|
-
|
|
1765
|
-
| Property | Value |
|
|
1766
|
-
|----------|-------|
|
|
1767
|
-
| ID | `missing-output-format` |
|
|
1768
|
-
| Description | Reviewer persona does not define structured output format |
|
|
1769
|
-
| Severity | WARN |
|
|
1770
|
-
| What it checks | Persona SKILL.md body contains "Output Format" or "Output Schema" section header |
|
|
1771
|
-
| Auto-fixable | No |
|
|
1772
|
-
|
|
1773
|
-
**Rule: `missing-severity-levels`**
|
|
1774
|
-
|
|
1775
|
-
| Property | Value |
|
|
1776
|
-
|----------|-------|
|
|
1777
|
-
| ID | `missing-severity-levels` |
|
|
1778
|
-
| Description | Reviewer persona does not define severity classifications |
|
|
1779
|
-
| Severity | WARN |
|
|
1780
|
-
| What it checks | Persona SKILL.md body contains at least two of: CRITICAL, ERROR, WARN, INFO |
|
|
1781
|
-
| Auto-fixable | No |
|
|
1782
|
-
|
|
1783
|
-
**Rule: `hardcoded-paths`**
|
|
1784
|
-
|
|
1785
|
-
| Property | Value |
|
|
1786
|
-
|----------|-------|
|
|
1787
|
-
| ID | `hardcoded-paths` |
|
|
1788
|
-
| Description | Absolute filesystem paths that will not work across machines |
|
|
1789
|
-
| Severity | ERROR |
|
|
1790
|
-
| What it checks | Regex: `/(?:^|\s)(\/(?:Users|home|var|tmp|opt)\/\S+)/` |
|
|
1791
|
-
| Auto-fixable | No |
|
|
1792
|
-
|
|
1793
|
-
**Rule: `hardcoded-model`**
|
|
1794
|
-
|
|
1795
|
-
| Property | Value |
|
|
1796
|
-
|----------|-------|
|
|
1797
|
-
| ID | `hardcoded-model` |
|
|
1798
|
-
| Description | Model name appears in prose text instead of frontmatter |
|
|
1799
|
-
| Severity | WARN |
|
|
1800
|
-
| What it checks | Regex in SKILL.md body: `/\b(claude-opus|claude-sonnet|claude-haiku|gpt-4|gpt-3\.5)\b/i` |
|
|
1801
|
-
| Auto-fixable | No |
|
|
1802
|
-
|
|
1803
|
-
**Rule: `unused-trait`**
|
|
1804
|
-
|
|
1805
|
-
| Property | Value |
|
|
1806
|
-
|----------|-------|
|
|
1807
|
-
| ID | `unused-trait` |
|
|
1808
|
-
| Description | Trait file exists but is not referenced by any persona |
|
|
1809
|
-
| Severity | WARN |
|
|
1810
|
-
| What it checks | Cross-reference `core/traits/` files against all `persona.config.json` trait entries |
|
|
1811
|
-
| Auto-fixable | No |
|
|
1812
|
-
|
|
1813
|
-
**Rule: `missing-anti-patterns`**
|
|
1814
|
-
|
|
1815
|
-
| Property | Value |
|
|
1816
|
-
|----------|-------|
|
|
1817
|
-
| ID | `missing-anti-patterns` |
|
|
1818
|
-
| Description | Trait does not include a "What Not To Do" section |
|
|
1819
|
-
| Severity | INFO |
|
|
1820
|
-
| What it checks | Trait markdown contains heading with "Not" or "Anti" or "Don't" |
|
|
1821
|
-
| Auto-fixable | No |
|
|
1822
|
-
|
|
1823
|
-
**Rule: `missing-activation-condition`**
|
|
1824
|
-
|
|
1825
|
-
| Property | Value |
|
|
1826
|
-
|----------|-------|
|
|
1827
|
-
| ID | `missing-activation-condition` |
|
|
1828
|
-
| Description | Trait does not specify when it should be active |
|
|
1829
|
-
| Severity | WARN |
|
|
1830
|
-
| What it checks | Trait markdown contains heading with "When" or "Activation" or "Applies" |
|
|
1831
|
-
| Auto-fixable | No |
|
|
1832
|
-
|
|
1833
|
-
**Rule: `duplicate-instruction`**
|
|
1834
|
-
|
|
1835
|
-
| Property | Value |
|
|
1836
|
-
|----------|-------|
|
|
1837
|
-
| ID | `duplicate-instruction` |
|
|
1838
|
-
| Description | Same instruction appears in multiple files |
|
|
1839
|
-
| Severity | WARN |
|
|
1840
|
-
| What it checks | Normalized instruction text comparison across all persona and trait files (case-insensitive, whitespace-normalized) |
|
|
1841
|
-
| Auto-fixable | No |
|
|
1842
|
-
|
|
1843
|
-
**Rule: `no-examples`**
|
|
1844
|
-
|
|
1845
|
-
| Property | Value |
|
|
1846
|
-
|----------|-------|
|
|
1847
|
-
| ID | `no-examples` |
|
|
1848
|
-
| Description | Persona has no example input/output to guide behavior |
|
|
1849
|
-
| Severity | INFO |
|
|
1850
|
-
| What it checks | SKILL.md body contains "Example" heading or code block with input/output labels |
|
|
1851
|
-
| Auto-fixable | No |
|
|
1852
|
-
|
|
1853
|
-
#### Security Rules
|
|
1854
|
-
|
|
1855
|
-
**Rule: `credential-in-prompt`**
|
|
1856
|
-
|
|
1857
|
-
| Property | Value |
|
|
1858
|
-
|----------|-------|
|
|
1859
|
-
| ID | `credential-in-prompt` |
|
|
1860
|
-
| Description | API key, token, or password detected in prompt text |
|
|
1861
|
-
| Severity | ERROR |
|
|
1862
|
-
| What it checks | Regex patterns: `/sk-[a-zA-Z0-9]{20,}/`, `/ghp_[a-zA-Z0-9]{36}/`, `/AKIA[A-Z0-9]{16}/`, `/xox[bpsa]-[a-zA-Z0-9-]+/`, `/eyJ[a-zA-Z0-9_-]+\.[a-zA-Z0-9_-]+/`, `/password\s*[:=]\s*["'][^"']+["']/i` |
|
|
1863
|
-
| Auto-fixable | No |
|
|
1864
|
-
|
|
1865
|
-
**Rule: `internal-url`**
|
|
1866
|
-
|
|
1867
|
-
| Property | Value |
|
|
1868
|
-
|----------|-------|
|
|
1869
|
-
| ID | `internal-url` |
|
|
1870
|
-
| Description | Internal URL that should not appear in distributed prompts |
|
|
1871
|
-
| Severity | ERROR |
|
|
1872
|
-
| What it checks | Regex: `/https?:\/\/[^\s]*\.(internal|local|corp|intranet)\b/`, private IP ranges |
|
|
1873
|
-
| Auto-fixable | No |
|
|
1874
|
-
|
|
1875
|
-
**Rule: `pii-in-example`**
|
|
1876
|
-
|
|
1877
|
-
| Property | Value |
|
|
1878
|
-
|----------|-------|
|
|
1879
|
-
| ID | `pii-in-example` |
|
|
1880
|
-
| Description | Real names, emails, or identifiers in persona examples |
|
|
1881
|
-
| Severity | ERROR |
|
|
1882
|
-
| What it checks | Email regex in non-config files, common name patterns adjacent to "example" or "sample" |
|
|
1883
|
-
| Auto-fixable | No |
|
|
1884
|
-
|
|
1885
|
-
### 6.2 Custom Rule Format
|
|
1886
|
-
|
|
1887
|
-
Organizations define custom rules in `agentboot.config.json`:
|
|
1888
|
-
|
|
1889
|
-
```json
|
|
1890
|
-
{
|
|
1891
|
-
"lint": {
|
|
1892
|
-
"rules": {
|
|
1893
|
-
"prompt-too-long": { "warn": 300, "error": 600 },
|
|
1894
|
-
"vague-instruction": "error"
|
|
1895
|
-
},
|
|
1896
|
-
"custom": [
|
|
1897
|
-
{
|
|
1898
|
-
"id": "no-passive-voice",
|
|
1899
|
-
"pattern": "should be|could be|might be",
|
|
1900
|
-
"message": "Use imperative voice: 'Verify X' not 'X should be verified'",
|
|
1901
|
-
"severity": "warn",
|
|
1902
|
-
"scope": "persona"
|
|
1903
|
-
}
|
|
1904
|
-
]
|
|
1905
|
-
}
|
|
1906
|
-
}
|
|
1907
|
-
```
|
|
1908
|
-
|
|
1909
|
-
**Custom rule fields:**
|
|
1910
|
-
|
|
1911
|
-
| Field | Type | Required | Description |
|
|
1912
|
-
|-------|------|----------|-------------|
|
|
1913
|
-
| `id` | string | Yes | Unique rule identifier (kebab-case) |
|
|
1914
|
-
| `pattern` | string | Yes | Regex pattern to match (case-insensitive) |
|
|
1915
|
-
| `message` | string | Yes | Human-readable error message |
|
|
1916
|
-
| `severity` | string | Yes | `info`, `warn`, `error` |
|
|
1917
|
-
| `scope` | string | No | Where to apply: `persona`, `trait`, `instruction`, `all` (default: `all`) |
|
|
1918
|
-
|
|
1919
|
-
### 6.3 Token Counting Algorithm
|
|
1920
|
-
|
|
1921
|
-
Token estimation is performed locally without an API call. AgentBoot uses a character-based approximation calibrated against the `cl100k_base` tokenizer (used by Claude models).
|
|
1922
|
-
|
|
1923
|
-
**Algorithm:**
|
|
1924
|
-
|
|
1925
|
-
```
|
|
1926
|
-
function estimateTokens(text: string): number {
|
|
1927
|
-
// Approximation: 1 token ~= 4 characters for English text
|
|
1928
|
-
// Calibrated against cl100k_base tokenizer on a corpus of
|
|
1929
|
-
// CLAUDE.md files, trait definitions, and persona prompts.
|
|
1930
|
-
//
|
|
1931
|
-
// For code-heavy content, ratio is closer to 1:3.5
|
|
1932
|
-
// For prose-heavy content, ratio is closer to 1:4.5
|
|
1933
|
-
// We use 4 as a balanced default.
|
|
1934
|
-
|
|
1935
|
-
const charCount = text.length;
|
|
1936
|
-
const baseEstimate = Math.ceil(charCount / 4);
|
|
1937
|
-
|
|
1938
|
-
// Adjust for markdown formatting overhead
|
|
1939
|
-
const headingCount = (text.match(/^#{1,6}\s/gm) || []).length;
|
|
1940
|
-
const codeBlockCount = (text.match(/```/g) || []).length / 2;
|
|
1941
|
-
const listItemCount = (text.match(/^[\s]*[-*]\s/gm) || []).length;
|
|
1942
|
-
|
|
1943
|
-
const formattingOverhead = headingCount * 2 + codeBlockCount * 4 + listItemCount;
|
|
1944
|
-
|
|
1945
|
-
return baseEstimate + formattingOverhead;
|
|
1946
|
-
}
|
|
1947
|
-
```
|
|
1948
|
-
|
|
1949
|
-
**Accuracy:** This approximation is within +/-15% of the actual tokenizer output for typical AgentBoot content. For exact counting, organizations can install `tiktoken` or `gpt-tokenizer` and AgentBoot will use them if available (runtime detection).
|
|
1950
|
-
|
|
1951
|
-
**Budget calculation per persona:**
|
|
1952
|
-
|
|
1953
|
-
```
|
|
1954
|
-
total_tokens =
|
|
1955
|
-
estimateTokens(SKILL.md body)
|
|
1956
|
-
+ sum(estimateTokens(trait) for each composed trait)
|
|
1957
|
-
+ sum(estimateTokens(instruction) for each always-on instruction)
|
|
1958
|
-
+ sum(estimateTokens(rule) for likely-to-activate path-scoped rules)
|
|
1959
|
-
+ estimateTokens(generated frontmatter YAML)
|
|
1960
|
-
```
|
|
1961
|
-
|
|
1962
|
-
---
|
|
1963
|
-
|
|
1964
|
-
## 7. Test Specification
|
|
1965
|
-
|
|
1966
|
-
### 7.1 Test YAML Format (Canonical Schema)
|
|
1967
|
-
|
|
1968
|
-
This section resolves inconsistencies between `prompt-optimization.md` and `test-plan.md`. The canonical format is defined here.
|
|
1969
|
-
|
|
1970
|
-
**File location:** `tests/behavioral/{persona-name}.test.yaml` for behavioral tests, `tests/eval/{persona-name}.eval.yaml` for LLM-as-judge.
|
|
1971
|
-
|
|
1972
|
-
**Canonical behavioral test schema:**
|
|
1973
|
-
|
|
1974
|
-
```yaml
|
|
1975
|
-
# Required fields
|
|
1976
|
-
persona: string # Persona ID to test
|
|
1977
|
-
model: string # Model override for tests (default: haiku)
|
|
1978
|
-
max_turns: integer # Max turns per test case (default: 5)
|
|
1979
|
-
max_budget_usd: number # Cost cap per test case (default: 0.50)
|
|
1980
|
-
|
|
1981
|
-
# Optional
|
|
1982
|
-
flake_tolerance: string # Pass threshold: "2 of 3" (default), "3 of 3", "1 of 3"
|
|
1983
|
-
|
|
1984
|
-
# Test file setup (shared across cases unless overridden)
|
|
1985
|
-
setup:
|
|
1986
|
-
files: # Files to create in the test workspace
|
|
1987
|
-
- path: string # Relative file path
|
|
1988
|
-
content: string # File content (multi-line via | or >)
|
|
1989
|
-
|
|
1990
|
-
# Test cases
|
|
1991
|
-
cases:
|
|
1992
|
-
- name: string # Test case identifier (kebab-case)
|
|
1993
|
-
prompt: string # Prompt to send to the persona
|
|
1994
|
-
|
|
1995
|
-
# Optional per-case setup override
|
|
1996
|
-
setup_override:
|
|
1997
|
-
files:
|
|
1998
|
-
- path: string
|
|
1999
|
-
content: string
|
|
2000
|
-
|
|
2001
|
-
# Assertions (all optional; at least one required)
|
|
2002
|
-
expect:
|
|
2003
|
-
findings_min: integer # Minimum number of findings
|
|
2004
|
-
findings_max: integer # Maximum number of findings
|
|
2005
|
-
severity_includes: string[] # At least one finding has this severity
|
|
2006
|
-
severity_excludes: string[] # No finding has this severity
|
|
2007
|
-
text_matches: # Regex patterns that must match
|
|
2008
|
-
- pattern: string # Regex
|
|
2009
|
-
in: string # Where to match: "findings", "output", "all"
|
|
2010
|
-
text_excludes: # Regex patterns that must NOT match
|
|
2011
|
-
- pattern: string
|
|
2012
|
-
in: string
|
|
2013
|
-
confidence_min: number # All findings have confidence >= N
|
|
2014
|
-
output_contains: string[] # Literal strings in output
|
|
2015
|
-
output_structure: # Structural assertions
|
|
2016
|
-
has_sections: string[] # Named sections must be present
|
|
2017
|
-
json_schema: string # Path to JSON schema file for output validation
|
|
2018
|
-
token_max: integer # Output token budget
|
|
2019
|
-
duration_max_ms: integer # Maximum execution time
|
|
2020
|
-
```
|
|
2021
|
-
|
|
2022
|
-
**Resolution of inconsistencies:**
|
|
2023
|
-
|
|
2024
|
-
| Inconsistency | prompt-optimization.md | test-plan.md | Canonical Decision |
|
|
2025
|
-
|---------------|----------------------|--------------|-------------------|
|
|
2026
|
-
| Prompt field name | `input` | `prompt` | **`prompt`** -- more descriptive for the test's purpose |
|
|
2027
|
-
| Text assertion name | `text_includes` | `text_matches` with `pattern` | **`text_matches`** with `pattern` and `in` fields -- more flexible |
|
|
2028
|
-
| Setup mechanism | Inline code in `input` | `setup.files` with separate paths | **Both supported**: `setup.files` for file-based tests, inline code in `prompt` for simple cases |
|
|
2029
|
-
| Flake tolerance syntax | Not specified | `flake_tolerance: 2 of 3` | **`flake_tolerance: "2 of 3"`** -- human-readable string |
|
|
2030
|
-
|
|
2031
|
-
### 7.2 Assertion Types (Canonical List)
|
|
2032
|
-
|
|
2033
|
-
| Assertion | Type | Semantics |
|
|
2034
|
-
|-----------|------|-----------|
|
|
2035
|
-
| `findings_min` | integer | Output must contain at least N findings. Findings are detected by structured output parsing (severity labels). |
|
|
2036
|
-
| `findings_max` | integer | Output must contain at most N findings. Used for false positive checks. |
|
|
2037
|
-
| `severity_includes` | string[] | At least one finding must have a severity from this list. Values: `CRITICAL`, `ERROR`, `WARN`, `INFO`. |
|
|
2038
|
-
| `severity_excludes` | string[] | No finding may have a severity from this list. |
|
|
2039
|
-
| `text_matches` | array of `{pattern, in}` | Regex pattern must match somewhere in the specified scope. `in` values: `findings` (finding text only), `output` (full output), `all` (default). |
|
|
2040
|
-
| `text_excludes` | array of `{pattern, in}` | Regex pattern must NOT match in the specified scope. |
|
|
2041
|
-
| `confidence_min` | number (0.0-1.0) | Every finding with a confidence score must have confidence >= N. |
|
|
2042
|
-
| `output_contains` | string[] | Each literal string must appear somewhere in the output. |
|
|
2043
|
-
| `output_structure` | object | Structural checks. `has_sections`: list of section names (headers) that must be present. |
|
|
2044
|
-
| `json_schema` | string | Path to a JSON schema file. Output is parsed as JSON and validated against the schema. |
|
|
2045
|
-
| `token_max` | integer | Output must not exceed N tokens (estimated). |
|
|
2046
|
-
| `duration_max_ms` | integer | Test execution must complete within N milliseconds. |
|
|
2047
|
-
| `judge_score_min` | object | (Eval tests only) Minimum scores from LLM judge. Keys are dimension names, values are minimum scores (1-5) or `"pass"`. |
|
|
2048
|
-
|
|
2049
|
-
### 7.3 Test Runner Algorithm
|
|
2050
|
-
|
|
2051
|
-
**Behavioral test execution:**
|
|
2052
|
-
|
|
2053
|
-
```
|
|
2054
|
-
function runBehavioralTests(testFile, options):
|
|
2055
|
-
config = parseYaml(testFile)
|
|
2056
|
-
results = []
|
|
2057
|
-
|
|
2058
|
-
for each case in config.cases:
|
|
2059
|
-
// Determine retry count from flake_tolerance
|
|
2060
|
-
[requiredPasses, totalRuns] = parseFlakeTolerance(config.flake_tolerance || "2 of 3")
|
|
2061
|
-
passCount = 0
|
|
2062
|
-
runResults = []
|
|
2063
|
-
|
|
2064
|
-
for run in 1..totalRuns:
|
|
2065
|
-
// Set up test workspace
|
|
2066
|
-
workspace = createTempDir()
|
|
2067
|
-
if case.setup_override:
|
|
2068
|
-
writeFiles(workspace, case.setup_override.files)
|
|
2069
|
-
else if config.setup:
|
|
2070
|
-
writeFiles(workspace, config.setup.files)
|
|
2071
|
-
|
|
2072
|
-
// Invoke persona
|
|
2073
|
-
output = exec(
|
|
2074
|
-
`claude -p \
|
|
2075
|
-
--agent ${config.persona} \
|
|
2076
|
-
--output-format json \
|
|
2077
|
-
--max-turns ${config.max_turns} \
|
|
2078
|
-
--max-budget-usd ${config.max_budget_usd} \
|
|
2079
|
-
--permission-mode bypassPermissions \
|
|
2080
|
-
--no-session-persistence \
|
|
2081
|
-
"${case.prompt}"`,
|
|
2082
|
-
{ cwd: workspace }
|
|
2083
|
-
)
|
|
2084
|
-
|
|
2085
|
-
// Parse output
|
|
2086
|
-
parsed = parseJsonOutput(output)
|
|
2087
|
-
|
|
2088
|
-
// Evaluate assertions
|
|
2089
|
-
assertionResults = evaluateAssertions(case.expect, parsed)
|
|
2090
|
-
passed = assertionResults.every(a => a.passed)
|
|
2091
|
-
|
|
2092
|
-
if passed: passCount++
|
|
2093
|
-
runResults.push({ run, passed, assertionResults, cost: parsed.cost })
|
|
2094
|
-
|
|
2095
|
-
// Cleanup
|
|
2096
|
-
removeTempDir(workspace)
|
|
2097
|
-
|
|
2098
|
-
// Early exit: if enough passes already, skip remaining runs
|
|
2099
|
-
if passCount >= requiredPasses: break
|
|
2100
|
-
|
|
2101
|
-
// Early exit: if impossible to reach required passes, skip remaining
|
|
2102
|
-
remainingRuns = totalRuns - run
|
|
2103
|
-
if passCount + remainingRuns < requiredPasses: break
|
|
2104
|
-
|
|
2105
|
-
results.push({
|
|
2106
|
-
name: case.name,
|
|
2107
|
-
passed: passCount >= requiredPasses,
|
|
2108
|
-
passCount,
|
|
2109
|
-
totalRuns: runResults.length,
|
|
2110
|
-
runs: runResults,
|
|
2111
|
-
totalCost: sum(runResults.map(r => r.cost))
|
|
2112
|
-
})
|
|
2113
|
-
|
|
2114
|
-
// Check budget
|
|
2115
|
-
runningCost += results.last().totalCost
|
|
2116
|
-
if runningCost > options.maxBudget:
|
|
2117
|
-
results.push({ name: "BUDGET_EXCEEDED", passed: false })
|
|
2118
|
-
break
|
|
2119
|
-
|
|
2120
|
-
return results
|
|
2121
|
-
```
|
|
2122
|
-
|
|
2123
|
-
**Cost tracking:**
|
|
2124
|
-
|
|
2125
|
-
Every test run tracks its cost via the JSON output from `claude -p`. The runner maintains a running total and aborts when the budget cap is reached. Partial results are reported.
|
|
2126
|
-
|
|
2127
|
-
**Flake tolerance parsing:**
|
|
2128
|
-
|
|
2129
|
-
```
|
|
2130
|
-
function parseFlakeTolerance(s: string): [required: number, total: number]
|
|
2131
|
-
// "2 of 3" -> [2, 3]
|
|
2132
|
-
// "3 of 3" -> [3, 3] (strict, no flake tolerance)
|
|
2133
|
-
// "1 of 3" -> [1, 3] (very lenient)
|
|
2134
|
-
match = s.match(/(\d+)\s+of\s+(\d+)/)
|
|
2135
|
-
return [parseInt(match[1]), parseInt(match[2])]
|
|
2136
|
-
```
|
|
2137
|
-
|
|
2138
|
-
### 7.4 Snapshot Format
|
|
2139
|
-
|
|
2140
|
-
Snapshots store structural summaries for regression comparison:
|
|
2141
|
-
|
|
2142
|
-
```json
|
|
2143
|
-
{
|
|
2144
|
-
"persona": "code-reviewer",
|
|
2145
|
-
"test_case": "sql-injection-detection",
|
|
2146
|
-
"snapshot_date": "2026-03-19",
|
|
2147
|
-
"model": "haiku",
|
|
2148
|
-
"findings_count": {
|
|
2149
|
-
"CRITICAL": 1,
|
|
2150
|
-
"ERROR": 0,
|
|
2151
|
-
"WARN": 0,
|
|
2152
|
-
"INFO": 0
|
|
2153
|
-
},
|
|
2154
|
-
"finding_patterns": ["SQL injection", "parameterized"],
|
|
2155
|
-
"total_tokens": 1200,
|
|
2156
|
-
"duration_ms": 8500,
|
|
2157
|
-
"output_hash": "sha256:abc123..."
|
|
2158
|
-
}
|
|
2159
|
-
```
|
|
2160
|
-
|
|
2161
|
-
**Snapshot comparison algorithm:**
|
|
2162
|
-
|
|
2163
|
-
```
|
|
2164
|
-
function compareSnapshots(baseline, current):
|
|
2165
|
-
diffs = []
|
|
2166
|
-
|
|
2167
|
-
// Compare finding counts
|
|
2168
|
-
for severity in [CRITICAL, ERROR, WARN, INFO]:
|
|
2169
|
-
if baseline.findings_count[severity] != current.findings_count[severity]:
|
|
2170
|
-
diffs.push({
|
|
2171
|
-
field: `findings_count.${severity}`,
|
|
2172
|
-
baseline: baseline.findings_count[severity],
|
|
2173
|
-
current: current.findings_count[severity],
|
|
2174
|
-
type: current > baseline ? "REGRESSION_CANDIDATE" : "IMPROVEMENT_CANDIDATE"
|
|
2175
|
-
})
|
|
2176
|
-
|
|
2177
|
-
// Compare finding patterns
|
|
2178
|
-
missingPatterns = baseline.finding_patterns.filter(p => !current.finding_patterns.includes(p))
|
|
2179
|
-
newPatterns = current.finding_patterns.filter(p => !baseline.finding_patterns.includes(p))
|
|
2180
|
-
|
|
2181
|
-
if missingPatterns.length > 0:
|
|
2182
|
-
diffs.push({ type: "MISSING_PATTERNS", patterns: missingPatterns })
|
|
2183
|
-
if newPatterns.length > 0:
|
|
2184
|
-
diffs.push({ type: "NEW_PATTERNS", patterns: newPatterns })
|
|
2185
|
-
|
|
2186
|
-
// Compare token usage (significant change = >20% difference)
|
|
2187
|
-
tokenDelta = abs(current.total_tokens - baseline.total_tokens) / baseline.total_tokens
|
|
2188
|
-
if tokenDelta > 0.20:
|
|
2189
|
-
diffs.push({ type: "TOKEN_CHANGE", baseline: baseline.total_tokens, current: current.total_tokens })
|
|
2190
|
-
|
|
2191
|
-
return { match: diffs.length === 0, diffs }
|
|
2192
|
-
```
|
|
2193
|
-
|
|
2194
|
-
### 7.5 LLM-as-Judge Eval Format
|
|
2195
|
-
|
|
2196
|
-
```yaml
|
|
2197
|
-
persona_under_test: string # Persona ID
|
|
2198
|
-
judge_model: string # Model for the judge (typically opus)
|
|
2199
|
-
max_budget_usd: number # Cost cap for the full eval
|
|
2200
|
-
|
|
2201
|
-
cases:
|
|
2202
|
-
- name: string # Eval case name
|
|
2203
|
-
input_file: string # Path to code fixture
|
|
2204
|
-
persona_prompt: string # Prompt sent to persona under test
|
|
2205
|
-
judge_prompt: string # Evaluation prompt sent to judge model
|
|
2206
|
-
# Supports placeholders: {input}, {persona_output}
|
|
2207
|
-
ground_truth: # Optional: known issues in the code
|
|
2208
|
-
- string
|
|
2209
|
-
expect:
|
|
2210
|
-
judge_score_min: # Minimum scores per dimension
|
|
2211
|
-
completeness: integer # 1-5 scale
|
|
2212
|
-
accuracy: integer
|
|
2213
|
-
specificity: integer
|
|
2214
|
-
prioritization: integer
|
|
2215
|
-
tone: integer
|
|
2216
|
-
overall: string # "pass" or "fail"
|
|
2217
|
-
```
|
|
2218
|
-
|
|
2219
|
-
**Eval execution:**
|
|
2220
|
-
|
|
2221
|
-
```
|
|
2222
|
-
function runEval(evalFile):
|
|
2223
|
-
config = parseYaml(evalFile)
|
|
2224
|
-
results = []
|
|
2225
|
-
|
|
2226
|
-
for each case in config.cases:
|
|
2227
|
-
// Step 1: Run the persona under test
|
|
2228
|
-
personaOutput = exec(`claude -p --agent ${config.persona_under_test} ...`, case.persona_prompt)
|
|
2229
|
-
|
|
2230
|
-
// Step 2: Send output to judge
|
|
2231
|
-
judgePrompt = case.judge_prompt
|
|
2232
|
-
.replace("{input}", readFile(case.input_file))
|
|
2233
|
-
.replace("{persona_output}", personaOutput)
|
|
2234
|
-
|
|
2235
|
-
judgeOutput = exec(`claude -p --model ${config.judge_model} ...`, judgePrompt)
|
|
2236
|
-
|
|
2237
|
-
// Step 3: Parse judge scores
|
|
2238
|
-
scores = parseJudgeScores(judgeOutput)
|
|
2239
|
-
|
|
2240
|
-
// Step 4: Evaluate against thresholds
|
|
2241
|
-
passed = true
|
|
2242
|
-
for [dimension, minScore] in case.expect.judge_score_min:
|
|
2243
|
-
if dimension == "overall":
|
|
2244
|
-
passed = passed && scores.overall == minScore
|
|
2245
|
-
else:
|
|
2246
|
-
passed = passed && scores[dimension] >= minScore
|
|
2247
|
-
|
|
2248
|
-
results.push({ name: case.name, passed, scores, personaOutput, judgeOutput })
|
|
2249
|
-
|
|
2250
|
-
return results
|
|
2251
|
-
```
|
|
2252
|
-
|
|
2253
|
-
---
|
|
2254
|
-
|
|
2255
|
-
## 8. Telemetry Specification
|
|
2256
|
-
|
|
2257
|
-
### 8.1 Canonical Telemetry Event Schema
|
|
2258
|
-
|
|
2259
|
-
This section resolves field name inconsistencies across source documents. The following is the canonical schema.
|
|
2260
|
-
|
|
2261
|
-
```json
|
|
2262
|
-
{
|
|
2263
|
-
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2264
|
-
"$id": "https://agentboot.dev/schema/telemetry-event/v1",
|
|
2265
|
-
"title": "AgentBoot Telemetry Event",
|
|
2266
|
-
"type": "object",
|
|
2267
|
-
"required": ["event", "persona_id", "timestamp"],
|
|
2268
|
-
"properties": {
|
|
2269
|
-
"event": {
|
|
2270
|
-
"type": "string",
|
|
2271
|
-
"enum": ["persona_invocation", "persona_error", "hook_execution", "session_summary"],
|
|
2272
|
-
"description": "Event type"
|
|
2273
|
-
},
|
|
2274
|
-
"persona_id": {
|
|
2275
|
-
"type": "string",
|
|
2276
|
-
"description": "Persona identifier (e.g., 'security-reviewer')"
|
|
2277
|
-
},
|
|
2278
|
-
"persona_version": {
|
|
2279
|
-
"type": "string",
|
|
2280
|
-
"description": "Persona version from config or package.json"
|
|
2281
|
-
},
|
|
2282
|
-
"model": {
|
|
2283
|
-
"type": "string",
|
|
2284
|
-
"description": "Model used for this invocation"
|
|
2285
|
-
},
|
|
2286
|
-
"scope": {
|
|
2287
|
-
"type": "string",
|
|
2288
|
-
"description": "Scope path: 'org:team:group' format (e.g., 'acme:platform:api')"
|
|
2289
|
-
},
|
|
2290
|
-
"input_tokens": {
|
|
2291
|
-
"type": "integer",
|
|
2292
|
-
"description": "Input token count for this invocation"
|
|
2293
|
-
},
|
|
2294
|
-
"output_tokens": {
|
|
2295
|
-
"type": "integer",
|
|
2296
|
-
"description": "Output token count"
|
|
2297
|
-
},
|
|
2298
|
-
"thinking_tokens": {
|
|
2299
|
-
"type": "integer",
|
|
2300
|
-
"description": "Extended thinking token count (billed as output)"
|
|
2301
|
-
},
|
|
2302
|
-
"tool_calls": {
|
|
2303
|
-
"type": "integer",
|
|
2304
|
-
"description": "Number of tool calls in this invocation"
|
|
2305
|
-
},
|
|
2306
|
-
"duration_ms": {
|
|
2307
|
-
"type": "integer",
|
|
2308
|
-
"description": "Wall-clock duration in milliseconds"
|
|
2309
|
-
},
|
|
2310
|
-
"cost_usd": {
|
|
2311
|
-
"type": "number",
|
|
2312
|
-
"description": "Estimated cost in USD"
|
|
2313
|
-
},
|
|
2314
|
-
"findings_count": {
|
|
2315
|
-
"type": "object",
|
|
2316
|
-
"description": "Finding counts by severity",
|
|
2317
|
-
"properties": {
|
|
2318
|
-
"CRITICAL": { "type": "integer" },
|
|
2319
|
-
"ERROR": { "type": "integer" },
|
|
2320
|
-
"WARN": { "type": "integer" },
|
|
2321
|
-
"INFO": { "type": "integer" }
|
|
2322
|
-
}
|
|
2323
|
-
},
|
|
2324
|
-
"suggestions": {
|
|
2325
|
-
"type": "integer",
|
|
2326
|
-
"description": "Number of non-finding suggestions"
|
|
2327
|
-
},
|
|
2328
|
-
"timestamp": {
|
|
2329
|
-
"type": "string",
|
|
2330
|
-
"format": "date-time",
|
|
2331
|
-
"description": "ISO 8601 timestamp"
|
|
2332
|
-
},
|
|
2333
|
-
"session_id": {
|
|
2334
|
-
"type": "string",
|
|
2335
|
-
"description": "Session identifier (opaque string)"
|
|
2336
|
-
},
|
|
2337
|
-
"gate_result": {
|
|
2338
|
-
"type": "string",
|
|
2339
|
-
"enum": ["passed", "failed", "skipped"],
|
|
2340
|
-
"description": "CI gate result (if applicable)"
|
|
2341
|
-
}
|
|
2342
|
-
}
|
|
2343
|
-
}
|
|
2344
|
-
```
|
|
2345
|
-
|
|
2346
|
-
**Field name resolution:**
|
|
2347
|
-
|
|
2348
|
-
| Source doc field | Canonical field | Reason |
|
|
2349
|
-
|-----------------|----------------|--------|
|
|
2350
|
-
| `persona` (ci-cd-automation.md) | `persona_id` | Consistent with other `_id` suffixed fields |
|
|
2351
|
-
| `scope: "team:platform/api"` (prompt-optimization.md) | `scope: "acme:platform:api"` | Use colon separators consistently, include org |
|
|
2352
|
-
| `version` (ci-cd-automation.md) | `persona_version` | Avoid ambiguity with AgentBoot version |
|
|
2353
|
-
|
|
2354
|
-
### 8.2 Hook Generation
|
|
2355
|
-
|
|
2356
|
-
AgentBoot generates audit trail hooks that emit telemetry events. All hooks are `async: true` to avoid blocking the developer.
|
|
2357
|
-
|
|
2358
|
-
**Generated hooks:**
|
|
2359
|
-
|
|
2360
|
-
| Hook Event | Matcher | Script | What It Emits |
|
|
2361
|
-
|------------|---------|--------|---------------|
|
|
2362
|
-
| `SubagentStart` | (all) | `.claude/hooks/agentboot-telemetry.sh` | `persona_invocation` event with start timestamp, persona_id, model |
|
|
2363
|
-
| `SubagentStop` | (all) | `.claude/hooks/agentboot-telemetry.sh` | Completes the `persona_invocation` event with duration, tokens, cost, findings |
|
|
2364
|
-
| `PostToolUse` | `Edit\|Write\|Bash` | `.claude/hooks/agentboot-telemetry.sh` | Increments `tool_calls` counter |
|
|
2365
|
-
| `SessionEnd` | (all) | `.claude/hooks/agentboot-telemetry.sh` | `session_summary` event with total cost, total invocations |
|
|
2366
|
-
|
|
2367
|
-
**Hook script behavior:**
|
|
2368
|
-
|
|
2369
|
-
The generated hook script is a single shell script that reads the hook input JSON from stdin, extracts relevant fields, and appends an NDJSON line to the telemetry log.
|
|
2370
|
-
|
|
2371
|
-
```bash
|
|
2372
|
-
#!/bin/bash
|
|
2373
|
-
# .claude/hooks/agentboot-telemetry.sh
|
|
2374
|
-
# Generated by AgentBoot. Do not edit manually.
|
|
2375
|
-
|
|
2376
|
-
TELEMETRY_LOG="${AGENTBOOT_TELEMETRY_LOG:-$HOME/.agentboot/telemetry.ndjson}"
|
|
2377
|
-
INPUT=$(cat)
|
|
2378
|
-
|
|
2379
|
-
EVENT_NAME=$(echo "$INPUT" | jq -r '.hook_event_name')
|
|
2380
|
-
AGENT_TYPE=$(echo "$INPUT" | jq -r '.agent_type // empty')
|
|
2381
|
-
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
|
|
2382
|
-
|
|
2383
|
-
case "$EVENT_NAME" in
|
|
2384
|
-
SubagentStart)
|
|
2385
|
-
echo "{\"event\":\"persona_invocation\",\"persona_id\":\"$AGENT_TYPE\",\"timestamp\":\"$TIMESTAMP\",\"status\":\"started\"}" >> "$TELEMETRY_LOG"
|
|
2386
|
-
;;
|
|
2387
|
-
SubagentStop)
|
|
2388
|
-
# Extract completion data from hook input
|
|
2389
|
-
echo "{\"event\":\"persona_invocation\",\"persona_id\":\"$AGENT_TYPE\",\"timestamp\":\"$TIMESTAMP\",\"status\":\"completed\"}" >> "$TELEMETRY_LOG"
|
|
2390
|
-
;;
|
|
2391
|
-
SessionEnd)
|
|
2392
|
-
echo "{\"event\":\"session_summary\",\"timestamp\":\"$TIMESTAMP\"}" >> "$TELEMETRY_LOG"
|
|
2393
|
-
;;
|
|
2394
|
-
esac
|
|
2395
|
-
|
|
2396
|
-
echo '{"continue":true}' # Always allow continuation
|
|
2397
|
-
```
|
|
2398
|
-
|
|
2399
|
-
**Async behavior:** All telemetry hooks set `"async": true` in the hook configuration. This means the hook runs in the background and does not block the main agent execution. If the hook fails (e.g., cannot write to log file), the failure is silently ignored.
|
|
2400
|
-
|
|
2401
|
-
### 8.3 NDJSON Output Format
|
|
2402
|
-
|
|
2403
|
-
Telemetry is stored as Newline-Delimited JSON (NDJSON). Each line is a complete, valid JSON object.
|
|
2404
|
-
|
|
2405
|
-
**File location:** `~/.agentboot/telemetry.ndjson` (user-level default, configurable via `agentboot.config.json` or `$AGENTBOOT_TELEMETRY_LOG`)
|
|
2406
|
-
|
|
2407
|
-
**Example:**
|
|
2408
|
-
```
|
|
2409
|
-
{"event":"persona_invocation","persona_id":"code-reviewer","persona_version":"1.2.0","model":"sonnet","scope":"acme:platform:api","input_tokens":8420,"output_tokens":3200,"thinking_tokens":12000,"tool_calls":7,"duration_ms":45000,"cost_usd":0.089,"findings_count":{"CRITICAL":0,"ERROR":1,"WARN":3,"INFO":2},"suggestions":2,"timestamp":"2026-03-19T14:30:00Z","session_id":"abc123"}
|
|
2410
|
-
{"event":"persona_invocation","persona_id":"security-reviewer","persona_version":"1.2.0","model":"opus","scope":"acme:platform:api","input_tokens":12400,"output_tokens":5100,"thinking_tokens":18000,"tool_calls":12,"duration_ms":92000,"cost_usd":0.57,"findings_count":{"CRITICAL":1,"ERROR":2,"WARN":1,"INFO":0},"suggestions":0,"timestamp":"2026-03-19T14:32:00Z","session_id":"abc123"}
|
|
2411
|
-
```
|
|
2412
|
-
|
|
2413
|
-
**Rotation:** AgentBoot does not implement log rotation in V1. Organizations should configure external rotation (logrotate, or periodic archival). The `agentboot metrics` command reads all NDJSON files matching the configured pattern.
|
|
2414
|
-
|
|
2415
|
-
### 8.4 `agentboot metrics` Aggregation Algorithm
|
|
2416
|
-
|
|
2417
|
-
```
|
|
2418
|
-
function aggregateMetrics(events, filters):
|
|
2419
|
-
// Apply filters
|
|
2420
|
-
filtered = events
|
|
2421
|
-
.filter(e => e.event === "persona_invocation" && e.status === "completed")
|
|
2422
|
-
.filter(e => !filters.persona || e.persona_id === filters.persona)
|
|
2423
|
-
.filter(e => !filters.team || e.scope.includes(filters.team))
|
|
2424
|
-
.filter(e => !filters.period || isWithinPeriod(e.timestamp, filters.period))
|
|
2425
|
-
|
|
2426
|
-
// Group by persona
|
|
2427
|
-
byPersona = groupBy(filtered, "persona_id")
|
|
2428
|
-
|
|
2429
|
-
// Compute aggregates per persona
|
|
2430
|
-
for [personaId, events] in byPersona:
|
|
2431
|
-
yield {
|
|
2432
|
-
persona_id: personaId,
|
|
2433
|
-
invocation_count: events.length,
|
|
2434
|
-
avg_input_tokens: mean(events.map(e => e.input_tokens)),
|
|
2435
|
-
avg_output_tokens: mean(events.map(e => e.output_tokens)),
|
|
2436
|
-
avg_cost_usd: mean(events.map(e => e.cost_usd)),
|
|
2437
|
-
total_cost_usd: sum(events.map(e => e.cost_usd)),
|
|
2438
|
-
avg_duration_ms: mean(events.map(e => e.duration_ms)),
|
|
2439
|
-
avg_tool_calls: mean(events.map(e => e.tool_calls)),
|
|
2440
|
-
findings_distribution: {
|
|
2441
|
-
CRITICAL: sum(events.map(e => e.findings_count?.CRITICAL || 0)),
|
|
2442
|
-
ERROR: sum(events.map(e => e.findings_count?.ERROR || 0)),
|
|
2443
|
-
WARN: sum(events.map(e => e.findings_count?.WARN || 0)),
|
|
2444
|
-
INFO: sum(events.map(e => e.findings_count?.INFO || 0))
|
|
2445
|
-
},
|
|
2446
|
-
model_distribution: countBy(events, "model"),
|
|
2447
|
-
period: { start: min(timestamps), end: max(timestamps) }
|
|
2448
|
-
}
|
|
2449
|
-
```
|
|
2450
|
-
|
|
2451
|
-
---
|
|
2452
|
-
|
|
2453
|
-
## 9. Plugin and Marketplace Specification
|
|
2454
|
-
|
|
2455
|
-
### 9.1 Plugin Structure
|
|
2456
|
-
|
|
2457
|
-
The `agentboot export --format plugin` command produces a directory compatible with the plugin system.
|
|
2458
|
-
|
|
2459
|
-
**Plugin directory layout:**
|
|
2460
|
-
|
|
2461
|
-
```
|
|
2462
|
-
dist/plugin/
|
|
2463
|
-
.claude-plugin/
|
|
2464
|
-
plugin.json # Plugin manifest
|
|
2465
|
-
agents/
|
|
2466
|
-
code-reviewer/
|
|
2467
|
-
CLAUDE.md # Agent definition with frontmatter
|
|
2468
|
-
security-reviewer/
|
|
2469
|
-
CLAUDE.md
|
|
2470
|
-
skills/
|
|
2471
|
-
review-code/
|
|
2472
|
-
SKILL.md # Skill with context: fork
|
|
2473
|
-
review-security/
|
|
2474
|
-
SKILL.md
|
|
2475
|
-
hooks/
|
|
2476
|
-
hooks.json # Hook definitions
|
|
2477
|
-
rules/
|
|
2478
|
-
gotchas-postgres.md # Path-scoped rules
|
|
2479
|
-
traits/
|
|
2480
|
-
critical-thinking.md # Shared traits
|
|
2481
|
-
```
|
|
2482
|
-
|
|
2483
|
-
**plugin.json schema:**
|
|
2484
|
-
|
|
2485
|
-
```json
|
|
2486
|
-
{
|
|
2487
|
-
"name": "{org}@{org}-personas",
|
|
2488
|
-
"version": "1.2.0",
|
|
2489
|
-
"description": "Agentic personas for {org}",
|
|
2490
|
-
"author": "{org}",
|
|
2491
|
-
"license": "Apache-2.0",
|
|
2492
|
-
"agentboot_version": "0.1.0",
|
|
2493
|
-
"personas": [
|
|
2494
|
-
{
|
|
2495
|
-
"id": "code-reviewer",
|
|
2496
|
-
"name": "Code Reviewer",
|
|
2497
|
-
"description": "Reviews code changes for quality, consistency, and bugs",
|
|
2498
|
-
"model": "sonnet",
|
|
2499
|
-
"agent_path": "agents/code-reviewer/CLAUDE.md",
|
|
2500
|
-
"skill_path": "skills/review-code/SKILL.md"
|
|
2501
|
-
}
|
|
2502
|
-
],
|
|
2503
|
-
"traits": [
|
|
2504
|
-
{
|
|
2505
|
-
"id": "critical-thinking",
|
|
2506
|
-
"path": "traits/critical-thinking.md"
|
|
2507
|
-
}
|
|
2508
|
-
]
|
|
2509
|
-
}
|
|
2510
|
-
```
|
|
2511
|
-
|
|
2512
|
-
### 9.2 marketplace.json Format
|
|
2513
|
-
|
|
2514
|
-
The marketplace index file lives in the marketplace repository root.
|
|
2515
|
-
|
|
2516
|
-
```json
|
|
2517
|
-
{
|
|
2518
|
-
"$schema": "https://agentboot.dev/schema/marketplace/v1",
|
|
2519
|
-
"name": "{org}-personas",
|
|
2520
|
-
"description": "Agentic personas marketplace for {org}",
|
|
2521
|
-
"maintainer": "{org}",
|
|
2522
|
-
"url": "https://github.com/{org}/{org}-personas",
|
|
2523
|
-
"entries": [
|
|
2524
|
-
{
|
|
2525
|
-
"type": "plugin",
|
|
2526
|
-
"name": "{org}@{org}-personas",
|
|
2527
|
-
"version": "1.2.0",
|
|
2528
|
-
"description": "Full persona suite for {org}",
|
|
2529
|
-
"published_at": "2026-03-19T14:30:00Z",
|
|
2530
|
-
"sha256": "abc123...",
|
|
2531
|
-
"path": "releases/v1.2.0/"
|
|
2532
|
-
},
|
|
2533
|
-
{
|
|
2534
|
-
"type": "trait",
|
|
2535
|
-
"name": "critical-thinking",
|
|
2536
|
-
"version": "1.0.0",
|
|
2537
|
-
"description": "Apply systematic skepticism to every claim and finding",
|
|
2538
|
-
"published_at": "2026-03-15T10:00:00Z",
|
|
2539
|
-
"path": "traits/critical-thinking/"
|
|
2540
|
-
},
|
|
2541
|
-
{
|
|
2542
|
-
"type": "domain",
|
|
2543
|
-
"name": "healthcare",
|
|
2544
|
-
"version": "1.0.0",
|
|
2545
|
-
"description": "HIPAA compliance domain layer",
|
|
2546
|
-
"published_at": "2026-03-10T08:00:00Z",
|
|
2547
|
-
"path": "domains/healthcare/"
|
|
2548
|
-
}
|
|
2549
|
-
]
|
|
2550
|
-
}
|
|
2551
|
-
```
|
|
2552
|
-
|
|
2553
|
-
### 9.3 How `agentboot publish` Works
|
|
2554
|
-
|
|
2555
|
-
1. Read compiled plugin output from `dist/plugin/`
|
|
2556
|
-
2. Validate plugin structure (plugin.json exists, all referenced files exist)
|
|
2557
|
-
3. If `--bump` specified, increment version in `plugin.json` and `package.json`
|
|
2558
|
-
4. Copy plugin directory to marketplace repository at `releases/v{version}/`
|
|
2559
|
-
5. Compute SHA-256 hash of the release directory
|
|
2560
|
-
6. Add or update entry in `marketplace.json`
|
|
2561
|
-
7. Git commit: `chore: publish {plugin-name} v{version}`
|
|
2562
|
-
8. Git push
|
|
2563
|
-
|
|
2564
|
-
### 9.4 How `agentboot search` Works
|
|
2565
|
-
|
|
2566
|
-
1. Fetch `marketplace.json` from configured marketplace URL(s)
|
|
2567
|
-
2. Cache locally at `~/.agentboot/cache/marketplace-{hash}.json` (TTL: 1 hour)
|
|
2568
|
-
3. Search entries by:
|
|
2569
|
-
- Full-text match on `name` and `description`
|
|
2570
|
-
- Filter by `type` if `--type` specified
|
|
2571
|
-
4. Rank results by relevance (name match > description match)
|
|
2572
|
-
5. Display results with install command for each
|
|
2573
|
-
|
|
2574
|
-
**Default marketplace registries:**
|
|
2575
|
-
- Public AgentBoot marketplace: `https://github.com/agentboot/marketplace` (when available)
|
|
2576
|
-
- Org marketplace: from `agentboot.config.json` or `connect` config
|
|
2577
|
-
|
|
2578
|
-
---
|
|
2579
|
-
|
|
2580
|
-
## 10. Knowledge Server Specification (MCP)
|
|
2581
|
-
|
|
2582
|
-
### 10.1 MCP Tool Definitions
|
|
2583
|
-
|
|
2584
|
-
AgentBoot exposes knowledge as MCP tools. The tool interface stays stable across all three knowledge stages.
|
|
2585
|
-
|
|
2586
|
-
**Tool: `agentboot_kb_search`**
|
|
2587
|
-
|
|
2588
|
-
| Property | Value |
|
|
2589
|
-
|----------|-------|
|
|
2590
|
-
| Name | `agentboot_kb_search` |
|
|
2591
|
-
| Description | Search organizational knowledge base for relevant rules, gotchas, patterns, and domain knowledge |
|
|
2592
|
-
| Input schema | `{ "query": string, "type?": "gotcha" | "trait" | "adr" | "pattern" | "all", "limit?": integer }` |
|
|
2593
|
-
| Output | Array of `{ id, type, title, content, relevance_score, paths, tags }` |
|
|
2594
|
-
|
|
2595
|
-
**Tool: `agentboot_kb_get`**
|
|
2596
|
-
|
|
2597
|
-
| Property | Value |
|
|
2598
|
-
|----------|-------|
|
|
2599
|
-
| Name | `agentboot_kb_get` |
|
|
2600
|
-
| Description | Get a specific knowledge item by ID |
|
|
2601
|
-
| Input schema | `{ "id": string }` |
|
|
2602
|
-
| Output | `{ id, type, title, content, metadata, related_ids }` |
|
|
2603
|
-
|
|
2604
|
-
**Tool: `agentboot_kb_list`**
|
|
2605
|
-
|
|
2606
|
-
| Property | Value |
|
|
2607
|
-
|----------|-------|
|
|
2608
|
-
| Name | `agentboot_kb_list` |
|
|
2609
|
-
| Description | List knowledge items by category or tag |
|
|
2610
|
-
| Input schema | `{ "type?": string, "tag?": string, "limit?": integer, "offset?": integer }` |
|
|
2611
|
-
| Output | Array of `{ id, type, title, summary, tags }` |
|
|
2612
|
-
|
|
2613
|
-
**Tool: `agentboot_persona_invoke`**
|
|
2614
|
-
|
|
2615
|
-
| Property | Value |
|
|
2616
|
-
|----------|-------|
|
|
2617
|
-
| Name | `agentboot_persona_invoke` |
|
|
2618
|
-
| Description | Invoke an AgentBoot persona against provided input |
|
|
2619
|
-
| Input schema | `{ "persona": string, "input": string, "format?": "text" | "json" }` |
|
|
2620
|
-
| Output | Persona output (structured findings or text) |
|
|
2621
|
-
|
|
2622
|
-
### 10.2 SQLite Schema (Stage 2)
|
|
2623
|
-
|
|
2624
|
-
Generated by `agentboot build --index`. Stores structured metadata extracted from markdown frontmatter.
|
|
2625
|
-
|
|
2626
|
-
```sql
|
|
2627
|
-
-- Knowledge items (gotchas, traits, patterns, ADRs)
|
|
2628
|
-
CREATE TABLE knowledge_items (
|
|
2629
|
-
id TEXT PRIMARY KEY, -- e.g., "gotcha:postgres-rls"
|
|
2630
|
-
type TEXT NOT NULL, -- "gotcha", "trait", "adr", "pattern", "instruction"
|
|
2631
|
-
title TEXT NOT NULL,
|
|
2632
|
-
content TEXT NOT NULL, -- Full markdown content
|
|
2633
|
-
summary TEXT, -- First paragraph or description
|
|
2634
|
-
source_path TEXT NOT NULL, -- Relative file path
|
|
2635
|
-
created_at TEXT NOT NULL, -- ISO 8601
|
|
2636
|
-
updated_at TEXT NOT NULL
|
|
2637
|
-
);
|
|
2638
|
-
|
|
2639
|
-
-- Tags for categorization
|
|
2640
|
-
CREATE TABLE tags (
|
|
2641
|
-
item_id TEXT NOT NULL REFERENCES knowledge_items(id),
|
|
2642
|
-
tag TEXT NOT NULL,
|
|
2643
|
-
PRIMARY KEY (item_id, tag)
|
|
2644
|
-
);
|
|
2645
|
-
|
|
2646
|
-
-- Path scoping (which file patterns this item applies to)
|
|
2647
|
-
CREATE TABLE path_scopes (
|
|
2648
|
-
item_id TEXT NOT NULL REFERENCES knowledge_items(id),
|
|
2649
|
-
glob_pattern TEXT NOT NULL, -- e.g., "**/*.ts", "src/api/**"
|
|
2650
|
-
PRIMARY KEY (item_id, glob_pattern)
|
|
2651
|
-
);
|
|
2652
|
-
|
|
2653
|
-
-- Related items (bidirectional links)
|
|
2654
|
-
CREATE TABLE related_items (
|
|
2655
|
-
item_id TEXT NOT NULL REFERENCES knowledge_items(id),
|
|
2656
|
-
related_id TEXT NOT NULL REFERENCES knowledge_items(id),
|
|
2657
|
-
relation_type TEXT NOT NULL, -- "supersedes", "related", "conflicts", "extends"
|
|
2658
|
-
PRIMARY KEY (item_id, related_id)
|
|
2659
|
-
);
|
|
2660
|
-
|
|
2661
|
-
-- Full-text search index
|
|
2662
|
-
CREATE VIRTUAL TABLE knowledge_fts USING fts5(
|
|
2663
|
-
title,
|
|
2664
|
-
content,
|
|
2665
|
-
summary,
|
|
2666
|
-
content='knowledge_items',
|
|
2667
|
-
content_rowid='rowid'
|
|
2668
|
-
);
|
|
2669
|
-
|
|
2670
|
-
-- Indexes
|
|
2671
|
-
CREATE INDEX idx_items_type ON knowledge_items(type);
|
|
2672
|
-
CREATE INDEX idx_tags_tag ON tags(tag);
|
|
2673
|
-
CREATE INDEX idx_paths_pattern ON path_scopes(glob_pattern);
|
|
2674
|
-
```
|
|
2675
|
-
|
|
2676
|
-
### 10.3 Vector Index Specification (Stage 3)
|
|
2677
|
-
|
|
2678
|
-
Generated by `agentboot build --embeddings`. Adds semantic search via sqlite-vss.
|
|
2679
|
-
|
|
2680
|
-
```sql
|
|
2681
|
-
-- Vector embeddings for semantic search
|
|
2682
|
-
CREATE VIRTUAL TABLE knowledge_vectors USING vss0(
|
|
2683
|
-
embedding(1536) -- Dimension matches embedding model output
|
|
2684
|
-
);
|
|
2685
|
-
|
|
2686
|
-
-- Mapping between vectors and knowledge items
|
|
2687
|
-
CREATE TABLE vector_mappings (
|
|
2688
|
-
vector_rowid INTEGER PRIMARY KEY,
|
|
2689
|
-
item_id TEXT NOT NULL REFERENCES knowledge_items(id),
|
|
2690
|
-
chunk_index INTEGER NOT NULL DEFAULT 0, -- For items split into chunks
|
|
2691
|
-
chunk_text TEXT NOT NULL
|
|
2692
|
-
);
|
|
2693
|
-
```
|
|
2694
|
-
|
|
2695
|
-
**Embedding model:** The embedding model is configurable. Default: `voyage-3` (1536 dimensions). Alternatives: any model producing fixed-dimension vectors compatible with sqlite-vss.
|
|
2696
|
-
|
|
2697
|
-
**Chunking strategy:**
|
|
2698
|
-
1. Split content at heading boundaries (## or ###)
|
|
2699
|
-
2. If a chunk exceeds 512 tokens, split at paragraph boundaries
|
|
2700
|
-
3. Each chunk stores its own embedding
|
|
2701
|
-
4. Search returns the parent knowledge item, not individual chunks
|
|
2702
|
-
|
|
2703
|
-
### 10.4 How `agentboot build --index` Works
|
|
2704
|
-
|
|
2705
|
-
```
|
|
2706
|
-
function buildIndex(config):
|
|
2707
|
-
1. Collect all knowledge sources:
|
|
2708
|
-
- core/traits/*.md
|
|
2709
|
-
- .claude/rules/*.md (gotchas)
|
|
2710
|
-
- core/instructions/*.md
|
|
2711
|
-
- Domain layer files from extend.domains
|
|
2712
|
-
2. For each file:
|
|
2713
|
-
a. Parse frontmatter (title, paths, tags, description)
|
|
2714
|
-
b. Generate ID from type and filename: "{type}:{filename-without-ext}"
|
|
2715
|
-
c. Extract summary (first paragraph of body)
|
|
2716
|
-
3. Create SQLite database at .agentboot/knowledge.db
|
|
2717
|
-
4. Insert all items, tags, path_scopes
|
|
2718
|
-
5. Build FTS5 index
|
|
2719
|
-
6. Print: "Indexed {n} knowledge items ({gotchas} gotchas, {traits} traits, ...)"
|
|
2720
|
-
```
|
|
2721
|
-
|
|
2722
|
-
### 10.5 How `agentboot build --embeddings` Works
|
|
2723
|
-
|
|
2724
|
-
```
|
|
2725
|
-
function buildEmbeddings(config):
|
|
2726
|
-
1. Require --index to have been run (knowledge.db must exist)
|
|
2727
|
-
2. Load all items from knowledge_items table
|
|
2728
|
-
3. For each item:
|
|
2729
|
-
a. Chunk content (heading boundaries, max 512 tokens per chunk)
|
|
2730
|
-
b. Call embedding API for each chunk
|
|
2731
|
-
c. Store vector in knowledge_vectors table
|
|
2732
|
-
d. Store mapping in vector_mappings table
|
|
2733
|
-
4. Print: "Generated {n} embeddings for {m} knowledge items"
|
|
2734
|
-
5. Print estimated cost: "{tokens} tokens, ~${cost}"
|
|
2735
|
-
```
|
|
2736
|
-
|
|
2737
|
-
**Cost considerations:** Embedding generation requires API calls. For a typical org (50-500 knowledge items), the one-time cost is <$1. Incremental rebuilds only process changed items.
|
|
2738
|
-
|
|
2739
|
-
---
|
|
2740
|
-
|
|
2741
|
-
## 11. Open Questions
|
|
2742
|
-
|
|
2743
|
-
Open questions discovered during technical specification have been resolved or deferred.
|
|
2744
|
-
See the internal tracking document for remaining items.
|
|
2745
|
-
|
|
2746
|
-
Key resolved decisions applied to this spec:
|
|
2747
|
-
- CLI argument parser: commander
|
|
2748
|
-
- Token counting: estimation first (character-based ~1:4 ratio), move to tiktoken later
|
|
2749
|
-
- Test file convention: tests/behavioral/, tests/eval/, tests/snapshots/, tests/fixtures/
|
|
2750
|
-
- Telemetry log: target repo + user-level at ~/.agentboot/telemetry.ndjson (configurable)
|
|
2751
|
-
- Lint --fix: use Haiku to apply suggestions with human confirmation
|
|
2752
|
-
- Overlap analysis: normalized hash + Jaccard similarity for V1
|
|
2753
|
-
- MCP server: combined binary, both stdio + HTTP, separate later
|
|
2754
|
-
- Scope merging: rules=mandatory, personas=optional. No `required` field on personas — use rules
|
|
2755
|
-
- Plugin validation: implement AgentBoot's own (don't depend on claude plugin validate)
|