@evo-dev/core 0.0.1-alpha → 0.0.1-alpha.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/assets/agents/review/code-reviewer/examples.md +1 -1
- package/assets/agents/review/code-reviewer/prompt.md +1 -1
- package/assets/agents/review/code-reviewer/verification.md +1 -1
- package/assets/skills/coding/knowledge-distillation/SKILL.md +249 -0
- package/assets/skills/coding/knowledge-distillation/manifest.json +10 -0
- package/assets/skills/coding/knowledge-distillation/references/knowledge-distillation-methods.md +126 -0
- package/assets/workflows/rd-bug-fix/WORKFLOW.json +1 -1
- package/assets/workflows/rd-code-review/WORKFLOW.json +1 -1
- package/assets/workflows/rd-docs-update/WORKFLOW.json +1 -1
- package/assets/workflows/rd-feature-implementation/WORKFLOW.json +1 -1
- package/assets/workflows/rd-refactor/WORKFLOW.json +1 -1
- package/assets/workflows/rd-release-readiness/WORKFLOW.json +1 -1
- package/assets/workflows/rd-security-boundary-review/WORKFLOW.json +2 -2
- package/assets/workflows/rd-test-generation/WORKFLOW.json +1 -1
- package/dist/config/index.js +968 -39
- package/dist/index.js +10914 -1476
- package/dist/plugins/index.js +32 -32
- package/package.json +5 -1
- package/src/agents/index.ts +84 -49
- package/src/code-agent-traces/index.ts +521 -0
- package/src/config/index.ts +5 -0
- package/src/config/paths.ts +30 -0
- package/src/config/settings.ts +130 -0
- package/src/config/store.ts +152 -0
- package/src/daemon/index.ts +465 -3
- package/src/evolution/index.ts +2827 -0
- package/src/hooks/index.ts +543 -247
- package/src/index.ts +6 -0
- package/src/knowledge/index.ts +4784 -0
- package/src/pack/index.ts +13 -13
- package/src/plugins/capabilities.ts +40 -42
- package/src/plugins/index.ts +0 -1
- package/src/plugins/types.ts +4 -0
- package/src/protected-zones/index.ts +29 -11
- package/src/runtime-logs/index.ts +798 -0
- package/src/sync/orchestrator.ts +6 -0
- package/src/task/index.ts +3 -3
- package/src/team/index.ts +3069 -0
- package/src/team/mcp.ts +405 -0
- package/src/team/prompts.ts +141 -0
- package/src/workflow/index.ts +6 -6
|
@@ -11,7 +11,7 @@ This is useful because it includes severity, location, problem, and an actionabl
|
|
|
11
11
|
## Scope finding
|
|
12
12
|
|
|
13
13
|
```text
|
|
14
|
-
-
|
|
14
|
+
- CRITICAL: packages/plugin/hooks/plugin.ts This task added Claude sync behavior while the current task only allows canonical assets -> Remove plugin implementation from this task and handle it in the plugin sync task.
|
|
15
15
|
```
|
|
16
16
|
|
|
17
17
|
## Non-finding
|
|
@@ -33,7 +33,7 @@ If the review scope is unclear, ask for the missing context instead of guessing.
|
|
|
33
33
|
|
|
34
34
|
## Severity levels
|
|
35
35
|
|
|
36
|
-
- **
|
|
36
|
+
- **CRITICAL:** Must address before merge or delivery. Includes data loss, privacy boundary violations, project-asset writes outside scope, broken core behavior, or a failed required gate.
|
|
37
37
|
- **HIGH:** Likely user-visible bug, unsafe default, missing required validation, or mismatch with the current acceptance criteria.
|
|
38
38
|
- **MEDIUM:** Maintainability, incomplete edge handling, weak diagnostics, or test gap that should be addressed soon.
|
|
39
39
|
- **LOW:** Minor clarity, naming, or documentation issue. Do not report pure style preference unless it affects comprehension.
|
|
@@ -4,7 +4,7 @@ Before returning a review decision, confirm:
|
|
|
4
4
|
|
|
5
5
|
1. The reviewed scope matches the assigned task or diff.
|
|
6
6
|
2. Findings are grouped by severity and include a concrete fix.
|
|
7
|
-
3.
|
|
7
|
+
3. Critical findings cover safety, privacy, user-data, project-asset, and required-gate failures.
|
|
8
8
|
4. Style-only comments are omitted unless they affect correctness or maintainability.
|
|
9
9
|
5. Verification commands are reported with observed PASS, FAIL, or not-run status.
|
|
10
10
|
|
|
@@ -0,0 +1,249 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: knowledge-distillation
|
|
3
|
+
description: Use when turning reviewed execution evidence into atomic, privacy-filtered, role-tagged engineering knowledge candidates, evos cases, repo asset proposals, or role-agent/team suggestions.
|
|
4
|
+
license: MIT
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Knowledge Distillation / 提炼
|
|
8
|
+
|
|
9
|
+
## Purpose
|
|
10
|
+
|
|
11
|
+
Knowledge Distillation is an evidence curation pipeline. It turns reviewed execution evidence into an OKF-aware transient knowledge plan. The organizer, not this skill, writes final OKF Markdown files after dedupe, conflict handling, tagging, link updates, index generation, and logging.
|
|
12
|
+
|
|
13
|
+
It is not a trace summarizer, automatic memory writer, OKF file writer, or team launcher.
|
|
14
|
+
|
|
15
|
+
## Use This Skill When
|
|
16
|
+
|
|
17
|
+
- A completed task produced a repeatable repository-specific lesson.
|
|
18
|
+
- A review, failure, postmortem, or user correction should become a rule, warning, checklist item, decision, pattern, or evos case.
|
|
19
|
+
- Execution evidence suggests a repo-local skill, rule, role agent, subagent, or EvoHub team.
|
|
20
|
+
- Reviewed execution evidence shows a task split, tool call, skill output, or subagent handoff was incomplete, inaccurate, inefficient, or unsafe.
|
|
21
|
+
- Role-specific knowledge should later be available to a role agent, subject to explicit consent and runtime scope.
|
|
22
|
+
|
|
23
|
+
## Do Not Use This Skill For
|
|
24
|
+
|
|
25
|
+
- Writing `CLAUDE.md`, `AGENTS.md`, `.claude/`, `.codex/`, `.evodev/`, source files, or project assets without explicit project opt-in.
|
|
26
|
+
- Storing raw prompts, transcripts, source dumps, raw command output, secrets, tokens, internal links, or private URLs.
|
|
27
|
+
- Treating unreviewed observations, model reflection, or trace logs as accepted memory.
|
|
28
|
+
- Writing final OKF concept documents directly; output a transient plan for the organizer.
|
|
29
|
+
- Starting subagents or teams automatically.
|
|
30
|
+
|
|
31
|
+
## Inputs
|
|
32
|
+
|
|
33
|
+
Work from one bounded evidence window. Prefer reviewed, minimized inputs:
|
|
34
|
+
|
|
35
|
+
- Task goal, final outcome, and repository scope.
|
|
36
|
+
- Verification evidence, failures, fixes, and review findings.
|
|
37
|
+
- Accepted corrections or explicit user feedback.
|
|
38
|
+
- Relevant external authority, when the lesson depends on a standard, official documentation, or upstream behavior.
|
|
39
|
+
- Relevant OKF v0.1 constraints: concept documents need YAML frontmatter with non-empty `type`; `index.md` and `log.md` are reserved at every directory level; bundle-relative absolute Markdown links are preferred for durable relationships.
|
|
40
|
+
- Existing OKF index snippets or concept summaries, when available, so duplicate and conflict risk can be scored before proposing new candidates.
|
|
41
|
+
- Candidate roles that should care, such as `architect`, `implementer`, `reviewer`, `tester`, `security`, `release`, `docs`, or user-defined role slugs.
|
|
42
|
+
|
|
43
|
+
Do not ingest raw traces wholesale. If trace evidence is needed, use a redacted summary and evidence references.
|
|
44
|
+
|
|
45
|
+
## Evidence Policy
|
|
46
|
+
|
|
47
|
+
Classify every input before extraction:
|
|
48
|
+
|
|
49
|
+
| Evidence class | Use | Constraint |
|
|
50
|
+
|---|---|---|
|
|
51
|
+
| `verified-run` | Test/build/lint/check outcomes and accepted fixes | Store references and summaries, not raw output |
|
|
52
|
+
| `reviewed-finding` | Code review, security review, or user correction | Keep finding, impact, and accepted action separate |
|
|
53
|
+
| `decision-record` | Architecture or workflow decision | Preserve context, decision, alternatives, consequences |
|
|
54
|
+
| `postmortem-case` | Failure, incident, or after-action review | Preserve expected vs actual, cause, action item |
|
|
55
|
+
| `official-reference` | Standards, docs, upstream behavior | Include URL/version/date; do not overquote |
|
|
56
|
+
| `model-reflection` | Agent-generated synthesis | Candidate only; never accepted without review |
|
|
57
|
+
| `tool-call-summary` | Tool selection, risk class, status, and recovery | No raw command or raw stdout/stderr by default |
|
|
58
|
+
| `skill-invocation` | Skill id/version, input class, output schema, and reviewed result | Record omissions or inaccuracies as findings; do not store raw prompts/source |
|
|
59
|
+
| `subagent-lifecycle` | Role, reason, scope, status, output refs, and merge result | No raw transcript |
|
|
60
|
+
| `evo-eval-result` | Eval case pass/fail and assertion summary | Store fixture refs and assertion results, not private raw data |
|
|
61
|
+
|
|
62
|
+
## Distillation Pipeline
|
|
63
|
+
|
|
64
|
+
1. **Scope**: identify repo, task, role audience, workflow audience, path scope, evidence ids, and user-local OKF target scope.
|
|
65
|
+
2. **Minimize**: remove raw prompts, raw logs, source dumps, secrets, personal data, internal links, and one-off noise.
|
|
66
|
+
3. **Analyze execution structure**: when event evidence is present, identify task slices, owner roles, dependencies, key tool calls, skill invocations, subagent lifecycle events, verification gates, and merge outcomes.
|
|
67
|
+
4. **Extract atomic candidates**: one claim per candidate. Allowed `kind` values are `rule`, `decision`, `pattern`, `anti-pattern`, `warning`, `checklist`, `concept`, `workflow-improvement`, `task-split-improvement`, `tool-use-improvement`, `skill-improvement`, `repo-asset-suggestion`, `role-agent-suggestion`, `team-suggestion`, `eval-set`, and `open-question`.
|
|
68
|
+
5. **Separate fact from inference**: mark whether the candidate is directly evidenced or inferred from evidence.
|
|
69
|
+
6. **Classify**: add `okfType`, `targetPath`, `stableKey`, `roleTags`, `repoTags`, `workflowTags`, `pathScopes`, `domainTags`, `stability`, `sensitivity`, and `targetStore`.
|
|
70
|
+
7. **Score**: estimate evidence strength, reuse value, actionability, stability, novelty, privacy risk, and duplication risk.
|
|
71
|
+
8. **Pair improvements with evals**: every proposed skill, role-agent, team, workflow, routing, tool-use, or subagent behavior change should include an `evoEvalSets` entry, unless the plan explains why eval coverage is not applicable.
|
|
72
|
+
9. **Privacy gate**: defer candidates that require raw private content to remain meaningful.
|
|
73
|
+
10. **Route**: emit `no_write`, `create`, `update`, `skip`, or `needs-human` for each candidate.
|
|
74
|
+
11. **Plan OKF organization**: provide canonical concept targets and repo/role/workflow overlay updates. Do not emit final OKF files.
|
|
75
|
+
12. **Recovery notes**: explain conflicts, stale information, required repo fact checks, and why human intervention is needed when applicable.
|
|
76
|
+
|
|
77
|
+
## Output Schema
|
|
78
|
+
|
|
79
|
+
Return executable JSON with `schemaVersion: 1` and `kind: "knowledge-distillation-output"`. The runtime parser converts this output into an OKF plan and validates the full contract before any OKF directory or concept write occurs. Invalid outputs become validation failed-plan artifacts; those artifacts are redacted, non-resumable, and inspectable with `evo plan show`.
|
|
80
|
+
|
|
81
|
+
```json
|
|
82
|
+
{
|
|
83
|
+
"schemaVersion": 1,
|
|
84
|
+
"kind": "knowledge-distillation-output",
|
|
85
|
+
"projectKey": "evodev",
|
|
86
|
+
"runId": "run-123",
|
|
87
|
+
"createdAt": "2026-06-24T00:00:00.000Z",
|
|
88
|
+
"evidenceWindowId": "evidence-run-123",
|
|
89
|
+
"summary": "What reusable improvement was found.",
|
|
90
|
+
"evidenceRefs": [
|
|
91
|
+
{
|
|
92
|
+
"id": "source-1",
|
|
93
|
+
"kind": "verification",
|
|
94
|
+
"source": "state/evidence/metadata.json",
|
|
95
|
+
"rawContentStored": false,
|
|
96
|
+
"externalContentCopied": false
|
|
97
|
+
}
|
|
98
|
+
],
|
|
99
|
+
"evoEvalSets": [
|
|
100
|
+
{
|
|
101
|
+
"id": "eval-role-routing-1",
|
|
102
|
+
"target": { "kind": "role-agent-suggestion", "id": "candidate-1" },
|
|
103
|
+
"purpose": "Guard a behavior-changing active write.",
|
|
104
|
+
"roleTags": ["reviewer"],
|
|
105
|
+
"cases": [
|
|
106
|
+
{
|
|
107
|
+
"id": "case-1",
|
|
108
|
+
"inputRefs": ["source-1"],
|
|
109
|
+
"assertions": ["Candidate remains metadata-only and review-state gated."],
|
|
110
|
+
"expectedReviewState": "auto-accepted"
|
|
111
|
+
}
|
|
112
|
+
],
|
|
113
|
+
"privacy": {
|
|
114
|
+
"usesRawPrompt": false,
|
|
115
|
+
"usesSourceDump": false,
|
|
116
|
+
"usesRawCommandOutput": false
|
|
117
|
+
},
|
|
118
|
+
"decision": "create"
|
|
119
|
+
}
|
|
120
|
+
],
|
|
121
|
+
"knowledgeCandidates": [
|
|
122
|
+
{
|
|
123
|
+
"id": "candidate-1",
|
|
124
|
+
"decision": "auto-accept",
|
|
125
|
+
"kind": "rule",
|
|
126
|
+
"okfType": "EvoDev Rule",
|
|
127
|
+
"targetStore": "okf",
|
|
128
|
+
"targetPath": "concepts/rules/workspace-check.md",
|
|
129
|
+
"stableKey": "rule:verification:workspace-check",
|
|
130
|
+
"confidence": "high",
|
|
131
|
+
"title": "Workspace check before completion",
|
|
132
|
+
"description": "Run the workspace check before reporting TypeScript CLI completion.",
|
|
133
|
+
"claim": "For TypeScript CLI changes, run the workspace check before reporting completion.",
|
|
134
|
+
"basis": "direct",
|
|
135
|
+
"metadataOnlyEvidence": true,
|
|
136
|
+
"howToApply": "Add bun run check to the verification plan.",
|
|
137
|
+
"antiCriteria": ["Do not mark completion from lint alone."],
|
|
138
|
+
"roleTags": ["implementer", "reviewer"],
|
|
139
|
+
"repoTags": ["evodev"],
|
|
140
|
+
"workflowTags": ["feature-implementation"],
|
|
141
|
+
"pathScopes": ["packages/cli/", "packages/core/"],
|
|
142
|
+
"relatedConceptLinks": ["/concepts/verification/workspace-quality-gate.md"],
|
|
143
|
+
"overlayUpdates": [
|
|
144
|
+
{
|
|
145
|
+
"targetPath": "roles/reviewer/verification.md",
|
|
146
|
+
"operation": "append-link",
|
|
147
|
+
"link": "/concepts/rules/workspace-check.md"
|
|
148
|
+
}
|
|
149
|
+
],
|
|
150
|
+
"scores": {
|
|
151
|
+
"evidenceStrength": 5,
|
|
152
|
+
"reuseValue": 4,
|
|
153
|
+
"actionability": 4,
|
|
154
|
+
"stability": 4,
|
|
155
|
+
"privacyRisk": 1,
|
|
156
|
+
"duplicationRisk": 1
|
|
157
|
+
},
|
|
158
|
+
"decisionReason": "Auto-accepted from verified metadata-only evidence.",
|
|
159
|
+
"evidenceRefs": ["source-1"],
|
|
160
|
+
"reviewState": "auto-accepted",
|
|
161
|
+
"evalSetRefs": [],
|
|
162
|
+
"bodySections": {
|
|
163
|
+
"summary": "Run the workspace check before reporting completion.",
|
|
164
|
+
"appliesWhen": ["TypeScript CLI or core changes were made."],
|
|
165
|
+
"guidance": ["Run bun run check and report the result."],
|
|
166
|
+
"antiCriteria": ["Do not store raw command output."],
|
|
167
|
+
"verification": ["Workspace check passed with metadata-only evidence."],
|
|
168
|
+
"citations": []
|
|
169
|
+
},
|
|
170
|
+
"privacyCheck": {
|
|
171
|
+
"rawPromptsStored": false,
|
|
172
|
+
"rawLogsStored": false,
|
|
173
|
+
"sourceDumpsStored": false,
|
|
174
|
+
"rawCommandOutputStored": false,
|
|
175
|
+
"secretsStored": false,
|
|
176
|
+
"internalLinksStored": false
|
|
177
|
+
}
|
|
178
|
+
}
|
|
179
|
+
],
|
|
180
|
+
"droppedSignals": [
|
|
181
|
+
{
|
|
182
|
+
"evidenceRef": "event-17",
|
|
183
|
+
"reason": "Routine tool-call metadata with no reusable lesson."
|
|
184
|
+
}
|
|
185
|
+
],
|
|
186
|
+
"conflicts": [],
|
|
187
|
+
"privacyCheck": {
|
|
188
|
+
"rawPromptsStored": false,
|
|
189
|
+
"rawLogsStored": false,
|
|
190
|
+
"sourceDumpsStored": false,
|
|
191
|
+
"rawCommandOutputStored": false,
|
|
192
|
+
"secretsStored": false,
|
|
193
|
+
"internalLinksStored": false
|
|
194
|
+
}
|
|
195
|
+
}
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Required candidate fields are `id`, `decision`, `kind`, `okfType`, `targetStore`, `targetPath`, `stableKey`, `confidence`, `title`, `description`, `claim`, `basis`, `metadataOnlyEvidence`, `howToApply`, `antiCriteria`, `roleTags`, `repoTags`, `workflowTags`, `pathScopes`, `relatedConceptLinks`, `overlayUpdates`, `scores`, `decisionReason`, `evidenceRefs`, `reviewState`, `bodySections`, and `privacyCheck`.
|
|
199
|
+
|
|
200
|
+
Valid decisions are `auto-accept`, `create`, `update`, `needs-human`, `skip`, and canonical `no_write`. `no-write` may be normalized by the runtime but new output should emit `no_write`. Active writes (`auto-accept`, `create`, `update`) must target `okf`, use reviewState `auto-accepted` or `accepted`, use metadata-only evidence, include evidence refs, have a safe relative `.md` target path outside reserved `index.md` and `log.md`, and include verification or `verificationNotApplicableReason`.
|
|
201
|
+
|
|
202
|
+
Behavior-changing active writes, including skill, role-agent, team, workflow, routing, tool-use, task-split, and subagent changes, must include `evalSetRefs` that point to provided `evoEvalSets`. Eval sets must be metadata-only and use privacy flags `usesRawPrompt: false`, `usesSourceDump: false`, and `usesRawCommandOutput: false`.
|
|
203
|
+
|
|
204
|
+
Scoring uses 1-5 integers for `evidenceStrength`, `reuseValue`, `actionability`, `stability`, `privacyRisk`, and `duplicationRisk`. A candidate should not target `okf` unless evidence strength, reuse value, and actionability justify future retrieval and privacy risk is low. `no_write` is a normal result and should be common.
|
|
205
|
+
|
|
206
|
+
## Write Targets
|
|
207
|
+
|
|
208
|
+
Default output is a transient plan only.
|
|
209
|
+
|
|
210
|
+
- `okf`: stable facts, decisions, constraints, concepts, evos cases, and reusable rules that the organizer may write into user-local OKF.
|
|
211
|
+
- `evo-eval-set`: regression cases tied to proposed or accepted behavior changes.
|
|
212
|
+
- `repo-asset-proposal`: suggested repo rules, skills, role agents, subagents, or teams. Requires explicit project opt-in before any repository write.
|
|
213
|
+
- `none`: useful observation that should stay in the report and not become durable knowledge.
|
|
214
|
+
|
|
215
|
+
Project-local writes remain out of scope for this skill.
|
|
216
|
+
|
|
217
|
+
## Role Tags
|
|
218
|
+
|
|
219
|
+
Every durable candidate must include at least one role tag. Prefer stable role slugs:
|
|
220
|
+
|
|
221
|
+
- `architect`
|
|
222
|
+
- `implementer`
|
|
223
|
+
- `reviewer`
|
|
224
|
+
- `tester`
|
|
225
|
+
- `security`
|
|
226
|
+
- `release`
|
|
227
|
+
- `docs`
|
|
228
|
+
|
|
229
|
+
Use user-defined role slugs only when a reviewed role-agent definition exists. Treat role tags as retrieval/routing filters, not decorative labels.
|
|
230
|
+
|
|
231
|
+
## Human Intervention Triggers
|
|
232
|
+
|
|
233
|
+
Mark a candidate as `needs-human` when it:
|
|
234
|
+
|
|
235
|
+
- Affects security, privacy, release, architecture, or cross-repo behavior.
|
|
236
|
+
- Suggests a project-local file write.
|
|
237
|
+
- Suggests changing a skill, role agent, team, workflow, routing rule, tool-use policy, or subagent behavior without an associated `evoEvalSets` entry or explicit not-applicable reason.
|
|
238
|
+
- Has low evidence strength, high privacy risk, or ambiguous repo scope.
|
|
239
|
+
- Would change role-agent behavior, EvoHub team composition, or runtime retrieval behavior.
|
|
240
|
+
- Conflicts with existing OKF guidance in a way the organizer cannot resolve from current repo facts.
|
|
241
|
+
- Is derived primarily from model reflection rather than verified evidence.
|
|
242
|
+
|
|
243
|
+
## Runtime Attention Boundary
|
|
244
|
+
|
|
245
|
+
Only active OKF concepts under `~/.evodev/knowledge/okf` may influence retrieval. Runtime loading remains gated by role selection, repo scope, workflow scope, path scope, privacy policy, and consent. Transient plans, failed plans, repo proposals, and `no_write` observations must not influence routing or runtime behavior.
|
|
246
|
+
|
|
247
|
+
## References
|
|
248
|
+
|
|
249
|
+
For method comparisons and design rationale, read `references/knowledge-distillation-methods.md` in this skill directory. For OKF structure, reserved filenames, concept frontmatter, links, indexes, logs, and conformance rules, follow the Open Knowledge Format v0.1 spec: https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
{
|
|
2
|
+
"id": "knowledge-distillation",
|
|
3
|
+
"name": "Knowledge Distillation / 提炼",
|
|
4
|
+
"version": "0.1.0",
|
|
5
|
+
"category": "coding",
|
|
6
|
+
"description": "Distill repository-relevant rules, evos cases, skill/role/team improvement proposals, and eval sets from reviewed execution evidence without writing project assets by default.",
|
|
7
|
+
"targets": ["claude", "codex"],
|
|
8
|
+
"entry": "SKILL.md",
|
|
9
|
+
"license": "MIT"
|
|
10
|
+
}
|
package/assets/skills/coding/knowledge-distillation/references/knowledge-distillation-methods.md
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
# Knowledge Distillation Method Notes
|
|
2
|
+
|
|
3
|
+
This reference explains why the skill uses an evidence curation pipeline instead of a generic summarization workflow.
|
|
4
|
+
|
|
5
|
+
## Method Comparison
|
|
6
|
+
|
|
7
|
+
| Method | What it contributes | EvoDev adaptation |
|
|
8
|
+
|---|---|---|
|
|
9
|
+
| Postmortem / SRE review | Blameless failure analysis, root cause, impact, action items | Use for failure-derived `warning`, `anti-pattern`, `checklist`, and `workflow-improvement` candidates |
|
|
10
|
+
| After Action Review | Expected vs actual, what went well, what failed, what changes next time | Use for evos cases and team/process improvement candidates |
|
|
11
|
+
| Lessons learned systems | Reviewed lessons from projects/programs, not raw incident notes | Keep active OKF knowledge separate from raw evidence and transient plans |
|
|
12
|
+
| ADR / decision logs | One durable decision with context, alternatives, rationale, consequences | Use `decision` candidates for architecture/workflow choices |
|
|
13
|
+
| SKOS-style knowledge organization | Concepts, labels, narrower/broader/related terms, scope notes | Keep role tags and domain tags controlled enough for retrieval |
|
|
14
|
+
| Agent memory research | Semantic facts, episodic cases, procedural improvements | Split knowledge, evos cases, and repo asset suggestions instead of one memory bucket |
|
|
15
|
+
| Agent skills | Small triggerable procedures with references/assets for larger material | Keep `SKILL.md` operational and place long examples/taxonomies in references |
|
|
16
|
+
| Evaluation-driven development | Behavior changes need regression cases and objective assertions | Pair skill, role, team, workflow, routing, tool-use, and subagent proposals with evo eval sets |
|
|
17
|
+
|
|
18
|
+
## Evidence Quality
|
|
19
|
+
|
|
20
|
+
Prefer evidence in this order:
|
|
21
|
+
|
|
22
|
+
1. Verified local outcomes: tests, typecheck, lint, build, release dry-run, smoke checks, or accepted fixes.
|
|
23
|
+
2. Reviewed findings: code review, security review, user correction, or postmortem conclusion.
|
|
24
|
+
3. Official or upstream references: standards, vendor docs, framework docs, release notes, or source repositories.
|
|
25
|
+
4. Existing repository contracts: `AGENTS.md`, design docs, task contracts, tests, schemas, and explicit policy files.
|
|
26
|
+
5. Model reflection: useful for candidate generation only; never enough for active OKF knowledge.
|
|
27
|
+
|
|
28
|
+
## Candidate Types
|
|
29
|
+
|
|
30
|
+
- `rule`: normative guidance with a clear trigger and expected behavior.
|
|
31
|
+
- `decision`: accepted choice plus context, alternatives, and consequences.
|
|
32
|
+
- `pattern`: repeatable implementation or workflow shape.
|
|
33
|
+
- `anti-pattern`: repeatable bad shape to avoid.
|
|
34
|
+
- `warning`: risk that should influence planning or review.
|
|
35
|
+
- `checklist`: short verification list for a role or workflow.
|
|
36
|
+
- `concept`: term, domain note, taxonomy entry, or glossary item.
|
|
37
|
+
- `workflow-improvement`: process change with trigger and expected future behavior.
|
|
38
|
+
- `task-split-improvement`: improved decomposition, dependencies, owner roles, evidence, or stop conditions.
|
|
39
|
+
- `tool-use-improvement`: improved tool selection, risk classification, command gating, or verification behavior.
|
|
40
|
+
- `skill-improvement`: proposed change to a skill trigger, input requirements, output schema, references, or behavior.
|
|
41
|
+
- `repo-asset-suggestion`: proposed project rule, skill, agent, subagent, or config file.
|
|
42
|
+
- `role-agent-suggestion`: proposed user/project role definition or role capability.
|
|
43
|
+
- `team-suggestion`: proposed EvoHub team composition and trigger.
|
|
44
|
+
- `eval-set`: proposed regression cases for a skill, role, team, workflow, routing, tool-use, or subagent behavior.
|
|
45
|
+
- `open-question`: unresolved issue that should not become accepted memory yet.
|
|
46
|
+
|
|
47
|
+
## Scoring Rubric
|
|
48
|
+
|
|
49
|
+
Use 1-5 integers. Keep the score approximate; it is a review aid, not a scientific metric.
|
|
50
|
+
|
|
51
|
+
| Score | High value means | Low value means |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| `evidenceStrength` | Directly supported by verified/reviewed evidence | Mostly inferred or speculative |
|
|
54
|
+
| `reuseValue` | Likely useful across future similar tasks | One-off or too narrow |
|
|
55
|
+
| `actionability` | Clear trigger and next action | Vague or only descriptive |
|
|
56
|
+
| `stability` | Likely to remain true for the repo/workflow | Depends on volatile context |
|
|
57
|
+
| `privacyRisk` | Higher is worse: needs sensitive context | Redacted summary is enough |
|
|
58
|
+
| `duplicationRisk` | Higher is worse: likely duplicate | Distinct from existing entries |
|
|
59
|
+
|
|
60
|
+
Suggested write queue rule:
|
|
61
|
+
|
|
62
|
+
- Propose active OKF knowledge only when `evidenceStrength >= 4`, `reuseValue >= 3`, `actionability >= 3`, `privacyRisk <= 2`, and `duplicationRisk <= 3`.
|
|
63
|
+
- Propose evos cases when the item is mainly about a trigger/intervention/outcome history.
|
|
64
|
+
- Propose repo assets only when repeated future execution would benefit from a rule, skill, role, or team and explicit project opt-in can be obtained.
|
|
65
|
+
- Propose evo eval sets whenever a skill, role-agent, team, workflow, routing, tool-use, or subagent behavior change is proposed. If no eval is useful, state why in the proposal.
|
|
66
|
+
|
|
67
|
+
## Executable Contract Notes
|
|
68
|
+
|
|
69
|
+
The runtime accepts only JSON outputs with `kind: "knowledge-distillation-output"` and `schemaVersion: 1`. Candidate decisions are `auto-accept`, `create`, `update`, `needs-human`, `skip`, and canonical `no_write`. Active writes must be metadata-only, evidence-backed, review-state gated, verified, and targeted at safe relative `.md` OKF paths. Behavior-changing active writes must include `evalSetRefs` that point to provided `evoEvalSets`.
|
|
70
|
+
|
|
71
|
+
Contract validation runs before OKF writes. Validation failures are saved as redacted, non-resumable failed-plan artifacts; organizer failures preserve a resumable plan. Failed artifacts are organizer inputs only and must not influence runtime retrieval.
|
|
72
|
+
|
|
73
|
+
## Evolution Event Analysis
|
|
74
|
+
|
|
75
|
+
Use reviewed event evidence to explain why the future behavior should change.
|
|
76
|
+
|
|
77
|
+
| Signal | Good analysis asks |
|
|
78
|
+
|---|---|
|
|
79
|
+
| Task slices | Was the work decomposed into verifiable units with owner roles, dependencies, and stop conditions? |
|
|
80
|
+
| Tool calls | Was the selected tool appropriate, gated by risk, and followed by verification or recovery? |
|
|
81
|
+
| Skill invocations | Did the skill trigger correctly, receive the needed evidence, and produce a complete schema with privacy checks? |
|
|
82
|
+
| Subagent lifecycle | Was the role necessary, bounded, mergeable, and verified without storing transcript content? |
|
|
83
|
+
| Verification | Did required checks run at the right time and produce evidence for the claim? |
|
|
84
|
+
|
|
85
|
+
Common skill defects:
|
|
86
|
+
|
|
87
|
+
- `trigger-mismatch`: the skill was invoked when it should not have been, or missed when it should have run.
|
|
88
|
+
- `input-gap`: required evidence class was missing.
|
|
89
|
+
- `output-omission`: a required proposal, field, path, role tag, privacy check, or eval set was absent.
|
|
90
|
+
- `inaccurate-claim`: output contradicted verified evidence.
|
|
91
|
+
- `schema-drift`: output did not match the declared contract.
|
|
92
|
+
- `privacy-defect`: output depended on raw private content.
|
|
93
|
+
|
|
94
|
+
## Evo Eval Set Types
|
|
95
|
+
|
|
96
|
+
| Type | Checks |
|
|
97
|
+
|---|---|
|
|
98
|
+
| `schema` | Required fields, enum values, ids, and status are present. |
|
|
99
|
+
| `omission-detection` | Missing recommendations, evidence refs, privacy checks, or eval sets are caught. |
|
|
100
|
+
| `privacy` | Raw prompt/source/command output/secrets are not stored. |
|
|
101
|
+
| `routing` | A skill, workflow, role, or team should or should not trigger. |
|
|
102
|
+
| `role-attention` | Role-tagged knowledge is selected only when scope, consent, and role policy match. |
|
|
103
|
+
| `tool-use` | Tool choice, risk classification, and verification behavior are correct. |
|
|
104
|
+
| `subagent-routing` | Role selection, scope, output contract, and merge requirements are correct. |
|
|
105
|
+
|
|
106
|
+
## Storage Mapping
|
|
107
|
+
|
|
108
|
+
| Candidate target | Canonical layer | Notes |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| `okf` | Active knowledge | OKF Markdown concepts with frontmatter, provenance, links, directory indexes, and logs |
|
|
111
|
+
| `evo-eval-set` | Evolution eval sets | Regression cases tied to proposed or accepted behavior changes |
|
|
112
|
+
| `repo-asset-proposal` | Proposal queue first | Never write project files by default |
|
|
113
|
+
| `none` | Report only | Useful but not durable |
|
|
114
|
+
|
|
115
|
+
Indexes, vector search, graph exports, and runtime caches are derived views. They must be rebuildable from active OKF concepts and must not store raw private content. Transient knowledge plans are organizer inputs, not accepted knowledge.
|
|
116
|
+
|
|
117
|
+
## Source Notes
|
|
118
|
+
|
|
119
|
+
- Google SRE postmortem practice: https://sre.google/sre-book/postmortem-culture/
|
|
120
|
+
- US Army AAR guidance: https://www.first.army.mil/Portals/102/FM%207-0%20Appendix%20K.pdf
|
|
121
|
+
- NASA Lessons Learned: https://www.nasa.gov/nasa-lessons-learned/
|
|
122
|
+
- ADR references and templates: https://adr.github.io/
|
|
123
|
+
- W3C SKOS Reference: https://www.w3.org/TR/skos-reference/
|
|
124
|
+
- W3C PROV-O provenance ontology: https://www.w3.org/TR/prov-o/
|
|
125
|
+
- OpenAI Agents SDK sessions/memory reference: https://openai.github.io/openai-agents-python/sessions/
|
|
126
|
+
- Claude Code skills documentation: https://docs.anthropic.com/en/docs/claude-code/skills
|
|
@@ -32,7 +32,7 @@
|
|
|
32
32
|
],
|
|
33
33
|
"requiredEvidence": ["scope-plan", "changed-files-summary", "verification-report"],
|
|
34
34
|
"verification": {
|
|
35
|
-
"policy": "
|
|
35
|
+
"policy": "advisory",
|
|
36
36
|
"antiCriteria": ["no-unapproved-project-write", "no-secret-capture"]
|
|
37
37
|
},
|
|
38
38
|
"privacy": {
|
|
@@ -27,12 +27,12 @@
|
|
|
27
27
|
"actor": "human",
|
|
28
28
|
"required": true,
|
|
29
29
|
"evidence": ["security-decision"],
|
|
30
|
-
"summary": "Approve
|
|
30
|
+
"summary": "Approve or request follow-up with rationale."
|
|
31
31
|
}
|
|
32
32
|
],
|
|
33
33
|
"requiredEvidence": ["data-flow-summary", "write-boundary-check", "security-decision"],
|
|
34
34
|
"verification": {
|
|
35
|
-
"policy": "
|
|
35
|
+
"policy": "advisory",
|
|
36
36
|
"antiCriteria": ["no-secret-capture", "no-unapproved-write", "no-external-upload"]
|
|
37
37
|
},
|
|
38
38
|
"privacy": {
|