harness-everything 0.11.6 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +45 -0
- package/README.md +7 -3
- package/behavioral-evals/README.md +37 -0
- package/behavioral-evals/cases/self-evolve-retrieval-recurrence.yaml +48 -0
- package/behavioral-evals/paired-benchmark.js +211 -5
- package/ci/contract-integrity.test.js +68 -2
- package/ci/mechanism-2a-rule-of-3.test.js +71 -0
- package/ci/mechanism-2ah-workflow-lifecycle.test.js +276 -0
- package/ci/mechanism-2c-state-persist.test.js +45 -2
- package/ci/mechanism-2t-issue-20.test.js +0 -8
- package/ci/mechanism-30-opencode-plugin-loadability.test.js +39 -7
- package/ci/mechanism-32-paired-benchmark.test.js +139 -0
- package/ci/openai-plugin-package.test.js +17 -0
- package/ci/self-evolve-loop.test.js +37 -2
- package/ci/telemetry.test.js +31 -1
- package/docs/architecture.md +17 -0
- package/docs/audit.md +52 -5
- package/docs/release-evidence.md +3 -0
- package/docs/workflow-runtime.md +13 -0
- package/docs/workflows/environment-detection.md +47 -37
- package/docs/workflows/eval-harness.md +45 -38
- package/docs/workflows/fable-discipline.md +45 -33
- package/docs/workflows/git-commit.md +53 -45
- package/docs/workflows/grill-me.md +36 -38
- package/docs/workflows/grill-with-docs.md +36 -46
- package/docs/workflows/harness-everything.md +59 -57
- package/docs/workflows/improve-codebase-architecture.md +36 -44
- package/docs/workflows/install-cognitive-os.md +36 -29
- package/docs/workflows/repo-docs.md +29 -40
- package/docs/workflows/rewrite-commits.md +40 -36
- package/docs/workflows/security-review.md +34 -53
- package/docs/workflows/skill-creator.md +39 -56
- package/docs/workflows/skill-style.md +35 -40
- package/docs/workflows/using-git-worktrees.md +40 -44
- package/docs/workflows/verification-loop.md +43 -48
- package/docs/workflows/verify-before-claim.md +37 -40
- package/docs/workflows/zoom-out.md +39 -51
- package/grill-with-docs/ADR-FORMAT.md +22 -0
- package/grill-with-docs/SKILL.md +1 -1
- package/harness-everything/SKILL.md +1 -1
- package/harness-everything/scripts/kernel-router.js +38 -10
- package/hooks/scripts/lib/learning-opportunity.js +22 -0
- package/hooks/scripts/lib/telemetry.js +40 -0
- package/hooks/scripts/lib/workflow-isolation.js +30 -1
- package/hooks/scripts/lib/workflow-runtime.js +18 -1
- package/hooks/scripts/rule-of-3-tracker.js +17 -9
- package/hooks/scripts/state-persist.js +19 -7
- package/hooks/scripts/workflow-disposition.js +19 -8
- package/hooks/scripts/workflow-gate.js +21 -4
- package/multi-agent-workspace/SKILL.md +1 -1
- package/multi-agent-workspace/scripts/index_memory.js +28 -0
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/contract-integrity/scripts/audit.js +536 -0
- package/plugins/harness-everything/hooks/scripts/lib/learning-opportunity.js +22 -0
- package/plugins/harness-everything/hooks/scripts/lib/telemetry.js +40 -0
- package/plugins/harness-everything/hooks/scripts/lib/workflow-isolation.js +30 -1
- package/plugins/harness-everything/hooks/scripts/lib/workflow-runtime.js +18 -1
- package/plugins/harness-everything/hooks/scripts/rule-of-3-tracker.js +17 -9
- package/plugins/harness-everything/hooks/scripts/state-persist.js +19 -7
- package/plugins/harness-everything/hooks/scripts/workflow-disposition.js +19 -8
- package/plugins/harness-everything/hooks/scripts/workflow-gate.js +21 -4
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/skills/grill-with-docs/ADR-FORMAT.md +22 -0
- package/plugins/harness-everything/skills/grill-with-docs/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +38 -10
- package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +1 -1
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/index_memory.js +28 -0
- package/plugins/harness-everything/skills/self-evolve/SKILL.md +1 -1
- package/plugins/harness-everything/skills/self-evolve/scripts/lesson-candidate.js +52 -0
- package/plugins/harness-everything/skills/to-spec/SKILL.md +1 -1
- package/plugins/harness-everything/skills/to-spec/references/process.md +26 -0
- package/plugins/harness-everything/skills/to-spec/templates/cli-reference.md +14 -0
- package/plugins/harness-everything/skills/to-spec/templates/dev-doc.md +14 -0
- package/plugins/harness-everything/skills/to-spec/templates/feature-spec.md +14 -0
- package/plugins/harness-everything/skills/to-spec/templates/schema-doc.md +14 -0
- package/plugins/harness-everything/skills/to-tickets/SKILL.md +1 -1
- package/plugins/harness-everything/skills/to-tickets/references/publishing-and-templates.md +20 -1
- package/plugins/harness-everything/skills/verification-loop/SKILL.md +3 -2
- package/plugins/harness-everything/skills/verification-loop/references/verification-phases.md +25 -1
- package/plugins/harness-everything/skills/verification-loop/scripts/contract-integrity-audit.js +31 -0
- package/plugins/harness-everything/skills/verification-loop/templates/verification-report.template.md +3 -0
- package/scripts/sync-openai-plugin.js +12 -0
- package/self-evolve/SKILL.md +1 -1
- package/self-evolve/scripts/lesson-candidate.js +52 -0
- package/telemetry/schemas/event.schema.json +141 -22
- package/telemetry/scripts/report.js +42 -0
- package/to-spec/SKILL.md +1 -1
- package/to-spec/references/process.md +26 -0
- package/to-spec/templates/cli-reference.md +14 -0
- package/to-spec/templates/dev-doc.md +14 -0
- package/to-spec/templates/feature-spec.md +14 -0
- package/to-spec/templates/schema-doc.md +14 -0
- package/to-tickets/SKILL.md +1 -1
- package/to-tickets/references/publishing-and-templates.md +20 -1
- package/verification-loop/SKILL.md +3 -2
- package/verification-loop/references/verification-phases.md +25 -1
- package/verification-loop/scripts/contract-integrity-audit.js +31 -0
- package/verification-loop/templates/verification-report.template.md +3 -0
- package/docs/issue-20-rollup.md +0 -47
- package/docs/reviews/pr-142-issue-131.md +0 -89
|
@@ -1,64 +1,71 @@
|
|
|
1
1
|
# Workflow: Eval Harness
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> Score an agent execution log or conversation history on correctness, token efficiency, anti-loop focus, and environment awareness, and save the result as a scorecard when a benchmark is requested.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Source of truth: `eval-harness/SKILL.md`.
|
|
6
6
|
|
|
7
7
|
## 1. Skill Behavior Workflow
|
|
8
8
|
|
|
9
|
-
This section visualizes how the `eval-harness` skill executes internally, detailing the sequence of operations, state transitions, and evaluation steps.
|
|
10
|
-
|
|
11
9
|
```mermaid
|
|
12
10
|
graph TD
|
|
13
|
-
|
|
14
|
-
ParseLog -->
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
FileFallback -->|evals/ Exists| WriteEvals["Save Scorecard to evals/scorecard.md"]
|
|
21
|
-
FileFallback -->|No evals/ Folder| WritePlatform["Save Scorecard to .github/harness-everything/evals/scorecard.md"]
|
|
22
|
-
|
|
23
|
-
RunScript --> End([Evaluation Scorecard Logged])
|
|
24
|
-
WriteEvals --> End
|
|
25
|
-
WritePlatform --> End
|
|
11
|
+
BenchReq([Benchmark request with log or history]) --> ParseLog[Parse actions errors tokens timing]
|
|
12
|
+
ParseLog --> ScoreDims[Score four dimensions 0-10 via scoring-rubric]
|
|
13
|
+
ScoreDims --> RunEval[Run scripts evaluate.js with scores and insights]
|
|
14
|
+
RunEval --> EvalOk{Evaluate script succeeded?}
|
|
15
|
+
EvalOk -->|yes| SavedCard([Scorecard saved])
|
|
16
|
+
EvalOk -->|no or missing| MdFallback[Write evals scorecard.md fallback]
|
|
17
|
+
MdFallback --> SavedCard
|
|
26
18
|
```
|
|
27
19
|
|
|
28
|
-
|
|
20
|
+
```mermaid
|
|
21
|
+
graph TD
|
|
22
|
+
ScoreStart[Start scoring] --> Correctness[Score correctness 0-10]
|
|
23
|
+
ScoreStart --> Efficiency[Score efficiency 0-10]
|
|
24
|
+
ScoreStart --> AntiLoop[Score anti-loop focus 0-10]
|
|
25
|
+
ScoreStart --> EnvAware[Score environment awareness 0-10]
|
|
26
|
+
Correctness --> CombineScores[Combine into four-dimension card]
|
|
27
|
+
Efficiency --> CombineScores
|
|
28
|
+
AntiLoop --> CombineScores
|
|
29
|
+
EnvAware --> CombineScores
|
|
30
|
+
CombineScores --> EmitCard[Emit via script or markdown fallback]
|
|
31
|
+
```
|
|
29
32
|
|
|
30
33
|
## 2. Triggering and Routing Path
|
|
31
34
|
|
|
32
|
-
This diagram illustrates how the `eval-harness` skill is triggered through user requests or developer actions, and how it integrates or chains together with other companion skills in the Harness OS ecosystem to form unified workflows.
|
|
33
|
-
|
|
34
35
|
```mermaid
|
|
35
36
|
graph LR
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
37
|
+
UserBench[User asks to run benchmark] --> EvalSkill[eval-harness SKILL]
|
|
38
|
+
LogInput[Execution log or conversation history provided] --> EvalSkill
|
|
39
|
+
EvalSkill --> RubricStep[Apply references scoring-rubric.md]
|
|
40
|
+
RubricStep --> ScriptStep[Run scripts evaluate.js]
|
|
41
|
+
ScriptStep --> OutputLoc[Write to workspace evals or github fallback dir]
|
|
39
42
|
```
|
|
40
43
|
|
|
41
|
-
|
|
44
|
+
## 3. Real-World Use Case
|
|
42
45
|
|
|
43
|
-
|
|
46
|
+
A reviewer wants to compare two runs of a file-migration task.
|
|
44
47
|
|
|
45
|
-
|
|
48
|
+
1. Collect the execution log with tool calls, errors, tokens, and timing.
|
|
49
|
+
2. Score correctness, efficiency, anti-loop focus, and environment awareness 0–10 using `eval-harness/references/scoring-rubric.md`.
|
|
50
|
+
3. Run `node <skill-dir>/scripts/evaluate.js <A> <B> <C> <D> "insights"`.
|
|
51
|
+
4. On script failure or absence, write `<workspace>/evals/scorecard.md`, or `<workspace>/.github/harness-everything/evals/` if needed.
|
|
52
|
+
5. Leave the evaluated code unchanged; output is only the scorecard and insights.
|
|
46
53
|
|
|
47
54
|
```mermaid
|
|
48
55
|
graph TD
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
56
|
+
ReviewAsk[Reviewer asks to score migration run] --> GatherLog[Gather execution log]
|
|
57
|
+
GatherLog --> ApplyRubric[Score four dimensions with rubric]
|
|
58
|
+
ApplyRubric --> TryScript[Try evaluate.js]
|
|
59
|
+
TryScript --> ScriptOk{Script ok?}
|
|
60
|
+
ScriptOk -->|yes| DoneScore([Scorecard emitted])
|
|
61
|
+
ScriptOk -->|no| WriteMd[Write scorecard.md fallback]
|
|
62
|
+
WriteMd --> DoneScore
|
|
54
63
|
```
|
|
55
64
|
|
|
56
|
-
---
|
|
57
|
-
|
|
58
65
|
## 4. Verification Check
|
|
59
66
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
- [ ]
|
|
63
|
-
- [ ]
|
|
64
|
-
- [ ]
|
|
67
|
+
- [ ] Input included both a benchmark request and an execution log or conversation history; tasks without a log were declined
|
|
68
|
+
- [ ] All four dimensions were scored 0–10 using `eval-harness/references/scoring-rubric.md`: correctness, efficiency, anti-loop focus, environment awareness
|
|
69
|
+
- [ ] `eval-harness/scripts/evaluate.js` was run with four scores plus insights; markdown fallback was used only if the script failed or was missing
|
|
70
|
+
- [ ] Scorecard was written under `<workspace>/evals/` or `<workspace>/.github/harness-everything/evals/`
|
|
71
|
+
- [ ] No evaluated code was written or fixed, and no general review was substituted for dimension scoring
|
|
@@ -1,59 +1,71 @@
|
|
|
1
1
|
# Workflow: Fable Discipline
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> Act as a background guard while staged multi-agent work runs, keeping output compact, stage scope tight, handoffs explicit, and diverging builds halted.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Source of truth: `fable-discipline/SKILL.md`.
|
|
6
6
|
|
|
7
7
|
## 1. Skill Behavior Workflow
|
|
8
8
|
|
|
9
|
-
This section visualizes how the `fable-discipline` skill executes internally, detailing the sequence of operations, state transitions, and evaluation steps.
|
|
10
|
-
|
|
11
9
|
```mermaid
|
|
12
10
|
graph TD
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
11
|
+
FableActive([fable-mode active with staged work]) --> CompactMilestone[Compact state after milestone]
|
|
12
|
+
CompactMilestone --> ScopeCheck[Enforce stage scope and output limits]
|
|
13
|
+
ScopeCheck --> HandoffStep[Require state manifest at handoff]
|
|
14
|
+
HandoffStep --> BuildHealth{Build errors diverging?}
|
|
15
|
+
BuildHealth -->|yes| HaltZoom[HALT and call zoom-out]
|
|
16
|
+
BuildHealth -->|no| KeepGuard[Continue background guard]
|
|
17
|
+
KeepGuard --> CompactMilestone
|
|
18
|
+
HaltZoom --> MapErrors[Map error dependency graph for human]
|
|
21
19
|
```
|
|
22
20
|
|
|
23
|
-
|
|
21
|
+
```mermaid
|
|
22
|
+
graph TD
|
|
23
|
+
IncomingAgent[Incoming stage agent] --> VerifyManifest[Verify outgoing state manifest first]
|
|
24
|
+
VerifyManifest --> ManifestOk{Manifest complete?}
|
|
25
|
+
ManifestOk -->|yes| DoStage[Run stage within allowed files]
|
|
26
|
+
ManifestOk -->|no| AskManifest[Require missing APIs and IO details]
|
|
27
|
+
AskManifest --> VerifyManifest
|
|
28
|
+
DoStage --> OneCommit[One commit per independent logic block]
|
|
29
|
+
OneCommit --> LeaveManifest[Leave manifest for next agent]
|
|
30
|
+
```
|
|
24
31
|
|
|
25
32
|
## 2. Triggering and Routing Path
|
|
26
33
|
|
|
27
|
-
This diagram illustrates how the `fable-discipline` skill is triggered through user requests or developer actions, and how it integrates or chains together with other companion skills in the Harness OS ecosystem to form unified workflows.
|
|
28
|
-
|
|
29
34
|
```mermaid
|
|
30
35
|
graph LR
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
36
|
+
FableMode[fable-mode active] --> FableDisc[fable-discipline shadow guard]
|
|
37
|
+
FableDisc --> WorkspaceGuard[Constrain multi-agent-workspace mutations]
|
|
38
|
+
WorkspaceGuard --> FableMode
|
|
39
|
+
FableDisc --> DivergedBuild{Diverging build errors?}
|
|
40
|
+
DivergedBuild -->|yes| ZoomOut[zoom-out]
|
|
41
|
+
DivergedBuild -->|no| FableMode
|
|
34
42
|
```
|
|
35
43
|
|
|
36
|
-
|
|
44
|
+
## 3. Real-World Use Case
|
|
37
45
|
|
|
38
|
-
|
|
46
|
+
Long architectural migration runs under `fable-mode` with sub-agents handling separate stages.
|
|
39
47
|
|
|
40
|
-
|
|
48
|
+
1. After each milestone, compact decisions and drop unneeded history; avoid broad regex and avoid reading irrelevant files over 1000 lines.
|
|
49
|
+
2. Confirm working directory before touching core architecture; keep one commit per independent logic block.
|
|
50
|
+
3. Outgoing agent leaves a manifest with completed APIs and expected inputs/outputs; incoming agent verifies it first.
|
|
51
|
+
4. If fixes create more breakage, halt immediately, call `zoom-out`, and map the error dependency graph instead of continued patching.
|
|
41
52
|
|
|
42
53
|
```mermaid
|
|
43
54
|
graph TD
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
55
|
+
LongTask[Migration stage completes] --> CompactState[Compact milestone state]
|
|
56
|
+
CompactState --> CheckBloat[Check for bloat or oversized reads]
|
|
57
|
+
CheckBloat --> Handoff[Write and verify handoff manifest]
|
|
58
|
+
Handoff --> BuildCheck{Build diverging?}
|
|
59
|
+
BuildCheck -->|yes| HaltNow[HALT and zoom-out]
|
|
60
|
+
BuildCheck -->|no| NextStage[Start next stage guarded]
|
|
49
61
|
```
|
|
50
62
|
|
|
51
|
-
---
|
|
52
|
-
|
|
53
63
|
## 4. Verification Check
|
|
54
64
|
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
- [ ]
|
|
58
|
-
- [ ]
|
|
59
|
-
- [ ]
|
|
65
|
+
- [ ] Ran only as a background shadow for the whole `fable-mode` duration; not used standalone, not used for small tasks or routine single-file edits
|
|
66
|
+
- [ ] After each milestone, state was compacted and unneeded history dropped; no broad regex without precise conditions and no irrelevant 1000+ line reads
|
|
67
|
+
- [ ] Working directory was confirmed before core changes, with one commit per independent logic block
|
|
68
|
+
- [ ] Every sub-agent handoff via `multi-agent-workspace` left a state manifest with completed APIs and expected inputs/outputs, verified by the incoming agent first
|
|
69
|
+
- [ ] On diverging build errors, execution halted immediately and called `zoom-out` with an error dependency map instead of continued patching
|
|
70
|
+
- [ ] Made no direct state mutations of its own; only constrained `fable-mode` and `multi-agent-workspace` mutations
|
|
71
|
+
- [ ] Detail followed `fable-discipline/references/discipline-rules.md` where needed
|
|
@@ -1,71 +1,79 @@
|
|
|
1
1
|
# Workflow: Git Commit
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> Create clean Angular-style commit messages for an explicit commit request or a finished task phase, after checking repo state, submodules, and staged changes.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Source of truth: `git-commit/SKILL.md`.
|
|
6
6
|
|
|
7
7
|
## 1. Skill Behavior Workflow
|
|
8
8
|
|
|
9
|
-
This section visualizes how the `git-commit` skill executes internally, detailing the sequence of operations, state transitions, and evaluation steps.
|
|
10
|
-
|
|
11
9
|
```mermaid
|
|
12
10
|
graph TD
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
DirectCommit --> VerifyCommit["Run git log -1 to Verify"]
|
|
29
|
-
TempFileCommit --> VerifyCommit
|
|
30
|
-
VerifyCommit --> End([Staging Clean and Committed])
|
|
11
|
+
CommitTrig([Commit requested or phase concluded]) --> StatusCheck[Run git status]
|
|
12
|
+
StatusCheck --> RepoOk{Git repo present?}
|
|
13
|
+
RepoOk -->|no| InitOffer[Offer git init or skip]
|
|
14
|
+
RepoOk -->|yes| SubCheck[Check submodule changes per SUBMODULES guide]
|
|
15
|
+
SubCheck --> SubDirty{Submodules changed?}
|
|
16
|
+
SubDirty -->|yes| SubFirst[Commit submodules first]
|
|
17
|
+
SubDirty -->|no| DiffCheck[Run git diff-cached]
|
|
18
|
+
SubFirst --> DiffCheck
|
|
19
|
+
DiffCheck --> StagedOk{Anything staged?}
|
|
20
|
+
StagedOk -->|no| PromptStage[Prompt user and stage targeted files]
|
|
21
|
+
StagedOk -->|yes| FormatMsg[Format type-scope-subject message]
|
|
22
|
+
PromptStage --> FormatMsg
|
|
23
|
+
FormatMsg --> VerifyLog[Commit then verify with git log -1]
|
|
24
|
+
VerifyLog --> DoneClean([Tree updated and clean])
|
|
31
25
|
```
|
|
32
26
|
|
|
33
|
-
|
|
27
|
+
```mermaid
|
|
28
|
+
graph TD
|
|
29
|
+
FormatSimple[Formatted Angular message] --> EscCheck{Simple single line?}
|
|
30
|
+
EscCheck -->|yes| DirectM[git commit -m]
|
|
31
|
+
EscCheck -->|no multiline or Windows| TempF[Write commit-msg file then git commit -F and cleanup]
|
|
32
|
+
DirectM --> LogOne[git log -1 verifies commit]
|
|
33
|
+
TempF --> LogOne
|
|
34
|
+
LogOne --> CleanTree([Working tree clean])
|
|
35
|
+
```
|
|
34
36
|
|
|
35
37
|
## 2. Triggering and Routing Path
|
|
36
38
|
|
|
37
|
-
This diagram illustrates how the `git-commit` skill is triggered through user requests or developer actions, and how it integrates or chains together with other companion skills in the Harness OS ecosystem to form unified workflows.
|
|
38
|
-
|
|
39
39
|
```mermaid
|
|
40
40
|
graph LR
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
41
|
+
UserReq[User requests commit] --> GitSkill[git-commit SKILL]
|
|
42
|
+
PhaseDone[Task phase concludes with staged diff] --> GitSkill
|
|
43
|
+
GitSkill --> StatusGate[git status gate]
|
|
44
|
+
StatusGate --> SubGuide[guides SUBMODULES.md if needed]
|
|
45
|
+
SubGuide --> StyleGuide[guides ANGULAR_STYLE.md formatting]
|
|
46
|
+
StyleGuide --> CommitVerify[Commit and git log -1 check]
|
|
45
47
|
```
|
|
46
48
|
|
|
47
|
-
|
|
49
|
+
## 3. Real-World Use Case
|
|
48
50
|
|
|
49
|
-
|
|
51
|
+
Developer finishes an auth fix and asks for a commit.
|
|
50
52
|
|
|
51
|
-
|
|
53
|
+
1. Run `git status`; if not a repo, offer `git init` or skip.
|
|
54
|
+
2. If submodules changed, commit those first per `git-commit/guides/SUBMODULES.md`.
|
|
55
|
+
3. Run `git diff --cached`; if nothing staged, prompt before staging targeted files.
|
|
56
|
+
4. Format `<type>(<scope>): <subject>` per `git-commit/guides/ANGULAR_STYLE.md`.
|
|
57
|
+
5. Use `git commit -m` for a simple line, or `.git-commit-msg.txt` + `git commit -F` with cleanup for multiline/Windows, then verify with `git log -1`.
|
|
52
58
|
|
|
53
59
|
```mermaid
|
|
54
60
|
graph TD
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
+
DevDone[Auth fix finished and staged] --> CheckStatus[Run git status and cached diff]
|
|
62
|
+
CheckStatus --> FmtMsg[Format feat-auth style message]
|
|
63
|
+
FmtMsg --> PickMethod{Multiline or Windows?}
|
|
64
|
+
PickMethod -->|no| CommitM[git commit -m]
|
|
65
|
+
PickMethod -->|yes| CommitF[git commit -F with temp file]
|
|
66
|
+
CommitM --> VerifyOne[git log -1]
|
|
67
|
+
CommitF --> VerifyOne
|
|
68
|
+
VerifyOne --> CleanDone([Committed and verified])
|
|
61
69
|
```
|
|
62
70
|
|
|
63
|
-
---
|
|
64
|
-
|
|
65
71
|
## 4. Verification Check
|
|
66
72
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
- [ ]
|
|
70
|
-
- [ ]
|
|
71
|
-
- [ ]
|
|
73
|
+
- [ ] `git status` ran first; non-repo offered `git init` or skip, and empty staging prompted the user instead of committing nothing
|
|
74
|
+
- [ ] Submodule changes were handled first per `git-commit/guides/SUBMODULES.md`
|
|
75
|
+
- [ ] Staged diff was reviewed with `git diff --cached`, with targeted staging only after confirmation
|
|
76
|
+
- [ ] Message followed `<type>(<scope>): <subject>` per `git-commit/guides/ANGULAR_STYLE.md`
|
|
77
|
+
- [ ] Multiline/Windows path used `.git-commit-msg.txt` + `git commit -F` with cleanup; simple path used `git commit -m`; result verified with `git log -1`
|
|
78
|
+
- [ ] No branch, rebase, merge, or push was performed, and no staging happened without confirmation
|
|
79
|
+
- [ ] Detail followed `git-commit/references/commit-flow.md` where needed
|
|
@@ -1,62 +1,60 @@
|
|
|
1
1
|
# Workflow: Grill Me
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> A demanding challenger that pressure-tests vague plans and architectures one question at a time, exposes loopholes and undefined boundaries, counters sycophantic agreement, keeps the CONTEXT.md glossary current, and passes settled decisions to to-spec for formal specs and ADRs.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Source of truth: `grill-me/SKILL.md`.
|
|
6
6
|
|
|
7
7
|
## 1. Skill Behavior Workflow
|
|
8
8
|
|
|
9
|
-
This section visualizes how the `grill-me` skill executes internally, detailing the sequence of operations, state transitions, and evaluation steps.
|
|
10
|
-
|
|
11
9
|
```mermaid
|
|
12
10
|
graph TD
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
11
|
+
Propose["Vague plan or architecture proposal received"] --> Discover["Scan plan code plus CONTEXT.md, README.md, docs/adr/"]
|
|
12
|
+
Discover --> DomainTerms["Grill using project domain model and terminology"]
|
|
13
|
+
DomainTerms --> AskOne["Ask exactly ONE question with attached insight"]
|
|
14
|
+
AskOne --> ResolveBranch["Resolve decision-tree branch before moving on"]
|
|
15
|
+
ResolveBranch --> UpdateGlossary["Update CONTEXT.md glossary inline"]
|
|
16
|
+
UpdateGlossary --> ConsensusCheck{"Consensus on all branches?"}
|
|
17
|
+
ConsensusCheck -->|No| AskOne
|
|
18
|
+
ConsensusCheck -->|Yes| HandoffSpec["Hand off to to-spec to preview outline and publish"]
|
|
19
|
+
HandoffSpec --> RouteExec["Route execution to to-tickets, fable-mode, or tdd"]
|
|
21
20
|
```
|
|
22
21
|
|
|
23
|
-
---
|
|
24
|
-
|
|
25
22
|
## 2. Triggering and Routing Path
|
|
26
23
|
|
|
27
|
-
This diagram illustrates how the `grill-me` skill is triggered through user requests or developer actions, and how it integrates or chains together with other companion skills in the Harness OS ecosystem to form unified workflows.
|
|
28
|
-
|
|
29
24
|
```mermaid
|
|
30
25
|
graph LR
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
26
|
+
VaguePlan["Input: vague plan proposal"] --> GrillMe["grill-me / SKILL.md"]
|
|
27
|
+
EvaluateArch["Input: evaluate architecture request"] --> GrillMe
|
|
28
|
+
ExplicitAsk["Input: explicit grill me request"] --> GrillMe
|
|
29
|
+
GrillMe --> ToSpec["to-spec: preview outline and publish PRD, CLI/API reference, Schema doc, or ADR"]
|
|
30
|
+
ToSpec --> ToTickets["to-tickets for execution"]
|
|
31
|
+
ToSpec --> FableMode["fable-mode for execution"]
|
|
32
|
+
ToSpec --> TDD["tdd for execution"]
|
|
35
33
|
```
|
|
36
34
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
## 3. Real-World Use Case Flowchart
|
|
40
|
-
|
|
41
|
-
Here we model concrete real-world scenarios and use cases of the `grill-me` skill, illustrating standard success paths, error handling, or recovery loops.
|
|
35
|
+
## 3. Real-World Use Case
|
|
42
36
|
|
|
43
37
|
```mermaid
|
|
44
38
|
graph TD
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
39
|
+
CronProposal["Proposal: nightly cron syncs tables directly"] --> GrillStart["grill-me starts single-question loop"]
|
|
40
|
+
GrillStart --> Q1["Q1: What happens on mid-sync failure with locks held?"]
|
|
41
|
+
Q1 --> A1["Answer: wrap in transactions, define lock behavior"]
|
|
42
|
+
A1 --> Glossary1["Update CONTEXT.md glossary for sync and lock terms"]
|
|
43
|
+
Glossary1 --> Q2["Q2: What are boundary conditions for partial state?"]
|
|
44
|
+
Q2 --> Consensus["Consensus reached on failure and boundary handling"]
|
|
45
|
+
Consensus --> SpecHandoff["to-spec publishes ADR and spec"]
|
|
52
46
|
```
|
|
53
47
|
|
|
54
|
-
|
|
48
|
+
Concrete example: a developer proposes a direct table-sync cron job. `grill-me` scans `CONTEXT.md` and related code, asks one question about mid-sync failure, resolves it, updates the glossary, asks the next question about monitoring and boundary conditions, then hands the hardened decisions to `to-spec` for a PRD or ADR. Execution follows via `to-tickets`, `fable-mode`, or `tdd`.
|
|
55
49
|
|
|
56
|
-
|
|
50
|
+
Deep detail: `grill-me/references/grilling-playbook.md`.
|
|
57
51
|
|
|
58
|
-
|
|
52
|
+
## 4. Verification Check
|
|
59
53
|
|
|
60
|
-
- [ ]
|
|
61
|
-
- [ ]
|
|
62
|
-
- [ ]
|
|
54
|
+
- [ ] Exactly ONE question asked at a time; questionnaires prohibited
|
|
55
|
+
- [ ] Each question carries the agent's insight and its branch is resolved before advancing
|
|
56
|
+
- [ ] Questions use the project's domain model and terminology from `CONTEXT.md`, `README.md`, and `docs/adr/`
|
|
57
|
+
- [ ] `CONTEXT.md` glossary updated inline as terms resolve; document creation left to `to-spec`
|
|
58
|
+
- [ ] On consensus, `to-spec` invoked to preview outline and publish PRD, CLI/API reference, Schema doc, or ADR
|
|
59
|
+
- [ ] No code implementation or ticket breakdown inside this skill; execution routed to `to-tickets`, `fable-mode`, or `tdd`
|
|
60
|
+
- [ ] Not used for casual Q and A, direct spec writing, or ticket breakdown of an approved spec
|
|
@@ -1,69 +1,59 @@
|
|
|
1
|
-
# Workflow: Grill
|
|
1
|
+
# Workflow: Grill With Docs
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> Tests a plan against the recorded domain language and past decisions, tightens fuzzy terminology into canonical terms, and writes CONTEXT.md and ADR updates inline as choices settle.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Source of truth: `grill-with-docs/SKILL.md`.
|
|
6
6
|
|
|
7
7
|
## 1. Skill Behavior Workflow
|
|
8
8
|
|
|
9
|
-
This section visualizes how the `grill-with-docs` skill executes internally, detailing the sequence of operations, state transitions, and evaluation steps.
|
|
10
|
-
|
|
11
9
|
```mermaid
|
|
12
10
|
graph TD
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
ResolveADRPath -->|No Folder Found| WriteFallback["Write to committable docs/adr"]
|
|
23
|
-
|
|
24
|
-
WriteDocs --> ToSpec["Hand off to to-spec for Outline Preview & Spec/ADR Publishing"]
|
|
25
|
-
WriteFallback --> ToSpec
|
|
26
|
-
ToSpec --> End([Route to to-tickets / fable-mode / tdd])
|
|
11
|
+
PlanInput["Plan needing domain alignment"] --> ResolveStorage["Resolve storage with project-docs-resolver.js; CONTEXT-MAP.md root wins"]
|
|
12
|
+
ResolveStorage --> ChallengeTerms["Challenge glossary conflicts; sharpen fuzzy terms to canonical ones"]
|
|
13
|
+
ChallengeTerms --> ScenarioTest["Stress-test relationships with concrete scenarios; cross-check claims against code"]
|
|
14
|
+
ScenarioTest --> InlineUpdate["Update context glossary inline, never batched; zero implementation details"]
|
|
15
|
+
InlineUpdate --> ADRGate{"ADR warranted: hard to reverse plus surprising without context plus real trade-off?"}
|
|
16
|
+
ADRGate -->|Yes| WriteADR["Offer and write ADR at resolved location"]
|
|
17
|
+
ADRGate -->|No| SkipADR["Skip ADR; keep pure glossary"]
|
|
18
|
+
WriteADR --> Handoff["Hand off aligned design to to-spec"]
|
|
19
|
+
SkipADR --> Handoff
|
|
27
20
|
```
|
|
28
21
|
|
|
29
|
-
---
|
|
30
|
-
|
|
31
22
|
## 2. Triggering and Routing Path
|
|
32
23
|
|
|
33
|
-
This diagram illustrates how the `grill-with-docs` skill is triggered through user requests or developer actions, and how it integrates or chains together with other companion skills in the Harness OS ecosystem to form unified workflows.
|
|
34
|
-
|
|
35
24
|
```mermaid
|
|
36
25
|
graph LR
|
|
37
|
-
|
|
38
|
-
GrillMe["grill-me
|
|
39
|
-
GWD
|
|
40
|
-
|
|
41
|
-
|
|
26
|
+
DomainStress["Input: stress-testing a plan against domain language and documented decisions"] --> GWD["grill-with-docs / SKILL.md"]
|
|
27
|
+
GWD --> GrillMe["grill-me for unverified design"]
|
|
28
|
+
GWD --> ToSpec["to-spec for aligned design"]
|
|
29
|
+
ToSpec --> ToTickets["to-tickets for execution"]
|
|
30
|
+
ToSpec --> FableMode["fable-mode for execution"]
|
|
31
|
+
ToSpec --> TDD["tdd for execution"]
|
|
42
32
|
```
|
|
43
33
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
## 3. Real-World Use Case Flowchart
|
|
47
|
-
|
|
48
|
-
Here we model concrete real-world scenarios and use cases of the `grill-with-docs` skill, illustrating standard success paths, error handling, or recovery loops.
|
|
34
|
+
## 3. Real-World Use Case
|
|
49
35
|
|
|
50
36
|
```mermaid
|
|
51
37
|
graph TD
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
SyncCode --> Done([Zero architectural drift achieved])
|
|
38
|
+
RetryProposal["Proposal: change notification retry limits"] --> ResolveDocs["Resolve docs location; read CONTEXT.md and rate-limit ADR"]
|
|
39
|
+
ResolveDocs --> Conflict["Find glossary conflict: retry versus backoff"]
|
|
40
|
+
Conflict --> Scenario["Scenario test: burst retry against overload protection limit"]
|
|
41
|
+
Scenario --> GlossaryFix["Inline glossary fix to canonical terms"]
|
|
42
|
+
GlossaryFix --> ADRDecision["ADR offered because limit change is hard to reverse and involves trade-off"]
|
|
43
|
+
ADRDecision --> AlignedHandoff["Aligned design to to-spec; execution to to-tickets, fable-mode, or tdd"]
|
|
59
44
|
```
|
|
60
45
|
|
|
61
|
-
|
|
46
|
+
Concrete example: a retry-limit change is checked against the existing rate-limiting ADR and code. Fuzzy retry language is sharpened, the glossary is fixed inline, and only because the decision is hard to reverse and surprising without context is an ADR written. An unverified design would go to `grill-me` first instead.
|
|
62
47
|
|
|
63
|
-
|
|
48
|
+
Deep detail: `grill-with-docs/references/session-playbook.md`. Formats: `grill-with-docs/ADR-FORMAT.md`, `grill-with-docs/CONTEXT-FORMAT.md`.
|
|
64
49
|
|
|
65
|
-
|
|
50
|
+
## 4. Verification Check
|
|
66
51
|
|
|
67
|
-
- [ ]
|
|
68
|
-
- [ ]
|
|
69
|
-
- [ ]
|
|
52
|
+
- [ ] Storage resolved with `multi-agent-workspace/scripts/project-docs-resolver.js`; `<workspace>/CONTEXT-MAP.md` root wins
|
|
53
|
+
- [ ] Fuzzy terms sharpened to canonical ones; glossary stays pure with zero implementation details
|
|
54
|
+
- [ ] Relationships tested with concrete scenarios and claims cross-checked against code
|
|
55
|
+
- [ ] Context glossary updated inline, never batched
|
|
56
|
+
- [ ] ADR offered only when hard to reverse plus surprising without context plus a real trade-off
|
|
57
|
+
- [ ] ADRs meet the 3-part bar; handoff to `to-spec` after alignment
|
|
58
|
+
- [ ] Routing respected: unverified design to `grill-me`; aligned design to `to-spec`; execution to `to-tickets`, `fable-mode`, or `tdd`
|
|
59
|
+
- [ ] Not used for verifying unstable designs without a grilling pass, and not used for spec publishing without grilling
|