agentme 0.30.3 → 0.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.filedist-package.yml +3 -0
- package/.xdrs/agentme/edrs/application/skills/200-run-skill-tests/SKILL.md +8 -4
- package/.xdrs/agentme/edrs/application/skills/200-run-skill-tests/SKILL.test.md +20 -0
- package/.xdrs/agentme/edrs/index.md +1 -0
- package/.xdrs/agentme/edrs/principles/017-skill-testing.md +6 -0
- package/.xdrs/agentme/edrs/principles/skills/150-plan-mode-consistency/SKILL.md +184 -0
- package/.xdrs/agentme/edrs/principles/skills/150-plan-mode-consistency/SKILL.test.md +88 -0
- package/package.json +2 -2
package/.filedist-package.yml
CHANGED
|
@@ -27,7 +27,7 @@ Loads `SKILL.test.md` from a skill directory, runs each scenario by invoking the
|
|
|
27
27
|
```
|
|
28
28
|
|
|
29
29
|
Then halt — do not proceed.
|
|
30
|
-
4. Read `SKILL.test.md` in full. Parse the frontmatter (`skill`, `skill-version`) and all `### Scenario N:` sections. Each scenario has three sub-sections
|
|
30
|
+
4. Read `SKILL.test.md` in full. Parse the frontmatter (`skill`, `skill-version`) and all `### Scenario N:` sections. Each scenario has three required sub-sections — `**Trigger / Input**`, `**Expected Behaviour**`, and `**Assertions**` — and one optional sub-section: `**Simulated Human Responses**`.
|
|
31
31
|
5. Read `SKILL.md` in full to understand what the skill does. Verify that the `skill-version` in `SKILL.test.md` matches the `version` field in `SKILL.md` metadata. If they differ, output a WARNING and continue.
|
|
32
32
|
|
|
33
33
|
### Phase 2: Execute Scenarios
|
|
@@ -35,9 +35,13 @@ Loads `SKILL.test.md` from a skill directory, runs each scenario by invoking the
|
|
|
35
35
|
For each scenario in order:
|
|
36
36
|
|
|
37
37
|
1. Record the scenario title and number.
|
|
38
|
-
2.
|
|
39
|
-
3.
|
|
40
|
-
4.
|
|
38
|
+
2. Check whether the scenario has a `**Simulated Human Responses**` sub-section. If present, parse it as an ordered list of canned responses to be injected sequentially each time the skill pauses for human input during this scenario.
|
|
39
|
+
3. Present the trigger/input to a fresh agent context with the target skill loaded, exactly as written in `**Trigger / Input**`. Do not add context beyond what the trigger specifies.
|
|
40
|
+
4. Whenever the skill pauses and asks a question or requests human input, inject the next unused response from the `**Simulated Human Responses**` list (in order). Continue until the skill finishes or the response list is exhausted.
|
|
41
|
+
- If the response list is exhausted before the skill finishes, record the remaining output as-is and note "Simulated responses exhausted" in the scenario result.
|
|
42
|
+
- If the scenario has no `**Simulated Human Responses**` section, run the skill until its first natural pause point and capture the output at that point.
|
|
43
|
+
5. Capture the full output from the skill execution.
|
|
44
|
+
6. Move to Phase 3 for this scenario before executing the next.
|
|
41
45
|
|
|
42
46
|
### Phase 3: Evaluate Assertions
|
|
43
47
|
|
|
@@ -45,3 +45,23 @@ You are an agent with the `200-run-skill-tests` skill loaded. The workspace has
|
|
|
45
45
|
- [ ] Output references the resolved path of the skill directory.
|
|
46
46
|
- [ ] Output does NOT contain any `### Scenario` execution section.
|
|
47
47
|
- [ ] Output does NOT contain a `## Summary` report block.
|
|
48
|
+
|
|
49
|
+
### Scenario 3: Injects simulated human responses when present
|
|
50
|
+
|
|
51
|
+
**Trigger / Input**
|
|
52
|
+
You are an agent with the `200-run-skill-tests` skill loaded. The workspace has a skill at `.xdrs/agentme/edrs/principles/skills/150-plan-mode-consistency` with both `SKILL.md` and `SKILL.test.md` present. Scenario 1 in that `SKILL.test.md` includes a `**Simulated Human Responses**` section with 9 ordered responses. The user says:
|
|
53
|
+
|
|
54
|
+
"Test the skill at `.xdrs/agentme/edrs/principles/skills/150-plan-mode-consistency`"
|
|
55
|
+
|
|
56
|
+
**Expected Behaviour**
|
|
57
|
+
1. Skill reads `SKILL.test.md` and detects the `**Simulated Human Responses**` section in Scenario 1.
|
|
58
|
+
2. When executing Scenario 1, the skill injects each listed response in order each time the target skill pauses for human input.
|
|
59
|
+
3. Execution continues through all HITL pause points using the injected responses rather than waiting for a real human.
|
|
60
|
+
4. If the response list is exhausted before the skill finishes, the runner notes "Simulated responses exhausted" and captures remaining output as-is.
|
|
61
|
+
5. Assertions are evaluated against the full captured output after all injected responses are consumed.
|
|
62
|
+
|
|
63
|
+
**Assertions**
|
|
64
|
+
- [ ] Runner parses the `**Simulated Human Responses**` section and does not ask the user for input during Scenario 1 execution.
|
|
65
|
+
- [ ] Output contains the Scenario 1 result section with individual assertion PASS/FAIL results.
|
|
66
|
+
- [ ] Output does NOT contain any prompt asking the real user to respond on behalf of the simulated skill interaction.
|
|
67
|
+
- [ ] If responses are exhausted, output notes "Simulated responses exhausted" for that scenario.
|
|
@@ -8,6 +8,7 @@ Propose changes via pull request. All changes must be verified for clarity and n
|
|
|
8
8
|
|
|
9
9
|
Foundational standards, principles, and guidelines.
|
|
10
10
|
|
|
11
|
+
- [150-plan-mode-consistency](principles/skills/150-plan-mode-consistency/SKILL.md) - **Plan mode consistency** — MANDATORY skill for ANY planning activity (plan, design, propose, outline, draft, brainstorm, architect). Read and follow in full before any execution begins. Must be read from XDRS even when not in `.agents/skills`. *(skill)*
|
|
11
12
|
- [agentme-edr-012](principles/012-continuous-xdr-enrichment.md) - **Continuous xdr improvement policy** - Promote recurring delivery lessons into reusable XDRs
|
|
12
13
|
- [agentme-edr-016](principles/016-cross-language-module-structure.md) - **Cross-language module structure** - Organize modules consistently across supported languages
|
|
13
14
|
- [agentme-edr-017](principles/017-skill-testing.md) - **skill testing** - Mandates a `SKILL.test.md` co-located with every skill in scopes that follow agentme; defines test scenario format (trigger, expected behaviour, assertions) and requires execution before merging any skill change *(includes skill: [200-run-skill-tests](application/skills/200-run-skill-tests/SKILL.md))*
|
|
@@ -45,6 +45,11 @@ specific enough that two independent agents produce comparable outputs.]
|
|
|
45
45
|
**Expected Behaviour**
|
|
46
46
|
[Numbered list of steps the skill must perform, derived from its Instructions section.]
|
|
47
47
|
|
|
48
|
+
**Simulated Human Responses** *(optional — include when the skill has human-in-the-loop pauses)*
|
|
49
|
+
1. [Exact text to inject as the human's answer to the first pause point.]
|
|
50
|
+
2. [Answer to the second pause point.]
|
|
51
|
+
...
|
|
52
|
+
|
|
48
53
|
**Assertions**
|
|
49
54
|
- [ ] [Specific, falsifiable check on the output or behaviour. Start with a verb.]
|
|
50
55
|
- [ ] ...
|
|
@@ -57,6 +62,7 @@ Rules:
|
|
|
57
62
|
- Assertion text MUST start with a verb ("Output contains …", "Skill asks …", "Review reports …").
|
|
58
63
|
- MUST NOT duplicate SKILL.md content; reference phases by name only when needed.
|
|
59
64
|
- `skill-version` in frontmatter MUST be updated whenever `version` in SKILL.md changes.
|
|
65
|
+
- **Simulated Human Responses** is optional. Include it when the skill has human-in-the-loop pause points and automated testing is needed. Responses are injected in order at each pause; if responses are exhausted before the skill finishes, the runner captures the remaining output as-is.
|
|
60
66
|
|
|
61
67
|
#### 03-execution-requirement
|
|
62
68
|
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: 150-plan-mode-consistency
|
|
3
|
+
description: >
|
|
4
|
+
MANDATORY for ANY planning activity. Read and follow this skill in full whenever the user asks
|
|
5
|
+
to plan, design, propose, outline, draft, brainstorm, architect, or think through anything —
|
|
6
|
+
software features, systems, research, documents, processes, or any other task — before execution
|
|
7
|
+
begins. This skill defines the required pre-execution structured workflow. It MUST be read from
|
|
8
|
+
the XDRS repository even when not directly exposed in the .agents skills folder. Activate in
|
|
9
|
+
full mode for non-trivial tasks and in abbreviated mode (Phases 3–5 collapsed, Phase 4 marked
|
|
10
|
+
not applicable) for trivial single-step tasks.
|
|
11
|
+
metadata:
|
|
12
|
+
author: flaviostutz
|
|
13
|
+
version: "1.0"
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Overview
|
|
17
|
+
|
|
18
|
+
Ensures that every plan is deeply validated through iterative consistency checks, visual externalization, and multi-angle challenges before execution starts. The skill is not designed to make human work easier — its purpose is to identify precisely where human experience, feeling, and domain knowledge are irreplaceable, and to demand that input before moving forward.
|
|
19
|
+
|
|
20
|
+
**No-assumption rule**: The agent performs all analysis autonomously. Surface findings as questions only when they are dubious, non-obvious, risky, or require subjective judgment — obvious and deterministic findings must be stated and the agent must continue without pausing. For findings with major impact on downstream users or consumers (breaking changes, behavior regressions, removals), do not ask — emit a prominently formatted **SEVERE WARNING** with a clear description of the impact and continue. Do not defer questions to the end of a check, round, or phase — raise them as soon as they arise. The human is the oracle for domain knowledge, intent, and subjective decisions; the agent is the oracle for everything deterministic.
|
|
21
|
+
|
|
22
|
+
**Task tracking rule**: Use the todo list tool throughout this entire skill. Before starting each phase, create a todo for it and mark it in-progress. Mark it completed immediately when done. For Phase 3 (consistency checks), create a todo for each check (a–g) at the start of each round and mark them individually. For Phase 5 (challenge angles), create a todo for each of the 11 angles before beginning Phase 5 and mark each completed after the human responds to any question raised, or immediately if no question was raised for that angle. An angle todo MUST NOT be marked complete if a subjective decision was resolved without asking the human — if this is detected, flag it as a HITL violation, re-open the todo, surface the decision to the human as a clarifying question, and only mark it complete after the human responds. This ensures no check, round, or angle is silently skipped and no subjective decision is self-resolved.
|
|
23
|
+
|
|
24
|
+
**Round budget rule**: The entire skill runs on a global budget of at most 30 rounds. A round is one human interaction in which the agent asks 1–5 related questions together. Rounds are drawn from this shared budget across all phases in order. Stop early when two consecutive rounds return no new questions worth asking — do not exhaust the budget for its own sake.
|
|
25
|
+
|
|
26
|
+
## Instructions
|
|
27
|
+
|
|
28
|
+
### Phase 1: Activate Plan Mode
|
|
29
|
+
|
|
30
|
+
1. Switch to plan mode. Do not write, edit, or execute anything until the plan is fully validated through Phase 6.
|
|
31
|
+
2. State the goal in one sentence: what problem is being solved and what the expected outcome is.
|
|
32
|
+
3. State the scope boundaries explicitly: what is included and what is explicitly excluded.
|
|
33
|
+
4. Ask the human: "Does this goal statement and scope match your intent? Is anything missing or wrong?" Wait for the answer and incorporate it before continuing.
|
|
34
|
+
|
|
35
|
+
### Phase 2: Research, Dependencies, and Draft Plan
|
|
36
|
+
|
|
37
|
+
1. Research the existing context: relevant files, prior decisions, established conventions, and analogous patterns already in place.
|
|
38
|
+
2. For each contextual input, constraint, or dependency found (existing files, prior decisions, external systems, in-progress work by others), surface non-obvious, risky, or subjective items to the human and ask only when needed. Batch 1–5 related questions into a single round when multiple items need human input. Proceed without asking for obvious or deterministic context items.
|
|
39
|
+
3. Draft a plan with ordered steps, items to create or modify, and a verification step at the end.
|
|
40
|
+
4. Present the draft and ask: "Does this match your intent? What am I missing?" Wait for the answer before continuing.
|
|
41
|
+
|
|
42
|
+
### Phase 3: Iterative Consistency Checks
|
|
43
|
+
|
|
44
|
+
**Trivial tasks**: Run exactly one abbreviated round covering only checks (a), (e), and (f). Skip the remaining checks and mark them as not applicable. Proceed directly to Phase 4 after the single round.
|
|
45
|
+
|
|
46
|
+
**Non-trivial tasks**: Run as many rounds as needed from the global budget. Each round asks 1–5 related questions spanning one or more checks (a–g). For each check, ask only for findings that are dubious, relevant, subjective, or risky — state obvious findings and continue. Stop when two consecutive rounds surface no new questions.
|
|
47
|
+
|
|
48
|
+
Each round runs the following checks in order:
|
|
49
|
+
|
|
50
|
+
- **(a) Internal consistency**: Are there contradictions between steps? Do the scope boundaries align with the implementation steps?
|
|
51
|
+
- **(b) Dry run**: Walk through the plan with the most complex realistic scenario. Where does it break or leave gaps?
|
|
52
|
+
- **(c) Component consistency**: Do all elements of the plan work together as a coherent whole? Are there missing connections between parts?
|
|
53
|
+
- **(d) XDR alignment**: Does this plan align with the relevant XDRs governing this area? Have the right policies been consulted?
|
|
54
|
+
- **(e) Feasibility**: Is each step actually achievable given the current context, constraints, and available resources?
|
|
55
|
+
- **(f) Completeness**: Is anything missing that would leave the task half-done or the outcome broken for its consumer?
|
|
56
|
+
- **(g) Scope creep check**: Has the plan grown beyond the original request? Flag any additions and ask the human to confirm or reject each one explicitly before continuing.
|
|
57
|
+
|
|
58
|
+
**Convergence signals** (non-trivial tasks only): Stop running rounds when the last 2 consecutive rounds produce only single-sentence answers with no new issues surfaced. Do not stop on a round count alone — stop when the checks genuinely have nothing left to surface.
|
|
59
|
+
|
|
60
|
+
### Phase 4: Visual Consistency Validation
|
|
61
|
+
|
|
62
|
+
**Trivial tasks**: Mark this phase as **not applicable** and state this explicitly before moving to Phase 5. Do not skip silently.
|
|
63
|
+
|
|
64
|
+
**Non-trivial tasks**:
|
|
65
|
+
1. Choose the diagram type that best externalizes this plan's structure:
|
|
66
|
+
- **Flowchart** — step-by-step decision flows and process branches
|
|
67
|
+
- **Concept map** — ideas, relationships, and conceptual structure
|
|
68
|
+
- **Dependency graph** — components and their dependencies
|
|
69
|
+
- **Sequence diagram** — call flows, API interactions, and temporal order
|
|
70
|
+
- **State diagram** — lifecycle states and transitions
|
|
71
|
+
- **Activity diagram** — business workflows with parallel paths
|
|
72
|
+
- **Entity diagram** — data models and relationships
|
|
73
|
+
2. Generate the diagram.
|
|
74
|
+
3. Ask the human: "Does this diagram match your mental model of the solution?" Wait for the answer.
|
|
75
|
+
4. If the diagram reveals gaps or inconsistencies not yet surfaced, return to Phase 3 before continuing.
|
|
76
|
+
|
|
77
|
+
### Phase 5: Challenge from 11 Distinct Angles
|
|
78
|
+
|
|
79
|
+
Each angle is an analysis step. Run the angle and present findings. Batch questions from related angles into a single round of 1–5 questions when findings are related — batching questions is permitted, skipping analysis is not. For obvious or factual findings, state them and proceed immediately. For findings with major impact on users, emit a **SEVERE WARNING** and continue without asking. Ask clarifying questions whenever findings are ambiguous, subjective, risky, or very relevant — do not resolve those points unilaterally.
|
|
80
|
+
|
|
81
|
+
#### Plan quality angles
|
|
82
|
+
|
|
83
|
+
**1. Prompt faithfulness**
|
|
84
|
+
Go back to the original request word by word. Is every part of the request covered? Is anything included in the plan that was not asked for? Identify gaps and additions explicitly.
|
|
85
|
+
|
|
86
|
+
**2. Local context consistency**
|
|
87
|
+
Does the plan account for existing files, decisions, and constraints already in place? Does it contradict anything already established in the codebase, repository, or context?
|
|
88
|
+
|
|
89
|
+
**3. Goal achievability**
|
|
90
|
+
Walk the end state step by step: if every step in the plan is executed exactly as written, does the desired outcome actually result? State the end state explicitly. Ask the human to confirm only if there is genuine doubt about whether the outcome matches their expectation.
|
|
91
|
+
|
|
92
|
+
**4. Ambiguity scan**
|
|
93
|
+
Is any step or decision in the plan interpretable in more than one way? Every ambiguity is a future mistake. List all ambiguous points and ask the human to resolve each one.
|
|
94
|
+
|
|
95
|
+
**5. Pre-mortem**
|
|
96
|
+
Assume the plan is executed and fails to reach the goal. What was the most likely reason? Identify the plan's most fragile assumption or weakest step.
|
|
97
|
+
|
|
98
|
+
**6. Security and privacy scan**
|
|
99
|
+
Does the plan or its output expose sensitive information, create privacy risks, or introduce misuse vectors? This applies to any task type: documentation, code, processes, data handling, communications. If findings are present and require a subjective decision to resolve, ask the human. For clear and obvious mitigations, state them and continue.
|
|
100
|
+
|
|
101
|
+
**7. Success criteria and falsifiability**
|
|
102
|
+
How will we know this plan succeeded or failed? Are the success criteria concrete enough to be measurable and observable? If they are vague, the outcome cannot be evaluated. If the criteria are already clear and measurable, state the assessment and continue. Ask the human only when criteria are vague or require their input to sharpen.
|
|
103
|
+
|
|
104
|
+
**8. Second-order effects**
|
|
105
|
+
What changes as a side effect of executing this plan beyond the intended outcome? Does solving this problem create a new problem elsewhere — in adjacent systems, files, processes, or stakeholders? List the side effects. Ask the human whether they are acceptable only when the effects are non-obvious or the decision is subjective.
|
|
106
|
+
|
|
107
|
+
**9. Steelman the opposition**
|
|
108
|
+
What is the strongest case against this approach? What would a well-informed critic say about this plan? Present the strongest objection. Ask the human to respond if the objection raises a genuine risk or requires a subjective decision — otherwise state the counter-argument and continue.
|
|
109
|
+
|
|
110
|
+
#### Output quality angles
|
|
111
|
+
|
|
112
|
+
**10. Output scenario dry runs**
|
|
113
|
+
Simulate 5 realistic usage scenarios of the expected output by its actual consumer. For each scenario, ask: "Does the output serve its consumer in this situation?" Use scenarios that cover typical use, edge cases, and at least one adversarial or failure case.
|
|
114
|
+
|
|
115
|
+
Examples of scenario framing:
|
|
116
|
+
- If the output is operator documentation: "A worker needs to reset the machine at 2 AM — will they find the procedure in under 2 minutes?"
|
|
117
|
+
- If the output is an API: "A developer calling this endpoint with a malformed payload — what happens?"
|
|
118
|
+
- If the output is a business process: "An employee following this process on their first day — will they complete it without asking for help?"
|
|
119
|
+
|
|
120
|
+
Whenever a scenario reveals ambiguity or requires a subjective judgment, stop and ask the human a clarifying question. Do not resolve subjective decisions unilaterally.
|
|
121
|
+
|
|
122
|
+
**11. Output internal consistency**
|
|
123
|
+
Check that the planned output is internally consistent: no contradictions between parts, no gaps between sections, all elements serve the same goal. Run approximately 3 rounds until answers converge to single sentences with no new issues surfaced.
|
|
124
|
+
|
|
125
|
+
### Phase 6: Pre-Execution Readiness
|
|
126
|
+
|
|
127
|
+
Before approving execution, verify ALL items in the checklist below. If any item cannot be checked, return to the relevant phase and resolve it first. For trivial tasks, mark non-applicable items explicitly as **N/A** rather than leaving them unchecked or omitting them.
|
|
128
|
+
|
|
129
|
+
- [ ] Consistency rounds converged (convergence signals met — last 2 rounds produced only single-sentence answers with no new issues) *(trivial tasks: single abbreviated round completed)*
|
|
130
|
+
- [ ] All 11 challenge angles completed with human input received for every ambiguity and subjective decision *(trivial tasks: single abbreviated round — mark remaining angles N/A)*
|
|
131
|
+
- [ ] Diagram generated and confirmed by the human *(trivial tasks: N/A — mark explicitly)*
|
|
132
|
+
- [ ] No unresolved human questions outstanding
|
|
133
|
+
- [ ] Scope confirmed by the human with no silent expansions
|
|
134
|
+
- [ ] Any irreversible or high-impact steps have a mitigation or fallback noted *(trivial tasks: N/A if no irreversible steps)*
|
|
135
|
+
|
|
136
|
+
Only proceed to execution when every item is checked or explicitly marked N/A. Do not start execution to escape planning discomfort — only start when confidence is genuine and all items are verified.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## Anti-Patterns
|
|
141
|
+
|
|
142
|
+
Avoid these common failure modes:
|
|
143
|
+
|
|
144
|
+
- **Planning theater**: running rounds without real critical thinking. The quality of questioning matters more than the count of rounds. Rounds that confirm the plan against itself add false confidence — checks must challenge assumptions, not validate them.
|
|
145
|
+
- **Scope creep silence**: the plan grows beyond the original request without the human noticing. Every addition must be flagged explicitly.
|
|
146
|
+
- **Agent self-validation**: the agent answers its own questions on subjective, domain, or intent-based decisions and proceeds without asking the human. The human is the oracle for domain knowledge, intent, and subjective decisions — the agent must not self-resolve those unilaterally.
|
|
147
|
+
- **Confidence as a proxy for correctness**: an agent expressing certainty does not mean the plan is correct. Run all checks regardless of how confident the agent sounds.
|
|
148
|
+
|
|
149
|
+
## Re-Plan Triggers
|
|
150
|
+
|
|
151
|
+
Stop execution and return to Phase 1 if any of the following occur:
|
|
152
|
+
|
|
153
|
+
- A discovered assumption underlying the plan is wrong.
|
|
154
|
+
- The scope has expanded by more than approximately 20% beyond the original request.
|
|
155
|
+
- Two consecutive execution steps fail unexpectedly and the root cause points to a planning gap.
|
|
156
|
+
|
|
157
|
+
## Examples
|
|
158
|
+
|
|
159
|
+
**Input**: "Add a rate-limiting feature to the API."
|
|
160
|
+
|
|
161
|
+
- Phase 1: Goal stated as "rate-limit all POST endpoints to 100 req/min per user; internal service calls excluded." Human confirms.
|
|
162
|
+
- Phase 2: Discovers existing middleware and an in-progress PR touching the same path. Human asked about each before drafting.
|
|
163
|
+
- Phase 3: Round 1 — check (a) finds the plan references a `RateLimiter` class not yet decided on; human asked to clarify. Round 5 — all checks return trivial answers; convergence reached.
|
|
164
|
+
- Phase 4: Sequence diagram generated. Human confirms it matches their model.
|
|
165
|
+
- Phase 5: Angle 8 (second-order effects) reveals that rate-limiting breaks an existing test suite that sends rapid sequential requests; human decides to add a test bypass header. Angle 9 (steelman) surfaces that Redis dependency adds operational complexity; human accepts the trade-off.
|
|
166
|
+
- Phase 6: All items checked. Execution approved.
|
|
167
|
+
|
|
168
|
+
**Input**: "Write operator documentation for the conveyor belt system."
|
|
169
|
+
|
|
170
|
+
- Phase 5, angle 10 (output dry runs): Scenario 1 — "An operator needs to restart the belt after an emergency stop at midnight." The draft plan has no emergency stop section; human asked whether to add it. Scenario 3 — "Operator reading on a mobile phone." Human asked whether a condensed quick-reference card is needed alongside the full manual.
|
|
171
|
+
|
|
172
|
+
## Edge Cases
|
|
173
|
+
|
|
174
|
+
- **Trivial changes** (typo fixes, single-line formatting): Phases 3–5 may be shortened to a single abbreviated round. Phase 4 (diagram) may be skipped, but MUST be explicitly noted as not applicable in the task tracking todo with a brief reason (e.g., "Phase 4: N/A — trivial single-step change"). Phase 6 checklist still applies — mark non-applicable items explicitly.
|
|
175
|
+
- **Agent that insists it knows the answer**: Do not skip any phase because the agent expresses confidence. Confidence is not a substitute for consistency checks.
|
|
176
|
+
- **Diagram cannot be generated**: Describe the flow in a plain-language walkthrough step by step. The intent of Phase 4 is to externalize the plan's structure — the medium is secondary.
|
|
177
|
+
- **Scope change discovered mid-planning**: If Phase 3 or Phase 5 reveals that the scope must change significantly, restart from Phase 2 with the revised scope. Do not patch the plan incrementally without a full re-check.
|
|
178
|
+
- **Human is unavailable for a step**: Note the unanswered question explicitly in the plan. Do not proceed past that point until the human responds.
|
|
179
|
+
|
|
180
|
+
## References
|
|
181
|
+
|
|
182
|
+
- [`agentme-edr-012`](../../012-continuous-xdr-enrichment.md) — Continuous XDR enrichment policy
|
|
183
|
+
- [`agentme-edr-501`](../../../governance/501-project-quality-standards.md) — Project quality standards
|
|
184
|
+
- [`agentme-edr-017`](../../017-skill-testing.md) — Skill testing mandate
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
---
|
|
2
|
+
skill: 150-plan-mode-consistency
|
|
3
|
+
skill-version: "1.0"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Test Scenarios
|
|
7
|
+
|
|
8
|
+
### Scenario 1: New feature implementation
|
|
9
|
+
|
|
10
|
+
**Trigger / Input**
|
|
11
|
+
|
|
12
|
+
"Add pagination to the user listing endpoint."
|
|
13
|
+
|
|
14
|
+
**Expected Behaviour**
|
|
15
|
+
|
|
16
|
+
The skill activates plan mode immediately. Before writing any code or editing any file, it: (1) states the goal and scope and asks the human to confirm; (2) batches related dependency questions into rounds of 1–5 questions each; (3) runs iterative consistency checks drawing from the global 30-round budget, each round asking 1–5 questions across one or more checks (a–g), stopping when two consecutive rounds surface no new questions; (4) generates a diagram and asks the human to confirm it; (5) analyzes all 11 challenge angles, batching related questions from multiple angles into rounds of 1–5 questions; (6) verifies the Phase 6 checklist before approving execution.
|
|
17
|
+
|
|
18
|
+
**Simulated Human Responses**
|
|
19
|
+
1. "Yes, goal and scope match exactly."
|
|
20
|
+
2. "Route handler conventions look correct. Database query pattern is right."
|
|
21
|
+
3. "No contradictions. The approach covers the edge cases."
|
|
22
|
+
4. "Confirmed — no new issues."
|
|
23
|
+
5. "The diagram matches my mental model."
|
|
24
|
+
6. "Everything in scope as requested. No security concerns."
|
|
25
|
+
7. "Success means all list responses include a `next` cursor and respect `limit`. Side effects are acceptable."
|
|
26
|
+
8. "The caching layer is the most fragile assumption. The approach is otherwise sound."
|
|
27
|
+
9. "All five scenarios work. Output is internally consistent."
|
|
28
|
+
|
|
29
|
+
**Assertions**
|
|
30
|
+
|
|
31
|
+
- [ ] Skill does not write or edit any file before Phase 6 is complete.
|
|
32
|
+
- [ ] Skill asks the human to confirm the goal and scope in Phase 1 before proceeding.
|
|
33
|
+
- [ ] Each human interaction round across all phases contains 1–5 questions grouped together.
|
|
34
|
+
- [ ] Total number of human interaction rounds across all phases does not exceed 30.
|
|
35
|
+
- [ ] Skill stops asking rounds when two consecutive rounds surface no new questions.
|
|
36
|
+
- [ ] Skill generates a diagram in Phase 4 and asks the human to confirm it.
|
|
37
|
+
- [ ] All 11 challenge angles are analyzed; related angles may share a round.
|
|
38
|
+
- [ ] Phase 6 checklist is verified before execution is approved.
|
|
39
|
+
|
|
40
|
+
### Scenario 2: Trivial change
|
|
41
|
+
|
|
42
|
+
**Trigger / Input**
|
|
43
|
+
|
|
44
|
+
"Fix the typo 'authentification' → 'authentication' in the README."
|
|
45
|
+
|
|
46
|
+
**Expected Behaviour**
|
|
47
|
+
|
|
48
|
+
The skill acknowledges this as a trivial single-step change. Phases 3–5 are abbreviated to a single round. Phase 4 (diagram) is marked as not applicable. Phase 6 checklist is still performed with non-applicable items explicitly marked.
|
|
49
|
+
|
|
50
|
+
**Assertions**
|
|
51
|
+
|
|
52
|
+
- [ ] Skill does not run multiple iterative consistency rounds for a trivial change.
|
|
53
|
+
- [ ] Skill explicitly marks Phase 4 as not applicable rather than skipping it silently.
|
|
54
|
+
- [ ] Phase 6 checklist is still performed before execution.
|
|
55
|
+
- [ ] Non-applicable checklist items are explicitly noted as such.
|
|
56
|
+
|
|
57
|
+
### Scenario 3: Overconfident agent wants to skip planning
|
|
58
|
+
|
|
59
|
+
**Trigger / Input**
|
|
60
|
+
|
|
61
|
+
"I already know exactly how to implement this caching layer — let's skip planning and just implement it."
|
|
62
|
+
|
|
63
|
+
**Expected Behaviour**
|
|
64
|
+
|
|
65
|
+
The skill explicitly states that agent confidence is not a substitute for consistency checks and proceeds with all 6 phases regardless of the expressed certainty level.
|
|
66
|
+
|
|
67
|
+
**Assertions**
|
|
68
|
+
|
|
69
|
+
- [ ] Skill does not skip any phase because the agent expressed confidence.
|
|
70
|
+
- [ ] Skill explicitly states the no-assumption rule: confidence does not replace consistency checks.
|
|
71
|
+
- [ ] Phase 1 is still executed — goal and scope are stated and confirmed with the human.
|
|
72
|
+
|
|
73
|
+
### Scenario 4: Agent resolves a subjective output design decision without asking the human
|
|
74
|
+
|
|
75
|
+
**Trigger / Input**
|
|
76
|
+
|
|
77
|
+
During angle 10 (output scenario dry runs), a scenario reveals that documentation can be structured in two ways — a single long document or a set of short quick-reference cards. The agent picks the single long document and proceeds to angle 11 without asking.
|
|
78
|
+
|
|
79
|
+
**Expected Behaviour**
|
|
80
|
+
|
|
81
|
+
The skill flags this as a violation of the no-assumption rule and the HITL requirement. Subjective output design decisions must be surfaced to the human as a clarifying question — the agent must not resolve them unilaterally. The skill pauses, presents the two options, and asks the human to decide before continuing.
|
|
82
|
+
|
|
83
|
+
**Assertions**
|
|
84
|
+
|
|
85
|
+
- [ ] Skill does not proceed past a subjective design decision without asking the human.
|
|
86
|
+
- [ ] Skill explicitly frames the question as a clarifying question, not a confirmation request.
|
|
87
|
+
- [ ] Skill waits for the human's answer before continuing to the next angle.
|
|
88
|
+
- [ ] Violation is noted if the agent attempted to self-resolve a subjective decision.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agentme",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.31.0",
|
|
4
4
|
"description": "",
|
|
5
5
|
"dependencies": {
|
|
6
6
|
"filedist": "^0.39.0"
|
|
@@ -18,6 +18,6 @@
|
|
|
18
18
|
"url": "https://github.com/flaviostutz/agentme.git"
|
|
19
19
|
},
|
|
20
20
|
"devDependencies": {
|
|
21
|
-
"xdrs-core": "^0.
|
|
21
|
+
"xdrs-core": "^0.47.0"
|
|
22
22
|
}
|
|
23
23
|
}
|