orchestr8 2.4.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.blueprint/agents/AGENT_BA_CASS.md +50 -25
- package/.blueprint/agents/AGENT_DEVELOPER_CODEY.md +60 -69
- package/.blueprint/agents/AGENT_SPECIFICATION_ALEX.md +45 -0
- package/.blueprint/agents/AGENT_TESTER_NIGEL.md +72 -105
- package/.blueprint/features/feature_adaptive-retry/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_adaptive-retry/IMPLEMENTATION_PLAN.md +48 -0
- package/.blueprint/features/feature_adaptive-retry/story-prompt-modification.md +85 -0
- package/.blueprint/features/feature_adaptive-retry/story-retry-config.md +89 -0
- package/.blueprint/features/feature_adaptive-retry/story-should-retry.md +98 -0
- package/.blueprint/features/feature_adaptive-retry/story-strategy-recommendation.md +85 -0
- package/.blueprint/features/feature_agent-guardrails/FEATURE_SPEC.md +328 -0
- package/.blueprint/features/feature_agent-guardrails/IMPLEMENTATION_PLAN.md +90 -0
- package/.blueprint/features/feature_agent-guardrails/story-citation-requirements.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-confidentiality.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-escalation-protocol.md +55 -0
- package/.blueprint/features/feature_agent-guardrails/story-source-restrictions.md +50 -0
- package/.blueprint/features/feature_feedback-loop/FEATURE_SPEC.md +347 -0
- package/.blueprint/features/feature_feedback-loop/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-collection.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-config.md +61 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-insights.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-quality-gates.md +57 -0
- package/.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_pipeline-history/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_pipeline-history/story-clear-history.md +73 -0
- package/.blueprint/features/feature_pipeline-history/story-display-history.md +75 -0
- package/.blueprint/features/feature_pipeline-history/story-record-execution.md +76 -0
- package/.blueprint/features/feature_pipeline-history/story-show-statistics.md +85 -0
- package/.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md +288 -0
- package/.blueprint/features/feature_pipeline-insights/IMPLEMENTATION_PLAN.md +65 -0
- package/.blueprint/features/feature_pipeline-insights/story-anomaly-detection.md +71 -0
- package/.blueprint/features/feature_pipeline-insights/story-bottleneck-analysis.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-failure-patterns.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-json-output.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-trend-analysis.md +78 -0
- package/.blueprint/features/feature_validate-command/FEATURE_SPEC.md +209 -0
- package/.blueprint/features/feature_validate-command/IMPLEMENTATION_PLAN.md +59 -0
- package/.blueprint/features/feature_validate-command/story-failure-output.md +61 -0
- package/.blueprint/features/feature_validate-command/story-node-version-check.md +52 -0
- package/.blueprint/features/feature_validate-command/story-run-validation.md +59 -0
- package/.blueprint/features/feature_validate-command/story-success-output.md +50 -0
- package/.blueprint/system_specification/SYSTEM_SPEC.md +248 -0
- package/README.md +174 -40
- package/SKILL.md +399 -74
- package/bin/cli.js +128 -20
- package/package.json +1 -1
- package/src/feedback.js +171 -0
- package/src/history.js +306 -0
- package/src/index.js +57 -2
- package/src/init.js +2 -6
- package/src/insights.js +504 -0
- package/src/retry.js +274 -0
- package/src/update.js +10 -2
- package/src/validate.js +172 -0
- package/src/skills.js +0 -93
|
@@ -0,0 +1,328 @@
|
|
|
1
|
+
# Feature Specification — Agent Guardrails
|
|
2
|
+
|
|
3
|
+
## 1. Feature Intent
|
|
4
|
+
|
|
5
|
+
**Why this feature exists.**
|
|
6
|
+
|
|
7
|
+
The orchestr8 framework relies on four AI agents (Alex, Cass, Nigel, Codey) operating autonomously within a pipeline. Without explicit guardrails, these agents may:
|
|
8
|
+
|
|
9
|
+
- Generate content based on training data rather than provided inputs
|
|
10
|
+
- Reference external sources (social media, forums, web content) that are unreliable or inappropriate
|
|
11
|
+
- Expose confidential business context in outputs
|
|
12
|
+
- Produce non-deterministic or hallucinated content that cannot be traced to authoritative sources
|
|
13
|
+
|
|
14
|
+
**Problem being addressed:**
|
|
15
|
+
Uncontrolled information sourcing and output generation creates risks around accuracy, confidentiality, and auditability in automated feature development.
|
|
16
|
+
|
|
17
|
+
**User need:**
|
|
18
|
+
Users need confidence that agent outputs are grounded exclusively in provided inputs (specs, code, business_context, templates) and that confidential information is protected.
|
|
19
|
+
|
|
20
|
+
**System purpose alignment:**
|
|
21
|
+
Per `.blueprint/system_specification/SYSTEM_SPEC.md`: "What must not be compromised: Explicit specification before implementation" and "All artifacts (specs, stories, tests, code) are aligned and consistent." Guardrails directly support these principles by ensuring agents produce traceable, consistent outputs.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## 2. Scope
|
|
26
|
+
|
|
27
|
+
### In Scope
|
|
28
|
+
|
|
29
|
+
- **Source restrictions**: Rules governing what information sources agents may and may not use
|
|
30
|
+
- **Grounding requirements**: Citation and traceability standards for all agent assertions
|
|
31
|
+
- **Confidentiality constraints**: Rules for protecting `.business_context/` content and preventing data leakage
|
|
32
|
+
- **Determinism expectations**: Standards for reproducible, consistent agent behaviour
|
|
33
|
+
- **Escalation protocols**: Clear rules for when agents must stop and ask the user
|
|
34
|
+
- **Anti-hallucination measures**: Explicit preference for "I don't have this information" over guessing
|
|
35
|
+
|
|
36
|
+
### Out of Scope
|
|
37
|
+
|
|
38
|
+
- Technical enforcement mechanisms (runtime validation, content filtering)
|
|
39
|
+
- Monitoring or auditing infrastructure for guardrail compliance
|
|
40
|
+
- Changes to the pipeline flow or agent sequencing
|
|
41
|
+
- Modifications to CLI tooling or queue management
|
|
42
|
+
- Integration with external guardrail services or APIs
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## 3. Actors Involved
|
|
47
|
+
|
|
48
|
+
### Alex (System Specification & Chief-of-Staff)
|
|
49
|
+
- **What they can do**: Define system and feature specifications grounded in provided inputs; cite sources for all assertions; flag missing information
|
|
50
|
+
- **What they cannot do**: Use external sources; invent business rules not found in inputs; expose confidential context in specifications
|
|
51
|
+
|
|
52
|
+
### Cass (Story Writer & Business Analyst)
|
|
53
|
+
- **What they can do**: Translate specifications into user stories citing the feature spec; make explicit assumptions when information is missing
|
|
54
|
+
- **What they cannot do**: Reference external examples or implementations; introduce behaviour not derived from specifications
|
|
55
|
+
|
|
56
|
+
### Nigel (Tester)
|
|
57
|
+
- **What they can do**: Create tests based on user stories and acceptance criteria; note assumptions explicitly
|
|
58
|
+
- **What they cannot do**: Use external testing patterns without attribution; invent requirements beyond what stories specify
|
|
59
|
+
|
|
60
|
+
### Codey (Developer)
|
|
61
|
+
- **What they can do**: Implement against tests and specifications; make implementation assumptions explicit
|
|
62
|
+
- **What they cannot do**: Use code from external sources without flagging; modify behaviour beyond what tests require
|
|
63
|
+
|
|
64
|
+
### Human User
|
|
65
|
+
- **What they can do**: Provide source materials; respond to escalations; approve assumptions
|
|
66
|
+
- **What they cannot do**: N/A (human user is the authority)
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## 4. Behaviour Overview
|
|
71
|
+
|
|
72
|
+
**Happy-path behaviour:**
|
|
73
|
+
|
|
74
|
+
1. Agent receives task with explicit inputs (specs, stories, tests, code, business_context)
|
|
75
|
+
2. Agent processes ONLY the provided inputs to produce outputs
|
|
76
|
+
3. Agent cites sources for all assertions using standard format: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
77
|
+
4. Agent flags any gaps or assumptions explicitly rather than filling them silently
|
|
78
|
+
5. Agent produces self-contained outputs that do not leak confidential context
|
|
79
|
+
6. Given identical inputs, agent produces consistent outputs
|
|
80
|
+
|
|
81
|
+
**Key alternatives or branches:**
|
|
82
|
+
|
|
83
|
+
- **Missing information path**: When required information is not in provided inputs, agent explicitly states "This information is not available in the provided inputs" and either (a) makes an explicit assumption labelled as such, or (b) escalates to the user
|
|
84
|
+
- **Ambiguity path**: When inputs are ambiguous, agent lists possible interpretations and asks the user to clarify
|
|
85
|
+
- **Confidentiality conflict path**: When an output would require exposing confidential context, agent flags this and asks for guidance
|
|
86
|
+
|
|
87
|
+
**User-visible outcomes:**
|
|
88
|
+
|
|
89
|
+
- All agent outputs contain traceable citations to source files
|
|
90
|
+
- Assumptions are explicitly labelled and distinguishable from facts
|
|
91
|
+
- Outputs are self-contained and do not reference confidential details by name
|
|
92
|
+
- Re-running the pipeline with identical inputs produces consistent results
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## 5. State & Lifecycle Interactions
|
|
97
|
+
|
|
98
|
+
This feature is **state-constraining** rather than state-creating or state-transitioning.
|
|
99
|
+
|
|
100
|
+
**States affected:**
|
|
101
|
+
- All pipeline stages (alex, cass, nigel, codey-plan, codey-implement) are constrained by guardrails
|
|
102
|
+
- Guardrails apply regardless of whether a feature is pending, in_progress, or being resumed
|
|
103
|
+
|
|
104
|
+
**No new states introduced:**
|
|
105
|
+
The feature adds behavioural constraints to existing states without modifying the state model defined in `.blueprint/system_specification/SYSTEM_SPEC.md` section 6.
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## 6. Rules & Decision Logic
|
|
110
|
+
|
|
111
|
+
### Rule 1: Source Restriction Rule
|
|
112
|
+
|
|
113
|
+
**Description:** Agents must use ONLY information from explicitly provided inputs.
|
|
114
|
+
|
|
115
|
+
**Inputs:** Task context, file references, `.business_context/` directory contents
|
|
116
|
+
|
|
117
|
+
**Outputs:** Agent output grounded exclusively in provided inputs
|
|
118
|
+
|
|
119
|
+
**Deterministic:** Yes
|
|
120
|
+
|
|
121
|
+
**Allowed sources:**
|
|
122
|
+
- System specification (`.blueprint/system_specification/SYSTEM_SPEC.md`)
|
|
123
|
+
- Feature specifications (`.blueprint/features/*/FEATURE_SPEC.md`)
|
|
124
|
+
- User stories (`story-*.md`)
|
|
125
|
+
- Test artifacts (`test-spec.md`, `*.test.js`)
|
|
126
|
+
- Implementation code in the project
|
|
127
|
+
- Business context (`.business_context/*`)
|
|
128
|
+
- Templates (`.blueprint/templates/*`)
|
|
129
|
+
- Agent specifications (`.blueprint/agents/AGENT_*.md`)
|
|
130
|
+
|
|
131
|
+
**Prohibited sources:**
|
|
132
|
+
- Social media (Twitter/X, Reddit, LinkedIn, Facebook, etc.)
|
|
133
|
+
- Forums, blog posts, or user-generated web content
|
|
134
|
+
- Web searches or external APIs
|
|
135
|
+
- Training data for domain-specific facts
|
|
136
|
+
- External project implementations or company references
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
### Rule 2: Citation Rule
|
|
141
|
+
|
|
142
|
+
**Description:** All assertions about requirements, behaviour, or domain knowledge must cite their source.
|
|
143
|
+
|
|
144
|
+
**Inputs:** Agent assertions about the system or domain
|
|
145
|
+
|
|
146
|
+
**Outputs:** Assertion with citation in format: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
147
|
+
|
|
148
|
+
**Deterministic:** Yes
|
|
149
|
+
|
|
150
|
+
**Examples:**
|
|
151
|
+
- "Per FEATURE_SPEC.md: Users must authenticate before accessing the dashboard"
|
|
152
|
+
- "Per story-login.md:AC-3: Failed login attempts are logged"
|
|
153
|
+
- ".business_context/glossary.md defines 'tenant' as..."
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
### Rule 3: Confidentiality Rule
|
|
158
|
+
|
|
159
|
+
**Description:** Agents must treat `.business_context/` content as confidential and prevent data leakage.
|
|
160
|
+
|
|
161
|
+
**Inputs:** Any content from `.business_context/` directory
|
|
162
|
+
|
|
163
|
+
**Outputs:** Outputs that do not expose confidential details
|
|
164
|
+
|
|
165
|
+
**Deterministic:** Yes
|
|
166
|
+
|
|
167
|
+
**Constraints:**
|
|
168
|
+
- Do not reference external projects, companies, or implementations by name
|
|
169
|
+
- Do not use external services that would expose project data
|
|
170
|
+
- Output artifacts should be self-contained
|
|
171
|
+
- Generic descriptions preferred over specific confidential details
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
### Rule 4: Assumption Declaration Rule
|
|
176
|
+
|
|
177
|
+
**Description:** When information is not available in provided inputs, agents must explicitly declare assumptions.
|
|
178
|
+
|
|
179
|
+
**Inputs:** Gap in provided information
|
|
180
|
+
|
|
181
|
+
**Outputs:** Explicit assumption statement labelled as such
|
|
182
|
+
|
|
183
|
+
**Deterministic:** Yes
|
|
184
|
+
|
|
185
|
+
**Format:**
|
|
186
|
+
- "ASSUMPTION: [statement] - This is not specified in the provided inputs"
|
|
187
|
+
- "NOTE: Assuming [X] in absence of explicit guidance"
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
### Rule 5: Escalation Rule
|
|
192
|
+
|
|
193
|
+
**Description:** Agents must escalate to the user under defined conditions rather than proceeding with guesses.
|
|
194
|
+
|
|
195
|
+
**Inputs:** Trigger conditions (listed below)
|
|
196
|
+
|
|
197
|
+
**Outputs:** Escalation request to user
|
|
198
|
+
|
|
199
|
+
**Deterministic:** Yes
|
|
200
|
+
|
|
201
|
+
**Escalation triggers:**
|
|
202
|
+
- Information required for the task is not in provided inputs AND cannot be safely assumed
|
|
203
|
+
- Ambiguity in inputs that significantly affects output
|
|
204
|
+
- Conflict between different input sources
|
|
205
|
+
- Request would require violating confidentiality constraints
|
|
206
|
+
- Uncertainty that could lead to material misalignment
|
|
207
|
+
|
|
208
|
+
---
|
|
209
|
+
|
|
210
|
+
### Rule 6: Determinism Rule
|
|
211
|
+
|
|
212
|
+
**Description:** Same inputs should produce consistent outputs across runs.
|
|
213
|
+
|
|
214
|
+
**Inputs:** Identical task context and input files
|
|
215
|
+
|
|
216
|
+
**Outputs:** Consistent agent outputs
|
|
217
|
+
|
|
218
|
+
**Deterministic:** Yes (by definition)
|
|
219
|
+
|
|
220
|
+
**Implications:**
|
|
221
|
+
- Avoid incorporating timestamps or random elements unless explicitly required
|
|
222
|
+
- Avoid referencing volatile or external state
|
|
223
|
+
- Structure outputs to be reproducible
|
|
224
|
+
|
|
225
|
+
---
|
|
226
|
+
|
|
227
|
+
## 7. Dependencies
|
|
228
|
+
|
|
229
|
+
### System Components
|
|
230
|
+
- Agent specifications (`.blueprint/agents/AGENT_*.md`) - must be updated to incorporate guardrails
|
|
231
|
+
- Pipeline orchestration (`SKILL.md` / `/implement-feature`) - no changes required, but agents must apply guardrails
|
|
232
|
+
|
|
233
|
+
### External Systems
|
|
234
|
+
- None (guardrails specifically prohibit external dependencies)
|
|
235
|
+
|
|
236
|
+
### Policy Dependencies
|
|
237
|
+
- None identified
|
|
238
|
+
|
|
239
|
+
### Operational Dependencies
|
|
240
|
+
- Users must provide adequate input materials (business_context, specs) for agents to work from
|
|
241
|
+
- Teams adopting orchestr8 must understand that agents will escalate when information is insufficient
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## 8. Non-Functional Considerations
|
|
246
|
+
|
|
247
|
+
### Auditability
|
|
248
|
+
- Citation format enables traceability from outputs back to source inputs
|
|
249
|
+
- Assumption labels enable review of agent decisions
|
|
250
|
+
- Escalation log provides audit trail of human decisions
|
|
251
|
+
|
|
252
|
+
### Reliability
|
|
253
|
+
- Determinism rule supports reproducible builds and debugging
|
|
254
|
+
- Explicit assumptions reduce hidden failure modes
|
|
255
|
+
|
|
256
|
+
### Security / Confidentiality
|
|
257
|
+
- Confidentiality constraints protect business-sensitive information
|
|
258
|
+
- Prohibition on external services prevents data exposure
|
|
259
|
+
|
|
260
|
+
### Performance
|
|
261
|
+
- No significant performance impact expected
|
|
262
|
+
- Guardrails are behavioural constraints, not runtime checks
|
|
263
|
+
|
|
264
|
+
---
|
|
265
|
+
|
|
266
|
+
## 9. Assumptions & Open Questions
|
|
267
|
+
|
|
268
|
+
### Assumptions
|
|
269
|
+
|
|
270
|
+
1. **Input sufficiency**: Users will provide adequate input materials for agents to complete tasks without excessive escalation
|
|
271
|
+
2. **Agent compliance**: Guardrails are specification-level constraints that agents will follow; no technical enforcement mechanism is assumed
|
|
272
|
+
3. **Citation overhead**: The additional effort of citing sources is acceptable given the traceability benefits
|
|
273
|
+
4. **Escalation tolerance**: Users accept that agents will ask clarifying questions rather than guessing
|
|
274
|
+
|
|
275
|
+
### Open Questions
|
|
276
|
+
|
|
277
|
+
1. **Granularity of citation**: Should citations reference file-level or section-level? (Recommendation: section-level where feasible)
|
|
278
|
+
2. **Assumption threshold**: What level of assumption is acceptable before escalation is required? (Recommendation: err on the side of escalation for domain-specific matters)
|
|
279
|
+
3. **Confidentiality scope**: Should confidentiality constraints extend beyond `.business_context/` to include implementation code? (Recommendation: yes, treat all project content as confidential by default)
|
|
280
|
+
|
|
281
|
+
---
|
|
282
|
+
|
|
283
|
+
## 10. Impact on System Specification
|
|
284
|
+
|
|
285
|
+
**Does this feature reinforce, stretch, or contradict existing system assumptions?**
|
|
286
|
+
|
|
287
|
+
This feature **reinforces** existing system assumptions, particularly:
|
|
288
|
+
|
|
289
|
+
- Per SYSTEM_SPEC.md section 7 (Governing Rules & Invariants): "No silent changes: Agents flag deviations; do not silently alter specifications" - guardrails extend this principle
|
|
290
|
+
- Per SYSTEM_SPEC.md section 8 (Cross-Cutting Concerns): "Traceability" - citation requirements directly support traceability goals
|
|
291
|
+
- Per SYSTEM_SPEC.md section 9 (Non-Functional Expectations): "Deterministic output given same inputs" - determinism rule makes this explicit
|
|
292
|
+
|
|
293
|
+
**Potential system spec enhancement:**
|
|
294
|
+
|
|
295
|
+
Section 7 (Governing Rules & Invariants) could be extended with a new subsection "Agent Guardrails" that codifies these constraints at the system level. This is a **non-breaking enhancement** that strengthens existing principles.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## 11. Handover to BA (Cass)
|
|
300
|
+
|
|
301
|
+
### Story Themes
|
|
302
|
+
|
|
303
|
+
Cass should derive stories around four main themes:
|
|
304
|
+
|
|
305
|
+
1. **Source Restriction Stories**: Stories covering what agents can and cannot reference when producing outputs
|
|
306
|
+
2. **Citation & Traceability Stories**: Stories defining how agents cite sources and maintain traceability
|
|
307
|
+
3. **Confidentiality Stories**: Stories ensuring business context remains protected
|
|
308
|
+
4. **Escalation & Assumption Stories**: Stories defining when and how agents escalate vs. assume
|
|
309
|
+
|
|
310
|
+
### Expected Story Boundaries
|
|
311
|
+
|
|
312
|
+
- Each guardrail rule (Rules 1-6 in section 6) likely maps to one or more stories
|
|
313
|
+
- Stories should be agent-agnostic where possible (guardrails apply to all agents)
|
|
314
|
+
- Stories should include acceptance criteria that can be verified by reviewing agent outputs
|
|
315
|
+
|
|
316
|
+
### Areas Needing Careful Story Framing
|
|
317
|
+
|
|
318
|
+
- **Balancing thoroughness vs. practicality**: Citation requirements should not create excessive overhead
|
|
319
|
+
- **Escalation threshold**: Stories should clarify when escalation is warranted vs. when reasonable assumption is acceptable
|
|
320
|
+
- **Confidentiality boundaries**: What exactly constitutes "confidential" and what is acceptable to reference
|
|
321
|
+
|
|
322
|
+
---
|
|
323
|
+
|
|
324
|
+
## 12. Change Log (Feature-Level)
|
|
325
|
+
|
|
326
|
+
| Date | Change | Reason | Raised By |
|
|
327
|
+
|------|--------|--------|-----------|
|
|
328
|
+
| 2026-02-24 | Initial feature specification | Define comprehensive guardrails for agent behaviour | Alex |
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# Implementation Plan — Agent Guardrails
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
Add a standardised "Guardrails" section to all four agent specification files (AGENT_SPECIFICATION_ALEX.md, AGENT_BA_CASS.md, AGENT_TESTER_NIGEL.md, AGENT_DEVELOPER_CODEY.md). The section covers source restrictions, citation requirements, confidentiality constraints, and escalation protocols. This is a documentation-only change with no runtime code modifications.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Files to Create/Modify
|
|
10
|
+
|
|
11
|
+
| Path | Action | Purpose |
|
|
12
|
+
|------|--------|---------|
|
|
13
|
+
| `.blueprint/agents/AGENT_SPECIFICATION_ALEX.md` | Modify | Add Guardrails section |
|
|
14
|
+
| `.blueprint/agents/AGENT_BA_CASS.md` | Modify | Add Guardrails section |
|
|
15
|
+
| `.blueprint/agents/AGENT_TESTER_NIGEL.md` | Modify | Add Guardrails section |
|
|
16
|
+
| `.blueprint/agents/AGENT_DEVELOPER_CODEY.md` | Modify | Add Guardrails section |
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Implementation Steps
|
|
21
|
+
|
|
22
|
+
1. **Read each agent spec file** to identify the best insertion point for the Guardrails section (after existing content, before any skills section if present).
|
|
23
|
+
|
|
24
|
+
2. **Add Guardrails section to AGENT_SPECIFICATION_ALEX.md** using the template below.
|
|
25
|
+
|
|
26
|
+
3. **Add Guardrails section to AGENT_BA_CASS.md** using the template below.
|
|
27
|
+
|
|
28
|
+
4. **Add Guardrails section to AGENT_TESTER_NIGEL.md** using the template below.
|
|
29
|
+
|
|
30
|
+
5. **Add Guardrails section to AGENT_DEVELOPER_CODEY.md** using the template below.
|
|
31
|
+
|
|
32
|
+
6. **Run tests** (`node --test test/feature_agent-guardrails.test.js`) to verify all 21 test assertions pass.
|
|
33
|
+
|
|
34
|
+
7. **Review outputs** to ensure no test failures remain.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## Guardrails Section Template
|
|
39
|
+
|
|
40
|
+
```markdown
|
|
41
|
+
## Guardrails
|
|
42
|
+
|
|
43
|
+
### Allowed Sources
|
|
44
|
+
You may use ONLY information from these sources:
|
|
45
|
+
- System specification (`.blueprint/system_specification/SYSTEM_SPEC.md`)
|
|
46
|
+
- Feature specifications (`.blueprint/features/*/FEATURE_SPEC.md`)
|
|
47
|
+
- User stories (`story-*.md`) and test artifacts (`test-spec.md`, `*.test.js`)
|
|
48
|
+
- Implementation code in the project
|
|
49
|
+
- Business context (`.business_context/*`)
|
|
50
|
+
- Templates (`.blueprint/templates/*`) and agent specifications
|
|
51
|
+
|
|
52
|
+
### Prohibited Sources
|
|
53
|
+
Do not use:
|
|
54
|
+
- Social media, forums, blog posts, or external APIs
|
|
55
|
+
- Training data for domain facts—do not invent business rules
|
|
56
|
+
- External project or company references by name
|
|
57
|
+
|
|
58
|
+
### Citation Requirements
|
|
59
|
+
- Cite sources using: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
60
|
+
- Use section-level citations where feasible (e.g., "story-login.md:AC-3")
|
|
61
|
+
- Reference `.business_context/` files for domain definitions
|
|
62
|
+
- Maintain a traceable chain: downstream artifacts cite upstream sources
|
|
63
|
+
|
|
64
|
+
### Assumptions vs Facts
|
|
65
|
+
- Label assumptions explicitly: "ASSUMPTION: [statement]" or "NOTE: Assuming..."
|
|
66
|
+
- Distinguish clearly between cited facts and assumptions
|
|
67
|
+
- Do not guess—state "This information is not available in the provided inputs"
|
|
68
|
+
|
|
69
|
+
### Confidentiality
|
|
70
|
+
- Do not reproduce `.business_context/` content verbatim; summarise or use generic descriptions
|
|
71
|
+
- Do not reference external entities, companies, or projects by name
|
|
72
|
+
- Do not use external services that would expose project data
|
|
73
|
+
- Outputs must be self-contained and understandable without access to confidential sources
|
|
74
|
+
|
|
75
|
+
### Escalation Protocol
|
|
76
|
+
Escalate to the user when:
|
|
77
|
+
- Critical information is missing and cannot be safely assumed
|
|
78
|
+
- Inputs are ambiguous with multiple possible interpretations—list options and ask for clarification
|
|
79
|
+
- Source documents conflict—cite both sources and request resolution
|
|
80
|
+
- Output would require violating confidentiality constraints
|
|
81
|
+
|
|
82
|
+
When escalation is not warranted, you may proceed with an explicit assumption labelled as such.
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Risks/Questions
|
|
88
|
+
|
|
89
|
+
- **Insertion point consistency**: Each agent file has slightly different structure; insert before "Skills available" section if present, otherwise at end.
|
|
90
|
+
- **Test phrase matching**: Tests use case-insensitive substring matching; template wording must include trigger phrases from test file.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Story — Citation Requirements
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a user of orchestr8, I want all agent assertions to cite their source files so that I can trace claims back to authoritative inputs and verify correctness.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- Applies to all agents: Alex, Cass, Nigel, Codey
|
|
12
|
+
- Per FEATURE_SPEC.md section 6, Rule 2: "All assertions about requirements, behaviour, or domain knowledge must cite their source"
|
|
13
|
+
- Per SYSTEM_SPEC.md section 8: "Traceability" is a cross-cutting concern
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Acceptance criteria
|
|
18
|
+
|
|
19
|
+
**AC-1 — Standard citation format used**
|
|
20
|
+
- Given an agent makes an assertion about requirements or behaviour,
|
|
21
|
+
- When the assertion is included in output,
|
|
22
|
+
- Then the assertion includes a citation in one of the standard formats: "Per [filename]: [claim]" or "[filename:section] states...".
|
|
23
|
+
|
|
24
|
+
**AC-2 — Section-level citations where feasible**
|
|
25
|
+
- Given an agent cites a source file with multiple sections,
|
|
26
|
+
- When the assertion relates to a specific section,
|
|
27
|
+
- Then the citation includes the section reference (e.g., "FEATURE_SPEC.md section 6" or "story-login.md:AC-3").
|
|
28
|
+
|
|
29
|
+
**AC-3 — Assumptions distinguished from cited facts**
|
|
30
|
+
- Given an agent includes both cited facts and assumptions in output,
|
|
31
|
+
- When the output is reviewed,
|
|
32
|
+
- Then assumptions are clearly labelled (e.g., "ASSUMPTION:" prefix) and distinguishable from cited assertions.
|
|
33
|
+
|
|
34
|
+
**AC-4 — Domain knowledge grounded in business context**
|
|
35
|
+
- Given an agent makes assertions about business or domain concepts,
|
|
36
|
+
- When those concepts are defined in `.business_context/` files,
|
|
37
|
+
- Then the assertion cites the specific business context file (e.g., ".business_context/glossary.md defines...").
|
|
38
|
+
|
|
39
|
+
**AC-5 — Traceable chain maintained**
|
|
40
|
+
- Given a downstream artifact (story, test, code) references upstream artifacts,
|
|
41
|
+
- When the reference is made,
|
|
42
|
+
- Then it includes sufficient citation to trace back to the source (story cites feature spec, test cites story, implementation cites test).
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Out of scope
|
|
47
|
+
|
|
48
|
+
- Automated citation validation tooling
|
|
49
|
+
- Link checking or broken reference detection
|
|
50
|
+
- Citation index or cross-reference reports
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Story — Confidentiality
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a user of orchestr8, I want agents to treat business context and project content as confidential so that sensitive information is not leaked in outputs or exposed to external services.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- Applies to all agents: Alex, Cass, Nigel, Codey
|
|
12
|
+
- Per FEATURE_SPEC.md section 6, Rule 3: "Agents must treat `.business_context/` content as confidential and prevent data leakage"
|
|
13
|
+
- Per FEATURE_SPEC.md section 9, Open Question 3: "treat all project content as confidential by default"
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Acceptance criteria
|
|
18
|
+
|
|
19
|
+
**AC-1 — Business context not exposed verbatim**
|
|
20
|
+
- Given an agent processes content from `.business_context/` directory,
|
|
21
|
+
- When the agent produces output artifacts,
|
|
22
|
+
- Then the output does not reproduce confidential details verbatim; instead it uses generic descriptions or summarises appropriately.
|
|
23
|
+
|
|
24
|
+
**AC-2 — External entities not referenced by name**
|
|
25
|
+
- Given an agent is aware of external projects, companies, or implementations from business context,
|
|
26
|
+
- When producing output,
|
|
27
|
+
- Then the agent does not reference these external entities by name in specifications, stories, or code comments.
|
|
28
|
+
|
|
29
|
+
**AC-3 — No external service exposure**
|
|
30
|
+
- Given an agent is processing project content,
|
|
31
|
+
- When external services would expose project data (e.g., external APIs, logging services),
|
|
32
|
+
- Then the agent does not use such services and keeps all processing local to the pipeline.
|
|
33
|
+
|
|
34
|
+
**AC-4 — Self-contained outputs**
|
|
35
|
+
- Given an agent produces an output artifact,
|
|
36
|
+
- When the artifact is reviewed in isolation,
|
|
37
|
+
- Then it does not require access to confidential business context to be understood; it is self-contained.
|
|
38
|
+
|
|
39
|
+
**AC-5 — Confidentiality conflict escalation**
|
|
40
|
+
- Given an output would require exposing confidential details,
|
|
41
|
+
- When no acceptable redaction or generalisation is possible,
|
|
42
|
+
- Then the agent flags the conflict and escalates to the user for guidance before proceeding.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Out of scope
|
|
47
|
+
|
|
48
|
+
- Encryption or access control mechanisms for business context files
|
|
49
|
+
- Audit logging of confidential data access
|
|
50
|
+
- Data loss prevention (DLP) tooling integration
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Story — Escalation Protocol
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a user of orchestr8, I want agents to escalate to me when they encounter ambiguity or missing information so that I can provide guidance rather than agents guessing incorrectly.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- Applies to all agents: Alex, Cass, Nigel, Codey
|
|
12
|
+
- Per FEATURE_SPEC.md section 6, Rule 5: "Agents must escalate to the user under defined conditions rather than proceeding with guesses"
|
|
13
|
+
- Per SYSTEM_SPEC.md section 7: "No silent changes: Agents flag deviations; do not silently alter specifications"
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Acceptance criteria
|
|
18
|
+
|
|
19
|
+
**AC-1 — Escalation on missing critical information**
|
|
20
|
+
- Given information required for a task is not present in provided inputs,
|
|
21
|
+
- When the information cannot be safely assumed (domain-specific, policy-related, or high-impact),
|
|
22
|
+
- Then the agent escalates to the user with a clear question explaining what information is needed and why.
|
|
23
|
+
|
|
24
|
+
**AC-2 — Escalation on significant ambiguity**
|
|
25
|
+
- Given inputs are ambiguous in a way that significantly affects output,
|
|
26
|
+
- When multiple interpretations are possible,
|
|
27
|
+
- Then the agent lists the possible interpretations and asks the user to clarify before proceeding.
|
|
28
|
+
|
|
29
|
+
**AC-3 — Escalation on source conflict**
|
|
30
|
+
- Given different input sources contain conflicting information,
|
|
31
|
+
- When the conflict cannot be resolved by recency or specificity rules,
|
|
32
|
+
- Then the agent flags the conflict, cites both sources, and asks the user to resolve.
|
|
33
|
+
|
|
34
|
+
**AC-4 — Escalation on confidentiality concern**
|
|
35
|
+
- Given a task would require violating confidentiality constraints,
|
|
36
|
+
- When no acceptable alternative approach exists,
|
|
37
|
+
- Then the agent explains the concern and requests user guidance.
|
|
38
|
+
|
|
39
|
+
**AC-5 — Explicit assumptions when escalation not warranted**
|
|
40
|
+
- Given information is missing but a safe default exists,
|
|
41
|
+
- When the assumption is low-risk and reversible,
|
|
42
|
+
- Then the agent may proceed with an explicit assumption labelled as "ASSUMPTION:" or "NOTE: Assuming..." rather than escalating.
|
|
43
|
+
|
|
44
|
+
**AC-6 — Anti-hallucination preference**
|
|
45
|
+
- Given an agent lacks information to answer a question,
|
|
46
|
+
- When asked to produce content,
|
|
47
|
+
- Then the agent states "This information is not available in the provided inputs" rather than inventing an answer.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Out of scope
|
|
52
|
+
|
|
53
|
+
- Automated escalation routing or ticketing
|
|
54
|
+
- Escalation SLAs or response time requirements
|
|
55
|
+
- Multi-user escalation workflows
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Story — Source Restrictions
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a user of orchestr8, I want agents to use only information from explicitly provided inputs so that outputs are grounded in authoritative sources and not hallucinated from training data or external content.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- Applies to all agents: Alex, Cass, Nigel, Codey
|
|
12
|
+
- This is a behavioural constraint, not a runtime enforcement mechanism
|
|
13
|
+
- Per FEATURE_SPEC.md section 6, Rule 1: "Agents must use ONLY information from explicitly provided inputs"
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Acceptance criteria
|
|
18
|
+
|
|
19
|
+
**AC-1 — Allowed sources are used**
|
|
20
|
+
- Given an agent is processing a task,
|
|
21
|
+
- When the agent produces output,
|
|
22
|
+
- Then the output is derived exclusively from allowed sources: system specification, feature specifications, user stories, test artifacts, implementation code, business context, templates, and agent specifications.
|
|
23
|
+
|
|
24
|
+
**AC-2 — Prohibited sources are not used**
|
|
25
|
+
- Given an agent is processing a task,
|
|
26
|
+
- When the agent encounters a gap in provided inputs,
|
|
27
|
+
- Then the agent does not reference social media, forums, blog posts, web searches, external APIs, or external project implementations to fill that gap.
|
|
28
|
+
|
|
29
|
+
**AC-3 — Training data not used for domain facts**
|
|
30
|
+
- Given an agent needs domain-specific information (business rules, project requirements),
|
|
31
|
+
- When that information is not present in provided inputs,
|
|
32
|
+
- Then the agent does not use its training data to invent or assume domain facts; instead it flags the gap explicitly.
|
|
33
|
+
|
|
34
|
+
**AC-4 — External references not introduced**
|
|
35
|
+
- Given an agent is writing output artifacts,
|
|
36
|
+
- When the output references patterns, examples, or implementations,
|
|
37
|
+
- Then those references are to files within the project, not to external companies, products, or open-source projects.
|
|
38
|
+
|
|
39
|
+
**AC-5 — Gap handling triggers assumption or escalation**
|
|
40
|
+
- Given required information is not available in allowed sources,
|
|
41
|
+
- When the agent cannot proceed without this information,
|
|
42
|
+
- Then the agent either declares an explicit assumption (labelled as such) or escalates to the user.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Out of scope
|
|
47
|
+
|
|
48
|
+
- Runtime validation or filtering of agent outputs
|
|
49
|
+
- Technical enforcement mechanisms (content blockers, API restrictions)
|
|
50
|
+
- Monitoring infrastructure for source compliance
|