engineering-intelligence 1.3.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -0
- package/dist/adapters/index.js +9 -9
- package/dist/adapters/index.js.map +1 -1
- package/dist/templates.d.ts +1 -1
- package/dist/templates.d.ts.map +1 -1
- package/dist/templates.js +70 -0
- package/dist/templates.js.map +1 -1
- package/dist/visualizer/index.d.ts.map +1 -1
- package/dist/visualizer/index.js +107 -0
- package/dist/visualizer/index.js.map +1 -1
- package/package.json +1 -1
- package/templates/canonical/rules/engineering-intelligence.md +9 -2
- package/templates/canonical/skills/adr-compliance-checker/SKILL.md +43 -0
- package/templates/canonical/skills/aidlc-lifecycle-engine/SKILL.md +39 -0
- package/templates/canonical/skills/api-backward-compatibility-engine/SKILL.md +80 -0
- package/templates/canonical/skills/api-snapshot-testing-engine/SKILL.md +75 -0
- package/templates/canonical/skills/context-budget-optimizer/SKILL.md +97 -0
- package/templates/canonical/skills/context-sync-engine/SKILL.md +26 -4
- package/templates/canonical/skills/contract-test-generator/SKILL.md +40 -0
- package/templates/canonical/skills/convention-detector/SKILL.md +16 -2
- package/templates/canonical/skills/database-migration-safety-engine/SKILL.md +79 -0
- package/templates/canonical/skills/dead-code-detector/SKILL.md +34 -0
- package/templates/canonical/skills/engineering-change-review/SKILL.md +17 -2
- package/templates/canonical/skills/engineering-intelligence-skill/SKILL.md +92 -8
- package/templates/canonical/skills/environment-variable-auditor/SKILL.md +47 -0
- package/templates/canonical/skills/graph-engine/SKILL.md +22 -5
- package/templates/canonical/skills/impact-analysis-engine/SKILL.md +30 -9
- package/templates/canonical/skills/knowledge-base-validator/SKILL.md +19 -3
- package/templates/canonical/skills/llm-prompt-injection-guard/SKILL.md +47 -0
- package/templates/canonical/skills/memory-sync-engine/SKILL.md +20 -0
- package/templates/canonical/skills/operations-readiness-engine/SKILL.md +17 -4
- package/templates/canonical/skills/requirement-scoper/SKILL.md +9 -0
- package/templates/canonical/skills/security-audit-engine/SKILL.md +30 -3
- package/templates/canonical/skills/staleness-detector/SKILL.md +19 -0
- package/templates/canonical/skills/testing-intelligence-engine/SKILL.md +37 -1
- package/templates/canonical/skills/type-safety-engine/SKILL.md +81 -0
- package/templates/canonical/workflows/engineering-intelligence.md +6 -4
- package/templates/canonical/workflows/initialize-engineering-intelligence.md +3 -2
|
@@ -32,15 +32,29 @@ Classify the incoming request before starting:
|
|
|
32
32
|
|
|
33
33
|
### 1. Pre-Flight: Read Intelligence
|
|
34
34
|
|
|
35
|
-
|
|
36
|
-
- `knowledge-base/` —
|
|
37
|
-
- `.engineering-intelligence/aidlc/` —
|
|
38
|
-
- `.engineering-intelligence/memory/` — decisions, constraints, patterns
|
|
39
|
-
- `.engineering-intelligence/context/` — module
|
|
40
|
-
- `.engineering-intelligence/graph/` —
|
|
35
|
+
Use `context-budget-optimizer` before loading broad intelligence. Do not read all of `knowledge-base/` or all graph JSON by default. Build `.engineering-intelligence/context/context-manifest.md`, then load only relevant slices:
|
|
36
|
+
- `knowledge-base/` — only H2 sections relevant to the changed modules, APIs, schemas, or risk areas
|
|
37
|
+
- `.engineering-intelligence/aidlc/` — `aidlc-state.md`, active checkpoint, active unit, acceptance criteria, and execution-plan rows relevant to the request
|
|
38
|
+
- `.engineering-intelligence/memory/` — only matching decisions, constraints, conventions, regression patterns, and ADR references
|
|
39
|
+
- `.engineering-intelligence/context/` — module/service/runtime rows near the change scope
|
|
40
|
+
- `.engineering-intelligence/graph/` — only relevant nodes/edges by graph proximity
|
|
41
41
|
|
|
42
42
|
**If intelligence is missing or stale**: Run `initialize-intelligence-skill` first.
|
|
43
43
|
|
|
44
|
+
Token rule: keep initial intelligence loading under 40% of the available context budget whenever possible. Lazy-load safety-gate evidence only when the trigger applies.
|
|
45
|
+
|
|
46
|
+
#### Pre-Flight Freshness Gate
|
|
47
|
+
|
|
48
|
+
Before impact analysis or code edits:
|
|
49
|
+
|
|
50
|
+
1. Resolve the likely changed modules from the request, graph proximity, or `change-detection-engine`.
|
|
51
|
+
2. Run `staleness-detector` scoped to those modules and related knowledge/context/memory artifacts.
|
|
52
|
+
3. If any freshness score is below `60`, run `incremental-sync-engine` for the stale artifacts before editing product code, or explicitly mark stale context in the impact report with the affected documents and scores.
|
|
53
|
+
4. If any score is below `50`, implementation is blocked until incremental sync runs or the user explicitly accepts stale-context risk.
|
|
54
|
+
5. Use the `staleness-detector` **Pre-Implementation Drift Trigger** decision: `Proceed`, `Sync before implementation`, or `Block implementation`.
|
|
55
|
+
6. Carry that exact decision into the impact report's freshness-gate line.
|
|
56
|
+
7. Skip stale H2 sections that carry low confidence metadata unless they are refreshed or verified against source.
|
|
57
|
+
|
|
44
58
|
### 2. Impact Analysis: Write Report
|
|
45
59
|
|
|
46
60
|
Before any code edit, write `.engineering-intelligence/reports/IMP-XXX-<summary>.md`:
|
|
@@ -55,6 +69,7 @@ Before any code edit, write `.engineering-intelligence/reports/IMP-XXX-<summary>
|
|
|
55
69
|
|
|
56
70
|
## Analysis
|
|
57
71
|
- Mode: <proposal|diff>
|
|
72
|
+
- Freshness gate: <passed|synced|stale risk accepted> with scores
|
|
58
73
|
- Graph inputs consulted: <list>
|
|
59
74
|
- Directly affected: <files, modules, services>
|
|
60
75
|
- Indirectly affected: <downstream consumers, dependent services>
|
|
@@ -62,6 +77,10 @@ Before any code edit, write `.engineering-intelligence/reports/IMP-XXX-<summary>
|
|
|
62
77
|
|
|
63
78
|
## Validation Requirements
|
|
64
79
|
- <test types needed>
|
|
80
|
+
- Type safety: <required|not applicable>
|
|
81
|
+
- API compatibility: <required|not applicable>
|
|
82
|
+
- Migration safety: <required|not applicable>
|
|
83
|
+
- Acceptance mapping: required
|
|
65
84
|
- <manual verification needed>
|
|
66
85
|
|
|
67
86
|
## Intelligence Artifacts Affected
|
|
@@ -94,9 +113,21 @@ Before any code edit, write `.engineering-intelligence/reports/IMP-XXX-<summary>
|
|
|
94
113
|
- Follow existing coding patterns from `.engineering-intelligence/memory/coding-patterns.md`
|
|
95
114
|
- Read conventions from `knowledge-base/16-conventions.md` and `.engineering-intelligence/memory/coding-patterns.md` — match naming patterns, import style, error handling patterns, and code structure
|
|
96
115
|
- If conventions document is missing or outdated, run `convention-detector` first
|
|
116
|
+
- After generating or modifying each file, compare it against `coding-patterns.md` for naming, import order, error handling, logging, folder structure, test style, and framework idioms. Auto-correct minor violations. Structural convention violations become review findings and block completion when critical.
|
|
97
117
|
- Respect architectural boundaries from `.engineering-intelligence/memory/architecture-decisions.md`
|
|
98
118
|
- Consult `dangerous-areas.md` before modifying flagged code
|
|
99
119
|
|
|
120
|
+
#### Strict TDD Red-Green Gate
|
|
121
|
+
|
|
122
|
+
When TDD delivery mode is selected:
|
|
123
|
+
|
|
124
|
+
1. Add or update the required tests first.
|
|
125
|
+
2. Run the new/targeted tests before implementation code changes.
|
|
126
|
+
3. Confirm they fail for the expected reason.
|
|
127
|
+
4. Save failing output in `.engineering-intelligence/aidlc/construction/<unit>/build-and-test/build-and-test-summary.md`.
|
|
128
|
+
5. Only then implement production code.
|
|
129
|
+
6. If this sequence is skipped, mark the construction unit blocked unless the user explicitly approves non-TDD execution.
|
|
130
|
+
|
|
100
131
|
### 4. Add/Update Tests
|
|
101
132
|
|
|
102
133
|
- Add tests proportional to the change risk level
|
|
@@ -110,10 +141,32 @@ Before any code edit, write `.engineering-intelligence/reports/IMP-XXX-<summary>
|
|
|
110
141
|
|
|
111
142
|
- Run linters, type checks, and test suites available in the project
|
|
112
143
|
- Use environmental backpressure: analyze failed diagnostics, fix, and rerun the relevant command until it passes or a blocker is recorded
|
|
144
|
+
- Run `type-safety-engine` for typed projects or record why no type system applies
|
|
145
|
+
- Run `api-backward-compatibility-engine` when API, event, webhook, SDK, route, or schema contracts changed
|
|
146
|
+
- Run `api-snapshot-testing-engine` when API response behavior can be replayed or sampled
|
|
147
|
+
- Run `database-migration-safety-engine` when schema, ORM model, migration, index, or data persistence contracts changed
|
|
148
|
+
- Run `security-audit-engine` in targeted dependency-risk mode when package manifests add or upgrade dependencies; critical CVEs block completion
|
|
149
|
+
- Run `environment-variable-auditor` when environment variable reads, validation schemas, deployment config, or CI secrets change
|
|
150
|
+
- Run `adr-compliance-checker` when accepted ADRs or architecture decisions apply to the changed area
|
|
151
|
+
- Run `llm-prompt-injection-guard` when user-controlled data reaches prompts, RAG, agent tools, LLM calls, or durable AI memory
|
|
113
152
|
- Write `.engineering-intelligence/aidlc/construction/<unit>/build-and-test/build-and-test-summary.md` for non-trivial units
|
|
114
153
|
- **Never claim validation passed unless it actually ran and passed**
|
|
115
154
|
- Record partial or failed validation honestly
|
|
116
155
|
|
|
156
|
+
#### Acceptance Criteria Verification Matrix
|
|
157
|
+
|
|
158
|
+
Before Definition of Done can pass, map every criterion from `.engineering-intelligence/aidlc/agile/acceptance-criteria.md` to evidence:
|
|
159
|
+
|
|
160
|
+
```markdown
|
|
161
|
+
## Acceptance Criteria Verification Matrix
|
|
162
|
+
| Criterion | Evidence Type | Evidence | Result | Open Item |
|
|
163
|
+
|---|---|---|---|---|
|
|
164
|
+
| Given..., when..., then... | automated test | `test/file.test.ts` / command result | pass | — |
|
|
165
|
+
| ... | manual verification | <steps required> | pending | <owner/reason> |
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Missing mappings block the Done gate and must be copied into the CHG record as open items.
|
|
169
|
+
|
|
117
170
|
### 6. Incremental Sync
|
|
118
171
|
|
|
119
172
|
Use `incremental-sync-engine` to update only affected artifacts:
|
|
@@ -148,6 +201,28 @@ Create `.changes/CHG-XXX-<summary>.md`:
|
|
|
148
201
|
- <tests added/modified>
|
|
149
202
|
- <test results: passed/failed/skipped>
|
|
150
203
|
|
|
204
|
+
## Acceptance Criteria Verification
|
|
205
|
+
<copy the Acceptance Criteria Verification Matrix, including open items>
|
|
206
|
+
|
|
207
|
+
## Safety Gates
|
|
208
|
+
- Freshness gate: <passed|synced|stale risk accepted>
|
|
209
|
+
- Type safety: <passed|failed|not applicable>
|
|
210
|
+
- API compatibility: <passed|failed|not applicable>
|
|
211
|
+
- API snapshots: <passed|failed|not applicable>
|
|
212
|
+
- Migration safety: <passed|failed|not applicable>
|
|
213
|
+
- Dependency security: <passed|failed|not applicable>
|
|
214
|
+
- Environment variables: <passed|failed|not applicable>
|
|
215
|
+
- ADR compliance: <passed|failed|not applicable>
|
|
216
|
+
- LLM prompt injection: <passed|failed|not applicable>
|
|
217
|
+
- Convention enforcement: <passed|findings>
|
|
218
|
+
|
|
219
|
+
## Rollback
|
|
220
|
+
- Code rollback: <git revert command or branch rollback>
|
|
221
|
+
- Data rollback: <down migration / compensating operation / N/A with justification>
|
|
222
|
+
- Feature flag rollback: <toggle / N/A with justification>
|
|
223
|
+
- Infrastructure rollback: <IaC rollback / N/A with justification>
|
|
224
|
+
- Irreversible steps requiring approval: <list or none>
|
|
225
|
+
|
|
151
226
|
## Related Reports
|
|
152
227
|
- IMP-XXX: <link to impact report>
|
|
153
228
|
- REV-XXX: <link to review report, if applicable>
|
|
@@ -178,6 +253,7 @@ Summarize to the user:
|
|
|
178
253
|
## Quality Gates
|
|
179
254
|
|
|
180
255
|
- [ ] Impact report written before any code edit
|
|
256
|
+
- [ ] Pre-Flight Freshness Gate passed, synced stale artifacts, or stale risk was explicit in the impact report
|
|
181
257
|
- [ ] All changed behavior has corresponding test coverage
|
|
182
258
|
- [ ] Validation was actually executed (not just claimed)
|
|
183
259
|
- [ ] Only affected intelligence artifacts were updated
|
|
@@ -185,6 +261,14 @@ Summarize to the user:
|
|
|
185
261
|
- [ ] Story meets Definition of Ready before implementation starts
|
|
186
262
|
- [ ] Story meets Definition of Done before final report
|
|
187
263
|
- [ ] Acceptance criteria are mapped to validation evidence
|
|
264
|
+
- [ ] Acceptance Criteria Verification Matrix has no unmapped criteria unless recorded as open items
|
|
265
|
+
- [ ] Type safety, API compatibility, and migration safety gates ran when applicable
|
|
266
|
+
- [ ] API snapshot replay ran for changed API behavior when feasible
|
|
267
|
+
- [ ] Dependency security ran for new or upgraded packages
|
|
268
|
+
- [ ] Environment variable audit ran when config/env usage changed
|
|
269
|
+
- [ ] ADR compliance checked applicable accepted decisions
|
|
270
|
+
- [ ] LLM prompt injection guard ran for LLM/user-input paths
|
|
271
|
+
- [ ] Medium-and-above risk changes include rollback instructions or explicit N/A justification
|
|
188
272
|
- [ ] Environmental backpressure was used for validation failures
|
|
189
273
|
- [ ] Change record references the correct impact report
|
|
190
274
|
- [ ] High-risk changes went through review gate
|
|
@@ -192,6 +276,6 @@ Summarize to the user:
|
|
|
192
276
|
|
|
193
277
|
## Cross-References
|
|
194
278
|
|
|
195
|
-
- Depends on: `initialize-intelligence-skill` (prerequisite), `change-detection-engine`, `impact-analysis-engine`, `graph-engine`
|
|
196
|
-
- Uses during execution: `testing-intelligence-engine`, `incremental-sync-engine`, `change-history-engine`
|
|
279
|
+
- Depends on: `initialize-intelligence-skill` (prerequisite), `context-budget-optimizer`, `change-detection-engine`, `impact-analysis-engine`, `graph-engine`, `staleness-detector`
|
|
280
|
+
- Uses during execution: `testing-intelligence-engine`, `api-snapshot-testing-engine`, `type-safety-engine`, `api-backward-compatibility-engine`, `database-migration-safety-engine`, `security-audit-engine`, `environment-variable-auditor`, `adr-compliance-checker`, `llm-prompt-injection-guard`, `incremental-sync-engine`, `change-history-engine`
|
|
197
281
|
- Optional: `engineering-change-review` (for high-risk), `refactoring-planner` (for refactors), `convention-detector` (for convention compliance)
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: environment-variable-auditor
|
|
3
|
+
description: Audits environment variable usage against examples, validation schemas, CI secrets, and deployment configuration.
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Environment Variable Auditor
|
|
8
|
+
|
|
9
|
+
Use this skill when code adds, removes, or changes environment variables, configuration schemas, deployment manifests, or CI/CD secrets.
|
|
10
|
+
|
|
11
|
+
## Procedure
|
|
12
|
+
|
|
13
|
+
1. Detect environment variable reads:
|
|
14
|
+
- Node: `process.env.*`
|
|
15
|
+
- Python: `os.environ`, `os.getenv`
|
|
16
|
+
- Go: `os.Getenv`
|
|
17
|
+
- Ruby: `ENV[]`
|
|
18
|
+
- Java/Kotlin/C#: equivalent environment APIs
|
|
19
|
+
2. Compare against `.env.example`, `.env.sample`, README setup docs, deployment manifests, CI secret declarations, and validation schemas such as Zod, envalid, pydantic, dotenv-safe, or custom config loaders.
|
|
20
|
+
3. Flag:
|
|
21
|
+
- missing example entries
|
|
22
|
+
- missing validation/default
|
|
23
|
+
- stale example vars no longer used
|
|
24
|
+
- required vars not present in CI/deployment docs
|
|
25
|
+
- secrets accidentally committed or documented with real values
|
|
26
|
+
4. Block completion for newly required production env vars without docs and validation.
|
|
27
|
+
|
|
28
|
+
## Output
|
|
29
|
+
|
|
30
|
+
Write `.engineering-intelligence/aidlc/construction/<unit>/environment-variable-audit.md`:
|
|
31
|
+
|
|
32
|
+
```markdown
|
|
33
|
+
# Environment Variable Audit: <unit>
|
|
34
|
+
|
|
35
|
+
| Variable | Usage | Example Present | Validation Present | Deployment/CI Present | Risk |
|
|
36
|
+
|---|---|---|---|---|---|
|
|
37
|
+
|
|
38
|
+
## Required Fixes
|
|
39
|
+
- <missing or stale env handling>
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Quality Gates
|
|
43
|
+
|
|
44
|
+
- [ ] Env reads were scanned
|
|
45
|
+
- [ ] Example files and validation schemas were checked
|
|
46
|
+
- [ ] CI/deployment declarations were checked when present
|
|
47
|
+
- [ ] New required env vars are documented and validated
|
|
@@ -70,6 +70,8 @@ Each JSON graph uses this envelope:
|
|
|
70
70
|
| `configures` | Configuration dependency |
|
|
71
71
|
| `implicit-dependency` | Shared database tables, shared config |
|
|
72
72
|
| `event-coupling` | Event-driven coupling |
|
|
73
|
+
| `imports-type` | Type-only or annotation-level dependency between interfaces, aliases, declarations, schemas, or annotations |
|
|
74
|
+
| `co-change` | Git history coupling above confidence threshold |
|
|
73
75
|
|
|
74
76
|
### 2. `service-graph.json` — Service Communication Topology
|
|
75
77
|
|
|
@@ -102,6 +104,7 @@ Each JSON graph uses this envelope:
|
|
|
102
104
|
| `service-layer` | Business logic service |
|
|
103
105
|
| `repository` | Data access layer |
|
|
104
106
|
| `background` | Background job or worker |
|
|
107
|
+
| `function` | Function or method on a critical path |
|
|
105
108
|
|
|
106
109
|
| Edge Relation | Description |
|
|
107
110
|
|---|---|
|
|
@@ -136,6 +139,8 @@ Each JSON graph uses this envelope:
|
|
|
136
139
|
| `store` | Persistence |
|
|
137
140
|
| `sink` | Data output/consumer |
|
|
138
141
|
| `validator` | Validation step |
|
|
142
|
+
| `sensitive-source` | PII, token, payment, health, credential, or regulated data origin |
|
|
143
|
+
| `sensitive-sink` | Sink that persists, transmits, logs, or exports sensitive data |
|
|
139
144
|
|
|
140
145
|
| Edge Relation | Description |
|
|
141
146
|
|---|---|
|
|
@@ -163,9 +168,13 @@ Derive Mermaid diagrams from the JSON graphs. Include:
|
|
|
163
168
|
2. Build all five graphs from scratch
|
|
164
169
|
3. Assign `verified` confidence to edges backed by explicit code evidence
|
|
165
170
|
4. Assign `inferred` confidence to edges derived from patterns or naming conventions
|
|
166
|
-
5. Integrate git change coupling data from `git-intelligence-engine` as `
|
|
167
|
-
6.
|
|
168
|
-
7.
|
|
171
|
+
5. Integrate git change coupling data from `git-intelligence-engine` as `co-change` edges when coupling strength is above `0.7`, confidence is `inferred`, and evidence cites commit ranges or reports
|
|
172
|
+
6. Build `imports-type` edges from type checker/compiler API evidence when typed languages are detected
|
|
173
|
+
7. Build function-level call graph nodes for critical paths in `runtime-graph.json`
|
|
174
|
+
8. Annotate runtime edges with `metadata.hotness` derived from `critical-paths.md`, test coverage frequency, and git co-change strength
|
|
175
|
+
9. Classify sensitive data nodes in `data-flow-graph.json` with `metadata.sensitivity` (`pii`, `token`, `payment`, `credential`, `health`, `regulated`, `none`) and track propagation to sinks
|
|
176
|
+
10. Mark unresolvable relationships as `unknown` and add to `unknowns` array
|
|
177
|
+
11. Generate Mermaid diagrams in `architecture-map.md`
|
|
169
178
|
|
|
170
179
|
### Incremental Mode (post-change update)
|
|
171
180
|
|
|
@@ -173,8 +182,9 @@ Derive Mermaid diagrams from the JSON graphs. Include:
|
|
|
173
182
|
2. Identify affected nodes and edges across all graphs
|
|
174
183
|
3. Update, add, or remove only affected nodes and edges
|
|
175
184
|
4. Preserve stable node IDs — only change an ID if the represented element was renamed
|
|
176
|
-
5.
|
|
177
|
-
6.
|
|
185
|
+
5. Update `imports-type`, `co-change`, function call, hotness, and sensitive data annotations where affected
|
|
186
|
+
6. Re-derive affected sections of `architecture-map.md`
|
|
187
|
+
7. Require full remapping only when structural impact cannot be bounded
|
|
178
188
|
|
|
179
189
|
## Rules
|
|
180
190
|
|
|
@@ -184,6 +194,8 @@ Derive Mermaid diagrams from the JSON graphs. Include:
|
|
|
184
194
|
- Derive Mermaid diagrams in `architecture-map.md` from the JSON graph content — never hand-author diagrams that contradict the JSON
|
|
185
195
|
- For ordinary changes, update only impacted nodes, edges, maps, and connected context documents
|
|
186
196
|
- Rebuild all graphs on initialization, explicit mapping request, or when structural impact cannot be bounded
|
|
197
|
+
- Do not add `co-change` edges below `0.7` coupling strength
|
|
198
|
+
- Sensitive data propagation to unencrypted channels, logs, or unvalidated sinks must become a security finding in the impact report
|
|
187
199
|
|
|
188
200
|
## Quality Gates
|
|
189
201
|
|
|
@@ -192,6 +204,11 @@ Derive Mermaid diagrams from the JSON graphs. Include:
|
|
|
192
204
|
- [ ] No `unknown` relationships are left out of the `unknowns` array
|
|
193
205
|
- [ ] `architecture-map.md` Mermaid diagrams are syntactically valid
|
|
194
206
|
- [ ] Node IDs are stable across incremental updates
|
|
207
|
+
- [ ] Type dependencies are represented as `imports-type` where applicable
|
|
208
|
+
- [ ] Co-change edges above `0.7` are represented or explicitly unavailable
|
|
209
|
+
- [ ] Runtime critical paths include function-level nodes where detectable
|
|
210
|
+
- [ ] Sensitive data nodes and propagation edges are annotated in `data-flow-graph.json`
|
|
211
|
+
- [ ] Hot path edges include `metadata.hotness` when evidence exists
|
|
195
212
|
|
|
196
213
|
## Cross-References
|
|
197
214
|
|
|
@@ -23,10 +23,13 @@ Determine what can break before changing code. Produce a reusable impact report
|
|
|
23
23
|
3. **Trace Direct Impact** — Identify:
|
|
24
24
|
- Files directly modified or proposed for modification
|
|
25
25
|
- Functions, classes, types, and interfaces changed
|
|
26
|
+
- TypeScript interfaces, type aliases, enums, declaration files, Python annotations, Pydantic/dataclass models, and other typed contracts changed
|
|
26
27
|
- API contracts affected (routes, request/response shapes)
|
|
27
28
|
- Database schemas or migrations affected
|
|
28
29
|
- Configuration changes
|
|
29
30
|
|
|
31
|
+
For typed languages, invoke `type-safety-engine` in dependency-tracing mode. Add type-only dependencies as `imports-type` edges in `dependency-graph.json` and include them in direct impact when the changed symbol is a shared type or annotation.
|
|
32
|
+
|
|
30
33
|
4. **Trace Indirect Impact** — Using graph intelligence, identify:
|
|
31
34
|
- Downstream consumers of changed modules (from dependency-graph)
|
|
32
35
|
- Services communicating with changed services (from service-graph)
|
|
@@ -35,10 +38,14 @@ Determine what can break before changing code. Produce a reusable impact report
|
|
|
35
38
|
- Data pipelines affected (from data-flow-graph)
|
|
36
39
|
- Test suites covering affected code
|
|
37
40
|
- Cross-reference with git change coupling data from `git-intelligence-engine` for hidden dependencies
|
|
41
|
+
- Query paths referencing changed schema fields from schema-to-query mapping
|
|
42
|
+
- API clients or contract tests affected by additive, deprecated, or breaking API changes
|
|
43
|
+
|
|
44
|
+
5. **Traverse Sensitive Data Paths** — Query `data-flow-graph.json` for sensitive-source to sensitive-sink reachability involving the changed scope. Sensitive data propagation to unencrypted channels, logs, analytics events, prompt/RAG memory, or unvalidated sinks escalates risk and becomes a security finding in the impact report.
|
|
38
45
|
|
|
39
|
-
|
|
46
|
+
6. **Trace Transitive Impact** — Follow 2nd and 3rd order effects through the graph. Identify files that are indirectly affected by consumers of the directly affected modules. Walk the dependency chain until impact attenuates or reaches a service boundary.
|
|
40
47
|
|
|
41
|
-
|
|
48
|
+
7. **Score Risk** — Assign risk based on:
|
|
42
49
|
|
|
43
50
|
| Factor | Low | Medium | High | Critical |
|
|
44
51
|
|---|---|---|---|---|
|
|
@@ -48,8 +55,9 @@ Determine what can break before changing code. Produce a reusable impact report
|
|
|
48
55
|
| **API contract** | None | Additive | Deprecated | Breaking |
|
|
49
56
|
| **Test coverage** | Well-tested | Partial | Sparse | None |
|
|
50
57
|
| **Change coupling** | None | Low (1-2 coupled files) | Medium (3-5 coupled files) | High (6+ coupled files) |
|
|
58
|
+
| **Hot path** | Cold path | Normal | Critical path | Revenue/security/SLO path |
|
|
51
59
|
|
|
52
|
-
|
|
60
|
+
8. **Identify Validation Needs** — Map impact to required validation:
|
|
53
61
|
|
|
54
62
|
| Impact Area | Validation Required |
|
|
55
63
|
|---|---|
|
|
@@ -60,9 +68,9 @@ Determine what can break before changing code. Produce a reusable impact report
|
|
|
60
68
|
| UI change | Visual regression, accessibility |
|
|
61
69
|
| Infrastructure | Deploy to staging, smoke test |
|
|
62
70
|
|
|
63
|
-
|
|
71
|
+
9. **Map Intelligence Artifacts** — Determine which intelligence artifacts need synchronization after the change is implemented.
|
|
64
72
|
|
|
65
|
-
> **Surprise Impact Detection**: Flag any dependency discovered during analysis that is NOT in the current graph
|
|
73
|
+
> **Surprise Impact Detection**: Flag any dependency discovered during analysis that is NOT in the current graph. Submit the edge to `graph-engine` incremental mode as an `inferred` edge with evidence so the graph learns from the surprise. Surprise impacts must also be reported in the Unknowns section of the impact report until the graph update is complete.
|
|
66
74
|
|
|
67
75
|
## Output Format
|
|
68
76
|
|
|
@@ -83,9 +91,10 @@ Write `.engineering-intelligence/reports/IMP-XXX-<slug>.md`:
|
|
|
83
91
|
- Missing: <graphs not available>
|
|
84
92
|
|
|
85
93
|
## Direct Impact
|
|
86
|
-
| File/Module | Change Type | Risk |
|
|
87
|
-
|
|
88
|
-
| path/to/file.ts | Modified function `handleAuth` | high |
|
|
94
|
+
| File/Module | Change Type | Dependency Type | Risk |
|
|
95
|
+
|---|---|---|---|
|
|
96
|
+
| path/to/file.ts | Modified function `handleAuth` | runtime call | high |
|
|
97
|
+
| path/to/types.ts | Modified interface `User` | imports-type | high |
|
|
89
98
|
|
|
90
99
|
## Indirect Impact
|
|
91
100
|
| Affected Area | Relationship | Confidence |
|
|
@@ -93,6 +102,13 @@ Write `.engineering-intelligence/reports/IMP-XXX-<slug>.md`:
|
|
|
93
102
|
| UserService | Imports changed module | verified |
|
|
94
103
|
| /api/users endpoint | Calls modified handler | inferred |
|
|
95
104
|
|
|
105
|
+
## Contract And Data Impact
|
|
106
|
+
| Surface | Classification | Required Gate |
|
|
107
|
+
|---|---|---|
|
|
108
|
+
| API <name> | additive | api-backward-compatibility-engine |
|
|
109
|
+
| Table.column | schema-to-query impact | database-migration-safety-engine |
|
|
110
|
+
| Sensitive data path | unencrypted sink | security-audit-engine |
|
|
111
|
+
|
|
96
112
|
## Risk Assessment
|
|
97
113
|
- Overall risk: <level>
|
|
98
114
|
- Key risk factors: <list>
|
|
@@ -130,13 +146,18 @@ Write `.engineering-intelligence/reports/IMP-XXX-<slug>.md`:
|
|
|
130
146
|
- [ ] Scope is explicitly stated (not assumed)
|
|
131
147
|
- [ ] Graph inputs are listed (consulted, refreshed, or missing)
|
|
132
148
|
- [ ] Direct and indirect impact are separated
|
|
149
|
+
- [ ] Type-level dependencies are traced for typed languages
|
|
150
|
+
- [ ] API and schema/query impact are classified when relevant
|
|
151
|
+
- [ ] Sensitive data path traversal was performed for data-flow changes
|
|
152
|
+
- [ ] Surprise impacts were submitted to graph-engine incremental mode
|
|
133
153
|
- [ ] Risk score is justified with evidence
|
|
134
154
|
- [ ] Validation requirements are specific (not generic)
|
|
135
155
|
- [ ] Report ends with the "did not modify product code" statement
|
|
136
156
|
|
|
137
157
|
## Cross-References
|
|
138
158
|
|
|
139
|
-
- Depends on: `change-detection-engine`, `graph-engine`, `git-intelligence-engine`
|
|
159
|
+
- Depends on: `change-detection-engine`, `graph-engine`, `git-intelligence-engine`, `type-safety-engine`
|
|
140
160
|
- Consults: `data-flow-graph.json` (for data pipeline impact)
|
|
161
|
+
- Consults: `api-backward-compatibility-engine`, `database-migration-safety-engine` when contracts or schemas change
|
|
141
162
|
- Used by: `engineering-intelligence-skill`, `incremental-sync-engine`, `analyze-impact` workflow
|
|
142
163
|
- Consumed by: `engineering-change-review`, `testing-intelligence-engine`
|
|
@@ -38,7 +38,11 @@ Systematically audit every significant claim in `knowledge-base/*.md` against ac
|
|
|
38
38
|
- Distribution across statuses
|
|
39
39
|
- Overall document confidence: High (>90% ✅), Medium (70-90% ✅), Low (<70% ✅)
|
|
40
40
|
|
|
41
|
-
5. **
|
|
41
|
+
5. **Cross-Document Consistency Check** — Extract key claims from all knowledge documents and compare claims across documents for contradictions, such as different framework versions, conflicting ownership, incompatible API signatures, or mismatched data model descriptions.
|
|
42
|
+
|
|
43
|
+
6. **Auto-Heal Unsupported Claims** — During explicit synchronization workflows only, re-extract the smallest affected section for unsupported or stale claims, update that section with fresh evidence citations, and record the heal. Escalate claims requiring product judgment instead of guessing.
|
|
44
|
+
|
|
45
|
+
7. **Write Report** — Generate `knowledge-base/15-validation-report.md`
|
|
42
46
|
|
|
43
47
|
## Output Format
|
|
44
48
|
|
|
@@ -75,6 +79,16 @@ Scope: <documents validated>
|
|
|
75
79
|
|
|
76
80
|
- <areas where code has diverged from docs>
|
|
77
81
|
|
|
82
|
+
## Cross-Document Contradictions
|
|
83
|
+
|
|
84
|
+
| Claim A | Document A | Claim B | Document B | Suggested Canonical Resolution |
|
|
85
|
+
|---|---|---|---|---|
|
|
86
|
+
|
|
87
|
+
## Auto-Heal Actions
|
|
88
|
+
|
|
89
|
+
| Document | Section | Action | Evidence | Result |
|
|
90
|
+
|---|---|---|---|---|
|
|
91
|
+
|
|
78
92
|
## Recommended Actions
|
|
79
93
|
|
|
80
94
|
- <specific documents needing update>
|
|
@@ -83,8 +97,8 @@ Scope: <documents validated>
|
|
|
83
97
|
|
|
84
98
|
## Rules
|
|
85
99
|
|
|
86
|
-
- Do NOT silently rewrite knowledge documents during validation
|
|
87
|
-
-
|
|
100
|
+
- Do NOT silently rewrite knowledge documents during read-only validation
|
|
101
|
+
- Auto-heal unsupported claims only as part of an explicit synchronization workflow and record every edit
|
|
88
102
|
- Report honestly — a low-confidence score is valuable information
|
|
89
103
|
- Flag areas where you lack sufficient context to validate
|
|
90
104
|
|
|
@@ -94,6 +108,8 @@ Scope: <documents validated>
|
|
|
94
108
|
- [ ] Each finding has an evidence path or explicit "no evidence found"
|
|
95
109
|
- [ ] Summary table has accurate counts
|
|
96
110
|
- [ ] Stale documentation risks are identified
|
|
111
|
+
- [ ] Cross-document contradictions are listed or explicitly absent
|
|
112
|
+
- [ ] Auto-heal actions are recorded when synchronization mode is active
|
|
97
113
|
- [ ] Recommended actions are actionable
|
|
98
114
|
|
|
99
115
|
## Cross-References
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: llm-prompt-injection-guard
|
|
3
|
+
description: Detects user-input-to-LLM prompt injection paths, unsafe RAG ingestion, unvalidated LLM outputs, and poisoned AI memory/documentation flows.
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# LLM Prompt Injection Guard
|
|
8
|
+
|
|
9
|
+
Use this skill for AI-augmented applications, RAG pipelines, agent tools, prompt builders, chat handlers, knowledge ingestion, or any code that sends user-controlled data to an LLM.
|
|
10
|
+
|
|
11
|
+
## Procedure
|
|
12
|
+
|
|
13
|
+
1. Detect LLM calls, prompt templates, tool outputs, embedding pipelines, document ingestion, and agent memory writes.
|
|
14
|
+
2. Trace user-controlled sources into prompts, system messages, tool descriptions, retrieval documents, logs, knowledge-base files, and memory files.
|
|
15
|
+
3. Flag missing controls:
|
|
16
|
+
- no prompt boundary separation
|
|
17
|
+
- no input sanitization or quoting
|
|
18
|
+
- no output schema validation
|
|
19
|
+
- tool results trusted without validation
|
|
20
|
+
- externally sourced content written into durable memory without provenance
|
|
21
|
+
- secrets or policies exposed to user-influenced context
|
|
22
|
+
4. Require adversarial tests for high-risk LLM paths.
|
|
23
|
+
|
|
24
|
+
## Output
|
|
25
|
+
|
|
26
|
+
Write `.engineering-intelligence/reports/LLM-PROMPT-INJECTION-<slug>.md`:
|
|
27
|
+
|
|
28
|
+
```markdown
|
|
29
|
+
# LLM Prompt Injection Review: <summary>
|
|
30
|
+
|
|
31
|
+
## Data Paths
|
|
32
|
+
| Source | LLM / Memory Sink | Control Present | Risk | Evidence |
|
|
33
|
+
|---|---|---|---|---|
|
|
34
|
+
|
|
35
|
+
## Findings
|
|
36
|
+
- <prompt injection or output validation risk>
|
|
37
|
+
|
|
38
|
+
## Required Tests
|
|
39
|
+
- <adversarial test cases>
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Quality Gates
|
|
43
|
+
|
|
44
|
+
- [ ] LLM calls and memory/document ingestion paths were inventoried
|
|
45
|
+
- [ ] User-controlled sources were traced to LLM and durable-memory sinks
|
|
46
|
+
- [ ] Output validation was checked
|
|
47
|
+
- [ ] High-risk paths have adversarial tests or blocking findings
|
|
@@ -22,6 +22,11 @@ Maintain durable, long-lived engineering memory. Memory is for decisions and pat
|
|
|
22
22
|
| `coding-patterns.md` | Recurring conventions, idioms, naming rules, file organization | Refactors that establish new patterns, convention changes |
|
|
23
23
|
| `project-constraints.md` | Performance budgets, compatibility requirements, SLA targets, regulatory | Infrastructure changes, new compliance requirements |
|
|
24
24
|
| `technology-decisions.md` | Stack choices, framework versions, deprecation timelines, migration plans | Dependency updates, technology migrations |
|
|
25
|
+
| `regression-patterns.md` | Recurring bug categories and proven regression test templates | Bugfixes that reveal reusable failure patterns |
|
|
26
|
+
|
|
27
|
+
## Regression Pattern Ownership
|
|
28
|
+
|
|
29
|
+
Testing Intelligence Engine owns detection and proposal of regression patterns during bugfix validation. Memory Sync owns durable persistence to `.engineering-intelligence/memory/regression-patterns.md` after confirming the pattern is reusable and evidence-backed. Testing Intelligence must not directly persist durable memory unless it is explicitly running through Memory Sync.
|
|
25
30
|
|
|
26
31
|
## Staleness Detection Rules
|
|
27
32
|
|
|
@@ -58,12 +63,25 @@ A memory entry may be stale if:
|
|
|
58
63
|
- Is this a decision or just an implementation detail?
|
|
59
64
|
- Is this captured better elsewhere (knowledge-base, context)?
|
|
60
65
|
|
|
66
|
+
5. **Memory Pruning Audit** — During initialization, major refactors, or explicit sync:
|
|
67
|
+
- Flag entries where referenced code no longer exists
|
|
68
|
+
- Flag entries superseded for more than six months
|
|
69
|
+
- Flag patterns contradicted by current `convention-detector` output
|
|
70
|
+
- Flag decisions that conflict with accepted ADRs
|
|
71
|
+
- Propose retirement with evidence; do not delete historical decisions silently
|
|
72
|
+
|
|
73
|
+
6. **Regression Pattern Update** — For bugfixes:
|
|
74
|
+
- Classify the bug category (pagination boundary, null/empty collection, race condition, permission bypass, retry/idempotency, schema drift, API contract mismatch, etc.)
|
|
75
|
+
- Match against `.engineering-intelligence/memory/regression-patterns.md`
|
|
76
|
+
- Add or update a reusable regression test template when the pattern is durable
|
|
77
|
+
|
|
61
78
|
## Rules
|
|
62
79
|
|
|
63
80
|
- **Durable only**: Do not store transient implementation notes or unverified assumptions
|
|
64
81
|
- **Evidence required**: Every entry must cite source evidence
|
|
65
82
|
- **Leave unchanged when appropriate**: Most changes do NOT affect durable memory — it's correct to update nothing
|
|
66
83
|
- **Status tracking**: Mark superseded decisions as `Superseded` rather than deleting them — history matters
|
|
84
|
+
- **Pruning with evidence**: Retire or deprecate stale memory only with evidence and status updates
|
|
67
85
|
- **No product code**: Memory synchronization never modifies product code
|
|
68
86
|
|
|
69
87
|
## Quality Gates
|
|
@@ -73,6 +91,8 @@ A memory entry may be stale if:
|
|
|
73
91
|
- [ ] New entries are truly durable (not transient implementation details)
|
|
74
92
|
- [ ] All entries have evidence citations
|
|
75
93
|
- [ ] Superseded entries are marked, not deleted
|
|
94
|
+
- [ ] Memory pruning audit was run for initialization and major refactors
|
|
95
|
+
- [ ] Regression patterns were checked for bugfixes
|
|
76
96
|
|
|
77
97
|
## Cross-References
|
|
78
98
|
|
|
@@ -8,13 +8,16 @@ version: 1.0.0
|
|
|
8
8
|
|
|
9
9
|
Use this skill when a change affects deployment, infrastructure, runtime behavior, SLOs, data migrations, incident response, or production monitoring.
|
|
10
10
|
|
|
11
|
+
For medium-and-above risk changes, use the rollback planning section even when the change is not deployment-bound. Rollback planning is a release safety gate, not only an infrastructure concern.
|
|
12
|
+
|
|
11
13
|
## Procedure
|
|
12
14
|
|
|
13
15
|
1. Identify deployment target, environment variables, infrastructure files, CI/CD gates, and runtime services.
|
|
14
16
|
2. Define release strategy: normal deploy, canary, blue/green, feature flag, or manual rollout.
|
|
15
17
|
3. Define rollback plan and data recovery constraints.
|
|
16
18
|
4. Define observability: metrics, traces, logs, dashboards, alerts, and ownership.
|
|
17
|
-
5.
|
|
19
|
+
5. Inject or flag observability gaps for new functions, endpoints, jobs, service interactions, and business events.
|
|
20
|
+
6. Confirm production readiness with objective gates.
|
|
18
21
|
|
|
19
22
|
## Outputs
|
|
20
23
|
|
|
@@ -34,16 +37,24 @@ Write `.engineering-intelligence/aidlc/operations/operations-readiness.md`:
|
|
|
34
37
|
| Signal | Source | Threshold | Owner | Runbook |
|
|
35
38
|
|---|---|---|---|---|
|
|
36
39
|
|
|
40
|
+
## Observability Injection
|
|
41
|
+
| New / Changed Path | Entry Log | Exit Log | Trace Span | Error Metric | Business Event | Status |
|
|
42
|
+
|---|---|---|---|---|---|---|
|
|
43
|
+
|
|
37
44
|
## Rollback
|
|
38
|
-
- Code rollback:
|
|
39
|
-
- Data rollback:
|
|
40
|
-
-
|
|
45
|
+
- Code rollback: `git revert <commit>` or <branch rollback sequence>
|
|
46
|
+
- Data rollback: <down migration / compensating SQL / irreversible approval>
|
|
47
|
+
- Feature flag rollback: <toggle and expected effect>
|
|
48
|
+
- Infrastructure rollback: <IaC rollback sequence>
|
|
49
|
+
- External dependency rollback: <vendor/config rollback>
|
|
41
50
|
|
|
42
51
|
## Readiness Gates
|
|
43
52
|
- [ ] Build passes
|
|
44
53
|
- [ ] Tests pass
|
|
45
54
|
- [ ] Migrations are backward compatible or downtime is approved
|
|
46
55
|
- [ ] Alerts/runbooks exist for changed failure modes
|
|
56
|
+
- [ ] Rollback planner covers code, data, feature flag, and infrastructure rollback for medium+ risk
|
|
57
|
+
- [ ] New endpoints/functions/services have logging, tracing, metrics, and business-event coverage or explicit gaps
|
|
47
58
|
- [ ] Secrets and environment variables are documented securely
|
|
48
59
|
```
|
|
49
60
|
|
|
@@ -52,3 +63,5 @@ Write `.engineering-intelligence/aidlc/operations/operations-readiness.md`:
|
|
|
52
63
|
- Never execute production deployment without explicit user approval.
|
|
53
64
|
- Mark unknown production constraints in `open-questions.md`.
|
|
54
65
|
- For database changes, require backward-compatible migration planning unless downtime is approved.
|
|
66
|
+
- Medium-and-above risk changes require rollback procedures in both `operations-readiness.md` and the CHG record.
|
|
67
|
+
- Irreversible steps require explicit human approval.
|
|
@@ -21,6 +21,13 @@ Act as a detailed Business Analyst and Technical Architect persona. Analyze the
|
|
|
21
21
|
- Identify domain logic in `knowledge-base/` matching the request category
|
|
22
22
|
- Query dependency/service graphs to locate related modules and boundaries
|
|
23
23
|
- Read architecture memory to understand tech constraints and guidelines
|
|
24
|
+
- Scan relevant modules for implicit invariants and dominant implementation constraints before asking questions:
|
|
25
|
+
- retry strategies, timeout values, rate limits, pagination defaults
|
|
26
|
+
- locking/idempotency patterns
|
|
27
|
+
- validation style and error codes
|
|
28
|
+
- logging/tracing conventions
|
|
29
|
+
- framework patterns used in 80% or more of similar code
|
|
30
|
+
- Surface confirmed dominant patterns as constraints in the requirements document.
|
|
24
31
|
|
|
25
32
|
2. **Formulate Scoping Questions** — Identify gaps, ambiguities, and edge cases. Ask the user 3 to 5 targeted questions covering:
|
|
26
33
|
- **Business Value & Scope**: What are the limits of the MVP?
|
|
@@ -47,6 +54,7 @@ The final requirements document `knowledge-base/19-requirements.md` must follow
|
|
|
47
54
|
- **Logic & Configuration**: <Exact details on implementation strategy, libraries, config parameters>
|
|
48
55
|
- **System Boundaries & Dependencies**: <Files/modules affected based on graph mappings>
|
|
49
56
|
- **Edge Cases & Failure Modes**: <Exactly how to handle failures, retries, limits>
|
|
57
|
+
- **Implicit Codebase Constraints**: <Dominant existing patterns discovered from relevant modules, with evidence paths>
|
|
50
58
|
|
|
51
59
|
## 3. Agile Delivery Model
|
|
52
60
|
- **Epic**: <epic or initiative>
|
|
@@ -81,5 +89,6 @@ Provide the exact prompt to pass to the coding agent to execute this change:
|
|
|
81
89
|
- [ ] Clear business goals and technical boundaries defined.
|
|
82
90
|
- [ ] At least 3 scoping questions asked and logged.
|
|
83
91
|
- [ ] User story, acceptance criteria, priority, dependencies, and Ready/Done gates are documented.
|
|
92
|
+
- [ ] Implicit codebase constraints were mined and cited before questions were finalized.
|
|
84
93
|
- [ ] Finalized prompt maps exact files and modules.
|
|
85
94
|
- [ ] Output does not contain any code modification.
|