@trohde/earos 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +156 -0
- package/assets/init/.agents/skills/earos-artifact-gen/SKILL.md +106 -0
- package/assets/init/.agents/skills/earos-artifact-gen/references/interview-guide.md +313 -0
- package/assets/init/.agents/skills/earos-artifact-gen/references/output-guide.md +367 -0
- package/assets/init/.agents/skills/earos-assess/SKILL.md +212 -0
- package/assets/init/.agents/skills/earos-assess/references/calibration-benchmarks.md +160 -0
- package/assets/init/.agents/skills/earos-assess/references/output-templates.md +311 -0
- package/assets/init/.agents/skills/earos-assess/references/scoring-protocol.md +281 -0
- package/assets/init/.agents/skills/earos-calibrate/SKILL.md +153 -0
- package/assets/init/.agents/skills/earos-calibrate/references/agreement-metrics.md +188 -0
- package/assets/init/.agents/skills/earos-calibrate/references/calibration-protocol.md +263 -0
- package/assets/init/.agents/skills/earos-create/SKILL.md +257 -0
- package/assets/init/.agents/skills/earos-create/references/criterion-writing-guide.md +268 -0
- package/assets/init/.agents/skills/earos-create/references/dependency-rules.md +193 -0
- package/assets/init/.agents/skills/earos-create/references/rubric-interview-guide.md +123 -0
- package/assets/init/.agents/skills/earos-create/references/validation-checklist.md +238 -0
- package/assets/init/.agents/skills/earos-profile-author/SKILL.md +251 -0
- package/assets/init/.agents/skills/earos-profile-author/references/criterion-writing-guide.md +280 -0
- package/assets/init/.agents/skills/earos-profile-author/references/design-methods.md +158 -0
- package/assets/init/.agents/skills/earos-profile-author/references/profile-checklist.md +173 -0
- package/assets/init/.agents/skills/earos-remediate/SKILL.md +118 -0
- package/assets/init/.agents/skills/earos-remediate/references/output-template.md +199 -0
- package/assets/init/.agents/skills/earos-remediate/references/remediation-patterns.md +330 -0
- package/assets/init/.agents/skills/earos-report/SKILL.md +85 -0
- package/assets/init/.agents/skills/earos-report/references/portfolio-template.md +181 -0
- package/assets/init/.agents/skills/earos-report/references/single-artifact-template.md +168 -0
- package/assets/init/.agents/skills/earos-review/SKILL.md +130 -0
- package/assets/init/.agents/skills/earos-review/references/challenge-patterns.md +163 -0
- package/assets/init/.agents/skills/earos-review/references/output-template.md +180 -0
- package/assets/init/.agents/skills/earos-template-fill/SKILL.md +177 -0
- package/assets/init/.agents/skills/earos-template-fill/references/evidence-writing-guide.md +186 -0
- package/assets/init/.agents/skills/earos-template-fill/references/section-rubric-mapping.md +200 -0
- package/assets/init/.agents/skills/earos-validate/SKILL.md +113 -0
- package/assets/init/.agents/skills/earos-validate/references/fix-patterns.md +281 -0
- package/assets/init/.agents/skills/earos-validate/references/validation-checks.md +287 -0
- package/assets/init/.claude/CLAUDE.md +4 -0
- package/assets/init/AGENTS.md +293 -0
- package/assets/init/CLAUDE.md +635 -0
- package/assets/init/README.md +507 -0
- package/assets/init/calibration/gold-set/.gitkeep +0 -0
- package/assets/init/calibration/results/.gitkeep +0 -0
- package/assets/init/core/core-meta-rubric.yaml +643 -0
- package/assets/init/docs/consistency-report.md +325 -0
- package/assets/init/docs/getting-started.md +194 -0
- package/assets/init/docs/profile-authoring-guide.md +51 -0
- package/assets/init/docs/terminology.md +126 -0
- package/assets/init/earos.manifest.yaml +104 -0
- package/assets/init/evaluations/.gitkeep +0 -0
- package/assets/init/examples/aws-event-driven-order-processing/artifact.yaml +2056 -0
- package/assets/init/examples/aws-event-driven-order-processing/evaluation.yaml +973 -0
- package/assets/init/examples/aws-event-driven-order-processing/report.md +244 -0
- package/assets/init/examples/example-solution-architecture.evaluation.yaml +136 -0
- package/assets/init/examples/multi-cloud-data-analytics/artifact.yaml +715 -0
- package/assets/init/overlays/data-governance.yaml +94 -0
- package/assets/init/overlays/regulatory.yaml +154 -0
- package/assets/init/overlays/security.yaml +92 -0
- package/assets/init/profiles/adr.yaml +225 -0
- package/assets/init/profiles/capability-map.yaml +223 -0
- package/assets/init/profiles/reference-architecture.yaml +426 -0
- package/assets/init/profiles/roadmap.yaml +205 -0
- package/assets/init/profiles/solution-architecture.yaml +227 -0
- package/assets/init/research/architecture-assessment-rubrics-research.docx +0 -0
- package/assets/init/research/architecture-assessment-rubrics-research.md +566 -0
- package/assets/init/research/reference-architecture-research.md +751 -0
- package/assets/init/standard/EAROS.md +1426 -0
- package/assets/init/standard/schemas/artifact.schema.json +1295 -0
- package/assets/init/standard/schemas/artifact.uischema.json +65 -0
- package/assets/init/standard/schemas/evaluation.schema.json +284 -0
- package/assets/init/standard/schemas/rubric.schema.json +383 -0
- package/assets/init/templates/evaluation-record.template.yaml +58 -0
- package/assets/init/templates/new-profile.template.yaml +65 -0
- package/bin.js +188 -0
- package/dist/assets/_basePickBy-BVu6YmSW.js +1 -0
- package/dist/assets/_baseUniq-CWRzQDz_.js +1 -0
- package/dist/assets/arc-CyDBhtDM.js +1 -0
- package/dist/assets/architectureDiagram-2XIMDMQ5-BH6O4dvN.js +36 -0
- package/dist/assets/blockDiagram-WCTKOSBZ-2xmwdjpg.js +132 -0
- package/dist/assets/c4Diagram-IC4MRINW-BNmPRFJF.js +10 -0
- package/dist/assets/channel-CiySTNoJ.js +1 -0
- package/dist/assets/chunk-4BX2VUAB-DGQTvirp.js +1 -0
- package/dist/assets/chunk-55IACEB6-DNMAQAC_.js +1 -0
- package/dist/assets/chunk-FMBD7UC4-BJbVTQ5o.js +15 -0
- package/dist/assets/chunk-JSJVCQXG-BCxUL74A.js +1 -0
- package/dist/assets/chunk-KX2RTZJC-H7wWZOfz.js +1 -0
- package/dist/assets/chunk-NQ4KR5QH-BK4RlTQF.js +220 -0
- package/dist/assets/chunk-QZHKN3VN-0chxDV5g.js +1 -0
- package/dist/assets/chunk-WL4C6EOR-DexfQ-AV.js +189 -0
- package/dist/assets/classDiagram-VBA2DB6C-D7luWJQn.js +1 -0
- package/dist/assets/classDiagram-v2-RAHNMMFH-D7luWJQn.js +1 -0
- package/dist/assets/clone-ylgRbd3D.js +1 -0
- package/dist/assets/cose-bilkent-S5V4N54A-DS2IOCfZ.js +1 -0
- package/dist/assets/cytoscape.esm-CyJtwmzi.js +331 -0
- package/dist/assets/dagre-KLK3FWXG-BbSoTTa3.js +4 -0
- package/dist/assets/defaultLocale-DX6XiGOO.js +1 -0
- package/dist/assets/diagram-E7M64L7V-C9TvYgv0.js +24 -0
- package/dist/assets/diagram-IFDJBPK2-DowUMWrg.js +43 -0
- package/dist/assets/diagram-P4PSJMXO-BL6nrnQF.js +24 -0
- package/dist/assets/erDiagram-INFDFZHY-rXPRl8VM.js +70 -0
- package/dist/assets/flowDiagram-PKNHOUZH-DBRM99-W.js +162 -0
- package/dist/assets/ganttDiagram-A5KZAMGK-INcWFsBT.js +292 -0
- package/dist/assets/gitGraphDiagram-K3NZZRJ6-DMwpfE91.js +65 -0
- package/dist/assets/graph-DLQn37b-.js +1 -0
- package/dist/assets/index-BFFITMT8.js +650 -0
- package/dist/assets/index-H7f6VTz1.css +1 -0
- package/dist/assets/infoDiagram-LFFYTUFH-B0f4TWRM.js +2 -0
- package/dist/assets/init-Gi6I4Gst.js +1 -0
- package/dist/assets/ishikawaDiagram-PHBUUO56-CsU6XimZ.js +70 -0
- package/dist/assets/journeyDiagram-4ABVD52K-CQ7ibNib.js +139 -0
- package/dist/assets/kanban-definition-K7BYSVSG-DzEN7THt.js +89 -0
- package/dist/assets/katex-B1X10hvy.js +261 -0
- package/dist/assets/layout-C0dvb42R.js +1 -0
- package/dist/assets/linear-j4a8mGj7.js +1 -0
- package/dist/assets/mindmap-definition-YRQLILUH-DP8iEuCf.js +68 -0
- package/dist/assets/ordinal-Cboi1Yqb.js +1 -0
- package/dist/assets/pieDiagram-SKSYHLDU-BpIAXgAm.js +30 -0
- package/dist/assets/quadrantDiagram-337W2JSQ-DrpXn5Eg.js +7 -0
- package/dist/assets/requirementDiagram-Z7DCOOCP-Bg7EwHlG.js +73 -0
- package/dist/assets/sankeyDiagram-WA2Y5GQK-BWagRs1F.js +10 -0
- package/dist/assets/sequenceDiagram-2WXFIKYE-q5jwhivG.js +145 -0
- package/dist/assets/stateDiagram-RAJIS63D-B_J9pE-2.js +1 -0
- package/dist/assets/stateDiagram-v2-FVOUBMTO-Q_1GcybB.js +1 -0
- package/dist/assets/timeline-definition-YZTLITO2-dv0jgQ0z.js +61 -0
- package/dist/assets/treemap-KZPCXAKY-Dt1dkIE7.js +162 -0
- package/dist/assets/vennDiagram-LZ73GAT5-BdO5RgRZ.js +34 -0
- package/dist/assets/xychartDiagram-JWTSCODW-CpDVe-8v.js +7 -0
- package/dist/index.html +23 -0
- package/export-docx.js +1583 -0
- package/init.js +353 -0
- package/manifest-cli.mjs +207 -0
- package/package.json +83 -0
- package/schemas/artifact.schema.json +1295 -0
- package/schemas/artifact.uischema.json +65 -0
- package/schemas/evaluation.schema.json +284 -0
- package/schemas/rubric.schema.json +383 -0
- package/serve.js +238 -0
|
@@ -0,0 +1,186 @@
|
|
|
1
|
+
# Evidence Writing Guide — EAROS Template Fill
|
|
2
|
+
|
|
3
|
+
This file shows how to write architecture content that produces strong EAROS evidence. Read this when helping authors draft specific sections or reviewing content they've provided.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## The Core Principle: Explicit Over Implicit
|
|
8
|
+
|
|
9
|
+
EAROS evaluators score what is stated, not what they can infer. An author who understands this writes explicitly: stating their reasoning, naming their stakeholders, listing their assumptions, and mapping their decisions to drivers.
|
|
10
|
+
|
|
11
|
+
The single most common improvement that raises EAROS scores is replacing implied content with stated content.
|
|
12
|
+
|
|
13
|
+
**Implied (scores 1–2):** "The event-driven architecture enables scalability."
|
|
14
|
+
|
|
15
|
+
**Stated (scores 3–4):** "The event-driven architecture was chosen to address the scalability driver (Driver-3: handle 10x traffic peaks). Alternatives considered: synchronous REST (rejected: cascade failure risk under load), message queue fan-out (rejected: requires managed MQ service not approved in current cloud account)."
|
|
16
|
+
|
|
17
|
+
The content is the same architectural idea — but the explicit version provides evidence anchors an assessor can score against each level descriptor.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Writing Patterns by Section Type
|
|
22
|
+
|
|
23
|
+
### Pattern 1 — The Stakeholder Table
|
|
24
|
+
|
|
25
|
+
**Weak (scores 1):**
|
|
26
|
+
> "Audience: Technical stakeholders and business owners."
|
|
27
|
+
|
|
28
|
+
**Adequate (scores 2):**
|
|
29
|
+
> "Stakeholders: Solution Architect, Platform Team, Security Review Board, Service Owner."
|
|
30
|
+
|
|
31
|
+
**Strong (scores 3–4):**
|
|
32
|
+
|
|
33
|
+
| Stakeholder | Role | Primary Concern | Addressed In |
|
|
34
|
+
|------------|------|----------------|-------------|
|
|
35
|
+
| Solution Architect | Document owner | Completeness, architectural soundness | All sections |
|
|
36
|
+
| Platform Team | Operations | Deployment topology, runbook requirements | Section 5, Appendix B |
|
|
37
|
+
| Security Review Board | Governance | Control compliance, threat model | Section 6, Appendix A |
|
|
38
|
+
| Service Owner | Business | Cost model, SLA commitments | Section 7 |
|
|
39
|
+
|
|
40
|
+
**Why the table works:** It provides direct evidence for STK-01 (named stakeholders with concerns) AND for CVP-01 (views addressing stakeholder concerns). One table serves multiple criteria.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
### Pattern 2 — The Scope Block
|
|
45
|
+
|
|
46
|
+
**Weak (scores 0–1):**
|
|
47
|
+
> "This document covers the payments architecture."
|
|
48
|
+
|
|
49
|
+
**Adequate (scores 2):**
|
|
50
|
+
> "In scope: Payments service, API gateway, upstream banking core.
|
|
51
|
+
> Out of scope: authentication, reporting."
|
|
52
|
+
|
|
53
|
+
**Strong (scores 3–4):**
|
|
54
|
+
```
|
|
55
|
+
IN SCOPE:
|
|
56
|
+
- Payments Service (new) — core payment processing logic
|
|
57
|
+
- Notification Service (existing, modified) — payment confirmation events
|
|
58
|
+
- Banking Core API (existing, upstream) — account validation and settlement
|
|
59
|
+
|
|
60
|
+
OUT OF SCOPE:
|
|
61
|
+
- Authentication/Authorization — handled by IAM Platform (see IAM-ARCH-2024-001)
|
|
62
|
+
- Analytics Pipeline — separate initiative (ANALYTICS-2025 roadmap item)
|
|
63
|
+
- Mobile App — consumer of this service, not modified in this initiative
|
|
64
|
+
|
|
65
|
+
ASSUMPTIONS:
|
|
66
|
+
- Banking Core API contract is stable for 12 months (contact: payments-arch@company.com)
|
|
67
|
+
- Mobile app team provides test harness for integration testing by Q2 2026
|
|
68
|
+
- Existing AWS EU-West-1 account remains the deployment target
|
|
69
|
+
|
|
70
|
+
CONSTRAINTS:
|
|
71
|
+
- No new PII data stores — any personal data must flow through the approved Data Platform
|
|
72
|
+
- GDPR data residency requirements apply — all processing within EU
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
**Why this works:** Each section maps directly to what SCP-01 requires. An assessor can find evidence for each level descriptor immediately. Score 3 requires all four blocks present; score 4 adds consistency verification (e.g., scope boundary tested across all views).
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
### Pattern 3 — The Decision Record
|
|
80
|
+
|
|
81
|
+
**Weak (scores 1):**
|
|
82
|
+
> "We chose event-driven architecture for scalability."
|
|
83
|
+
|
|
84
|
+
**Adequate (scores 2):**
|
|
85
|
+
> "Event-driven architecture was chosen because it enables decoupling between producers and consumers, supporting the scalability requirements."
|
|
86
|
+
|
|
87
|
+
**Strong (scores 3–4):**
|
|
88
|
+
```
|
|
89
|
+
Decision: Adopt event-driven architecture using Apache Kafka for inter-service communication
|
|
90
|
+
|
|
91
|
+
Context: Payment volume is expected to scale 10x during peak periods (Driver-3). The current
|
|
92
|
+
synchronous REST integration pattern creates cascade failures when Banking Core API
|
|
93
|
+
latency increases (observed in P95 incident Aug 2025, Incident-2025-0143).
|
|
94
|
+
|
|
95
|
+
Options considered:
|
|
96
|
+
A. Synchronous REST (rejected): cascade failure risk, confirmed by incident analysis
|
|
97
|
+
B. Message queue fan-out (rejected): requires managed MQ service not in approved catalog
|
|
98
|
+
C. Event-driven Kafka (selected): approved platform service, proven at 2x current volume
|
|
99
|
+
|
|
100
|
+
Rationale: Option C addresses the scalability driver without introducing unapproved
|
|
101
|
+
dependencies. Operational overhead (Kafka expertise) accepted — Platform Team
|
|
102
|
+
confirmed capability.
|
|
103
|
+
|
|
104
|
+
Revisit trigger: If Kafka proves operationally burdensome by Q3 2026 review, re-evaluate B.
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**Why this works:** Provides TRC-01 evidence (link to driver), RAT-01 evidence (trade-offs considered), and ACT-01 evidence (revisit condition named). One decision record serves three criteria.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
### Pattern 4 — The Risk Table
|
|
112
|
+
|
|
113
|
+
**Absent (scores 0):**
|
|
114
|
+
> "Risks: TBD"
|
|
115
|
+
|
|
116
|
+
**Weak (scores 1):**
|
|
117
|
+
> "Risks: Performance, security, integration."
|
|
118
|
+
|
|
119
|
+
**Adequate (scores 2):**
|
|
120
|
+
|
|
121
|
+
| Risk | Likelihood | Impact | Mitigation |
|
|
122
|
+
|------|-----------|--------|------------|
|
|
123
|
+
| API latency degradation | Medium | High | Circuit breaker pattern |
|
|
124
|
+
| Data loss on Kafka failure | Low | Critical | Persistent disk, replication factor 3 |
|
|
125
|
+
|
|
126
|
+
**Strong (scores 3–4):**
|
|
127
|
+
|
|
128
|
+
| Risk | Likelihood | Impact | Mitigation | Owner | Residual Risk |
|
|
129
|
+
|------|-----------|--------|------------|-------|--------------|
|
|
130
|
+
| Banking Core API SLA breach | Medium | High | Circuit breaker + fallback to cached data; see RUNBOOK-PAY-003 | Platform Eng | Low — fallback tested to 15min outage |
|
|
131
|
+
| Kafka consumer lag during peak | Medium | Medium | Auto-scaling consumer group; alert at 5min lag | Payments Eng | Medium — depends on Auto-scaling SLA |
|
|
132
|
+
| PII data in event payload | Low | Critical | Event schema validation gate; DLP scanning on all topics | Security | Low — schema registry prevents unknown fields |
|
|
133
|
+
|
|
134
|
+
**The most commonly missing columns:** Residual Risk and Owner. "Mitigation: TBD" or "Owner: TBD" caps RAT-01 at score 2.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
### Pattern 5 — The Compliance Section
|
|
139
|
+
|
|
140
|
+
**The most common failure mode (scores 0):**
|
|
141
|
+
> "The solution will comply with all applicable security and regulatory standards."
|
|
142
|
+
|
|
143
|
+
**Marginal (scores 1):**
|
|
144
|
+
> "The solution addresses GDPR and ISO 27001 requirements."
|
|
145
|
+
|
|
146
|
+
**Adequate (scores 2):**
|
|
147
|
+
> "GDPR controls applied: data minimization (only payment reference stored, not card number), right to erasure implemented via Payments Data API. ISO 27001: encryption at rest and in transit implemented."
|
|
148
|
+
|
|
149
|
+
**Strong (scores 3–4):**
|
|
150
|
+
|
|
151
|
+
| Control | Standard | How Addressed | Evidence |
|
|
152
|
+
|---------|----------|--------------|---------|
|
|
153
|
+
| Data minimization | GDPR Art. 5(1)(c) | Card numbers never stored — only payment tokens via PCI-DSS vault | Section 4.2, Data Flow Diagram |
|
|
154
|
+
| Encryption at rest | ISO 27001 A.10.1 | AES-256 on all storage; key rotation quarterly | Section 5.3 |
|
|
155
|
+
| Access control | Enterprise Security Baseline v3 | RBAC via IAM Platform; no direct DB access | Section 5.4 |
|
|
156
|
+
| PCI DSS SAQ-A | PCI DSS 3.2.1 | Delegated to payment gateway (Stripe); scope reduction confirmed by Security Review 2025-Q3 | Appendix A |
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## The "Explicit Over Implicit" Checklist
|
|
161
|
+
|
|
162
|
+
Use this to review any section before submission:
|
|
163
|
+
|
|
164
|
+
- [ ] Are stakeholders **named** (not "technical teams") with **specific concerns**?
|
|
165
|
+
- [ ] Is scope **listed** (not described) with explicit in-scope AND out-of-scope?
|
|
166
|
+
- [ ] Are assumptions **stated** (not implied from the design choices)?
|
|
167
|
+
- [ ] Are drivers **referenced by name** in decision rationale (not just alluded to)?
|
|
168
|
+
- [ ] Are risks **in a table** with mitigations AND owners AND residual risk?
|
|
169
|
+
- [ ] Is compliance **mapped to specific named controls** (not stated as assertion)?
|
|
170
|
+
- [ ] Are component names **consistent** between text and all diagrams?
|
|
171
|
+
- [ ] Does each diagram have a **legend** or annotation explaining its notation?
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Common Writing Anti-Patterns
|
|
176
|
+
|
|
177
|
+
| Anti-Pattern | EAROS Problem | Fix |
|
|
178
|
+
|-------------|--------------|-----|
|
|
179
|
+
| "The architecture follows best practices" | Assertion without evidence; CMP-01 = 0 | Name the specific practices and how they're applied |
|
|
180
|
+
| "Risks will be managed" | Not a risk statement; RAT-01 = 0–1 | Add: what risk, likelihood, impact, mitigation, owner |
|
|
181
|
+
| "See attached diagram" | Diagram without narrative; CVP-01 = 1 | Add: what the diagram shows, what to look for, what boundaries mean |
|
|
182
|
+
| "Compliant with enterprise standards" | No named standard; CMP-01 = 0 | Name each: GDPR, ISO 27001, PCI-DSS, etc. |
|
|
183
|
+
| "To be determined" in gate criterion section | Immediate gate risk | These must be resolved before submission |
|
|
184
|
+
| Generic stakeholders | STK-01 = 1 | Name roles with specific concerns |
|
|
185
|
+
| Scope as a paragraph | Hard to extract in/out/assumptions; SCP-01 = 2 | Use structured lists or tables |
|
|
186
|
+
| One diagram only | CVP-01 = 1 | Add context, deployment, and data flow views |
|
|
@@ -0,0 +1,200 @@
|
|
|
1
|
+
# Section-to-Rubric Mapping — EAROS Template Fill
|
|
2
|
+
|
|
3
|
+
This file maps common architecture document sections to the EAROS criteria they address. Use it when walking authors through their document or checking completeness.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Why This Mapping Matters
|
|
8
|
+
|
|
9
|
+
Authors structure documents for readability; EAROS evaluators assess documents for criterion coverage. These don't always align. An author can have a well-structured document that misses 3 core criteria because those concerns were spread across sections without explicit treatment.
|
|
10
|
+
|
|
11
|
+
This mapping shows where criterion coverage is expected so authors can write explicitly, not just organically.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Core Criteria — Section Mapping and Scoring Boundaries
|
|
16
|
+
|
|
17
|
+
### STK-01 — Stakeholder Identification
|
|
18
|
+
**Criterion question:** Are the decision audience and key stakeholders identified with their primary concerns?
|
|
19
|
+
|
|
20
|
+
**Expected in sections:** Introduction, Purpose, Audience, or dedicated Stakeholders section
|
|
21
|
+
|
|
22
|
+
**What assessors look for:**
|
|
23
|
+
- Named stakeholders (roles, not just "the business")
|
|
24
|
+
- Primary concern mapped to each stakeholder
|
|
25
|
+
- Document's decision relevance to each stakeholder
|
|
26
|
+
|
|
27
|
+
**Score 2 vs. 3 boundary:** Listed vs. listed-with-concerns-mapped
|
|
28
|
+
|
|
29
|
+
**Strong (score 3–4):**
|
|
30
|
+
> | Stakeholder | Role | Primary Concern | Addressed In |
|
|
31
|
+
> |------------|------|----------------|-------------|
|
|
32
|
+
> | Solution Architect | Document owner | Design completeness, soundness | All sections |
|
|
33
|
+
> | Platform Team | Operations | Deployment topology, runbook completeness | Section 5, Appendix B |
|
|
34
|
+
> | Security Review Board | Governance | Control compliance, threat model | Section 6, Appendix A |
|
|
35
|
+
> | Service Owner | Business | Cost model, SLA commitments | Section 7 |
|
|
36
|
+
|
|
37
|
+
**Weak (score 1):**
|
|
38
|
+
> "Audience: Technical teams and business stakeholders."
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
### SCP-01 — Scope and Boundaries ⚠️ GATE (major or critical depending on profile)
|
|
43
|
+
**Criterion question:** Are scope, out-of-scope elements, constraints, and assumptions explicitly stated?
|
|
44
|
+
|
|
45
|
+
**Expected in sections:** Scope, Boundaries, Constraints and Assumptions, or Introduction
|
|
46
|
+
|
|
47
|
+
**What assessors look for:**
|
|
48
|
+
- Explicit in-scope list (named components/systems)
|
|
49
|
+
- Explicit out-of-scope list (what isn't covered and why)
|
|
50
|
+
- Stated assumptions (especially ones that affect the design)
|
|
51
|
+
- Constraints (regulatory, technical, organizational)
|
|
52
|
+
|
|
53
|
+
**Score 2 vs. 3 boundary:** Scope defined vs. scope + exclusions + assumptions all stated
|
|
54
|
+
|
|
55
|
+
**Strong (score 3–4):**
|
|
56
|
+
> IN SCOPE: Payments service, Notification service, upstream Banking Core API
|
|
57
|
+
> OUT OF SCOPE: Authentication (handled by IAM platform — see IAM-2024-001), analytics pipeline
|
|
58
|
+
> ASSUMPTIONS: Banking Core API versioned contract stable for 12 months
|
|
59
|
+
> CONSTRAINTS: Must operate within existing AWS EU-West-1 account; GDPR data residency applies
|
|
60
|
+
|
|
61
|
+
**Weak (score 0–1):**
|
|
62
|
+
> "This document covers the payments service architecture."
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
### CVP-01 — Content and Viewpoints
|
|
67
|
+
**Criterion question:** Are the chosen views appropriate for the stated stakeholders and decision purpose?
|
|
68
|
+
|
|
69
|
+
**Expected in sections:** Architecture Views, Solution Overview, or any section with diagrams
|
|
70
|
+
|
|
71
|
+
**What assessors look for:**
|
|
72
|
+
- Multiple views (context, component, deployment, data flow — not just one diagram)
|
|
73
|
+
- Connection between views and stakeholder concerns
|
|
74
|
+
- Annotated diagrams with legends
|
|
75
|
+
|
|
76
|
+
**Score 2 vs. 3 boundary:** Multiple views present vs. views explicitly mapped to stakeholder concerns
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
### TRC-01 — Traceability
|
|
81
|
+
**Criterion question:** Are architecture decisions traceable to business drivers or requirements?
|
|
82
|
+
|
|
83
|
+
**Expected in sections:** Architecture Decisions, Design Rationale, Decision Log
|
|
84
|
+
|
|
85
|
+
**What assessors look for:**
|
|
86
|
+
- Explicit links from decisions to the business drivers that motivated them
|
|
87
|
+
- Decision record format: context → options → decision → rationale
|
|
88
|
+
- References back to requirement IDs or named principles
|
|
89
|
+
|
|
90
|
+
**Score 2 vs. 3 boundary:** Decisions exist vs. decisions with explicit driver references
|
|
91
|
+
|
|
92
|
+
**Strong (score 3):**
|
|
93
|
+
> "Decision: Adopt event-driven Kafka. Context: scalability driver (Driver-3). Options: REST synchronous (rejected: cascade failure risk), Kafka (selected). Rationale: proven at 2× current volume, approved platform service."
|
|
94
|
+
|
|
95
|
+
**Weak (score 1):**
|
|
96
|
+
> "We chose event-driven architecture for scalability."
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
### CON-01 — Internal Consistency
|
|
101
|
+
**Criterion question:** Is terminology and component naming consistent across all sections and diagrams?
|
|
102
|
+
|
|
103
|
+
**Expected everywhere:** Checked across all sections and diagrams (not a specific section)
|
|
104
|
+
|
|
105
|
+
**What assessors look for:**
|
|
106
|
+
- Same name for the same component in all diagrams and text
|
|
107
|
+
- Scope boundary consistent between all views
|
|
108
|
+
- API contracts consistent between description and sequence diagrams
|
|
109
|
+
|
|
110
|
+
**Score 2 vs. 3 boundary:** Minor inconsistencies vs. fully consistent with a glossary
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
### RAT-01 — Risk and Assumptions ⚠️ GATE (major in most profiles)
|
|
115
|
+
**Criterion question:** Are risks, assumptions, and constraints identified with mitigations and owners?
|
|
116
|
+
|
|
117
|
+
**Expected in sections:** Risks, RAID Log, Assumptions, or Risk Register
|
|
118
|
+
|
|
119
|
+
**What assessors look for:**
|
|
120
|
+
- Risk register with all columns: Risk, Likelihood, Impact, Mitigation, Owner, Residual Risk
|
|
121
|
+
- Architectural trade-offs explicitly named
|
|
122
|
+
- Open questions flagged with planned resolution
|
|
123
|
+
|
|
124
|
+
**Score 2 vs. 3 boundary:** Risks listed vs. risks with mitigations AND owners
|
|
125
|
+
|
|
126
|
+
**Strong (score 3–4):**
|
|
127
|
+
> | Risk | Likelihood | Impact | Mitigation | Owner | Residual |
|
|
128
|
+
> |------|-----------|--------|------------|-------|---------|
|
|
129
|
+
> | Banking Core API SLA breach | Medium | High | Circuit breaker + fallback to cached data | Platform Eng | Low |
|
|
130
|
+
|
|
131
|
+
**Weak (score 0–1):**
|
|
132
|
+
> "Risks: Performance, security, integration issues."
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
### CMP-01 — Compliance and Standards ⚠️ GATE (critical in many profiles)
|
|
137
|
+
**Criterion question:** Does the design address applicable compliance frameworks and enterprise standards?
|
|
138
|
+
|
|
139
|
+
**Expected in sections:** Compliance, Security, Standards References, or Architecture Decisions
|
|
140
|
+
|
|
141
|
+
**What assessors look for:**
|
|
142
|
+
- Named standards (not "industry standards" — specific names like GDPR, ISO 27001, PCI-DSS)
|
|
143
|
+
- Specific controls mapped to specific design elements
|
|
144
|
+
- Named exceptions with approval path
|
|
145
|
+
|
|
146
|
+
**Score 2 vs. 3 boundary:** Standards mentioned vs. specific controls mapped to design
|
|
147
|
+
|
|
148
|
+
**The critical anti-pattern** (scores 0):
|
|
149
|
+
> "The solution will comply with all applicable security and regulatory standards."
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
### ACT-01 — Actions and Decisions
|
|
154
|
+
**Criterion question:** Are the key decisions and required actions clearly stated with owners?
|
|
155
|
+
|
|
156
|
+
**Expected in sections:** Decision, Recommendations, Next Steps, or Action Log
|
|
157
|
+
|
|
158
|
+
**What assessors look for:**
|
|
159
|
+
- Clear decision statement (what was decided, not just what was considered)
|
|
160
|
+
- Named actions with owners and target dates
|
|
161
|
+
- Decision authority identified
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
### MNT-01 — Maintainability and Ownership
|
|
166
|
+
**Criterion question:** Is the document owned, versioned, and maintainable?
|
|
167
|
+
|
|
168
|
+
**Expected in sections:** Document Control, cover page, header/footer
|
|
169
|
+
|
|
170
|
+
**What assessors look for:**
|
|
171
|
+
- Named owner (team or role)
|
|
172
|
+
- Version number and date
|
|
173
|
+
- Change history or changelog
|
|
174
|
+
- Review trigger or next review date
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## Gate Summary by Profile
|
|
179
|
+
|
|
180
|
+
| Profile | Critical Gates | Major Gates |
|
|
181
|
+
|---------|---------------|-------------|
|
|
182
|
+
| Solution architecture | CMP-01 | SCP-01, RAT-01 |
|
|
183
|
+
| Reference architecture | None | RA-VIEW-01, RA-IMPL-01 (see profile) |
|
|
184
|
+
| ADR | SCP-01 | CON-01, RAT-01 |
|
|
185
|
+
| Capability map | None | SCP-01, TRC-01 |
|
|
186
|
+
| Roadmap | None | ACT-01, TRC-01 |
|
|
187
|
+
|
|
188
|
+
> Always verify from the loaded profile YAML — this table is indicative only. Gate assignments are defined in the `gate` field of each criterion.
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
## Common Completeness Failures
|
|
193
|
+
|
|
194
|
+
1. **Scope without assumptions** — Section exists but assumptions unstated → SCP-01 capped at 2
|
|
195
|
+
2. **Risks without owners** — Risk table has "Owner: TBD" → RAT-01 capped at 2
|
|
196
|
+
3. **Compliance by assertion** — "The solution will comply with all standards" → CMP-01 = 0
|
|
197
|
+
4. **Single diagram** — One architecture diagram presented as complete → CVP-01 = 1
|
|
198
|
+
5. **Traceability implied** — Decisions made with no reference to drivers → TRC-01 = 1–2
|
|
199
|
+
6. **Generic stakeholders** — "Audience: technical teams" → STK-01 = 1
|
|
200
|
+
7. **Out-of-scope omitted** — Scope section exists but no explicit exclusions → SCP-01 = 2
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: earos-validate
|
|
3
|
+
description: "Run a project health check on the EAROS repository. Validates all YAML rubric files against schemas, checks ID uniqueness, verifies cross-references, detects missing v2 fields, and reports on documentation accuracy. Triggers when the user wants to validate the EAROS repo, check rubric health, run a consistency check, verify schemas, find missing fields, or says \"validate the rubrics\", \"check the EAROS repo\", \"run a health check\", \"check for schema errors\", \"find inconsistencies\", or \"is the rubric set valid\"."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# EAROS Validate — Repository Health Check
|
|
7
|
+
|
|
8
|
+
You run a systematic health check on the EAROS repository. This catches errors that accumulate silently during development: ID conflicts across files, missing required fields added in v2, documentation claims that no longer match the YAML, gate configurations that contradict the status logic.
|
|
9
|
+
|
|
10
|
+
**Why this matters:** A rubric with a duplicate criterion ID will produce ambiguous evaluation records. A profile with a missing `decision_tree` field will calibrate unreliably. Documentation that says "10 criteria" when there are 11 creates confusion for authors and reviewers. These errors compound. A weekly or pre-commit health check prevents this.
|
|
11
|
+
|
|
12
|
+
## What to Load First
|
|
13
|
+
|
|
14
|
+
Read before running any checks:
|
|
15
|
+
1. `earos.manifest.yaml` — the authoritative registry of all rubric files; Check 8 validates it
|
|
16
|
+
2. `standard/schemas/rubric.schema.json` — schema for all rubric/profile/overlay YAML files
|
|
17
|
+
3. `standard/schemas/evaluation.schema.json` — schema for evaluation record files
|
|
18
|
+
|
|
19
|
+
Then read all YAML files in: `core/`, `profiles/`, `overlays/`, `examples/`
|
|
20
|
+
|
|
21
|
+
## The Eight Checks
|
|
22
|
+
|
|
23
|
+
Run all eight. Do not stop at the first error.
|
|
24
|
+
|
|
25
|
+
**Check 1 — Schema conformance**
|
|
26
|
+
For each rubric YAML, verify required top-level fields, `scoring` and `outputs` sub-fields, and kind-specific requirements (profiles must have `inherits`, overlays must not, overlays must use `append_to_base_rubric`).
|
|
27
|
+
|
|
28
|
+
**Check 2 — Criterion v2 field completeness**
|
|
29
|
+
Every criterion must have all 13 v2 fields: `id`, `question`, `description`, `metric_type`, `scale`, `gate`, `required_evidence`, `scoring_guide` (keys "0"–"4"), `anti_patterns`, `examples.good`, `examples.bad`, `decision_tree`, `remediation_hints`.
|
|
30
|
+
|
|
31
|
+
**Check 3 — ID uniqueness**
|
|
32
|
+
Collect all rubric IDs, dimension IDs, and criterion IDs. No duplicates allowed across any files. Criterion ID conflicts across profiles cause ambiguity in evaluation records.
|
|
33
|
+
|
|
34
|
+
**Check 4 — Cross-reference validation**
|
|
35
|
+
Profile `inherits` references must resolve to real rubric IDs. Gate configurations must have valid `severity` values and non-empty `failure_effect`. Dimension weights outside 0.5–2.0 should be flagged.
|
|
36
|
+
|
|
37
|
+
**Check 5 — Evaluation record schema check**
|
|
38
|
+
For each evaluation record in `examples/`: required fields, valid `status` values, valid `judgment_type` and `confidence` values per criterion. Status must match the gate failures and overall score.
|
|
39
|
+
|
|
40
|
+
**Check 6 — Documentation accuracy**
|
|
41
|
+
Check CLAUDE.md claims ("9 dimensions", "10 criteria", profile lists) against actual YAML content. Check README.md profile and overlay lists against actual files.
|
|
42
|
+
|
|
43
|
+
**Check 7 — YAML style conventions**
|
|
44
|
+
Two-space indentation, quoted numeric keys in `scoring_guide`, kebab-case filenames, no version number in filename (version is tracked inside the file only).
|
|
45
|
+
|
|
46
|
+
**Check 8 — Manifest-filesystem consistency**
|
|
47
|
+
Read `earos.manifest.yaml`. For each entry in `core`, `profiles`, and `overlays`:
|
|
48
|
+
- Verify the file exists on disk at the listed path
|
|
49
|
+
- Verify `rubric_id` in the manifest matches `rubric_id` in the file
|
|
50
|
+
- Verify `title` and `artifact_type` are consistent
|
|
51
|
+
|
|
52
|
+
Then verify completeness: every `.yaml` file in `core/`, `profiles/`, and `overlays/` must appear in the manifest. Any file on disk that is absent from the manifest is an ERROR. Any manifest entry whose file is missing is an ERROR.
|
|
53
|
+
|
|
54
|
+
> Read `references/validation-checks.md` for the complete check procedures with exact field paths and error message formats. Read it before running any checks — it contains the precision needed to produce actionable error messages.
|
|
55
|
+
|
|
56
|
+
## Severity Classification
|
|
57
|
+
|
|
58
|
+
| Severity | Meaning |
|
|
59
|
+
|----------|---------|
|
|
60
|
+
| **ERROR** | Missing required field, schema violation, duplicate ID, gate-status contradiction |
|
|
61
|
+
| **WARNING** | Style issue, extreme dimension weight, advisory-level inconsistency |
|
|
62
|
+
|
|
63
|
+
Errors must be fixed before the repository can be used in production. Warnings should be reviewed.
|
|
64
|
+
|
|
65
|
+
## Output Format
|
|
66
|
+
|
|
67
|
+
```markdown
|
|
68
|
+
# EAROS Repository Validation Report
|
|
69
|
+
Date: [today]
|
|
70
|
+
Files checked: [N rubric files] + [N evaluation records]
|
|
71
|
+
|
|
72
|
+
## Summary
|
|
73
|
+
| Check | Errors | Warnings |
|
|
74
|
+
|-------|--------|---------|
|
|
75
|
+
| Schema conformance | [N] | [N] |
|
|
76
|
+
| Criterion v2 completeness | [N] | [N] |
|
|
77
|
+
| ID uniqueness | [N] | [N] |
|
|
78
|
+
| Cross-references | [N] | [N] |
|
|
79
|
+
| Evaluation records | [N] | [N] |
|
|
80
|
+
| Documentation accuracy | [N] | [N] |
|
|
81
|
+
| YAML style | [N] | [N] |
|
|
82
|
+
| Manifest consistency | [N] | [N] |
|
|
83
|
+
| TOTAL | [N] | [N] |
|
|
84
|
+
|
|
85
|
+
Overall health: [Clean / Warnings only / Errors found]
|
|
86
|
+
|
|
87
|
+
## Errors (must fix)
|
|
88
|
+
[FILE] [DESCRIPTION] — each with exact field path and criterion ID where applicable
|
|
89
|
+
|
|
90
|
+
## Warnings (should review)
|
|
91
|
+
[FILE] [DESCRIPTION]
|
|
92
|
+
|
|
93
|
+
## Recommended Actions
|
|
94
|
+
[Numbered list, prioritised by severity]
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
> For common issues and how to fix them, read `references/fix-patterns.md`.
|
|
98
|
+
|
|
99
|
+
## Non-Negotiable Rules
|
|
100
|
+
|
|
101
|
+
1. **Report, don't auto-fix.** Flag problems; do not silently correct them. The user reviews and approves all changes.
|
|
102
|
+
2. **Be precise.** "profiles/reference-architecture.yaml CRITERION RA-VIEW-01 MISSING: decision_tree" is useful. "Some criteria have missing fields" is not.
|
|
103
|
+
3. **Count accurately.** Verify documentation claims against actual YAML — do not rely on memory or prior knowledge.
|
|
104
|
+
4. **Errors vs. warnings.** Missing required fields are errors. Style deviations are warnings. Never downgrade an error to a warning.
|
|
105
|
+
|
|
106
|
+
## When to Read References
|
|
107
|
+
|
|
108
|
+
| When | Read |
|
|
109
|
+
|------|------|
|
|
110
|
+
| Before running any checks | `references/validation-checks.md` |
|
|
111
|
+
| Check field paths for scoring and outputs | `references/validation-checks.md` |
|
|
112
|
+
| After finding errors — how to fix | `references/fix-patterns.md` |
|
|
113
|
+
| User asks how to fix a specific error | `references/fix-patterns.md` |
|