@trohde/earos 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +156 -0
- package/assets/init/.agents/skills/earos-artifact-gen/SKILL.md +106 -0
- package/assets/init/.agents/skills/earos-artifact-gen/references/interview-guide.md +313 -0
- package/assets/init/.agents/skills/earos-artifact-gen/references/output-guide.md +367 -0
- package/assets/init/.agents/skills/earos-assess/SKILL.md +212 -0
- package/assets/init/.agents/skills/earos-assess/references/calibration-benchmarks.md +160 -0
- package/assets/init/.agents/skills/earos-assess/references/output-templates.md +311 -0
- package/assets/init/.agents/skills/earos-assess/references/scoring-protocol.md +281 -0
- package/assets/init/.agents/skills/earos-calibrate/SKILL.md +153 -0
- package/assets/init/.agents/skills/earos-calibrate/references/agreement-metrics.md +188 -0
- package/assets/init/.agents/skills/earos-calibrate/references/calibration-protocol.md +263 -0
- package/assets/init/.agents/skills/earos-create/SKILL.md +257 -0
- package/assets/init/.agents/skills/earos-create/references/criterion-writing-guide.md +268 -0
- package/assets/init/.agents/skills/earos-create/references/dependency-rules.md +193 -0
- package/assets/init/.agents/skills/earos-create/references/rubric-interview-guide.md +123 -0
- package/assets/init/.agents/skills/earos-create/references/validation-checklist.md +238 -0
- package/assets/init/.agents/skills/earos-profile-author/SKILL.md +251 -0
- package/assets/init/.agents/skills/earos-profile-author/references/criterion-writing-guide.md +280 -0
- package/assets/init/.agents/skills/earos-profile-author/references/design-methods.md +158 -0
- package/assets/init/.agents/skills/earos-profile-author/references/profile-checklist.md +173 -0
- package/assets/init/.agents/skills/earos-remediate/SKILL.md +118 -0
- package/assets/init/.agents/skills/earos-remediate/references/output-template.md +199 -0
- package/assets/init/.agents/skills/earos-remediate/references/remediation-patterns.md +330 -0
- package/assets/init/.agents/skills/earos-report/SKILL.md +85 -0
- package/assets/init/.agents/skills/earos-report/references/portfolio-template.md +181 -0
- package/assets/init/.agents/skills/earos-report/references/single-artifact-template.md +168 -0
- package/assets/init/.agents/skills/earos-review/SKILL.md +130 -0
- package/assets/init/.agents/skills/earos-review/references/challenge-patterns.md +163 -0
- package/assets/init/.agents/skills/earos-review/references/output-template.md +180 -0
- package/assets/init/.agents/skills/earos-template-fill/SKILL.md +177 -0
- package/assets/init/.agents/skills/earos-template-fill/references/evidence-writing-guide.md +186 -0
- package/assets/init/.agents/skills/earos-template-fill/references/section-rubric-mapping.md +200 -0
- package/assets/init/.agents/skills/earos-validate/SKILL.md +113 -0
- package/assets/init/.agents/skills/earos-validate/references/fix-patterns.md +281 -0
- package/assets/init/.agents/skills/earos-validate/references/validation-checks.md +287 -0
- package/assets/init/.claude/CLAUDE.md +4 -0
- package/assets/init/AGENTS.md +293 -0
- package/assets/init/CLAUDE.md +635 -0
- package/assets/init/README.md +507 -0
- package/assets/init/calibration/gold-set/.gitkeep +0 -0
- package/assets/init/calibration/results/.gitkeep +0 -0
- package/assets/init/core/core-meta-rubric.yaml +643 -0
- package/assets/init/docs/consistency-report.md +325 -0
- package/assets/init/docs/getting-started.md +194 -0
- package/assets/init/docs/profile-authoring-guide.md +51 -0
- package/assets/init/docs/terminology.md +126 -0
- package/assets/init/earos.manifest.yaml +104 -0
- package/assets/init/evaluations/.gitkeep +0 -0
- package/assets/init/examples/aws-event-driven-order-processing/artifact.yaml +2056 -0
- package/assets/init/examples/aws-event-driven-order-processing/evaluation.yaml +973 -0
- package/assets/init/examples/aws-event-driven-order-processing/report.md +244 -0
- package/assets/init/examples/example-solution-architecture.evaluation.yaml +136 -0
- package/assets/init/examples/multi-cloud-data-analytics/artifact.yaml +715 -0
- package/assets/init/overlays/data-governance.yaml +94 -0
- package/assets/init/overlays/regulatory.yaml +154 -0
- package/assets/init/overlays/security.yaml +92 -0
- package/assets/init/profiles/adr.yaml +225 -0
- package/assets/init/profiles/capability-map.yaml +223 -0
- package/assets/init/profiles/reference-architecture.yaml +426 -0
- package/assets/init/profiles/roadmap.yaml +205 -0
- package/assets/init/profiles/solution-architecture.yaml +227 -0
- package/assets/init/research/architecture-assessment-rubrics-research.docx +0 -0
- package/assets/init/research/architecture-assessment-rubrics-research.md +566 -0
- package/assets/init/research/reference-architecture-research.md +751 -0
- package/assets/init/standard/EAROS.md +1426 -0
- package/assets/init/standard/schemas/artifact.schema.json +1295 -0
- package/assets/init/standard/schemas/artifact.uischema.json +65 -0
- package/assets/init/standard/schemas/evaluation.schema.json +284 -0
- package/assets/init/standard/schemas/rubric.schema.json +383 -0
- package/assets/init/templates/evaluation-record.template.yaml +58 -0
- package/assets/init/templates/new-profile.template.yaml +65 -0
- package/bin.js +188 -0
- package/dist/assets/_basePickBy-BVu6YmSW.js +1 -0
- package/dist/assets/_baseUniq-CWRzQDz_.js +1 -0
- package/dist/assets/arc-CyDBhtDM.js +1 -0
- package/dist/assets/architectureDiagram-2XIMDMQ5-BH6O4dvN.js +36 -0
- package/dist/assets/blockDiagram-WCTKOSBZ-2xmwdjpg.js +132 -0
- package/dist/assets/c4Diagram-IC4MRINW-BNmPRFJF.js +10 -0
- package/dist/assets/channel-CiySTNoJ.js +1 -0
- package/dist/assets/chunk-4BX2VUAB-DGQTvirp.js +1 -0
- package/dist/assets/chunk-55IACEB6-DNMAQAC_.js +1 -0
- package/dist/assets/chunk-FMBD7UC4-BJbVTQ5o.js +15 -0
- package/dist/assets/chunk-JSJVCQXG-BCxUL74A.js +1 -0
- package/dist/assets/chunk-KX2RTZJC-H7wWZOfz.js +1 -0
- package/dist/assets/chunk-NQ4KR5QH-BK4RlTQF.js +220 -0
- package/dist/assets/chunk-QZHKN3VN-0chxDV5g.js +1 -0
- package/dist/assets/chunk-WL4C6EOR-DexfQ-AV.js +189 -0
- package/dist/assets/classDiagram-VBA2DB6C-D7luWJQn.js +1 -0
- package/dist/assets/classDiagram-v2-RAHNMMFH-D7luWJQn.js +1 -0
- package/dist/assets/clone-ylgRbd3D.js +1 -0
- package/dist/assets/cose-bilkent-S5V4N54A-DS2IOCfZ.js +1 -0
- package/dist/assets/cytoscape.esm-CyJtwmzi.js +331 -0
- package/dist/assets/dagre-KLK3FWXG-BbSoTTa3.js +4 -0
- package/dist/assets/defaultLocale-DX6XiGOO.js +1 -0
- package/dist/assets/diagram-E7M64L7V-C9TvYgv0.js +24 -0
- package/dist/assets/diagram-IFDJBPK2-DowUMWrg.js +43 -0
- package/dist/assets/diagram-P4PSJMXO-BL6nrnQF.js +24 -0
- package/dist/assets/erDiagram-INFDFZHY-rXPRl8VM.js +70 -0
- package/dist/assets/flowDiagram-PKNHOUZH-DBRM99-W.js +162 -0
- package/dist/assets/ganttDiagram-A5KZAMGK-INcWFsBT.js +292 -0
- package/dist/assets/gitGraphDiagram-K3NZZRJ6-DMwpfE91.js +65 -0
- package/dist/assets/graph-DLQn37b-.js +1 -0
- package/dist/assets/index-BFFITMT8.js +650 -0
- package/dist/assets/index-H7f6VTz1.css +1 -0
- package/dist/assets/infoDiagram-LFFYTUFH-B0f4TWRM.js +2 -0
- package/dist/assets/init-Gi6I4Gst.js +1 -0
- package/dist/assets/ishikawaDiagram-PHBUUO56-CsU6XimZ.js +70 -0
- package/dist/assets/journeyDiagram-4ABVD52K-CQ7ibNib.js +139 -0
- package/dist/assets/kanban-definition-K7BYSVSG-DzEN7THt.js +89 -0
- package/dist/assets/katex-B1X10hvy.js +261 -0
- package/dist/assets/layout-C0dvb42R.js +1 -0
- package/dist/assets/linear-j4a8mGj7.js +1 -0
- package/dist/assets/mindmap-definition-YRQLILUH-DP8iEuCf.js +68 -0
- package/dist/assets/ordinal-Cboi1Yqb.js +1 -0
- package/dist/assets/pieDiagram-SKSYHLDU-BpIAXgAm.js +30 -0
- package/dist/assets/quadrantDiagram-337W2JSQ-DrpXn5Eg.js +7 -0
- package/dist/assets/requirementDiagram-Z7DCOOCP-Bg7EwHlG.js +73 -0
- package/dist/assets/sankeyDiagram-WA2Y5GQK-BWagRs1F.js +10 -0
- package/dist/assets/sequenceDiagram-2WXFIKYE-q5jwhivG.js +145 -0
- package/dist/assets/stateDiagram-RAJIS63D-B_J9pE-2.js +1 -0
- package/dist/assets/stateDiagram-v2-FVOUBMTO-Q_1GcybB.js +1 -0
- package/dist/assets/timeline-definition-YZTLITO2-dv0jgQ0z.js +61 -0
- package/dist/assets/treemap-KZPCXAKY-Dt1dkIE7.js +162 -0
- package/dist/assets/vennDiagram-LZ73GAT5-BdO5RgRZ.js +34 -0
- package/dist/assets/xychartDiagram-JWTSCODW-CpDVe-8v.js +7 -0
- package/dist/index.html +23 -0
- package/export-docx.js +1583 -0
- package/init.js +353 -0
- package/manifest-cli.mjs +207 -0
- package/package.json +83 -0
- package/schemas/artifact.schema.json +1295 -0
- package/schemas/artifact.uischema.json +65 -0
- package/schemas/evaluation.schema.json +284 -0
- package/schemas/rubric.schema.json +383 -0
- package/serve.js +238 -0
|
@@ -0,0 +1,507 @@
|
|
|
1
|
+
# EaROS — Enterprise Architecture Rubric Operational Standard
|
|
2
|
+
|
|
3
|
+
[](https://creativecommons.org/licenses/by/4.0/)
|
|
4
|
+
[](CHANGELOG.md)
|
|
5
|
+
[](https://github.com/ThomasRohde/EAROS)
|
|
6
|
+
|
|
7
|
+
**Version 2.0.0 · March 2026** · [github.com/ThomasRohde/EAROS](https://github.com/ThomasRohde/EAROS)
|
|
8
|
+
|
|
9
|
+
EaROS is a structured, extensible framework for evaluating enterprise architecture artifacts. It provides a universal rubric foundation, artifact-specific profiles, and cross-cutting overlays that together enable consistent, evidence-anchored assessment — by human reviewers and AI agents alike.
|
|
10
|
+
|
|
11
|
+
<p align="center">
|
|
12
|
+
<img src="images/hero-home.png" width="49%" alt="EaROS Home Screen">
|
|
13
|
+
<img src="images/hero-editor.png" width="49%" alt="EaROS Editor Screen">
|
|
14
|
+
</p>
|
|
15
|
+
|
|
16
|
+
> EaROS is to architecture review what a marking rubric is to an exam: it makes the criteria explicit, the scoring reproducible, and the feedback actionable.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## What is EaROS?
|
|
21
|
+
|
|
22
|
+
Architecture artifacts — solution designs, ADRs, capability maps, reference architectures, roadmaps — are evaluated constantly, but rarely consistently. Different reviewers apply different mental models. Review boards drift. AI-generated assessments hallucinate quality where there is none.
|
|
23
|
+
|
|
24
|
+
EaROS solves this by codifying evaluation criteria into governed, machine-readable rubrics. Each criterion has precise descriptors for every score level, mandatory evidence requirements, and unambiguous pass/fail gates. The result is architecture governance that scales: from a single architect reviewing a colleague's design, to an AI agent running nightly quality checks across hundreds of artifacts.
|
|
25
|
+
|
|
26
|
+
### Design Principles
|
|
27
|
+
|
|
28
|
+
1. **Concern-driven, not document-driven** — assess what matters to stakeholders, not just completeness
|
|
29
|
+
2. **Evidence first** — every score requires a cited excerpt or reference, not an impression
|
|
30
|
+
3. **Gates before averages** — critical failures cannot be hidden by high scores elsewhere
|
|
31
|
+
4. **Explainability over false precision** — ordinal scores with verbal anchors beat decimal averages
|
|
32
|
+
5. **Separate observation from inference** — what the artifact says vs. what it implies are different things
|
|
33
|
+
6. **Rubrics are governed assets** — versioned, owned, calibrated, not ad hoc checklists
|
|
34
|
+
7. **Agentic use must remain auditable** — AI evaluations must cite evidence and flag uncertainty
|
|
35
|
+
8. **Machine-readable where possible** — artifacts in structured formats are assessed more reliably
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## The Three-Layer Model
|
|
40
|
+
|
|
41
|
+
```mermaid
|
|
42
|
+
flowchart TD
|
|
43
|
+
O["<b>OVERLAYS</b> (cross-cutting concerns)<br/>security · data-governance · regulatory"]
|
|
44
|
+
P["<b>PROFILES</b> (artifact-specific extensions)<br/>solution-architecture · reference-architecture · adr · capability-map · roadmap"]
|
|
45
|
+
C["<b>CORE</b> (universal foundation — all artifacts)<br/>core-meta-rubric.yaml<br/>9 dimensions · 0–4 ordinal scale · 3 pass thresholds"]
|
|
46
|
+
|
|
47
|
+
O --- P --- C
|
|
48
|
+
|
|
49
|
+
style O fill:#f3e5f5,stroke:#ab47bc,stroke-width:2px,color:#000
|
|
50
|
+
style P fill:#e3f2fd,stroke:#42a5f5,stroke-width:2px,color:#000
|
|
51
|
+
style C fill:#e8f5e9,stroke:#66bb6a,stroke-width:2px,color:#000
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
**Core** defines the nine dimensions that apply to every architecture artifact: stakeholder fit, scope clarity, concern coverage, traceability, internal consistency, risk coverage, compliance, actionability, and maintainability.
|
|
55
|
+
|
|
56
|
+
**Profiles** extend the core with artifact-specific dimensions. The solution-architecture profile adds optioning and quality-attribute criteria. The reference-architecture profile adds views, prescriptiveness, golden-path, and reusability criteria. Each profile inherits the core and adds 4–12 additional criteria.
|
|
57
|
+
|
|
58
|
+
**Overlays** inject cross-cutting criteria on top of any core+profile combination. Apply the security overlay when reviewing a design that touches authentication or data handling, regardless of its artifact type.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Three Schema Types
|
|
63
|
+
|
|
64
|
+
EaROS uses three distinct JSON Schemas in `standard/schemas/`, forming a deliberate derivation chain:
|
|
65
|
+
|
|
66
|
+
```mermaid
|
|
67
|
+
flowchart LR
|
|
68
|
+
R["<b>rubric.schema.json</b><br/>governs all rubric YAML files (core, profiles, overlays)"]
|
|
69
|
+
A["<b>artifact.schema.json</b><br/>governs architecture artifact documents"]
|
|
70
|
+
E["<b>evaluation.schema.json</b><br/>governs evaluation record output files"]
|
|
71
|
+
|
|
72
|
+
R -->|required_evidence fields drive| A
|
|
73
|
+
A -->|sections map to evidence requirements| E
|
|
74
|
+
|
|
75
|
+
style R fill:#fff3e0,stroke:#ffb74d,stroke-width:2px,color:#000
|
|
76
|
+
style A fill:#e0f7fa,stroke:#4dd0e1,stroke-width:2px,color:#000
|
|
77
|
+
style E fill:#fce4ec,stroke:#f06292,stroke-width:2px,color:#000
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
| Schema | Validates | Kind discriminator |
|
|
81
|
+
|--------|-----------|--------------------|
|
|
82
|
+
| `rubric.schema.json` | Core rubrics, profiles, overlays | `kind: core_rubric`, `profile`, `overlay` |
|
|
83
|
+
| `evaluation.schema.json` | Evaluation records | `kind: evaluation` |
|
|
84
|
+
| `artifact.schema.json` | Architecture artifact documents | `kind: artifact` |
|
|
85
|
+
|
|
86
|
+
A well-completed artifact document satisfies the evidence requirements that rubric criteria require. When a profile adds criteria with new `required_evidence` fields, the artifact schema should be extended to add the corresponding sections. This chain makes EaROS end-to-end: rubric defines what counts as evidence → artifact schema structures how evidence is captured → evaluation schema records how it is scored. The artifact schema is also used by the editor's JSON Forms to render a structured artifact creation form.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Repository Structure
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
EAROS/
|
|
94
|
+
├── earos.manifest.yaml Inventory of all rubric files (single source of truth)
|
|
95
|
+
│
|
|
96
|
+
├── standard/ Standard documents and JSON schemas
|
|
97
|
+
│ ├── EAROS.md The EaROS standard (canonical reference)
|
|
98
|
+
│ ├── EAROS_Standard_v2.docx Word version of the standard
|
|
99
|
+
│ └── schemas/
|
|
100
|
+
│ ├── rubric.schema.json JSON Schema for core rubric / profile / overlay files
|
|
101
|
+
│ ├── evaluation.schema.json JSON Schema for evaluation record files
|
|
102
|
+
│ └── artifact.schema.json JSON Schema for architecture artifact documents
|
|
103
|
+
│ (derived from rubric required_evidence fields)
|
|
104
|
+
│
|
|
105
|
+
├── core/
|
|
106
|
+
│ └── core-meta-rubric.yaml Universal rubric — applies to all artifacts
|
|
107
|
+
│
|
|
108
|
+
├── profiles/ Artifact-specific rubric extensions
|
|
109
|
+
│ ├── solution-architecture.yaml
|
|
110
|
+
│ ├── reference-architecture.yaml
|
|
111
|
+
│ ├── adr.yaml
|
|
112
|
+
│ ├── capability-map.yaml
|
|
113
|
+
│ └── roadmap.yaml
|
|
114
|
+
│
|
|
115
|
+
├── overlays/ Cross-cutting concern injectors
|
|
116
|
+
│ ├── security.yaml
|
|
117
|
+
│ ├── data-governance.yaml
|
|
118
|
+
│ └── regulatory.yaml
|
|
119
|
+
│
|
|
120
|
+
├── templates/ Blank templates for assessors and authors
|
|
121
|
+
│ ├── new-profile.template.yaml Scaffold for creating a new profile
|
|
122
|
+
│ └── evaluation-record.template.yaml Blank evaluation record
|
|
123
|
+
│
|
|
124
|
+
├── tools/
|
|
125
|
+
│ ├── scoring-sheets/ Excel-based tool for manual assessment
|
|
126
|
+
│ │ └── EAROS_Scoring_Sheet_v2.xlsx
|
|
127
|
+
│ ├── validate.py Python schema validation utility
|
|
128
|
+
│ └── editor/ Browser-based YAML editor + CLI (React + JSON Forms + Vite)
|
|
129
|
+
│ ├── bin.js CLI entry point (earos command)
|
|
130
|
+
│ ├── scaffold/ Bundled workspace scaffold (copied by earos init)
|
|
131
|
+
│ │ ├── core/, profiles/, overlays/, templates/, standard/schemas/
|
|
132
|
+
│ │ ├── .agents/skills/ 10 agent skills (agent-agnostic)
|
|
133
|
+
│ │ ├── .claude/CLAUDE.md Claude Code discovery shim (points to AGENTS.md)
|
|
134
|
+
│ │ ├── AGENTS.md Full project guide for AI agents
|
|
135
|
+
│ │ └── earos.manifest.yaml
|
|
136
|
+
│ ├── src/
|
|
137
|
+
│ │ ├── components/ HomeScreen, AssessmentWizard, ArtifactEditor,
|
|
138
|
+
│ │ │ RubricEditor, CriterionScorer, HelpDialog, …
|
|
139
|
+
│ │ └── utils/ schemaLoader, validate, yaml helpers
|
|
140
|
+
│ └── README.md
|
|
141
|
+
│
|
|
142
|
+
├── examples/
|
|
143
|
+
│ └── example-solution-architecture.evaluation.yaml Worked evaluation
|
|
144
|
+
│
|
|
145
|
+
├── calibration/ Calibration infrastructure
|
|
146
|
+
│ ├── gold-set/ Reference artifacts with known scores
|
|
147
|
+
│ └── results/ Calibration run outputs
|
|
148
|
+
│
|
|
149
|
+
├── research/ Research underpinning the standard (63 sources)
|
|
150
|
+
│ ├── architecture-assessment-rubrics-research.md
|
|
151
|
+
│ └── reference-architecture-research.md
|
|
152
|
+
│
|
|
153
|
+
├── presentations/ Slide decks for rollout and training
|
|
154
|
+
│ ├── EAROS_v2_Part1_Overview.pptx
|
|
155
|
+
│ ├── EAROS_v2_Part2_Scoring.pptx
|
|
156
|
+
│ └── EAROS_v2_Part3_Implementation.pptx
|
|
157
|
+
│
|
|
158
|
+
├── docs/ How-to guides
|
|
159
|
+
│ ├── getting-started.md
|
|
160
|
+
│ ├── profile-authoring-guide.md
|
|
161
|
+
│ └── terminology.md Glossary of all EaROS, statistical, and architecture terms
|
|
162
|
+
│
|
|
163
|
+
└── .agents/skills/ Claude Code skills (wraps .agents/skills/ for Claude Code)
|
|
164
|
+
├── earos-assess/
|
|
165
|
+
├── earos-review/
|
|
166
|
+
├── earos-template-fill/
|
|
167
|
+
├── earos-artifact-gen/
|
|
168
|
+
├── earos-create/
|
|
169
|
+
├── earos-profile-author/
|
|
170
|
+
├── earos-calibrate/
|
|
171
|
+
├── earos-report/
|
|
172
|
+
├── earos-validate/
|
|
173
|
+
└── earos-remediate/
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## File Naming Conventions
|
|
179
|
+
|
|
180
|
+
The `kind` field is the universal type discriminator — it determines how a file is interpreted, not its filename suffix or path. Version is tracked inside the file (`version: 2.0.0`), never in the filename.
|
|
181
|
+
|
|
182
|
+
| File type | Pattern | Example |
|
|
183
|
+
|-----------|---------|---------|
|
|
184
|
+
| Rubric definitions (core, profiles, overlays) | `<name>.yaml` | `reference-architecture.yaml` |
|
|
185
|
+
| Evaluation records | `<name>.evaluation.yaml` | `payments-api.evaluation.yaml` |
|
|
186
|
+
| Templates | `<name>.template.yaml` | `evaluation-record.template.yaml` |
|
|
187
|
+
| JSON schemas | `<name>.schema.json` | `rubric.schema.json` |
|
|
188
|
+
|
|
189
|
+
- Kebab-case throughout; no spaces in filenames
|
|
190
|
+
- No version numbers in filenames — use `version:` inside the file
|
|
191
|
+
|
|
192
|
+
---
|
|
193
|
+
|
|
194
|
+
## Scoring Model
|
|
195
|
+
|
|
196
|
+
### 0–4 Ordinal Scale
|
|
197
|
+
|
|
198
|
+
| Score | Label | Meaning |
|
|
199
|
+
|-------|-------|---------|
|
|
200
|
+
| 4 | Strong | Fully addressed, well evidenced, internally consistent, decision-ready |
|
|
201
|
+
| 3 | Good | Clearly addressed with adequate evidence and only minor gaps |
|
|
202
|
+
| 2 | Partial | Explicitly addressed but coverage incomplete, inconsistent, or weakly evidenced |
|
|
203
|
+
| 1 | Weak | Acknowledged or implied, but inadequate for decision support |
|
|
204
|
+
| 0 | Absent | No meaningful evidence, or evidence directly contradicts the criterion |
|
|
205
|
+
| N/A | Not applicable | Criterion genuinely does not apply to this artifact (requires justification) |
|
|
206
|
+
|
|
207
|
+
### Gate Types
|
|
208
|
+
|
|
209
|
+
Gates prevent weak scores being masked by high averages elsewhere.
|
|
210
|
+
|
|
211
|
+
| Gate Type | Effect |
|
|
212
|
+
|-----------|--------|
|
|
213
|
+
| `none` | Contributes to score only; no gate logic |
|
|
214
|
+
| `advisory` | Weak performance triggers a recommendation |
|
|
215
|
+
| `major` | Significant weakness may cap the status (e.g., cannot pass above `conditional_pass`) |
|
|
216
|
+
| `critical` | Failure blocks pass status entirely; triggers `reject` regardless of average |
|
|
217
|
+
|
|
218
|
+
### Status Thresholds
|
|
219
|
+
|
|
220
|
+
| Status | Threshold |
|
|
221
|
+
|--------|-----------|
|
|
222
|
+
| **Pass** | No critical gate failure + overall ≥ 3.2 + no dimension < 2.0 |
|
|
223
|
+
| **Conditional Pass** | No critical gate failure + overall 2.4–3.19 |
|
|
224
|
+
| **Rework Required** | Overall < 2.4, or repeated weak dimensions |
|
|
225
|
+
| **Reject** | Any critical gate failure, or mandatory control breach |
|
|
226
|
+
| **Not Reviewable** | Evidence too incomplete to score responsibly |
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
## Getting Started
|
|
231
|
+
|
|
232
|
+
### Install
|
|
233
|
+
|
|
234
|
+
```bash
|
|
235
|
+
npm install -g @trohde/earos
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
### Scaffold a new workspace
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
earos init my-architecture
|
|
242
|
+
cd my-architecture
|
|
243
|
+
earos
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
`earos init` copies a complete, ready-to-use EaROS workspace into `my-architecture/` — all rubrics, schemas, templates, and agent skills bundled. The browser editor opens automatically at `http://localhost:3000`.
|
|
247
|
+
|
|
248
|
+
---
|
|
249
|
+
|
|
250
|
+
## earos init — Agent-Agnostic Workspace
|
|
251
|
+
|
|
252
|
+
`earos init [dir]` scaffolds a self-contained EaROS workspace that works with any AI coding agent:
|
|
253
|
+
|
|
254
|
+
```
|
|
255
|
+
my-architecture/
|
|
256
|
+
├── earos.manifest.yaml Inventory of all rubric files
|
|
257
|
+
├── AGENTS.md Full project guide — read by Cursor, Copilot, Windsurf, etc.
|
|
258
|
+
├── .claude/CLAUDE.md Thin Claude Code shim — points to AGENTS.md for discovery
|
|
259
|
+
├── .agents/skills/ 10 EaROS skills in agent-agnostic format
|
|
260
|
+
│ ├── earos-assess/
|
|
261
|
+
│ ├── earos-review/
|
|
262
|
+
│ ├── earos-template-fill/
|
|
263
|
+
│ ├── earos-artifact-gen/
|
|
264
|
+
│ ├── earos-create/
|
|
265
|
+
│ ├── earos-profile-author/
|
|
266
|
+
│ ├── earos-calibrate/
|
|
267
|
+
│ ├── earos-report/
|
|
268
|
+
│ ├── earos-validate/
|
|
269
|
+
│ └── earos-remediate/
|
|
270
|
+
├── core/core-meta-rubric.yaml
|
|
271
|
+
├── profiles/ 5 artifact profiles (solution-architecture, reference-architecture,
|
|
272
|
+
│ adr, capability-map, roadmap)
|
|
273
|
+
├── overlays/ 3 cross-cutting overlays (security, data-governance, regulatory)
|
|
274
|
+
├── standard/schemas/ JSON Schemas for rubrics, evaluations, and artifacts
|
|
275
|
+
├── templates/ Blank templates for new rubrics and evaluation records
|
|
276
|
+
├── evaluations/ Your evaluation records go here
|
|
277
|
+
└── calibration/ Calibration artifacts and results
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
**Agent-agnostic design:** Skills live in `.agents/skills/` — a convention understood by Cursor, GitHub Copilot, Windsurf, and other AI coding tools. `AGENTS.md` at the repo root is the primary guide for any agent. Claude Code additionally discovers `.claude/CLAUDE.md`, which is a thin shim that points agents to `AGENTS.md` and `.agents/skills/`. Every skill reads the actual YAML rubric files at runtime, so assessments always use the latest rubric version regardless of which agent runs them.
|
|
281
|
+
|
|
282
|
+
---
|
|
283
|
+
|
|
284
|
+
## CLI Commands
|
|
285
|
+
|
|
286
|
+
```bash
|
|
287
|
+
earos # Open the web editor (http://localhost:3000)
|
|
288
|
+
earos init [dir] # Scaffold a new agent-agnostic EaROS workspace
|
|
289
|
+
earos validate <file> # Validate a rubric/evaluation/artifact YAML against schemas
|
|
290
|
+
earos manifest # Regenerate earos.manifest.yaml from the filesystem
|
|
291
|
+
earos manifest add <path> # Add a single file to the manifest
|
|
292
|
+
earos manifest check # Verify manifest matches filesystem (exits non-zero on drift)
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
After creating a new rubric, always run `earos manifest add <path>` to register it.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## For Governance Teams — Creating and Managing Rubrics
|
|
300
|
+
|
|
301
|
+
1. Run `earos` to open the editor, then select **Create Rubric** or **Edit Rubric**
|
|
302
|
+
2. Use the `earos-create` agent skill to author a new profile or overlay via guided interview
|
|
303
|
+
3. Validate YAML: `earos validate path/to/rubric.yaml`
|
|
304
|
+
4. Add to the manifest: `earos manifest add path/to/rubric.yaml`
|
|
305
|
+
|
|
306
|
+
## For Reviewers — Assessing an Artifact
|
|
307
|
+
|
|
308
|
+
1. **Identify the artifact type** — solution architecture, ADR, capability map, reference architecture, or roadmap
|
|
309
|
+
2. **Select the rubric set:** always start with `core/core-meta-rubric.yaml`, add the matching profile from `profiles/`, add applicable overlays from `overlays/`
|
|
310
|
+
3. **Score using one of:**
|
|
311
|
+
- **Browser editor:** `earos` — guided wizard with criterion-by-criterion scoring
|
|
312
|
+
- **Spreadsheet:** open `tools/scoring-sheets/EAROS_Scoring_Sheet_v2.xlsx`
|
|
313
|
+
- **Agent skill:** use the `earos-assess` skill in your AI coding agent
|
|
314
|
+
4. **Check gates** — any critical gate failure overrides the aggregate score
|
|
315
|
+
5. **Determine status** against the thresholds above
|
|
316
|
+
|
|
317
|
+
See [`docs/getting-started.md`](docs/getting-started.md) for a full walkthrough.
|
|
318
|
+
|
|
319
|
+
## For Architects — Writing an Assessment-Ready Artifact
|
|
320
|
+
|
|
321
|
+
1. Use the `earos-artifact-gen` skill — it interviews you and produces a schema-compliant artifact YAML
|
|
322
|
+
2. Or use the `earos-template-fill` skill — it guides you through writing each section to satisfy rubric evidence requirements
|
|
323
|
+
3. Or open the editor and select **Create Artifact** on the home screen
|
|
324
|
+
4. Validate against `standard/schemas/artifact.schema.json`
|
|
325
|
+
|
|
326
|
+
## AI-Agent Assessment
|
|
327
|
+
|
|
328
|
+
EaROS is designed for automated evaluation. The YAML rubric files are the machine-readable specification; the evaluation record schema defines the output format.
|
|
329
|
+
|
|
330
|
+
**Minimal agent prompt pattern:**
|
|
331
|
+
|
|
332
|
+
```
|
|
333
|
+
You are an architecture quality assessor. Apply the EaROS rubric defined in
|
|
334
|
+
[rubric YAML] to the artifact below. For each criterion:
|
|
335
|
+
1. Extract the relevant evidence from the artifact (direct quote or reference)
|
|
336
|
+
2. Score 0–4 against the level descriptors
|
|
337
|
+
3. If you cannot find evidence, score N/A and explain
|
|
338
|
+
4. Flag any gate criteria that fail
|
|
339
|
+
5. Classify evidence as observed / inferred / external
|
|
340
|
+
6. Report confidence (high/medium/low) separately from the score
|
|
341
|
+
Produce output conforming to evaluation.schema.json.
|
|
342
|
+
|
|
343
|
+
<artifact>
|
|
344
|
+
[artifact content]
|
|
345
|
+
</artifact>
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
The rubric files define an 8-step evaluation DAG:
|
|
349
|
+
|
|
350
|
+
```mermaid
|
|
351
|
+
flowchart LR
|
|
352
|
+
S1[structural<br>validation] --> S2[content<br>extraction]
|
|
353
|
+
S2 --> S3[criterion<br>scoring]
|
|
354
|
+
S3 --> S4[cross_reference<br>validation]
|
|
355
|
+
S4 --> S5[dimension<br>aggregation]
|
|
356
|
+
S5 --> S6[challenge<br>pass]
|
|
357
|
+
S6 --> S7[calibration]
|
|
358
|
+
S7 --> S8[status<br>determination]
|
|
359
|
+
|
|
360
|
+
style S1 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
361
|
+
style S2 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
362
|
+
style S3 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
363
|
+
style S4 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
364
|
+
style S5 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
365
|
+
style S6 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
366
|
+
style S7 fill:#f5f5f5,stroke:#9e9e9e,color:#000
|
|
367
|
+
style S8 fill:#4caf50,stroke:#2e7d32,color:#fff
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
Calibrate your agent against `calibration/gold-set/` before production use. Target inter-rater reliability of Cohen's κ > 0.70.
|
|
371
|
+
|
|
372
|
+
---
|
|
373
|
+
|
|
374
|
+
## Agent Skills
|
|
375
|
+
|
|
376
|
+
EaROS ships 10 agent skills for end-to-end workflows. In scaffolded workspaces (`earos init`) they live in `.agents/skills/` — readable by any AI coding agent (Cursor, Copilot, Windsurf, Claude Code, etc.). In the EaROS development repo they additionally appear as Claude Code skills in `.agents/skills/`.
|
|
377
|
+
|
|
378
|
+
Every skill reads the actual YAML rubric files at runtime — no embedded rubric content — so assessments always use the latest rubric version.
|
|
379
|
+
|
|
380
|
+
| Skill | Purpose |
|
|
381
|
+
|-------|---------|
|
|
382
|
+
| `earos-assess` | Run a full EaROS evaluation on any architecture artifact (8-step DAG, RULERS protocol) |
|
|
383
|
+
| `earos-review` | Challenge an existing evaluation record — check for over-scoring and unsupported claims |
|
|
384
|
+
| `earos-template-fill` | Guide an artifact author through writing an assessment-ready document |
|
|
385
|
+
| `earos-artifact-gen` | Guided interview → produces a schema-compliant artifact YAML document |
|
|
386
|
+
| `earos-create` | Create a new rubric from scratch — profile, overlay, or core rubric |
|
|
387
|
+
| `earos-profile-author` | Technical YAML authoring guide — v2 field structure, schema compliance |
|
|
388
|
+
| `earos-calibrate` | Run calibration exercises and compute inter-rater reliability metrics |
|
|
389
|
+
| `earos-report` | Generate executive reports and portfolio dashboards from evaluation records |
|
|
390
|
+
| `earos-validate` | Health-check the repository — schema validation, ID uniqueness, cross-reference checks |
|
|
391
|
+
| `earos-remediate` | Generate a prioritized improvement plan from an EaROS evaluation record |
|
|
392
|
+
|
|
393
|
+
---
|
|
394
|
+
|
|
395
|
+
## The EaROS Editor
|
|
396
|
+
|
|
397
|
+
A browser-based tool for creating and editing EaROS rubrics, running assessments, and authoring artifact documents. Built with React + JSON Forms + Material UI + Vite.
|
|
398
|
+
|
|
399
|
+
When installed globally (`npm install -g @trohde/earos`), use the `earos` command from any workspace. When working in the development repo directly:
|
|
400
|
+
|
|
401
|
+
```bash
|
|
402
|
+
cd tools/editor
|
|
403
|
+
npm install
|
|
404
|
+
node bin.js # open home screen in browser (http://localhost:3000)
|
|
405
|
+
node bin.js ../../profiles/adr.yaml # open editor with a file pre-loaded
|
|
406
|
+
node bin.js validate path/to/file.yaml # validate a file, exit 0 (valid) / 1 (errors)
|
|
407
|
+
node bin.js manifest # regenerate earos.manifest.yaml from filesystem
|
|
408
|
+
node bin.js manifest add <path> # add a single file to the manifest
|
|
409
|
+
node bin.js manifest check # check manifest matches filesystem; exits non-zero on drift
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
**Home screen — 3×2 card layout:**
|
|
413
|
+
|
|
414
|
+
| Audience | Card 1 | Card 2 |
|
|
415
|
+
|----------|--------|--------|
|
|
416
|
+
| **Governance Teams** | Create Rubric | Edit Rubric |
|
|
417
|
+
| **Reviewers** | New Assessment | Continue Assessment |
|
|
418
|
+
| **Architects** | Create Artifact | Edit Artifact |
|
|
419
|
+
|
|
420
|
+
**Key features:**
|
|
421
|
+
- **Assessment wizard** — guided criterion-by-criterion scoring with evidence capture, gate tracking, and automatic status determination
|
|
422
|
+
- **Artifact editor** — structured document editor driven by `artifact.schema.json`; shows EaROS evidence requirements inline
|
|
423
|
+
- **Rubric editor** — tabbed JSON Forms view: Metadata / Dimensions & Criteria / Scoring & Outputs / Agent & Calibration
|
|
424
|
+
- **Manifest-driven sidebar** — browse and load any rubric file via `earos.manifest.yaml`; no hardcoded paths
|
|
425
|
+
- **Schemas loaded via API** — schemas are served from the canonical `standard/schemas/` paths at runtime
|
|
426
|
+
- **Live YAML preview** — right panel updates in real time; copy-to-clipboard button
|
|
427
|
+
- **Real-time validation** — status bar shows error count and first errors as you type
|
|
428
|
+
|
|
429
|
+
See [`tools/editor/README.md`](tools/editor/README.md) for full documentation.
|
|
430
|
+
|
|
431
|
+
---
|
|
432
|
+
|
|
433
|
+
## Extending EaROS
|
|
434
|
+
|
|
435
|
+
### Creating a New Profile
|
|
436
|
+
|
|
437
|
+
Use [`templates/new-profile.template.yaml`](templates/new-profile.template.yaml) as your scaffold, or run the `earos-create` agent skill for a guided interview. The [`docs/profile-authoring-guide.md`](docs/profile-authoring-guide.md) describes five design methods:
|
|
438
|
+
|
|
439
|
+
- **Method A: Decision-Centred** — for ADRs, investment reviews
|
|
440
|
+
- **Method B: Viewpoint-Centred** — for capability maps, reference architectures
|
|
441
|
+
- **Method C: Lifecycle-Centred** — for transitions, roadmaps, handover docs
|
|
442
|
+
- **Method D: Risk-Centred** — for security, regulatory, resilience architecture
|
|
443
|
+
- **Method E: Pattern-Library** — for recurring platform services
|
|
444
|
+
|
|
445
|
+
Rules: set `kind: profile`, `inherits: [EAROS-CORE-002]`, add 5–12 criteria (the core already has 10), include all required fields for every criterion (`question`, `description`, `scoring_guide`, `required_evidence`, `anti_patterns`, `examples.good`, `examples.bad`, `decision_tree`, `remediation_hints`), validate against `rubric.schema.json`, then add to the manifest.
|
|
446
|
+
|
|
447
|
+
### Creating a New Overlay
|
|
448
|
+
|
|
449
|
+
Use `kind: overlay` and `artifact_type: any`. Overlays use `scoring.method: append_to_base_rubric` — they add criteria on top of any core+profile combination. Apply overlays by context, not artifact type. Overlays are additive and cannot weaken base gates.
|
|
450
|
+
|
|
451
|
+
### Calibrating Before Production
|
|
452
|
+
|
|
453
|
+
1. Score the artifacts in `calibration/gold-set/` independently
|
|
454
|
+
2. Compare against reference scores using `calibration/results/`
|
|
455
|
+
3. Resolve disagreements against the level descriptors
|
|
456
|
+
4. Iterate until κ > 0.70 on well-defined criteria, > 0.50 on subjective ones
|
|
457
|
+
|
|
458
|
+
---
|
|
459
|
+
|
|
460
|
+
## Glossary
|
|
461
|
+
|
|
462
|
+
New to EaROS terminology? The standard uses precise vocabulary from statistics, architecture practice, and the EaROS framework itself. See [`docs/terminology.md`](docs/terminology.md) for definitions of all key terms, organized into three categories:
|
|
463
|
+
|
|
464
|
+
- **Statistical and calibration terms** — Cohen's kappa, weighted kappa, ICC, Spearman's rho, Wasserstein distance, inter-rater reliability, calibration
|
|
465
|
+
- **EaROS-specific terms** — core meta-rubric, profile, overlay, gate, evidence anchor, evidence class, RULERS protocol, DAG evaluation flow, challenge pass, rubric locking, decision tree
|
|
466
|
+
- **Architecture terms** — architecture artifact, viewpoint, quality attribute, quality attribute scenario, fitness function, ADR, golden path, concern
|
|
467
|
+
|
|
468
|
+
---
|
|
469
|
+
|
|
470
|
+
## Standards and Research Foundation
|
|
471
|
+
|
|
472
|
+
EaROS draws on and extends:
|
|
473
|
+
|
|
474
|
+
- **TOGAF** Architecture Content Framework and governance practices
|
|
475
|
+
- **arc42** template structure and quality criteria
|
|
476
|
+
- **C4 Model** viewpoint hierarchy
|
|
477
|
+
- **RULERS** protocol for evidence-anchored rubric scoring
|
|
478
|
+
- **LLM-Rubric** and **AutoRubric** research on AI-agent evaluation reliability
|
|
479
|
+
- AWS/Azure/Google Well-Architected Frameworks
|
|
480
|
+
|
|
481
|
+
Full research documentation is in [`research/`](research/).
|
|
482
|
+
|
|
483
|
+
---
|
|
484
|
+
|
|
485
|
+
## Author
|
|
486
|
+
|
|
487
|
+
Thomas Rohde · [rohde.thomas@gmail.com](mailto:rohde.thomas@gmail.com) · [github.com/ThomasRohde](https://github.com/ThomasRohde)
|
|
488
|
+
|
|
489
|
+
---
|
|
490
|
+
|
|
491
|
+
## Contributing
|
|
492
|
+
|
|
493
|
+
Contributions to profiles, overlays, calibration artifacts, and documentation are welcome. When contributing:
|
|
494
|
+
|
|
495
|
+
- Use `<artifact-type>.yaml` for rubric files (no version number in filename)
|
|
496
|
+
- Validate YAML files against the JSON schemas in `standard/schemas/`
|
|
497
|
+
- Include level descriptors (0–4) and evidence requirements for every new criterion
|
|
498
|
+
- Add worked examples to `examples/` when introducing new profiles
|
|
499
|
+
- Document changes in `CHANGELOG.md`
|
|
500
|
+
|
|
501
|
+
---
|
|
502
|
+
|
|
503
|
+
## License
|
|
504
|
+
|
|
505
|
+
This work is licensed under the [Creative Commons Attribution 4.0 International License](LICENSE) (CC BY 4.0).
|
|
506
|
+
|
|
507
|
+
You are free to use, adapt, and build upon EaROS for any purpose, including commercial, provided you give appropriate credit.
|
|
File without changes
|
|
File without changes
|