rosett-ai 1.3.3

data/doc/DESIGN_AUDIT.md

@@ -0,0 +1,235 @@
+# Design Document Audit Report
+
+**Date**: 2026-03-15
+**Scope**: All 36 design documents in `conf/design/`
+**Method**: Schema validation + content analysis of every document
+
+## Inventory
+
+| # | Name | Domain | Version | Status | Constraints | Criteria |
+|---|------|--------|---------|--------|-------------|----------|
+| 1 | aaif_alignment | core | 0.1.0 | draft | 10 | 13 |
+| 2 | accessibility | accessibility | 1.0.0 | approved | 13 | 7 |
+| 3 | ai_authorship | core | 0.1.0 | draft | 10 | 12 |
+| 4 | ai_provenance | security | 0.1.0 | draft | 10 | 12 |
+| 5 | ai_tool_configuration | core | 0.1.0 | draft | 10 | 13 |
+| 6 | architecture | core | 1.0.0 | approved | 6 | 8 |
+| 7 | autocompletion | ui | 0.1.0 | draft | 6 | 8 |
+| 8 | behaviour_composition | compiler | 0.1.0 | draft | 10 | 13 |
+| 9 | ci_pipeline | ci | 1.0.0 | approved | 10 | 12 |
+| 10 | claude_code_configuration | core | 0.1.0 | approved | 21 | 18 |
+| 11 | compiler | compiler | 1.0.0 | approved | 10 | 9 |
+| 12 | comply | security | 0.1.0 | draft | 7 | 7 |
+| 13 | content_packs | licensing | 1.0.0 | approved | 9 | 8 |
+| 14 | desktop_integration | ui | 0.1.0 | draft | 11 | 11 |
+| 15 | distribution | ci | 0.2.0 | draft | 18 | 13 |
+| 16 | documentation | documentation | 0.1.0 | approved | 9 | 10 |
+| 17 | engine_architecture | core | 0.1.0 | draft | 11 | 12 |
+| 18 | gui_plugins | ui | 0.1.0 | draft | 12 | 15 |
+| 19 | i18n | i18n | 1.0.0 | approved | 10 | 8 |
+| 20 | licensing_system | licensing | 1.0.0 | approved | 13 | 9 |
+| 21 | lifecycle_management | core | 1.0.0 | approved | 9 | 7 |
+| 22 | mcp_integration | core | 0.1.0 | draft | 12 | 13 |
+| 23 | mcp_settings | core | 0.1.0 | draft | 7 | 7 |
+| 24 | monitoring_observability | operations | 1.0.0 | draft | 9 | 13 |
+| 25 | policy_management | security | 0.1.0 | draft | 10 | 12 |
+| 26 | project_management | core | 0.1.0 | draft | 6 | 6 |
+| 27 | release_management | release | 0.1.0 | approved | 11 | 10 |
+| 28 | security | security | 1.1.0 | approved | 17 | 12 |
+| 29 | smart_ui_feedback | ui | 0.1.0 | draft | 7 | 7 |
+| 30 | styles | styles | 1.0.0 | approved | 11 | 8 |
+| 31 | test_peer_review | testing | 0.1.0 | draft | 8 | 8 |
+| 32 | testing | testing | 1.0.0 | approved | 11 | 9 |
+| 33 | ui_framework | ui | 1.0.0 | approved | 8 | 8 |
+| 34 | usage_optimization | operations | 1.0.0 | draft | 13 | 11 |
+| 35 | version_management | ci | 1.0.0 | approved | 6 | 6 |
+| 36 | workflow | core | 0.1.0 | draft | 8 | 7 |
+
+**Totals**: 17 approved, 19 draft | 14 unique domains | 362 constraints | 360 criteria
+
+## 1. Consistency
+
+### Version Numbering
+
+Convention: approved = 1.0.0+, draft = 0.x.0.
+
+**Exception**: `claude_code_configuration` is **approved at 0.1.0** — breaks the pattern.
+All other approved docs are 1.0.0 (or `security` at 1.1.0, `distribution` at 0.2.0 draft).
+
+### Status Values
+
+Two-state system: `approved` (17) and `draft` (19). No intermediate states.
+Clean and consistent.
+
+### Domain Taxonomy (14 domains)
+
+| Domain | Count | Examples |
+|--------|-------|---------|
+| core | 11 | architecture, engine_architecture, mcp_integration |
+| ui | 6 | ui_framework, desktop_integration, autocompletion |
+| security | 4 | security, ai_provenance, policy_management, comply |
+| ci | 3 | ci_pipeline, distribution, version_management |
+| testing | 2 | testing, test_peer_review |
+| compiler | 2 | compiler, behaviour_composition |
+| licensing | 2 | licensing_system, content_packs |
+| operations | 2 | monitoring_observability, usage_optimization |
+| accessibility | 1 | accessibility |
+| documentation | 1 | documentation |
+| i18n | 1 | i18n |
+| release | 1 | release_management |
+| styles | 1 | styles |
+
+Flat taxonomy — no hierarchy. `core` dominates (30% of docs).
+
+### Intent Style
+
+Mixed: 22 imperative ("Define...", "Establish..."), 14 declarative ("Developers need...",
+"Software distributed..."). Older approved docs tend imperative; newer drafts mix styles.
+
+### Constraint and Criteria Counts
+
+- **Constraints**: 6-21 per doc (median 10). Variation correlates with complexity.
+- **Criteria**: 6-18 per doc (median 9). Same pattern.
+
+## 2. Contradictions
+
+**None found.** Key points of alignment:
+
+- **Technology stack**: Ruby, Thor, YAML.safe_load, RSpec, Mutant — consistent everywhere
+- **MCP overlap resolved**: `mcp_integration` and `mcp_settings` cross-reference each other
+  via `depends_on`. Settings is narrower (trust model for server config).
+- **Security hierarchy**: `policy_management` explicitly defers to `security` —
+  "Policies must never weaken security constraints from security.yml"
+- **Authorship vs Provenance**: Clean separation documented —
+  provenance = machine data, authorship = human-facing policies
+- **Dependency graph**: DAG with no cycles. `security` is the root.
+
+## 3. Tensions
+
+### Engine-Agnostic vs Claude-Specific
+
+Two overlapping configuration systems:
+
+| Document | Status | Scope |
+|----------|--------|-------|
+| `claude_code_configuration` | approved | Claude Code settings (4 scopes) |
+| `ai_tool_configuration` | draft | Engine-agnostic AI tool config |
+
+**Tension**: `claude_code_configuration` is Claude-specific but approved.
+`ai_tool_configuration` is engine-agnostic but still draft. Migration path unclear.
+
+**Recommendation**: Promote `ai_tool_configuration` to approved and document
+`claude_code_configuration` as the Claude engine's reference implementation,
+or explicitly deprecate it.
+
+### TUI-First vs Desktop Complexity
+
+Core principle: "TUI-first, CLI fully functional without any GUI package installed."
+
+But `desktop_integration` and `gui_plugins` introduce D-Bus, StatusNotifierItem,
+and compositor-specific focus monitors — significant complexity for P3 features.
+
+**Risk**: Desktop features may absorb disproportionate development bandwidth.
+Not a contradiction (D-Bus can run headless), but a scope management concern.
+
+### Offline-First vs API Features
+
+Consistent constraint: "Must work offline" (licensing, provenance, MCP admin, comply).
+API features are opt-in (`--api` flag on adopt, MCP executor). Tension is managed.
+
+## 4. Completeness
+
+### Constraint Quality
+
+**Strong (approved docs)**: Specific, actionable, testable.
+
+- `security`: "Never use YAML.load — always YAML.safe_load with explicit permitted_classes"
+- `testing`: "Mutation score must meet threshold before code can merge"
+- `compiler`: "Compilation of 100 behaviour files completes in under 5 seconds"
+
+**Weaker (some drafts)**: Broad, aspirational.
+
+- `project_management`: "All project operations must work within the .rosett-ai/ directory scope"
+- `workflow`: "Workflows must not access network unless explicitly declared"
+
+### Acceptance Criteria Testability
+
+Most criteria are testable. Draft docs have ~10-20% process/governance criteria
+that are harder to verify programmatically ("Project configuration changes must
+be logged for audit trail" — no format specified).
+
+### Missing Cross-Cutting Concerns
+
+| Missing Topic | Suggested Priority | Notes |
+|---------------|-------------------|-------|
+| Backward compatibility | P2 | Deprecation policy, migration paths, output versioning |
+| Error handling patterns | P1 | Structured errors, exit codes, localized messages |
+| Threat model | P1 | Adversary scenarios, supply chain risk for content packs |
+
+### Gaps Between Documents
+
+- **GUI testing strategy**: `ui_framework` mentions shared_examples but no
+  integration/accessibility test design
+- **Package signing key management**: `distribution` covers GPG-signed packages
+  but no doc on key rotation or signing ceremonies
+- **Multi-user environments**: No doc on namespace collision or permission
+  conflicts when multiple users share a system
+
+## 5. Harmony
+
+### Coherence Assessment
+
+**Yes.** The 36 docs follow a clear layered architecture:
+
+| Layer | Priority | Docs | Purpose |
+|-------|----------|------|---------|
+| Foundation | P1 | 6 | Security, testing, CI, styles, versions |
+| Core Product | P2 | 11 | Engines, compiler, AI integration, compliance |
+| UX Polish | P3 | 8 | Accessibility, i18n, desktop, autocompletion |
+| Monetization | P4 | 2 | Licensing, content packs |
+| Dev Tooling | P5 | 2 | Release management, documentation |
+| Optimization | P6 | 2 | Monitoring, performance |
+
+### Dependency Graph
+
+- `security` is the root — 16 docs depend on it
+- `architecture` is the structural backbone — 12 docs depend on it
+- `engine_architecture` enables the plugin ecosystem — 9 docs depend on it
+- No cycles detected. Clean DAG.
+
+### Implicit Assumptions
+
+These are assumed but not always stated explicitly:
+
+1. **Linux-only** — stated in `architecture`, assumed everywhere
+2. **Debian-first** — .deb packaging, RPM as Phase 2
+3. **Single-user** — XDG paths assume one user per HOME
+4. **Ruby 3.3.10** — scattered references, not centrally declared
+5. **Zeitwerk lazy loading** — mentioned in `usage_optimization`, assumed in code
+
+## Summary
+
+### Strengths
+
+1. **No contradictions** — technology choices align, dependencies are clean
+2. **Strong approved docs** — P1 foundation is detailed, specific, testable
+3. **Coherent layering** — P1-P6 priority structure is logical
+4. **Sound dependency graph** — DAG, no cycles, clear hierarchy
+
+### Issues
+
+| # | Issue | Severity | Action |
+|---|-------|----------|--------|
+| 1 | `claude_code_configuration` approved at 0.1.0 | Low | Bump to 1.0.0 or document exception |
+| 2 | Engine-agnostic vs Claude-specific tension | Medium | Clarify relationship |
+| 3 | 19 draft docs vs 17 approved (backlog) | Low | Expected — drafts are future work |
+| 4 | Intent style inconsistency | Low | Standardize to imperative voice |
+| 5 | Missing cross-cutting docs | Medium | Consider error handling, threat model |
+| 6 | Implicit assumptions not documented | Low | Add to architecture.yml |
+
+### Overall Verdict
+
+**Production-ready design corpus.** The 36 documents form a coherent, well-structured
+architecture with no contradictions and a sound dependency graph. The main risks are
+scope management (19 draft docs) and one unresolved config tension. Recommended
+clarifications are minor — no structural changes needed.