rosett-ai 1.3.3

data/conf/review/design-questionnaire.yml

@@ -0,0 +1,153 @@
+---
+# Design Document Peer Review Questionnaire
+# Version: 1.0.0
+# Used by: rai design peer review process
+#
+# Each question produces a verdict: pass, concern, fail
+# Questions marked severity: critical block sign-off if verdict is fail
+
+questions:
+  Q1:
+    title: Intent Clarity
+    description: |
+      Is the intent specific, actionable, and written in imperative voice?
+      Does it state WHY this document exists, WHAT problem it solves, and
+      WHO benefits?
+    answer_type: verdict
+    severity: critical
+    guidance: |
+      The intent must be imperative ("Establish X", "Define Y", "Enforce Z"),
+      not declarative ("This document describes..."). It should be specific
+      enough that a developer knows what to build without reading the rest
+      of the document.
+
+  Q2:
+    title: Constraint Quality
+    description: |
+      Are constraints testable boundaries (not aspirational)?
+      Could each constraint be verified by a binary pass/fail check?
+    answer_type: verdict
+    severity: critical
+    guidance: |
+      Good: "Never use YAML.load — always YAML.safe_load"
+      Bad: "Should follow best practices for YAML parsing"
+      Each constraint must be enforceable by code, CI, or review.
+      Vague or aspirational language ("strive for", "aim to") fails.
+
+  Q3:
+    title: Criteria Testability
+    description: |
+      Can each acceptance criterion be verified by a concrete test?
+      Are criteria measurable (not process-oriented)?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Good: "RuboCop security cops report zero violations"
+      Bad: "Team follows security best practices"
+      Process criteria ("reviewed by", "documented in") are acceptable
+      only if they specify a verifiable artefact.
+
+  Q4:
+    title: Dependency Soundness
+    description: |
+      Are depends_on entries accurate, complete, and acyclic?
+      Does every listed dependency exist as a design document?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Cross-reference depends_on entries against existing filenames in
+      conf/design/. Missing entries indicate undocumented dependencies.
+      Circular dependencies are always a fail.
+
+  Q5:
+    title: Scope Boundaries
+    description: |
+      Is scope clearly bounded with explicit exclusions?
+      Are overlaps with other design documents resolved?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Look for implicit scope expansion. A security document should not
+      define UI patterns. An engine architecture document should not
+      define compiler internals. Explicit "out of scope" items or
+      constraints that defer to other documents are a good signal.
+
+  Q6:
+    title: Implementation Readiness
+    description: |
+      Is there enough detail for a developer to start implementing?
+      Are examples and anti-patterns provided?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      A document with intent and constraints but no examples is
+      incomplete. Examples should cover happy path and error cases.
+      Anti-patterns should list common wrong approaches. A developer
+      should not need to ask clarifying questions.
+
+  Q7:
+    title: Security Alignment
+    description: |
+      Does the document align with security.yml constraints?
+      Are security implications explicitly addressed?
+    answer_type: verdict
+    severity: critical
+    guidance: |
+      Every document that touches file I/O, user input, shell commands,
+      or external data must reference security.yml constraints. Documents
+      that have no security surface should state so explicitly.
+
+  Q8:
+    title: Engine Agnosticism
+    description: |
+      Is the document engine-agnostic where appropriate?
+      Are engine-specific details properly isolated?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Core and compiler documents must not assume a specific AI engine.
+      Engine-specific details belong in engine architecture or engine-
+      specific documents. References to "Claude" in generic documents
+      should use "engine" or "AI tool" instead.
+
+  Q9:
+    title: Convention Compliance
+    description: |
+      Does the document follow schema conventions? Version pattern
+      consistent? Fields used consistently across documents?
+    answer_type: verdict
+    severity: minor
+    guidance: |
+      Draft = 0.x.0, Approved = 1.x.0+. Required fields present.
+      snake_case name matches filename. Domain is valid enum value.
+      Optional fields (examples, anti_patterns, preferences) used
+      where they add value.
+
+  Q10:
+    title: Completeness
+    description: |
+      Are all relevant fields populated? Do examples cover both
+      happy path and error cases?
+    answer_type: verdict
+    severity: minor
+    guidance: |
+      Check for missing optional sections that would improve clarity.
+      Examples should have scenario + expected + not (anti-example).
+      Anti-patterns section should exist for any document with
+      non-obvious wrong approaches.
+
+verdicts:
+  pass: "Question answered satisfactorily, no concerns"
+  concern: "Minor observation, acceptable but noted"
+  fail: "Issue found that requires attention"
+
+file_verdicts:
+  approved: "All questions pass or concern, no critical/major fails"
+  approved_with_notes: "Minor concerns recorded, acceptable as-is"
+  needs_fix: "One or more findings require document changes"
+  rejected: "Fundamental issues, document needs rewriting"
+
+finding_severities:
+  critical: "Blocks sign-off, must be resolved"
+  major: "Should be resolved, can be deferred with justification"
+  minor: "Noted for improvement, does not block sign-off"

data/conf/review/questionnaire.yml

@@ -0,0 +1,146 @@
+---
+# Test Peer Review Questionnaire Template
+# Version: 1.0.0
+# Used by: Rosett-AI test peer review process (see conf/design/test_peer_review.yml)
+#
+# Each question produces a verdict: pass, concern, fail
+# Questions marked severity: critical block sign-off if verdict is fail
+
+questions:
+  Q1:
+    title: Intent Clarity
+    description: |
+      Does the spec file have a clear, documented purpose?
+      Can you explain what it tests in one sentence?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Look at the top-level describe block and any comments. The file
+      should test one class or module. The test names should make the
+      purpose obvious without reading the implementation.
+
+  Q2:
+    title: Assertion Quality
+    description: |
+      Do assertions use concrete expected values (not computed from
+      the implementation)? Are matchers appropriate and specific?
+    answer_type: verdict
+    severity: critical
+    guidance: |
+      Watch for: `expect(result).to eq(input.map(&:name))` — this
+      mirrors the implementation. Should be: `expect(result).to eq(['alice', 'bob'])`.
+      Also check for overly loose matchers like `be_truthy` where
+      `be true` is appropriate.
+
+  Q3:
+    title: Edge Case Coverage
+    description: |
+      Are empty inputs, nil values, boundary conditions, and error
+      paths tested?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Every public method should have tests for: empty/nil input,
+      single element, maximum/boundary values, invalid types, and
+      expected error classes with messages.
+
+  Q4:
+    title: Mock Appropriateness
+    description: |
+      Are mocks only used at boundaries (IO, network, shell)?
+      Is the unit under test never mocked?
+    answer_type: verdict
+    severity: critical
+    guidance: |
+      Mocking the class under test is always wrong. Mocking internal
+      collaborators indicates tight coupling. Only mock: file system,
+      network calls, shell commands, external services.
+
+  Q5:
+    title: Implementation Independence
+    description: |
+      Would tests still pass if the implementation were completely
+      rewritten with the same public API?
+    answer_type: verdict
+    severity: critical
+    guidance: |
+      Tests that check internal state, private method calls, or
+      instance variable values are coupled to implementation. Tests
+      should only verify observable behaviour through public methods.
+
+  Q6:
+    title: Mutation Resilience
+    description: |
+      Are assertions specific enough that mutating the implementation
+      would cause test failures?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Look for tests that would still pass if: return values were
+      swapped, conditions were inverted, loops were removed, or
+      constants were changed. Each assertion should catch at least
+      one category of mutation.
+
+  Q7:
+    title: Naming Quality
+    description: |
+      Do describe/context/it blocks read as behaviour specifications,
+      not implementation descriptions?
+    answer_type: verdict
+    severity: minor
+    guidance: |
+      Good: `it 'returns empty array when no behaviours are enabled'`
+      Bad: `it 'calls select on the array and filters by enabled flag'`
+      Names should describe WHAT not HOW.
+
+  Q8:
+    title: Fixture Quality
+    description: |
+      Are test fixtures representative, minimal, and free of
+      sensitive data?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Fixtures should be the minimum needed to exercise the behaviour.
+      No real credentials, API keys, or personal data. YAML fixtures
+      must use YAML.safe_load in tests.
+
+  Q9:
+    title: Completeness
+    description: |
+      Are all public methods of the class under test covered?
+      Are both happy and unhappy paths tested?
+    answer_type: verdict
+    severity: major
+    guidance: |
+      Cross-reference the class under test with the spec. Every
+      public method should have at least one test. Missing methods
+      should be listed as findings.
+
+  Q10:
+    title: Documentation
+    description: |
+      Is the test's intent documented? Would a new developer
+      understand why each test exists?
+    answer_type: verdict
+    severity: minor
+    guidance: |
+      Complex test setups should have a brief comment explaining why.
+      Unusual assertions should be explained. The spec file as a whole
+      should be self-documenting through good naming (Q7).
+
+verdicts:
+  pass: "Question answered satisfactorily, no concerns"
+  concern: "Minor observation, acceptable but noted"
+  fail: "Issue found that requires attention"
+
+file_verdicts:
+  approved: "All questions pass or concern, no critical/major fails"
+  approved_with_notes: "Minor concerns recorded, acceptable as-is"
+  needs_fix: "One or more findings require code changes"
+  rejected: "Fundamental issues, test needs rewriting"
+
+finding_severities:
+  critical: "Blocks sign-off, must be resolved"
+  major: "Should be resolved, can be deferred with justification"
+  minor: "Noted for improvement, does not block sign-off"