rosett-ai 1.3.3

data/doc/ENGINES.md

@@ -0,0 +1,567 @@
+# Engines
+
+raictl uses a pluggable engine architecture. Each engine translates generic YAML
+rules into a specific AI tool's native format.
+
+Engines are distributed as **external gems** (`rosett-ai-engine-<name>`) that
+self-register with rosett-ai core's plugin registry at load time. Core provides
+the base classes and contracts; each gem owns its backend, detector, executor,
+config compiler, manifests, schemas, and specs.
+
+## Plugin Discovery
+
+```mermaid
+graph TB
+    subgraph Core["rosett-ai core"]
+        REG["RosettAi::Plugins::Registry"]
+        DISC["discover! → Gem.find_files('rosett_ai_engine/*/register.rb')"]
+        CONTRACT["RosettAi::Plugins::EngineContract"]
+        BASE["RosettAi::Engines::BaseConfigCompiler"]
+    end
+
+    subgraph Gems["External Engine Gems"]
+        GEM_C["rosett-ai-engine-claude<br/>lib/rosett_ai_engine/claude/register.rb"]
+        GEM_O["rosett-ai-engine-*<br/>lib/rosett_ai_engine/*/register.rb"]
+    end
+
+    subgraph Engines["Discovered Engine Gems"]
+        CLAUDE["rosett-ai-engine-claude"]
+        GENERIC["rosett-ai-engine-generic"]
+        AGENTS["rosett-ai-engine-agents-md"]
+        CURSOR["rosett-ai-engine-cursor"]
+        COPILOT["rosett-ai-engine-copilot"]
+        WINDSURF["rosett-ai-engine-windsurf"]
+        GOOSE["rosett-ai-engine-goose"]
+        AIDER["rosett-ai-engine-aider"]
+        OLLAMA["rosett-ai-engine-ollama"]
+        NEOX["rosett-ai-engine-gpt-neox"]
+        ACME["rosett-ai-engine-acme (example)"]
+    end
+
+    DISC --> GEM_C
+    DISC --> GEM_O
+    GEM_C -->|register :engine| REG
+    GEM_O -->|register :engine| REG
+    REG --> Engines
+    CONTRACT -.->|implements| GEM_C
+    BASE -.->|inherits| GEM_C
+```
+
+## Plugin Types
+
+raictl supports three plugin contract types:
+
+| Type | Contract Module | Discovery Pattern | Gem Prefix |
+|------|----------------|-------------------|------------|
+| Engine | `RosettAi::Plugins::EngineContract` | `rosett_ai_engine/*/register.rb` | `rosett-ai-engine-` |
+| GUI | `RosettAi::Plugins::GuiContract` | `rosett_ai_gui/*/register.rb` | `rosett-ai-gui-` |
+| MCP | `RosettAi::Plugins::McpContract` | `rosett_ai_mcp/*/register.rb` | `rosett-ai-mcp-` |
+
+## Engine Registry
+
+All engines are external gems discovered dynamically by
+`RosettAi::Plugins::Registry.discover!` via `Gem.find_files`. Core contains
+no built-in engine code — only base classes (`RosettAi::Compiler::Backend`,
+`RosettAi::Engines::Detector`, `RosettAi::Engines::BaseConfigCompiler`) and
+deprecated constant aliases for backward compatibility.
+
+## Engine Comparison
+
+| Feature | Claude | Cursor | Copilot | Windsurf | AGENTS.md | Goose | Aider | Generic | Ollama | GPT-NeoX |
+|---------|--------|--------|---------|----------|-----------|-------|-------|---------|--------|----------|
+| Backend | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
+| Detector | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -- | Yes | Yes |
+| Executor | Yes | -- | -- | -- | -- | Yes (CLI) | Yes (CLI) | -- | Yes | Yes |
+| Config Compiler | Yes | -- | -- | -- | -- | -- | -- | -- | -- | -- |
+| Key Map | Yes | -- | -- | -- | -- | -- | -- | -- | -- | -- |
+| Scope Router | Yes | -- | -- | -- | -- | -- | -- | -- | -- | -- |
+| Domain Transformer | Yes | -- | -- | -- | -- | -- | -- | -- | -- | -- |
+| Manifest | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -- | Yes | Yes |
+| Output Format | Markdown | Markdown | Markdown | Markdown | Markdown | Markdown | Markdown | Markdown | Markdown | Markdown |
+| Rule IDs | Yes | No | No | No | No | No | No | No | No | No |
+| Priority Annotations | Yes | No | No | No | Sorted | No | No | No | No | No |
+| Metadata Comments | HTML | None | None | None | None | None | None | None | None | None |
+
+## Engine Component Diagram
+
+```mermaid
+graph TB
+    subgraph Required["Required Components"]
+        BACK["backend.rb<br/>Renders YAML → output format"]
+    end
+
+    subgraph Detection["Detection (optional)"]
+        DET["detector.rb<br/>Checks if AI tool is installed"]
+        MANIF["manifest.yml<br/>Capabilities + detection rules"]
+    end
+
+    subgraph Config["Config (optional, Claude only)"]
+        CC["config_compiler.rb"]
+        KM["key_map.rb"]
+        SR["scope_router.rb"]
+        DT["domain_transformer.rb"]
+    end
+
+    subgraph Execution["Execution (optional)"]
+        EXEC["executor.rb<br/>API (HTTP/SDK) or CLI subprocess<br/>invocation for quorum"]
+    end
+
+    Required --- Detection
+    Required --- Config
+    Required --- Execution
+```
+
+## Claude Engine
+
+The Claude engine is the most feature-rich, handling both rule compilation
+and Claude Code settings management.
+
+### Components
+
+```text
+lib/rosett_ai/engines/claude/
+├── backend.rb             # Markdown with HTML comment metadata
+├── config_compiler.rb     # YAML scope → JSON settings
+├── detector.rb            # Detects claude binary + ~/.claude/
+├── domain_transformer.rb  # Transform keys between domains
+├── executor.rb            # Invoke Claude API (anthropic gem)
+├── key_map.rb             # Map YAML keys → JSON settings keys
+└── scope_router.rb        # Route settings to correct scope file
+```
+
+### Compilation Output Format
+
+```markdown
+<!-- rosett-ai-claude-managed -->
+<!-- Source: conf/behaviour/security.yml -->
+<!-- Category: behaviour | Version: 1.1.0 -->
+
+# security
+
+Security rules for Claude Code
+
+## Rules
+
+### rule_001 (priority: 90)
+
+Always use YAML.safe_load — never YAML.load
+
+### rule_002 (priority: 85)
+
+Use array-form system() — never string interpolation
+```
+
+### Config Compilation Flow
+
+```mermaid
+sequenceDiagram
+    participant CLI
+    participant Compiler as ConfigCompiler
+    participant ScopeRouter
+    participant KeyMap
+    participant FS as Filesystem
+
+    CLI->>Compiler: compile(scope: "managed")
+    Compiler->>ScopeRouter: route(scope)
+    ScopeRouter-->>Compiler: conf/engines/claude/config/managed.yml
+    Compiler->>Compiler: load YAML
+    Compiler->>KeyMap: transform(yaml_keys)
+    KeyMap-->>Compiler: json_keys
+    Compiler->>FS: write settings.json
+    FS-->>Compiler: success
+    Compiler-->>CLI: CompileResult
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect Claude] --> B{claude binary in PATH?}
+    B -->|Yes| C{~/.claude/ exists?}
+    B -->|No| D[Not detected]
+    C -->|Yes| E{~/.claude/settings.json exists?}
+    C -->|No| F[Partially detected]
+    E -->|Yes| G[Fully detected + available]
+    E -->|No| F
+```
+
+## AGENTS.md Engine
+
+Compiles rules into the [AGENTS.md](https://github.com/anthropics/agents-md)
+format — bullet-list rules under an `## Instructions` heading. No rule IDs,
+no metadata, priority-sorted.
+
+### Components
+
+```text
+lib/rosett_ai/engines/agents_md/
+├── backend.rb    # Bullet-list renderer
+└── detector.rb   # Always available (no external tool needed)
+```
+
+### Output Format
+
+```markdown
+## Instructions
+
+- Always use YAML.safe_load — never YAML.load
+- Use array-form system() — never string interpolation
+- All external input must be validated before use
+```
+
+### Design Decisions
+
+```mermaid
+flowchart LR
+    A[Rules YAML] --> B{Has priority?}
+    B -->|Yes| C[Sort descending]
+    B -->|No| D[Default priority 50]
+    C --> E[Render as bullet list]
+    D --> E
+    E --> F[Prepend '## Instructions']
+    F --> G[Write AGENTS.md]
+```
+
+## Generic Engine
+
+Minimal engine producing plain markdown without tool-specific formatting.
+Useful for engines not yet supported or for human-readable documentation.
+
+### Components
+
+```text
+lib/rosett_ai/engines/generic/
+└── backend.rb    # Plain markdown renderer
+```
+
+## Ollama Engine
+
+Compiles rules into prompts suitable for Ollama's local LLM serving.
+
+### Components
+
+```text
+lib/rosett_ai/engines/ollama/
+├── backend.rb     # Ollama-compatible text renderer
+├── detector.rb    # Detects ollama binary + localhost:11434
+└── executor.rb    # Invokes Ollama API for quorum
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect Ollama] --> B{ollama binary in PATH?}
+    B -->|Yes| C{localhost:11434 responding?}
+    B -->|No| D[Not detected]
+    C -->|Yes| E[Fully detected + available]
+    C -->|No| F[Binary found, server not running]
+```
+
+## GPT-NeoX Engine
+
+Compiles rules into prompts for GPT-NeoX models served via vLLM.
+Separate from Ollama for deployment independence.
+
+### Components
+
+```text
+lib/rosett_ai/engines/gpt_neox/
+├── backend.rb     # vLLM-compatible text renderer
+├── detector.rb    # Detects vLLM endpoint via env/config
+└── executor.rb    # Invokes vLLM API for quorum
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect GPT-NeoX] --> B{VLLM_ENDPOINT env var set?}
+    B -->|Yes| C{Endpoint responding?}
+    B -->|No| D{Config has vllm_endpoint?}
+    C -->|Yes| E[Fully detected]
+    C -->|No| F[Configured but unreachable]
+    D -->|Yes| C
+    D -->|No| G[Not detected]
+```
+
+## Goose Engine
+
+Compiles rules into markdown for [Goose](https://block.github.io/goose/) by Block.
+Uses the generic backend with a Goose-specific managed marker. Supports
+quorum participation via CLI subprocess executor.
+
+### Components
+
+```text
+lib/rosett_ai/engines/goose/
+├── backend.rb     # Goose-compatible markdown renderer
+├── detector.rb    # Detects goose binary + ~/.config/goose/
+└── executor.rb    # CLI subprocess executor (Open3.capture3)
+```
+
+### Output Marker
+
+```markdown
+<!-- rosett-ai-goose-managed -->
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect Goose] --> B{goose binary in PATH?}
+    B -->|Yes| C{~/.config/goose/ exists?}
+    B -->|No| D[Not detected]
+    C -->|Yes| E[Fully detected + available]
+    C -->|No| F[Binary found, no config]
+```
+
+### Executor
+
+CLI subprocess executor using `Open3.capture3` with `GOOSE_MODE=auto`.
+Prompts passed via tempfile. Returns structured JSON for quorum aggregation.
+
+## Aider Engine
+
+Compiles rules into markdown for [Aider](https://aider.chat/), the AI pair
+programming tool. Uses the generic backend with an Aider-specific managed marker.
+Supports quorum participation via CLI subprocess executor.
+
+### Components
+
+```text
+lib/rosett_ai/engines/aider/
+├── backend.rb     # Aider-compatible markdown renderer
+├── detector.rb    # Detects aider binary
+└── executor.rb    # CLI subprocess executor (Open3.capture3)
+```
+
+### Output Marker
+
+```markdown
+<!-- rosett-ai-aider-managed -->
+```
+
+### Executor
+
+CLI subprocess executor using `Open3.capture3` with
+`--message-file <tempfile> --no-auto-commits --no-git --yes`.
+Returns structured JSON for quorum aggregation.
+
+## Cursor Engine
+
+Compiles rules into a `.cursorrules` file for [Cursor](https://cursor.com/) by
+Anysphere. Compilation-only engine (no executor).
+
+### Components
+
+```text
+lib/rosett_ai/engines/cursor/
+├── backend.rb     # Cursor-compatible markdown renderer
+└── detector.rb    # Detects cursor binary + ~/.cursor/
+```
+
+### Output Marker
+
+```markdown
+<!-- rosett-ai-cursor-managed -->
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect Cursor] --> B{cursor binary in PATH?}
+    B -->|Yes| C{~/.cursor/ exists?}
+    B -->|No| D[Not detected]
+    C -->|Yes| E[Fully detected + available]
+    C -->|No| F[Binary found, no config]
+```
+
+## Copilot Engine
+
+Compiles rules into `.github/copilot-instructions.md` for
+[GitHub Copilot](https://github.com/features/copilot).
+Compilation-only engine (no executor).
+
+### Components
+
+```text
+lib/rosett_ai/engines/copilot/
+├── backend.rb     # Copilot-compatible markdown renderer
+└── detector.rb    # Detects .github/ config directory
+```
+
+### Output Marker
+
+```markdown
+<!-- rosett-ai-copilot-managed -->
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect Copilot] --> B{.github/ directory exists?}
+    B -->|Yes| C[Detected — advisory only]
+    B -->|No| D[Not detected]
+```
+
+## Windsurf Engine
+
+Compiles rules into a `.windsurfrules` file for
+[Windsurf](https://codeium.com/windsurf) by Codeium.
+Compilation-only engine (no executor).
+
+### Components
+
+```text
+lib/rosett_ai/engines/windsurf/
+├── backend.rb     # Windsurf-compatible markdown renderer
+└── detector.rb    # Detects windsurf binary
+```
+
+### Output Marker
+
+```markdown
+<!-- rosett-ai-windsurf-managed -->
+```
+
+### Detection Logic
+
+```mermaid
+flowchart TD
+    A[Detect Windsurf] --> B{windsurf binary in PATH?}
+    B -->|Yes| C[Detected + available]
+    B -->|No| D[Not detected]
+```
+
+## Capability Manifests
+
+Each engine declares capabilities via `conf/engines/<name>/manifest.yml`:
+
+```yaml
+name: agents_md
+display_name: "AGENTS.md"
+version: "1.0.0"
+
+detection:
+  always_available: true
+
+capabilities:
+  compile_rules: true
+  compile_config: false
+  sensitive_filtering: false
+  rule_ids: false
+  priority_annotations: false
+  metadata_comments: false
+
+execution:
+  api_gem: null
+  cli_binary: null
+  structured_output: false
+```
+
+### Capability Gap Warnings
+
+```mermaid
+flowchart TD
+    A[Compile request] --> B{Load manifest}
+    B --> C{Behaviour has sensitive: true?}
+    C -->|Yes| D{Engine supports sensitive_filtering?}
+    C -->|No| E[Proceed normally]
+    D -->|Yes| F[Apply redaction + compile]
+    D -->|No| G{--strict mode?}
+    G -->|Yes| H[ERROR: capability gap]
+    G -->|No| I["WARN: included without redaction"]
+    I --> J[Compile anyway]
+```
+
+## Executor Resolution
+
+All engines with an executor implement a uniform `analyze(prompt)` method that
+returns structured JSON with `findings`, `overall_status`, and `summary` keys.
+The `ExecutorResolver` selects the correct constructor kwargs based on the
+engine manifest's `execution` section:
+
+```mermaid
+flowchart TD
+    A[RuleAdopter.resolve_executor] --> B[ExecutorResolver.new]
+    B --> C{Read manifest execution}
+    C -->|api_gem set| D["Pass api_key:, base_url:<br/>(SDK-based: Claude)"]
+    C -->|api_type: http| E["Pass endpoint:, model:<br/>(HTTP API: Ollama, GPT-NeoX)"]
+    C -->|api_type: cli| F2["Pass binary:<br/>(CLI subprocess: Goose, Aider)"]
+    D --> F[Executor.new with kwargs]
+    E --> F
+    F2 --> F
+    F --> G[executor.analyze prompt]
+    G --> H["JSON {findings, overall_status, summary}"]
+```
+
+## Quorum Module
+
+The quorum module dispatches adopt analysis to multiple engines in parallel
+and aggregates their findings. Requires `--api --quorum` flags.
+
+```mermaid
+flowchart LR
+    subgraph Dispatch["Dispatcher"]
+        D1["Thread per engine"]
+    end
+
+    subgraph Collect["Collector"]
+        C1["Normalize responses"]
+    end
+
+    subgraph Compare["Comparator"]
+        CP1["Cross-engine: promoted"]
+        CP2["Single-engine: annotated"]
+    end
+
+    PROMPT["Adopt prompt"] --> Dispatch
+    Dispatch --> |Claude| Collect
+    Dispatch --> |Goose| Collect
+    Dispatch --> |Aider| Collect
+    Dispatch --> |Ollama| Collect
+    Dispatch --> |GPT-NeoX| Collect
+    Collect --> Compare
+    Compare --> RESULT["Aggregated findings"]
+```
+
+### Components
+
+```text
+lib/rosett_ai/quorum/
+├── collector.rb          # Normalize per-engine responses
+├── comparator.rb         # Aggregate cross-engine findings
+├── dispatcher.rb         # Thread-based parallel dispatch
+└── strategies/
+    └── adopt.rb          # Adopt-specific orchestration
+```
+
+### Usage
+
+```bash
+# Single engine (default)
+bin/raictl adopt --api
+
+# Multi-engine quorum analysis
+bin/raictl adopt --api --quorum
+```
+
+## Adding a New Engine
+
+1. Create directory: `lib/rosett_ai/engines/<name>/`
+2. Implement `backend.rb` (required)
+3. Optionally add `detector.rb`, `executor.rb`, `manifest.yml`
+4. Register in `lib/rosett_ai/engines/registry.rb`
+5. Add target profile in `conf/targets/<name>.yml` (if different output dir)
+6. Add specs in `spec/rosett_ai/engines/<name>/`
+7. Use shared examples: `it_behaves_like 'a compiler backend'`
+
+See [CONTRIBUTING.md](../CONTRIBUTING.md) for the full guide.
+
+---
+
+*Generated: 2026-03-02 | rosett-ai v1.0.0*