@miller-tech/uap 1.39.0 → 1.40.1

package/docs/reference/PATTERNS.md

@@ -0,0 +1,102 @@
+# Pattern Library Reference
+
+> Universal Agent Protocol (UAP) v1.40.0
+
+UAP ships an execution-pattern library: a set of reusable problem-solving
+strategies that are matched to the current task and surfaced to the agent.
+Patterns are defined as markdown files under `.factory/patterns/`, catalogued
+in `.factory/patterns/index.json`, and retrieved on demand via a Qdrant-backed
+RAG flow (`uap patterns query`).
+
+## The 23 Patterns
+
+The canonical roster lives in `.factory/patterns/index.json`. Each pattern has
+a numeric (or string) id, a markdown body file, a title, an abbreviation, a
+category, and a keyword set used for retrieval.
+
+| ID | Title | Abbreviation | Category | What it does |
+|----|-------|--------------|----------|--------------|
+| P12 | Output Existence Verification | OE-Verify | Verification | Confirm the artifact a task was supposed to produce actually exists before claiming done. **Always enforced.** |
+| P13 | Iterative Refinement Loop | Iter-Ref | Testing | Run tests, fix, and re-run in a loop until they pass. |
+| P14 | Output Format Validation | Format-Check | Verification | Validate that produced output matches the required format (JSON/YAML/CSV). |
+| P16 | Task-First Execution | Task-First | Execution | When the task is clear, execute directly instead of over-planning. |
+| P17 | Constraint Extraction | Extract-Constraints | Planning | Pull hard constraints ("exactly", "only", "no more than") out of the prompt before acting. |
+| P19 | Impossible Task Refusal | Refuse-Impossible | Safety | Detect and refuse tasks that are fundamentally impossible. |
+| P20 | Adversarial Thinking | Adversarial | Security | Reason about how something could be bypassed/exploited/evaded. |
+| P21 | Chess Engine Integration | Chess-Engine | Domain-Specific | Delegate chess reasoning (FEN, best move, checkmate) to an engine. |
+| P22 | Git Recovery Forensics | Git-Recovery | Recovery | Recover lost/corrupted git state via reflog and forensic inspection. |
+| P23 | Compression Impossibility Detection | Compress-Check | Verification | Detect already-compressed data so re-compression isn't attempted. |
+| P24 | Polyglot Code Construction | Polyglot | Code-Golf | Construct source that compiles/runs as multiple languages. |
+| P25 | Service Configuration Pipeline | Service-Config | DevOps | Configure, validate, and reload a service/daemon as a pipeline. |
+| P26 | Near-Miss Iteration | Near-Miss | Testing | When a test result is a small gap away, tweak and re-check. |
+| P28 | Service Smoke Test | Smoke-Test | Verification | After deploy/start, run a health check / smoke test to verify. |
+| P30 | Performance Threshold Tuning | Perf-Threshold | Optimization | Measure performance and tune against a percentage/threshold target. |
+| P31 | Round-Trip Verification | Round-Trip | Verification | Encode then decode (or compress/decompress) and verify the round trip. |
+| P32 | CLI Execution Verification | CLI-Verify | Verification | Verify a CLI/binary actually runs as expected. |
+| P33 | Numerical Stability Testing | Num-Stable | Testing | Test floating-point/numerical work for precision and stability. |
+| P34 | Image-to-Structured Pipeline | Image-Structured | Domain-Specific | Convert images (OCR/diagram/board) into structured data. |
+| P35 | Decoder-First Analysis | Decoder-First | Analysis | Build/understand the decoder/parser first when reverse-engineering a format. **Always enforced.** |
+| P36 | Competition Domain Research | Competition-Research | Research | Research the competitive domain (win rate, leaderboard, tournament) before optimizing. |
+| P37 | Ambiguity Detection & Resolution | Ambiguity-Detect | Planning | Detect vague/unspecified requirements and clarify before acting. |
+| IaC | Infrastructure as Code Parity | IaC-Parity | Infrastructure | Keep infrastructure changes reflected in IaC (terraform/kubernetes) for reproducibility. |
+
+### Always-on patterns
+
+Two patterns are unconditionally included regardless of the matched task, set
+in `src/coordination/pattern-router.ts`:
+
+```js
+const alwaysIncludeIds = ['P12', 'P35']; // Output Existence, Decoder-First
+```
+
+- **P12 (Output Existence Verification)** — guards against "claiming done"
+  without producing the artifact.
+- **P35 (Decoder-First Analysis)** — anchors format/reverse-engineering work.
+
+## How pattern RAG works
+
+Pattern retrieval is semantic, not keyword-based. The flow:
+
+1. **Indexing** (`uap patterns index`) runs a generated Python indexer
+   (`agents/scripts/index_patterns_to_qdrant.py`). It scans multiple sources —
+   `CLAUDE.md` (with `@include` resolution), additional source files
+   (`AGENTS.md`, etc.), `*/SKILL.md` skill files, and `.factory/patterns/*.md`
+   (using `index.json` to preserve the canonical `PNN: Title` identity and
+   keyword set). Documents are de-duplicated by content hash, embedded, and
+   upserted into the Qdrant collection.
+2. **Querying** (`uap patterns query "<task>"`) runs the generated query script
+   (`agents/scripts/query_patterns.py`), embeds the query with the same model,
+   and runs a cosine `query_points` search against the collection, returning the
+   top-K hits above the score threshold.
+3. **Fallback** — if the Python query script is absent, the CLI falls back to a
+   direct Qdrant `scroll` + keyword match (less accurate) and prints a notice.
+
+### RAG defaults
+
+From `src/cli/patterns.ts` (`getPatternRagConfig`), overridable via
+`.uap.json` under `memory.patternRag`:
+
+| Setting | Default |
+|---------|---------|
+| Collection | `agent_patterns` |
+| Embedding model | `all-MiniLM-L6-v2` |
+| Vector size | `384` (Cosine distance) |
+| Score threshold | `0.35` |
+| Top-K | `2` |
+| Index script | `./agents/scripts/index_patterns_to_qdrant.py` |
+| Query script | `./agents/scripts/query_patterns.py` |
+| Source file | `CLAUDE.md` |
+| Max body chars | `400` (display); indexer truncates bodies at 2000 |
+
+The query script requires a Python with `sentence-transformers` and
+`qdrant-client`. The CLI auto-discovers `agents/.venv/bin/python`,
+`.venv/bin/python`, then `python3`/`python`, and can bootstrap a venv.
+
+## `uap patterns` CLI
+
+| Command | Description |
+|---------|-------------|
+| `uap patterns status` | Show RAG config, Qdrant collection point count, Python/script availability. |
+| `uap patterns index` | (Re)index pattern/doc sources into the Qdrant collection. `--verbose` for full output. |
+| `uap patterns query "<task>"` | Semantic search for task-relevant patterns. Flags: `--top <n>`, `--min-score <f>`, `--format text\|json`. |
+| `uap patterns generate` | Generate the Python index/query scripts. `--force` to overwrite. |

package/docs/reference/PLATFORMS.md

@@ -0,0 +1,83 @@
+# Platform & Harness Reference
+
+> Universal Agent Protocol (UAP) v1.40.0
+
+UAP integrates with 9 agent harnesses. The canonical list is `ALL_TARGETS` in
+`src/cli/hooks.ts`. Every platform receives the UAP enforcement layer: lifecycle
+hooks, the DB-driven policy gate, and (where supported) the hierarchical MCP
+router.
+
+## Supported platforms
+
+| Target flag | Platform | Config location | Integration style |
+|-------------|----------|-----------------|-------------------|
+| `claude` | Claude Code | `.claude/settings.local.json` + `.claude/hooks/` | Native hook events |
+| `factory` | Factory.AI Droid | `.factory/settings.local.json` + `.factory/hooks/` | Native hook events (`$FACTORY_PROJECT_DIR`) |
+| `cursor` | Cursor | `.cursor/hooks.json` + `.cursor/hooks/` | Native hook events |
+| `vscode` | VSCode | `.claude/settings.local.json` + `.claude/hooks/` | Claude Code hook format (via third-party skills) |
+| `opencode` | OpenCode | `.opencode/plugin/uap-session-hooks.ts` + `.opencode/hooks/` | TypeScript plugin |
+| `codex` | Codex CLI | `AGENTS.md`, `.codex/config.toml`, `.codex/hooks/`, `.agents/skills/` | AGENTS.md + skills + MCP server |
+| `forgecode` | ForgeCode | `.forge/forgecode.plugin.sh` + `.forge/hooks/` | ZSH plugin |
+| `omp` | Oh-My-Pi | `.uap/omp/settings.json` + `.uap/omp/hooks/{pre,post}/` | omp pre/post hook dirs |
+| `hermes` | Hermes Agent (NousResearch) | `~/.hermes/config.yaml` + `~/.hermes/agent-hooks/` | **Global** YAML config + MCP |
+
+> **Hermes is global.** Its config lives under `~/.hermes` (or `$HERMES_HOME`),
+> so it is excluded from the default project install loop (`PROJECT_TARGETS`).
+> Installing it requires an explicit `-t hermes`. `status`/`doctor` still report it.
+
+## Support matrix
+
+| Platform | Lifecycle hooks | Worktree/Bash guards | Policy gate | MCP router |
+|----------|-----------------|----------------------|-------------|------------|
+| Claude Code | Native | Yes | Hard (PreToolUse) | Yes |
+| Factory.AI | Native | Yes | Hard (PreToolUse) | Yes |
+| Cursor | Native | Yes | Hard (preToolUse) | Yes |
+| VSCode | Native (Claude format) | Yes | Hard (PreToolUse) | Yes |
+| OpenCode | Plugin events | Yes | Hard (`tool.execute.before` → exit 2 blocks) | Yes |
+| Codex CLI | Advisory (no native pre-tool event) | Advisory | Hard for MCP-routed tools; advisory for native edit/bash | Yes (`[mcp_servers.uap]`) |
+| ForgeCode | ZSH plugin | Yes (scripts) | Via gate script | — |
+| Oh-My-Pi | pre/post dirs | Yes | Yes (`uap-policy-gate.sh`) | — |
+| Hermes | `pre_tool_call` / `on_session_start` | `pre_tool_call` matcher | Hard via `uap-policy-gate-hermes.sh` (emits block JSON) | Yes (`mcp_servers.uap`) |
+
+> **Gating notes.** Codex CLI has no native pre-tool-use hook event, so policy
+> gating is **hard** only for tools routed through the UAP MCP server
+> (`execute_tool` runs the PolicyGate) and **advisory** for Codex-native
+> edit/bash. `uap hooks doctor` reports Codex as MCP-gated. Hermes hooks are
+> fail-open by design, but the UAP gate always emits a decision JSON so real
+> blocks are enforced; Hermes prompts once to approve each hook command
+> (`~/.hermes/shell-hooks-allowlist.json`) unless `hooks_auto_accept: true`.
+
+## Lifecycle hooks installed
+
+The shared hook scripts (copied from `templates/hooks/`) are wired into each
+platform's lifecycle events:
+
+| Script | Event | Purpose |
+|--------|-------|---------|
+| `session-start.sh` | SessionStart | Injects recent memory context. |
+| `pre-tool-use-edit-write.sh` | PreToolUse (Edit/Write/MultiEdit) | Worktree file guard — **blocks** edits outside worktree dirs. |
+| `pre-tool-use-bash.sh` | PreToolUse (Bash) | Dangerous-command guard — **blocks** force push, `terraform apply`, etc. |
+| `uap-policy-gate.sh` | PreToolUse | DB-driven policy gate (`policies.db` + `.policy-tools/*.py`). |
+| `post-tool-use-edit-write.sh` | PostToolUse (Edit/Write) | Build gate + backup reminder after edits. |
+| `pre-compact.sh` | PreCompact | Flushes a compaction marker to memory. |
+| `post-compact.sh` | PostCompact | Re-injects policy awareness after compaction. |
+| `stop.sh` | Stop | Completion-gate checklist + session cleanup. |
+| `session-end.sh` | SessionEnd | Agent deregistration + backup retention. |
+| `loop-protection.sh` | — | Loop/spawn protection. |
+
+## Install & verify
+
+```bash
+uap hooks install              # install for all PROJECT_TARGETS (excludes hermes)
+uap hooks install -t claude    # install for a single platform
+uap hooks install -t hermes    # required to install the global Hermes config
+uap hooks status               # per-platform script + settings status (all targets)
+uap hooks doctor               # health check across all targets
+```
+
+The MCP router is configured separately and across all harnesses with:
+
+```bash
+uap mcp-setup                  # configure the hierarchical MCP router everywhere
+uap mcp-router start           # run the stdio MCP router server
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@miller-tech/uap",
-  "version": "1.39.0",
+  "version": "1.40.1",
   "description": "Autonomous AI agent memory system with CLAUDE.md protocol enforcement",
   "type": "module",
   "main": "dist/index.js",

package/docs/AGENTS.md

@@ -1,423 +0,0 @@
-# UAP Project Code Analysis and Documentation Audit Report
-
-## Executive Summary
-
-The Universal Agent Protocol (UAP) is a comprehensive AI agent framework with extensive functionality for persistent memory, multi-agent coordination, pattern-based workflows, and policy enforcement. The project contains approximately 143 TypeScript source files, 74 test files, and over 90,000 total files (including benchmark results and agent sessions).
-
-## Documentation Quality Assessment
-
-### Current State: GOOD TO VERY GOOD
-
-**Strengths:**
-
-- Comprehensive README.md (542 lines) with feature overview, CLI reference, and architecture diagrams
-- Extensive CLI reference documentation (620 lines in UAP_CLI_REFERENCE.md)
-- Architecture documentation covering system design and components
-- Benchmark results and validation plans documented
-- Integration guides for multiple platforms (Claude, Factory, OpenCode, etc.)
-
-**Well-documented:**
-
-- Hierarchical memory (Hot/Warm/Cold tiers)
-- Embedding service with multiple providers
-- Knowledge graph implementation
-- Semantic compression
-- Write gate quality filtering
-- Daily log staging area
-
-**Under-documented:**
-
-- Predictive memory service
-- Context pruner (token-budget-aware)
-- Ambiguity detector (P37 pattern)
-- Prepopulation from docs/git history
-- Memory maintenance routines
-- Correction propagator for cross-tier updates
-
-### 2. Multi-Agent Coordination (8 modules)
-
-**Well-documented:**
-
-- Coordination service and database
-- Agent lifecycle management
-- Work claims and announcements
-- Messaging system with channels
-- Deploy batching with configurable windows
-
-**Under-documented:**
-
-- Capability router (18 capability types)
-- Auto-agent registration
-- Pattern router implementation
-- Adaptive patterns with success tracking
-
-### 3. Policy Enforcement (8 modules)
-
-**Well-documented:**
-
-- Policy schema and database manager
-- Policy memory CRUD operations
-- Enforcement levels (REQUIRED/RECOMMENDED/OPTIONAL)
-- Audit trail functionality
-
-**Under-documented:**
-
-- Policy gate middleware implementation
-- Enforced tool router
-- Python enforcement tools
-- Policy converter to CLAUDE.md format
-
-### 4. Pattern System (23+ patterns)
-
-**Well-documented:**
-
-- Pattern list with descriptions
-- Critical patterns (P12, P35) always active
-- Pattern RAG management
-
-**Under-documented:**
-
-- Pattern matching algorithm
-- Pattern library storage format
-- Pattern success tracking metrics
-- Custom pattern creation guide
-
-### 5. Worktree System
-
-**Well-documented:**
-
-- CLI commands (create, list, pr, cleanup, finish)
-- Git workflow integration
-- Exempt paths documentation
-
-**Under-documented:**
-
-- Worktree file guard enforcement mechanism
-- Branch naming conventions
-- Conflict resolution strategies
-- Prune functionality details
-
-### 6. Hooks System
-
-**Well-documented:**
-
-- Session start hook (5 steps)
-- Pre-compact hook (4 steps)
-- Platform-specific installations
-- Hook status checking
-
-**Under-documented:**
-
-- Pre-tool-use hooks (mentioned but not detailed)
-- Post-tool-use hooks
-- Custom hook creation guide
-- Hook failure recovery
-
-### 7. MCP Router
-
-**Well-documented:**
-
-- 98% token reduction claim
-- Meta-tool routing concept
-- Config parser and fuzzy search
-
-**Under-documented:**
-
-- Client pool management
-- Output compression algorithm
-- Session statistics tracking
-- Tool discover/execute definitions
-
-### 8. Multi-Model Architecture
-
-**Well-documented:**
-
-- 3-tier execution model
-- 13 model profiles
-- Dynamic temperature and rate limiting
-- Model analytics
-
-**Under-documented:**
-
-- Task planner decomposition algorithm
-- Plan validator cycle detection
-- Unified router logic
-- Execution profile loading
-
-### 9. Browser Automation
-
-**Well-documented:**
-
-- CloakBrowser integration
-- Playwright drop-in compatibility
-- Basic usage example
-
-**Under-documented:**
-
-- Stealth techniques
-- Humanize mode details
-- Error handling
-- Performance characteristics
-
-### 10. Task Management
-
-**Well-documented:**
-
-- Task types and statuses
-- Priority levels (P0-P4)
-- Dependencies and claims
-- JSONL sync format
-
-**Under-documented:**
-
-- Event bus implementation
-- Decoder gate mechanism
-- Task classifier (9 categories)
-- Compaction/archive logic
-
-### 11. Droids & Skills
-
-**Well-documented:**
-
-- 8 expert droids listed
-- 33 skills categorized
-- Skill documentation command
-
-**Under-documented:**
-
-- Droid creation process
-- Skill loading mechanism
-- Custom droid development
-- Skill composition patterns
-
-## Documentation Gaps by Priority
-
-### CRITICAL (Must-Have)
-
-1. **Inline JSDoc for Public APIs**
-   - Missing from src/index.ts exports (340 lines of exports)
-   - Affects: Memory system, coordination, policies, models, MCP router
-   - Impact: Breaks IDE autocomplete and API documentation generation
-
-2. **CLI Command Completeness**
-   - Dashboard has 11 views but only partially documented
-   - Model commands (8 subcommands) under-documented
-   - Policy commands (15 subcommands) need examples
-   - Worktree finish command behavior not explained
-
-3. **Configuration Schema Documentation**
-   - .uap.json schema incomplete
-   - Missing validation rules
-   - Environment variables not fully listed
-
-4. **Database Schema Accuracy**
-   - API_REFERENCE.md shows outdated table structures
-   - Missing tables: deploy_queue, pattern_index, task_events
-   - Qdrant collection schemas not documented
-
-5. **Hook System Implementation Details**
-   - Pre-edit build gate enforcement mechanism
-   - Worktree file guard blocking logic
-   - Completion gate verification steps
-
-### HIGH PRIORITY (Should-Have)
-
-6. **Memory System Architecture Diagram**
-   - Current diagram has placeholder text
-   - 4-layer architecture needs visual representation
-   - Data flow between tiers unclear
-
-7. **Pattern Library Reference**
-   - All 23+ patterns need detailed descriptions
-   - Pattern selection criteria missing
-   - Pattern composition examples
-
-8. **Multi-Agent Coordination Flow**
-   - Heartbeat mechanism details
-   - Overlap detection algorithm
-   - Message priority handling
-   - Deadlock prevention
-
-9. **Policy Enforcement Workflow**
-   - Policy evaluation order
-   - Violation handling procedures
-   - Audit trail query examples
-
-10. **Testing and Quality Gates**
-    - Test coverage requirements (50% threshold)
-    - Build gate enforcement
-    - Completion gate verification steps
-
-### MEDIUM PRIORITY (Nice-to-Have)
-
-11. **Benchmark Methodology**
-    - Terminal-Bench adapter details
-    - Harbor integration workflow
-    - A/B comparison methodology
-
-12. **Deployment Guides**
-    - Production Qdrant setup
-    - CI/CD pipeline configuration
-    - Horizontal scaling patterns
-
-13. **Troubleshooting Matrix**
-    - Error code reference
-    - Common failure modes
-    - Recovery procedures
-
-14. **Integration Patterns**
-    - RTK token compression details
-    - Platform-specific optimizations
-    - Custom adapter development
-
-15. **Performance Optimization Guide**
-    - Token budget management
-    - Cache warm strategies
-    - Memory pruning thresholds
-
-## Specific Recommendations
-
-### Immediate Actions (Week 1-2)
-
-1. **Add JSDoc to src/index.ts**
-
-   ```typescript
-   /**
-    * Hierarchical memory manager with hot/warm/cold tiering
-    * @see src/memory/hierarchical-memory.ts for implementation details
-    */
-   export { HierarchicalMemoryManager } from './memory/hierarchical-memory.js';
-   - Replace placeholder text in docs/INDEX.md
-   - Create Mermaid diagrams for memory flow
-   - Document data persistence layers
-
-   ```
-
-2. **Complete CLI Reference**
-   - Add all 109 commands with examples
-   - Include exit codes and error messages
-   - Document shell completion setup
-
-3. **Fix Version Mismatch**
-   - Update INDEX.md to v1.20.32
-   - Ensure CHANGELOG.md is current
-   - Sync README version numbers
-
-4. **Document Database Schemas**
-   - Update API_REFERENCE.md with actual tables
-   - Add Qdrant collection schemas
-   - Include migration guides
-
-### Short-Term Actions (Week 3-4)
-
-6. **Create Module-Level Documentation**
-   - Add README.md to each src/ subdirectory
-   - Document module responsibilities and dependencies
-   - Include usage examples
-
-7. **Build Pattern Library Reference**
-   - Document all 23+ patterns with use cases
-   - Add pattern selection decision tree
-   - Create pattern composition examples
-
-8. **Enhance Hook Documentation**
-   - Document all hook types (pre/post tool-use)
-   - Add hook failure recovery guide
-   - Include custom hook templates
-
-9. **Complete Configuration Guide**
-   - Full .uap.json schema with validation
-   - Environment variable reference
-   - Platform-specific configs
-
-10. **Add Testing Documentation**
-    - Test coverage requirements
-    - Build gate enforcement
-    - Completion gate checklist
-
-### Long-Term Actions (Month 2+)
-
-11. **Generate API Documentation**
-    - Use TypeDoc for TypeScript APIs
-    - Integrate with GitHub Pages
-    - Keep in sync with code changes
-
-12. **Create Video Tutorials**
-    - Quick start walkthrough
-    - Advanced feature demonstrations
-    - Troubleshooting guides
-
-13. **Develop Interactive Examples**
-    - Code sandbox for CLI commands
-    - Memory system visualization
-    - Multi-agent simulation
-
-14. **Establish Documentation Review Process**
-    - Require docs updates with PRs
-    - Add docs linting to CI
-    - Schedule quarterly reviews
-
-15. **Create Contribution Guide**
-    - Documentation standards
-    - Template examples
-    - Review process
-
-## Proposed Documentation Structure
-
-### Current Structure (Adequate)
-
-```
-docs/
-├── getting-started/      ✓ Good coverage
-├── architecture/         ⚠ Needs updates
-├── reference/            ✓ CLI ref good, API ref outdated
-├── deployment/           ⚠ Incomplete
-├── benchmarks/           ✓ Comprehensive
-├── operations/           ⚠ Minimal
-├── integrations/         ✓ Good coverage
-├── research/             ⚠ Academic focus
-└── archive/              ℹ Historical reference
-```
-
-### Proposed Enhanced Structure:
-
--74 test files covering major components
-
-- Tests for: policies, memory, coordination, models, benchmarks, CLI
-- Vitest configuration with coverage thresholds (50%)
-
-### Gaps
-
-1. **Browser module** - Minimal test coverage
-2. **MCP router** - Only output-compressor tested
-3. **Dashboard** - Server and event-stream not tested
-4. **Telemetry** - Session telemetry untested
-5. **Benchmarks** - Agent implementations not tested
-
-### Recommendations
-
-1. Add JSDoc to test files for clarity
-2. Create integration test suite
-3. Document test coverage requirements
-4. Add performance test benchmarks
-
-## Conclusion
-
-The UAP project has **solid foundational documentation** with comprehensive CLI references and good architectural overviews. However, there are significant gaps in:
-
-1. code documentation (JSDoc)
-2. Module-level documentation for internal components
-3. Accurate database schemas
-4. Configuration schema completeness
-5. Hook system implementation details
-
-**Priority Ranking:**
-
-- **Must-Have**: JSDoc, CLI completeness, config schema, DB schemas, hook docs
-  - Architecture diagrams, pattern library, coordination flows, policy workflows, testing docs
-- **Nice-to-Have**: Video tutorials, interactive examples, contribution guide, API generation
-- Critical fixes: 2-3 weeks
-- High priority: 4-6 weeks
-- Medium priority: 8-12 weeks
-
-The documentation quality is sufficient for current users but needs improvement to support new contributors and maintain long-term project health.