@deimoscloud/coreai 0.1.0

package/ARCHITECTURE.md

@@ -0,0 +1,547 @@
+# CoreAI Architecture
+
+Technical documentation for understanding and modifying the CoreAI multi-agent framework.
+
+## Overview
+
+CoreAI enables multiple AI agents to collaborate on software development tasks using Claude Code. Agents communicate through file-based messaging (inboxes), follow defined workflows, and use MCP (Model Context Protocol) tools for external integrations.
+
+---
+
+## Core Concepts
+
+### 1. Agents
+
+Agents are Claude Code personas defined in markdown files. Each agent has:
+
+- **Identity**: Name, role, responsibilities
+- **Tools**: MCP tools they can access (Jira, GitHub, etc.)
+- **Behavior**: Startup protocol, work patterns, completion protocol
+- **Context**: KnowledgeLibrary directory for state and communication
+
+**Agent Types:**
+
+| Type | Purpose | Examples |
+|------|---------|----------|
+| Manager | Coordinate, delegate, validate | engineering-manager |
+| IC Engineer | Implement features, fix bugs | backend-engineer, frontend-engineer |
+| Reviewer | Code review, specialized guidance | security-engineer, qa-engineer |
+
+**File Location:** `.claude/agents/<agent-name>.md`
+
+### 2. Commands (Skills)
+
+Commands are user-invocable workflows defined in markdown. They provide:
+
+- **Instructions**: Step-by-step process for the agent to follow
+- **Templates**: Output formats, message structures
+- **Integrations**: Which MCP tools to use
+
+**File Location:** `.claude/commands/<command-name>.md`
+
+**Invocation:** User types `/command-name` or agent is instructed to use it
+
+### 3. KnowledgeLibrary
+
+File-based storage for agent state and inter-agent communication.
+
+```
+KnowledgeLibrary/
+├── context.txt           # Project-wide state (sprint, priorities)
+├── architecture.txt      # Technical decisions
+├── prd.txt               # Product requirements
+│
+└── <agent-name>/
+    ├── inbox/            # Incoming messages from other agents
+    │   └── processed/    # Archived completed messages
+    ├── outbox/           # Copies of sent messages
+    ├── context/
+    │   └── current.txt   # Agent's current state and tasks
+    ├── control/
+    │   ├── objectives.txt    # Goals and success criteria
+    │   ├── decisions.txt     # Decision log
+    │   └── dependencies.txt  # Blocking/blocked-by tracking
+    ├── history/          # Historical records
+    └── tech/             # Technical notes, research
+```
+
+### 4. Workflows
+
+State machines defining how work progresses through the system.
+
+**Ticket Implementation Workflow:**
+```
+BACKLOG → IN_PROGRESS → PR_CREATED → IN_REVIEW → APPROVED → MERGED → DONE
+```
+
+**Code Review Workflow:**
+```
+ASSIGNED → REVIEWING → DECISION_POSTED
+```
+
+Defined in: `coreai/WORKFLOWS.md`
+
+---
+
+## Communication Model
+
+### Why File-Based Messaging?
+
+Claude Code agents cannot directly invoke each other. The Task tool spawns subagents that **lose MCP access** (no Jira, GitHub, etc.). File-based messaging solves this:
+
+1. Agent A writes message to Agent B's inbox
+2. Agent A tells user: "Please invoke @agent-b"
+3. User invokes Agent B (gets full MCP access)
+4. Agent B reads inbox, processes task
+5. Agent B writes response to Agent A's inbox
+6. User invokes Agent A to continue
+
+### Message Format
+
+Inbox messages follow a standard template:
+
+```markdown
+---
+type: task-assignment | completion-report | review-request | feedback
+from: <sender-agent>
+to: <recipient-agent>
+date: YYYY-MM-DD HH:MM
+ticket: PROJ-XX (optional)
+priority: P0-P3 (optional)
+---
+
+## Subject
+
+### Details
+[Content]
+
+### Expected Actions
+1. Action 1
+2. Action 2
+```
+
+**Filename Convention:** `YYYYMMDD_HHMM-<from-agent>-<ticket-or-subject>.md`
+
+### Message Flow Example
+
+```
+┌─────────────────┐     writes to inbox     ┌─────────────────┐
+│  Engineering    │ ───────────────────────▶│ Backend         │
+│  Manager        │                         │ Engineer        │
+└────────┬────────┘                         └────────┬────────┘
+         │                                           │
+         │ tells user:                               │ reads inbox
+         │ "invoke @backend-engineer"                │ works on task
+         │                                           │
+         ▼                                           ▼
+┌─────────────────┐                         ┌─────────────────┐
+│  User           │ ─── invokes agent ────▶ │ Backend         │
+│                 │                         │ Engineer        │
+└─────────────────┘                         └────────┬────────┘
+                                                     │
+                                                     │ writes completion
+                                                     │ to EM inbox
+                                                     ▼
+                                            ┌─────────────────┐
+                                            │ EM Inbox        │
+                                            └─────────────────┘
+```
+
+---
+
+## MCP Integration
+
+MCP (Model Context Protocol) provides external tool access. Agents declare which MCP tools they need in their frontmatter:
+
+```yaml
+---
+tools: Read, Write, Edit, Glob, Grep, mcp__github, mcp__atlassian
+---
+```
+
+### Available MCP Servers
+
+| Server | Tools | Purpose |
+|--------|-------|---------|
+| `mcp__github` | PR management, issues, repo operations | GitHub integration |
+| `mcp__atlassian` | Jira tickets, Confluence pages | Atlassian integration |
+| `mcp__postgres` | Database queries | PostgreSQL access |
+| `mcp__firebase` | Firebase operations | Firebase/GCP access |
+
+### MCP Configuration
+
+MCP servers are configured in `mcp.json` or `.claude/mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "github": {
+      "command": "npx",
+      "args": ["-y", "@modelcontextprotocol/server-github"],
+      "env": {
+        "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_TOKEN}"
+      }
+    }
+  }
+}
+```
+
+**Critical:** Agents invoked via Task tool lose MCP access. Always invoke agents directly via `@agent-name`.
+
+### CLI MCP Integration Layer
+
+The CoreAI CLI provides a programmatic MCP integration layer (`src/adapters/mcp/`) for connecting to MCP servers and mapping their tools to adapter interfaces.
+
+#### Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                       Adapter Layer                              │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐           │
+│  │ IssueTracker │  │ GitProvider  │  │    State     │           │
+│  │   Adapter    │  │   Adapter    │  │   Adapter    │           │
+│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘           │
+└─────────┼─────────────────┼─────────────────┼───────────────────┘
+          │                 │                 │
+          └────────────┬────┴─────────────────┘
+                       │
+          ┌────────────▼────────────┐
+          │     Tool Mapper         │
+          │  (MCP tools → Adapter   │
+          │    interface methods)   │
+          └────────────┬────────────┘
+                       │
+          ┌────────────▼────────────┐
+          │      MCP Client         │
+          │  (SDK wrapper)          │
+          └────────────┬────────────┘
+                       │
+          ┌────────────▼────────────┐
+          │    Server Discovery     │
+          │  (config file parsing)  │
+          └─────────────────────────┘
+```
+
+#### Components
+
+| Component | File | Purpose |
+|-----------|------|---------|
+| Types | `src/adapters/mcp/types.ts` | Server configs, tool definitions, errors |
+| Client | `src/adapters/mcp/client.ts` | MCP SDK wrapper with simplified API |
+| Discovery | `src/adapters/mcp/discovery.ts` | Config file discovery and parsing |
+| Mapper | `src/adapters/mcp/mapper.ts` | Tool-to-adapter mapping with known servers |
+| Registry | `src/adapters/mcp/registry.ts` | Central registry for connection management |
+
+#### Server Discovery
+
+The CLI discovers MCP servers from multiple locations (in priority order):
+
+1. **Project-level**: `mcp.json`, `.mcp.json`, `.claude/mcp.json`
+2. **Global config**: `~/.config/claude/mcp.json`, `~/.claude/mcp.json`
+3. **Claude Desktop**: `~/Library/Application Support/Claude/claude_desktop_config.json`
+
+```typescript
+import { discoverMcpServers } from '@deimoscloud/coreai';
+
+const servers = discoverMcpServers({
+  projectRoot: '/path/to/project',
+  includeGlobal: true,
+});
+```
+
+#### Tool Mapping
+
+MCP tools are mapped to adapter interface methods using either:
+
+1. **Known server mappings** - Pre-defined mappings for popular servers (GitHub, Jira, Confluence, Filesystem)
+2. **Auto-discovery** - Heuristic-based mapping using tool naming patterns with confidence scoring
+
+```typescript
+import { getMappingsForServer, McpTool } from '@deimoscloud/coreai';
+
+const tools: McpTool[] = await client.listTools();
+const mappings = getMappingsForServer('github', tools);
+
+// mappings contains:
+// { toolName: 'create_pull_request', adapterType: 'git', method: 'createPullRequest', ... }
+```
+
+#### Supported Transports
+
+| Transport | Config Field | Description |
+|-----------|--------------|-------------|
+| stdio | `command`, `args`, `env`, `cwd` | Local process communication |
+| http | `url`, `headers` | HTTP-based remote servers |
+| sse | `url`, `headers` | Server-Sent Events for streaming |
+
+---
+
+## Template System
+
+### Placeholders
+
+Templates use double-brace placeholders that get replaced during installation:
+
+| Placeholder | Source | Example |
+|-------------|--------|---------|
+| `{{PROJECT_NAME}}` | coreai.json: project.name | "SurfTrack" |
+| `{{JIRA_PROJECT}}` | coreai.json: project.jiraProject | "SUR" |
+| `{{GITHUB_REPO}}` | coreai.json: project.githubRepo | "owner/repo" |
+| `{{CONFLUENCE_SPACE}}` | coreai.json: project.confluenceSpace | "SUR" |
+| `{{CONFLUENCE_URL}}` | coreai.json: project.confluenceUrl | "https://..." |
+| `{{JIRA_URL}}` | coreai.json: project.jiraUrl | "https://..." |
+| `{{LINT_CMD}}` | coreai.json: qualityChecks.lint | "./gradlew ktlintCheck" |
+| `{{STATIC_ANALYSIS_CMD}}` | coreai.json: qualityChecks.staticAnalysis | "./gradlew detekt" |
+| `{{TEST_CMD}}` | coreai.json: qualityChecks.test | "./gradlew test" |
+| `{{BUILD_CMD}}` | coreai.json: qualityChecks.build | "./gradlew assembleDebug" |
+
+### Agent-Specific Placeholders
+
+Used in agent templates (`_templates/`):
+
+| Placeholder | Purpose |
+|-------------|---------|
+| `{{AGENT_NAME}}` | Lowercase hyphenated name (e.g., "backend-engineer") |
+| `{{AGENT_DISPLAY_NAME}}` | Title case name (e.g., "Backend Engineer") |
+| `{{AGENT_DESCRIPTION}}` | Role description |
+| `{{TEAM_MEMBERS_LIST}}` | List of team members (EM only) |
+
+### Substitution Process
+
+`install.sh` and `add-agent.sh` use `sed` to replace placeholders:
+
+```bash
+sed -i.bak \
+    -e "s|{{PROJECT_NAME}}|$PROJECT_NAME|g" \
+    -e "s|{{JIRA_PROJECT}}|$JIRA_PROJECT|g" \
+    "$TARGET_FILE"
+```
+
+---
+
+## File Structure
+
+### Framework Files (coreai/)
+
+```
+coreai/
+├── README.md              # Installation guide
+├── ARCHITECTURE.md        # This file
+├── AGENT_SPEC.md          # Agent behavior specification
+├── WORKFLOWS.md           # Workflow state machines
+│
+├── scripts/
+│   ├── install.sh         # Main installer
+│   └── add-agent.sh       # Add new agent
+│
+├── agents/
+│   ├── _templates/               # Agent templates for add-agent.sh
+│   │   ├── ic-engineer.md        # IC engineer template
+│   │   └── reviewer.md           # Reviewer template
+│   └── examples/                 # Ready-to-use agents (copied on install)
+│       ├── engineering-manager.md  # Required core agent
+│       ├── backend-engineer.md
+│       ├── android-engineer.md
+│       ├── wearos-engineer.md
+│       ├── devops-engineer.md
+│       ├── solutions-architect.md
+│       ├── qa-engineer.md
+│       ├── security-engineer.md
+│       ├── product-manager.md
+│       └── frontend-engineer.md
+│
+├── commands/
+│   ├── core/              # Always installed
+│   │   ├── delegate.md
+│   │   ├── check-inbox.md
+│   │   ├── review.md
+│   │   ├── sprint-status.md
+│   │   ├── git-commit.md
+│   │   └── pr-create.md
+│   └── optional/          # Install as needed
+│       ├── worktree-setup.md
+│       ├── worktree-cleanup.md
+│       ├── jira-create.md
+│       ├── jira-transition.md
+│       └── docs-update.md
+│
+├── templates/
+│   ├── inbox-messages/    # Message templates
+│   └── project-files/     # Config templates
+│
+└── knowledge-library/     # KnowledgeLibrary templates
+```
+
+### Installed Files (Target Project)
+
+```
+project/
+├── .claude/
+│   ├── agents/            # Copied from coreai/agents/
+│   └── commands/          # Copied from coreai/commands/
+│
+├── KnowledgeLibrary/      # Created by install.sh
+│   ├── context.txt
+│   ├── architecture.txt
+│   ├── prd.txt
+│   └── <agent>/           # Per-agent directories
+│
+├── coreai.json            # Project configuration
+└── .mcp.json              # MCP server configuration
+```
+
+---
+
+## Key Design Decisions
+
+### 1. Engineers Own Ticket Lifecycle
+
+The Engineering Manager delegates tasks but does NOT move tickets through states. Engineers are responsible for:
+
+- Moving ticket to "In Progress" when starting
+- Creating the PR
+- Moving ticket to "In Review" when PR is ready
+
+This reduces back-and-forth and gives engineers ownership.
+
+### 2. File-Based Over Database
+
+Using files for communication instead of a database:
+
+- **Pro:** Works offline, no infrastructure needed
+- **Pro:** Human-readable, auditable
+- **Pro:** Git-trackable for history
+- **Con:** No real-time notifications
+- **Con:** Requires manual processing
+
+### 3. Validation Checkpoints
+
+The EM validates completion reports before assigning reviewers:
+
+- Ticket is in correct state
+- PR exists and CI is passing
+- All acceptance criteria addressed
+
+This catches issues early and enforces workflow compliance.
+
+### 4. Quality-Gated Commits
+
+The `/git-commit` command requires all quality checks to pass:
+
+1. ktlint/linting
+2. Static analysis (detekt)
+3. Unit tests
+4. Build
+
+This prevents broken code from being committed.
+
+---
+
+## Modifying the Framework
+
+### Adding a New Command
+
+1. Create `coreai/commands/core/<command>.md` or `optional/<command>.md`
+2. Add placeholders for project-specific values
+3. Update `install.sh` to copy the command
+4. Update `README.md` command list
+
+### Adding a New Agent Template
+
+1. Create `coreai/agents/_templates/<type>.md`
+2. Include startup protocol, work patterns, completion protocol
+3. Use `{{AGENT_NAME}}` and other placeholders
+4. Update `add-agent.sh` to support the new type
+
+### Adding a New Placeholder
+
+1. Add to `coreai.json` template (`templates/project-files/coreai.json.example`)
+2. Add `sed` replacement in `install.sh` and `add-agent.sh`
+3. Document in this file's Placeholders section
+
+### Changing Workflow States
+
+1. Update `coreai/WORKFLOWS.md` with new states
+2. Update affected commands (especially `/jira-transition`)
+3. Update agent templates that reference workflows
+
+### Adding MCP Integration
+
+1. Add server config to `templates/project-files/mcp.json.template`
+2. Document required environment variables
+3. Update agent frontmatter to include new tools
+4. Add usage examples to relevant commands
+
+---
+
+## Troubleshooting
+
+### Agent Can't Access Jira/GitHub
+
+**Cause:** Agent was invoked via Task tool instead of directly.
+
+**Fix:** Always use `@agent-name` to invoke agents, never Task tool.
+
+### Placeholders Not Replaced
+
+**Cause:** `sed` command failed or placeholder syntax wrong.
+
+**Fix:** Check for typos in placeholder names. Ensure `{{` and `}}` are used.
+
+### Inbox Messages Not Found
+
+**Cause:** Wrong directory path or filename convention.
+
+**Fix:** Verify KnowledgeLibrary structure exists. Check filename follows `YYYYMMDD_HHMM-agent-subject.md` format.
+
+### Quality Checks Failing
+
+**Cause:** Commands in `coreai.json` don't match project setup.
+
+**Fix:** Update `qualityChecks` in `coreai.json` to match actual project commands.
+
+---
+
+## CLI Development Status
+
+The CoreAI CLI (`@deimoscloud/coreai`) is under active development. Current implementation status:
+
+### Implemented
+
+- **Configuration System** - YAML-based config loading with environment variable support
+- **Agent System** - Agent definition loading, compilation, and resolution
+- **Adapter Interfaces** - Abstract interfaces for issue trackers, git providers, documentation, and state
+- **MCP Integration** - Server discovery, client wrapper, and tool-to-adapter mapping
+- **MCP Server Registry** - Central registry for managing MCP server connections
+- **Native Adapter Fallbacks** - Filesystem and GitHub adapters when MCP servers unavailable
+- **Cache System** - CacheProvider interface, FileCacheProvider implementation with full CRUD operations
+- **Context Loader** - CacheManager and ContextLoader with cache-first resolution strategy
+- **Sync Commands** - `coreai sync`, `coreai cache clear`, `coreai cache status` CLI commands
+- **Command Framework** - Command registration, runtime config, graceful degradation
+- **Core CLI Commands** - `coreai init`, `coreai build`, `coreai validate`
+- **Skill Generation** - Generate Claude skills from config to `.claude/commands/`
+- **KnowledgeLibrary** - Local agent state management with inbox/outbox directories
+- **State Commands** - `coreai status` for agent states and pending messages
+- **CLI Binary** - Complete command-line interface with all Phase 1 commands
+
+### CLI Commands Reference
+
+| Command | Description |
+|---------|-------------|
+| `coreai init` | Initialize a new CoreAI project |
+| `coreai build` | Compile agent definitions to `.claude/agents/` |
+| `coreai build --init-knowledge-library` | Build agents + create KnowledgeLibrary directories |
+| `coreai validate` | Validate config and project setup |
+| `coreai status` | Show agent states and pending messages |
+| `coreai status --detailed` | Show detailed message information |
+| `coreai sync` | Sync shared context from remote sources |
+| `coreai cache status` | Show cache statistics |
+| `coreai cache clear` | Clear the local cache |
+| `coreai agents list` | List available agents |
+| `coreai agents show <name>` | Show agent details |
+| `coreai skills generate` | Generate Claude skills from templates |
+| `coreai skills list` | List available skills |
+
+---
+
+*Last Updated: 2026-01-22*