kc-beta 0.2.1 → 0.3.1

package/QUICKSTART.md

@@ -0,0 +1,149 @@
+# KC Agent CLI (Beta) — Quickstart
+
+## Install
+
+```bash
+npm install -g kc-beta
+```
+
+Requires **Node.js 20+**.
+
+## Setup
+
+Run the onboarding wizard to configure your LLM provider and API keys:
+
+```bash
+kc-beta onboard
+```
+
+You'll be prompted to choose:
+
+1. **Language** — English or 中文
+2. **LLM Provider** — SiliconFlow, Aliyun Bailian, Anthropic, OpenAI, VolcanoCloud, Zhipu GLM, MiniMax, OpenRouter, or custom
+3. **API Key** — your provider API key (supports both API keys and Aliyun/VolcanoCloud coding plan keys)
+4. **Model Discovery** — KC auto-discovers available models via API or curated lists and suggests tier assignments with capability-based ranking
+5. **Conductor Model** — the main model that drives the agent
+6. **Worker LLM Tiers** — tier1 (best) through tier4 (cheapest) for verification tasks
+7. **VLM Tiers** — vision models for OCR/document parsing (tier1-3)
+8. **Worker Provider** (optional) — use a different provider for worker LLMs (defaults to conductor provider)
+
+Config is saved to `~/.kc_agent/config.json` and shared across projects.
+
+### Edit Settings Later
+
+```bash
+kc-beta config
+```
+
+Category-based editor for: LLM provider, model tiers, VLM tiers, worker LLM provider, quality thresholds, language.
+
+## Create a Project
+
+```bash
+kc-beta init my-project
+kc-beta init my-project --lang=zh   # Chinese skills
+```
+
+This creates a workspace with:
+
+```
+my-project/
+  .env              # Project-level config (overrides global)
+  Rules/            # Put regulation documents here
+  Samples/          # Put sample documents here
+  Input/            # Production batches
+  Output/           # Verification results
+  skills/           # Meta-methodology skills (en or zh)
+```
+
+## Start the Agent
+
+```bash
+kc-beta             # default language from config
+kc-beta --en        # this session in English (does not change config)
+kc-beta --zh        # this session in Chinese (does not change config)
+```
+
+Launch from your project directory — KC has full read/write access to the folder you launch from:
+
+```bash
+cd my-project
+kc-beta
+```
+
+The agent starts in **BOOTSTRAP** phase. It will:
+
+1. Set up the workspace structure
+2. Detect regulations and samples in your project directory
+3. Ask about your verification scenario
+
+Once regulations and samples are in place, the agent advances through 6 phases automatically:
+
+| Phase | What happens |
+|-------|-------------|
+| **BOOTSTRAP** | Workspace setup, understand the scenario |
+| **EXTRACTION** | Decompose regulations into atomic rules |
+| **SKILL_AUTHORING** | Write verification skills for each rule |
+| **SKILL_TESTING** | Test skills, iterate via evolution loop |
+| **DISTILLATION** | Convert skills to worker LLM workflows |
+| **PRODUCTION_QC** | Run workflows on production docs with QC |
+
+## Slash Commands
+
+| Command | Description |
+|---------|-------------|
+| `/help` | Show available commands |
+| `/status` | Session info, model, phase, context usage |
+| `/clear` | Clear conversation history |
+| `/compact` | Summarize older messages to reduce context usage |
+| `/sessions` | List all sessions |
+| `/resume <name>` | Resume a previous session (restores phase + pipeline state) |
+| `/rename <name>` | Rename current session |
+| `/exit` | Save state and quit |
+
+## Keyboard Shortcuts
+
+- **Enter** — Send message
+- **Ctrl+C** — Clear queue (if streaming) or save & exit
+- **Ctrl+D** — Save & exit
+
+## Status Bar
+
+The status bar shows:
+- Session ID and current phase
+- **Context usage**: `CTX: 45.2k/200k (23%)` — turns green/yellow/red as context fills
+
+## Per-Project Config
+
+Override global settings in your project's `.env`:
+
+```env
+LLM_API_KEY=sk-xxx
+LLM_BASE_URL=https://api.siliconflow.cn/v1
+
+TIER1=Pro/zai-org/GLM-5
+TIER2=
+TIER3=
+TIER4=
+
+SKILL_ACCURACY=0.9
+WORKFLOW_ACCURACY=0.9
+MAX_ITERATIONS=20
+
+# Optional: web search via Tavily
+TAVILY_API_KEY=tvly-xxx
+```
+
+Legacy keys (`SILICONFLOW_API_KEY`, `SILICONFLOW_BASE_URL`) are still accepted for backward compatibility.
+
+## Web Search
+
+KC can search the web using Tavily when information is not available in your provided documents. Set `TAVILY_API_KEY` in your `.env` or global config. KC prioritizes your domain documents over web results.
+
+## Troubleshooting
+
+- **"No API key configured"** — Run `kc-beta onboard` first
+- **Connection errors** — Check your API key and base URL. KC retries up to 10 times with exponential backoff on transient failures.
+- **Context too long** — Use `/compact` to summarize older messages, or let automatic windowing handle it
+- **Resume after crash** — Use `/resume <session-name>` to pick up where you left off
+- **Node version** — Requires Node.js 20+. Check with `node --version`

package/README.md

@@ -0,0 +1,207 @@
+# KC Agent CLI (`kc-beta`)
+
+> Build, distill, and run document verification systems with an LLM agent.
+> Pure Node.js. One binary. Bring your own model.
+
+KC is a coding agent purpose-built for **rule-based document verification**:
+read a regulation, decompose it into atomic verification rules, write skills
+to check each rule against sample documents, and (optionally) distill those
+skills into cheap worker-LLM workflows for production batch processing.
+
+It is designed for the developer at a bank, insurer, or law firm who needs
+to verify hundreds of documents against dozens of compliance rules — and
+wants the system to be transparent, testable, and ownable.
+
+---
+
+## Quick Install
+
+```bash
+npm install -g kc-beta
+kc-beta onboard      # configure provider + API key
+cd my-project        # a folder containing rules/ and samples/
+kc-beta              # launch the agent
+```
+
+Requires **Node.js 20+**. See [QUICKSTART.md](./QUICKSTART.md) for the full setup walkthrough.
+
+---
+
+## What It Does
+
+KC drives a single coding agent through six phases:
+
+| Phase | What it does |
+|-------|-------------|
+| **BOOTSTRAP**       | Set up the workspace, detect rules/samples in your project |
+| **EXTRACTION**      | Decompose regulation documents into atomic, testable rules |
+| **SKILL_AUTHORING** | Write a verification skill for each rule (Anthropic skill-creator format) |
+| **SKILL_TESTING**   | Run skills on samples, iterate via the evolution loop |
+| **DISTILLATION**    | Convert proven skills into cheap worker-LLM workflows |
+| **PRODUCTION_QC**   | Run workflows on production batches with confidence-based sampling |
+
+The conductor LLM (your main model) drives all reasoning. Worker LLM tools
+are gated to DISTILL phases only, so the build phase is always grounded in
+high-quality output.
+
+---
+
+## Architecture
+
+```
+~/.kc_agent/
+  config.json                       # provider, API key, model tiers
+  workspaces/<sessionId>/           # KC's working files
+    rules/, rule_skills/, workflows/, samples/, output/, logs/
+    AGENT.md                        # per-project context (KC can edit)
+    tasks.json                      # ralph-loop task list
+    session-state.json              # phase + pipeline state for /resume
+
+your-project/                       # where you launched kc-beta
+  rules/        # source regulations (KC reads with scope="project")
+  samples/      # sample documents
+  Output/       # KC writes user-facing reports here
+```
+
+**Dual-directory design.** KC has full read/write to its own workspace plus
+*scoped* read/write to your project directory. Source files stay in your
+project; KC's working artifacts stay in `~/.kc_agent/workspaces/`.
+
+**Phase-gated tools.** Worker LLM, workflow runner, tier downgrade, and QC
+sampling tools only register during DISTILL phases. BUILD phases force the
+conductor to do the intellectual work directly — the results are the
+ground-truth baseline for distillation.
+
+**Skills as first-class deliverables.** Every rule produces a self-contained
+skill folder (SKILL.md + scripts + references + samples). For complex rules
+that worker LLMs can't reliably handle, the skill itself — run by a capable
+agent — is the production solution.
+
+---
+
+## Provider Support
+
+10 providers configured out of the box:
+
+- **SiliconFlow** (default, recommended for China)
+- **Aliyun Bailian** (with coding-plan key support)
+- **VolcanoCloud** (ByteDance Doubao)
+- **Anthropic** (Messages API native)
+- **OpenAI**
+- **Zhipu GLM**
+- **MiniMax**
+- **OpenRouter**
+- **AWS Bedrock** (stub)
+- **Custom** (any OpenAI-compatible endpoint)
+
+Model assignments live in [`src/model-tiers.json`](./src/model-tiers.json) —
+edit directly to update tier-1 through tier-4 LLM and tier-1 through tier-3
+VLM (vision) models per provider, no code changes needed.
+
+You can use **separate providers** for the conductor and worker LLMs (e.g.,
+Anthropic conductor + SiliconFlow workers).
+
+---
+
+## Ralph-Loop Autonomous Execution
+
+When KC extracts rules, it automatically generates a per-rule task list and
+processes them one at a time. Between tasks the conductor's context is
+compacted aggressively, so context stays bounded even with 50+ rules.
+
+```
+SKILL_AUTHORING  [████████░░░░] 8/12
+✓ R001  Registered capital check
+✓ R002  Net asset adequacy
+▸ R003  Related-party disclosure   ← current
+·  R004  Risk capital calculation
+...
+```
+
+Use `/tasks` to see the full list. The agent decides *how* to do each task;
+the task manager only tells it *what's next*.
+
+---
+
+## Slash Commands
+
+```
+/help                Show available commands
+/status              Session, model, phase, context usage
+/tasks               Show task list and progress
+/clear               Clear conversation (workspace preserved)
+/compact             Summarize older messages via the conductor
+/sessions            List all sessions
+/resume <name>       Resume a previous session
+/rename <name>       Rename current session
+/exit                Save state and quit
+```
+
+`--en` / `--zh` flags override language for one session without writing config.
+
+---
+
+## Optional Plugins
+
+Some heavyweight features ship as **meta-meta skills** the agent invokes on
+demand, rather than always-on dependencies:
+
+- **`pdf-review-dashboard`** — Two-column HTML dashboard (PDF on the left,
+  verification results on the right, click-to-jump) for manual review and
+  ground-truth collection.
+- **`auto-model-selection`** — Use [Context7](https://github.com/upstash/context7)
+  CLI to fetch current model listings when the bundled `model-tiers.json`
+  is stale or you've switched providers.
+
+Both are bundled in `template/skills/{en,zh}/meta-meta/` and discovered by
+the skill loader at startup.
+
+---
+
+## Configuration
+
+Global config: `~/.kc_agent/config.json` (set by `kc-beta onboard`).
+Per-project override: `<project>/.env`.
+
+Edit anytime with the category-based editor:
+
+```bash
+kc-beta config
+```
+
+Categories: LLM Provider, Model Tiers, VLM Tiers, Worker LLM Provider,
+Quality Thresholds, Language.
+
+---
+
+## Documentation
+
+- [QUICKSTART.md](./QUICKSTART.md) — full setup and slash command reference
+- [DEV_LOG.md](./DEV_LOG.md) — release history and design rationale
+- [docs/global_update_design_v3.md](./docs/global_update_design_v3.md) — v3 design plan and progress tracker
+- [docs/initial_spec_draft.md](./docs/initial_spec_draft.md) — original architectural spec
+
+---
+
+## Status
+
+**v0.3.1 — beta.** Production-readiness update covering the seven blocks
+of the v3 design plan: dual-directory permissions, AGENT.md per-project
+context, PDF review dashboard skill, parsing/extraction skill rewrites,
+production-experience meta-skill polish, model-tier baseline + Context7
+plugin, and ralph-loop autonomous task execution.
+
+We are inviting a small group of developer users to test before public launch.
+Bug reports and PRs welcome at <https://github.com/kitchen-engineer42/kc-cli>.
+
+---
+
+## License
+
+MIT. Bundled meta-skills under `template/skills/` are proprietary —
+distributed via npm but not open-source. See `template/skills/LICENSE` for
+terms.
+
+---
+
+*Built by Memium / kitchen-engineer42.*

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "kc-beta",
-  "version": "0.2.1",
+  "version": "0.3.1",
   "description": "KC Agent — LLM document verification agent (pure Node.js CLI)",
   "type": "module",
   "bin": {
@@ -9,8 +9,18 @@
   "files": [
     "bin/",
     "src/",
-    "template/"
+    "template/",
+    "README.md",
+    "QUICKSTART.md"
   ],
+  "homepage": "https://github.com/kitchen-engineer42/kc-cli",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/kitchen-engineer42/kc-cli.git"
+  },
+  "bugs": {
+    "url": "https://github.com/kitchen-engineer42/kc-cli/issues"
+  },
   "engines": {
     "node": ">=20.0.0"
   },

package/src/agent/context.js

@@ -32,9 +32,11 @@ outcome. Handle ambiguity explicitly — note it, ask the developer user. After
 audit which regulation sections are not yet covered.
 
 ### Entity Extraction
-Prefer regex/Python for predictable formats. Use LLM only when semantic understanding \
-is required. Every extraction captures: value, evidence, source location, confidence, \
-method used.
+Choose the cheapest method that meets accuracy threshold. Regex is the smallest \
+"model" — zero cost, instant, deterministic. Worker LLM handles semantic tasks \
+regex cannot (contextual interpretation, misleading language, adequacy judgment). \
+Try different methods, find the cost-accuracy balance. Every extraction captures: \
+value, evidence, source location, confidence, method used.
 
 ### Skill Authoring
 Write each rule into a skill folder following the Anthropic skill-creator format. A \
@@ -79,13 +81,15 @@ unclear regulations with them. Present results and let them judge.`;
 export class ContextAssembler {
   /**
    * @param {object} [opts]
+   * @param {string} [opts.agentMd] - Content of workspace AGENT.md (per-project context)
    * @param {string} [opts.pipelineState]
    * @param {string} [opts.workspaceState]
    * @param {string} [opts.skillIndex] - Brief index of available meta skills
    * @returns {string}
    */
-  build({ pipelineState, workspaceState, skillIndex } = {}) {
+  build({ agentMd, pipelineState, workspaceState, skillIndex } = {}) {
     const parts = [AGENT_IDENTITY];
+    if (agentMd) parts.push(agentMd);
     if (skillIndex) parts.push(skillIndex);
     if (pipelineState) parts.push(pipelineState);
     if (workspaceState) parts.push(workspaceState);

package/src/agent/engine.js

@@ -1,3 +1,5 @@
+import fs from "node:fs";
+import path from "node:path";
 import { AgentEvent } from "./events.js";
 import { ContextAssembler } from "./context.js";
 import { ConversationHistory } from "./history.js";
@@ -20,6 +22,7 @@ import { TierDowngradeTool } from "./tools/tier-downgrade.js";
 import { AgentTool } from "./tools/agent-tool.js";
 import { WebSearchTool } from "./tools/web-search.js";
 import { SkillLoader } from "./skill-loader.js";
+import { TaskManager } from "./task-manager.js";
 import { Phase } from "./pipelines/index.js";
 import { ProjectInitializer } from "./pipelines/initializer.js";
 import { RuleExtractionPipeline } from "./pipelines/extraction.js";
@@ -56,7 +59,7 @@ export class AgentEngine {
     this.context = new ContextAssembler();
 
     // Workspace + structural components
-    this.workspace = new Workspace(config.kcWorkspaceRoot, sessionId);
+    this.workspace = new Workspace(config.kcWorkspaceRoot, sessionId, config.projectDir);
     this.history = new ConversationHistory(this.workspace.cwd);
     this.versionManager = new VersionManager(this.workspace.cwd);
     this.cornerCases = new CornerCaseRegistry(this.workspace.cwd);
@@ -74,6 +77,9 @@ export class AgentEngine {
     // Session state persistence
     this.sessionState = new SessionState(this.workspace.cwd);
 
+    // Task manager (ralph-loop)
+    this.taskManager = new TaskManager(this.workspace.cwd);
+
     // Build all tool instances (but register phase-appropriate ones)
     this._buildTools = this._createAllTools();
     this._phaseSummaries = [];
@@ -102,12 +108,20 @@ export class AgentEngine {
    * re-register per phase without recreating.
    */
   _createAllTools() {
+    // Worker LLM uses separate config if set, otherwise falls back to conductor
+    const workerApiKey = this.config.effectiveWorkerApiKey();
+    const workerBaseUrl = this.config.effectiveWorkerBaseUrl();
+    const workerAuthType = this.config.effectiveWorkerAuthType();
+
     const workerLlm = new WorkerLLMCallTool(this.workspace, {
-      apiKey: this.config.llmApiKey,
-      baseUrl: this.config.llmBaseUrl,
-      authType: this.config.authType,
+      apiKey: workerApiKey,
+      baseUrl: workerBaseUrl,
+      authType: workerAuthType,
     });
 
+    // OCR/VLM uses worker config (VLM is a type of worker LLM)
+    const vlmModel = this.config.vlmTier1 || "";
+
     return {
       // Always available (BUILD + DISTILL)
       core: [
@@ -116,9 +130,9 @@ export class AgentEngine {
         new DocumentParseTool(this.workspace, {
           mineruApiUrl: this.config.mineruApiUrl,
           mineruApiKey: this.config.mineruApiKey,
-          llmApiKey: this.config.llmApiKey,
-          llmBaseUrl: this.config.llmBaseUrl,
-          ocrModel: this.config.ocrModelTier1,
+          llmApiKey: workerApiKey,
+          llmBaseUrl: workerBaseUrl,
+          ocrModel: vlmModel,
         }),
         new DocumentSearchTool(this.workspace),
         new RuleCatalogTool(this.workspace),
@@ -156,15 +170,57 @@ export class AgentEngine {
     }
   }
 
+  /**
+   * Read AGENT.md from workspace (per-project context).
+   * Returns content string or empty string if not found.
+   */
+  _readAgentMd() {
+    const agentMdPath = path.join(this.workspace.cwd, "AGENT.md");
+    try {
+      if (fs.existsSync(agentMdPath)) {
+        return fs.readFileSync(agentMdPath, "utf-8");
+      }
+    } catch { /* ignore */ }
+    return "";
+  }
+
+  /**
+   * Build the workspace/project directory state string for the system prompt.
+   */
+  _buildWorkspaceState() {
+    const lines = [
+      `## Directory Layout`,
+      `**KC Workspace:** ${this.workspace.cwd}`,
+      `  Use scope="workspace" (default). Write all working files here (rules, skills, workflows, results, logs).`,
+    ];
+    if (this.workspace.projectDir) {
+      lines.push(
+        `**Project Directory:** ${this.workspace.projectDir}`,
+        `  Use scope="project" to read/write files in the user's project folder.`,
+        `  This is where the user's source regulations, samples, and reference documents are.`,
+        ``,
+        `Read source documents from the project directory. Write KC outputs to the workspace.`,
+        `Write user-facing exports (reports, results) to the project directory when the user asks.`,
+      );
+    }
+
+    // Task progress (ralph-loop)
+    const taskContext = this.taskManager.describeForContext();
+    if (taskContext) lines.push("", taskContext);
+
+    return lines.join("\n");
+  }
+
   /**
    * Get current context usage statistics.
    * @returns {{ totalTokens: number, limit: number, percentage: number }}
    */
   getContextStats() {
     const systemPrompt = this.context.build({
+      agentMd: this._readAgentMd(),
       skillIndex: this._skillLoader.formatForContext(),
       pipelineState: this.pipelines[this.currentPhase]?.describeState?.() || null,
-      workspaceState: `Your workspace directory is: ${this.workspace.cwd}`,
+      workspaceState: this._buildWorkspaceState(),
     });
     const systemTokens = estimateTokens(systemPrompt);
     const messageTokens = estimateMessagesTokens(this.history.messages);
@@ -270,6 +326,14 @@ export class AgentEngine {
       engine._phaseSummaries = data.phaseSummaries || [];
       engine._registerToolsForPhase(engine.currentPhase);
 
+      // Restore project directory from saved state
+      if (data.projectDir) {
+        if (fs.existsSync(data.projectDir)) {
+          engine.workspace.projectDir = data.projectDir;
+        }
+        // If dir no longer exists, projectDir stays as whatever was passed at launch
+      }
+
       // Restore pipeline milestones
       const milestones = data.pipelineMilestones || {};
       for (const [phase, mData] of Object.entries(milestones)) {
@@ -309,9 +373,10 @@ export class AgentEngine {
     const pipelineState = pipeline?.describeState?.() || null;
 
     const systemPrompt = this.context.build({
+      agentMd: this._readAgentMd(),
       skillIndex: this._skillLoader.formatForContext(),
       pipelineState,
-      workspaceState: `Your workspace directory is: ${this.workspace.cwd}`,
+      workspaceState: this._buildWorkspaceState(),
     });
     const tools = this.toolRegistry.schemasOpenai();
 
@@ -433,6 +498,10 @@ export class AgentEngine {
                 });
                 this.currentPhase = pEvent.nextPhase;
                 this._registerToolsForPhase(this.currentPhase);
+
+                // Ralph-loop: create per-rule tasks for the new phase
+                this._createTasksForPhase(this.currentPhase);
+
                 this.saveState();
               }
               yield new AgentEvent({
@@ -450,4 +519,80 @@ export class AgentEngine {
       }
     }
   }
+
+  /**
+   * Create per-rule tasks when entering a new phase.
+   * Reads the rule catalog and creates one task per rule for the given phase.
+   */
+  _createTasksForPhase(phase) {
+    const catalogPath = path.join(this.workspace.cwd, "rules", "catalog.json");
+    if (!fs.existsSync(catalogPath)) return;
+
+    try {
+      const catalog = JSON.parse(fs.readFileSync(catalogPath, "utf-8"));
+      const rules = Array.isArray(catalog) ? catalog : [];
+      if (rules.length > 0) {
+        this.taskManager.createRuleTasks(rules, phase);
+      }
+    } catch { /* skip if catalog can't be read */ }
+  }
+
+  /**
+   * Ralph-loop: run a turn, then auto-continue through pending tasks.
+   * Compacts context aggressively between tasks to prevent context blowup.
+   * If no tasks exist, behaves identically to runTurn().
+   *
+   * @param {string} userMessage
+   * @yields {AgentEvent}
+   */
+  async *runTaskLoop(userMessage) {
+    // Run the initial turn (user's request)
+    yield* this.runTurn(userMessage);
+
+    // Auto-continue through pending tasks
+    while (this.taskManager.getNextPending()) {
+      // Context safety: force compaction if above 70%, or light compaction if history is long
+      const stats = this.getContextStats();
+      if (stats.percentage > 70) {
+        await this.compact();
+      } else if (this.history.messages.length > 15) {
+        await this.compact({ recentCount: 8 });
+      }
+
+      const task = this.taskManager.getNextPending();
+      this.taskManager.updateTask(task.id, { status: "in_progress" });
+
+      // Yield task progress event for TUI
+      yield new AgentEvent({
+        type: "task_progress",
+        data: {
+          taskId: task.id,
+          title: task.title,
+          ruleId: task.ruleId,
+          status: "in_progress",
+          progress: this.taskManager.progress,
+        },
+      });
+
+      // Synthesize a task-focused prompt
+      const taskPrompt = `Continue with next task: ${task.title}` +
+        (task.ruleId ? ` (rule: ${task.ruleId})` : "");
+
+      yield* this.runTurn(taskPrompt);
+
+      this.taskManager.updateTask(task.id, { status: "completed" });
+      this.taskManager.save();
+      this.saveState();
+
+      yield new AgentEvent({
+        type: "task_progress",
+        data: {
+          taskId: task.id,
+          title: task.title,
+          status: "completed",
+          progress: this.taskManager.progress,
+        },
+      });
+    }
+  }
 }