@hongmaple0820/scale-engine 0.25.0 → 0.27.0

package/docs/AI_ENGINEERING_OS_POSITIONING.md

@@ -0,0 +1,462 @@
+# SCALE Engine Strategic Positioning
+
+> Date: 2026-05-20  
+> Status: strategic direction with a v0.27.0 runtime baseline
+> Audience: maintainers, contributors, roadmap reviewers, and product-facing documentation owners
+
+SCALE Engine should be positioned as an **Agent Governance Runtime** evolving toward an **AI Engineering OS**.
+
+The project is no longer best described as a prompt toolbox. Its durable value is the runtime layer around AI coding agents:
+
+- workflow state machines
+- hard gates and verification evidence
+- hook-based tool interception
+- role and permission boundaries
+- artifact persistence
+- context budgets
+- memory provider routing
+- skill, MCP, CLI, and adapter orchestration
+
+The core thesis is:
+
+> Use system constraints, evidence, and runtime gates to replace agent self-discipline.
+
+This positioning is intentionally stronger than "prompt engineering", but it must stay evidence-backed. SCALE can describe the direction as AI Engineering OS; it should only describe measurable gains after benchmark, eval, or runtime evidence exists.
+
+## Reference Inputs
+
+This document consolidates:
+
+- the current SCALE Engine architecture and documentation surfaces
+- maintainer review of SCALE as an Agent Governance Runtime
+- the external Harness Engineering framing in [SCALE OS v10.0: AI 编码的认知操作系统](https://segmentfault.com/a/1190000047756584)
+
+External performance claims from ecosystem articles are positioning inputs, not SCALE Engine release claims. Public SCALE claims should still be backed by local evals, runtime evidence, or release reports.
+
+## 1. Market Category
+
+SCALE sits between four emerging categories:
+
+| Category | SCALE relationship |
+| --- | --- |
+| AI Engineering OS | Long-term positioning: one governed operating layer for agent-driven engineering |
+| Agent Governance Runtime | Current strongest fit: gates, hooks, evidence, role boundaries, and policy enforcement |
+| Workflow Orchestration Runtime | Current fit: FSM, phase commands, artifacts, verification, and ship flow |
+| Harness Engineering Infrastructure | Methodology fit: constraints + feedback + workflow + continuous improvement |
+
+SCALE should avoid being framed as only:
+
+- prompt templates
+- agent rules
+- a Claude/Cursor/Codex config generator
+- an AutoGPT-style chain executor
+- a generic skills catalog
+
+Those may exist around the project, but they are not the core defensible layer.
+
+## 2. Problem Definition
+
+AI coding failures are not only model-quality failures. They are engineering runtime failures:
+
+| Failure mode | Typical prompt-only response | SCALE response |
+| --- | --- | --- |
+| Fake completion | "Please verify before finishing" | verification gates and final evidence checks |
+| Skipped tests | reminder text | FSM and verification status before completion |
+| Repeated blind retries | "try a different approach" | retry and behavior detectors |
+| Context overload | longer instructions | context budgets, lazy loading, scoped packs |
+| Agent drift | more rules | persisted workflow state and phase boundaries |
+| Hallucinated delivery | review prompt | runtime evidence ledger and ship gates |
+| Lost learning | chat history | memory artifacts, failure replay, lessons, rule candidates |
+| Multi-agent confusion | role descriptions | role gateway and tool permission boundaries |
+| Tool overreach | trust agent judgment | hook interception and policy gateway |
+
+The strategic target is not to make the model "more obedient". The target is to make non-compliant behavior observable, blockable, and recoverable.
+
+## 3. Current Strengths
+
+### 3.1 Runtime Constraints
+
+SCALE already has the right architectural instinct: lower critical rules from prompt text into runtime checks.
+
+Relevant surfaces:
+
+- `docs/ENGINEERING_STANDARDS.md`
+- `docs/RUNTIME_EVIDENCE.md`
+- `docs/DEPENDENCY_AUDIT.md`
+- `src/workflow/gates/GateSystem.ts`
+- `src/guardrails/Gateway.ts`
+- `src/artifact/fsm.ts`
+
+This is the primary moat. Prompt rules can be ignored; runtime gates can block progress.
+
+### 3.2 Workflow State Machine
+
+The workflow is driven by artifact state, not by chat momentum.
+
+Strategic value:
+
+- prevents premature completion
+- forces phase-specific evidence
+- makes stalled or skipped phases visible
+- supports resume and handoff across long sessions
+- gives agent platforms a shared lifecycle model
+
+The FSM should remain strict at phase boundaries and flexible inside each phase.
+
+### 3.3 Hook and Gateway Layer
+
+Hooks, pre-tool checks, post-tool checks, stop checks, and role-aware gateway decisions form the AI runtime interceptor layer.
+
+Strategic value:
+
+- agents do not receive raw, unlimited tool authority
+- unsafe operations can be blocked before execution
+- tool output can be converted into evidence
+- repeated failure patterns can be detected outside the model
+
+This layer makes SCALE closer to an admission controller than a prompt pack.
+
+### 3.4 Evidence-Backed Delivery
+
+SCALE's strongest anti-hallucination capability is engineering hallucination control:
+
+- no test evidence means no verified claim
+- no runtime evidence means no product-smoke claim
+- no reviewed file scope means no governed ship
+- no dependency audit evidence means weaker security confidence
+
+This reduces fake completion more reliably than instruction text.
+
+It does not fully solve reasoning hallucination. Architecture decisions, root-cause analysis, and technical tradeoffs still need evaluator intelligence.
+
+### 3.5 Adapter and Platform Surface
+
+The agent-platform adapters let SCALE act as a shared governance layer for different coding agents.
+
+Strategic value:
+
+- one governance model across Claude Code, Codex, Cursor, Gemini, Windsurf, Kiro, Cline, and related tools
+- fewer duplicated rule files
+- lower switching cost between agents
+- consistent evidence and workflow semantics
+
+Adapter expansion should not become the main roadmap by itself. The strategic value comes from shared governance semantics, not from the count of supported agents.
+
+## 4. Honest Capability Assessment
+
+SCALE can already claim:
+
+- stronger engineering governance than prompt-only rules
+- structured workflow execution with phase and artifact state
+- hard verification gates for delivery claims
+- evidence-based runtime reporting
+- first-class supply-chain audit direction
+- growing adapter coverage
+- memory and skill orchestration foundations
+
+SCALE should not yet overclaim:
+
+- fully autonomous self-evolution
+- human-level long-term memory
+- guaranteed token reduction percentages
+- guaranteed hallucination reduction percentages
+- adaptive cognitive planning
+- universal skill routing intelligence
+
+Use target ranges only in roadmap or evaluation documents, not as product claims, until eval evidence supports them.
+
+## 5. Current Gaps
+
+### 5.1 Memory Architecture
+
+Current state is closer to engineering knowledge persistence than true cognitive memory.
+
+Existing strengths:
+
+- artifacts persist decisions and work state
+- memory brain stores evidence-backed learnings
+- failure replay can preserve incidents
+- provider routing gives the right extension point
+
+Missing layers:
+
+| Memory type | Target meaning |
+| --- | --- |
+| Working memory | short-lived task context with strict token budget |
+| Episodic memory | past task episodes, failures, fixes, and outcomes |
+| Semantic memory | stable project facts and domain concepts |
+| Procedural memory | reusable ways of doing work |
+| Strategy memory | learned routing, verification, and recovery strategies |
+
+The next memory work should focus on provider-backed retrieval quality, not more local file accumulation.
+
+### 5.2 Context Compiler
+
+SCALE has context structure and budgets. It does not yet have a full context compiler.
+
+Current capability:
+
+- categorize context
+- budget context
+- lazy-load selected material
+- assemble role/task-specific packs
+
+Target capability:
+
+- rank relevance
+- slice semantically
+- compress adaptively
+- route retrieval by task intent
+- explain why each context item was included
+- measure token saved vs evidence lost
+
+This is the highest-leverage path for token reduction.
+
+### 5.3 Adaptive Workflow
+
+The current workflow is mostly rule-driven.
+
+The target workflow should adapt based on:
+
+- task risk
+- code ownership boundaries
+- prior failure rate
+- changed-file blast radius
+- missing evidence
+- tool reliability
+- agent capability confidence
+
+The system should not make every task heavy. It should apply stricter gates when risk rises and keep small documentation or config changes lightweight.
+
+### 5.4 Skill Routing Intelligence
+
+SCALE already models skills, MCP, CLI, browser, desktop automation, and evidence requirements.
+
+The missing layer is strategy:
+
+- when to call a skill
+- why that skill is preferred
+- what evidence it must produce
+- what to do when it fails
+- when to switch to MCP or CLI
+- when to avoid tool use entirely
+
+Skill routing should become a planned execution graph, not an ad hoc recommendation list.
+
+### 5.5 Evaluator Intelligence
+
+Current gates are strong for engineering completion, but weaker for reasoning quality.
+
+Needed evaluator layers:
+
+- critique loop for architecture and root cause
+- uncertainty scoring
+- adversarial review on high-risk changes
+- tradeoff comparison
+- failure hypothesis ranking
+- "evidence is insufficient" verdicts
+
+This is the path to reducing reasoning hallucination rather than only delivery hallucination.
+
+### 5.6 Self-Optimization Loop
+
+Evolution should mean more than summarizing lessons.
+
+The target loop:
+
+```text
+failure evidence
+  -> defect record
+  -> root-cause classification
+  -> lesson candidate
+  -> rule candidate
+  -> hook or gate proposal
+  -> shadow validation
+  -> regression check
+  -> promoted governance behavior
+```
+
+The promotion step must remain evidence-backed. Automatically generating rules without validation risks turning mistakes into permanent friction.
+
+## 6. Roadmap Direction
+
+### 6.1 Immediate Patch: 0.26.1
+
+Goal: publish the security patch before expanding strategic scope.
+
+Primary outcomes:
+
+- remove `verify-task` shell execution risk from the published package
+- document safe verification command semantics
+- pin and override flagged dependency versions where applicable
+- preserve production dependency audit health
+
+This is a trust-maintenance release. It should not be mixed with large roadmap changes.
+
+### 6.2 0.27.0: Cognitive Runtime Layer
+
+Theme: make context, memory, and skill use more intelligent and explainable.
+
+Core work:
+
+| Module | Outcome |
+| --- | --- |
+| Context Compiler | relevance-ranked, budgeted, explainable context packs |
+| Memory Provider Runtime | gbrain, agentmemory, code memory, and local memory as provider choices |
+| Skill Routing Engine | task-intent routing with evidence requirements and fallback decisions |
+| Governance ROI | quantify token cost, evidence quality, and gate friction |
+
+Implemented baseline in v0.27.0:
+
+```bash
+scale ai-os plan \
+  --task-id TASK-123 \
+  --task "Fix OAuth callback auth token handling and verify browser flow" \
+  --level L \
+  --files src/auth/oauth.ts,src/ui/callback.tsx \
+  --budget 8000 \
+  --json
+```
+
+The command returns one runtime plan containing:
+
+- `governance`: progressive mode, risk signals, required behaviors
+- `context`: Context Compiler ranking, included sections, omitted sections, token savings
+- `memory`: provider order, selected providers, fallback status, recalled items, memory context pack
+- `skillPlan`: detected intents plus executable skill/artifact/verification steps
+- `adaptiveWorkflow`: risk-adaptive gates and exit criteria for the task
+- `roi`: benefit and overhead modules for context, memory, skill routing, and governance
+
+Exit criteria:
+
+- each context item has an inclusion reason: baseline implemented by `ContextCompiler`
+- memory recall has provider, score, and evidence source: baseline implemented by Memory Provider Router
+- skill recommendations include why, when, and required proof: baseline implemented by skill execution plans
+- context pack generation reports token budget and omissions: baseline implemented by `context.pack.compiler`
+
+### 6.3 0.28.0: Adaptive Governance
+
+Theme: deepen adaptive governance beyond the v0.27.0 baseline.
+
+Core work:
+
+| Module | Outcome |
+| --- | --- |
+| Adaptive Workflow Router | production policy controls for dynamic gate profiles beyond the v0.27.0 planning output |
+| Evaluator Intelligence | critique and uncertainty gates for architecture/root-cause work |
+| Tool Strategy Planner | cost, retry, fallback, and evidence graph for tools |
+| Evolution Shadow Promotion | lessons become rules only after validation |
+
+Exit criteria:
+
+- small tasks can stay lightweight with evidence
+- risky tasks escalate automatically
+- reasoning-heavy tasks get critique/evaluator gates
+- evolution proposals can be traced to failure evidence and validation results
+
+### 6.4 0.29.0+: Agent Engineering OS
+
+Theme: integrate governance, memory, context, and tools into an operating layer.
+
+Target capabilities:
+
+- unified agent workspace policy
+- provider-neutral memory and code intelligence
+- cross-agent execution ledger
+- adaptive workflow templates
+- measurable token and quality reports
+- ecosystem-safe skill and MCP lifecycle governance
+
+## 7. Measurement Plan
+
+Strategic claims must be tied to measurement.
+
+| Claim | Required metric |
+| --- | --- |
+| Fewer fake completions | final-check failure rate before/after gates |
+| Fewer skipped steps | FSM blocked transition count and successful recovery rate |
+| Fewer blind retries | repeated-command detector hits and fix iteration count |
+| Lower token use | context pack token count vs baseline full-context load |
+| Better memory | recall precision, accepted memory rate, contradiction count |
+| Better skill use | recommended skill acceptance rate and evidence completion rate |
+| Better workflow quality | pass@1, average fix iterations, failure replay closure rate |
+| Safer dependencies | dependency audit block count and reviewed baseline count |
+
+Target ranges can be tracked internally, but public claims should use measured values from `WORKFLOW_EVAL`, runtime evidence, or release reports.
+
+## 8. Messaging Rules
+
+Use:
+
+- "Agent Governance Runtime"
+- "AI Engineering OS direction"
+- "runtime constraints instead of prompt-only discipline"
+- "evidence-backed workflow gates"
+- "provider-based memory and context orchestration"
+
+Avoid:
+
+- "fully autonomous engineer"
+- "guaranteed 90% AI coding rate"
+- "eliminates hallucination"
+- "zero human governance"
+- "universal memory"
+- "all tools are safe by default"
+
+The product message should be ambitious, but the engineering message must stay falsifiable.
+
+## 9. Non-Goals
+
+SCALE should not try to own every layer.
+
+Non-goals:
+
+- replacing all agent platforms
+- building a full IDE
+- becoming a generic automation shell
+- implementing every memory backend internally
+- copying external skills without attribution
+- turning every task into heavyweight enterprise ceremony
+
+The correct posture is:
+
+> Govern agent engineering work, integrate external capability providers, and require evidence at the boundaries.
+
+## 10. Documentation Placement
+
+Recommended documentation split:
+
+| Surface | Content |
+| --- | --- |
+| `README.md` / `README.en.md` | concise positioning, installation, core value, current capabilities |
+| `docs/AI_ENGINEERING_OS_POSITIONING.md` | strategic category, gaps, roadmap, messaging rules |
+| `docs/CONTEXT_BUDGET.md` | context budget and compiler mechanics |
+| `docs/MEMORY_BRAIN.md` / `docs/MEMORY_FABRIC.md` | memory provider and recall behavior |
+| `docs/SKILL_RADAR.md` / `docs/TOOL_ORCHESTRATION.md` | skill and tool routing behavior |
+| `docs/WORKFLOW_EVAL.md` | measurable evidence and improvement claims |
+
+README should not absorb this whole strategy. It should link here and keep the first screen user-focused.
+
+## 11. Strategic Summary
+
+SCALE's strongest current differentiator is not more prompts. It is a runtime governance model for AI engineering:
+
+```text
+Agent intent
+  -> governed workflow state
+  -> scoped context
+  -> role/tool policy
+  -> evidence-producing execution
+  -> verification gates
+  -> memory and evolution feedback
+```
+
+The next stage is to make this runtime more cognitive:
+
+- compile context, do not just load it
+- route memory, do not just store it
+- plan skill use, do not just recommend it
+- adapt workflow, do not just enforce one path
+- validate evolution, do not just summarize lessons
+
+If these are implemented with measurable evidence, SCALE can credibly move from "AI workflow engine" to "AI Engineering OS".

package/docs/BACKGROUND_HUNTER.md

@@ -1,62 +1,62 @@
-# Background Hunter
-
-Background Hunter is the readonly proactive scan layer for SCALE Engine V2.
-It turns existing governance signals into an actionable hunt queue without editing application code.
-
-## Boundary
-
-Default behavior is intentionally conservative:
-
-- scan only, no automatic code changes
-- no automatic LLM repair
-- no automatic commit or pull request
-- no release bypass
-- ignore decisions are explicit and written to `.scale/hunt/ignored-findings.json`
-
-The hunter reuses existing checks instead of creating a second rule system. The first implementation consumes:
-
-- `EngineeringStandards`
-- `ReviewAnalyzer` when status and diff input are provided by callers
-
-## Commands
-
-```bash
-scale hunt scan
-scale hunt scan --json
-scale hunt report
-scale hunt diagnose <finding-id>
-scale hunt ignore <finding-id> --reason "Accepted legacy debt tracked elsewhere"
-```
-
-`hunt scan` and `hunt report` do not modify source files. They classify findings as `open` or `ignored`.
-
-`hunt diagnose <finding-id>` creates a normal `DiagnosticLoop` from the finding. This keeps the debugging workflow evidence-first:
-
-- reproducible command
-- expected failure
-- changed files
-- verification commands
-- hypotheses and cleanup checklist
-
-`hunt ignore` records the finding id and stable fingerprint. The same finding will remain visible in the report as `ignored`, but it is removed from the open queue.
-
-## Finding Identity
-
-Every finding gets:
-
-- `id`: short deterministic SHA-256 id derived from the fingerprint
-- `fingerprint`: stable source/rule/path/line/message tuple
-- `source`: currently `engineering-standards` or `review-analyzer`
-- `diagnosticInput`: ready-to-use `DiagnosticLoopInput`
-
-This allows repeated scans to avoid noisy duplicates and lets teams explicitly accept or defer known debt.
-
-## Recommended Flow
-
-1. Run `scale hunt scan --json`.
-2. Triage open findings.
-3. For real issues, run `scale hunt diagnose <finding-id> --json`.
-4. Fix through the normal plan/TDD/verify workflow.
-5. For accepted legacy debt, run `scale hunt ignore <finding-id> --reason "..."`
-
-Do not promote Background Hunter to automatic repair until the project has enough evidence that its findings are stable and low-noise.
+# Background Hunter
+
+Background Hunter is the readonly proactive scan layer for SCALE Engine V2.
+It turns existing governance signals into an actionable hunt queue without editing application code.
+
+## Boundary
+
+Default behavior is intentionally conservative:
+
+- scan only, no automatic code changes
+- no automatic LLM repair
+- no automatic commit or pull request
+- no release bypass
+- ignore decisions are explicit and written to `.scale/hunt/ignored-findings.json`
+
+The hunter reuses existing checks instead of creating a second rule system. The first implementation consumes:
+
+- `EngineeringStandards`
+- `ReviewAnalyzer` when status and diff input are provided by callers
+
+## Commands
+
+```bash
+scale hunt scan
+scale hunt scan --json
+scale hunt report
+scale hunt diagnose <finding-id>
+scale hunt ignore <finding-id> --reason "Accepted legacy debt tracked elsewhere"
+```
+
+`hunt scan` and `hunt report` do not modify source files. They classify findings as `open` or `ignored`.
+
+`hunt diagnose <finding-id>` creates a normal `DiagnosticLoop` from the finding. This keeps the debugging workflow evidence-first:
+
+- reproducible command
+- expected failure
+- changed files
+- verification commands
+- hypotheses and cleanup checklist
+
+`hunt ignore` records the finding id and stable fingerprint. The same finding will remain visible in the report as `ignored`, but it is removed from the open queue.
+
+## Finding Identity
+
+Every finding gets:
+
+- `id`: short deterministic SHA-256 id derived from the fingerprint
+- `fingerprint`: stable source/rule/path/line/message tuple
+- `source`: currently `engineering-standards` or `review-analyzer`
+- `diagnosticInput`: ready-to-use `DiagnosticLoopInput`
+
+This allows repeated scans to avoid noisy duplicates and lets teams explicitly accept or defer known debt.
+
+## Recommended Flow
+
+1. Run `scale hunt scan --json`.
+2. Triage open findings.
+3. For real issues, run `scale hunt diagnose <finding-id> --json`.
+4. Fix through the normal plan/TDD/verify workflow.
+5. For accepted legacy debt, run `scale hunt ignore <finding-id> --reason "..."`
+
+Do not promote Background Hunter to automatic repair until the project has enough evidence that its findings are stable and low-noise.