@harness-engineering/intelligence 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Intense Visions, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,441 @@
+# @harness-engineering/intelligence
+
+Intelligence pipeline for spec enrichment, complexity modeling, and pre-execution simulation. Augments the hybrid orchestrator's routing with graph-backed analysis and tiered simulation.
+
+## Architecture
+
+```
+                         ┌─────────────────────┐
+                         │    Work Item Input   │
+                         │  Roadmap · JIRA ·    │
+                         │  GitHub · Linear ·   │
+                         │  Manual text         │
+                         └──────────┬──────────┘
+                                    ▼
+                         ┌─────────────────────┐
+                         │      Adapters        │
+                         │  toRawWorkItem()     │
+                         │  jiraToRawWorkItem() │
+                         │  githubToRawWorkItem │
+                         │  linearToRawWorkItem │
+                         │  manualToRawWorkItem │
+                         └──────────┬──────────┘
+                                    ▼
+┌────────────────────────────────────────────────────────────────┐
+│                    IntelligencePipeline                        │
+│                                                                │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │  SEL — Spec Enrichment Layer                             │  │
+│  │  LLM analysis + graph-validated system discovery         │  │
+│  │  → EnrichedSpec (intent, affected systems, unknowns)     │  │
+│  └──────────────────────┬───────────────────────────────────┘  │
+│                         ▼                                      │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │  CML — Complexity Modeling Layer                         │  │
+│  │  Structural (graph blast radius) + Semantic (ambiguity)  │  │
+│  │  + Historical (past outcome failure rate)                │  │
+│  │  → ComplexityScore → ConcernSignal[]                     │  │
+│  └──────────────────────┬───────────────────────────────────┘  │
+│                         ▼                                      │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │  PESL — Pre-Execution Simulation Layer                   │  │
+│  │  Graph-only checks (quick-fix) or full LLM simulation    │  │
+│  │  (guided-change) with abort on low confidence            │  │
+│  │  → SimulationResult (confidence, abort flag)             │  │
+│  └──────────────────────────────────────────────────────────┘  │
+│                                                                │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │  Outcome Recording                                       │  │
+│  │  ExecutionOutcomeConnector → graph nodes + edges         │  │
+│  │  Feeds back into CML historical dimension                │  │
+│  └──────────────────────────────────────────────────────────┘  │
+└────────────────────────────────────────────────────────────────┘
+```
+
+## Quick Start
+
+```ts
+import {
+  IntelligencePipeline,
+  AnthropicAnalysisProvider,
+  toRawWorkItem,
+} from '@harness-engineering/intelligence';
+import { GraphStore } from '@harness-engineering/graph';
+
+// 1. Create the pipeline
+const provider = new AnthropicAnalysisProvider({
+  apiKey: process.env.ANTHROPIC_API_KEY!,
+});
+const store = new GraphStore();
+await store.load('.harness/graph');
+
+const pipeline = new IntelligencePipeline(provider, store);
+
+// 2. Preprocess an issue (SEL → CML → signals)
+const result = await pipeline.preprocessIssue(issue, scopeTier, escalationConfig);
+// result.spec    — EnrichedSpec with intent, affected systems, unknowns
+// result.score   — ComplexityScore with overall 0-1, risk level, reasoning
+// result.signals — ConcernSignal[] for routeIssue()
+
+// 3. Simulate before dispatch (PESL)
+if (result.spec && result.score) {
+  const simulation = await pipeline.simulate(result.spec, result.score, scopeTier);
+  // simulation.abort — true if confidence too low
+  // simulation.predictedFailures — what might go wrong
+  // simulation.testGaps — missing test coverage
+}
+
+// 4. Record execution outcomes (feedback loop)
+await pipeline.recordOutcome({
+  issueId: 'CORE-42',
+  result: 'success',
+  retryCount: 0,
+  failureReasons: [],
+  durationMs: 45000,
+  affectedSystemNodeIds: ['node-abc'],
+});
+```
+
+## Tier-Based Behavior
+
+The pipeline adapts its behavior based on the orchestrator's scope tier:
+
+| Scope Tier                            | SEL  | CML  | PESL                | Routing                                 |
+| ------------------------------------- | ---- | ---- | ------------------- | --------------------------------------- |
+| `autoExecute` (quick-fix, diagnostic) | Skip | Skip | Graph-only          | Always dispatch locally                 |
+| `signalGated` (guided-change)         | Run  | Run  | Full LLM simulation | Dispatch if no concern signals          |
+| `alwaysHuman` (full-exploration)      | Run  | Skip | Skip                | Always escalate (enrichment as context) |
+
+## Layers
+
+### SEL — Spec Enrichment Layer
+
+Converts raw work items into structured `EnrichedSpec` objects via LLM analysis and graph validation.
+
+```ts
+import { enrich, GraphValidator } from '@harness-engineering/intelligence';
+
+const validator = new GraphValidator(store);
+const spec = await enrich(rawWorkItem, provider, validator);
+// spec.intent — what the task is trying to accomplish
+// spec.affectedSystems — graph-validated system references
+// spec.unknowns — explicitly identified knowledge gaps
+// spec.ambiguities — areas needing clarification
+```
+
+### CML — Complexity Modeling Layer
+
+Scores task complexity across three dimensions:
+
+- **Structural** — Graph blast radius via `CascadeSimulator`, normalized to 0-1
+- **Semantic** — Unknowns, ambiguities, and risk signals from SEL enrichment
+- **Historical** — Smoothed failure rate from past execution outcomes on the same affected systems
+
+```ts
+import { scoreCML, scoreToConcernSignals } from '@harness-engineering/intelligence';
+
+const score = scoreCML(enrichedSpec, store);
+// score.overall — 0-1 weighted composite
+// score.riskLevel — 'low' | 'medium' | 'high' | 'critical'
+// score.recommendedRoute — 'local' | 'human' | 'simulation-required'
+
+const signals = scoreToConcernSignals(score);
+// signals fed into existing routeIssue() for signalGated tiers
+```
+
+### PESL — Pre-Execution Simulation Layer
+
+Simulates execution before code is written. Tiered by cost:
+
+- **Graph-only** (quick-fix/diagnostic) — `CascadeSimulator` blast radius + test gap detection. Fast (<2s), no LLM cost.
+- **Full simulation** (guided-change) — LLM-assisted plan expansion, failure injection, test projection. Merged with graph baseline.
+
+```ts
+import { PeslSimulator } from '@harness-engineering/intelligence';
+
+const simulator = new PeslSimulator(provider, store);
+const result = await simulator.simulate(spec, score, scopeTier);
+
+if (result.abort) {
+  // Confidence too low — escalate to human instead of dispatching
+  console.log('Predicted failures:', result.predictedFailures);
+  console.log('Test gaps:', result.testGaps);
+}
+```
+
+### Adapters
+
+Pure mapping functions for converting external data into `RawWorkItem`:
+
+```ts
+import {
+  toRawWorkItem, // Roadmap Issue → RawWorkItem
+  jiraToRawWorkItem, // JIRA issue → RawWorkItem
+  githubToRawWorkItem, // GitHub issue/PR → RawWorkItem
+  linearToRawWorkItem, // Linear issue → RawWorkItem
+  manualToRawWorkItem, // Free text → RawWorkItem
+} from '@harness-engineering/intelligence';
+```
+
+Each adapter accepts a pre-fetched data object and produces a `RawWorkItem`. No API clients are included — adapters are pure functions.
+
+### Effectiveness — Agent Performance Introspection
+
+Analyzes persona-attributed `execution_outcome` nodes in the graph to score per-persona accuracy, detect blind spots, and recommend which persona to route new issues to.
+
+```ts
+import {
+  computePersonaEffectiveness,
+  detectBlindSpots,
+  recommendPersona,
+} from '@harness-engineering/intelligence';
+
+// Per-(persona, system) success rates with Laplace smoothing
+const scores = computePersonaEffectiveness(store, { persona: 'backend-dev' });
+// scores[0].successRate — smoothed success rate in [0, 1]
+// scores[0].sampleSize  — total observations
+
+// Blind spots: (persona, system) pairs with high failure rates
+const spots = detectBlindSpots(store, { minFailures: 2, minFailureRate: 0.5 });
+// spots[0].failureRate — raw failure rate
+// spots[0].failures    — absolute failure count
+
+// Recommend personas for an issue given its affected systems
+const recs = recommendPersona(store, {
+  systemNodeIds: ['module-auth', 'module-db'],
+  candidatePersonas: ['backend-dev', 'fullstack'],
+  minSamples: 3,
+});
+// recs[0].persona — best candidate
+// recs[0].score   — mean smoothed success rate across requested systems
+```
+
+### Specialization — Persistent Agent Expertise Tracking
+
+Extends effectiveness with temporal decay, task-type categorization, consistency scoring, expertise levels, and persistent profile storage. Recent outcomes are weighted more heavily than old ones via exponential decay.
+
+```ts
+import {
+  computeSpecialization,
+  computeExpertiseLevel,
+  buildSpecializationProfile,
+  weightedRecommendPersona,
+  decayWeight,
+  temporalSuccessRate,
+  loadProfiles,
+  saveProfiles,
+  refreshProfiles,
+} from '@harness-engineering/intelligence';
+
+// Compute specialization entries for (persona, system, taskType) tuples
+const entries = computeSpecialization(store, {
+  persona: 'backend-dev',
+  taskType: 'bug-fix',
+  temporal: { halfLifeDays: 30 },
+});
+// entries[0].score.composite         — weighted combination of temporal, consistency, volume
+// entries[0].expertiseLevel          — 'novice' | 'competent' | 'proficient' | 'expert'
+
+// Classify expertise from raw numbers
+const level = computeExpertiseLevel(25, 0.8); // → 'proficient'
+
+// Build a full profile for a persona (strengths, weaknesses, overall level)
+const profile = buildSpecializationProfile(store, 'backend-dev', {
+  temporal: { halfLifeDays: 30 },
+});
+// profile.strengths   — top 3 entries by composite score
+// profile.weaknesses  — entries with temporal success rate < 0.5
+// profile.overallLevel — median expertise across all entries
+
+// Weighted recommendation incorporating specialization multipliers
+const weighted = weightedRecommendPersona(store, {
+  systemNodeIds: ['module-auth'],
+  taskType: 'bug-fix',
+});
+// weighted[0].weightedScore — baseScore * specializationMultiplier
+
+// Temporal decay helpers
+const weight = decayWeight(15, 30); // weight at 15 days with 30-day half-life
+const rate = temporalSuccessRate([{ result: 'success', timestamp: '2026-04-01T00:00:00Z' }], {
+  halfLifeDays: 30,
+});
+
+// Persist profiles to .harness/specialization-profiles.json
+const profiles = loadProfiles('/path/to/project');
+saveProfiles('/path/to/project', profiles);
+
+// Recompute and persist profiles for all personas with outcomes
+const refreshed = refreshProfiles('/path/to/project', store);
+```
+
+### Outcome Recording
+
+Execution results feed back into the graph for future CML scoring:
+
+```ts
+import { ExecutionOutcomeConnector } from '@harness-engineering/intelligence';
+
+const connector = new ExecutionOutcomeConnector(store);
+await connector.ingest({
+  issueId: 'CORE-42',
+  result: 'failure',
+  retryCount: 2,
+  failureReasons: ['Migration script failed'],
+  durationMs: 120000,
+  affectedSystemNodeIds: ['module-auth'],
+});
+// Creates execution_outcome node with outcome_of edges to affected systems
+// Future CML scoring queries these outcomes for historical failure rates
+```
+
+## Orchestrator Integration
+
+The intelligence pipeline integrates into the orchestrator's tick cycle at two points:
+
+1. **Pre-routing** (in `asyncTick`) — `preprocessIssue()` runs SEL + CML, producing concern signals that feed into `routeIssue()`. For `alwaysHuman` tiers, the enriched spec is attached to the `EscalateEffect` as context for the human reviewer.
+
+2. **Post-routing** (in `asyncTick`) — `simulate()` runs PESL for locally-routed issues. If simulation recommends abort (`confidence < 0.3`), the dispatch is converted to an `EscalateEffect` instead.
+
+3. **Post-execution** (in `emitWorkerExit`) — `recordOutcome()` ingests the execution result into the graph, feeding the CML historical dimension for future scoring.
+
+### Dashboard
+
+The dashboard's **Attention** page displays escalated interactions. When the intelligence pipeline is enabled, escalations include enriched context:
+
+- **Enriched spec summary** — Intent, affected systems, unknowns
+- **Concern signals** — What triggered the escalation (high complexity, large blast radius, high ambiguity)
+- **PESL abort reasons** — Predicted failures and test gaps (when simulation recommends abort)
+
+This context helps human reviewers make faster, more informed decisions about escalated work items.
+
+## Configuration
+
+The pipeline uses the same LLM connection as the orchestrator's agent backend — Anthropic API, OpenAI API, or a local LLM (Ollama, LM Studio, etc.):
+
+```yaml
+intelligence:
+  enabled: true
+```
+
+No separate API key needed. The pipeline derives its provider from the existing agent config:
+
+| Agent Backend                     | Intelligence Uses                 |
+| --------------------------------- | --------------------------------- |
+| `anthropic` / `claude`            | Anthropic Messages API (same key) |
+| `openai`                          | OpenAI Chat API (same key)        |
+| `localBackend: openai-compatible` | Local endpoint (same URL)         |
+
+Override models per-layer:
+
+```yaml
+intelligence:
+  enabled: true
+  models:
+    sel: llama3.2 # fast model for enrichment
+    pesl: deepseek-r1 # reasoning model for simulation
+```
+
+For thinking models (Qwen3, DeepSeek-R1), disable reasoning for structured output:
+
+```yaml
+intelligence:
+  enabled: true
+  promptSuffix: '/no_think' # Qwen3 — disable thinking for structured output
+  jsonMode: false # Qwen3 hangs with Ollama's JSON grammar constraint
+  requestTimeoutMs: 90000 # fail fast instead of waiting 10 min
+  failureCacheTtlMs: 300000 # don't retry failed analyses for 5 min
+```
+
+When `enabled: false` (default), the pipeline is completely skipped. See the [Intelligence Pipeline Guide](../../docs/guides/intelligence-pipeline.md) for full configuration reference.
+
+## Dependencies
+
+```
+@harness-engineering/types  → shared type definitions
+@harness-engineering/graph  → GraphStore, CascadeSimulator, node/edge types
+@anthropic-ai/sdk           → LLM calls via AnalysisProvider
+openai                      → OpenAI-compatible analysis provider
+zod                         → response schema validation
+```
+
+The intelligence package has **no dependency** on `@harness-engineering/orchestrator`. The dependency flows one way: `orchestrator → intelligence → graph → types`.
+
+## API Reference
+
+### Pipeline
+
+| Export                 | Description                                  |
+| ---------------------- | -------------------------------------------- |
+| `IntelligencePipeline` | Main pipeline class composing SEL, CML, PESL |
+| `PreprocessResult`     | Return type of `preprocessIssue()`           |
+
+### Adapters
+
+| Export                | Description                     |
+| --------------------- | ------------------------------- |
+| `toRawWorkItem`       | Roadmap `Issue` → `RawWorkItem` |
+| `jiraToRawWorkItem`   | JIRA issue → `RawWorkItem`      |
+| `githubToRawWorkItem` | GitHub issue/PR → `RawWorkItem` |
+| `linearToRawWorkItem` | Linear issue → `RawWorkItem`    |
+| `manualToRawWorkItem` | Free text → `RawWorkItem`       |
+
+### Types
+
+| Export             | Description                                          |
+| ------------------ | ---------------------------------------------------- |
+| `RawWorkItem`      | Generic work item input                              |
+| `EnrichedSpec`     | SEL output with intent, affected systems, unknowns   |
+| `ComplexityScore`  | CML output with overall score, risk level, reasoning |
+| `SimulationResult` | PESL output with confidence, abort flag, predictions |
+| `ExecutionOutcome` | Outcome data for graph ingestion                     |
+
+### Effectiveness
+
+| Export                        | Description                                                    |
+| ----------------------------- | -------------------------------------------------------------- |
+| `computePersonaEffectiveness` | Per-(persona, system) Laplace-smoothed success rates           |
+| `detectBlindSpots`            | Find (persona, system) pairs with high failure rates           |
+| `recommendPersona`            | Recommend personas for an issue given affected system node IDs |
+| `PersonaEffectivenessScore`   | Type: smoothed success rate for a (persona, system) pair       |
+| `BlindSpot`                   | Type: (persona, system) pair with consistent failures          |
+| `PersonaRecommendation`       | Type: persona recommendation with score and coverage stats     |
+
+### Specialization
+
+| Export                       | Description                                                                     |
+| ---------------------------- | ------------------------------------------------------------------------------- |
+| `computeSpecialization`      | Compute specialization entries for (persona, system, taskType) tuples           |
+| `computeExpertiseLevel`      | Classify expertise level from sample size and success rate                      |
+| `buildSpecializationProfile` | Build a full profile for a persona (strengths, weaknesses, overall level)       |
+| `weightedRecommendPersona`   | Persona recommendation with specialization multipliers                          |
+| `decayWeight`                | Exponential decay weight for an outcome at a given age                          |
+| `temporalSuccessRate`        | Temporally-weighted success rate from timestamped outcomes                      |
+| `loadProfiles`               | Load persisted specialization profiles from disk                                |
+| `saveProfiles`               | Save specialization profiles to `.harness/specialization-profiles.json`         |
+| `refreshProfiles`            | Recompute and persist profiles for all personas with outcomes                   |
+| `SpecializationScore`        | Type: composite score (temporal, consistency, volume, composite)                |
+| `SpecializationEntry`        | Type: single entry for a (persona, system, taskType) bucket                     |
+| `SpecializationProfile`      | Type: full persona profile with strengths, weaknesses, overall level            |
+| `WeightedRecommendation`     | Type: recommendation with base score, specialization multiplier, weighted score |
+| `ExpertiseLevel`             | Type: `'novice' \| 'competent' \| 'proficient' \| 'expert'`                     |
+| `TaskType`                   | Type: task type categorization                                                  |
+| `TemporalConfig`             | Type: temporal decay configuration (half-life, reference time)                  |
+| `ProfileStore`               | Type: persisted store of specialization profiles                                |
+
+### Analysis
+
+| Export                             | Description                                |
+| ---------------------------------- | ------------------------------------------ |
+| `AnthropicAnalysisProvider`        | Anthropic-backed `AnalysisProvider`        |
+| `OpenAICompatibleAnalysisProvider` | OpenAI/local LLM-backed `AnalysisProvider` |
+| `enrich`                           | SEL enrichment function                    |
+| `GraphValidator`                   | Graph-based affected system resolver       |
+| `scoreCML`                         | CML scoring function                       |
+| `scoreToConcernSignals`            | Score → `ConcernSignal[]` conversion       |
+| `PeslSimulator`                    | Tiered simulation facade                   |
+| `ExecutionOutcomeConnector`        | Outcome graph ingestion                    |
+| `computeHistoricalComplexity`      | CML historical dimension                   |
+
+## License
+
+MIT