@smartergpt/lexrunner 1.0.0 → 1.3.0

package/docs/adr/ADR-008-lex-packaging.md

@@ -1,243 +0,0 @@
-# ADR-008: Lex Packaging Strategy
-
-**Status:** Accepted
-**Date:** 2026-01-01
-**Authors:** LexRunner Team
-
----
-
-## Context
-
-LexRunner depends on Lex (MIT-licensed core) for memory, policy, atlas, and error handling functionality. The question is how to package and distribute this dependency to ensure:
-
-1. **Stable versioning** — Predictable updates and rollback capability
-2. **Security** — Supply chain integrity and provenance
-3. **Developer experience** — Simple installation and updates
-4. **Licensing clarity** — MIT (Lex) vs. Proprietary (LexRunner) boundaries
-
-Options considered:
-
-1. **npm registry publish** — Publish Lex to public npm registry
-2. **Git submodule** — Track Lex as a Git submodule in LexRunner repo
-3. **Vendoring** — Copy Lex source into `vendor/lex/` directory
-
----
-
-## Decision
-
-We adopt **npm registry publish** as the packaging strategy for Lex.
-
-### Implementation
-
-Lex is published to the public npm registry as:
-
-- **Package name:** `@smartergpt/lex`
-- **Current version:** `2.1.1` (used by LexRunner)
-- **License:** MIT
-- **Registry:** https://registry.npmjs.org
-
-LexRunner consumes Lex via standard npm dependency mechanism:
-
-```json
-{
-  "dependencies": {
-    "@smartergpt/lex": "^2.1.1"
-  }
-}
-```
-
-### Export Paths
-
-Lex provides modular exports for selective imports:
-
-- `@smartergpt/lex` — Core exports
-- `@smartergpt/lex/types` — Frame types and schemas
-- `@smartergpt/lex/errors` — AXError types and utilities
-- `@smartergpt/lex/policy` — Policy validation
-- `@smartergpt/lex/atlas` — Code atlas and dependency analysis
-- `@smartergpt/lex/aliases` — Module ID resolution
-- `@smartergpt/lex/logger` — Structured logging
-
-### Versioning Policy
-
-- **Semantic Versioning:** Lex follows semver strictly
-  - **MAJOR** — Breaking API changes
-  - **MINOR** — New features, backward compatible
-  - **PATCH** — Bug fixes, backward compatible
-- **LexRunner Pinning:** Use caret range (`^2.1.1`) to allow MINOR/PATCH updates
-- **Testing:** CI validates LexRunner against pinned Lex version before merge
-
----
-
-## Rationale
-
-### Why npm Publish?
-
-1. **Industry Standard** — npm is the standard distribution mechanism for TypeScript/Node.js packages
-2. **Dependency Management** — npm/package-lock.json ensures reproducible builds
-3. **Security Scanning** — npm audit and GitHub Dependabot provide vulnerability detection
-4. **Version Control** — Semantic versioning and package.json constraints prevent breaking changes
-5. **Distribution** — Public registry enables community adoption of Lex independently
-6. **Build Artifacts** — Published package includes only built `dist/` files, not source
-
-### Why Not Git Submodule?
-
-- **Build Complexity** — Requires building Lex from source during LexRunner build
-- **Version Pinning** — Git commit hashes are less discoverable than semver
-- **Dependency Resolution** — npm handles transitive dependencies; Git submodules don't
-- **Breaking Changes** — Harder to prevent breaking changes from upstream
-
-### Why Not Vendoring?
-
-- **Duplication** — Copies Lex source into LexRunner repo
-- **License Violations** — Risk of modifying Lex code without proper attribution
-- **Sync Burden** — Manual process to pull upstream changes
-- **Security** — No automated vulnerability scanning for vendored code
-- **Repo Size** — Increases LexRunner repository size unnecessarily
-
----
-
-## Consequences
-
-### Positive
-
-- **Simple Updates** — `npm update @smartergpt/lex` to get latest compatible version
-- **Security Scanning** — GitHub Dependabot alerts on vulnerabilities in Lex
-- **Reproducible Builds** — package-lock.json ensures exact version consistency
-- **Clear Licensing** — npm package.json declares MIT license for Lex
-- **Public Distribution** — Lex can be adopted independently by other projects
-- **Build Artifacts Only** — Published package contains pre-built dist/ files
-
-### Negative
-
-- **Registry Dependency** — Requires npm registry availability for installation
-- **Supply Chain Risk** — Potential for npm package hijacking (mitigated by package-lock.json)
-- **Version Lag** — LexRunner may lag behind latest Lex releases (intentional for stability)
-
----
-
-## Security Implications
-
-### Supply Chain
-
-- **Package Integrity** — npm uses SHA-512 integrity hashes in package-lock.json
-- **Provenance** — GitHub Actions publishes Lex with provenance attestation
-- **Audit Trail** — npm registry provides download and publish audit logs
-- **Dependabot** — Automated alerts for known vulnerabilities
-
-### Versioning & Updates
-
-1. **Controlled Updates** — Use caret range (`^2.1.1`) to allow PATCH/MINOR, block MAJOR
-2. **Testing Gate** — CI must pass before merging Lex updates
-3. **Review Process** — PRs with Lex updates require maintainer approval
-4. **Rollback** — Revert package.json change if update breaks LexRunner
-
-### Rollback Strategy
-
-If a Lex update breaks LexRunner:
-
-1. **Immediate Rollback** — Revert package.json to previous version
-2. **Lock Specific Version** — Change `^2.1.1` to `2.1.1` to prevent auto-updates
-3. **Root Cause Analysis** — Identify breaking change in Lex
-4. **Upstream Issue** — File issue in Lex repository
-5. **Fix Forward** — Once Lex releases patch, update LexRunner
-
----
-
-## Update Workflow
-
-### Checking for Updates
-
-Use the provided script to check for Lex updates:
-
-```bash
-./scripts/package-lex.sh check
-```
-
-### Updating Lex
-
-Manual update process:
-
-```bash
-# 1. Check for updates
-npm outdated @smartergpt/lex
-
-# 2. Update to latest compatible version
-npm update @smartergpt/lex
-
-# 3. Or update to specific version
-npm install @smartergpt/lex@2.1.1
-
-# 4. Run tests
-npm test
-
-# 5. Commit if tests pass
-git add package.json package-lock.json
-git commit -m "chore: update @smartergpt/lex to 2.1.1"
-```
-
-Automated update (via Dependabot):
-
-1. **Dependabot PR** — GitHub creates PR with Lex update
-2. **CI Validation** — Automated tests run
-3. **Review** — Maintainer reviews changes
-4. **Merge** — Merge if tests pass and no breaking changes
-
----
-
-## Licensing Documentation
-
-### Lex (MIT)
-
-- **Package:** `@smartergpt/lex`
-- **License:** MIT (open source)
-- **Attribution:** Required (see NOTICE.md)
-- **Repository:** https://github.com/Guffawaffle/lex
-
-### LexRunner (Proprietary)
-
-- **Package:** `lexrunner`
-- **License:** UNLICENSED (proprietary)
-- **Attribution:** Uses Lex (MIT) as dependency
-- **Repository:** https://github.com/Guffawaffle/lexrunner
-
-### License Compliance
-
-LexRunner's license compliance is validated via:
-
-- **CI Gate:** `scripts/check-license-compliance.mjs`
-- **Verification:** Ensures proper Lex attribution in NOTICE.md
-- **Enforcement:** CI fails if compliance checks fail
-
----
-
-## Alternatives Considered
-
-### 1. Private npm Registry
-
-**Rejected** — Lex is MIT-licensed and intended for public distribution. Private registry adds unnecessary complexity and limits community adoption.
-
-### 2. Monorepo with Lex
-
-**Rejected** — Separating Lex and LexRunner as independent repositories preserves licensing boundaries and allows independent evolution.
-
-### 3. Inline Lex Code
-
-**Rejected** — Violates DRY principle and creates license compliance issues. LexRunner must consume Lex as a dependency, not duplicate source code.
-
----
-
-## References
-
-- [Lex npm Package](https://www.npmjs.com/package/@smartergpt/lex)
-- [Lex Repository](https://github.com/Guffawaffle/lex)
-- [Lex Public API](../LEX_PUBLIC_API.md)
-- [License Compliance Guide](../LICENSING.md)
-- [ADR-000: Product Naming and Branding](./ADR-000-product-naming-and-branding.md)
-
----
-
-## Sign-Off
-
-Accepted by: LexRunner Team
-Date: 2026-01-01

package/docs/adr/ADR-009-ax-test-output-adapters.md

@@ -1,440 +0,0 @@
-# ADR-009: AX Test Output Adapters
-
-**Status:** Proposed
-**Date:** 2026-01-04
-**Authors:** lexrunner team
-
----
-
-## Context
-
-Test runners emit output in wildly different formats:
-
-| Runner           | Native Output              |
-| ---------------- | -------------------------- |
-| Vitest           | JSON, TAP, default console |
-| Jest             | JSON, JUnit XML            |
-| PHPUnit          | XML, JSON                  |
-| pytest           | JSON, JUnit XML            |
-| go test          | JSON lines                 |
-| Node test runner | TAP, spec                  |
-
-Agents consuming test results need **consistent, actionable output** regardless of the underlying test platform. Current CI gates capture pass/fail status but lack:
-
-1. **Structured failure details** with file:line locations
-2. **`nextActions[]`** suggesting recovery steps (AX pattern)
-3. **Diff context** showing expected vs actual
-4. **Summary-first output** for token-efficient agent consumption
-
----
-
-## Decision
-
-LexRunner will provide a **Test Gate Adapter system** that normalizes any test runner output into an AX-compliant schema.
-
-### Architecture
-
-```
-┌─────────────────────────────────────────────────────────────┐
-│  lexrunner gate test                                        │
-│                                                             │
-│   Input Adapters          Core            Output Formats    │
-│  ┌──────────────┐    ┌──────────┐    ┌──────────────┐      │
-│  │ vitest-json  │───▶│          │───▶│ ax-summary   │      │
-│  │ jest-json    │───▶│ Normalize│───▶│ (JSON)       │      │
-│  │ junit-xml    │───▶│    to    │───▶│              │      │
-│  │ tap          │───▶│ AXTest   │    └──────────────┘      │
-│  │ pytest-json  │───▶│ Result   │    ┌──────────────┐      │
-│  │ go-test-json │───▶│          │───▶│ pr-comment   │      │
-│  │ phpunit-xml  │───▶│          │───▶│ (Markdown)   │      │
-│  └──────────────┘    └──────────┘    └──────────────┘      │
-│                                                             │
-└─────────────────────────────────────────────────────────────┘
-```
-
-### AXTestResult Schema (v1.0.0)
-
-> **Design principle:** This schema is a **verifiable contract**, not just a convenient shape.
-> Normalization is never a one-way door — `raw` preserves original data for auditability.
-
-```typescript
-interface AXTestResult {
-  schemaVersion: "1.0.0";
-  timestamp: string; // ISO 8601
-
-  summary: {
-    total: number;
-    passed: number;
-    failed: number;
-    skipped: number;
-    durationMs: number;
-  };
-
-  // Only populated for failures - passing tests omitted for token efficiency
-  failures: AXTestFailure[];
-
-  // Optional coverage summary (percentages, 0-100)
-  coverage?: {
-    linesPct: number;
-    branchesPct: number;
-    functionsPct: number;
-    statementsPct?: number;
-  };
-
-  // Run metadata for reproducibility
-  run?: {
-    cwd?: string; // Working directory
-    command?: string; // Exact command executed
-    ci?: boolean; // Running in CI environment
-    os?: string; // e.g., "linux", "darwin", "win32"
-    nodeVersion?: string; // e.g., "20.10.0"
-  };
-
-  // Adapter metadata
-  adapter: {
-    name: string; // e.g., "vitest-json"
-    version: string;
-    source: string; // Original output file/stream
-  };
-
-  // Original unparsed output — normalization is never a one-way door
-  raw?: string;
-}
-
-interface AXTestFailure {
-  // Identity & grouping (enables flaky detection, trend analysis)
-  failureId: string; // Deterministic hash of (file + name + error.type + canonicalized message)
-  signature?: string; // Normalized signature with volatile values scrubbed (timestamps, ports)
-
-  // Location
-  file: string; // Relative path
-  line: number; // 1-indexed
-  column?: number;
-
-  name: string; // Test name / describe path
-  suite?: string; // Parent suite/describe
-
-  error: {
-    message: string;
-    type?: string; // e.g., "AssertionError", "TypeError"
-    stack?: string; // Human-readable stack trace
-  };
-
-  // Structured stack for agent navigation
-  stackFrames?: Array<{
-    file: string;
-    line: number;
-    column?: number;
-    function?: string;
-  }>;
-
-  // Assertion details
-  assertion?: {
-    operator?: string; // e.g., "toBe", "toEqual", "strictEqual"
-    expectedType?: string; // e.g., "string", "LexSonaError"
-    actualType?: string; // e.g., "undefined", "TypeError"
-  };
-
-  diff?: {
-    expected: string;
-    actual: string;
-    unified?: string; // Unified diff format when applicable
-    contextLines?: number; // Lines of context included
-  };
-
-  // AX-compliant recovery guidance (string or structured)
-  nextActions: Array<string | AXNextAction>;
-  nextActionsMeta?: {
-    confidence: "high" | "medium" | "low";
-  };
-
-  // Duration for identifying slow tests
-  durationMs?: number;
-
-  // Original failure data from adapter — preserves runner-specific details
-  raw?: unknown;
-}
-
-// Structured next action for agent execution
-interface AXNextAction {
-  kind: "rerun" | "inspect" | "fix" | "doc";
-  cmd?: string; // Executable command
-  note: string; // Human-readable description
-}
-```
-
-### CLI Interface
-
-```bash
-# Explicit adapter (REQUIRED in CI mode for determinism)
-lexrunner gate test --adapter=vitest --input=test-results.json
-
-# Strict mode: require explicit adapter, fail on ambiguity
-lexrunner gate test --strict --input=test-results.json
-# Error: --adapter required in strict mode
-
-# Auto-detect from file extension/content (local DX only, warns in CI)
-lexrunner gate test --input=test-results.json
-# Warning: Auto-detected adapter 'vitest-json'. Use --adapter for CI.
-
-# Pipe mode (for integration)
-vitest run --reporter=json | lexrunner gate test --adapter=vitest
-
-# Output formats
-lexrunner gate test --input=results.json --output=ax-summary.json
-lexrunner gate test --input=results.json --format=markdown > pr-comment.md
-
-# Include raw output for auditability
-lexrunner gate test --adapter=vitest --input=results.json --include-raw
-```
-
-> **CI Recommendation:** Always use `--adapter` explicitly or `--strict` mode.
-> Auto-detect is convenient for local development but can silently choose
-> the wrong parser if output formats evolve between runner versions.
-
-### Adapter Interface
-
-```typescript
-interface TestAdapter {
-  name: string;
-  version: string;
-
-  // File extensions this adapter handles
-  extensions: string[];
-
-  // Content detection (for auto-detect)
-  detect(content: string): boolean;
-
-  // Parse raw output into normalized form
-  parse(input: string | Buffer): AXTestResult;
-}
-```
-
-### Built-in Adapters (v1.0.0)
-
-| Adapter        | Priority | Rationale                                    |
-| -------------- | -------- | -------------------------------------------- |
-| `junit-xml`    | P0       | Universal CI lingua franca across ecosystems |
-| `vitest-json`  | P0       | LexSona, LexRunner use Vitest                |
-| `jest-json`    | P0       | Huge footprint, close to Vitest patterns     |
-| `node-tap`     | P1       | Lex uses Node test runner with TAP           |
-| `pytest-json`  | P1       | Very common in mixed stacks                  |
-| `go-test-json` | P2       | Extremely structured, popular in infra repos |
-| `phpunit-xml`  | P2       | Enterprise/legacy usage                      |
-
-> **Note:** TAP is worth having but is one of the messier formats.
-> Treat as "best effort + raw preservation" for edge cases.
-
-### `nextActions[]` Generation
-
-Adapters generate contextual suggestions based on failure patterns.
-Actions can be strings (for display) or structured objects (for execution).
-
-#### Rerun Templates (Per-Runner)
-
-Each adapter provides runner-specific rerun commands:
-
-| Runner  | Rerun Template                             |
-| ------- | ------------------------------------------ |
-| Vitest  | `vitest run -t "<name>"`                   |
-| Jest    | `jest -t "<name>" --runInBand`             |
-| pytest  | `pytest -k "<expr>" -q`                    |
-| go test | `go test ./... -run "<regex>"`             |
-| PHPUnit | `phpunit --filter "<name>"`                |
-| Node    | `node --test --test-name-pattern="<name>"` |
-
-#### Common Failure Patterns
-
-```typescript
-const FAILURE_PATTERNS: Record<string, PatternHandler> = {
-  // Assertion failures
-  "snapshot-mismatch": {
-    detect: (msg) => msg.includes("snapshot") || msg.includes("toMatchSnapshot"),
-    actions: [
-      { kind: "fix", cmd: "npm test -- -u", note: "Update snapshots" },
-      { kind: "inspect", note: "Verify snapshot changes are intentional" },
-    ],
-    confidence: "high",
-  },
-
-  // Module resolution
-  "module-not-found": {
-    detect: (msg) => msg.includes("Cannot find module") || msg.includes("ERR_MODULE_NOT_FOUND"),
-    actions: [
-      { kind: "inspect", note: "Verify import path is correct" },
-      { kind: "inspect", note: "Check tsconfig paths configuration" },
-      { kind: "fix", cmd: "npm install", note: "Reinstall dependencies" },
-      { kind: "inspect", note: "Check for workspace hoisting issues" },
-    ],
-    confidence: "high",
-  },
-
-  // Timeouts
-  timeout: {
-    detect: (msg) => msg.includes("timeout") || msg.includes("exceeded"),
-    actions: [
-      { kind: "inspect", note: "Identify slow async operation" },
-      { kind: "fix", note: "Consider using fake timers" },
-      { kind: "fix", note: "Increase test timeout if legitimately slow" },
-    ],
-    confidence: "medium",
-  },
-
-  // Unhandled promises
-  "unhandled-promise": {
-    detect: (msg) => msg.includes("unhandled") && msg.includes("promise"),
-    actions: [
-      { kind: "fix", note: "Await the promise or return it from test" },
-      { kind: "fix", note: "Add error handler to promise chain" },
-    ],
-    confidence: "high",
-  },
-
-  // Port/network collisions
-  "port-in-use": {
-    detect: (msg) => msg.includes("EADDRINUSE") || msg.includes("address already in use"),
-    actions: [
-      { kind: "fix", note: "Use random port allocation" },
-      { kind: "fix", note: "Ensure proper teardown in afterEach/afterAll" },
-      { kind: "inspect", note: "Check for leaked server instances" },
-    ],
-    confidence: "high",
-  },
-
-  // Time/date flakes
-  "date-flake": {
-    detect: (msg) => msg.includes("Date") || msg.includes("timestamp") || msg.includes("timezone"),
-    actions: [
-      { kind: "fix", note: "Use frozen time (vi.useFakeTimers, jest.useFakeTimers)" },
-      { kind: "fix", note: "Avoid locale-sensitive date formatting in assertions" },
-    ],
-    confidence: "medium",
-  },
-
-  // Order dependence
-  "order-dependent": {
-    detect: (msg, ctx) => ctx?.passedInIsolation === false,
-    actions: [
-      { kind: "fix", note: "Reset shared state in beforeEach" },
-      { kind: "inspect", note: "Run with --sequence.shuffle to detect order dependence" },
-    ],
-    confidence: "low",
-  },
-
-  // Null/undefined access
-  "type-error-null": {
-    detect: (msg, ctx) =>
-      ctx?.error?.type === "TypeError" &&
-      (msg.includes("undefined") || msg.includes("null") || msg.includes("Cannot read")),
-    actions: [
-      { kind: "fix", note: "Add null/undefined guard clause" },
-      { kind: "fix", note: "Use optional chaining (?.)" },
-      { kind: "inspect", note: "Trace to first non-test frame in stack" },
-    ],
-    confidence: "high",
-  },
-};
-```
-
----
-
-## Consequences
-
-### Positive
-
-- **Universal interface** — One schema for all test runners
-- **Agent-optimized** — Summary-first, failures-only, actionable
-- **Extensible** — Community can contribute adapters
-- **CI integration** — Emit PR comments, artifacts automatically
-
-### Negative
-
-- **Adapter maintenance** — Each runner version may need updates
-- **Lossy normalization** — Some runner-specific details may be lost
-- **Parse complexity** — Some formats (TAP) are ambiguous
-
-### Mitigations
-
-- Adapter versioning tied to runner versions
-- `raw` field in schema (both top-level and per-failure) preserves original data
-- `failureId` and `signature` fields enable cross-run correlation
-- Strict parsing with fallback to generic failure + raw preservation
-- `--strict` mode in CI prevents silent adapter mismatches
-
----
-
-## Implementation Phases
-
-### Phase 1: Internal Gold Standard
-
-Make Lex ecosystem tests emit AX-compliant output natively:
-
-- Configure Vitest reporters in LexSona/LexRunner
-- Add post-process script to Lex (Node test runner)
-- Validate all CI gates emit `ax-summary.json`
-
-### Phase 2: LexRunner Gate Command
-
-Ship `lexrunner gate test` with core adapters:
-
-- `vitest-json`, `node-tap`, `junit-xml`
-- CLI with pipe support
-- Markdown output for PR comments
-
-### Phase 3: Ecosystem Expansion
-
-- Additional adapters (pytest, go, phpunit)
-- Adapter registry/plugin system
-- Historical failure analysis (patterns across runs)
-
----
-
-## Monetization Considerations
-
-| Model             | Description                                           |
-| ----------------- | ----------------------------------------------------- |
-| **OSS Core**      | Base adapters (vitest, jest, junit) always free       |
-| **Adapter Packs** | Enterprise adapters (pytest, go, dotnet) as paid tier |
-| **Cloud API**     | Hosted endpoint: POST output → GET AX summary         |
-| **Analytics**     | Failure pattern detection, flaky test identification  |
-
----
-
-## References
-
-- [AX.md](../AX.md) — Agent Experience design principles
-- [ADR-003](ADR-003-gate-uniform-execution.md) — Gate uniform execution
-- [gates.md](../gates.md) — Gate specification
-- [AGENTS.md](/AGENTS.md) — Runner operating principles
-
----
-
-## Appendix: Naming Decision
-
-**Schema name:** `AXTestResult` (internal consistency with AX vocabulary)
-
-**Public documentation:** Describe as "Agent-optimized test report (AX)" to be self-documenting.
-
-**Alternative considered:** `AgentTestReport` — more self-documenting but loses brand continuity.
-
-**Decision:** Keep `AXTestResult` for schema/code, use "Agent-optimized" in user-facing docs.
-
----
-
-## Changelog
-
-- **2026-01-04:** Initial proposal
-- **2026-01-04:** Incorporated Lex feedback:
-  - Added `raw` field at both top-level and per-failure
-  - Added `failureId` and `signature` for failure identity/grouping
-  - Added `stackFrames[]` for structured stack traces
-  - Added `assertion` details and enhanced `diff` with unified format
-  - Added `run` metadata for reproducibility
-  - Made `nextActions` support structured objects with `kind`
-  - Added `nextActionsMeta.confidence` for guidance weighting
-  - Clarified `coverage` units as percentages
-  - Added `--strict` mode recommendation for CI
-  - Expanded adapter priority rationale
-  - Added per-runner rerun templates
-  - Expanded failure pattern library
-- [AGENTS.md](/AGENTS.md) — Runner operating principles