@smartergpt/lexrunner 1.0.0

package/docs/adr/ADR-008-lex-packaging.md

@@ -0,0 +1,243 @@
+# ADR-008: Lex Packaging Strategy
+
+**Status:** Accepted
+**Date:** 2026-01-01
+**Authors:** LexRunner Team
+
+---
+
+## Context
+
+LexRunner depends on Lex (MIT-licensed core) for memory, policy, atlas, and error handling functionality. The question is how to package and distribute this dependency to ensure:
+
+1. **Stable versioning** — Predictable updates and rollback capability
+2. **Security** — Supply chain integrity and provenance
+3. **Developer experience** — Simple installation and updates
+4. **Licensing clarity** — MIT (Lex) vs. Proprietary (LexRunner) boundaries
+
+Options considered:
+
+1. **npm registry publish** — Publish Lex to public npm registry
+2. **Git submodule** — Track Lex as a Git submodule in LexRunner repo
+3. **Vendoring** — Copy Lex source into `vendor/lex/` directory
+
+---
+
+## Decision
+
+We adopt **npm registry publish** as the packaging strategy for Lex.
+
+### Implementation
+
+Lex is published to the public npm registry as:
+
+- **Package name:** `@smartergpt/lex`
+- **Current version:** `2.1.1` (used by LexRunner)
+- **License:** MIT
+- **Registry:** https://registry.npmjs.org
+
+LexRunner consumes Lex via standard npm dependency mechanism:
+
+```json
+{
+  "dependencies": {
+    "@smartergpt/lex": "^2.1.1"
+  }
+}
+```
+
+### Export Paths
+
+Lex provides modular exports for selective imports:
+
+- `@smartergpt/lex` — Core exports
+- `@smartergpt/lex/types` — Frame types and schemas
+- `@smartergpt/lex/errors` — AXError types and utilities
+- `@smartergpt/lex/policy` — Policy validation
+- `@smartergpt/lex/atlas` — Code atlas and dependency analysis
+- `@smartergpt/lex/aliases` — Module ID resolution
+- `@smartergpt/lex/logger` — Structured logging
+
+### Versioning Policy
+
+- **Semantic Versioning:** Lex follows semver strictly
+  - **MAJOR** — Breaking API changes
+  - **MINOR** — New features, backward compatible
+  - **PATCH** — Bug fixes, backward compatible
+- **LexRunner Pinning:** Use caret range (`^2.1.1`) to allow MINOR/PATCH updates
+- **Testing:** CI validates LexRunner against pinned Lex version before merge
+
+---
+
+## Rationale
+
+### Why npm Publish?
+
+1. **Industry Standard** — npm is the standard distribution mechanism for TypeScript/Node.js packages
+2. **Dependency Management** — npm/package-lock.json ensures reproducible builds
+3. **Security Scanning** — npm audit and GitHub Dependabot provide vulnerability detection
+4. **Version Control** — Semantic versioning and package.json constraints prevent breaking changes
+5. **Distribution** — Public registry enables community adoption of Lex independently
+6. **Build Artifacts** — Published package includes only built `dist/` files, not source
+
+### Why Not Git Submodule?
+
+- **Build Complexity** — Requires building Lex from source during LexRunner build
+- **Version Pinning** — Git commit hashes are less discoverable than semver
+- **Dependency Resolution** — npm handles transitive dependencies; Git submodules don't
+- **Breaking Changes** — Harder to prevent breaking changes from upstream
+
+### Why Not Vendoring?
+
+- **Duplication** — Copies Lex source into LexRunner repo
+- **License Violations** — Risk of modifying Lex code without proper attribution
+- **Sync Burden** — Manual process to pull upstream changes
+- **Security** — No automated vulnerability scanning for vendored code
+- **Repo Size** — Increases LexRunner repository size unnecessarily
+
+---
+
+## Consequences
+
+### Positive
+
+- **Simple Updates** — `npm update @smartergpt/lex` to get latest compatible version
+- **Security Scanning** — GitHub Dependabot alerts on vulnerabilities in Lex
+- **Reproducible Builds** — package-lock.json ensures exact version consistency
+- **Clear Licensing** — npm package.json declares MIT license for Lex
+- **Public Distribution** — Lex can be adopted independently by other projects
+- **Build Artifacts Only** — Published package contains pre-built dist/ files
+
+### Negative
+
+- **Registry Dependency** — Requires npm registry availability for installation
+- **Supply Chain Risk** — Potential for npm package hijacking (mitigated by package-lock.json)
+- **Version Lag** — LexRunner may lag behind latest Lex releases (intentional for stability)
+
+---
+
+## Security Implications
+
+### Supply Chain
+
+- **Package Integrity** — npm uses SHA-512 integrity hashes in package-lock.json
+- **Provenance** — GitHub Actions publishes Lex with provenance attestation
+- **Audit Trail** — npm registry provides download and publish audit logs
+- **Dependabot** — Automated alerts for known vulnerabilities
+
+### Versioning & Updates
+
+1. **Controlled Updates** — Use caret range (`^2.1.1`) to allow PATCH/MINOR, block MAJOR
+2. **Testing Gate** — CI must pass before merging Lex updates
+3. **Review Process** — PRs with Lex updates require maintainer approval
+4. **Rollback** — Revert package.json change if update breaks LexRunner
+
+### Rollback Strategy
+
+If a Lex update breaks LexRunner:
+
+1. **Immediate Rollback** — Revert package.json to previous version
+2. **Lock Specific Version** — Change `^2.1.1` to `2.1.1` to prevent auto-updates
+3. **Root Cause Analysis** — Identify breaking change in Lex
+4. **Upstream Issue** — File issue in Lex repository
+5. **Fix Forward** — Once Lex releases patch, update LexRunner
+
+---
+
+## Update Workflow
+
+### Checking for Updates
+
+Use the provided script to check for Lex updates:
+
+```bash
+./scripts/package-lex.sh check
+```
+
+### Updating Lex
+
+Manual update process:
+
+```bash
+# 1. Check for updates
+npm outdated @smartergpt/lex
+
+# 2. Update to latest compatible version
+npm update @smartergpt/lex
+
+# 3. Or update to specific version
+npm install @smartergpt/lex@2.1.1
+
+# 4. Run tests
+npm test
+
+# 5. Commit if tests pass
+git add package.json package-lock.json
+git commit -m "chore: update @smartergpt/lex to 2.1.1"
+```
+
+Automated update (via Dependabot):
+
+1. **Dependabot PR** — GitHub creates PR with Lex update
+2. **CI Validation** — Automated tests run
+3. **Review** — Maintainer reviews changes
+4. **Merge** — Merge if tests pass and no breaking changes
+
+---
+
+## Licensing Documentation
+
+### Lex (MIT)
+
+- **Package:** `@smartergpt/lex`
+- **License:** MIT (open source)
+- **Attribution:** Required (see NOTICE.md)
+- **Repository:** https://github.com/Guffawaffle/lex
+
+### LexRunner (Proprietary)
+
+- **Package:** `lexrunner`
+- **License:** UNLICENSED (proprietary)
+- **Attribution:** Uses Lex (MIT) as dependency
+- **Repository:** https://github.com/Guffawaffle/lexrunner
+
+### License Compliance
+
+LexRunner's license compliance is validated via:
+
+- **CI Gate:** `scripts/check-license-compliance.mjs`
+- **Verification:** Ensures proper Lex attribution in NOTICE.md
+- **Enforcement:** CI fails if compliance checks fail
+
+---
+
+## Alternatives Considered
+
+### 1. Private npm Registry
+
+**Rejected** — Lex is MIT-licensed and intended for public distribution. Private registry adds unnecessary complexity and limits community adoption.
+
+### 2. Monorepo with Lex
+
+**Rejected** — Separating Lex and LexRunner as independent repositories preserves licensing boundaries and allows independent evolution.
+
+### 3. Inline Lex Code
+
+**Rejected** — Violates DRY principle and creates license compliance issues. LexRunner must consume Lex as a dependency, not duplicate source code.
+
+---
+
+## References
+
+- [Lex npm Package](https://www.npmjs.com/package/@smartergpt/lex)
+- [Lex Repository](https://github.com/Guffawaffle/lex)
+- [Lex Public API](../LEX_PUBLIC_API.md)
+- [License Compliance Guide](../LICENSING.md)
+- [ADR-000: Product Naming and Branding](./ADR-000-product-naming-and-branding.md)
+
+---
+
+## Sign-Off
+
+Accepted by: LexRunner Team
+Date: 2026-01-01

package/docs/adr/ADR-009-ax-test-output-adapters.md

@@ -0,0 +1,440 @@
+# ADR-009: AX Test Output Adapters
+
+**Status:** Proposed
+**Date:** 2026-01-04
+**Authors:** lexrunner team
+
+---
+
+## Context
+
+Test runners emit output in wildly different formats:
+
+| Runner           | Native Output              |
+| ---------------- | -------------------------- |
+| Vitest           | JSON, TAP, default console |
+| Jest             | JSON, JUnit XML            |
+| PHPUnit          | XML, JSON                  |
+| pytest           | JSON, JUnit XML            |
+| go test          | JSON lines                 |
+| Node test runner | TAP, spec                  |
+
+Agents consuming test results need **consistent, actionable output** regardless of the underlying test platform. Current CI gates capture pass/fail status but lack:
+
+1. **Structured failure details** with file:line locations
+2. **`nextActions[]`** suggesting recovery steps (AX pattern)
+3. **Diff context** showing expected vs actual
+4. **Summary-first output** for token-efficient agent consumption
+
+---
+
+## Decision
+
+LexRunner will provide a **Test Gate Adapter system** that normalizes any test runner output into an AX-compliant schema.
+
+### Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│  lexrunner gate test                                        │
+│                                                             │
+│   Input Adapters          Core            Output Formats    │
+│  ┌──────────────┐    ┌──────────┐    ┌──────────────┐      │
+│  │ vitest-json  │───▶│          │───▶│ ax-summary   │      │
+│  │ jest-json    │───▶│ Normalize│───▶│ (JSON)       │      │
+│  │ junit-xml    │───▶│    to    │───▶│              │      │
+│  │ tap          │───▶│ AXTest   │    └──────────────┘      │
+│  │ pytest-json  │───▶│ Result   │    ┌──────────────┐      │
+│  │ go-test-json │───▶│          │───▶│ pr-comment   │      │
+│  │ phpunit-xml  │───▶│          │───▶│ (Markdown)   │      │
+│  └──────────────┘    └──────────┘    └──────────────┘      │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### AXTestResult Schema (v1.0.0)
+
+> **Design principle:** This schema is a **verifiable contract**, not just a convenient shape.
+> Normalization is never a one-way door — `raw` preserves original data for auditability.
+
+```typescript
+interface AXTestResult {
+  schemaVersion: "1.0.0";
+  timestamp: string; // ISO 8601
+
+  summary: {
+    total: number;
+    passed: number;
+    failed: number;
+    skipped: number;
+    durationMs: number;
+  };
+
+  // Only populated for failures - passing tests omitted for token efficiency
+  failures: AXTestFailure[];
+
+  // Optional coverage summary (percentages, 0-100)
+  coverage?: {
+    linesPct: number;
+    branchesPct: number;
+    functionsPct: number;
+    statementsPct?: number;
+  };
+
+  // Run metadata for reproducibility
+  run?: {
+    cwd?: string; // Working directory
+    command?: string; // Exact command executed
+    ci?: boolean; // Running in CI environment
+    os?: string; // e.g., "linux", "darwin", "win32"
+    nodeVersion?: string; // e.g., "20.10.0"
+  };
+
+  // Adapter metadata
+  adapter: {
+    name: string; // e.g., "vitest-json"
+    version: string;
+    source: string; // Original output file/stream
+  };
+
+  // Original unparsed output — normalization is never a one-way door
+  raw?: string;
+}
+
+interface AXTestFailure {
+  // Identity & grouping (enables flaky detection, trend analysis)
+  failureId: string; // Deterministic hash of (file + name + error.type + canonicalized message)
+  signature?: string; // Normalized signature with volatile values scrubbed (timestamps, ports)
+
+  // Location
+  file: string; // Relative path
+  line: number; // 1-indexed
+  column?: number;
+
+  name: string; // Test name / describe path
+  suite?: string; // Parent suite/describe
+
+  error: {
+    message: string;
+    type?: string; // e.g., "AssertionError", "TypeError"
+    stack?: string; // Human-readable stack trace
+  };
+
+  // Structured stack for agent navigation
+  stackFrames?: Array<{
+    file: string;
+    line: number;
+    column?: number;
+    function?: string;
+  }>;
+
+  // Assertion details
+  assertion?: {
+    operator?: string; // e.g., "toBe", "toEqual", "strictEqual"
+    expectedType?: string; // e.g., "string", "LexSonaError"
+    actualType?: string; // e.g., "undefined", "TypeError"
+  };
+
+  diff?: {
+    expected: string;
+    actual: string;
+    unified?: string; // Unified diff format when applicable
+    contextLines?: number; // Lines of context included
+  };
+
+  // AX-compliant recovery guidance (string or structured)
+  nextActions: Array<string | AXNextAction>;
+  nextActionsMeta?: {
+    confidence: "high" | "medium" | "low";
+  };
+
+  // Duration for identifying slow tests
+  durationMs?: number;
+
+  // Original failure data from adapter — preserves runner-specific details
+  raw?: unknown;
+}
+
+// Structured next action for agent execution
+interface AXNextAction {
+  kind: "rerun" | "inspect" | "fix" | "doc";
+  cmd?: string; // Executable command
+  note: string; // Human-readable description
+}
+```
+
+### CLI Interface
+
+```bash
+# Explicit adapter (REQUIRED in CI mode for determinism)
+lexrunner gate test --adapter=vitest --input=test-results.json
+
+# Strict mode: require explicit adapter, fail on ambiguity
+lexrunner gate test --strict --input=test-results.json
+# Error: --adapter required in strict mode
+
+# Auto-detect from file extension/content (local DX only, warns in CI)
+lexrunner gate test --input=test-results.json
+# Warning: Auto-detected adapter 'vitest-json'. Use --adapter for CI.
+
+# Pipe mode (for integration)
+vitest run --reporter=json | lexrunner gate test --adapter=vitest
+
+# Output formats
+lexrunner gate test --input=results.json --output=ax-summary.json
+lexrunner gate test --input=results.json --format=markdown > pr-comment.md
+
+# Include raw output for auditability
+lexrunner gate test --adapter=vitest --input=results.json --include-raw
+```
+
+> **CI Recommendation:** Always use `--adapter` explicitly or `--strict` mode.
+> Auto-detect is convenient for local development but can silently choose
+> the wrong parser if output formats evolve between runner versions.
+
+### Adapter Interface
+
+```typescript
+interface TestAdapter {
+  name: string;
+  version: string;
+
+  // File extensions this adapter handles
+  extensions: string[];
+
+  // Content detection (for auto-detect)
+  detect(content: string): boolean;
+
+  // Parse raw output into normalized form
+  parse(input: string | Buffer): AXTestResult;
+}
+```
+
+### Built-in Adapters (v1.0.0)
+
+| Adapter        | Priority | Rationale                                    |
+| -------------- | -------- | -------------------------------------------- |
+| `junit-xml`    | P0       | Universal CI lingua franca across ecosystems |
+| `vitest-json`  | P0       | LexSona, LexRunner use Vitest                |
+| `jest-json`    | P0       | Huge footprint, close to Vitest patterns     |
+| `node-tap`     | P1       | Lex uses Node test runner with TAP           |
+| `pytest-json`  | P1       | Very common in mixed stacks                  |
+| `go-test-json` | P2       | Extremely structured, popular in infra repos |
+| `phpunit-xml`  | P2       | Enterprise/legacy usage                      |
+
+> **Note:** TAP is worth having but is one of the messier formats.
+> Treat as "best effort + raw preservation" for edge cases.
+
+### `nextActions[]` Generation
+
+Adapters generate contextual suggestions based on failure patterns.
+Actions can be strings (for display) or structured objects (for execution).
+
+#### Rerun Templates (Per-Runner)
+
+Each adapter provides runner-specific rerun commands:
+
+| Runner  | Rerun Template                             |
+| ------- | ------------------------------------------ |
+| Vitest  | `vitest run -t "<name>"`                   |
+| Jest    | `jest -t "<name>" --runInBand`             |
+| pytest  | `pytest -k "<expr>" -q`                    |
+| go test | `go test ./... -run "<regex>"`             |
+| PHPUnit | `phpunit --filter "<name>"`                |
+| Node    | `node --test --test-name-pattern="<name>"` |
+
+#### Common Failure Patterns
+
+```typescript
+const FAILURE_PATTERNS: Record<string, PatternHandler> = {
+  // Assertion failures
+  "snapshot-mismatch": {
+    detect: (msg) => msg.includes("snapshot") || msg.includes("toMatchSnapshot"),
+    actions: [
+      { kind: "fix", cmd: "npm test -- -u", note: "Update snapshots" },
+      { kind: "inspect", note: "Verify snapshot changes are intentional" },
+    ],
+    confidence: "high",
+  },
+
+  // Module resolution
+  "module-not-found": {
+    detect: (msg) => msg.includes("Cannot find module") || msg.includes("ERR_MODULE_NOT_FOUND"),
+    actions: [
+      { kind: "inspect", note: "Verify import path is correct" },
+      { kind: "inspect", note: "Check tsconfig paths configuration" },
+      { kind: "fix", cmd: "npm install", note: "Reinstall dependencies" },
+      { kind: "inspect", note: "Check for workspace hoisting issues" },
+    ],
+    confidence: "high",
+  },
+
+  // Timeouts
+  timeout: {
+    detect: (msg) => msg.includes("timeout") || msg.includes("exceeded"),
+    actions: [
+      { kind: "inspect", note: "Identify slow async operation" },
+      { kind: "fix", note: "Consider using fake timers" },
+      { kind: "fix", note: "Increase test timeout if legitimately slow" },
+    ],
+    confidence: "medium",
+  },
+
+  // Unhandled promises
+  "unhandled-promise": {
+    detect: (msg) => msg.includes("unhandled") && msg.includes("promise"),
+    actions: [
+      { kind: "fix", note: "Await the promise or return it from test" },
+      { kind: "fix", note: "Add error handler to promise chain" },
+    ],
+    confidence: "high",
+  },
+
+  // Port/network collisions
+  "port-in-use": {
+    detect: (msg) => msg.includes("EADDRINUSE") || msg.includes("address already in use"),
+    actions: [
+      { kind: "fix", note: "Use random port allocation" },
+      { kind: "fix", note: "Ensure proper teardown in afterEach/afterAll" },
+      { kind: "inspect", note: "Check for leaked server instances" },
+    ],
+    confidence: "high",
+  },
+
+  // Time/date flakes
+  "date-flake": {
+    detect: (msg) => msg.includes("Date") || msg.includes("timestamp") || msg.includes("timezone"),
+    actions: [
+      { kind: "fix", note: "Use frozen time (vi.useFakeTimers, jest.useFakeTimers)" },
+      { kind: "fix", note: "Avoid locale-sensitive date formatting in assertions" },
+    ],
+    confidence: "medium",
+  },
+
+  // Order dependence
+  "order-dependent": {
+    detect: (msg, ctx) => ctx?.passedInIsolation === false,
+    actions: [
+      { kind: "fix", note: "Reset shared state in beforeEach" },
+      { kind: "inspect", note: "Run with --sequence.shuffle to detect order dependence" },
+    ],
+    confidence: "low",
+  },
+
+  // Null/undefined access
+  "type-error-null": {
+    detect: (msg, ctx) =>
+      ctx?.error?.type === "TypeError" &&
+      (msg.includes("undefined") || msg.includes("null") || msg.includes("Cannot read")),
+    actions: [
+      { kind: "fix", note: "Add null/undefined guard clause" },
+      { kind: "fix", note: "Use optional chaining (?.)" },
+      { kind: "inspect", note: "Trace to first non-test frame in stack" },
+    ],
+    confidence: "high",
+  },
+};
+```
+
+---
+
+## Consequences
+
+### Positive
+
+- **Universal interface** — One schema for all test runners
+- **Agent-optimized** — Summary-first, failures-only, actionable
+- **Extensible** — Community can contribute adapters
+- **CI integration** — Emit PR comments, artifacts automatically
+
+### Negative
+
+- **Adapter maintenance** — Each runner version may need updates
+- **Lossy normalization** — Some runner-specific details may be lost
+- **Parse complexity** — Some formats (TAP) are ambiguous
+
+### Mitigations
+
+- Adapter versioning tied to runner versions
+- `raw` field in schema (both top-level and per-failure) preserves original data
+- `failureId` and `signature` fields enable cross-run correlation
+- Strict parsing with fallback to generic failure + raw preservation
+- `--strict` mode in CI prevents silent adapter mismatches
+
+---
+
+## Implementation Phases
+
+### Phase 1: Internal Gold Standard
+
+Make Lex ecosystem tests emit AX-compliant output natively:
+
+- Configure Vitest reporters in LexSona/LexRunner
+- Add post-process script to Lex (Node test runner)
+- Validate all CI gates emit `ax-summary.json`
+
+### Phase 2: LexRunner Gate Command
+
+Ship `lexrunner gate test` with core adapters:
+
+- `vitest-json`, `node-tap`, `junit-xml`
+- CLI with pipe support
+- Markdown output for PR comments
+
+### Phase 3: Ecosystem Expansion
+
+- Additional adapters (pytest, go, phpunit)
+- Adapter registry/plugin system
+- Historical failure analysis (patterns across runs)
+
+---
+
+## Monetization Considerations
+
+| Model             | Description                                           |
+| ----------------- | ----------------------------------------------------- |
+| **OSS Core**      | Base adapters (vitest, jest, junit) always free       |
+| **Adapter Packs** | Enterprise adapters (pytest, go, dotnet) as paid tier |
+| **Cloud API**     | Hosted endpoint: POST output → GET AX summary         |
+| **Analytics**     | Failure pattern detection, flaky test identification  |
+
+---
+
+## References
+
+- [AX.md](../AX.md) — Agent Experience design principles
+- [ADR-003](ADR-003-gate-uniform-execution.md) — Gate uniform execution
+- [gates.md](../gates.md) — Gate specification
+- [AGENTS.md](/AGENTS.md) — Runner operating principles
+
+---
+
+## Appendix: Naming Decision
+
+**Schema name:** `AXTestResult` (internal consistency with AX vocabulary)
+
+**Public documentation:** Describe as "Agent-optimized test report (AX)" to be self-documenting.
+
+**Alternative considered:** `AgentTestReport` — more self-documenting but loses brand continuity.
+
+**Decision:** Keep `AXTestResult` for schema/code, use "Agent-optimized" in user-facing docs.
+
+---
+
+## Changelog
+
+- **2026-01-04:** Initial proposal
+- **2026-01-04:** Incorporated Lex feedback:
+  - Added `raw` field at both top-level and per-failure
+  - Added `failureId` and `signature` for failure identity/grouping
+  - Added `stackFrames[]` for structured stack traces
+  - Added `assertion` details and enhanced `diff` with unified format
+  - Added `run` metadata for reproducibility
+  - Made `nextActions` support structured objects with `kind`
+  - Added `nextActionsMeta.confidence` for guidance weighting
+  - Clarified `coverage` units as percentages
+  - Added `--strict` mode recommendation for CI
+  - Expanded adapter priority rationale
+  - Added per-runner rerun templates
+  - Expanded failure pattern library
+- [AGENTS.md](/AGENTS.md) — Runner operating principles