indusagi-coding-agent 0.1.42 → 0.1.43

package/guides/OBSERVE_DEEP_VALIDATION.md

@@ -0,0 +1,252 @@
+# Observe deep validation
+
+Date: 2026-03-30
+
+This document records the larger end-to-end validation run for package-based Observe integration in `indusagi-screth` using local `../indusvx-screth` behind the `indusagi` package import path.
+
+## What was validated
+
+The deeper validation used:
+
+- local package dependency: `"indusagi": "file:../indusvx-screth"`
+- app-level Observe bootstrap in `app/runtime/observe.ts`
+- runtime wiring through:
+  - `app/main.ts`
+  - `app/runtime/sdk.ts`
+  - `app/runtime/agent-session.ts`
+- a larger harness:
+  - `app/dev/observe-deep.ts`
+- npm script:
+  - `npm run observe:deep`
+
+## Commands run
+
+### `indusvx-screth`
+
+```bash
+npm run build
+npm test
+```
+
+### `indusagi-screth`
+
+```bash
+npm run build
+npm test
+npm run observe:deep
+```
+
+## Result summary
+
+### Build/test
+
+- `indusvx-screth` build: ✅
+- `indusvx-screth` tests: ✅ (`56` files, `92` tests)
+- `indusagi-screth` build: ✅
+- `indusagi-screth` tests: ✅ (`13` files, `781` tests)
+- `indusagi-screth` deep Observe validation: ✅
+
+## Deep validation output
+
+Output root:
+
+- `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/`
+
+Key files:
+
+- summary JSON:
+  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/summary.json`
+- harness report:
+  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/report.md`
+- control plane:
+  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/agent/observe/control-plane.json`
+- runtime records:
+  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/agent/observe/runtime-records/*.ndjson`
+- artifact payloads:
+  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/agent/observe/artifacts/*`
+- generated workspace:
+  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/workspace/`
+
+Dashboard URL during validation:
+
+- `http://127.0.0.1:61184/api/observe/ui`
+
+## What the deep harness exercised
+
+The deeper test covered all of the following in one run:
+
+- package-based import resolution through `indusagi/...`
+- Observe bootstrap from app runtime
+- real `createAgentSession(...)` path
+- agent prompt flow
+- multi-turn tool-call execution
+- read tool usage
+- write tool usage
+- edit tool usage
+- explicit `session.executeBash(...)`
+- session fork
+- session switch
+- session new-session flow
+- trace mirroring into control plane
+- artifact persistence
+- dataset creation
+- experiment creation
+- score creation
+- HTTP API validation
+- privacy redaction validation
+
+## Observed stats
+
+From the deep run summary:
+
+- total records: `83`
+- control-plane traces: `7`
+- control-plane sessions: `3`
+- control-plane artifacts: `1`
+- control-plane experiments: `1`
+
+By record type:
+
+- `entry: 50`
+- `span: 20`
+- `trace: 7`
+- `event: 6`
+
+By kind:
+
+- `session.entry: 7`
+- `agent.event: 43`
+- `tool_call: 4`
+- `turn: 4`
+- `agent_run: 4`
+- `state: 2`
+- `annotation: 3`
+- `delta: 1`
+- `model_generation: 1`
+- `command_execution: 1`
+- `custom: 10`
+- `session_fork: 1`
+- `session_switch: 2`
+
+## Concrete output checks
+
+### Workspace mutation checks
+
+The deep test verified that tools really changed files.
+
+Confirmed:
+
+- `workspace/src/index.ts` was edited to:
+
+```ts
+export const APP_NAME = "observe-deep-test-updated";
+```
+
+- `workspace/reports/generated-report.md` was created and contained:
+
+```md
+# Generated Report
+
+This file was created by the deep Observe validation harness.
+```
+
+### Control-plane checks
+
+Confirmed persisted entities in control plane:
+
+- sessions
+- traces
+- artifacts
+- prompts
+- datasets
+- dataset items
+- experiments
+- experiment runs
+- scores
+
+### API checks
+
+Confirmed these endpoints responded during the run:
+
+- `/api/observe/health`
+- `/api/observe/stats`
+- `/api/observe/records`
+- `/api/observe/control-plane/sessions`
+- `/api/observe/control-plane/traces`
+- `/api/observe/control-plane/artifacts`
+- `/api/observe/control-plane/experiments`
+- `/api/observe/ui`
+
+## Privacy validation
+
+A privacy probe intentionally emitted secret-like values via Observe runtime data:
+
+- `apiKey`
+- `token`
+- `password`
+- `authorization`
+
+The run verified that raw secrets did **not** appear in:
+
+- runtime NDJSON exports
+- Observe API responses
+- control-plane JSON
+
+Observed redaction example in runtime output:
+
+```json
+{
+  "apiKey": "[REDACTED]",
+  "token": "[REDACTED]",
+  "password": "[REDACTED]",
+  "nested": {
+    "authorization": "[REDACTED]"
+  }
+}
+```
+
+## What is working well now
+
+The package-based local integration is working well enough to validate that:
+
+- `indusagi-screth` can use local `indusvx-screth` through `indusagi/...` imports
+- app-level Observe bootstrap is active
+- runtime traces/spans/events are emitted
+- tool-call spans are emitted
+- model-generation spans are emitted
+- command execution spans are emitted
+- session fork/switch/new traces are emitted
+- control-plane persistence works
+- artifact storage works
+- dashboard/API works
+- privacy redaction works in exported/queryable telemetry
+
+## Remaining gaps / not yet covered by this deep harness
+
+This run did **not** fully exercise:
+
+- MCP connection/request/call flows
+- TUI Observe panel/overlay integration
+- subagent/task trace propagation
+- real networked model providers
+- memory package replacement parity beyond app-level instrumentation hooks
+
+## Important caveat found
+
+One practical gap still visible in the deep validation output:
+
+- session files were still being created under the default global session directory path rather than being fully isolated under the custom deep-test agent directory
+
+So Observe itself worked well, but session persistence path ownership/isolation still deserves cleanup if fully self-contained deep-test environments are required.
+
+## Overall conclusion
+
+This bigger test passed.
+
+The current package-based approach is now validated as:
+
+- stronger than the original smoke harness
+- capable of producing rich Observe telemetry from the real app runtime path
+- good for local product-level observability validation
+
+It is **not yet complete full-product coverage** because MCP, TUI, and subagent paths still need dedicated deep tests.

package/guides/OBSERVE_LOCAL_TESTING.md

@@ -0,0 +1,183 @@
+# Observe local testing in `indusagi-screth`
+
+This guide shows the most reliable way to test the new clean-room Observe stack from the sibling `indusvx-screth/` package **inside the `indusagi-screth` repo context**.
+
+## Why this path
+
+A direct full package swap from published `indusagi` to local `indusvx-screth` is not the safest first step because `indusagi-screth` still imports extra surfaces like `indusagi/memory` that are not part of the current `indusvx-screth` exports.
+
+So the working approach added here is:
+
+1. build the sibling `../indusvx-screth` package locally
+2. run a smoke harness from `indusagi-screth`
+3. dynamically load `../indusvx-screth/dist/{agent,ai,observe}.js`
+4. run a real local agent pass against the `indusagi-screth` repo
+5. persist and expose Observe runtime + control-plane outputs
+
+This exercises:
+
+- runtime traces/spans/events
+- tool lifecycle spans
+- model generation spans
+- file exporter output
+- file-backed control-plane persistence
+- artifact blob persistence
+- HTTP stats/query/dashboard
+- prompt/dataset/experiment/score smoke objects
+- privacy/redaction pipeline
+
+## Commands
+
+From `indusagi-screth/`:
+
+```bash
+npm run observe:smoke
+```
+
+That runs a one-shot smoke test.
+
+To keep the dashboard server running for manual inspection:
+
+```bash
+npm run observe:smoke:serve
+```
+
+## What the smoke harness does
+
+The harness lives at:
+
+- `app/dev/observe-smoke.ts`
+
+It:
+
+1. builds and loads local `../indusvx-screth`
+2. creates an Observe runtime with privacy controls enabled
+3. writes runtime records with `FileObserveExporter`
+4. mirrors trace records into a durable `FileObserveControlPlaneStore`
+5. stores artifact blobs with `FileObserveArtifactContentStore`
+6. starts `ObserveHttpServer`
+7. runs a local `Agent` using:
+   - mock model
+   - Observe runtime
+   - real local `read` and `ls` tools
+8. creates smoke prompt/dataset/experiment/score objects linked to the produced trace
+
+## Output locations
+
+Each run writes into:
+
+- `.indusagi/observe-smoke/<timestamp>/`
+
+Inside that directory you will see:
+
+- `runtime-records/` → exported runtime records
+- `control-plane.json` → durable control-plane snapshot
+- `artifacts/` → external artifact blob files
+
+## What to inspect
+
+### 1. Runtime records
+
+Inspect the NDJSON output under:
+
+- `.indusagi/observe-smoke/<timestamp>/runtime-records/`
+
+Look for records like:
+
+- `recordType: "trace"`
+- `recordType: "span"`
+- `kind: "agent_run"`
+- `kind: "turn"`
+- `kind: "tool_call"`
+- `kind: "model_generation"`
+
+### 2. Control-plane snapshot
+
+Open:
+
+- `.indusagi/observe-smoke/<timestamp>/control-plane.json`
+
+Look for:
+
+- sessions
+- traces
+- prompts
+- datasets
+- experiments
+- experimentRuns
+- scores
+- artifacts
+
+### 3. Dashboard
+
+When the harness prints a URL like:
+
+- `http://127.0.0.1:4319/api/observe/ui`
+
+open it in your browser.
+
+You should see:
+
+- runtime stats
+- latest traces
+- artifacts
+- API endpoint map
+
+### 4. Query endpoints
+
+You can also query the server directly:
+
+```bash
+curl http://127.0.0.1:4319/api/observe/health
+curl http://127.0.0.1:4319/api/observe/stats
+curl "http://127.0.0.1:4319/api/observe/records?limit=20"
+curl http://127.0.0.1:4319/api/observe/control-plane/traces
+curl http://127.0.0.1:4319/api/observe/control-plane/artifacts
+```
+
+Adjust the port to whatever the smoke harness prints.
+
+## What counts as a pass
+
+A good run should produce all of the following:
+
+- at least one trace record
+- at least one model generation span
+- at least one tool call span
+- a persisted control-plane trace
+- at least one artifact
+- a working dashboard URL
+- no leaked fake secrets in exported payloads
+
+## Manual deep-checks
+
+After `npm run observe:smoke:serve`, manually verify:
+
+### Tool path
+
+The run should show tool activity for:
+
+- `read package.json`
+- `ls app`
+
+### Privacy path
+
+Search the exported NDJSON for raw secrets. The harness enables privacy controls, so sensitive-looking values should be redacted or omitted before export.
+
+### Persistence path
+
+Stop the process and confirm that:
+
+- runtime files remain on disk
+- `control-plane.json` remains readable
+- artifact blob files remain in `artifacts/`
+
+## Current limitation
+
+This harness tests observability **in the `indusagi-screth` repo context** using the local sibling `indusvx-screth` runtime surfaces, but it does **not** yet replace every `indusagi/*` import across the full `indusagi-screth` app.
+
+That is intentional: it gives a working, low-risk validation path first.
+
+## Next integration step after smoke success
+
+If the smoke harness looks good, the next deeper step is to wire Observe directly into `indusagi-screth/app/runtime/sdk.ts` so the main coding-agent runtime itself passes a real Observe runtime into the created `Agent` instance.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "indusagi-coding-agent",
-  "version": "0.1.42",
-  "description": "Local TypeScript coding-agent workspace built around the sibling indusvx-screth_copy package.",
+  "version": "0.1.43",
+  "description": "Terminal-first TypeScript coding agent built on the published indusagi package.",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -33,7 +33,11 @@
     "clean": "shx rm -rf dist",
     "sync:export-vendors": "node tasks/sync-export-vendors.mjs",
     "copy:export-assets": "node tasks/copy-export-assets.mjs",
-    "build": "npm run sync:export-vendors && tsc -p tsconfig.build.json && npm run copy:export-assets && shx chmod +x dist/cli.js",
+    "build": "npm run build:observe-local && npm run sync:export-vendors && tsc -p tsconfig.build.json && npm run copy:export-assets && shx chmod +x dist/cli.js",
+    "build:observe-local": "node -e \"console.log('Using published indusagi@0.12.23')\"",
+    "observe:smoke": "npm run build && node dist/dev/observe-smoke.js",
+    "observe:smoke:serve": "npm run build && node dist/dev/observe-smoke.js --serve",
+    "observe:deep": "npm run build && node dist/dev/observe-deep.js",
     "test": "vitest --run",
     "verify": "npm run build && npm test"
   },
@@ -48,9 +52,12 @@
     "file-type": "^21.1.1",
     "glob": "^11.0.3",
     "highlight.js": "^11.11.1",
-    "indusagi": "^0.12.22",
+    "i": "^0.3.7",
+    "indusagi": "^0.12.23",
+    "indusagi-coding-agent": "^0.1.42",
     "marked": "^17.0.5",
     "minimatch": "^10.0.3",
+    "npm": "^11.12.0",
     "strip-ansi": "^7.1.2",
     "uuid": "^11.1.0"
   },