indusagi-coding-agent 0.1.59 → 0.1.60

package/guides/OBSERVE_DEEP_VALIDATION.md

@@ -1,252 +0,0 @@
-# Observe deep validation
-
-Date: 2026-03-30
-
-This document records the larger end-to-end validation run for package-based Observe integration in `indusagi-screth` using local `../indusvx-screth` behind the `indusagi` package import path.
-
-## What was validated
-
-The deeper validation used:
-
-- local package dependency: `"indusagi": "file:../indusvx-screth"`
-- app-level Observe bootstrap in `app/runtime/observe.ts`
-- runtime wiring through:
-  - `app/main.ts`
-  - `app/runtime/sdk.ts`
-  - `app/runtime/agent-session.ts`
-- a larger harness:
-  - `app/dev/observe-deep.ts`
-- npm script:
-  - `npm run observe:deep`
-
-## Commands run
-
-### `indusvx-screth`
-
-```bash
-npm run build
-npm test
-```
-
-### `indusagi-screth`
-
-```bash
-npm run build
-npm test
-npm run observe:deep
-```
-
-## Result summary
-
-### Build/test
-
-- `indusvx-screth` build: ✅
-- `indusvx-screth` tests: ✅ (`56` files, `92` tests)
-- `indusagi-screth` build: ✅
-- `indusagi-screth` tests: ✅ (`13` files, `781` tests)
-- `indusagi-screth` deep Observe validation: ✅
-
-## Deep validation output
-
-Output root:
-
-- `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/`
-
-Key files:
-
-- summary JSON:
-  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/summary.json`
-- harness report:
-  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/report.md`
-- control plane:
-  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/agent/observe/control-plane.json`
-- runtime records:
-  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/agent/observe/runtime-records/*.ndjson`
-- artifact payloads:
-  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/agent/observe/artifacts/*`
-- generated workspace:
-  - `.indusagi/observe-deep/2026-03-30T13-37-06-998Z/workspace/`
-
-Dashboard URL during validation:
-
-- `http://127.0.0.1:61184/api/observe/ui`
-
-## What the deep harness exercised
-
-The deeper test covered all of the following in one run:
-
-- package-based import resolution through `indusagi/...`
-- Observe bootstrap from app runtime
-- real `createAgentSession(...)` path
-- agent prompt flow
-- multi-turn tool-call execution
-- read tool usage
-- write tool usage
-- edit tool usage
-- explicit `session.executeBash(...)`
-- session fork
-- session switch
-- session new-session flow
-- trace mirroring into control plane
-- artifact persistence
-- dataset creation
-- experiment creation
-- score creation
-- HTTP API validation
-- privacy redaction validation
-
-## Observed stats
-
-From the deep run summary:
-
-- total records: `83`
-- control-plane traces: `7`
-- control-plane sessions: `3`
-- control-plane artifacts: `1`
-- control-plane experiments: `1`
-
-By record type:
-
-- `entry: 50`
-- `span: 20`
-- `trace: 7`
-- `event: 6`
-
-By kind:
-
-- `session.entry: 7`
-- `agent.event: 43`
-- `tool_call: 4`
-- `turn: 4`
-- `agent_run: 4`
-- `state: 2`
-- `annotation: 3`
-- `delta: 1`
-- `model_generation: 1`
-- `command_execution: 1`
-- `custom: 10`
-- `session_fork: 1`
-- `session_switch: 2`
-
-## Concrete output checks
-
-### Workspace mutation checks
-
-The deep test verified that tools really changed files.
-
-Confirmed:
-
-- `workspace/src/index.ts` was edited to:
-
-```ts
-export const APP_NAME = "observe-deep-test-updated";
-```
-
-- `workspace/reports/generated-report.md` was created and contained:
-
-```md
-# Generated Report
-
-This file was created by the deep Observe validation harness.
-```
-
-### Control-plane checks
-
-Confirmed persisted entities in control plane:
-
-- sessions
-- traces
-- artifacts
-- prompts
-- datasets
-- dataset items
-- experiments
-- experiment runs
-- scores
-
-### API checks
-
-Confirmed these endpoints responded during the run:
-
-- `/api/observe/health`
-- `/api/observe/stats`
-- `/api/observe/records`
-- `/api/observe/control-plane/sessions`
-- `/api/observe/control-plane/traces`
-- `/api/observe/control-plane/artifacts`
-- `/api/observe/control-plane/experiments`
-- `/api/observe/ui`
-
-## Privacy validation
-
-A privacy probe intentionally emitted secret-like values via Observe runtime data:
-
-- `apiKey`
-- `token`
-- `password`
-- `authorization`
-
-The run verified that raw secrets did **not** appear in:
-
-- runtime NDJSON exports
-- Observe API responses
-- control-plane JSON
-
-Observed redaction example in runtime output:
-
-```json
-{
-  "apiKey": "[REDACTED]",
-  "token": "[REDACTED]",
-  "password": "[REDACTED]",
-  "nested": {
-    "authorization": "[REDACTED]"
-  }
-}
-```
-
-## What is working well now
-
-The package-based local integration is working well enough to validate that:
-
-- `indusagi-screth` can use local `indusvx-screth` through `indusagi/...` imports
-- app-level Observe bootstrap is active
-- runtime traces/spans/events are emitted
-- tool-call spans are emitted
-- model-generation spans are emitted
-- command execution spans are emitted
-- session fork/switch/new traces are emitted
-- control-plane persistence works
-- artifact storage works
-- dashboard/API works
-- privacy redaction works in exported/queryable telemetry
-
-## Remaining gaps / not yet covered by this deep harness
-
-This run did **not** fully exercise:
-
-- MCP connection/request/call flows
-- TUI Observe panel/overlay integration
-- subagent/task trace propagation
-- real networked model providers
-- memory package replacement parity beyond app-level instrumentation hooks
-
-## Important caveat found
-
-One practical gap still visible in the deep validation output:
-
-- session files were still being created under the default global session directory path rather than being fully isolated under the custom deep-test agent directory
-
-So Observe itself worked well, but session persistence path ownership/isolation still deserves cleanup if fully self-contained deep-test environments are required.
-
-## Overall conclusion
-
-This bigger test passed.
-
-The current package-based approach is now validated as:
-
-- stronger than the original smoke harness
-- capable of producing rich Observe telemetry from the real app runtime path
-- good for local product-level observability validation
-
-It is **not yet complete full-product coverage** because MCP, TUI, and subagent paths still need dedicated deep tests.

package/guides/OBSERVE_LOCAL_TESTING.md

@@ -1,183 +0,0 @@
-# Observe local testing in `indusagi-screth`
-
-This guide shows the most reliable way to test the new clean-room Observe stack from the sibling `indusvx-screth/` package **inside the `indusagi-screth` repo context**.
-
-## Why this path
-
-A direct full package swap from published `indusagi` to local `indusvx-screth` is not the safest first step because `indusagi-screth` still imports extra surfaces like `indusagi/memory` that are not part of the current `indusvx-screth` exports.
-
-So the working approach added here is:
-
-1. build the sibling `../indusvx-screth` package locally
-2. run a smoke harness from `indusagi-screth`
-3. dynamically load `../indusvx-screth/dist/{agent,ai,observe}.js`
-4. run a real local agent pass against the `indusagi-screth` repo
-5. persist and expose Observe runtime + control-plane outputs
-
-This exercises:
-
-- runtime traces/spans/events
-- tool lifecycle spans
-- model generation spans
-- file exporter output
-- file-backed control-plane persistence
-- artifact blob persistence
-- HTTP stats/query/dashboard
-- prompt/dataset/experiment/score smoke objects
-- privacy/redaction pipeline
-
-## Commands
-
-From `indusagi-screth/`:
-
-```bash
-npm run observe:smoke
-```
-
-That runs a one-shot smoke test.
-
-To keep the dashboard server running for manual inspection:
-
-```bash
-npm run observe:smoke:serve
-```
-
-## What the smoke harness does
-
-The harness lives at:
-
-- `app/dev/observe-smoke.ts`
-
-It:
-
-1. builds and loads local `../indusvx-screth`
-2. creates an Observe runtime with privacy controls enabled
-3. writes runtime records with `FileObserveExporter`
-4. mirrors trace records into a durable `FileObserveControlPlaneStore`
-5. stores artifact blobs with `FileObserveArtifactContentStore`
-6. starts `ObserveHttpServer`
-7. runs a local `Agent` using:
-   - mock model
-   - Observe runtime
-   - real local `read` and `ls` tools
-8. creates smoke prompt/dataset/experiment/score objects linked to the produced trace
-
-## Output locations
-
-Each run writes into:
-
-- `.indusagi/observe-smoke/<timestamp>/`
-
-Inside that directory you will see:
-
-- `runtime-records/` → exported runtime records
-- `control-plane.json` → durable control-plane snapshot
-- `artifacts/` → external artifact blob files
-
-## What to inspect
-
-### 1. Runtime records
-
-Inspect the NDJSON output under:
-
-- `.indusagi/observe-smoke/<timestamp>/runtime-records/`
-
-Look for records like:
-
-- `recordType: "trace"`
-- `recordType: "span"`
-- `kind: "agent_run"`
-- `kind: "turn"`
-- `kind: "tool_call"`
-- `kind: "model_generation"`
-
-### 2. Control-plane snapshot
-
-Open:
-
-- `.indusagi/observe-smoke/<timestamp>/control-plane.json`
-
-Look for:
-
-- sessions
-- traces
-- prompts
-- datasets
-- experiments
-- experimentRuns
-- scores
-- artifacts
-
-### 3. Dashboard
-
-When the harness prints a URL like:
-
-- `http://127.0.0.1:4319/api/observe/ui`
-
-open it in your browser.
-
-You should see:
-
-- runtime stats
-- latest traces
-- artifacts
-- API endpoint map
-
-### 4. Query endpoints
-
-You can also query the server directly:
-
-```bash
-curl http://127.0.0.1:4319/api/observe/health
-curl http://127.0.0.1:4319/api/observe/stats
-curl "http://127.0.0.1:4319/api/observe/records?limit=20"
-curl http://127.0.0.1:4319/api/observe/control-plane/traces
-curl http://127.0.0.1:4319/api/observe/control-plane/artifacts
-```
-
-Adjust the port to whatever the smoke harness prints.
-
-## What counts as a pass
-
-A good run should produce all of the following:
-
-- at least one trace record
-- at least one model generation span
-- at least one tool call span
-- a persisted control-plane trace
-- at least one artifact
-- a working dashboard URL
-- no leaked fake secrets in exported payloads
-
-## Manual deep-checks
-
-After `npm run observe:smoke:serve`, manually verify:
-
-### Tool path
-
-The run should show tool activity for:
-
-- `read package.json`
-- `ls app`
-
-### Privacy path
-
-Search the exported NDJSON for raw secrets. The harness enables privacy controls, so sensitive-looking values should be redacted or omitted before export.
-
-### Persistence path
-
-Stop the process and confirm that:
-
-- runtime files remain on disk
-- `control-plane.json` remains readable
-- artifact blob files remain in `artifacts/`
-
-## Current limitation
-
-This harness tests observability **in the `indusagi-screth` repo context** using the local sibling `indusvx-screth` runtime surfaces, but it does **not** yet replace every `indusagi/*` import across the full `indusagi-screth` app.
-
-That is intentional: it gives a working, low-risk validation path first.
-
-## Next integration step after smoke success
-
-If the smoke harness looks good, the next deeper step is to wire Observe directly into `indusagi-screth/app/runtime/sdk.ts` so the main coding-agent runtime itself passes a real Observe runtime into the created `Agent` instance.