@kitlangton/motel 0.2.4 → 0.2.6

package/AGENTS.md

@@ -31,6 +31,18 @@
 - Effect LSP diagnostics over the whole project: `bunx effect-language-service diagnostics --project tsconfig.json --format text`
 - Effect LSP interactive setup wizard: `bunx effect-language-service setup`
 
+## Release Strategy
+- npm package: `@kitlangton/motel`
+- Current published npm `latest`: `0.2.4` (`npm view @kitlangton/motel dist-tags --json`)
+- Tags are versioned as `vX.Y.Z` (`git tag --sort=-version:refname` shows `v0.2.4`, `v0.2.3`, ...)
+- Publishing is handled by GitHub Actions in `.github/workflows/publish.yml`, not by local manual `npm publish`
+- The publish workflow triggers on `git push` of tags matching `v*` or via manual `workflow_dispatch`
+- The workflow runs `bun install --frozen-lockfile`, `bun run typecheck`, `bun run test`, then `npm publish --provenance`
+- `npm publish` runs `prepublishOnly`, which builds the web UI via `bun run web:build`
+- Before tagging a release, make sure the committed `package.json` version matches the intended git tag exactly
+- Preferred release flow: update `package.json` version, commit the release changes, create tag `v<package.json version>`, push the commit and tag, then verify the GitHub Actions publish and npm dist-tags
+- Do not create or push release tags from a dirty worktree with unrelated uncommitted changes; ask before including unrelated edits in a release
+
 ## Effect LSP
 The repo is wired up with `@effect/language-service` as a `tsconfig.json` `plugins` entry. Editors that pick up the TypeScript workspace plugin (Zed, VSCode, Cursor, NVim via vtsls) will surface Effect-specific diagnostics, quick fixes, and refactors inline. In Zed this requires selecting the workspace TypeScript version — it does so automatically when `node_modules/typescript` is present.
 
@@ -82,11 +94,11 @@ The repo is wired up with `@effect/language-service` as a `tsconfig.json` `plugi
 - `src/runtime.ts` wires the Effect beta runtime and OTEL trace + log exporters.
 - `src/localServer.ts` starts the local Bun OTLP/query server.
 - `src/httpApi.ts` defines the typed Effect HttpApi surface and OpenAPI spec for the local server.
+- `src/httpListPolicy.ts` owns pure HTTP list/search parameter decoding, bounds, cursors, and pagination metadata shaping.
 - `src/server.ts` runs the local server without the TUI.
 - `src/instructions.ts` contains the copied setup instructions for other Effect apps.
-- `src/services/TelemetryStore.ts` persists traces and logs in SQLite and exposes indexed queries.
-- `src/services/TraceQueryService.ts` reads traces from the local store.
-- `src/services/LogQueryService.ts` reads logs from the local store.
+- `src/services/TelemetryStore.ts` persists traces and logs in SQLite and exposes indexed queries through writer and read-only service identifiers.
+- `src/services/TelemetryQuery.ts` proxies read-only store calls to `src/services/telemetryQueryWorker.ts`, keeping synchronous Bun SQLite queries off the HTTP event loop.
 - `src/config.ts` is the source of truth for ports and env-driven OTEL settings.
 - `web/` is a Vite + React SPA for the browser-based UI (Tailwind CSS, `@effect/atom-react`, `AtomHttpApi`).
 - `web/src/api.ts` creates the typed `AtomHttpApi.Service` client from `src/httpApi.ts`.
@@ -95,15 +107,14 @@ The repo is wired up with `@effect/language-service` as a `tsconfig.json` `plugi
 - The server in `src/localServer.ts` serves `web/dist/` as static files with SPA fallback for non-API routes.
 
 ## Tests
-- `bun test` runs the suite. Three kinds of tests live in the repo:
+- `bun run test` runs the suite. Three kinds of tests live in the repo:
   - `src/telemetry.test.ts` exercises the SQLite TelemetryStore with
     OTLP payloads end-to-end.
   - `src/ui/waterfallNav.test.ts` unit-tests the pure collapse/expand
     resolver (no UI).
   - `src/ui/*.repro.test.ts` drive the real TUI under `tuistory` to
-    reproduce regressions; each has a sibling `*.repro.seed.ts` that
-    seeds a deterministic trace into SQLite in a child process. These
-    are auto-skipped when `tuistory` isn't installed.
+    reproduce regressions, using deterministic traces seeded into SQLite
+    by child processes. They become no-op passes when `tuistory` is absent.
 
 ## Effect Observability Guidance
 - Inspect the target repo’s existing Effect runtime and observability wiring before adding anything new.
@@ -128,12 +139,16 @@ The repo is wired up with `@effect/language-service` as a `tsconfig.json` `plugi
 - `MOTEL_OTEL_EXPORTER_URL`: defaults to `http://127.0.0.1:27686/v1/traces`
 - `MOTEL_OTEL_LOGS_EXPORTER_URL`: defaults to `http://127.0.0.1:27686/v1/logs`
 - `MOTEL_OTEL_QUERY_URL`: defaults to `http://127.0.0.1:27686`
-- `MOTEL_OTEL_DB_PATH`: defaults to `.motel-data/telemetry.sqlite`
+- `MOTEL_OTEL_DB_PATH`: defaults to `${XDG_STATE_HOME:-~/.local/state}/motel/telemetry.sqlite` (one shared DB per machine; daemon log + lock + instance registry live in the same directory)
+- `MOTEL_RUNTIME_DIR`: overrides the daemon log, lock, and instance-registry directory (primarily for isolated tests and custom managed instances)
 - `MOTEL_OTEL_TRACE_LOOKBACK_MINUTES`: defaults to `1440` (24h)
 - `MOTEL_OTEL_TRACE_LIMIT`: defaults to `100`
 - `MOTEL_OTEL_LOG_LIMIT`: defaults to `80`
 - `MOTEL_OTEL_RETENTION_HOURS`: defaults to `168` (7d)
 - `MOTEL_OTEL_MAX_DB_SIZE_MB`: defaults to `1024` (size-based retention cap)
+- `MOTEL_OTEL_RETENTION_TRACE_BATCH`: defaults to `100` completed traces per cleanup pass
+- `MOTEL_OTEL_RETENTION_LOG_BATCH`: defaults to `5000` logs per cleanup pass
+- `MOTEL_OTEL_RETENTION_INTERVAL_SECONDS`: defaults to `10`
 
 ## TUI Keys
 - `?`: toggle shortcut help

package/README.md

@@ -62,8 +62,19 @@ http://127.0.0.1:27686/v1/traces
 http://127.0.0.1:27686/v1/logs
 ```
 
-Motel keeps everything in a local SQLite database at
-`.motel-data/telemetry.sqlite`. No Docker, no cloud account.
+Motel keeps everything in a machine-global local SQLite database at
+`${XDG_STATE_HOME:-~/.local/state}/motel/telemetry.sqlite`. One managed
+daemon is shared across local projects. No Docker, no cloud account.
+
+The store retains seven days of telemetry by default and targets a 1 GB
+active-data ceiling using bounded background batches. Recent data is preserved while
+the oldest completed traces and logs are removed first. Configure the policy
+with `MOTEL_OTEL_RETENTION_HOURS`, `MOTEL_OTEL_MAX_DB_SIZE_MB`,
+`MOTEL_OTEL_RETENTION_TRACE_BATCH`, `MOTEL_OTEL_RETENTION_LOG_BATCH`, and
+`MOTEL_OTEL_RETENTION_INTERVAL_SECONDS`. Existing databases created without
+incremental auto-vacuum are never silently rewritten at startup; deleted pages
+are reused, but shrinking such a historical file requires an explicit offline
+SQLite `VACUUM` chosen by the user.
 
 ## How agents connect
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kitlangton/motel",
-  "version": "0.2.4",
+  "version": "0.2.6",
   "description": "A local OpenTelemetry ingest + TUI viewer for development, backed by SQLite.",
   "type": "module",
   "license": "MIT",
@@ -27,14 +27,22 @@
     "web"
   ],
   "engines": {
-    "bun": ">=1.1.0"
+    "bun": ">=1.3.0"
   },
   "publishConfig": {
     "access": "public"
   },
+  "overrides": {
+    "@grpc/grpc-js": "1.14.4",
+    "protobufjs": "7.6.4",
+    "shell-quote": "1.9.0",
+    "ws": "8.21.0"
+  },
   "files": [
     "src",
     "web/dist",
+    "skills/motel-debug/SKILL.md",
+    "skills/motel-debug/references/effect.md",
     "LICENSE",
     "README.md",
     "AGENTS.md"
@@ -68,30 +76,38 @@
     "web:dev": "bun run --cwd web dev",
     "web:build": "bun run --cwd web build",
     "story:chat": "bun run src/storybook/aiChatStory.tsx",
+    "bench:motel-cold-start": "bun run scripts/bench-motel-cold-start.ts",
+    "bench:ingest-logs": "bun run scripts/bench-ingest-logs.ts",
+    "bench:ingest-traces": "bun run scripts/bench-ingest-traces.ts",
+    "bench:search-spans": "bun run scripts/bench-search-spans.ts",
+    "release:validate": "bun run scripts/validate-release.ts",
     "typecheck": "tsc --noEmit",
+    "web:typecheck": "tsc --noEmit -p web/tsconfig.json",
     "prepublishOnly": "bun run web:build"
   },
   "devDependencies": {
-    "@effect/language-service": "^0.85.1",
-    "@types/bun": "^1.3.12",
-    "@types/react": "^19.2.14",
-    "typescript": "^6.0.2"
+    "@effect/language-service": "^0.86.2",
+    "@types/bun": "^1.3.14",
+    "@types/react": "^19.2.17",
+    "typescript": "^6.0.3"
   },
   "dependencies": {
-    "@effect/atom-react": "^4.0.0-beta.49",
-    "@effect/opentelemetry": "^4.0.0-beta.49",
-    "@effect/platform-bun": "^4.0.0-beta.50",
+    "@effect/atom-react": "4.0.0-beta.90",
+    "@effect/opentelemetry": "4.0.0-beta.90",
+    "@effect/platform-bun": "4.0.0-beta.90",
     "@opentelemetry/api": "^1.9.0",
-    "@opentelemetry/exporter-logs-otlp-http": "^0.214.0",
-    "@opentelemetry/exporter-trace-otlp-http": "^0.211.0",
-    "@opentelemetry/sdk-logs": "^0.214.0",
-    "@opentelemetry/sdk-node": "^0.214.0",
-    "@opentelemetry/sdk-trace-base": "^2.5.0",
-    "@opentelemetry/sdk-trace-node": "^2.6.1",
-    "@opentui/core": "^0.1.99",
-    "@opentui/react": "^0.1.99",
-    "effect": "^4.0.0-beta.49",
-    "react": "^19.2.5",
+    "@opentelemetry/exporter-logs-otlp-http": "^0.219.0",
+    "@opentelemetry/exporter-trace-otlp-http": "^0.219.0",
+    "@opentelemetry/otlp-transformer": "0.214.0",
+    "@opentelemetry/sdk-logs": "^0.219.0",
+    "@opentelemetry/sdk-node": "^0.219.0",
+    "@opentelemetry/sdk-trace-base": "^2.8.0",
+    "@opentelemetry/sdk-trace-node": "^2.8.0",
+    "@opentui/core": "0.4.2",
+    "@opentui/react": "0.4.2",
+    "effect": "4.0.0-beta.90",
+    "protobufjs": "7.6.4",
+    "react": "^19.2.7",
     "scheduler": "^0.27.0"
   }
 }

package/skills/motel-debug/SKILL.md

@@ -0,0 +1,203 @@
+---
+name: motel-debug
+description: Debug applications with motel, a local OpenTelemetry ingest and query server. Use when the user wants runtime-evidence debugging with traces or logs, wants temporary debug instrumentation that can be removed later, or needs a repo wired to send OTLP/HTTP telemetry to a local motel server. If the target repo uses Effect or @effect/*, also read references/effect.md.
+---
+
+# Motel Debug
+
+You are in **debug mode**. Debug with runtime evidence, not guesswork.
+
+Agents guess based on code alone. You need actual runtime data. Motel is the local OpenTelemetry server that collects traces and logs — use it as your evidence loop.
+
+Default local server details:
+
+- Base URL: `http://127.0.0.1:27686`
+- OTLP traces: `POST /v1/traces`
+- OTLP logs: `POST /v1/logs`
+- Query API: `GET /api/*`
+- OpenAPI: `GET /openapi.json`
+- Header: `Content-Type: application/json`
+- Auth: none by default
+
+If the user provides a different motel URL, use that instead of the default.
+
+## Workflow
+
+### 1. Verify motel is running — and start it if not
+
+Check `GET /api/health`. If it returns 200, continue.
+
+If it fails (connection refused, timeout, non-200), motel isn't running.
+Start it as a background daemon — **do not** launch the TUI, which is
+interactive and will block your shell:
+
+```bash
+motel start
+```
+
+`motel start` ensures the machine-global managed daemon is running, writes
+runtime files under `${XDG_STATE_HOME:-~/.local/state}/motel/`, and returns a
+JSON status blob. It is idempotent and shared across local projects. If motel isn't on `PATH`, fall
+back to `bunx @kitlangton/motel start`.
+
+After starting, re-check `GET /api/health` (may take 1–2s to become
+ready). If it still fails, read `${XDG_STATE_HOME:-~/.local/state}/motel/daemon.log` for the error
+and surface it to the user.
+
+Other lifecycle commands, for reference:
+
+```bash
+motel status   # JSON status (running? pid? originating workdir?)
+motel stop     # stop the shared managed daemon for all local projects
+```
+
+Discover reporting services with `GET /api/services` when needed.
+
+### 2. Generate hypotheses
+
+Before touching any code, generate **3-5 specific hypotheses** about why the bug occurs. Be precise — "the cache key doesn't include the user ID" is better than "something is wrong with caching."
+
+### 3. Instrument with tagged debug blocks
+
+Add the minimum instrumentation needed to confirm or reject **all** hypotheses in parallel. Every debug block must:
+
+- Be wrapped in `#region motel debug` / `#endregion motel debug` markers
+- Include a `debug.hypothesis` attribute linking it to a specific hypothesis
+- Use whatever tracing/logging mechanism the codebase already has (spans, structured logs, annotations — not raw `fetch` calls)
+
+Tag every piece of debug instrumentation with structured attributes so you can query it later. Reuse these keys:
+
+| Key | Purpose |
+|-----|---------|
+| `debug.session` | Groups all instrumentation for this debug session |
+| `debug.hypothesis` | Links to a specific hypothesis (e.g. `"cache-miss"`, `"A"`) |
+| `debug.step` | Position in the flow (e.g. `"entry"`, `"before-write"`, `"after-read"`) |
+| `debug.label` | Human-readable description of what this point captures |
+
+Choose log placements based on your hypotheses:
+
+- Function entry with parameters
+- Function exit with return values
+- Values before and after critical operations
+- Branch execution paths (which if/else ran)
+- State mutations and intermediate values
+- Suspected error or edge-case values
+
+Guidelines:
+
+- At least 1 instrumentation point is required; never skip instrumentation
+- Do not exceed 10 — if you think you need more, narrow your hypotheses
+- Typical range is 2-6
+
+### 4. Reproduce the issue
+
+- If a failing test exists, run it directly
+- If reproduction is straightforward (CLI command, curl, simple script), write and run it yourself
+- Otherwise, ask the user to reproduce — provide clear numbered steps and remind them to restart if needed
+- Once a reproduction pathway is established, reuse it for all subsequent iterations
+
+### 5. Analyze evidence
+
+Query motel for the debug instrumentation:
+
+```bash
+curl "http://127.0.0.1:27686/api/spans/search?service=<service>&attr.debug.hypothesis=<id>"
+curl "http://127.0.0.1:27686/api/logs/search?service=<service>&attr.debug.session=<session>"
+curl "http://127.0.0.1:27686/api/traces/search?service=<service>&attr.debug.hypothesis=<id>"
+```
+
+For each hypothesis, evaluate: **CONFIRMED**, **REJECTED**, or **INCONCLUSIVE** — cite specific spans, logs, or attribute values as evidence.
+
+### 6. Fix only with evidence
+
+Do **not** fix without runtime evidence. When you fix:
+
+- Keep all debug instrumentation in place — do not remove it yet
+- Make the fix as small and targeted as possible
+- Reuse existing architecture and patterns; do not overengineer
+
+### 7. Verify the fix
+
+Reproduce the issue again with instrumentation still active. Compare before/after evidence:
+
+- Cite specific log lines or span attributes that prove the fix works
+- If the fix failed: **revert code changes from rejected hypotheses** (do not let speculative fixes accumulate), generate new hypotheses from different subsystems, add more instrumentation, and iterate
+- Iteration is expected. Taking longer with more data yields better fixes.
+
+### 8. Clean up
+
+Only after the fix is verified **and** the user confirms there are no remaining issues:
+
+- Run the cleanup script or remove blocks manually (see Cleanup section below)
+- Run `git diff` to confirm only the intentional fix remains
+
+## Instrumentation Rules
+
+Wrap every temporary debug block in these exact markers:
+
+```ts
+// #region motel debug
+// temporary debug instrumentation
+// #endregion motel debug
+```
+
+Use whatever the codebase already provides for tracing and logging. The markers are language-comment wrappers — adapt the comment syntax for non-JS/TS files (e.g. `# #region motel debug` for Python).
+
+**Do not:**
+- Log secrets, tokens, passwords, or raw PII
+- Remove instrumentation before post-fix verification succeeds
+- Use `setTimeout`, `sleep`, or artificial delays as a "fix"
+- Let code changes from rejected hypotheses accumulate — revert them
+
+## Query Patterns
+
+Two filter prefixes for attribute search:
+
+| Prefix | Match type | Example |
+|--------|-----------|---------|
+| `attr.<key>=<value>` | Exact match | `attr.debug.hypothesis=cache-miss` |
+| `attrContains.<key>=<substring>` | Case-insensitive substring | `attrContains.ai.prompt.messages=hello world` |
+
+```bash
+curl http://127.0.0.1:27686/api/health
+curl http://127.0.0.1:27686/api/services
+
+# Trace search
+curl "http://127.0.0.1:27686/api/traces/search?service=<service>&operation=<text>&attr.debug.session=<session>"
+
+# Span search (supports traceId to scope to one trace)
+curl "http://127.0.0.1:27686/api/spans/search?service=<service>&traceId=<trace-id>&attr.debug.hypothesis=<id>"
+curl "http://127.0.0.1:27686/api/spans/search?service=<service>&attrContains.ai.prompt.messages=<phrase>"
+
+# Log search (supports severity filter, case-insensitive body search)
+curl "http://127.0.0.1:27686/api/logs/search?service=<service>&severity=ERROR&body=<text>"
+curl "http://127.0.0.1:27686/api/logs/search?service=<service>&attrContains.debug.label=<substring>"
+
+# AI call search (compact summaries with previews)
+curl "http://127.0.0.1:27686/api/ai/calls?model=gpt-5.4&sessionId=<session>"
+curl "http://127.0.0.1:27686/api/ai/calls?text=<phrase>&status=error"
+
+# AI call detail (full prompt/response payloads)
+curl "http://127.0.0.1:27686/api/ai/calls/<span-id>"
+
+# AI stats
+curl "http://127.0.0.1:27686/api/ai/stats?groupBy=model&agg=total_input_tokens"
+
+curl http://127.0.0.1:27686/openapi.json
+```
+
+List and search responses include `meta.nextCursor` when more data is available.
+
+Motel gives you trace-correlated data — you can see which span a debug log belongs to, the parent operation, timing, and the full trace tree. Use `GET /api/traces/<trace-id>/spans` and `GET /api/spans/<span-id>/logs` to navigate the correlation.
+
+For AI/LLM calls, use `/api/ai/calls` for compact searchable summaries (with prompt/response previews and token usage), and `/api/ai/calls/<span-id>` for full payloads.
+
+## Effect
+
+If the target repo uses Effect, read `references/effect.md` before changing runtime wiring or adding instrumentation.
+
+## Cleanup
+
+Use the bundled script at `scripts/clear-motel-debug.ts` when you want deterministic cleanup. It removes every block between `#region motel debug` and `#endregion motel debug` in JS/TS files and fails on unmatched markers.
+
+If you cannot run the script, delete every marked block manually and then grep for `#region motel debug` to confirm none remain.

package/skills/motel-debug/references/effect.md

@@ -0,0 +1,38 @@
+# Effect Notes
+
+Apply these only when the target repo already uses Effect or `@effect/*`.
+
+## Runtime
+
+- Inspect the existing runtime and observability wiring before adding anything new.
+- Prefer the repo's existing Effect-native observability APIs if they already exist.
+- If `effect/unstable/observability` is already the best fit, prefer it over adding new OTEL packages.
+- Merge telemetry into the main runtime once, not per feature or per request path.
+
+## Instrumentation
+
+- Prefer `Effect.fn("...")` for meaningful workflow spans.
+- Add a few child spans around boundaries that are likely to fail or add latency.
+- Emit `Effect.logInfo`, `Effect.logWarning`, and `Effect.logError` with structured fields.
+- Put searchable values in annotations/attributes, not only in the free-form log body.
+- Reuse stable debug keys such as `debug.session`, `debug.hypothesis`, `debug.step`, and `debug.label`.
+
+## Debug Blocks
+
+Wrap temporary debug-only Effect instrumentation in removable markers.
+
+```ts
+// #region motel debug
+const program = Effect.fn("feature/doThing")(function*() {
+	yield* Effect.logInfo("entered doThing", {
+		debug: {
+			session: "abc123",
+			hypothesis: "cache-miss",
+			step: "entry",
+		},
+	})
+})
+// #endregion motel debug
+```
+
+Keep those blocks until the fix is verified, then remove them with the cleanup script.

package/src/App.tsx

@@ -7,22 +7,21 @@ import { Divider, FooterHints, HelpModal, PlainLine, SplitDivider, TextLine } fr
 import { useAppLayout } from "./ui/app/useAppLayout.ts"
 import { useTraceScreenData } from "./ui/app/useTraceScreenData.ts"
 import { TraceWorkspace } from "./ui/app/TraceWorkspace.tsx"
+import { startupBenchMark } from "./startupBench.js"
 import {
 	type AttrFacetState,
 	attrPickerIndexAtom,
 	attrPickerInputAtom,
 	attrPickerModeAtom,
 	attrFacetStateAtom,
-	chatDetailChunkIdAtom,
-	chatDetailScrollOffsetAtom,
 	noticeAtom,
-	persistSelectedTheme,
 	selectedAttrIndexAtom,
-	selectedChatChunkIdAtom,
 	selectedThemeAtom,
 	waterfallFilterModeAtom,
 	waterfallFilterTextAtom,
-} from "./ui/state.ts"
+} from "./ui/atoms.ts"
+import { chatDetailChunkIdAtom, chatDetailScrollOffsetAtom, selectedChatChunkIdAtom } from "./ui/aiState.ts"
+import { persistSelectedTheme } from "./ui/persistence.ts"
 import type { ThemeName } from "./ui/theme.ts"
 import { applyTheme, colors, SEPARATOR, themeLabel } from "./ui/theme.ts"
 import { useKeyboardNav } from "./ui/useKeyboardNav.ts"
@@ -32,6 +31,8 @@ import { getVisibleSpans } from "./ui/waterfallModel.ts"
 
 const NOTICE_TIMEOUT_MS = 2500
 
+startupBenchMark("app_module_loaded")
+
 const buildHeaderModel = ({
 	headerFooterWidth,
 	selectedTraceService,
@@ -177,6 +178,7 @@ const AppOverlays = ({
 )
 
 export const App = () => {
+	startupBenchMark("app_render_started")
 	const { width, height } = useTerminalDimensions()
 	const [notice, setNotice] = useAtom(noticeAtom)
 	const [selectedTheme] = useAtom(selectedThemeAtom)
@@ -261,6 +263,10 @@ export const App = () => {
 		persistSelectedTheme(selectedTheme)
 	}, [selectedTheme])
 
+	useEffect(() => {
+		startupBenchMark("app_effects_committed")
+	}, [])
+
 	const { spanNavActive } = useKeyboardNav({
 		selectedTrace,
 		filteredTraces,
@@ -337,6 +343,7 @@ export const App = () => {
 	// above an empty column and leaves a visible stale sliver when
 	// toggling tab back and forth with the trace view.
 	const showSplit = isWideLayout && detailView !== "service-logs"
+	startupBenchMark("app_render_ready")
 
 	return (
 		<box width={width ?? 100} height={height ?? 24} flexGrow={1} flexDirection="column" backgroundColor={RGBA.fromHex(colors.screenBg)}>