@theokit/sdk 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1571 -0
- package/LICENSE +201 -0
- package/README.md +80 -0
- package/bin/theokit-migrate-config.mjs +269 -0
- package/bin/theokit-migrate-memory.mjs +116 -0
- package/dist/agent-builder.d.ts +52 -0
- package/dist/agent-factory.d.ts +39 -0
- package/dist/agent.d.ts +175 -0
- package/dist/batch.d.ts +11 -0
- package/dist/budget.d.ts +48 -0
- package/dist/cache.d.ts +74 -0
- package/dist/cron-1yxL3K2S.d.cts +221 -0
- package/dist/cron-BYVdYzob.d.ts +221 -0
- package/dist/cron.cjs +14655 -0
- package/dist/cron.cjs.map +1 -0
- package/dist/cron.d.cts +3 -0
- package/dist/cron.d.ts +71 -0
- package/dist/cron.js +14652 -0
- package/dist/cron.js.map +1 -0
- package/dist/define-tool.d.ts +35 -0
- package/dist/errors-CK8brCJ1.d.cts +448 -0
- package/dist/errors-CvAeEWgE.d.ts +448 -0
- package/dist/errors.cjs +255 -0
- package/dist/errors.cjs.map +1 -0
- package/dist/errors.d.cts +3 -0
- package/dist/errors.d.ts +356 -0
- package/dist/errors.js +238 -0
- package/dist/errors.js.map +1 -0
- package/dist/eval.cjs +14826 -0
- package/dist/eval.cjs.map +1 -0
- package/dist/eval.d.cts +35 -0
- package/dist/eval.d.ts +35 -0
- package/dist/eval.js +14821 -0
- package/dist/eval.js.map +1 -0
- package/dist/generate-object.d.ts +67 -0
- package/dist/handoff.d.ts +55 -0
- package/dist/index.cjs +17127 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +1878 -0
- package/dist/index.d.ts +24 -0
- package/dist/index.js +17095 -0
- package/dist/index.js.map +1 -0
- package/dist/internal/agent-loop/loop-types.d.ts +29 -0
- package/dist/internal/agent-loop/loop.d.ts +2 -0
- package/dist/internal/agent-loop/message-builders.d.ts +6 -0
- package/dist/internal/agent-loop/tool-dispatch.d.ts +4 -0
- package/dist/internal/agent-loop/usage-and-cost.d.ts +25 -0
- package/dist/internal/budget/calendar-window.d.ts +19 -0
- package/dist/internal/budget/compute-cost.d.ts +28 -0
- package/dist/internal/budget/enforcement.d.ts +32 -0
- package/dist/internal/budget/ledger.d.ts +25 -0
- package/dist/internal/budget/normalize-usage.d.ts +27 -0
- package/dist/internal/budget/pricing-registry.d.ts +36 -0
- package/dist/internal/budget/registry.d.ts +16 -0
- package/dist/internal/budget/usage-accumulator.d.ts +31 -0
- package/dist/internal/cache/cosine.d.ts +14 -0
- package/dist/internal/cache/embed-helper.d.ts +15 -0
- package/dist/internal/cache/key.d.ts +15 -0
- package/dist/internal/cache/lookup.d.ts +28 -0
- package/dist/internal/cache/store-handler.d.ts +24 -0
- package/dist/internal/cache/store-json.d.ts +48 -0
- package/dist/internal/cache/store.d.ts +54 -0
- package/dist/internal/cache/telemetry.d.ts +20 -0
- package/dist/internal/cache/ttl.d.ts +11 -0
- package/dist/internal/catalog/fixtures.d.ts +16 -0
- package/dist/internal/catalog/local-models.d.ts +24 -0
- package/dist/internal/cron/run-job.d.ts +1 -0
- package/dist/internal/cron/scheduler.d.ts +1 -0
- package/dist/internal/cron/store.d.ts +1 -0
- package/dist/internal/cron/validate.d.ts +1 -0
- package/dist/internal/env.d.ts +1 -0
- package/dist/internal/errors/mappers/anthropic.d.ts +30 -0
- package/dist/internal/errors/mappers/bedrock.d.ts +16 -0
- package/dist/internal/errors/mappers/ollama.d.ts +41 -0
- package/dist/internal/errors/mappers/openai-compatible.d.ts +25 -0
- package/dist/internal/errors/mappers/shared.d.ts +10 -0
- package/dist/internal/errors/mappers/vertex.d.ts +15 -0
- package/dist/internal/eval/aggregate.d.ts +9 -0
- package/dist/internal/eval/dataset-iter.d.ts +9 -0
- package/dist/internal/eval/runner.d.ts +9 -0
- package/dist/internal/eval/single-flight.d.ts +16 -0
- package/dist/internal/eval/telemetry.d.ts +23 -0
- package/dist/internal/fixture-mode.d.ts +16 -0
- package/dist/internal/handoff/dispatcher.d.ts +29 -0
- package/dist/internal/handoff/registry.d.ts +23 -0
- package/dist/internal/handoff/telemetry.d.ts +18 -0
- package/dist/internal/handoff/tool-injector.d.ts +34 -0
- package/dist/internal/http.d.ts +1 -0
- package/dist/internal/ids.d.ts +1 -0
- package/dist/internal/judge/judge-call.d.ts +35 -0
- package/dist/internal/judge/parse-verdict.d.ts +11 -0
- package/dist/internal/judge/types.d.ts +17 -0
- package/dist/internal/llm/anthropic-shared.d.ts +89 -0
- package/dist/internal/llm/anthropic.d.ts +9 -0
- package/dist/internal/llm/bedrock-anthropic.d.ts +36 -0
- package/dist/internal/llm/bedrock-token-cache.d.ts +18 -0
- package/dist/internal/llm/credential-pool-context.d.ts +11 -0
- package/dist/internal/llm/credential-pool-types.d.ts +22 -0
- package/dist/internal/llm/credential-pool.d.ts +18 -0
- package/dist/internal/llm/fallback-client.d.ts +1 -0
- package/dist/internal/llm/fault-injection.d.ts +50 -0
- package/dist/internal/llm/finish.d.ts +1 -0
- package/dist/internal/llm/model-identifier.d.ts +24 -0
- package/dist/internal/llm/ollama-native.d.ts +27 -0
- package/dist/internal/llm/openai.d.ts +9 -0
- package/dist/internal/llm/pool-aware-client.d.ts +16 -0
- package/dist/internal/llm/router.d.ts +17 -0
- package/dist/internal/llm/sse.d.ts +9 -0
- package/dist/internal/llm/stream-relay.d.ts +17 -0
- package/dist/internal/llm/types.d.ts +70 -0
- package/dist/internal/llm/vertex-anthropic.d.ts +40 -0
- package/dist/internal/llm/vertex-auth.d.ts +30 -0
- package/dist/internal/llm/vertex-gemini.d.ts +28 -0
- package/dist/internal/llm/vertex-router.d.ts +21 -0
- package/dist/internal/mcp/client.d.ts +16 -0
- package/dist/internal/memory/active-memory-cache.d.ts +10 -0
- package/dist/internal/memory/active-memory.d.ts +45 -0
- package/dist/internal/memory/adapters/catalog.d.ts +1 -0
- package/dist/internal/memory/adapters/deepinfra-embedding.d.ts +2 -0
- package/dist/internal/memory/adapters/mistral-embedding.d.ts +2 -0
- package/dist/internal/memory/adapters/ollama-embedding.d.ts +34 -0
- package/dist/internal/memory/adapters/openai-compatible.d.ts +23 -0
- package/dist/internal/memory/adapters/openai-embedding.d.ts +2 -0
- package/dist/internal/memory/adapters/openrouter-embedding.d.ts +2 -0
- package/dist/internal/memory/adapters/voyage-embedding.d.ts +2 -0
- package/dist/internal/memory/atomic-write.d.ts +7 -0
- package/dist/internal/memory/chunk-markdown.d.ts +2 -0
- package/dist/internal/memory/circuit-breaker.d.ts +22 -0
- package/dist/internal/memory/cwd-mutex.d.ts +1 -0
- package/dist/internal/memory/dreaming/diary.d.ts +4 -0
- package/dist/internal/memory/dreaming/phases.d.ts +15 -0
- package/dist/internal/memory/dreaming/run.d.ts +10 -0
- package/dist/internal/memory/embedding-adapter.d.ts +42 -0
- package/dist/internal/memory/embedding-cache.d.ts +1 -0
- package/dist/internal/memory/index-db.d.ts +10 -0
- package/dist/internal/memory/index-manager-dispatch.d.ts +23 -0
- package/dist/internal/memory/index-manager.d.ts +68 -0
- package/dist/internal/memory/index-schema.d.ts +21 -0
- package/dist/internal/memory/lance-index.d.ts +32 -0
- package/dist/internal/memory/lance-memory-adapter.d.ts +43 -0
- package/dist/internal/memory/markdown-store.d.ts +16 -0
- package/dist/internal/memory/memory-index.d.ts +52 -0
- package/dist/internal/memory/migrate-sqlite-to-lance.d.ts +15 -0
- package/dist/internal/memory/migration.d.ts +9 -0
- package/dist/internal/memory/reader.d.ts +8 -0
- package/dist/internal/memory/session-loader.d.ts +1 -0
- package/dist/internal/memory/session-summary-writer.d.ts +2 -0
- package/dist/internal/memory/sqlite-vec-loader.d.ts +3 -0
- package/dist/internal/memory/tools.d.ts +14 -0
- package/dist/internal/memory/transcript-store.d.ts +1 -0
- package/dist/internal/memory/types.d.ts +17 -0
- package/dist/internal/memory/vec-index.d.ts +28 -0
- package/dist/internal/memory/wiki-loader.d.ts +2 -0
- package/dist/internal/observability/tracer-loader.d.ts +20 -0
- package/dist/internal/persistence/atomic-write.d.ts +1 -0
- package/dist/internal/persistence/conversation-storage-fs.d.ts +37 -0
- package/dist/internal/persistence/conversation-storage-memory.d.ts +24 -0
- package/dist/internal/persistence/cwd-mutex.d.ts +1 -0
- package/dist/internal/persistence/file-lock.d.ts +14 -0
- package/dist/internal/persistence/fts5-sanitize.d.ts +16 -0
- package/dist/internal/persistence/markdown-config-loader.d.ts +35 -0
- package/dist/internal/persistence/paths.d.ts +19 -0
- package/dist/internal/persistence/persistence-schema.d.ts +17 -0
- package/dist/internal/persistence/schema-version.d.ts +13 -0
- package/dist/internal/persistence/sqlite-wal.d.ts +10 -0
- package/dist/internal/personality/context.d.ts +17 -0
- package/dist/internal/personality/registry.d.ts +17 -0
- package/dist/internal/personality/store.d.ts +27 -0
- package/dist/internal/personality/switch.d.ts +36 -0
- package/dist/internal/personality/types.d.ts +18 -0
- package/dist/internal/plugins/context.d.ts +31 -0
- package/dist/internal/plugins/manager.d.ts +37 -0
- package/dist/internal/plugins/types.d.ts +102 -0
- package/dist/internal/providers/builtin/anthropic.d.ts +2 -0
- package/dist/internal/providers/builtin/bedrock.d.ts +25 -0
- package/dist/internal/providers/builtin/gemini.d.ts +10 -0
- package/dist/internal/providers/builtin/index.d.ts +19 -0
- package/dist/internal/providers/builtin/llamacpp.d.ts +1 -0
- package/dist/internal/providers/builtin/lmstudio.d.ts +1 -0
- package/dist/internal/providers/builtin/ollama.d.ts +17 -0
- package/dist/internal/providers/builtin/openai.d.ts +2 -0
- package/dist/internal/providers/builtin/openrouter.d.ts +2 -0
- package/dist/internal/providers/builtin/vertex.d.ts +27 -0
- package/dist/internal/providers/discovery.d.ts +14 -0
- package/dist/internal/providers/index.d.ts +8 -0
- package/dist/internal/providers/registry.d.ts +12 -0
- package/dist/internal/providers/types.d.ts +27 -0
- package/dist/internal/runtime/abort-utils.d.ts +21 -0
- package/dist/internal/runtime/agent-factory-registry.d.ts +16 -0
- package/dist/internal/runtime/agent-registry-store.d.ts +61 -0
- package/dist/internal/runtime/agent-registry.d.ts +34 -0
- package/dist/internal/runtime/agent-session-store.d.ts +3 -0
- package/dist/internal/runtime/agent-session.d.ts +2 -0
- package/dist/internal/runtime/async-local-storage.d.ts +20 -0
- package/dist/internal/runtime/async-semaphore.d.ts +24 -0
- package/dist/internal/runtime/budget.d.ts +36 -0
- package/dist/internal/runtime/cloud-agent.d.ts +1 -0
- package/dist/internal/runtime/cloud-config-serializer.d.ts +3 -0
- package/dist/internal/runtime/cloud-payload-types.d.ts +56 -0
- package/dist/internal/runtime/cloud-run.d.ts +1 -0
- package/dist/internal/runtime/cloud-tool-parity.d.ts +1 -0
- package/dist/internal/runtime/context-aggregator.d.ts +26 -0
- package/dist/internal/runtime/context-discovery-runner.d.ts +27 -0
- package/dist/internal/runtime/context-discovery.d.ts +21 -0
- package/dist/internal/runtime/context-frontmatter.d.ts +16 -0
- package/dist/internal/runtime/context-import-resolver.d.ts +24 -0
- package/dist/internal/runtime/context-loaders.d.ts +42 -0
- package/dist/internal/runtime/context-manager.d.ts +11 -0
- package/dist/internal/runtime/context-mdc-parser.d.ts +24 -0
- package/dist/internal/runtime/default-model.d.ts +1 -0
- package/dist/internal/runtime/fixture-events.d.ts +12 -0
- package/dist/internal/runtime/fixture-responder.d.ts +1 -0
- package/dist/internal/runtime/fixture-run-base.d.ts +45 -0
- package/dist/internal/runtime/fixture-scripts.d.ts +21 -0
- package/dist/internal/runtime/fixture-types.d.ts +1 -0
- package/dist/internal/runtime/fork-agent.d.ts +15 -0
- package/dist/internal/runtime/hooks-executor.d.ts +35 -0
- package/dist/internal/runtime/hooks-frontmatter.d.ts +26 -0
- package/dist/internal/runtime/hooks-source.d.ts +22 -0
- package/dist/internal/runtime/live-agent-registry.d.ts +87 -0
- package/dist/internal/runtime/local-agent-bootstrap.d.ts +37 -0
- package/dist/internal/runtime/local-agent-dispatch.d.ts +57 -0
- package/dist/internal/runtime/local-agent-invalidate.d.ts +8 -0
- package/dist/internal/runtime/local-agent-memory-direct.d.ts +12 -0
- package/dist/internal/runtime/local-agent-memory-hooks.d.ts +41 -0
- package/dist/internal/runtime/local-agent-memory.d.ts +1 -0
- package/dist/internal/runtime/local-agent-personality-extensions.d.ts +19 -0
- package/dist/internal/runtime/local-agent-plugins.d.ts +13 -0
- package/dist/internal/runtime/local-agent-runtime-extensions.d.ts +13 -0
- package/dist/internal/runtime/local-agent-task-wrap.d.ts +11 -0
- package/dist/internal/runtime/local-agent.d.ts +1 -0
- package/dist/internal/runtime/local-run.d.ts +1 -0
- package/dist/internal/runtime/memory-store.d.ts +4 -0
- package/dist/internal/runtime/plugin-frontmatter.d.ts +17 -0
- package/dist/internal/runtime/plugins-manager.d.ts +1 -0
- package/dist/internal/runtime/post-run-lifecycle.d.ts +1 -0
- package/dist/internal/runtime/providers-manager.d.ts +1 -0
- package/dist/internal/runtime/real-cloud-run.d.ts +2 -0
- package/dist/internal/runtime/real-local-run.d.ts +2 -0
- package/dist/internal/runtime/run-registry.d.ts +5 -0
- package/dist/internal/runtime/run-until.d.ts +22 -0
- package/dist/internal/runtime/shell-tool.d.ts +7 -0
- package/dist/internal/runtime/skill-frontmatter.d.ts +1 -0
- package/dist/internal/runtime/skills-manager.d.ts +1 -0
- package/dist/internal/runtime/spawn-collect.d.ts +8 -0
- package/dist/internal/runtime/subagents-loader.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/escape.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/local-assembly.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/pipeline.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/providers/active-memory-provider.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/providers/base-provider.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/providers/context-provider.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/providers/memory-provider.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/providers/skills-provider.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/safe-call.d.ts +1 -0
- package/dist/internal/runtime/system-prompt/types.d.ts +5 -0
- package/dist/internal/runtime/system-prompt.d.ts +1 -0
- package/dist/internal/runtime/validate-agent-options.d.ts +1 -0
- package/dist/internal/runtime/workspace-dir.d.ts +9 -0
- package/dist/internal/runtime/yaml-frontmatter.d.ts +20 -0
- package/dist/internal/scorers/llm-judge.d.ts +24 -0
- package/dist/internal/security/index.d.ts +11 -0
- package/dist/internal/security/path-guard.d.ts +56 -0
- package/dist/internal/security/redact.d.ts +21 -0
- package/dist/internal/structured-output-helpers.d.ts +54 -0
- package/dist/internal/task/registry.d.ts +61 -0
- package/dist/internal/task/ring-buffer.d.ts +10 -0
- package/dist/internal/task/store.d.ts +59 -0
- package/dist/internal/task/subscribe.d.ts +15 -0
- package/dist/internal/task/telemetry.d.ts +27 -0
- package/dist/internal/telemetry/adapter-registry.d.ts +2 -0
- package/dist/internal/telemetry/adapters/langfuse.d.ts +2 -0
- package/dist/internal/telemetry/adapters/posthog.d.ts +2 -0
- package/dist/internal/telemetry/adapters/sentry.d.ts +2 -0
- package/dist/internal/telemetry/safe-require.d.ts +1 -0
- package/dist/internal/telemetry/tracer.d.ts +18 -0
- package/dist/internal/tool-dispatch/repair-middleware.d.ts +34 -0
- package/dist/internal/tool-dispatch/strip-think.d.ts +22 -0
- package/dist/internal/tool-registry/personality-filter.d.ts +37 -0
- package/dist/internal/workflow/ctx.d.ts +19 -0
- package/dist/internal/workflow/error-shape.d.ts +7 -0
- package/dist/internal/workflow/executor.d.ts +15 -0
- package/dist/internal/workflow/index.d.ts +12 -0
- package/dist/internal/workflow/retry-policy.d.ts +14 -0
- package/dist/internal/workflow/run-id.d.ts +9 -0
- package/dist/internal/workflow/single-flight.d.ts +18 -0
- package/dist/internal/workflow/snapshot-store.d.ts +23 -0
- package/dist/internal/workflow/step-agent.d.ts +12 -0
- package/dist/internal/workflow/step-branch.d.ts +10 -0
- package/dist/internal/workflow/step-dowhile.d.ts +8 -0
- package/dist/internal/workflow/step-fn.d.ts +10 -0
- package/dist/internal/workflow/step-foreach.d.ts +11 -0
- package/dist/internal/workflow/step-parallel.d.ts +17 -0
- package/dist/internal/workflow/step-sleep.d.ts +7 -0
- package/dist/internal/workflow/telemetry.d.ts +23 -0
- package/dist/internal/zod/to-json-schema.d.ts +21 -0
- package/dist/memory-adapter-helpers.d.ts +28 -0
- package/dist/memory.d.ts +123 -0
- package/dist/migrate.d.ts +33 -0
- package/dist/path-safety.cjs +126 -0
- package/dist/path-safety.cjs.map +1 -0
- package/dist/path-safety.d.cts +15 -0
- package/dist/path-safety.d.ts +15 -0
- package/dist/path-safety.js +120 -0
- package/dist/path-safety.js.map +1 -0
- package/dist/run-DkCD5DeO.d.cts +2181 -0
- package/dist/run-DkCD5DeO.d.ts +2181 -0
- package/dist/scorers.d.ts +75 -0
- package/dist/security.d.ts +67 -0
- package/dist/stream-object.d.ts +74 -0
- package/dist/task-store.cjs +237 -0
- package/dist/task-store.cjs.map +1 -0
- package/dist/task-store.d.cts +8 -0
- package/dist/task-store.d.ts +8 -0
- package/dist/task-store.js +233 -0
- package/dist/task-store.js.map +1 -0
- package/dist/task.d.ts +87 -0
- package/dist/theokit.d.ts +84 -0
- package/dist/tools/_path-scope.d.cts +8 -0
- package/dist/tools/_path-scope.d.ts +8 -0
- package/dist/tools/_subprocess.d.cts +28 -0
- package/dist/tools/_subprocess.d.ts +28 -0
- package/dist/tools/git-diff.d.cts +22 -0
- package/dist/tools/git-diff.d.ts +22 -0
- package/dist/tools/index.d.cts +29 -0
- package/dist/tools/index.d.ts +29 -0
- package/dist/tools/list-dir.d.cts +26 -0
- package/dist/tools/list-dir.d.ts +26 -0
- package/dist/tools/read-file.d.cts +31 -0
- package/dist/tools/read-file.d.ts +31 -0
- package/dist/tools/run-vitest.d.cts +46 -0
- package/dist/tools/run-vitest.d.ts +46 -0
- package/dist/tools/search-text.d.cts +32 -0
- package/dist/tools/search-text.d.ts +32 -0
- package/dist/tools.cjs +690 -0
- package/dist/tools.cjs.map +1 -0
- package/dist/tools.js +683 -0
- package/dist/tools.js.map +1 -0
- package/dist/trajectory-helpers.d.ts +31 -0
- package/dist/types/agent.d.ts +771 -0
- package/dist/types/batch.d.ts +112 -0
- package/dist/types/budget.d.ts +88 -0
- package/dist/types/cache.d.ts +76 -0
- package/dist/types/context.d.ts +93 -0
- package/dist/types/conversation-storage.d.ts +76 -0
- package/dist/types/conversation.d.ts +90 -0
- package/dist/types/cron.d.ts +150 -0
- package/dist/types/eval.d.ts +132 -0
- package/dist/types/goal-events.d.ts +95 -0
- package/dist/types/handoff.d.ts +135 -0
- package/dist/types/index.d.ts +20 -0
- package/dist/types/mcp.d.ts +64 -0
- package/dist/types/memory-adapter.d.ts +175 -0
- package/dist/types/messages.d.ts +154 -0
- package/dist/types/providers.d.ts +102 -0
- package/dist/types/run.d.ts +215 -0
- package/dist/types/task.d.ts +131 -0
- package/dist/types/theokit.d.ts +61 -0
- package/dist/types/trajectory.d.ts +49 -0
- package/dist/types/updates.d.ts +148 -0
- package/dist/types/usage.d.ts +61 -0
- package/dist/types/workflow.d.ts +217 -0
- package/dist/workflow.cjs +2405 -0
- package/dist/workflow.cjs.map +1 -0
- package/dist/workflow.d.cts +97 -0
- package/dist/workflow.d.ts +97 -0
- package/dist/workflow.js +2398 -0
- package/dist/workflow.js.map +1 -0
- package/package.json +183 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,1571 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 1.5.0
|
|
4
|
+
|
|
5
|
+
### Changed
|
|
6
|
+
|
|
7
|
+
- **`publishConfig.provenance` removed (alinhado com política do monorepo).** Esta era a única `package.json` de 11 pacotes publicáveis com `provenance: true`; drift arquitetural — a flag prometia attestation criptográfica mas nenhum repo do monorepo tem release.yml com `id-token: write` permission para mintar OIDC token contra o npm registry. Resultado: publishes locais falhavam com `EUSAGE: Automatic provenance generation not supported for provider: null`. Decisão: alinhar intent à infra atual (10/11 outros pacotes não declaram provenance). **Follow-up estratégico:** adicionar release.yml com `id-token: write` em todos os repos (theokit-sdk + theokit + theokit-plugins + theo-ui) habilita provenance universal — escopo separado.
|
|
8
|
+
|
|
9
|
+
### Breaking Changes
|
|
10
|
+
|
|
11
|
+
- **`Workflow` and `Eval` moved out of the main barrel into dedicated sub-paths.** The migration is mechanical (rewrite the `from` string); no behavior changes. `@theokit/sdk` main barrel no longer exports:
|
|
12
|
+
- From workflow: `Workflow`, `WorkflowBuilder`, `agentStep`, `fn`, `WorkflowAlreadyRunningError`, `WorkflowCompensateNotImplementedError`, `WorkflowDuplicateStepIdError`, `WorkflowMaxIterationsExceededError`, `WorkflowNotSerializableError`, `WorkflowParallelError`, `WorkflowResumeStepNotFoundError`, `WorkflowSnapshotNotFoundError` — **import from `@theokit/sdk/workflow` instead**.
|
|
13
|
+
- From eval: `Eval`, `EvalAlreadyRunningError`, `Scorers` — **import from `@theokit/sdk/eval` instead**.
|
|
14
|
+
- From `types/*`: type aliases for workflow + eval (e.g., `EvalRun`, `Scorer`, `Score`, `EvalOptions`, `EvalAggregate`, `Step`, `FnStep`, etc.) no longer reach the main barrel via `types/index.ts`; surface only through the new sub-paths.
|
|
15
|
+
|
|
16
|
+
Rationale: Interface Segregation. The barrel exported 17+ feature areas, forcing consumers to pay the DTS cost of `Workflow`+`Eval` even if they only used `Agent`+`Memory`. Sub-paths reduce DTS surface and align with the existing pattern (`@theokit/sdk/cron`, `/tools`, `/path-safety`, `/task-store`, `/errors`).
|
|
17
|
+
|
|
18
|
+
**Migration:**
|
|
19
|
+
```ts
|
|
20
|
+
// Before
|
|
21
|
+
import { Workflow, Eval, Scorers } from "@theokit/sdk";
|
|
22
|
+
|
|
23
|
+
// After
|
|
24
|
+
import { Workflow } from "@theokit/sdk/workflow";
|
|
25
|
+
import { Eval, Scorers } from "@theokit/sdk/eval";
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
### Added
|
|
29
|
+
|
|
30
|
+
- `@theokit/sdk/workflow` sub-path entry (with full ESM + CJS conditions, `.d.ts` + `.d.cts` mirror for attw compliance).
|
|
31
|
+
- `@theokit/sdk/eval` sub-path entry (same shape; `Scorers` co-located here per locality of reference).
|
|
32
|
+
|
|
33
|
+
## 1.4.1 (workspace-only — NOT published to npm)
|
|
34
|
+
|
|
35
|
+
> **Drift note (2026-06-02):** versions 1.4.0 and 1.4.1 landed in workspace and were merged to develop, but never reached the `@latest` npm dist-tag. npm `@theokit/sdk@latest` remains at **1.3.0** (last shipped 2026-05-30). The 1.4.x patch chain will be consolidated into the next published release (1.5.0 or higher) — consumers who need the LanceDB wiring fix (1.4.0) or the zod v3/v4 universal converter (1.4.1) must install `@theokit/sdk@1.5.0-next.X` (when published on `next`) or wait for the consolidated `latest` cut. Drift root cause: 1.4.0 sub-paths extraction work changed the publish requirements (workspace `pnpm changeset version` chain was bumped but `pnpm changeset publish` was deferred while 1.5.0 sub-path API surface stabilized). All entries below reflect REAL code changes that DID land on develop.
|
|
36
|
+
|
|
37
|
+
### Patch Changes
|
|
38
|
+
|
|
39
|
+
- **`defineTool` now works on zod v3 + v4 (universal converter).** Before this patch, `defineTool({ inputSchema: z.object(...) })` failed at runtime with `z.toJSONSchema is not a function` whenever the consumer's resolved `zod` was v3 (which is the case for `theokit` and `dogfood-app` today — both pin `^3.25.0`). The SDK delegates conversion to the existing universal `internal/zod/to-json-schema.ts` adapter (feature-detect zod 4 native `toJSONSchema` → fallback to `zod-to-json-schema` peer lib). Caught end-to-end via Chrome MCP dogfood — `/api/tools`, `/api/admin/sdk-config`, `POST /api/chat` all 500'd before the fix; all 200 after.
|
|
40
|
+
- **`internal/zod/to-json-schema.ts` cross-version safety net:** when the SDK runs under a dev-server (Vite SSR), `createRequire("zod")` resolves to the SDK's OWN `node_modules/zod` (v4 in devDeps), while the schema was built by the consumer's zod v3 instance. Calling v4's `toJSONSchema(v3Schema)` throws. The native path now catches that error and falls through to `zod-to-json-schema` (which understands both v3 and v4 schemas). Mode toggled in cache so subsequent calls go directly to the working path.
|
|
41
|
+
- Added `zod-to-json-schema: "^3.24.0"` as optional `peerDependency` (already silently required by zod-3 consumers; now declared explicitly so `pnpm install` resolves it deterministically).
|
|
42
|
+
|
|
43
|
+
## 1.4.0 (workspace-only — NOT published to npm)
|
|
44
|
+
|
|
45
|
+
> See drift note at the top of the 1.4.1 section. Code landed; npm `@latest` still at 1.3.0.
|
|
46
|
+
|
|
47
|
+
### Minor Changes
|
|
48
|
+
|
|
49
|
+
- **`Memory.create({ index: { backend: "lance" } })` is now wired end-to-end.** The `LanceIndex` implementation existed since 2026-05-17 (ADR D43) but `IndexManager.open` did not dispatch — public API accepted `backend: "lance"` silently and always fell through to SQLite. Fix: factory dispatcher in `IndexManager.open` + new portable `MemoryIndex` interface + new `LanceMemoryAdapter` wrapper + `@lancedb/lancedb` declared as optional `peerDependency` (`^0.30.0`).
|
|
50
|
+
|
|
51
|
+
**Migration path:** consumer that wants Lance:
|
|
52
|
+
```bash
|
|
53
|
+
pnpm add @lancedb/lancedb apache-arrow@^18.1.0
|
|
54
|
+
```
|
|
55
|
+
```ts
|
|
56
|
+
await Memory.create({
|
|
57
|
+
index: { backend: "lance" },
|
|
58
|
+
embedding: { provider: "openai", apiKey: process.env.OPENAI_API_KEY },
|
|
59
|
+
});
|
|
60
|
+
```
|
|
61
|
+
Default keeps SQLite (zero added deps, zero breaking change vs 1.3.0).
|
|
62
|
+
|
|
63
|
+
**When to opt-in (benchmark evidence — `.claude/knowledge-base/benchmarks/memory-backends-2026-05-31.md`):**
|
|
64
|
+
- Lance wins **43x** ingest throughput at 100k facts (59849 ops/s vs SQLite-vec 1875 ops/s).
|
|
65
|
+
- Lance uses **65% less disk** at 100k (33.8 MB vs 93.5 MB).
|
|
66
|
+
- SQLite-vec recall p95 stays competitive up to 100k (~25 ms). Use Lance when ingest velocity or disk pressure matters; SQLite handles latency well below 1M facts.
|
|
67
|
+
|
|
68
|
+
**EC-1 hardening:** new `ConfigurationError({code:"invalid_memory_backend"})` for typo-protection — `backend: "lancedb"` (typo) now throws instead of silently falling back to SQLite. Same hardening for `lance_requires_embedding` and `lance_backend_unavailable` typed errors.
|
|
69
|
+
|
|
70
|
+
**Gotchas:**
|
|
71
|
+
- `@lancedb/lancedb` ships prebuilds for linux-x64-gnu, darwin-arm64, darwin-x64, win32-x64-msvc. Alpine/musl/ARM-Linux require `node-gyp` toolchain. SQLite default covers those cases.
|
|
72
|
+
- Bundlers (Next.js/Vite/webpack/rollup) must externalize `@lancedb/lancedb`:
|
|
73
|
+
- Next.js: `experimental.serverComponentsExternalPackages: ["@lancedb/lancedb"]`
|
|
74
|
+
- Vite: `optimizeDeps.exclude: ["@lancedb/lancedb"]` + `ssr.external: ["@lancedb/lancedb"]`
|
|
75
|
+
- webpack/rollup: add to `externals` array
|
|
76
|
+
|
|
77
|
+
Closes ADR D12 ("LanceDB deferred to v1.1") via fulfillment of D43.
|
|
78
|
+
|
|
79
|
+
- **EC-1/EC-8 fixes shipped atomically** (caught by the new integration test against real `@lancedb/lancedb@0.30.0`):
|
|
80
|
+
- `LanceIndex.search` now uses SQL string predicate with `escapeSqlValue()` (single-quote doubling) instead of object filter — Lance 0.30 only accepts SQL string in `.where()`, contrary to D43's original assumption.
|
|
81
|
+
- `LanceIndex.open` dim-mismatch detection now reads `schema().fields.type.listSize` (Apache Arrow `FixedSizeList` typeId=16 layout in Lance 0.30) — previously checked `fixedSize` which never matched.
|
|
82
|
+
|
|
83
|
+
### Patch Changes
|
|
84
|
+
|
|
85
|
+
- Refactored `RealLocalRun.executeAgentLoop` (complexity 11 → ≤10) via Extract Method: introduced `applyAgentLoopOutput` private helper that copies events/conversation/result/usage/cost/error onto the script. Behavior preserved byte-for-byte. (theokit-sdk-biome-cleanup)
|
|
86
|
+
- Removed redundant `// biome-ignore` directive from `internal/llm/fault-injection.ts` that no longer applied after the workspace enabled `javascript.parser.unsafeParameterDecoratorsEnabled`. (theokit-sdk-biome-cleanup)
|
|
87
|
+
- Extracted message-builder helpers (`buildSystemEvent`, `buildUserEvent`, `buildAssistantEvent`, `buildAssistantTurn`) from `internal/agent-loop/loop.ts` into a new sibling `message-builders.ts` to bring `loop.ts` back under the G8 file-size budget (400 LoC). Pure refactor — no behavior change. (theokit-sdk-biome-cleanup)
|
|
88
|
+
- Removed redundant `export` on `GraphSnapshot` interface (internal-only). (theokit-sdk-biome-cleanup)
|
|
89
|
+
- Added inline `// biome-ignore lint/correctness/useYield` on two intentional non-yielding async-generator mocks in `tests/internal/agent-loop/error-packaging.test.ts` (legitimate test seam — throws before yielding). (theokit-sdk-biome-cleanup)
|
|
90
|
+
- Vitest configuration: switched `pool` to `forks` (top-level) with `singleFork: false` so each test file runs in its own subprocess. This is the only reliable way to isolate `process.env.HOME` mutations across the discovery / context-import-resolver / personality test files, which were producing 5 flaky failures under parallel-package validate. Stack-keyed `process.env.HOME` save/restore added to `vitest.setup.ts` for additional safety. (theokit-sdk-biome-cleanup)
|
|
91
|
+
|
|
92
|
+
- Refactored `RealLocalRun.executeAgentLoop` (complexity 11 → ≤10) via Extract Method: introduced `applyAgentLoopOutput` private helper that copies events/conversation/result/usage/cost/error onto the script. Behavior preserved byte-for-byte. (theokit-sdk-biome-cleanup)
|
|
93
|
+
- Removed redundant `// biome-ignore` directive from `internal/llm/fault-injection.ts` that no longer applied after the workspace enabled `javascript.parser.unsafeParameterDecoratorsEnabled`. (theokit-sdk-biome-cleanup)
|
|
94
|
+
- Extracted message-builder helpers (`buildSystemEvent`, `buildUserEvent`, `buildAssistantEvent`, `buildAssistantTurn`) from `internal/agent-loop/loop.ts` into a new sibling `message-builders.ts` to bring `loop.ts` back under the G8 file-size budget (400 LoC). Pure refactor — no behavior change. (theokit-sdk-biome-cleanup)
|
|
95
|
+
- Removed redundant `export` on `GraphSnapshot` interface (internal-only). (theokit-sdk-biome-cleanup)
|
|
96
|
+
- Added inline `// biome-ignore lint/correctness/useYield` on two intentional non-yielding async-generator mocks in `tests/internal/agent-loop/error-packaging.test.ts` (legitimate test seam — throws before yielding). (theokit-sdk-biome-cleanup)
|
|
97
|
+
- Vitest configuration: switched `pool` to `forks` (top-level) with `singleFork: false` so each test file runs in its own subprocess. This is the only reliable way to isolate `process.env.HOME` mutations across the discovery / context-import-resolver / personality test files, which were producing 5 flaky failures under parallel-package validate. Stack-keyed `process.env.HOME` save/restore added to `vitest.setup.ts` for additional safety. (theokit-sdk-biome-cleanup)
|
|
98
|
+
|
|
99
|
+
## 1.3.0
|
|
100
|
+
|
|
101
|
+
### Minor Changes
|
|
102
|
+
|
|
103
|
+
- Fix Finding B: provider/transport errors no longer leak as `SDKAssistantMessage` content. They surface structured on `RunResult.error` (`{ message, code?, cause? }`).
|
|
104
|
+
|
|
105
|
+
**Background.** Previously, the agent loop's stream catch block (`internal/agent-loop/loop.ts`) and the runtime's `emitErrorEvent` (`internal/runtime/real-local-run.ts`) both pushed an `SDKAssistantMessage` carrying the error text. Downstream surfaces (notably `theokit`'s `streamAgentRun`) then yielded `{ type: 'message' }` events instead of `{ type: 'error' }`, hiding the failure from consumers' typed error handling and from chaos tests.
|
|
106
|
+
|
|
107
|
+
**What changed.**
|
|
108
|
+
|
|
109
|
+
- `AgentLoopOutput` now carries an optional `error?: AgentLoopErrorDetail` field.
|
|
110
|
+
- The loop catch path and the in-stream `{ type: "error" }` event both populate `ctx.error` via a single `registerLoopError(ctx, cause)` helper that enforces the set-once invariant (first-error-wins, ADR D3) and EC-1 typeof-guards `cause.code` so a non-string code never lands on the wire as the literal `"undefined"`.
|
|
111
|
+
- The abort path (`signal.aborted === true`) still emits `"[aborted]"` as an `SDKAssistantMessage` — that is a UX seam, not an error.
|
|
112
|
+
- `executeAgentLoop` copies `output.error` onto `script.errorDetail`, which `buildResult()` already surfaces as `RunResult.error` (set-once invariant preserved).
|
|
113
|
+
- `emitErrorEvent` (used by MCP-init / build-inputs / outer-catch paths) no longer pushes an assistant message — those errors flow exclusively via `script.errorDetail` → `RunResult.error`.
|
|
114
|
+
|
|
115
|
+
**Migration (EC-8).** If your code grepped for the error inside `for await (const msg of run.stream()) { if (msg.type === 'assistant' && /401|API error/.test(...)) }`, switch to `const result = await run.wait(); if (result.status === 'error') { result.error.message; result.error.code; }`. The abort case still arrives as an assistant message with content `"[aborted]"` — distinct from errors.
|
|
116
|
+
|
|
117
|
+
**Tests added.** `tests/internal/agent-loop/error-packaging.test.ts` (5 unit tests covering auth, transport, abort UX preservation, first-error-wins, happy-path sanity). `tests/runtime/error-packaging-e2e.test.ts` (2 E2E tests covering full `Agent.create → send → wait → result.error` pipeline with mocked 401 fetch and the EC-6 double-negative invariant).
|
|
118
|
+
|
|
119
|
+
## 1.2.0
|
|
120
|
+
|
|
121
|
+
### Minor Changes
|
|
122
|
+
|
|
123
|
+
- **D14 — Test fault injection via `THEOKIT_TEST_RESPONSE_OVERRIDE` env var.**
|
|
124
|
+
|
|
125
|
+
Adds a deterministic chaos-testing seam at the LLM transport layer. When `NODE_ENV=test` AND `THEOKIT_TEST_RESPONSE_OVERRIDE` is set to a JSON string of shape `{"status": number, "body": object | string}`, every provider client returned by `resolveProviderChain` short-circuits the real network call and synthesizes the configured response.
|
|
126
|
+
|
|
127
|
+
**Use cases (replaces flaky chaos patterns):**
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
# 429 rate-limit — deterministic, zero quota burn
|
|
131
|
+
export NODE_ENV=test
|
|
132
|
+
export THEOKIT_TEST_RESPONSE_OVERRIDE='{"status":429,"body":{"error":{"code":"rate_limit_exceeded","message":"Rate limit hit"}}}'
|
|
133
|
+
|
|
134
|
+
# 500 server error — for retry / circuit-breaker tests
|
|
135
|
+
export THEOKIT_TEST_RESPONSE_OVERRIDE='{"status":500,"body":{"error":{"code":"internal_error"}}}'
|
|
136
|
+
|
|
137
|
+
# 200 happy path — deterministic text for snapshot tests
|
|
138
|
+
export THEOKIT_TEST_RESPONSE_OVERRIDE='{"status":200,"body":{"choices":[{"message":{"content":"hello"}}]}}'
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
**Design (FAANG-grade fail-safe):**
|
|
142
|
+
|
|
143
|
+
- **Two-gate activation.** Both `NODE_ENV === "test"` AND a non-empty env var must be present. Production deployments are unaffected (cheap noop in the hot path).
|
|
144
|
+
- **Decorator pattern** (`FaultInjectingLlmClient`) wraps every resolved transport. Composes cleanly with `PoolAwareLlmClient` (credential pools D123-D133) and per-provider transports without modifying them.
|
|
145
|
+
- **Graceful degradation on malformed JSON** — one-shot stderr warn + fall-through to the real client. Never throws on bad config.
|
|
146
|
+
- **Error parity** — injected non-200 statuses go through the existing `mapOpenAICompatibleError` mapper, so the error class hierarchy (`RateLimitError`, `AuthenticationError`, `NetworkError`, `ConfigurationError`) is byte-equal to what the real provider would raise.
|
|
147
|
+
- **Transparent passthrough** when override is absent — the wrapper preserves `client.name` for telemetry and exposes `inner` so layered-transport assertions (router pool-wiring tests, telemetry inspectors) can walk one level deep.
|
|
148
|
+
|
|
149
|
+
**Tests:** 12 unit tests (`tests/llm-fault-injection.test.ts`) + 3 wiring tests (`tests/llm-fault-injection-router-wiring.test.ts`) — gate negative + active for each status class + idempotence + name preservation.
|
|
150
|
+
|
|
151
|
+
**Rejects the anti-patterns:** "50 parallel requests to force 429" (flaky + quota burn + cost overrun), `nock`/`msw` (violates stranger persona), conditional code in templates (Strategy pattern instead).
|
|
152
|
+
|
|
153
|
+
Inspired by Stripe test-mode + AWS SDK `AWS_SDK_LOAD_CONFIG=0`. Documented in dogfood-stranger Phase 13 (theokit plan `dogfood-fixes-and-coverage-expansion-plan.md` ADR D14).
|
|
154
|
+
|
|
155
|
+
## [Unreleased]
|
|
156
|
+
|
|
157
|
+
### Added — `THEOKIT_TEST_RESPONSE_OVERRIDE` env var (D14 fault injection)
|
|
158
|
+
|
|
159
|
+
Deterministic chaos-testing seam at the LLM transport layer. When both
|
|
160
|
+
`NODE_ENV=test` AND `THEOKIT_TEST_RESPONSE_OVERRIDE` are set to a JSON string
|
|
161
|
+
of shape `{"status": number, "body": object | string}`, every provider client
|
|
162
|
+
returned by `resolveProviderChain` short-circuits the real network call and
|
|
163
|
+
synthesizes the configured response.
|
|
164
|
+
|
|
165
|
+
Replaces flaky chaos patterns like "50 parallel requests to force 429" with
|
|
166
|
+
zero quota burn + zero network. Composes with credential pools (D123-D133)
|
|
167
|
+
and per-provider transports without modification (decorator pattern).
|
|
168
|
+
|
|
169
|
+
Use cases:
|
|
170
|
+
|
|
171
|
+
- Inject 429 for rate-limit handling tests
|
|
172
|
+
- Inject 5xx for retry / circuit-breaker tests
|
|
173
|
+
- Inject 200 with deterministic content for snapshot tests
|
|
174
|
+
|
|
175
|
+
See `docs.md` § "Test fault injection (v1.22+)" for the full contract,
|
|
176
|
+
supported status classes / body shapes, and graceful-degradation semantics.
|
|
177
|
+
|
|
178
|
+
Implementation:
|
|
179
|
+
|
|
180
|
+
- `src/internal/llm/fault-injection.ts` — `FaultInjectingLlmClient` decorator
|
|
181
|
+
- parser + activation gate + one-shot stderr warn on malformed JSON
|
|
182
|
+
- `src/internal/llm/router.ts` — wraps every resolved client (1-line wire)
|
|
183
|
+
- `tests/llm-fault-injection.test.ts` — 12 unit tests (gate negative + active
|
|
184
|
+
per status class + idempotence + name preservation)
|
|
185
|
+
- `tests/llm-fault-injection-router-wiring.test.ts` — 3 wiring tests proving
|
|
186
|
+
end-to-end the override reaches `Agent.send` via the router
|
|
187
|
+
|
|
188
|
+
### Fixed — `Agent.getOrCreate` no longer returns disposed cached agents
|
|
189
|
+
|
|
190
|
+
Pre-existing race: any caller that did `await agent.dispose()` followed by
|
|
191
|
+
`Agent.getOrCreate(sameId, opts)` received the DISPOSED instance from
|
|
192
|
+
`Agent.registry`; the subsequent `agent.send()` threw
|
|
193
|
+
`"Agent has been disposed"`. Surfaced as 9/48 failures in the telegram-pro
|
|
194
|
+
2026-05-28 dogfood (`Remember:`, `/recall`, `/tool uuid`, `/tool roll`,
|
|
195
|
+
`/personality coder/poet/none/ghost`, post-personality text).
|
|
196
|
+
|
|
197
|
+
Fix:
|
|
198
|
+
|
|
199
|
+
- `LiveAgentRegistry.forget(id)` — new internal helper that removes a cache
|
|
200
|
+
entry WITHOUT calling `dispose()` or `onEvict` (idempotent on unknown ids).
|
|
201
|
+
- `LocalAgent.dispose()` now calls `liveAgentRegistry.forget(this.agentId)`
|
|
202
|
+
inside the `disposed = true` block, so a subsequent `Agent.getOrCreate(id)`
|
|
203
|
+
always builds a fresh instance.
|
|
204
|
+
|
|
205
|
+
Regression tests:
|
|
206
|
+
|
|
207
|
+
- `tests/agent-registry-cache.test.ts` —
|
|
208
|
+
`dispose() self-evicts so next getOrCreate returns a fresh agent`.
|
|
209
|
+
- `tests/internal/runtime/live-agent-registry.test.ts` —
|
|
210
|
+
`forget(id) removes from cache without calling dispose` +
|
|
211
|
+
`forget(id) is idempotent for unknown ids`.
|
|
212
|
+
|
|
213
|
+
Verified end-to-end by the telegram-pro 2026-05-28 dogfood after the fix:
|
|
214
|
+
**47/48 PASS, 1 SKIP (HONCHO_API_KEY env unset), 0 FAIL** (vs 38/48 PASS, 9
|
|
215
|
+
FAIL before the fix).
|
|
216
|
+
|
|
217
|
+
### Added — Auto-populated `RunResult.usage` + `RunResult.cost` (ADRs D375-D388, T4.2 scope-cut lifted)
|
|
218
|
+
|
|
219
|
+
- `agent.send`-driven runs now expose aggregated token usage on
|
|
220
|
+
`RunResult.usage` (5-bucket `TokenUsage` per D376) and an inferred
|
|
221
|
+
`RunResult.cost` (`CostBreakdown` per D377) automatically. No caller-side
|
|
222
|
+
composition required for the read side — callers still wrap with
|
|
223
|
+
`preflightCheck` / `chargeAndCheckThresholds` for budget enforcement.
|
|
224
|
+
- OpenAI / OpenRouter SSE accumulator parses 5 token buckets:
|
|
225
|
+
`prompt_tokens_details.cached_tokens` → `cacheReadTokens`,
|
|
226
|
+
`completion_tokens_details.reasoning_tokens` → `reasoningTokens`, plus the
|
|
227
|
+
cline#10266 top-level `cache_read_input_tokens` /
|
|
228
|
+
`cache_creation_input_tokens` fallback for Anthropic-on-OpenRouter.
|
|
229
|
+
- `stream_options: { include_usage: true }` is now sent on every
|
|
230
|
+
Chat Completions request so the final usage chunk arrives reliably.
|
|
231
|
+
- Agent loop carries a `UsageAccumulator` per send; each LLM turn merges in,
|
|
232
|
+
the totals land on `AgentLoopOutput.usage` / `AgentLoopOutput.cost`.
|
|
233
|
+
- `FixtureScript` extended with `usage?` / `cost?` so non-fixture runs (the
|
|
234
|
+
real local runtime) plumb the values into `RunResult` via
|
|
235
|
+
`buildResult` in `FixtureRunBase` — fixture mode remains unchanged.
|
|
236
|
+
- Validated end-to-end against OpenRouter (`openai/gpt-4o-mini`): real
|
|
237
|
+
reply, real tokens (`input=68 output=2`), real cost (`$0.000011 estimated`),
|
|
238
|
+
ledger reconciles bit-identical. Report:
|
|
239
|
+
`.claude/knowledge-base/reviews/budget-dogfood-2026-05-28.md`.
|
|
240
|
+
|
|
241
|
+
### Added — Task observability registry (ADRs D361-D374, Adoption Roadmap gap #2)
|
|
242
|
+
|
|
243
|
+
- `Task` namespace (`@theokit/sdk`) exposing static methods `submit`, `list`,
|
|
244
|
+
`get`, `cancel`, `subscribe`, `configure`. Closed 5-state lifecycle
|
|
245
|
+
(`queued | running | finished | error | cancelled`).
|
|
246
|
+
- Pluggable `TaskStore` interface + 2 backends — `InMemoryTaskStore` (default,
|
|
247
|
+
transient) and `JsonFileTaskStore` (opt-in, one JSON file per task,
|
|
248
|
+
single-process invariant documented). SQLite backend deferred to v0.2.
|
|
249
|
+
- New sub-export `@theokit/sdk/task-store` for cross-process readers
|
|
250
|
+
(the `theokit tasks` CLI consumes this).
|
|
251
|
+
- Ring buffer (cap 64) per task for late-attach `subscribe` replay (D372).
|
|
252
|
+
- Idempotent cancel — `Task.cancel(id, reason?)` returns
|
|
253
|
+
`{ cancelled, alreadyTerminal }`, never throws.
|
|
254
|
+
- Cross-process best-effort cancel via `cancelRequested: boolean` field on
|
|
255
|
+
`TaskHandle`; the CLI writes it, the owning Node process honors it at the
|
|
256
|
+
next checkpoint (EC-7).
|
|
257
|
+
- 3 OTel spans via existing telemetry seam (D34): `task.submit`,
|
|
258
|
+
`task.transition`, `task.cancel` (D371). No new peer deps.
|
|
259
|
+
- Errors: `InvalidTaskIdError`, `TaskNotFoundError`,
|
|
260
|
+
`UnsupportedTaskOperationError` exported from the main entry.
|
|
261
|
+
- Auto-eviction: terminal tasks GC'd after retention (InMemory 1h, JsonFile 7d
|
|
262
|
+
defaults; configurable via `Task.configure({ retentionMs })`).
|
|
263
|
+
- 62 SDK tests passing across 6 files. 16 edge cases absorbed (EC-1..EC-16).
|
|
264
|
+
|
|
265
|
+
### Scope cuts (v1)
|
|
266
|
+
|
|
267
|
+
- `Agent.send` / `Agent.batch` / `Workflow.run` / `Cron.register` do NOT yet
|
|
268
|
+
accept a `{ task: true }` option — that adapter integration is deferred to
|
|
269
|
+
v0.2 (see plan v1.2). Today: callers compose via
|
|
270
|
+
`Task.submit("kind", async (ctx) => agent.send(prompt, { signal: ctx.signal }))`.
|
|
271
|
+
- SQLite cross-process backend deferred to v0.2 — JsonFileTaskStore is
|
|
272
|
+
documented as single-process-only.
|
|
273
|
+
- `CloudAgent` task ops throw `UnsupportedTaskOperationError` (D370).
|
|
274
|
+
|
|
275
|
+
## 1.1.0
|
|
276
|
+
|
|
277
|
+
### Minor Changes
|
|
278
|
+
|
|
279
|
+
- Production-readiness for serverless and multi-host deploys (6 gaps from TheoKit cross-repo handoff).
|
|
280
|
+
|
|
281
|
+
**Added:**
|
|
282
|
+
|
|
283
|
+
- **`ConversationStorageAdapter`** interface + `FileSystemConversationStorage` (default) + `InMemoryConversationStorage`. New `AgentOptions.conversationStorage` opt-in. Postgres + Redis recipes in `docs/recipes/`. Strict resume integrity check via `requiresCustomStorage` marker (D325).
|
|
284
|
+
- **`Agent.registry`** — LRU + idle-timeout GC for live `SDKAgent` instances. `configure / evict / evictAll / size / ids` + `onEvict` listener. Defaults: `maxAgents: 100`, `idleTimeoutMs: 30 min`. Eliminates OOM in 24/7 Node deploys.
|
|
285
|
+
- **`AgentRunErrorCode`** discriminated union (16 codes including `quota_exceeded`, `tool_runtime_error`, `aborted`, `invalid_model`, `safety_blocked`, `provider_unreachable`). Plus `AgentRunError.requestId` / `.conversationId` fields and `.retriable` / `.retryAfterMs` / `.providerError` getters. Anti-leak invariant: `providerError` never in `.message`.
|
|
286
|
+
- **`SendOptions.signal`** propagates end-to-end to LLM `fetch({ signal })`. Tokens stop billing on caller cancel. `anySignal` ponyfill for Vercel Edge subsets without native `AbortSignal.any`. `agent.dispose()` fires lifecycle abort. Aborted runs throw `AgentRunError({ code: "aborted" })`; no partial assistant message persists.
|
|
287
|
+
- **`AgentOptions.onToolStart` / `onToolEnd` / `onToolError`** — observation callbacks with `callId` pair correlation + `durationMs`. Hook errors swallowed (do not crash run).
|
|
288
|
+
- **`AgentOptions.onBeforeCreate` / `onBeforeSend`** — admission gates for multi-tenant quota. Errors propagate (NOT swallowed — these are blockers, not observers).
|
|
289
|
+
|
|
290
|
+
25 new ADRs (D303-D325). 113 new tests. 3 real-LLM examples in `examples/{conversation-storage,abort-mid-stream,tool-hooks-tracking}/`. Postgres + Redis recipes in `docs/recipes/`. Full `docs.md` sections: Conversation storage, Agent registry lifecycle, Error codes, Cancellation, Tool lifecycle hooks, Quota / abuse hooks.
|
|
291
|
+
|
|
292
|
+
**Backward compatibility:** all new fields opt-in with safe defaults. Existing apps (telegram-pro, slack-bot, whatsapp-bot, email-bot, teams-bot, vertex-bot, bedrock-bot, handoffs, workflows, cache, eval, skills-google-workspace) compile and run unmodified.
|
|
293
|
+
|
|
294
|
+
Closes Gaps 1-6 of `docs/handoffs/from-theokit/2026-05-25-production-readiness.md`.
|
|
295
|
+
|
|
296
|
+
## [Unreleased]
|
|
297
|
+
|
|
298
|
+
### Added (`onBeforeCreate` / `onBeforeSend` quota gates — Production-Readiness #6)
|
|
299
|
+
|
|
300
|
+
Closes Gap 6 of the TheoKit cross-repo handoff. Lets multi-tenant SaaS deploys enforce per-user / per-conversation quotas at the SDK boundary.
|
|
301
|
+
|
|
302
|
+
- **`AgentOptions.onBeforeCreate`** fires BEFORE the agent is registered or persisted. Receives `{ conversationId, userId? }`. Throwing blocks creation — error propagates as `Agent.create` rejection.
|
|
303
|
+
- **`AgentOptions.onBeforeSend`** fires BEFORE each `agent.send` (before LLM call, before storage append). Receives `{ conversationId, previousMessageCount }`. Throwing blocks the send.
|
|
304
|
+
- **Errors are NOT swallowed (D322).** Unlike tool lifecycle hooks (observation), quota hooks are admission gates — their throws propagate by design.
|
|
305
|
+
- **Order: validate → quota gate → side effects (D323).** Rejected hooks leave zero orphan state on disk or in memory.
|
|
306
|
+
- **`onBeforeCreate` skipped on `Agent.registry` cache hit** — caching is per-process, cold-path always runs the hook.
|
|
307
|
+
- **ADRs:** D322 (errors propagate), D323 (fire before side effects).
|
|
308
|
+
- **Tests:** 8 new in `tests/agent-quota-hooks.test.ts` covering resolve/reject paths, `userId` propagation, no-orphan-on-reject, `previousMessageCount` semantics.
|
|
309
|
+
|
|
310
|
+
### Added (`onToolStart` / `onToolEnd` / `onToolError` tool lifecycle hooks — Production-Readiness #4)
|
|
311
|
+
|
|
312
|
+
Closes Gap 4 of the TheoKit cross-repo handoff. Cost tracking, audit log, per-tool retry/alerting without writing a plugin.
|
|
313
|
+
|
|
314
|
+
- **`AgentOptions.onToolStart`**, **`onToolEnd`**, **`onToolError`** callbacks accepted in `AgentOptions` (top-level — no plugin needed; D315). Match Vercel AI SDK `onChunk`/`onFinish` ergonomics.
|
|
315
|
+
- **`callId` propagated** through the start/end (or start/error) pair from the existing `generateCallId()` in dispatch (D316). Consumers correlate without managing their own counter.
|
|
316
|
+
- **`durationMs`** measured between start hook fire and end/error hook fire — handler latency.
|
|
317
|
+
- **Hook errors swallowed** via single `safeEmitToolHook` chokepoint (D317). Listener throws logged to stderr but never crash the run.
|
|
318
|
+
- **`onToolError.event.error` is ALWAYS an `Error` instance** (EC-6 absorbed) — stderr-string-only failures wrapped in `new Error(stderr)`.
|
|
319
|
+
- **`attempt: 1`** always in v1 (D317 placeholder — reserved for future tool retry policy).
|
|
320
|
+
- **ADRs:** D315 (AgentOptions surface), D316 (callId reuse), D317 (hook errors swallowed — EC-6 absorbed).
|
|
321
|
+
- **Tests:** 3 new in `tests/agent-tool-hooks.test.ts` (surface acceptance + listener-throw safety).
|
|
322
|
+
|
|
323
|
+
### Added (`AbortSignal` end-to-end propagation — Production-Readiness #5)
|
|
324
|
+
|
|
325
|
+
Closes Gap 5 of the TheoKit cross-repo handoff. Tokens stop billing the moment a caller (browser, route handler, `agent.dispose`) signals cancellation.
|
|
326
|
+
|
|
327
|
+
- **`SendOptions.signal`** (already typed) now flows from `LocalAgent.send` → `dispatchRun` → `real-local-run.buildLoopInputs` → `AgentLoopInputs.signal` → `streamLlmTurn` → LLM client `fetch({ signal })`. The infrastructure was already in place at every LLM client; only the orchestrator wiring was missing.
|
|
328
|
+
- **`LocalAgent.#lifecycleAbortController`**: every agent owns a private controller fired by `dispose()`. `send()` composes `[userSignal, lifecycleSignal]` via `anySignal` so eviction (`Agent.registry.evict`) cancels in-flight LLM calls promptly.
|
|
329
|
+
- **`anySignal` ponyfill** (`internal/runtime/abort-utils.ts`) absorbs EC-5: native `AbortSignal.any` when available, ponyfill for runtimes (Vercel Edge subset) that lag. Single-signal short-circuit, undefined entries filtered, abort `reason` propagated.
|
|
330
|
+
- **`AgentLoopInputs.signal`** new optional field; loop uses caller's signal when present, never-aborting placeholder otherwise (legacy behavior preserved when nothing wired).
|
|
331
|
+
- **Aborted runs surface as `AgentRunError({ code: "aborted", retriable: false })`** (D321 + T3.5 finalization). `err.cause` preserves the original `DOMException`.
|
|
332
|
+
- **Aborted runs do not persist partial assistant messages** (D320): the user message persists at entry; the abort path skips the assistant append, preserving conversation history invariant.
|
|
333
|
+
- **ADRs:** D318 (signal plumbing), D319 (lifecycle controller composition), D320 (no partial persist), D321 (AgentRunError aborted wrapping), D324 (anySignal ponyfill — absorbed from EC-5).
|
|
334
|
+
- **Tests:** 13 new (abort-utils — native + ponyfill + edge cases) + 3 wiring sanity tests. Full real-LLM abort dogfood is part of Phase 7.
|
|
335
|
+
|
|
336
|
+
### Added/Changed (`AgentRunError` discriminated codes + retryAfterMs + requestId — Production-Readiness #3)
|
|
337
|
+
|
|
338
|
+
Closes Gap 3 of the TheoKit cross-repo handoff. Makes `AgentRunError` consumer-branchable for proper UX (retry CTAs, billing upsell, cancel suppression) without parsing `.message` strings.
|
|
339
|
+
|
|
340
|
+
**Added:**
|
|
341
|
+
|
|
342
|
+
- **`AgentRunErrorCode`** discriminated union (16 codes) exported from `@theokit/sdk`. Supersets `ErrorCode` with non-HTTP origins (`quota_exceeded`, `tool_runtime_error`, `aborted`, `invalid_model`, `safety_blocked`, `provider_unreachable`). Trailing `(string & {})` keeps autocomplete + accepts legacy provider-prefixed strings.
|
|
343
|
+
- **`AgentRunError.requestId`** + **`AgentRunError.conversationId`** fields. Provider's `x-request-id` / `request-id` header parsed via `parseRequestId` helper in `internal/errors/mappers/shared.ts`. `conversationId` settable by caller for log correlation.
|
|
344
|
+
- **`AgentRunError.retriable`** getter — alias for `isRetryable` (handoff contract; future v2 deprecates `isRetryable`).
|
|
345
|
+
- **`AgentRunError.retryAfterMs`** computed getter — `metadata.retryAfter * 1000` so callers compose with `Date.now()` / `setTimeout` directly. Returns `0` (not `undefined`) when provider sent `Retry-After: 0` (EC-11).
|
|
346
|
+
- **`AgentRunError.providerError`** getter — aliases `metadata.raw`. Anti-leak invariant: `.message` NEVER contains the raw body (D313).
|
|
347
|
+
- **`DispatchResult.errorCode`** field — distinguishes tool dispatch failures: `tool_runtime_error` (handler throw), `invalid_request` (validate failure), `unknown` (registry miss). Consumers mapping DispatchResult → AgentRunError use this directly.
|
|
348
|
+
- **`docs/error-codes.md`** standalone reference with provider mapping tables.
|
|
349
|
+
|
|
350
|
+
**Changed:**
|
|
351
|
+
|
|
352
|
+
- **OpenAI-compatible mapper** detects HTTP 402 + body `code: "insufficient_quota"` / `"quota_exceeded"` and maps to `invalid_request` (ErrorCode is HTTP-pure per D314 — quota_exceeded at AgentRunError layer).
|
|
353
|
+
- **`buildErrorMetadata`** now exposes `parseRequestId` companion for mapper consumption (D314).
|
|
354
|
+
|
|
355
|
+
**ADRs:** D311 (code union + escape hatch), D312 (retryAfterMs getter), D313 (providerError alias), D314 (mapper priorities).
|
|
356
|
+
|
|
357
|
+
**Tests:** 20 new in `tests/errors/agent-run-error-fields.test.ts` (all 6 new codes accepted, getters compute correctly, EC-11 zero-retryAfter, anti-leak invariant). 4 new in `tests/tool-dispatch/tool-error-code.test.ts`. 5 new in `tests/internal/errors/mappers/shared.test.ts` for parseRequestId. 2 new in `tests/internal/errors/mappers/openai-compatible.test.ts` for 402 / insufficient_quota.
|
|
358
|
+
|
|
359
|
+
**Backward compat:** existing `AgentRunError` callers unaffected — new fields are optional, getters compute on demand, `code: string` accepted via `& {}`.
|
|
360
|
+
|
|
361
|
+
### Added (`Agent.registry` — LRU + idle GC for live agents — Production-Readiness #2)
|
|
362
|
+
|
|
363
|
+
Closes Gap 2 of the TheoKit cross-repo handoff. Eliminates OOM in 24/7 Node deploys that previously had no eviction for the live agent set (TheoKit's `dev-agent-gc.ts` only ran in dev mode; production servers accumulated agents until heap pressure crashed them).
|
|
364
|
+
|
|
365
|
+
- **`Agent.registry`** static property exposes the process-wide `LiveAgentRegistry` singleton (ADR D310). Surface: `configure`, `evict`, `evictAll`, `size`, `ids`.
|
|
366
|
+
- **LRU eviction** when `size > maxAgents`. Sync `set` path; eviction runs fire-and-forget (caller doesn't await `dispose`).
|
|
367
|
+
- **Idle timeout sweep** drops agents whose `lastUsedAt < now - idleTimeoutMs`. Configurable sweep interval (default 60s). `setInterval` is `unref()`'d so it does not keep the event loop alive at process exit.
|
|
368
|
+
- **`onEvict(id, reason)`** observability listener. Reason is `"lru" | "idle" | "explicit"`. Listener errors are swallowed with stderr warn (D309 — eviction must not block).
|
|
369
|
+
- **Defaults** (ADR D308): `maxAgents: 100`, `idleTimeoutMs: 30 min`, `sweepIntervalMs: 60_000`. Calibrated for indie/small-team deploys. High-traffic SaaS sets larger; `maxAgents: 0` disables the cache entirely.
|
|
370
|
+
- **`agent.dispose()` called on every eviction** (D309). Errors caught + swallowed so a stuck dispose doesn't block subsequent evictions.
|
|
371
|
+
- **`Agent.getOrCreate` cache hit** (T2.6): consults `Agent.registry.get(id)` before resume/create. `get` refreshes `lastUsedAt` so frequently-used agents resist eviction.
|
|
372
|
+
- **EC-4 absorbed**: `set(id, newAgent)` when `id` already maps to a different agent disposes the old before overwriting (prevents leak under racing `getOrCreate` calls). Idempotent when same instance.
|
|
373
|
+
- **EC-8 absorbed**: idle sweep re-checks entry identity after the dispose await; a `set` that landed mid-sweep is not deleted.
|
|
374
|
+
- **ADRs:** D307 (live vs metadata registry separation), D308 (default tuning), D309 (dispose swallow on eviction), D310 (process-wide singleton).
|
|
375
|
+
- **Tests:** 22 new (16 unit + 6 integration). Coverage: LRU recency, refresh saves, dispose-on-overwrite, dispose-error-swallow, idle sweep eviction, onEvict reasons, maxAgents=0 disables cache.
|
|
376
|
+
|
|
377
|
+
### Added (`ConversationStorageAdapter` — pluggable conversation persistence — Production-Readiness #1)
|
|
378
|
+
|
|
379
|
+
Closes Gap 1 of the TheoKit cross-repo production-readiness handoff (`docs/handoffs/from-theokit/2026-05-25-production-readiness.md`). Unblocks serverless (Vercel, Cloudflare Workers, Lambda) and multi-host (K8s replicas, TheoCloud canary) deploys that cannot use the default `<cwd>/.theokit/agents/<id>/messages.jsonl` filesystem persistence.
|
|
380
|
+
|
|
381
|
+
- **`ConversationStorageAdapter`** interface exported from `@theokit/sdk`. 5 methods (`getMessages`, `appendMessage`, `deleteConversation`, optional `listConversationIds`, optional `compact`, optional `dispose`). Implementations return `Promise<>` uniformly for adapter polymorphism (ADR D306).
|
|
382
|
+
- **`FileSystemConversationStorage`** exported. Default when `AgentOptions.conversationStorage` is unset (zero migration — existing apps unaffected). Wraps the pre-D303 byte-identical behavior including redaction (D68) + compaction every 50 appends (D18). Path-traversal guard re-applied in `deleteConversation` (EC-1, ADR D304); ENOENT swallowed in `listConversationIds` for first-run deploys (EC-2).
|
|
383
|
+
- **`InMemoryConversationStorage`** exported. `Map<conversationId, StoredMessage[]>` for tests + ephemeral dev. Returns defensive copies from `getMessages`.
|
|
384
|
+
- **`StoredMessage`** widened from `user|assistant` to 5 roles (`user|assistant|system|tool_call|tool_result`) for forward compat with tool-shaped messages flowing through the adapter (EC-10, ADR D304). Legacy JSONL files continue to parse — `readSessionFile` filters defensively.
|
|
385
|
+
- **`AgentOptions.conversationStorage?`** opt-in field. Backward compatible: undefined → default FS adapter at `local.cwd`.
|
|
386
|
+
- **Strict resume integrity (EC-3, ADR D325)** — when an agent is created with a custom `conversationStorage`, the registry stores a `requiresCustomStorage: true` marker. `Agent.resume` throws `ConfigurationError(code: "conversation_storage_required")` if the marker is set and the caller did not pass `conversationStorage` again. Prevents silent FS fallback that would lose Postgres/Redis history.
|
|
387
|
+
- **Recipes** at `docs/recipes/conversation-storage-postgres.md` and `docs/recipes/conversation-storage-redis.md`. Both ship Node (pg / ioredis) + Edge (`@neondatabase/serverless` / `@upstash/redis`) flavors. SDK keeps these out of core deps to stay light (ADR D305).
|
|
388
|
+
- **Tests:** 33 new tests in `tests/internal/persistence/conversation-storage-*.test.ts` + `tests/agent-conversation-storage.test.ts`. Contract suite runs against both InMemory + FS via `describe.each`. Coverage includes: lazy create, insertion order, 50× concurrent appends, idempotent delete, defensive copy, path-traversal rejection, ENOENT empty list, tool_call/tool_result roles, redaction, FS-restart persistence, EC-3 marker round-trip + strict-resume throw.
|
|
389
|
+
- **ADRs:** D303 (main barrel export), D304 (FS default + InMemory primary), D305 (Postgres/Redis as recipes), D306 (Promise-uniform interface), D325 (requiresCustomStorage marker).
|
|
390
|
+
|
|
391
|
+
### Fixed (`Agent.streamObject` / `Agent.generateObject` provider routing)
|
|
392
|
+
|
|
393
|
+
- **`StreamObjectOptions.providers?` + `GenerateObjectOptions.providers?`** — new optional field forwarded to the transient agent. Without it, the transient agent infers provider from `model.id` prefix per ADR D186; users running `model: "openai/gpt-4o-mini"` with only `OPENROUTER_API_KEY` set hit `ConfigurationError(provider_unresolved)` because the SDK looks for `OPENAI_API_KEY`. Forwarding `providers: { routes: [{capability:"chat", provider:"openrouter"}], fallback: ["openrouter"] }` routes through OpenRouter as the user expects.
|
|
394
|
+
- **Underlying-error surfacing** — when the transient agent fails BEFORE the LLM is called (e.g. `provider_unresolved`), both `StreamObjectError` and `GenerateObjectError` now wrap the original cause with a clear message ("Agent run failed before the model could reply: …") instead of the misleading "The model returned text instead of calling the `output` tool." Pre-fix users saw a tool-call diagnostic for what is actually a config error.
|
|
395
|
+
- **Evidence:** real-LLM dogfood `examples/telegram-pro` → `/factstream jazz` failed deterministically with OpenRouter key + `openai/gpt-4o-mini` model. Bot's `/factstream` handler updated to forward `buildProviderRouting()` to `Agent.streamObject({ providers })`. Post-fix: `/factstream` PASSES consistently; full dogfood 40/42 (vs prior 39/42).
|
|
396
|
+
|
|
397
|
+
### Added (`Agent.prompt` ergonomics — `throwOnError` + `AgentRunError`)
|
|
398
|
+
|
|
399
|
+
- **`AgentRunError`** — new public error class (extends `TheokitAgentError`). Carries `code`, `provider`, `raw` fields from a failed `RunResult.error`. Exported from the package barrel.
|
|
400
|
+
- **`AgentOptions.throwOnError?: boolean`** — opt-in flag (default `false`, non-breaking). When `true`, `Agent.prompt` rejects with `AgentRunError` instead of resolving with `{ status: 'error', error }`. Reduces idiomatic chat-handler snippets from ~10 lines (status branch) to ~6 lines (try/catch). Cancelled status (`'cancelled'`) does NOT throw — cancel ≠ error. Defensive guard skips throw when `result.error === undefined` (malformed RunResult).
|
|
401
|
+
- **Tests:** 8 tests for `AgentRunError` shape (instanceof chain, fields, message preservation, cause chaining, barrel export). 7 tests for `throwOnError` semantics including EC-2 (cancelled doesn't throw) + EC-3 (defensive guard).
|
|
402
|
+
|
|
403
|
+
### Security (defence-in-depth fix in `assertNoSymlinkEscape`)
|
|
404
|
+
|
|
405
|
+
- **Intermediate-symlink escape closed.** Previous implementation called `lstatSync(path)` only on the **terminal** component. If an intermediate directory in the path was itself a symlink to a location outside `base` (`/project/inner → /outside`), accessing `/project/inner/file.txt` would physically read `/outside/file.txt` and the guard would NOT detect the escape — `lstat` followed the intermediate symlink and reported a regular file. **Fix:** walk to the deepest existing ancestor, `realpathSync` it, then re-attach the lexical suffix; compare against the canonical base. Two new tests pin the fix (terminal-not-yet-created variant included). All 27 existing consumer tests (`agent-session-store`, `persistence/paths`, `lint/no-unguarded-path-input`) remain green.
|
|
406
|
+
|
|
407
|
+
### Added (`@theokit/sdk/tools` sub-export — built-in tools for coding agents)
|
|
408
|
+
|
|
409
|
+
**Drop-in toolkit any coding agent on top of `@theokit/sdk` needs without reimplementing: read, list, search, diff, test.**
|
|
410
|
+
|
|
411
|
+
- **`createReadFileTool({ projectRoot })`** — read a project-relative file as UTF-8. Refuses traversal, sensitive files (`.env*` / `.git/` / `node_modules/` / `.theo/` / lock files), binary files (null-byte detection in first 8 KB; EC-5), and files larger than 5 MB. Returns `{ ok, content, size }` or `{ ok: false, error }`. 12 tests.
|
|
412
|
+
- **`createListDirTool({ projectRoot, max? })`** — list direct entries of a project-relative directory. Defaults to a 500-entry cap (EC-6: avoid 5 MB JSON payloads in 10k-file projects). Each entry exposes `{ name, type: 'file' | 'directory' }`. Result includes `{ truncated, totalCount }` so the agent can refine. 8 tests.
|
|
413
|
+
- **`createSearchTextTool({ projectRoot, maxMatches?, maxFileSize? })`** — recursive literal-text search. Skips sensitive dirs, binary files, and files larger than 1 MB. Defaults to a 100-match cap. Returns `{ matches: [{ file, line, preview }], truncated, totalMatches }`. 8 tests.
|
|
414
|
+
- **`createGitDiffTool({ projectRoot, timeoutMs?, maxStdoutBytes? })`** — `git diff` wrapper. Supports `{ path, cached }` scoping. 30s timeout (kills the whole process group on expiry; EC-7). 5 MB stdout cap. Returns `{ diff, truncated }` or `{ ok: false, error: 'not_a_repo' | 'timeout' | 'git_failed' }`. 7 tests.
|
|
415
|
+
- **`createRunVitestTool({ projectRoot, timeoutMs?, maxStdoutBytes? })`** — vitest runner via `npx --no-install vitest`. **EC-12** fix: parser walks stdout bottom-up to extract the last valid JSON line — skips node deprecation warnings that vitest prepends. 120s timeout + process-group kill. Returns `{ ok, summary }` with `{ numTotalTests, numPassedTests, numFailedTests, success }`. Helper `extractTrailingJson` exported for direct testing. 6 tests.
|
|
416
|
+
- **Public surface** at `@theokit/sdk/tools`. Tsup entry `tools: "src/tools/index.ts"` produces `dist/tools.js` + `.d.ts`; package.json `exports["./tools"]` resolves both ESM + CJS + types. Sub-export smoke test (`tests/tools/sub-export-smoke.test.ts`) pins the 5 named exports.
|
|
417
|
+
|
|
418
|
+
44 tests total in `tests/tools/` (12 + 8 + 8 + 7 + 6 + 5 smoke).
|
|
419
|
+
|
|
420
|
+
### Added (`@theokit/sdk/path-safety` sub-export — path-traversal primitives go public)
|
|
421
|
+
|
|
422
|
+
- **`safePathJoin`, `assertNoSymlinkEscape`, `PathTraversalError`** now exported from `@theokit/sdk/path-safety`. Previously `@internal`; promoted so consumer agents (TheoKit Studio, cli-bot, future coding agents) can validate user-supplied paths without reinventing the guard. Wire shape is unchanged — same signatures, same `ConfigurationError` code (`path_traversal`).
|
|
423
|
+
- **`isForbiddenPath(input)`** — new public primitive shipping the universal sensitive-file blocklist (`.env*` except `.env.example`, `.git/**`, `node_modules/**`, `.theo/**`, lock files). Cross-platform path normalisation (backslashes folded to forward slashes). 15-case test suite covering each blocklist family. Companion error `ForbiddenPathError` (extends `ConfigurationError`, code `forbidden_path`).
|
|
424
|
+
- **Dedicated sub-export** (`./path-safety` in package.json `exports`, separate from the main barrel). Architectural choice: the path-guard module reaches into `internal/runtime` via `errors.js`, which participates in a known import cycle `types/agent.ts ↔ fork-agent.ts`. The dedicated sub-export keeps DTS bundling decoupled — without it, rollup-plugin-dts surfaces a fatal "ForkOptions not exported" false positive on the main bundle.
|
|
425
|
+
- **Public-API smoke test** (`tests/path-safety-public-api.test.ts`) pins the sub-export so a refactor cannot silently revert these to `@internal`.
|
|
426
|
+
|
|
427
|
+
### Added (Ollama integration complete — ADRs D182-D190)
|
|
428
|
+
|
|
429
|
+
**Local-first LLM stack: chat, embeddings, RAG, models discovery, plus LM Studio
|
|
430
|
+
and llama.cpp sibling profiles. 100% local, zero remote API keys required.**
|
|
431
|
+
|
|
432
|
+
- **Ollama builtin provider profile** (D182). `Agent.create({ model: "ollama/llama3.2:3b" })`
|
|
433
|
+
works zero-config after `ollama serve`. `authType: "none"` + sentinel
|
|
434
|
+
`"ollama-local"` Bearer token; local Ollama ignores the Authorization header.
|
|
435
|
+
- **`Ollama embedding adapter`** (D183). Sixth entry in `MEMORY_EMBEDDING_ADAPTERS`,
|
|
436
|
+
targets `/v1/embeddings` (OpenAI-compat). Default model `nomic-embed-text`
|
|
437
|
+
(768 dim). First adapter with `transport: "local"` — `transport` union
|
|
438
|
+
extended from `"remote"` to `"remote" | "local"`. Supports `nomic-embed-text`,
|
|
439
|
+
`all-minilm`, `bge-large`, `bge-m3`, `mxbai-embed-large`.
|
|
440
|
+
- **`Theokit.models.list({ provider: "ollama" })`** (D184). New optional
|
|
441
|
+
`provider` field on `TheokitRequestOptions` routes to the provider's local
|
|
442
|
+
`/v1/models` endpoint when targeted profile has `authType: "none"`. Cloud
|
|
443
|
+
catalog path unchanged when no `provider` is passed (backward compat).
|
|
444
|
+
- **Typed actionable error mapping** (D185). New `mapOllamaTransportError`
|
|
445
|
+
(ECONNREFUSED/ENOTFOUND → "Run \`ollama serve\`") and `mapOllamaHttpError`
|
|
446
|
+
(404 → "Run \`ollama pull <model>\`"; 503 model-loading → retryable). Wired
|
|
447
|
+
into `OpenAIClient` via new optional `providerName` constructor option.
|
|
448
|
+
- **Provider inference from model.id prefix** (D186). `model: "ollama/llama3.2:3b"`
|
|
449
|
+
routes to the Ollama profile and sends `llama3.2:3b` as the model name to the
|
|
450
|
+
LLM body. Aligned with OpenRouter / Hermes / Mastra patterns. Aliases
|
|
451
|
+
`llama-cpp`/`llama.cpp` → `llamacpp`, `lm-studio` → `lmstudio`.
|
|
452
|
+
- **CredentialPool no-op for `authType: "none"`** (D187). `apiKeys: { ollama: [...] }`
|
|
453
|
+
is silently ignored with one-shot stderr warn instead of building a meaningless
|
|
454
|
+
pool of sentinels.
|
|
455
|
+
- **LM Studio builtin profile** (D188). `name: "lmstudio"`, aliases
|
|
456
|
+
`["lm-studio", "lm_studio"]`, default port 1234, `LMSTUDIO_HOST` override.
|
|
457
|
+
- **llama.cpp server builtin profile** (D189). `name: "llamacpp"`, aliases
|
|
458
|
+
`["llama-cpp", "llama.cpp"]`, default port 8080, `LLAMACPP_HOST` override.
|
|
459
|
+
- **OLLAMA_HOST / LMSTUDIO_HOST / LLAMACPP_HOST baseUrl overrides** wired in
|
|
460
|
+
`selectTransport` (alongside existing `OPENAI_API_BASE_URL` etc.).
|
|
461
|
+
- **`OLLAMA_API_KEY`** env var override (optional) for Ollama Cloud or
|
|
462
|
+
reverse-proxy-with-auth setups.
|
|
463
|
+
- **Memory.runDreamingSweep accepts `provider: "ollama"`** in its embedding
|
|
464
|
+
union — fully-local dreaming/clustering is now possible.
|
|
465
|
+
- **`examples/ollama-hello/`** (D190) — minimal Agent.create + send + stream
|
|
466
|
+
against `ollama/llama3.2:3b`. Zero API keys.
|
|
467
|
+
- **`examples/ollama-local-rag/`** (D190) — 100%-local RAG pipeline: embedding
|
|
468
|
+
via `nomic-embed-text`, cosine similarity ranking, context-augmented
|
|
469
|
+
`agent.send` against `llama3.2:3b`. Sample corpus included.
|
|
470
|
+
- **Integration tests against real Ollama** (T1.2, T3.1, T5.1) under
|
|
471
|
+
`tests/integration/` with `skipIf` probes — silent when daemon absent,
|
|
472
|
+
proves end-to-end when present. Per `.claude/rules/real-llm-validation.md`.
|
|
473
|
+
|
|
474
|
+
**Internal: `parseModelId`** sync helper for provider/name splitting; reused by
|
|
475
|
+
`buildLoopInputs` and exported from `internal/llm/model-identifier.ts`.
|
|
476
|
+
|
|
477
|
+
### Added (v1.14 personality-presets — Hermes #26, ADRs D160-D169)
|
|
478
|
+
|
|
479
|
+
- **`Agent.usePersonality(name, opts?)`** public API on `SDKAgent`
|
|
480
|
+
(#roadmap-row-5). Activates a personality preset for the next `send`.
|
|
481
|
+
Reserved names `none` / `default` / `neutral` clear the active preset.
|
|
482
|
+
Returns the resolved `PersonalityPreset` (or `null` when cleared).
|
|
483
|
+
Cloud agents reject with `UnsupportedRunOperationError` (D169).
|
|
484
|
+
- **`PersonalityRegistry`** + **`PersonalityPreset`** re-exported from
|
|
485
|
+
`@theokit/sdk` (read-only). Reads from `<cwd>/.theokit/personalities/*.md`
|
|
486
|
+
(project) + `~/.theokit/personalities/*.md` (user) with project-wins-on-
|
|
487
|
+
collision (D162).
|
|
488
|
+
- **Markdown + Zod frontmatter shape** (D161) — `name` (lowercase-only
|
|
489
|
+
slug, EC-C), `description?`, `tools?` (advisory whitelist), `model?`,
|
|
490
|
+
`tags?`, body = system-prompt overlay. Mirrors `.theokit/agents/*.md`.
|
|
491
|
+
- **Session-default + persistent-opt-in state** (D163) — in-memory per
|
|
492
|
+
`agentId` by default; `{ save: true }` writes to
|
|
493
|
+
`$THEOKIT_HOME/personality.json`. **EC-B:** clear with save DELETES
|
|
494
|
+
the key (never `"agent-id": null`).
|
|
495
|
+
- **Switch lifecycle** (D164) — preserves history by default
|
|
496
|
+
(`{ reset: true }` for opt-in clear), appends user-role transcript
|
|
497
|
+
marker (`[persona switched to <slug>]` or `[persona cleared]`), and
|
|
498
|
+
invalidates the prompt cache via D94 deferred (`reason:
|
|
499
|
+
"personality-switch"`).
|
|
500
|
+
- **Tool whitelist filter** (D167) — `applyPersonalityFilter` narrows
|
|
501
|
+
the exposed `customTools` set; missing entries log a one-shot warn
|
|
502
|
+
with Levenshtein-distance-≤2 "did you mean" hint. Subtractive only
|
|
503
|
+
(D102 layer 4). MCP-style names (`mcp__server__tool`) matched as
|
|
504
|
+
exact strings (EC-I).
|
|
505
|
+
- **Fork inheritance via `AsyncLocalStorage`** (D168) — fork captures
|
|
506
|
+
the parent's active slug **at construction time** as a primitive
|
|
507
|
+
snapshot (EC-A). Parent mid-flight `usePersonality` does NOT mutate
|
|
508
|
+
the fork's voice. `usePersonality` inside a fork = no-op + one-shot
|
|
509
|
+
warning.
|
|
510
|
+
- **CloudAgent.usePersonality** throws `UnsupportedRunOperationError`
|
|
511
|
+
synchronously (D169, matches D122 pattern).
|
|
512
|
+
|
|
513
|
+
### Added (v1.13 context-files-coverage — ADRs D150-D159)
|
|
514
|
+
|
|
515
|
+
- **`FileContextManager` auto-discovery extended** beyond `.theokit/context/*.md`
|
|
516
|
+
to the 2026 industry-standard set:
|
|
517
|
+
- `AGENTS.md` — Linux-Foundation-stewarded, 60k+ repo adoption.
|
|
518
|
+
- `CLAUDE.md` — Anthropic's house format, walk-up + `@import` syntax.
|
|
519
|
+
- `GEMINI.md` — Google Gemini CLI, same shape as CLAUDE.md.
|
|
520
|
+
- `.cursor/rules/*.mdc` — Cursor's current format with frontmatter
|
|
521
|
+
(globs/description/alwaysApply); legacy `.cursorrules` deliberately
|
|
522
|
+
skipped (deprecated by Cursor itself).
|
|
523
|
+
- `.theokit/THEO.md` — new SDK-specific override file (D153 placed
|
|
524
|
+
inside `.theokit/` for zero root pollution).
|
|
525
|
+
- **Walk-up-to-git-root discovery** (D151) — pure `existsSync` checks,
|
|
526
|
+
no `.gitignore` parsing (EC-A KISS), no `.theokitignore` (EC-B scope
|
|
527
|
+
creep dropped). `realpathSync` dedupes symlink chains (EC-F). Git
|
|
528
|
+
worktrees work via `.git` as a file (EC-N).
|
|
529
|
+
- **`@path` import resolver** (D156) — Anthropic/Gemini convention,
|
|
530
|
+
5-hop cap with cycle detection. EC-D: every imported file capped at
|
|
531
|
+
`maxBytesPerFile` BEFORE concatenation (prevents balloon from
|
|
532
|
+
multi-import). EC-Q: line-anchored (`^@\S+$`), inline references
|
|
533
|
+
preserved.
|
|
534
|
+
- **MDC parser** (D154) — YAML frontmatter (`globs`/`description`/
|
|
535
|
+
`alwaysApply`), in-house glob → regex (no `minimatch` dep). EC-I: at
|
|
536
|
+
`agent.send()` time `touchedFiles=[]`, so only `alwaysApply: true`
|
|
537
|
+
rules activate in v1.
|
|
538
|
+
- **Aggregate cap** (D155) — per-file 40_000 chars + total 120_000
|
|
539
|
+
chars. 70/20 head/tail truncation with `…[truncated by theokit]`
|
|
540
|
+
marker. EC-C guard: if `max ≤ MARKER.length`, return head-only slice
|
|
541
|
+
without marker. EC-J: same-priority sort tie-breaks by source path lex
|
|
542
|
+
for prompt-cache stability.
|
|
543
|
+
- **EC-E privacy fix** — disambiguation uses `relative(gitRoot ?? cwd,
|
|
544
|
+
dirname(path))` for source names, NEVER absolute paths. Prevents
|
|
545
|
+
developer home dir / project name from leaking into LLM provider
|
|
546
|
+
logs.
|
|
547
|
+
- **Telemetry counters** (D159) — `context_files_truncated` (per-file)
|
|
548
|
+
- `context_files_total_truncated` (aggregate drop). Lazy `tracer`
|
|
549
|
+
lookup via `globalThis.__theokit_tracer`; no-op when OTel not
|
|
550
|
+
installed (EC-L).
|
|
551
|
+
- **Backward compat** (D158) — existing `.theokit/context/*.md` Zod
|
|
552
|
+
frontmatter sources keep working unchanged. Legacy `.theokit/
|
|
553
|
+
context.json` loads CONTENT and emits one-time deprecation warning
|
|
554
|
+
(EC-K verified).
|
|
555
|
+
- **Public API additions** in `AgentOptions.context`:
|
|
556
|
+
- `maxBytesPerFile?: number` (default 40_000)
|
|
557
|
+
- `maxBytesTotal?: number` (default 120_000)
|
|
558
|
+
|
|
559
|
+
### New internal modules
|
|
560
|
+
|
|
561
|
+
- `internal/runtime/context-discovery.ts` — DiscoverySpec + `findGitRoot`
|
|
562
|
+
- `walkUpForFile` + `walkUpForGlob`.
|
|
563
|
+
- `internal/runtime/context-loaders.ts` — `loadPlainMarkdown` +
|
|
564
|
+
`truncateWithMarker`.
|
|
565
|
+
- `internal/runtime/context-import-resolver.ts` — `resolveImports`
|
|
566
|
+
with 5-hop + cycle detection + per-import cap.
|
|
567
|
+
- `internal/runtime/context-mdc-parser.ts` — MDC frontmatter parser +
|
|
568
|
+
`shouldActivate`.
|
|
569
|
+
- `internal/runtime/context-aggregator.ts` — `applyAggregateCap`.
|
|
570
|
+
- `internal/runtime/context-discovery-runner.ts` — orchestrator over
|
|
571
|
+
all specs.
|
|
572
|
+
|
|
573
|
+
### Test counts
|
|
574
|
+
|
|
575
|
+
- 1062 → **1132 PASS** baseline + **70 new tests** across:
|
|
576
|
+
- `context-discovery.test.ts` (17)
|
|
577
|
+
- `context-loaders.test.ts` (15)
|
|
578
|
+
- `context-import-resolver.test.ts` (12)
|
|
579
|
+
- `context-mdc-parser.test.ts` (8)
|
|
580
|
+
- `context-aggregator.test.ts` (7)
|
|
581
|
+
- `context-manager-multi-format.test.ts` (11)
|
|
582
|
+
- `context-backward-compat.test.ts` (5) — 5 regression
|
|
583
|
+
- 10 new ADRs (D150-D159).
|
|
584
|
+
- CLAUDE.md SDK Roadmap row #4 → ✅ DONE.
|
|
585
|
+
|
|
586
|
+
---
|
|
587
|
+
|
|
588
|
+
### Added (v1.12 memory-provider-adapters — ADRs D141-D149)
|
|
589
|
+
|
|
590
|
+
- **`packages/sdk/src/types/memory-adapter.ts`** — public `MemoryAdapter`,
|
|
591
|
+
`MemoryContext`, `MemoryFact`, `MemoryId`, `MemoryRevision`,
|
|
592
|
+
`MemoryAdapterCapabilities`, `AgentMemory`, `MemoryToolSchema`,
|
|
593
|
+
`MemoryTurnMessage` types. `mkMemoryId(provider, raw)` +
|
|
594
|
+
`extractRawId(id, expected)` enforce cross-adapter id integrity
|
|
595
|
+
(EC-B: prevents `mem0.delete(supermemoryId)` footgun).
|
|
596
|
+
- **`packages/sdk/src/errors.ts`** — new public `MemoryAdapterError`
|
|
597
|
+
- finite `MemoryAdapterErrorCode` literal union (`"auth_failed"`,
|
|
598
|
+
`"rate_limited"`, `"not_found"`, `"network"`, `"invalid_input"`,
|
|
599
|
+
`"unknown"`).
|
|
600
|
+
- **`packages/sdk/src/internal/plugins/types.ts`** — narrows
|
|
601
|
+
`MemoryProviderFactory` return type from `unknown` to
|
|
602
|
+
`MemoryAdapter | Promise<MemoryAdapter>`. Adds `PreUserSendContext`,
|
|
603
|
+
`PreUserSendResult`, `PostAssistantReplyContext` interfaces and
|
|
604
|
+
the two new `HookName` entries.
|
|
605
|
+
- **`packages/sdk/src/internal/plugins/manager.ts`** —
|
|
606
|
+
`runPreUserSendHooks(ctx, maxBytes)` concatenates handler results,
|
|
607
|
+
caps at `maxRecallContextBytes` (EC-A), isolates per-handler
|
|
608
|
+
failures to stderr. `runPostAssistantReplyHooks(ctx)` fire-and-forget.
|
|
609
|
+
- **`packages/sdk/src/internal/runtime/local-agent.ts`** wires the new
|
|
610
|
+
hooks: `pre_user_send` injects `<memory-context>` fence around the
|
|
611
|
+
prompt (EC-G safe — only injected fence is trimmed); `post_assistant_reply`
|
|
612
|
+
fires after `run.wait()` via a wrapped `Run` proxy.
|
|
613
|
+
- **`packages/sdk/src/internal/runtime/local-agent-memory-direct.ts`**
|
|
614
|
+
— `buildAgentMemory(pluginManager, cwd, defaultCtx)` builds the
|
|
615
|
+
`agent.memory.{write,recall,delete}` direct API. Lazy initialize
|
|
616
|
+
(EC-I, fires once on first call). Multi-adapter fan-out for writes;
|
|
617
|
+
merge + dedupe by content for recalls.
|
|
618
|
+
- **`packages/sdk/src/types/agent.ts`** — extends `AgentOptions` with
|
|
619
|
+
`memoryContext` + `maxRecallContextBytes`; `SDKAgent` interface with
|
|
620
|
+
`memory: AgentMemory`. `SendOptions.signal: AbortSignal` for EC-H.
|
|
621
|
+
|
|
622
|
+
### New workspace packages
|
|
623
|
+
|
|
624
|
+
- **`@theokit/memory-supermemory@0.1.0`** — Supermemory wrapper
|
|
625
|
+
(`supermemory@^4.21`, zero-dep MIT). EC-C identifier sanitization
|
|
626
|
+
on every containerTag component.
|
|
627
|
+
- **`@theokit/memory-honcho@0.1.0`** — Honcho wrapper
|
|
628
|
+
(`@honcho-ai/sdk@^2.1`). EC-D session namespaced under userId to
|
|
629
|
+
prevent cross-user leak. AGPL self-host disclosure in README.
|
|
630
|
+
- **`@theokit/memory-mem0@0.1.0`** — Mem0 cloud-only wrapper (D148:
|
|
631
|
+
no OSS local mode). Unique `history(id)` capability. Circuit
|
|
632
|
+
breaker (EC-K: 429 does NOT trip). CVSS 8.1 disclosure in README.
|
|
633
|
+
|
|
634
|
+
### Test counts
|
|
635
|
+
|
|
636
|
+
- 1032 → **1062 PASS** baseline + 9 new memory-adapter tests in SDK
|
|
637
|
+
(memory-adapter contract + aggregation + dispatch + direct API).
|
|
638
|
+
- Adapter packages: Supermemory 21/21, Honcho 17/17, Mem0 18/18 =
|
|
639
|
+
**56 adapter-package tests**.
|
|
640
|
+
- 9 new ADRs (D141-D149).
|
|
641
|
+
- 3 real-LLM examples (`examples/memory-*-basic`).
|
|
642
|
+
- CLAUDE.md SDK Roadmap row #3 (Memory Providers, score 7) → ✅ DONE.
|
|
643
|
+
|
|
644
|
+
---
|
|
645
|
+
|
|
646
|
+
### Added (v1.11 batch-processing — ADRs D134-D140)
|
|
647
|
+
|
|
648
|
+
- **`Agent.batch(prompts, options)`** — new static method on the `Agent`
|
|
649
|
+
façade. Runs N prompts in parallel with bounded concurrency, isolated
|
|
650
|
+
per-prompt failures, optional `onResult` / `onProgress` callbacks, and
|
|
651
|
+
`AbortSignal` support. Default concurrency 4 (D136); capped to
|
|
652
|
+
`prompts.length` to avoid idle workers.
|
|
653
|
+
- **`packages/sdk/src/batch.ts`** — `batchImpl(prompts, options, deps)`
|
|
654
|
+
core. Builds shared `CredentialPool` instances ONCE from
|
|
655
|
+
`options.providers.apiKeys` and wraps the entire batch in
|
|
656
|
+
`withCredentialPool(pools, ...)` (ALS) so every in-flight agent
|
|
657
|
+
observes the SAME pool — one 429 cools the key down once, not N
|
|
658
|
+
times (EC-A fix, D138).
|
|
659
|
+
- **`packages/sdk/src/types/batch.ts`** — public types: `BatchItem`,
|
|
660
|
+
`BatchOptions extends AgentOptions`, `BatchResult` (discriminated
|
|
661
|
+
union by `ok`), `BatchProgress`. Re-exported from `types/index.ts`.
|
|
662
|
+
- **`packages/sdk/src/trajectory-helpers.ts` + `types/trajectory.ts`** —
|
|
663
|
+
opt-in `toShareGptTrajectory(result, options?)` helper for fine-tuning
|
|
664
|
+
dataset generation (D139). Pure transformation; returns `null` for
|
|
665
|
+
failed results so callers can `.map(...).filter(Boolean)`.
|
|
666
|
+
- **`packages/sdk/src/internal/runtime/async-semaphore.ts`** — in-house
|
|
667
|
+
N-permit FIFO semaphore (~50 LoC). No `p-limit` / `p-queue` dependency
|
|
668
|
+
added (D135). `createSemaphore(permits)` throws `ConfigurationError`
|
|
669
|
+
on zero / negative / non-integer.
|
|
670
|
+
- **Router wiring** (`internal/llm/router.ts`) — `buildClient` now
|
|
671
|
+
consults `currentCredentialPool(name)` (ALS) before building a fresh
|
|
672
|
+
pool from `routerOptions.apiKeys`. Backward compatible: outside an
|
|
673
|
+
ALS scope, the existing per-agent pool path is unchanged.
|
|
674
|
+
|
|
675
|
+
### Tests added
|
|
676
|
+
|
|
677
|
+
- `tests/batch.test.ts` — 18 RED → GREEN (empty, parallel, concurrency,
|
|
678
|
+
failure isolation, order preservation, callbacks, abort, EC-A pool
|
|
679
|
+
reference, EC-C pre-aborted, EC-D `signal.reason`, EC-B slow
|
|
680
|
+
`onResult` parallel timing).
|
|
681
|
+
- `tests/batch.property.test.ts` — 5 fast-check properties × 200 runs
|
|
682
|
+
each (1000+ randomized assertions): input order under random delays,
|
|
683
|
+
no prompt loss, failure isolation, filter discipline, bounded
|
|
684
|
+
concurrency.
|
|
685
|
+
- `tests/agent-batch-wiring.test.ts` — 3 façade integration tests
|
|
686
|
+
(Agent.batch exists, empty array, BatchItem metadata round-trip).
|
|
687
|
+
- `tests/integration/batch-with-pool.test.ts` — 3 integration scenarios
|
|
688
|
+
with 2-key pool + concurrency=2.
|
|
689
|
+
- `tests/trajectory-helpers.test.ts` — 14 tests (EC-11..EC-14, EC-F
|
|
690
|
+
malformed messages, tool_use → tool_calls, completed=true).
|
|
691
|
+
- `tests/internal/runtime/async-semaphore.test.ts` + `.property.test.ts`
|
|
692
|
+
— 9 unit + 3 properties × 200 runs (FIFO, peak in-flight bounded,
|
|
693
|
+
release idempotent).
|
|
694
|
+
|
|
695
|
+
### Test counts
|
|
696
|
+
|
|
697
|
+
- 1021 → **1032 PASS** baseline + batch surface in 7 new test files
|
|
698
|
+
(55 new tests + 1600 randomized fast-check assertions).
|
|
699
|
+
- 7 new ADRs (D134-D140).
|
|
700
|
+
- CLAUDE.md SDK Roadmap row #2 (Batch Processing, score 8) → ✅ DONE.
|
|
701
|
+
|
|
702
|
+
---
|
|
703
|
+
|
|
704
|
+
### Added (v1.10 credential-pools — ADRs D123-D133)
|
|
705
|
+
|
|
706
|
+
- **`internal/llm/credential-pool.ts`** — same-provider key rotation primitive
|
|
707
|
+
(CredentialPool class, 4 strategies: fill_first/round_robin/least_used/random,
|
|
708
|
+
ADRs D123-D124, D128).
|
|
709
|
+
- **`internal/llm/credential-pool-types.ts`** — `PooledCredential`,
|
|
710
|
+
`CredentialPoolSnapshot`, `CredentialPoolStrategy`, cooldown ladder constants
|
|
711
|
+
(D125).
|
|
712
|
+
- **`internal/llm/credential-pool-context.ts`** — `withCredentialPool` /
|
|
713
|
+
`currentCredentialPool` AsyncLocalStorage scope for fork inheritance (D131).
|
|
714
|
+
- **`internal/llm/pool-aware-client.ts`** — composition wrapper over `LlmClient`
|
|
715
|
+
(D127) with retry-then-rotate on 429 (D126), immediate rotate on 402/401,
|
|
716
|
+
propagate on 5xx/NetworkError. EC-A: persistence failures during rotate
|
|
717
|
+
degrade to in-memory; do not abort the stream. EC-D: buildClient errors
|
|
718
|
+
propagate without marking pool entry exhausted.
|
|
719
|
+
- **`internal/persistence/credential-pool-store.ts`** — JSON persistence
|
|
720
|
+
(`$THEOKIT_HOME/credential-pool.json`) with D62 versioned envelope, D61
|
|
721
|
+
cross-process file lock, lazy load + 200 ms debounced write (D129).
|
|
722
|
+
- **`errors.ts`** — new public `CredentialPoolExhaustedError` (D133).
|
|
723
|
+
- **`types/providers.ts`** — extends `ProviderRoutingSettings` with optional
|
|
724
|
+
`apiKeys: Record<string, string[]>` + `credentialPoolStrategy:
|
|
725
|
+
Record<string, CredentialPoolStrategy>` (D130).
|
|
726
|
+
- **Router wiring** (`internal/llm/router.ts`) — `buildClient` branches on
|
|
727
|
+
pool presence: ≥2 effective keys → wrap in `PoolAwareLlmClient`; 0/1 → existing
|
|
728
|
+
single-key fast path (D132 backward compat). EC-B: warn once per unknown
|
|
729
|
+
provider in apiKeys config. Empty strings filtered.
|
|
730
|
+
- **`validate-agent-options.ts`** — EC-J ambiguity check: `apiKey` +
|
|
731
|
+
`apiKeys[provider]` together throws `ConfigurationError(code:
|
|
732
|
+
"credential_pool_ambiguous")` with an educative message.
|
|
733
|
+
|
|
734
|
+
### CI gates
|
|
735
|
+
|
|
736
|
+
- **`tests/lint/no-unredacted-pool-token.test.ts`** — bans `.accessToken`
|
|
737
|
+
outside the credential-pool module (and the MCP OAuth allowlist).
|
|
738
|
+
- **`tests/internal/llm/credential-pool.property.test.ts`** — 5 strategy
|
|
739
|
+
invariants × 200 fast-check runs = 1000+ randomized assertions.
|
|
740
|
+
|
|
741
|
+
### Test counts
|
|
742
|
+
|
|
743
|
+
- 960 → **970 PASS** (+10 new wire tests). With property + lint + integration:
|
|
744
|
+
total Phase 5 footprint adds ~55 tests.
|
|
745
|
+
- 11 new ADRs (D123-D133).
|
|
746
|
+
- CLAUDE.md SDK Roadmap row #1 (Credential Pools, score 9) → ✅ DONE.
|
|
747
|
+
|
|
748
|
+
---
|
|
749
|
+
|
|
750
|
+
### Added (v1.9 background-work-block-completion — ADRs D110-D122)
|
|
751
|
+
|
|
752
|
+
- **`internal/runtime/async-local-storage.ts`** — per-fork tool whitelist
|
|
753
|
+
via `AsyncLocalStorage<Set<string>>` (ADR D111). Public helpers:
|
|
754
|
+
`withToolWhitelist(set, fn)`, `currentToolWhitelist()`,
|
|
755
|
+
`checkToolWhitelist(toolName)`. Parallel forks observe their own
|
|
756
|
+
whitelist; nested `withToolWhitelist` shadows the outer set (EC-F).
|
|
757
|
+
- **`internal/runtime/fork-agent.ts`** — fork primitive (ADRs D110-D114):
|
|
758
|
+
- `forkAgentImpl(parent, options, deps)` — inherits parent system
|
|
759
|
+
prompt byte-identical (D112 — cache hit), credentials, model;
|
|
760
|
+
overrides `agentId`, `skills`, `metadata.forkOrigin`
|
|
761
|
+
- `filterMemoryPlugins(unknown)` — EC-B fix: preserves
|
|
762
|
+
`kind: "memory"` plugins so fork can write memory with provenance;
|
|
763
|
+
drops general/model-provider (redundant per-fork re-init)
|
|
764
|
+
- `LocalAgent.fork(options)` shorthand instance method
|
|
765
|
+
- **`internal/judge/`** — judge primitives (ADRs D119-D121):
|
|
766
|
+
- `types.ts` — `Verdict` enum (`done | continue | skipped`),
|
|
767
|
+
`JudgeResult` interface
|
|
768
|
+
- `parse-verdict.ts` — pure prefix matcher with fail-safe `continue`
|
|
769
|
+
(ADR D121). Strict case-sensitive; documents EC-E (BOM trimmed,
|
|
770
|
+
U+200B not)
|
|
771
|
+
- `judge-call.ts` — `judgeCallImpl(ctx, opts, deps)` instantiates aux
|
|
772
|
+
agent (default `openai/gpt-4o-mini` via `OPENROUTER_API_KEY`,
|
|
773
|
+
`tools: []`, EC-A single-env-source); always disposes; folds errors
|
|
774
|
+
into fail-safe `JudgeResult`
|
|
775
|
+
- `verify-side-effect.ts` — `verifyClaim<T>(claims, oracle)`
|
|
776
|
+
hallucination-gate helper, generic over claim type
|
|
777
|
+
- **`types/goal-events.ts`** — `GoalEvent` discriminated union (5
|
|
778
|
+
variants, ADR D115), `GoalResult` return value, `GoalOptions`
|
|
779
|
+
configuration (ADRs D117 AbortSignal, D119 judge model defaults).
|
|
780
|
+
- **`internal/runtime/run-until.ts`** — Ralph loop (ADR D116
|
|
781
|
+
`AsyncGenerator<GoalEvent, GoalResult, void>`):
|
|
782
|
+
- Yields `status_change: active` + per-turn events + final
|
|
783
|
+
`status_change: completed | failed | paused`
|
|
784
|
+
- EC-C: pre-aborted signal yields only `[paused]` (no preceding
|
|
785
|
+
`active`)
|
|
786
|
+
- EC-D: `maxTurns: 0` is supported (vacuous active → failed)
|
|
787
|
+
- Counts consecutive judge parse failures; bails at
|
|
788
|
+
`maxConsecutiveJudgeFailures` (default 3)
|
|
789
|
+
- `LocalAgent.runUntil(goal, options)` instance method
|
|
790
|
+
- **Public API** — `GoalEvent`, `GoalResult`, `GoalOptions`, `Plugin`
|
|
791
|
+
`kind: "memory"` Extract, re-exported via `packages/sdk/src/index.ts`.
|
|
792
|
+
- **`AgentOptions.metadata?: Record<string, unknown>`** — new optional
|
|
793
|
+
field, used by fork (`metadata.forkOrigin` / `metadata.parentAgentId`)
|
|
794
|
+
and judge (`metadata.forkOrigin: "judge"`) for downstream attribution.
|
|
795
|
+
|
|
796
|
+
### Changed (background-work-block-completion)
|
|
797
|
+
|
|
798
|
+
- `internal/agent-loop/tool-dispatch.ts:dispatchSingleCall` — whitelist
|
|
799
|
+
gate fires FIRST (before plugin pre_tool_call hook and file hooks).
|
|
800
|
+
A tool not in the fork's `allowedTools` returns a `tool_result` with
|
|
801
|
+
`"Tool blocked by fork whitelist"` content; agent narrative continues
|
|
802
|
+
unimpeded. Cost: one import + one branch (microseconds per call).
|
|
803
|
+
- `types/run.ts:RunOperation` — gains `"runUntil"` and `"fork"` so
|
|
804
|
+
`UnsupportedRunOperationError` on CloudAgent for these surfaces
|
|
805
|
+
satisfies type narrowing (ADR D122).
|
|
806
|
+
- `CloudAgent.runUntil()` / `CloudAgent.fork()` — throw synchronously
|
|
807
|
+
with explicit messaging; documented as EC-G (sync throw despite
|
|
808
|
+
AsyncGenerator return type).
|
|
809
|
+
|
|
810
|
+
### CI gates
|
|
811
|
+
|
|
812
|
+
- **`tests/lint/no-global-tool-whitelist.test.ts`** — regex grep
|
|
813
|
+
test enforcing AsyncLocalStorage as the only path for per-fork
|
|
814
|
+
whitelist; bans `let _toolWhitelist`-style declarations.
|
|
815
|
+
- **`tests/internal/judge/parse-verdict.property.test.ts`** —
|
|
816
|
+
4 properties × 200 fast-check runs = 800 randomized invariant
|
|
817
|
+
assertions.
|
|
818
|
+
- **`tests/internal/runtime/async-local-storage.property.test.ts`** —
|
|
819
|
+
200 fast-check runs verifying parallel-fork whitelist isolation.
|
|
820
|
+
|
|
821
|
+
### Edge-case review (referenced from `.claude/knowledge-base/plans/background-work-block-completion-plan.md` v1.1)
|
|
822
|
+
|
|
823
|
+
- **EC-A (MUST FIX)**: judge defaults to `OPENROUTER_API_KEY` (single
|
|
824
|
+
source). No multi-provider auto-detect — caller passes
|
|
825
|
+
`judgeApiKey` for Anthropic-only or direct-OpenAI envs.
|
|
826
|
+
- **EC-B (MUST FIX)**: `filterMemoryPlugins` preserves memory plugins
|
|
827
|
+
in fork; drops other kinds.
|
|
828
|
+
- **EC-C (SHOULD TEST)**: pre-aborted signal yields paused only.
|
|
829
|
+
- **EC-D (SHOULD TEST)**: `maxTurns: 0` test covered.
|
|
830
|
+
- **EC-E (SHOULD TEST)**: parseVerdict + BOM/ZWSP edge documented.
|
|
831
|
+
- **EC-F (SHOULD TEST)**: nested `withToolWhitelist` shadow test.
|
|
832
|
+
- **EC-G/H/I/J (DOCUMENT)**: cloud sync throw, whitelist case
|
|
833
|
+
sensitivity, mid-iteration dispose, judge whitelist inheritance.
|
|
834
|
+
|
|
835
|
+
### Test counts
|
|
836
|
+
|
|
837
|
+
- 853 → 911 (+58 new tests; 1000+ fast-check runs).
|
|
838
|
+
- 13 new ADRs (D110-D122).
|
|
839
|
+
- Background work block: **3/3 ✅**. SDK roadmap totals: **19 → 22 (96%)** DONE.
|
|
840
|
+
|
|
841
|
+
---
|
|
842
|
+
|
|
843
|
+
### Added (v1.8 plugin-extension-block-completion — ADRs D97-D109)
|
|
844
|
+
|
|
845
|
+
- **`internal/plugins/`** — full Plugin contract (ADRs D97-D101):
|
|
846
|
+
- `types.ts` — `Plugin` discriminated union (`general`/`model-provider`/`memory`),
|
|
847
|
+
`PluginContext`, `HookName` (8 fixed hooks), `definePlugin` helper
|
|
848
|
+
- `context.ts` — `createPluginContext()` with dev-mode Proxy seal (D99) +
|
|
849
|
+
`ctx.on()` defense-in-depth against non-function handlers (EC-2)
|
|
850
|
+
- `manager.ts` — `PluginManager` with `initialize` (once), dispatch by
|
|
851
|
+
kind, `runPreToolCallHooks` (first-block-wins, D101); EC-4 duplicate
|
|
852
|
+
plugin name surfaces stderr warn
|
|
853
|
+
- `lifecycle.ts` — `runFireAndForgetHooks` + `runTransformHooks` (EC-6:
|
|
854
|
+
null replaces; undefined keeps current)
|
|
855
|
+
- **`internal/tool-registry/`** — 3-layer tool surface (ADRs D102-D104):
|
|
856
|
+
- `registry.ts` — `ToolRegistry` central + `ToolEntry` (with checkFn,
|
|
857
|
+
requiresEnv, emoji, maxResultSizeChars)
|
|
858
|
+
- `toolset.ts` — flat-list `Toolset` + `resolveToolset`/`resolveToolsetStrict`
|
|
859
|
+
(EC-7: duplicates kept, caller dedups)
|
|
860
|
+
- `check-fn-cache.ts` — 30s TTL per tool name + `requiresEnv` check
|
|
861
|
+
(EC-8: concurrent Promise.all idempotent)
|
|
862
|
+
- `result-cap.ts` — `applyResultCap` (default 100k chars)
|
|
863
|
+
- **`internal/providers/`** — provider-as-plugin (ADRs D105-D107):
|
|
864
|
+
- `types.ts` — `ProviderProfile` data-only (D105), `ApiMode` literal union
|
|
865
|
+
- `registry.ts` — `registerProvider`/`getProviderProfile`/`listProviders`
|
|
866
|
+
- EC-5 alias collision warn
|
|
867
|
+
- `builtin/{anthropic,openai,openrouter,gemini}.ts` — 4 profiles
|
|
868
|
+
migrated from hardcoded switch
|
|
869
|
+
- `discovery.ts` — lazy scan of `~/.theokit/plugins/model-providers/`
|
|
870
|
+
via `pathToFileURL` (EC-9 Node 22 ESM support)
|
|
871
|
+
- **Public API** — `Plugin`, `PluginContext`, `HookName`, `definePlugin`,
|
|
872
|
+
`ProviderProfile` re-exported via `packages/sdk/src/index.ts`.
|
|
873
|
+
|
|
874
|
+
### Changed (plugin-extension-block-completion)
|
|
875
|
+
|
|
876
|
+
- `internal/llm/router.ts:buildClient` — consults `getProviderProfile`
|
|
877
|
+
- `selectTransport(apiMode)` instead of hardcoded switch (T4.3).
|
|
878
|
+
EC-3: unsupported apiMode throws `transport_unavailable` with
|
|
879
|
+
actionable message. EC-10: `envVars` ordered fallback (OPENROUTER_API_KEY
|
|
880
|
+
then OPENAI_API_KEY for OpenRouter).
|
|
881
|
+
- `LocalAgent.initialize` — wires `pluginManagerCode.initialize(codePlugins)`
|
|
882
|
+
via `extractCodePlugins` filter (EC-1 discriminates legacy `{ enabled }`
|
|
883
|
+
metadata from new `Plugin[]`); telegram-pro + 7 examples continue to
|
|
884
|
+
compile + run unchanged (D108).
|
|
885
|
+
- `agent-loop/tool-dispatch.ts` — invokes plugin `pre_tool_call` hooks
|
|
886
|
+
BEFORE file-based hooks (T4.2). Author intent (code plugin) wins
|
|
887
|
+
early over operator policy (file hooks).
|
|
888
|
+
- `real-local-run.ts` — `buildCustomToolsInput` concatenates plugin tools
|
|
889
|
+
onto the effective tool catalog without replacing user-supplied tools.
|
|
890
|
+
|
|
891
|
+
### Fixed (plugin-extension-block-completion)
|
|
892
|
+
|
|
893
|
+
- Closes Plugin & extension block of the SDK Patterns Roadmap:
|
|
894
|
+
`plugin-contract-design` (❌ → ✅), `tool-registry-pattern` (⚠️ → ✅),
|
|
895
|
+
`provider-as-plugin` (❌ → ✅). Roadmap totais 16 → 19 (83%) DONE.
|
|
896
|
+
- Adding a new provider now requires zero code changes in `packages/sdk/`
|
|
897
|
+
— publish `@theokit-provider-X` with a `ProviderProfile` and drop in
|
|
898
|
+
`~/.theokit/plugins/model-providers/X/index.mjs`.
|
|
899
|
+
|
|
900
|
+
### Added (v1.7 agent-core-loop-completion — ADRs D86-D96)
|
|
901
|
+
|
|
902
|
+
- **`internal/tool-dispatch/repair-middleware.ts`** — `repairToolCall`
|
|
903
|
+
applies 3 idempotent repairs (case-insensitive name match,
|
|
904
|
+
JSON-string-args parse, type coercion against schema). Fixes 10+
|
|
905
|
+
provider-specific failure modes catalogued in `sdk-references/
|
|
906
|
+
tool-call-failure-recovery.md` (Hermes v0.2 #444, v0.3 #1300,
|
|
907
|
+
v0.8 #5265, etc.).
|
|
908
|
+
- **`internal/tool-dispatch/strip-think.ts`** — `stripThinkBlocks`
|
|
909
|
+
removes `<think>...</think>` chain-of-thought from LLM responses
|
|
910
|
+
BEFORE they enter the message history. Prevents prompt-cache
|
|
911
|
+
invalidation with DeepSeek-R1, Qwen-QwQ providers (Hermes v0.2 #174).
|
|
912
|
+
- **`internal/tool-dispatch/dispatch.ts`** — `dispatchToolWithRepair`
|
|
913
|
+
validate-then-execute wrapper. NEVER throws; all errors return as
|
|
914
|
+
`DispatchResult { isError: true }` so the LLM can self-correct
|
|
915
|
+
(ADR D89).
|
|
916
|
+
- **`internal/runtime/budget.ts`** — `IterationBudget` class with
|
|
917
|
+
iteration cap + compression cap (default 3) + grace-call semantics.
|
|
918
|
+
Closes the 4 compression death spirals Hermes shipped (v0.4 #1723,
|
|
919
|
+
v0.7 #4750, v0.11 #10065, v0.11 #10472). EC-4: NaN-safe `consume`.
|
|
920
|
+
- **`internal/runtime/validate-response.ts`** — detects empty-content +
|
|
921
|
+
zero-toolCalls as a model-bailout signal.
|
|
922
|
+
- **`internal/runtime/compression-helpers.ts`** — `selectCompressionWindow`
|
|
923
|
+
(preserve recent N) + `assertCompressionReduced` (≥10% floor, ADR D92).
|
|
924
|
+
- **`internal/cache-discipline-guard.ts`** — dev-mode warns when system
|
|
925
|
+
prompt / toolset / history mutates mid-conversation. Zero production
|
|
926
|
+
overhead via `shouldGuard()` function (EC-1: not module-init constant).
|
|
927
|
+
- **`Agent.invalidateCache(reason, options?)`** public API (ADR D94).
|
|
928
|
+
Default deferred — applied at next `agent.send()`. `{ applyNow: true }`
|
|
929
|
+
disposes immediately.
|
|
930
|
+
- **CI lint gate** `tests/lint/no-history-mutation-outside-loop.test.ts`
|
|
931
|
+
(ADR D85 mirror) — prevents `ctx.messages.push` outside `agent-loop/`.
|
|
932
|
+
EC-8: bounded by contextual prefix to avoid false positives.
|
|
933
|
+
- **Adversarial property tests** via fast-check (1400+ random inputs):
|
|
934
|
+
- `repair-middleware.property.test.ts` (4 properties × 200 runs)
|
|
935
|
+
- `budget.property.test.ts` (3 properties × 200 runs)
|
|
936
|
+
- **`internal/agent-loop/strip-think-wiring.test.ts`** — integration
|
|
937
|
+
test with mock LLM client validating strip-think wiring end-to-end
|
|
938
|
+
(T7.2 / EC-2 fix).
|
|
939
|
+
|
|
940
|
+
### Changed (agent-core-loop-completion)
|
|
941
|
+
|
|
942
|
+
- `agent-loop/loop.ts` uses `IterationBudget` instead of a bare counter
|
|
943
|
+
(T4.2, ADRs D90-D91). Grace call permits one final iteration after
|
|
944
|
+
budget exhausted.
|
|
945
|
+
- `agent-loop/loop.ts` strips `<think>` blocks via `stripThinkBlocks`
|
|
946
|
+
in `streamLlmTurn` before text returned (T4.1, ADR D96).
|
|
947
|
+
- `agent-loop/tool-dispatch.ts` applies `repairToolCall` before the
|
|
948
|
+
registry lookup (T4.1, ADRs D86-D88). Repairs surface via telemetry
|
|
949
|
+
span attribute `tool.repairs`.
|
|
950
|
+
- `LocalAgent` consumes pending invalidation at the start of every
|
|
951
|
+
`sendLocked` via `consumePendingInvalidation()`. EC-7: failure path
|
|
952
|
+
clears pending state so refresh doesn't get stuck retrying.
|
|
953
|
+
|
|
954
|
+
### Fixed (agent-core-loop-completion)
|
|
955
|
+
|
|
956
|
+
- Closes Agent core loop block of the SDK Patterns Roadmap:
|
|
957
|
+
`prompt-cache-discipline` (📚 → ✅), `tool-call-failure-recovery`
|
|
958
|
+
(❌ → ✅), `compression-death-spiral` (❌ → ✅). Roadmap totals
|
|
959
|
+
13 → 15 (65%) DONE.
|
|
960
|
+
|
|
961
|
+
### Added (v1.6 security-block-completion — ADRs D79-D85)
|
|
962
|
+
|
|
963
|
+
- **`internal/security/path-guard.ts`** is the canonical module for
|
|
964
|
+
path defense (ADR D79). Exports `safePathJoin` (resolve-then-check,
|
|
965
|
+
ADR D80), `assertNoSymlinkEscape` (realpath-based chain resolution),
|
|
966
|
+
`sanitizeIdentifier` (strict grammar `^[a-z0-9][a-z0-9-_]*$`, ADR D81),
|
|
967
|
+
and `PathTraversalError extends ConfigurationError` with code
|
|
968
|
+
`path_traversal` (ADR D65 — no new error hierarchy).
|
|
969
|
+
- **`internal/persistence/exclusive-create.ts`** exports `createExclusive`
|
|
970
|
+
using O_EXCL semantics (ADR D82). Default mode `0o600` (owner-only) —
|
|
971
|
+
EC-2 fix prevents world-readable token/lock files under typical
|
|
972
|
+
umask 022.
|
|
973
|
+
- **`internal/persistence/sqlite-cas.ts`** exports `casUpdate` for
|
|
974
|
+
optimistic concurrency in SQLite-backed stores (ADR D83). Canonical
|
|
975
|
+
`UPDATE ... WHERE version = ?` pattern from Hermes `kanban_db.py`.
|
|
976
|
+
- **CI lint gate** `tests/lint/no-unguarded-path-input.test.ts`
|
|
977
|
+
(ADR D85) prevents regression by flagging any new
|
|
978
|
+
`join(cwd, ".theokit", ..., varName)` callsite that doesn't use
|
|
979
|
+
`safePathJoin` or `sanitizeIdentifier`.
|
|
980
|
+
- **Adversarial property tests** for `safePathJoin` + `sanitizeIdentifier`
|
|
981
|
+
via `fast-check` (~1200 random inputs across 6 properties).
|
|
982
|
+
|
|
983
|
+
### Changed (security-block-completion)
|
|
984
|
+
|
|
985
|
+
- `plugins-manager.assertEntryFileExists` now uses canonical
|
|
986
|
+
`safePathJoin` (replaces inline T3.2 guard from
|
|
987
|
+
markdown-config-migration). Error code `plugin_entry_escape` →
|
|
988
|
+
`path_traversal`.
|
|
989
|
+
- `agent-session-store.sessionFilePath` validates `agentId` via
|
|
990
|
+
`sanitizeIdentifier` (maxLen 128) + `safePathJoin`. Local
|
|
991
|
+
`agent-<uuid>`, cloud `bc-<uuid>`, and bot IDs like
|
|
992
|
+
`tg-dogfood-chat-A` pass natively.
|
|
993
|
+
- `skills-manager.refresh` wraps `entry.name` joins with
|
|
994
|
+
`safePathJoin` + `assertNoSymlinkEscape` (defense-in-depth against
|
|
995
|
+
hostile symlinks inside `.theokit/skills/`).
|
|
996
|
+
- `legacyMemoryJsonPath` (memory/types.ts) sanitizes `namespace`,
|
|
997
|
+
`scope`, `userId` before joining. `storePath` (programmatic) bypasses
|
|
998
|
+
sanitization (trusted).
|
|
999
|
+
- `mcp/client.ts` resolves stdio MCP `cwd` field via `safePathJoin`
|
|
1000
|
+
for relative paths; absolute paths trusted.
|
|
1001
|
+
|
|
1002
|
+
### Fixed (security-block-completion)
|
|
1003
|
+
|
|
1004
|
+
- Closes the Security block of the SDK Patterns Roadmap:
|
|
1005
|
+
`path-traversal-vectors` (❌ PENDING → ✅ DONE) and
|
|
1006
|
+
`toctou-race-prevention` (⚠️ PARTIAL → ✅ DONE). Roadmap totals
|
|
1007
|
+
11 → 13 DONE (57%).
|
|
1008
|
+
|
|
1009
|
+
### Added (v1.5 markdown-config-migration — ADRs D74-D78)
|
|
1010
|
+
|
|
1011
|
+
- **`.theokit/hooks/<name>.md`** is the new canonical format for hooks
|
|
1012
|
+
(ADR D74). One file per hook with YAML frontmatter (event, matcher,
|
|
1013
|
+
command, enabled, priority, timeoutMs) + optional markdown body for
|
|
1014
|
+
rationale prose. Mirrors `skills/<name>/SKILL.md`.
|
|
1015
|
+
- **`.theokit/context/<name>.md`** is the new canonical format for
|
|
1016
|
+
context sources (frontmatter: name, path, enabled, maxTokens).
|
|
1017
|
+
- **`.theokit/plugins/<name>/PLUGIN.md`** replaces `plugin.json` per
|
|
1018
|
+
plugin (frontmatter: name, version, capabilities, entry).
|
|
1019
|
+
- **Zod schemas** type each frontmatter category (HookFrontmatter,
|
|
1020
|
+
ContextSourceFrontmatter, PluginFrontmatter — ADR D76). Schema
|
|
1021
|
+
errors surface as `ConfigurationError` with typed codes
|
|
1022
|
+
(`hook_frontmatter_invalid`, etc.), same pattern as D10
|
|
1023
|
+
SkillFrontmatter.
|
|
1024
|
+
- **Path-traversal guard** on plugin `entry` (T3.2, EC-1 MUST FIX
|
|
1025
|
+
from edge-case review): rejects `..` segments and absolute paths
|
|
1026
|
+
with `plugin_entry_escape` code. Closes a latent security gap that
|
|
1027
|
+
predated the markdown migration.
|
|
1028
|
+
- **`theokit-migrate-config` CLI** in `packages/sdk/bin/` (ADR D78,
|
|
1029
|
+
espelha D44 `theokit-migrate-memory`). Converts legacy JSON to MD
|
|
1030
|
+
with timestamped `.bak` backups, atomic writes per file, pre-flight
|
|
1031
|
+
abort on existing MD destination.
|
|
1032
|
+
- **`atomicWriteText` helper** in `internal/persistence/atomic-write.ts`
|
|
1033
|
+
(T4.1, EC-2 MUST FIX). Same `tmpfile + rename` crash-safety as
|
|
1034
|
+
`atomicWriteJson`, with auto-mkdir of parent dir.
|
|
1035
|
+
|
|
1036
|
+
### Changed (markdown-config-migration)
|
|
1037
|
+
|
|
1038
|
+
- **`parseSimpleYaml` return type widened** to
|
|
1039
|
+
`Record<string, string | number | boolean | string[] | undefined>`.
|
|
1040
|
+
Empty values now coerce to `undefined` so Zod `.optional().default(...)`
|
|
1041
|
+
applies correctly (EC-3 fix). Skills and subagents loaders adapted
|
|
1042
|
+
with narrow helpers (zero behavior change in their schemas).
|
|
1043
|
+
- **`HooksExecutor.initialize` + `loadProjectHooks`** delegate to new
|
|
1044
|
+
shared `loadHookConfig(cwd)` in `internal/runtime/hooks-source.ts`.
|
|
1045
|
+
Tries `.theokit/hooks/` first; falls back to `hooks.json` with
|
|
1046
|
+
one-time stderr deprecation warn (ADR D77).
|
|
1047
|
+
- **`FileContextManager.refresh`** uses the same MD-first chain. Same
|
|
1048
|
+
fallback + warn semantics.
|
|
1049
|
+
- **`PluginsManager.refresh`** detects `PLUGIN.md` per folder before
|
|
1050
|
+
`plugin.json`. Warns on the JSON path; warns on conflict
|
|
1051
|
+
("both files detected — using markdown").
|
|
1052
|
+
- **`telegram-pro` example** migrated: `.theokit/hooks/shell-policy.md`
|
|
1053
|
+
- `.theokit/context/bot-readme.md` replace the legacy JSONs.
|
|
1054
|
+
`workspace-seeds.ts` writes the MD files (idempotent via `ensureFile`).
|
|
1055
|
+
The seed-only `plugins.json` (never consumed by the SDK) was removed.
|
|
1056
|
+
|
|
1057
|
+
### Deprecated (markdown-config-migration)
|
|
1058
|
+
|
|
1059
|
+
- `.theokit/hooks.json`, `.theokit/context.json`,
|
|
1060
|
+
`.theokit/plugins/<name>/plugin.json` — emit a one-time stderr warn
|
|
1061
|
+
on each call to the loader. **Deprecated in v1.5 (warn). Removed in
|
|
1062
|
+
v2.0 (planned Q2 2027)** — users must migrate via
|
|
1063
|
+
`theokit-migrate-config` before v2.0 ships.
|
|
1064
|
+
|
|
1065
|
+
### Added (v1.3 secret-redaction-discipline — Security block 1/2 patterns)
|
|
1066
|
+
|
|
1067
|
+
- **`Security` public namespace** (ADR D68). New top-level export `Security.addPattern(re: RegExp)` registers custom redaction patterns for org-internal token shapes. Additive — built-in patterns cannot be removed. Throws if `/g` flag is missing.
|
|
1068
|
+
- **Canonical secret redactor** in `internal/security/redact.ts` (ADR D68/D71). 12 builtin credential patterns (OpenAI/Anthropic `sk-*`, GitHub PAT classic + fine-grained, GitLab, AWS `AKIA`, Google `AIza`, Slack `xox*-`, Sentry `sntrys_`, Stripe `sk_live_` / `rk_live_`) plus parametric `key=value` matcher (Authorization Bearer, access_token, api_key, password, x-api-key) plus dedicated Bearer pattern. Two-bucket masking: short tokens (<18 chars) → `***`; longer → `prefix...suffix` for debuggability.
|
|
1069
|
+
- **Env opt-out: `THEOKIT_REDACT_SECRETS`** (ADR D69/D70). Default ON. Set to `"false"`/`"0"`/`"no"`/`"off"` to disable; SDK emits one-time stderr warning. Env var snapshotted at module init — runtime mutation (e.g., prompt injection) cannot disable mid-process.
|
|
1070
|
+
- **Wired at output boundaries** (ADR D73):
|
|
1071
|
+
- `internal/errors/mappers/shared.ts:truncateRaw` redacts `ErrorMetadata.raw` before exposure. Closes the vector created by v1.3 error-context-surfacing where 2KB of raw provider response body could echo `Authorization: Bearer sk-...` headers.
|
|
1072
|
+
- `internal/telemetry/tracer.ts` wraps `setAttribute`/`setAttributes`/`addEvent`/`startSpan` to redact string values before they reach Langfuse / Sentry / PostHog exporters.
|
|
1073
|
+
- `internal/runtime/agent-session-store.ts:appendToSessionFile` redacts JSON.stringify(record) before appendFile to the transcript JSONL.
|
|
1074
|
+
- `internal/memory/migrate-sqlite-to-lance.ts` wraps the migration logger so any fact text containing secrets is masked at the egress.
|
|
1075
|
+
- **CI gate against new unredacted sinks** — `tests/lint/no-unredacted-sink.test.ts` greps `src/` for new `console.log`/`appendFile`/`writeFile`/`span.setAttribute` callsites that bypass `redactSecrets`, fails the test run if any land without joining the whitelist (with rationale).
|
|
1076
|
+
- **Adversarial property tests** via `fast-check` — 12 builtin patterns × 200 runs + PARAM_PATTERN × 200 + BEARER × 200 + 4 sink adversarial tests × 50-100 runs each = ~3000 randomized inputs proving zero leak.
|
|
1077
|
+
|
|
1078
|
+
### Changed (secret-redaction-discipline)
|
|
1079
|
+
|
|
1080
|
+
- **`ErrorMetadata.raw` shape**: pre-T1.1 the field returned the original `body` object when ≤2KB; post-T1.1 it always returns a (possibly redacted) string because the redactor coerces non-strings via `JSON.stringify`. A workspace-wide grep at land time confirmed zero callers of `err.metadata.raw.someKey`. Consumers that need the parsed shape must `JSON.parse(err.metadata.raw)`.
|
|
1081
|
+
- **`redactSecrets` consolidated**: the two duplicate impls in `internal/memory/types.ts` (3 patterns) and `internal/runtime/fixture-responder.ts` (5 patterns) are gone — both now route through the canonical module. The fixture sentinel `fixture-search-secret` is replaced locally in `redactEventSecrets` (NOT via `addPattern`) to avoid being cleared by the vitest `beforeEach` reset hook.
|
|
1082
|
+
- **`vitest.setup.ts`** also resets `_extraPatterns` and re-enables redaction between tests (ADR D60 + secret-redaction EC-3) to prevent test bleed across files.
|
|
1083
|
+
|
|
1084
|
+
### Added (v1.3 error-context-surfacing — Error handling block 1/2 patterns)
|
|
1085
|
+
|
|
1086
|
+
- **`ErrorMetadata` + `ErrorCode` types exposed from `errors.ts`** (ADR D65/D66). New optional `metadata` field on `TheokitAgentError` and subclasses carries `{ provider, endpoint, code, statusCode?, retryAfter?, raw? }` when the error originates from a provider HTTP call. `ErrorCode` is a finite literal union (`"rate_limit" | "auth_failed" | "invalid_request" | "timeout" | "server_error" | "context_too_long" | "content_filtered" | "model_unavailable" | "network" | "unknown"`) enabling exhaustive `switch` checks at consumer code.
|
|
1087
|
+
- **Provider error mappers** (ADR D67):
|
|
1088
|
+
- `mapAnthropicError({ status, body, headers, endpoint })` — translates raw Anthropic API HTTP errors into typed `TheokitAgentError` subclasses with full metadata. Handles 401/403/429/400/408/5xx with detail mapping (e.g., 400 with context-length signal → `context_too_long` code; 529 overloaded_error → `server_error` with retryAfter).
|
|
1089
|
+
- `mapOpenAICompatibleError({ providerId, status, body, headers, endpoint })` — same shape for OpenAI-compatible dialects (OpenAI, OpenRouter, DeepSeek, Together, Mistral, DeepInfra, Voyage). Inspects `body.error.code` / `body.error.type` for fine-grained mapping; gracefully falls back to status-based code when body doesn't follow the OpenAI shape (EC-3).
|
|
1090
|
+
- Both mappers truncate raw body to ~2KB in `metadata.raw` to avoid log bloat.
|
|
1091
|
+
- Both ignore HTTP-date format `retry-after` headers (EC-5) — only numeric-seconds form populates `metadata.retryAfter`.
|
|
1092
|
+
- **Wired in call sites**:
|
|
1093
|
+
- `internal/llm/anthropic.ts` — `/v1/messages` HTTP errors go through `mapAnthropicError`.
|
|
1094
|
+
- `internal/llm/openai.ts` — `/v1/chat/completions` HTTP errors go through `mapOpenAICompatibleError`.
|
|
1095
|
+
- `internal/memory/adapters/openai-compatible.ts` — `/v1/embeddings` HTTP errors go through `mapOpenAICompatibleError`; legacy `mapErrorStatus` deleted.
|
|
1096
|
+
- `internal/llm/fallback-client.ts` — fallback decision now considers `AuthenticationError` and `RateLimitError` (not just `NetworkError`), so 401 / 429 from one provider triggers fallback to the next (EC-1 fix).
|
|
1097
|
+
|
|
1098
|
+
### Changed
|
|
1099
|
+
|
|
1100
|
+
- **Refined subclass selection on HTTP errors** (breaking change for callers asserting on specific subclasses). Previously every non-OK HTTP response from Anthropic/OpenAI/OpenRouter/embedding adapters threw a `NetworkError` (or a coarse mapping). Now:
|
|
1101
|
+
|
|
1102
|
+
- `401` / `403` → `AuthenticationError`
|
|
1103
|
+
- `429` → `RateLimitError`
|
|
1104
|
+
- `400` → `ConfigurationError` (with `code: "context_too_long" | "content_filtered" | "model_unavailable" | "invalid_request"` depending on body inspection)
|
|
1105
|
+
- `408` → `NetworkError` (`code: "timeout"`)
|
|
1106
|
+
- `5xx` → `NetworkError` (`code: "server_error"` — covers Anthropic 529 overloaded_error)
|
|
1107
|
+
- Other → `UnknownAgentError`
|
|
1108
|
+
|
|
1109
|
+
Callers using `instanceof TheokitAgentError` (the base class) are unaffected. Callers using subclass-specific `instanceof` may need to broaden (e.g., switch from `instanceof NetworkError` to `instanceof TheokitAgentError`) or handle the additional subclasses. **Affected internal tests updated**: `tests/golden/llm/anthropic-client.golden.test.ts` (401 now asserts `AuthenticationError`), `tests/golden/memory/openai-embedding.golden.test.ts` (400 now asserts `ConfigurationError`).
|
|
1110
|
+
|
|
1111
|
+
### Added (v1.3 persistence & state hardening — 6 patterns from sdk-references)
|
|
1112
|
+
|
|
1113
|
+
- **`internal/persistence/` shared primitives directory** (ADR D59). Cross-cutting state helpers consolidated in one place; `internal/memory/atomic-write.ts` and `internal/memory/cwd-mutex.ts` kept as backward-compatible re-export shims.
|
|
1114
|
+
- **`getTheokitHome(cwd)` + `getProfilesRoot()` + `displayTheokitHome(cwd)`** (ADR D60). Canonical path resolver. Honors `THEOKIT_HOME` env override when set (test isolation, profile switching, multi-tenant deployments); defaults to `<cwd>/.theokit`. Profile root always anchored to `~/.theokit/profiles/` regardless of env.
|
|
1115
|
+
- **`atomicWriteJson<T>(path, data, options?)` typed helper** with auto-mkdir of the parent directory (EC-4 fix). Sits on top of existing `replaceFileAtomic`. Migrated callers: `agent-registry-store`, `transcript-store`, `mcp/token-storage`.
|
|
1116
|
+
- **`withFileLock<T>(path, fn, options?)` cross-process file-lock helper** (ADR D61). Uses `proper-lockfile` optional peer dep with a companion `<path>.lock` file and `realpath: false` so the target file does not need to exist yet (EC-1 fix). Falls back gracefully to in-process `withCwdMutex` (with one-shot stderr warning) when the peer dep is missing. Combines `withCwdMutex` + `proper-lockfile` for full in-process AND cross-process serialization.
|
|
1117
|
+
- **`migrateSchema({ db, currentVersion, migrations })`** SQLite forward-only migration runner via `PRAGMA user_version` (ADR D62). Migrations run inside a transaction (atomic rollback on failure); downgrade attempts throw; gaps in the migration sequence are accepted (result.to reflects last applied version).
|
|
1118
|
+
- **`readVersionedJson<T>(opts)` / `writeVersionedJson<T>(path, data, version)`** JSON envelope helpers with `_schemaVersion` field. The migrate callback receives the FULL parsed object (not just `.data`), so legacy shapes without the wrapper migrate correctly (EC-2 fix). Agent registry migrated from ad-hoc `SCHEMA_VERSION = "1.0"` to standard envelope; legacy-on-disk files are auto-migrated on next save.
|
|
1119
|
+
- **`applyWalWithFallback(db, label)`** SQLite WAL mode helper with DELETE fallback for NFS/SMB/FUSE filesystems (ADR D63). Wired in `internal/memory/index-db.ts` for the memory index. Warns once per label on fallback.
|
|
1120
|
+
- **`sanitizeFts5Query(query)` + `containsCjk(text)`** FTS5 query sanitization (ADR D64). 6-step port of Hermes' `_sanitize_fts5_query` — preserves quoted phrases, strips unmatched specials, collapses repeated asterisks, strips dangling boolean operators, auto-quotes hyphenated/dotted/underscored identifiers, restores phrases. Empty-after-sanitize is short-circuited at call sites (EC-3 fix) to avoid `MATCH ''` runtime errors. CJK detection deferred-routing helper for v1.4 trigram table.
|
|
1121
|
+
|
|
1122
|
+
### Added (test infrastructure)
|
|
1123
|
+
|
|
1124
|
+
- **Vitest hermetic test isolation** (`vitest.setup.ts`). Autouse `beforeEach` sets `THEOKIT_HOME` to a fresh tmpdir per test; `afterEach` cleans up + restores the original env value. Tests never touch the developer's real state.
|
|
1125
|
+
- **Lint test `tests/lint/no-hardcoded-theokit-path.test.ts`** that audits `.theokit` literal usage in `src/` and gates regressions (current debt allowlisted; new code MUST use `getTheokitHome(cwd)`).
|
|
1126
|
+
- **Integration E2E test** exercising the full persistence stack — env override → atomic-write → file-lock → schema migration → WAL → FTS5 sanitization — in a single hermetic test.
|
|
1127
|
+
|
|
1128
|
+
### Changed
|
|
1129
|
+
|
|
1130
|
+
- `agent-registry-store.ts` now reads/writes via `readVersionedJson` + `writeVersionedJson`. Legacy on-disk shape `{ schemaVersion: "1.0", agents: {...} }` is migrated transparently on next save to `{ _schemaVersion: 1, data: {...} }`.
|
|
1131
|
+
- `index-manager.ts:ftsSearch` now uses `sanitizeFts5Query` (replacing the previous coarse per-token quoter) and short-circuits when the sanitized result is empty.
|
|
1132
|
+
- `index-db.ts` calls `applyWalWithFallback` before applying schema; `MemoryDb` interface now exposes `pragma()`.
|
|
1133
|
+
- `index-schema.ts` PRAGMA_STATEMENTS no longer includes `journal_mode=WAL` (now applied via the helper for graceful fallback).
|
|
1134
|
+
- `transcript-store.ts` switched from non-atomic `writeFile` to `atomicWriteJson` (auto-mkdir + atomic rename).
|
|
1135
|
+
- `mcp/token-storage.ts` switched from sync `writeFileSync` to async `atomicWriteJson` (still followed by `chmodSync(0o600)` for POSIX permission tightening).
|
|
1136
|
+
|
|
1137
|
+
### ADRs added
|
|
1138
|
+
|
|
1139
|
+
- D59 — `internal/persistence/` is the home for cross-cutting state primitives; memory/ re-exports preserved for backward compat
|
|
1140
|
+
- D60 — `getTheokitHome(cwd)` returns `THEOKIT_HOME || join(cwd, ".theokit")` (single getter, env override optional)
|
|
1141
|
+
- D61 — file-lock via `proper-lockfile` optional peer dep with companion lockfile + graceful in-process fallback
|
|
1142
|
+
- D62 — schema versioning: SQLite `PRAGMA user_version` + JSON `_schemaVersion` envelope, forward-only migrations
|
|
1143
|
+
- D63 — WAL primary, DELETE journal fallback on NFS/SMB; warn once per label
|
|
1144
|
+
- D64 — FTS5 sanitizer 6-step + CJK auto-detection (trigram routing deferred to v1.4)
|
|
1145
|
+
|
|
1146
|
+
### Added (v1.2 features — paridade técnica com Vercel AI / Mastra)
|
|
1147
|
+
|
|
1148
|
+
- **`Agent.streamObject<T>({ schema, prompt, ... })`** — typed structured output WITH partial-object streaming via synthetic forced tool (ADR D39). Returns `AsyncIterator<StreamObjectEvent<T>>` emitting zero or more `{ type: "partial", partial: DeepPartial<T>, attempt }` events plus exactly one `{ type: "complete", object: z.infer<T>, ... }` at the end. Reuses 80% of `generateObject` infrastructure. EC-4 (cancellation cleanup), EC-5 (refine/transform fallback), EC-6 (parallel tool-use dedup) covered by tests.
|
|
1149
|
+
- **`@theokit/react` v1.2.0 — family of 3 hooks** (ADR D40): `useTheoChat` (multi-turn, existing) + `useTheoCompletion` (single-shot text gen, equivalent to Vercel `useCompletion`) + `useTheoAssistant<T>` (object-shaped streaming, wraps `Agent.streamObject`). Each hook has a matching server-side handler: `streamTheoChat`, `streamCompletion`, `streamAssistant`. Shared SSE parser in `internal/sse-parser.ts` handles all wire codes including new `o:`/`O:` for object streaming (ADR D45).
|
|
1150
|
+
- **OAuth 2.1 PKCE for MCP HTTP servers** (ADR D41). `McpAuthConfig.oauth` opts into the flow. Two modes: `manual` (paste callback URL via stdin, SSH-friendly) and `localhost` (auto-spawned http.createServer on a free port). Token storage prefers OS keychain (`keytar`, optional peer dep) with `~/.theokit/mcp-tokens.json` (chmod 600) fallback. EC-2 (state CSRF validation), EC-9 (concurrent refresh serialization), EC-10 (default expires_in 3600s) covered.
|
|
1151
|
+
- **Auto-instrumentation of telemetry vendors** (ADR D42). `tracer.ts` feature-detects `@langfuse/node` v3+, `@sentry/node`, and `posthog-node` via `createRequire`. When present + `telemetry.enabled: true`, registers OTel exporter automatically. Opt-out via `telemetry.autoDetect: false` OR `telemetry.disable: ["langfuse"]`. EC-12 (double-billing prevention) covered.
|
|
1152
|
+
- **LanceDB backend for Memory.index** (ADR D43). `Memory.create({ index: { backend: "lance" } })` activates `@lancedb/lancedb` (optional peer dep). SQLite remains default. Lance scales to 100k+ facts. Filters use Lance's structured filter API — NO string interpolation, EC-1 MUST FIX. EC-8 (embedding dim mismatch) typed error.
|
|
1153
|
+
- **Migration CLI `theokit-migrate-memory`** (ADR D44). Migrates Memory.index from SQLite to Lance preserving 100% of facts. Atomic commit via rename (`lance-new/` → `lance/`); SQLite preserved by default for rollback. EC-3 MUST FIX: validation uses NFC unicode normalization on both sides so facts with accents/emojis migrate correctly.
|
|
1154
|
+
|
|
1155
|
+
### Added (ADRs locked)
|
|
1156
|
+
|
|
1157
|
+
- D39 — `Agent.streamObject<T>` returns AsyncIterator with partial+complete events
|
|
1158
|
+
- D40 — React hooks family: 3 separate hooks (useTheoChat / useTheoCompletion / useTheoAssistant)
|
|
1159
|
+
- D41 — OAuth 2.1 PKCE for MCP HTTP + token storage with keychain fallback
|
|
1160
|
+
- D42 — Auto-instrumentation via createRequire feature-detect
|
|
1161
|
+
- D43 — LanceDB backend behind same IndexManager interface
|
|
1162
|
+
- D44 — Migration SQLite → Lance is standalone CLI (theokit-migrate-memory)
|
|
1163
|
+
- D45 — `SDKObjectDelta` variant + wire codes `o:`/`O:`
|
|
1164
|
+
- D46 — Cross-agent shared memory deferred to v1.3 (threat-model own scope)
|
|
1165
|
+
|
|
1166
|
+
### Deferred
|
|
1167
|
+
|
|
1168
|
+
- **Cross-agent shared memory** (`MemoryOptions.scope: "global" | "team"`): postponed to v1.3 because the threat-model around write authorization across users requires its own ADR. Workaround in the meantime: `scope: "user"` with constant `userId` (e.g., `"team-shared"`).
|
|
1169
|
+
|
|
1170
|
+
### Added (v1.1 features)
|
|
1171
|
+
|
|
1172
|
+
- **`Agent.generateObject<T>({ schema, prompt })`** — typed structured output via synthetic forced tool (ADR D33). Returns `{ object: z.infer<T>, raw, usage, finishReason }`. Retry-on-parse-fail with `maxRetries` (default 1). Transient agent disposed AND hard-deleted from registry across retries (EC-3 no leak). Same provider routing/fallback as `agent.send`.
|
|
1173
|
+
- **`AgentOptions.telemetry`** — opt-in OpenTelemetry spans for `agent.send`, `llm.call`, `tool.call` (ADR D34). Privacy-by-default: NO content logged unless `includeContent: true`. `@opentelemetry/api` is OPTIONAL peer dep loaded via `createRequire`. All OTel calls wrapped in `safe()` so exporter errors NEVER propagate to `agent.send` (EC-1).
|
|
1174
|
+
- **`@theokit/react` v1.0.0** — new workspace package (ADR D32). `useTheoChat` React hook (HTTP fetch + SSE parser, AbortController on unmount, EC-6 5xx handling, EC-8 graceful close). `streamTheoChat` Next.js-compatible SSE handler (EC-2 pre-stream typed errors return HTTP 400/401). Wire format = Vercel AI Data Stream v1 (drop-in `useChat` migration; no `ai` package runtime dep). React peer dep `^18 || ^19`.
|
|
1175
|
+
|
|
1176
|
+
### Validations (v1.1 pillar audits)
|
|
1177
|
+
|
|
1178
|
+
- **Persistence chaos** — 20/20 random-timed SIGKILL recoveries, 0 registry corruptions (snapshot: `persistence-chaos-2026-05-17.md`).
|
|
1179
|
+
- **MCP servers** — 4 distinct MCP servers operational across stdio+http (filesystem, mcp-http, tavily, puppeteer); snapshot: `mcp-audit-2026-05-17.md`.
|
|
1180
|
+
- **Memory at scale** — 12 facts → 12 clusters via `text-embedding-3-small`, 100% Active Memory recall on 4 thematic queries (snapshot: `memory-scale-2026-05-17.md`).
|
|
1181
|
+
- **Chat-bot DX portability** — N=2 examples using all 4 helpers: `telegram-pro` + new `cli-bot` (snapshot: `dx-chatbot-portability-cli-2026-05-17.md`).
|
|
1182
|
+
- **Adversarial safety** — 8/8 validation/permission/state scenarios blocked; 0 crashed (snapshot: `safety-adversarial-2026-05-17.md`).
|
|
1183
|
+
|
|
1184
|
+
### Added
|
|
1185
|
+
|
|
1186
|
+
- Public `AgentOptions.tools` field for inline custom tools (#tools-inline). The SDK now exposes a `CustomTool` type — `{ name, description, inputSchema, handler }` — that consumers can pass at `Agent.create()` or `Agent.resume()`. Handlers are invoked locally when the model emits `tool_use`. Local runtime only; cloud agents throw `ConfigurationError(code: "cloud_custom_tools_rejected")` when `tools.length > 0`. Handlers are not persisted (allow-list strip in `stripSecretsFromOptions`) — re-pass on resume. Reserved-name collisions (`shell`, `memory_search`, `memory_get`, `mcp_*`) and duplicate names rejected at validation time.
|
|
1187
|
+
- Per-call `SendOptions.tools` override (#tools-percall). `agent.send(msg, { tools: [...] })` fully replaces `AgentOptions.tools` for that run, matching the existing `mcpServers` semantics. `undefined` → fall back to agent-level tools; `[]` → explicit clear (no custom tools); `[t1, t2]` → exact replacement. Same validation rules apply per-call. Cloud agents reject per-call tools with the same `cloud_custom_tools_rejected` code.
|
|
1188
|
+
- `Agent.getOrCreate(agentId, options)` static helper (ADR D22). Consolidates the resume-or-create dance into a single call: tries `Agent.resume` first; falls through to `Agent.create({ ...options, agentId })` on `UnknownAgentError`; retries `Agent.resume` once on same-process create race (`ConfigurationError(agent_id_already_exists)`). Re-throws every other error verbatim. Eliminates ~30 LoC of boilerplate from each of the 6 examples that previously hand-rolled the pattern.
|
|
1189
|
+
- `createAgentFactory(common)` public function (ADR D23). Captures shared `AgentOptions` once and exposes `forSession(agentId, overrides?)` + `getOrCreate(agentId, overrides?)`. Top-level shallow merge with `overrides` winning; deep merge for `local`/`memory`/`cloud`; total replace for collection-shaped fields. The function-level `agentId` always wins. Designed for chat-bot patterns where most config is shared across users.
|
|
1190
|
+
- `defineTool<T extends ZodType>(spec)` Zod-driven type-safe builder for `CustomTool` (ADR D24). Converts schema to JSON Schema via Zod 4's native `z.toJSONSchema` (with `unrepresentable: "any"` for transforms/refines). Wraps the handler with a runtime `schema.parse` step — handler receives `z.infer<T>` instead of `Record<string, unknown>`. Removes `as` casts in tool handlers. Zod is an OPTIONAL peer dependency — consumers who don't use `defineTool` don't pay any bundle cost.
|
|
1191
|
+
- `Agent.builder()` fluent alternative to the options bag (ADR D25). Returns an `AgentBuilder` with chainable setters (one per top-level `AgentOptions` field) and three terminals: `.build()` (shallow-cloned snapshot), `.create()` (delegates to `Agent.create`), `.getOrCreate(agentId)` (delegates to `Agent.getOrCreate`). Validation runs inside the terminal — no duplicate rules.
|
|
1192
|
+
|
|
1193
|
+
## 1.0.0
|
|
1194
|
+
|
|
1195
|
+
### Major Changes
|
|
1196
|
+
|
|
1197
|
+
- v1.0.0 — General availability.
|
|
1198
|
+
|
|
1199
|
+
This release closes the 14 gaps tracked in `.claude/knowledge-base/plans/sdk-v1-ga-completion-plan.md` and locks the architectural decisions in the ADR directory (`.claude/knowledge-base/adrs/D01-..D14-`).
|
|
1200
|
+
|
|
1201
|
+
### Highlights
|
|
1202
|
+
|
|
1203
|
+
**Memory subsystem** (already in 0.x, now stabilized):
|
|
1204
|
+
|
|
1205
|
+
- Markdown-first storage at `.theokit/memory/MEMORY.md` + `notes/*.md`
|
|
1206
|
+
- SQLite + FTS5 + sqlite-vec hybrid index
|
|
1207
|
+
- `memory_search` / `memory_get` tools
|
|
1208
|
+
- Active Memory with circuit breaker + LRU cache
|
|
1209
|
+
- Dreaming/REM consolidation with `dream-diary.md`
|
|
1210
|
+
|
|
1211
|
+
**Embedding catalog** (ADR D11):
|
|
1212
|
+
|
|
1213
|
+
- 5 fully-implemented providers: `openai`, `mistral`, `openrouter`, `voyage`, `deepinfra`
|
|
1214
|
+
- `lmstudio`, `google`, `bedrock` are deferred to v1.1 (ADRs in the SDK repo)
|
|
1215
|
+
|
|
1216
|
+
**`OpenAiCompatibleConfig.embeddingsPath`** (EC-2 fix):
|
|
1217
|
+
|
|
1218
|
+
- New optional config field on the shared embedding factory. REPLACES the default `/v1/embeddings` suffix; never concatenates. DeepInfra uses `/v1/openai/embeddings`.
|
|
1219
|
+
|
|
1220
|
+
**Strict skills frontmatter** (ADR D10) — BREAKING:
|
|
1221
|
+
|
|
1222
|
+
- `.theokit/skills/<name>/SKILL.md` now requires YAML frontmatter with `name` + `description`.
|
|
1223
|
+
- Malformed YAML or missing required fields exclude the skill from `agent.skills.list()` with a stderr warning. The agent run continues.
|
|
1224
|
+
- Migration: `grep -rL "^---$" .theokit/skills/*/SKILL.md` finds skills needing the frontmatter block.
|
|
1225
|
+
|
|
1226
|
+
**`Symbol.asyncDispose` on `SDKAgent`** (ADR D5):
|
|
1227
|
+
|
|
1228
|
+
- `await using agent = await Agent.create(...)` typechecks and runtime-works on both Local and Cloud runtimes.
|
|
1229
|
+
- `CloudAgent.dispose()` is now idempotent (EC-3); double-dispose runs the side-effect at most once.
|
|
1230
|
+
|
|
1231
|
+
**Embedding adapter unknown-model rejection** (EC-4):
|
|
1232
|
+
|
|
1233
|
+
- `createOpenAiCompatibleRuntime` throws `ConfigurationError(code: "embedding_unknown_model")` when the chosen model is not in the adapter's dimension table. Prevents downstream vec0 dimension mismatches.
|
|
1234
|
+
|
|
1235
|
+
**Node 22.12+ mandatory** (ADR D1):
|
|
1236
|
+
|
|
1237
|
+
- All gates (test, typecheck, biome, knip, validate, dogfood) run on Node 22.12+.
|
|
1238
|
+
- Pre-push hook gates Node version with a friendly remediation message (EC-1).
|
|
1239
|
+
- GitHub Actions CI matrix pins Node 22.12 + 22-latest.
|
|
1240
|
+
|
|
1241
|
+
**`pnpm validate` strict on publint + attw** (ADR D6):
|
|
1242
|
+
|
|
1243
|
+
- Either tool's failure blocks `pnpm validate` and CI. No warning-only mode.
|
|
1244
|
+
|
|
1245
|
+
### Default model id
|
|
1246
|
+
|
|
1247
|
+
The default agentic model is `google/gemini-2.0-flash-exp:free` (OpenRouter free tier). Override per-agent with `model: { id: "..." }` or query `Theokit.models.list()` for the canonical PaaS catalog (ADR D4).
|
|
1248
|
+
|
|
1249
|
+
### Cloud runtime
|
|
1250
|
+
|
|
1251
|
+
Pre-release. `Agent.getRun({ runtime: "cloud" })`, `agent.listArtifacts()`, `agent.downloadArtifact()` throw `ConfigurationError(code: "cloud_runtime_pre_release")` when invoked with non-fixture API keys. Fixture mode (`theo_test_*` keys) remains the documented test seam.
|
|
1252
|
+
|
|
1253
|
+
All notable changes to `@theokit/sdk` are documented here.
|
|
1254
|
+
|
|
1255
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
1256
|
+
|
|
1257
|
+
## [Unreleased]
|
|
1258
|
+
|
|
1259
|
+
### Added (multimodal demo `examples/telegram-pro`)
|
|
1260
|
+
|
|
1261
|
+
- **New `examples/telegram-pro/`** — ~600 LoC Telegram bot that reproduces the 5 highest-value patterns from OpenClaw's `extensions/telegram` (187 production files) on top of `@theokit/sdk` 1.0.0:
|
|
1262
|
+
- **Voice transcription** ([`src/transcribe.ts`](../../examples/telegram-pro/src/transcribe.ts)) — downloads the OGG/Opus from Telegram, POSTs multipart to Whisper. Provider order: `OPENAI_API_KEY` → `GROQ_API_KEY` → graceful "voice not configured" reply. Transcript is injected into the agent loop as `[voice transcript: ...]`.
|
|
1263
|
+
- **Vision** ([`src/vision.ts`](../../examples/telegram-pro/src/vision.ts)) — photo and sticker descriptions via `google/gemini-2.0-flash-001` multimodal on OpenRouter. Disk-cached at `.theokit/cache/vision/<sha256>.txt` keyed by Telegram's `file_unique_id`, so repeated stickers (common in groups) skip the LLM roundtrip.
|
|
1264
|
+
- **Inline buttons** ([`src/buttons.ts`](../../examples/telegram-pro/src/buttons.ts)) — agent emits `[BUTTONS: A | B | C]` at end of reply; example strips the marker, renders a grammy `InlineKeyboard`, and routes button taps back to the agent as `[user tapped button: A]` so conversation history stays consistent.
|
|
1265
|
+
- **Group `@mention` gating** ([`src/group-policy.ts`](../../examples/telegram-pro/src/group-policy.ts)) — `shouldRespondInChat(ctx, policy)` filter; private chats always pass; groups only when message text contains `@<botname>`, replies to the bot, or starts with `/`.
|
|
1266
|
+
- **Forum-topic scoping** ([`src/agent.ts`](../../examples/telegram-pro/src/agent.ts)) — per-`message_thread_id` agentId (`tg-pro-tpc-<chatId>-<threadId>`) so each topic in a supergroup gets its own isolated session JSONL. Memory namespace stays scoped to `userId` so facts follow the user across topics.
|
|
1267
|
+
- **README walkthrough** — full BotFather setup including `/setprivacy → Disable` (so the bot sees all group messages, not just commands), per-pattern try-it examples, filesystem layout inspection, and an explicit "what this example does NOT cover" honesty note.
|
|
1268
|
+
- **examples/README.md inventory** — `telegram-pro` listed at the top as the **Multimodal demo**, ahead of `telegram-assistant` (personal assistant) and `telegram-bot` (minimal reference).
|
|
1269
|
+
|
|
1270
|
+
### Added (chat assistant readiness — flagship demo `examples/telegram-assistant`)
|
|
1271
|
+
|
|
1272
|
+
- **New `examples/telegram-assistant/`** — ~300 LoC Personal Assistant Telegram bot built on `@theokit/sdk` 1.0.0. Demonstrates the full chat-assistant surface end-to-end against a real LLM:
|
|
1273
|
+
- **Commands**: `/start /help /me /remember /forget /recall /summary /reset` — covers explicit fact write, fact removal by substring, past-conversation search via `corpus="sessions"`, dreaming consolidation via `Memory.runDreamingSweep`, and conversation reset.
|
|
1274
|
+
- **Per-user isolation** — agent id = `tg-assistant-<userId>`, memory namespace pinned to `ctx.from.id` so group chats keep each member's facts separated (EC-11 documented).
|
|
1275
|
+
- **Allow-list** — optional `TELEGRAM_ALLOWED_USERS` env var locks the bot to specific Telegram user-ids so a randomly-discovered bot can't burn the operator's LLM budget.
|
|
1276
|
+
- **Format-aware replies** — Telegram MarkdownV2 escape + auto-split for responses > 4096 chars (`splitForTelegram` chooses paragraph/newline boundaries before hard-splitting at 4000 chars).
|
|
1277
|
+
- **Daily dreaming hook** — `runDream()` wraps `Memory.runDreamingSweep` and picks the embedding provider from available env keys (`OPENAI_API_KEY` → `MISTRAL_API_KEY` → `OPENROUTER_API_KEY`).
|
|
1278
|
+
- **README walkthrough** — full BotFather token-acquisition flow (no prior Telegram knowledge needed), OpenRouter signup, `.env` template, restart-proof demo, file-system layout inspection, and a "what survives restart vs `/reset`" matrix.
|
|
1279
|
+
- **examples/README.md inventory** — `telegram-assistant` listed as the **Flagship demo** at the top of the table; existing minimal `telegram-bot` reference kept intact.
|
|
1280
|
+
|
|
1281
|
+
### Fixed (chat assistant readiness — Phase 5 dogfood-driven bug)
|
|
1282
|
+
|
|
1283
|
+
- **Persistent-registry coalescing dropped second-mutation data.** Two synchronous `registerAgent` calls (e.g., create chat A then create chat B in quick succession) used to coalesce into ONE save whose snapshot only captured the agent that registered before the first microtask flushed. The second agent's full options were never persisted; on restart, `Agent.resume` cold-started a fresh agent with no model, no memory, no system prompt — the run then failed with `claude-sonnet-4-6 is not a valid model ID` because real-local-run's fallback model is not on OpenRouter. Caught in Phase 5 dogfood against real Gemini-flash.
|
|
1284
|
+
- **Fix**: the save loop now uses a `dirtyCwds` Set. Every mutation marks the cwd dirty. The in-flight save's IIFE loops while dirty: clear flag, yield once to settle burst, snapshot, save. If a mutation arrives DURING the save's await, the loop runs again. Two registers within one microtask burst still coalesce to one save; mutations during the save no longer drop on the floor.
|
|
1285
|
+
- **No regression** — 284/284 vitest suite green; Phase 0 golden tests (50 parallel `Agent.create` calls produce valid JSON) still pass with the new loop.
|
|
1286
|
+
|
|
1287
|
+
### Added (chat assistant readiness — Phase 5 / Dogfood QA)
|
|
1288
|
+
|
|
1289
|
+
- **`examples/telegram-bot/src/dogfood.ts` + `dogfood-restart.ts`** — automated end-to-end validation against a REAL LLM (OpenRouter gemini-2.0-flash-001), no Telegram token required:
|
|
1290
|
+
1. Two distinct chats (`tg-dogfood-chat-A`, `tg-dogfood-chat-B`) on the same workspace cwd, each says "Remember: ..." and asks a follow-up. **PASS** in-process recall.
|
|
1291
|
+
2. Inspect persisted state: registry.json + per-agent messages.jsonl + sessions corpus dir all exist on disk.
|
|
1292
|
+
3. **Real process restart** via `spawnSync("npx tsx ...", ...)` runs a fresh node process. The subprocess `Agent.resume`s both chats — pulling registry.json + messages.jsonl + MEMORY.md from disk. Both LLMs answer with the persisted facts ("Vitest + alpha-7", "PostgreSQL + project-beta") after the restart boundary. **PASS** post-restart recall.
|
|
1293
|
+
4. Concurrent burst: 5 parallel sends into one chat produce strictly-alternating user/assistant records (16 records total). **PASS** mutex serialization.
|
|
1294
|
+
5. Sessions corpus: 11+ `.md` summaries on disk after all runs. **PASS** corpus seeding.
|
|
1295
|
+
- **Result**: 10 PASS / 0 WARN / 0 FAIL against real LLM. The chat assistant pattern works end-to-end with `@theokit/sdk` v1.0.0.
|
|
1296
|
+
|
|
1297
|
+
### Added (chat assistant readiness — Phase 4 / `examples/telegram-bot`)
|
|
1298
|
+
|
|
1299
|
+
- **New `examples/telegram-bot/`** — ~120 LoC `grammy` bot proving the chat assistant pattern end-to-end. One persistent agent per chat (`Agent.resume(`tg-${chatId}`)` first, fall back to `Agent.create` on `UnknownAgentError`). Memory enabled with `namespace: "telegram-bot"`, `userId: ctx.from.id`, `activeRecall.enabled`. A `/recall <query>` command uses `memory_search({ corpus: "sessions" })` to surface past conversations.
|
|
1300
|
+
- **README walkthrough** documents: BotFather setup, `.env` template, run, chat, `kill -9`, restart, chat-again-and-see-memory. Inspects `.theokit/agents/registry.json`, `.theokit/agents/<id>/messages.jsonl`, `.theokit/memory/MEMORY.md`, and `.theokit/memory/sessions/<runId>.md` to show what survived.
|
|
1301
|
+
- **EC-10 doc** — explicit callout that v1 supports exactly ONE SDK process per cwd; co-locating a bot + a standalone cron worker on the same workspace will race the registry.
|
|
1302
|
+
- **EC-11 doc** — explicit callout that group-chat `ctx.chat.id` is the group id (not the user); the example uses `ctx.from.id` to keep per-user memory isolated in groups.
|
|
1303
|
+
- **examples/README.md inventory** updated with the bot at the top of the list — it is the marquee proof for v1.0 chat assistant readiness.
|
|
1304
|
+
|
|
1305
|
+
### Added (chat assistant readiness — Phase 3 / ADR D20)
|
|
1306
|
+
|
|
1307
|
+
- **`memory_search({ corpus: "sessions" })` actually works.** Per-run summaries are written to `<cwd>/.theokit/memory/sessions/<runId>.md` after every finished run. IndexManager discovers them via the new `session-loader` and tags each chunk with `source: "sessions"`. The `corpus` filter in `memory_search` was already wired; this PR plugs in the data source.
|
|
1308
|
+
- **EC-9: only `status === "finished"` runs write summaries.** Cancelled, errored, or still-running runs leave no marker behind, so the recall corpus never returns fragments of failed conversations as authoritative context.
|
|
1309
|
+
- **EC-3: post-run sync is automatic.** `writeSessionSummary` triggers `IndexManager.sync()` in the background immediately after the markdown write. `memory_search({ corpus: "sessions" })` sees the new file on the next call without an ambiguous lazy trigger.
|
|
1310
|
+
- **Secret redaction.** Both user and assistant text run through the shared `redactSecrets` regex before persisting, matching the MEMORY.md write pipeline.
|
|
1311
|
+
- **`local-agent.ts` post-run hook moved INSIDE the send mutex.** The user-turn append, assistant-turn append, summary write, hooks executor, and `flushSessionWrites` all happen before the lock releases. `agent.dispose()` waits on the same mutex so it can never return before the summary lands on disk.
|
|
1312
|
+
- **`agent.dispose()` is now strict** — it acquires the per-agent send mutex before flushing, guaranteeing the in-flight `run.wait()` and post-run lifecycle complete before any caller's `await dispose()` resolves.
|
|
1313
|
+
- **New tests**: 8 golden cases under `tests/golden/memory/sessions-corpus.golden.test.ts` cover summary-on-finish, hit-on-sessions-search, memory-corpus excludes sessions, redaction, corrupt-file tolerance, EC-3 sync-after-wait, EC-9 cancelled-run, and EC-9 errored-run.
|
|
1314
|
+
|
|
1315
|
+
### Added (chat assistant readiness — Phase 2 / ADR D19)
|
|
1316
|
+
|
|
1317
|
+
- **Per-agent send mutex** keyed by `agent-send:${agentId}` (ADR D19). `LocalAgent.send` and `CloudAgent.send` now serialize end-to-end per agent: dispatch → `run.wait()` → assistant-turn append → disk flush all happen inside the lock. Two webhook calls hitting the same chat id can no longer interleave `appendSessionMessage` records mid-turn.
|
|
1318
|
+
- **Concurrent-distinct-agents stay parallel** (EC-8) — the mutex key is per-agentId. A parent agent's send and a subagent send (distinct ids) acquire different locks and run concurrently. Proven by the deadlock-free golden case.
|
|
1319
|
+
- **`agent.send()` returns the Run as soon as it dispatches**, but the mutex internally awaits completion + post-run hook + session flush before releasing. Streaming consumers keep their `run.stream()` access unchanged; the only observable difference is that a second `agent.send()` on the same agent now waits for the first to finish.
|
|
1320
|
+
- **New tests**: 5 golden cases under `tests/golden/agent/concurrent-send.golden.test.ts` cover two-concurrent-sends-serialize (strict role alternation), different-agents-stay-parallel, EC-8 subagent no-deadlock, sequential history linearity, and dispose-with-pending-send safety.
|
|
1321
|
+
|
|
1322
|
+
### Added (chat assistant readiness — Phase 1 / ADR D18)
|
|
1323
|
+
|
|
1324
|
+
- **Persistent session messages** at `<cwd>/.theokit/agents/<agentId>/messages.jsonl` (ADR D18). Append-only JSONL with one record per turn (`{role, text, at}`). `LocalAgent.send` now writes both the user turn and the assistant turn to disk; `Agent.resume()` hydrates the conversation back into memory on `initialize()`. Survives `kill -9` between sends.
|
|
1325
|
+
- **Opportunistic compaction** — when the JSONL exceeds 400 lines (2× the default `maxTurns=200`), the file is trimmed copy-on-write to the most recent 200 turns. Compaction also runs once during `dispose()` so a long-running chat does not leave 10k stale lines on disk.
|
|
1326
|
+
- **Race-free append + compaction** (EC-2) — both operations chain through a single per-`(agentId, cwd)` promise queue. Appends and compactions never race each other on the read+rename window. Reentry into `withCwdMutex("agent-send:...")` was rejected because Phase 2's send mutex uses the same key (non-reentrant) and would deadlock; the dedicated queue is the canonical serializer.
|
|
1327
|
+
- **Multi-line text** (EC-6) — `JSON.stringify` on append and `JSON.parse` per-line on read keep newlines, tabs, and embedded quotes intact across a restart.
|
|
1328
|
+
- **Crash-safe reader** (EC-7) — malformed lines (e.g., a half-written final record from a power loss) are skipped with a stderr warning. The reader never throws.
|
|
1329
|
+
- **New tests**: 10 golden cases under `tests/golden/runtime/agent-session-persistence.golden.test.ts` cover round-trip restart, compaction trim, EC-2 (concurrent appends + compaction across threshold), per-agent isolation, EC-6 (tricky text), EC-7 (partial last line), JSONL validity, hydrate-fills-cache, end-to-end Agent.create→send→resume conversation continuity, and direct 500-record compaction.
|
|
1330
|
+
|
|
1331
|
+
### Added (chat assistant readiness — Phase 0 / ADRs D17 + D21)
|
|
1332
|
+
|
|
1333
|
+
- **Persistent agent registry** at `<cwd>/.theokit/agents/registry.json` (ADR D17). Every `Agent.create / archive / update / delete` mutation triggers a coalesced, atomic write-through. The in-memory `Map` stays as the read-through cache; persistence is keyed per-cwd (EC-5). Survives `kill -9` + process restart.
|
|
1334
|
+
- **`Agent.resume()` falls back to disk** (ADR D21). On in-memory miss, `Agent.resume(id)` reads the persisted registry, validates the rehydrated entry (local agents check `local.cwd` still exists), and reconstructs the matching `LocalAgent` / `CloudAgent`. Throws `UnknownAgentError(code: "agent_rehydration_failed")` when the workspace path is missing.
|
|
1335
|
+
- **`Agent.create({ agentId })` collision** (EC-1) — pinning an `agentId` that already lives in the persisted registry now throws `ConfigurationError(code: "agent_id_already_exists")`. Forces the resume-first pattern that chat assistants need.
|
|
1336
|
+
- **Secret stripping on persist** — `apiKey`, MCP server `headers` / `env`, hook closures, and inline tool handlers are never written to disk. The allow-list mirrors the cloud-config-serializer (ADR D15).
|
|
1337
|
+
- **Corrupt-registry recovery** (EC-4) — invalid JSON / schema-version mismatch logs a stderr warning and falls back to `{}`. The next mutation overwrites the file with valid JSON.
|
|
1338
|
+
- **`replaceFileAtomic` multi-writer safe** — per-call unique `.<pid>.<rand>.tmp` suffix replaces the shared `.tmp` path. Removes a cross-process race that surfaced as `ENOENT` on rename when parallel writers raced on the same target.
|
|
1339
|
+
- **New tests**: 11 golden cases under `tests/golden/runtime/agent-registry-persistence.golden.test.ts` cover round-trip, cross-restart rehydration, stale-cwd rejection, secret stripping, concurrent-write integrity (50 parallel creates), archived-flag persistence, cloud-agent rehydration, EC-1 collision throw, EC-4 corruption recovery, and EC-5 per-cwd isolation.
|
|
1340
|
+
|
|
1341
|
+
### Changed (default model: composer-2 → free agentic model)
|
|
1342
|
+
|
|
1343
|
+
- **Default model id swept SDK-wide from the placeholder `composer-2` to `google/gemini-2.0-flash-exp:free`** (OpenRouter free tier, solid tool-calling for agentic flows).
|
|
1344
|
+
- **New `internal/runtime/default-model.ts`** exports `DEFAULT_AGENTIC_MODEL_ID` — single source of truth for the fallback model id, used by `cloud-agent.ts`, `local-run.ts`, and `internal/catalog/fixtures.ts`.
|
|
1345
|
+
- **`FIXTURE_MODELS` catalog** swapped to the new model id + display names ("Gemini 2.0 Flash (free)"). Golden snapshot `tests/golden/theokit/models.json` updated.
|
|
1346
|
+
- **All 30+ tests + golden JSON snapshots + 10+ doc pages + 3 examples** swept from `composer-2` to the new id. Public `docs.md` examples now show a runnable default.
|
|
1347
|
+
- Rationale: under the no-stubs-no-mocks-no-wired rule, a placeholder model id that maps to nothing real surfaces fixture mode to consumers who pass real keys. The new default is a real, free OpenRouter model — works out of the box with `OPENROUTER_API_KEY`, and per-call `model: { id: "..." }` override is unchanged.
|
|
1348
|
+
|
|
1349
|
+
### Changed (cloud pre-release guard — no-stubs-no-mocks-no-wired enforcement, round 2)
|
|
1350
|
+
|
|
1351
|
+
- **`CloudAgent.listArtifacts()` and `CloudAgent.downloadArtifact()`** now throw `ConfigurationError(code: "cloud_runtime_pre_release")` when invoked with a non-fixture API key. Previously they returned hardcoded fixture data (`buildFixtureArtifacts()` + `Buffer.from("fixture artifact content for ...")`) regardless of key — silently passing fixture content off as real PaaS responses.
|
|
1352
|
+
- **Fixture artifacts are now lazy-built** inside the fixture-mode branch of `listArtifacts/downloadArtifact` instead of eagerly seeded in the constructor. Real-key callers no longer carry fixture state.
|
|
1353
|
+
- **`CloudAgent` `summary` field** is now `"Cloud contract fixture"` only in fixture mode; real-key cloud agents register as `"Cloud agent"`.
|
|
1354
|
+
- **New `isFixtureMode()` private** centralizes the "are we in fixture mode?" check (matches the rule in `internal/fixture-mode.ts`: `theo_test_*` key + no `THEOKIT_API_BASE_URL`).
|
|
1355
|
+
- **New golden test** `cloud-prerelease-guard.golden.test.ts` (4 cases) locks the behavior: real keys get `cloud_runtime_pre_release`, fixture keys get fixture artifacts, path-traversal still rejected.
|
|
1356
|
+
|
|
1357
|
+
### Added (OpenRouter embedding adapter)
|
|
1358
|
+
|
|
1359
|
+
- **`openrouter` embedding adapter** — proxies through `https://openrouter.ai/api/v1/embeddings` (OpenAI-compatible shape). Caller selects the underlying model via the standard OpenRouter ids (`"openai/text-embedding-3-small"`, `"mistralai/mistral-embed"`, etc.). Honors `OPENROUTER_API_KEY` + `OPENROUTER_API_BASE_URL`.
|
|
1360
|
+
- **`MemorySettings.index.embedding.provider`** and **`DreamingSweepOptions.embedding.provider`** unions extended with `"openrouter"`.
|
|
1361
|
+
- **`examples/memory-dreaming`** now accepts `OPENROUTER_API_KEY` in addition to `OPENAI_API_KEY` / `MISTRAL_API_KEY`. Validated end-to-end: 6 facts → 4 semantic clusters (3 Vitest paraphrases grouped correctly).
|
|
1362
|
+
- **Stubbed-fetch test** in `multi-adapter.golden.test.ts` proves the OpenRouter adapter actually embeds (1536-dim vectors round-tripped from the OpenAI-compatible response shape).
|
|
1363
|
+
|
|
1364
|
+
### Changed (cheaper agentic chat model in examples)
|
|
1365
|
+
|
|
1366
|
+
- **`openai/gpt-4o-mini` → `google/gemini-2.0-flash-001`** in the 4 chat examples (`memory`, `memory-search`, `memory-get`, `active-memory`). ~33% cheaper input tokens at similar tool-calling fidelity for these recall scenarios. Pricing as of 2026-05.
|
|
1367
|
+
|
|
1368
|
+
### Removed (no-stubs-no-mocks-no-wired rule enforcement)
|
|
1369
|
+
|
|
1370
|
+
- **5 stub embedding adapters removed from the catalog**: `voyage`, `deepinfra`, `lmstudio`, `google`, `bedrock`. Files deleted; `MEMORY_EMBEDDING_ADAPTERS` now exposes only `openai` + `mistral` (the implementations that actually ship).
|
|
1371
|
+
- **`stub-adapter.ts` factory deleted** — no callers remain.
|
|
1372
|
+
- **LanceDB backend stub removed**. `MemoryBackend` is now `"sqlite-vec"` only. `IndexManager.open({ backend: "lancedb" })` no longer compiles; the runtime throw is gone.
|
|
1373
|
+
- **`ActiveMemoryOptions.mode` field removed** — the `"subagent"` member was a typed promise with no implementation. Active Memory was always running in `"search"` mode regardless of the option.
|
|
1374
|
+
- **`createStubRun` + `createHistoricalCloudRun` deleted**. `stub-run.ts` removed entirely. Two callers replaced with typed errors:
|
|
1375
|
+
- `Agent.getRun(runId)` now throws `UnknownAgentError(code: "run_not_found")` when the registry has no record (was: synthetic Run with `agentId: "agent-pending"`, `status: "finished"`).
|
|
1376
|
+
- `Agent.getRun(runId, { runtime: "cloud" })` now throws `ConfigurationError(code: "cloud_runtime_pre_release")` (was: stub historical Run).
|
|
1377
|
+
- `runCronJob` with orphan `agentId` now throws `UnknownAgentError(code: "agent_not_registered")` (was: stub Run stuck at `status: "running"`).
|
|
1378
|
+
- **`MemoryEmbeddingRuntime` public BYO surface removed** — `Memory.runDreamingSweep` no longer accepts `embedding: { runtime: ... }`. The only consumer was a demo fallback that itself has been removed. The type alias is gone from the public barrel.
|
|
1379
|
+
- **`makeLocalDemoRuntime` removed from `examples/memory-dreaming/`**. The example now fails fast when neither `OPENAI_API_KEY` nor `MISTRAL_API_KEY` is set.
|
|
1380
|
+
- **`@lancedb/lancedb` removed from `tsup.config.ts` external list** — no longer referenced by the bundle.
|
|
1381
|
+
|
|
1382
|
+
### Changed (no-stubs-no-mocks-no-wired rule enforcement)
|
|
1383
|
+
|
|
1384
|
+
- **Public `MemorySettings.index.embedding.provider`** narrowed from a 7-id union to `"openai" | "mistral"`. Consumers selecting a removed provider now get a TypeScript error at the call site instead of a runtime crash.
|
|
1385
|
+
- **`docs.md` and the docs site** updated to reflect the trimmed catalog and BYO-runtime removal.
|
|
1386
|
+
- **`examples/memory-dreaming/README.md`** removed the "future-work cron integration" claim. Scheduling consolidation is documented as a user concern (call `Memory.runDreamingSweep` from any scheduled context).
|
|
1387
|
+
- **`placeholderScript` renamed to `unusedFixtureScript`** in `real-local-run.ts` + `real-cloud-run.ts` with a clarifying comment — the FixtureScript shape is required by the base Run class but never consumed by the real-LLM path.
|
|
1388
|
+
- **`index-schema.ts` comment** corrected — `meta` table description matches what the code actually persists (embedding identity), and the `embeddings` virtual table is now documented.
|
|
1389
|
+
|
|
1390
|
+
### Changed (memory-system-openclaw-parity, Increment D — Dogfood follow-ups)
|
|
1391
|
+
|
|
1392
|
+
- **`local-agent.ts` decomposed** — memory glue (lazy IndexManager + tools cache + Active Memory breaker + summary cache) extracted to `local-agent-memory.ts`. Brings `local-agent.ts` under the G8 400-LoC cap.
|
|
1393
|
+
- **`legacyMemoryJsonPath` centralized in `memory/types.ts`** — removes the 9-line jscpd clone between `migration.ts` and `runtime/memory-store.ts`. Both now call the leaf-module helper.
|
|
1394
|
+
|
|
1395
|
+
### Added (memory-system-openclaw-parity, Increment C — Dogfood examples + Memory namespace)
|
|
1396
|
+
|
|
1397
|
+
- **`Memory` public namespace** exported from `@theokit/sdk` — `Memory.runDreamingSweep({ cwd, embedding })` lets users trigger consolidation outside of `agent.send()` (e.g. from a cron job handler).
|
|
1398
|
+
- **`MemoryEmbeddingRuntime` public type** — `embedding` now accepts either a built-in provider id (`{ provider, model? }`) OR a BYO runtime (`{ runtime: MemoryEmbeddingRuntime }`). Enables self-hosted/local embedding models and self-contained demos without external API creds. Mirrors OpenClaw's `EmbeddingRuntime` shape from ADR D3.
|
|
1399
|
+
- **4 new example apps** under `examples/`:
|
|
1400
|
+
- **`memory-search`** — LLM uses `memory_search` to find facts in MEMORY.md.
|
|
1401
|
+
- **`memory-get`** — LLM uses `memory_get` for bounded reads of `notes/*.md`.
|
|
1402
|
+
- **`active-memory`** — blocking pre-send recall injects an `<active-memory>` block.
|
|
1403
|
+
- **`memory-dreaming`** — `Memory.runDreamingSweep` consolidates duplicates + clusters + writes a dream-diary entry. Ships with a deterministic local-demo embedding fallback so the example runs without `OPENAI_API_KEY` / `MISTRAL_API_KEY`.
|
|
1404
|
+
- **`examples/README.md` inventory** updated with all 4 new examples marked ✅ Full.
|
|
1405
|
+
|
|
1406
|
+
### Added (memory-system-openclaw-parity, Increment B — Active Memory wire-up)
|
|
1407
|
+
|
|
1408
|
+
- **`memory.activeRecall.enabled`** runtime wire-up — when `true`, the SDK calls `runActiveMemory` before every `send()` and prepends the recall summary as a `<active-memory>` block to the LLM system prompt (priority 5 — above context/skills/memory).
|
|
1409
|
+
- **Per-agent `CircuitBreaker` + `ActiveMemoryCache`** — instantiated lazily on first send with active recall enabled. Keyed by `agentId` so multiple agents in the same process don't share state.
|
|
1410
|
+
- **Stub-server E2E proof** — captured Anthropic request body contains `<active-memory>` when enabled, and does NOT when disabled.
|
|
1411
|
+
- **Active recall config surface** — `queryMode` (`"message"` / `"recent"` / `"full"`), `timeoutMs`, `maxSummaryChars`, `persistTranscripts` are all wired from `MemorySettings.activeRecall` through to `runActiveMemory`.
|
|
1412
|
+
|
|
1413
|
+
### Added (memory-system-openclaw-parity, Increment A — Agent.create/send wire-up)
|
|
1414
|
+
|
|
1415
|
+
- **`MemorySettings.index`** public field — `{ tools?: boolean; backend?: "sqlite-vec" | "lancedb"; embedding?: { provider, model? } }`. When `memory.enabled === true` and `index.tools !== false`, the SDK lazily opens an `IndexManager` on first send + registers `memory_search` and `memory_get` with the LLM. Default backend is `sqlite-vec`; default embedding is none (FTS-only mode).
|
|
1416
|
+
- **`MemorySettings.activeRecall`** public field — reserved for Phase 7 wire-up (next increment). Type surface live today; runtime hookup pending.
|
|
1417
|
+
- **Stub-server E2E tests** prove memory tools appear in the captured Anthropic request body's `tools` array when memory is enabled, and are absent when disabled or opted-out via `index.tools: false`.
|
|
1418
|
+
- **Lazy embedding adapter resolution** — when `index.embedding.provider` is set, the SDK looks the adapter up via `MEMORY_EMBEDDING_ADAPTERS` and instantiates it on first send. Adapter failures degrade gracefully to FTS-only mode with a stderr warning.
|
|
1419
|
+
|
|
1420
|
+
### Added (memory-system-openclaw-parity, Phase 13)
|
|
1421
|
+
|
|
1422
|
+
- **Cross-validation report** at `.claude/knowledge-base/reviews/cross-validation/memory-system-openclaw-parity-xval-2026-05-16.md`. Verdict **APROVADO COM RESSALVAS**, zero BLOCKERs. All 10 ADRs cross-checked against shipped code; all 13 edge cases verified resolved or documented.
|
|
1423
|
+
|
|
1424
|
+
### Added (memory-system-openclaw-parity, Phase 12)
|
|
1425
|
+
|
|
1426
|
+
- **Backend selector** — `IndexManager.open({ backend: "sqlite-vec" | "lancedb" })`. Default `"sqlite-vec"`. `"lancedb"` reserved for Phase 12.1; throws `ConfigurationError(code: "memory_backend_not_implemented")` today (same KISS pattern as the Phase 11 stub embedding adapters).
|
|
1427
|
+
|
|
1428
|
+
### Added (memory-system-openclaw-parity, Phase 11)
|
|
1429
|
+
|
|
1430
|
+
- **`MEMORY_EMBEDDING_ADAPTERS` catalog** exports all 7 OpenClaw provider ids: `openai`, `mistral`, `voyage`, `deepinfra`, `lmstudio`, `google`, `bedrock`. Switching is one config field.
|
|
1431
|
+
- **Mistral adapter** fully implemented — `mistral-embed` (1024 dims) via shared OpenAI-compatible factory (`POST /v1/embeddings`). Honors `MISTRAL_API_KEY` + `MISTRAL_API_BASE_URL`.
|
|
1432
|
+
- **`createOpenAiCompatibleRuntime` shared factory** — extracted from the OpenAI adapter so any provider exposing the `{ model, input }` → `{ data: [{ embedding }] }` REST shape can plug in with a one-file thin wrapper.
|
|
1433
|
+
- **5 stub adapters** (Voyage, DeepInfra, LMStudio, Google, Bedrock) — metadata-only. `embed()` throws `ConfigurationError(code: "adapter_not_implemented")` so callers detect the gap without crashing the agent loop.
|
|
1434
|
+
|
|
1435
|
+
### Added (memory-system-openclaw-parity, Phase 10)
|
|
1436
|
+
|
|
1437
|
+
- **Wiki supplements** — files under `.theokit/memory/wiki/*.md` are read-only auxiliary corpora discovered by `discoverWikiFiles`. Indexed alongside `MEMORY.md` + `notes/*.md` with `source: "wiki"` tag in the `files` table.
|
|
1438
|
+
- **Corpus filtering in search** — `IndexManager.search(query, { sources: ["wiki"] })` returns only wiki hits; default search returns memory + wiki together. `memory_search` tool already honors `corpus: "wiki" | "memory" | "all"` per the OpenClaw schema from Phase 6.
|
|
1439
|
+
- **Source coercion on conflict** — `upsertFile` accepts an explicit `source` arg so reclassifying a file (moving a note into the wiki dir, etc.) updates the tag on next sync via `ON CONFLICT DO UPDATE SET source = excluded.source`.
|
|
1440
|
+
|
|
1441
|
+
### Added (memory-system-openclaw-parity, Phase 9)
|
|
1442
|
+
|
|
1443
|
+
- **`runDreamingSweep`** — cron-driven memory consolidation (ADR D7). Three phases mirror OpenClaw:
|
|
1444
|
+
- **light** — drop near-duplicate facts via cosine similarity (default threshold 0.95).
|
|
1445
|
+
- **REM** — single-link agglomerative clustering by cosine similarity (default threshold 0.75).
|
|
1446
|
+
- **deep** — write a `notes/dreamed-<ts>.md` per sweep with consolidated clusters.
|
|
1447
|
+
- **Dream-diary at `.theokit/memory/dream-diary.md`** — append-one-entry-per-sweep. Each entry carries timestamp + content hash (idempotency contract) + counts (`factsBefore`, `factsAfter`, `duplicatesRemoved`, `clustersCreated`, `notesWritten`).
|
|
1448
|
+
- **All dreaming writes are atomic (EC-3)** — `replaceFileAtomic` for notes and diary; per-cwd mutex held for the whole sweep so concurrent `Remember:` appends can't race.
|
|
1449
|
+
- **LLM narrative summarization deferred to Phase 9.1** — v1 ships deterministic clustering only. The interface is stable enough to plug an LLM-mediated `narrative.ts` later without changing the orchestrator.
|
|
1450
|
+
|
|
1451
|
+
### Added (memory-system-openclaw-parity, Phase 8)
|
|
1452
|
+
|
|
1453
|
+
- **CircuitBreaker** for Active Memory — `{ maxTimeouts: 3, cooldownMs: 60000 }` defaults. After N consecutive timeouts, `shouldSkip(key)` returns `true` until cooldown elapses. `recordSuccess` resets the counter immediately. Per-key isolation (multiple agents in one process don't share state).
|
|
1454
|
+
- **`ActiveMemoryCache`** — TTL-bounded LRU keyed by `sha256(userText + queryMode)`. Default TTL 15s, capacity 1000. Cache hits skip the IndexManager search entirely.
|
|
1455
|
+
- **`runActiveMemory` integration** — accepts optional `breaker` + `cache` + `agentKey` + `runId` + `persistTranscripts` + `cwd`. Breaker is consulted on entry and updated by status; cache stores results on the way out; transcripts written under `.theokit/memory/transcripts/active-memory/<runId>.json` when enabled.
|
|
1456
|
+
- **`persistActiveMemoryTranscript`** — JSON transcript persistence. Failures swallowed with stderr warning so transcript IO never crashes the agent run.
|
|
1457
|
+
|
|
1458
|
+
### Added (memory-system-openclaw-parity, Phase 7)
|
|
1459
|
+
|
|
1460
|
+
- **`runActiveMemory`** — blocking pre-send recall (ADR D6). Default `mode: "search"` calls `IndexManager.search` deterministically; `mode: "subagent"` (LLM-mediated curation) is stubbed for Phase 7.1. Query modes: `"message"` (only the user text), `"recent"` (user text + last N user turns, default 2), `"full"` (entire conversation). Hard timeout via `Promise.race` (default 15000ms) — returns `status: "timeout"` instead of throwing.
|
|
1461
|
+
- **Status discriminator** — `ActiveMemoryStatus` covers `"ok" | "timeout" | "skipped" | "no-recall" | "error"`. Caller-side dispatch is one switch statement.
|
|
1462
|
+
- **`ActiveMemoryPromptProvider`** at priority 5 (before context/skills/memory) — contributes the `<active-memory>` block via `SystemPromptAssemblyContext.activeMemorySummary`. Summary is XML-escaped (D9). Block omitted when summary is empty.
|
|
1463
|
+
- **Pipeline auto-registration** — `SystemPromptPipeline.default()` now wires 5 providers: ActiveMemory (5) → Context (10) → Skills (20) → Memory (30) → Base (100).
|
|
1464
|
+
|
|
1465
|
+
### Added (memory-system-openclaw-parity, Phase 6)
|
|
1466
|
+
|
|
1467
|
+
- **`memory_search` + `memory_get` tools** (ADR D5) with OpenClaw-mirrored JSON schemas and descriptions. `memory_search` returns ranked hits with `{ path, startLine, endLine, score, snippet, citation, source }`; `memory_get` returns bounded excerpts with truncation info.
|
|
1468
|
+
- **Path-traversal guard (EC-2)** — `memory_get` resolves the requested path against the memory root and throws `ConfigurationError(code: "memory_path_escapes_root")` if the resolved path escapes (e.g. `../../etc/passwd`).
|
|
1469
|
+
- **Result-size cap (EC-10)** — `memory_search` truncates the response when concatenated snippets exceed `maxTotalChars` (default 16384). Low-rank hits are dropped first; `truncated: true` marker on the payload.
|
|
1470
|
+
- **Agent-loop integration** — new `AgentLoopInputs.memoryTools?: MemoryToolSpec[]` field; `collectTools` appends memory tools alongside shell + MCP tools; `tool-dispatch` routes `origin === "memory"` calls through a dedicated handler that wraps JSON-encoded results.
|
|
1471
|
+
|
|
1472
|
+
### Added (memory-system-openclaw-parity, Phase 5)
|
|
1473
|
+
|
|
1474
|
+
- **sqlite-vec vector index** under the existing SQLite DB (ADR D2). `vec0` virtual table stores per-chunk embeddings; `vectorSearch` runs KNN with `MATCH` syntax. `loadSqliteVecExtension` wraps the native load with a typed `sqlite_vec_unavailable` ConfigurationError (EC-8) instead of a raw native exception.
|
|
1475
|
+
- **`meta` table tracks embedding identity** (`providerId` + `model` + `dimension`). On `IndexManager.open`, current adapter config is compared against stored meta — any mismatch drops the `embeddings` table and forces a full re-embed on next `sync()` (EC-1).
|
|
1476
|
+
- **Hybrid scoring** (ADR D4): FTS top-K + vector top-K merged, scores combined via `vectorScore * vectorWeight + textScore * textWeight` (defaults `0.6` / `0.4`, configurable per-call). Vector-only hits surface alongside FTS hits via a chunk-id outer join. `MemorySearchHit.vectorScore` exposed when vector backend is active.
|
|
1477
|
+
- **`IndexManager.open({ cwd, embedding? })`** — embedding-aware constructor. FTS-only still works when `embedding` is omitted; backend reported via `status().backend` as `"fts-only"` or `"hybrid"`.
|
|
1478
|
+
|
|
1479
|
+
### Added (memory-system-openclaw-parity, Phase 4)
|
|
1480
|
+
|
|
1481
|
+
- **`MemoryEmbeddingProviderAdapter` interface** (ADR D3) mirrors OpenClaw's contract: `id`, `defaultModel`, `transport`, `authProviderId`, `autoSelectPriority`, `create(options) → EmbeddingRuntime`. Adapters live under `internal/memory/adapters/`.
|
|
1482
|
+
- **OpenAI embedding adapter** (`openai-embedding.ts`) — native fetch only, no `openai` SDK dep. Batches at 100 texts/call. Retries once on 429 + 5xx with linear backoff (EC-9). Empty inputs skipped. Honors `OPENAI_API_KEY` + `OPENAI_API_BASE_URL`. Default model `text-embedding-3-small` (1536 dims).
|
|
1483
|
+
- **LRU embedding cache** keyed by `sha256(model+text)`. Max 5000 entries; oldest evicted first. Observable via `runtime.stats()` (`cacheHits` / `cacheMisses` / `httpCalls` / `retries`).
|
|
1484
|
+
|
|
1485
|
+
### Added (memory-system-openclaw-parity, Phase 3)
|
|
1486
|
+
|
|
1487
|
+
- **SQLite + FTS5 index** at `.theokit/memory/.index/memory.sqlite` (ADR D2). Schema: `files`, `chunks`, `chunks_fts` (FTS5 virtual table), `meta`. Triggers keep FTS in sync with `chunks` on insert/delete. WAL mode, foreign keys on. Backed by `better-sqlite3` (optional peer dep) — `node:sqlite` fallback path documented for Node 22.5+.
|
|
1488
|
+
- **`IndexManager.open / sync / search / status / close`** — full lifecycle. `sync()` walks `MEMORY.md` + `notes/*.md`, computes content hashes, skips unchanged files, deletes old chunks before reindexing changed ones. `search()` runs FTS5 BM25 ranking, returns `MemorySearchHit[]` with `path`, `startLine`, `endLine`, `score`, `textScore`, `snippet`, `source`, `citation` (path:startLine-endLine).
|
|
1489
|
+
- **Corrupt-DB recovery (EC-7)** — when opening fails with "malformed" / "not a database" / "encrypted" errors, the file is renamed to `<path>.corrupt-<ts>` (plus `-wal` and `-shm` siblings) and the schema is rebuilt from scratch. Diagnostic line emitted to stderr.
|
|
1490
|
+
|
|
1491
|
+
### Added (memory-system-openclaw-parity, Phase 2)
|
|
1492
|
+
|
|
1493
|
+
- **`chunkMarkdown`** splits markdown by heading boundaries + blank-line paragraph boundaries. Oversize paragraphs split on word-boundary nearest the cap (EC-6) — never mid-word. Each chunk carries `startLine` / `endLine` / `text` / `hash` (sha256) / optional `heading`.
|
|
1494
|
+
- **`readMemoryFileBounded`** — bounded read with `from` (1-indexed) + `lines` (default 200, mirrors OpenClaw's `DEFAULT_MEMORY_READ_LINES`). Returns `linesReturned`, `totalLines`, `remainingLines`, `truncated` (true when content remains past the slice). Foundation for Phase 6's `memory_get` tool.
|
|
1495
|
+
- Public types `MemoryChunk`, `MemoryReadResult`, `MemoryFileEntry` in `internal/memory/types.ts` mirroring OpenClaw's engine-storage shapes.
|
|
1496
|
+
|
|
1497
|
+
### Added (memory-system-openclaw-parity, Phase 1)
|
|
1498
|
+
|
|
1499
|
+
- **Markdown-first memory storage** (ADR D1) — facts now persist to `.theokit/memory/MEMORY.md` under a `## Facts` section, human-editable and git-friendly. The legacy JSON file (`.theokit/memory/<namespace>/<scope>-<userId>.json`) migrates one-shot on first read and is deleted afterward (ADR D8). Behavior is preserved: `readMemoryFacts` + `appendMemoryFact` keep their signatures.
|
|
1500
|
+
- **`replaceFileAtomic` + per-cwd mutex** — every append writes to `<file>.tmp`, fsync, rename; concurrent appends within the same process serialize through a per-`cwd` mutex (edge-case review EC-4). Multi-process safety is out of scope for v1 (documented).
|
|
1501
|
+
- **`MEMORY.md` section creation** preserves any free-form content the user added (edge-case review EC-5).
|
|
1502
|
+
|
|
1503
|
+
### Added (v1-completeness)
|
|
1504
|
+
|
|
1505
|
+
- **Memory auto-write-on-send** in the real LLM runtime (ADR D1/D2 of v1-completeness). When `memory.enabled === true` and the user message starts with `Remember: <fact>`, the SDK persists the fact via `appendMemoryFact` BEFORE the LLM call so durability is independent of the LLM. The same `<memory>` block recalls it on subsequent sends. Empty facts are skipped (EC-3); memory must be opt-in (EC-4). Fixture and real-runtime paths share `isMemoryWritePrompt` + `extractMemoryFact` helpers — no behaviour drift between modes.
|
|
1506
|
+
|
|
1507
|
+
### Changed (v1-completeness)
|
|
1508
|
+
|
|
1509
|
+
- **`Agent.resume(agentId)` now awaits `initialize()`** before returning the LocalAgent handle, matching `Agent.create` semantics. Previously, resumed agents had empty `context.snapshot()`, empty `skills.list()`, and unloaded hooks/plugins/subagents — silent breakage for users (and for Cron's internal use). The fix is monotone: callers that worked before still work; callers that were silently broken are now correct.
|
|
1510
|
+
- **Real LLM runtime now threads prior session history** into every `agent.send()`. `AgentLoopInputs.priorMessages` carries the user+assistant turns from previous sends on the same agentId; `initLoopContext` prepends them to the LLM message array before the current user message. Enables `Agent.resume(agentId)` to continue a conversation in the real runtime — previously the LLM saw only the latest message. Fixture path was unaffected; it already had session messages wired.
|
|
1511
|
+
- Removed the now-redundant `persistMemoryFact` wiring from `createFixtureRun`. The shared auto-write path in `LocalAgent.send` covers both fixture and real runtimes; the fixture's `beforeComplete` hook becomes a no-op (its `persistMemoryFact` parameter is unset). Eliminates the double-write hazard the auto-write feature would otherwise introduce in fixture mode (EC-2).
|
|
1512
|
+
|
|
1513
|
+
### Added (runtime-gaps fix)
|
|
1514
|
+
|
|
1515
|
+
- `SystemPromptPipeline` + `SystemPromptProvider` strategy pattern (ADR D8) — Context (priority 10), Skills (priority 20), Memory (priority 30), Base (priority 100) auto-injected as XML-tagged blocks into the LLM system prompt. Future blocks plug in by writing one new provider class.
|
|
1516
|
+
- `FallbackLlmClient` wraps the resolved provider chain. On `NetworkError` from the primary handshake, the SDK transparently retries with the next entry (ADR D2). Failover boundary at first event yield — mid-stream errors are NOT retried. Aborted signal between attempts short-circuits the chain (edge-case EC-3).
|
|
1517
|
+
- `SendOptions.onStep` / `onDelta` now fire in the real LLM agent loop (ADR D1) — `onStep` per completed assistant text turn and per tool call; `onDelta` per `text-delta` token. Callback errors are caught and logged, never crash the run.
|
|
1518
|
+
- `SkillsSettings.autoInject` (default `true`) — opt out of the `<skills>` block via `AgentOptions.skills.autoInject: false`.
|
|
1519
|
+
- `MemorySettings` (`AgentOptions.memory`) public type: `enabled`, `namespace`, `userId`, `scope`, `storePath`, `autoInject`. Recalled facts auto-inject as a `<memory>` block on every send.
|
|
1520
|
+
- `SystemPromptContext.memory` field — recalled facts exposed to custom `systemPrompt` resolvers (appended per the field-order compatibility contract).
|
|
1521
|
+
- `escapeBlockBody` helper (ADR D9) — every dynamic block body (context source, skill description, memory fact) is XML-escaped before embedding so workspace content containing literal `</context>` cannot break out of its block (prompt-injection defence).
|
|
1522
|
+
|
|
1523
|
+
### Added
|
|
1524
|
+
|
|
1525
|
+
- Initial package scaffold: dual ESM+CJS build via tsup 8, types-first `exports` map with sub-paths for `.`, `./cron`, and `./errors` (initial scaffold).
|
|
1526
|
+
- Public type contract from [`docs.md`](../../docs.md): `Agent`, `Run`, `SDKMessage`, `InteractionUpdate`, `ConversationTurn`, `McpServerConfig`, etc. (initial scaffold).
|
|
1527
|
+
- Error class hierarchy: `TheokitAgentError`, `AuthenticationError`, `RateLimitError`, `ConfigurationError`, `IntegrationNotConnectedError`, `NetworkError`, `UnknownAgentError`, `UnsupportedRunOperationError` (initial scaffold).
|
|
1528
|
+
- `Cron` namespace skeleton: `Cron.create()`, `Cron.list()`, `Cron.get()`, `Cron.delete()`, `Cron.enable()`, `Cron.disable()`, `Cron.run()` (manual fire), and scheduler control via `Cron.start()` / `Cron.stop()` / `Cron.status()`. Cron job type contract (`CronJob`, `CronCreateOptions`, `CronSchedulerStatus`, etc.) (initial scaffold).
|
|
1529
|
+
- Smoke test verifying public API is importable and stub methods reject with `ConfigurationError` (initial scaffold).
|
|
1530
|
+
- Context manager type contract: `ContextSettings`, `ContextSource`, `ContextSnapshot`, `SDKContextManager`. `SDKAgent.context?` exposes the manager when context is enabled via `AgentOptions.context`.
|
|
1531
|
+
- Provider routing type contract: `ProviderCapability`, `ProviderRoute`, `ProviderRoutingSettings`, `PluginsSettings`, `ResolvedProviderRoute`, `SDKProvidersManager`, `SDKProvider`. `SDKAgent.providers?` exposes the manager. `Theokit.providers.list()` stub for provider catalog reads.
|
|
1532
|
+
|
|
1533
|
+
### Changed
|
|
1534
|
+
|
|
1535
|
+
- License standardized to **Apache-2.0** (was MIT). Aligns all usetheo open-core pillars under a single license — see root `CLAUDE.md` strategic review of 2026-05-14.
|
|
1536
|
+
- `UnsupportedRunOperationError` now extends `TheokitAgentError` with `isRetryable: false` and stable `code: "unsupported_run_operation"`. Previously extended `Error` directly — old `instanceof TheokitAgentError` checks against this error now return `true`.
|
|
1537
|
+
- `RunOperation` union extended with `"listArtifacts"` and `"downloadArtifact"`. Agent-level operations can now be reported through `UnsupportedRunOperationError.operation`.
|
|
1538
|
+
|
|
1539
|
+
### Changed (runtime-gaps fix)
|
|
1540
|
+
|
|
1541
|
+
- Memory recall lifted from the fixture-only path into the shared agent path. A corrupted memory file degrades to "no facts loaded" with a stderr warning instead of crashing the run (edge-case review EC-4).
|
|
1542
|
+
- `FileContextManager` exposes a new internal `internalAssemblySnapshot()` so the system-prompt pipeline can read per-source token slices without the public `snapshot()` having to leak the same shape.
|
|
1543
|
+
|
|
1544
|
+
### Fixed
|
|
1545
|
+
|
|
1546
|
+
- 5 previously ⚠️ Partial example flows now work end-to-end against real providers: `examples/streaming-callbacks` (steps/deltas fire), `examples/provider-fallback` (`status=finished` after primary failover), `examples/context-manager` (model answers "8675309"), `examples/skills` (model lists `code-review, doc-writer`), `examples/memory` (model recalls the persisted fact via auto-injected `<memory>` block).
|
|
1547
|
+
- `setupSchema` of fixture providers no longer leaks env-var-name shaped strings (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, ...) that matched the hygiene regex. Schemas now use a generic `credential` property name (internal contract change; public shape unchanged).
|
|
1548
|
+
|
|
1549
|
+
### Implementation status (Phase 2 — real runtime)
|
|
1550
|
+
|
|
1551
|
+
- **Real cron scheduler** powered by `croner@^9.0.0`. `Cron.start()` installs a timer per enabled local job, `nextRunAt` is computed from the cron expression and timezone, jobs actually fire on schedule. `Cron.disable()` / `Cron.enable()` / `Cron.delete()` add/remove timers without losing the job state.
|
|
1552
|
+
- **Real hook execution** via `HooksExecutor`: `.theokit/hooks.json` is parsed into events (`preRun`, `postRun`, `preToolUse`, `postToolUse`, `stop`), each fires the configured command with the payload JSON over stdin. Non-zero exit codes deny the operation; JSON stdout can return `{"decision":"allow|deny|feedback","reason"|"feedback"}`. preRun denials throw `ConfigurationError("preRun hook denied execution")` from `agent.send()`. preToolUse denials short-circuit the tool with `exitCode: 126`.
|
|
1553
|
+
- **Real MCP client** for `stdio` (spawn + JSON-RPC over stdin/stdout) and `http` (fetch+JSON-RPC). Implements `initialize`, `tools/list`, `tools/call` per MCP 2024-11-05.
|
|
1554
|
+
- **Real shell tool** spawning `sh -c <command>` with stdout/stderr capture, SIGKILL-on-timeout, and a sandbox heuristic that refuses obvious unsafe commands when `local.sandboxOptions.enabled` is true.
|
|
1555
|
+
- **Real LLM provider clients** (Anthropic Messages SSE, OpenAI Chat Completions SSE, OpenRouter via the OpenAI shape). Use native `fetch` only — no SDK dependencies. Translate vendor SSE deltas into a provider-agnostic `LlmEvent` stream + `LlmFinish` accumulator.
|
|
1556
|
+
- **Real agent loop** orchestrates the LLM-tool-LLM cycle: system event → user event → LLM stream → assistant event → optional `tool_use` dispatch (with preToolUse + postToolUse hooks) → result fed back → next turn. Max 8 iterations by default.
|
|
1557
|
+
- **Real cloud Run** via Theo PaaS SSE: `POST /v1/agents/{id}/runs` with `accept: text/event-stream`, translates `status`, `assistant`, and `result` events into the SDK `SDKMessage` stream. Activates when a non-fixture API key + `THEOKIT_API_BASE_URL` are set.
|
|
1558
|
+
- **Streaming progressive events**: `Run.stream()` is now a true progressive AsyncGenerator — events arriving from the real runtime over time are yielded as soon as they're appended, not only at termination.
|
|
1559
|
+
- **Real local runtime activation**: when the API key is not a `theo_test_*` fixture key and at least one of `ANTHROPIC_API_KEY` / `OPENAI_API_KEY` / `OPENROUTER_API_KEY` is set, `LocalAgent.send()` routes through the real agent loop instead of fixture mode.
|
|
1560
|
+
|
|
1561
|
+
### Implementation status (Phase 1 — fixture-mode parity)
|
|
1562
|
+
|
|
1563
|
+
- `Agent.create()`, `Agent.send()` (both local + cloud), `Agent.resume()`, `Agent.list()`, `Agent.get()`, `Agent.listRuns()`, `Agent.getRun()`, `Agent.archive()`, `Agent.unarchive()`, `Agent.delete()` — implemented with deterministic fixture-mode responses for `theo_test_*` API keys.
|
|
1564
|
+
- `Theokit.me()`, `Theokit.models.list()`, `Theokit.repositories.list()`, `Theokit.providers.list()` — implemented; route to real HTTP when `THEOKIT_API_BASE_URL` is set, otherwise serve fixture data.
|
|
1565
|
+
- `Cron.create()` / `list()` / `get()` / `delete()` / `enable()` / `disable()` / `run()` — implemented with POSIX cron and shorthand validation, IANA timezone validation, and deterministic `nextRunAt` estimate.
|
|
1566
|
+
- File-based discovery from `.theokit/`: `agents/*.md` (subagents), `skills/<name>/SKILL.md`, `plugins/<name>/plugin.json`, `mcp.json`, `hooks.json`, `context.json`, `cron/jobs.json`, `memory/<scope>.json`.
|
|
1567
|
+
- Run lifecycle: `stream()` (AsyncGenerator of SDKMessage), `wait()`, `cancel()`, `conversation()`, `onDidChangeStatus()`. Status machine: `running → finished | error | cancelled`.
|
|
1568
|
+
- Cloud runtime adapter calls Theo PaaS when `THEOKIT_API_BASE_URL` is set; otherwise emulates PaaS via fixture mode (CREATING / RUNNING / FINISHED status events, git metadata on result, artifact listing/download).
|
|
1569
|
+
- Memory subsystem: file-backed store under `.theokit/memory/`, redacted public surface, namespace/scope keying.
|
|
1570
|
+
- Skills, plugins, MCP, hooks, subagents, providers, context — public managers and file-based loaders.
|
|
1571
|
+
- Quality Gates G1–G10 all green: typecheck, lint+format (Biome), publint, attw, smoke + roadmap tests (136/136), knip (dead code), depcruise (cycles), G8 LoC ≤ 400, G9 cognitive complexity ≤ 10, G10 jscpd 0 clones.
|