@vantageos/agent-engine 0.1.0-alpha.0

package/README.md

@@ -0,0 +1,257 @@
+# @vantageos/agent-engine
+
+Reusable Convex component (pattern `@convex-dev/*`, house precedent
+`vantage-doc-forge`): durable agent engine. **An agent is a DATA
+AGGREGATE** — root sheet + child tables (skills, recursive subagents,
+connections, habits, sandbox config, per-tool approvals) — **never compiled
+code**. Instant creation, zero deployment. Built on `@convex-dev/workflow`
++ `@convex-dev/workpool` (+ `@convex-dev/agent` for threads/context).
+
+Consumers, in order: VantagePeers (Sigma, I1 long-task survival),
+EveVantage (Hestia), VantageCRM (Theta).
+
+Bible: `elpiarthera/ElPi-Corp` (private) pinned @ `60b69bfc` —
+`analysis/evevantage/SYNTHESE.md` §4/§7,
+`analysis/evevantage/i0-research-convex-components.md`,
+`analysis/evevantage/mvp-evevantage.md` §2. Upstream reference:
+https://github.com/vercel/eve (reasoning source, never an opaque
+dependency).
+
+## EVE PARITY BY DESIGN
+
+Non-negotiable rule (SYNTHESE §4.e-ter): slots, contracts and vocabulary
+mirror `vercel/eve`. Every deliberate divergence is marked **"notre
+ligne"** below, in this table and in the code comments next to the field.
+
+| Eve slot / concept | Source (vercel/eve) | Our table / contract | Notes |
+|---|---|---|---|
+| Authored agent = a folder, `compileAgent` | `packages/eve/src/compiler/compile-from-memory.ts:86-87` | `agents` root row | **notre ligne** — we never compile; the row IS the agent |
+| `name` derived from folder path (no `name` field) | `agent-definition.ts:281-284` | `agents.name` | **notre ligne** — required field, since we have no folder |
+| `description` | `agent-definition.ts:291` | `agents.description` | direct parity |
+| `instructions` (`instructions.md`) | `shared/instructions-definition.ts:1-18` | `agents.instructions` | direct parity |
+| `model` (required) | `agent-definition.ts:304` | `agents.model` | direct parity |
+| reasoning effort | `agent-definition.ts:315-319` | `agents.reasoningEffort` | direct parity |
+| model provider / API key | not in the Eve repo (env-only) | `agents.modelProvider` + `modelKeys` table | **notre ligne** — BYOK (mvp §2.3), server-only table split from `agents` so list/get can never leak the key |
+| `limits.maxSubagents`, `maxInputTokensPerSession` | `agent-definition.ts:163-207` | `agents.maxSubagents`, `agents.maxInputTokensPerSession` | direct parity |
+| `dynamicSkills` / skills | `manifest.ts:722-725`, `agent-definition.ts:139` | `skills` child table | direct parity |
+| `dynamicTools` / tools | `manifest.ts:722-725` | resolved at read-time from `connections` + `toolApprovals` | direct parity |
+| Tool `approval: Approval<...>` (`always`/`once`/`never`) | `public/definitions/tool.ts:117-125` | `toolApprovals` child table | direct parity |
+| MCP/OpenAPI connections (`defineMcpClientConnection`) | `public/definitions/connections/mcp.ts:23-48` | `connections` child table | direct parity. Our VantagePeers tools (memory, tasks, messages, notes) are wired here as an MCP connection, on by default — **notre ligne**, mvp §2.2 |
+| Declared subagents, root-only `agent` tool | https://eve.dev/docs/subagents | `subagents` child table (`parentAgentId` -> `childAgentId`, recursive) | direct parity. **Isolation strict, "inherits nothing from the root's authored slots"** — a subagent row only declares the link + exposed tool name; it never inlines the parent's skills/connections/approvals |
+| `sandbox` | `agent-definition.ts:150-160` | `sandboxConfig` child table (Vercel Sandbox, egress policy, allowlist) | direct parity, I0 §8 |
+| `schedules` / hooks -> "habits" | `manifest.ts:143`, `:726` | `habits` child table (dynamic cron per agent/org, `@convex-dev/crons`) | **notre ligne (naming)** — Eve calls these schedules/hooks; we call them habits (product decision) and back them with runtime-registered crons instead of static config |
+| Session/turn resolver boundaries (`ALLOWED_DYNAMIC_INSTRUCTION_EVENTS = {session.started, turn.started}`) | `dynamic-tool-definition.ts:40-46` | `sessionEvents.name` union (`session.started`, `turn.started`, ...) + `engine/resolvers.ts:resolveManifest` | direct parity — same event names, same "reread the aggregate, never cache" contract |
+| `subagents` used for inter-agent collaboration | eve docs | **not used** — collaboration goes through VantagePeers MCP tools instead | **notre ligne** — explicit Laurent decision (mvp-evevantage.md §2.5): "la collaboration entre agents n'arrive pas par les `subagents` d'eve" |
+| `outputSchema`, `compaction`, `modelOptions`, `channels` | `agent-definition.ts:124-146,313-330`, `manifest.ts:56-77` | out of scope this lot | left for a later I-cycle, documented as deliberately deferred (mvp §2.5) |
+
+## Data aggregate — schema (`convex/schema.ts`)
+
+Root sheet: `agents`.
+
+Child tables (>= 7, every index starts with `orgId`):
+
+1. `skills` — contextual procedures
+2. `subagents` — recursive declared delegation, strict isolation
+3. `connections` — MCP/OpenAPI (VantagePeers tools included)
+4. `habits` — dynamic crons (habits, notre ligne naming)
+5. `sandboxConfig` — Vercel Sandbox settings
+6. `toolApprovals` — `always` / `once` / `never` per tool
+7. `modelKeys` — BYOK, server-only, split from `agents`
+
+Engine tables: `sessions`, `sessionEvents`, `awaitEventTimeouts`.
+
+**Hard rule**: `orgId` is derived server-side from the authenticated
+identity (`convex/lib/auth.ts:requireOrgId`, reads the Clerk `org_id`
+claim) — it is never accepted as a client argument, and it is the FIRST
+field of every `.index(...)` call. Deny by default: cross-org reads return
+`null` / cross-org writes throw, indistinguishable from "not found"
+(`convex/lib/auth.ts:assertSameOrg`).
+
+## Engine — durable sessions (`convex/engine/`)
+
+- `engine/workflows.ts` — `WorkflowManager` from `@convex-dev/workflow`.
+  The session loop is exported as `sessionWorkflowV1`. **Versioned
+  workflows / drain-before-deploy discipline** (workflow#35): a breaking
+  change to the loop ships as `sessionWorkflowV2`, never an in-place edit
+  of `V1` while sessions still reference it.
+- `engine/resolvers.ts` — `resolveManifest`, the dynamic resolver. Rereads
+  the full agent aggregate (skills, tool approvals, connections) on every
+  call; callers invoke it at `session.started` / `turn.started` boundaries
+  only, mirroring eve's `ALLOWED_DYNAMIC_INSTRUCTION_EVENTS`.
+- `engine/sessions.ts` — `start` / `recordTurnStarted`, the mutations that
+  write the named `sessionEvents` rows Eve-style.
+- `engine/awaitEventTimeout.ts` — **self-built `awaitEvent` timeout**
+  (workflow#177: Convex ships no native timeout for `step.awaitEvent`).
+  `schedule` writes an `awaitEventTimeouts` row and schedules an
+  `internalMutation` (`fire`) via `ctx.scheduler.runAfter`; if the real
+  event arrives first, `cancel` cancels the scheduled function. `fire`
+  inserts a blank `sandbox.timeout` `sessionEvents` row so the awaiting
+  workflow step wakes up regardless.
+
+`ai` is pinned to `6.0.35` (peer dependency of `@convex-dev/agent@0.6.4`,
+agent#79 warns the v5->v6 migration broke other consumers — we track
+majors deliberately, never float). Counters/rollups, when needed, go
+through `@convex-dev/sharded-counter` / `@convex-dev/aggregate` — no
+canary/alpha third-party dependency for a guardrail (I0 §7.4).
+
+## Long-task survival (I1) — `engine/durableJob.ts`
+
+SYNTHESE.md §5/§7: "le plafond de durée des actions Convex est notre
+plafond aujourd'hui, sur tous nos produits" -- `durableJobs` is the
+GENERIC (shape-agnostic, unlike `sessions`) primitive that lets a
+consumer's task survive past that ceiling by expressing it as N
+independently-bounded steps. It **ENVELOPS** `@convex-dev/workflow` (via
+the same `WorkflowManager` + versioned-workflow discipline as
+`engine/workflows.ts`), it does not reimplement durable execution.
+
+Contract (see `convex/engine/durableJob.ts` for the full commentary and
+`examples/vantage-peers-long-task/` for the worked integration a consumer
+reads without needing to know our internals):
+
+- `start(kind, stepFunctionHandle, totalSteps, maxAttemptsPerStep)` —
+  enqueue a job driven by a step function the CALLER owns (component
+  isolation: we invoke it by opaque `FunctionHandle`, we never read the
+  caller's own tables).
+- `getStatus(jobId)` — poll `status` / `cursor` / `lastError` /
+  `abandonedAtStep`.
+- `cancel(jobId)` — best-effort stop.
+- **Pause at step boundary**: one loop iteration == one workflow step ==
+  one durability boundary (`@convex-dev/workflow` persists the journal
+  after each step returns).
+- **Deterministic replay**: `runStepInternal` no-ops for any `stepIndex <
+  job.cursor`, so a resumed workflow's journal replay never re-runs a
+  completed step.
+- **Abandon after N**: the caller's step function is retried up to
+  `maxAttemptsPerStep` times inside one workflow step; once exhausted the
+  job is marked `abandoned` with `abandonedAtStep` + `lastError`, and no
+  further step runs.
+
+## `convex/_generated/` freshness -- two gates, not one
+
+`convex/_generated/api.d.ts` carries an "AUTOMATICALLY GENERATED" header
+but nothing stops a hand-edit of its *content* (as opposed to adding or
+removing an entire module) from being committed as-is. That gap was
+measured for real on PR #2 (T2-BIS): a manual edit of a single existing
+line (`FunctionReference<any, "public">` flipped to `"internal">`) passed
+the original structural gate with `OK`, because that gate only diffs the
+*list* of modules present under `convex/` against the modules imported by
+`api.d.ts` -- it never recomputes a byte.
+
+Two gates now exist, and they check different things:
+
+1. **Structural, offline, secret-free** --
+   `scripts/check-generated-freshness.mjs` (`npm run check:generated`).
+   Runs in CI (`.github/workflows/ci.yml`). Catches a function file
+   added/renamed/removed without regenerating bindings. **Does NOT catch a
+   hand-edit of existing generated content.**
+2. **Content, requires a live deployment** --
+   `scripts/check-generated-content.mjs` (`npm run
+   check:generated:content`). Actually runs `npx convex codegen` against
+   the dev deployment and diffs `convex/_generated/` byte-for-byte
+   (`git diff --exit-code`). Catches everything, including a single hand-
+   edited character.
+
+   **Three-state exit contract** (never two -- a two-state gate cannot
+   distinguish "I measured, it's clean" from "I could not measure at
+   all", and collapsing those is either a silent fail-open or a red that
+   proves nothing; see flotte rule `three-state-verdict.md`):
+
+   | Exit | Meaning | When |
+   | ---- | ------- | ---- |
+   | `0` | **PASS** | A real measurement was taken (`npx convex codegen` ran) and `convex/_generated/` is byte-identical to its output. |
+   | `1` | **VIOLATION** | A real measurement was taken and it found >=1 byte of divergence -- hand-edited, stale, or generated against a different schema. |
+   | `2` | **REFUSAL TO JUDGE** | `npx convex codegen` itself could not run (no live, authenticated Convex deployment reachable -- `CONVEX_DEPLOYMENT` / `.env.local` missing or invalid). No measurement was taken; this is neither a clean bill of health nor a proven violation. |
+
+### CI cannot run the content gate -- explicit limit, not a rounding error
+
+`npx convex codegen` has no offline mode: it always needs
+`CONVEX_DEPLOYMENT` / a deploy key against a live deployment. This repo
+refuses to store that as a CI secret (plan B, PR #1). **Consequence,
+stated plainly: CI cannot detect a hand-edited `convex/_generated/`
+content change.** CI only runs the structural gate. The content gate is
+the only thing that catches a content-level tamper, and it can only run
+where the secret lives: **locally, pre-commit**, via `.env.local`.
+
+To install the pre-commit gate once per clone:
+
+```
+git config core.hooksPath scripts/git-hooks
+```
+
+This points git at the versioned hook `scripts/git-hooks/pre-commit`
+(plain git hook, not a Claude hook -- Claude hooks under `.claude/hooks/`
+are Pi-owned only). The hook no-ops unless a staged file is under
+`convex/`, and then runs `npm run check:generated:content`, reading its
+exit code explicitly rather than treating it as a boolean:
+
+- `0` (PASS) -- commit proceeds.
+- `1` (VIOLATION) -- commit is **blocked**, message names the divergence.
+- `2` (REFUSAL TO JUDGE, e.g. `.env.local` deployment secret missing or
+  invalid) -- commit is **also blocked**, with a distinct message that
+  says explicitly this is *unverified*, not a proven violation. This is a
+  deliberate fail-closed choice: an unverified change to `convex/_generated/`
+  content must never slip through silently just because the local
+  instrument (a live Convex deployment) was unavailable. The trade-off is
+  that a developer without a working `.env.local` cannot commit to
+  `convex/**` at all until they fix their local deployment credentials --
+  judged the safer failure mode than a fail-open that would let a hand-
+  edited, un-regenerated `convex/_generated/` through whenever the secret
+  happens to be missing.
+
+**If `core.hooksPath` is never configured, this protection does not
+exist on that clone.** Nothing currently makes that configuration
+mandatory or automatic -- that decision (e.g. via a repo bootstrap script
+that all contributors run) is Pi's to make, not this PR's.
+
+## Tests (TDD, RED then GREEN)
+
+- `convex/agents.test.ts` — org isolation (family 1)
+- `convex/engineResolvers.test.ts` — dynamic resolver, no-redeploy (family 2)
+- `convex/engineAwaitEventTimeout.test.ts` — awaitEvent timeout wakes the
+  workflow when the real event never arrives (family 3)
+- `convex/engineDurableJob.test.ts` — I1 long-task survival contract
+  (family 4): a 25-step job survives to `completed` via the component; a
+  replayed already-completed step is a proven no-op (resume after forced
+  interruption); a step that always fails is retried and the job is
+  `abandoned` (never touching the next step); a step that fails once then
+  succeeds still reaches `completed`; cross-org `getStatus` is denied.
+
+Run: `npx vitest run` (or `npm test`).
+
+## Packaging & publication (T2-TER)
+
+`@vantageos/agent-engine` is published to npm under the `alpha` dist-tag
+(`npm view @vantageos/agent-engine version` / `dist-tags`, read from the
+registry after publish, never from `npm publish`'s own exit status). The
+package exposes a resolvable `"./convex.config"` subpath (types + JS,
+built by `npm run build` from `convex/component/convex.config.ts` via
+`tsc -p tsconfig.build.json` into `dist/component/`), so a consumer's
+`convex/convex.config.ts` can do:
+
+```ts
+import { defineApp } from "convex/server";
+import agentEngine from "@vantageos/agent-engine/convex.config";
+
+const app = defineApp();
+app.use(agentEngine);
+export default app;
+```
+
+**Known remaining gap, stated here where it is read (not only in a
+message):** `convex/component/convex.config.ts` currently declares only
+the component's identity (`defineComponent("agentEngine")`) and its
+sub-component composition (workflow + 2 workpools + agent). It does not
+yet re-host this repo's schema/functions (`convex/schema.ts`,
+`convex/agents.ts`, `convex/skills.ts`, `convex/engine/**`) inside the
+component boundary, so `components.agentEngine.engine.durableJob.*` etc.
+are not yet callable from a consumer through the published component --
+see `examples/vantage-peers-long-task/README.md` "Packaging note" for the
+full detail and exactly what remains.
+
+## Status
+
+T1 (schema + durable session engine, PR #1) merged @
+`9f3401ba63e6e663af95e9e4f5da42bc30b3550b`. I1 (long-task survival
+contract, `engine/durableJob.ts`) — not merged, Eta reviews, merge on Pi
+token. T2-TER (this PR, npm packaging + component identity + publication)
+— see "Packaging & publication" above; **not merged**.

package/dist/component/convex.config.d.ts

@@ -0,0 +1,2 @@
+declare const component: import("convex/server").ComponentDefinition<any, {}>;
+export default component;

package/dist/component/convex.config.js

@@ -0,0 +1,42 @@
+import { defineComponent } from "convex/server";
+import workflow from "@convex-dev/workflow/convex.config";
+import workpool from "@convex-dev/workpool/convex.config";
+import agent from "@convex-dev/agent/convex.config";
+// @vantageos/agent-engine — Convex COMPONENT definition (pattern
+// `@convex-dev/*`, verified against the published shape of
+// `@convex-dev/workflow` — `defineComponent(name)` + a `convex.config.ts`
+// at the root of the *package that gets published*, not merely at the
+// root of the git repo).
+//
+// This is what a consumer registers via:
+//   import agentEngine from "@vantageos/agent-engine/convex.config";
+//   app.use(agentEngine);
+//
+// NOTRE LIGNE / KNOWN GAP (say it here, at the point of read, not only in
+// a message — T2-TER instruction): this file declares agent-engine's
+// COMPONENT IDENTITY and its sub-component composition
+// (workflow + 2 named workpools + agent), which is exactly what makes the
+// `@vantageos/agent-engine/convex.config` import RESOLVE for a consumer
+// and what makes `app.use(agentEngine)` build without throwing.
+//
+// It does NOT yet re-host this repo's schema (convex/schema.ts) and
+// functions (convex/agents.ts, convex/skills.ts, convex/engine/**) inside
+// this component boundary, so a consumer cannot yet call
+// `components.agentEngine.agents.create` / `components.agentEngine.engine
+// .durableJob.start` etc. through the published component -- those
+// functions still only exist in this repo's OWN dev/test deployment
+// (convex/convex.config.ts, defineApp, used by `npx convex dev` and
+// convex-test). That migration (moving schema.ts + agents.ts + skills.ts +
+// engine/** under convex/component/, regenerating this component's OWN
+// convex/component/_generated/ against a real component-hosting
+// deployment, and re-verifying every existing test) is real, non-trivial
+// work that was NOT attempted in this packaging pass -- doing it
+// unverified would have violated Day 128 (never weaken/bypass a test to
+// make it pass) by shipping schema/function code with no working
+// convex-test harness behind it. It is the next, separately-scoped task.
+const component = defineComponent("agentEngine");
+component.use(workflow);
+component.use(workpool, { name: "agentTurnPool" });
+component.use(workpool, { name: "wakeupPool" });
+component.use(agent);
+export default component;

package/package.json

@@ -0,0 +1,50 @@
+{
+  "name": "@vantageos/agent-engine",
+  "version": "0.1.0-alpha.0",
+  "description": "Reusable Convex component: durable agent engine. Agent = data aggregate (root sheet + child tables), never compiled code. Built on @convex-dev/workflow + @convex-dev/workpool (+ @convex-dev/agent). Eve parity by design.",
+  "type": "module",
+  "license": "UNLICENSED",
+  "publishConfig": {
+    "access": "public",
+    "tag": "alpha"
+  },
+  "files": [
+    "dist",
+    "README.md"
+  ],
+  "exports": {
+    "./package.json": "./package.json",
+    "./convex.config": {
+      "types": "./dist/component/convex.config.d.ts",
+      "default": "./dist/component/convex.config.js"
+    },
+    "./convex.config.js": {
+      "types": "./dist/component/convex.config.d.ts",
+      "default": "./dist/component/convex.config.js"
+    }
+  },
+  "scripts": {
+    "dev": "convex dev",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit",
+    "build": "tsc -p tsconfig.build.json",
+    "prepublishOnly": "npm run build",
+    "check:generated": "node scripts/check-generated-freshness.mjs",
+    "check:generated:content": "node scripts/check-generated-content.mjs"
+  },
+  "dependencies": {
+    "@convex-dev/agent": "0.6.4",
+    "@convex-dev/workflow": "0.4.4",
+    "@convex-dev/workpool": "0.4.7",
+    "ai": "6.0.35",
+    "convex": "1.42.1",
+    "convex-helpers": "0.1.103"
+  },
+  "devDependencies": {
+    "@edge-runtime/vm": "5.0.0",
+    "convex-test": "0.0.54",
+    "typescript": "5.7.3",
+    "vitest": "2.1.9"
+  }
+}