@entrinsik/vite-plugin-informer 2.7.0-beta.0 → 2.10.0

package/README.md

@@ -0,0 +1,129 @@
+# @entrinsik/vite-plugin-informer
+
+Vite plugin and deploy tool for Informer App development: a local dev server wired to a live Informer, and a deploy pipeline that assembles your project and ships it to an App's library.
+
+## Quick start
+
+```bash
+npm create vite@latest my-app -- --template react
+cd my-app
+npm install -D @entrinsik/vite-plugin-informer
+npx informer-init                          # wires vite config, informer.yaml, .env, deploy scripts
+npm run dev                                # local dev against the Informer in .env
+npm run deploy                             # assemble + upload + deploy
+```
+
+## What a deploy ships
+
+The assembler uploads your built frontend plus these source trees, matching how an App's library is structured in Informer:
+
+| Folder | Becomes |
+|---|---|
+| `server/` | API routes, dispatched under the App's `/api` surface |
+| `webhooks/` | Publicly callable webhook routes |
+| `tools/` | AI tools, private to the App's agents |
+| `mcp/` | Tools exposed over the App's MCP endpoint |
+| `channels/` | Live channel join/leave handlers |
+| `migrations/` | Workspace database migrations, run in order on deploy |
+| `embeddings/` | Declarative embedding use cases (see below) |
+| `lib/`, `shared/` | Modules importable by the trees above |
+
+`informer.yaml` declares dependencies, raw API access, roles, and (optionally) a minimum Informer version.
+
+## Server compatibility
+
+A deploy begins by asking the target server what it is (`GET /api/about`, one unauthenticated request) and adapts to the answer, so one version of this plugin can drive a fleet on mixed releases.
+
+Everything below needs **Informer 2026.1.3**: App Channels (`channels/`, the `channels:` block, `broadcast()`), the embedding pump (`embeddings/`, `embed()`), staged uploads/downloads (`uploads`/`downloads` on the handler bag, `__INFORMER__.upload()`), and the `platform` descriptor itself.
+
+Against an older server the CLI:
+
+- **Leaves `channels/` and `embeddings/` on your disk.** Those releases don't recognise the folders as server-side, so anything uploaded there is served to any App viewer — your workspace SQL and channel authorization logic included. The rest of the App deploys normally; the warning names what was skipped and why.
+- **Names every feature that would be inert**, so a green deploy can't be mistaken for a working one.
+- **Enforces `requires: { informer: … }` itself.** Releases before 2026.1.3 ignore the block, so without this an App that declared a floor would deploy clean onto a server that can't run it. The deploy is refused before the App's library is touched.
+
+A server whose version isn't comparable (a `dev` build, or an `/about` behind a proxy) is treated as current and warned about — gating it would break deploys to builds made from source.
+
+Feature-detect at runtime with optional chaining, since `platform` is absent entirely before 2026.1.3:
+
+```javascript
+if (platform?.capabilities?.channels) broadcast('tickets', { … });
+```
+
+A marketplace publish is unaffected: the archive always carries every source tree, because the server that installs it isn't known at publish time.
+
+## Embeddings
+
+Apps can maintain vector embeddings over their own workspace data declaratively. You ship an `embeddings/` folder with **one file per use case**; the Informer platform acts as an *embedding pump* that asks your App what is pending, chunks and embeds the content in billed batches, and hands the vectors back for your App to store in its own workspace tables. You never call an embedding provider yourself, and the platform never keeps a copy of your corpus — your own tables are the ledger.
+
+Each use case file exports `GET` and `POST` (a missing handler fails the deploy) and, optionally, a `config` literal; omitted keys take the defaults below, and a use case with no triggers in its config only runs on deploy and on a manual run:
+
+```javascript
+// embeddings/tickets.js
+export const config = { chunking: 'prose', on: ['ticket.created'], cron: '0 * * * *', revision: 1 };
+
+// GET: the rows that still need embedding. The query IS the watermark:
+// never embedded, stale, or written under an older revision.
+export async function GET({ query, batch }) {
+  return await query(`
+    SELECT t.id, t.subject || ' ' || t.body AS content
+    FROM tickets t
+    LEFT JOIN ticket_embeddings e ON e.ticket_id = t.id AND e.seq = 0
+    WHERE e.ticket_id IS NULL OR e.embedded_at < t.updated_at OR e.revision <> $2
+    ORDER BY t.id
+    LIMIT $1
+  `, [batch.limit, batch.revision]);
+}
+
+// POST: store the finished vectors wherever your migrations put them.
+export async function POST({ query, batch }) {
+  for (const doc of batch.docs) {
+    await query('DELETE FROM ticket_embeddings WHERE ticket_id = $1', [doc.id]);
+    for (const c of doc.chunks) {
+      await query(
+        'INSERT INTO ticket_embeddings (ticket_id, seq, embedding, content, revision) VALUES ($1, $2, $3::vector, $4, $5)',
+        [doc.id, c.seq, JSON.stringify(c.embedding), c.content, batch.revision]
+      );
+    }
+  }
+  return { stored: batch.docs.length };
+}
+```
+
+How it works, briefly:
+
+- **Your storage is the ledger.** Because `GET` anti-joins your own embedding table, the pump is idempotent and crash-tolerant: a run that dies mid-way finds the same rows pending next time.
+- **Triggers converge.** Deploy pokes each use case (initial backfill), `emit()` events matching `on:` poke, `cron` desugars into a scheduled poke, and the platform sweep runs whatever is poked. A manual run claims the same single-flight lease directly, so nothing double-embeds.
+- **Failures are bounded.** A run that fails for a reason the next attempt might not hit (a provider blip, a handler timeout) retries with a doubling delay and is parked after five consecutive failures; a run that cannot succeed as-is (a rejected key, an exhausted budget, a contract violation) is parked at once. Every park keeps the error on the use case's status row; the next trigger gives it one more attempt, and a successful run resets the count.
+- **`revision` is the invalidation.** The string the pump hands you couples your config (chunking profile, budgets, `config.revision`) to the resolved platform embedding model. Store it beside each vector and compare it in `GET`: bumping the config, changing chunking, or an admin repointing the embedding model all surface as pending re-embed work automatically.
+- **Unembeddable documents are tombstoned** platform-side by content hash and re-reported with `skipped: true` on every run — stamp them in your own table and exclude them from `GET`.
+- **Workspace pgvector.** Apps with the embeddings capability get the `vector` extension provisioned where the server may create extensions (otherwise the deploy log names the cause and a migration using the `vector` type fails), so migrations can declare real `vector(...)` columns; `ON DELETE CASCADE` foreign keys are the cleanup story.
+- **Query time.** Every handler bag (`server/`, `webhooks/`, `tools/`, `mcp/`) gets `embed(name, text)`, which returns `{ embedding, revision }`. `name` must be a declared use case. Filter on the revision as well as ordering by distance — an admin repointing the embedding model puts your query vectors in a new space while the stored rows are still in the old one, and an unfiltered `ORDER BY` then returns near-random neighbours with no error:
+
+```javascript
+const { embedding, revision } = await embed('tickets', q);
+const hits = await query(
+  `SELECT ticket_id, content
+     FROM ticket_embeddings
+    WHERE revision = $2
+    ORDER BY embedding <=> $1::vector
+    LIMIT 10`,
+  [JSON.stringify(embedding), revision]
+);
+```
+- **Billing.** Embedding calls bill to the App, one usage entry per batch of up to 128 chunks regardless of how many provider requests run underneath; pump compute meters against the App's compute budget.
+- **`config` is read as a literal.** Deploy lifts `export const config = { … }` out of the source and evaluates it with none of the file's imports in scope, so it must be self-contained: no imported constants, no spreads of imported objects, no `export { config }` re-export. Other modules may import the literal from the use-case file; the reverse fails the deploy. The same self-contained-literal rule applies to `export const config`, `schema`, and `description` in `server/`, `webhooks/`, `tools/`, and `mcp/`; there a `config` violation fails the deploy while a `schema` or `description` violation is a deploy warning (nothing is published for that export).
+
+### Dev loop
+
+`npm run deploy` uploads `embeddings/` like any other source tree — there is no manifest block to add (plugin ≥ 2.10.0; earlier versions never upload the folder, so the use case silently does not exist on the server). `npm run dev` never runs the pump, and its `embed()` throws an explanation rather than embedding (the dev mirror reports `platform.capabilities.embeddings: false`), so a search route is exercised against a deployed App. Feature-detect with `platform?.capabilities?.embeddings` — `typeof embed === 'function'` is true on both, since the binding exists either way. Then, in the App admin panel, the **Embeddings** tab lists each use case with its pump status (queued, running, up to date, last run, last error, skipped docs) and a **Run now** action; the same surface exists as API routes (`GET /apps/{id}/embeddings`, `POST /apps/{id}/embeddings/{name}/_run`). Deploy, run, read the error, fix, repeat.
+
+### Upgrading an App that already has an `embeddings/` folder
+
+`embeddings/` is a server-side folder from this release on, like `server/`: its files are uploaded with the library, never served to browsers, and scanned as pump handlers. An App that kept anything else there (assets, data files) loses those files in the browser after redeploying with plugin 2.10.0 or later. A `.js` file there that exports `GET` or `POST` but not both fails the deploy; one that exports neither is never scanned as a use case at all and lands as a warning on an otherwise successful deploy. Move such content elsewhere before upgrading; the deploy names every stray entry as a warning.
+
+### Older Informer releases
+
+The `embeddings/` folder, `embed()`, and the pump routes exist only on Informer releases that ship the feature. An App that must also run on older servers feature-detects with optional chaining (`platform?.capabilities?.embeddings` on the handler bag, `window.__INFORMER__.platform?.capabilities?.embeddings` in the browser) — releases before 2026.1.3 inject no `platform` descriptor at all, so a bare `platform.capabilities.embeddings` throws there instead of reporting `false`. It gates its vector DDL with a `-- requires: embeddings` header so the migration is skipped where pgvector is absent, and states a floor in `informer.yaml` (`requires: { informer: '>=…' }`) only if it cannot work without the feature.
+
+Full reference: the **Embeddings** article under Developer → API → App in the Informer docs.

package/bin/ci.js

@@ -14,7 +14,9 @@
  *         INFORMER_MARKETPLACES: ${{ vars.INFORMER_MARKETPLACES }}
  *
  * Everything interesting lives here rather than in YAML, so fixing it is an
- * npm release rather than a pull request into every customer's repo.
+ * npm release rather than a pull request into every customer's repo. And
+ * everything testable lives in src/fan-out.js rather than here — this file
+ * is only argv/env handling, console output, and the exit code.
  *
  * For a single marketplace with a `lmpub_` key — locally, or on CI that is not
  * GitHub Actions — use `informer-publish` instead. This command needs the OIDC
@@ -24,8 +26,8 @@
 import { appendFile } from 'node:fs/promises';
 import { resolve } from 'node:path';
 import { publish } from '../src/publish.js';
-import { preparePublish, tagVersion } from '../src/publish-payload.js';
-import { parseMarketplaces, normalizeMarketplaceUrl } from '../src/marketplaces.js';
+import { preparePublish, tagVersion, isVersionTag, looksLikeVersion } from '../src/publish-payload.js';
+import { resolveTargets, fanOut, summaryTable, failedTargets } from '../src/fan-out.js';
 import { canMintIdToken, mintIdToken } from '../src/github-oidc.js';
 import { loadEnv, parseModeArg } from '../src/env.js';
 
@@ -34,12 +36,12 @@ loadEnv({ mode });
 
 const projectRoot = resolve('.');
 
-// --- Targets: explicit flags replace the configured list entirely, so a
-//     workflow_dispatch can re-publish to a subset without editing config. ---
 let targets;
 try {
-    const explicit = argValues('--marketplace').map(normalizeMarketplaceUrl);
-    targets = explicit.length ? [...new Set(explicit)] : parseMarketplaces(process.env.INFORMER_MARKETPLACES);
+    targets = resolveTargets({
+        explicit: argValues('--marketplace'),
+        configured: process.env.INFORMER_MARKETPLACES
+    });
 } catch (err) {
     fail(err.message);
 }
@@ -60,13 +62,28 @@ if (!canMintIdToken()) {
     );
 }
 
-// --- Build the payload ONCE. Every marketplace gets identical bytes. ---
+// The version comes from the tag that triggered the run. On a branch push or
+// a workflow_dispatch against a branch, GITHUB_REF_NAME is the branch name —
+// and without this guard the run would publish a version literally called
+// "main", on the STABLE channel (no prerelease dash), to every marketplace.
+// The predicate is shared with informer-publish (src/publish-payload.js).
+const explicitVersion = argValue('--version');
+if (explicitVersion && !looksLikeVersion(explicitVersion)) {
+    fail(`--version "${explicitVersion}" doesn't look like a version (x.y.z, or x.y.z-beta.n).`);
+}
+if (!explicitVersion && !isVersionTag(process.env.GITHUB_REF_TYPE, process.env.GITHUB_REF_NAME)) {
+    fail(
+        `informer-ci publishes the version named by the tag that triggered the run, ` +
+        `but this run's ref is ${process.env.GITHUB_REF_TYPE || 'ref'} "${process.env.GITHUB_REF_NAME || '(unset)'}" — not a version tag.\n` +
+        '  Run the workflow against a vX.Y.Z tag, or pass --version <x.y.z>.'
+    );
+}
+const refVersion = tagVersion(process.env.GITHUB_REF_NAME);
+
+// --- Build the payload once: every marketplace gets the same file set. ---
 let payload;
 try {
-    payload = await preparePublish({
-        projectRoot,
-        version: argValue('--version') || tagVersion(process.env.GITHUB_REF_NAME)
-    });
+    payload = await preparePublish({ projectRoot, version: explicitVersion || refVersion });
 } catch (err) {
     fail(err.message);
 }
@@ -79,28 +96,30 @@ console.log(
     `${files.length} files${screenshots.length ? `, ${screenshots.length} screenshot${screenshots.length === 1 ? '' : 's'}` : ''}`
 );
 
-// --- Fan out. One target failing must not stop the others: a marketplace that
-//     is down should not withhold the release from the ones that are up. ---
-const results = [];
-for (const marketplaceUrl of targets) {
-    const label = hostOf(marketplaceUrl);
-    try {
-        // Audience is the marketplace itself, so a token minted for one
-        // License Manager is not replayable at another.
-        const token = await mintIdToken(marketplaceUrl);
-        const result = await publish({ marketplaceUrl, token, files, icon, screenshots, fields: payload.fields });
+const results = await fanOut({
+    targets,
+    mint: mintIdToken,
+    publishOne: ({ marketplaceUrl, token }) =>
+        publish({ marketplaceUrl, token, files, icon, screenshots, fields: payload.fields }),
+    onSuccess: (label, result) => {
         for (const w of result.warnings || []) console.warn(`  warning (${label}): ${w}`);
         console.log(`  ✓ ${label} — published ${slug} v${version} (${result.channel || channel})`);
-        results.push({ label, ok: true });
-    } catch (err) {
-        console.error(`  ✗ ${label} — ${err.message}`);
-        results.push({ label, ok: false, error: err.message });
+    },
+    onFailure: (label, message) => {
+        console.error(`  ✗ ${label} — ${message}`);
     }
-}
+});
 
 await writeSummary(results);
 
-const failed = results.filter(r => !r.ok);
+// A row that published but whose reporting threw: the ✓ line above is missing
+// for it, so say what happened rather than leaving a silent gap. Not a failure
+// — the release is live, and exiting 1 here invites a re-run that 409s.
+for (const r of results.filter(r => r.reportError)) {
+    console.warn(`  warning (${r.label}): published, but reporting failed — ${r.reportError}`);
+}
+
+const failed = failedTargets(results);
 if (failed.length) {
     console.error(`\n${failed.length} of ${results.length} marketplaces failed.`);
     process.exit(1);
@@ -122,32 +141,14 @@ function argValues(flag) {
     return out;
 }
 
-function hostOf(url) {
-    try {
-        return new URL(url).host;
-    } catch {
-        return url;
-    }
-}
-
-/**
- * A per-target line in the job summary. Without this the Actions UI shows one
- * opaque step for what is really N independent publishes — the observability
- * the old matrix-of-jobs gave up when fan-out moved in here.
- */
 async function writeSummary(rows) {
     const path = process.env.GITHUB_STEP_SUMMARY;
     if (!path || !rows.length) return;
-    const body = [
-        `### ${name} v${version} (${channel})`,
-        '',
-        '| Marketplace | Result |',
-        '| --- | --- |',
-        ...rows.map(r => `| ${r.label} | ${r.ok ? 'published' : `failed — ${r.error.replace(/\|/g, '\\|')}`} |`),
-        ''
-    ].join('\n');
-    await appendFile(path, body).catch(() => {
-        // A summary is a nicety; never fail a good publish over it.
+    const body = summaryTable({ title: `${name} v${version} (${channel})`, rows });
+    await appendFile(path, body).catch(err => {
+        // A summary is a nicety; never fail a good publish over it — but say
+        // why it's missing rather than leaving its absence a mystery.
+        console.warn(`  warning: could not write the job summary: ${err.message}`);
     });
 }
 

package/bin/publish.js

@@ -10,7 +10,7 @@
 
 import { resolve } from 'node:path';
 import { publish } from '../src/publish.js';
-import { preparePublish, tagVersion } from '../src/publish-payload.js';
+import { preparePublish, tagVersion, isVersionTag } from '../src/publish-payload.js';
 import { loadEnv, parseModeArg } from '../src/env.js';
 
 const mode = parseModeArg(process.argv);
@@ -39,12 +39,19 @@ if (process.env.INFORMER_MARKETPLACES && process.env.GITHUB_ACTIONS) {
 
 const projectRoot = resolve('.');
 
+// Prefer an explicit --version, then the git tag (CI: GITHUB_REF_NAME=vX.Y.Z),
+// then package.json. A branch ref is NOT a version tag: without the gate, a
+// branch push on Actions published a version literally called "main" — so a
+// non-tag ref falls through to package.json, and says so.
+const refIsTag = isVersionTag(process.env.GITHUB_REF_TYPE, process.env.GITHUB_REF_NAME);
+if (process.env.GITHUB_REF_NAME && !refIsTag && !argValue('--version')) {
+    console.warn(`Note: ref "${process.env.GITHUB_REF_NAME}" is not a version tag — using the package.json version.`);
+}
+
 let payload;
 try {
     payload = await preparePublish({
-        // Prefer an explicit --version, then the git tag (CI: GITHUB_REF_NAME=vX.Y.Z),
-        // then package.json.
-        version: argValue('--version') || tagVersion(process.env.GITHUB_REF_NAME),
+        version: argValue('--version') || (refIsTag ? tagVersion(process.env.GITHUB_REF_NAME) : undefined),
         projectRoot
     });
 } catch (err) {

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@entrinsik/vite-plugin-informer",
-    "version": "2.7.0-beta.0",
+    "version": "2.10.0",
     "description": "Vite plugin and deploy tool for Informer App development",
     "scripts": {
         "test": "node --test"

package/src/agent-dev.js

@@ -2,7 +2,18 @@ import { readFile, readdir, stat, access } from 'node:fs/promises';
 import { join } from 'node:path';
 import { parse as parseUrl } from 'node:url';
 import yaml from 'yaml';
-import { loadDependencies, buildDevContext, loadAppEnv, buildDevCrypto, buildDevMessaging, normalizeFetchPath } from './dev-dependencies.js';
+import { manifestBlock, buildDevContext, buildDevCrypto, buildDevMessaging, normalizeFetchPath } from './dev-dependencies.js';
+import { createDevChannels, createDevEmit } from './dev-channels.js';
+import { devPlatform } from './dev-platform.js';
+
+// embed() is present on a real install even without the embeddings
+// capability, where it throws a written explanation. Mirroring that here
+// keeps the dev failure the same lesson as the deployed one, instead of a
+// bare "embed is not a function" that reads like a missing binding.
+const embed = async () => {
+    throw new Error('embed() is not available in the dev mirror: the embeddings capability needs a real Informer (platform.capabilities.embeddings is false)');
+};
+
 const parseYaml = yaml.parse;
 
 const MAX_STEPS = 20;
@@ -149,9 +160,11 @@ async function readSSE(response) {
  *
  * @param {Object} viteServer - Vite dev server instance
  * @param {Object} opts - Configuration
+ * @param {ReturnType<typeof createDevChannels>} [opts.channels] - the dev channels hub
+ *   behind `broadcast()` and the `channels:` relay (a private one when omitted)
  * @returns {Function} Connect middleware
  */
-export function createAgentMiddleware(viteServer, { serverOrigin, authHeader, devWorkspaceId, projectRoot, devBindings, appToken }) {
+export function createAgentMiddleware(viteServer, { serverOrigin, authHeader, devWorkspaceId, projectRoot, devBindings, appToken, channels = createDevChannels() }) {
 
     // query() — proxies to the workspace _sql endpoint (same as server-routes.js)
     async function query(sql, params) {
@@ -226,12 +239,6 @@ export function createAgentMiddleware(viteServer, { serverOrigin, authHeader, de
         ? async (path, opts = {}) => await fetchAs(appAuth, path, { ...opts, headers: { ...(opts.headers || {}), 'x-informer-app-depth': '1' } })
         : null;
 
-    // emit() — no-op in dev mode (logs to console)
-    function emit(event, payload) {
-        console.log(`[agent-dev] emit("${event}",`, JSON.stringify(payload), ')');
-        return { ok: true };
-    }
-
     // notify/email — delivery is a console-logged no-op in dev, but required-field
     // validation mirrors prod so a tool that passes here won't 500 in production.
     const { notify, email } = buildDevMessaging('[agent-dev]');
@@ -319,10 +326,17 @@ export function createAgentMiddleware(viteServer, { serverOrigin, authHeader, de
                 // Build the typed dependency context + env the same way the
                 // server-routes dev middleware does, so tools using
                 // `context.<slot>.<method>(...)` and `env` work locally and match
-                // the prod sandbox bag.
-                const deps = await loadDependencies(projectRoot);
+                // the prod sandbox bag. The manifest is already in hand.
+                const deps = manifestBlock(yaml, 'dependencies');
                 const context = buildDevContext({ deps, apiFetch, devBindings, appFetch });
-                const env = await loadAppEnv(projectRoot);
+                const env = manifestBlock(yaml, 'env');
+
+                // emit() writes no app_event row in dev, but still relays a
+                // listed event to its `channels:` channel; broadcast() publishes
+                // a live frame to the page (see dev-channels.js).
+                const manifestChannels = manifestBlock(yaml, 'channels');
+                const emit = createDevEmit({ channels, manifestChannels, logPrefix: '[agent-dev]' });
+                const { broadcast } = channels;
 
                 // Load tool handlers via ssrLoadModule
                 const localTools = await scanLocalTools(projectRoot);
@@ -441,7 +455,7 @@ export function createAgentMiddleware(viteServer, { serverOrigin, authHeader, de
 
                         if (tool) {
                             try {
-                                result = await tool.handler({ args: tc.input, run: { agentName, trigger: triggerEvent }, context, query, fetch: apiFetch, emit, notify, email, crypto: cryptoHelper, markdown, log, env });
+                                result = await tool.handler({ args: tc.input, run: { agentName, trigger: triggerEvent }, context, query, fetch: apiFetch, emit, broadcast, notify, email, embed, crypto: cryptoHelper, markdown, log, env, platform: devPlatform() });
                             } catch (err) {
                                 console.error(`[agent-dev] Tool "${tc.name}" failed:`, err.message);
                                 result = { error: err.message };

package/src/assemble.js

@@ -22,7 +22,11 @@ import { join, relative, posix } from 'node:path';
 // the Vite build lands it at the library root). Without README.md here, a
 // CLI-deployed app could never light those surfaces up via its README.
 const ROOT_CONFIG_FILES = ['informer.yaml', 'data-access.yaml', 'API.md', 'README.md'];
-const SOURCE_DIRS = ['migrations', 'tools', 'mcp', 'server', 'webhooks', 'lib', 'shared'];
+// channels/ holds live-channel join/leave handlers (App Channels); the server
+// scans and bundles it like server/ and webhooks/. `embeddings` holds the pump
+// use-case files: server-side like server/, so they ship with the library and
+// the platform can scan them.
+const SOURCE_DIRS = ['migrations', 'tools', 'mcp', 'server', 'webhooks', 'channels', 'embeddings', 'lib', 'shared'];
 
 // Entries never worth shipping in an app's server-side library: OS/editor
 // dotfiles (.DS_Store, .env), nested dependency trees, and test files. Applied