@geraldmaron/construct 1.0.18 → 1.0.19

package/templates/docs/skill-artifact.md

@@ -1,9 +1,11 @@
 ---
 name: <verb-led-id>
 scope: "<one paragraph: what it covers, what it does not>"
-observable_outcome: <how someone outside the role tells this skill happened>
+observable_outcome: <how someone outside the role tells this skill happened — a concrete artifact, not "understands X">
+prerequisites:
+  - <skill-id or capability that must exist before this one is useful>
 provenance:
-  - <citation: post-mortem, public framework, competency model>
+  - <citation: post-mortem, public framework, competency model, with URL>
 roles:
   - <role-id that uses this skill>
 ---
@@ -12,16 +14,34 @@ roles:
 
 ## What this skill produces
 
-<Concrete output. Bloom-style: not "knows X" but "produces Y".>
+<Concrete output, Bloom-style: not "knows X" but "produces Y". Name the artifact and what makes it correct. Example: "a runbook whose diagnostic flowchart branches to a fix in ≤3 steps", not "understands incident response".>
 
 ## When to invoke it
 
-<Triggers. What the operator or the upstream persona is doing when this skill is the right call.>
+<Triggers. What the operator or upstream persona is doing when this skill is the right call. Include the signal that distinguishes it from adjacent skills.>
 
-## How it composes
+## Prerequisites and composition
 
-<Other skills typically chained before or after.>
+<What must be true or done first, and the skills typically chained before and after. A skill with unmet prerequisites produces confident-looking but wrong output.>
+
+## Competency rubric
+
+<What the skill looks like at each level. The author of an artifact should be able to place their work on this scale.>
+
+| Level | What it looks like |
+|-------|--------------------|
+| Novice | <produces the artifact but misses edge cases / evidence / failure paths> |
+| Competent | <covers the happy path and the common failure modes; cites sources> |
+| Expert | <anticipates the non-obvious failure, calibrates confidence, leaves the next person a reproducible trail> |
+
+## Failure modes
+
+<How this skill breaks when misapplied — distinct from anti-patterns. What does a wrong-but-plausible output look like, and what signal reveals it? When should the operator escalate or stop?>
 
 ## Anti-patterns
 
-<Ways this skill gets misapplied; what looks similar but is not this.>
+<Ways this skill gets misapplied; what looks similar but is not this skill.>
+
+## Worked example
+
+<A short, concrete instance: the trigger, the artifact produced, and why it meets the observable outcome. One real example removes more ambiguity than a page of guidance.>

package/templates/docs/strategy.md

@@ -15,8 +15,8 @@ Owning workflow: skills/docs/strategy-workflow.md.
 <!-- 1–3 sentences. Where are we going in 3–5 years? -->
 
 ## Bets
-<!-- Explicit choices. Each bet: what we're doing and why. -->
-- **Bet 1**: …
+<!-- Explicit choices. Each bet: what we're doing, why, the leading indicator that tells us early if it's working, and the kill criterion that ends it. A bet with no kill criterion is a belief, not a bet. -->
+- **Bet 1**: … **Why**: … **Leading indicator**: … **Kill criterion**: {what observation would make us stop}
 - **Bet 2**: …
 
 ## Non-bets
@@ -29,6 +29,27 @@ Owning workflow: skills/docs/strategy-workflow.md.
 ## North Star Metric
 <!-- Single measurable outcome. Format: Metric / Baseline / Target / Owner -->
 
+## Metrics
+<!-- The North Star is lagging. Name the leading indicators that move first and predict it, so a failing bet is visible before the lagging metric confirms it. -->
+
+| Metric | Type (leading/lagging) | Baseline | Target | Owner |
+|--------|------------------------|----------|--------|-------|
+| {name} | leading / lagging | {value} | {value} | {who} |
+
+## Milestones
+<!-- What must be true by when. Each milestone is a checkpoint that confirms or kills a bet. -->
+
+| Milestone | By when | Confirms / kills |
+|-----------|---------|------------------|
+| {what}    | {date}  | {which bet}      |
+
+## Risks
+<!-- What could invalidate the strategy. Risk register, not a worry list. -->
+
+| Risk | Likelihood | Impact | Mitigation / leading signal |
+|------|-----------|--------|------------------------------|
+| {what} | high/med/low | high/med/low | {response} |
+
 ## Competitive Positioning
 <!-- How we differentiate. 2–4 sentences. -->
 

package/lib/bootstrap/lazy-install.mjs

@@ -1,161 +0,0 @@
-/**
- * lib/bootstrap/lazy-install.mjs — on-demand resource installer with consent gating.
- *
- * When a runtime code path needs a resource that is not present, call
- * `lazyInstall(id)`. The function:
- *
- *   1. Reads cached operator consent from config.env (consent key set by the
- *      resource definition in `lib/bootstrap/resources.mjs`).
- *   2. If consent is 'never', returns { installed: false, fallback: <text> }
- *      without prompting or printing anything.
- *   3. If a TTY is available, prompts the user with the resource displayName,
- *      estimated download size, and [y/n/never].
- *   4. On 'y' or an existing 'yes' consent: calls resource.install(), waits
- *      for it to complete, updates config.env, and returns { installed: true }.
- *   5. On 'n': returns { installed: false, fallback: <text> } without saving.
- *   6. On 'never': writes 'never' to config.env and returns { installed: false }.
- *
- * Hooks that run without a TTY (most hook invocations) skip to step 2 and
- * use the fallback silently.
- *
- * Consent is cached per-resource in config.env so the prompt never repeats
- * across process restarts once the user has answered.
- */
-
-import fs from 'node:fs';
-import path from 'node:path';
-import os from 'node:os';
-import readline from 'node:readline';
-
-import { getResource, probeResource } from './resources.mjs';
-
-const CONFIG_ENV = path.join(os.homedir(), '.construct', 'config.env');
-
-function readConfigEnv() {
-  try {
-    return fs.readFileSync(CONFIG_ENV, 'utf8');
-  } catch {
-    return '';
-  }
-}
-
-function readConsentFromEnv(consentKey) {
-  const lines = readConfigEnv().split('\n');
-  for (const line of lines) {
-    const trimmed = line.trim();
-    if (!trimmed || trimmed.startsWith('#')) continue;
-    const eq = trimmed.indexOf('=');
-    if (eq === -1) continue;
-    const k = trimmed.slice(0, eq).trim();
-    const v = trimmed.slice(eq + 1).trim().replace(/^['"]|['"]$/g, '');
-    if (k === consentKey) return v;
-  }
-  return null;
-}
-
-function writeConsentToEnv(consentKey, value) {
-  let content = readConfigEnv();
-  const regex = new RegExp(`^${consentKey}=.*$`, 'm');
-  const line = `${consentKey}=${value}`;
-  if (regex.test(content)) {
-    content = content.replace(regex, line);
-  } else {
-    content = content.trimEnd() + (content ? '\n' : '') + line + '\n';
-  }
-  fs.mkdirSync(path.dirname(CONFIG_ENV), { recursive: true });
-  fs.writeFileSync(CONFIG_ENV, content, { mode: 0o600 });
-}
-
-function hasTty() {
-  return Boolean(process.stdin.isTTY && process.stderr.isTTY);
-}
-
-function formatBytes(bytes) {
-  if (!bytes) return '';
-  if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`;
-  return `${Math.round(bytes / (1024 * 1024))} MB`;
-}
-
-async function promptUser(question) {
-  return new Promise((resolve) => {
-    const rl = readline.createInterface({ input: process.stdin, output: process.stderr });
-    rl.question(question, (answer) => {
-      rl.close();
-      resolve(answer.trim().toLowerCase());
-    });
-  });
-}
-
-/**
- * Attempt to install a resource lazily, respecting cached consent.
- *
- * @param {string} id - resource id from the registry
- * @param {object} [opts]
- * @param {boolean} [opts.silent] - skip TTY prompt, use cached consent only
- * @returns {{ installed: boolean, fallback?: string, error?: string }}
- */
-export async function lazyInstall(id, { silent = false } = {}) {
-  const resource = getResource(id);
-  if (!resource) {
-    return { installed: false, error: `unknown resource: ${id}` };
-  }
-
-  const probe = await probeResource(resource);
-  if (probe.present && probe.healthy) {
-    return { installed: true };
-  }
-
-  const { consentKey, displayName, downloadSize, fallback } = resource;
-  const fallbackMsg = typeof fallback === 'function' ? fallback() : (fallback || '');
-
-  const cached = readConsentFromEnv(consentKey);
-
-  if (cached === 'never') {
-    return { installed: false, fallback: fallbackMsg };
-  }
-
-  if (cached === 'yes') {
-    return await performInstall(resource, fallbackMsg);
-  }
-
-  if (silent || !hasTty()) {
-    return { installed: false, fallback: fallbackMsg };
-  }
-
-  const size = downloadSize ? ` (~${formatBytes(downloadSize)})` : '';
-  const answer = await promptUser(
-    `\n[construct] ${displayName} required${size}. Install now? [y/n/never] `
-  );
-
-  if (answer === 'y' || answer === 'yes') {
-    writeConsentToEnv(consentKey, 'yes');
-    return await performInstall(resource, fallbackMsg);
-  }
-
-  if (answer === 'never') {
-    writeConsentToEnv(consentKey, 'never');
-    process.stderr.write(
-      `[construct] "${displayName}" will not be prompted again. Using fallback: ${fallbackMsg}\n`
-    );
-    return { installed: false, fallback: fallbackMsg };
-  }
-
-  return { installed: false, fallback: fallbackMsg };
-}
-
-async function performInstall(resource, fallbackMsg) {
-  if (typeof resource.install !== 'function') {
-    return { installed: false, fallback: fallbackMsg, error: 'resource has no install()' };
-  }
-  try {
-    process.stderr.write(`[construct] Installing ${resource.displayName}...\n`);
-    const result = await resource.install();
-    if (result?.success) {
-      process.stderr.write(`[construct] ${resource.displayName} installed.\n`);
-      return { installed: true };
-    }
-    return { installed: false, fallback: fallbackMsg, error: result?.log };
-  } catch (err) {
-    return { installed: false, fallback: fallbackMsg, error: err?.message || String(err) };
-  }
-}

package/lib/embed/jobs/vector-sync.mjs

@@ -1,198 +0,0 @@
-/**
- * lib/embed/jobs/vector-sync.mjs — Automatic embedding pipeline for observations.
- *
- * Runs on a schedule (default: every 5 minutes) to embed new/modified observations
- * and store them in Postgres (if available) and/or the local vector index.
- *
- * Design:
- *   - Scans .cx/observations/ for items not yet embedded
- *   - Embeds each using the configured model (local ONNX, OpenAI, Ollama)
- *   - Stores in Postgres construct_observations table (if DATABASE_URL set)
- *   - Falls back to local vectors.json if Postgres unavailable
- *   - Batched processing to avoid overwhelming the embedding API
- *   - Deduplicated via content hash comparison
- */
-import { existsSync, readFileSync, writeFileSync, statSync, mkdirSync } from 'node:fs';
-import { join, dirname } from 'node:path';
-import { homedir } from 'node:os';
-
-const DEFAULT_BATCH_SIZE = 32;
-const DEFAULT_INTERVAL_MS = 5 * 60 * 1000;
-
-/**
- * Find observations that haven't been embedded yet.
- * @param {string} rootDir
- * @returns {Array<{ id: string, project: string, role: string, category: string, summary: string, content: string, tags: string[], confidence: number, source: string, filePath: string }>}
- */
-function findUnembeddedObservations(rootDir) {
-  const obsDir = join(rootDir, '.cx', 'observations');
-  if (!existsSync(obsDir)) return [];
-
-  const indexPath = join(obsDir, 'index.json');
-  if (!existsSync(indexPath)) return [];
-
-  let index;
-  try {
-    index = JSON.parse(readFileSync(indexPath, 'utf8'));
-  } catch (err) {
-    process.stderr.write('[jobs/vector-sync.mjs] findUnembedded: ' + (err?.message ?? String(err)) + '\n');
-    return [];
-  }
-
-  // Check which observations have been embedded
-  const vectorsPath = join(obsDir, 'vectors.json');
-  const embeddedIds = new Set();
-  if (existsSync(vectorsPath)) {
-    try {
-      const vectors = JSON.parse(readFileSync(vectorsPath, 'utf8'));
-      for (const v of (vectors.records || vectors)) {
-        if (v.id) embeddedIds.add(v.id);
-      }
-    } catch { /* ignore */ }
-  }
-
-  // Also check Postgres if available
-  // (This is handled by the caller via VectorClient.isHealthy())
-
-  return (index || [])
-    .filter((obs) => obs.id && !embeddedIds.has(obs.id))
-    .map((obs) => ({
-      ...obs,
-      filePath: join(obsDir, `${obs.id}.json`),
-    }))
-    .filter((obs) => existsSync(obs.filePath));
-}
-
-/**
- * Run the vector sync job.
- * @param {object} opts
- * @param {string} opts.rootDir - Root directory for observation store
- * @param {object} opts.env - Environment variables
- * @param {object} opts.logger - Optional logger with debug/info/error methods
- * @param {object} opts.vectorClient - Optional VectorClient instance for SQL storage
- * @returns {Promise<{ synced: number, errors: number, durationMs: number }>}
- */
-export async function runVectorSync({ rootDir, env = process.env, logger, vectorClient } = {}) {
-  const startTime = Date.now();
-  const batchSize = parseInt(env.CONSTRUCT_EMBEDDING_BATCH_SIZE, 10) || DEFAULT_BATCH_SIZE;
-
-  try {
-    const unembedded = findUnembeddedObservations(rootDir);
-    if (!unembedded.length) {
-      logger?.debug?.('Vector sync: no new observations to embed');
-      return { synced: 0, errors: 0, durationMs: Date.now() - startTime };
-    }
-
-    let synced = 0;
-    let errors = 0;
-
-    // Dynamically import embedding engine
-    const { embedBatch } = await import('../../storage/embeddings-engine.mjs');
-
-    for (let i = 0; i < unembedded.length; i += batchSize) {
-      const batch = unembedded.slice(i, i + batchSize);
-      const texts = batch.map((o) => `${o.summary}\n${o.content}`);
-
-      try {
-        const embeddings = await embedBatch(texts, { env });
-
-        for (let j = 0; j < batch.length; j++) {
-          const obs = batch[j];
-          const embedding = embeddings[j].embedding;
-
-          // Store in Postgres if available
-          if (vectorClient && await vectorClient.isHealthy()) {
-            try {
-              await vectorClient.storeObservation({
-                id: obs.id,
-                project: obs.project || rootDir.split('/').pop(),
-                role: obs.role,
-                category: obs.category,
-                summary: obs.summary,
-                content: obs.content,
-                tags: obs.tags || [],
-                confidence: obs.confidence ?? 0.8,
-                source: obs.source,
-                embedding,
-              });
-            } catch (err) {
-              logger?.error?.(`Vector sync: failed to store observation ${obs.id} in SQL: ${err.message}`);
-              errors++;
-              continue;
-            }
-          }
-
-          // Always update local vectors.json
-          updateLocalVectorIndex(rootDir, obs, embedding);
-          synced++;
-        }
-      } catch (err) {
-        logger?.error?.(`Vector sync: batch embedding failed: ${err.message}`);
-        errors += batch.length;
-      }
-    }
-
-    const durationMs = Date.now() - startTime;
-    logger?.info?.(`Vector sync: embedded ${synced} observation(s) in ${durationMs}ms (${errors} errors)`);
-    return { synced, errors, durationMs };
-  } catch (err) {
-    const durationMs = Date.now() - startTime;
-    logger?.error?.(`Vector sync failed: ${err.message}`);
-    return { synced: 0, errors: 1, durationMs };
-  }
-}
-
-/**
- * Update the local vectors.json index with a new embedding.
- * @param {string} rootDir
- * @param {object} obs - Observation record
- * @param {Float32Array} embedding
- */
-function updateLocalVectorIndex(rootDir, obs, embedding) {
-  const obsDir = join(rootDir, '.cx', 'observations');
-  const vectorsPath = join(obsDir, 'vectors.json');
-
-  let vectors;
-  if (existsSync(vectorsPath)) {
-    try {
-      vectors = JSON.parse(readFileSync(vectorsPath, 'utf8'));
-    } catch (err) {
-      process.stderr.write('[jobs/vector-sync.mjs] updateLocalVectorIndex: ' + (err?.message ?? String(err)) + '\n');
-      vectors = { version: 1, model: 'hashing-bow-v1', updatedAt: null, records: [] };
-    }
-  } else {
-    vectors = { version: 1, model: 'hashing-bow-v1', updatedAt: null, records: [] };
-  }
-
-  // Add or update the record
-  const existingIdx = vectors.records.findIndex((r) => r.id === obs.id);
-  const record = {
-    id: obs.id,
-    embedding: Array.from(embedding),
-    summary: obs.summary,
-    content: obs.content,
-    tags: obs.tags || [],
-    project: obs.project,
-    role: obs.role,
-    category: obs.category,
-    updatedAt: new Date().toISOString(),
-  };
-
-  if (existingIdx >= 0) {
-    vectors.records[existingIdx] = record;
-  } else {
-    vectors.records.push(record);
-  }
-
-  // Cap at 1000 records (sliding window)
-  if (vectors.records.length > 1000) {
-    vectors.records = vectors.records.slice(-1000);
-  }
-
-  vectors.updatedAt = new Date().toISOString();
-
-  mkdirSync(dirname(vectorsPath), { recursive: true });
-  writeFileSync(vectorsPath, JSON.stringify(vectors, null, 2), 'utf8');
-}
-
-export { DEFAULT_BATCH_SIZE, DEFAULT_INTERVAL_MS };

package/lib/knowledge/postgres-search.mjs

@@ -1,132 +0,0 @@
-/**
- * lib/knowledge/postgres-search.mjs — Tag-aware vector search via pgvector + GIN.
- *
- * Runs when DATABASE_URL is configured. Chooses between two execution paths
- * based on tag selectivity:
- *
- * - High selectivity (tag matches <5% of corpus): GIN prefilter first, then
- *   HNSW ANN on the filtered set.
- * - Low selectivity (tag matches ≥5% of corpus): HNSW ANN first with
- *   hnsw.iterative_scan=relaxed_order, postfilter by tag.
- *
- * Falls back gracefully when DATABASE_URL is absent or the pgvector extension
- * is not installed; callers should use knowledgeSearch() from search.mjs in
- * that case.
- */
-
-// Selectivity threshold: if the tagged fraction exceeds this, use postfilter.
-const HIGH_SELECTIVITY_THRESHOLD = 0.05;
-
-/**
- * Execute a tag-filtered vector search against Postgres + pgvector.
- *
- * @param {object} opts
- * @param {Float32Array|number[]} opts.queryEmbedding   — query vector
- * @param {string}   opts.project       — project id (construct_documents.project)
- * @param {string[]} [opts.tags]        — tag filter
- * @param {'any'|'all'} [opts.tagMatch] — 'any' (default) or 'all'
- * @param {number}   [opts.topK]        — result limit (default 20)
- * @param {object}   opts.db            — postgres client (from lib/db.mjs or postgres pkg)
- * @returns {Promise<Array<{id, title, score}>>}
- */
-export async function postgresTagSearch({ queryEmbedding, project, tags = [], tagMatch = 'any', topK = 20, db }) {
-  if (!db) throw new Error('postgres-search: db client required');
-  if (!queryEmbedding?.length) throw new Error('postgres-search: queryEmbedding required');
-
-  const vecLiteral = `[${Array.from(queryEmbedding).join(',')}]`;
-
-  if (!tags.length) {
-    // No tag filter — plain ANN.
-    const rows = await db`
-      select d.id, d.title, e.embedding <=> ${vecLiteral}::vector as distance
-      from construct_embeddings e
-      join construct_documents d on d.id = e.document_id
-      where d.project = ${project}
-      order by e.embedding <=> ${vecLiteral}::vector
-      limit ${topK}
-    `;
-    return rows.map((r) => ({ id: r.id, title: r.title, score: 1 - parseFloat(r.distance) }));
-  }
-
-  // Estimate selectivity for this tag set.
-  const tagJsonb = JSON.stringify(tags);
-  const [{ total }] = await db`select count(*) as total from construct_documents where project = ${project}`;
-  const totalCount = parseInt(total, 10) || 1;
-
-  let taggedCount;
-  if (tagMatch === 'all') {
-    const [{ cnt }] = await db`
-      select count(*) as cnt from construct_documents
-      where project = ${project} and tags @> ${tagJsonb}::jsonb
-    `;
-    taggedCount = parseInt(cnt, 10);
-  } else {
-    // 'any' — count docs with at least one matching tag.
-    const [{ cnt }] = await db`
-      select count(*) as cnt from construct_documents
-      where project = ${project}
-        and (${db.unsafe(tags.map((_, i) => `tags ? $${i + 2}`).join(' or '))})
-    `;
-    taggedCount = parseInt(cnt, 10);
-  }
-
-  const selectivity = taggedCount / totalCount;
-
-  if (selectivity < HIGH_SELECTIVITY_THRESHOLD) {
-    // High selectivity: GIN prefilter → ANN on filtered set.
-    if (tagMatch === 'all') {
-      const rows = await db`
-        with filtered as (
-          select id from construct_documents
-          where project = ${project} and tags @> ${tagJsonb}::jsonb
-        )
-        select d.id, d.title, e.embedding <=> ${vecLiteral}::vector as distance
-        from construct_embeddings e
-        join filtered f on f.id = e.document_id
-        join construct_documents d on d.id = e.document_id
-        order by e.embedding <=> ${vecLiteral}::vector
-        limit ${topK}
-      `;
-      return rows.map((r) => ({ id: r.id, title: r.title, score: 1 - parseFloat(r.distance) }));
-    }
-    // 'any' — OR-expand tag list.
-    const tagConditions = tags.map((t) => `tags ? '${t.replace(/'/g, "''")}'`).join(' or ');
-    const rows = await db.unsafe(`
-      with filtered as (
-        select id from construct_documents
-        where project = $1 and (${tagConditions})
-      )
-      select d.id, d.title, e.embedding <=> $2::vector as distance
-      from construct_embeddings e
-      join filtered f on f.id = e.document_id
-      join construct_documents d on d.id = e.document_id
-      order by e.embedding <=> $2::vector
-      limit $3
-    `, [project, vecLiteral, topK]);
-    return rows.map((r) => ({ id: r.id, title: r.title, score: 1 - parseFloat(r.distance) }));
-  }
-
-  // Low selectivity: HNSW iterative scan → postfilter.
-  await db`set hnsw.iterative_scan = relaxed_order`;
-  if (tagMatch === 'all') {
-    const rows = await db`
-      select d.id, d.title, e.embedding <=> ${vecLiteral}::vector as distance
-      from construct_embeddings e
-      join construct_documents d on d.id = e.document_id
-      where d.project = ${project} and d.tags @> ${tagJsonb}::jsonb
-      order by e.embedding <=> ${vecLiteral}::vector
-      limit ${topK}
-    `;
-    return rows.map((r) => ({ id: r.id, title: r.title, score: 1 - parseFloat(r.distance) }));
-  }
-  const tagConditions = tags.map((t) => `d.tags ? '${t.replace(/'/g, "''")}'`).join(' or ');
-  const rows = await db.unsafe(`
-    select d.id, d.title, e.embedding <=> $1::vector as distance
-    from construct_embeddings e
-    join construct_documents d on d.id = e.document_id
-    where d.project = $2 and (${tagConditions})
-    order by e.embedding <=> $1::vector
-    limit $3
-  `, [vecLiteral, project, topK]);
-  return rows.map((r) => ({ id: r.id, title: r.title, score: 1 - parseFloat(r.distance) }));
-}