incitaciones 0.3.1 → 0.4.0

package/pi-package/skills/agent-value-alignment/references/layer-4-infrastructure.md

@@ -0,0 +1,58 @@
+# Layer 4 — Infrastructure: State, Memory, and Observability
+
+**Question:** How is state preserved, memory managed, and drift observed in production?
+
+## Checkpointing: fault tolerance vs. alignment
+
+- **Fault-tolerant checkpointing** asks: "can we resume after a crash?"
+- **Alignment-oriented checkpointing** asks: "is the agent still pursuing the right goal, and can we roll back to a known-good state if it has drifted?"
+
+The latter requires **goal-state verification at each checkpoint**, not just state persistence.
+
+**LangGraph** (1.0 GA, Oct 2025) is the most mature production option: checkpoints at every super-step, pluggable backends (`PostgresSaver` for production), first-class `interrupt()` API for HITL resumable via `Command(resume=...)`, time-travel debugging, state mutation. Powers agents at ~400 companies (LinkedIn, Uber, Replit) with 8% latency overhead. **MemGPT/Letta** provides OS-inspired tiered memory (main context / recall DB / archival vector store) with LLM-managed tier movement.
+
+Caveat: OpenAI Agents SDK and AutoGen lack native durable execution — teams must add Temporal/DBOS. Google ADK loses in-memory session state on Cloud Run restarts. Configure persistent storage from day one; retrofitting is significantly harder.
+
+## Memory architecture as a first-class concern
+
+"Memory deserves the same level of engineering investment as the LLM itself" (Du, 2026). Three dimensions:
+
+- **Temporal scope** — working (subject to lost-in-the-middle), episodic (concrete experience records), semantic (abstracted knowledge; over-generalization risk), procedural (reusable skills; Voyager without its skill library showed 15.3× slower progression).
+- **Representational substrate** — context-resident text (compression drift), vector stores, structured stores, executable repositories.
+- **Control policy** — heuristic, prompted self-control (MemGPT; silent orchestration failure risk), learned/RL (e.g., AgeMem, per Du 2026; most capable, requires training infra).
+
+Empirical stakes: Generative Agents without reflection degenerated to repetitive responses within 48 simulated hours; on MemoryArena (2026), active memory agents achieved 80%+ task completion vs. ~45% for long-context-only baselines. Models near-saturating LoCoMo plummet to 40–60% on MemoryArena — a deep gap between passive recall and active, decision-relevant memory.
+
+## Safe stopping, rollback, and HITL
+
+Safe stopping criteria (distinct from error conditions): uncertainty threshold, scope boundary, resource limit, anomaly detection, human review trigger. LangGraph's `interrupt()` provides the mechanism; the *design* challenge is choosing when to pause, what to surface, and how to incorporate feedback: approve / modify / rollback-and-retry / abort.
+
+**Cost calculus favors false positives:** unnecessary human reviews are far cheaper than undetected goal drift leading to irreversible actions. Configure HITL at all irreversible action boundaries.
+
+## Observability for drift detection
+
+**Quantitative signals:** task completion rate, goal retention score, scope violation rate, rollback rate, HITL trigger rate, token efficiency (useful work / total tokens — flags unproductive spinning).
+
+**Qualitative signals:** execution traces, reasoning traces, state snapshots, anomaly detection.
+
+**Decay detection dashboard** tiers these into:
+- Behavioral health (weekly) — eval pass rate, goal achievement, scope violations, escalations, errors.
+- Adoption health (weekly) — WAU, tasks/user, retention, satisfaction.
+- Value health (monthly) — primary value metric, cost/task, cost/user.
+
+**Automatic review triggers:** eval pass rate drops >10% week-over-week; any red metric persists 2+ weeks; model or prompt change deployed (re-run full eval suite).
+
+## AUDIT checks
+
+- [ ] Checkpointing is alignment-oriented (goal-state verification, not just resume).
+- [ ] Persistent backend configured from day one (not in-memory in production).
+- [ ] Memory architecture is a deliberate choice across all three dimensions, not an afterthought.
+- [ ] HITL interrupts configured at all irreversible action boundaries.
+- [ ] Tracing/observability deployed before production (LangSmith or equivalent).
+- [ ] Decay detection dashboard exists with the three tiers and automatic triggers.
+- [ ] Full eval suite re-runs on every model/prompt change.
+
+## ESTABLISH (Day 5)
+
+- Day 5a (30 min): Schedule the first Value Realization Review for 90 days out.
+- Day 5b (30 min – ½ day): Stand up the decay detection dashboard. *30 min only if metrics are already instrumented; up to half a day if instrumentation must be built.* Do not skip — without it, drift cannot be detected before lagging outcome metrics confirm it.

package/pi-package/skills/agent-value-alignment/references/report-template.md

@@ -0,0 +1,113 @@
+# Agent Value Alignment — Report Template
+
+Use this for both AUDIT and ESTABLISH modes. Replace `[ ]` placeholders.
+
+## Severity rubric
+
+Apply consistently to every finding so two auditors rate the same gap alike:
+
+| Severity | Definition |
+|---|---|
+| **CRITICAL** | Breaks the governing invariant (an unenforced decision / unverified clause / unacted eval result) **or** leaves an irreversible-action boundary unguarded (no HITL, no rollback) |
+| **HIGH** | A Layer-1 or Layer-2 artifact is missing, unfalsifiable, or unsigned (e.g., no kill criteria, no BSD, vague VP) — the loop cannot close without it |
+| **MEDIUM** | An AUDIT check fails but a compensating control exists (e.g., eval suite present but golden dataset lacks failure-class examples) |
+| **LOW** | Cadence or documentation gap (e.g., Value Realization Review not yet scheduled, dashboard not yet live) |
+
+A layer is 🔴 if it has ≥1 CRITICAL finding; 🟡 if it has ≥1 HIGH/MEDIUM but no CRITICAL; 🟢 if all AUDIT checks pass.
+
+---
+
+## Agent Value Alignment Report
+
+**Target:** [agent/tool name and version]
+**Mode:** [AUDIT | ESTABLISH]
+**Date:** [YYYY-MM-DD]
+**Reviewer:** [name/role]
+**Sources consulted:** [system prompt, BSD, eval suite, dashboard, etc.]
+
+---
+
+### Executive Summary
+
+[2–4 sentences. State the single most important finding: is the closed loop intact, and if not, where is it broken?]
+
+**Overall verdict:** [ALIGNED | PARTIAL | NOT_ALIGNED]
+
+### Layer Scores
+
+| Layer | Status | Critical gaps |
+|---|---|---|
+| 1 — Value | [🟢/🟡/🔴] | [count] |
+| 2 — Governance | [🟢/🟡/🔴] | [count] |
+| 3 — Runtime Prompt | [🟢/🟡/🔴] | [count] |
+| 4 — Infrastructure | [🟢/🟡/🔴] | [count] |
+| Integration | [🟢/🟡/🔴] | [count] |
+
+Status: 🟢 all AUDIT checks pass · 🟡 ≥1 check fails, no critical · 🔴 ≥1 critical gap.
+
+---
+
+### Findings
+
+#### [LAYER-N-FINDING-001] [CRITICAL|HIGH|MEDIUM|LOW] — [Location]
+- **Layer:** [1 Value | 2 Governance | 3 Runtime | 4 Infra | Integration]
+- **What's wrong:** [specific gap, citing the artifact and location]
+- **Why it matters:** [which root cause or failure mode this enables]
+- **Recommendation:** [specific action — which artifact to create/change, by whom]
+- **Invariant violated:** [enforcement | verification | action] (if applicable)
+
+[repeat for each finding]
+
+---
+
+### Invariant Check
+
+The governing invariant: *every governance decision → prompt clause → eval test → governance review*.
+
+- Decisions with no prompt clause (not enforced): [list, or "none"]
+- Prompt clauses with no eval test (not verified): [list, or "none"]
+- Eval results not reviewed in governance (not acted upon): [list, or "none"]
+
+---
+
+### Value Proposition (as currently stated)
+
+> "[verbatim current VP, or 'NOT DEFINED']"
+
+**Falsifiable?** [Yes/No — which slot is missing]
+**Propagated verbatim across prompt + BSD + review agenda?** [Yes/No]
+
+---
+
+### Kill Criteria Status
+
+[Cite each criterion, its threshold, the current value, and status: on-track / at-risk / triggered. Or "NOT DEFINED".]
+
+---
+
+### Recommended Actions (priority order)
+
+1. [Action] — Owner: [role] — Layer: [N] — Due: [date]
+2. [Action] — Owner: [role] — Layer: [N] — Due: [date]
+3. [Action] — Owner: [role] — Layer: [N] — Due: [date]
+
+### Next Review
+
+**Value Realization Review scheduled:** [date, or "NOT SCHEDULED — schedule within 90 days"]
+**Decay dashboard live:** [Yes/No]
+
+---
+
+### For ESTABLISH mode only: Minimum Viable Artifact Set
+
+| Artifact | Status | Location |
+|---|---|---|
+| Falsifiable Value Proposition | [Done/In progress/Not started] | [path] |
+| Kill Criteria (signed) | | |
+| Agent Charter | | |
+| Goal Sandwich system prompt | | |
+| 10 eval test cases | | |
+| Value Realization Review scheduled | | |
+| Decay detection dashboard | | |
+
+*Do not consider the loop closed until every artifact above is "Done" AND the invariant check shows zero unenforced/unverified/unacted items.*

package/scripts/generate-pi-resources.mjs

@@ -2,6 +2,8 @@
 import fs from "node:fs";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
+import { homedir } from "node:os";
+import process from "node:process";
 
 const __filename = fileURLToPath(import.meta.url);
 const __dirname = path.dirname(__filename);
@@ -32,6 +34,15 @@ function readText(filePath) {
   return fs.readFileSync(filePath, "utf8");
 }
 
+function getPackageVersion() {
+  try {
+    const pkg = JSON.parse(readText(path.join(repoRoot, "package.json")));
+    return `npm:${pkg.version}`;
+  } catch {
+    return new Date().toISOString().slice(0, 10);
+  }
+}
+
 function generateSkill(prompt) {
   const distilledPath = path.join(repoRoot, prompt.distilled);
   const skillDir = path.join(skillsOut, prompt.name);
@@ -83,6 +94,113 @@ function generatePromptTemplate(prompt) {
   fs.writeFileSync(path.join(promptsOut, `${prompt.name}.md`), template, "utf8");
 }
 
+/**
+ * Detect flat-skills directories that already have incitaciones skills installed,
+ * and re-sync them with the latest pi-package content. This ensures that
+ * pi update / npm version bumps keep flat-install locations current.
+ *
+ * Flat installs (e.g. ~/.agents/skills/) are created by install.sh and
+ * npx incitaciones install, but are NOT managed by pi's package reader
+ * (which reads directly from pi-package/skills/). Without this re-sync,
+ * a version bump leaves those directories stale.
+ */
+function reSyncFlatInstallSkills(manifest) {
+  const version = getPackageVersion();
+  const now = new Date().toISOString().replace("Z", "000Z");
+  const home = homedir();
+
+  // Candidate directories to check for incitaciones flat installs
+  const candidates = [
+    path.join(home, ".agents", "skills"),
+    path.join(home, ".config", "agents", "skills"),
+  ];
+
+  // Also check .agents/skills in cwd for project-local installs
+  try {
+    candidates.push(path.join(process.cwd(), ".agents", "skills"));
+  } catch {}
+
+  // Deduplicate
+  const checked = new Set();
+  const dirs = candidates.filter((d) => {
+    if (checked.has(d)) return false;
+    checked.add(d);
+    return fs.existsSync(d);
+  });
+
+  let synced = 0;
+  for (const dir of dirs) {
+    // Check if this directory has incitaciones-installed skills
+    // by reading the first SKILL.md we find and checking its frontmatter
+    let hasIncitaciones = false;
+    try {
+      for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
+        if (!entry.isDirectory()) continue;
+        const skillFile = path.join(dir, entry.name, "SKILL.md");
+        if (!fs.existsSync(skillFile)) continue;
+        const content = fs.readFileSync(skillFile, "utf8");
+        if (content.includes("installed-from: incitaciones") || content.includes("installed-from: \"incitaciones\"")) {
+          hasIncitaciones = true;
+          break;
+        }
+      }
+    } catch {
+      // Permission issues, etc. — skip
+      continue;
+    }
+
+    if (!hasIncitaciones) continue;
+
+    // Re-sync: for each prompt in manifest, copy from pi-package/skills/ to flat dir
+    for (const prompt of manifest.prompts) {
+      const srcDir = path.join(skillsOut, prompt.name);
+      if (!fs.existsSync(srcDir)) continue;
+
+      const dstDir = path.join(dir, prompt.name);
+      fs.rmSync(dstDir, { recursive: true, force: true });
+      fs.mkdirSync(dstDir, { recursive: true });
+
+      // Generate flat-format frontmatter for SKILL.md
+      const distilledPath = path.join(repoRoot, prompt.distilled);
+      if (!fs.existsSync(distilledPath)) continue;
+
+      const description = escapeYamlDoubleQuoted(prompt.description || `Incitaciones prompt: ${prompt.name}`);
+      const frontmatter = [
+        "---",
+        `name: ${prompt.name}`,
+        `description: "${description}"`,
+        "installed-from: incitaciones",
+        `installed-version: "${version}"`,
+        `installed-at: "${now}"`,
+        "---",
+        "",
+      ].join("\n");
+
+      fs.writeFileSync(path.join(dstDir, "SKILL.md"), frontmatter + readText(distilledPath), "utf8");
+
+      // Copy references/ from the source skill directory
+      if (distilledPath.endsWith(`${path.sep}SKILL.md`)) {
+        const srcRefDir = path.dirname(distilledPath);
+        for (const entry of fs.readdirSync(srcRefDir, { withFileTypes: true })) {
+          if (entry.name === "SKILL.md") continue;
+          const s = path.join(srcRefDir, entry.name);
+          const d = path.join(dstDir, entry.name);
+          if (entry.isDirectory()) {
+            fs.cpSync(s, d, { recursive: true });
+          } else {
+            fs.copyFileSync(s, d);
+          }
+        }
+      }
+    }
+    synced++;
+  }
+
+  if (synced > 0) {
+    console.log(`Re-synced incitaciones skills in ${synced} flat-install director(ies)`);
+  }
+}
+
 function main() {
   const manifest = JSON.parse(readText(manifestPath));
   resetDir(outRoot);
@@ -117,6 +235,9 @@ function main() {
   fs.writeFileSync(path.join(outRoot, "README.md"), readme, "utf8");
 
   console.log(`Generated pi resources for ${manifest.prompts.length} prompts in ${path.relative(repoRoot, outRoot)}/`);
+
+  // Re-sync any existing flat-install skill directories (e.g. ~/.agents/skills/)
+  reSyncFlatInstallSkills(manifest);
 }
 
-main();
+main();