@prismer/runtime 2.0.8 → 2.2.63

package/built-in-skills/skill-creator/references/skill-authoring/GUIDE.md

@@ -0,0 +1,134 @@
+---
+name: skill-authoring
+scope: common
+description: Low-level helper for the DRAFT-GATED (review) tier of skill authoring — it is NOT the entry point for creating skills. To make/build/author a skill, use `skill-creator` (the single canonical skill author). This page only documents skill-creator's draft tier — generating a `status=draft` skill via `POST /api/im/skills/draft` (`cloud skill draft create`) for human review in Studio Authoring, plus the skill.json + sampleTasks/acceptanceCriteria test contract. Do NOT route here when the user wants to create a skill — route to skill-creator and add `--draft` only when human review is required.
+license: MIT
+compatibility:
+  - prismer-sdk
+  - hermes
+  - claude-code
+  - codex
+---
+
+# Skill Authoring — the draft-gated tier of `skill-creator`
+
+> **Not an authoring entry point.** Creating or editing a skill is the job of
+> **`skill-creator`** — the single canonical skill author. This page only
+> documents one of skill-creator's two publish trust tiers (**draft-gated**),
+> for the case where a skill must be reviewed by a human before it goes live.
+> If the user says "make this a skill" / "create a skill for X", **use
+> `skill-creator`**, not this.
+
+## Where this fits
+
+`skill-creator` drives the full lifecycle author→validate→test→package→publish.
+At the publish step there are two trust tiers ([[16-skill-role-authoring-as-agent-capability]] §5.3):
+
+- **Direct create (default)** — `cloud skill create ./<dir>` → `status=active`,
+  `publishScope=workspace` (**private**, installable — NOT the public marketplace).
+  Putting it in the marketplace is a separate `cloud skill publish`. Under `skill-builder`.
+- **Draft-gated (explicit, this page)** — `cloud skill draft create` →
+  `status=draft`, Studio Authoring reviews and promotes it. Opt into this **only
+  when human review is required** — the default is direct publish (decision 4,
+  locked 2026-06-24).
+
+Both tiers are reached from inside `skill-creator`; this is a publish-verb
+choice, not a separate skill to author in.
+
+## Draft submission (the review-gated path)
+
+When the workflow requires human review before a skill is live, submit a draft
+instead of publishing directly:
+
+```bash
+cloud skill draft create ./<bundle-dir>   # optional: --workspace-id <id> --json
+# → POST /api/im/skills/draft. Cloud reads the directory, runs the 7 validation
+#   gates, and returns { id, slug, status: draft }. Does NOT publish.
+```
+
+> ⚠️ **`<bundle-dir>` is a directory** holding `SKILL.md` + `skill.json` (+ an
+> optional `scripts/`). The CLI reads and packages the directory on every run, so
+> edits on disk take effect immediately — there is no separate manifest to
+> regenerate. There is **no** `--slug` / `--manifest` flag; the slug is taken from
+> the SKILL.md frontmatter `name`.
+
+The draft lands in `status=draft`; the workspace owner reviews and promotes it
+via Studio Authoring (`/evolution → Studio → Authoring`). Do **not** auto-publish
+or chain into install — promotion is the owner's call.
+
+> Do NOT call `POST /api/im/skills` (direct publish) when the intent is review-gated —
+> that bypasses the draft state. Use `--draft` deliberately.
+
+## `skill.json` — where security + the test contract live
+
+**Read this first — it is the #1 draft-tier gotcha.** The bundle must contain a
+`skill.json` alongside `SKILL.md` (cloud locates each file by name, so a
+`scripts/` dir sorted in between is fine). Cloud reads `security` and the test
+contract from **`skill.json`**, and **NEVER** from the `SKILL.md` YAML
+frontmatter. If you put `security:` / `dataAccess:` in the
+SKILL.md frontmatter, gate 5 still fails with "security.dataAccess must be
+non-empty" no matter how you format the YAML — it is reading a different file.
+
+### `security` (blocking — gate 5)
+
+```json
+{
+  "schemaVersion": 1,
+  "slug": "<slug>",
+  "security": {
+    "dataAccess": ["workspace-assets"],
+    "humanApprovalRequiredFor": []
+  }
+}
+```
+
+- **`security.dataAccess`** — non-empty array. Scopes: `workspace-assets` |
+  `memory` | `external-network` | `secrets` | `filesystem`.
+- **`security.humanApprovalRequiredFor`** — **required non-empty IF** `dataAccess`
+  contains a **sensitive** scope (`external-network`, `secrets`, `filesystem`).
+  List the actions needing human approval (e.g. `["filesystem-write"]`).
+- **Least privilege:** if the skill only reads env + prints (no arbitrary user
+  files), declare `["memory"]` — a non-sensitive scope — and
+  `humanApprovalRequiredFor` may stay empty. Only reach for `filesystem` when the
+  skill genuinely reads/writes user files, and then you must also declare
+  approval.
+
+### test contract (drives `cloud skill test`)
+
+The same `skill.json` also carries the VERIFICATION contract exercised by
+`cloud skill test --agent` in lifecycle step ③:
+
+- `runtime.kind` = `'inline-script'` (or `'http-endpoint'`), `runtime.requires`
+  declaring `bins`/`env` the script needs.
+- `inputs` / `outputs` describing the call contract.
+- **`sampleTasks[]`** — at least 2 concrete tasks. EACH MUST have
+  `acceptanceCriteria[]` written as substrings/regex the dispatch OUTPUT must
+  contain (e.g. `"\"status\":\\s*200"`, `"results"`). These are the auto-scored
+  tests: `cloud skill test` dispatches each sampleTask to a real agent and
+  matches its output against the criteria. A sampleTask with no
+  acceptanceCriteria scores `inconclusive` (NOT a pass), so always write them.
+
+> Derive acceptanceCriteria from the spec: required response fields, status
+> codes, schema keys. Tight enough to catch a broken call, not so tight they
+> depend on volatile data.
+
+## Draft validation gates (self-check before submit)
+
+The cloud server runs these at `createDraft` time and rejects with HTTP 400 on a
+blocking failure. Run the same checks locally first (or just `cloud skill validate ./<dir>`):
+
+| Gate           | Check                                                            | Blocking |
+|----------------|------------------------------------------------------------------|----------|
+| `manifest`     | files[] complete; merkle root reproducible                       | yes      |
+| `frontmatter`  | `name` matches `^[a-z][a-z0-9-]*$`; description ≥ 50 quality units (ASCII=1, non-ASCII=2); YAML `|`/`>` supported | yes      |
+| `package`      | bundle contains SKILL.md + skill.json (located by name)          | yes      |
+| `requires`     | runtime.requires declares env/bins/python/node explicitly        | warn     |
+| `security`     | **skill.json** security.dataAccess non-empty; sensitive scopes (external-network/secrets/filesystem) require security.humanApprovalRequiredFor. NOT read from SKILL.md frontmatter. | yes      |
+| `sample`       | at least 1 sampleTask + 1 acceptance criterion                   | warn     |
+| `runtime`      | sandbox executes sample task                                     | deferred |
+
+## Reference
+
+- The skill author (use this): `built-in-skills/skill-creator/SKILL.md`
+- Direct-publish tier: `built-in-skills/skill-creator/references/skill-builder/GUIDE.md`
+- Bundle + catalog contract: `public/docs/Standardization/01-skill-standard.md` (SS-01)

package/built-in-skills/{skill-authoring/skill.json → skill-creator/references/skill-authoring/draft-contract.json}

@@ -17,7 +17,7 @@
     "kind": "text-workflow",
     "executableJson": {
       "kind": "cli",
-      "command": "cloud skill draft create --slug {{slug}} --manifest {{manifest_path}}"
+      "command": "cloud skill draft create {{bundle_dir}}"
     },
     "requires": {
       "env": ["PRISMER_API_KEY"],
@@ -27,7 +27,7 @@
   },
   "inputs": [
     { "name": "sourceKind", "type": "enum<inline-spec|doc-url|code-source|service-endpoint>", "required": true, "description": "Source kind chosen by the user or inferred from the slash command" },
-    { "name": "intent", "type": "string", "required": true, "description": "Free-form description of what the produced skill should do; ≥ 50 chars" },
+    { "name": "intent", "type": "string", "required": true, "description": "Free-form description of what the produced skill should do; ≥ 50 quality units (ASCII=1, non-ASCII=2)" },
     { "name": "sourceRefs", "type": "array<string>", "required": false, "description": "URLs / repo paths / endpoint URLs depending on sourceKind" }
   ],
   "outputs": [
@@ -47,7 +47,7 @@
       ],
       "acceptanceCriteria": [
         "IMSkill row exists with status=draft and slug matches user-provided value",
-        "manifest files[0].path === 'SKILL.md' and files[1].path === 'skill.json'",
+        "manifest contains a SKILL.md and a skill.json (located by basename, any order — the package gate does NOT require them at positions [0]/[1])",
         "GET /api/im/skills/search?q=<slug> returns 0 rows (draft filter applies)"
       ]
     }

package/built-in-skills/skill-creator/references/skill-builder/GUIDE.md

@@ -0,0 +1,167 @@
+---
+name: skill-builder
+scope: common
+description: Low-level helper for the DIRECT-PUBLISH tier of skill authoring — it is NOT the entry point for creating skills. To make/build/author/package a skill, use `skill-creator` (the single canonical skill author). This page only documents `skill-creator`'s direct-publish tier (`cloud skill create --install`, or the portable `scripts/ingest.mjs` ingest fallback) and the SS-01 bundle/ingest format. Do NOT route here when the user wants to create a skill — route to skill-creator.
+license: MIT
+compatibility:
+  - prismer-sdk
+  - hermes
+  - claude-code
+  - codex
+metadata:
+  category: authoring
+allowed-tools:
+  - Read
+  - Write
+  - Bash
+  - WebFetch
+---
+
+# Skill Builder — the direct-publish tier of `skill-creator`
+
+> **Not an authoring entry point.** Creating, editing, or packaging a skill is
+> the job of **`skill-creator`** — the single canonical skill author. This page
+> only exists to document one of skill-creator's two publish trust tiers
+> (**direct publish**) plus the low-level `scripts/ingest.mjs` ingest helper.
+> If the user says "turn this into a skill" / "make a skill" / "package this
+> workflow", **use `skill-creator`**, not this.
+
+## Where this fits
+
+`skill-creator` drives the full lifecycle author→validate→test→package→publish.
+At the publish step there are two trust tiers ([[16-skill-role-authoring-as-agent-capability]] §5.3):
+
+- **Direct create (default)** — `cloud skill create ./<dir> [--install]` →
+  `status=active`, `publishScope=workspace` (**private**, installable — NOT public).
+  Marketplace publish is the separate `cloud skill publish`. This page.
+- **Draft-gated (explicit)** — `cloud skill draft create` → `status=draft`,
+  Studio Authoring promotes it. Documented under `skill-authoring`.
+
+Both tiers are reached from inside `skill-creator`. Neither is a separate
+"author a skill" skill — they're publish-verb choices.
+
+## SS-01 bundle format (reference)
+
+The canonical contract is **SS-01** (`public/docs/Standardization/01-skill-standard.md`).
+A skill is a directory named exactly `<slug>/`:
+
+```
+<slug>/
+├── SKILL.md        # required — frontmatter + body (injected into agent prompt)
+├── scripts/        # optional — runnable code
+├── references/     # optional — docs loaded as needed
+└── assets/         # optional — templates / images
+```
+
+`SKILL.md` frontmatter (SS-01 §3.2 — aligns with agentskills.io):
+
+```markdown
+---
+name: <slug>                 # must equal the dir name
+scope: common                # common | persistence | coding
+description: <one line; what it does AND when to trigger>
+license: MIT
+compatibility:
+  - prismer-sdk
+  - claude-code
+metadata:
+  category: <category>
+---
+
+# <Title>
+<body: imperative instructions, progressive disclosure, ≤500 lines>
+```
+
+`scope` buckets the skill for per-role native-skill filtering (release203/13):
+`common` (default, any agent) / `persistence` (memory/state) / `coding`
+(code-agent targets). It lives inside SKILL.md and round-trips automatically.
+
+## Direct-publish ingest
+
+The preferred path is the `cloud skill create` CLI (driven from `skill-creator`):
+
+```bash
+cloud skill create ./<slug> --install --json
+# POST /api/im/skills (+ install). Reads api key from prismer config.
+```
+
+The legacy script entrypoint is retained as a Cloud CLI adapter. It requires
+`cloud` on PATH (or `PRISMER_CLOUD_BIN` pointing to the current build):
+
+```bash
+# Credentials come from the injected environment, never literal command text.
+node references/skill-builder/scripts/ingest.mjs ./<slug> --install
+```
+
+The adapter delegates frontmatter parsing, complete resource/NOTICE inclusion,
+manifest hashing and install error handling to `cloud skill create`. It does
+not perform raw HTTP fallback or implement a separate YAML parser.
+
+If `cloud` is present but `skill create` is missing, that is CLI version skew,
+not permission to bypass the lifecycle. Report `cloud --version` and use the
+current repository build. The script fallback is not a workaround for a
+partially stale CLI because its auth/scope/readback behavior is precisely what
+the authoring flow needs to verify.
+
+The bundle reader accepts YAML literal/folded block scalars in frontmatter, so
+upstream `description: |` / `description: >` content must not be rewritten to a
+single line merely for ingest. The 50-unit description quality floor counts
+non-ASCII code points as two units, avoiding meaningless padding of concise CJK
+descriptions while retaining the English quality bar.
+
+> **The catalog slug is server-assigned**, not your frontmatter `name`. Community
+> creates are prefixed (`weather-lookup` → `community-weather-lookup`). Use the
+> slug returned in the 201 response for install, `cloud skill show`, and a role's
+> `requiredSkills`. A `409` means the slug already exists — pick a new name;
+> there is no in-place overwrite.
+
+## After publishing: managing a live skill
+
+Publishing is not a one-way door. Every verb below is a `cloud skill` subcommand
+(the `prismer skill` daemon CLI has the identical set).
+
+**The one fact everything else follows from: an installed skill is a LIVE
+REFERENCE, not a snapshot.** `im_agent_skills` stores only `skillId` +
+`installedRevision`; every daemon sync re-reads the catalog row's current
+`contentManifest`. So **editing a published skill's content propagates to every
+agent that has it installed** — no re-install, no opt-in. That is why the server
+rejects a content edit with `422 changelog_required` unless you pass
+`--changelog "<what changed>"`: consumers are entitled to know what changed under
+them. Metadata-only edits (description / icon / tags) need no changelog.
+
+| Verb                                        | What it does                                                                            | Effect on people already using it                                        |
+| ------------------------------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
+| `cloud skill delist <slug> [--reason ...]`  | Removes it from Marketplace search. Content, installs, syncs all keep working.          | **None.** They keep the skill and keep receiving content updates.        |
+| `cloud skill relist <slug>`                 | Puts it back (re-runs the publish gate, so an admin takedown still blocks it).          | None.                                                                     |
+| `cloud skill deprecate <slug> --reason <r> [--successor <slug>]` | Marks it superseded. It stays installable — new installs get a **warning, not a block**. | None; they see the reason + successor.                                    |
+| `cloud skill undeprecate <slug>`            | Clears the mark.                                                                        | None.                                                                     |
+| `cloud skill archive <slug> [--confirm]`    | **Retires it. UNBINDS every agent that has it installed.**                              | **Destructive.** With live consumers the server refuses until `--confirm`. |
+| `cloud skill transfer <slug> --to <imUserId>` | Offers ownership. Two-phase — nothing moves until the recipient runs `transfer-accept`. | None. (`transfer-abort` withdraws the offer.)                             |
+| `cloud skill published`                     | Your publication inventory: slug, listing state, **consumer count**.                    | —                                                                         |
+
+**delist vs archive is the distinction that matters.** Delist is a visibility
+flip: it hides the skill from the Marketplace and nothing else. Archive retires
+it and unbinds every installed agent — that is why it has a consumer door:
+
+```bash
+$ cloud skill archive weather-lookup
+Error: 3 agents are still using this skill — archiving unbinds them.
+       Re-run with --confirm to archive anyway: cloud skill archive weather-lookup --confirm
+```
+
+Reach for delist when you no longer want new users; reach for archive only when
+you intend to cut existing ones off. Deprecate is the polite middle: it keeps
+working and points people at the successor. There is **no hard delete** — a
+published skill is never destroyed, only moved between these states.
+
+Admin takedown is a separate, admin-only action. A taken-down skill cannot be
+re-listed by its owner (`cloud skill relist` returns "was taken down by an
+administrator"); an admin must reinstate it first.
+
+## Reference
+
+- The skill author (use this): `built-in-skills/skill-creator/SKILL.md`
+- Bundle + catalog contract: `public/docs/Standardization/01-skill-standard.md` (SS-01)
+- Draft-gated tier: `built-in-skills/skill-creator/references/skill-authoring/GUIDE.md`
+- Role side: `built-in-skills/role-builder/SKILL.md` (SS-02)
+- Post-publish endpoint contract: `docs/api/publish-lifecycle.md` (product204/21)

package/built-in-skills/skill-creator/references/skill-builder/scripts/ingest.mjs

@@ -0,0 +1,93 @@
+#!/usr/bin/env node
+// skill-builder ingest — material→bundle→live catalog, via SDK + API key.
+//
+// Zero runtime dependencies (Node ≥18 built-ins only). Reads a SS-01 skill
+// bundle directory, parses SKILL.md frontmatter, builds an SS-01 contentManifest
+// with a server-matching merkle revision, and POSTs it to the real catalog
+// endpoint authenticated with an `sk-prismer-*` API key.
+//
+// Usage:
+//   PRISMER_API_KEY=sk-prismer-... [PRISMER_CLOUD_BASE=http://127.0.0.1:3000] \
+//     node scripts/ingest.mjs <bundle-dir> [--install] [--agent <imUserId>] [--json]
+//
+// Env:
+//   PRISMER_API_KEY   (required) Bearer token; admin not required for skill create.
+//   PRISMER_CLOUD_BASE (default http://127.0.0.1:3000) cloud base URL.
+
+import { spawnSync } from 'node:child_process';
+
+function die(msg, code = 1) {
+  console.error(`[skill-builder] ✗ ${msg}`);
+  process.exit(code);
+}
+
+const args = process.argv.slice(2);
+const secretOptions = new Set(['--api-key', '--token', '--jwt', '--secret', '--password']);
+let bundleDir;
+let wantInstall = false;
+let asJson = false;
+let agentId;
+for (let index = 0; index < args.length; index += 1) {
+  const arg = args[index];
+  const equalsAt = arg.indexOf('=');
+  const option = equalsAt === -1 ? arg : arg.slice(0, equalsAt);
+  if (secretOptions.has(option)) {
+    die(`${option} is forbidden; credentials must come from environment variables`, 2);
+  }
+  if (arg === '--install') {
+    wantInstall = true;
+    continue;
+  }
+  if (arg === '--json') {
+    asJson = true;
+    continue;
+  }
+  if (arg === '--agent') {
+    const value = args[index + 1];
+    if (!value || value.startsWith('--')) die('--agent requires an imUserId', 2);
+    agentId = value;
+    index += 1;
+    continue;
+  }
+  if (arg.startsWith('--')) die(`unknown argument at position ${index + 1}`, 2);
+  if (bundleDir) die('only one bundle directory may be supplied', 2);
+  bundleDir = arg;
+}
+
+const API_KEY = process.env.PRISMER_API_KEY;
+const BASE = (process.env.PRISMER_CLOUD_BASE || 'http://127.0.0.1:3000').replace(/\/$/, '');
+
+if (!bundleDir) die('usage: node ingest.mjs <bundle-dir> [--install] [--agent <imUserId>] [--json]');
+if (!API_KEY) die('PRISMER_API_KEY env is required (sk-prismer-*)');
+let target;
+try {
+  target = new URL(BASE);
+} catch {
+  die('PRISMER_CLOUD_BASE must be a valid URL', 2);
+}
+if (
+  !['http:', 'https:'].includes(target.protocol) ||
+  target.username ||
+  target.password ||
+  target.search ||
+  target.hash || target.pathname !== '/'
+) {
+  die('PRISMER_CLOUD_BASE must be a plain http(s) origin without credentials, query, or hash', 2);
+}
+const isLocalTarget = ['localhost', '127.0.0.1', '::1', 'host.docker.internal'].includes(target.hostname);
+const allowRemote = ['1', 'true', 'yes'].includes(String(process.env.PRISMER_ALLOW_REMOTE_WRITE).toLowerCase());
+if (!isLocalTarget && !allowRemote) {
+  die('remote mutation is blocked; set PRISMER_ALLOW_REMOTE_WRITE=1 after confirming the target environment', 2);
+}
+
+// Keep the legacy entrypoint, but use the canonical structured parser and CLI.
+const command = ['skill', 'create', bundleDir];
+if (wantInstall) command.push('--install');
+if (agentId) command.push('--agent', agentId);
+if (asJson) command.push('--json');
+const result = spawnSync(process.env.PRISMER_CLOUD_BIN || 'cloud', command, {
+  stdio: 'inherit', shell: false, timeout: 120_000,
+  env: { ...process.env, PRISMER_CLOUD_BASE: BASE },
+});
+if (result.error) die('Cloud CLI failed or timed out; inspect remote state before retrying', 5);
+process.exitCode = result.status ?? 5;

package/built-in-skills/skill-creator/scripts/aggregate_benchmark.py

@@ -150,7 +150,8 @@ def load_run_results(benchmark_dir: Path) -> dict:
                 metrics = grading.get("execution_metrics", {})
                 result["tool_calls"] = metrics.get("total_tool_calls", 0)
                 if not result.get("tokens"):
-                    result["tokens"] = metrics.get("output_chars", 0)
+                    result["tokens"] = metrics.get("total_tokens", 0)
+                result["output_chars"] = metrics.get("output_chars", 0)
                 result["errors"] = metrics.get("errors_encountered", 0)
 
                 # Extract expectations — viewer requires fields: text, passed, evidence
@@ -268,7 +269,7 @@ def generate_benchmark(benchmark_dir: Path, skill_name: str = "", skill_path: st
             "analyzer_model": "<model-name>",
             "timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
             "evals_run": eval_ids,
-            "runs_per_configuration": 3
+            "runs_per_configuration": {config: len(entries) for config, entries in results.items()}
         },
         "runs": runs,
         "run_summary": run_summary,