openpond-sdk 0.3.3 → 0.5.0

package/MODEL_STARTERS.md

@@ -119,3 +119,29 @@ Keep hidden case inputs and expected results in the private initial-state asset.
 Ordinary task input is policy-visible. The service passes only each hidden case's
 explicit `input` to candidate code; expected results stay with the authored grader.
 See the Evals JavaScript environment contract for service fields and Node execution.
+
+### Complete Taskset evaluation runs
+
+`OpenPondModelTasksetRunsClient` from `openpond-sdk/model-taskset-runs` exposes
+`create`, `get`, `list`, `cancel` and `result` through
+`/v1/model-taskset-runs`. A request pins a Model attachment and an ordered
+population of `{ receiptId, taskId, seed, fixtureId }` members. Receipt IDs are
+unique within the run; hosted seeds are canonical integer strings between
+`0` and `2147483647`. Requests support up to 10,000 members and never accept
+replacement world state, verifier bytes or caller-generated scores.
+
+Use `{ kind: "fixture" }` with each member's authored fixture ID to check a
+package without claiming model execution. Use a `hosted_chat` policy with a
+model ID and sampling/output limits for a model evaluation; those members
+have `fixtureId: null`. A model evaluation pins both the provider snapshot and
+the requested sampling configuration. Reusing an operation ID must return the
+same admitted request or report a conflict.
+
+Details retain the request, immutable public run manifest, policy snapshot and
+current summary. Terminal results retain canonical receipts in admitted order
+and the authored metric only for completed evaluations. Failed and cancelled
+runs cannot publish a metric value; completed but entirely unscorable runs
+retain a null metric value. The client verifies workspace, manifest,
+population, grading role, metric references and result hashes on readback.
+Hosts remain responsible for owner-authenticated evidence and durable
+dispatch/cancellation; a content hash is not producer authentication.

package/TASKSETS.md

@@ -190,3 +190,58 @@ Model selection and retry receipts atomically. Bound Reward packages and reviewe
 batches use their respective authoring graphs and are rejected by this ordinary
 package compiler. Draft persistence, HTTP operations and editor controls are host
 integration work.
+
+## Shared draft documents and compilation
+
+`openpond-sdk/taskset-drafts` exports the complete `TasksetDraftSchema`, the
+authoring validator and the immutable `TasksetDraftWorkspaceSchema`. Desktop
+imports these same contracts and authoring functions. A workspace contains one
+draft document and hashed source files; managed form manifests are synthesized
+from the document instead of stored as a second editable copy.
+
+Use `saveTasksetDraftWorkspaceDocument` for form edits and
+`saveTasksetDraftWorkspaceFile` for file edits. Both require the expected draft
+revision and return a new workspace. File edits also compare the previous file
+hash, preserve exact UTF-8/base64 bytes and reject managed manifests or retained
+source archives. Published workspaces cannot be edited. The JSON workspace is
+limited to 64 MiB; editable file content is limited to 6 MB.
+
+Server-side package helpers are exported from `openpond-sdk/taskset-packages`:
+
+```ts
+const projected = prepareImportedTasksetPackage({
+  package: source, profileId: owner.scopeId, name, createdAt: now,
+});
+const initialized = prepareTasksetDraftSource({
+  source, preparation, expectedModelRevision,
+  sourceDraft: tasksetDraftFromTaskset(projected.taskset, now),
+});
+const workspace = materializeTasksetDraftWorkspace({ initialized, source });
+const publishedPackage = compileModelTasksetDraftWorkspace({
+  workspace, preparation, adapterId: "my-authoring-host", now,
+});
+```
+
+`tasksetDraftFromTaskset` is exported from `openpond-sdk/taskset-drafts`.
+Retain the original `preparation` and source package before initialization.
+Compilation verifies that preparation, applies the same publication validation
+as Desktop, pins changed code and private assets, and seals an owned package.
+It does not execute a model, run graders, select a Taskset or commit host state.
+Imported source still requires explicit source review before publication.
+
+## Hosted draft client contract
+
+`OpenPondTasksetDraftClient` from `openpond-sdk/taskset-drafts` defines the
+scoped `/v1/models/:modelId/taskset-drafts` protocol. Construct it in server code
+with `baseUrl`, `apiKey` and `teamId`. It supports source inspection,
+initialization, listing, document/file reads and edits, explicit Model refresh,
+validation, publication and deletion. Hosting these endpoints requires a server
+implementation; exporting this client alone does not make them available.
+
+Validation pins the exact draft revision and workspace hash and returns the
+compiled package hash and Taskset reference. Publication requires those same
+values plus an operation ID. The host must atomically compare the draft and
+Model revisions, publish/select the package, finalize the draft and retain the
+retry receipt. The client verifies response scope, revisions and file hashes;
+it never retries writes automatically. Hosts own authorization, durable storage,
+pagination, transaction isolation and idempotent recovery.

package/dist/actions-local.js

@@ -1,27 +1,26 @@
+import {
+  PROJECT_ACTION_CONFIG_PATH,
+  isProjectActionDefinition,
+  loadProjectActionConfiguration,
+  resolveProjectActionRuntime
+} from "./chunk-P27DQ65Z.js";
+
 // ../actions/src/local.ts
 import { randomUUID } from "node:crypto";
 import { spawn } from "node:child_process";
-import { promises as fs5 } from "node:fs";
-import path5 from "node:path";
-
-// ../actions/src/build.ts
 import { promises as fs4 } from "node:fs";
 import path4 from "node:path";
+
+// ../actions/src/build.ts
+import { promises as fs3 } from "node:fs";
+import path3 from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
 import { build } from "esbuild";
 
-// ../actions/src/define-action.ts
-import { z } from "zod";
-function isProjectActionDefinition(value) {
-  return Boolean(
-    value && typeof value === "object" && value.kind === "openpond-project-action" && typeof value.id === "string" && typeof value.run === "function"
-  );
-}
-
 // ../actions/src/schema.ts
-import { z as z2 } from "zod";
+import { z } from "zod";
 function projectActionJsonSchema(schema) {
-  return z2.toJSONSchema(schema, { unrepresentable: "any" });
+  return z.toJSONSchema(schema, { unrepresentable: "any" });
 }
 
 // ../actions/src/catalog.ts
@@ -90,92 +89,42 @@ function toCatalogEntry(action) {
   };
 }
 
-// ../actions/src/configuration.ts
+// ../actions/src/discovery.ts
 import { promises as fs } from "node:fs";
 import path from "node:path";
-import { z as z3 } from "zod";
-var PROJECT_ACTION_CONFIG_PATH = "openpond/project-actions.json";
-var connectionSchema = z3.object({
-  values: z3.record(z3.string(), z3.unknown()).optional().default({}),
-  environment: z3.record(z3.string(), z3.string().trim().min(1)).optional().default({})
-}).strict();
-var configSchema = z3.object({
-  sourceDirectory: z3.string().trim().min(1).optional(),
-  outputDirectory: z3.string().trim().min(1).optional(),
-  environment: z3.record(z3.string(), z3.string().trim().min(1)).optional().default({}),
-  connections: z3.record(z3.string(), connectionSchema).optional().default({})
-}).strict();
-async function loadProjectActionConfiguration(projectRoot) {
-  const resolvedRoot = path.resolve(projectRoot);
-  const configPath = path.join(resolvedRoot, PROJECT_ACTION_CONFIG_PATH);
-  const raw = await fs.readFile(configPath, "utf8").catch((error) => {
-    if (error.code === "ENOENT") return null;
-    throw error;
-  });
-  if (raw === null) return configSchema.parse({});
-  let value;
-  try {
-    value = JSON.parse(raw);
-  } catch (error) {
-    const message = error instanceof Error ? error.message : String(error);
-    throw new Error(`${PROJECT_ACTION_CONFIG_PATH} is not valid JSON: ${message}`);
-  }
-  return configSchema.parse(value);
-}
-function resolveProjectActionRuntime(config, hostEnvironment = process.env) {
-  const environment = {};
-  for (const [runtimeName, hostName] of Object.entries(config.environment)) {
-    const value = hostEnvironment[hostName];
-    if (value !== void 0) environment[runtimeName] = value;
-  }
-  const connections = {};
-  for (const [name, descriptor] of Object.entries(config.connections)) {
-    const value = { ...descriptor.values };
-    for (const [property, hostName] of Object.entries(descriptor.environment)) {
-      const environmentValue = hostEnvironment[hostName];
-      if (environmentValue !== void 0) value[property] = environmentValue;
-    }
-    connections[name] = value;
-  }
-  return { environment, connections };
-}
-
-// ../actions/src/discovery.ts
-import { promises as fs2 } from "node:fs";
-import path2 from "node:path";
 var ACTION_FILE = /\.(?:[cm]?[jt]s)$/i;
 var IGNORED_FILE = /(?:\.d\.ts|\.test\.[cm]?[jt]s|\.spec\.[cm]?[jt]s)$/i;
 async function discoverProjectActionFiles(input) {
-  const projectRoot = path2.resolve(input.projectRoot);
-  const sourceRoot = path2.resolve(projectRoot, input.sourceDirectory ?? "openpond/actions");
+  const projectRoot = path.resolve(input.projectRoot);
+  const sourceRoot = path.resolve(projectRoot, input.sourceDirectory ?? "openpond/actions");
   if (!isWithin(projectRoot, sourceRoot)) {
     throw new Error("Project Action source directory must stay inside the Project root.");
   }
-  const stats = await fs2.stat(sourceRoot).catch(() => null);
+  const stats = await fs.stat(sourceRoot).catch(() => null);
   if (!stats?.isDirectory()) {
     throw new Error(`Project Action source directory does not exist: ${sourceRoot}`);
   }
   const files = await walk(sourceRoot);
-  const actions = files.filter((file) => ACTION_FILE.test(file) && !IGNORED_FILE.test(file)).map((file) => path2.relative(projectRoot, file).split(path2.sep).join("/")).sort();
+  const actions = files.filter((file) => ACTION_FILE.test(file) && !IGNORED_FILE.test(file)).map((file) => path.relative(projectRoot, file).split(path.sep).join("/")).sort();
   if (actions.length === 0) {
     throw new Error(`No Project Action files were found under ${sourceRoot}.`);
   }
   return { projectRoot, sourceRoot, files: actions };
 }
 async function walk(directory) {
-  const entries = await fs2.readdir(directory, { withFileTypes: true });
+  const entries = await fs.readdir(directory, { withFileTypes: true });
   const files = [];
   for (const entry of entries.sort((left, right) => left.name.localeCompare(right.name))) {
     if (entry.name.startsWith(".") || entry.name === "node_modules") continue;
-    const target = path2.join(directory, entry.name);
+    const target = path.join(directory, entry.name);
     if (entry.isDirectory()) files.push(...await walk(target));
     else if (entry.isFile()) files.push(target);
   }
   return files;
 }
 function isWithin(root, target) {
-  const relative = path2.relative(root, target);
-  return relative === "" || !relative.startsWith("..") && !path2.isAbsolute(relative);
+  const relative = path.relative(root, target);
+  return relative === "" || !relative.startsWith("..") && !path.isAbsolute(relative);
 }
 
 // ../actions/src/hash.ts
@@ -196,15 +145,15 @@ function sortValue(value) {
 }
 
 // ../actions/src/setup.ts
-import { promises as fs3 } from "node:fs";
+import { promises as fs2 } from "node:fs";
 import { createRequire } from "node:module";
-import path3 from "node:path";
+import path2 from "node:path";
 async function validateProjectActionStaticSetup(projectRoot, requirements) {
   for (const requirement of requirements) {
     if (requirement.required === false) continue;
     if (requirement.kind === "package") {
       try {
-        createRequire(path3.join(projectRoot, "package.json")).resolve(requirement.name);
+        createRequire(path2.join(projectRoot, "package.json")).resolve(requirement.name);
       } catch {
         throw new Error(`Project Action package is not installed: ${requirement.name}`);
       }
@@ -228,10 +177,10 @@ async function validateProjectActionRunSetup(projectRoot, requirements, request)
 }
 async function executableExists(name) {
   if (!name.trim() || name.includes("/") || name.includes("\\")) return false;
-  for (const directory of (process.env.PATH ?? "").split(path3.delimiter).filter(Boolean)) {
-    const candidates = process.platform === "win32" ? (process.env.PATHEXT ?? ".EXE;.CMD;.BAT;.COM").split(";").map((extension) => path3.join(directory, `${name}${extension}`)) : [path3.join(directory, name)];
+  for (const directory of (process.env.PATH ?? "").split(path2.delimiter).filter(Boolean)) {
+    const candidates = process.platform === "win32" ? (process.env.PATHEXT ?? ".EXE;.CMD;.BAT;.COM").split(";").map((extension) => path2.join(directory, `${name}${extension}`)) : [path2.join(directory, name)];
     for (const candidate of candidates) {
-      if (await fs3.access(candidate).then(() => true).catch(() => false)) return true;
+      if (await fs2.access(candidate).then(() => true).catch(() => false)) return true;
     }
   }
   return false;
@@ -245,24 +194,24 @@ async function buildProjectActions(input) {
     projectRoot: input.projectRoot,
     sourceDirectory: input.sourceDirectory ?? configuration.sourceDirectory
   });
-  const outputDirectory = path4.resolve(
+  const outputDirectory = path3.resolve(
     discovered.projectRoot,
     input.outputDirectory ?? configuration.outputDirectory ?? DEFAULT_PROJECT_ACTION_OUTPUT_DIRECTORY
   );
   assertWithinProject(discovered.projectRoot, outputDirectory);
-  await fs4.mkdir(outputDirectory, { recursive: true });
-  const entryPath = path4.join(outputDirectory, "entry.mjs");
-  const bundlePath = path4.join(outputDirectory, "bundle.mjs");
-  const runnerPath = path4.join(outputDirectory, "runner.mjs");
-  const registryPath = path4.join(outputDirectory, "action-registry.json");
-  const manifestPath = path4.join(outputDirectory, "build-manifest.json");
+  await fs3.mkdir(outputDirectory, { recursive: true });
+  const entryPath = path3.join(outputDirectory, "entry.mjs");
+  const bundlePath = path3.join(outputDirectory, "bundle.mjs");
+  const runnerPath = path3.join(outputDirectory, "runner.mjs");
+  const registryPath = path3.join(outputDirectory, "action-registry.json");
+  const manifestPath = path3.join(outputDirectory, "build-manifest.json");
   const imports = discovered.files.map((file, index) => {
-    const relative = path4.relative(outputDirectory, path4.join(discovered.projectRoot, file)).split(path4.sep).join("/");
+    const relative = path3.relative(outputDirectory, path3.join(discovered.projectRoot, file)).split(path3.sep).join("/");
     const specifier = relative.startsWith(".") ? relative : `./${relative}`;
     return `import * as actionModule${index} from ${JSON.stringify(specifier)};`;
   });
   const modules = discovered.files.map((_file, index) => `actionModule${index}`).join(", ");
-  await fs4.writeFile(
+  await fs3.writeFile(
     entryPath,
     [
       ...imports,
@@ -291,22 +240,22 @@ async function buildProjectActions(input) {
     await validateProjectActionStaticSetup(discovered.projectRoot, action.setup);
   }
   const registry = createProjectActionRegistry(actions);
-  await fs4.writeFile(registryPath, canonicalJson(registry), "utf8");
-  await fs4.writeFile(runnerPath, runtimeSource(), { encoding: "utf8", mode: 493 });
-  const bundleHash = sha256(await fs4.readFile(bundlePath));
+  await fs3.writeFile(registryPath, canonicalJson(registry), "utf8");
+  await fs3.writeFile(runnerPath, runtimeSource(), { encoding: "utf8", mode: 493 });
+  const bundleHash = sha256(await fs3.readFile(bundlePath));
   const registryHash = sha256(canonicalJson(registry));
   const manifest = {
     schemaVersion: "openpond.projectActionBuild.v1",
-    sourceDirectory: path4.relative(discovered.projectRoot, discovered.sourceRoot).split(path4.sep).join("/"),
+    sourceDirectory: path3.relative(discovered.projectRoot, discovered.sourceRoot).split(path3.sep).join("/"),
     sourceFiles: discovered.files,
-    bundleFile: path4.basename(bundlePath),
-    runnerFile: path4.basename(runnerPath),
-    registryFile: path4.basename(registryPath),
+    bundleFile: path3.basename(bundlePath),
+    runnerFile: path3.basename(runnerPath),
+    registryFile: path3.basename(registryPath),
     bundleHash,
     registryHash
   };
-  await fs4.writeFile(manifestPath, canonicalJson(manifest), "utf8");
-  await fs4.rm(entryPath, { force: true });
+  await fs3.writeFile(manifestPath, canonicalJson(manifest), "utf8");
+  await fs3.rm(entryPath, { force: true });
   return {
     projectRoot: discovered.projectRoot,
     outputDirectory,
@@ -322,17 +271,17 @@ function dependencySearchPaths(projectRoot) {
   const candidates = [];
   let current = projectRoot;
   while (true) {
-    candidates.push(path4.join(current, "node_modules"));
-    const parent = path4.dirname(current);
+    candidates.push(path3.join(current, "node_modules"));
+    const parent = path3.dirname(current);
     if (parent === current) break;
     current = parent;
   }
-  candidates.push(path4.resolve(path4.dirname(fileURLToPath(import.meta.url)), "../../../node_modules"));
+  candidates.push(path3.resolve(path3.dirname(fileURLToPath(import.meta.url)), "../../../node_modules"));
   return [...new Set(candidates)];
 }
 function assertWithinProject(projectRoot, target) {
-  const relative = path4.relative(projectRoot, target);
-  if (relative.startsWith("..") || path4.isAbsolute(relative)) {
+  const relative = path3.relative(projectRoot, target);
+  if (relative.startsWith("..") || path3.isAbsolute(relative)) {
     throw new Error("Project Action output directory must stay inside the Project root.");
   }
 }
@@ -410,7 +359,7 @@ try {
 // ../actions/src/local.ts
 var localConcurrency = /* @__PURE__ */ new Map();
 function createLocalActionRunner(options) {
-  const projectRoot = path5.resolve(options.projectRoot);
+  const projectRoot = path4.resolve(options.projectRoot);
   const buildPolicy = options.build ?? "if-missing";
   let currentBuild = null;
   const ensureBuild = async () => {
@@ -418,7 +367,7 @@ function createLocalActionRunner(options) {
     if (currentBuild) return currentBuild;
     if (buildPolicy === "never" || buildPolicy === "if-missing") {
       const configuration = await loadProjectActionConfiguration(projectRoot);
-      const outputDirectory = path5.resolve(
+      const outputDirectory = path4.resolve(
         projectRoot,
         options.outputDirectory ?? configuration.outputDirectory ?? DEFAULT_PROJECT_ACTION_OUTPUT_DIRECTORY
       );
@@ -463,17 +412,17 @@ function createLocalActionRunner(options) {
       };
       await validateProjectActionRunSetup(projectRoot, catalogAction.setupRequirements, resolvedRequest);
       const runId = request.runId?.trim() || randomUUID();
-      const actionOutputDirectory = path5.resolve(
-        request.outputDirectory ?? path5.join(projectRoot, ".openpond", "outputs", runId)
+      const actionOutputDirectory = path4.resolve(
+        request.outputDirectory ?? path4.join(projectRoot, ".openpond", "outputs", runId)
       );
       assertWithinProject2(projectRoot, actionOutputDirectory, "Project Action output directory");
-      await fs5.mkdir(actionOutputDirectory, { recursive: true });
+      await fs4.mkdir(actionOutputDirectory, { recursive: true });
       const release = await acquireLocalActionSlot(
         `${projectRoot}\0${request.actionId}`,
         catalogAction.implementation.concurrency,
         resolvedRequest.signal
       );
-      const responseFile = path5.join(built.outputDirectory, `.run-${randomUUID()}.json`);
+      const responseFile = path4.join(built.outputDirectory, `.run-${randomUUID()}.json`);
       const startedAt = Date.now();
       let childResult;
       let payload;
@@ -495,12 +444,12 @@ function createLocalActionRunner(options) {
             connections: resolvedRequest.connections ?? {}
           }
         });
-        payload = JSON.parse(await fs5.readFile(responseFile, "utf8"));
+        payload = JSON.parse(await fs4.readFile(responseFile, "utf8"));
       } catch (error) {
         const message = error instanceof Error ? error.message : String(error);
         throw new Error(redactRuntimeValues(message, resolvedRequest));
       } finally {
-        await fs5.rm(responseFile, { force: true }).catch(() => void 0);
+        await fs4.rm(responseFile, { force: true }).catch(() => void 0);
         release();
       }
       return {
@@ -557,11 +506,11 @@ async function acquireLocalActionSlot(key, limit, signal) {
   };
 }
 async function loadBuild(projectRoot, outputDirectory) {
-  const manifestPath = path5.join(outputDirectory, "build-manifest.json");
-  const registryPath = path5.join(outputDirectory, "action-registry.json");
+  const manifestPath = path4.join(outputDirectory, "build-manifest.json");
+  const registryPath = path4.join(outputDirectory, "action-registry.json");
   const [manifestText, registryText] = await Promise.all([
-    fs5.readFile(manifestPath, "utf8").catch(() => null),
-    fs5.readFile(registryPath, "utf8").catch(() => null)
+    fs4.readFile(manifestPath, "utf8").catch(() => null),
+    fs4.readFile(registryPath, "utf8").catch(() => null)
   ]);
   if (!manifestText || !registryText) return null;
   const manifest = JSON.parse(manifestText);
@@ -569,8 +518,8 @@ async function loadBuild(projectRoot, outputDirectory) {
   return {
     projectRoot,
     outputDirectory,
-    bundlePath: path5.join(outputDirectory, manifest.bundleFile),
-    runnerPath: path5.join(outputDirectory, manifest.runnerFile),
+    bundlePath: path4.join(outputDirectory, manifest.bundleFile),
+    runnerPath: path4.join(outputDirectory, manifest.runnerFile),
     registryPath,
     manifestPath,
     registry,
@@ -645,8 +594,8 @@ function executeChild(input) {
   });
 }
 function assertWithinProject2(projectRoot, target, label) {
-  const relative = path5.relative(projectRoot, target);
-  if (relative.startsWith("..") || path5.isAbsolute(relative)) {
+  const relative = path4.relative(projectRoot, target);
+  if (relative.startsWith("..") || path4.isAbsolute(relative)) {
     throw new Error(`${label} must stay inside the Project root.`);
   }
 }