opencode-agent-skill 7.7.0 → 10.0.0

package/docs/V9-SPEED-INTELLIGENCE.md

@@ -0,0 +1,102 @@
+# UES 9.0 Speed & Intelligence
+
+UES 9.0 builds on the V8 evidence and reliability gates. The goal is not to make the underlying model generate tokens faster; it is to reduce time-to-correct-result by finding the right code sooner, shrinking unnecessary context, avoiding unsafe retries and requiring evidence before completion.
+
+## Execution profiles
+
+UES classifies work into three execution profiles:
+
+- **FAST**: small/low-risk work, up to 2 skills, 12k context budget, targeted verification, no durable worktree/critic overhead by default.
+- **STANDARD**: medium work, up to 4 skills, 24k context budget, semantic+Git context and targeted/affected verification.
+- **DEEP**: long-horizon or high-risk work, up to 5 skills, 48k context budget, durable state, worktree isolation, critic/integration verification and full CI.
+
+Risk still overrides speed. Authentication, security, payment, schema/migration, production/deploy and public-contract work remains fail-closed.
+
+## Incremental evidence index
+
+The V9 index caches bounded source metadata under `.ues-cache/semantic-index-v1.json`. Unchanged files reuse cached evidence while changed files are reparsed. It records:
+
+- concrete source paths;
+- bounded symbol definitions with line numbers;
+- lexical identifier counts;
+- cache reuse/reparse statistics.
+
+This is intentionally labelled **syntax-aware lexical evidence**. It must not be presented as proof of program semantics or as a full AST/LSP call graph.
+
+Commands:
+
+```bash
+ocskill index status .
+ocskill index build .
+ocskill index rebuild .
+ocskill aci search "checkout total" .
+ocskill aci refs calculateTotal .
+ocskill aci view src/checkout.ts . --line 120 --lines 80
+ocskill aci text "exact marker" .
+```
+
+## Weak-model ACI
+
+The ACI keeps search/view output bounded and evidence-first. Path traversal outside the repository is rejected. Large/binary files are refused by the bounded viewer. Reference results distinguish lexical references from concrete definition lines so an agent cannot honestly claim deeper semantic certainty than the tool measured.
+
+## Runtime traces
+
+Operational trace events are appended under `.ues-traces/*.jsonl`. Common credentials/tokens are redacted before persistence, oversized payloads are hashed/truncated, and traces explicitly exclude hidden chain-of-thought.
+
+```bash
+ocskill trace show <trace-id> .
+```
+
+## Verification sandbox
+
+When Docker or Podman is available, deterministic verification commands can run with:
+
+- network disabled by default;
+- Linux capabilities dropped;
+- no-new-privileges;
+- read-only container root;
+- bounded PIDs/memory/CPU;
+- only the requested workspace bind-mounted;
+- no host secrets forwarded by default.
+
+This protects verification commands; it does **not** claim to sandbox the OpenCode model process itself.
+
+## Benchmark confidence gate
+
+The matrix now embeds paired baseline/UES evidence and computes:
+
+- paired wins/losses/ties;
+- pass-rate delta;
+- exact two-sided sign-test p-value;
+- per-suite regression checks;
+- mean duration ratio;
+- optional cost ratio.
+
+Normal reporting:
+
+```bash
+npm run evals:matrix -- --model provider/model --trials 3
+```
+
+Fail-closed release gate:
+
+```bash
+npm run evals:matrix:gate -- --model provider/model --trials 3
+```
+
+The gate requires enough paired samples, positive uplift, more UES wins than losses, statistical support, no suite regression and acceptable speed. This is evidence for the measured benchmark only; it is not evidence that UES turns a weaker model into a different model.
+
+## Lock and Windows hardening
+
+State locking now uses owner tokens and heartbeats. Stale takeover renames the old lock before removal, and release only removes a lock whose token still belongs to the caller. Windows CLI/router invocation resolves Node-backed shims and refuses unrecognized batch shims rather than falling back to shell execution.
+
+## Release validation
+
+Before publishing 9.0.0:
+
+```bash
+npm run ci
+npm run evals:matrix:gate -- --model provider/model --trials 3
+```
+
+If GitHub-hosted Actions still fails before the first workflow step, treat that as an infrastructure/repository-action issue rather than test evidence; local CI and the paired benchmark remain required release gates.

package/evals/live/tasks.json

@@ -11,13 +11,13 @@
     {
       "id": "auth-resource-ownership",
       "fixture": "fixtures/engineering-bench",
-      "prompt": "Fix canEditResource(user, resource) in src/authz.mjs. Missing inputs deny. Suspended users always deny. Admin users may edit any non-missing resource. Non-admin users may edit only when user.id equals resource.ownerId AND user.tenantId equals resource.tenantId. Return a boolean and do not mutate inputs.",
+      "prompt": "Fix canEditResource(user, resource) in src/authz.mjs. Missing inputs deny. Treat user.role === 'admin' as admin. Treat user.suspended === true as suspended; suspended users always deny, including admins. Admin users may edit any non-missing resource. Non-admin users may edit only when user.id equals resource.ownerId AND user.tenantId equals resource.tenantId. Return a boolean and do not mutate inputs.",
       "grader": "graders/engineering-bench.mjs"
     },
     {
       "id": "api-pagination-contract",
       "fixture": "fixtures/engineering-bench",
-      "prompt": "Fix paginate(items, options) in src/pagination.mjs. pages are 1-indexed; defaults are page=1 and pageSize=10. items must be an array. page/pageSize must be positive integers. Return {items,total,page,pageSize,totalPages}; out-of-range pages return an empty items array. Use TypeError for wrong types and RangeError for invalid numeric ranges.",
+      "prompt": "Fix paginate(items, options) in src/pagination.mjs. Pages are 1-indexed; defaults are page=1 and pageSize=10. items must be an array. page/pageSize must be finite integers: non-number, non-finite, or non-integer values throw TypeError; integer values <= 0 throw RangeError. Return {items,total,page,pageSize,totalPages}; out-of-range pages return an empty items array.",
       "grader": "graders/engineering-bench.mjs"
     },
     {
@@ -29,13 +29,13 @@
     {
       "id": "payment-idempotency",
       "fixture": "fixtures/engineering-bench",
-      "prompt": "Fix applyPaymentEvent(order, event) in src/payment.mjs. Do not mutate order. Duplicate event.id values already in order.processedEvents must be idempotent. A new event must have a non-empty id. For status='succeeded', amountCents and currency must exactly match order.totalCents/order.currency or throw RangeError, then mark status paid. Failed events must not mark paid. Record each new event id exactly once.",
+      "prompt": "Fix applyPaymentEvent(order, event) in src/payment.mjs. Do not mutate order. Duplicate event.id values already in order.processedEvents must be idempotent. A new event.id must be a non-empty string; malformed, missing, or empty ids throw TypeError. For status='succeeded', amountCents and currency must exactly match order.totalCents/order.currency or throw RangeError, then mark status paid. Failed events must not mark paid. Record each new event id exactly once.",
       "grader": "graders/engineering-bench.mjs"
     },
     {
       "id": "webhook-ordering",
       "fixture": "fixtures/engineering-bench",
-      "prompt": "Fix advancePaymentState(state, event) in src/webhook.mjs. Ignore duplicate/stale events whose integer sequence <= state.lastSequence. Newer events must follow legal transitions pending->authorized|failed, authorized->paid|failed, paid->refunded; same-status newer events are allowed. Illegal transitions throw RangeError. Return a new state and never mutate inputs.",
+      "prompt": "Fix advancePaymentState(state, event) in src/webhook.mjs. Ignore duplicate/stale events whose integer sequence <= state.lastSequence by returning the original state object unchanged (the exact same reference). Newer events must follow legal transitions pending->authorized|failed, authorized->paid|failed, paid->refunded; same-status newer events are allowed. Illegal transitions throw RangeError. For accepted newer events, return a new state object and never mutate inputs.",
       "grader": "graders/engineering-bench.mjs"
     },
     {
@@ -77,7 +77,7 @@
     {
       "id": "money-integer-invariants",
       "fixture": "fixtures/engineering-bench",
-      "prompt": "Fix orderTotal(lines) in src/money.mjs. Each line uses integer unitPriceCents>=0, integer quantity>0, optional integer discountCents>=0. Line total is unitPriceCents*quantity-discountCents and may not go below zero. Return an integer cent total. Malformed numeric types use TypeError; invalid ranges use RangeError.",
+      "prompt": "Fix orderTotal(lines) in src/money.mjs. Each line uses integer unitPriceCents>=0, integer quantity>0, optional integer discountCents>=0. Non-number, non-finite, or non-integer numeric fields throw TypeError. Range violations throw RangeError. discountCents may not exceed unitPriceCents*quantity; treat that as a RangeError rather than clamping. Return the sum as an integer cent total.",
       "grader": "graders/engineering-bench.mjs"
     },
     {
@@ -113,7 +113,7 @@
     {
       "id": "cache-invalidation-tags",
       "fixture": "fixtures/engineering-bench",
-      "prompt": "Fix tagsForProductMutation(product) in src/cache-tags.mjs. product must contain a non-empty id. Return unique tags in this order: 'products', 'product:<id>', then optional 'category:<categoryId>' and 'seller:<sellerId>' when those values are non-null/non-empty. Do not emit undefined/empty tags and do not mutate product.",
+      "prompt": "Fix tagsForProductMutation(product) in src/cache-tags.mjs. product must contain a non-empty string id; malformed product/id or an empty id throws TypeError. Return unique tags in this order: 'products', 'product:<id>', then optional 'category:<categoryId>' and 'seller:<sellerId>' when those values are non-null/non-empty. Do not emit undefined/empty tags and do not mutate product.",
       "grader": "graders/engineering-bench.mjs"
     },
     {

package/evals/polyglot/fixtures/polyglot-bench/api/generated/client.ts

@@ -0,0 +1,2 @@
+// GENERATED FILE - DO NOT EDIT
+export type User = { id: string; name: string }

package/evals/polyglot/fixtures/polyglot-bench/api/openapi.json

@@ -0,0 +1,25 @@
+{
+  "openapi": "3.1.0",
+  "info": {
+    "title": "Demo",
+    "version": "1.0.0"
+  },
+  "components": {
+    "schemas": {
+      "User": {
+        "type": "object",
+        "required": [
+          "id"
+        ],
+        "properties": {
+          "id": {
+            "type": "string"
+          },
+          "name": {
+            "type": "string"
+          }
+        }
+      }
+    }
+  }
+}

package/evals/polyglot/fixtures/polyglot-bench/db/migrations/20260920_add_order_key.sql

@@ -0,0 +1 @@
+ALTER TABLE orders ADD COLUMN order_key TEXT NOT NULL;

package/evals/polyglot/fixtures/polyglot-bench/dotnet/OrderService.cs

@@ -0,0 +1,8 @@
+public static class OrderService
+{
+    public static bool CanUpdate(User user, Order order)
+    {
+        if (user == null || order == null) return false;
+        return user.Role == "Admin";
+    }
+}

package/evals/polyglot/fixtures/polyglot-bench/java/PriceService.java

@@ -0,0 +1,5 @@
+public final class PriceService {
+    public static long totalCents(long unitPriceCents, long quantity, long discountCents) {
+        return unitPriceCents * quantity;
+    }
+}

package/evals/polyglot/fixtures/polyglot-bench/monorepo/package.json

@@ -0,0 +1,6 @@
+{
+  "private": true,
+  "workspaces": [
+    "apps/*"
+  ]
+}

package/evals/polyglot/fixtures/polyglot-bench/monorepo/packages/api/package.json

@@ -0,0 +1,4 @@
+{
+  "name": "@demo/api",
+  "version": "1.0.0"
+}

package/evals/polyglot/fixtures/polyglot-bench/monorepo/packages/web/package.json

@@ -0,0 +1,7 @@
+{
+  "name": "@demo/web",
+  "version": "1.0.0",
+  "dependencies": {
+    "@demo/api": "^1.0.0"
+  }
+}

package/evals/polyglot/fixtures/polyglot-bench/monorepo/pnpm-lock.yaml

@@ -0,0 +1,5 @@
+lockfileVersion: '9.0'
+
+importers:
+  .:
+    dependencies: {}

package/evals/polyglot/fixtures/polyglot-bench/next/app/api/products/route.ts

@@ -0,0 +1,7 @@
+export async function GET() {
+  try {
+    return Response.json({ products: [] }, { status: 200 })
+  } catch (error: any) {
+    return Response.json({ error: String(error), stack: error?.stack }, { status: 500 })
+  }
+}

package/evals/polyglot/fixtures/polyglot-bench/python/tenant_auth.py

@@ -0,0 +1,4 @@
+def can_edit(user, resource):
+    if not user or not resource:
+        return False
+    return user.get("role") == "admin"

package/evals/polyglot/fixtures/polyglot-bench/react-native/keyboard.ts

@@ -0,0 +1,3 @@
+export function keyboardOffset(platform: string, safeAreaTop: number, headerHeight: number) {
+  return safeAreaTop + headerHeight
+}

package/evals/polyglot/graders/polyglot-bench.mjs

@@ -0,0 +1,101 @@
+import assert from "node:assert/strict"
+import { createHash } from "node:crypto"
+import { readFile } from "node:fs/promises"
+import path from "node:path"
+
+const root = process.env.UES_EVAL_WORKSPACE || process.cwd()
+const task = process.env.UES_EVAL_TASK
+
+const text = async (file) => readFile(path.join(root, file), "utf8")
+const sha = (value) => createHash("sha256").update(value).digest("hex")
+const noDangerousSql = (value) => assert.doesNotMatch(value, /\b(?:DROP|TRUNCATE|DELETE\s+FROM)\b/i)
+
+switch (task) {
+  case "polyglot-python-tenant-auth": {
+    const source = await text("python/tenant_auth.py")
+    assert.match(source, /def\s+can_edit\s*\(/)
+    assert.match(source, /suspended/i)
+    assert.match(source, /tenant/i)
+    assert.match(source, /owner_id/)
+    assert.match(source, /user(?:\.|\[).*id|user\.id|user\.get\(["']id["']\)/)
+    assert.match(source, /admin/i)
+    assert.doesNotMatch(source, /return\s+True\s*$/m)
+    break
+  }
+  case "polyglot-java-money-validation": {
+    const source = await text("java/PriceService.java")
+    assert.match(source, /long\s+totalCents\s*\(/)
+    assert.match(source, /IllegalArgumentException/)
+    assert.match(source, /unitPriceCents\s*\*\s*quantity/)
+    assert.match(source, /discountCents/)
+    assert.match(source, /discountCents\s*>\s*(?:gross|unitPriceCents\s*\*\s*quantity)/)
+    assert.doesNotMatch(source, /\b(?:double|float)\b/)
+    break
+  }
+  case "polyglot-dotnet-order-authorization": {
+    const source = await text("dotnet/OrderService.cs")
+    assert.match(source, /bool\s+CanUpdate\s*\(/)
+    assert.match(source, /Suspended/)
+    assert.match(source, /TenantId/)
+    assert.match(source, /Admin/)
+    assert.match(source, /OwnerId\s*==\s*user\.Id|order\.OwnerId\s*==\s*user\.Id/)
+    assert.doesNotMatch(source, /return\s+true\s*;/i)
+    break
+  }
+  case "polyglot-nextjs-api-errors": {
+    const source = await text("next/app/api/products/route.ts")
+    assert.match(source, /export\s+async\s+function\s+GET/)
+    assert.match(source, /NOT_FOUND/)
+    assert.match(source, /status\s*:\s*404/)
+    assert.match(source, /INTERNAL/)
+    assert.match(source, /status\s*:\s*500/)
+    assert.doesNotMatch(source, /error\.stack|stack\s*:/)
+    break
+  }
+  case "polyglot-react-native-keyboard": {
+    const source = await text("react-native/keyboard.ts")
+    assert.match(source, /platform\s*===\s*["']ios["']/)
+    assert.match(source, /safeAreaTop\s*\+\s*headerHeight/)
+    assert.match(source, /return\s+headerHeight/)
+    assert.match(source, /Number\.isFinite/)
+    assert.match(source, /android/)
+    break
+  }
+  case "polyglot-safe-sql-migration": {
+    const source = await text("db/migrations/20260920_add_order_key.sql")
+    noDangerousSql(source)
+    const add = source.search(/ADD\s+COLUMN\s+order_key/i)
+    const backfill = source.search(/UPDATE\s+orders[\s\S]*order_key/i)
+    const notNull = source.search(/order_key[\s\S]*SET\s+NOT\s+NULL/i)
+    const unique = source.search(/CREATE\s+UNIQUE\s+INDEX/i)
+    assert.ok(add >= 0 && backfill > add && notNull > backfill && unique > notNull)
+    assert.match(source, /order_key\s*=\s*[^;]*\bid\b/i)
+    break
+  }
+  case "polyglot-monorepo-workspace-boundary": {
+    const rootPkg = JSON.parse(await text("monorepo/package.json"))
+    const webPkg = JSON.parse(await text("monorepo/packages/web/package.json"))
+    assert.ok(Array.isArray(rootPkg.workspaces))
+    assert.ok(rootPkg.workspaces.includes("packages/*"))
+    assert.equal(webPkg.dependencies?.["@demo/api"], "workspace:*")
+    const lock = await text("monorepo/pnpm-lock.yaml")
+    assert.equal(sha(lock), "130256c4e0c0b4db7ec66736ba9afbadb60e908556b993e92fd5c70244bba8d8")
+    break
+  }
+  case "polyglot-generated-contract-discipline": {
+    const api = JSON.parse(await text("api/openapi.json"))
+    const user = api.components?.schemas?.User
+    assert.ok(user?.properties?.id)
+    assert.ok(user?.properties?.name)
+    assert.ok(user?.properties?.displayName)
+    assert.ok(user.required?.includes("id"))
+    assert.ok(user.required?.includes("displayName"))
+    const generated = await text("api/generated/client.ts")
+    assert.equal(sha(generated), "3b4a1e4902a2d678c751f9673a74a5efd352e434142195747268516a30b0ce88")
+    break
+  }
+  default:
+    throw new Error("Unknown polyglot task: " + task)
+}
+
+console.log("polyglot grader PASS: " + task)

package/evals/polyglot/tasks.json

@@ -0,0 +1,54 @@
+{
+  "version": 1,
+  "description": "Polyglot engineering-surface benchmark covering Python, Java/Spring, .NET, Next.js, React Native, SQL migration, monorepo dependency boundaries and generated-code discipline.",
+  "tasks": [
+    {
+      "id": "polyglot-python-tenant-auth",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair python/tenant_auth.py. can_edit(user, resource) must deny missing inputs and suspended users, allow admins only inside the same tenant, and allow members only when both tenant IDs match and resource.owner_id equals user.id. Keep the function pure and return a boolean.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-java-money-validation",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair java/PriceService.java. totalCents(unitPriceCents, quantity, discountCents) must reject negative unit price/discount and non-positive quantity with IllegalArgumentException, calculate integer-cent total exactly, and reject discounts larger than the gross line total. Do not switch to floating point.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-dotnet-order-authorization",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair dotnet/OrderService.cs. CanUpdate(User user, Order order) must deny null inputs and suspended users, require matching TenantId for every role, allow Admin within the same tenant, and otherwise require order.OwnerId == user.Id. Return bool without mutating inputs.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-nextjs-api-errors",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair next/app/api/products/route.ts. GET must return JSON with status 200 on success, map a NOT_FOUND error to a sanitized 404 body {error:{code:'NOT_FOUND',message:'Not found'}}, and map unknown errors to 500 INTERNAL without exposing stack/message details. Preserve the route export.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-react-native-keyboard",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair react-native/keyboard.ts. keyboardOffset(platform, safeAreaTop, headerHeight) must validate non-negative finite offsets, accept only ios/android, return safeAreaTop+headerHeight for iOS and headerHeight for Android, and throw on invalid input.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-safe-sql-migration",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair db/migrations/20260920_add_order_key.sql as a forward-compatible migration: add order_key as nullable first, backfill existing rows deterministically from id, then make it NOT NULL and add a UNIQUE index. Do not drop/truncate tables or delete data.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-monorepo-workspace-boundary",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair the monorepo dependency boundary in monorepo/package.json and monorepo/packages/web/package.json. Root workspaces must include packages/* and web must depend on @demo/api using workspace:*. Do not edit the generated monorepo/pnpm-lock.yaml.",
+      "grader": "graders/polyglot-bench.mjs"
+    },
+    {
+      "id": "polyglot-generated-contract-discipline",
+      "fixture": "fixtures/polyglot-bench",
+      "prompt": "Repair api/openapi.json so the User schema preserves the legacy name field and also exposes displayName, with id and displayName required. Do not edit api/generated/client.ts because it is generated output; only the source contract should change.",
+      "grader": "graders/polyglot-bench.mjs"
+    }
+  ]
+}