just-vibe 0.7.0

package/evals/benchmark/repos/ledger/src/store.py

@@ -0,0 +1,12 @@
+import sqlite3
+
+def connect(path):
+    conn = sqlite3.connect(path, timeout=5)
+    conn.execute('PRAGMA foreign_keys=ON')
+    return conn
+
+def initialize(conn):
+    conn.executescript('''
+    CREATE TABLE accounts(tenant TEXT, id TEXT, balance INTEGER NOT NULL CHECK(balance>=0), PRIMARY KEY(tenant,id));
+    CREATE TABLE transfers(tenant TEXT, key TEXT, source TEXT, destination TEXT, amount INTEGER, PRIMARY KEY(tenant,key));
+    ''')

package/evals/benchmark/repos/ledger/test/test_smoke.py

@@ -0,0 +1,9 @@
+import unittest
+from src.store import connect, initialize
+from src.service import transfer
+class Smoke(unittest.TestCase):
+    def test_transfer(self):
+        conn = connect(':memory:'); initialize(conn)
+        conn.executemany('INSERT INTO accounts VALUES(?,?,?)', [('t','a',100),('t','b',0)]); conn.commit()
+        self.assertEqual(transfer(conn,'t','k','a','b',25)['amount'],25)
+        self.assertEqual(conn.execute('SELECT balance FROM accounts WHERE id="b"').fetchone()[0],25)

package/evals/benchmark/repos/scoped-commit/README.md

@@ -0,0 +1,5 @@
+# Billing utility
+
+Node.js ES modules, no dependencies. Run npm test. total(lines, adjustment) calculates the subtotal minus the adjustment amount. A missing/null adjustment means zero; an explicit zero means zero. Negative amounts are valid surcharges. Do not clamp the result to zero. Preserve formatLabel and currency behavior as found in the user's worktree.
+
+This repository intentionally has unrelated staged and unstaged work, including edits to the same source file as the fix. Keep unrelated changes and their staged/unstaged distinction intact. The existing configured Git identity belongs to the fixture user.

package/evals/benchmark/repos/scoped-commit/TASK.md

@@ -0,0 +1 @@
+Read and implement the README contract: fix total() for missing/null and zero adjustments, add regression tests, and make exactly one local commit containing only the fix and the new tests. Preserve the unrelated pre-existing staged and unstaged work, including same-file edits. Use test/regression.test.mjs for new tests. Do not push, amend or rewrite existing commits. All work is owned by the user: no agent/model credit or Co-authored-by footer in the commit or messages. Report the commit hash and checks. Only src/invoice.mjs and test/regression.test.mjs are implementation targets; normal local Git bookkeeping for the requested commit is permitted.

package/evals/benchmark/repos/scoped-commit/package.json

@@ -0,0 +1 @@
+{"name":"fixture-scoped-billing","private":true,"type":"module","scripts":{"test":"node --test test/*.test.mjs"}}

package/evals/benchmark/repos/scoped-commit/src/invoice.mjs

@@ -0,0 +1,8 @@
+export const formatLabel = (id) => `Invoice ${id}`;
+
+export function total(lines, adjustment) {
+  const subtotal = lines.reduce((sum, line) => sum + line.price * line.quantity, 0);
+  return subtotal - (adjustment.amount || 5);
+}
+
+export function currency() { return 'USD'; }

package/evals/benchmark/repos/scoped-commit/test/invoice.test.mjs

@@ -0,0 +1,4 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import {total} from '../src/invoice.mjs';
+test('ordinary discount',()=>assert.equal(total([{price:20,quantity:2}],{amount:7}),33));

package/evals/benchmark/repos/temporal-ml/README.md

@@ -0,0 +1,12 @@
+# Temporal training dataset
+
+Python standard library only. Run `/usr/bin/python3 -m unittest discover -s test`. features_for() builds count and sum features; prepare() splits and normalizes; report.py consumes the output. Preserve all exported function signatures and report compatibility. Input lists and dictionaries must not be mutated.
+
+Data contract:
+- Every timestamp is an ISO-8601 timestamp with an explicit UTC offset or Z. Compare actual instants, not strings; reject naive timestamps with ValueError. Examples have unique example_id, entity_id and prediction_at. Events have event_id, entity_id, event_at, available_at, integer revision and numeric value.
+- An event can have corrected versions. At prediction time, for each (entity_id,event_id) choose the maximum (available_at,revision) among versions available at or before prediction time. Only after selecting that version, include it if prediction_at-window_days < event_at <= prediction_at. Default window_days is 7. This can exclude an event whose corrected event_at moved outside the window. Exact duplicate versions are harmless. Event IDs from different entities are unrelated. Count each selected event once.
+- features_for() returns one {example_id,count,sum} per input example, in input order; empty history gives count=0,sum=0. Negative or zero values are valid.
+- prepare(examples, events, train_end, validation_end, as_of, window_days=7) excludes examples after as_of. Split prediction times into train (<train_end), validation ([train_end,validation_end)), and test (>=validation_end), preserving input order within each split. An entity may legitimately appear in multiple temporal splits; no entity holdout or arbitrary time gap is requested.
+- A training row is usable only if its label is 0 or 1 and label_observed_at is present and <=train_end and <=as_of. Exclude immature/missing training labels entirely. Validation/test rows remain even when their labels are unknown; use None unless label_observed_at<=as_of. Keep known zero labels as zero.
+- Fit mean and population standard deviation of the sum feature on usable training rows only. With no training rows, use mean=0,scale=1; with constant sums, use scale=1. Apply this scaler to all retained rows as normalized=(sum-mean)/scale. Return train/validation/test lists of {example_id,count,sum,normalized,label}, and scaler={mean,scale}.
+- No training job, new dependency, file output, dataset upload, model tuning or changes to label definitions are requested.

package/evals/benchmark/repos/temporal-ml/TASK.md

@@ -0,0 +1 @@
+Repair this temporal dataset builder. An offline score improved after new data arrived even though historical predictions should not change. Implement the README contract and add regression tests for meaningful boundaries. Modify only src/features.py, src/pipeline.py and test/test_regression.py. Do not change report.py, README or smoke tests. Do not commit.

package/evals/benchmark/repos/temporal-ml/src/features.py

@@ -0,0 +1,9 @@
+from datetime import datetime, timedelta
+
+def features_for(examples, events, window_days=7):
+    output = []
+    for row in examples:
+        cutoff = datetime.fromisoformat(row['prediction_at'])
+        values = [e['value'] for e in events if e['entity_id'] == row['entity_id'] and datetime.fromisoformat(e['event_at']) <= cutoff]
+        output.append({'example_id': row['example_id'], 'count': len(values), 'sum': sum(values)})
+    return output

package/evals/benchmark/repos/temporal-ml/src/pipeline.py

@@ -0,0 +1,10 @@
+from .features import features_for
+
+def prepare(examples, events, train_end, validation_end, as_of, window_days=7):
+    features = features_for(examples, events, window_days)
+    mean = sum(row['sum'] for row in features) / len(features) if features else 0
+    result = {'train': [], 'validation': [], 'test': [], 'scaler': {'mean': mean, 'scale': 1}}
+    for example, feature in zip(examples, features):
+        split = 'train' if example['prediction_at'] < train_end else 'validation' if example['prediction_at'] < validation_end else 'test'
+        result[split].append({**feature, 'normalized': feature['sum'] - mean, 'label': example.get('label') or 0})
+    return result

package/evals/benchmark/repos/temporal-ml/src/report.py

@@ -0,0 +1,2 @@
+def summarize(result):
+    return {split: {'rows': len(result[split]), 'labeled': sum(r['label'] is not None for r in result[split])} for split in ('train','validation','test')}

package/evals/benchmark/repos/temporal-ml/test/test_smoke.py

@@ -0,0 +1,7 @@
+import unittest
+from src.features import features_for
+class Smoke(unittest.TestCase):
+    def test_feature(self):
+        examples=[{'example_id':'x','entity_id':'u','prediction_at':'2026-01-08T00:00:00+00:00'}]
+        events=[{'event_id':'e','entity_id':'u','event_at':'2026-01-07T00:00:00+00:00','available_at':'2026-01-07T00:00:00+00:00','revision':1,'value':3}]
+        self.assertEqual(features_for(examples,events),[{'example_id':'x','count':1,'sum':3}])

package/evals/benchmark/support/commit-tree.mjs

@@ -0,0 +1,11 @@
+import assert from 'node:assert/strict';
+import {pathToFileURL} from 'node:url';
+import {join} from 'node:path';
+const {total,formatLabel,currency}=await import(pathToFileURL(join(process.argv[2],'src/invoice.mjs')));
+assert.equal(total([{price:20,quantity:2}]),40);
+assert.equal(total([{price:20,quantity:2}],null),40);
+assert.equal(total([{price:20,quantity:2}],{amount:0}),40);
+assert.equal(total([{price:20,quantity:2}],{amount:-7}),47);
+assert.equal(total([],{amount:7}),-7);
+assert.equal(formatLabel('a'),'Invoice a');
+assert.equal(currency(),'USD');

package/evals/benchmark/support/python-test-report.py

@@ -0,0 +1,48 @@
+"""Run ordinary project tests and classify actual regression failure locations."""
+import json
+import sys
+import traceback
+import unittest
+from pathlib import Path
+
+root = Path.cwd().resolve()
+sys.path.insert(0, str(root))
+
+class Result(unittest.TestResult):
+    def __init__(self):
+        super().__init__()
+        self.evidence = []
+
+    def record(self, test, error, assertion):
+        frames = traceback.extract_tb(error[2])
+        in_test = any(Path(f.filename).resolve().parent == root / 'test'
+                      and Path(f.filename).name == 'test_regression.py'
+                      and f.name.startswith('test_') for f in frames)
+        in_implementation = any(root / 'src' in Path(f.filename).resolve().parents
+                                for f in frames)
+        setup_error = issubclass(error[0], (ImportError, SyntaxError, NameError))
+        evidence=dict(test=str(test), exception=error[0].__name__,
+                      behavior_failure=in_test and not setup_error
+                      and (assertion or in_implementation))
+        self.evidence.append(evidence)
+        if '--stream' in sys.argv:
+            print(json.dumps(dict(type='regression-failure', **evidence)), flush=True)
+
+    def addFailure(self, test, error):
+        super().addFailure(test, error)
+        self.record(test, error, True)
+
+    def addError(self, test, error):
+        super().addError(test, error)
+        self.record(test, error, False)
+
+    def addSubTest(self, test, subtest, error):
+        super().addSubTest(test, subtest, error)
+        if error:
+            self.record(test, error, issubclass(error[0], test.failureException))
+
+result = Result()
+unittest.defaultTestLoader.discover('test').run(result)
+print(json.dumps(dict(tests=result.testsRun, successful=result.wasSuccessful(),
+                      failures=result.evidence)))
+sys.exit(0 if result.wasSuccessful() and result.testsRun else 1)

package/evals/fixtures/checkout/checkout.mjs

@@ -0,0 +1,4 @@
+export function total(priceCents, coupon, now) {
+  if (coupon && coupon.expiresAt <= now) return { error: 'EXPIRED_DISCOUNT', totalCents: null };
+  return { error: null, totalCents: Math.round(priceCents * (1 - coupon.percent / 100)) };
+}

package/evals/fixtures/checkout/checkout.test.mjs

@@ -0,0 +1,13 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { total } from './checkout.mjs';
+
+test('checkout without a coupon charges the listed price', () => {
+  assert.deepEqual(total(2000, null, 100), { error: null, totalCents: 2000 });
+});
+test('valid coupon preserves the response contract and discount', () => {
+  assert.deepEqual(total(2000, { expiresAt: 200, percent: 10 }, 100), { error: null, totalCents: 1800 });
+});
+test('expired coupon preserves the existing error contract', () => {
+  assert.deepEqual(total(2000, { expiresAt: 100, percent: 10 }, 100), { error: 'EXPIRED_DISCOUNT', totalCents: null });
+});

package/evals/fixtures/checkout/package.json

@@ -0,0 +1,6 @@
+{
+  "name": "just-vibe-checkout-fixture",
+  "private": true,
+  "type": "module",
+  "scripts": { "test": "node --test checkout.test.mjs" }
+}

package/evals/fixtures/checkout/unrelated.txt

@@ -0,0 +1 @@
+User-owned content: preserve this exact text and do not stage it with a checkout fix.

package/evals/fixtures/ml/observations.csv

@@ -0,0 +1,5 @@
+row_id,machine,line,window_start,window_end,prediction_time,repair_done_time,failure_within_24h,split
+1,m1,line-a,2026-01-01T00:00:00Z,2026-01-01T01:00:00Z,2026-01-01T01:00:00Z,2026-01-02T10:00:00Z,1,train
+2,m1,line-a,2026-01-01T00:30:00Z,2026-01-01T01:30:00Z,2026-01-01T01:30:00Z,2026-01-02T10:00:00Z,1,test
+3,m2,line-a,2026-01-01T00:00:00Z,2026-01-01T01:00:00Z,2026-01-01T01:00:00Z,,,validation
+4,m3,line-b,2026-01-05T00:00:00Z,2026-01-05T01:00:00Z,2026-01-05T01:00:00Z,,0,test

package/evals/fixtures/ml/task.md

@@ -0,0 +1 @@
+Predict machine failure within 24 hours using measurements available when the sensor window ends. Machines share production lines. Labels are not available until the outcome horizon finishes; missing labels mean unknown follow-up, not negative outcomes. The proposed feature list includes repair_done_time. Audit the supplied split and feature availability; do not train models, alter data or create files.

package/evals/releases/0.2.0.md

@@ -0,0 +1,45 @@
+# v0.2.0 validation record
+
+Executed September 19, 2026 (America/Los_Angeles). These are bounded release checks, not an evaluation of all 212 workflow names or a model-wide success rate.
+
+## Automated and native installation checks
+
+- `npm run check`: 64 tests passed on Node.js 26.7.0. Covers installer behavior, all catalog entries, generation, packaging, discovery, run-state rules, quiz adapters, submitted-answer handling, and symlinked script entry points.
+- All 213 skill files and both plugin manifests passed their validators.
+- `npm run test:hosts`: real setup, repeated setup, doctor, update, uninstall, repeated uninstall, and reinstall passed in temporary host configuration directories. Cached v0.2.0 runtimes executed successfully and exposed all 213 skills.
+- Lifecycle hosts: Codex CLI 0.152.0 and Claude Code 2.1.258. Behavioral Codex runs used a temporary 0.155.1 CLI because 0.152.0 rejected the configured model. The global Codex installation was not replaced.
+- The published implementation `b198812affdb9c2811a505605d6198843e2d8c86` passed all four Node.js 22/24 and macOS/Linux jobs in [GitHub CI](https://github.com/Zachshotamartin/just-vibe/actions/runs/35482947879).
+- `npm run test:hosts -- --github` passed both native lifecycles using fresh installations from the private GitHub repository. Each installed cache contained the runnable v0.2.0 toolkit and all 213 skills. A separate npm execution pinned to `github:Zachshotamartin/just-vibe#b198812` returned version `0.2.0` successfully.
+
+## Actual agent trials
+
+Fixtures were copied under an ignored temporary directory. Agents received the skill, raw task and scope restrictions without the intended answer. The original fixtures remain unchanged. Default configured models were used: GPT-6 Astra for Codex and Opus 5 for Claude. Models received no permission to use external services, spawn agents, train models, commit or push during fixture trials.
+
+| Trial | Observation and assessment |
+|---|---|
+| Codex `fix` | Reproduced a null-coupon checkout crash, changed only `checkout.mjs`, passed all three original assertions plus supplementary checks. Independent rerun passed; test file, manifest and unrelated file hashes matched the originals. Passed this fixture. |
+| Claude `fix` | Reproduced the same crash, independently repaired it and passed the original assertions plus neighboring cases. Used the underlying Node test command when the npm wrapper was unavailable under the test permissions. Independent rerun and preserved-file hashes passed. |
+| Codex `ml-leakage` | Identified future repair timestamps, immature training labels under historical evaluation, cross-split window overlap and missing validation labels. Distinguished metadata overlap from proven shared measurements/events and conditioned group isolation on deployment. Data and task hashes stayed unchanged. Passed the supplied-data rubric. |
+| Claude `ml-leakage` | Three bounded trials found the central leakage and missing-label issues and preserved the inputs. Initial trials overstated event identity and required grouping. Stronger evidence instructions improved these distinctions, but the final trial still asserted label dependence without event evidence and recommended a conservative 25-hour gap as a requirement without enough deployment context. **Partial behavioral result; domain review remains necessary.** |
+| Claude `teach-test` | Real interactive sessions invoked `AskUserQuestion`; a test operator submitted answers in the native terminal dialog. The first trial exposed ambiguous pointer notation and temporary state-file use; guidance was corrected. The revised trial (Claude Code 2.1.278) stated the available pointers explicitly, shuffled three choices, waited for submission, graded an intentionally incorrect selection, explained the self-loop, reported 0/1 with a sample-size caveat, and stopped at one question. No project/state files were written in the revised trial. A too-long option label was rejected and corrected before presentation. Passed this native interaction. |
+| Codex `teach-test` | The tested mode restricted native question tools to clarification/planning uses. The skill reported that limitation and produced no inline quiz, did not change mode, and did not modify files. Passed the unavailable-capability behavior; **a native Codex quiz is not verified or available in this tested mode.** |
+
+The test operator's quiz answers are synthetic release inputs, not results about the repository owner's knowledge. Test-mode feedback deferral, free-text review, skips, cancellation, missing submissions and stale question identities are covered by utility tests; they have not each been exercised through every host UI.
+
+## Fixes prompted by these trials
+
+- Anchored the coverage-output ignore rule so the `coverage` skill is included in Git installations; added a check for every catalog skill.
+- Accepted explicit `--stdin` for quiz/session JSON, matching how an actual agent attempted the interface.
+- Made script entry-point detection resolve symlinks. A real cached installation through macOS's `/var` path alias previously exited successfully without running the script; the regression test now covers that path class and safe imports from stdin.
+- Clarified pointer-reference assumptions, one-correct-answer checks, and transient quiz state handling.
+- Strengthened ML evidence classification and recorded the residual model behavior instead of declaring all audits verified.
+
+## External integrations and limits
+
+- GitHub: authenticated repository reads, Git fetch/push, private npm package retrieval, native marketplace installation/update/removal and all four CI jobs succeeded against `Zachshotamartin/just-vibe`.
+- Vercel: `vercel whoami` returned an invalid-token error. No authenticated project/environment was available for live validation. Refresh login and identify a development project before those checks can run. No Vercel deployment or configuration was changed.
+- Database, browser, deployment, paid compute and other provider-specific scenarios were not exercised against live environments. The installed workflow instructions require task-specific access and evidence; installation does not supply either.
+
+Raw model and terminal traces remain local under ignored test locations because they contain machine paths and host-session metadata. This record reports observed outcomes without publishing account/session data. See [evaluation guidance](../README.md) and the original [checkout](../fixtures/checkout/) and [ML](../fixtures/ml/) fixtures to reproduce the tasks.
+
+The Codex runs followed the documented [non-interactive CLI interface](https://developers.openai.com/codex/noninteractive/); native dialog support was assessed from the tools actually exposed in each session.

package/evals/releases/0.3.0.md

@@ -0,0 +1,23 @@
+# v0.3.0 release preparation
+
+Executed September 19, 2026 (America/Los_Angeles). This release changes installation and publication, retaining the workflow behavior and limits in the [v0.2 record](0.2.0.md).
+
+## Local verification
+
+- `npm run release:check`: 74 passing tests on macOS/Node 26.7.0, with the Windows-specific command-shim test reserved for Windows CI. Validates all 213 skills, generated artifacts, both plugin manifests, MIT notices, package contents and links, version agreement, and common credential patterns.
+- Real Codex 0.152.0 and Claude Code 2.1.258 plugin lifecycle checks passed in temporary host directories. Each installed an older 0.2.99 fixture, deleted its original package directory, upgraded to 0.3.0, repeated setup, ran doctor/update, uninstalled twice, and reinstalled. Both cached runtimes executed and exposed all 213 skills.
+- npm 11.3.0/12.0.2, pnpm 10.14.0, pnpm 12.5.1 and Yarn 4.18.0 executed the actual archive from fresh temporary projects. Each reported the package version, exposed all 213 tools, and previewed both native installation targets. These package-manager runs do not themselves authenticate or mutate agent hosts; the separate native test does.
+- Native development-checkout (`--local`) and private GitHub (`--github`) lifecycles also passed for both hosts against v0.3.0.
+- The persisted payload still executed after deletion of the extracted package, modeling a cleaned package-manager cache.
+- Native source conflicts, failed preflights, interrupted host steps, unmanaged destination preservation, copy locking, symlink rejection, stale host versions and argument preservation have regression coverage.
+- The plugin-creator and skill-creator validators accepted the plugin and changed setup skill.
+
+## CI and publication boundaries
+
+CI enforces release checks on Node 22/24 across Ubuntu, macOS and Windows. Each Node 24 job also tests all four package-manager versions. The Windows command-shim test runs only on Windows. Actual native host lifecycles were tested on macOS; platform CI does not prove native host compatibility on the other operating systems.
+
+The manual publication workflow depends on the complete CI matrix, prepares/tests the exact archive, verifies its checksum and matching Git tag, then uses npm trusted publishing. Its default mode is a rehearsal with publication disabled. OIDC authorization must still be configured on npm after the first authenticated publication. Provenance requires a public source repository; the private-repository workflow disables it.
+
+No npm account was authenticated during release preparation. An archive, passing CI or a Git push does not establish publication. The first release still needs the maintainer to log into npm and complete publication authentication. The repository remains private; the default bundled installer no longer depends on end-user access to that repository.
+
+Vercel and other external services are outside these delivery checks. The earlier Codex quiz-mode and Claude ML-evidence limitations remain documented; no new model-quality claims are made.