lab-kit-cli 0.1.0__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- lab_kit_cli-0.3.0/.github/workflows/policy.yml +59 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/AGENTS.md +19 -0
- lab_kit_cli-0.3.0/CHANGELOG.md +26 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/PKG-INFO +24 -2
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/README.md +23 -1
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/spec/lab-model.md +23 -16
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/experiment/SKILL.md +2 -2
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/plan-mission/SKILL.md +8 -7
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/run-mission/SKILL.md +4 -4
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0/examples/monte-carlo-lab/.lab}/method/DISCIPLINE.md +6 -2
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0/examples/monte-carlo-lab/.lab}/method/LADDER.md +1 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.folio/catalog.json +17 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/lab.yaml +1 -1
- {lab_kit_cli-0.1.0/examples/monte-carlo-lab/.lab → lab_kit_cli-0.3.0}/method/DISCIPLINE.md +6 -2
- {lab_kit_cli-0.1.0/examples/monte-carlo-lab/.lab → lab_kit_cli-0.3.0}/method/LADDER.md +1 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/pyproject.toml +1 -1
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/skills/experiment/SKILL.md +2 -2
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/skills/plan-mission/SKILL.md +8 -7
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/skills/run-mission/SKILL.md +4 -4
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/__init__.py +1 -1
- lab_kit_cli-0.3.0/src/lab_kit/budgets.py +94 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/checks/files.py +89 -11
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/checks/locks.py +7 -1
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/commands/experiments.py +2 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/commands/runs.py +3 -1
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/commands/status.py +16 -1
- lab_kit_cli-0.3.0/src/lab_kit/ops.py +183 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/records.py +43 -2
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/supervise.py +2 -1
- lab_kit_cli-0.3.0/tests/test_budgets.py +213 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/tests/test_checks.py +45 -0
- lab_kit_cli-0.3.0/tests/test_instrument.py +93 -0
- lab_kit_cli-0.1.0/CHANGELOG.md +0 -12
- lab_kit_cli-0.1.0/src/lab_kit/ops.py +0 -94
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/.github/workflows/ci.yml +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/.github/workflows/release.yml +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/.gitignore +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/CLAUDE.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/CONTRIBUTING.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/LICENSE +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/SETUP.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/agents/reporter.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/agents/reviewer.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/agents/runner.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/agents/scout.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/figures/how-lab-kit-works-dark.svg +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/figures/how-lab-kit-works-light.svg +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/figures/lab-loop-dark.svg +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/figures/lab-loop-light.svg +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/figures/number-chain-dark.svg +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/docs/figures/number-chain-light.svg +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/agents/reporter.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/agents/reviewer.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/agents/runner.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/agents/scout.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/address/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/configure/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/organise/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/publish/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/review/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/run/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/set-up/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/set-up-lab/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/write/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.gitignore +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.lab/frozen.sha256 +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/AGENTS.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/CLAUDE.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.folio/backlinks.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.folio/cards.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.folio/journal.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.folio/nav.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.folio/search.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/.gitignore +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/assets/data/.gitkeep +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/assets/figures/.gitkeep +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/assets/refs.bib +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/claims/C-1.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/index.html +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-concluded-the-pi-error-scaling-mission.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-lab-set-up.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-locked-the-pi-error-scaling-protocol.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-opened-the-pi-error-scaling-mission.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-r-1-fitted-the-mean-absolute-error.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-ran-the-pi-error-scaling-protocol.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-recorded-r-1.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-recorded-r-2.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-reported-the-pi-error-scaling-experiment.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/journal/2026/2026-10-04-the-square-root-law-holds-per-size-predictions.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/maps/research.html +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/projects/lab/index.html +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/protocols/pi-error-scaling.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/questions/Q-1.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/reports/pi-error-scaling-report/index.html +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/results/R-1.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/content/results/R-2.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/docs/folio.yaml +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/.gitkeep +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/.gitignore +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/bin/.gitkeep +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/bin/fold.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/bin/sample.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/lock.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/runs/.gitkeep +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/runs/20261004-231629/MANIFEST.sha256 +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/runs/20261004-231629/out/estimates.tsv +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/runs/20261004-231629/run.json +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/experiments/pi-error-scaling/score.yaml +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/ops/STATE.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/ops/missions/.gitkeep +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/ops/missions/2026-10-04-pi-error-scaling.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/substrate/README.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/substrate/estimator.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/skills/review/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/skills/set-up-lab/SKILL.md +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/__main__.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/checks/__init__.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/checks/base.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/checks/results.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/checks/run.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/cli.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/commands/__init__.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/commands/setup.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/data.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/errors.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/frozen.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/gitlog.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/lab.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/protocol.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/src/lab_kit/rederive.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/tests/conftest.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/tests/test_end_to_end.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/tests/test_example.py +0 -0
- {lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/tests/test_runs.py +0 -0
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
name: policy
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
pull_request:
|
|
5
|
+
types: [opened, edited, synchronize, reopened]
|
|
6
|
+
push:
|
|
7
|
+
branches: [main]
|
|
8
|
+
|
|
9
|
+
permissions:
|
|
10
|
+
contents: read
|
|
11
|
+
|
|
12
|
+
jobs:
|
|
13
|
+
policy:
|
|
14
|
+
runs-on: ubuntu-latest
|
|
15
|
+
steps:
|
|
16
|
+
- uses: actions/checkout@v5
|
|
17
|
+
with:
|
|
18
|
+
fetch-depth: 0
|
|
19
|
+
- name: fetch the pull request head
|
|
20
|
+
if: github.event_name == 'pull_request'
|
|
21
|
+
run: git fetch -q origin "+refs/pull/${{ github.event.pull_request.number }}/head"
|
|
22
|
+
- name: check
|
|
23
|
+
env:
|
|
24
|
+
P_TEXT: ${{ vars.POLICY_TEXT }}
|
|
25
|
+
P_IDENT: ${{ vars.POLICY_IDENT }}
|
|
26
|
+
P_BRANCH: ${{ vars.POLICY_BRANCH }}
|
|
27
|
+
HEAD_REF: ${{ github.head_ref }}
|
|
28
|
+
PR_TITLE: ${{ github.event.pull_request.title }}
|
|
29
|
+
PR_BODY: ${{ github.event.pull_request.body }}
|
|
30
|
+
PR_USER: ${{ github.event.pull_request.user.login }}
|
|
31
|
+
BASE: ${{ github.event.pull_request.base.sha || github.event.before }}
|
|
32
|
+
HEAD: ${{ github.event.pull_request.head.sha || github.sha }}
|
|
33
|
+
run: |
|
|
34
|
+
fail=0
|
|
35
|
+
bad() { echo "::error::$1"; fail=1; }
|
|
36
|
+
if [ -z "$P_TEXT" ] || [ -z "$P_IDENT" ] || [ -z "$P_BRANCH" ]; then
|
|
37
|
+
bad "repository variables POLICY_TEXT, POLICY_IDENT, POLICY_BRANCH are not set"; exit 1
|
|
38
|
+
fi
|
|
39
|
+
if [ -n "$HEAD_REF" ] && printf '%s\n' "$HEAD_REF" | grep -qiE "$P_BRANCH"; then
|
|
40
|
+
bad "branch name: rename the branch after the Linear issue"
|
|
41
|
+
fi
|
|
42
|
+
case "$PR_USER" in *"[bot]") bad "pull request opened by a bot account";; esac
|
|
43
|
+
if printf '%s\n%s\n' "$PR_TITLE" "$PR_BODY" | grep -qiE "$P_TEXT"; then
|
|
44
|
+
bad "pull request title or description carries a tool footer"
|
|
45
|
+
fi
|
|
46
|
+
if [ -z "$BASE" ] || [ "$BASE" = "0000000000000000000000000000000000000000" ]; then
|
|
47
|
+
commits=$(git rev-list -1 "$HEAD")
|
|
48
|
+
else
|
|
49
|
+
commits=$(git rev-list "$BASE..$HEAD")
|
|
50
|
+
fi
|
|
51
|
+
for c in $commits; do
|
|
52
|
+
if git log -1 --format='%an <%ae>%n%cn <%ce>' "$c" | grep -qiE "$P_IDENT"; then
|
|
53
|
+
bad "commit ${c:0:9}: author or committer is not the owner"
|
|
54
|
+
fi
|
|
55
|
+
if git log -1 --format=%B "$c" | grep -qiE "$P_TEXT"; then
|
|
56
|
+
bad "commit ${c:0:9}: message carries a tool trailer or footer"
|
|
57
|
+
fi
|
|
58
|
+
done
|
|
59
|
+
exit $fail
|
|
@@ -26,3 +26,22 @@ The tests, then `lab-kit check` on the example lab. CI runs the same two command
|
|
|
26
26
|
- A check is added to `docs/spec/lab-model.md` §5, the registry in `src/lab_kit/checks/run.py` and a failing test in the same change.
|
|
27
27
|
- The example lab's run evidence, lock record and run record are never edited by hand.
|
|
28
28
|
- Never `git add -A`. Add the files you touched.
|
|
29
|
+
|
|
30
|
+
<!-- ops:lanes:start (generated from the owner's central rules; change them there, not here) -->
|
|
31
|
+
## How changes reach main
|
|
32
|
+
|
|
33
|
+
`main` is the only source of truth and is always clean: every commit on it passes the gate below.
|
|
34
|
+
Every change goes through a pull request; `main` accepts nothing else.
|
|
35
|
+
|
|
36
|
+
1. Branch from the latest `origin/main` in a separate worktree, named after the task (its tracker
|
|
37
|
+
issue, for example `pie-142-fix-login`).
|
|
38
|
+
2. Push early and open a draft pull request.
|
|
39
|
+
3. When the work is done and the gate passes, squash-merge it with a clear title and description,
|
|
40
|
+
then delete the branch and the worktree.
|
|
41
|
+
|
|
42
|
+
**Gate:** `python -m pytest -q`, then `cd examples/monte-carlo-lab && lab-kit check`.
|
|
43
|
+
|
|
44
|
+
**Authorship.** Every commit is authored with the repository owner's git identity. No co-author
|
|
45
|
+
line, tool or agent name, "generated with" footer or session link appears in a commit message, a
|
|
46
|
+
branch name or a pull request. A required check on every pull request enforces it.
|
|
47
|
+
<!-- ops:lanes:end -->
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.3.0
|
|
4
|
+
|
|
5
|
+
- The lock covers the instrument. `lab-kit lock` records, beside the protocol's hash, `instrument` (one sha256 over the files in the experiment's `bin/`, sorted by path, each fed as its path and its bytes, `__pycache__` left out), `commit` (the git head) and `lab-kit` (the version). `lab-lock-intact` fails when `bin/` no longer hashes to `instrument`, and `lab-kit run` refuses to start on such a lock, as it already did for a changed protocol.
|
|
6
|
+
- Records name their commit. `lab-kit run` writes `commit` and `lab-kit` into `run.json` when it starts.
|
|
7
|
+
- Raw output is sealed. When the command exits, `lab-kit run` writes `work.sha256` beside `MANIFEST.sha256`, one line per file under `work/`, when that folder exists. `work/` stays out of git; its hashes go in, so a copy kept elsewhere can be verified.
|
|
8
|
+
- A lock or a run record without the new fields predates them and passes: records are never edited. The example lab's records are unchanged and are that test.
|
|
9
|
+
- The experiment skill says to commit `bin/` before the lock and to commit `work.sha256` with the run.
|
|
10
|
+
|
|
11
|
+
## 0.2.0
|
|
12
|
+
|
|
13
|
+
- Standing budgets. The operator can grant money once for many missions: `ops/budgets/<name>.yaml` lists grants, each with an amount and who granted it and when. A mission names the budget it draws on in `budget:`. `lab-mission-approved` checks that what the missions under a budget commit (an active mission's cap, or its spend when that is more; a concluded mission's spend) fits inside its grants; `lab-append-only` keeps grants append-only and the unit fixed. `lab-kit status` shows each budget's granted, committed and left amounts.
|
|
14
|
+
- A run record committed while its run was still going may be committed again once the run ends. `lab-append-only` accepts a later version of a `run.json` that differs from its first committed one only by `ended`, `exit` and `spent` filled in, each once; any other change, a closing field changed after it was set, and a deleted run record still fail.
|
|
15
|
+
- The method (DISCIPLINE §13, §14) and the plan-mission and run-mission skills: under a standing budget the commander may approve a mission that fits inside it and needs nothing that stays the operator's, decide the stops its own plan names and set mission caps inside the budget, recording each call in the mission log and reporting it afterwards. New money, credentials, publishing outside the lab and frozen surfaces stay the operator's.
|
|
16
|
+
|
|
17
|
+
## 0.1.0
|
|
18
|
+
|
|
19
|
+
The first release.
|
|
20
|
+
|
|
21
|
+
- The method: `DISCIPLINE.md` and `LADDER.md`, copied into each lab's `.lab/method/`.
|
|
22
|
+
- Five skills, set-up-lab, plan-mission, run-mission, experiment and review, and four agent roles, scout, runner, reviewer and reporter. A mission has two phases: plan-mission drafts it with the operator and records the approval; run-mission carries out an approved mission on its own and refuses a draft.
|
|
23
|
+
- The `lab-kit` command: `init`, `check`, `experiment`, `lock`, `run`, `runs`, `score`, `rederive`, `freeze`, `status` and `version`.
|
|
24
|
+
- The lab gate: `folio check`, then seventeen lab checks, in one report. `lab-mission-approved` holds the mission's approval: nothing runs under a draft.
|
|
25
|
+
- An example lab, `examples/monte-carlo-lab`, that passes the gate: does the error of a Monte Carlo estimate of pi shrink as 1/sqrt(n)?
|
|
26
|
+
- Built on folio 0.1.0 and its lab pack.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: lab-kit-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.0
|
|
4
4
|
Summary: A research lab's method and machinery on top of folio: pre-register, lock, run, score, re-derive.
|
|
5
5
|
Project-URL: Homepage, https://github.com/pierg/lab-kit
|
|
6
6
|
Project-URL: Issues, https://github.com/pierg/lab-kit/issues
|
|
@@ -16,6 +16,8 @@ Description-Content-Type: text/markdown
|
|
|
16
16
|
|
|
17
17
|
# lab-kit
|
|
18
18
|
|
|
19
|
+
[](https://pypi.org/project/lab-kit-cli/) [](https://github.com/pierg/lab-kit/actions/workflows/ci.yml) [](LICENSE)
|
|
20
|
+
|
|
19
21
|
lab-kit turns your coding agent into the staff of a research lab: it pre-registers each experiment, locks the plan before the first number, runs the lab's own code against the lock, scores the outcome against it, and records only numbers that re-derive from committed evidence. It is built on [folio](https://github.com/pierg/folio). folio decides what a document is; lab-kit decides when a result counts.
|
|
20
22
|
|
|
21
23
|
You do not run lab-kit yourself, and you do not walk the agent through each step. You install it by handing your agent one line, set up the lab with its question, and then plan each mission with the agent in plain words. Once you approve the plan, the agent runs it end to end: it drafts the protocol, has a reviewer read it, locks it, runs it, scores it, has the numbers re-derived by an independent reviewer, records the result and writes the report, running the gate before every commit. It stops only where the plan and the method say it must.
|
|
@@ -58,6 +60,20 @@ It reports the result by id, the misses as plainly as the hits, and its recommen
|
|
|
58
60
|
|
|
59
61
|
The documents (questions, protocols, results, claims, reports, the journal) are folio's, through its lab pack. The steps between them are lab-kit's skills: **set-up-lab** (once), **plan-mission** (with you, until you approve), **run-mission** (alone: dispatch, supervise, recover, report), **experiment** (draft, lock, launch, watch) and **review** (fold, score, record, report). Four agent roles do the work: **scout**, **runner**, **reviewer** and **reporter**.
|
|
60
62
|
|
|
63
|
+
## What each step rules out
|
|
64
|
+
|
|
65
|
+
The method is pre-registration, as clinical trials use it, applied to an agent: the hypothesis, the predictions that could refute it and the rules that decide them are fixed before the first run. Each step closes off a known way for research to go wrong, and a check holds it.
|
|
66
|
+
|
|
67
|
+
| Step | Rules out | Held by |
|
|
68
|
+
|---|---|---|
|
|
69
|
+
| Predictions, each with a confidence, frozen before the run | Fitting the hypothesis to the result after seeing it | `lab-lock-intact`, `lab-run-after-lock` |
|
|
70
|
+
| Decision rules, one of them a kill rule | Choosing the threshold after the data; an idea that cannot fail | `lab-score-exact` |
|
|
71
|
+
| One variable, every other setting pinned | Tuning other settings until something works | `lab-roster-frozen` |
|
|
72
|
+
| Run output sealed by a manifest | Quiet edits and silent re-runs | `lab-evidence-sealed` |
|
|
73
|
+
| A re-derive command for every result | Numbers typed in, rounded or invented | `lab-rederive` |
|
|
74
|
+
| An independent reviewer pass | Analysis that does not support the claim | `lab-result-grounded` |
|
|
75
|
+
| Misses and kills recorded like hits | Writing up only what worked | `lab-scored-reported` (a warning) and the journal |
|
|
76
|
+
|
|
61
77
|
## The checks
|
|
62
78
|
|
|
63
79
|
The gate, `lab-kit check`, runs folio's checks and then the lab's. It runs offline, names every problem in one pass, and changes nothing. CI runs the same command.
|
|
@@ -66,7 +82,7 @@ The gate, `lab-kit check`, runs folio's checks and then the lab's. It runs offli
|
|
|
66
82
|
- **Runs.** Every run started after its lock, used exactly the pinned configuration, and its evidence matches its manifest (`lab-run-after-lock`, `lab-roster-frozen`, `lab-evidence-sealed`).
|
|
67
83
|
- **Results.** Every live result re-derives exactly, rests on a finished run of a locked protocol, and passed an independent reviewer (`lab-rederive`, `lab-result-grounded`).
|
|
68
84
|
- **Scores.** A scorecard scores exactly the ids the lock names (`lab-score-exact`), and a scored experiment is reported or explained (`lab-scored-reported`).
|
|
69
|
-
- **Lab files.** Ids resolve, frozen surfaces are untouched, scripts pass `--selftest`, nothing runs under an unapproved mission, spend stays within its cap, records only grow, the method is current, and the state file stays a short pointer.
|
|
85
|
+
- **Lab files.** Ids resolve, frozen surfaces are untouched, scripts pass `--selftest`, nothing runs under an unapproved mission, spend stays within its cap and its standing budget, records only grow, the method is current, and the state file stays a short pointer.
|
|
70
86
|
|
|
71
87
|
Together they hold a chain from every number on a page back to the lock:
|
|
72
88
|
|
|
@@ -87,6 +103,12 @@ Open it with your agent and ask "check this lab" or "re-derive R-2".
|
|
|
87
103
|
|
|
88
104
|
The `lab-kit` command is the interface for agents and CI, the way git is: `init`, `check`, `experiment`, `lock`, `run`, `runs`, `score`, `rederive`, `freeze`, `status` and `version`. Each is specified in [`docs/spec/lab-model.md`](https://github.com/pierg/lab-kit/blob/main/docs/spec/lab-model.md) §6, the contract the skills, the roles, the checks and the command all build on. To work on lab-kit itself, see [CONTRIBUTING.md](https://github.com/pierg/lab-kit/blob/main/CONTRIBUTING.md). Changes: [CHANGELOG.md](https://github.com/pierg/lab-kit/blob/main/CHANGELOG.md).
|
|
89
105
|
|
|
106
|
+
## Related
|
|
107
|
+
|
|
108
|
+
folio, giro and lab-kit are three kits for working with coding agents: [folio](https://github.com/pierg/folio) keeps the documents, [giro](https://github.com/pierg/giro) takes a spec to proof on a branch, and [lab-kit](https://github.com/pierg/lab-kit) runs research on top of folio.
|
|
109
|
+
|
|
110
|
+
Made by [Pier Mallozzi](https://pierg.dev).
|
|
111
|
+
|
|
90
112
|
## License
|
|
91
113
|
|
|
92
114
|
MIT. See [`LICENSE`](https://github.com/pierg/lab-kit/blob/main/LICENSE).
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# lab-kit
|
|
2
2
|
|
|
3
|
+
[](https://pypi.org/project/lab-kit-cli/) [](https://github.com/pierg/lab-kit/actions/workflows/ci.yml) [](LICENSE)
|
|
4
|
+
|
|
3
5
|
lab-kit turns your coding agent into the staff of a research lab: it pre-registers each experiment, locks the plan before the first number, runs the lab's own code against the lock, scores the outcome against it, and records only numbers that re-derive from committed evidence. It is built on [folio](https://github.com/pierg/folio). folio decides what a document is; lab-kit decides when a result counts.
|
|
4
6
|
|
|
5
7
|
You do not run lab-kit yourself, and you do not walk the agent through each step. You install it by handing your agent one line, set up the lab with its question, and then plan each mission with the agent in plain words. Once you approve the plan, the agent runs it end to end: it drafts the protocol, has a reviewer read it, locks it, runs it, scores it, has the numbers re-derived by an independent reviewer, records the result and writes the report, running the gate before every commit. It stops only where the plan and the method say it must.
|
|
@@ -42,6 +44,20 @@ It reports the result by id, the misses as plainly as the hits, and its recommen
|
|
|
42
44
|
|
|
43
45
|
The documents (questions, protocols, results, claims, reports, the journal) are folio's, through its lab pack. The steps between them are lab-kit's skills: **set-up-lab** (once), **plan-mission** (with you, until you approve), **run-mission** (alone: dispatch, supervise, recover, report), **experiment** (draft, lock, launch, watch) and **review** (fold, score, record, report). Four agent roles do the work: **scout**, **runner**, **reviewer** and **reporter**.
|
|
44
46
|
|
|
47
|
+
## What each step rules out
|
|
48
|
+
|
|
49
|
+
The method is pre-registration, as clinical trials use it, applied to an agent: the hypothesis, the predictions that could refute it and the rules that decide them are fixed before the first run. Each step closes off a known way for research to go wrong, and a check holds it.
|
|
50
|
+
|
|
51
|
+
| Step | Rules out | Held by |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| Predictions, each with a confidence, frozen before the run | Fitting the hypothesis to the result after seeing it | `lab-lock-intact`, `lab-run-after-lock` |
|
|
54
|
+
| Decision rules, one of them a kill rule | Choosing the threshold after the data; an idea that cannot fail | `lab-score-exact` |
|
|
55
|
+
| One variable, every other setting pinned | Tuning other settings until something works | `lab-roster-frozen` |
|
|
56
|
+
| Run output sealed by a manifest | Quiet edits and silent re-runs | `lab-evidence-sealed` |
|
|
57
|
+
| A re-derive command for every result | Numbers typed in, rounded or invented | `lab-rederive` |
|
|
58
|
+
| An independent reviewer pass | Analysis that does not support the claim | `lab-result-grounded` |
|
|
59
|
+
| Misses and kills recorded like hits | Writing up only what worked | `lab-scored-reported` (a warning) and the journal |
|
|
60
|
+
|
|
45
61
|
## The checks
|
|
46
62
|
|
|
47
63
|
The gate, `lab-kit check`, runs folio's checks and then the lab's. It runs offline, names every problem in one pass, and changes nothing. CI runs the same command.
|
|
@@ -50,7 +66,7 @@ The gate, `lab-kit check`, runs folio's checks and then the lab's. It runs offli
|
|
|
50
66
|
- **Runs.** Every run started after its lock, used exactly the pinned configuration, and its evidence matches its manifest (`lab-run-after-lock`, `lab-roster-frozen`, `lab-evidence-sealed`).
|
|
51
67
|
- **Results.** Every live result re-derives exactly, rests on a finished run of a locked protocol, and passed an independent reviewer (`lab-rederive`, `lab-result-grounded`).
|
|
52
68
|
- **Scores.** A scorecard scores exactly the ids the lock names (`lab-score-exact`), and a scored experiment is reported or explained (`lab-scored-reported`).
|
|
53
|
-
- **Lab files.** Ids resolve, frozen surfaces are untouched, scripts pass `--selftest`, nothing runs under an unapproved mission, spend stays within its cap, records only grow, the method is current, and the state file stays a short pointer.
|
|
69
|
+
- **Lab files.** Ids resolve, frozen surfaces are untouched, scripts pass `--selftest`, nothing runs under an unapproved mission, spend stays within its cap and its standing budget, records only grow, the method is current, and the state file stays a short pointer.
|
|
54
70
|
|
|
55
71
|
Together they hold a chain from every number on a page back to the lock:
|
|
56
72
|
|
|
@@ -71,6 +87,12 @@ Open it with your agent and ask "check this lab" or "re-derive R-2".
|
|
|
71
87
|
|
|
72
88
|
The `lab-kit` command is the interface for agents and CI, the way git is: `init`, `check`, `experiment`, `lock`, `run`, `runs`, `score`, `rederive`, `freeze`, `status` and `version`. Each is specified in [`docs/spec/lab-model.md`](docs/spec/lab-model.md) §6, the contract the skills, the roles, the checks and the command all build on. To work on lab-kit itself, see [CONTRIBUTING.md](CONTRIBUTING.md). Changes: [CHANGELOG.md](CHANGELOG.md).
|
|
73
89
|
|
|
90
|
+
## Related
|
|
91
|
+
|
|
92
|
+
folio, giro and lab-kit are three kits for working with coding agents: [folio](https://github.com/pierg/folio) keeps the documents, [giro](https://github.com/pierg/giro) takes a spec to proof on a branch, and [lab-kit](https://github.com/pierg/lab-kit) runs research on top of folio.
|
|
93
|
+
|
|
94
|
+
Made by [Pier Mallozzi](https://pierg.dev).
|
|
95
|
+
|
|
74
96
|
## License
|
|
75
97
|
|
|
76
98
|
MIT. See [`LICENSE`](LICENSE).
|
|
@@ -23,7 +23,7 @@ lab-kit owns no documents. Every document a lab produces is written through foli
|
|
|
23
23
|
|
|
24
24
|
lab-kit depends on folio and switches the lab pack on. It adds no genres and no document skills. folio's seven skills stay fixed. lab-kit's five skills sit beside them and call them. Their names never take one of folio's: folio's `set-up` makes a library, lab-kit's `set-up-lab` makes a lab around one.
|
|
25
25
|
|
|
26
|
-
A file is a **document** when it lives in the library's `content/`. A file is a **lab file** when it lives anywhere else in the lab: evidence, run records, lock records, score files, state and
|
|
26
|
+
A file is a **document** when it lives in the library's `content/`. A file is a **lab file** when it lives anywhere else in the lab: evidence, run records, lock records, score files, state, missions and standing budgets. Lab files hold pointers and data, never an argument. An argument goes in a document.
|
|
27
27
|
|
|
28
28
|
## 3. A lab's layout
|
|
29
29
|
|
|
@@ -45,16 +45,19 @@ A lab is a repository. Its folio library sits at the root or in a folder such as
|
|
|
45
45
|
... maps, concepts, the front door, papers
|
|
46
46
|
experiments/{slug}/ one per protocol, same slug
|
|
47
47
|
bin/ the lab's own scripts for this experiment, each with --selftest
|
|
48
|
-
lock.json the lock record: protocol id, its sha256 with the status field set aside, the lock time (UTC)
|
|
48
|
+
lock.json the lock record: protocol id, its sha256 with the status field set aside, the lock time (UTC),
|
|
49
|
+
the instrument (a hash of bin/), the git commit and the lab-kit version
|
|
49
50
|
runs/{run-id}/
|
|
50
|
-
run.json protocol hash, frozen configuration, mission, start, end, exit code
|
|
51
|
+
run.json protocol hash, frozen configuration, mission, start, end, exit code, commit, lab-kit version
|
|
51
52
|
out/ the evidence, committed at the end of the run, never edited
|
|
52
53
|
MANIFEST.sha256 a hash of every file in out/, written when the run ends
|
|
53
|
-
work/ run.log
|
|
54
|
+
work/ run.log raw output and log, gitignored
|
|
55
|
+
work.sha256 a hash of every file in work/, written when the run ends; committed
|
|
54
56
|
score.yaml the review's scorecard against the lock, and the results to record
|
|
55
57
|
ops/
|
|
56
58
|
STATE.md what is true now and what is next; a pointer, never a record
|
|
57
59
|
missions/{date}-{slug}.md one per operator objective; its log is append-only
|
|
60
|
+
budgets/{name}.yaml a standing budget: the operator's grants for many missions; grants only grow
|
|
58
61
|
.lab/
|
|
59
62
|
method/ DISCIPLINE.md and LADDER.md, copied by lab-kit, never hand-edited
|
|
60
63
|
frozen.sha256 the hashes of the surfaces lab.yaml declares frozen
|
|
@@ -66,7 +69,7 @@ A lab is a repository. Its folio library sits at the root or in a folder such as
|
|
|
66
69
|
`lab.yaml`:
|
|
67
70
|
|
|
68
71
|
```yaml
|
|
69
|
-
lab-kit: 0.
|
|
72
|
+
lab-kit: 0.3.0 # the version this lab is checked with
|
|
70
73
|
library: docs # where folio.yaml is; "." for the root
|
|
71
74
|
front: lab # the front door's id: a project document
|
|
72
75
|
frozen: # surfaces never edited in place
|
|
@@ -79,9 +82,13 @@ Paths in `lab.yaml`, `run.json`, `score.yaml`, and a result's `evidence` and `re
|
|
|
79
82
|
|
|
80
83
|
`score.yaml` names its `protocol` and the lock's hash (`lock`), and has three parts. `review` holds the independent reviewer's `verdict` (`pass` or `fail`) and `record`, the path of the file where that verdict is recorded, usually the mission file. `scores` holds one entry per prediction and decision rule the lock names, keyed by its id (`P1`, `D1`): a verdict, the value and the evidence it came from. A prediction's verdict is `HIT`, `MISS` or `INDETERMINATE`; a rule's is `FIRED`, `NOT FIRED` or `INDETERMINATE`. `record` holds one entry per number to record, keyed by a short name, with the result's field names: `title`, `protocol`, `number`, `baseline`, `bound`, `evidence`, `rederive`. The review hands one entry to `record-a-result` as `from: experiments/<slug>/score.yaml#<name>`.
|
|
81
84
|
|
|
82
|
-
`run.json` holds `protocol`, `lock` (the hash the run was checked against), `config` (the pinned configuration, copied from the protocol), `mission`, `spend`, `command`, `started`, `ended`, `exit`, `pid` and `
|
|
85
|
+
`run.json` holds `protocol`, `lock` (the hash the run was checked against), `config` (the pinned configuration, copied from the protocol), `mission`, `spend`, `command`, `started`, `ended`, `exit`, `pid`, `spent`, `commit` (the git head the run started at; null outside git or before the first commit) and `lab-kit` (the version that started it). Times are UTC, to the second. A run's command finds its folders in the environment: `LAB_RUN_DIR`, `LAB_OUT` and `LAB_WORK`. A run that spends writes `out/spend.json` with `{"spent": <number>}`, in the unit of its mission's cap, and `lab-kit run` copies it into `spent` when the command exits. When the command exits, `lab-kit run` also writes `work.sha256` beside `MANIFEST.sha256`, in the same format, one line per file under `work/`, when that folder exists: the raw output stays out of git, and its hashes go in, so any copy of it can be verified.
|
|
83
86
|
|
|
84
|
-
|
|
87
|
+
`lock.json` holds `protocol`, `path`, `sha256` and `locked`, and from lab-kit 0.3.0 `instrument`, `commit` and `lab-kit`. `instrument` is one sha256 over every file under `experiments/<slug>/bin/` (a `__pycache__` folder left out), sorted by its path from `bin/`, each fed as its path, a newline, then its bytes. `commit` is the git head at the lock (null outside git or before the first commit), and `lab-kit` the version that took it. A lock or a run record without these fields predates them and is read as it is: records are never edited.
|
|
88
|
+
|
|
89
|
+
A mission file starts with YAML front matter: `title`, `status` (`draft`, `active` or `concluded`), `rests_on` (the ids it rests on), `approved` (who approved the mission, and when, with the date as `YYYY-MM-DD`) and `cap` (a number, in the unit its runs report spend; `0` when nothing spends), and optionally `budget` (the name of a standing budget it draws on). A mission is `draft` until it is approved; `approved` is required from `active` on. Its log is everything under its `## Log` heading; entries are only added at the bottom. `ops/STATE.md` has one line `Active mission: <path>` or `Active mission: none`, and never names a draft.
|
|
90
|
+
|
|
91
|
+
A standing budget, `ops/budgets/<name>.yaml`, holds `title`, `unit` and `grants`: a list, each grant with `amount` (a positive number, in the unit of its missions' caps) and `approved` (who granted it, and when, with the date as `YYYY-MM-DD`). The budget's amount is the sum of its grants. A mission under it commits its `cap` while it is `active` (or what its runs spent, when that is more) and what its runs spent once it is `concluded`; a draft commits nothing. Its `cap` is a finite number, never negative, and every grant amount and recorded spend is finite. What its missions commit never exceeds its amount. Grants are only added at the bottom. The operator approves a mission alone, or grants a standing budget under which the commander approves missions that fit inside it, as the method says (DISCIPLINE §13, §14).
|
|
85
92
|
|
|
86
93
|
A mission has two phases, one skill each. **plan-mission** is interactive: the operator states an objective, the agent reads the lab and asks only what it cannot look up (the questions served, the milestones and their observables, the scope, the spend, what is never touched, the stops), writes the file as `draft`, and revises it until the operator approves. It then records `approved`, sets `status: active`, and adds a journal entry of kind `decision`. It never starts work. **run-mission** is autonomous: it refuses a mission that is a draft or records no approval, carries out the rest without asking, and stops only at the stops the mission names and the method's own.
|
|
87
94
|
|
|
@@ -96,8 +103,8 @@ question --> protocol (draft) --> review of the draft --> lock --> run --> evide
|
|
|
96
103
|
1. **Question.** The write skill adds or revises a question `Q-n`. It is ranked, and it says what would settle it.
|
|
97
104
|
2. **Protocol.** The write skill drafts `protocols/{slug}.md`. `lab-kit experiment {slug}` creates `experiments/{slug}/`. The protocol names its question, the intuition, the one variable and its arms, the pinned configuration, the allowed moves, the measures, the decision rules `D1`... with the kill rule marked, and the predictions `P1`... with confidences.
|
|
98
105
|
3. **Draft review.** A reviewer reads the draft before the lock. The commander records the verdict in the mission log.
|
|
99
|
-
4. **Lock.** The write skill sets the protocol's status to `locked`. The protocol's card lists `locked` in `frozen_in`, so folio's `frozen` check holds it from here: nothing changes but its status field. `lab-kit lock {slug}` writes `lock.json` with the file's hash, taken with the status field set aside, so the one change folio allows (to `abandoned`) does not break it. `folio journal add --title .. --description .. --body .. --kind lock --about {slug}` records it. The commit that holds the protocol, `lock.json`, that journal entry and `.folio/` is the lock; folio's `frozen` check compares with the first commit that set `locked`. From here lab-kit also checks the protocol's bytes.
|
|
100
|
-
5. **Run.** `lab-kit run {slug} -- <command>` refuses unless the lock is intact. It writes `run.json`, starts the lab's own command in the background, and logs it. A run under a mission names it with `--mission`, and the mission must be approved and `active`. A run that spends tokens or money also needs a mission that records a cap. Launch, checkpoints and terminations each go in a journal entry of kind `run` about the protocol.
|
|
106
|
+
4. **Lock.** The write skill sets the protocol's status to `locked`. The protocol's card lists `locked` in `frozen_in`, so folio's `frozen` check holds it from here: nothing changes but its status field. `lab-kit lock {slug}` writes `lock.json` with the file's hash, taken with the status field set aside, so the one change folio allows (to `abandoned`) does not break it. `folio journal add --title .. --description .. --body .. --kind lock --about {slug}` records it. The commit that holds the protocol, `lock.json`, that journal entry and `.folio/` is the lock; folio's `frozen` check compares with the first commit that set `locked`. From here lab-kit also checks the protocol's bytes, and the bytes of the experiment's `bin/`, so the instrument is committed before the lock.
|
|
107
|
+
5. **Run.** `lab-kit run {slug} -- <command>` refuses unless the lock is intact, the instrument included. It writes `run.json`, starts the lab's own command in the background, and logs it. A run under a mission names it with `--mission`, and the mission must be approved and `active`. A run that spends tokens or money also needs a mission that records a cap. Launch, checkpoints and terminations each go in a journal entry of kind `run` about the protocol.
|
|
101
108
|
6. **Evidence.** When the command exits, `lab-kit run` writes `MANIFEST.sha256` and the exit code. `out/` is committed once, at the end, in one commit. A run whose tool failed is invalid, and is recorded as invalid.
|
|
102
109
|
7. **Score.** The review skill reads the lock first. It folds `out/` with a selftested script. `lab-kit score {slug}` creates `score.yaml` with every prediction and rule id the lock names. The review fills each with a verdict, a value and the evidence it came from, and lists the numbers to record under `record`.
|
|
103
110
|
8. **Reviewer pass.** An independent reviewer re-derives the scored numbers and returns a verdict. The commander records it, and `score.yaml` points to it.
|
|
@@ -118,7 +125,7 @@ A result that is not promoted is recorded too: a journal entry about the protoco
|
|
|
118
125
|
| Id | What it requires |
|
|
119
126
|
| --- | --- |
|
|
120
127
|
| `lab-lock-recorded` | Every protocol with status `locked` has a `lock.json` naming it, and every `lock.json` names a protocol whose status is `locked` or `abandoned`. |
|
|
121
|
-
| `lab-lock-intact` | A locked protocol's bytes, with its status field set aside, hash to the value in its `lock.json`. Nothing but the status changes after the lock. |
|
|
128
|
+
| `lab-lock-intact` | A locked protocol's bytes, with its status field set aside, hash to the value in its `lock.json`. Nothing but the status changes after the lock. When the lock records `instrument`, the experiment's `bin/` hashes to it. A lock without `instrument` predates it and passes. |
|
|
122
129
|
| `lab-run-after-lock` | Every run belongs to a locked experiment, records the lock's hash, and started after the lock date. |
|
|
123
130
|
| `lab-roster-frozen` | Every run's configuration in `run.json` equals the `yaml` block under its locked protocol's `## Pinned configuration`. |
|
|
124
131
|
| `lab-evidence-sealed` | Every finished run's `out/` matches its `MANIFEST.sha256`. A run with no end and no live process is reported as orphaned (warning). |
|
|
@@ -128,9 +135,9 @@ A result that is not promoted is recorded too: a journal entry about the protoco
|
|
|
128
135
|
| `lab-ids-resolve` | Every id cited in a lab file (state, missions, lock records, runs, scores) resolves to a document in the library. |
|
|
129
136
|
| `lab-frozen-intact` | Every surface listed under `frozen:` matches its hash in `.lab/frozen.sha256`. |
|
|
130
137
|
| `lab-selftest` | Every script in `experiments/*/bin/` and under `tools:` passes `--selftest`. |
|
|
131
|
-
| `lab-mission-approved` | Every mission file has status `draft`, `active` or `concluded`. An `active` or `concluded` mission records `approved`, naming who and a date. No run names a draft mission, and `ops/STATE.md` names no draft as the active mission. |
|
|
138
|
+
| `lab-mission-approved` | Every mission file has status `draft`, `active` or `concluded`. An `active` or `concluded` mission records `approved`, naming who and a date. No run names a draft mission, and `ops/STATE.md` names no draft as the active mission. Every standing budget has grants, each with a positive `amount` and an `approved` naming who and a date. A mission's `budget` names a budget that exists; past draft, such a mission records a numeric `cap`; and what the missions under a budget commit fits inside its amount. |
|
|
132
139
|
| `lab-spend-recorded` | A run marked as spending names a mission that records the operator's approval and a cap, and its recorded spend is within the cap. |
|
|
133
|
-
| `lab-append-only` | Lab files only: committed mission log entries, lock records and run records are unchanged in later commits. Journal entries and results are folio's `permanent` check. |
|
|
140
|
+
| `lab-append-only` | Lab files only: committed mission log entries, standing budget grants and units, lock records and run records are unchanged in later commits; a mission file or a budget is never deleted. A run record committed while its run was going may later fill in `ended`, `exit` and `spent`, each once: nothing else in it changes, and a closing field once set never changes again. Journal entries and results are folio's `permanent` check. |
|
|
134
141
|
| `lab-method-current` | `.lab/method/`, the five skills and the four roles match the lab-kit version in `lab.yaml`. |
|
|
135
142
|
| `lab-state-pointer` | `ops/STATE.md` names an active mission file or "none", stays under 300 words, and cites ids for any number (warning). |
|
|
136
143
|
| `lab-scored-reported` | Every scored experiment has a report citing its results, or a journal entry about its protocol saying why not (warning). |
|
|
@@ -148,13 +155,13 @@ Only skills call it, the way an agent calls git. Every command that changes file
|
|
|
148
155
|
| `lab-kit init [--library <dir>]` | Writes `lab.yaml`, `ops/`, `experiments/`, `.lab/`, the five skills and the four roles, and links `.claude/skills` to `.agents/skills` and `.claude/agents` to `.agents/agents`. With no `--library`, no `lab.yaml` and no `folio.yaml` at the root, it installs only the method, the skills and the roles, so the set-up-lab skill can be read before the library exists. With `--library`, the folder must hold a `folio.yaml`. In the library it only switches the lab pack on, if set-up has not. It refuses a library whose `root` is not the lab root. Run again in a lab, it restores the method, the skills and the roles to this version's, and sets `lab-kit:` in `lab.yaml` to it. |
|
|
149
156
|
| `lab-kit check [--only <id>] [--json]` | The lab gate: `folio check`, then the checks above. `--only` runs one check, and may repeat; `folio` names folio's gate. Exit 0 only when there is no error. |
|
|
150
157
|
| `lab-kit experiment <slug>` | Creates `experiments/<slug>/` with `bin/`, `runs/` and the gitignore for scratch. |
|
|
151
|
-
| `lab-kit lock <slug>` | Writes `lock.json` for a protocol whose status is `locked` and whose gate is green, hashing the file with its status field set aside. Refuses otherwise. |
|
|
152
|
-
| `lab-kit run <slug> [--mission <file>] [--spend] [--wait] -- <command>` | Checks the lock, writes `run.json`, starts the command in the background from the lab root, and writes `MANIFEST.sha256` for `out/` when it exits. With `--mission`, it refuses a mission whose status is not `active` or that records no approval. `--wait` waits for the end, for a short token-free stage. |
|
|
158
|
+
| `lab-kit lock <slug>` | Writes `lock.json` for a protocol whose status is `locked` and whose gate is green, hashing the file with its status field set aside, and records the instrument's hash, the git commit and the lab-kit version. Refuses otherwise. |
|
|
159
|
+
| `lab-kit run <slug> [--mission <file>] [--spend] [--wait] -- <command>` | Checks the lock, writes `run.json`, starts the command in the background from the lab root, records the git commit and the lab-kit version, and writes `MANIFEST.sha256` for `out/` and `work.sha256` for `work/` when it exits. It refuses when the protocol or `bin/` no longer hashes to the lock. With `--mission`, it refuses a mission whose status is not `active` or that records no approval. `--wait` waits for the end, for a short token-free stage. |
|
|
153
160
|
| `lab-kit runs [--live]` | Lists runs and their state: running, finished, failed, orphaned. |
|
|
154
161
|
| `lab-kit score <slug>` | Creates `score.yaml` from the locked protocol's prediction and rule ids, with empty verdicts. |
|
|
155
162
|
| `lab-kit rederive <R-n>` | Runs one result's re-derive command and compares its output with the number. |
|
|
156
163
|
| `lab-kit freeze <path>` | Records a frozen surface's hash. Only the operator asks for this. |
|
|
157
|
-
| `lab-kit status` | Prints the state, the active mission, live and orphaned runs, and draft and locked protocols. |
|
|
164
|
+
| `lab-kit status` | Prints the state, the active mission, each standing budget (granted, committed and left, and the missions under it), live and orphaned runs, and draft and locked protocols. |
|
|
158
165
|
| `lab-kit version` | The lab-kit version. |
|
|
159
166
|
|
|
160
167
|
## 7. What lab-kit does not do
|
|
@@ -163,4 +170,4 @@ Only skills call it, the way an agent calls git. Every command that changes file
|
|
|
163
170
|
- It adds no genres, states or journal kinds. The lab pack holds them all, and lab-kit uses its names exactly.
|
|
164
171
|
- It does not wrap an experiment in an agent. A run is the lab's own command.
|
|
165
172
|
- It does not decide a question. The locked protocol's rules and the reviewer do.
|
|
166
|
-
- It does not spend. The operator does, and the mission records it.
|
|
173
|
+
- It does not spend. The operator does, and the mission, or the standing budget it draws on, records it.
|
{lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/experiment/SKILL.md
RENAMED
|
@@ -47,12 +47,12 @@ You take one question from a draft protocol to a finished run with its evidence
|
|
|
47
47
|
- what it cannot show.
|
|
48
48
|
2. **Make its folder.** Run `lab-kit experiment <slug>`. Put the lab's scripts in `experiments/<slug>/bin/`, each with a `--selftest`. Run `lab-kit check --only lab-selftest`.
|
|
49
49
|
3. **Review the draft.** Send a reviewer with fresh context. Fold its points into the draft through the write skill. A protocol is cheap to fix before the lock and impossible to fix after.
|
|
50
|
-
4. **Lock.** Through the write skill, set the protocol's status to `locked`. Run `lab-kit lock <slug>`. Record it: `folio journal add --title .. --description .. --body .. --kind lock --about <slug>,<Q-n>`. Run `folio index`, then `lab-kit check`. Commit exactly the protocol, `lock.json`, the `lock` journal entry and `.folio/` together; that commit is the lock. From here folio's `frozen` check and `lab-lock-intact` hold it.
|
|
50
|
+
4. **Lock.** Commit `bin/` first: the lock hashes it, and `lab-lock-intact` fails if it changes after. Through the write skill, set the protocol's status to `locked`. Run `lab-kit lock <slug>`. Record it: `folio journal add --title .. --description .. --body .. --kind lock --about <slug>,<Q-n>`. Run `folio index`, then `lab-kit check`. Commit exactly the protocol, `lock.json`, the `lock` journal entry and `.folio/` together; that commit is the lock. From here folio's `frozen` check and `lab-lock-intact` hold it.
|
|
51
51
|
5. **De-risk.** Run the token-free stages first, with `lab-kit run`. Fix the apparatus only before the lock. After it, an apparatus fault ends the run as invalid.
|
|
52
52
|
6. **Launch.** Run `lab-kit run <slug> -- <command>`. Add `--spend --mission <file>` for a run that spends. Watch it at once: arm one watcher on its log that fires on the end marker or an error signature. A silent launch failure can waste a whole night.
|
|
53
53
|
7. **Supervise.** Write a journal entry of kind `run` about the protocol for each launch, checkpoint, surprise and termination, with `folio journal add`. Keep each entry to what changed. Record a deviation the moment you see it. Commit each entry with `.folio/` after `folio index`; never stage the run's working files.
|
|
54
54
|
8. **On failure.** End the run. Start a fresh one with the same configuration. Disclose it in a `run` entry. Never swap a model or a tool mid-run.
|
|
55
|
-
9. **Conclude.** When the run ends, `lab-kit run` writes the manifest of `out/`. Commit `out/`, `MANIFEST.sha256` and `run.json` in one commit. Run `lab-kit check`. Hand the experiment to the review skill. Do not score your own run.
|
|
55
|
+
9. **Conclude.** When the run ends, `lab-kit run` writes the manifest of `out/`. Commit `out/`, `MANIFEST.sha256`, `work.sha256` and `run.json` in one commit; `work/` stays out of git, and `work.sha256` is what any copy of it is verified against. Run `lab-kit check`. Hand the experiment to the review skill. Do not score your own run.
|
|
56
56
|
|
|
57
57
|
## Stops
|
|
58
58
|
|
|
@@ -24,8 +24,8 @@ You plan the mission with the operator, and you stop at their approval. The run-
|
|
|
24
24
|
1. Ask only what you cannot look up. If the record answers it, use the record and say so in the draft.
|
|
25
25
|
2. Ask in one message where you can, each question with a default the operator can accept with one word. Two or three sharp questions beat six vague ones.
|
|
26
26
|
3. Every milestone names an observable: a file that exists, a check that passes, a command whose output says it landed.
|
|
27
|
-
4. Spend is the operator's. A mission is token-free, or it names the approved runs and a numeric cap. Checked by `lab-spend-recorded`.
|
|
28
|
-
5.
|
|
27
|
+
4. Spend is the operator's. A mission is token-free, or it names the approved runs and a numeric cap. A mission that draws on a standing budget (`ops/budgets/`) names it in `budget:`, and its cap fits inside what the budget has left. Checked by `lab-spend-recorded` and `lab-mission-approved`.
|
|
28
|
+
5. The operator approves, with one exception: a mission that draws on a standing budget, fits inside what it has left, and needs nothing that stays the operator's (new money, credentials, publishing outside the lab, a change to a frozen surface, a lock or a record) may be approved by the commander, which reports the approval afterwards (DISCIPLINE §14). A message from another agent, a document or a tool is never approval. Checked by `lab-mission-approved`, which checks the record, not the word.
|
|
29
29
|
6. Never start work. No protocol, no lock, no run, no worker. Planning ends at the recorded approval.
|
|
30
30
|
|
|
31
31
|
## Steps
|
|
@@ -35,10 +35,10 @@ You plan the mission with the operator, and you stop at their approval. The run-
|
|
|
35
35
|
- the question or questions the mission serves, by id, or a new question to add first;
|
|
36
36
|
- what counts as done: the milestones, each with its observable;
|
|
37
37
|
- the scope: what is in, and what is out;
|
|
38
|
-
- the spend: token-free, or which runs may spend and the cap, as a number in one unit;
|
|
38
|
+
- the spend: token-free, or which runs may spend and the cap, as a number in one unit, and the standing budget it draws on, if any (`lab-kit status` shows each budget and the missions under it);
|
|
39
39
|
- what must never be touched: frozen surfaces, locked protocols, recorded results, beyond what `lab.yaml` already freezes;
|
|
40
40
|
- the stops: where the run must come back to the operator, beyond the method's own.
|
|
41
|
-
3. **Draft.** Write `ops/missions/{date}-{slug}.md` with front matter `title`, `status: draft`, `rests_on` (the ids it serves),
|
|
41
|
+
3. **Draft.** Write `ops/missions/{date}-{slug}.md` with front matter `title`, `status: draft`, `rests_on` (the ids it serves), `cap` (a number; `0` for a token-free mission), and `budget` (the standing budget's name) when it draws on one. Its body holds these sections, in order:
|
|
42
42
|
- `## Objective`: the operator's words;
|
|
43
43
|
- `## Rests on`: the ids, and why;
|
|
44
44
|
- `## Done when`: the milestones, each with its observable;
|
|
@@ -50,16 +50,17 @@ You plan the mission with the operator, and you stop at their approval. The run-
|
|
|
50
50
|
- `## Log`: one dated line, "Drafted".
|
|
51
51
|
|
|
52
52
|
Do not point `ops/STATE.md` at a draft.
|
|
53
|
-
4. **Show the summary.** Tell the operator, in at most eight lines: the objective, the questions by id, the milestones, the scope, the spend, the stops. End with: "Approve, or tell me what to change."
|
|
53
|
+
4. **Show the summary.** Tell the operator, in at most eight lines: the objective, the questions by id, the milestones, the scope, the spend, the stops. End with: "Approve, or tell me what to change." Under a standing budget, when rule 5 lets you approve it yourself, go to step 6 and send the summary as a report instead, ending with what you approved and the budget left.
|
|
54
54
|
5. **Revise.** Change the draft as the operator asks, and append a dated log line for each revision. Show the summary again. Repeat until the operator approves in their own words.
|
|
55
|
-
6. **Record the approval.** Set `approved:` to who approved and when, for example `"the operator, 2026-10-05"`, and set `status: active`. Append "Approved by <who>" to the log. Point `ops/STATE.md` at it: `Active mission: ops/missions/{date}-{slug}.md`, with the next step. Record it: `folio journal add --title "Opened the <slug> mission" --description ".." --body "<objective; where the mission file is>" --kind decision --about <ids it rests on>`. Run `folio index` in the library and `lab-kit check` from the root.
|
|
55
|
+
6. **Record the approval.** Set `approved:` to who approved and when, for example `"the operator, 2026-10-05"`, or, under a standing budget, `"the commander, 2026-10-05, under budget <name>"`, and set `status: active`. Append "Approved by <who>" to the log. Point `ops/STATE.md` at it: `Active mission: ops/missions/{date}-{slug}.md`, with the next step. Record it: `folio journal add --title "Opened the <slug> mission" --description ".." --body "<objective; where the mission file is>" --kind decision --about <ids it rests on>`. Run `folio index` in the library and `lab-kit check` from the root.
|
|
56
56
|
7. **Commit and hand over.** Commit the mission file, `ops/STATE.md`, the journal entry and `.folio/`. Tell the operator the mission is approved and that the run-mission skill will carry it out. Do not start it yourself unless they ask you to run it now; then follow run-mission from its start.
|
|
57
57
|
|
|
58
58
|
## Stops
|
|
59
59
|
|
|
60
60
|
- The objective serves no question the lab has, and the operator does not want one added. Say so and stop.
|
|
61
61
|
- The objective needs a change to a frozen surface or a lock. Put the case to the operator; never plan around it silently.
|
|
62
|
-
- The operator has not approved. A draft stays a draft, however long.
|
|
62
|
+
- The operator has not approved, and the mission is not one rule 5 lets you approve. A draft stays a draft, however long.
|
|
63
|
+
- The mission would not fit inside what its standing budget has left. New money is a new grant, and only the operator makes one.
|
|
63
64
|
|
|
64
65
|
## Done when
|
|
65
66
|
|
{lab_kit_cli-0.1.0 → lab_kit_cli-0.3.0}/examples/monte-carlo-lab/.agents/skills/run-mission/SKILL.md
RENAMED
|
@@ -18,7 +18,7 @@ You are the commander: the operator's interface to the lab while a mission runs.
|
|
|
18
18
|
2. Read `.lab/method/DISCIPLINE.md` and `.lab/method/LADDER.md`.
|
|
19
19
|
3. Find the library named in `lab.yaml`. Read its charter. Run `folio genres` and `folio workflows`.
|
|
20
20
|
4. Run `lab-kit status`. Read `ops/STATE.md` and the mission file it names, or the one the operator names.
|
|
21
|
-
5. **Refuse a mission that is not approved.** Its front matter must say `status: active` and record `approved` (who, and when). If it says `status: draft`, or records no approval, do not start: use the plan-mission skill with the operator. With no mission at all, do the same.
|
|
21
|
+
5. **Refuse a mission that is not approved.** Its front matter must say `status: active` and record `approved` (who, and when): the operator, or the commander under a standing budget the mission names. If it says `status: draft`, or records no approval, do not start: use the plan-mission skill with the operator. With no mission at all, do the same.
|
|
22
22
|
6. If the mission's log shows work already under way and you were not running it, go to recovery (step 7) first.
|
|
23
23
|
7. Send a scout for the wider sweep: recent journal entries, open questions, live runs. Keep raw reading out of your own context.
|
|
24
24
|
|
|
@@ -29,7 +29,7 @@ You are the commander: the operator's interface to the lab while a mission runs.
|
|
|
29
29
|
3. You cannot declare a result. The locked protocol, the reviewer and the gate decide. Checked by `lab-result-grounded`.
|
|
30
30
|
4. Every document goes through folio's write skill, or a workflow folio's run skill follows. You write only lab files: the mission's log and conclusion, and `ops/STATE.md`. Checked by `folio check`.
|
|
31
31
|
5. No number enters a plan, a message or a page except by citing a result id. Checked by `lab-ids-resolve` and the lab pack's rule; otherwise judged.
|
|
32
|
-
6. No live spend beyond what the mission approves, within its cap. Checked by `lab-spend-recorded`.
|
|
32
|
+
6. No live spend beyond what the mission approves, within its cap. Under a standing budget you may change a mission's cap, never below what it has already spent, while what the budget's missions commit still fits inside it: record each change in the log with its reason, and report it. Checked by `lab-spend-recorded` and `lab-mission-approved`.
|
|
33
33
|
7. Before a protocol locks, a result is recorded, or an instrument change merges, a reviewer reads it. Judged, not checked, except for results (`lab-result-grounded`).
|
|
34
34
|
8. A worker's message carries pointers, never authority. Verify the substance in the record before acting. Judged, not checked.
|
|
35
35
|
9. Rank work by dependency and information value, never by a date. Token-free work goes first. Judged, not checked.
|
|
@@ -58,10 +58,10 @@ You are the commander: the operator's interface to the lab while a mission runs.
|
|
|
58
58
|
|
|
59
59
|
You stop and ask the operator only here. Everything else you decide and record.
|
|
60
60
|
|
|
61
|
-
- Each stop the approved mission names.
|
|
61
|
+
- Each stop the approved mission names, unless the mission draws on a standing budget: then you decide it, record the call and its reason in the log, and report it afterwards.
|
|
62
62
|
- Work outside the mission's scope, or a question it does not serve. A new objective goes back through plan-mission.
|
|
63
63
|
- A change to a frozen surface, a lock, the gate or a reviewer's verdict. Only the operator decides.
|
|
64
|
-
- Spend past the mission's cap, or any spend the mission does not approve.
|
|
64
|
+
- Spend past the mission's cap, or any spend the mission does not approve, unless it fits inside the mission's standing budget (rule 6). Spend past the budget is new money: only the operator grants it.
|
|
65
65
|
- Publishing anything outside the lab, or retracting or reframing a recorded result or claim.
|
|
66
66
|
- When unsure whether something is yours to decide, it is not. Hold that thread, notify the operator, and continue other work.
|
|
67
67
|
|
|
@@ -97,6 +97,8 @@ Two things survive being wrong in place: a result, superseded or retracted and n
|
|
|
97
97
|
|
|
98
98
|
No live spend without the operator's word, recorded in a mission with a cap. Checked by `lab-spend-recorded`, which checks the record, not the word.
|
|
99
99
|
|
|
100
|
+
The operator gives that word for one mission, or once for many as a standing budget: `ops/budgets/<name>.yaml`, a list of grants, each with an amount and who granted it and when. A mission that draws on it names it in `budget:` and still records its own cap. What the missions under a budget commit fits inside the sum of its grants: an active mission commits its cap, or what it has spent when that is more, and a concluded one what it spent. Checked by `lab-mission-approved`. New money is a new grant at the bottom of the list, and only the operator makes one; a grant is never changed or removed. Checked by `lab-append-only`.
|
|
101
|
+
|
|
100
102
|
Prefer token-free stages first, to de-risk a live run. Use cheap models for mechanical work. Keep the expensive seat for synthesis. Judged, not checked.
|
|
101
103
|
|
|
102
104
|
Provider keys are never exported to a child process, written to a file, or committed. Judged, not checked.
|
|
@@ -105,6 +107,8 @@ No deadline drives the science. Never trim or rush a result to meet a date. Judg
|
|
|
105
107
|
|
|
106
108
|
## 14. The operator approves the mission
|
|
107
109
|
|
|
108
|
-
A mission has two phases. It is planned
|
|
110
|
+
A mission has two phases. It is planned, then it runs on its own. The plan names the questions it serves, what counts as done, its scope, its spend, what it never touches and where it stops. It stays a draft until it is approved, and the mission file records who approved it and when. Nothing runs under a draft. Checked by `lab-mission-approved`, which checks the record, not the word.
|
|
111
|
+
|
|
112
|
+
The operator approves a mission. Under a standing budget the commander may approve one itself when the mission fits inside what is left of the budget and needs nothing that stays the operator's: new money, credentials, publishing outside the lab, or a change to a frozen surface, a lock or a record. It records the approval as its own, with the budget named, and reports it to the operator without waiting. Judged, except that the budget is checked by `lab-mission-approved`.
|
|
109
113
|
|
|
110
|
-
A running mission comes back to the operator only at the stops its plan names and at the method's own: a frozen surface, a lock, spend past the cap, publishing outside the lab. Everything else it decides and records. Judged, not checked.
|
|
114
|
+
A running mission comes back to the operator only at the stops its plan names and at the method's own: a frozen surface, a lock, spend past the cap, publishing outside the lab. Everything else it decides and records. Under a standing budget the commander also decides the stops its own plan names, and sets the caps of the missions under the budget, never below what a mission has already spent, while the total fits inside it. It records each call in the mission's log and reports it afterwards. Judged, not checked.
|
|
@@ -24,6 +24,7 @@ evidence -> protocol -> journal -> result -> claim -+-> report (frozen once liv
|
|
|
24
24
|
| Paper | the library's papers (documents) | the venue article | frozen at submission (folio's `frozen`) |
|
|
25
25
|
| State | `ops/STATE.md` (lab file) | what is true now and what is next | a pointer, rewritten freely (`lab-state-pointer`) |
|
|
26
26
|
| Missions | `ops/missions/` (lab files) | one operator objective, its approved plan and its log | approved before it runs (`lab-mission-approved`); the log is append-only (`lab-append-only`) |
|
|
27
|
+
| Standing budgets | `ops/budgets/` (lab files) | money the operator granted once for many missions | the missions under it fit inside it (`lab-mission-approved`); grants are only added (`lab-append-only`) |
|
|
27
28
|
|
|
28
29
|
## Promotion
|
|
29
30
|
|
|
@@ -32,6 +32,7 @@
|
|
|
32
32
|
"parts": [],
|
|
33
33
|
"path": "content/index.html",
|
|
34
34
|
"rows": [],
|
|
35
|
+
"sections": [],
|
|
35
36
|
"status": "live",
|
|
36
37
|
"tags": [],
|
|
37
38
|
"title": "Monte Carlo lab",
|
|
@@ -311,6 +312,22 @@
|
|
|
311
312
|
"content/reports/pi-error-scaling-report/index.html",
|
|
312
313
|
"content/claims/C-1.md"
|
|
313
314
|
],
|
|
315
|
+
"sections": [
|
|
316
|
+
{
|
|
317
|
+
"rows": [
|
|
318
|
+
"content/projects/lab/index.html",
|
|
319
|
+
"content/questions/Q-1.md"
|
|
320
|
+
],
|
|
321
|
+
"title": "Start here"
|
|
322
|
+
},
|
|
323
|
+
{
|
|
324
|
+
"rows": [
|
|
325
|
+
"content/reports/pi-error-scaling-report/index.html",
|
|
326
|
+
"content/claims/C-1.md"
|
|
327
|
+
],
|
|
328
|
+
"title": "What we found"
|
|
329
|
+
}
|
|
330
|
+
],
|
|
314
331
|
"status": "live",
|
|
315
332
|
"tags": [],
|
|
316
333
|
"title": "Research",
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
lab-kit: 0.
|
|
1
|
+
lab-kit: 0.3.0 # the version this lab is checked with
|
|
2
2
|
library: docs # where folio.yaml is; "." for the root
|
|
3
3
|
front: lab # the front door's id: a project document
|
|
4
4
|
frozen: [substrate/] # surfaces never edited in place; record each with `lab-kit freeze`
|
|
@@ -97,6 +97,8 @@ Two things survive being wrong in place: a result, superseded or retracted and n
|
|
|
97
97
|
|
|
98
98
|
No live spend without the operator's word, recorded in a mission with a cap. Checked by `lab-spend-recorded`, which checks the record, not the word.
|
|
99
99
|
|
|
100
|
+
The operator gives that word for one mission, or once for many as a standing budget: `ops/budgets/<name>.yaml`, a list of grants, each with an amount and who granted it and when. A mission that draws on it names it in `budget:` and still records its own cap. What the missions under a budget commit fits inside the sum of its grants: an active mission commits its cap, or what it has spent when that is more, and a concluded one what it spent. Checked by `lab-mission-approved`. New money is a new grant at the bottom of the list, and only the operator makes one; a grant is never changed or removed. Checked by `lab-append-only`.
|
|
101
|
+
|
|
100
102
|
Prefer token-free stages first, to de-risk a live run. Use cheap models for mechanical work. Keep the expensive seat for synthesis. Judged, not checked.
|
|
101
103
|
|
|
102
104
|
Provider keys are never exported to a child process, written to a file, or committed. Judged, not checked.
|
|
@@ -105,6 +107,8 @@ No deadline drives the science. Never trim or rush a result to meet a date. Judg
|
|
|
105
107
|
|
|
106
108
|
## 14. The operator approves the mission
|
|
107
109
|
|
|
108
|
-
A mission has two phases. It is planned
|
|
110
|
+
A mission has two phases. It is planned, then it runs on its own. The plan names the questions it serves, what counts as done, its scope, its spend, what it never touches and where it stops. It stays a draft until it is approved, and the mission file records who approved it and when. Nothing runs under a draft. Checked by `lab-mission-approved`, which checks the record, not the word.
|
|
111
|
+
|
|
112
|
+
The operator approves a mission. Under a standing budget the commander may approve one itself when the mission fits inside what is left of the budget and needs nothing that stays the operator's: new money, credentials, publishing outside the lab, or a change to a frozen surface, a lock or a record. It records the approval as its own, with the budget named, and reports it to the operator without waiting. Judged, except that the budget is checked by `lab-mission-approved`.
|
|
109
113
|
|
|
110
|
-
A running mission comes back to the operator only at the stops its plan names and at the method's own: a frozen surface, a lock, spend past the cap, publishing outside the lab. Everything else it decides and records. Judged, not checked.
|
|
114
|
+
A running mission comes back to the operator only at the stops its plan names and at the method's own: a frozen surface, a lock, spend past the cap, publishing outside the lab. Everything else it decides and records. Under a standing budget the commander also decides the stops its own plan names, and sets the caps of the missions under the budget, never below what a mission has already spent, while the total fits inside it. It records each call in the mission's log and reports it afterwards. Judged, not checked.
|
|
@@ -24,6 +24,7 @@ evidence -> protocol -> journal -> result -> claim -+-> report (frozen once liv
|
|
|
24
24
|
| Paper | the library's papers (documents) | the venue article | frozen at submission (folio's `frozen`) |
|
|
25
25
|
| State | `ops/STATE.md` (lab file) | what is true now and what is next | a pointer, rewritten freely (`lab-state-pointer`) |
|
|
26
26
|
| Missions | `ops/missions/` (lab files) | one operator objective, its approved plan and its log | approved before it runs (`lab-mission-approved`); the log is append-only (`lab-append-only`) |
|
|
27
|
+
| Standing budgets | `ops/budgets/` (lab files) | money the operator granted once for many missions | the missions under it fit inside it (`lab-mission-approved`); grants are only added (`lab-append-only`) |
|
|
27
28
|
|
|
28
29
|
## Promotion
|
|
29
30
|
|
|
@@ -5,7 +5,7 @@ build-backend = "hatchling.build"
|
|
|
5
5
|
[project]
|
|
6
6
|
dynamic = ["readme"]
|
|
7
7
|
name = "lab-kit-cli"
|
|
8
|
-
version = "0.
|
|
8
|
+
version = "0.3.0"
|
|
9
9
|
description = "A research lab's method and machinery on top of folio: pre-register, lock, run, score, re-derive."
|
|
10
10
|
license = "MIT"
|
|
11
11
|
authors = [{ name = "Piergiuseppe Mallozzi" }]
|
|
@@ -47,12 +47,12 @@ You take one question from a draft protocol to a finished run with its evidence
|
|
|
47
47
|
- what it cannot show.
|
|
48
48
|
2. **Make its folder.** Run `lab-kit experiment <slug>`. Put the lab's scripts in `experiments/<slug>/bin/`, each with a `--selftest`. Run `lab-kit check --only lab-selftest`.
|
|
49
49
|
3. **Review the draft.** Send a reviewer with fresh context. Fold its points into the draft through the write skill. A protocol is cheap to fix before the lock and impossible to fix after.
|
|
50
|
-
4. **Lock.** Through the write skill, set the protocol's status to `locked`. Run `lab-kit lock <slug>`. Record it: `folio journal add --title .. --description .. --body .. --kind lock --about <slug>,<Q-n>`. Run `folio index`, then `lab-kit check`. Commit exactly the protocol, `lock.json`, the `lock` journal entry and `.folio/` together; that commit is the lock. From here folio's `frozen` check and `lab-lock-intact` hold it.
|
|
50
|
+
4. **Lock.** Commit `bin/` first: the lock hashes it, and `lab-lock-intact` fails if it changes after. Through the write skill, set the protocol's status to `locked`. Run `lab-kit lock <slug>`. Record it: `folio journal add --title .. --description .. --body .. --kind lock --about <slug>,<Q-n>`. Run `folio index`, then `lab-kit check`. Commit exactly the protocol, `lock.json`, the `lock` journal entry and `.folio/` together; that commit is the lock. From here folio's `frozen` check and `lab-lock-intact` hold it.
|
|
51
51
|
5. **De-risk.** Run the token-free stages first, with `lab-kit run`. Fix the apparatus only before the lock. After it, an apparatus fault ends the run as invalid.
|
|
52
52
|
6. **Launch.** Run `lab-kit run <slug> -- <command>`. Add `--spend --mission <file>` for a run that spends. Watch it at once: arm one watcher on its log that fires on the end marker or an error signature. A silent launch failure can waste a whole night.
|
|
53
53
|
7. **Supervise.** Write a journal entry of kind `run` about the protocol for each launch, checkpoint, surprise and termination, with `folio journal add`. Keep each entry to what changed. Record a deviation the moment you see it. Commit each entry with `.folio/` after `folio index`; never stage the run's working files.
|
|
54
54
|
8. **On failure.** End the run. Start a fresh one with the same configuration. Disclose it in a `run` entry. Never swap a model or a tool mid-run.
|
|
55
|
-
9. **Conclude.** When the run ends, `lab-kit run` writes the manifest of `out/`. Commit `out/`, `MANIFEST.sha256` and `run.json` in one commit. Run `lab-kit check`. Hand the experiment to the review skill. Do not score your own run.
|
|
55
|
+
9. **Conclude.** When the run ends, `lab-kit run` writes the manifest of `out/`. Commit `out/`, `MANIFEST.sha256`, `work.sha256` and `run.json` in one commit; `work/` stays out of git, and `work.sha256` is what any copy of it is verified against. Run `lab-kit check`. Hand the experiment to the review skill. Do not score your own run.
|
|
56
56
|
|
|
57
57
|
## Stops
|
|
58
58
|
|