@synapse-research/synapse 0.2.11 → 0.2.13

package/dist/public/synapse-plugin/skills/synapse/references/03-experiment-workflow.md

@@ -1,319 +0,0 @@
-# Experiment Workflow
-
-This guide covers the current experiment lifecycle: planning or revising experiment specs, executing approved experiments, reporting progress, and saving final reports.
-
-All execution happens through Synapse MCP tools plus Claude Code's own tools (Bash for remote shells, Task for sub-agent dispatch). The Synapse Claude Code plugin never requires a custom runtime on GPU nodes — remote work is driven over SSH using the access bundle.
-
----
-
-## Experiment Lifecycle
-
-```text
-draft --> pending_review --> pending_start --> in_progress --> completed
-```
-
-- `draft` — being authored or revised
-- `pending_review` — waiting for human review
-- `pending_start` — approved and ready for execution
-- `in_progress` — actively running
-- `completed` — results submitted
-
-Each experiment card must represent **one independent run**. Do not bundle comparison runs, ablations, or parameter sweeps into a single card — create multiple cards instead.
-
----
-
-## Getting Assigned Experiments
-
-```text
-synapse_get_assigned_experiments()
-synapse_get_assigned_experiments({ researchProjectUuid: "..." })
-synapse_get_assigned_experiments({ statuses: ["pending_start", "in_progress"] })
-```
-
-For full details:
-
-```text
-synapse_get_experiment({ experimentUuid: "..." })
-```
-
----
-
-## Foundational First Experiment
-
-If a project has no completed experiments yet, treat the first experiment as foundational infrastructure, not a normal research run. Bundle three deliverables:
-
-1. **Data preparation** — normalize the raw dataset into a single canonical format every future experiment will consume. Script lives under the project's repo (if `synapse_get_repo_access` shows one is configured).
-2. **Baseline** — the simplest reasonable approach, run end-to-end, with metrics recorded via `synapse_submit_experiment_results`. This becomes the reference all future experiments compare against.
-3. **Evaluation script** — a canonical eval harness that future experiments call. Committed alongside data prep.
-
-If the project has a repo, commit all three onto the base branch (or a per-experiment branch merged back into the base branch). Every subsequent experiment branches from that base so it inherits prep + eval.
-
----
-
-## Planning Or Revising An Experiment
-
-When Synapse asks you to flesh out a quick experiment idea or revise a reviewer-returned experiment:
-
-```text
-# Mark that you are drafting
-synapse_update_experiment_status({
-  experimentUuid: "...",
-  status: "draft",
-  liveStatus: "writing",
-  liveMessage: "Drafting experiment plan"
-})
-
-# Save the full plan
-synapse_update_experiment_plan({
-  experimentUuid: "...",
-  title: "Refined title",
-  description: "## Objective\n\n...",
-  researchQuestionUuid: "...",
-  priority: "high"
-})
-
-# Hand it back for review
-synapse_update_experiment_status({
-  experimentUuid: "...",
-  status: "pending_review"
-})
-```
-
-When revising based on reviewer feedback, read the full comment thread first — feedback is often scattered across multiple comments:
-
-```text
-synapse_get_comments({
-  targetType: "experiment",
-  targetUuid: "..."
-})
-```
-
-Write the plan in the same language as the project description. After revising, reply to the reviewer with `synapse_add_comment` using the exact mention format `@[name](actorType:uuid)`.
-
-### Plan content
-
-A good experiment plan covers:
-- **Objective** — the one thing this experiment is trying to learn
-- **Methodology** — enough detail that another agent (or sub-agent) could execute it unattended
-- **Expected outcomes** — what success and failure look like against the project's evaluation method
-- **Implementation steps** — data prep, training/inference, evaluation, analysis
-- **Resource requirements** — expected GPU count, wall clock, dataset size
-
----
-
-## Creating A New Experiment
-
-When the user asks you to author a brand-new experiment outside autonomous loop, create it directly:
-
-```text
-synapse_create_experiment({
-  researchProjectUuid: "...",
-  title: "Baseline reproduction with revised tokenizer",
-  description: "## Objective\n\n...",
-  researchQuestionUuid: "...",
-  priority: "high"
-})
-```
-
-Defaults:
-- `status = pending_review` for the normal agent-created path
-- Use `status: "draft"` if you want to keep refining before sending it to review
-
-Typical follow-up after creating a draft:
-
-```text
-synapse_update_experiment_plan({ experimentUuid: "...", description: "## Objective\n\n..." })
-synapse_update_experiment_status({ experimentUuid: "...", status: "pending_review" })
-```
-
----
-
-## Execution Checklist
-
-This is the detailed flow for moving an experiment through `in_progress` to `completed`. It assumes the experiment is already `pending_start`.
-
-1. **Build an internal todo list** covering: compute reservation, repo setup, data prep, run, monitoring, commit, submission. Keeps complex experiments honest.
-
-2. **Inspect available compute**
-
-   ```text
-   synapse_list_compute_nodes({
-     researchProjectUuid: "...",
-     onlyAvailable: true
-   })
-   ```
-
-   If the project has a `computePoolUuid`, your reservations must stay inside that pool.
-
-3. **Reserve GPUs** — either inline via `synapse_start_experiment({ gpuUuids: [...] })` or explicitly ahead of time:
-
-   ```text
-   synapse_reserve_gpus({
-     experimentUuid: "...",
-     gpuUuids: ["gpu-uuid-1", "gpu-uuid-2"]
-   })
-   ```
-
-4. **Start the experiment** — moves it to `in_progress` and creates/updates the experiment result document:
-
-   ```text
-   synapse_start_experiment({
-     experimentUuid: "...",
-     workingNotes: "Starting with baseline configuration"
-   })
-   ```
-
-5. **Fetch SSH access** — never assume a server-local key path exists:
-
-   ```text
-   synapse_get_node_access_bundle({
-     experimentUuid: "...",
-     nodeUuid: "..."
-   })
-   ```
-
-   Returns host / user / port and `privateKeyPemBase64`. Decode, write to a local PEM, `chmod 600`, then SSH using the returned host/user/port and the PEM.
-
-6. **Check out the repo** — if the project is repo-backed:
-
-   ```text
-   synapse_get_repo_access({ experimentUuid: "..." })
-   ```
-
-   Clone on the remote node, then check out the experiment's base branch (or create a per-experiment branch off the base). Subsequent experiments must inherit the base branch's data prep + eval scripts.
-
-7. **Run the workload in a persistent remote shell** — use `tmux` (or `screen`) so the session survives disconnects, and run Python with unbuffered output so logs do not stall tool calls:
-
-   ```bash
-   tmux new -d -s exp-<short> 'cd ~/work && PYTHONUNBUFFERED=1 python -u train.py --config exp.yaml 2>&1 | tee run.log'
-   ```
-
-8. **Monitoring — long runs (>30 min)**: poll from the main agent on a cadence, or set up a cron / periodic job on the remote node that calls back with a progress update. Use `synapse_report_experiment_progress` to push each milestone:
-
-   ```text
-   synapse_report_experiment_progress({
-     experimentUuid: "...",
-     message: "Epoch 15/100, loss: 0.342, val_acc: 87.2%",
-     phase: "training",
-     liveStatus: "running"
-   })
-   ```
-
-   `liveStatus` values:
-   - `checking_resources` while probing compute
-   - `queuing` while waiting for GPUs (status-only, does not create a progress-log row)
-   - `running` during active execution
-
-   `phase` labels: `setup`, `training`, `evaluation`, `analysis`.
-
-9. **Monitoring — short runs** (a few minutes): skip the cron, report progress inline at setup / mid-training / evaluation / analysis transitions.
-
-10. **Commit code and artifacts** — commit configs, scripts, and meaningful artifacts to the experiment branch (or base branch) and capture the commit SHA to include in the submission.
-
-11. **Submit results**
-
-    ```text
-    synapse_submit_experiment_results({
-      experimentUuid: "...",
-      outcome: "success",
-      experimentResults: {
-        "accuracy": 0.923,
-        "summary": "Ablation outperformed baseline",
-        "branch": "exp-ablation-3",
-        "commit": "abc1234"
-      }
-    })
-    ```
-
-    `outcome` is optional, typically `success`, `failure`, or `inconclusive`. Submitting moves the experiment to `completed`, refreshes the experiment result document, and triggers the project synthesis refresh.
-
-12. **Save the dedicated experiment report** — when the flow asks for a full writeup:
-
-    ```text
-    synapse_save_experiment_report({
-      experimentUuid: "...",
-      title: "Experiment Report: Baseline vs Ablation",
-      content: "# Objective\n\n..."
-    })
-    ```
-
-    Use python + a plotting library to generate charts and embed them in the markdown where they help. Do **not** post the report as a comment — always use `synapse_save_experiment_report` so the dedicated result document exists.
-
-13. **Match the project description's language** — if the project brief is in Chinese, write plan, progress messages, and report in Chinese.
-
----
-
-## Handling Failures And Inconclusive Runs
-
-A failed or inconclusive experiment is still a valid submission — it is data. Do not leave it stuck in `in_progress`:
-
-```text
-synapse_submit_experiment_results({
-  experimentUuid: "...",
-  outcome: "failure",
-  experimentResults: {
-    "error": "OOM at batch size 32 on 2x L40S",
-    "completedEpochs": 3,
-    "lastLoss": 1.82,
-    "nextSteps": "Retry with gradient accumulation or batch 16"
-  }
-})
-```
-
-Follow up with a comment explaining the failure so the reviewer can decide whether to revise and retry:
-
-```text
-synapse_add_comment({
-  targetType: "experiment",
-  targetUuid: "...",
-  content: "Failed due to OOM. Proposing a revised plan..."
-})
-```
-
----
-
-## Reviving A Stuck Experiment
-
-An experiment stuck in `in_progress` (sub-agent crashed, SSH died, user killed the run) can be unblocked in two ways:
-
-1. **Resume** — respawn a sub-agent with the same experiment UUID. The `SubagentStart` hook reuses the existing Synapse session if still active; otherwise it reopens the closed one. The sub-agent re-fetches node access and continues.
-2. **Close out** — if the run cannot be resumed, submit a failure:
-
-   ```text
-   synapse_submit_experiment_results({
-     experimentUuid: "...",
-     outcome: "failure",
-     experimentResults: { "error": "Sub-agent lost remote session, no checkpoint" }
-   })
-   ```
-
----
-
-## Handling Rejection During Review
-
-When a reviewer sends `pending_review` back to `draft`:
-
-1. Read the full comment thread — feedback may span multiple comments and a re-read of the plan.
-2. Flip to `draft` with `liveStatus: "writing"`.
-3. Revise title / description / researchQuestionUuid / priority via `synapse_update_experiment_plan`.
-4. Move it back with `synapse_update_experiment_status({ status: "pending_review" })`.
-5. `synapse_add_comment` on the experiment with `@[reviewerName](actorType:uuid)` so the reviewer sees the update.
-
----
-
-## Typical Execution Flow (Short Form)
-
-1. `synapse_checkin()` — see assigned experiments
-2. `synapse_get_experiment()` — understand the task
-3. `synapse_create_experiment()` — only if authoring new
-4. If drafting or revising: `synapse_update_experiment_status` + `synapse_update_experiment_plan`
-5. `synapse_list_compute_nodes()` — inspect availability
-6. `synapse_start_experiment()` with optional inline / explicit `synapse_reserve_gpus()`
-7. `synapse_get_node_access_bundle()` — write PEM locally, chmod 600, SSH
-8. Run workload in tmux + unbuffered python
-9. `synapse_report_experiment_progress()` at milestones
-10. `synapse_submit_experiment_results()` — success, failure, or inconclusive
-11. `synapse_save_experiment_report()` if a dedicated report is required
-12. `synapse_add_comment()` for durable findings and mention the reviewer
-
-For parallel multi-experiment dispatch (main agent orchestrates, sub-agents execute), see **[05-session-sub-agent.md](05-session-sub-agent.md)**.

package/dist/public/synapse-plugin/skills/synapse/references/04-autonomous-loop.md

@@ -1,223 +0,0 @@
-# Autonomous Loop (Claude Code-Client)
-
-This guide covers the **CC-client autonomous loop**: the main Claude Code agent drives research on a project without waiting for per-step human instruction. It proposes experiments, dispatches sub-agents to execute them, monitors progress, synthesizes results, and iterates.
-
-The CC-client loop runs entirely inside the current Claude Code session. It does not depend on the server-side `ResearchProject.autonomousLoopEnabled` flag — that flag is for realtime-transport agents. A user opens Claude Code, says "enable autonomous loop on project X", and the main agent takes over.
-
----
-
-## Architecture
-
-```
-Main CC agent (orchestrator)
-  ├── synapse_get_project_full_context     (each iteration)
-  ├── synapse_get_assigned_experiments     (each iteration)
-  ├── synapse_list_compute_nodes           (before proposing)
-  ├── Task()   → sub-agent → executes experiment
-  ├── Task()   → sub-agent → executes experiment     (parallel)
-  ├── synapse_save_project_synthesis       (when new results exist)
-  ├── synapse_propose_experiment           (when queue empty)
-  └── synapse_add_comment                  (audit trail)
-```
-
-- The main agent **never** runs training itself. It monitors and proposes.
-- Sub-agents spawned via the Task tool are auto-enrolled by the `SubagentStart` hook: session created, UUID injected, execution workflow attached.
-- State persists in Synapse (experiments, results, synthesis document). No extra local state is required.
-
----
-
-## Modes
-
-### Full Auto (default)
-
-- Proposals created as `pending_start` and immediately dispatched to sub-agents.
-- No human approval between iterations.
-- User can stop at any time by interrupting the session or saying "stop autonomous loop".
-
-### Review
-
-- Proposals created as `pending_review`.
-- The main agent pauses until a human approves, then returns to the loop.
-- Use when the user wants a sanity check on each proposed experiment.
-
-Choose the mode at loop start. Default is full auto unless the user explicitly asks for review.
-
----
-
-## Before Starting
-
-1. `synapse_checkin()` — refresh identity, roles, assignments.
-2. `synapse_get_research_project({ researchProjectUuid })` — confirm the target project and check its `autonomousLoopEnabled` flag.
-3. If `autonomousLoopEnabled = true` and a realtime loop agent is already assigned, warn the user: running the CC-client loop against the same project will double-dispatch proposals. Ask the user to either disable the server-side loop or run the CC loop on a different project.
-4. `synapse_get_project_full_context({ researchProjectUuid })` — load the brief, evaluation methods, past experiments, and latest synthesis.
-5. If the project has no completed experiments, do not enter the loop cold. Offer the foundational path first (see **[03-experiment-workflow.md](03-experiment-workflow.md)** — Foundational First Experiment). Once the baseline exists, the loop has something to build on.
-6. Collect budgets from the user: `maxIterations`, `maxExperimentsProposed`, optional `maxComputeHours`.
-
----
-
-## Iteration Procedure
-
-Each iteration runs the same five steps:
-
-### Step 1: Refresh state
-
-```text
-synapse_get_project_full_context({ researchProjectUuid: "..." })
-synapse_get_assigned_experiments({
-  researchProjectUuid: "...",
-  statuses: ["pending_start", "in_progress"]
-})
-```
-
-### Step 2: Decide what this iteration does
-
-Priority order:
-
-- **Monitor** — if any experiment is `in_progress`, do not propose. Poll `synapse_get_experiment` for each, write a short progress note, and yield.
-- **Dispatch** — if any experiment is `pending_start` and mode is `full_auto`, spawn a Task sub-agent per experiment (see Step 3). Do not wait for them to finish — sub-agents run in parallel.
-- **Synthesize** — if the queue is empty and new completed experiments exist, read the current `project_synthesis` document with `synapse_get_documents({ type: "project_synthesis" })` + `synapse_get_document`. Update it with `synapse_save_project_synthesis` only if the new evidence is not already covered.
-- **Propose** — if the queue is empty and synthesis is current, call `synapse_propose_experiment`. One independent run per proposal — split comparisons, ablations, and parameter sweeps into separate proposals.
-
-### Step 3: Dispatch sub-agents for `pending_start` experiments
-
-```text
-Task({
-  subagent_type: "general-purpose",
-  name: "exp-<short-id>",
-  prompt: "Your Synapse experiment UUID: <experiment-uuid>. Execute the experiment end to end following the experiments skill."
-})
-```
-
-The `SubagentStart` hook automatically creates/reuses a Synapse session, injects the session UUID, and includes the execution workflow. The main agent does not call `synapse_create_session`.
-
-Dispatch all `pending_start` experiments concurrently if compute allows. Let the sub-agents do the work — the main agent only tracks them via `synapse_get_assigned_experiments` and `synapse_get_experiment` on subsequent iterations.
-
-### Step 4: Respect compute reality
-
-Before proposing new experiments:
-
-```text
-synapse_list_compute_nodes({
-  researchProjectUuid: "...",
-  onlyAvailable: true
-})
-```
-
-Do not over-propose concurrent experiments beyond what free GPUs can accommodate. If the project has a `computePoolUuid`, all proposals must fit within that pool.
-
-### Step 5: Log and check stop conditions
-
-Leave an audit trail for the human:
-
-```text
-synapse_add_comment({
-  targetType: "research_project",
-  targetUuid: "...",
-  content: "Iteration 5 — dispatched exp <uuid1>, <uuid2>; no proposal (queue saturated)."
-})
-```
-
-Then evaluate stop conditions (next section). If none trigger, continue to the next iteration.
-
----
-
-## Writing A Proposal
-
-`synapse_propose_experiment` creates a new experiment card. A good proposal includes:
-
-- **Motivation** — what prior result or gap this addresses
-- **Hypothesis** — what you expect to learn
-- **Method** — enough detail that a sub-agent can execute without further prompting
-- **Success criteria** — how the result will be judged against the project's evaluation method
-- **Compute fit** — realistic given current availability
-
-```text
-synapse_propose_experiment({
-  researchProjectUuid: "...",
-  title: "Ablation: remove cross-attention from layer 6",
-  description: "## Motivation\n\nExperiment <uuid> showed..."
-})
-```
-
-In full auto mode the resulting experiment lands as `pending_start` and is auto-assigned back to the loop agent for execution on the next iteration. In review mode it lands as `pending_review`.
-
-**One independent run per proposal.** Do not bundle comparisons, ablations, or parameter sweeps into one card — create multiple proposals instead.
-
----
-
-## Stop Conditions
-
-Exit the loop cleanly when any of the following hold:
-
-- `maxIterations` reached
-- `maxExperimentsProposed` reached
-- `maxComputeHours` consumed across loop-dispatched experiments
-- synthesis unchanged across K consecutive iterations (no-progress signal)
-- all research questions are `completed` and no promising direction remains
-- compute pool exhausted and no experiment is making forward progress
-- user interrupts or says "stop"
-
-On exit, summarize for the user:
-
-- experiments dispatched and their outcomes
-- whether the synthesis changed materially
-- open questions and recommended next steps the human should consider
-
----
-
-## Mutual Exclusion With The Server-Side Loop
-
-The server-side autonomous loop (`autonomousLoopEnabled`) is designed for realtime-transport agents and dispatches proposals through a different path than the CC-client loop. Running both against the same project will cause double-dispatch.
-
-Before entering the CC-client loop:
-
-- If `autonomousLoopEnabled = true` → warn the user, do not start.
-- If the user insists, ask them to disable the server-side loop first, or use a different project.
-
-Do not try to take over the server-side flag from inside Claude Code.
-
----
-
-## Compact End-to-End Example
-
-```text
-# Turn 1 — enter loop
-synapse_checkin()
-synapse_get_research_project({ researchProjectUuid })          # check flag
-synapse_get_project_full_context({ researchProjectUuid })
-
-# Iteration 1
-synapse_get_assigned_experiments({ researchProjectUuid, statuses: ["pending_start", "in_progress"] })
-# queue empty, no new results
-synapse_list_compute_nodes({ researchProjectUuid, onlyAvailable: true })
-synapse_propose_experiment({ researchProjectUuid, title, description })
-synapse_add_comment({ targetType: "research_project", targetUuid, content: "Iter 1 — proposed <title>" })
-
-# Iteration 2 (same turn or next turn)
-synapse_get_assigned_experiments(...)                           # one pending_start now
-Task({ name: "exp-ablation-1", prompt: "Your Synapse experiment UUID: <uuid>. Execute end to end." })
-synapse_add_comment({ ..., content: "Iter 2 — dispatched exp <uuid>" })
-
-# Iteration 3
-synapse_get_assigned_experiments(...)                           # in_progress
-# monitor only, no proposal
-
-# Iteration 4
-synapse_get_assigned_experiments(...)                           # completed
-synapse_get_documents({ researchProjectUuid, type: "project_synthesis" })
-synapse_save_project_synthesis({ researchProjectUuid, title, content })
-# queue empty again → next propose
-
-# ...until a stop condition fires
-```
-
----
-
-## Tips
-
-- Do not re-propose an experiment that has already completed. Always check `synapse_get_project_full_context` for prior work first.
-- Build on failures — an `outcome: "failure"` experiment is signal, not noise.
-- Stay aligned with the project's research questions and evaluation method.
-- Proposals should be specific enough that a sub-agent can execute them unattended.
-- Use the compute availability summary to keep proposals realistic.
-- Leave a comment on the project each iteration — it is the user's only visible log when the loop runs for hours.