@paradigma-inc/flywheel 0.1.11 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/skills/flywheel/SKILL.md +7 -25
- package/skills/flywheel/agents/openai.yaml +2 -2
- package/skills/flywheel/example-workflows/organizing-exploring-and-iterating-on-a-research-topic.md +8 -8
- package/skills/flywheel/example-workflows/reproducing-papers-on-a-budget.md +8 -8
- package/skills/flywheel/getting-started/flywheel-tutorial-overview.md +1 -1
- package/skills/flywheel/references/experiment-design-protocol.md +204 -0
- package/skills/flywheel/references/flywheel-mcp-tool-map.md +160 -0
- package/skills/flywheel/web-ui/flywheel-webui-map.md +1 -1
- package/skills/flywheel-auto/SKILL.md +156 -0
- package/skills/flywheel-auto/agents/openai.yaml +12 -0
- package/skills/flywheel-auto/evals/evals.json +20 -0
- package/skills/flywheel-auto/references/ARTIFACTS.md +218 -0
- package/skills/flywheel-auto/references/INTERFACES.md +201 -0
- package/skills/{flywheel/reference → flywheel-auto/references}/experiment-design-protocol.md +9 -5
- package/skills/flywheel-lookahead/SKILL.md +145 -0
- package/skills/flywheel-lookahead/agents/openai.yaml +12 -0
- package/skills/flywheel-lookahead/evals/evals.json +20 -0
- package/skills/flywheel-lookahead/references/ARTIFACTS.md +218 -0
- package/skills/flywheel-lookahead/references/INTERFACES.md +201 -0
- package/skills/flywheel-lookahead/references/flywheel-mcp-tool-map.md +160 -0
- package/skills/flywheel-reproduce/SKILL.md +212 -0
- package/skills/flywheel-reproduce/agents/openai.yaml +12 -0
- package/skills/flywheel-reproduce/evals/evals.json +20 -0
- package/skills/flywheel-reproduce/references/ARTIFACTS.md +218 -0
- package/skills/flywheel-reproduce/references/INTERFACES.md +201 -0
- package/skills/flywheel-reproduce/references/experiment-design-protocol.md +204 -0
- package/skills/flywheel-reproduce/references/flywheel-mcp-tool-map.md +160 -0
- package/skills/flywheel-to-graph/SKILL.md +187 -0
- package/skills/flywheel-to-graph/agents/openai.yaml +12 -0
- package/skills/flywheel-to-graph/evals/evals.json +20 -0
- package/skills/flywheel-to-graph/references/ARTIFACTS.md +218 -0
- package/skills/flywheel-to-graph/references/INTERFACES.md +201 -0
- package/skills/flywheel-to-graph/references/flywheel-mcp-tool-map.md +160 -0
- package/skills/flywheel/reference/command-presets.md +0 -230
- package/skills/flywheel/reference/wiki-porting.md +0 -92
- /package/skills/{flywheel/reference → flywheel-auto/references}/flywheel-mcp-tool-map.md +0 -0
package/package.json
CHANGED
package/skills/flywheel/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: flywheel
|
|
3
|
-
description:
|
|
3
|
+
description: General Flywheel guidance for experiment design, MCP/tool-contract questions, and setup or troubleshooting across supported hosts. Use for platform support or cross-cutting guidance, not concrete graph-authoring tasks covered by the specialized Flywheel skills.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Flywheel Skill
|
|
@@ -12,9 +12,6 @@ Use this skill to structure research work and to route setup/support questions t
|
|
|
12
12
|
Use this skill when the conversation is about any of the following:
|
|
13
13
|
|
|
14
14
|
- planning experiments, hypotheses, comparisons, or next-branch decisions
|
|
15
|
-
- turning source material into a Flywheel graph
|
|
16
|
-
- using command-like Flywheel workflows such as `/to-graph`, `/reproduce`,
|
|
17
|
-
`/lookahead`, or `/fsd`
|
|
18
15
|
- running or interpreting empirical work in Flywheel
|
|
19
16
|
- choosing between `insight` and `empirical` nodes
|
|
20
17
|
- setting up, updating, authenticating, or troubleshooting Flywheel MCP
|
|
@@ -26,24 +23,18 @@ If the request is ambiguous, default to experiment-design guidance before platfo
|
|
|
26
23
|
|
|
27
24
|
1. Classify the request as research guidance, platform support, or mixed.
|
|
28
25
|
2. Load only the minimum files needed for the current task.
|
|
29
|
-
3.
|
|
30
|
-
`/reproduce`, `/lookahead`, or `/fsd`, load
|
|
31
|
-
`reference/command-presets.md` first.
|
|
32
|
-
4. For research requests, run a short design pass before execution:
|
|
26
|
+
3. For research requests, run a short design pass before execution:
|
|
33
27
|
ask 1-2 questions to clarify objective, assumptions, and decision criteria.
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
28
|
+
4. If design is solid and the user wants to proceed, shift to execution support.
|
|
29
|
+
5. Use Flywheel MCP tools for live behavior/contract checks when available.
|
|
30
|
+
6. Summarize evidence and propose the most likely next branch.
|
|
37
31
|
|
|
38
32
|
## Routing Map
|
|
39
33
|
|
|
40
34
|
Choose one primary file first, then add at most one supporting file only if needed:
|
|
41
35
|
|
|
42
|
-
- `
|
|
43
|
-
- `
|
|
44
|
-
and `/fsd`
|
|
45
|
-
- `reference/wiki-porting.md` when the user asks to port a markdown/wiki corpus
|
|
46
|
-
- `reference/flywheel-mcp-tool-map.md` for exact tool surface and contract behavior
|
|
36
|
+
- `references/experiment-design-protocol.md` for structured experiment design and readiness checks
|
|
37
|
+
- `references/flywheel-mcp-tool-map.md` for exact tool surface and contract behavior
|
|
47
38
|
- `getting-started/flywheel-tutorial-overview.md` for orientation and learning path
|
|
48
39
|
- `getting-started/flywheel-quickstart.md` for fastest first-use setup and auth recovery
|
|
49
40
|
- `setting-up-flywheel/` for host-specific installation and update procedures
|
|
@@ -59,12 +50,3 @@ Choose one primary file first, then add at most one supporting file only if need
|
|
|
59
50
|
- Prefer concrete steps and decision points over long narrative explanations.
|
|
60
51
|
- Do not run broad document reads; load only what is required by the current request.
|
|
61
52
|
- Do not start compute-heavy execution before design intent is clear.
|
|
62
|
-
- Keep `/reproduce` narrow: validate existing knowledge claims rather than
|
|
63
|
-
expanding an open-ended research frontier.
|
|
64
|
-
- Treat `/lookahead` as planning only and `/fsd` as budgeted autonomous
|
|
65
|
-
frontier advancement.
|
|
66
|
-
|
|
67
|
-
## Wiki Porting
|
|
68
|
-
|
|
69
|
-
When asked to port a markdown wiki, load `reference/wiki-porting.md` before
|
|
70
|
-
acting.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
interface:
|
|
2
2
|
display_name: "Flywheel"
|
|
3
|
-
short_description: "Guide Flywheel
|
|
4
|
-
default_prompt: "Use $flywheel to answer Flywheel experiment,
|
|
3
|
+
short_description: "Guide Flywheel experiment design, setup, tools, and workflows"
|
|
4
|
+
default_prompt: "Use $flywheel to answer Flywheel experiment, setup, MCP tool, compute, campaign, or Web UI questions."
|
|
5
5
|
|
|
6
6
|
dependencies:
|
|
7
7
|
tools:
|
package/skills/flywheel/example-workflows/organizing-exploring-and-iterating-on-a-research-topic.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Organizing, Exploring, and Iterating on a Research Topic
|
|
2
2
|
|
|
3
|
-
> The tool lists below are inferred from the current [Flywheel MCP tool map](../
|
|
3
|
+
> The tool lists below are inferred from the current [Flywheel MCP tool map](../references/flywheel-mcp-tool-map.md). They are likely tool sequences for each step, not verbatim execution logs.
|
|
4
4
|
|
|
5
5
|
## Image Description
|
|
6
6
|
|
|
@@ -173,8 +173,8 @@ at this point i started testing this out, would i get really the right spectra?
|
|
|
173
173
|
|
|
174
174
|
- `mcp__flywheel__flywheel_branch_node` to create the empirical branch for the spectra sanity check.
|
|
175
175
|
- `mcp__flywheel__flywheel_stage_node_update` to record the hypothesis and evaluation criteria.
|
|
176
|
-
- `
|
|
177
|
-
- `
|
|
176
|
+
- `mcp__flywheel__flywheel_request_compute_grant_approval`, `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to run the experiments on managed compute.
|
|
177
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach the singular-value plot as an artifact.
|
|
178
178
|
- `mcp__flywheel__flywheel_commit_node` to mark the empirical step as completed.
|
|
179
179
|
|
|
180
180
|
### Image Description
|
|
@@ -217,9 +217,9 @@ but which way should we schedule? not clear. is it sgd first or muon first? how
|
|
|
217
217
|
|
|
218
218
|
- `mcp__flywheel__flywheel_branch_node` to split the root into separate schedule and learning-rate branches.
|
|
219
219
|
- `mcp__flywheel__flywheel_stage_node_update` to encode the hypotheses and sweep definitions.
|
|
220
|
-
- `
|
|
220
|
+
- `mcp__flywheel__flywheel_request_compute_grant_approval` and `mcp__flywheel__flywheel_list_compute_grants` to secure spend before launch.
|
|
221
221
|
- `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to provision GPUs and keep them available to the agent.
|
|
222
|
-
- `
|
|
222
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach result plots and cards as branches finish.
|
|
223
223
|
- `mcp__flywheel__flywheel_get_node_tree` or `mcp__flywheel__flywheel_list_executions` to monitor the graph as the runs progress.
|
|
224
224
|
- `mcp__flywheel__flywheel_commit_node` to finalize each completed empirical branch.
|
|
225
225
|
|
|
@@ -249,7 +249,7 @@ i won't spoil for you how this ends. because this will soon be a public graph in
|
|
|
249
249
|
|
|
250
250
|
### Likely Flywheel MCP tools for this step
|
|
251
251
|
|
|
252
|
-
- `
|
|
253
|
-
- `
|
|
254
|
-
- `
|
|
252
|
+
- `mcp__flywheel__flywheel_get_node_sharing` to inspect the current sharing state before changing it.
|
|
253
|
+
- `mcp__flywheel__flywheel_set_sharing_for_node` or `mcp__flywheel__flywheel_set_sharing_for_nodes` to share the graph with co-authors or make it public.
|
|
254
|
+
- `mcp__flywheel__flywheel_get_node_sharing` again to verify that the sharing settings are correct.
|
|
255
255
|
- `mcp__flywheel__flywheel_export_summary` or `mcp__flywheel__flywheel_export_subgraph` to package the graph for outside review or reuse.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Reproducing Papers on a Budget
|
|
2
2
|
|
|
3
|
-
> The tool lists below are inferred from the current [Flywheel MCP tool map](../
|
|
3
|
+
> The tool lists below are inferred from the current [Flywheel MCP tool map](../references/flywheel-mcp-tool-map.md). They are likely tool sequences for each step, not verbatim execution logs.
|
|
4
4
|
|
|
5
5
|
## [@tensorqt](https://x.com/tensorqt) · [13 Mar 2026, 9:34 pm](https://x.com/tensorqt/status/2032555892399305171)
|
|
6
6
|
|
|
@@ -11,7 +11,7 @@ A really interesting way of using Flywheel is to reproduce papers on a budget: I
|
|
|
11
11
|
- `mcp__flywheel__flywheel_stage_node_create` to create the root reproduction graph or first node for the paper.
|
|
12
12
|
- `mcp__flywheel__flywheel_branch_node` to split the paper into parallel validation branches.
|
|
13
13
|
- `mcp__flywheel__flywheel_stage_node_update` to capture the paper claim, reproduction plan, and budget constraint.
|
|
14
|
-
- `
|
|
14
|
+
- `mcp__flywheel__flywheel_request_compute_grant_approval` to enforce the "$10 max" spend before managed compute starts.
|
|
15
15
|
- `mcp__flywheel__flywheel_commit_node` to persist the planned graph structure.
|
|
16
16
|
|
|
17
17
|
### Image Description
|
|
@@ -41,7 +41,7 @@ The model decides to play it safe: it starts by showing self-attention correlati
|
|
|
41
41
|
- `mcp__flywheel__flywheel_branch_node` to open the first empirical branch for the baseline correlation check.
|
|
42
42
|
- `mcp__flywheel__flywheel_stage_node_update` to record the hypothesis and experiment setup.
|
|
43
43
|
- `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to provision and use the small A10 GPU.
|
|
44
|
-
- `
|
|
44
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach `attn_similarity_by_layer.png`.
|
|
45
45
|
- `mcp__flywheel__flywheel_commit_node` to mark the step as a completed empirical result.
|
|
46
46
|
|
|
47
47
|
### Image Description
|
|
@@ -61,7 +61,7 @@ Step two is analogous: orthogonality before and after XSA.
|
|
|
61
61
|
- `mcp__flywheel__flywheel_branch_node` to create the next experimental branch.
|
|
62
62
|
- `mcp__flywheel__flywheel_stage_node_update` to record the orthogonality hypothesis for XSA.
|
|
63
63
|
- `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to run the projection check.
|
|
64
|
-
- `
|
|
64
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach `orthogonality_by_layer.png`.
|
|
65
65
|
- `mcp__flywheel__flywheel_commit_node` to finalize the branch.
|
|
66
66
|
|
|
67
67
|
### Image Description
|
|
@@ -81,7 +81,7 @@ Then, it moves to overhead of the orthogonal projection.
|
|
|
81
81
|
- `mcp__flywheel__flywheel_branch_node` to fork an efficiency-comparison branch.
|
|
82
82
|
- `mcp__flywheel__flywheel_stage_node_update` to define the runtime and VRAM hypothesis.
|
|
83
83
|
- `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to run the matched SA vs XSA benchmark.
|
|
84
|
-
- `
|
|
84
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach `efficiency_comparison.png`.
|
|
85
85
|
- `mcp__flywheel__flywheel_commit_node` to finalize the overhead measurement.
|
|
86
86
|
|
|
87
87
|
### Image Description
|
|
@@ -101,7 +101,7 @@ Up to now, no trouble. The agent then decides to train a 50M parameter model wit
|
|
|
101
101
|
- `mcp__flywheel__flywheel_branch_node` to create the larger training branch.
|
|
102
102
|
- `mcp__flywheel__flywheel_stage_node_update` to record the 50M-parameter comparison and evaluation criteria.
|
|
103
103
|
- `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to run the longer training jobs.
|
|
104
|
-
- `
|
|
104
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach `run_metrics.csv` and `quality_pairs.csv`.
|
|
105
105
|
- `mcp__flywheel__flywheel_commit_node` to record that the empirical result did not reproduce the expected gain.
|
|
106
106
|
|
|
107
107
|
### Image Description
|
|
@@ -121,7 +121,7 @@ Also, across sequence length, the trend seems to worsen.
|
|
|
121
121
|
- `mcp__flywheel__flywheel_branch_node` to spin off an analysis branch for context length.
|
|
122
122
|
- `mcp__flywheel__flywheel_get_node` and `mcp__flywheel__flywheel_list_artifacts` to gather results from the completed training branches.
|
|
123
123
|
- `mcp__flywheel__flywheel_stage_node_update` to record the interpretation that the delta worsens at longer context.
|
|
124
|
-
- `
|
|
124
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach `length_trend.png`.
|
|
125
125
|
- `mcp__flywheel__flywheel_commit_node` to persist the analysis.
|
|
126
126
|
|
|
127
127
|
### Image Description
|
|
@@ -141,7 +141,7 @@ At the same time, baseline seems to also win the LR sweeps.
|
|
|
141
141
|
- `mcp__flywheel__flywheel_branch_node` to create a robustness branch for LR sweeps and sink-style perturbations.
|
|
142
142
|
- `mcp__flywheel__flywheel_stage_node_update` to capture the robustness hypothesis.
|
|
143
143
|
- `mcp__flywheel__flywheel_compute_acquire`, `mcp__flywheel__flywheel_compute_status`, and `mcp__flywheel__flywheel_compute_connection` to run the compact sweep experiments.
|
|
144
|
-
- `
|
|
144
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, and `mcp__flywheel__flywheel_finalize_artifact_uploads` to attach `robustness_deltas.png`.
|
|
145
145
|
- `mcp__flywheel__flywheel_commit_node` to finalize the robustness verdict.
|
|
146
146
|
|
|
147
147
|
### Image Description
|
|
@@ -26,5 +26,5 @@ While in SWE we use git to manage changes to code, in research we are interested
|
|
|
26
26
|
- [Using local hardware with Flywheel](../usage-and-workflows/using-local-hardware-with-flywheel.md)
|
|
27
27
|
- [Participating in a Campaign](../campaigns/participating-in-a-campaign.md)
|
|
28
28
|
- [OAuth client registration FAQ](../setting-up-flywheel/how-can-i-get-an-authorized-client_id-for-the-oauth-flow.md)
|
|
29
|
-
- [Flywheel MCP tool map](../
|
|
29
|
+
- [Flywheel MCP tool map](../references/flywheel-mcp-tool-map.md)
|
|
30
30
|
- [Example workflows](../example-workflows/)
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
# Experiment Design Protocol
|
|
2
|
+
|
|
3
|
+
Use this when the user needs help turning research intent into a well-formed experiment or exploration before spending compute.
|
|
4
|
+
|
|
5
|
+
## Goal
|
|
6
|
+
|
|
7
|
+
Help the user clarify what they are trying to learn, shape the work around that question, and avoid wasting compute before the design is solid.
|
|
8
|
+
|
|
9
|
+
## Operating rules
|
|
10
|
+
|
|
11
|
+
- Treat design and execution as separate phases.
|
|
12
|
+
- Optimize for clarity of experimental purpose, not speed to a first run.
|
|
13
|
+
- Adapt depth to the user's experience and the clarity already present in the conversation.
|
|
14
|
+
- Apply epistemic discipline: separate what is known from what is assumed, and name uncertainty instead of smoothing it over.
|
|
15
|
+
- Support both hypothesis-driven and exploratory work.
|
|
16
|
+
- Support simple runs and complex shapes such as multi-stage pipelines, sweep-then-deep-dive, multi-arm comparisons, and custom structures.
|
|
17
|
+
- Keep all 10 brief fields, but allow exploratory fields to be marked `exploratory` or `TBD` rather than fabricated.
|
|
18
|
+
- Use quick Socratic questioning to surface assumptions, confidence, and what would change the user's mind. Keep it to 1-2 short questions per turn.
|
|
19
|
+
- Propose defaults for structural choices such as experiment shape, stop condition, or artifact plan. Use questions rather than defaults for epistemic choices such as beliefs, assumptions, and what evidence would matter.
|
|
20
|
+
- When a reasoning or design gap is visible, raise it as a question rather than an assertion.
|
|
21
|
+
- If the core gate is satisfied and the user wants to proceed, stop asking more design questions.
|
|
22
|
+
- Use Flywheel `insight` nodes to preserve design context when it will help across turns or sessions.
|
|
23
|
+
|
|
24
|
+
## Phase 1: Clarify what the user is trying to learn
|
|
25
|
+
|
|
26
|
+
Start with:
|
|
27
|
+
|
|
28
|
+
1. What are you trying to learn or decide?
|
|
29
|
+
2. Is this mainly hypothesis-driven or exploratory right now?
|
|
30
|
+
|
|
31
|
+
Keep this phase quick. Ask 1-2 short questions per turn, and use light Socratic questioning as an epistemic check after the user states the learning goal: briefly surface what seems known versus assumed before moving on.
|
|
32
|
+
|
|
33
|
+
If the work is hypothesis-driven, ask:
|
|
34
|
+
|
|
35
|
+
- What is the hypothesis?
|
|
36
|
+
- Compared to what baseline or alternative?
|
|
37
|
+
- What result would matter?
|
|
38
|
+
|
|
39
|
+
If the work is exploratory, ask:
|
|
40
|
+
|
|
41
|
+
- What is the big question?
|
|
42
|
+
- What would you need to learn first before tackling it?
|
|
43
|
+
- What is the cheapest or cleanest way to learn that first piece?
|
|
44
|
+
- What signal or pattern are you looking for?
|
|
45
|
+
|
|
46
|
+
If the user is still fuzzy after this phase, stay in planning mode. If needed, create or update an `insight` node rather than an `empirical` node.
|
|
47
|
+
|
|
48
|
+
## Phase 2: Shape the work
|
|
49
|
+
|
|
50
|
+
Choose or define the experiment shape through quick Socratic questioning:
|
|
51
|
+
|
|
52
|
+
- single focused run
|
|
53
|
+
- multi-stage pipeline
|
|
54
|
+
- sweep then deep-dive
|
|
55
|
+
- multi-arm comparison
|
|
56
|
+
- custom shape
|
|
57
|
+
|
|
58
|
+
If the user is unsure about structure, propose a default shape, stop condition, or artifact plan instead of extending the question loop.
|
|
59
|
+
|
|
60
|
+
Then fill the experiment brief:
|
|
61
|
+
|
|
62
|
+
- `question`: the decision or learning goal
|
|
63
|
+
- `hypothesis`: the claim being tested
|
|
64
|
+
- `comparator`: the baseline or alternative
|
|
65
|
+
- `unit_of_work`: what one run, branch, or stage actually changes
|
|
66
|
+
- `primary_metric`: the main number or observable to inspect
|
|
67
|
+
- `artifact_plan`: which artifact will help interpret the result
|
|
68
|
+
- `budget_cap`: max spend or runtime for the current stage
|
|
69
|
+
- `stop_condition`: when to stop rather than letting the run expand
|
|
70
|
+
- `interpretation`: what would count as signal, no signal, or ambiguity
|
|
71
|
+
- `next_branch_if_inconclusive`: the follow-up branch if the result is unclear
|
|
72
|
+
|
|
73
|
+
For exploratory work, `hypothesis` or `comparator` may be marked `exploratory` or `TBD`, but the learning goal still needs to be explicit.
|
|
74
|
+
|
|
75
|
+
## Phase 3: Run the adaptive design gate
|
|
76
|
+
|
|
77
|
+
Frame the gate as preventing waste, not enforcing bureaucracy.
|
|
78
|
+
|
|
79
|
+
Always check:
|
|
80
|
+
|
|
81
|
+
- the question or goal is explicit
|
|
82
|
+
- at least one metric or observable is defined
|
|
83
|
+
- a budget cap or stop condition exists
|
|
84
|
+
|
|
85
|
+
For hypothesis-driven work, also check:
|
|
86
|
+
|
|
87
|
+
- there is a falsifiable hypothesis
|
|
88
|
+
- there is a comparator or baseline
|
|
89
|
+
|
|
90
|
+
For exploratory work, instead check:
|
|
91
|
+
|
|
92
|
+
- the user can say what they are looking for
|
|
93
|
+
- the first learning step is scoped well enough to run
|
|
94
|
+
|
|
95
|
+
Additional checks when relevant:
|
|
96
|
+
|
|
97
|
+
- an artifact plan or `no_artifacts_reason` exists
|
|
98
|
+
- the run shape matches the question and is not changing too many important things without purpose
|
|
99
|
+
- an interpretation rule or next branch is defined
|
|
100
|
+
|
|
101
|
+
If the core gate passes and the user wants to proceed, let them run even if some non-core details are still `TBD`.
|
|
102
|
+
|
|
103
|
+
When blocked, ask only the next necessary question instead of reopening the whole brief.
|
|
104
|
+
|
|
105
|
+
## Phase 4: Confirm the plan
|
|
106
|
+
|
|
107
|
+
Before any compute request or training launch, restate:
|
|
108
|
+
|
|
109
|
+
- what we are trying to learn
|
|
110
|
+
- the experiment shape
|
|
111
|
+
- the metric or observable
|
|
112
|
+
- the artifact plan
|
|
113
|
+
- the budget or stop condition
|
|
114
|
+
- what result would change the next step
|
|
115
|
+
|
|
116
|
+
If the run is expensive or high-risk, ask for explicit confirmation.
|
|
117
|
+
|
|
118
|
+
## Phase 5: Drive Flywheel
|
|
119
|
+
|
|
120
|
+
Use Flywheel in layers when possible.
|
|
121
|
+
|
|
122
|
+
Before execution, load `references/flywheel-mcp-tool-map.md` and verify the
|
|
123
|
+
exact tool surface exposed by the current MCP host before critical flows.
|
|
124
|
+
|
|
125
|
+
### Design layer
|
|
126
|
+
|
|
127
|
+
Use an `insight` node to capture rationale, open questions, experiment shape, and any decomposition needed for exploratory or multi-stage work.
|
|
128
|
+
|
|
129
|
+
Typical flow:
|
|
130
|
+
|
|
131
|
+
1. `mcp__flywheel__flywheel_stage_node_create`
|
|
132
|
+
2. `mcp__flywheel__flywheel_stage_node_update`
|
|
133
|
+
3. `mcp__flywheel__flywheel_commit_node`
|
|
134
|
+
|
|
135
|
+
### Execution layer
|
|
136
|
+
|
|
137
|
+
Only after the design gate passes, create or branch the `empirical` node for the runnable part of the work.
|
|
138
|
+
|
|
139
|
+
Typical flow:
|
|
140
|
+
|
|
141
|
+
1. `mcp__flywheel__flywheel_branch_node` or `mcp__flywheel__flywheel_stage_node_create`
|
|
142
|
+
2. `mcp__flywheel__flywheel_stage_node_update` with the explicit run summary and the local question or hypothesis for that branch
|
|
143
|
+
3. `mcp__flywheel__flywheel_request_compute_grant_approval` only after the user accepts the design
|
|
144
|
+
4. `mcp__flywheel__flywheel_list_compute_grants` or `mcp__flywheel__flywheel_list_approval_sessions` when you need to resolve the approved `compute_grant_id`
|
|
145
|
+
5. `mcp__flywheel__flywheel_compute_acquire` and related compute tools only when execution is actually needed
|
|
146
|
+
6. `mcp__flywheel__flywheel_prepare_artifact_uploads`, raw upload to the returned signed URLs, then `mcp__flywheel__flywheel_finalize_artifact_uploads`
|
|
147
|
+
7. Do a brief epistemic check before commit: verify what the evidence actually shows, whether it matches the interpretation rule from the brief, and whether any gap between the data and the hoped-for story needs to be named explicitly in the node summary.
|
|
148
|
+
8. `mcp__flywheel__flywheel_commit_node`
|
|
149
|
+
|
|
150
|
+
Important notes:
|
|
151
|
+
|
|
152
|
+
- Exploratory work can stay in `insight` nodes until a specific empirical probe is ready.
|
|
153
|
+
- Because `empirical` commits require a non-empty `hypothesis`, turn each runnable exploratory probe into a concrete local question or hypothesis for that branch.
|
|
154
|
+
- For multi-stage or multi-arm work, use branches to represent stages or arms and keep summaries clear about how each branch feeds the next.
|
|
155
|
+
- Completed empirical work needs artifacts or a `no_artifacts_reason`.
|
|
156
|
+
|
|
157
|
+
## Adaptive question flow
|
|
158
|
+
|
|
159
|
+
Ask in short batches of 1-2 questions per turn. Keep the flow light, Socratic, and epistemic rather than exhaustive.
|
|
160
|
+
|
|
161
|
+
1. What are you trying to learn or decide?
|
|
162
|
+
2. Is this hypothesis-driven or exploratory?
|
|
163
|
+
3. Briefly separate what the user seems to know from what they seem to be assuming before locking the design.
|
|
164
|
+
4. If hypothesis-driven: what is the hypothesis and compared to what?
|
|
165
|
+
5. If exploratory: what is the first thing you need to learn and what is the cheapest way to learn it?
|
|
166
|
+
6. What experiment shape fits this work?
|
|
167
|
+
7. What metric or observable and artifact will you inspect?
|
|
168
|
+
8. What budget or stop condition keeps this from wasting compute?
|
|
169
|
+
9. What interpretation rule will distinguish evidence from expectation?
|
|
170
|
+
10. If the result is ambiguous, what is the next branch?
|
|
171
|
+
|
|
172
|
+
## Output template
|
|
173
|
+
|
|
174
|
+
Use this shape when turning a vague request into an executable plan:
|
|
175
|
+
|
|
176
|
+
```md
|
|
177
|
+
Experiment brief
|
|
178
|
+
|
|
179
|
+
- Question:
|
|
180
|
+
- Hypothesis:
|
|
181
|
+
- Comparator:
|
|
182
|
+
- Unit of work:
|
|
183
|
+
- Primary metric or observable:
|
|
184
|
+
- Artifact plan:
|
|
185
|
+
- Budget/time cap:
|
|
186
|
+
- Stop condition:
|
|
187
|
+
- Interpretation rule:
|
|
188
|
+
- Next branch if inconclusive:
|
|
189
|
+
|
|
190
|
+
Experiment type: hypothesis-driven | exploratory
|
|
191
|
+
Design gate: ready | blocked
|
|
192
|
+
Remaining gap:
|
|
193
|
+
Recommended next action:
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
## Generalization rule
|
|
197
|
+
|
|
198
|
+
Reuse the same protocol across domains by changing the unit of work and artifact type:
|
|
199
|
+
|
|
200
|
+
- model training -> metrics tables, loss curves, checkpoints
|
|
201
|
+
- benchmark comparisons -> score tables, latency plots, error slices
|
|
202
|
+
- prompt evaluations -> rubric tables, failure examples, sampled outputs
|
|
203
|
+
- product experiments -> funnels, event tables, user-segment slices
|
|
204
|
+
- scientific workflows -> figures, logs, result tables, notebooks
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# Flywheel MCP Tool Map
|
|
2
|
+
|
|
3
|
+
This reference describes the Flywheel MCP tool families and the common runtime
|
|
4
|
+
contract expected by the public Flywheel skill.
|
|
5
|
+
|
|
6
|
+
Use it as a routing guide, not as a session snapshot. Always verify the exact
|
|
7
|
+
tool surface exposed by your current MCP host before executing critical flows.
|
|
8
|
+
|
|
9
|
+
## Core Contract Expectations
|
|
10
|
+
|
|
11
|
+
- Node lifecycle and graph-mutation flows use optimistic locking
|
|
12
|
+
(`expected_revision`).
|
|
13
|
+
- Node commits require `kind`, `outcome`, and `summary`.
|
|
14
|
+
- `kind` is typically `insight` or `empirical`.
|
|
15
|
+
- `insight` commits require non-empty `insights`.
|
|
16
|
+
- `empirical` commits require a non-empty `hypothesis`; completed empirical
|
|
17
|
+
commits also require artifacts or a `no_artifacts_reason`.
|
|
18
|
+
- Artifact publish is a two-step flow:
|
|
19
|
+
prepare upload, upload raw bytes to returned signed URLs, then finalize
|
|
20
|
+
the upload batch.
|
|
21
|
+
|
|
22
|
+
## Tool Families
|
|
23
|
+
|
|
24
|
+
### Session, Auth, and Contract
|
|
25
|
+
|
|
26
|
+
- `mcp__flywheel__flywheel_auth_status`
|
|
27
|
+
- `mcp__flywheel__flywheel_get_contract`
|
|
28
|
+
- `mcp__flywheel__flywheel_get_contract_section`
|
|
29
|
+
- `mcp__flywheel__flywheel_get_credits_balance`
|
|
30
|
+
|
|
31
|
+
### Node Discovery and Read
|
|
32
|
+
|
|
33
|
+
- `mcp__flywheel__flywheel_list_nodes`
|
|
34
|
+
- `mcp__flywheel__flywheel_get_node`
|
|
35
|
+
- `mcp__flywheel__flywheel_get_node_tree`
|
|
36
|
+
- `mcp__flywheel__flywheel_get_node_ancestry`
|
|
37
|
+
- `mcp__flywheel__flywheel_summarize_node_tree`
|
|
38
|
+
- `mcp__flywheel__flywheel_get_campaign_snapshot`
|
|
39
|
+
- `mcp__flywheel__flywheel_list_audit`
|
|
40
|
+
|
|
41
|
+
### Node Mutation, Branching, and Commit
|
|
42
|
+
|
|
43
|
+
- `mcp__flywheel__flywheel_stage_node_create`
|
|
44
|
+
- `mcp__flywheel__flywheel_stage_node_update`
|
|
45
|
+
- `mcp__flywheel__flywheel_commit_node`
|
|
46
|
+
- `mcp__flywheel__flywheel_branch_node`
|
|
47
|
+
- `mcp__flywheel__flywheel_merge_nodes`
|
|
48
|
+
- `mcp__flywheel__flywheel_add_parent`
|
|
49
|
+
- `mcp__flywheel__flywheel_remove_parent`
|
|
50
|
+
- `mcp__flywheel__flywheel_delete_node`
|
|
51
|
+
- `mcp__flywheel__flywheel_bulk_delete_nodes`
|
|
52
|
+
|
|
53
|
+
### Access Policy and Collaboration
|
|
54
|
+
|
|
55
|
+
- `mcp__flywheel__flywheel_get_node_sharing`
|
|
56
|
+
- `mcp__flywheel__flywheel_set_sharing_for_node`
|
|
57
|
+
- `mcp__flywheel__flywheel_set_sharing_for_nodes`
|
|
58
|
+
|
|
59
|
+
### Tags and Graph Annotation
|
|
60
|
+
|
|
61
|
+
- `mcp__flywheel__flywheel_create_node_tag`
|
|
62
|
+
- `mcp__flywheel__flywheel_update_node_tag`
|
|
63
|
+
- `mcp__flywheel__flywheel_delete_node_tag`
|
|
64
|
+
- `mcp__flywheel__flywheel_set_node_tag_assignments`
|
|
65
|
+
|
|
66
|
+
### Artifacts
|
|
67
|
+
|
|
68
|
+
- `mcp__flywheel__flywheel_list_artifacts`
|
|
69
|
+
- `mcp__flywheel__flywheel_get_artifact`
|
|
70
|
+
- `mcp__flywheel__flywheel_get_artifact_preview`
|
|
71
|
+
- `mcp__flywheel__flywheel_prepare_artifact_uploads`
|
|
72
|
+
- `mcp__flywheel__flywheel_finalize_artifact_uploads`
|
|
73
|
+
- `mcp__flywheel__flywheel_set_artifact_note`
|
|
74
|
+
- `mcp__flywheel__flywheel_delete_artifact`
|
|
75
|
+
|
|
76
|
+
Common artifact types include:
|
|
77
|
+
`text`, `table`, `json`, `image`, `banner`, `html`, `plotly_html`, `vega`,
|
|
78
|
+
`checkpoint`, and `diff_carousel`.
|
|
79
|
+
|
|
80
|
+
### Export and Import
|
|
81
|
+
|
|
82
|
+
- `mcp__flywheel__flywheel_export_subgraph`
|
|
83
|
+
- `mcp__flywheel__flywheel_import_subgraph`
|
|
84
|
+
- `mcp__flywheel__flywheel_export_summary`
|
|
85
|
+
- `mcp__flywheel__flywheel_export_summary_stream`
|
|
86
|
+
- `mcp__flywheel__flywheel_export_summary_pdf`
|
|
87
|
+
- `mcp__flywheel__flywheel_export_summary_render_pdf`
|
|
88
|
+
|
|
89
|
+
### Executions
|
|
90
|
+
|
|
91
|
+
- `mcp__flywheel__flywheel_launch_execution`
|
|
92
|
+
- `mcp__flywheel__flywheel_list_executions`
|
|
93
|
+
- `mcp__flywheel__flywheel_terminate_execution`
|
|
94
|
+
|
|
95
|
+
### Managed Compute
|
|
96
|
+
|
|
97
|
+
- `mcp__flywheel__flywheel_approval_session_heartbeat`
|
|
98
|
+
- `mcp__flywheel__flywheel_list_approval_sessions`
|
|
99
|
+
- `mcp__flywheel__flywheel_expire_approval_session`
|
|
100
|
+
- `mcp__flywheel__flywheel_request_compute_grant_approval`
|
|
101
|
+
- `mcp__flywheel__flywheel_list_compute_grants`
|
|
102
|
+
- `mcp__flywheel__flywheel_compute_list_options`
|
|
103
|
+
- `mcp__flywheel__flywheel_compute_acquire`
|
|
104
|
+
- `mcp__flywheel__flywheel_compute_status`
|
|
105
|
+
- `mcp__flywheel__flywheel_compute_connection`
|
|
106
|
+
- `mcp__flywheel__flywheel_compute_release`
|
|
107
|
+
- `mcp__flywheel__flywheel_compute_release_all`
|
|
108
|
+
|
|
109
|
+
### Campaign Budgets (Organizer Flows)
|
|
110
|
+
|
|
111
|
+
- `mcp__flywheel__flywheel_list_campaign_budgets`
|
|
112
|
+
- `mcp__flywheel__flywheel_create_campaign_budget`
|
|
113
|
+
- `mcp__flywheel__flywheel_update_campaign_budget`
|
|
114
|
+
- `mcp__flywheel__flywheel_revoke_campaign_budget`
|
|
115
|
+
|
|
116
|
+
### Migration Helpers
|
|
117
|
+
|
|
118
|
+
Some installations may expose migration-only helper tools with hashed names.
|
|
119
|
+
Treat these as specialized one-off tools, not part of day-to-day research
|
|
120
|
+
workflows.
|
|
121
|
+
|
|
122
|
+
## Practical Tool Sequences
|
|
123
|
+
|
|
124
|
+
### Insight Node Flow
|
|
125
|
+
|
|
126
|
+
1. `mcp__flywheel__flywheel_stage_node_create`
|
|
127
|
+
2. `mcp__flywheel__flywheel_stage_node_update`
|
|
128
|
+
3. `mcp__flywheel__flywheel_commit_node`
|
|
129
|
+
|
|
130
|
+
### Empirical Node With Artifacts
|
|
131
|
+
|
|
132
|
+
1. `mcp__flywheel__flywheel_stage_node_create`
|
|
133
|
+
2. `mcp__flywheel__flywheel_stage_node_update`
|
|
134
|
+
3. Run experiment or compute steps
|
|
135
|
+
4. `mcp__flywheel__flywheel_prepare_artifact_uploads`
|
|
136
|
+
5. Upload artifact bytes to signed URLs
|
|
137
|
+
6. `mcp__flywheel__flywheel_finalize_artifact_uploads`
|
|
138
|
+
7. `mcp__flywheel__flywheel_commit_node`
|
|
139
|
+
|
|
140
|
+
### Managed Compute Flow
|
|
141
|
+
|
|
142
|
+
1. `mcp__flywheel__flywheel_approval_session_heartbeat`
|
|
143
|
+
2. `mcp__flywheel__flywheel_request_compute_grant_approval`
|
|
144
|
+
3. `mcp__flywheel__flywheel_list_compute_grants` (if needed)
|
|
145
|
+
4. `mcp__flywheel__flywheel_compute_acquire`
|
|
146
|
+
5. `mcp__flywheel__flywheel_compute_status` until ready
|
|
147
|
+
6. `mcp__flywheel__flywheel_compute_connection`
|
|
148
|
+
7. `mcp__flywheel__flywheel_compute_release` (or `..._release_all`)
|
|
149
|
+
|
|
150
|
+
### Share a Graph With Collaborators
|
|
151
|
+
|
|
152
|
+
1. `mcp__flywheel__flywheel_get_node_sharing`
|
|
153
|
+
2. `mcp__flywheel__flywheel_set_sharing_for_node` (or `..._for_nodes`)
|
|
154
|
+
3. `mcp__flywheel__flywheel_export_summary` or `..._export_subgraph` for handoff
|
|
155
|
+
|
|
156
|
+
## Safety Notes
|
|
157
|
+
|
|
158
|
+
- Prefer `get_contract` before implementing strict assumptions in automation.
|
|
159
|
+
- Avoid call-order assumptions not mandated by contract.
|
|
160
|
+
- Keep checks bounded: list/read first, then mutate only the intended nodes.
|
|
@@ -24,5 +24,5 @@ Use this map when a user asks where to find a feature in the web interface.
|
|
|
24
24
|
## Pairing With Other Docs
|
|
25
25
|
|
|
26
26
|
- Start with `web-ui/the-flywheel-web-ui.md` for overview behavior.
|
|
27
|
-
- Use `
|
|
27
|
+
- Use `references/flywheel-mcp-tool-map.md` when the answer depends on exact MCP
|
|
28
28
|
tool capabilities or runtime contract.
|