openpond 0.0.34 → 0.0.35

package/dist/web/courses/post-training/scripts/script_02.md

@@ -0,0 +1,49 @@
+# Definitions
+
+Post-training from first principles · Lesson 2 of 10 · 6:14
+
+## Learning objective
+
+Decode the notation, objectives, estimators, and acronyms used throughout modern reinforcement fine-tuning.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+OpenPond lesson intro, annotated policy notation, separate logits and softmax explainers, a rollout tuple, concrete reward examples, reward-to-advantage relationships, PPO and GRPO definition cards, baseline estimators, a gradient step, probability ratios and clipping, distribution metrics, and full acronym maps.
+
+## Narration transcript
+
+The expression pi theta of a given state is a compact description of model behavior. Pi names the policy: a probability distribution, not one completed answer. Theta means all adjustable model parameters. The state is the context available now, and an action can be one token, one tool call, or another sampled decision.
+
+Before sampling, the network produces raw scores called logits. A logit is not a percentage and logits do not need to add to one. They can be positive or negative. Only their relative differences matter: a larger logit means the model prefers that action compared with the alternatives available at that position.
+
+Softmax converts logits into probabilities. It exponentiates each logit, then divides by the sum of all exponentiated logits. This makes every result positive and makes the distribution sum to one. In the example, logits three, one, and point two become probabilities of roughly eighty-four, eleven, and five percent. Temperature divides the logits before softmax. Lower temperature sharpens the distribution; higher temperature flattens it.
+
+A rollout, also called a trajectory and written tau, is the interaction record used for learning. The behavior policy mu generated each sampled action. Its log-probability is stored beside the action so training can later compare the old behavior distribution with the current policy. Environment observations affect later states, but only the model's sampled actions receive policy-gradient terms.
+
+Reward is the scalar number emitted by an evaluator. A math checker might return one for an exact answer and zero otherwise. A code environment might return one only when every hidden test passes. A tool task might inspect whether the final database state matches a target. The rule can be binary or graded, but the optimizer sees the number, not the evaluator's full explanation. That richer explanation is feedback.
+
+Reward, return, and advantage are related but not interchangeable. Reward is the scalar emitted at a step or at the end. Return, G at time t, combines future rewards and can discount distant outcomes with gamma. Advantage subtracts a baseline from that return. Positive advantage means the action did better than expected; negative advantage means it did worse. PPO learns a value baseline, while GRPO estimates one from sibling responses.
+
+PPO means Proximal Policy Optimization. It is a policy-gradient method that trains a second network, called a critic or value function, to estimate expected return. The observed return minus that expectation becomes advantage. PPO also clips the probability-ratio incentive so one sampled action cannot drive an arbitrarily large update.
+
+GRPO means Group Relative Policy Optimization. It removes the learned critic and samples several sibling responses for the same prompt. Each response is compared with the group's reward mean and standard deviation. Better-than-group responses receive positive advantage; worse responses receive negative advantage. GRPO keeps policy ratios and clipping.
+
+Both methods need a baseline because raw reward lacks context. PPO's baseline comes from the critic, and Generalized Advantage Estimation combines value predictions across time. GRPO's baseline comes from sibling rewards. If every sibling receives the same reward, the group standard deviation is zero and there is no relative advantage to learn from.
+
+An objective, often written L of theta, turns the training goal into one number. The gradient is the local slope of that objective with respect to every adjustable parameter. Backpropagation computes those slopes. The optimizer applies a learning rate alpha to make a small update from theta to theta prime. In policy-gradient training, the RL method determines the credit weights inside the loss; ordinary differentiation performs the parameter update.
+
+PPO and GRPO use a probability ratio. The numerator is the current policy's probability for the stored action. The denominator is the behavior probability recorded during rollout. A ratio above one means that action became more likely. Clipping with epsilon caps the incentive once the ratio moves too far in one update. It limits this sampled term, but does not prove that the entire model stayed close.
+
+Entropy measures how spread out one policy distribution is. KL divergence compares two distributions and is commonly used to measure movement from a reference model or mismatch between teacher and student. Its direction matters. Cross-entropy uses target probabilities q to weight the log of student probabilities p. Distillation usually minimizes this quantity so the student assigns probability where the teacher does.
+
+RL means reinforcement learning. RFT means reinforcement fine-tuning: applying an RL-style objective to an already pretrained model. RLVR adds verifiable rewards, such as executable tests. PPO and GRPO are policy update rules. PPO is Proximal Policy Optimization. GRPO is Group Relative Policy Optimization. GAE is Generalized Advantage Estimation. Pass-at-k asks whether at least one of k sampled attempts succeeds. On-policy and off-policy describe who generated the data.
+
+The teacher-guided methods use dense distributions instead of only scalar outcomes. OPSD is On-Policy Self-Distillation and conditions the teacher on a trusted solution. SDFT is Self-Distillation Fine-Tuning and supplies a demonstration. SDPO is Self-Distillation Policy Optimization and supplies failure feedback. In each case the student creates the prefix and the frozen teacher receives extra evidence. SRPO, Self-Rewarding Policy Optimization, routes successful and unsuccessful samples through different signals.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_03.md

@@ -0,0 +1,35 @@
+# On-policy and off-policy data
+
+Post-training from first principles · Lesson 3 of 10 · 1:06
+
+## Learning objective
+
+Why learner rollouts and teacher or stored data support different updates.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+on- and off-policy sources, a concrete rollout record, stored-data schemas, and objective routing.
+
+## Narration transcript
+
+On-policy data comes from the learner being updated. Policy version twelve generates an attempt that will update version twelve.
+
+Off-policy data comes from a teacher, human, or older checkpoint. These labels describe source, not quality.
+
+An RL rollout stores the problem, sampled actions, generating checkpoint, behavior probabilities, environment observations, and reward.
+
+Text alone cannot connect evaluated behavior to an update. The missing record matters as much as the final response.
+
+Other sources carry different signals: teacher targets, chosen and rejected responses, or old actions with original probabilities and rewards.
+
+PPO and GRPO usually score learner attempts. Offline distillation uses teacher examples. On-policy distillation labels the student's current prefix.
+
+Inside any RL rollout, the final patch is only one part of a sequence of actions and observations.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_04.md

@@ -0,0 +1,43 @@
+# Rewards and credit assignment
+
+Post-training from first principles · Lesson 4 of 10 · 3:00
+
+## Learning objective
+
+Follow a code-repair trajectory from actions and observations to advantage.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+trajectories, rollout tuples, feedback versus reward, credit assignment, discounted return, advantage, entropy, pass@k, reference KL.
+
+## Narration transcript
+
+An RL rollout is a sequence, not only a final response. Credit assignment asks which decisions inside that sequence should become more or less likely.
+
+The issue and repository form state zero. The behavior policy samples “inspect files.” File contents are an environment observation, not another sampled action. The next state contains those contents. The policy then writes raise Cancelled Error, and test output closes the trajectory.
+
+The tuple makes those roles explicit. Tau contains states, sampled actions, behavior log-probabilities, observations, and terminal reward. Gradients apply to action positions controlled by the policy. Observations condition later actions but are masked out of the policy-gradient loss.
+
+Reward and feedback carry different information. A patch can receive reward zero while the environment still knows that eight tests passed, cancellation failed, and the expected exception was Cancelled Error. Reward is the scalar consumed by an objective. Feedback is the richer evidence the environment produced.
+
+A terminal score also leaves the cause ambiguous. Was the wrong file inspected? Was the timeout branch incorrect? Was “None” the decisive token? Response-level credit applies one outcome broadly. Step- or token-level signals can be denser, but density alone does not prove causal accuracy.
+
+For multi-step problems, return carries future rewards backward. G at time t is the discounted sum of later rewards. With gamma point nine and success two steps later, the earlier action receives return point eight one. PPO critics and Generalized Advantage Estimation, or GAE, build on this idea.
+
+Advantage then asks: better than what was expected? If observed return is point eight and the baseline is point five, advantage is positive point three. A return of point two gives negative point three.
+
+Positive advantage raises the sampled behavior’s probability; negative advantage lowers it. PPO learns a critic for the baseline. GRPO replaces that critic with statistics from sibling responses.
+
+Learning also changes exploration. Pass-at-one can rise while entropy and pass-at-k fall if the model collapses onto one strategy. Reliability and breadth therefore need separate metrics.
+
+Finally, a frozen reference policy anchors behavior outside the optimized task. Kullback–Leibler divergence, shortened to KL, measures distribution shift. Too little constraint permits drift and reward exploits; too much prevents useful learning.
+
+A test-based reward is useful only when the test measures the intended outcome reliably.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_05.md

@@ -0,0 +1,43 @@
+# Verifiable rewards
+
+Post-training from first principles · Lesson 5 of 10 · 2:53
+
+## Learning objective
+
+See how tests create scalable rewards—and how a model can exploit the checker.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+the cancellation test as verifier, RLVR definition, verifier equation, executable loop, secondary applications, verifier errors, reward hacking, and hidden evaluation.
+
+## Narration transcript
+
+Every reward-based update depends on whether its evaluator measures the right thing. This is the verifier problem.
+
+RLVR means Reinforcement Learning with Verifiable Rewards. For our repair, the verifier applies the patch, runs the cancellation tests, and returns a reproducible outcome. The same idea applies to checked math answers, tool states, and agent environments.
+
+A preference such as “patch A is cleaner” depends on a judge. A verifiable reward applies an explicit rule. Verifiable does not mean infallible; it means the same patch should receive the same result under the same evaluator state.
+
+Write the rule as r equals V of x, y, and e. X is the public task. Y is the sampled attempt. E is evaluator-only state such as hidden tests. For a binary verifier, reward is zero or one.
+
+This equation separates the setting from the optimizer. RLVR supplies reward. PPO, GRPO, or another policy optimizer supplies the update. DeepSeek-R1 is an important example of reasoning training built from this combination.
+
+The full loop is prompt, policy attempt, execution, verification, reward, and policy update. Automatic checks scale because a person does not need to score every trajectory. The verifier also becomes both the task specification and an attack surface.
+
+RLVR applies beyond short math answers. Code can compile and run tests. Tool agents can be checked against a target database state. Search can verify retrieved evidence. Scientific tasks can use simulators and constraints. It is a poor fit when quality is inherently subjective, outcomes cannot be reproduced, or the checker is easy to game.
+
+Verifier errors have two directions. A false negative rejects a valid solution. A false positive is more dangerous under optimization because a wrong solution receives positive gradient and can become an exploit.
+
+Suppose visible tests cover only three examples. Hard-coding those outputs and implementing the general rule both receive reward one. The optimizer cannot distinguish them. Hidden tests, randomized cases, invariants, and shadow verifiers make the measured objective closer to the intended task.
+
+That requires an information boundary. The policy sees the public task and allowed observations. The evaluator alone sees hidden checks and anti-cheat state. Training-only teacher evidence belongs in a third protected channel.
+
+So “RLVR with GRPO” is precise: the first term names how outcomes are labeled, and the second names how those labels become an update.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_06.md

@@ -0,0 +1,41 @@
+# PPO and GRPO
+
+Post-training from first principles · Lesson 6 of 10 · 2:54
+
+## Learning objective
+
+Compare PPO's learned critic with GRPO's sibling-response baseline.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+PPO's critic on the cancellation trajectory, GRPO sibling patch attempts, worked group advantages, clipping, mixed-group probability, zero-variance groups, and direct comparison.
+
+## Narration transcript
+
+A trustworthy reward still has to become a learning signal. PPO and GRPO differ mainly in how they decide whether an outcome was better or worse than expected.
+
+PPO means Proximal Policy Optimization. For the successful inspect, patch, and test trajectory, observed return is point eight. A learned critic expected point five. Their difference, positive point three, is the advantage. PPO combines that advantage with a clipped probability ratio and updates the policy.
+
+GRPO means Group Relative Policy Optimization. It keeps rollouts, rewards, probability ratios, and clipping, but removes the critic. Instead, it samples several patches for the same cancellation bug and compares their verifier scores.
+
+Our six sibling patches produce rewards one, zero, zero, one, one, zero. The mean is one half and the standard deviation is one half. Normalizing gives every passing patch advantage positive one and every failing patch negative one.
+
+That comparison is useful but coarse. Raise Timeout Error and return None both failed, so binary GRPO treats them alike even if one was closer. The group says which siblings did better; it does not explain why.
+
+The clipping graph controls how much one sampled action can influence a single update. If its probability moves from point one zero to point one three, the ratio is one point three. With a cap at one point two, additional positive credit stops growing. Clipping limits a local incentive; it does not guarantee that the whole policy stayed close.
+
+A group teaches only when it contains contrast. If one patch succeeds with probability p, a group of size G is mixed with probability one minus p to the G minus one minus p, all to the G. That subtracts all-success and all-failure groups.
+
+Near zero success, almost every group fails. Near one, almost every group passes. Either case produces little relative signal. Larger groups widen the useful middle but require more patch generation and test execution.
+
+This is why prompts need to sit near the learning frontier: difficult enough to fail, possible enough to solve. If every cancellation patch receives the same reward, normalized group advantage is zero.
+
+Both PPO and GRPO need trajectories, rewards, behavior log-probabilities, clipped ratios, and gradients. PPO learns expected value with a critic. GRPO estimates a baseline from sibling patches. The cancellation environment stays the same; only the baseline estimator changes.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_07.md

@@ -0,0 +1,37 @@
+# Distillation
+
+Post-training from first principles · Lesson 7 of 10 · 2:42
+
+## Learning objective
+
+Transfer a teacher's token distribution instead of copying one final answer.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+cancellation-token hard and soft targets, token cross-entropy, student update, teacher temperature, KL direction, prefix provenance, and privileged teacher context.
+
+## Narration transcript
+
+Outcome rewards compress an attempt to a scalar. Distillation supplies richer guidance by specifying a probability distribution over plausible next tokens.
+
+A one-hot target says “Cancelled Error” and assigns every alternative zero. A teacher distribution can say that “Cancelled Error” is most likely, “Timeout Error” is also plausible, “return” is weak, and “retry” is unlikely. That structure contains more information than the one sampled token.
+
+Teacher probability q weights the log of student probability p over the vocabulary. Minimizing that cross-entropy moves the student toward the teacher. In our example, the student initially favors “return,” while the teacher favors “Cancelled Error.” After an update, the two distributions move closer.
+
+Teacher temperature controls how much of that structure is visible. Low temperature makes the target almost one-hot. Higher temperature reveals alternatives in the tail; too much can magnify noise. Teacher-target temperature is separate from the rollout temperature used to sample behavior.
+
+KL direction changes the lesson too. Forward KL penalizes the student for missing teacher-supported alternatives. Reverse KL strongly penalizes student probability where the teacher assigns little mass, often favoring a narrower mode. “We used KL” is incomplete without direction and temperature.
+
+Prefix provenance determines which states receive teacher targets. Offline distillation uses fixed teacher trajectories. On-policy distillation lets the student reach its own strange cancellation state, then asks the teacher what should come next there.
+
+The teacher does not need larger weights. The same frozen model can see the student's failed prefix plus extra training-only evidence: a verified repair, an expert demonstration, or a test explanation. That evidence changes the teacher distribution.
+
+Training transfers the useful part of that privileged view into student weights. Deployment removes the evidence. A verified solution, demonstration, or failure explanation produces a different teacher target.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_08.md

@@ -0,0 +1,37 @@
+# OPSD, SDFT, and SDPO
+
+Post-training from first principles · Lesson 8 of 10 · 2:43
+
+## Learning objective
+
+Compare trusted solutions, demonstrations, and failure feedback at one prefix.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+one failed cancellation patch with three teacher evidence channels, OPSD/SDFT/SDPO worked examples, evidence comparison, and shared failure modes.
+
+## Narration transcript
+
+OPSD, SDFT, and SDPO share one mechanism. A frozen teacher scores the student's prefix while receiving additional evidence. The evidence source defines the method.
+
+The student distribution is p theta given the issue and its prefix. Teacher target q scores the same next token at the same prefix, but also conditions on evidence e. Training matches q into p; deployment removes e.
+
+On-Policy Self-Distillation, or OPSD, gives the teacher a trusted solution. Here that is the verified repair: preserve cleanup and raise Cancelled Error. The teacher uses it to guide the student's own prefix without placing the privileged patch in the deployed student's prompt. The OPSD paper evaluated this mechanism on reasoning tasks; the animation maps the information flow onto our code case.
+
+Self-Distillation Fine-Tuning, or SDFT, gives the teacher an expert demonstration. A related repair shows the sequence inspect the flag, run cleanup, then raise the exception. Unlike offline imitation, SDFT scores the current student's return-None prefix, so the demonstration guides the state the student actually reached.
+
+Self-Distillation Policy Optimization, or SDPO, gives the teacher feedback about the current failure. The scalar reward is zero, but the test trace says test-cancel failed, expected Cancelled Error, and return skipped cancellation. The teacher converts that explanation into dense token probabilities along the failed prefix.
+
+The choice follows the evidence. A trusted solution supports OPSD. A demonstration supports SDFT. An explanatory failure trace supports SDPO. If all you have is a binary outcome, GRPO may be the honest baseline; inventing a generic “feedback” field would hide these different trust boundaries.
+
+None of the dense methods is automatically safe. A privileged patch can leak. A demonstration can be irrelevant. A teacher can misread feedback and densify an error. New capability can overwrite old behavior, entropy can collapse, and teacher forward passes can erase claimed efficiency savings.
+
+So the durable comparison is not acronym against acronym. Hold the student prefix fixed, vary the evidence, record the cost, and measure both the cancellation repair and unrelated retained capability.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_09.md

@@ -0,0 +1,45 @@
+# Credible experiments
+
+Post-training from first principles · Lesson 9 of 10 · 3:32
+
+## Learning objective
+
+Build versioned datasets, fair baselines, and claims that survive scrutiny.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+Taskset boundaries, Hugging Face import, baselines, evaluation, compute, experimental campaigns, claim table, synthesis.
+
+## Narration transcript
+
+A research result requires many versioned tasks, protected information boundaries, matched baselines, and held-out evaluation.
+
+Policy-visible input includes the issue, repository revision, allowed tools, and public tests. Training-only evidence may contain an expert repair, privileged patch, or public diagnostic. Evaluator-only hidden tests, anti-exploit checks, and held-out repository clusters must remain outside both.
+
+The Taskset preserves those boundaries together with source revision, license, split, verifier version, and content hash. Related issues from the same repository should be clustered before splitting; otherwise memorization can look like generalization.
+
+Hugging Face support is worthwhile when it behaves as a reproducible importer. Pin the repository revision. Inspect the dataset card and license. Choose configuration and split explicitly. Map columns through a reviewed transform, log rejected rows, preserve original identifiers, and materialize an immutable snapshot. A training run should never silently follow a changing main branch.
+
+Every proposed method needs a simple credible baseline. Begin with the base model and the most direct token-imitation objective supported by the data. Add offline distillation or preference learning when those signals are relevant. Compare with GRPO or another RLVR baseline before claiming value from OPSD, SDFT, or SDPO.
+
+Equal optimizer tokens and equal total compute answer different questions. A distillation method may use fewer rollouts while spending additional teacher forward passes. Report both controls.
+
+Evaluation should span four dimensions. Capability measures task success. Diversity measures pass-at-k, entropy, and distinct strategies. Retention tests old domains and distribution drift. Integrity audits reward exploits, leakage, and disagreement with a shadow verifier. A single average can hide serious regressions.
+
+Compute accounting includes student rollout tokens, teacher-scored tokens, backward tokens, verifier time, external calls, failed jobs, memory, storage, and wall clock.
+
+The primary study compares GRPO, SDPO, and a routed success-and-failure method on code repair while separating public diagnostics from hidden tests. Verified math can replicate the GRPO and OPSD claims with exact graders. A tool protocol can replicate token imitation and SDFT while measuring retention. The extra domains test whether a conclusion transfers; they do not replace the main story halfway through it.
+
+The paper should bind every claim to a dataset revision, metric, run set, seed, confidence interval, and ablation. That structure keeps a local result from turning into a universal claim.
+
+The final map is simple. Demonstrations produce imitation targets. Preferences produce relative targets. Verifiable outcomes produce rewards. Privileged solutions and failure explanations produce context-conditioned teacher distributions.
+
+Evidence determines the target. The target determines the update. The update changes future behavior. Evaluation measures what improved—and what changed elsewhere.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/courses/post-training/scripts/script_10.md

@@ -0,0 +1,47 @@
+# Technical appendix
+
+Post-training from first principles · Lesson 10 of 10 · 3:12
+
+## Learning objective
+
+Inspect implementation choices and paper-specific results after the core mechanisms are clear.
+
+## Using this with an LLM
+
+Use this script as lesson source material. Preserve its distinctions and caveats when summarizing, making flash cards, proposing experiments, or answering questions. The narration is the source of truth; ask for missing experimental details instead of inventing them.
+
+## Visual context
+
+Length-normalization equations and pass-at-k curves; full versus top-k teacher-logit storage; paper-reported OPSD, SDFT, and SDPO bars; and an SRPO success/failure routing diagram.
+
+## Narration transcript
+
+### GRPO details
+
+The first appendix isolates two GRPO evaluation choices that matter after the main mechanism is understood.
+
+Length normalization changes gradient weight. A fourteen-hundred-token correct answer and a one-hundred-eighty-token correct answer may receive the same reward but contribute very different numbers of token terms. Sequence averaging, token averaging, overlong penalties, and group normalization therefore change the effective objective.
+
+Pass-at-one measures reliability from one sample. Pass-at-k asks whether any of k samples succeeds and exposes remaining search diversity. A model can improve pass-at-one while collapsing onto one strategy, so report both metrics with the sampling configuration.
+
+### Distillation systems
+
+The systems problem is storage. A large vocabulary creates one teacher logit for every token at every trajectory position.
+
+Top-k compression stores the most likely teacher logits and approximates the remaining tail. This reduces memory and bandwidth but changes the target distribution. Measure divergence from full logits on a validation sample, and count teacher-forward cost when comparing efficiency.
+
+### Paper details and SRPO
+
+The reported graphs belong in an appendix because they come from different models, datasets, and experimental budgets.
+
+The OPSD study reports its largest displayed aggregate gain at the smallest Qwen3 model. At one point seven billion parameters, the displayed base score is thirty-seven point one, GRPO is thirty-seven point seven, and OPSD is forty-three point four. The gaps are smaller at four and eight billion parameters. That is a hypothesis for replication, not a universal model-size law.
+
+The SDFT study emphasizes knowledge acquisition and retention. Its displayed strict aggregate moves from eighty to eighty-nine, while its out-of-distribution aggregate moves from eighty to ninety-eight. The SDPO study reports forty-one point two for GRPO and forty-eight point eight for SDPO on its code benchmark. Their bars answer different questions and should not be read as a controlled head-to-head comparison.
+
+SRPO means Sample-Routed Policy Optimization. It sends verified successes toward a GRPO-style outcome update and failures with explanations toward an SDPO-style corrective target.
+
+The useful principle is signal routing. A successful rollout already shows what worked and can support sparse selection. A failed rollout with diagnostics may reveal how to recover and can support dense correction. The router preserves that asymmetry instead of forcing every sample through one uniform loss.
+
+## Provenance
+
+This is the production narration for the OpenPond learning series, generated from the canonical course script. Equations, diagrams, and cited paper results remain in the accompanying video and research document.

package/dist/web/index.html

@@ -7,8 +7,8 @@
     <link rel="icon" type="image/png" sizes="16x16" href="./favicon-16x16.png" />
     <link rel="apple-touch-icon" href="./openpond-icon.png" />
     <title>OpenPond App</title>
-    <script type="module" crossorigin src="./assets/index-CKmte5ok.js"></script>
-    <link rel="stylesheet" crossorigin href="./assets/index-8H-GDlM5.css">
+    <script type="module" crossorigin src="./assets/index-B5AN29EO.js"></script>
+    <link rel="stylesheet" crossorigin href="./assets/index-COGlKIJ4.css">
   </head>
   <body>
     <div id="root"></div>

package/dist/web/tutorials/how-to-make-an-agent-create.vtt

@@ -0,0 +1,49 @@
+WEBVTT
+
+C01
+00:00:03.670 --> 00:00:09.855
+Open Lab, choose New agent, and start the guided creation flow.
+
+C02
+00:00:09.575 --> 00:00:17.817
+Choose From prompt to describe the Agent directly, or From chats to include examples of the work it should handle.
+
+C04
+00:00:17.537 --> 00:00:27.159
+This synthetic example starts with three chats: Acme has renewal blockers, Northstar is expanding, and the team needs a weekly account review. Add them with the Agent purpose.
+
+C06
+00:00:26.879 --> 00:00:36.819
+Review the selected excerpts as examples. OpenPond turns their recurring goals, decisions, corrections, and answer patterns into the first Agent plan.
+
+C07
+00:00:36.539 --> 00:00:46.658
+The draft converts those examples into the Agent's purpose, chat behavior, runnable actions, output formats, and Evals for your review.
+
+C08
+00:00:46.378 --> 00:00:52.652
+The right side panel asks how the Agent should prioritize conflicting account risks.
+
+C09
+00:00:52.372 --> 00:01:00.753
+Before creation, review the proposed name, chat endpoint, runnable actions, Evals, and output formats.
+
+C09A
+00:01:00.473 --> 00:01:09.071
+If the plan misses a requirement, open Edit plan and describe the change in plain language before any Agent source is created.
+
+C09B
+00:01:08.791 --> 00:01:16.049
+Review the revised plan, then click Confirm plan to create the Agent and run its Evals.
+
+C10
+00:01:15.769 --> 00:01:24.189
+This is the creation result: OpenPond built the Agent and passed its behavior Evals against the Account Health examples.
+
+C11
+00:01:23.909 --> 00:01:31.346
+After the Evals pass, the Agent is saved to your Profile and is ready to use.
+
+C12
+00:01:31.066 --> 00:01:37.658
+The Agent page shows its purpose, chat entrypoint, and runnable account actions.

package/dist/web/tutorials/how-to-make-an-agent-improve.vtt

@@ -0,0 +1,45 @@
+WEBVTT
+
+I00
+00:00:03.670 --> 00:00:14.365
+The current Agent finds the right facts, but buries the priority after the account summary. Improve will make the billing and P1 decision lead every high-risk result.
+
+I01
+00:00:14.085 --> 00:00:23.707
+Open Improve agent on the existing Agent, then choose whether to describe the correction directly or include a chat that demonstrates it.
+
+I02
+00:00:23.427 --> 00:00:33.853
+Choose From chats, ask every high-risk response to lead with the billing and P1 decision, and attach the correction conversation that demonstrates it.
+
+I03
+00:00:33.573 --> 00:00:43.156
+Review the new lead-with-priority behavior, the existing capabilities that must stay intact, and the Evals OpenPond will rerun.
+
+I04
+00:00:42.876 --> 00:00:49.468
+Review the planned change and confirm that it preserves citations and every existing action.
+
+I05
+00:00:49.188 --> 00:00:58.900
+The same locked Evals make the decision clear: the current Agent misses the new lead sentence, while the updated Agent passes every original and new Eval.
+
+I06
+00:00:58.620 --> 00:01:07.666
+Apply the update only after that comparison. The existing Agent updates in place, with the successful Evals recorded.
+
+I07
+00:01:07.386 --> 00:01:16.879
+The improved chat now places disputed billing and the open P1 case ahead of adoption decline while preserving source citations.
+
+I08
+00:01:16.599 --> 00:01:25.069
+The direct triage action repeats the billing-first result, proving the change works outside chat without breaking the existing workflow.
+
+I09
+00:01:24.789 --> 00:01:32.940
+Commit the final Agent source with a clear Git message so the exact Profile version remains reviewable and shareable.
+
+I10
+00:01:32.660 --> 00:01:45.898
+Open Sync to review the committed Profile changes. Confirm sync publishes this version so teammates can use the Agent in chat and hosted sandboxes can attach the same source.

package/dist/web/tutorials/how-to-make-an-agent-use.vtt

@@ -0,0 +1,33 @@
+WEBVTT
+
+C13Q
+00:00:03.670 --> 00:00:12.050
+Use opens a normal chat with the Agent attached. Type a natural-language question about Acme before sending it.
+
+C13
+00:00:11.770 --> 00:00:21.532
+Acme is the at-risk example: the Agent answers in chat with renewal timing, usage decline, the disputed invoice, and the open P1 case.
+
+C14
+00:00:21.252 --> 00:00:29.850
+A follow-up keeps Acme context and assigns the first actions to Revenue Operations and Support instead of restarting the conversation.
+
+C15
+00:00:29.570 --> 00:00:38.208
+The action picker offers chat plus each runnable account action, so you can choose conversation or a focused task.
+
+C16
+00:00:37.928 --> 00:00:48.493
+Northstar is the healthy expansion example from the source chats. Its direct summary identifies rising usage and the request for twenty-five more seats.
+
+C17
+00:00:48.213 --> 00:00:56.772
+Running renewal triage for Acme returns high risk, cites the billing and support blockers, and orders the next steps.
+
+C18
+00:00:56.492 --> 00:01:05.309
+The weekly review action combines the portfolio and returns visible Markdown, CSV, and JSON artifacts for downstream use.
+
+C19
+00:01:05.029 --> 00:01:12.337
+After restarting OpenPond, the same Agent, chat history, and actions remain available.