lunarlabs-core 0.1.0__tar.gz

lunarlabs_core-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Lunar Labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lunarlabs_core-0.1.0/PKG-INFO

@@ -0,0 +1,151 @@
+Metadata-Version: 2.4
+Name: lunarlabs-core
+Version: 0.1.0
+Summary: The product-neutral Lunar Labs spine: capture, deterministic verification, fail-open escalation, training-pair extraction, and the fitness gate.
+Author: Lunar Labs
+License: MIT
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# lunarlabs-core
+
+**The product-neutral spine shared by Lunar Labs products.**
+
+```
+candidate (cheap/local) → verify (deterministic oracle) → pass?  ✓ done
+                                                        → fail/abstain?  ↑ teacher (frontier) → verify
+                                                                              ↓
+                                        append-only trace capture (both verdicts label-worthy)
+                                                                              ↓
+                    training-pair extraction (retrieval-task Q→A, NEVER raw transcripts)
+                                                                              ↓
+                          nightly consolidation → FITNESS GATE (promote only if deflection
+                                                  improves without regression) → ship or roll back
+```
+
+Two products are built on this spine:
+
+- **Artemis** — a small-model-first cascade for tool calls. It uses the spine to
+  deflect spend from frontier models to on-device models; its four oracle
+  families (agreement, provenance, structural, execution) are product-specific
+  and stay in Artemis.
+- **Avalon** — an adaptive AI runtime. It uses the same spine for its own
+  capture → verify → consolidate → gate loop over runtime behavior. Avalon's
+  vision is much larger than this library; `lunarlabs-core` is only the shared
+  contracts both products import so the safety semantics (abstention, fail-open,
+  gating) are defined — and tested — exactly once.
+
+This library is stdlib-only and deliberately boring. What is shared is the
+*contract*; what is measured, drafted, and trained stays in each product.
+
+## What the spine defines
+
+| module | contract | who consumes it |
+|---|---|---|
+| `verdict.py` | The three-valued verdict: PASS / FAIL / **ABSTAIN**, expressed as `ok` × `label`. | Both products' oracles. |
+| `verifier.py` | The `Oracle` protocol (`output → Verdict`), `all_of` composition, one trivial example oracle. | Both products plug in their own domain oracles. |
+| `escalation.py` | `verified_cascade`: candidate-first routing, **fail-open** on any internal error, `LUNARLABS_*` kill switch. | Artemis's router; Avalon's runtime decision path. |
+| `trace.py` | `TraceRecord` + append-only JSONL `TraceWriter`, with the dual-label capture gate as the only write path. | Artemis's escalation capture; Avalon's observed-event capture. |
+| `pairs.py` | `TrainingPair`: retrieval-task Q→A with provenance. **Never raw transcripts.** | Both products' nightly consolidation / LoRA pipelines. |
+| `fitness.py` | `fitness_gate`: pure decision over measured metrics — promote only on meaningful deflection gain with no keep-set regression. | Both products' nightly retrain jobs. |
+
+## The ABSTAIN third verdict is load-bearing
+
+Every oracle answers three things, not two. When a check cannot *mechanically*
+decide, it declines:
+
+- **PASS + label** — route on it, and it is strong enough to write training data.
+- **FAIL + label** — real evidence of a wrong answer; the pair may be captured.
+- **ABSTAIN** (`ok=False, label=False`) — escalate (costs money, never ships a
+  wrong answer) but the trace writer **refuses the pair**, so it poisons
+  nothing. Training on abstentions would teach the model the reference's
+  arbitrary surface conventions, which raises future agreement, which inflates
+  measured deflection: a self-fulfilling metric.
+
+`TraceWriter.record_escalation` enforces both conjuncts — the teacher's pass
+must be label-worthy AND the candidate's failure must be label-worthy — so a
+weak oracle cannot poison the retrain it feeds.
+
+## Fail-open, and the kill switch
+
+Every spine-internal failure — candidate down, timeout, malformed reply, a bug
+in the verifier itself — routes to the teacher instead of raising. Breaking
+the product loses the user permanently; a missed deflection costs a fraction
+of a cent. Nothing in `verified_cascade` raises past its caller.
+
+```bash
+export LUNARLABS_DISABLED=1     # every call goes straight to the teacher
+export LUNARLABS_DEFLECT=off    # same effect, second switch
+```
+
+Both are read fresh on every call — flipping the env var takes effect without
+a restart or redeploy.
+
+## Training pairs are retrieval-task Q→A — never raw transcripts
+
+From the JARVIS research: a raw transcript trains surface imitation (phrasing,
+length, the teacher's tics) and drowns the signal you paid an escalation for.
+A Q→A pair isolates exactly the thing the candidate got wrong: *given THIS
+observed context, the correct answer is THAT.* Every `TrainingPair` therefore
+has `task_type="retrieval"`, a question projected from the trace's context,
+the teacher's verified output as the answer, and full provenance (source trace
+id, candidate failure stage/reasons). A pair without provenance is
+unauditable and should be refused downstream.
+
+## Usage
+
+```python
+from lunarlabs_core import (
+    TraceWriter, verified_cascade, non_empty_oracle,
+    extract_pairs, write_pairs_jsonl, FitnessMetrics, fitness_gate,
+)
+
+trace = TraceWriter("traces.jsonl")
+
+result = verified_cascade(
+    candidate=lambda: small_model.answer(request),   # cheap path first
+    teacher=lambda: frontier_model.answer(request),  # only on verified failure
+    verify=my_oracle,                                # your deterministic oracle
+    context={"request_id": "..."},                   # opaque to the spine
+    trace=trace,
+)
+result.handled_by    # "candidate" | "teacher"
+result.failed_open   # True if the spine broke and got out of the way
+
+# Nightly consolidation:
+pairs = extract_pairs(trace.records())
+write_pairs_jsonl(pairs, "train.jsonl")
+
+decision = fitness_gate(
+    FitnessMetrics(deflection_rate=0.61, keep_set_score=0.92),  # base, measured by you
+    FitnessMetrics(deflection_rate=0.68, keep_set_score=0.92),  # candidate retrain
+)
+print(decision.summary())   # [PROMOTED] deflection 61% -> 68% (more deflection, no regression)
+```
+
+## What is deliberately NOT here
+
+- Artemis's four oracle families, its providers/proxy, pricing, audit and
+  dashboard — cost/tool-call-specific, they stay in Artemis.
+- Avalon's runtime, planners, and consolidation wiring.
+- Any trainer. `fitness.py` decides; it does not train. Each product owns its
+  LoRA/fine-tune stack and its metric measurement.
+
+## Docs
+
+- `docs/data-volume-falsifier.sql` — the JARVIS data-volume falsifier: SELECT-only
+  per-day counts of Carter-signal (user messages, observed-provenance commitments,
+  episodic memories, user facts, eval scorecards) over the last 30 days, to test
+  whether the real system produces enough usable training pairs/day to feed the
+  nightly trainer (the "~50/day starves it" question). Run with
+  `psql "$DATABASE_URL" -f docs/data-volume-falsifier.sql`.
+
+## Development
+
+```bash
+python -m pytest        # stdlib-only package; pytest is the only dev dependency
+```
+
+MIT © Lunar Labs

lunarlabs_core-0.1.0/README.md

@@ -0,0 +1,140 @@
+# lunarlabs-core
+
+**The product-neutral spine shared by Lunar Labs products.**
+
+```
+candidate (cheap/local) → verify (deterministic oracle) → pass?  ✓ done
+                                                        → fail/abstain?  ↑ teacher (frontier) → verify
+                                                                              ↓
+                                        append-only trace capture (both verdicts label-worthy)
+                                                                              ↓
+                    training-pair extraction (retrieval-task Q→A, NEVER raw transcripts)
+                                                                              ↓
+                          nightly consolidation → FITNESS GATE (promote only if deflection
+                                                  improves without regression) → ship or roll back
+```
+
+Two products are built on this spine:
+
+- **Artemis** — a small-model-first cascade for tool calls. It uses the spine to
+  deflect spend from frontier models to on-device models; its four oracle
+  families (agreement, provenance, structural, execution) are product-specific
+  and stay in Artemis.
+- **Avalon** — an adaptive AI runtime. It uses the same spine for its own
+  capture → verify → consolidate → gate loop over runtime behavior. Avalon's
+  vision is much larger than this library; `lunarlabs-core` is only the shared
+  contracts both products import so the safety semantics (abstention, fail-open,
+  gating) are defined — and tested — exactly once.
+
+This library is stdlib-only and deliberately boring. What is shared is the
+*contract*; what is measured, drafted, and trained stays in each product.
+
+## What the spine defines
+
+| module | contract | who consumes it |
+|---|---|---|
+| `verdict.py` | The three-valued verdict: PASS / FAIL / **ABSTAIN**, expressed as `ok` × `label`. | Both products' oracles. |
+| `verifier.py` | The `Oracle` protocol (`output → Verdict`), `all_of` composition, one trivial example oracle. | Both products plug in their own domain oracles. |
+| `escalation.py` | `verified_cascade`: candidate-first routing, **fail-open** on any internal error, `LUNARLABS_*` kill switch. | Artemis's router; Avalon's runtime decision path. |
+| `trace.py` | `TraceRecord` + append-only JSONL `TraceWriter`, with the dual-label capture gate as the only write path. | Artemis's escalation capture; Avalon's observed-event capture. |
+| `pairs.py` | `TrainingPair`: retrieval-task Q→A with provenance. **Never raw transcripts.** | Both products' nightly consolidation / LoRA pipelines. |
+| `fitness.py` | `fitness_gate`: pure decision over measured metrics — promote only on meaningful deflection gain with no keep-set regression. | Both products' nightly retrain jobs. |
+
+## The ABSTAIN third verdict is load-bearing
+
+Every oracle answers three things, not two. When a check cannot *mechanically*
+decide, it declines:
+
+- **PASS + label** — route on it, and it is strong enough to write training data.
+- **FAIL + label** — real evidence of a wrong answer; the pair may be captured.
+- **ABSTAIN** (`ok=False, label=False`) — escalate (costs money, never ships a
+  wrong answer) but the trace writer **refuses the pair**, so it poisons
+  nothing. Training on abstentions would teach the model the reference's
+  arbitrary surface conventions, which raises future agreement, which inflates
+  measured deflection: a self-fulfilling metric.
+
+`TraceWriter.record_escalation` enforces both conjuncts — the teacher's pass
+must be label-worthy AND the candidate's failure must be label-worthy — so a
+weak oracle cannot poison the retrain it feeds.
+
+## Fail-open, and the kill switch
+
+Every spine-internal failure — candidate down, timeout, malformed reply, a bug
+in the verifier itself — routes to the teacher instead of raising. Breaking
+the product loses the user permanently; a missed deflection costs a fraction
+of a cent. Nothing in `verified_cascade` raises past its caller.
+
+```bash
+export LUNARLABS_DISABLED=1     # every call goes straight to the teacher
+export LUNARLABS_DEFLECT=off    # same effect, second switch
+```
+
+Both are read fresh on every call — flipping the env var takes effect without
+a restart or redeploy.
+
+## Training pairs are retrieval-task Q→A — never raw transcripts
+
+From the JARVIS research: a raw transcript trains surface imitation (phrasing,
+length, the teacher's tics) and drowns the signal you paid an escalation for.
+A Q→A pair isolates exactly the thing the candidate got wrong: *given THIS
+observed context, the correct answer is THAT.* Every `TrainingPair` therefore
+has `task_type="retrieval"`, a question projected from the trace's context,
+the teacher's verified output as the answer, and full provenance (source trace
+id, candidate failure stage/reasons). A pair without provenance is
+unauditable and should be refused downstream.
+
+## Usage
+
+```python
+from lunarlabs_core import (
+    TraceWriter, verified_cascade, non_empty_oracle,
+    extract_pairs, write_pairs_jsonl, FitnessMetrics, fitness_gate,
+)
+
+trace = TraceWriter("traces.jsonl")
+
+result = verified_cascade(
+    candidate=lambda: small_model.answer(request),   # cheap path first
+    teacher=lambda: frontier_model.answer(request),  # only on verified failure
+    verify=my_oracle,                                # your deterministic oracle
+    context={"request_id": "..."},                   # opaque to the spine
+    trace=trace,
+)
+result.handled_by    # "candidate" | "teacher"
+result.failed_open   # True if the spine broke and got out of the way
+
+# Nightly consolidation:
+pairs = extract_pairs(trace.records())
+write_pairs_jsonl(pairs, "train.jsonl")
+
+decision = fitness_gate(
+    FitnessMetrics(deflection_rate=0.61, keep_set_score=0.92),  # base, measured by you
+    FitnessMetrics(deflection_rate=0.68, keep_set_score=0.92),  # candidate retrain
+)
+print(decision.summary())   # [PROMOTED] deflection 61% -> 68% (more deflection, no regression)
+```
+
+## What is deliberately NOT here
+
+- Artemis's four oracle families, its providers/proxy, pricing, audit and
+  dashboard — cost/tool-call-specific, they stay in Artemis.
+- Avalon's runtime, planners, and consolidation wiring.
+- Any trainer. `fitness.py` decides; it does not train. Each product owns its
+  LoRA/fine-tune stack and its metric measurement.
+
+## Docs
+
+- `docs/data-volume-falsifier.sql` — the JARVIS data-volume falsifier: SELECT-only
+  per-day counts of Carter-signal (user messages, observed-provenance commitments,
+  episodic memories, user facts, eval scorecards) over the last 30 days, to test
+  whether the real system produces enough usable training pairs/day to feed the
+  nightly trainer (the "~50/day starves it" question). Run with
+  `psql "$DATABASE_URL" -f docs/data-volume-falsifier.sql`.
+
+## Development
+
+```bash
+python -m pytest        # stdlib-only package; pytest is the only dev dependency
+```
+
+MIT © Lunar Labs

lunarlabs_core-0.1.0/lunarlabs_core/__init__.py

@@ -0,0 +1,35 @@
+"""lunarlabs-core: the product-neutral spine shared by Lunar Labs products.
+
+    capture  ->  verify/oracle  ->  escalate (fail-open)  ->
+    training-pair extraction (retrieval Q->A)  ->
+    nightly consolidation  ->  fitness gate / rollback
+
+Products (Artemis, Avalon) plug in their own drafters, oracles, and metrics;
+the contracts live here.
+"""
+from .verdict import Verdict, abstain, failed, passed
+from .verifier import Oracle, all_of, non_empty_oracle
+from .trace import TraceRecord, TraceWriter
+from .escalation import (
+    DEFLECT_ENV,
+    DISABLED_ENV,
+    Drafter,
+    EscalationResult,
+    deflection_disabled,
+    verified_cascade,
+)
+from .fitness import FitnessMetrics, GateDecision, fitness_gate
+from .pairs import TrainingPair, extract_pairs, pair_from_trace, write_pairs_jsonl
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "Verdict", "abstain", "failed", "passed",
+    "Oracle", "all_of", "non_empty_oracle",
+    "TraceRecord", "TraceWriter",
+    "Drafter", "EscalationResult", "deflection_disabled", "verified_cascade",
+    "DISABLED_ENV", "DEFLECT_ENV",
+    "FitnessMetrics", "GateDecision", "fitness_gate",
+    "TrainingPair", "extract_pairs", "pair_from_trace", "write_pairs_jsonl",
+    "__version__",
+]

lunarlabs_core-0.1.0/lunarlabs_core/escalation.py

@@ -0,0 +1,129 @@
+"""The escalation contract: candidate first, teacher only when the check fails.
+
+The spine's thesis in one function. The candidate (small, cheap, local,
+private) produces an output. A deterministic verifier checks it. Pass — done.
+Fail or abstain — escalate to the teacher (frontier, strong, expensive),
+verify again, and capture the (candidate-failed, teacher-passed) pair as
+training data. The system relies on the teacher less every night the
+candidate retrains on its own misses.
+
+FAIL OPEN. Every spine-internal failure — candidate down, timeout, malformed
+reply, a bug in the verifier itself — routes to the teacher instead of
+raising. Breaking the product loses the user permanently; a missed deflection
+costs a fraction of a cent. That asymmetry is not close, so nothing in this
+module is allowed to raise past `verified_cascade`.
+
+KILL SWITCH. Set LUNARLABS_DISABLED=1 (or LUNARLABS_DEFLECT=off) in the
+environment and every call goes straight to the teacher — no code change, no
+redeploy. The env is read fresh on every call, deliberately: flipping the
+variable must take effect without restarting anything.
+"""
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional
+
+from .trace import TraceWriter
+from .verdict import Verdict
+from .verifier import Oracle
+
+_TRUE = {"1", "true", "yes", "on"}
+_OFF = {"off", "0", "false", "no"}
+
+DISABLED_ENV = "LUNARLABS_DISABLED"
+DEFLECT_ENV = "LUNARLABS_DEFLECT"
+
+# Produces an output (text / tool call / plan / anything). Zero-arg so the
+# caller closes over its own request state.
+Drafter = Callable[[], Any]
+
+
+def deflection_disabled(env: Optional[Dict[str, str]] = None) -> bool:
+    """The kill switch. Read fresh on every call, deliberately."""
+    e = os.environ if env is None else env
+    if str(e.get(DISABLED_ENV, "")).strip().lower() in _TRUE:
+        return True
+    return str(e.get(DEFLECT_ENV, "")).strip().lower() in _OFF
+
+
+@dataclass
+class EscalationResult:
+    output: Any                   # the chosen output
+    handled_by: str               # "candidate" | "teacher"
+    verified: bool                # did the chosen output pass verification?
+    escalated: bool
+    candidate_verdict: Verdict
+    teacher_verdict: Optional[Verdict] = None
+    failed_open: bool = False     # the spine broke and got out of the way
+    failure: str = ""             # why, for the operator
+    captured: bool = False        # did this call write a training pair?
+
+
+def verified_cascade(
+    candidate: Drafter,
+    teacher: Drafter,
+    verify: Oracle,
+    *,
+    context: Optional[Dict[str, Any]] = None,
+    trace: Optional[TraceWriter] = None,
+    enabled: Optional[bool] = None,
+) -> EscalationResult:
+    """Run the candidate, verify, escalate on failure, capture the fix.
+
+    `enabled=None` reads the kill switch from the environment; pass a bool to
+    decide programmatically (tests, per-request overrides).
+    """
+    off = (not enabled) if enabled is not None else deflection_disabled()
+    candidate_out: Any = None
+    cv = Verdict(False, ["deflection disabled"], "disabled", label=False)
+    failure = ""
+
+    if not off:
+        try:
+            candidate_out = candidate()
+            cv = verify(candidate_out)
+        except Exception as exc:  # noqa: BLE001 — fail open, never break the caller
+            candidate_out, failure = None, f"{type(exc).__name__}: {exc}"
+            cv = Verdict(False, [failure], "spine_error", label=False)
+
+        if cv.ok:
+            return EscalationResult(
+                output=candidate_out, handled_by="candidate",
+                verified=True, escalated=False, candidate_verdict=cv,
+            )
+
+    teacher_out = teacher()
+    try:
+        tv = verify(teacher_out)
+    except Exception as exc:  # noqa: BLE001 — a broken oracle must not eat the
+        # teacher's answer. Hand it back unverified and say why.
+        failure = failure or f"{type(exc).__name__}: {exc}"
+        tv = Verdict(False, [failure], "spine_error", label=False)
+
+    # Only capture when the candidate actually produced a wrong answer. A
+    # crash is not evidence about the model, and disabled routing produced
+    # nothing to learn from. record_escalation enforces both label flags.
+    captured = False
+    if trace is not None and candidate_out is not None and not failure:
+        try:
+            captured = trace.record_escalation(
+                context or {}, candidate_out, cv, teacher_out, tv,
+            ) is not None
+        except Exception:  # noqa: BLE001 — capture is a bonus, never a blocker
+            pass
+
+    # A broken oracle must not eat the teacher's answer: when the verifier
+    # crashed (spine_error) rather than ruled, hand the teacher's output back
+    # unverified. When the teacher genuinely failed verification, prefer the
+    # candidate's attempt — both are known-wrong, but the candidate's failure
+    # at least carries the verifier's reasons.
+    chosen = (teacher_out
+              if tv.ok or tv.stage == "spine_error" or candidate_out is None
+              else candidate_out)
+    return EscalationResult(
+        output=chosen, handled_by="teacher",
+        verified=tv.ok, escalated=True, candidate_verdict=cv,
+        teacher_verdict=tv, failed_open=bool(failure), failure=failure,
+        captured=captured,
+    )

lunarlabs_core-0.1.0/lunarlabs_core/fitness.py

@@ -0,0 +1,79 @@
+"""The fitness gate: promote a retrain only if it is measurably better.
+
+    traces  ->  train candidate  ->  FITNESS GATE  ->  promote or rollback
+
+The gate is the safety keystone of the nightly consolidation: a retrained
+candidate is promoted only if it deflects MORE work to the cheap path without
+regressing on a held-out "keep" set. That prevents the classic disaster of
+silently shipping a worse model every night.
+
+This module is the CONTRACT, not the wiring. It is a pure function over
+measured metrics — it does not know what a model is, how deflection was
+measured, or what the keep set contains. Each product measures its own
+`FitnessMetrics` (Artemis: cascade replay over eval cases; Avalon: whatever its
+runtime's held-out tasks are) and hands the numbers here. The rule is what is
+shared:
+
+    1. REGRESSION CHECK FIRST. If the candidate's keep-set score dropped by
+       more than `regression_tolerance`, reject — regardless of the gain.
+    2. Then require a MEANINGFUL gain: candidate deflection must beat base
+       deflection by at least `min_gain`. Equal is not better; retraining has
+       a cost and a risk, so the bar is strictly above noise.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Optional
+
+
+@dataclass
+class FitnessMetrics:
+    """Measured performance of one model generation, produced by the product."""
+    deflection_rate: float            # fraction of work handled by the cheap path, 0..1
+    keep_set_score: Optional[float] = None  # held-out regression score, higher is better;
+                                            # None means "no keep set was run"
+
+
+@dataclass
+class GateDecision:
+    promoted: bool
+    base_deflection: float
+    candidate_deflection: float
+    reason: str
+
+    def summary(self) -> str:
+        arrow = f"{self.base_deflection:.0%} -> {self.candidate_deflection:.0%}"
+        verb = "PROMOTED" if self.promoted else "REJECTED"
+        return f"[{verb}] deflection {arrow} ({self.reason})"
+
+
+def fitness_gate(
+    base: FitnessMetrics,
+    candidate: FitnessMetrics,
+    *,
+    min_gain: float = 0.01,
+    regression_tolerance: float = 0.0,
+) -> GateDecision:
+    """Decide whether `candidate` may replace `base`.
+
+    `min_gain`: the smallest deflection improvement that justifies a promote.
+    `regression_tolerance`: how much keep-set score the candidate may lose
+    (use a small positive value if the keep-set metric is noisy; 0.0 is strict).
+    """
+    if base.keep_set_score is not None and candidate.keep_set_score is not None:
+        if candidate.keep_set_score < base.keep_set_score - regression_tolerance:
+            return GateDecision(
+                False, base.deflection_rate, candidate.deflection_rate,
+                f"regressed on keep set "
+                f"{base.keep_set_score:.0%}->{candidate.keep_set_score:.0%}",
+            )
+
+    if candidate.deflection_rate >= base.deflection_rate + min_gain:
+        return GateDecision(
+            True, base.deflection_rate, candidate.deflection_rate,
+            "more deflection, no regression",
+        )
+    return GateDecision(
+        False, base.deflection_rate, candidate.deflection_rate,
+        "no meaningful gain",
+    )

lunarlabs_core-0.1.0/lunarlabs_core/pairs.py

@@ -0,0 +1,89 @@
+"""Training-pair extraction: traces become Q->A pairs, never raw transcripts.
+
+THE FORMAT RULE, and it is not negotiable: every training pair is stored as a
+RETRIEVAL-TASK question-answer pair — a question whose answering requires
+recalling or deriving a specific fact from what the system observed, answered
+by the verified teacher output. Pairs are NEVER stored as raw transcripts.
+
+Why (from the JARVIS research): a raw transcript trains the model to imitate
+the surface of a conversation — phrasing, length, the teacher's tics — and the
+signal you actually paid an escalation for is drowned in tokens that carry no
+information. Worse, transcripts teach continuity that never happened: the
+model learns to complete conversations, not to retrieve facts. A Q->A pair
+isolates exactly the thing the candidate got wrong: "given THIS situation,
+the correct answer is THAT." Retrieval form is what lets the nightly retrain
+move the fitness metric instead of the perplexity metric.
+
+Concretely: the QUESTION is built from the trace record's context (the
+situation the candidate faced), and the ANSWER is the teacher's verified
+output. `question_from_context` is the one place a product plugs in how its
+context becomes a question; the default just asks for the correct output given
+the serialized context.
+
+Provenance rides along on every pair: which trace it came from, what the
+candidate got wrong, and when. A pair without provenance is unauditable and
+should be refused by any downstream trainer.
+"""
+from __future__ import annotations
+
+import json
+import time
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Any, Callable, Dict, List, Optional, Union
+
+# The default context->question projection. Products with structured context
+# (Artemis: messages + tools; Avalon: observed runtime state) should pass their
+# own `question_from_context` that phrases the retrieval task in their domain.
+def default_question(context: Dict[str, Any]) -> str:
+    return (
+        "Given the following observed context, produce the correct output.\n\n"
+        + json.dumps(context, sort_keys=True, default=str)
+    )
+
+
+@dataclass
+class TrainingPair:
+    """One retrieval-task Q->A pair with full provenance."""
+    question: str
+    answer: str
+    task_type: str = "retrieval"      # the format contract; do not set to "transcript"
+    source_trace_id: str = ""
+    candidate_failure_stage: str = ""
+    candidate_failure_reasons: List[str] = field(default_factory=list)
+    ts: float = field(default_factory=time.time)
+
+
+def pair_from_trace(
+    record: Dict[str, Any],
+    *,
+    question_from_context: Optional[Callable[[Dict[str, Any]], str]] = None,
+) -> TrainingPair:
+    """Extract one Q->A pair from a trace record (see trace.TraceRecord)."""
+    project = question_from_context or default_question
+    answer = record.get("teacher_output")
+    return TrainingPair(
+        question=project(record.get("context") or {}),
+        answer=answer if isinstance(answer, str) else json.dumps(answer, default=str),
+        source_trace_id=record.get("record_id", ""),
+        candidate_failure_stage=record.get("candidate_failure_stage", ""),
+        candidate_failure_reasons=list(record.get("candidate_failure_reasons") or []),
+    )
+
+
+def extract_pairs(
+    records: List[Dict[str, Any]],
+    *,
+    question_from_context: Optional[Callable[[Dict[str, Any]], str]] = None,
+) -> List[TrainingPair]:
+    return [pair_from_trace(r, question_from_context=question_from_context)
+            for r in records]
+
+
+def write_pairs_jsonl(pairs: List[TrainingPair], out: Union[str, Path]) -> int:
+    """Export pairs as JSONL for the nightly consolidation. Returns count."""
+    out = Path(out)
+    with out.open("w") as f:
+        for p in pairs:
+            f.write(json.dumps(asdict(p), default=str) + "\n")
+    return len(pairs)