just-vibe 0.7.0

package/plugins/just-vibe/skills/ml-monitor/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-monitor
+description: "Define operational and model-quality monitoring, including delayed labels Use to design or implement requested ML telemetry; ops-alerts designs response-worthy alert behavior."
+---
+
+# ml-monitor
+
+Define operational and model-quality monitoring, including delayed labels
+
+## Choose this workflow
+
+Use to design or implement requested ML telemetry; ops-alerts designs response-worthy alert behavior.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML deployment methods](../../references/packs/ml-deployment.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; serving/batch system, model objectives, telemetry, delayed-label process, and response ownership.
+
+versioned model and preprocessing artifacts, input/output schema, runtime/dependencies, operating targets, and authorized environment. Validate artifact trust before loading formats that can execute code. Packaging or writing monitoring configuration does not deploy a model or enable a hosted service.
+
+Resolve any task-specific tools, target identity and evidence before dependent actions. No external connection is assumed.
+
+## Scope
+
+Operational health, data quality, model performance, and label-arrival monitoring.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Separate leading signals from outcome metrics, define joins and delay windows, choose thresholds and runbook actions, and implement only requested instrumentation/configuration.
+- Separate service, feature, prediction and delayed-outcome signals; define stable joins, label-lag windows and model-version attribution.
+
+## Decision branches
+
+- **When recent predictions have not had time to receive labels:** Exclude them from matured quality denominators and report pending follow-up separately.
+
+## Deliver and verify
+
+- Monitoring design or code with metric definitions, privacy controls, and alert tests.
+- Signal/window/join/threshold/action contract and delayed-label test cases.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Delayed labels do not make recent unlabeled cases look correct; model-version changes remain distinguishable.
+
+## Stop and recover
+
+- Do not activate external alerts or promise ongoing observation without an actual authorized runtime/scheduler.
+
+## Example requests
+
+- **Normal (plan):** Design quality monitoring with delayed labels and model-version separation.
+- **edge (plan):** Monitor a model whose outcomes arrive thirty days after prediction.
+- **blocked (inspect):** Plan monitoring without activating alerts or claiming an ongoing watcher exists.

package/plugins/just-vibe/skills/ml-package/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-package
+description: "Package preprocessing, artifacts, dependencies, and interfaces Use to create a reproducible inference artifact; ml-serving implements the serving boundary."
+---
+
+# ml-package
+
+Package preprocessing, artifacts, dependencies, and interfaces
+
+## Choose this workflow
+
+Use to create a reproducible inference artifact; ml-serving implements the serving boundary.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML deployment methods](../../references/packs/ml-deployment.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **apply**. Apply; model/preprocessing artifacts, runtime, interface, and output location.
+
+versioned model and preprocessing artifacts, input/output schema, runtime/dependencies, operating targets, and authorized environment. Validate artifact trust before loading formats that can execute code. Packaging or writing monitoring configuration does not deploy a model or enable a hosted service.
+
+Declared evidence requirements: `ml.artifacts`. Use actual host discovery or adequate supplied artifacts; unavailable evidence remains blocked/unknown.
+
+## Scope
+
+Reproducible inference package and metadata; no registry upload or deployment implicitly.
+
+Only the requested local changes; external actions require their exact action and target in session authorization.
+
+## Execute
+
+- Verify artifact provenance, bundle preprocessing and schema, pin compatible dependencies, record versions/checksums, and run known-input smoke/parity checks.
+- Bundle preprocessing, feature order/schema, model identity, dependency constraints and known-input expectations; validate fresh-load parity in an isolated supported environment.
+
+## Decision branches
+
+- **When serialization may execute code and provenance is untrusted:** Inspect provenance and use a safe supported loading path or stop before loading.
+
+## Deliver and verify
+
+- Package, manifest, loading instructions, and expected-output fixtures.
+- Artifact manifest, checksums, input/output schema and fresh-load fixture results.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- A fresh supported environment reproduces fixture outputs within tolerance; incompatible artifact/schema versions fail clearly.
+
+## Stop and recover
+
+- Do not load untrusted executable serialization or bundle training data/secrets unnecessarily. Missing preprocessing prevents a complete package claim.
+
+## Example requests
+
+- **Normal (apply):** Package the model with preprocessing, schema, dependencies, and parity fixtures.
+- **edge (apply):** Package a model whose categorical encoder and feature order were saved separately.
+- **blocked (inspect):** Inspect an artifact manifest without loading untrusted executable serialization.

package/plugins/just-vibe/skills/ml-parity/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-parity
+description: "Check training preprocessing against production inference Use to compare training and serving transformations; ml-drift compares populations over time."
+---
+
+# ml-parity
+
+Check training preprocessing against production inference
+
+## Choose this workflow
+
+Use to compare training and serving transformations; ml-drift compares populations over time.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML deployment methods](../../references/packs/ml-deployment.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **inspect**. Inspect; training and serving pipelines/artifacts plus representative versioned inputs.
+
+versioned model and preprocessing artifacts, input/output schema, runtime/dependencies, operating targets, and authorized environment. Validate artifact trust before loading formats that can execute code. Packaging or writing monitoring configuration does not deploy a model or enable a hosted service.
+
+Declared evidence requirements: `ml.artifacts`. Use actual host discovery or adequate supplied artifacts; unavailable evidence remains blocked/unknown.
+
+## Scope
+
+Preprocessing, feature order, types, defaults, model version, and numerical parity.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Align raw inputs and versions, compare each transformation boundary, localize first divergence, evaluate declared tolerances, and propose or apply requested fixes.
+- Feed identical raw rows through each pipeline and compare schema, feature names/order, transformations and model outputs at each boundary.
+
+## Decision branches
+
+- **When model/artifact versions differ:** Resolve version identity before attributing output differences solely to preprocessing.
+
+## Deliver and verify
+
+- Parity report with stage-level differences and regression fixtures.
+- First divergent boundary, aligned inputs/versions and semantic versus numerical differences.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Reordered features are detected; expected floating-point variance is separated from semantic mismatch.
+
+## Stop and recover
+
+- Missing training transforms or serving access limits coverage. Do not compare outputs from different model versions as a pure preprocessing test.
+
+## Example requests
+
+- **Normal (inspect):** Compare training and serving transformations on these exact versioned inputs.
+- **edge (inspect):** Diagnose high offline scores but poor serving due to reordered features.
+- **blocked (inspect):** Assess parity with missing training transforms; keep unobserved stages unknown.

package/plugins/just-vibe/skills/ml-report/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-report
+description: "Document data, results, limitations, and intended use Use to communicate established ML evidence; ml-evaluate creates evaluation results."
+---
+
+# ml-report
+
+Document data, results, limitations, and intended use
+
+## Choose this workflow
+
+Use to communicate established ML evidence; ml-evaluate creates evaluation results.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML evaluation methods](../../references/packs/ml-evaluation.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; task, dataset/model manifests, evaluation results, intended use, and audience.
+
+frozen model/artifact, evaluation dataset identity, labels where needed, metric definitions, and task/operating context. Report sample counts and uncertainty appropriate to dependencies; avoid repeated test-set tuning. Exploratory findings need fresh confirmation before strong generalization claims.
+
+Declared evidence requirements: `ml.artifacts`. Use actual host discovery or adequate supplied artifacts; unavailable evidence remains blocked/unknown.
+
+## Scope
+
+Accurate model documentation and release assessment; no invented experiments or approval.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Reconcile evidence, describe training/evaluation conditions, summarize baseline and slice results, document limitations and excluded uses, and identify missing release evidence.
+- Reconcile every number with a run and denominator, separate validation selection from independent test evidence and document deployment population/excluded uses.
+
+## Decision branches
+
+- **When evidence is missing for a key cohort or release gate:** Keep the limitation visible and withhold the corresponding suitability claim.
+
+## Deliver and verify
+
+- Model report/card with provenance, metrics, operating assumptions, and open risks.
+- Model/data/version summary, baseline/slice evidence, limitations and release gaps.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Every numerical claim traces to an actual run; missing cohort evidence is disclosed rather than generalized away.
+
+## Stop and recover
+
+- Do not present a report as deployment authorization or claim suitability for untested populations.
+
+## Example requests
+
+- **Normal (plan):** Write a model report using these actual runs and identify unsupported uses.
+- **edge (plan):** Write a model report with strong average performance but poor sparse-cohort evidence.
+- **blocked (inspect):** Draft a report with missing test results; do not invent metrics or deployment approval.

package/plugins/just-vibe/skills/ml-reproduce/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-reproduce
+description: "Reproduce a result from code, data, and configuration Use to repeat a specified run; ml-baseline defines a new benchmark."
+---
+
+# ml-reproduce
+
+Reproduce a result from code, data, and configuration
+
+## Choose this workflow
+
+Use to repeat a specified run; ml-baseline defines a new benchmark.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML experimentation methods](../../references/packs/ml-experiments.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; claimed result, code/data/artifact identities, environment, tolerance, and budget.
+
+dataset/split manifests, fixed objective/metric, environment/dependencies, baseline where applicable, and explicit compute limits. Record code revision, configuration, seeds, artifact paths, and resource use. Local smoke checks do not imply authorization for paid training. Never optimize on the held-out test set.
+
+Resolve any task-specific tools, target identity and evidence before dependent actions. No external connection is assumed.
+
+## Scope
+
+Re-run a defined result with explicit reproducibility criteria.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Verify immutable inputs and dependency versions, reconstruct the procedure, run authorized bounded work, compare outputs/metrics within justified tolerance, and isolate deviations.
+- Resolve exact data/artifact/code/dependency identities, reconstruct preprocessing and evaluation, and declare nondeterminism tolerances before execution.
+
+## Decision branches
+
+- **When original assets are unavailable and substitutes are necessary:** Label the result a reimplementation or approximate reproduction and list each substitution.
+
+## Deliver and verify
+
+- Reproduction record, matched/different conditions, measured result, and discrepancy analysis.
+- Reproduction manifest, deviations, observed differences and tolerance justification.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Data/version mismatch is discovered before claiming reproduction; nondeterministic hardware differences use declared tolerances.
+
+## Stop and recover
+
+- Missing original assets may make exact reproduction impossible. Do not silently substitute a different dataset or model and call it reproduced.
+
+## Example requests
+
+- **Normal (plan):** Plan reproducing this result with exact artifact identities and a two-hour budget.
+- **edge (plan):** Reproduce a GPU run on another supported device with explicit tolerances.
+- **blocked (inspect):** Assess reproducibility when the original dataset snapshot is missing.

package/plugins/just-vibe/skills/ml-robustness/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-robustness
+description: "Test missing inputs, noise, distribution changes, and boundaries Use for bounded valid perturbation tests; ml-drift compares observed populations."
+---
+
+# ml-robustness
+
+Test missing inputs, noise, distribution changes, and boundaries
+
+## Choose this workflow
+
+Use for bounded valid perturbation tests; ml-drift compares observed populations.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML evaluation methods](../../references/packs/ml-evaluation.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; model, plausible perturbations, operating bounds, metrics, and evaluation budget.
+
+frozen model/artifact, evaluation dataset identity, labels where needed, metric definitions, and task/operating context. Report sample counts and uncertainty appropriate to dependencies; avoid repeated test-set tuning. Exploratory findings need fresh confirmation before strong generalization claims.
+
+Resolve any task-specific tools, target identity and evidence before dependent actions. No external connection is assumed.
+
+## Scope
+
+Missing inputs, noise, boundary cases, and realistic distribution changes.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Define validity-preserving perturbations, protect labels from invalid transformations, run bounded authorized tests, compare baseline sensitivity, and identify failure envelopes.
+- Define which changes should preserve labels and expected behavior, cap the sweep and compare both failure rate and input validity against a baseline.
+
+## Decision branches
+
+- **When a perturbation changes the true label or leaves the valid domain:** Classify it separately from an invariance failure.
+
+## Deliver and verify
+
+- Robustness protocol/results and prioritized mitigations.
+- Perturbation contract, tested envelope, failures and unsupported regions.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Missing required inputs fail predictably; label-changing perturbations are not scored as ordinary invariance tests.
+
+## Stop and recover
+
+- No claims of universal robustness from a finite suite. Large synthetic sweeps stop at the declared compute cap.
+
+## Example requests
+
+- **Normal (plan):** Plan plausible missing-input and noise tests with fixed labels and bounded compute.
+- **edge (plan):** Test missing optional fields while rejecting transformations that change the outcome.
+- **blocked (inspect):** Design robustness tests without running a large synthetic inference sweep.

package/plugins/just-vibe/skills/ml-rollout/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-rollout
+description: "Prepare shadow, canary, or staged deployment and rollback criteria Use for a scoped model promotion plan or transition; ml-report documents offline evidence."
+---
+
+# ml-rollout
+
+Prepare shadow, canary, or staged deployment and rollback criteria
+
+## Choose this workflow
+
+Use for a scoped model promotion plan or transition; ml-report documents offline evidence.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML deployment methods](../../references/packs/ml-deployment.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; candidate/stable models, target service, evaluation evidence, traffic constraints, and rollback thresholds.
+
+versioned model and preprocessing artifacts, input/output schema, runtime/dependencies, operating targets, and authorized environment. Validate artifact trust before loading formats that can execute code. Packaging or writing monitoring configuration does not deploy a model or enable a hosted service.
+
+Resolve any task-specific tools, target identity and evidence before dependent actions. No external connection is assumed.
+
+## Scope
+
+Shadow/canary/staged rollout and recovery; execution only on explicitly authorized targets.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Check package/schema compatibility, define cohort routing and comparison metrics, preserve fallback artifacts, prepare staged gates, and verify authorized transitions against evidence.
+- Check feature/schema/artifact compatibility, define shadow/canary routing and predeclared stop criteria, and preserve a usable fallback plus traffic-switch mechanism.
+
+## Decision branches
+
+- **When fallback cannot consume the new schema or features:** Resolve that compatibility before describing rollback as available.
+
+## Deliver and verify
+
+- Rollout plan or execution record with actual traffic/version state and rollback criteria.
+- Cohort/gate/metric plan, candidate/fallback identities and verified transitions.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Canary failures halt progression; rollback accounts for incompatible feature/schema changes.
+
+## Stop and recover
+
+- No automatic full rollout from good offline metrics. Shadow traffic may incur cost and privacy obligations covered by the requested scope.
+
+## Example requests
+
+- **Normal (plan):** Prepare a canary rollout and rollback plan; do not change production traffic.
+- **edge (plan):** Plan a canary when the previous model expects an older feature schema.
+- **blocked (inspect):** Assess rollout readiness without live routing access or shadow-inference budget.

package/plugins/just-vibe/skills/ml-serving/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-serving
+description: "Implement online inference with validation and observable errors Use to implement model service behavior; ml-rollout plans traffic transition."
+---
+
+# ml-serving
+
+Implement online inference with validation and observable errors
+
+## Choose this workflow
+
+Use to implement model service behavior; ml-rollout plans traffic transition.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML deployment methods](../../references/packs/ml-deployment.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **apply**. Apply; model package, request/response contract, latency/resource constraints, and target runtime.
+
+versioned model and preprocessing artifacts, input/output schema, runtime/dependencies, operating targets, and authorized environment. Validate artifact trust before loading formats that can execute code. Packaging or writing monitoring configuration does not deploy a model or enable a hosted service.
+
+Resolve any task-specific tools, target identity and evidence before dependent actions. No external connection is assumed.
+
+## Scope
+
+Online inference service code and isolated validation; live hosting requires a deployment request.
+
+Only the requested local changes; external actions require their exact action and target in session authorization.
+
+## Execute
+
+- Validate inputs, manage model lifecycle/readiness, enforce resource/time limits, map errors, add redacted observability, and test concurrent valid/invalid requests.
+- Define readiness for the correct artifact, input bounds, batching/concurrency and deadlines; validate shapes/types before inference and preserve version in responses/telemetry.
+
+## Decision branches
+
+- **When model loading fails or an incompatible schema arrives:** Fail readiness or return a typed request error without serving an unidentified fallback.
+
+## Deliver and verify
+
+- Service, configuration, operational checks, and performance evidence if measured.
+- Serving contract, lifecycle, failure handling and concurrent valid/invalid checks.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- Invalid shapes/types fail safely; readiness does not pass before the correct model is usable.
+
+## Stop and recover
+
+- No live endpoint provisioning or model registry mutation implicitly. Do not log raw sensitive inference inputs by default.
+
+## Example requests
+
+- **Normal (apply):** Implement local inference with validation, readiness, and safe error handling.
+- **edge (apply):** Serve a model with bounded batch size and a failed startup load.
+- **blocked (inspect):** Design serving code without provisioning an endpoint or logging raw sensitive inputs.

package/plugins/just-vibe/skills/ml-slices/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-slices
+description: "Compare meaningful cohorts or operating conditions Use for cohort performance comparisons; ml-error-analysis investigates individual failure mechanisms."
+---
+
+# ml-slices
+
+Compare meaningful cohorts or operating conditions
+
+## Choose this workflow
+
+Use for cohort performance comparisons; ml-error-analysis investigates individual failure mechanisms.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML evaluation methods](../../references/packs/ml-evaluation.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **inspect**. Inspect; predictions/labels, meaningful cohorts, minimum sample guidance, and operating context.
+
+frozen model/artifact, evaluation dataset identity, labels where needed, metric definitions, and task/operating context. Report sample counts and uncertainty appropriate to dependencies; avoid repeated test-set tuning. Exploratory findings need fresh confirmation before strong generalization claims.
+
+Declared evidence requirements: `ml.artifacts`. Use actual host discovery or adequate supplied artifacts; unavailable evidence remains blocked/unknown.
+
+## Scope
+
+Performance across cohorts/time/conditions and coverage gaps.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Predefine important slices where possible, compute counts and metrics consistently, account for dependent samples, flag small groups, and distinguish exploratory comparisons.
+- Define important slices and overlap, compute consistent counts/metrics and distinguish planned from exploratory comparisons with small-sample limits.
+
+## Decision branches
+
+- **When a cohort has no outcomes or very few positives:** Report unavailable/unstable evidence rather than a confident zero or perfect score.
+
+## Deliver and verify
+
+- Slice table, uncertainty, worst-supported conditions, and follow-up data needs.
+- Slice definition, denominator, metric, uncertainty and coverage gaps.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- A tiny cohort's extreme score is qualified; missing cohorts are shown as no evidence rather than zero performance.
+
+## Stop and recover
+
+- Avoid causal or fairness guarantees from a metric table alone. Do not expose identifying small-group records.
+
+## Example requests
+
+- **Normal (inspect):** Compare operating-condition cohorts and report uncertainty for small slices.
+- **edge (inspect):** Compare overlapping cohorts with one tiny high-error subgroup.
+- **blocked (inspect):** Assess slice coverage without sensitive row-level records or causal fairness claims.

package/plugins/just-vibe/skills/ml-split/SKILL.md

@@ -0,0 +1,58 @@
+---
+name: ml-split
+description: "Design splits respecting time, groups, entities, and dependencies Use to design evaluation partitions matching deployment; ml-leakage audits actual contamination evidence."
+---
+
+# ml-split
+
+Design splits respecting time, groups, entities, and dependencies
+
+## Choose this workflow
+
+Use to design evaluation partitions matching deployment; ml-leakage audits actual contamination evidence.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML data methods](../../references/packs/ml-data.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; task, entity/group/time dependencies, deployment regime, and dataset version.
+
+task definition, dataset identity, field semantics, entity/time keys, and permission to inspect bounded data. Record prediction moment, label horizon, sampling, and provenance. Preserve held-out evaluation boundaries; no data upload, label alteration, or feature fitting across splits implicitly.
+
+Declared evidence requirements: `data.read`. Use actual host discovery or adequate supplied artifacts; unavailable evidence remains blocked/unknown.
+
+## Scope
+
+Train/validation/test membership and fitting boundaries; write manifests only when requested.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Define deployment population, row/entity dependence, prediction times, model-fit cutoff, outcome horizon and label availability. Decide whether the question concerns future observations of known entities, unseen entities, or separate evaluations of both.
+- Specify exact train/validation/test endpoints and an as-of snapshot. Compare timezone-aware instants rather than timestamp strings. Exclude observations that would not yet exist, and distinguish event time from ingestion and label-observation time.
+- Determine training eligibility before fitting preprocessing: an otherwise training-period row with an immature outcome must not teach the historical model. Retain unknown held-out labels as unknown where the evaluation contract requires it; zero is a valid observed label.
+- Derive group separation, purging or gaps from the stated deployment question and actual dependence/availability evidence. Record deterministic membership and exclusions with reasons; do not invent a universal embargo duration.
+- Verify exact boundaries, label maturity, duplicates, relevant group overlap, empty partitions and transform fit membership. Report the number of rows and number with evaluable labels separately.
+
+## Decision branches
+
+- **When the model will serve both known and unseen entities:** Define separate evaluation questions instead of asserting one grouping rule answers both.
+
+## Deliver and verify
+
+- Deployment question, snapshot and split/eligibility rules, deterministic membership/exclusion evidence, transform fit population and overlap/maturity checks.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- No row, label or learned transform contains information unavailable at the simulated fit/prediction moment. Unknown outcomes stay unknown, and permitted known-entity overlap is distinguished from forbidden leakage.
+
+## Stop and recover
+
+- Do not use random splitting by habit or repeatedly tune the split to improve scores. Document unsupported generalization claims.
+
+## Example requests
+
+- **Normal (plan):** Design time/group splits for overlapping machine sensor windows.
+- **edge (plan):** Split overlapping windows for forecasting on known machines and evaluate unseen machines separately.
+- **blocked (inspect):** Plan a split with unknown label horizon; do not invent a universal gap duration.

package/plugins/just-vibe/skills/ml-threshold/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: ml-threshold
+description: "Choose thresholds against explicit costs or capacity limits Use to choose a decision cutoff under explicit costs/capacity; ml-evaluate measures fixed behavior."
+---
+
+# ml-threshold
+
+Choose thresholds against explicit costs or capacity limits
+
+## Choose this workflow
+
+Use to choose a decision cutoff under explicit costs/capacity; ml-evaluate measures fixed behavior.
+
+Read [shared execution](../../references/execution.md) for context/mode/authority handling and [ML evaluation methods](../../references/packs/ml-evaluation.md) for tool selection and operational details. Resolve these paths from this skill file; all runtime assets ship inside the plugin.
+
+## Input and mode
+
+Use the complete request appended to this invocation, preserving all constraints and references. Default mode: **plan**. Plan; scores/labels, explicit error costs or capacity, prevalence, and validation protocol.
+
+frozen model/artifact, evaluation dataset identity, labels where needed, metric definitions, and task/operating context. Report sample counts and uncertainty appropriate to dependencies; avoid repeated test-set tuning. Exploratory findings need fresh confirmation before strong generalization claims.
+
+Declared evidence requirements: `ml.artifacts`. Use actual host discovery or adequate supplied artifacts; unavailable evidence remains blocked/unknown.
+
+## Scope
+
+Select an operating point or decision policy; no production activation.
+
+None by default. Plan artifacts may be saved when requested.
+
+## Execute
+
+- Compare threshold tradeoffs, model workload/capacity, handle ties and uncertainty, choose using validation data, and reserve independent confirmation.
+- Compute validation tradeoffs with denominators and tie handling, translate them into expected workload under stated volume/prevalence and reserve independent confirmation.
+
+## Decision branches
+
+- **When no agreed cost or capacity preference distinguishes options:** Present the tradeoff curve and the missing decision rather than selecting an arbitrary optimum.
+
+## Deliver and verify
+
+- Threshold recommendation, confusion/workload estimates, assumptions, and sensitivity.
+- Threshold policy, expected workload assumptions and independent evaluation requirement.
+
+Verify these observable conditions when applicable to the actual task; do not claim they were exercised from merely reading this file:
+
+- A daily review cap is respected under declared volume assumptions; a changed prevalence is shown to affect workload/precision.
+
+## Stop and recover
+
+- Do not invent business costs or optimize against the held-out test set. Unresolved priorities yield a tradeoff curve rather than a forced value.
+
+## Example requests
+
+- **Normal (plan):** Choose validation thresholds when reviewers can inspect 200 transactions daily.
+- **edge (plan):** Select a daily review threshold with tied scores and a hard queue cap.
+- **blocked (inspect):** Compare cutoffs with unknown error costs; do not optimize on test labels.