@deeeed/metamask-harness 0.61.1 → 0.63.0

package/docs/RECIPES.md

@@ -19,6 +19,85 @@ mm-harness run <name> --describe
 Discovery reports parameters, selected adapter variant, nested recipes, source,
 and shadows. Use `--json` for exact metadata.
 
+## Optional recipe advisor
+
+`recipe-quality plan` assesses acceptance criteria against adapter-supported
+actions and library recipes before a draft exists. Provide a JSON task with an
+`acceptanceCriteria` array of 1–20 non-empty strings, plus optional `title` and
+`description`. Each result identifies a candidate to reuse or compose, or asks
+for more discovery or manual review. A shortlist is not the complete catalog,
+and a matching description does not establish runtime proof.
+
+```bash
+mm-harness recipe-quality plan --task task.json --adapter extension --advisor typesafe --output artifacts/recipe-plan.json --json
+mm-harness recipe-quality advise --recipe artifacts/recipe.json --task TASK.md --adapter extension --advisor typesafe --output artifacts/recipe-advice.json --json
+```
+
+Advice is disabled by default. Enable it for one call with `--advisor <provider>`,
+or save an engineer preference in the existing `~/.mm-harness/config.json`:
+
+```bash
+mm-harness config set advisor.provider typesafe
+mm-harness config set advisor.stages plan
+mm-harness config set advisor.enabled true
+mm-harness recipe-quality advisor-status --json
+```
+
+TypeSafe is the first provider and reads `TYPESAFE_API_KEY` from the execution
+host's environment. Keys never belong in the config. `advisor.model` or
+`--advisor-model <model-or-alias>` overrides its default, currently `jev-1.13.0`.
+Artifacts record the provider, requested model and returned version.
+
+Precedence is CLI options, then `MM_HARNESS_ADVISOR`,
+`MM_HARNESS_ADVISOR_MODEL`, `MM_HARNESS_ADVISOR_STAGES`, then engineer settings,
+then disabled. The advisor override is a provider name or `off`; stages are a
+comma-separated subset of `plan,draft,evidence`. A provider or model saved alone,
+or an API key, never enables advice. `--advisor off` overrides an enabled profile.
+A model override alone needs an already enabled provider.
+
+`advisor-status` is read-only and makes no provider call. It reports resolved
+stages, provider/model, key availability and setting sources without secrets.
+Missing keys report `skipped`; request failures report `unavailable`. Skills
+can record that output per task, invoke only the enabled stages, and continue
+normally when advice is unavailable. The initial profile enables planning only.
+Saved policy skips unselected stages with `stage_disabled`; an explicit
+`--advisor <provider>` call can request a stage directly. Pass `--advisor-stages`
+as well when a task must preserve its frozen stage policy.
+
+Farmslot can use its existing execution environment forwarding, for example
+`MM_HARNESS_ADVISOR=typesafe MM_HARNESS_ADVISOR_STAGES=plan`, or select a saved
+profile file through `MM_HARNESS_CONFIG`. The harness executing on that worker
+resolves the policy. Keep provider credentials in that host's secret environment,
+not project JSON or task inputs. No Farmslot-specific resolver is needed.
+
+`advise --recipe` requires the file path of a statically valid draft. Use `--params`
+with a JSON object for root parameter values. Its called recipe dependencies use
+the same library resolution as `run`, including `--library name=path` overrides.
+Optional `--evidence file.json` reads only that explicit text artifact;
+it does not inspect images or follow referenced files. Named credentials are
+redacted. Whole optional candidate contracts are omitted first to fit the bound;
+if the task, recipe, used contracts and evidence still exceed it, advice is
+skipped without truncating those inputs. Supply focused inputs without wallet
+secrets or unrelated logs.
+
+These results are advisory and their confidence is uncalibrated. The author
+decides whether to revise the recipe, then validates and executes it normally.
+Advice is kept separate from `recipe-quality.json` and cannot change a run's
+pass/fail status, authorize retries, or substitute for missing proof.
+
+An authoring agent can request `plan` before drafting, inspect the proposed
+contracts, then request `advise` on its statically valid draft. After executing
+the recipe, it can call `advise` again with `--evidence` pointing to focused JSON
+observations from the trace. Include the scenario parameters and observations
+for every claimed case; a filename or successful order submission is not proof
+of an observed position. The command reads that JSON only, not linked files.
+
+`completed` means the provider answered, not that the recipe passed. Address
+`attention`, discover missing information for `insufficient_context`, and inspect
+images for `needs_visual_review`. Revalidate and rerun any revised recipe through
+normal commands. `disabled`, `skipped`, and `unavailable` leave the authoring
+workflow in charge. These calls are explicit; `run` never invokes an advisor itself.
+
 ## Author and run
 
 An action performs one typed operation. A recipe owns a reusable graph or a

package/docs/SECURITY.md

@@ -117,3 +117,11 @@ Security-sensitive changes require:
 5. independent review of the exact diff.
 
 Use [QA](QA.md) for the runnable checklist.
+
+## Optional model advice
+
+`recipe-quality plan/advise --advisor typesafe` sends the declared task, relevant recipe and action contracts, effective parameters, validation findings, and explicitly supplied text evidence to `https://api.typesafe.ai/v1/systemone`. It is off by default and uses the engineer's `TYPESAFE_API_KEY`. Named credentials and known secret-input values are redacted; this is not a general guarantee that arbitrary prose is free of secrets. The provider does not receive images, and results remain separate advisory artifacts.
+
+Provider adapters own endpoints, credential names and response validation. Shared recipe policy uses typed semantic assessments; it does not select a model. `--advisor-model` changes the selected provider's model, not its endpoint or credential destination. Only implemented providers are accepted. Artifacts identify both the requested and returned model; confidence and probabilities are optional and are not treated as calibrated across providers.
+
+Saved advisor opt-in lives in the existing engineer config; CLI overrides take precedence over execution-environment overrides and saved preferences. `advisor-status` never sends context or returns credential values. Configuration accepts enabled/provider/model/stages only. A key alone does not opt in, and `--advisor off` disables an enabled profile. Advice input reads use one bounded file descriptor so replacing a path between checking and reading cannot switch the contents.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@deeeed/metamask-harness",
-  "version": "0.61.1",
+  "version": "0.63.0",
   "type": "module",
   "bin": {
     "mm-harness": "bin/mm-harness"

package/scripts/completions.sh

@@ -42,6 +42,7 @@ _mmh_flags_for() {
     fixtures) printf '%s' "--from --dev --force --fixture --adapter --target --json" ;;
     checklist) printf '%s' "" ;;
     execution-template) printf '%s' "--dir --domain-dir --project-worker --project-name --package-templates --package-id --flow --run-mode --platform --domain --id --provenance --include-shadowed --no-include-shadowed --title --force --json" ;;
+    recipe-quality) printf '%s' "--input --output --recipe --task --evidence --params --adapter --library --advisor --advisor-model --advisor-stages --json" ;;
     actions)  printf '%s' "--adapter --action --kind --raw --action-manifest --target --json" ;;
     call)     printf '%s' "--list --arg --adapter --target --artifacts-dir --action-manifest --heal --json" ;;
     run)      printf '%s' "--list --describe --plan --adapter --artifacts-dir --target --library --cdp-port --heal --record-video --json" ;;
@@ -80,6 +81,7 @@ if [ -n "${BASH_VERSION:-}" ]; then
       launch)      [ "$cword" -eq 2 ] && COMPREPLY=($(compgen -W "ios android" -- "$cur")) ;;
       fixtures)    [ "$cword" -eq 2 ] && COMPREPLY=($(compgen -W "init sync set reset generate finalize" -- "$cur")) ;;
       checklist)   [ "$cword" -eq 2 ] && COMPREPLY=($(compgen -W "mark closeout" -- "$cur")) ;;
+      recipe-quality) [ "$cword" -eq 2 ] && COMPREPLY=($(compgen -W "build plan advise advisor-status" -- "$cur")) ;;
       completions) [ "$cword" -eq 2 ] && COMPREPLY=($(compgen -W "install bash zsh status" -- "$cur")) ;;
       call)        [ "$cword" -eq 2 ] && COMPREPLY=($(compgen -W "$(_mmh_actions)" -- "$cur")) ;;
       run)         COMPREPLY=($(compgen -f -- "$cur"; compgen -W "$(_mmh_recipes)" -- "$cur")) ;;
@@ -114,6 +116,7 @@ if [ -n "${ZSH_VERSION:-}" ]; then
       launch)      (( CURRENT == 3 )) && { candidates=(ios android); compadd -a candidates; } ;;
       fixtures)    (( CURRENT == 3 )) && { candidates=(init sync set reset generate finalize); compadd -a candidates; } ;;
       checklist)   (( CURRENT == 3 )) && { candidates=(mark closeout); compadd -a candidates; } ;;
+      recipe-quality) (( CURRENT == 3 )) && { candidates=(build plan advise advisor-status); compadd -a candidates; } ;;
       completions) (( CURRENT == 3 )) && { candidates=(install bash zsh status); compadd -a candidates; } ;;
       call)        (( CURRENT == 3 )) && { candidates=("${(@f)$(_mmh_actions)}"); compadd -a candidates; } ;;
       run)         candidates=("${(@f)$(_mmh_recipes)}"); compadd -a candidates; _files ;;