@jenga-ai/agent 1.0.0

package/skills/todo/assets/todo_template.md

@@ -0,0 +1,3 @@
+# Todo
+
+<!-- Format: <mission title>: <E##_S##> (epic/story ref optional) -->

package/skills/train/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: train
+description: Scaffold and run ML training jobs. Use `new <type> <job-name>` to scaffold a job from a template, or `run <job-dir>` to execute the two-phase validate → train pipeline on an existing job.
+keywords:
+  - train
+  - ml training
+  - model training
+  - machine learning
+  - fine-tune
+examples:
+  - "scaffold a new training job"
+  - "train a classifier model"
+---
+
+# Train — ML Training Job Orchestrator
+
+## Usage
+
+```
+python skills/train/train_cli.py <subcommand> [args]
+
+Subcommands:
+  new <type> <job-name>   Scaffold a new training job from a template
+  run <job-dir>           Execute validate → train pipeline on an existing job
+
+Types: classifiers, transformers, nlp
+```
+
+If an unrecognised subcommand is given, argparse prints the usage message above and
+exits with a non-zero code.
+
+## Subcommands
+
+### `/train new <type> <job-name>`
+Scaffold a new training job from the appropriate template (positional / scriptable mode).
+
+**Supported types:** `classifiers`, `transformers`, `nlp`
+
+**Optional flags:**
+- `--model <value>` — Override the model name/type in `config.yaml`
+- `--epochs <n>` — Override the number of training epochs/iterations
+- `--batch-size <n>` — Override the batch size
+
+**Steps:**
+1. Validate `<type>` is one of: `classifiers`, `transformers`, `nlp`
+2. Copy the template directory from `.training/template/<type>/` to `jobs/<job-name>/`
+   - If `jobs/<job-name>/` already exists, auto-suffix: `jobs/<job-name>-1/`, `jobs/<job-name>-2/`, etc.
+3. Apply any CLI flag overrides to the scaffolded `config.yaml`
+4. Generate `start.sh` in the job directory (executable)
+5. Read the `workflow:` block from `config.yaml` and print next-step instructions
+6. Print: `✅ Scaffolded job '<job-name>' from template '<type>' at jobs/<job-name>/`
+
+### `/train new --interactive [--full]`
+Launch a guided wizard that scaffolds the job **and** configures `config.yaml` interactively.
+
+**Flags:**
+- `--interactive` / `-i` — Enable the wizard. Job name and type are collected via prompts.
+- `--full` — Expand the wizard to prompt for **every** configurable field (not just the critical ones).
+
+**Wizard flow:**
+1. Prompt for job name (freeform, used as directory name under `jobs/`)
+2. Prompt for job type (`classifiers`, `transformers`, or `nlp`)
+3. Scaffold the job directory from the template
+4. Prompt for critical config fields (2–3 key model params + data path per type):
+   - **classifiers**: train file, target column, model algorithm, n_estimators, test split
+   - **transformers**: model name, task, train file, epochs, learning rate
+   - **nlp**: model name, task, train file, iterations, learning rate
+5. With `--full`: additionally prompt for every remaining field in `config.yaml`
+6. Write user responses back into the scaffolded `config.yaml`
+
+Each prompt shows the field name, its current default, and an inline description hint.
+
+**Ctrl+C handling:**
+Pressing `Ctrl+C` at any point during the wizard interrupts cleanly. If a scaffold directory has already been created, the user is asked:
+```
+Delete scaffolded directory '<path>'? [y/N]
+```
+Answering `y` removes the directory; `N` (or Enter) keeps it.
+
+### `/train run <job-dir>`
+Execute the two-phase pre-flight → smoke test pipeline on an existing job directory.
+
+**Steps:**
+
+#### Phase A — Pre-flight validation
+1. Print: `[pre-flight] Running validate.py in <job-dir>...`
+2. Run `validate.py` as a subprocess inside `<job-dir>`:
+   ```bash
+   cd <job-dir> && python validate.py
+   ```
+3. Stream all stdout/stderr output to the terminal in real time.
+4. If `validate.py` exits non-zero:
+   - Print: `❌ [pre-flight] FAILED — halting before smoke test.`
+   - Surface the full error output.
+   - **Stop here. Do not proceed to Phase B.**
+5. If `validate.py` exits zero:
+   - Print: `✅ [pre-flight] Passed.`
+
+#### Phase B — Smoke test
+1. Print: `[smoke test] Running train.py --smoke in <job-dir>...`
+2. Run `train.py --smoke` as a subprocess inside `<job-dir>`:
+   ```bash
+   cd <job-dir> && python train.py --smoke
+   ```
+3. Stream all stdout/stderr output to the terminal in real time.
+4. If `train.py` exits non-zero:
+   - Print: `❌ [smoke test] FAILED.`
+   - Surface the full error output.
+5. If `train.py` exits zero:
+   - Print: `✅ [smoke test] Passed. Job '<job-dir>' completed successfully.`
+
+## Notes
+- `train.py` and `validate.py` must have no awareness of each other — the `/train` skill owns the gate.
+- Both subprocess calls must stream output (not buffer) so the user sees progress in real time.
+- Phase B is **never** invoked if Phase A fails.
+- The `--interactive` wizard requires `pyyaml` (`pip install pyyaml`) to patch `config.yaml`.

package/skills/train/assets/dashboard-templates/classifiers.html

@@ -0,0 +1,106 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>Classifier Training Dashboard — {{job_name}}</title>
+  <style>
+    *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+    body { font-family: system-ui, -apple-system, sans-serif; background: #f5f7fa; color: #1a1a2e; padding: 2rem; }
+    header { margin-bottom: 2rem; }
+    header h1 { font-size: 1.6rem; font-weight: 700; }
+    header p { color: #555; margin-top: .25rem; font-size: .9rem; }
+    .card { background: #fff; border-radius: 10px; box-shadow: 0 2px 8px rgba(0,0,0,.08); padding: 1.5rem; margin-bottom: 1.5rem; }
+    .card h2 { font-size: 1rem; font-weight: 600; color: #333; margin-bottom: 1rem; text-transform: uppercase; letter-spacing: .05em; }
+    .metrics-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 1rem; }
+    .metric-box { background: #f0f4ff; border-radius: 8px; padding: 1rem; text-align: center; }
+    .metric-box .value { font-size: 2rem; font-weight: 700; color: #3b5bdb; }
+    .metric-box .label { font-size: .8rem; color: #666; margin-top: .25rem; }
+    .bar-chart { display: flex; flex-direction: column; gap: .6rem; }
+    .bar-row { display: flex; align-items: center; gap .5rem; }
+    .bar-label { width: 100px; font-size: .85rem; color: #444; flex-shrink: 0; }
+    .bar-track { flex: 1; background: #e9ecef; border-radius: 4px; height: 20px; position: relative; }
+    .bar-fill { height: 100%; border-radius: 4px; background: #3b5bdb; transition: width .4s; }
+    .bar-val { width: 50px; text-align: right; font-size: .8rem; color: #444; }
+    table { width: 100%; border-collapse: collapse; }
+    th, td { padding: .6rem 1rem; text-align: left; font-size: .9rem; border-bottom: 1px solid #eee; }
+    th { background: #f8f9fa; font-weight: 600; color: #555; }
+    footer { text-align: center; font-size: .75rem; color: #999; margin-top: 2rem; }
+  </style>
+</head>
+<body>
+
+<header>
+  <h1>🤖 Classifier Training Dashboard</h1>
+  <p>Job: <strong>{{job_name}}</strong> &nbsp;|&nbsp; Model: <strong>{{model_type}}</strong></p>
+</header>
+
+<div class="card">
+  <h2>Key Metrics</h2>
+  <div class="metrics-grid">
+    <div class="metric-box">
+      <div class="value">{{accuracy}}</div>
+      <div class="label">Accuracy</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{f1_score}}</div>
+      <div class="label">F1 Score</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{precision}}</div>
+      <div class="label">Precision</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{recall}}</div>
+      <div class="label">Recall</div>
+    </div>
+  </div>
+</div>
+
+<div class="card">
+  <h2>Metric Comparison</h2>
+  <div class="bar-chart">
+    <div class="bar-row">
+      <span class="bar-label">Accuracy</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{accuracy}} * 100%)"></div></div>
+      <span class="bar-val">{{accuracy}}</span>
+    </div>
+    <div class="bar-row">
+      <span class="bar-label">F1 Score</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{f1_score}} * 100%)"></div></div>
+      <span class="bar-val">{{f1_score}}</span>
+    </div>
+    <div class="bar-row">
+      <span class="bar-label">Precision</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{precision}} * 100%)"></div></div>
+      <span class="bar-val">{{precision}}</span>
+    </div>
+    <div class="bar-row">
+      <span class="bar-label">Recall</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{recall}} * 100%)"></div></div>
+      <span class="bar-val">{{recall}}</span>
+    </div>
+  </div>
+</div>
+
+<div class="card">
+  <h2>Summary Table</h2>
+  <table>
+    <thead>
+      <tr><th>Metric</th><th>Value</th></tr>
+    </thead>
+    <tbody>
+      <tr><td>Accuracy</td><td>{{accuracy}}</td></tr>
+      <tr><td>F1 Score</td><td>{{f1_score}}</td></tr>
+      <tr><td>Precision</td><td>{{precision}}</td></tr>
+      <tr><td>Recall</td><td>{{recall}}</td></tr>
+      <tr><td>Model Type</td><td>{{model_type}}</td></tr>
+      <tr><td>Job Name</td><td>{{job_name}}</td></tr>
+    </tbody>
+  </table>
+</div>
+
+<footer>Generated by JengaAgent /train skill &nbsp;|&nbsp; {{job_name}}</footer>
+
+</body>
+</html>

package/skills/train/assets/dashboard-templates/nlp.html

@@ -0,0 +1,102 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>NLP Pipeline Dashboard — {{job_name}}</title>
+  <style>
+    *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+    body { font-family: system-ui, -apple-system, sans-serif; background: #f5f7fa; color: #1a1a2e; padding: 2rem; }
+    header { margin-bottom: 2rem; }
+    header h1 { font-size: 1.6rem; font-weight: 700; }
+    header p { color: #555; margin-top: .25rem; font-size: .9rem; }
+    .card { background: #fff; border-radius: 10px; box-shadow: 0 2px 8px rgba(0,0,0,.08); padding: 1.5rem; margin-bottom: 1.5rem; }
+    .card h2 { font-size: 1rem; font-weight: 600; color: #333; margin-bottom: 1rem; text-transform: uppercase; letter-spacing: .05em; }
+    .metrics-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 1rem; }
+    .metric-box { background: #fff8e1; border-radius: 8px; padding: 1rem; text-align: center; }
+    .metric-box .value { font-size: 2rem; font-weight: 700; color: #e67700; }
+    .metric-box .label { font-size: .8rem; color: #666; margin-top: .25rem; }
+    .bar-chart { display: flex; flex-direction: column; gap: .6rem; }
+    .bar-row { display: flex; align-items: center; gap: .5rem; }
+    .bar-label { width: 100px; font-size: .85rem; color: #444; flex-shrink: 0; }
+    .bar-track { flex: 1; background: #e9ecef; border-radius: 4px; height: 20px; position: relative; }
+    .bar-fill { height: 100%; border-radius: 4px; background: #e67700; transition: width .4s; }
+    .bar-val { width: 50px; text-align: right; font-size: .8rem; color: #444; }
+    table { width: 100%; border-collapse: collapse; }
+    th, td { padding: .6rem 1rem; text-align: left; font-size: .9rem; border-bottom: 1px solid #eee; }
+    th { background: #f8f9fa; font-weight: 600; color: #555; }
+    footer { text-align: center; font-size: .75rem; color: #999; margin-top: 2rem; }
+  </style>
+</head>
+<body>
+
+<header>
+  <h1>🔤 NLP Pipeline Dashboard</h1>
+  <p>Job: <strong>{{job_name}}</strong> &nbsp;|&nbsp; Model: <strong>{{model_name}}</strong> &nbsp;|&nbsp; Task: <strong>{{task}}</strong></p>
+</header>
+
+<div class="card">
+  <h2>Key Metrics</h2>
+  <div class="metrics-grid">
+    <div class="metric-box">
+      <div class="value">{{f1}}</div>
+      <div class="label">F1 Score</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{precision}}</div>
+      <div class="label">Precision</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{recall}}</div>
+      <div class="label">Recall</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{iterations}}</div>
+      <div class="label">Iterations</div>
+    </div>
+  </div>
+</div>
+
+<div class="card">
+  <h2>Score Breakdown</h2>
+  <div class="bar-chart">
+    <div class="bar-row">
+      <span class="bar-label">F1 Score</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{f1}} * 100%)"></div></div>
+      <span class="bar-val">{{f1}}</span>
+    </div>
+    <div class="bar-row">
+      <span class="bar-label">Precision</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{precision}} * 100%)"></div></div>
+      <span class="bar-val">{{precision}}</span>
+    </div>
+    <div class="bar-row">
+      <span class="bar-label">Recall</span>
+      <div class="bar-track"><div class="bar-fill" style="width: calc({{recall}} * 100%)"></div></div>
+      <span class="bar-val">{{recall}}</span>
+    </div>
+  </div>
+</div>
+
+<div class="card">
+  <h2>Summary Table</h2>
+  <table>
+    <thead>
+      <tr><th>Metric</th><th>Value</th></tr>
+    </thead>
+    <tbody>
+      <tr><td>F1 Score</td><td>{{f1}}</td></tr>
+      <tr><td>Precision</td><td>{{precision}}</td></tr>
+      <tr><td>Recall</td><td>{{recall}}</td></tr>
+      <tr><td>Iterations</td><td>{{iterations}}</td></tr>
+      <tr><td>Model Name</td><td>{{model_name}}</td></tr>
+      <tr><td>Task</td><td>{{task}}</td></tr>
+      <tr><td>Job Name</td><td>{{job_name}}</td></tr>
+    </tbody>
+  </table>
+</div>
+
+<footer>Generated by JengaAgent /train skill &nbsp;|&nbsp; {{job_name}}</footer>
+
+</body>
+</html>

package/skills/train/assets/dashboard-templates/transformers.html

@@ -0,0 +1,98 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>Transformer Fine-Tuning Dashboard — {{job_name}}</title>
+  <style>
+    *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+    body { font-family: system-ui, -apple-system, sans-serif; background: #f5f7fa; color: #1a1a2e; padding: 2rem; }
+    header { margin-bottom: 2rem; }
+    header h1 { font-size: 1.6rem; font-weight: 700; }
+    header p { color: #555; margin-top: .25rem; font-size: .9rem; }
+    .card { background: #fff; border-radius: 10px; box-shadow: 0 2px 8px rgba(0,0,0,.08); padding: 1.5rem; margin-bottom: 1.5rem; }
+    .card h2 { font-size: 1rem; font-weight: 600; color: #333; margin-bottom: 1rem; text-transform: uppercase; letter-spacing: .05em; }
+    .metrics-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(160px, 1fr)); gap: 1rem; }
+    .metric-box { background: #f0fff4; border-radius: 8px; padding: 1rem; text-align: center; }
+    .metric-box .value { font-size: 2rem; font-weight: 700; color: #2f9e44; }
+    .metric-box .label { font-size: .8rem; color: #666; margin-top: .25rem; }
+    .loss-section { margin-top: 1rem; }
+    .loss-row { display: flex; justify-content: space-between; padding: .5rem 0; border-bottom: 1px solid #eee; font-size: .9rem; }
+    .loss-row .loss-label { color: #555; }
+    .loss-row .loss-val { font-weight: 600; color: #2f9e44; }
+    table { width: 100%; border-collapse: collapse; }
+    th, td { padding: .6rem 1rem; text-align: left; font-size: .9rem; border-bottom: 1px solid #eee; }
+    th { background: #f8f9fa; font-weight: 600; color: #555; }
+    .perplexity-note { font-size: .8rem; color: #777; margin-top: .5rem; }
+    footer { text-align: center; font-size: .75rem; color: #999; margin-top: 2rem; }
+  </style>
+</head>
+<body>
+
+<header>
+  <h1>🤗 Transformer Fine-Tuning Dashboard</h1>
+  <p>Job: <strong>{{job_name}}</strong> &nbsp;|&nbsp; Model: <strong>{{model_name}}</strong></p>
+</header>
+
+<div class="card">
+  <h2>Key Metrics</h2>
+  <div class="metrics-grid">
+    <div class="metric-box">
+      <div class="value">{{perplexity}}</div>
+      <div class="label">Perplexity</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{eval_loss}}</div>
+      <div class="label">Eval Loss</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{train_loss}}</div>
+      <div class="label">Train Loss</div>
+    </div>
+    <div class="metric-box">
+      <div class="value">{{epochs}}</div>
+      <div class="label">Epochs</div>
+    </div>
+  </div>
+  <p class="perplexity-note">ℹ️ Lower perplexity = better language model. Derived from eval_loss (e^loss).</p>
+</div>
+
+<div class="card">
+  <h2>Loss Breakdown</h2>
+  <div class="loss-section">
+    <div class="loss-row">
+      <span class="loss-label">Evaluation Loss</span>
+      <span class="loss-val">{{eval_loss}}</span>
+    </div>
+    <div class="loss-row">
+      <span class="loss-label">Training Loss</span>
+      <span class="loss-val">{{train_loss}}</span>
+    </div>
+    <div class="loss-row">
+      <span class="loss-label">Perplexity (e^eval_loss)</span>
+      <span class="loss-val">{{perplexity}}</span>
+    </div>
+  </div>
+</div>
+
+<div class="card">
+  <h2>Summary Table</h2>
+  <table>
+    <thead>
+      <tr><th>Metric</th><th>Value</th></tr>
+    </thead>
+    <tbody>
+      <tr><td>Perplexity</td><td>{{perplexity}}</td></tr>
+      <tr><td>Eval Loss</td><td>{{eval_loss}}</td></tr>
+      <tr><td>Train Loss</td><td>{{train_loss}}</td></tr>
+      <tr><td>Epochs</td><td>{{epochs}}</td></tr>
+      <tr><td>Model Name</td><td>{{model_name}}</td></tr>
+      <tr><td>Job Name</td><td>{{job_name}}</td></tr>
+    </tbody>
+  </table>
+</div>
+
+<footer>Generated by JengaAgent /train skill &nbsp;|&nbsp; {{job_name}}</footer>
+
+</body>
+</html>

package/skills/train/assets/results-parsers/__init__.py

@@ -0,0 +1,9 @@
+"""
+results-parsers — ML training result extraction utilities.
+
+Each submodule exposes:  parse(job_dir: Path) -> dict
+Each submodule extracts type-specific metrics from job output artifacts.
+"""
+from pathlib import Path
+
+PARSERS = ["classifiers", "transformers", "nlp"]

package/skills/train/assets/results-parsers/classifiers.py

@@ -0,0 +1,84 @@
+"""
+classifiers.py — Results parser for sklearn-based classifier jobs.
+
+Extracts: accuracy, precision, recall, f1_score, model_type.
+Reads from results.json or scans training-results/ for JSON files.
+"""
+import json
+from pathlib import Path
+
+
+def parse(job_dir: Path) -> dict:
+    """
+    Parse training results for a classifiers job.
+
+    Returns a dict with keys:
+        accuracy, precision, recall, f1_score, model_type
+    Returns empty dict if no results can be found; never raises.
+    """
+    job_dir = Path(job_dir)
+    metrics = {}
+
+    # 1. Try results.json at root
+    results_json = job_dir / "results.json"
+    if results_json.exists():
+        try:
+            data = json.loads(results_json.read_text())
+            metrics = _extract_from_dict(data)
+            if metrics:
+                return metrics
+        except Exception:
+            pass
+
+    # 2. Scan training-results/ for any JSON files
+    training_results_dir = job_dir / "training-results"
+    if training_results_dir.exists():
+        for json_file in sorted(training_results_dir.rglob("*.json")):
+            try:
+                data = json.loads(json_file.read_text())
+                metrics = _extract_from_dict(data)
+                if metrics:
+                    return metrics
+            except Exception:
+                continue
+
+    return metrics
+
+
+def _extract_from_dict(data: dict) -> dict:
+    """Extract classifier-relevant keys from a parsed dict."""
+    metrics = {}
+    if not isinstance(data, dict):
+        return metrics
+
+    # Common field names produced by sklearn classification_report / custom scripts
+    field_map = {
+        "accuracy": ["accuracy", "test_accuracy", "val_accuracy", "acc"],
+        "precision": ["precision", "weighted avg.precision", "macro avg.precision"],
+        "recall": ["recall", "weighted avg.recall", "macro avg.recall"],
+        "f1_score": ["f1_score", "f1", "weighted avg.f1-score", "macro avg.f1-score"],
+        "model_type": ["model_type", "model", "algorithm", "classifier"],
+    }
+
+    for target_key, candidates in field_map.items():
+        for candidate in candidates:
+            # Support dot-path lookup (e.g. "weighted avg.precision")
+            value = _deep_get(data, candidate)
+            if value is not None:
+                metrics[target_key] = value
+                break
+
+    return metrics
+
+
+def _deep_get(data: dict, dotted_key: str):
+    """Retrieve a value from a nested dict using a dot-separated key path."""
+    keys = dotted_key.split(".")
+    current = data
+    for k in keys:
+        if not isinstance(current, dict):
+            return None
+        current = current.get(k)
+        if current is None:
+            return None
+    return current

package/skills/train/assets/results-parsers/nlp.py

@@ -0,0 +1,88 @@
+"""
+nlp.py — Results parser for spaCy / NLP pipeline training jobs.
+
+Extracts: f1, precision, recall, model_name, task, iterations.
+Reads from results.json or scans training-results/ for JSON files.
+"""
+import json
+from pathlib import Path
+
+
+def parse(job_dir: Path) -> dict:
+    """
+    Parse training results for an nlp job.
+
+    Returns a dict with keys:
+        f1, precision, recall, model_name, task, iterations
+    Returns empty dict if no results can be found; never raises.
+    """
+    job_dir = Path(job_dir)
+    metrics = {}
+
+    # 1. Try results.json at root
+    results_json = job_dir / "results.json"
+    if results_json.exists():
+        try:
+            data = json.loads(results_json.read_text())
+            metrics = _extract_from_dict(data)
+            if metrics:
+                return metrics
+        except Exception:
+            pass
+
+    # 2. Scan training-results/ for JSON files
+    training_results_dir = job_dir / "training-results"
+    if training_results_dir.exists():
+        for json_file in sorted(training_results_dir.rglob("*.json")):
+            try:
+                data = json.loads(json_file.read_text())
+                metrics = _extract_from_dict(data)
+                if metrics:
+                    return metrics
+            except Exception:
+                continue
+
+    # 3. Try spaCy training output (scores.json or metrics.json)
+    for scores_file in sorted(job_dir.rglob("scores.json")) + sorted(job_dir.rglob("metrics.json")):
+        try:
+            data = json.loads(scores_file.read_text())
+            metrics = _extract_from_dict(data)
+            if metrics:
+                return metrics
+        except Exception:
+            continue
+
+    return metrics
+
+
+def _extract_from_dict(data: dict) -> dict:
+    """Extract NLP-relevant keys from a parsed dict."""
+    metrics = {}
+    if not isinstance(data, dict):
+        return metrics
+
+    field_map = {
+        "f1": ["f1", "f1_score", "ents_f", "token_f", "tag_f", "sents_f", "score"],
+        "precision": ["precision", "ents_p", "token_p", "tag_p"],
+        "recall": ["recall", "ents_r", "token_r", "tag_r"],
+        "model_name": ["model_name", "model", "base_model"],
+        "task": ["task", "pipeline_component", "component"],
+        "iterations": ["iterations", "n_iter", "steps", "batches_trained"],
+    }
+
+    for target_key, candidates in field_map.items():
+        for candidate in candidates:
+            value = data.get(candidate)
+            if value is None:
+                # Try nested under "scores" or "results" key
+                for wrapper in ("scores", "results", "metrics"):
+                    nested = data.get(wrapper, {})
+                    if isinstance(nested, dict):
+                        value = nested.get(candidate)
+                    if value is not None:
+                        break
+            if value is not None:
+                metrics[target_key] = value
+                break
+
+    return metrics