harness-plugin 0.1.0__tar.gz

harness_plugin-0.1.0/.gitignore

@@ -0,0 +1,52 @@
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.venv/
+venv/
+env/
+dist/
+build/
+*.egg-info/
+*.egg
+.eggs/
+*.whl
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+htmlcov/
+.coverage
+.coverage.*
+coverage.xml
+*.cover
+.tox/
+.nox/
+.hypothesis/
+*.log
+.DS_Store
+Thumbs.db
+.env
+.env.*
+!.env.example
+catboost_info/
+docs/plans/
+.claude/
+.worktrees/
+.mcp.json
+harness-demo/
+projects/
+
+# HarnessML generated artifacts
+**/data/features/cache/
+**/outputs/
+**/experiments/
+!**/src/views/Experiments/
+
+# Jekyll
+docs/_site/
+
+# Harness Studio
+packages/harness-studio/frontend/node_modules/
+packages/harness-studio/frontend/dist/
+packages/harness-studio/frontend/.vite/
+.studio/

harness_plugin-0.1.0/PKG-INFO

@@ -0,0 +1,20 @@
+Metadata-Version: 2.4
+Name: harness-plugin
+Version: 0.1.0
+Summary: Claude Code plugin for AI-driven ML experimentation
+Project-URL: Homepage, https://github.com/msilverblatt/harness-ml
+Project-URL: Repository, https://github.com/msilverblatt/harness-ml
+Project-URL: Issues, https://github.com/msilverblatt/harness-ml/issues
+Author: Matt Silverblatt-Buser
+License-Expression: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Requires-Dist: click>=8.0
+Requires-Dist: harness-core<1.0,>=0.1.0
+Requires-Dist: mcp>=1.0

harness_plugin-0.1.0/README.md

@@ -0,0 +1,134 @@
+# Harness Plugin
+
+MCP (Model Context Protocol) server for [HarnessML](https://github.com/msilverblatt/harness-ml). Provides AI-driven ML experimentation through a thin async dispatcher with hot-reloadable handlers.
+
+## Architecture
+
+```
+mcp_server.py          # Tool signatures + docstrings (thin dispatcher)
+handlers/
+├── data.py            # 19 actions: add, validate, fill_nulls, inspect, profile, views...
+├── features.py        #  6 actions: add, add_batch, test, discover, diversity, auto_search
+├── models.py          # 10 actions: add, update, remove, list, show, presets, batch ops, clone
+├── config.py          # 12 actions: init, update_data, ensemble, backtest, show, targets...
+├── pipeline.py        # 13 actions: run_backtest, predict, diagnostics, compare, explain...
+├── experiments.py     # 10 actions: create, write_overlay, run, promote, quick_run, explore...
+├── competitions.py    # 13 actions: create, simulate, standings, brackets, score, adjust...
+├── _validation.py     # Fuzzy enum matching with "Did you mean?" hints
+└── _common.py         # Shared helpers (resolve_project_dir, parse_json_param)
+```
+
+## Key Design Principles
+
+**Thin dispatcher**: `mcp_server.py` contains only tool signatures and docstrings. All business logic lives in `handlers/*.py`.
+
+**Hot-reload in dev mode**: Set `HARNESS_DEV=1` to enable hot-reloading of handler code. Changes to handler files take effect immediately without restarting the server. Changes to tool signatures or docstrings in `mcp_server.py` still require a restart.
+
+**Fuzzy enum matching**: Invalid action names get helpful "Did you mean?" suggestions using edit distance matching. Cross-parameter hints guide users toward correct tool usage.
+
+**Event emission**: Every tool call emits a structured event to SQLite for Studio observability. Emission is fail-safe -- it never blocks or breaks tool execution.
+
+## Tools
+
+7 MCP tools exposing ~83 actions:
+
+| Tool | Actions | Purpose |
+|------|---------|---------|
+| `data` | 19 | Data ingestion, validation, profiling, views, sources |
+| `features` | 6 | Feature engineering: add, batch, discover, auto-search |
+| `models` | 10 | Model configuration: add, update, remove, presets, clone |
+| `configure` | 12 | Project setup: init, ensemble, backtest, targets, guardrails |
+| `pipeline` | 13 | Execution: backtest, predict, diagnostics, compare, explain |
+| `experiments` | 10 | Experiment management: create, run, promote, compare, journal |
+| `competitions` | 13 | Tournament simulation: brackets, scoring, strategies |
+
+## Handler Dispatch Pattern
+
+Each handler module follows the same pattern:
+
+```python
+ACTIONS = {
+    "add": _handle_add,
+    "remove": _handle_remove,
+    ...
+}
+
+async def dispatch(action: str, **kwargs) -> str:
+    err = validate_enum(action, set(ACTIONS), "action")
+    if err:
+        return err  # "Did you mean 'add'?"
+    result = ACTIONS[action](**kwargs)
+    if asyncio.iscoroutine(result):
+        result = await result
+    return result
+```
+
+## Setup
+
+Add to your `.mcp.json` (Claude Desktop, Claude Code, or any MCP host):
+
+```json
+{
+  "mcpServers": {
+    "harness-ml": {
+      "command": "uv",
+      "args": [
+        "--directory", "/path/to/harness-ml",
+        "run", "harness-ml"
+      ]
+    }
+  }
+}
+```
+
+For dev mode with hot-reload:
+
+```json
+{
+  "mcpServers": {
+    "harness-ml": {
+      "command": "uv",
+      "args": [
+        "--directory", "/path/to/harness-ml",
+        "run", "harness-ml"
+      ],
+      "env": {
+        "HARNESS_DEV": "1"
+      }
+    }
+  }
+}
+```
+
+## Event Emission
+
+Tool calls emit events to SQLite for Harness Studio observability:
+
+- Event type, tool name, action, parameters, result summary
+- Timestamps and session tracking
+- Fail-safe: exceptions in emission are swallowed, never breaking tool execution
+
+## Extending
+
+**Adding a new action** to an existing handler:
+
+1. Add handler function `_handle_my_action(**kwargs) -> str` in the handler module
+2. Add entry to the `ACTIONS` dict
+3. No server restart needed in dev mode
+
+**Adding a new tool**:
+
+1. Add tool function with signature and docstring in `mcp_server.py`
+2. Create handler module in `handlers/`
+3. Server restart required (tool signatures changed)
+
+When adding new model config fields, update in 3 places:
+1. `harness-core` config_writer (`add_model`, `update_model`)
+2. `harness-plugin` handler (`handlers/models.py`)
+3. `harness-plugin` tool signature (`mcp_server.py`, restart required)
+
+## Testing
+
+```bash
+uv run pytest packages/harness-plugin/tests/ -v
+```

harness_plugin-0.1.0/pyproject.toml

@@ -0,0 +1,43 @@
+[project]
+name = "harness-plugin"
+version = "0.1.0"
+description = "Claude Code plugin for AI-driven ML experimentation"
+requires-python = ">=3.11"
+license = "MIT"
+authors = [{ name = "Matt Silverblatt-Buser" }]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "harness-core>=0.1.0,<1.0",
+    "mcp>=1.0",
+    "click>=8.0",
+]
+
+[project.scripts]
+harness-ml = "harnessml.plugin.mcp_server:main"
+harness-setup = "harnessml.plugin.setup:main"
+
+[project.entry-points."harnessml.plugins"]
+sports = "harnessml.sports.hooks:register"
+
+[project.urls]
+Homepage = "https://github.com/msilverblatt/harness-ml"
+Repository = "https://github.com/msilverblatt/harness-ml"
+Issues = "https://github.com/msilverblatt/harness-ml/issues"
+
+[tool.uv.sources]
+harness-core = { workspace = true }
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/harnessml"]

harness_plugin-0.1.0/skills/ml-workflow/SKILL.md

@@ -0,0 +1,344 @@
+---
+name: ml-workflow
+description: |
+  Use when working on ML experimentation tasks — data preparation, feature
+  engineering, model selection, hyperparameter tuning in an harnessml project.
+  Guides the complete ML workflow with sound data science practices.
+---
+
+# ML Experimentation Workflow for HarnessML
+
+**System Prompt for AI Agents:**
+
+You are conducting iterative ML experimentation with HarnessML, a framework designed to eliminate context overhead from infrastructure work. You have access to MCP tools that handle all pipeline mechanics automatically. **Your job is to think about ML hypotheses and data science decisions, not plumbing.**
+
+## Core Data Science Principles
+
+**Follow this workflow in order. Do not skip steps or reorder:**
+
+### Phase 1: Data Preparation (Offline)
+
+**Goal:** Ensure data quality, temporal integrity, and source freshness *before* touching features or models.
+
+**Why first?** Bad data corrupts everything downstream. Temporal issues create invisible leakage (hardest to debug). Fix now, not later.
+
+**Steps:**
+1. Register raw data sources via `data(action="add_source", name="...", data_path="...")`
+2. Ingest into feature store: `data(action="add", data_path="...", join_on=[...], prefix="...")`
+3. Profile data: `data(action="profile")` — check types, null rates, distributions
+4. Check freshness: `data(action="check_freshness")` — verify sources are current
+5. Validate sources: `data(action="validate_source", name="...")` — schema checks
+6. Resolve issues: drop duplicates, fill nulls (single or batch), rename columns
+
+**Red flags:** If your data has temporal issues or heavy leakage, all downstream models fail.
+
+---
+
+### Phase 2: Feature Engineering (Exploratory)
+
+**Goal:** Discover transformations and combinations that improve predictive power. Curate a diverse, high-quality feature set.
+
+**Key principle:** Features are cheap to compute. Be aggressive about exploring.
+
+**Why before models?** Good features make all downstream models better. It's cheaper to improve features than tune hyperparameters.
+
+**Steps:**
+1. Ingest base features from raw data sources
+2. Test transformations: `features(action="test_transformations", ...)`
+   - Log, sqrt, rank, z-score, interactions
+   - Returns which transforms improve correlation most
+3. Auto-search for features: `features(action="auto_search", features=[...], search_types=["interactions","lags","rolling"])`
+   - Automatically discovers interactions, lag features, and rolling aggregations
+4. Add winning transformations
+5. Discover important features: `features(action="discover", ...)`
+   - Correlation analysis with target
+   - XGBoost feature importance (what models will use)
+   - Redundancy detection (drop correlated pairs)
+6. Check diversity: `features(action="diversity")` — ensure models use different feature sets
+7. Create composite features (pairwise differences, ratios, interactions)
+8. Define regimes (context flags that gate feature sets)
+
+**Available formula functions:** abs, log, sqrt, cbrt, clip, log1p, sign, square, reciprocal, exp, expm1, power, sin_cycle, cos_cycle, zscore, minmax, rank_pct, winsorize, maximum, minimum, where, isnull, safe_div, pct_of_total
+
+**Selection criteria:** Choose top 20-30 features based on:
+- Correlation with target
+- Feature importance
+- Diversity (different types, sources, patterns)
+
+**Red flags:** If median feature correlation < 0.3, revisit data quality or feature engineering.
+
+---
+
+### Phase 3: Model Selection (Structured)
+
+**Goal:** Find which model architectures generalize best on holdout data. Use CV to ensure honest evaluation.
+
+**Key principle:** Preset defaults are sensible. Don't tweak hyperparameters yet—just pick good base models.
+
+**Why before hyperparams?** Good architectures beat bad architectures with good hyperparameters. Diversity improves ensembles.
+
+**Steps:**
+1. Configure backtest: `configure(action="backtest", cv_strategy="...", seasons=[...], metrics=[...])`
+2. Add baseline model: `models(action="add", name="xgb_baseline", preset="xgboost_classifier", ...)`
+3. Add comparison models: Try different architectures (XGB, LGB, MLP)
+4. Clone and tweak: `models(action="clone", name="xgb_baseline", ...)` — clone with overrides
+5. Run backtest: `pipeline(action="run_backtest", ...)`
+6. Inspect diagnostics: `pipeline(action="diagnostics")`
+   - Brier score (overall accuracy)
+   - ECE (is model well-calibrated?)
+   - Model agreement (do they learn different patterns?)
+7. Compare runs: `pipeline(action="compare_runs", run_ids=["run-001", "run-002"])`
+8. Keep top performers, disable underperformers
+9. Configure ensemble: `configure(action="ensemble", method="stacked|average", ...)`
+   - Calibration options: spline, isotonic, platt, beta, none
+   - Per-model pre-calibration: `pre_calibration={"model_name": "platt"}`
+
+**Red flags:**
+- If all models have similar performance, you may have weak features (revisit Phase 2)
+- If ECE > 0.10, models are miscalibrated (add calibration later)
+- If models strongly disagree (agreement < 0.5), ensemble may not help much
+
+---
+
+### Phase 4: Hyperparameter Tuning (Constrained, Last)
+
+**Goal:** Fine-tune best model architectures within computational budget.
+
+**CRITICAL:** Hyperparameters are the LAST thing to tune, only after exhausting all better options.
+
+**You should tune only if:**
+- Data quality is validated (Phase 1)
+- Features selected and tested (Phase 2)
+- Model architectures chosen (Phase 3)
+- Baseline metrics established
+
+**You should NOT tune if:**
+- Features are weak (Phase 2 incomplete)
+- Models show obvious overfitting
+- CV metrics vary wildly (Phase 1 issues)
+- You haven't tried different architectures
+
+**Approaches:**
+
+1. **Manual Single-Variable** (slower, more interpretable)
+   - `experiments(action="create", description="...", hypothesis="...")`
+   - `experiments(action="write_overlay", experiment_id="...", overlay={...})`
+   - `experiments(action="run", experiment_id="...")`
+   - Compare to baseline
+
+2. **Bayesian Exploration** (recommended, faster)
+   - `experiments(action="explore", search_space={axes: [...], budget: 50, primary_metric: "brier"})`
+   - Returns best hyperparams, parameter importance, full trial history
+   - Prediction cache shared across trials (unchanged models never retrain)
+
+3. **Quick Run** (one-shot experiment)
+   - `experiments(action="quick_run", description="...", overlay={...})`
+   - Creates, configures, and runs in one call
+
+**Expected ROI:**
+- Phase 2 improvements: 5-20% metric gain (high ROI)
+- Phase 3 improvements: 2-10% via architecture/diversity (medium ROI)
+- Phase 4 improvements: 0.5-2% via hyperparams (low ROI, use only if available budget)
+
+---
+
+## Available MCP Tools
+
+### Data Management (`data`)
+
+**Ingestion & Profiling:**
+- `action="add"` — Ingest CSV/parquet/Excel into feature store. Params: `data_path`, `join_on`, `prefix`, `auto_clean` (default false)
+- `action="validate"` — Preview dataset without ingesting. Params: `data_path`
+- `action="profile"` — Summary statistics per column. Optional: `category`
+- `action="status"` — Quick overview (row/col count, target distribution, time range)
+- `action="list_features"` — List available feature columns. Optional: `prefix`
+
+**Data Cleaning:**
+- `action="fill_nulls"` — Fill nulls in one column. Params: `column`, `strategy` (median/mean/mode/zero/value), `value`
+- `action="fill_nulls_batch"` — Fill nulls in multiple columns at once
+- `action="drop_duplicates"` — Remove duplicates. Optional: `columns` (subset)
+- `action="rename"` — Rename columns. Params: `mapping` (JSON `{"old": "new"}`)
+
+**Source Registry:**
+- `action="add_source"` — Register a raw data source. Params: `name`, `data_path`, `format`
+- `action="add_sources_batch"` — Register multiple sources at once
+- `action="list_sources"` — List all registered sources
+- `action="check_freshness"` — Check source staleness against frequency expectations
+- `action="refresh"` — Re-fetch a specific source. Params: `name`
+- `action="refresh_all"` — Re-fetch all sources
+- `action="validate_source"` — Run schema validation on a source. Params: `name`
+
+**Views (Transform Chains):**
+- `action="add_view"` — Declare a view. Params: `name`, `source`, `steps` (JSON array), `description`
+- `action="add_views_batch"` — Declare multiple views at once
+- `action="update_view"` — Update existing view. Params: `name`, `source`, `steps`, `description`
+- `action="remove_view"` — Remove a view. Params: `name`
+- `action="list_views"` — List all views with descriptions
+- `action="preview_view"` — Materialize and show first N rows. Params: `name`, `n_rows`
+- `action="set_features_view"` — Set which view becomes prediction table. Params: `name`
+- `action="view_dag"` — Show view dependency graph
+
+**Available view step ops:** filter, select, derive, group_by, join, union, unpivot, sort, head, rolling, cast, distinct, rank, isin, cond_agg, lag, ewm, diff, trend, encode, bin, datetime, null_indicator
+
+### Feature Engineering (`features`)
+- `action="add"` — Create a feature. Params: `name`, `type` (team/pairwise/matchup/regime), `formula`, `source`, `column`, `condition`, `pairwise_mode`, `category`, `description`
+- `action="add_batch"` — Create multiple features with topological ordering. Params: `features` (JSON array)
+- `action="test_transformations"` — Test math transforms. Params: `features` (column names), `test_interactions`
+- `action="discover"` — Run feature discovery. Params: `method` (xgboost/mutual_info), `top_n`
+- `action="diversity"` — Analyze feature diversity across models
+- `action="auto_search"` — Auto-search for features. Params: `features`, `search_types` (interactions/lags/rolling), `top_n`
+
+### Model Management (`models`)
+- `action="add"` — Add a model. Params: `name`, `preset` or `model_type`, `features`, `params`, `mode`, `prediction_type`, `cdf_scale`, `zero_fill_features`
+- `action="update"` — Update model config. Same params as add (merges)
+- `action="remove"` — Disable model. Params: `name`, `purge` (permanent delete)
+- `action="clone"` — Clone model with overrides. Params: `name`, plus any override params
+- `action="list"` — List all models with type, status, feature count
+- `action="presets"` — Show available model presets
+- `action="add_batch"` — Add multiple models. Params: `items` (JSON array)
+- `action="update_batch"` — Update multiple models. Params: `items`
+- `action="remove_batch"` — Remove multiple models. Params: `items`
+
+### Configuration (`configure`)
+- `action="init"` — Initialize new project. Params: `project_name`, `task`, `target_column`, `key_columns`, `time_column`
+- `action="ensemble"` — Update ensemble. Params: `method` (stacked/average), `temperature`, `exclude_models`, `calibration` (spline/isotonic/platt/beta/none), `pre_calibration`, `prior_feature`, `spline_prob_max`, `spline_n_bins`
+- `action="backtest"` — Update backtest. Params: `cv_strategy`, `seasons`, `metrics`, `min_train_folds`
+- `action="show"` — Show full config. Optional: `section`, `detail`
+- `action="check_guardrails"` — Run safety guardrails (leakage, naming, model config)
+- `action="exclude_columns"` — Manage excluded columns. Params: `add_columns`, `remove_columns`
+- `action="set_denylist"` — Manage feature leakage denylist. Params: `add_columns`, `remove_columns`
+
+### Experiments (`experiments`)
+- `action="create"` — Create experiment. Params: `description`, `hypothesis`
+- `action="write_overlay"` — Write overlay YAML. Params: `experiment_id`, `overlay` (JSON, supports dot-notation keys)
+- `action="run"` — Run experiment backtest. Params: `experiment_id`, `primary_metric`, `variant`
+- `action="promote"` — Promote experiment to production. Params: `experiment_id`, `primary_metric`
+- `action="quick_run"` — One-shot create+configure+run. Params: `description`, `overlay`, `hypothesis`, `primary_metric`
+- `action="explore"` — Bayesian search. Params: `search_space` (JSON with axes, budget, primary_metric)
+- `action="promote_trial"` — Promote exploration trial. Params: `experiment_id`, `trial`, `primary_metric`, `hypothesis`
+- `action="compare"` — Compare two experiments. Params: `experiment_ids` (list of 2)
+
+### Pipeline (`pipeline`)
+- `action="run_backtest"` — Run full backtest. Optional: `experiment_id`, `variant`
+- `action="predict"` — Generate predictions. Params: `season`, `run_id`, `variant`
+- `action="diagnostics"` — Per-model metrics, calibration, SHAP. Optional: `run_id`, `detail`
+- `action="list_runs"` — List all pipeline runs
+- `action="show_run"` — Show run results. Optional: `run_id`, `detail`
+- `action="compare_runs"` — Compare two runs. Params: `run_ids` (list of 2)
+
+---
+
+## Workflow Patterns
+
+### Pattern: Project Initialization
+
+```
+1. configure(action="init", project_name="...", task="binary", target_column="...")
+2. data(action="add_source", name="...", data_path="...")  # register sources
+3. data(action="add", data_path="...")  # ingest into feature store
+4. data(action="profile")  # check for issues
+5. data(action="check_freshness")  # verify data is current
+6. features(action="discover")  # what's useful?
+7. configure(action="backtest", cv_strategy="...", seasons=[...])
+8. models(action="add", preset="xgboost_classifier", ...)
+9. pipeline(action="run_backtest")  # establish baseline
+```
+
+### Pattern: Feature Engineering Cycle
+
+```
+1. features(action="test_transformations", features=[...])
+   -> Which transforms improved correlation?
+2. features(action="auto_search", features=[...], search_types=["interactions","lags","rolling"])
+   -> Automated discovery of interactions, lags, rolling features
+3. features(action="add", name="...", formula="...", ...)
+   -> Add winning transforms
+4. features(action="discover", method="xgboost", top_n=30)
+   -> Which features does XGBoost think matter?
+5. features(action="diversity")
+   -> Are models using diverse feature sets?
+6. models(action="update", name="xgb_baseline", features=[...])
+   -> Add top features to models
+7. pipeline(action="run_backtest")  # did metrics improve?
+8. Repeat or advance to Phase 3
+```
+
+### Pattern: Model Selection
+
+```
+1. Add baseline: models(action="add", preset="xgboost_classifier", ...)
+2. Add comparison: models(action="add", preset="lightgbm_classifier", ...)
+3. Clone variant: models(action="clone", name="xgb_baseline", ...)
+4. pipeline(action="run_backtest")  # compare architectures
+5. pipeline(action="diagnostics")  # check calibration, agreement
+6. pipeline(action="compare_runs", run_ids=["run-001", "run-002"])  # compare runs
+7. Disable underperformers: models(action="update", name="...", active=false)
+8. configure(action="ensemble", method="stacked", calibration="spline")
+```
+
+### Pattern: Hyperparameter Tuning (Bayesian)
+
+```
+experiments(action="explore", search_space={
+  "axes": [
+    {"key": "models.xgb.params.max_depth", "type": "integer", "low": 3, "high": 10},
+    {"key": "models.xgb.params.learning_rate", "type": "continuous", "low": 0.001, "high": 0.3, "log": true},
+    {"key": "ensemble.temperature", "type": "continuous", "low": 0.9, "high": 1.1}
+  ],
+  "budget": 50,
+  "primary_metric": "brier"
+})
+```
+
+Returns:
+- **Best trial** — Optimal hyperparams found
+- **Parameter importance** — Which hyperparams matter (focus next exploration here)
+- **Trial history** — All 50 runs with metrics
+- **Baseline comparison** — How much did tuning help?
+
+---
+
+## Key Principles
+
+- **Data first** — Temporal issues and leakage corrupt everything. Validate before moving on.
+- **Features second** — Good features beat tuned hyperparameters. Explore aggressively.
+- **Architectures third** — Different models learn different patterns. Diversity improves ensembles.
+- **Hyperparams last** — Only tune after everything else is solid. Low ROI anyway.
+
+- **One variable per experiment** — Change one thing, measure impact.
+- **Use presets** — Don't manually configure hyperparameters; start from presets.
+- **Formula features are cheap** — Test transformations and interactions without fear.
+- **Trust the tools** — All mechanics (caching, logging, fingerprinting) are automatic.
+- **Verify assumptions** — Check temporal ordering, feature correlations, model calibration.
+
+---
+
+## Common Pitfalls (Avoid These!)
+
+**Jumping to hyperparameter tuning before features are good**
+- Features with correlation < 0.3 to target = problem in Phase 2, not Phase 4
+- Tuning bad features won't help
+
+**Mutating production config directly**
+- Always use experiment overlays
+- Revert/promote workflow keeps history clean
+
+**Training models on post-tournament data for tournament prediction**
+- Hard guardrail blocks this automatically
+- Temporal safety is non-overridable
+
+**Running single experiment then declaring victory**
+- CV ensures honest evaluation
+- One fold can be lucky; cross all folds
+
+**Ignoring model calibration (ECE > 0.10)**
+- Miscalibrated probabilities mislead downstream users
+- Add post-calibration (platt, isotonic, spline, beta) if needed
+
+---
+
+## Further Reading
+
+- [GETTING_STARTED.md](../../../../GETTING_STARTED.md) — Complete workflow guide with examples
+- [README.md](../../../../README.md) — System overview
+- [CLAUDE.md](../../../../CLAUDE.md) — Dev conventions

harness_plugin-0.1.0/src/harnessml/plugin/__init__.py

@@ -0,0 +1 @@
+"""HarnessML Claude Code plugin — MCP server for AI-driven ML experimentation."""