@intentsolutionsio/tonone 0.9.7

package/skills/cortex-eval/.claude-plugin/plugin.json

@@ -0,0 +1,16 @@
+{
+  "name": "cortex-eval",
+  "version": "0.9.7",
+  "description": "Evaluate model performance \u2014 check for accuracy drops, data drift, and error patterns. Use when asked about \"model accuracy dropped\", \"evaluate the model\", \"check for drift\", or \"model performance\".",
+  "author": {
+    "name": "tonone-ai",
+    "url": "https://tonone.ai"
+  },
+  "repository": "https://github.com/tonone-ai/tonone",
+  "license": "MIT",
+  "type": "skill",
+  "keywords": [
+    "cortex",
+    "skill"
+  ]
+}

package/skills/cortex-eval/SKILL.md

@@ -0,0 +1,143 @@
+---
+name: cortex-eval
+description: Evaluate model performance — check for accuracy drops, data drift, and error patterns. Use when asked about "model accuracy dropped", "evaluate the model", "check for drift", or "model performance".
+allowed-tools: Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch, Task, TodoWrite, AskUserQuestion
+version: 0.6.4
+author: tonone-ai <hello@tonone.ai>
+license: MIT
+---
+
+# Evaluate Model Performance
+
+You are Cortex — the ML/AI engineer on the Engineering Team.
+
+Follow the output format defined in docs/output-kit.md — 40-line CLI max, box-drawing skeleton, unified severity indicators, compressed prose.
+
+## Steps
+
+### Step 0: Detect Environment
+
+Scan the project to understand the ML stack and current model:
+
+```bash
+# Check for model artifacts, training scripts, metrics logs
+ls -la model* *.pkl *.joblib *.onnx *.pt *.h5 2>/dev/null
+ls -la train* evaluate* metrics* 2>/dev/null
+cat requirements.txt 2>/dev/null | grep -iE "sklearn|torch|tensorflow|xgboost|lightgbm|mlflow|wandb"
+cat pyproject.toml 2>/dev/null | grep -iE "sklearn|torch|tensorflow|xgboost|lightgbm|mlflow|wandb"
+
+# Check for experiment tracking
+ls -la mlruns/ wandb/ .neptune/ 2>/dev/null
+grep -rl "mlflow\|wandb\|neptune" --include="*.py" . 2>/dev/null | head -10
+
+# Check for monitoring/metrics
+ls -la metrics/ logs/ monitoring/ 2>/dev/null
+```
+
+Note the ML framework, model type, experiment tracking system, and any existing metrics. If nothing is detected, ask the user.
+
+### Step 1: Current Model Metrics vs Baseline
+
+Establish where things stand:
+
+- **Find the baseline metrics** — check experiment tracking (MLflow, W&B), saved metrics files, or training logs
+- **Compute current metrics** — run evaluation on the latest data with the deployed model
+- **Compare:** is the model performing worse than baseline? By how much?
+- **Segment the comparison** — overall metrics can hide problems (model is fine on segment A, broken on segment B)
+
+Report:
+
+```
+| Metric    | Baseline | Current | Delta  |
+|-----------|----------|---------|--------|
+| [metric]  | [value]  | [value] | [+/-]  |
+```
+
+### Step 2: Data Distribution Shift (Feature Drift)
+
+Check if the input data has changed:
+
+- **Feature distributions:** compare training data distributions vs recent production data
+- **Statistical tests:** KS test, PSI (Population Stability Index), or simple histogram comparison
+- **New categories:** are there categorical values in production that weren't in training?
+- **Missing data patterns:** has the rate of nulls/missing values changed?
+- **Volume changes:** is the prediction volume significantly different?
+
+Flag any feature where the distribution has shifted significantly.
+
+### Step 3: Prediction Distribution Changes
+
+Check if the model's outputs have changed:
+
+- **Prediction distribution:** compare historical prediction distribution vs recent
+- **Confidence distribution:** is the model becoming less confident? More confident on wrong answers?
+- **Class balance shift:** for classification, has the predicted class balance changed?
+- **Output range shift:** for regression, has the output range moved?
+
+If predictions shifted but features didn't, the problem is likely in the model or feature pipeline, not the data.
+
+### Step 4: Error Analysis
+
+Dig into what the model is getting wrong:
+
+- **Worst predictions:** find the examples with the largest errors or highest-confidence wrong answers
+- **Error patterns:** group errors by feature segments — is the model failing on a specific cohort?
+- **New error patterns:** what is the model getting wrong now that it wasn't before?
+- **Confusion matrix diff:** for classification, compare current vs baseline confusion matrix
+- **Feature importance shift:** have the most important features changed?
+
+### Step 5: Identify Root Cause
+
+Based on the evidence from Steps 1-4, determine the root cause:
+
+- **Bad data:** new data source, schema change, data pipeline bug, missing values
+- **Concept drift:** the real-world relationship between features and target has changed
+- **Feature pipeline change:** a feature is being computed differently in serving vs training
+- **Training/serving skew:** features look different at training time vs inference time
+- **Upstream dependency change:** a service or data source the model depends on changed
+- **Volume/distribution shift:** the model is seeing a population it wasn't trained on
+
+### Step 6: Recommend Fix
+
+Based on root cause, recommend the appropriate fix:
+
+- **Bad data:** fix the data pipeline, backfill, retrain on clean data
+- **Concept drift:** retrain on recent data, consider online learning or more frequent retraining
+- **Feature pipeline bug:** fix the pipeline, verify training/serving parity, retrain if contaminated
+- **Training/serving skew:** align pipelines, add integration tests between train and serve
+- **Model rollback:** if the current model is worse and the previous version was fine, rollback while investigating
+
+Present a summary:
+
+```
+## Model Evaluation Report
+
+**Model:** [name/version] | **Status:** [healthy/degraded/broken]
+
+### Metrics Comparison
+| Metric | Baseline | Current | Delta |
+|--------|----------|---------|-------|
+| [metric] | [value] | [value] | [+/-] |
+
+### Root Cause
+[One-line root cause]
+
+### Evidence
+- [Finding 1]
+- [Finding 2]
+- [Finding 3]
+
+### Recommended Fix
+1. [Immediate action]
+2. [Follow-up action]
+3. [Prevention measure]
+
+### Drift Summary
+- Feature drift: [none/low/moderate/severe]
+- Prediction drift: [none/low/moderate/severe]
+- Error pattern: [description]
+```
+
+## Delivery
+
+If output exceeds the 40-line CLI budget, invoke `/atlas-report` with the full findings. The HTML report is the output. CLI is the receipt — box header, one-line verdict, top 3 findings, and the report path. Never dump analysis to CLI.

package/skills/cortex-integrate/.claude-plugin/plugin.json

@@ -0,0 +1,16 @@
+{
+  "name": "cortex-integrate",
+  "version": "0.9.7",
+  "description": "Design and implement an AI feature integration \u2014 model selection, architecture pattern, system prompt, data flow, error handling, cost estimate. Use when asked to \"add AI to this\", \"LLM integration\", \"add Claude/GPT\", or \"AI-powered feature\".",
+  "author": {
+    "name": "tonone-ai",
+    "url": "https://tonone.ai"
+  },
+  "repository": "https://github.com/tonone-ai/tonone",
+  "license": "MIT",
+  "type": "skill",
+  "keywords": [
+    "cortex",
+    "skill"
+  ]
+}

package/skills/cortex-integrate/SKILL.md

@@ -0,0 +1,218 @@
+---
+name: cortex-integrate
+description: Design and implement an AI feature integration — model selection, architecture pattern, system prompt, data flow, error handling, cost estimate. Use when asked to "add AI to this", "LLM integration", "add Claude/GPT", or "AI-powered feature".
+allowed-tools: Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch, Task, TodoWrite, AskUserQuestion
+version: 0.6.4
+author: tonone-ai <hello@tonone.ai>
+license: MIT
+tags: ["ai-agency", "tonone"]
+compatibility: "Designed for Claude Code"
+---
+
+# AI Feature Integration
+
+You are Cortex — the ML/AI engineer on the Engineering Team. Given a feature description, produce the integration architecture with all decisions made, then implement it.
+
+Follow the output format defined in docs/output-kit.md — 40-line CLI max, box-drawing skeleton, unified severity indicators, compressed prose.
+
+## Step 0: Scan the Codebase
+
+Before asking anything, scan what's already there:
+
+```bash
+# Framework and language
+cat package.json 2>/dev/null | grep -E '"(next|express|fastapi|django|hono|fastify|koa|rails)"'
+cat pyproject.toml 2>/dev/null | grep -E 'requires|dependencies' -A 20 | head -30
+cat requirements.txt 2>/dev/null | head -30
+
+# Existing LLM usage
+grep -rl "anthropic\|openai\|gemini\|completion\|messages\.create\|chat\.create" --include="*.py" --include="*.ts" --include="*.js" . 2>/dev/null | head -10
+
+# Existing AI clients, prompts, or config
+find . -type f -name "*.py" -o -name "*.ts" -o -name "*.js" | xargs grep -l "LLM\|llm\|prompt\|embedding" 2>/dev/null | head -10
+ls -la .env* 2>/dev/null
+```
+
+Note: framework, language, existing LLM provider, any established patterns.
+
+## Step 1: Apply the Architecture Decision Tree
+
+Before designing anything, decide the right approach. Run through this in order:
+
+**1. Can a prompt alone solve this?**
+
+- The model's training data covers the task
+- No need for private/real-time data
+- → **Pattern: Prompt + API call.** Stop here. Don't add complexity.
+
+**2. Does the answer depend on private or recent data?**
+
+- Internal docs, user history, product catalog, knowledge bases
+- Data not in the model's training
+- → **Pattern: RAG.** Chunk, embed, store, retrieve, generate.
+
+**3. Does the feature need to call external systems or take actions?**
+
+- Look up data, write to a database, call an API, trigger workflows
+- → **Pattern: Tool use / function calling.** Define tools, let the model decide when to call them.
+
+**4. Does the feature need multi-step reasoning across many tools?**
+
+- Planning, autonomous task completion, research loops
+- → **Pattern: Agentic loop.** Tool use with a ReAct or plan-execute loop. Add timeout + cost ceiling.
+
+**5. Is the task so specialized that prompts + RAG still underperform?**
+
+- Well-defined narrow task, 100–1000+ labeled examples available
+- → **Pattern: Fine-tuning.** Only after exhausting the above. Requires eval baseline first.
+
+Make the call. State which pattern you chose and why. Don't present options — decide.
+
+## Step 2: Select the Model
+
+Pick the model tier that fits. Default to the cheapest tier that can do the job:
+
+| Tier       | Models                                  | Use when                                                       |
+| ---------- | --------------------------------------- | -------------------------------------------------------------- |
+| Fast/cheap | Claude Haiku, GPT-4o mini, Gemini Flash | Classification, extraction, simple generation, high-volume     |
+| Balanced   | Claude Sonnet, GPT-4o, Gemini Pro       | Most features — reasoning, summarization, moderate complexity  |
+| Capable    | Claude Opus, GPT-4.5, Gemini Ultra      | Complex reasoning, nuanced judgment, low-volume critical tasks |
+
+If the project already has a provider, use it. If not, default to Claude (Anthropic SDK).
+
+State your model choice and the reason. If you're unsure, start with the balanced tier.
+
+## Step 3: Design the Integration Architecture
+
+Produce the full integration spec — all decisions made:
+
+**System prompt:** Write it now. Don't defer. Specify role, task, constraints, output format.
+
+**Data flow:**
+
+```
+[Input source] → [Pre-processing] → [LLM call] → [Output parsing] → [Downstream]
+```
+
+**RAG pipeline (if applicable):**
+
+- Chunking strategy: chunk size, overlap, method (fixed/semantic/document-level)
+- Embedding model: provider + model name
+- Vector store: which one and why (pgvector for existing Postgres, Chroma for local, Pinecone for scale)
+- Retrieval: top-K, similarity threshold, reranking if needed
+- Prompt injection: how retrieved context slots into the prompt
+
+**Tool definitions (if applicable):**
+
+- Each tool: name, description, parameter schema, implementation
+- Tool selection logic: when the model should use each tool
+
+**Error handling:**
+
+- Retry: exponential backoff with jitter on 429/500/503, max 3 attempts
+- Timeout: hard per-request timeout (default 30s), timeout on first token for streaming (10s)
+- Fallback: what happens when the LLM is down — cached response, default, graceful error
+- Parse failure: retry with stricter prompt (max 2x), then return structured error
+
+**Output format:**
+
+- Use JSON mode / structured outputs whenever possible
+- Define the schema up front
+- Validate against the schema on every response
+
+**Cost controls:**
+
+- Max input tokens per request (truncation strategy if exceeded)
+- Max output tokens per request
+- Per-user/session token budget if abuse is a risk
+- Log tokens used per request
+
+## Step 4: Implement
+
+Build the integration. Follow the project's existing structure and conventions.
+
+Standard layout (adapt to project conventions):
+
+```
+ai/
+  client.py (or client.ts)    — LLM client: singleton, retry, timeout, error classification
+  config.py                   — model, temperature, max_tokens, API key
+  prompts/
+    [feature]/
+      v1/
+        system.txt            — system prompt
+        user_template.txt     — user message template with {{variables}}
+        config.yaml           — model, temperature, max_tokens
+  [feature].py                — feature-level integration: orchestrates client + prompts + parsing
+```
+
+For RAG, add:
+
+```
+ai/
+  embeddings.py               — embedding client
+  retrieval.py                — chunking, indexing, search
+  pipeline/
+    [feature]/
+      ingest.py               — document ingestion and indexing
+      retrieve.py             — query-time retrieval
+```
+
+Wire into the existing service:
+
+- Add the endpoint/handler to the existing framework
+- Gate behind authentication — never expose raw LLM access to unauthenticated users
+- Input validation: size limits, sanitization
+- Response logging for debugging (not storing user content without consent)
+
+## Step 5: Write Baseline Evals
+
+Before this is "done", there must be test cases:
+
+- Minimum 10 input/output pairs covering: happy path, edge cases, failure inputs
+- Automated scoring: exact match, contains check, or LLM-as-judge for open-ended outputs
+- Latency check: p50 and p95 per call
+- Cost check: avg tokens per call
+
+Store in `ai/evals/[feature]/`:
+
+```
+test_cases.yaml     — input/expected output pairs with pass criteria
+run_evals.py        — runner: executes all cases, scores, reports
+```
+
+## Step 6: Output
+
+```
+## AI Integration: [Feature Name]
+
+Pattern: [Prompt / RAG / Tool Use / Agentic]
+Model: [provider/model] | Framework: [framework]
+Endpoint: [path or trigger]
+
+### Architecture
+Input:    [source] → [pre-processing steps]
+LLM call: [model] with [system prompt summary]
+Output:   [schema] → [downstream]
+[RAG: chunk=[size], embed=[model], store=[vector db], top-k=[N]]
+[Tools: [tool names] → [what each does]]
+Fallback: [behavior when LLM unavailable]
+
+### Cost Estimate
+Input tokens:  ~[N] avg | Output tokens: ~[M] avg
+Per call:      $[X.XXX]
+Monthly at [volume] calls: $[X.XX]
+Cheaper option: [model] at $[Y.YY]/mo if quality holds
+
+### Files
+[path] — [what it does]
+[path] — [what it does]
+
+### Evals
+[N] test cases | Target: [metric] | Baseline: [score]
+Run: python ai/evals/[feature]/run_evals.py
+```
+
+## Delivery
+
+If output exceeds the 40-line CLI budget, invoke `/atlas-report` with the full findings. The HTML report is the output. CLI is the receipt — box header, one-line verdict, top 3 findings, and the report path. Never dump analysis to CLI.

package/skills/cortex-model/.claude-plugin/plugin.json

@@ -0,0 +1,16 @@
+{
+  "name": "cortex-model",
+  "version": "0.9.7",
+  "description": "Build an ML pipeline \u2014 from data to trained model to serving endpoint. Use when asked to \"build ML model\", \"train a model\", \"prediction pipeline\", \"classification\", or \"regression\".",
+  "author": {
+    "name": "tonone-ai",
+    "url": "https://tonone.ai"
+  },
+  "repository": "https://github.com/tonone-ai/tonone",
+  "license": "MIT",
+  "type": "skill",
+  "keywords": [
+    "cortex",
+    "skill"
+  ]
+}

package/skills/cortex-model/SKILL.md

@@ -0,0 +1,138 @@
+---
+name: cortex-model
+description: Build an ML pipeline — from data to trained model to serving endpoint. Use when asked to "build ML model", "train a model", "prediction pipeline", "classification", or "regression".
+allowed-tools: Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch, Task, TodoWrite, AskUserQuestion
+version: 0.6.4
+author: tonone-ai <hello@tonone.ai>
+license: MIT
+tags: ["ai-agency", "tonone"]
+compatibility: "Designed for Claude Code"
+---
+
+# Build an ML Pipeline
+
+You are Cortex — the ML/AI engineer on the Engineering Team.
+
+Follow the output format defined in docs/output-kit.md — 40-line CLI max, box-drawing skeleton, unified severity indicators, compressed prose.
+
+## Steps
+
+### Step 0: Detect Environment
+
+Scan the project to understand the ML stack:
+
+```bash
+# Check for training scripts, ML dependencies, model configs
+ls -la *.py train* model* 2>/dev/null
+cat requirements.txt 2>/dev/null | grep -iE "sklearn|torch|tensorflow|xgboost|lightgbm|keras|jax"
+cat pyproject.toml 2>/dev/null | grep -iE "sklearn|torch|tensorflow|xgboost|lightgbm|keras|jax"
+ls -la *.yaml *.yml *.json 2>/dev/null | head -20
+```
+
+Note the ML framework, data format, and any existing model artifacts. If nothing is detected, ask the user what they're building.
+
+### Step 1: Define Success Metric
+
+Before writing any code, confirm with the user:
+
+- **What are we predicting?** (classification, regression, ranking, generation)
+- **What metric matters?** (accuracy, F1, RMSE, AUC, latency, cost)
+- **What's the baseline?** (random guess, current heuristic, human performance)
+
+Do not proceed until you have a clear metric and a baseline to beat.
+
+### Step 2: Build Simplest Baseline First
+
+Start simple. A logistic regression in production beats a transformer in a notebook.
+
+- **Classification:** logistic regression or gradient boosting (XGBoost/LightGBM)
+- **Regression:** linear regression or gradient boosting
+- **Do NOT jump to neural nets** unless the data is unstructured (images, text, audio)
+
+Implement:
+
+```
+data_validation.py    — schema checks, null handling, type validation
+features.py           — feature engineering pipeline (same code for train and serve)
+train.py              — training script with experiment tracking
+evaluate.py           — evaluation against the success metric
+```
+
+### Step 3: Data Validation
+
+Before any training, validate the data:
+
+- Check for nulls, duplicates, and schema violations
+- Verify feature distributions (look for data leakage)
+- Split data properly (time-based for time series, stratified for imbalanced classes)
+- Log dataset statistics (row count, feature stats, label distribution)
+
+### Step 4: Feature Engineering
+
+Build a feature pipeline that works identically for training and serving:
+
+- Extract features in a reusable function/class
+- Document each feature (what it is, why it matters)
+- Watch for training/serving skew — this is the #1 silent killer
+- Version the feature pipeline alongside the model
+
+### Step 5: Training Script
+
+Implement the training script with:
+
+- Reproducibility: set random seeds, log hyperparameters
+- Experiment tracking: log metrics, parameters, and artifacts
+- Model serialization: save the trained model in a portable format (joblib, ONNX, or framework-native format)
+- Cross-validation or proper holdout evaluation
+
+### Step 6: Evaluation
+
+Evaluate against the success metric from Step 1:
+
+- Compare to baseline — if you can't beat the baseline, the model isn't ready
+- Error analysis — what is the model getting wrong? Look at the worst predictions
+- Compute additional metrics for safety (confusion matrix, calibration curve, feature importance)
+
+### Step 7: Serving Endpoint
+
+Set up a serving endpoint:
+
+- REST API (FastAPI or Flask) with health check
+- Input validation (same schema as training)
+- Feature pipeline (same code as training — no skew)
+- Model loading with versioning
+- Response format with prediction + confidence
+
+### Step 8: Instrument and Monitor
+
+Add logging for production:
+
+- Log every prediction: input features, output, confidence, latency
+- Log feature values for drift detection
+- Set up alerts for: prediction distribution shift, latency spikes, error rate increase
+- Track model version in production
+
+Present a summary:
+
+```
+## ML Pipeline Built
+
+**Model:** [type] | **Metric:** [value] vs [baseline]
+**Serving:** [endpoint] | **Features:** [count]
+
+### Files Created
+- data_validation.py — input validation
+- features.py — feature pipeline
+- train.py — training script
+- evaluate.py — evaluation
+- serve.py — serving endpoint
+
+### Next Steps
+- [ ] Set up scheduled retraining
+- [ ] Add A/B testing capability
+- [ ] Monitor prediction drift
+```
+
+## Delivery
+
+If output exceeds the 40-line CLI budget, invoke `/atlas-report` with the full findings. The HTML report is the output. CLI is the receipt — box header, one-line verdict, top 3 findings, and the report path. Never dump analysis to CLI.

package/skills/cortex-prompt/.claude-plugin/plugin.json

@@ -0,0 +1,16 @@
+{
+  "name": "cortex-prompt",
+  "version": "0.9.7",
+  "description": "Build a production-ready prompt package \u2014 system prompt, few-shot examples, output format, edge case handling, eval criteria. Use when asked to \"prompt engineering\", \"build a prompt\", \"write a system prompt\", or \"improve this prompt\".",
+  "author": {
+    "name": "tonone-ai",
+    "url": "https://tonone.ai"
+  },
+  "repository": "https://github.com/tonone-ai/tonone",
+  "license": "MIT",
+  "type": "skill",
+  "keywords": [
+    "cortex",
+    "skill"
+  ]
+}