@monoes/monomindcli 2.15.2 → 2.15.4

package/org-skills/security-reviewer/references/sast-tools.md

@@ -0,0 +1,117 @@
+# SAST Tools
+
+## JavaScript/TypeScript
+
+```bash
+# Dependency vulnerabilities
+npm audit
+npm audit --json > npm-audit.json
+
+# ESLint security plugin
+npm install eslint-plugin-security --save-dev
+npx eslint --ext .js,.ts . --plugin security
+
+# Snyk
+npx snyk test
+npx snyk code test
+```
+
+## Python
+
+```bash
+# Bandit - Python SAST
+pip install bandit
+bandit -r . -f json -o bandit-report.json
+bandit -r . -ll  # Only high severity
+
+# Safety - Dependency check
+pip install safety
+safety check
+safety check -r requirements.txt --json > safety-report.json
+
+# Pyup Safety
+pip install pyupio-safety
+pyupio-safety check
+```
+
+## Go
+
+```bash
+# GoSec - Go security checker
+go install github.com/securego/gosec/v2/cmd/gosec@latest
+gosec ./...
+gosec -fmt=json -out=gosec-report.json ./...
+
+# Go vulnerability database
+go install golang.org/x/vuln/cmd/govulncheck@latest
+govulncheck ./...
+```
+
+## Multi-Language Tools
+
+```bash
+# Semgrep - Universal SAST
+pip install semgrep
+semgrep --config=auto .
+semgrep --config=p/security-audit .
+semgrep --config=p/owasp-top-ten .
+
+# Trivy - Comprehensive scanner
+brew install trivy
+trivy fs .
+trivy fs --security-checks vuln,secret,config .
+
+# SonarQube (requires server)
+sonar-scanner -Dsonar.projectKey=myproject
+```
+
+## CI/CD Integration
+
+### GitHub Actions
+
+```yaml
+- name: Run Semgrep
+  uses: returntocorp/semgrep-action@v1
+  with:
+    config: p/security-audit
+
+- name: Run npm audit
+  run: npm audit --audit-level=high
+
+- name: Run Trivy
+  uses: aquasecurity/trivy-action@master
+  with:
+    scan-type: 'fs'
+    severity: 'CRITICAL,HIGH'
+```
+
+### GitLab CI
+
+```yaml
+security-scan:
+  image: returntocorp/semgrep
+  script:
+    - semgrep --config=auto --json -o semgrep.json .
+  artifacts:
+    reports:
+      sast: semgrep.json
+```
+
+## Quick Reference
+
+| Language | Primary Tool | Dependency Check |
+|----------|--------------|------------------|
+| JavaScript | ESLint + security | npm audit |
+| TypeScript | ESLint + security | npm audit |
+| Python | Bandit | Safety |
+| Go | GoSec | govulncheck |
+| Java | SpotBugs | OWASP Dependency-Check |
+| Ruby | Brakeman | bundler-audit |
+
+| Tool | Strengths | Best For |
+|------|-----------|----------|
+| Semgrep | Multi-language, custom rules | General SAST |
+| Trivy | Container + code + secrets | Comprehensive |
+| Bandit | Python-specific | Python projects |
+| GoSec | Go-specific | Go projects |
+| npm audit | Built-in, fast | Node.js deps |

package/org-skills/security-reviewer/references/secret-scanning.md

@@ -0,0 +1,125 @@
+# Secret Scanning
+
+## Gitleaks
+
+```bash
+# Install
+brew install gitleaks
+
+# Scan current directory
+gitleaks detect --source . --verbose
+
+# Scan with report
+gitleaks detect --source . -f json -r gitleaks-report.json
+
+# Scan git history
+gitleaks detect --source . --log-opts="--all"
+
+# Use baseline (ignore known)
+gitleaks detect --baseline-path .gitleaks-baseline.json
+```
+
+## TruffleHog
+
+```bash
+# Install
+pip install trufflehog
+
+# Scan filesystem
+trufflehog filesystem .
+
+# Scan git repo
+trufflehog git file://. --since-commit HEAD~100
+
+# Scan with JSON output
+trufflehog filesystem . --json > trufflehog-report.json
+```
+
+## Manual Grep Patterns
+
+```bash
+# Common secret patterns
+grep -rn "api_key\|apikey\|api-key" --include="*.{ts,js,py}" .
+grep -rn "secret\|password\|passwd" --include="*.{ts,js,py}" .
+grep -rn "private_key\|privatekey" --include="*.{ts,js,py}" .
+grep -rn "access_token\|accesstoken" --include="*.{ts,js,py}" .
+
+# AWS credentials
+grep -rn "AKIA[0-9A-Z]{16}" .
+grep -rn "aws_secret_access_key" .
+
+# Base64 encoded (potential secrets)
+grep -rn "[A-Za-z0-9+/]{40,}=" .
+
+# JWT tokens
+grep -rn "eyJ[A-Za-z0-9_-]*\.eyJ[A-Za-z0-9_-]*\." .
+```
+
+## Common Secret Patterns
+
+| Type | Pattern | Example |
+|------|---------|---------|
+| AWS Access Key | `AKIA[0-9A-Z]{16}` | AKIAIOSFODNN7EXAMPLE |
+| AWS Secret Key | 40 char base64 | wJalrXUtnFEMI/K7MDENG... |
+| GitHub Token | `ghp_[A-Za-z0-9]{36}` | ghp_xxxxxxxxxxxx |
+| Slack Token | `xox[baprs]-` | xoxb-xxx-xxx |
+| Stripe Key | `sk_live_[A-Za-z0-9]{24}` | sk_live_xxxx |
+| Private Key | `-----BEGIN.*PRIVATE KEY-----` | RSA/EC keys |
+| JWT | `eyJ[A-Za-z0-9_-]*\.eyJ` | Encoded tokens |
+
+## Pre-commit Hook
+
+```yaml
+# .pre-commit-config.yaml
+repos:
+  - repo: https://github.com/gitleaks/gitleaks
+    rev: v8.18.0
+    hooks:
+      - id: gitleaks
+```
+
+## CI/CD Integration
+
+```yaml
+# GitHub Actions
+- name: Gitleaks
+  uses: gitleaks/gitleaks-action@v2
+  env:
+    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+# GitLab CI
+secret_detection:
+  image: zricethezav/gitleaks
+  script:
+    - gitleaks detect --source . -f sarif -r gl-secret-detection-report.sarif
+  artifacts:
+    reports:
+      secret_detection: gl-secret-detection-report.sarif
+```
+
+## Remediation Steps
+
+1. **Rotate immediately** - Consider secret compromised
+2. **Remove from history** - Use git filter-branch or BFG
+3. **Add to .gitignore** - Prevent future commits
+4. **Use env variables** - Move to environment
+5. **Use secret manager** - AWS Secrets Manager, Vault
+
+```bash
+# Remove from git history (BFG)
+bfg --replace-text passwords.txt repo.git
+
+# Or git filter-branch
+git filter-branch --force --index-filter \
+  "git rm --cached --ignore-unmatch path/to/secret" \
+  --prune-empty --tag-name-filter cat -- --all
+```
+
+## Quick Reference
+
+| Tool | Best For | Speed |
+|------|----------|-------|
+| Gitleaks | Git history | Fast |
+| TruffleHog | Deep scanning | Medium |
+| grep | Quick checks | Fast |
+| GitHub Secret Scanning | GitHub repos | Auto |

package/org-skills/security-reviewer/references/vulnerability-patterns.md

@@ -0,0 +1,152 @@
+# Vulnerability Patterns
+
+## SQL Injection
+
+```typescript
+// VULNERABLE
+const query = `SELECT * FROM users WHERE id = ${userId}`;
+const query = `SELECT * FROM users WHERE name = '${name}'`;
+
+// SECURE - Parameterized queries
+const query = 'SELECT * FROM users WHERE id = $1';
+db.query(query, [userId]);
+
+// SECURE - ORM
+const user = await User.findOne({ where: { id: userId } });
+```
+
+## XSS (Cross-Site Scripting)
+
+```typescript
+// VULNERABLE - Direct HTML injection
+element.innerHTML = userInput;
+document.write(userInput);
+
+// SECURE - Text content
+element.textContent = userInput;
+
+// SECURE - Sanitization
+import DOMPurify from 'dompurify';
+element.innerHTML = DOMPurify.sanitize(userInput);
+
+// SECURE - React (auto-escaped)
+return <div>{userInput}</div>;
+
+// VULNERABLE - React dangerouslySetInnerHTML
+return <div dangerouslySetInnerHTML={{ __html: userInput }} />;
+```
+
+## Path Traversal
+
+```typescript
+// VULNERABLE
+const file = path.join(uploadDir, req.query.filename);
+res.sendFile(file);
+
+// SECURE - Validate and normalize
+const filename = path.basename(req.query.filename);
+const file = path.resolve(uploadDir, filename);
+
+if (!file.startsWith(path.resolve(uploadDir))) {
+  throw new Error('Invalid path');
+}
+res.sendFile(file);
+```
+
+## Command Injection
+
+```typescript
+// VULNERABLE
+exec(`ls ${userInput}`);
+exec('git clone ' + repoUrl);
+
+// SECURE - Use arrays, avoid shell
+execFile('ls', [userInput]);
+spawn('git', ['clone', repoUrl]);
+
+// SECURE - Validation
+const allowedCommands = ['status', 'log'];
+if (!allowedCommands.includes(cmd)) throw new Error('Invalid');
+```
+
+## IDOR (Insecure Direct Object Reference)
+
+```typescript
+// VULNERABLE - No authorization check
+app.get('/documents/:id', async (req, res) => {
+  const doc = await Document.findById(req.params.id);
+  res.json(doc);
+});
+
+// SECURE - Verify ownership
+app.get('/documents/:id', async (req, res) => {
+  const doc = await Document.findOne({
+    _id: req.params.id,
+    userId: req.user.id  // Ensure user owns document
+  });
+  if (!doc) return res.status(404).json({ error: 'Not found' });
+  res.json(doc);
+});
+```
+
+## Insecure Deserialization
+
+```typescript
+// VULNERABLE - Python pickle
+import pickle
+data = pickle.loads(user_input)
+
+// SECURE - Use JSON
+import json
+data = json.loads(user_input)
+
+// VULNERABLE - Node.js
+const obj = eval('(' + userInput + ')');
+
+// SECURE
+const obj = JSON.parse(userInput);
+```
+
+## Sensitive Data Exposure
+
+```typescript
+// VULNERABLE - Logging sensitive data
+logger.info('User login', { email, password });
+console.log('Token:', authToken);
+
+// SECURE - Redact sensitive fields
+logger.info('User login', { email, password: '[REDACTED]' });
+
+// VULNERABLE - Error response exposes internals
+res.status(500).json({ error: err.stack });
+
+// SECURE - Generic error
+res.status(500).json({ error: 'Internal server error' });
+```
+
+## Quick Reference
+
+| Vulnerability | Input Vector | Prevention |
+|---------------|--------------|------------|
+| SQL Injection | Query params | Parameterized queries |
+| XSS | User content | Output encoding |
+| Path Traversal | File paths | path.basename + validation |
+| Command Injection | Shell args | execFile, no shell |
+| IDOR | Resource IDs | Authorization checks |
+| Deserialization | Serialized data | JSON only |
+| Data Exposure | Logs, errors | Redaction, generic errors |
+
+## OWASP Top 10 Mapping
+
+| OWASP | Vulnerabilities |
+|-------|-----------------|
+| A01 Broken Access Control | IDOR, path traversal |
+| A02 Cryptographic Failures | Weak encryption, plaintext |
+| A03 Injection | SQL, XSS, command |
+| A04 Insecure Design | Missing auth, IDOR |
+| A05 Security Misconfiguration | Debug mode, default creds |
+| A06 Vulnerable Components | Outdated dependencies |
+| A07 Auth Failures | Weak passwords, session issues |
+| A08 Data Integrity | Insecure deserialization |
+| A09 Logging Failures | Missing logs, exposed data |
+| A10 SSRF | Unvalidated URLs |

package/org-skills/senior-data-scientist/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Alireza Rezvani
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/org-skills/senior-data-scientist/SKILL.md

@@ -0,0 +1,222 @@
+---
+name: senior-data-scientist
+description: "World-class senior data scientist skill specialising in statistical modeling, experiment design, causal inference, and predictive analytics. Covers A/B testing (sample sizing, two-proportion z-tests, Bonferroni correction), difference-in-differences, feature engineering pipelines (Scikit-learn, XGBoost), cross-validated model evaluation (AUC-ROC, AUC-PR, SHAP), and MLflow experiment tracking —..."
+tags: ["data","ai-ml"]
+tools: []
+license: MIT
+source: https://github.com/alirezarezvani/claude-skills
+source_path: "engineering-team/skills/senior-data-scientist"
+source_commit: 19392f7a08264ed00486a251f5b2098321771f94
+---
+# Senior Data Scientist
+
+World-class senior data scientist skill for production-grade AI/ML/Data systems.
+
+## Core Workflows
+
+### 1. Design an A/B Test
+
+```python
+import numpy as np
+from scipy import stats
+
+def calculate_sample_size(baseline_rate, mde, alpha=0.05, power=0.8):
+    """
+    Calculate required sample size per variant.
+    baseline_rate: current conversion rate (e.g. 0.10)
+    mde: minimum detectable effect (relative, e.g. 0.05 = 5% lift)
+    """
+    p1 = baseline_rate
+    p2 = baseline_rate * (1 + mde)
+    effect_size = abs(p2 - p1) / np.sqrt((p1 * (1 - p1) + p2 * (1 - p2)) / 2)
+    z_alpha = stats.norm.ppf(1 - alpha / 2)
+    z_beta = stats.norm.ppf(power)
+    n = ((z_alpha + z_beta) / effect_size) ** 2
+    return int(np.ceil(n))
+
+def analyze_experiment(control, treatment, alpha=0.05):
+    """
+    Run two-proportion z-test and return structured results.
+    control/treatment: dicts with 'conversions' and 'visitors'.
+    """
+    p_c = control["conversions"] / control["visitors"]
+    p_t = treatment["conversions"] / treatment["visitors"]
+    pooled = (control["conversions"] + treatment["conversions"]) / (control["visitors"] + treatment["visitors"])
+    se = np.sqrt(pooled * (1 - pooled) * (1 / control["visitors"] + 1 / treatment["visitors"]))
+    z = (p_t - p_c) / se
+    p_value = 2 * (1 - stats.norm.cdf(abs(z)))
+    ci_low = (p_t - p_c) - stats.norm.ppf(1 - alpha / 2) * se
+    ci_high = (p_t - p_c) + stats.norm.ppf(1 - alpha / 2) * se
+    return {
+        "lift": (p_t - p_c) / p_c,
+        "p_value": p_value,
+        "significant": p_value < alpha,
+        "ci_95": (ci_low, ci_high),
+    }
+
+# --- Experiment checklist ---
+# 1. Define ONE primary metric and pre-register secondary metrics.
+# 2. Calculate sample size BEFORE starting: calculate_sample_size(0.10, 0.05)
+# 3. Randomise at the user (not session) level to avoid leakage.
+# 4. Run for at least 1 full business cycle (typically 2 weeks).
+# 5. Check for sample ratio mismatch: abs(n_control - n_treatment) / expected < 0.01
+# 6. Analyze with analyze_experiment() and report lift + CI, not just p-value.
+# 7. Apply Bonferroni correction if testing multiple metrics: alpha / n_metrics
+```
+
+### 2. Build a Feature Engineering Pipeline
+
+```python
+import pandas as pd
+import numpy as np
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import StandardScaler, OneHotEncoder
+from sklearn.impute import SimpleImputer
+from sklearn.compose import ColumnTransformer
+
+def build_feature_pipeline(numeric_cols, categorical_cols, date_cols=None):
+    """
+    Returns a fitted-ready ColumnTransformer for structured tabular data.
+    """
+    numeric_pipeline = Pipeline([
+        ("impute", SimpleImputer(strategy="median")),
+        ("scale",  StandardScaler()),
+    ])
+    categorical_pipeline = Pipeline([
+        ("impute", SimpleImputer(strategy="most_frequent")),
+        ("encode", OneHotEncoder(handle_unknown="ignore", sparse_output=False)),
+    ])
+    transformers = [
+        ("num", numeric_pipeline, numeric_cols),
+        ("cat", categorical_pipeline, categorical_cols),
+    ]
+    return ColumnTransformer(transformers, remainder="drop")
+
+def add_time_features(df, date_col):
+    """Extract cyclical and lag features from a datetime column."""
+    df = df.copy()
+    df[date_col] = pd.to_datetime(df[date_col])
+    df["dow_sin"] = np.sin(2 * np.pi * df[date_col].dt.dayofweek / 7)
+    df["dow_cos"] = np.cos(2 * np.pi * df[date_col].dt.dayofweek / 7)
+    df["month_sin"] = np.sin(2 * np.pi * df[date_col].dt.month / 12)
+    df["month_cos"] = np.cos(2 * np.pi * df[date_col].dt.month / 12)
+    df["is_weekend"] = (df[date_col].dt.dayofweek >= 5).astype(int)
+    return df
+
+# --- Feature engineering checklist ---
+# 1. Never fit transformers on the full dataset — fit on train, transform test.
+# 2. Log-transform right-skewed numeric features before scaling.
+# 3. For high-cardinality categoricals (>50 levels), use target encoding or embeddings.
+# 4. Generate lag/rolling features BEFORE the train/test split to avoid leakage.
+# 5. Document each feature's business meaning alongside its code.
+```
+
+### 3. Train, Evaluate, and Select a Prediction Model
+
+```python
+from sklearn.model_selection import StratifiedKFold, cross_validate
+from sklearn.metrics import make_scorer, roc_auc_score, average_precision_score
+import xgboost as xgb
+import mlflow
+
+SCORERS = {
+    "roc_auc":  make_scorer(roc_auc_score, needs_proba=True),
+    "avg_prec": make_scorer(average_precision_score, needs_proba=True),
+}
+
+def evaluate_model(model, X, y, cv=5):
+    """
+    Cross-validate and return mean ± std for each scorer.
+    Use StratifiedKFold for classification to preserve class balance.
+    """
+    cv_results = cross_validate(
+        model, X, y,
+        cv=StratifiedKFold(n_splits=cv, shuffle=True, random_state=42),
+        scoring=SCORERS,
+        return_train_score=True,
+    )
+    summary = {}
+    for metric in SCORERS:
+        test_scores = cv_results[f"test_{metric}"]
+        summary[metric] = {"mean": test_scores.mean(), "std": test_scores.std()}
+        # Flag overfitting: large gap between train and test score
+        train_mean = cv_results[f"train_{metric}"].mean()
+        summary[metric]["overfit_gap"] = train_mean - test_scores.mean()
+    return summary
+
+def train_and_log(model, X_train, y_train, X_test, y_test, run_name):
+    """Train model and log all artefacts to MLflow."""
+    with mlflow.start_run(run_name=run_name):
+        model.fit(X_train, y_train)
+        proba = model.predict_proba(X_test)[:, 1]
+        metrics = {
+            "roc_auc":  roc_auc_score(y_test, proba),
+            "avg_prec": average_precision_score(y_test, proba),
+        }
+        mlflow.log_params(model.get_params())
+        mlflow.log_metrics(metrics)
+        mlflow.sklearn.log_model(model, "model")
+        return metrics
+
+# --- Model evaluation checklist ---
+# 1. Always report AUC-PR alongside AUC-ROC for imbalanced datasets.
+# 2. Check overfit_gap > 0.05 as a warning sign of overfitting.
+# 3. Calibrate probabilities (Platt scaling / isotonic) before production use.
+# 4. Compute SHAP values to validate feature importance makes business sense.
+# 5. Run a baseline (e.g. DummyClassifier) and verify the model beats it.
+# 6. Log every run to MLflow — never rely on notebook output for comparison.
+```
+
+### 4. Causal Inference: Difference-in-Differences
+
+```python
+import statsmodels.formula.api as smf
+
+def diff_in_diff(df, outcome, treatment_col, post_col, controls=None):
+    """
+    Estimate ATT via OLS DiD with optional covariates.
+    df must have: outcome, treatment_col (0/1), post_col (0/1).
+    Returns the interaction coefficient (treatment × post) and its p-value.
+    """
+    covariates = " + ".join(controls) if controls else ""
+    formula = (
+        f"{outcome} ~ {treatment_col} * {post_col}"
+        + (f" + {covariates}" if covariates else "")
+    )
+    result = smf.ols(formula, data=df).fit(cov_type="HC3")
+    interaction = f"{treatment_col}:{post_col}"
+    return {
+        "att":     result.params[interaction],
+        "p_value": result.pvalues[interaction],
+        "ci_95":   result.conf_int().loc[interaction].tolist(),
+        "summary": result.summary(),
+    }
+
+# --- Causal inference checklist ---
+# 1. Validate parallel trends in pre-period before trusting DiD estimates.
+# 2. Use HC3 robust standard errors to handle heteroskedasticity.
+# 3. For panel data, cluster SEs at the unit level (add groups= param to fit).
+# 4. Consider propensity score matching if groups differ at baseline.
+# 5. Report the ATT with confidence interval, not just statistical significance.
+```
+
+## Reference Documentation
+
+- **Statistical Methods:** `references/statistical_methods_advanced.md`
+- **Experiment Design Frameworks:** `references/experiment_design_frameworks.md`
+- **Feature Engineering Patterns:** `references/feature_engineering_patterns.md`
+
+## Common Commands
+
+```bash
+# Testing & linting
+python -m pytest tests/ -v --cov=src/
+python -m black src/ && python -m pylint src/
+
+# Bundled pipeline scaffolds (stdlib runners — extend the process() body with project logic)
+python3 scripts/experiment_designer.py --input experiment_spec.json --output experiment_design.json
+python3 scripts/feature_engineering_pipeline.py --input raw_features.json --output features.json
+python3 scripts/model_evaluation_suite.py --input model_predictions.json --output evaluation.json
+# Each prints a JSON run report ({status, processed_items, start/end_time}); any status other
+# than "completed" means the stage failed — fix before moving to the next pipeline stage.
+```