@caoscompanybr/merlin 3.5.0

package/.merlin-core/skills/general/secret-safe-commit/SKILL.md

@@ -0,0 +1,246 @@
+---
+name: secret-safe-commit
+description: Pre-commit gitleaks scanning + project bootstrap. Refuses commits that expose secrets. Backed by .gitleaks.toml allowlist, .pre-commit-config.yaml, and CI workflow templates. Used by /commit and /safe-commit.
+license: Apache-2.0
+metadata:
+  author: merlin-framework
+  version: "3.0.0"
+  auto_activate:
+    [
+      safe-commit,
+      gitleaks,
+      secret-scan,
+      pre-commit-hook,
+      leak-prevention,
+      api-key-leak,
+    ]
+  tool_reminders:
+    [
+      Pre-commit hooks can be bypassed with --no-verify — always pair with CI scan,
+      On any gitleaks finding ABORT the commit and recommend purge-leaked-secret skill,
+      Never edit .gitleaks.toml allowlist without documenting why each entry exists,
+      gitleaks redact mode hides values in output — use it by default to avoid leaking secrets in agent logs,
+      Test fixtures with intentionally fake secrets need explicit allowlist or --skip-secret-scan flag with reason,
+    ]
+---
+
+# Secret-Safe Commit Skill
+
+**Mission:** Make it physically impossible for a Merlin-project commit to ship
+a secret to git history without explicit human bypass + audit log. Every
+project gets the same gitleaks setup; every commit goes through the same
+pre-flight check.
+
+The skill exists because we once discovered 3 credential files
+sitting in a client project's working tree (Firebase Admin SDK private key, Google
+OAuth client secret, YouTube tokens) that any `git add -A` would have
+committed instantly. We cannot rely on humans noticing — we need the tooling
+to refuse.
+
+---
+
+## When this skill activates
+
+- User invokes `/safe-commit` or `/commit`
+- User mentions: "gitleaks", "secret scan", "pre-commit", "leaked", "credential"
+- A new project is being bootstrapped (initial commit not yet made)
+- After `purge-leaked-secret` cleans up an incident — to prevent recurrence
+
+---
+
+## What this skill provides
+
+### A. Bootstrap command (one-time per project)
+
+```bash
+# 1. Install pre-commit framework + gitleaks
+pip install pre-commit
+# OR via brew/apt: brew install gitleaks  /  apt install gitleaks
+
+# 2. Drop in the templates (from this skill's templates/ directory)
+cp <skill-dir>/templates/.pre-commit-config.yaml .
+cp <skill-dir>/templates/.gitleaks.toml .
+mkdir -p .github/workflows
+cp <skill-dir>/templates/secret-scan.yml .github/workflows/
+
+# 3. Activate pre-commit hooks
+pre-commit install
+
+# 4. Run baseline scan to detect existing secrets in working tree
+gitleaks detect --source . --redact --verbose
+
+# 5. Run baseline scan on git history (full repo)
+gitleaks detect --source . --log-opts="--all" --redact --verbose
+```
+
+The baseline scan in step 4-5 is critical — we need to know NOW if past
+commits have secrets, not discover it later. If findings appear, immediately
+invoke the **`purge-leaked-secret`** skill.
+
+### B. Per-commit pre-flight (every `/safe-commit`)
+
+```bash
+gitleaks protect --source . --staged --redact --verbose
+```
+
+`protect` mode (vs `detect`) only scans STAGED changes — fast (<2s typical).
+`--redact` hides the actual secret value in output (so the agent log doesn't
+leak the secret it just caught).
+
+### C. CI defense-in-depth
+
+The included `.github/workflows/secret-scan.yml` runs on every push + PR with
+`gitleaks detect`. This catches:
+
+- Commits made with `--no-verify` (bypassing pre-commit)
+- Commits made with `--skip-secret-scan` (legitimate edge cases that still need to be reviewed)
+- Secrets that match patterns added to the gitleaks DB AFTER the local scan ran
+
+### D. Allowlist convention (`.gitleaks.toml`)
+
+Allowlists are technical debt — every entry must be justified. Convention:
+
+```toml
+[allowlist]
+description = "<project> allowlist — every entry must be approved + documented"
+
+# Each entry has 3 required fields: regex/path, reason, approver, date
+[[allowlist.regexes]]
+description = "Test fixture for AWS regex testing — fake key, never real"
+regex = '''AKIAIOSFODNN7EXAMPLE'''
+# approver: <name>, date: <YYYY-MM-DD>, see: tests/aws-mock.ts
+```
+
+If an allowlist grows past 5 entries, run `engineering-audit` on it — most
+entries are usually code smells (real secrets in tests, fake secrets in prod
+code by mistake, etc.).
+
+---
+
+## The `--skip-secret-scan` flag (governance)
+
+When a user invokes `/safe-commit --skip-secret-scan`:
+
+1. **Require a written reason** in the commit body: `secret-scan-skip: <why>`
+2. **Print the warning** to console (not just log):
+   ```
+   ⚠️  SECRET SCAN BYPASSED
+       Reason: <reason>
+       This bypass is logged to .merlin-core/thoughts/shared/incidents/secret-scan-bypass.log
+       The CI scan will still run on push.
+   ```
+3. **Log to** `.merlin-core/thoughts/shared/incidents/secret-scan-bypass.log` with:
+   - Timestamp (ISO 8601)
+   - Git user (from `git config user.email`)
+   - Commit SHA (after commit completes)
+   - Files staged (full list)
+   - Reason text
+
+4. **Periodic review** of the bypass log — recommend monthly audit during
+   `engineering-audit` on the project. Repeated bypasses for the same reason
+   indicate a process gap (e.g., test fixtures need permanent allowlist entries).
+
+---
+
+## Common gotchas
+
+### Gotcha #1 — `pre-commit install` not run after `pip install pre-commit`
+
+`pip install pre-commit` only installs the binary. Each repo needs `pre-commit
+install` to wire it into `.git/hooks/pre-commit`. Without that, the
+`.pre-commit-config.yaml` is just a sitting file. Always verify with
+`cat .git/hooks/pre-commit | head -5` after install.
+
+### Gotcha #2 — gitleaks runs from repo root, not subdirectory
+
+If your project has a monorepo structure (e.g. `firebase/functions/`
+and `firebase/hosting/`), gitleaks scans the entire tree by default. This is
+usually what you want. To restrict, use `--source <subdir>` but be aware that
+secrets can leak across subdirs (e.g., a `.env` at root affects both).
+
+### Gotcha #3 — Custom rules need `[[rules]]` not `[rules]`
+
+TOML array-of-tables syntax is `[[double brackets]]`. `[single]` defines a
+table, not a list. Easy to get wrong; gitleaks will silently load 0 custom
+rules with the wrong syntax.
+
+### Gotcha #4 — `gitleaks detect` is slow on large repos
+
+`detect` mode scans the entire git log. On a small repo it's <5s; on
+larger repos (>10k commits) it can take minutes. For routine scans, use
+`gitleaks protect --staged` (fast, only staged). Run `detect` on the full
+history only at bootstrap and weekly via cron/GitHub Action.
+
+### Gotcha #5 — Agent log captures the redacted secret name
+
+When the agent logs `gitleaks` output, even with `--redact`, the rule name
+
+- file path is captured. This is fine usually, but if a rule is named
+  something sensitive (e.g., "<Company> Internal API Key"), the rule name
+  itself can be a leak. Use generic rule names or anonymize.
+
+---
+
+## Templates included in this skill
+
+| File                                | Purpose                                                                        | Drop-in path         |
+| ----------------------------------- | ------------------------------------------------------------------------------ | -------------------- |
+| `templates/.pre-commit-config.yaml` | Hooks: gitleaks + lint-staged + commitlint + check-merge-conflict              | Project root         |
+| `templates/.gitleaks.toml`          | Custom rules + allowlist (start with allowlist empty + project-specific rules) | Project root         |
+| `templates/secret-scan.yml`         | GitHub Action — runs on push + PR                                              | `.github/workflows/` |
+
+After dropping templates, edit the project-specific rules section of
+`.gitleaks.toml` to add patterns unique to the project (e.g., custom JWT
+prefixes, internal service tokens).
+
+---
+
+## How this composes with other skills
+
+- **`/safe-commit` command** — invokes Step 0 of this skill (`gitleaks protect --staged`) before drafting commit message
+- **`purge-leaked-secret`** skill — invoked when this skill DETECTS a leak (proactive → reactive)
+- **`engineering-audit`** skill — its security axis includes verifying gitleaks is installed + active in the audited project
+- **`ci-security-baseline`** skill (Tier 2 roadmap) — extends the GitHub Action with Trivy + npm audit + Semgrep
+- **`vps-security-hardening`** skill — references this skill in its "operational hygiene" section
+
+---
+
+## Verification (after bootstrapping a project)
+
+```bash
+# 1. Verify pre-commit installed
+test -f .git/hooks/pre-commit && echo "✓ pre-commit hook installed" || echo "✗ MISSING"
+
+# 2. Verify gitleaks runs
+gitleaks version && echo "✓ gitleaks available" || echo "✗ MISSING"
+
+# 3. Test with a fake secret (creates a fake AWS key, attempts commit, expects abort)
+echo 'AWS_KEY="AKIAIOSFODNN7EXAMPLE"' > /tmp/test-leak.env
+git add /tmp/test-leak.env 2>/dev/null
+gitleaks protect --source . --staged --redact && echo "✗ FAIL: should have detected" || echo "✓ gitleaks correctly aborted"
+git restore --staged /tmp/test-leak.env 2>/dev/null
+rm -f /tmp/test-leak.env
+
+# 4. Verify CI workflow file
+test -f .github/workflows/secret-scan.yml && echo "✓ CI workflow installed" || echo "✗ MISSING"
+```
+
+If any check fails, re-run the bootstrap. All four must pass before declaring
+the project "secret-safe".
+
+---
+
+## Maintenance
+
+- **Quarterly:** Review `.gitleaks.toml` allowlist — remove entries that no longer apply
+- **Monthly:** Review `.merlin-core/thoughts/shared/incidents/secret-scan-bypass.log` — flag repeated bypass patterns
+- **On gitleaks update:** Re-run `gitleaks detect --source . --log-opts="--all"` to catch newly-recognized secret patterns in old commits
+- **On new project:** Run the bootstrap as part of `merlin-tools upgrade`
+
+---
+
+## References
+
+- [Gitleaks official docs](https://gitleaks.org/)
+- [pre-commit framework](https://pre-commit.com/)
+- A fleet security audit (internal) — the origin incident that motivated this skill

package/.merlin-core/skills/general/secret-safe-commit/templates/.gitleaks.toml

@@ -0,0 +1,91 @@
+# Merlin baseline gitleaks config — extends defaults with project-specific rules.
+# Includes critical custom rules to catch markdown-table-formatted secrets that
+# default gitleaks rules miss (the format that once exposed a vendor production
+# secret in a real project — see purge-leaked-secret skill).
+#
+# Schema: gitleaks v8.x — `[allowlist]` uses simple list-of-strings, NOT
+# `[[allowlist.paths]]` with sub-tables (deprecated in v8).
+#
+# Docs: https://github.com/gitleaks/gitleaks/blob/master/README.md#configuration
+
+# Inherit all default rules (AWS, GitHub, Google, Stripe, OpenAI, etc.)
+[extend]
+useDefault = true
+
+# ---------- CRITICAL: catch markdown-table formatted secrets ----------
+# Default gitleaks rules require key=value or "key": "value" format.
+# Markdown handoff docs frequently use:   | key | `value` |   which slips through.
+[[rules]]
+id = "markdown-table-secret"
+description = "Secret embedded in a markdown table cell (| key | `value` |)"
+regex = '''(?i)\|\s*[^|]*(?:secret|password|token|api[_-]?key|client[_-]?secret|access[_-]?token|refresh[_-]?token|auth[_-]?token|private[_-]?key|webhook[_-]?secret)[^|]*\|\s*`([A-Za-z0-9_+\/=\-]{30,})`'''
+secretGroup = 1
+entropy = 3.5
+tags = ["markdown", "table", "high-priority"]
+
+# Catches: **Client Secret:** `Bp07...` or "Webhook secret: `whsec_...`"
+[[rules]]
+id = "backtick-secret-near-keyword"
+description = "High-entropy string in backticks near a secret keyword"
+regex = '''(?i)(?:secret|password|token|api[_-]?key|client[_-]?secret|access[_-]?token|webhook[_-]?secret)["'\s:*]*`([A-Za-z0-9_+\/=\-]{30,})`'''
+secretGroup = 1
+entropy = 3.5
+tags = ["markdown", "high-priority"]
+
+# ---------- Common provider rules (uncomment + adapt per project) ----------
+
+# Mercado Pago credentials
+# [[rules]]
+# id = "mercadopago-access-token"
+# description = "Mercado Pago Access Token (production or test)"
+# regex = '''(?i)(APP_USR|TEST)-[0-9]{8,}-[0-9]{6}-[a-f0-9]{32}-[0-9]{8,}'''
+# tags = ["key", "mercadopago"]
+
+# Resend API keys
+# [[rules]]
+# id = "resend-api-key"
+# description = "Resend API Key"
+# regex = '''re_[A-Za-z0-9_]{40,}'''
+# tags = ["key", "resend"]
+
+# Example vendor JWTs
+# [[rules]]
+# id = "vendor-jwt"
+# description = "Example vendor production JWT (RS256, 30-day)"
+# regex = '''eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIs[A-Za-z0-9_=\-]{100,}'''
+# tags = ["jwt", "vendor"]
+
+# Firebase service account private key block
+# [[rules]]
+# id = "firebase-private-key-block"
+# description = "Firebase service account private_key field"
+# regex = '''"private_key"\s*:\s*"-----BEGIN PRIVATE KEY-----'''
+# tags = ["key", "firebase"]
+
+# ---------- Allowlist — every entry MUST be documented in comments ----------
+# gitleaks v8 uses simple list-of-strings format. Documentation goes in comments
+# IMMEDIATELY ABOVE each entry (approver, date, reason).
+[allowlist]
+description = "Documented exceptions — review monthly"
+
+paths = [
+    # Skill templates contain secret regex patterns AS DOCUMENTATION (not real leaks)
+    # approver: <name>, date: <YYYY-MM-DD>, reason: skill templates illustrate gitleaks rules
+    '''\.merlin-core/skills/.+/templates/.+''',
+
+    # Lock files (large auto-generated, no secrets)
+    '''package-lock\.json''',
+    '''yarn\.lock''',
+]
+
+regexes = [
+    # Generic AWS example key from gitleaks docs (universally fake)
+    # approver: <name>, date: <YYYY-MM-DD>, reason: AWS official documentation example
+    '''AKIAIOSFODNN7EXAMPLE''',
+
+    # Redaction marker we use after purging real secrets (not an actual secret)
+    # approver: <name>, date: <YYYY-MM-DD>, reason: redaction placeholder
+    '''<REDACTED[^>]*>''',
+]
+
+# Add project-specific exceptions above — keep documented in comments

package/.merlin-core/skills/general/secret-safe-commit/templates/.pre-commit-config.yaml

@@ -0,0 +1,57 @@
+# Merlin baseline pre-commit config — secret scanning + lint + commit message format.
+# Drop into project root, then run: pre-commit install
+#
+# Hooks run in order. Any failure aborts the commit.
+
+repos:
+  # ---------- Secret scanning (THIS IS THE CRITICAL ONE) ----------
+  - repo: https://github.com/gitleaks/gitleaks
+    rev: v8.21.1
+    hooks:
+      - id: gitleaks
+        # Scan staged changes only — fast (<2s typical)
+        # --redact hides secret values in output (so logs don't leak)
+        args: ['protect', '--staged', '--redact', '--verbose']
+
+  # ---------- Generic safety hooks (small + universal) ----------
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.6.0
+    hooks:
+      - id: check-added-large-files
+        args: ['--maxkb=500']  # Block files >500KB (likely binaries or accidentally committed assets)
+      - id: check-merge-conflict   # Refuses commits with <<<<<<< markers
+      - id: check-yaml             # Validates YAML files (catches typos in CI configs early)
+      - id: check-json             # Validates JSON files (catches typos in package.json early)
+      - id: detect-private-key     # Refuses files containing -----BEGIN PRIVATE KEY----- markers
+        # Exclude our own config files that REFERENCE the BEGIN PRIVATE KEY pattern as documentation
+        exclude: '^(\.gitleaks\.toml|\.pre-commit-config\.yaml|\.merlin-core/skills/.+/templates/.+|\.merlin-core/thoughts/shared/(research|plans|handoffs)/.+\.md)$'
+      - id: detect-aws-credentials # Belt-and-suspenders with gitleaks for AWS
+        args: ['--allow-missing-credentials']
+      - id: end-of-file-fixer      # Ensures files end with newline (poka-yoke)
+      - id: trailing-whitespace    # Removes trailing whitespace on save
+
+  # ---------- Conventional commit message validation ----------
+  - repo: https://github.com/compilerla/conventional-pre-commit
+    rev: v3.4.0
+    hooks:
+      - id: conventional-pre-commit
+        stages: [commit-msg]
+        args: ['--strict', '--force-scope', 'feat', 'fix', 'docs', 'style', 'refactor', 'perf', 'test', 'chore', 'ci', 'build', 'security']
+
+  # ---------- Project-specific lint hooks (uncomment + adapt per project) ----------
+  # For Node projects:
+  # - repo: local
+  #   hooks:
+  #     - id: eslint
+  #       name: eslint
+  #       entry: npx eslint --fix
+  #       language: system
+  #       types_or: [javascript, ts, tsx]
+  #       require_serial: false
+  #
+  # For Python projects:
+  # - repo: https://github.com/astral-sh/ruff-pre-commit
+  #   rev: v0.6.9
+  #   hooks:
+  #     - id: ruff           # Lint
+  #     - id: ruff-format    # Format

package/.merlin-core/skills/general/secret-safe-commit/templates/secret-scan.yml

@@ -0,0 +1,48 @@
+# Defense-in-depth: scan all pushes + PRs for leaked secrets.
+# Catches commits made with --no-verify or --skip-secret-scan that bypassed the local hook.
+#
+# Drop into .github/workflows/secret-scan.yml
+
+name: Secret Scan (gitleaks)
+
+on:
+  push:
+    branches: ['**']
+  pull_request:
+    branches: ['**']
+  schedule:
+    # Weekly full-history rescan — catches patterns added to gitleaks DB after the fact
+    - cron: '0 4 * * 1'  # Monday 04:00 UTC
+  workflow_dispatch:  # Allow manual trigger
+
+permissions:
+  contents: read
+  security-events: write  # For uploading SARIF results
+
+jobs:
+  gitleaks:
+    name: gitleaks scan
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout (full history for accurate scan)
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # Required for full-history scan
+
+      - name: Run gitleaks
+        uses: gitleaks/gitleaks-action@v2
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          # GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }}  # Only needed for private orgs (free for public)
+          GITLEAKS_ENABLE_COMMENTS: true       # Comment on PR with findings
+          GITLEAKS_ENABLE_UPLOAD_ARTIFACT: true # Upload SARIF for security tab
+          GITLEAKS_ENABLE_SUMMARY: true        # Show summary in workflow log
+
+      - name: Notify on finding (Slack/Discord/etc — adapt per project)
+        if: failure()
+        run: |
+          echo "::error::Gitleaks detected leaked secrets. See findings above."
+          # Example: notify Slack
+          # curl -X POST -H 'Content-type: application/json' \
+          #   --data '{"text":"🚨 Secret leak detected in ${{ github.repository }} — see ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"}' \
+          #   ${{ secrets.SLACK_WEBHOOK_URL }}

package/.merlin-core/skills/general/semantic-search/SKILL.md

@@ -0,0 +1,79 @@
+---
+name: semantic-search
+description: Symbol-aware code search patterns that distinguish definitions from references, classes from instances, and imports from usages.
+license: Apache-2.0
+metadata:
+  author: merlin-framework
+  version: "3.0.0"
+  auto_activate: [find-definition, find-class, find-function, find-usage, go-to-symbol]
+  file_triggers: [*.js, *.ts, *.py, *.java, *.go]
+  tool_reminders: [For 'find definition': search for class/function/const DECLARATION patterns, not all mentions, For 'find usage': exclude import lines and the definition itself]
+---
+
+## Symbol-Aware Search Patterns
+
+When searching for code symbols, use language-specific patterns to distinguish definitions from references. This prevents false positives and gives more precise results.
+
+## JavaScript / TypeScript
+
+### Find Definition
+
+```
+class ClassName          → /class\s+ClassName\b/
+function funcName        → /function\s+funcName\b/
+const/let/var varName    → /(const|let|var)\s+varName\b/
+export default           → /export\s+default\s+(class|function)\s+Name/
+arrow function           → /const\s+funcName\s*=/
+interface/type           → /(interface|type)\s+TypeName\b/
+```
+
+### Find Usage (exclude definition + imports)
+
+```
+Exclude: /^import\s.*ClassName/
+Exclude: /^(class|function|const|let|var|interface|type)\s+SymbolName/
+Include: all other occurrences of SymbolName
+```
+
+### Find Exports
+
+```
+Named export   → /export\s+(const|function|class|type|interface)\s+Name/
+Re-export      → /export\s*\{[^}]*Name[^}]*\}\s*from/
+Default export → /export\s+default\s+Name/
+```
+
+## Python
+
+### Find Definition
+
+```
+class ClassName    → /^class\s+ClassName[\s(:]/
+def funcName       → /^def\s+funcName\s*\(/
+variable           → /^ClassName\s*=/  (module-level assignment)
+```
+
+### Find Usage (exclude definition + imports)
+
+```
+Exclude: /^(from\s.*import|import)\s.*ClassName/
+Exclude: /^(class|def)\s+SymbolName/
+```
+
+## Go
+
+### Find Definition
+
+```
+func funcName      → /^func\s+funcName\b/
+type TypeName      → /^type\s+TypeName\s+(struct|interface)/
+var/const          → /^(var|const)\s+varName\b/
+method             → /^func\s*\([^)]+\)\s*funcName\b/
+```
+
+## Search Strategy
+
+1. **Definition search**: Use the language-specific declaration patterns above
+2. **Usage search**: Grep for the symbol, then EXCLUDE import lines and the definition itself
+3. **Reference search**: Find all occurrences including imports and definition (full picture)
+4. **When LSP MCP is available**: Prefer `go_to_definition` and `find_references` tools for precise resolution