@cregis-dev/cckit 0.2.0

package/templates/ecc/skills/regex-vs-llm-structured-text/SKILL.md

@@ -0,0 +1,219 @@
+---
+name: regex-vs-llm-structured-text
+description: Decision framework for choosing between regex and LLM when parsing structured text — start with regex, add LLM only for low-confidence edge cases.
+---
+
+# Regex vs LLM for Structured Text Parsing
+
+A practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.
+
+## When to Activate
+
+- Parsing structured text with repeating patterns (questions, forms, tables)
+- Deciding between regex and LLM for text extraction
+- Building hybrid pipelines that combine both approaches
+- Optimizing cost/accuracy tradeoffs in text processing
+
+## Decision Framework
+
+```
+Is the text format consistent and repeating?
+├── Yes (>90% follows a pattern) → Start with Regex
+│   ├── Regex handles 95%+ → Done, no LLM needed
+│   └── Regex handles <95% → Add LLM for edge cases only
+└── No (free-form, highly variable) → Use LLM directly
+```
+
+## Architecture Pattern
+
+```
+Source Text
+    │
+    ▼
+[Regex Parser] ─── Extracts structure (95-98% accuracy)
+    │
+    ▼
+[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)
+    │
+    ▼
+[Confidence Scorer] ─── Flags low-confidence extractions
+    │
+    ├── High confidence (≥0.95) → Direct output
+    │
+    └── Low confidence (<0.95) → [LLM Validator] → Output
+```
+
+## Implementation
+
+### 1. Regex Parser (Handles the Majority)
+
+```python
+import re
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class ParsedItem:
+    id: str
+    text: str
+    choices: tuple[str, ...]
+    answer: str
+    confidence: float = 1.0
+
+def parse_structured_text(content: str) -> list[ParsedItem]:
+    """Parse structured text using regex patterns."""
+    pattern = re.compile(
+        r"(?P<id>\d+)\.\s*(?P<text>.+?)\n"
+        r"(?P<choices>(?:[A-D]\..+?\n)+)"
+        r"Answer:\s*(?P<answer>[A-D])",
+        re.MULTILINE | re.DOTALL,
+    )
+    items = []
+    for match in pattern.finditer(content):
+        choices = tuple(
+            c.strip() for c in re.findall(r"[A-D]\.\s*(.+)", match.group("choices"))
+        )
+        items.append(ParsedItem(
+            id=match.group("id"),
+            text=match.group("text").strip(),
+            choices=choices,
+            answer=match.group("answer"),
+        ))
+    return items
+```
+
+### 2. Confidence Scoring
+
+Flag items that may need LLM review:
+
+```python
+@dataclass(frozen=True)
+class ConfidenceFlag:
+    item_id: str
+    score: float
+    reasons: tuple[str, ...]
+
+def score_confidence(item: ParsedItem) -> ConfidenceFlag:
+    """Score extraction confidence and flag issues."""
+    reasons = []
+    score = 1.0
+
+    if len(item.choices) < 3:
+        reasons.append("few_choices")
+        score -= 0.3
+
+    if not item.answer:
+        reasons.append("missing_answer")
+        score -= 0.5
+
+    if len(item.text) < 10:
+        reasons.append("short_text")
+        score -= 0.2
+
+    return ConfidenceFlag(
+        item_id=item.id,
+        score=max(0.0, score),
+        reasons=tuple(reasons),
+    )
+
+def identify_low_confidence(
+    items: list[ParsedItem],
+    threshold: float = 0.95,
+) -> list[ConfidenceFlag]:
+    """Return items below confidence threshold."""
+    flags = [score_confidence(item) for item in items]
+    return [f for f in flags if f.score < threshold]
+```
+
+### 3. LLM Validator (Edge Cases Only)
+
+```python
+def validate_with_llm(
+    item: ParsedItem,
+    original_text: str,
+    client,
+) -> ParsedItem:
+    """Use LLM to fix low-confidence extractions."""
+    response = client.messages.create(
+        model="claude-haiku-4-5-20251001",  # Cheapest model for validation
+        max_tokens=500,
+        messages=[{
+            "role": "user",
+            "content": (
+                f"Extract the question, choices, and answer from this text.\n\n"
+                f"Text: {original_text}\n\n"
+                f"Current extraction: {item}\n\n"
+                f"Return corrected JSON if needed, or 'CORRECT' if accurate."
+            ),
+        }],
+    )
+    # Parse LLM response and return corrected item...
+    return corrected_item
+```
+
+### 4. Hybrid Pipeline
+
+```python
+def process_document(
+    content: str,
+    *,
+    llm_client=None,
+    confidence_threshold: float = 0.95,
+) -> list[ParsedItem]:
+    """Full pipeline: regex -> confidence check -> LLM for edge cases."""
+    # Step 1: Regex extraction (handles 95-98%)
+    items = parse_structured_text(content)
+
+    # Step 2: Confidence scoring
+    low_confidence = identify_low_confidence(items, confidence_threshold)
+
+    if not low_confidence or llm_client is None:
+        return items
+
+    # Step 3: LLM validation (only for flagged items)
+    low_conf_ids = {f.item_id for f in low_confidence}
+    result = []
+    for item in items:
+        if item.id in low_conf_ids:
+            result.append(validate_with_llm(item, content, llm_client))
+        else:
+            result.append(item)
+
+    return result
+```
+
+## Real-World Metrics
+
+From a production quiz parsing pipeline (410 items):
+
+| Metric | Value |
+|--------|-------|
+| Regex success rate | 98.0% |
+| Low confidence items | 8 (2.0%) |
+| LLM calls needed | ~5 |
+| Cost savings vs all-LLM | ~95% |
+| Test coverage | 93% |
+
+## Best Practices
+
+- **Start with regex** — even imperfect regex gives you a baseline to improve
+- **Use confidence scoring** to programmatically identify what needs LLM help
+- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)
+- **Never mutate** parsed items — return new instances from cleaning/validation steps
+- **TDD works well** for parsers — write tests for known patterns first, then edge cases
+- **Log metrics** (regex success rate, LLM call count) to track pipeline health
+
+## Anti-Patterns to Avoid
+
+- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)
+- Using regex for free-form, highly variable text (LLM is better here)
+- Skipping confidence scoring and hoping regex "just works"
+- Mutating parsed objects during cleaning/validation steps
+- Not testing edge cases (malformed input, missing fields, encoding issues)
+
+## When to Use
+
+- Quiz/exam question parsing
+- Form data extraction
+- Invoice/receipt processing
+- Document structure parsing (headers, sections, tables)
+- Any structured text with repeating patterns where cost matters

package/templates/ecc/skills/search-first/SKILL.md

@@ -0,0 +1,159 @@
+---
+name: search-first
+description: Research-before-coding workflow. Search for existing tools, libraries, and patterns before writing custom code. Invokes the researcher agent.
+---
+
+# /search-first — Research Before You Code
+
+Systematizes the "search for existing solutions before implementing" workflow.
+
+## Trigger
+
+Use this skill when:
+- Starting a new feature that likely has existing solutions
+- Adding a dependency or integration
+- The user asks "add X functionality" and you're about to write code
+- Before creating a new utility, helper, or abstraction
+
+## Workflow
+
+```
+┌─────────────────────────────────────────────┐
+│  1. NEED ANALYSIS                           │
+│     Define what functionality is needed      │
+│     Identify language/framework constraints  │
+├─────────────────────────────────────────────┤
+│  2. PARALLEL SEARCH (researcher agent)      │
+│     ┌──────────┐ ┌──────────┐ ┌──────────┐  │
+│     │  npm /   │ │  MCP /   │ │  GitHub / │  │
+│     │  PyPI    │ │  Skills  │ │  Web      │  │
+│     └──────────┘ └──────────┘ └──────────┘  │
+├─────────────────────────────────────────────┤
+│  3. EVALUATE                                │
+│     Score candidates (functionality, maint, │
+│     community, docs, license, deps)         │
+├─────────────────────────────────────────────┤
+│  4. DECIDE                                  │
+│     ┌─────────┐  ┌──────────┐  ┌─────────┐  │
+│     │  Adopt  │  │  Extend  │  │  Build   │  │
+│     │ as-is   │  │  /Wrap   │  │  Custom  │  │
+│     └─────────┘  └──────────┘  └─────────┘  │
+├─────────────────────────────────────────────┤
+│  5. IMPLEMENT                               │
+│     Install package / Configure MCP /       │
+│     Write minimal custom code               │
+└─────────────────────────────────────────────┘
+```
+
+## Decision Matrix
+
+| Signal | Action |
+|--------|--------|
+| Exact match, well-maintained, MIT/Apache | **Adopt** — install and use directly |
+| Partial match, good foundation | **Extend** — install + write thin wrapper |
+| Multiple weak matches | **Compose** — combine 2-3 small packages |
+| Nothing suitable found | **Build** — write custom, but informed by research |
+
+## How to Use
+
+### Quick Mode (inline)
+
+Before writing a utility or adding functionality, mentally run through:
+
+1. Is this a common problem? → Search npm/PyPI
+2. Is there an MCP for this? → Check `~/.claude/settings.json` and search
+3. Is there a skill for this? → Check `~/.claude/skills/`
+4. Is there a GitHub template? → Search GitHub
+
+### Full Mode (agent)
+
+For non-trivial functionality, launch the researcher agent:
+
+```
+Task(subagent_type="general-purpose", prompt="
+  Research existing tools for: [DESCRIPTION]
+  Language/framework: [LANG]
+  Constraints: [ANY]
+
+  Search: npm/PyPI, MCP servers, Claude Code skills, GitHub
+  Return: Structured comparison with recommendation
+")
+```
+
+## Search Shortcuts by Category
+
+### Development Tooling
+- Linting → `eslint`, `ruff`, `textlint`, `markdownlint`
+- Formatting → `prettier`, `black`, `gofmt`
+- Testing → `jest`, `pytest`, `go test`
+- Pre-commit → `husky`, `lint-staged`, `pre-commit`
+
+### AI/LLM Integration
+- Claude SDK → Context7 for latest docs
+- Prompt management → Check MCP servers
+- Document processing → `unstructured`, `pdfplumber`, `mammoth`
+
+### Data & APIs
+- HTTP clients → `httpx` (Python), `ky`/`got` (Node)
+- Validation → `zod` (TS), `pydantic` (Python)
+- Database → Check for MCP servers first
+
+### Content & Publishing
+- Markdown processing → `remark`, `unified`, `markdown-it`
+- Image optimization → `sharp`, `imagemin`
+
+## Integration Points
+
+### With planner agent
+The planner should invoke researcher before Phase 1 (Architecture Review):
+- Researcher identifies available tools
+- Planner incorporates them into the implementation plan
+- Avoids "reinventing the wheel" in the plan
+
+### With architect agent
+The architect should consult researcher for:
+- Technology stack decisions
+- Integration pattern discovery
+- Existing reference architectures
+
+### With iterative-retrieval skill
+Combine for progressive discovery:
+- Cycle 1: Broad search (npm, PyPI, MCP)
+- Cycle 2: Evaluate top candidates in detail
+- Cycle 3: Test compatibility with project constraints
+
+## Examples
+
+### Example 1: "Add dead link checking"
+```
+Need: Check markdown files for broken links
+Search: npm "markdown dead link checker"
+Found: textlint-rule-no-dead-link (score: 9/10)
+Action: ADOPT — npm install textlint-rule-no-dead-link
+Result: Zero custom code, battle-tested solution
+```
+
+### Example 2: "Add HTTP client wrapper"
+```
+Need: Resilient HTTP client with retries and timeout handling
+Search: npm "http client retry", PyPI "httpx retry"
+Found: got (Node) with retry plugin, httpx (Python) with built-in retry
+Action: ADOPT — use got/httpx directly with retry config
+Result: Zero custom code, production-proven libraries
+```
+
+### Example 3: "Add config file linter"
+```
+Need: Validate project config files against a schema
+Search: npm "config linter schema", "json schema validator cli"
+Found: ajv-cli (score: 8/10)
+Action: ADOPT + EXTEND — install ajv-cli, write project-specific schema
+Result: 1 package + 1 schema file, no custom validation logic
+```
+
+## Anti-Patterns
+
+- **Jumping to code**: Writing a utility without checking if one exists
+- **Ignoring MCP**: Not checking if an MCP server already provides the capability
+- **Over-customizing**: Wrapping a library so heavily it loses its benefits
+- **Dependency bloat**: Installing a massive package for one small feature