engsys 1.0.0

package/core/skills/agentic-eval/SKILL.md

@@ -0,0 +1,195 @@
+---
+name: agentic-eval
+description: |
+  Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:
+  - Implementing self-critique and reflection loops
+  - Building evaluator-optimizer pipelines for quality-critical generation
+  - Creating test-driven code refinement workflows
+  - Designing rubric-based or LLM-as-judge evaluation systems
+  - Adding iterative improvement to agent outputs (code, reports, analysis)
+  - Measuring and improving agent response quality
+---
+
+# Agentic Evaluation Patterns
+
+Patterns for self-improvement through iterative evaluation and refinement.
+
+## Overview
+
+Evaluation patterns enable agents to assess and improve their own outputs, moving beyond single-shot generation to iterative refinement loops.
+
+```text
+Generate → Evaluate → Critique → Refine → Output
+    ↑                              │
+    └──────────────────────────────┘
+```
+
+## When to Use
+
+- **Quality-critical generation**: Code, reports, analysis requiring high accuracy
+- **Tasks with clear evaluation criteria**: Defined success metrics exist
+- **Content requiring specific standards**: Style guides, compliance, formatting
+
+---
+
+## Pattern 1: Basic Reflection
+
+Agent evaluates and improves its own output through self-critique.
+
+```python
+def reflect_and_refine(task: str, criteria: list[str], max_iterations: int = 3) -> str:
+    """Generate with reflection loop."""
+    output = llm(f"Complete this task:\n{task}")
+
+    for i in range(max_iterations):
+        # Self-critique
+        critique = llm(f"""
+        Evaluate this output against criteria: {criteria}
+        Output: {output}
+        Rate each: PASS/FAIL with feedback as JSON.
+        """)
+
+        critique_data = json.loads(critique)
+        all_pass = all(c["status"] == "PASS" for c in critique_data.values())
+        if all_pass:
+            return output
+
+        # Refine based on critique
+        failed = {k: v["feedback"] for k, v in critique_data.items() if v["status"] == "FAIL"}
+        output = llm(f"Improve to address: {failed}\nOriginal: {output}")
+
+    return output
+```
+
+**Key insight**: Use structured JSON output for reliable parsing of critique results.
+
+---
+
+## Pattern 2: Evaluator-Optimizer
+
+Separate generation and evaluation into distinct components for clearer responsibilities.
+
+```python
+class EvaluatorOptimizer:
+    def __init__(self, score_threshold: float = 0.8):
+        self.score_threshold = score_threshold
+
+    def generate(self, task: str) -> str:
+        return llm(f"Complete: {task}")
+
+    def evaluate(self, output: str, task: str) -> dict:
+        return json.loads(llm(f"""
+        Evaluate output for task: {task}
+        Output: {output}
+        Return JSON: {{"overall_score": 0-1, "dimensions": {{"accuracy": ..., "clarity": ...}}}}
+        """))
+
+    def optimize(self, output: str, feedback: dict) -> str:
+        return llm(f"Improve based on feedback: {feedback}\nOutput: {output}")
+
+    def run(self, task: str, max_iterations: int = 3) -> str:
+        output = self.generate(task)
+        for _ in range(max_iterations):
+            evaluation = self.evaluate(output, task)
+            if evaluation["overall_score"] >= self.score_threshold:
+                break
+            output = self.optimize(output, evaluation)
+        return output
+```
+
+---
+
+## Pattern 3: Code-Specific Reflection
+
+Test-driven refinement loop for code generation.
+
+```python
+class CodeReflector:
+    def reflect_and_fix(self, spec: str, max_iterations: int = 3) -> str:
+        code = llm(f"Write Python code for: {spec}")
+        tests = llm(f"Generate pytest tests for: {spec}\nCode: {code}")
+
+        for _ in range(max_iterations):
+            result = run_tests(code, tests)
+            if result["success"]:
+                return code
+            code = llm(f"Fix error: {result['error']}\nCode: {code}")
+        return code
+```
+
+---
+
+## Evaluation Strategies
+
+### Outcome-Based
+
+Evaluate whether output achieves the expected result.
+
+```python
+def evaluate_outcome(task: str, output: str, expected: str) -> str:
+    return llm(f"Does output achieve expected outcome? Task: {task}, Expected: {expected}, Output: {output}")
+```
+
+### LLM-as-Judge
+
+Use LLM to compare and rank outputs.
+
+```python
+def llm_judge(output_a: str, output_b: str, criteria: str) -> str:
+    return llm(f"Compare outputs A and B for {criteria}. Which is better and why?")
+```
+
+### Rubric-Based
+
+Score outputs against weighted dimensions.
+
+```python
+RUBRIC = {
+    "accuracy": {"weight": 0.4},
+    "clarity": {"weight": 0.3},
+    "completeness": {"weight": 0.3}
+}
+
+def evaluate_with_rubric(output: str, rubric: dict) -> float:
+    scores = json.loads(llm(f"Rate 1-5 for each dimension: {list(rubric.keys())}\nOutput: {output}"))
+    return sum(scores[d] * rubric[d]["weight"] for d in rubric) / 5
+```
+
+---
+
+## Best Practices
+
+| Practice              | Rationale                                               |
+| --------------------- | ------------------------------------------------------- |
+| **Clear criteria**    | Define specific, measurable evaluation criteria upfront |
+| **Iteration limits**  | Set max iterations (3-5) to prevent infinite loops      |
+| **Convergence check** | Stop if output score isn't improving between iterations |
+| **Log history**       | Keep full trajectory for debugging and analysis         |
+| **Structured output** | Use JSON for reliable parsing of evaluation results     |
+
+---
+
+## Quick Start Checklist
+
+```markdown
+## Evaluation Implementation Checklist
+
+### Setup
+
+- [ ] Define evaluation criteria/rubric
+- [ ] Set score threshold for "good enough"
+- [ ] Configure max iterations (default: 3)
+
+### Implementation
+
+- [ ] Implement generate() function
+- [ ] Implement evaluate() function with structured output
+- [ ] Implement optimize() function
+- [ ] Wire up the refinement loop
+
+### Safety
+
+- [ ] Add convergence detection
+- [ ] Log all iterations for debugging
+- [ ] Handle evaluation parse failures gracefully
+```

package/core/skills/chrome-devtools/SKILL.md

@@ -0,0 +1,97 @@
+---
+name: chrome-devtools
+description: "Expert-level browser automation, debugging, and performance analysis using Chrome DevTools MCP. Use for interacting with web pages, capturing screenshots, analyzing network traffic, and profiling performance."
+license: MIT
+---
+
+# Chrome DevTools Agent
+
+## Overview
+
+A specialized skill for controlling and inspecting a live Chrome browser. This skill leverages the `chrome-devtools` MCP server to perform a wide range of browser-related tasks, from simple navigation to complex performance profiling.
+
+## When to Use
+
+Use this skill when:
+
+- **Browser Automation**: Navigating pages, clicking elements, filling forms, and handling dialogs.
+- **Visual Inspection**: Taking screenshots or text snapshots of web pages.
+- **Debugging**: Inspecting console messages, evaluating JavaScript in the page context, and analyzing network requests.
+- **Performance Analysis**: Recording and analyzing performance traces to identify bottlenecks and Core Web Vital issues.
+- **Emulation**: Resizing the viewport or emulating network/CPU conditions.
+
+## Tool Categories
+
+### 1. Navigation & Page Management
+
+- `new_page`: Open a new tab/page.
+- `navigate_page`: Go to a specific URL, reload, or navigate history.
+- `select_page`: Switch context between open pages.
+- `list_pages`: See all open pages and their IDs.
+- `close_page`: Close a specific page.
+- `wait_for`: Wait for specific text to appear on the page.
+
+### 2. Input & Interaction
+
+- `click`: Click on an element (use `uid` from snapshot).
+- `fill` / `fill_form`: Type text into inputs or fill multiple fields at once.
+- `hover`: Move the mouse over an element.
+- `press_key`: Send keyboard shortcuts or special keys (e.g., "Enter", "Control+C").
+- `drag`: Drag and drop elements.
+- `handle_dialog`: Accept or dismiss browser alerts/prompts.
+- `upload_file`: Upload a file through a file input.
+
+### 3. Debugging & Inspection
+
+- `take_snapshot`: Get a text-based accessibility tree (best for identifying elements).
+- `take_screenshot`: Capture a visual representation of the page or a specific element.
+- `list_console_messages` / `get_console_message`: Inspect the page's console output.
+- `evaluate_script`: Run custom JavaScript in the page context.
+- `list_network_requests` / `get_network_request`: Analyze network traffic and request details.
+
+### 4. Emulation & Performance
+
+- `resize_page`: Change the viewport dimensions.
+- `emulate`: Throttling CPU/Network or emulating geolocation.
+- `performance_start_trace`: Start recording a performance profile.
+- `performance_stop_trace`: Stop recording and save the trace.
+- `performance_analyze_insight`: Get detailed analysis from recorded performance data.
+
+## Workflow Patterns
+
+### Pattern A: Identifying Elements (Snapshot-First)
+
+Always prefer `take_snapshot` over `take_screenshot` for finding elements. The snapshot provides `uid` values which are required by interaction tools.
+
+```markdown
+1. `take_snapshot` to get the current page structure.
+2. Find the `uid` of the target element.
+3. Use `click(uid=...)` or `fill(uid=..., value=...)`.
+```
+
+### Pattern B: Troubleshooting Errors
+
+When a page is failing, check both console logs and network requests.
+
+```markdown
+1. `list_console_messages` to check for JavaScript errors.
+2. `list_network_requests` to identify failed (4xx/5xx) resources.
+3. `evaluate_script` to check the value of specific DOM elements or global variables.
+```
+
+### Pattern C: Performance Profiling
+
+Identify why a page is slow.
+
+```markdown
+1. `performance_start_trace(reload=true, autoStop=true)`
+2. Wait for the page to load/trace to finish.
+3. `performance_analyze_insight` to find LCP issues or layout shifts.
+```
+
+## Best Practices
+
+- **Context Awareness**: Always run `list_pages` and `select_page` if you are unsure which tab is currently active.
+- **Snapshots**: Take a new snapshot after any major navigation or DOM change, as `uid` values may change.
+- **Timeouts**: Use reasonable timeouts for `wait_for` to avoid hanging on slow-loading elements.
+- **Screenshots**: Use `take_screenshot` sparingly for visual verification, but rely on `take_snapshot` for logic.

package/core/skills/code-review/SKILL.md

@@ -0,0 +1,26 @@
+---
+name: code-review
+description: 'Local code review before push. Default code-review skill. Trigger for any explicit review request AND autonomously when the agent thinks a review is needed (code/PR/quality/security).'
+metadata:
+  version: '0.2.0'
+---
+
+# Local Code Review
+
+Local pre-push review uses the **built-in `/code-review` skill**. Run a local code review with
+`/code-review` before `git push` so PRs open already-reviewed and CI minutes aren't spent on a
+post-push review loop.
+
+## How to Review
+
+1. `git fetch origin` so `origin/main` is current before scoping the diff.
+2. Run a local code review with the built-in `/code-review` skill against `origin/main`.
+3. Triage findings by severity — fix **Critical** + **Warning**; **Info** at discretion.
+4. Re-run to confirm clean; cap at ~2 passes.
+5. After opening the PR, persist the local review findings as a PR comment so the closeout
+   ceremony can mine them.
+
+For deeper, security-focused passes, the built-in `/security-review` command is also available.
+
+> Note: prefer the built-in `/code-review` skill for local pre-push review rather than a
+> post-push cloud review tool, so PRs open already-reviewed.