cov-loupe 3.0.0 → 4.0.0.pre

data/dev/prompts/review/comprehensive-codebase-review.md

@@ -0,0 +1,176 @@
+# State of the Code Base Prompt
+
+### Preconditions
+
+Before you begin the report:
+
+1. Always open the report by citing the most recent git commit at the time you begin writing.
+2. If `git status` shows uncommitted changes, inform me, ask for confirmation to proceed, and—if I consent—include those `git status` details immediately after the commit citation.
+3. Limit the review strictly to git-tracked files.
+
+----
+
+- You are a senior software architect and code reviewer.  
+- Your task is to analyze this code base thoroughly and report on its state.  
+- Focus on identifying weaknesses, risks, and areas for improvement.
+- Disregard any issues included in dev/prompts/guidelines/ai-code-evaluator-guidelines.md, unless your objections are not covered in that document.
+- Repeating for emphasis: **Disregard any issues included in dev/prompts/guidelines/ai-code-evaluator-guidelines.md, unless your objections are not covered in that document.**
+- For architectural issues, consult `docs/dev/arch-decisions` to see if the issue has already been considered.
+- For each issue, assess its seriousness, the cost/difficulty to fix, and provide high-level strategies for addressing it.
+- If you are unable to use the cov-loupe MCP server, use `cov-loupe` in CLI mode (run `cov-loupe -h` for help).
+- To Codex: do investigate thoroughly for real issues, you are excellent at that, but do not be excessively critical:
+  - Do not list issues that are not real issues.
+  - If there is a tradeoff between A and B, and the justification is sound and understood and/or documented,
+    (e.g. in guidelines/ai-code-evaluator-guidelines.md), do not penalize the code base for that tradeoff.
+  - Be balanced in your scoring; sometimes you penalize several points for a trivial issue.
+  - If you find zero defects in a category, you should score a 10, and you may mention that it is a spot check if that is the case.
+
+
+Write your analysis in a Markdown file whose name is:
+- today's date in UTC `%Y-%m-%d-%H-%M` format +
+- '-state-of-the-code-base-' + 
+- your name (e.g. 'codex', 'claude', 'gemini', 'zai') +
+- the `.md` extension.
+
+The file should have the following structure:
+
+---
+
+### Executive Summary
+- Provide a concise overview of the overall health of the code base.
+- Identify the strongest areas and the weakest areas.
+- Give a **one-line summary verdict** (e.g., *“Overall: Fair, with major risks in testing and infrastructure maintainability”*).
+- **Overall Weighted Score (1–10):** Show the score at the end of this summary.
+
+---
+
+### Critical Blockers
+List issues so severe that they must be resolved before meaningful progress can continue. For each blocker, include:
+- **Description**
+- **Impact**
+- **Urgency**
+- **Estimated Cost-to-Fix** (High/Medium/Low)
+
+---
+
+### Architecture & Design
+- Summarize the overall architecture (monolith, microservices, layered, etc.).
+- Identify strengths and weaknesses.
+- Highlight areas where complexity, coupling, or technical debt are high.
+- Assess maintainability, scalability, and clarity.
+- **Score (1–10)**
+
+---
+
+### Code Quality
+- Identify recurring issues (duplication, inconsistent style, long methods, deeply nested logic, etc.).
+- Point out readability and maintainability concerns.
+- **Score (1–10)**
+
+---
+
+### Infrastructure Code
+- Evaluate Dockerfiles, CI/CD pipelines, and Infrastructure-as-Code (Terraform, Ansible, etc.).
+- Highlight brittle or outdated configurations.
+- Identify risks in automation, deployment, or scaling.
+- **Score (1–10)**
+
+---
+
+### Dependencies & External Integrations
+- List major dependencies (frameworks, libraries, services).
+- Note outdated or risky dependencies and upgrade costs.
+- Assess vendor lock-in and integration fragility.
+- **Score (1–10)**
+
+---
+
+### Test Coverage
+- Using the **cov-loupe MCP server**, analyze the test coverage:
+    - Include a summary table of coverage by file/module.
+    - Report coverage at a high and general level.
+    - Rank risks of lacking coverage in **descending order of magnitude**.
+- Highlight untested critical paths and potential consequences.
+- Do not output the entire table in the report, but maybe the 10 least covered files if you believe it would be helpful.
+- **Score (1–10)**
+
+---
+
+### Security & Reliability
+- Identify insecure coding practices, hardcoded secrets, or missing validations.
+- Assess error handling, fault tolerance, and resilience.
+- **Score (1–10)**
+
+---
+
+### Documentation & Onboarding
+- Evaluate inline docs, README quality, and onboarding flow.
+- Identify missing/outdated documentation.
+- **Score (1–10)**
+
+---
+
+### Performance & Efficiency
+- Highlight bottlenecks or inefficient patterns.
+- Suggest whether optimizations are low-cost or high-cost.
+- **Score (1–10)**
+
+---
+
+### Formatting & Style Conformance
+- Report **bad or erroneous formatting** (inconsistent whitespace, broken Markdown).
+- Note whether style is consistent enough for maintainability.
+- **Score (1–10)**
+
+---
+
+### Best Practices & Conciseness
+- Assess whether the code follows recognized best practices (naming, modularization, separation of concerns).
+- Evaluate verbosity vs. clarity — is the code concise without being cryptic?
+- **Score (1–10)**
+
+---
+
+### Prioritized Issue List
+Provide a table of the top issues found, with the following columns:
+
+| Issue | Severity | Cost-to-Fix | Impact if Unaddressed |
+|-------|----------|-------------|------------------------|
+| Example issue description | High | Medium | Major reliability risks |
+
+The order should take both severity and cost-to-fix into account such that performing them in descending order would
+result in the optimal value addition velocity.
+
+---
+
+### High-Level Recommendations
+- Suggest general strategies for improvement (e.g., refactoring approach, improving test coverage, upgrading dependencies, modularization).
+- Highlight where incremental vs. large-scale changes are most appropriate.
+
+---
+
+### Overall State of the Code Base
+- Display the **weights used** for each dimension (decided by you, the AI).
+- Show the **weights table** and the weighted score calculation.
+
+### Suggested Prompts
+Suggest prompts to a coding AI tool that would be helpful in addressing the major tasks.
+
+#### Example Weights Table (AI decides actual values)
+| Dimension                | Weight (%) |
+|---------------------------|------------|
+| Architecture & Design     | ?%         |
+| Code Quality              | ?%         |
+| Infrastructure Code       | ?%         |
+| Dependencies              | ?%         |
+| Test Coverage             | ?%         |
+| Security & Reliability    | ?%         |
+| Documentation             | ?%         |
+| Performance & Efficiency  | ?%         |
+| Formatting & Style        | ?%         |
+| Best Practices & Conciseness | ?%      |
+
+- **Weighted Score Calculation:** Multiply each section's score by its chosen weight, then sum to compute the **Overall Weighted Score (1–10)**.
+- Report the final **Overall Weighted Score** with justification.
+
+### Summarize suggested changes

data/dev/prompts/review/identify-action-items.md

@@ -0,0 +1,143 @@
+# Identify Action Items
+
+**Purpose:** Identify and prioritize issues that need fixing NOW. This is a problems-focused review (not a balanced assessment).
+
+## When to Use This
+
+- Planning sprints or development cycles
+- Identifying technical debt to tackle
+- Preparing bug-fix releases
+- Answering: "What should we fix next?"
+
+For a balanced assessment showing strengths AND weaknesses, use `comprehensive-codebase-review.md` instead.
+
+---
+
+## Preconditions
+
+Before you begin the report:
+
+1. Always open the report by citing the most recent git commit at the time you begin writing.
+2. Limit the review strictly to git-tracked files.
+3. If `git status` shows uncommitted changes, inform me, ask for confirmation to proceed, and—if I consent—include those `git status` details immediately after the commit citation.
+
+---
+
+## Your Role & Task
+
+- You are a senior software architect and code reviewer.
+- Your task is to analyze this code base thoroughly and identify issues needing immediate attention.
+- Focus on: bugs, defects, security vulnerabilities, performance bottlenecks, technical debt, weaknesses, risks, ambiguities, and other areas requiring improvement.
+- **This is NOT a balanced review** - focus on problems, not strengths.
+
+---
+
+## Exclusions & Balance Guidance
+
+### Consult Guidelines First
+
+**CRITICAL:** 
+
+- Disregard any issues included in `dev/prompts/guidelines/ai-code-evaluator-guidelines.md` unless your objections are not covered in that document.
+- For architectural issues, consult `docs/dev/arch-decisions` to see if the issue has already been considered.
+
+**Repeating for emphasis:** Disregard any issues included in `dev/prompts/guidelines/ai-code-evaluator-guidelines.md` unless your objections are not covered in that document.
+
+### Be Balanced (Not Excessively Critical)
+
+- Do not list issues that are not real issues.
+- If there is a tradeoff between A and B, and the justification is sound and documented (e.g., in ai-code-evaluator-guidelines.md), do not penalize the code base for that tradeoff.
+- Be fair in your severity assessments; sometimes trivial issues are overweighted.
+- Investigate thoroughly for real issues, but maintain perspective on their actual impact.
+
+---
+
+## Tooling
+
+- Use the `cov-loupe` MCP server *as an MCP server* (not a command line application with args) to find information about test coverage.
+- Only if you are unable to use the cov-loupe MCP server, use `cov-loupe` in CLI mode (run `cov-loupe -h` for help).
+
+---
+
+## Report Structure
+
+### For Each Issue Found
+
+Delimit each issue with horizontal lines and headlines. Number each issue.
+
+**Required sections:**
+
+1. **Headline & Description:** Clear, concise explanation of the issue.
+2. **Assessment:**
+   - **Severity:** High/Medium/Low
+   - **Effort to Fix:** High/Medium/Low
+   - **Impact if Unaddressed:** What happens if we don't fix this?
+3. **Strategy:** High-level approach for addressing the issue.
+4. **Actionable Prompt:** Provide a specific, copy-paste-ready prompt that can be given to an AI coding agent to fix or improve this specific issue.
+
+**Example format:**
+
+```markdown
+---
+
+## Issue: Insecure Password Storage
+
+### Description
+User passwords are stored in plain text in the database.
+
+### Assessment
+- **Severity:** High
+- **Effort to Fix:** Medium
+- **Impact if Unaddressed:** Critical security vulnerability; user accounts easily compromised.
+
+### Strategy
+Replace plain text storage with bcrypt hashing. Update authentication logic to hash passwords on registration and verify hashed passwords on login.
+
+### Actionable Prompt
+```
+Update the user authentication system to use bcrypt for password hashing. Specifically:
+1. Add bcrypt gem to Gemfile
+2. Update User model to hash passwords before saving
+3. Update authentication logic to compare hashed passwords
+4. Add migration to hash existing plain text passwords
+```
+```
+
+---
+
+### Summary Table
+
+At the end of the file, produce a markdown table that summarizes ALL issues, ordered by priority (considering severity, effort, and impact):
+
+| Brief Description (<= 50 chars) | Severity (H/M/L) | Effort (H/M/L) | Impact if Unaddressed | Link to Detail |
+| :--- | :---: | :---: | :--- | :--- |
+| ... | ... | ... | ... | [See below](#issue-title) |
+
+**Priority ordering:** Issues should be ordered to maximize value. Generally this means:
+- Critical severity with low-to-medium effort → highest priority
+- High severity regardless of effort → high priority
+- Medium severity with low effort → medium-high priority
+- Low severity with high effort → lowest priority
+
+Use your judgment to order issues for optimal value delivery.
+
+---
+
+## Output File
+
+Write your analysis in a Markdown file whose name is:
+
+- today's date in UTC `%Y-%m-%d-%H-%M` format +
+- `-prioritized-action-items-` +
+- your name (e.g. `codex`, `claude`, `gemini`, `zai`) +
+- the `.md` extension.
+
+**Example:** `2026-01-08-19-45-prioritized-action-items-claude.md`
+
+---
+
+## Constraints
+
+- **DO NOT MAKE ANY CODE CHANGES. REVIEW ONLY.**
+- Focus exclusively on identifying and prioritizing problems.
+- Do not include "strengths" or "what's going well" sections.

data/dev/prompts/review/verify-code-changes.md

@@ -0,0 +1,54 @@
+# Verify Code Changes
+
+I need you to test that the intention of a code change was accomplished successfully, i.e. that the change is:
+
+- correct
+- complete
+- concise
+- conforms to best practices
+- as simple as possible
+- is not more easily accomplished using tools available that might not have been considered, especially since coding tools do not generally search the web
+- is adequately tested (for Ruby Simplecov coverage, use the cov-loupe MCP server)
+
+### Parameters (I Will Give You...)
+
+- Comparison Specification (an argument to `git diff`) - the point of reference from which you can do a git diff to see the changes. Examples:
+  - a commit (e.g. HEAD, HEAD~~, HEAD~4, 45076963221647d724b9b52faa3690a6d83ae8d1)
+  - a branch name
+  - a tag
+  - anything else that can be passed to `git diff`
+- A description of the intended change. Examples of changes:
+  - Implemented feature
+  - Fixed bug
+  - Test code added
+  - Documentation task
+
+If I do not give you the compare point in this prompt, ask me for them.
+
+If I do not give you the intent to examine, use the commit message(s) and say that
+you are doing that so I can be prodded in case that was not my intent.
+
+Be mindful of the signal to noise ratio. Do not add anything to the report
+unless it adds value to the reader. Here is an example of a time wasting comment:
+
+"Approach X could have been used, but the current implementation is a better fit."
+
+----
+
+> Sure, I can verify the code changes for you. I will need you to give me:
+> - the commit, branch, etc. for me to use as the starting point of the comparison
+> - a description of the code change intention to verify
+----
+
+### Your Response
+
+Examine the diff thoroughly. In your response, be balanced, fair, organized, and thorough.
+
+Write your response as markdown text and save it to a file whose name is:
+
+- today's date in UTC `%Y-%m-%d-%H-%M` format +
+- your name (e.g. 'codex', 'claude', 'gemini', 'zai') +
+- "-code-review-#{change_intention_phrase}.md"
+
+
+

data/dev/prompts/validate/create-screencast-outline.md

@@ -0,0 +1,234 @@
+# Create Screencast Outline
+
+**Purpose:** Design a compelling 2-5 minute screencast that showcases cov-loupe's value, especially its AI integration capabilities.
+
+## When to Use This
+
+- Planning marketing materials
+- Preparing demo videos
+- Creating tutorial content
+- Showcasing the tool at conferences or meetups
+
+## Goals
+
+1. **Highlight AI Integration:** Show how AI assistants can use cov-loupe to reason deeply about test coverage
+2. **Demonstrate Unique Value:** Illustrate advantages over directly analyzing `.resultset.json` files
+3. **Show Practical Use Cases:** Present real-world scenarios where cov-loupe solves actual problems
+4. **Keep It Engaging:** Maintain viewer interest with concrete, impressive examples
+5. **Stay Concise:** Fit key points into 2-5 minutes
+
+## Research Phase
+
+Before creating the outline, study:
+
+### Documentation to Review
+- `README.md` - Overview and main features
+- `docs/user/**/*.md` - User-facing functionality
+- `docs/dev/**/*.md` - Advanced features and architecture
+
+### Key Questions to Answer
+1. **What makes cov-loupe special?** What can it do that manual JSON inspection can't?
+2. **What are the "wow moments"?** Features that clearly demonstrate value
+3. **How does AI integration work?** What prompts leverage cov-loupe effectively?
+4. **What problems does it solve?** Real pain points in coverage analysis
+
+## Deliverable Structure
+
+Create an outline with the following sections:
+
+### 1. Hook (10-15 seconds)
+- **Goal:** Grab attention immediately
+- **Content:** One compelling statement or question
+- **Example:** "What if your AI assistant could deeply understand your test coverage?"
+
+### 2. The Problem (20-30 seconds)
+- **Goal:** Establish why this matters
+- **Content:** The pain points cov-loupe addresses
+- **Questions to answer:**
+  - What's frustrating about traditional coverage analysis?
+  - Why is raw `.resultset.json` analysis problematic?
+  - What questions go unanswered with existing tools?
+
+### 3. The Solution (30-45 seconds)
+- **Goal:** Introduce cov-loupe
+- **Content:** Quick overview of what it does
+- **Key points:**
+  - CLI + MCP server architecture
+  - Structured data for AI assistants
+  - Multiple output formats
+  - Staleness detection
+
+### 4. Demo - "Wow Moments" (90-120 seconds)
+- **Goal:** Show concrete, impressive use cases
+- **Content:** 2-3 demonstrations that showcase unique value
+
+**For each demo:**
+- **Setup:** Briefly describe the scenario (5-10 sec)
+- **Action:** Show the command or AI prompt (10-15 sec)
+- **Result:** Highlight the valuable output (10-15 sec)
+- **Value:** Explain why this matters (5-10 sec)
+
+**Suggested demo types:**
+- AI assistant analyzing coverage and suggesting where to add tests
+- Identifying stale coverage data automatically
+- Finding files with coverage < 80% and generating actionable insights
+- Comparing coverage across different areas of codebase
+- AI-generated test strategy based on coverage gaps
+
+### 5. AI Integration Highlight (45-60 seconds)
+- **Goal:** Showcase the MCP server + AI workflow
+- **Content:** Show an AI assistant using cov-loupe as a tool
+- **Example flow:**
+  - User asks: "Which files need more test coverage?"
+  - AI uses cov-loupe tools to analyze
+  - AI provides prioritized list with reasoning
+  - AI suggests specific test scenarios
+
+### 6. Call to Action (15-20 seconds)
+- **Goal:** Drive next steps
+- **Content:** How to get started
+- **Include:**
+  - Installation command
+  - Link to documentation
+  - Where to find more examples
+
+## Amazing Use Cases to Find
+
+Look for scenarios that demonstrate:
+
+### Deep Analysis
+- **Coverage gap identification:** Finding untested edge cases
+- **Risk assessment:** Identifying high-value files with low coverage
+- **Trend analysis:** Tracking coverage changes over time
+
+### AI-Powered Insights
+- **Smart prioritization:** AI ranks files by coverage urgency + business impact
+- **Test strategy generation:** AI suggests what types of tests to write
+- **Coverage archaeology:** AI explains why certain code is untested
+
+### Developer Workflow
+- **Pre-commit checks:** Catching coverage drops before they merge
+- **Code review assistance:** AI comments on coverage in PRs
+- **Refactoring safety:** Verifying test coverage before major changes
+
+### Advantages Over Raw JSON
+- **Structured queries:** Easy access to specific metrics
+- **Path resolution:** Handles relative vs absolute paths
+- **Staleness detection:** Knows when coverage is out of date
+- **Multiple formats:** JSON, YAML, tables, detailed views
+- **AI-friendly:** Purpose-built for tool integration
+
+## Output Format
+
+Produce a structured outline like:
+
+```markdown
+# Screencast Outline: "cov-loupe + AI: Intelligent Test Coverage Analysis"
+
+**Total Duration:** 4:30
+
+## Section 1: Hook (0:00 - 0:15)
+"Your AI assistant can now understand your test coverage better than ever."
+
+[Screen: Show AI assistant interface with cov-loupe integration]
+
+## Section 2: The Problem (0:15 - 0:45)
+- Coverage reports are just numbers
+- .resultset.json is hard to parse
+- No context about staleness
+- AI assistants struggle with raw JSON
+
+[Screen: Show confusing .resultset.json file]
+
+## Section 3: The Solution (0:45 - 1:15)
+cov-loupe provides:
+- Clean CLI interface
+- MCP server for AI integration
+- Multiple output formats
+- Smart staleness detection
+
+[Screen: Quick demo of `cov-loupe list` command]
+
+## Section 4: Demo 1 - AI-Powered Coverage Analysis (1:15 - 2:00)
+**Scenario:** Find files that need tests
+
+**Action:**
+User: "Which files have the worst test coverage?"
+
+AI (using cov-loupe):
+- Queries coverage data
+- Identifies bottom 5 files
+- Provides context about each
+
+**Result:**
+Prioritized list with:
+- Coverage percentages
+- File purposes
+- Suggested test scenarios
+
+**Value:** Goes beyond numbers to actionable insights
+
+[Screen: Show AI interaction]
+
+## Section 5: Demo 2 - Stale Coverage Detection (2:00 - 2:35)
+**Scenario:** Verify coverage is up-to-date
+
+**Action:** `cov-loupe list --raise-on-stale`
+
+**Result:**
+- Detects modified files
+- Shows timestamp mismatches
+- Prevents false confidence
+
+**Value:** Ensures coverage data reflects current code
+
+[Screen: Show stale detection in action]
+
+## Section 6: AI Integration Deep Dive (2:35 - 3:30)
+**Scenario:** Generate test strategy for uncovered code
+
+**Action:**
+User: "Help me improve coverage for the authentication module"
+
+AI workflow:
+1. Uses `coverage_summary_tool` for auth files
+2. Identifies uncovered lines
+3. Analyzes code context
+4. Suggests specific test cases
+
+**Result:**
+Detailed test plan with:
+- Edge cases to cover
+- Mock requirements
+- Assertion suggestions
+
+**Value:** AI + cov-loupe = smarter test planning
+
+[Screen: Show full AI interaction]
+
+## Section 7: Call to Action (3:30 - 4:30)
+Get started:
+```bash
+gem install cov-loupe
+cov-loupe --help
+```
+
+Learn more:
+- Documentation: [link]
+- MCP integration guide: [link]
+- Example prompts: [link]
+
+[Screen: Show website/GitHub repo]
+```
+
+## Notes
+
+- **Find real examples:** Use actual cov-loupe features, not hypotheticals
+- **Show, don't tell:** Prefer demonstrations over explanations
+- **Keep pace brisk:** 2-5 minutes goes quickly, cut ruthlessly
+- **Emphasize AI value:** This is the key differentiator
+- **Make it reproducible:** Viewers should be able to try examples themselves
+
+## Time Estimates
+
+Provide timing for each section to ensure the total stays within 2-5 minutes. Be realistic about how long demonstrations take.