codescoop 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 html-scan contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,249 @@
+
+# CodeScoop
+
+> **Scoop the exact code you need for component debugging and migration.**
+
+[![npm version](https://img.shields.io/npm/v/codescoop.svg)](https://www.npmjs.com/package/codescoop)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Stop guessing which CSS rules apply to your HTML. Stop copy-pasting 5,000 lines of code.**
+
+CodeScoop is a forensic tool for your frontend. It extracts the dependency graph of a single component—HTML, specific CSS rules, JS events, and variables—so you can debug legacy sites or feed clean context to an LLM.
+
+## The Problem
+
+You are trying to fix a bug in a legacy WordPress header, or convert it to Next.js.
+- **The Debugging Nightmare:** You see `class="btn"`, but where is the style coming from? `style.css`? `bootstrap.min.css`? An inline script?
+- **The AI Nightmare:** You paste the HTML into ChatGPT. It hallucinates CSS that doesn't exist or misses the JS logic entirely because you didn't paste the right file.
+
+## The Solution
+
+```bash
+codescoop index.html -s "header"
+```
+
+**One command. Complete context.**
+
+CodeScoop analyzes your project and generates a forensic report containing:
+
+* The exact HTML structure
+* **The Winning CSS:** Specificity scoring shows you exactly which rule wins (and which are overridden)
+* **Ghost Classes:** Identifies classes in HTML that have *no* matching CSS
+* **JS Forensics:** Finds jQuery listeners and vanilla JS events targeting your elements
+* **Variable Resolution:** Inlines values for `var(--primary)` so you don't need to hunt them down
+
+---
+
+## Key Features
+
+### CSS Conflict Detection
+
+Don't just list files. **Solve specificity wars.** CodeScoop calculates specificity scores for every matching rule so you can see exactly why your style isn't applying.
+
+### Ghost Class Detection
+
+Cleaning up legacy code? CodeScoop flags classes in your HTML that have **zero matching CSS rules** in your project. Delete them with confidence.
+
+### Smart Extraction
+
+It doesn't dump the whole file. It extracts *only* the rules that affect your specific component.
+
+* *Result:* A 50-line context file instead of a 5,000-line dump.
+
+### React Conversion Mode (`--for-conversion`)
+
+If you *are* migrating, this flag analyzes JS patterns and suggests:
+
+* State variables (`useState`)
+* Event handlers to implement
+* Animation libraries to install
+
+### Works With Everything
+
+Static HTML, **Live URLs**, PHP, Blade, Twig, EJS, ERB, Handlebars, JSP, ASP.
+
+### Advanced Modern CSS Support
+
+* **Shadow DOM:** Native support for `::part()` and `::slotted()` selectors allows you to analyze Web Components styling.
+* **CSS Houdini:** Detects and reports custom properties defined with `@property`, preserving their type syntax and initial values.
+
+---
+
+## Quick Start
+
+```bash
+npm install -g codescoop
+
+# 1. Debug a specific component
+codescoop page.html -s ".navbar"
+
+# 2. Analyze a live site (e.g., WordPress local dev)
+codescoop http://localhost:8000 -s ".hero" --dir ./wp-content/themes/mytheme
+
+# 3. Get a migration-ready bundle for AI
+codescoop page.html -s ".card" --for-conversion
+```
+
+---
+
+## Usage Scenarios
+
+### 1. The "Why is this not working?" Audit
+
+You have a button that refuses to turn blue.
+
+```bash
+codescoop index.html -s ".btn-primary"
+```
+
+**Output:**
+
+```markdown
+## CSS Conflict Analysis for `.btn-primary`
+
+| Status | Specificity | File | Rule |
+|--------|-------------|------|------|
+| Winner | **(0, 2, 0)** | `theme.css:45` | `background: red !important;` |
+| Overridden | **(0, 1, 0)** | `main.css:12` | `background: blue;` |
+| Overridden | **(0, 0, 1)** | `reset.css:5` | `background: gray;` |
+```
+
+*Verdict: `theme.css` is overriding your change with `!important`.*
+
+### 2. The Legacy Cleanup
+
+You inherited a 5-year-old site. You want to know which classes are useless.
+
+```bash
+codescoop old-page.html -s "main"
+```
+
+**Output:**
+
+```markdown
+## Ghost Classes Detected
+These classes appear in the HTML but have NO matching CSS in the project:
+
+| Class | Location |
+|-------|----------|
+| `.clearfix-old` | `line 45` |
+| `.legacy-wrapper-v2` | `line 102` |
+| `.hidden-xs` | `line 15` |
+```
+
+*Action: Safe to delete these from your HTML.*
+
+### 3. The "Perfect Context" for AI
+
+You want to convert a widget to React.
+
+```bash
+codescoop widget.php -s ".sidebar-widget" --for-conversion
+```
+
+**Output:**
+
+* **Clean HTML:** JSX-ready structure
+* **Scoped CSS:** Only the rules needed for this widget
+* **JS Hints:** "Found `$('.widget').on('click')` -> Implement `onClick` handler"
+* **Variables:** `var(--spacing)` resolved to `16px`
+
+*Result: Paste this into Claude/GPT and get a working component on the first try.*
+
+---
+
+## How It Works
+
+```
+┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
+│  Your Input     │───▶│    CodeScoop     │───▶│  Forensic Rep   │
+│  HTML/URL/PHP   │    │                  │    │                 │
+└─────────────────┘    │  ┌────────────┐  │    │  • Specificity  │
+                       │  │ Parse HTML │  │    │  • Ghost Classes│
+┌─────────────────┐    │  ├────────────┤  │    │  • JS Events    │
+│  Project Files  │───▶│  │ Find CSS   │  │    │  • Variables    │
+│  CSS/SCSS/JS    │    │  ├────────────┤  │    │  • Clean Code   │
+└─────────────────┘    │  │ Calc Score │  │    └─────────────────┘
+                       │  ├────────────┤  │
+                       │  │ Find Ghosts│  │
+                       │  └────────────┘  │
+                       └──────────────────┘
+```
+
+---
+
+## CLI Reference
+
+```bash
+codescoop <source> [options]
+```
+
+| Option | Short | Description |
+| --- | --- | --- |
+| `--selector <sel>` | `-s` | CSS selector to target |
+| `--dir <path>` | `-d` | Project directory to scan (Required for live URLs) |
+| `--for-conversion` |  | Add React/Next.js migration hints |
+| `--compact` | `-c` | Minimal output (hides long code blocks) |
+| `--summary-only` |  | Just list files and ghost classes |
+| `--skip-minified` |  | Exclude `*.min.css` / `*.min.js` |
+| `--max-rules <n>` |  | Max CSS rules per file (Default: 50) |
+
+---
+
+## Why Not Just Copy-Paste?
+
+| Feature | Manual Copy-Paste | CodeScoop |
+| --- | --- | --- |
+| **Context** | Partial (whatever you see) | Complete (hidden dependencies) |
+| **CSS Conflicts** | Guesswork | **Specificity Scoring** |
+| **Dead Code** | Hard to find | **Ghost Class Detection** |
+| **Variables** | Undefined | Auto-resolved |
+| **Time** | 15+ mins per component | < 10 seconds |
+
+---
+
+---
+
+## How CodeScoop Handles Complexity (The "Black Holes")
+
+Frontend analysis is hard. Here is how CodeScoop solves the common "black holes" that break other tools:
+
+### 1. The "Dynamic Class" Trap (`.menu.is-open`)
+*   **Problem:** Classes like `.is-open` are often added by JS and missing from static HTML.
+*   **Solution:** CodeScoop uses **Greedy Prefix Matching**. If you target `.menu`, we automatically find `.menu.is-open`, `.menu:hover`, and `.menu::before`. We also filter common state classes (`is-*`, `has-*`) to prevent false alarms.
+
+### 2. Nested SCSS (`&__element`)
+*   **Problem:** BEM syntax like `&__item` is invisible to standard regex searches.
+*   **Solution:** We implement **AST Resolution**. CodeScoop parses your SCSS, unwraps the `&` nesting, and resolves the actual selector (e.g., resolving `&__item` to `.block__item`) to ensure no rule is left behind.
+
+### 3. Tailwind & Runtime CSS
+*   **Problem:** Utility classes (`p-4`, `flex`) clutter reports and look like "missing" styles if not built.
+*   **Solution:** Our **Smart Filter** distinguishes between likely utility classes (Tailwind/Bootstrap patterns) and your actual missing custom styles, keeping your "Ghost Class" report clean and actionable.
+
+### 4. The Import Maze
+*   **Problem:** Styles buried 5 levels deep in `@import` chains are often missed.
+*   **Solution:** CodeScoop performs a **Deep Project Scan**, indexing every CSS/SCSS file in your directory to find relevant rules regardless of where they are imported.
+
+### 5. Advanced Modern CSS
+*   **Problem:** Modern features like Shadow DOM (`::part`) and CSS Houdini (`@property`) are often ignored by traditional parsers.
+*   **Solution:** CodeScoop includes native support for these features, correctly identifying `::part()` and `::slotted()` rules and extracting structured `@property` definitions.
+
+---
+
+## Contributing
+
+Found a bug? Want a feature? PRs welcome!
+
+```bash
+git clone https://github.com/yourusername/codescoop.git
+cd codescoop && npm install
+node bin/codescoop.js test/sample.html -s ".navbar"
+```
+
+---
+
+<p align="center">
+<b>Debug faster. Migrate smarter. Scoop exactly what you need.</b>
+
+<code>npm install -g codescoop</code>
+</p>

package/bin/codescoop.js

@@ -0,0 +1,276 @@
+#!/usr/bin/env node
+
+/**
+ * CodeScoop CLI
+ * Extract component dependencies for AI-powered React/Next.js conversion
+ */
+
+const { program } = require('commander');
+const chalk = require('chalk');
+const path = require('path');
+const fs = require('fs');
+const { runAnalysis } = require('../src/index');
+const { runInteractiveMode } = require('../src/cli/interactive');
+const {
+  validateHTMLFile,
+  validateOutputPath,
+  validateProjectDir,
+  sanitizeSelector,
+  formatError
+} = require('../src/utils/validation');
+const { fetchURL, isURL } = require('../src/utils/url-fetcher');
+const { isTemplateFile, parseTemplateFile } = require('../src/utils/template-parser');
+
+// ASCII Art Banner
+const banner = `
+${chalk.magenta('╔═══════════════════════════════════════════╗')}
+${chalk.magenta('║')}  ${chalk.bold.white('🍒 CodeScoop')} ${chalk.gray('- Scoop code for AI')}     ${chalk.magenta('║')}
+${chalk.magenta('╚═══════════════════════════════════════════╝')}
+`;
+
+program
+  .name('codescoop')
+  .description('Scoop out component dependencies for AI-powered conversion')
+  .version('1.0.0')
+  .argument('<source>', 'HTML file, URL, or template (.php, .blade.php, etc.)')
+  .option('-s, --selector <selector>', 'CSS selector to target (e.g., ".navbar", "#header")')
+  .option('-l, --lines <range>', 'Line range to target (e.g., "45-80")')
+  .option('-o, --output <path>', 'Output file path (default: <component>-analysis.md)')
+  .option('-d, --dir <path>', 'Project directory to scan (default: directory containing file)')
+  .option('-m, --match-index <n>', 'Which match to use if multiple elements found (0-based)', '0')
+  .option('-c, --compact', 'Compact mode: limit output size for LLM consumption')
+  .option('--for-conversion', 'Generate React/Next.js conversion context for LLMs')
+  .option('--max-rules <n>', 'Max CSS rules per file (default: 20 in compact mode)', '20')
+  .option('--max-js <n>', 'Max JS references per file (default: 10 in compact mode)', '10')
+  .option('--summary-only', 'Only show summary and file list, no code blocks')
+  .option('--skip-minified', 'Skip minified files (*.min.css, *.min.js)')
+  .option('--no-interactive', 'Skip interactive mode, require --selector or --lines')
+  .option('--include-inline', 'Include inline <style> and <script> blocks (default: true)', true)
+  .option('--verbose', 'Show detailed logging')
+  .action(async (source, options) => {
+    console.log(banner);
+
+    let htmlPath;
+    let htmlContent = null;
+    let projectDir;
+    let sourceType = 'file'; // file, url, or template
+
+    try {
+      // ============================================
+      // STEP 1: Detect source type and get HTML
+      // ============================================
+
+      if (isURL(source)) {
+        // URL Mode - fetch live page
+        sourceType = 'url';
+        console.log(chalk.cyan(`🌐 Fetching URL: ${source}`));
+
+        const result = await fetchURL(source);
+        if (result.statusCode !== 200) {
+          console.error(chalk.red(`✖  HTTP ${result.statusCode} - Failed to fetch URL`));
+          process.exit(1);
+        }
+
+        htmlContent = result.html;
+        htmlPath = source;
+        console.log(chalk.green(`✓ Fetched ${(htmlContent.length / 1024).toFixed(1)}KB`));
+
+        // For URL mode, project dir must be specified
+        projectDir = options.dir ? path.resolve(options.dir) : null;
+        if (!projectDir) {
+          console.log(chalk.yellow(`⚠️  No --dir specified. Only inline styles/scripts will be analyzed.`));
+          console.log(chalk.gray(`   Use --dir /path/to/project to scan local CSS/JS files.`));
+        }
+
+      } else {
+        // File Mode - local file
+        htmlPath = path.resolve(source);
+
+        // Check file exists
+        if (!fs.existsSync(htmlPath)) {
+          console.error(chalk.red(`✖  File not found: ${htmlPath}`));
+          process.exit(1);
+        }
+
+        // Check if it's a template file
+        if (isTemplateFile(htmlPath)) {
+          sourceType = 'template';
+          console.log(chalk.cyan(`📄 Parsing template: ${path.basename(htmlPath)}`));
+
+          const parsed = parseTemplateFile(htmlPath);
+          htmlContent = parsed.html;
+
+          if (parsed.warnings.length > 0) {
+            parsed.warnings.forEach(w => console.warn(chalk.yellow(`⚠️  ${w}`)));
+          }
+
+          if (parsed.dynamicClasses.length > 0) {
+            console.log(chalk.yellow(`⚠️  Found ${parsed.dynamicClasses.length} dynamic class attributes`));
+          }
+
+          console.log(chalk.green(`✓ Extracted HTML from ${parsed.templateType} template`));
+
+        } else {
+          // Standard HTML file
+          sourceType = 'file';
+          const validation = validateHTMLFile(htmlPath);
+
+          validation.warnings.forEach(w => console.warn(chalk.yellow(`⚠️  ${w}`)));
+
+          if (!validation.valid) {
+            validation.errors.forEach(e => console.error(chalk.red(`✖  ${e}`)));
+            process.exit(1);
+          }
+        }
+
+        // Set project directory
+        projectDir = options.dir ? path.resolve(options.dir) : path.dirname(htmlPath);
+
+        // Validate project directory
+        const dirValidation = validateProjectDir(projectDir);
+        dirValidation.warnings.forEach(w => console.warn(chalk.yellow(`⚠️  ${w}`)));
+
+        if (!dirValidation.valid) {
+          dirValidation.errors.forEach(e => console.error(chalk.red(`✖  ${e}`)));
+          process.exit(1);
+        }
+      }
+
+      // ============================================
+      // STEP 2: Validate options
+      // ============================================
+
+      // Validate output path if provided
+      if (options.output) {
+        const outputValidation = validateOutputPath(options.output);
+        outputValidation.warnings.forEach(w => console.warn(chalk.yellow(`⚠️  ${w}`)));
+
+        if (!outputValidation.valid) {
+          outputValidation.errors.forEach(e => console.error(chalk.red(`✖  ${e}`)));
+          process.exit(1);
+        }
+      }
+
+      // Validate selector if provided
+      if (options.selector) {
+        const selectorValidation = sanitizeSelector(options.selector);
+        if (!selectorValidation.valid) {
+          console.error(chalk.red(`✖  Invalid selector: ${selectorValidation.error}`));
+          process.exit(1);
+        }
+        options.selector = selectorValidation.selector;
+      }
+
+      // Validate match index
+      const matchIndex = parseInt(options.matchIndex, 10);
+      if (isNaN(matchIndex) || matchIndex < 0) {
+        console.error(chalk.red(`✖  Invalid match-index: "${options.matchIndex}". Must be a non-negative number.`));
+        process.exit(1);
+      }
+
+      // Verbose logging
+      if (options.verbose) {
+        console.log(chalk.gray(`Source type: ${sourceType}`));
+        console.log(chalk.gray(`Source: ${htmlPath}`));
+        if (projectDir) console.log(chalk.gray(`Project directory: ${projectDir}`));
+      }
+
+      // ============================================
+      // STEP 3: Interactive mode if needed
+      // ============================================
+
+      if (!options.selector && !options.lines) {
+        if (options.interactive === false) {
+          console.error(chalk.red('Error: --selector or --lines required when --no-interactive is set'));
+          console.log(chalk.gray('\nExamples:'));
+          console.log(chalk.gray('  codescoop page.html -s ".navbar"'));
+          console.log(chalk.gray('  codescoop https://site.com --selector "header" --dir ./theme'));
+          console.log(chalk.gray('  codescoop template.php -s ".content"'));
+          process.exit(1);
+        }
+
+        // Interactive mode only works for local files
+        if (sourceType === 'url') {
+          console.error(chalk.red('Error: Interactive mode not available for URLs. Use --selector.'));
+          process.exit(1);
+        }
+
+        console.log(chalk.yellow('No selector specified. Launching interactive mode...\n'));
+        const selection = await runInteractiveMode(htmlPath);
+
+        if (!selection) {
+          console.log(chalk.gray('No selection made. Exiting.'));
+          process.exit(0);
+        }
+
+        options.selector = selection.selector;
+      }
+
+      // ============================================
+      // STEP 4: Run analysis
+      // ============================================
+
+      const result = await runAnalysis({
+        htmlPath,
+        htmlContent, // Pass pre-fetched content for URL/template modes
+        projectDir,
+        selector: options.selector,
+        lineRange: options.lines,
+        matchIndex: matchIndex,
+        outputPath: options.output,
+        includeInline: options.includeInline,
+        verbose: options.verbose,
+        sourceType,
+        // Compact mode options
+        compact: options.compact,
+        forConversion: options.forConversion,
+        maxRulesPerFile: parseInt(options.maxRules, 10) || 20,
+        maxJsPerFile: parseInt(options.maxJs, 10) || 10,
+        summaryOnly: options.summaryOnly,
+        skipMinified: options.skipMinified
+      });
+
+      // ============================================
+      // STEP 5: Output results
+      // ============================================
+
+      console.log(chalk.green(`\n✓ Analysis complete!`));
+      console.log(chalk.white(`  Output: ${result.outputPath}`));
+      console.log(chalk.gray(`  Found ${result.cssMatches} CSS rules, ${result.jsMatches} JS references`));
+
+      if (result.libraryCount > 0) {
+        console.log(chalk.cyan(`  Libraries detected: ${result.libraryCount}`));
+      }
+
+      if (result.missingImports && result.missingImports.length > 0) {
+        console.log(chalk.yellow(`\n⚠ ${result.missingImports.length} files contain relevant code but are NOT imported:`));
+        result.missingImports.slice(0, 5).forEach(file => {
+          console.log(chalk.yellow(`  - ${file}`));
+        });
+        if (result.missingImports.length > 5) {
+          console.log(chalk.yellow(`  ... and ${result.missingImports.length - 5} more`));
+        }
+      }
+
+    } catch (error) {
+      console.error(chalk.red(`\n✖  ${formatError(error, options.verbose)}`));
+      if (options.verbose) {
+        console.error(chalk.gray(error.stack));
+      }
+      process.exit(1);
+    }
+  });
+
+// Handle uncaught errors gracefully
+process.on('uncaughtException', (error) => {
+  console.error(chalk.red(`\n✖  Unexpected error: ${error.message}`));
+  console.error(chalk.gray('This is likely a bug. Please report it.'));
+  process.exit(1);
+});
+
+process.on('unhandledRejection', (reason) => {
+  console.error(chalk.red(`\n✖  Unhandled promise rejection: ${reason}`));
+  process.exit(1);
+});
+
+program.parse();

package/package.json

@@ -0,0 +1,75 @@
+{
+  "name": "codescoop",
+  "version": "1.0.0",
+  "description": "Extract HTML component dependencies for AI-powered React/Next.js conversion",
+  "main": "src/index.js",
+  "bin": {
+    "codescoop": "./bin/codescoop.js"
+  },
+  "scripts": {
+    "start": "node bin/codescoop.js"
+  },
+  "keywords": [
+    "html",
+    "css",
+    "javascript",
+    "react",
+    "nextjs",
+    "conversion",
+    "migration",
+    "jquery",
+    "wordpress",
+    "magento",
+    "component",
+    "extractor",
+    "analyzer",
+    "cli",
+    "llm",
+    "ai",
+    "gpt",
+    "claude",
+    "chatgpt",
+    "openai",
+    "anthropic",
+    "debugging",
+    "scss",
+    "dependencies",
+    "php",
+    "blade",
+    "twig"
+  ],
+  "author": "",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/yourusername/codescoop.git"
+  },
+  "bugs": {
+    "url": "https://github.com/yourusername/codescoop/issues"
+  },
+  "homepage": "https://github.com/yourusername/codescoop#readme",
+  "engines": {
+    "node": ">=16.0.0"
+  },
+  "files": [
+    "bin/",
+    "src/",
+    "README.md",
+    "LICENSE"
+  ],
+  "dependencies": {
+    "acorn": "^8.11.3",
+    "acorn-loose": "^8.4.0",
+    "acorn-walk": "^8.3.2",
+    "chalk": "^4.1.2",
+    "cheerio": "^1.0.0-rc.12",
+    "commander": "^12.0.0",
+    "glob": "^10.3.10",
+    "inquirer": "^8.2.6",
+    "js-beautify": "^1.14.11",
+    "postcss": "^8.4.35",
+    "postcss-scss": "^4.0.9",
+    "sass": "^1.71.1",
+    "specificity": "^1.0.0"
+  }
+}