@akotliar/sitemap-qa 1.0.0-alpha.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Alex Kotliar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,235 @@
+# Sitemap-QA
+
+> **Automated sitemap analysis for QA teams** — Detect test/qa/dev/staging URLs, admin paths, sensitive parameters, and URLs that shouldn't be publicly indexed.
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Node Version](https://img.shields.io/badge/node-%3E%3D20.0.0-brightgreen)](package.json)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.0-blue)](https://www.typescriptlang.org/)
+
+Sitemap-QA is a command-line tool that automatically discovers, parses, and analyzes website sitemaps to identify potential quality issues, security risks, and configuration problems. Built for QA teams to validate deployments, catch environment leakage, and identify URLs that shouldn't be publicly indexed.
+
+---
+
+## 🎯 Why Sitemap-QA?
+
+Unlike SEO-focused sitemap validators, Sitemap-QA is designed specifically for **QA validation and risk detection**:
+
+- ✅ **Detect environment leakage** — Find staging, dev, or test URLs that shouldn't be in production sitemaps
+- ✅ **Identify exposed admin paths** — Catch `/admin`, `/dashboard`, and internal routes in public indexes
+- ✅ **Flag sensitive parameters** — Detect API keys, tokens, or passwords in sitemap URLs
+- ✅ **Validate domain consistency** — Find protocol mismatches and subdomain issues
+- ✅ **Fast and automated** — Analyze thousands of URLs in seconds with detailed reports
+
+Perfect for CI/CD pipelines, pre-release validation, and security audits.
+
+---
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+npm install -g sitemap-qa
+```
+
+### Basic Usage
+
+```bash
+# Analyze a website's sitemap
+sitemap-qa analyze https://example.com
+
+# Generate JSON output for CI/CD
+sitemap-qa analyze https://example.com --output json > report.json
+
+# Increase verbosity for debugging
+sitemap-qa analyze https://example.com --verbose
+```
+
+---
+
+## 📋 Features
+
+### Automatic Sitemap Discovery
+- Checks `robots.txt` for sitemap declarations
+- Tests standard paths (`/sitemap.xml`, `/sitemap_index.xml`, etc.)
+- Recursively follows sitemap indexes
+- Handles multiple sitemaps and formats
+- Detects and processes malformed sitemap indexes (sitemaps listed in `<url>` blocks instead of `<sitemap>` blocks)
+
+### Risk Detection Patterns
+
+| Risk Category | Severity | Examples | Can Be Excluded |
+|--------------|----------|----------|-----------------|
+| **Environment Leakage** | High | `staging.example.com`, `/dev/`, `/test/` | ✅ Via patterns |
+| **Admin Paths** | High | `/admin`, `/dashboard`, `/config`, `/console` | ✅ Via patterns |
+| **Internal Content** | Medium | `/internal` paths | ✅ Via patterns |
+| **Sensitive Parameters** | High | `?token=`, `?apikey=`, `?password=` | ✅ Via patterns |
+| **Test Content** | Medium | `/test-`, `sample-`, `demo-` | ✅ Via patterns |
+| **Protocol Inconsistency** | Medium | HTTP URLs in HTTPS sitemaps | ❌ Always detected |
+| **Domain Mismatch** | Medium | Different domains in sitemap | ❌ Always detected |
+
+**Note:** Admin path patterns now properly match URLs with query parameters (e.g., `/admin?id=123`).
+
+### Output Formats
+
+#### HTML Report (Interactive)
+The HTML report provides an interactive, visually appealing view with:
+- Expandable/collapsible sections by severity
+- Download buttons to export all URLs per category
+- Clean, modern design with hover effects
+- Portable single-file format
+
+#### JSON Report (Machine-Readable)
+```json
+{
+  "analysis_metadata": {
+    "base_url": "https://example.com",
+    "tool_version": "1.0.0",
+    "analysis_type": "rule-based analysis",
+    "analysis_timestamp": "2025-12-11T00:00:00.000Z",
+    "execution_time_ms": 4523
+  },
+  "sitemaps_discovered": [
+    "https://example.com/sitemap.xml"
+  ],
+  "suspicious_groups": [
+    {
+      "category": "environment_leakage",
+      "severity": "high",
+      "count": 3,
+      "rationale": "Production sitemap contains staging URLs",
+      "sample_urls": ["..."],
+      "recommended_action": "Verify sitemap generation excludes non-production environments"
+    }
+  ],
+  "summary": {
+    "high_severity_count": 2,
+    "medium_severity_count": 1,
+    "low_severity_count": 0,
+    "total_risky_urls": 8,
+    "overall_status": "issues_found"
+  }
+}
+```
+
+---
+
+## 🛠️ CLI Options
+
+```
+Usage: sitemap-qa analyze [options] <url>
+
+Analyze a website's sitemap for quality issues
+
+Arguments:
+  url                          Base URL of the website to analyze
+
+Options:
+  --timeout <seconds>          HTTP request timeout in seconds (default: 30)
+  --output <format>            Output format: html or json (default: "html")
+  --output-dir <path>          Output directory for reports (default: "./sitemap-qa/report")
+  --output-file <path>         Custom output filename
+  --accepted-patterns <list>   Comma-separated patterns to exclude from risk detection
+  --verbose                    Enable verbose logging
+  -h, --help                   Display help for command
+```
+
+### Examples
+
+```bash
+# Basic analysis with HTML report (default)
+sitemap-qa analyze https://example.com
+
+# JSON output for CI/CD integration
+sitemap-qa analyze https://example.com --output json
+
+# Custom output directory
+sitemap-qa analyze https://example.com --output-dir ./reports
+
+# Exclude specific URL patterns from detection
+sitemap-qa analyze https://example.com --accepted-patterns "internal-*,test-*"
+
+# Increase timeout for slow servers
+sitemap-qa analyze https://example.com --timeout 60
+
+# Verbose mode for debugging
+sitemap-qa analyze https://example.com --verbose
+```
+
+---
+
+## 🔧 Configuration
+
+Create a `.sitemap-qa.config.json` file in your project root or `~/.sitemap-qa/config.json` for global settings:
+
+```json
+{
+  "timeout": 30,
+  "concurrency": 10,
+  "outputFormat": "html",
+  "outputDir": "./sitemap-qa/report",
+  "verbose": false,
+  "acceptedPatterns": [
+    "test-*",
+    "staging-*"
+  ]
+}
+```
+
+### Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `timeout` | number | `30` | HTTP request timeout in seconds (1-300) |
+| `concurrency` | number | `10` | Number of concurrent HTTP requests |
+| `outputFormat` | string | `"html"` | Output format: `"html"` or `"json"` |
+| `outputDir` | string | `"./sitemap-qa/report"` | Directory for generated reports |
+| `verbose` | boolean | `false` | Enable detailed logging |
+| `acceptedPatterns` | string[] | `[]` | URL patterns to exclude from risk detection |
+
+### Accepted Patterns
+
+Exclude specific URLs from risk detection using wildcard patterns:
+
+```json
+{
+  "acceptedPatterns": [
+    "testing-*",                 // Matches: test-player, test-player-stats
+    "https://example.com/admin/*",  // Matches: any URL under /admin/
+    "/special-case"                 // Matches: exact path segment
+  ]
+}
+```
+
+**Pattern Syntax:**
+- Use `*` as a wildcard (matches any characters within a path segment)
+- Patterns are case-insensitive
+- Special characters are automatically escaped
+- Patterns match against the full URL
+- Full URLs or path fragments both work
+
+**Priority:** CLI options > Project config (`.sitemap-qa.config.json`) > Global config (`~/.sitemap-qa/config.json`) > Defaults
+
+## 📝 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+---
+
+## 🙏 Acknowledgments
+
+Built with:
+- [Commander.js](https://github.com/tj/commander.js) - CLI framework
+- [Chalk](https://github.com/chalk/chalk) - Terminal styling
+- [Vitest](https://vitest.dev/) - Testing framework
+- [TypeScript](https://www.typescriptlang.org/) - Type safety
+
+---
+
+## 📧 Support
+
+- **Issues**: [GitHub Issues](https://github.com/akotliar/sitemap-qa/issues)-
+
+---
+
+**Made with ❤️ for QA teams everywhere**