@crownpeak/dqm-react-component-dev-mcp 1.2.0

package/data/probe/site/search-reference.md

@@ -0,0 +1,291 @@
+# Search Functionality Reference
+
+Complete reference documentation for Probe's search capabilities, including query syntax, ranking algorithms, and advanced search techniques.
+
+## SEARCH COMMAND
+
+```bash
+probe search <QUERY> [PATH] [OPTIONS]
+```
+
+### CORE PARAMETERS
+
+| Parameter | Description |
+|-----------|-------------|
+| `<QUERY>` | **Required**: Search terms or expression |
+| `[PATH]` | Directory to search (defaults to current directory) |
+
+### KEY OPTIONS
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `--files-only` | List matching files without code blocks | Off |
+| `--ignore <PATTERN>` | Additional patterns to ignore | None |
+| `--exclude-filenames, -n` | Exclude filenames from matching | Off |
+| `--reranker, -r <TYPE>` | Ranking algorithm: `hybrid`, `hybrid2`, `bm25`, `tfidf` | `hybrid` |
+| `--frequency, -s` | Enable smart token matching | On |
+| `--max-results <N>` | Limit number of results | No limit |
+| `--max-bytes <N>` | Limit total bytes of code returned | No limit |
+| `--max-tokens <N>` | Limit total tokens | No limit |
+| `--allow-tests` | Include test files and code | Off |
+| `--any-term` | Match any search term (OR logic) | Off |
+| `--no-merge` | Keep code blocks separate | Off |
+| `--merge-threshold <N>` | Max lines between blocks to merge | 5 |
+| `--session <ID>` | Session ID for caching results | None |
+| `--format <TYPE>` | Output format: `color`, `plain`, `markdown`, `json` | `color` |
+
+For complete option details, see `probe search --help`.
+
+## QUERY PROCESSING
+
+Probe enhances search queries through several techniques:
+
+### TOKENIZATION
+
+Breaks down terms into tokens:
+```
+findUserByEmail → [find, user, by, email]
+```
+
+### STEMMING
+
+Reduces words to their root form:
+```
+implementing, implementation → implement
+```
+
+### SMART PATTERN GENERATION
+
+- **Term Boundaries**: Understands where code tokens start/end
+- **Case Handling**: Works with camelCase, snake_case, etc.
+- **Compound Handling**: Breaks down compound terms
+
+## QUERY SYNTAX
+
+Probe supports an Elasticsearch-like query syntax:
+
+### BASIC TERMS
+
+```bash
+probe search "authentication"  # Single term
+probe search "user authentication"  # Multiple terms (AND logic)
+```
+
+### BOOLEAN OPERATORS
+
+```bash
+probe search "error AND handling"  # Require both terms
+probe search "login OR authentication"  # Match either term
+probe search "database NOT sqlite"  # Exclude term
+```
+
+### GROUPING
+
+```bash
+probe search "(error OR exception) AND (handle OR process)"
+```
+
+### TERM MODIFIERS
+
+```bash
+probe search "+authentication login"  # Required term
+probe search "database -sqlite"  # Excluded term
+probe search "\"handle error\""  # Exact phrase
+```
+
+### FIELD SPECIFIERS
+
+```bash
+probe search "function:authenticate"  # Search in function names
+```
+
+### SEARCH HINTS
+
+Search hints allow you to filter results by file properties. These filters are applied at the file discovery stage and removed from the query before content searching:
+
+```bash
+probe search "error AND ext:rs"              # Only search in .rs files
+probe search "class AND file:src/**/*.py"    # Only search in Python files under src/
+probe search "function AND dir:tests"        # Only search in files under tests/ directory
+probe search "struct AND type:rust"          # Only search in Rust files
+probe search "component AND lang:javascript" # Only search in JavaScript files
+```
+
+**Available Search Hints:**
+
+| Hint | Description | Examples |
+|------|-------------|----------|
+| `ext:<extension>` | Filter by file extension | `ext:rs`, `ext:py,js,ts` |
+| `file:<pattern>` | Filter by file path pattern (supports globs) | `file:src/**/*.rs`, `file:*test*` |
+| `path:<pattern>` | Alias for `file:` | `path:src/main.rs` |
+| `dir:<pattern>` | Filter by directory pattern | `dir:src`, `dir:tests` |
+| `type:<filetype>` | Filter by ripgrep file type | `type:rust`, `type:javascript` |
+| `lang:<language>` | Filter by programming language | `lang:rust`, `lang:python` |
+
+**Combining Search Hints:**
+
+```bash
+# Multiple filters (all must match)
+probe search "config AND ext:rs AND dir:src"
+
+# Multiple extensions (any can match)
+probe search "import AND ext:js,ts,jsx,tsx"
+
+# Complex combinations
+probe search "(error OR exception) AND ext:rs AND file:src/**/*"
+```
+
+### WILDCARDS
+
+```bash
+probe search "auth*"  # Matches "auth", "authentication", "authorize", etc.
+```
+
+## RANKING ALGORITHMS
+
+Probe uses sophisticated algorithms to rank search results:
+
+### TF-IDF RANKING
+
+Term Frequency-Inverse Document Frequency balances how often terms appear in a specific code block against how common they are across the codebase.
+
+#### HOW IT WORKS
+
+1. **Term Frequency (TF)**: How often a term appears in a code block
+   ```
+   TF(term, block) = (Number of times term appears in block) / (Total number of terms in block)
+   ```
+
+2. **Inverse Document Frequency (IDF)**: Measures how unique or rare a term is
+   ```
+   IDF(term) = ln(Total number of blocks / Number of blocks containing term)
+   ```
+
+3. **TF-IDF Score**: Combines these factors
+   ```
+   TF-IDF(term, block) = TF(term, block) * IDF(term)^2
+   ```
+
+Key benefits:
+- Rewards matches on rare, important terms
+- Penalizes common terms that appear everywhere
+- Considers term frequency within each code block
+
+### BM25 RANKING
+
+BM25 (Best Matching 25) is an improved version of TF-IDF that addresses some of its limitations.
+
+#### HOW IT WORKS
+
+```
+BM25(block, query) = ∑ IDF(term) * (TF(term, block) * (k1 + 1)) / (TF(term, block) + k1 * (1 - b + b * (block_length / average_block_length)))
+```
+
+Where:
+- `k1` (1.2): Controls term frequency saturation
+- `b` (0.75): Controls length normalization
+
+Key benefits:
+- Better handling of document length
+- Diminishing returns for repeated terms
+- More accurate for longer code blocks
+- Improved handling of edge cases
+
+### HYBRID RANKING
+
+Probe's default ranking algorithm combines multiple signals for superior results.
+
+#### HOW IT WORKS
+
+The hybrid algorithm considers:
+
+1. **Combined score**: Weighted combination of TF-IDF and BM25
+   ```
+   Combined = α * TF-IDF + (1-α) * BM25
+   ```
+
+2. **Position weights**: Terms in function names, class names, and identifiers receive higher scores
+
+3. **Block metrics**:
+   - Number of unique terms matched
+   - Total matches in the block
+   - Block type (methods score higher than comments)
+
+4. **File metrics**:
+   - File match rank
+   - Number of unique terms in the file
+   - Total matches in the file
+
+Key benefits:
+- More balanced scoring across different code structures
+- Better handling of both short and long code blocks
+- Prioritizes meaningful code over comments or boilerplate
+
+### HYBRID2 RANKING
+
+An enhanced version of the hybrid algorithm with improved relevance:
+- Better normalization of scores across different metrics
+- Enhanced weighting for structural elements
+- Improved handling of term proximity
+- More sophisticated position weighting
+
+## PRACTICAL EXAMPLES
+
+### FINDING ERROR HANDLING CODE
+
+```bash
+probe search "error handling try catch"
+```
+
+This search:
+1. Tokenizes to: ["error", "handl", "try", "catch"]
+2. Matches files containing these terms
+3. Ranks results based on term frequency and importance
+4. Returns complete code blocks with error handling logic
+
+### SEARCHING FOR AUTHENTICATION FLOWS
+
+```bash
+probe search "(login OR authenticate) AND (user OR account) NOT test"
+```
+
+This complex query:
+1. Finds code with either "login" or "authenticate"
+2. Requires either "user" or "account" to be present
+3. Excludes results containing "test"
+4. Returns ranked, complete code blocks
+
+### FINDING SPECIFIC API ENDPOINTS
+
+```bash
+probe search "function:create* api endpoint"
+```
+
+This search:
+1. Targets functions starting with "create"
+2. Requires "api" and "endpoint" terms
+3. Returns complete function definitions
+4. Ranks results with the most relevant endpoints first
+
+### LIMITING RESULTS FOR AI INTEGRATION
+
+```bash
+probe search "database connection pool" --max-tokens 4000 --format json
+```
+
+This search:
+1. Finds code related to database connection pools
+2. Limits results to fit within 4000 tokens
+3. Returns JSON-formatted output suitable for AI processing
+
+## PERFORMANCE TIPS
+
+- **Be specific**: More specific queries yield more relevant results
+- **Use field specifiers**: Target specific code elements with `function:`, `class:`, etc.
+- **Leverage boolean operators**: Combine terms with AND, OR, NOT for precision
+- **Control result size**: Use `--max-results`, `--max-bytes`, or `--max-tokens` for large codebases
+- **Session caching**: Use `--session` to avoid seeing the same code blocks repeatedly
+- **Experiment with rankers**: Try different ranking algorithms for different types of searches
+
+For more information on how Probe works internally, see [How Probe Works](how-it-works.md).
+For details on code extraction, see [Code Extraction Reference](code-extraction.md).

package/data/probe/site/supported-languages.md

@@ -0,0 +1,215 @@
+# Supported Languages
+
+Probe provides language-aware code search and extraction for a wide range of programming languages. This page details the supported languages and their specific features.
+
+> For a comprehensive overview of how Probe's language support system works, see the [Language Support Overview](/language-support-overview) page.
+
+## Language Support Table
+
+| Language | File Extensions | AST Parsing | Block Extraction |
+|----------|----------------|-------------|-----------------|
+| Rust | `.rs` | ✅ | ✅ |
+| JavaScript / JSX | `.js`, `.jsx` | ✅ | ✅ |
+| TypeScript / TSX | `.ts`, `.tsx` | ✅ | ✅ |
+| Python | `.py` | ✅ | ✅ |
+| Go | `.go` | ✅ | ✅ |
+| C / C++ | `.c`, `.h`, `.cpp`, `.cc`, `.cxx`, `.hpp`, `.hxx` | ✅ | ✅ |
+| Java | `.java` | ✅ | ✅ |
+| Ruby | `.rb` | ✅ | ✅ |
+| PHP | `.php` | ✅ | ✅ |
+| Swift | `.swift` | ✅ | ✅ |
+| C# | `.cs` | ✅ | ✅ |
+| Markdown | `.md`, `.markdown` | ✅ | ✅ |
+| YAML | `.yaml`, `.yml` | ✅ | ✅ |
+
+## Language Detection
+
+Probe automatically detects the language of a file based on its extension. This detection is used to:
+
+1. **Select the appropriate parser**: Each language has a specialized parser
+2. **Apply language-specific extraction rules**: Different languages have different code structures
+3. **Handle language-specific features**: Such as Python's significant whitespace or Rust's macros
+
+## Language-Specific Features
+
+Each language has specific features and capabilities in Probe:
+
+### Rust
+
+- **Function Extraction**: Extracts complete function definitions including attributes and documentation
+- **Struct/Enum Extraction**: Extracts complete struct and enum definitions
+- **Impl Block Extraction**: Extracts implementation blocks for types
+- **Macro Handling**: Properly handles macro definitions and invocations
+- **Module Awareness**: Understands Rust's module system
+- **Test Detection**: Identifies test functions marked with `#[test]` attribute or functions with names starting with `test_`
+
+### JavaScript / TypeScript
+
+- **Function Extraction**: Extracts function and arrow function definitions
+- **Class Extraction**: Extracts complete class definitions with methods
+- **JSX/TSX Support**: Properly handles JSX and TSX syntax
+- **Module Awareness**: Understands ES modules and CommonJS
+- **Type Definitions**: Extracts TypeScript interfaces and type definitions
+- **Test Detection**: Identifies test functions using Jest, Mocha, or other frameworks
+
+TypeScript adds additional support for interfaces, type aliases, enums, and namespaces.
+
+### Python
+
+- **Function Extraction**: Extracts function definitions with docstrings
+- **Class Extraction**: Extracts complete class definitions with methods
+- **Decorator Handling**: Properly handles decorated functions and classes
+- **Indentation Awareness**: Understands Python's significant whitespace
+- **Module Docstrings**: Includes module-level docstrings in relevant extractions
+- **Test Detection**: Identifies test functions using unittest, pytest, or other frameworks
+
+### Go
+
+- **Function Extraction**: Extracts function definitions with documentation
+- **Struct Extraction**: Extracts complete struct definitions
+- **Interface Extraction**: Extracts interface definitions
+- **Method Extraction**: Extracts methods associated with types
+- **Comment Handling**: Properly associates comments with code blocks
+- **Test Detection**: Identifies test functions with the `Test` prefix
+
+Go also implements special handling for nested struct types.
+
+### C / C++
+
+- **Function Extraction**: Extracts function definitions and declarations
+- **Class/Struct Extraction**: Extracts complete class and struct definitions
+- **Template Handling**: Properly handles template definitions
+- **Namespace Awareness**: Understands C++ namespaces
+- **Preprocessor Handling**: Includes relevant preprocessor directives
+- **Test Detection**: Identifies test functions based on naming conventions
+
+### Java
+
+- **Method Extraction**: Extracts method definitions with annotations
+- **Class Extraction**: Extracts complete class definitions
+- **Interface Extraction**: Extracts interface definitions
+- **Annotation Handling**: Properly handles annotated elements
+- **Package Awareness**: Understands Java's package system
+- **Test Detection**: Identifies test classes and methods using JUnit annotations
+
+### Ruby
+
+- **Method Extraction**: Extracts method definitions
+- **Class/Module Extraction**: Extracts complete class and module definitions
+- **Block Handling**: Properly handles Ruby blocks
+- **Mixin Awareness**: Understands Ruby's include and extend
+- **Documentation**: Includes RDoc comments in extractions
+- **Test Detection**: Identifies test methods in Test::Unit, RSpec, or Minitest
+
+### PHP
+
+- **Function Extraction**: Extracts function definitions
+- **Class Extraction**: Extracts complete class definitions with methods
+- **Namespace Awareness**: Understands PHP namespaces
+- **Attribute Handling**: Properly handles PHP 8 attributes
+- **Documentation**: Includes PHPDoc comments in extractions
+- **Test Detection**: Identifies test classes and methods using PHPUnit conventions
+
+### Swift
+
+- **Function Extraction**: Extracts function and method definitions
+- **Class/Struct Extraction**: Extracts complete class and struct definitions
+- **Protocol Extraction**: Extracts protocol definitions
+- **Extension Handling**: Properly handles Swift extensions
+- **Attribute Handling**: Includes relevant attributes in extractions
+- **Test Detection**: Identifies test classes and methods using XCTest conventions
+
+### C#
+
+- **Method Extraction**: Extracts method definitions with attributes
+- **Class Extraction**: Extracts complete class definitions
+- **Interface Extraction**: Extracts interface definitions
+- **Namespace Awareness**: Understands C# namespaces
+- **Attribute Handling**: Properly handles C# attributes
+- **Test Detection**: Identifies test classes and methods using NUnit, MSTest, or xUnit conventions
+
+### Markdown
+
+- **Section Extraction**: Extracts complete sections based on headings
+- **Code Block Extraction**: Extracts fenced code blocks
+- **List Extraction**: Extracts complete lists
+- **Table Extraction**: Extracts complete tables
+- **Frontmatter Handling**: Properly handles YAML frontmatter
+
+### YAML
+
+- **Key-Value Pair Extraction**: Extracts mapping pairs with keys and values
+- **Sequence Extraction**: Extracts arrays and lists
+- **Scalar Extraction**: Extracts literal and folded block scalars
+- **Comment Extraction**: Extracts comments and preserves them in context
+- **Document Structure**: Handles multi-document YAML streams
+- **Anchor and Alias Support**: Properly handles YAML anchors and aliases
+- **Test Detection**: Identifies test-related configuration keys and values
+
+## Test Detection
+
+Probe can detect test code at two levels:
+
+1. **File-level detection**: Identifies test files based on naming conventions and directory patterns
+2. **Node-level detection**: Identifies test functions and classes within files
+
+This allows users to include or exclude test code from search results using the `--allow-tests` flag.
+
+### File-level Test Detection
+
+Probe identifies test files based on common naming patterns:
+
+- **Rust**: `*_test.rs`, `*_tests.rs`, `test_*.rs`, `tests.rs`
+- **JavaScript/TypeScript**: `*.test.js`, `*.spec.js`, `*.test.ts`, `*.spec.ts`
+- **Python**: `test_*.py`, `*_test.py`
+- **Go**: `*_test.go`
+- **Java**: `*Test.java`, `*Tests.java`
+
+Probe also identifies test files based on directory patterns:
+- `/test/`, `/tests/`, `/spec/`, `/specs/`, `/__tests__/`
+
+### Node-level Test Detection
+
+Each language implementation has specific logic to identify test functions and classes:
+
+- **Rust**: Functions with `#[test]` attribute or names starting with `test_`
+- **JavaScript/TypeScript**: Functions using Jest, Mocha, or other test frameworks
+- **Python**: Functions using unittest, pytest, or other test frameworks
+- **Go**: Functions with the `Test` prefix
+- **Java**: Classes and methods with JUnit annotations
+
+## Pattern Matching Examples
+
+Probe supports language-specific pattern matching. Here are some examples:
+
+### Rust Patterns
+```
+fn $NAME($$$PARAMS) $$$BODY
+struct $NAME { $$$FIELDS }
+impl $TYPE { $$$METHODS }
+```
+
+### JavaScript/TypeScript Patterns
+```
+function $NAME($$$PARAMS) { $$$BODY }
+const $NAME = ($$$PARAMS) => $$$BODY
+class $NAME { $$$METHODS }
+```
+
+### Python Patterns
+```
+def $NAME($$$PARAMS): $$$BODY
+class $NAME: $$$METHODS
+@$DECORATOR\ndef $NAME($$$PARAMS): $$$BODY
+```
+
+### Go Patterns
+```
+func $NAME($$$PARAMS) $$$BODY
+type $NAME struct { $$$FIELDS }
+func ($RECEIVER $TYPE) $NAME($$$PARAMS) $$$BODY
+```
+
+## Adding Support for New Languages
+
+If you're interested in adding support for a language that's not currently supported, see the [Adding New Languages](/adding-languages) page for a detailed guide.

package/data/probe/site/use-cases/README.md

@@ -0,0 +1,8 @@
+# Use Cases & Workflows
+
+This directory contains scenario-based documentation for different Probe usage patterns.
+
+- **Web Interface**: Code as a source of truth for product functionality
+- **AI Code Editors & MCP**: Integration with AI-powered code editors
+- **CLI for AI Workflows**: Using Probe in the command line for AI-assisted workflows
+- **Developers & SDK**: Building custom tools with the Node.js SDK