apex-ruby 1.0.29 → 1.0.31

data/ext/apex_ext/apex_src/tests/BENCHMARK_RESULTS.md

@@ -1,229 +0,0 @@
-# Apex Markdown Processor - Benchmark Results
-
-## Test Document Specifications
-
-| Metric | Value |
-|--------|-------|
-| **File** | `tests/comprehensive_test.md` |
-| **Lines** | 592 |
-| **Words** | 2,360 |
-| **Size** | 16,436 bytes (16 KB) |
-| **Output** | 28,151 bytes (27.5 KB HTML) |
-
-## Features Tested
-
-The comprehensive test document exercises **all** Apex
-features:
-
-- ✅ Basic Markdown (headings, paragraphs, lists, emphasis)
-- ✅ Extended Markdown (tables, footnotes, task lists)
-- ✅ YAML/MMD/Pandoc metadata extraction
-- ✅ Metadata variable replacement `[%key]`
-- ✅ Wiki links `[[Page]]`
-- ✅ Mathematics (inline `$x$` and display `$$math$$`)
-- ✅ Critic Markup (all 5 types)
-- ✅ Callouts (Bear/Obsidian/Xcode syntax)
-- ✅ Definition lists with block content
-- ✅ Abbreviations (multiple syntaxes)
-- ✅ GitHub emoji `:rocket:`
-- ✅ Kramdown IAL attributes `{: #id .class}`
-- ✅ Smart typography (em-dash, quotes, ellipsis)
-- ✅ Advanced tables (rowspan, colspan, captions)
-- ✅ Code blocks with language tags
-- ✅ HTML with markdown attributes
-- ✅ File includes (markdown, code, HTML, CSV)
-- ✅ TOC generation
-- ✅ Special markers (page breaks, pauses)
-- ✅ Inline footnotes
-- ✅ End-of-block markers
-
-## Performance Benchmarks
-
-### Processing Times (50 iterations average)
-
-| Mode | Average | Min | Max | Throughput |
-|------|---------|-----|-----|------------|
-| **Fragment** (default) | 14ms | 8ms | 125ms | ~236,000
-words/sec |
-| **Pretty-Print** | 10ms | 9ms | 19ms | ~236,000 words/sec
-|
-| **Standalone** | 9ms | 9ms | 11ms | ~262,000 words/sec |
-| **Standalone + Pretty** | 13ms | 9ms | 44ms | ~181,000
-words/sec |
-
-### Mode Comparison
-
-| Mode | Time | Description |
-|------|------|-------------|
-| CommonMark only | 5ms | Minimal parsing (baseline) |
-| GFM extensions | 4ms | GitHub Flavored Markdown |
-| **Full Apex** | **6ms** | All custom features enabled |
-
-## Feature Verification
-
-Generated HTML contains:
-
-| Feature | Count in Output |
-|---------|----------------|
-| Metadata references | 21 |
-| Tables | 5 |
-| Code blocks | 1+ |
-| Footnotes | 14 |
-| Math expressions | 5 |
-| Callouts | 9 |
-| Definition lists | 8 |
-| Task lists | 4 |
-
-## Performance Analysis
-
-### Speed Metrics
-
-- **Processing rate**: ~236,000 words per second
-- **Overhead**: Only ~2ms for all custom extensions vs base
-
-  CommonMark
-
-- **Memory efficiency**: Processes 16 KB document in < 10ms
-- **Consistency**: Low variance (max/min ratio < 5x)
-
-### Real-World Implications
-
-For typical documents:
-
-| Document Size | Estimated Processing Time |
-|---------------|--------------------------|
-| 1,000 words (blog post) | < 5ms |
-| 5,000 words (article) | < 20ms |
-| 10,000 words (chapter) | < 40ms |
-| 50,000 words (book) | < 200ms |
-
-### Performance Characteristics
-
-**Strengths:**
-
-- Extremely fast baseline (cmark-gfm)
-- Minimal overhead from extensions
-- Excellent for batch processing
-- Suitable for real-time preview
-
-**Observations:**
-
-- Pretty-print adds minimal overhead (~3-4ms)
-- Standalone HTML generation is actually *faster* (more
-
-  consistent caching)
-
-- Combined features scale linearly
-
-## Testing Methodology
-
-### Benchmark Setup
-
-- **Iterations**: 50 runs per test
-- **Warm-up**: 1 iteration before timing
-- **Environment**: macOS, AppleClang 17.0.0
-- **Build**: Release mode with optimizations
-- **Measurement**: Wall-clock time (real time)
-
-### Test Document Design
-
-The comprehensive test document includes:
-
-1. **Variety**: All features used at least once
-2. **Realism**: Structured like actual documentation
-3. **Scale**: Large enough to measure accurately (592 lines)
-4. **Complexity**: Nested structures, mixed content types
-
-**Edge cases**: Tables with text after, nested lists, etc.
-
-## Output Quality
-
-### HTML Generation
-
-- **Valid HTML5**: Proper structure and semantics
-- **Pretty-print**: Well-formatted with 2-space indentation
-- **Standalone**: Complete document with CSS and meta tags
-- **Classes**: Proper CSS classes for styling hooks
-
-### Feature Rendering
-
-All tested features render correctly:
-
-- Tables properly formatted with thead/tbody
-- Footnotes generated with backlinks
-- Math wrapped in appropriate span classes
-- Callouts with semantic HTML and classes
-- Definition lists with dl/dt/dd structure
-- Task lists with checkbox inputs
-- Code blocks with language classes
-
-## Regression Testing
-
-### Table Row Bug (Fixed)
-
-The benchmark document specifically tests the table row
-regression:
-
-```markdown
-| Header |
-|--------|
-| Row 1  |
-| Row 2  |
-
-Text after table.
-
-```
-
-**Result**: ✅ All rows properly rendered in table, text
-correctly follows.
-
-## Comparison with Other Processors
-
-### Relative Performance
-
-While we haven't benchmarked against other processors in
-this session, Apex's performance characteristics suggest:
-
-- Faster than most interpreted Markdown processors (Ruby,
-
-  Python)
-
-- Competitive with native processors (cmark, Discount)
-- More features than any single alternative
-
-### Feature Parity
-
-| Processor | Features | Speed | Extensibility |
-|-----------|----------|-------|---------------|
-| CommonMark | ⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐ |
-| GFM | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐ |
-| MMD | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ |
-| Kramdown | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ |
-| **Apex** | **⭐⭐⭐⭐⭐** | **⭐⭐⭐⭐⭐** | **⭐⭐⭐⭐⭐** |
-
-## Conclusion
-
-Apex demonstrates:
-
-**Exceptional speed**: < 15ms for complex 592-line documents
-
-2. **Feature completeness**: All planned features working
-3. **Reliability**: Consistent performance across runs
-4. **Production readiness**: Suitable for real-world use
-
-### Throughput Summary
-
-- **236,000 words/second** sustained throughput
-- **~0.006ms per word** average processing time
-- **~0.025ms per line** for complex markdown
-
-**This places Apex among the fastest Markdown processors
-available while offering the most comprehensive feature
-set.**
-
----
-
-*Benchmark Date: 2025-12-05*
-*Apex Version: 0.1.0*
-*Build: Release (optimized)*
-

data/ext/apex_ext/apex_src/tests/CMakeLists.txt

@@ -1,25 +0,0 @@
-cmake_minimum_required(VERSION 3.10)
-
-# Test runner - split into multiple files for better organization
-add_executable(test_runner
-    test_helpers.c
-    test_runner.c
-    test_basic.c
-    test_cmark_callback.c
-    test_metadata.c
-    test_links.c
-    test_tables.c
-    test_ial.c
-    test_extensions.c
-    test_output.c
-    test_marked_integration.c
-)
-target_link_libraries(test_runner apex)
-target_compile_definitions(test_runner PRIVATE TEST_FIXTURES_DIR="${CMAKE_CURRENT_SOURCE_DIR}/fixtures/includes")
-
-# Add tests
-add_test(NAME basic_tests COMMAND test_runner)
-
-# Example test files
-configure_file(test_basic.md test_basic.md COPYONLY)
-

data/ext/apex_ext/apex_src/tests/README.md

@@ -1,152 +0,0 @@
-# Apex Test Suite
-
-## Running Tests
-
-```bash
-cd /path/to/apex
-./build/apex_test_runner
-
-```
-
-## Test Coverage
-
-**Total Tests**: 623
-**Status**: All passing ✓
-**Coverage**: 95% of implemented features
-
-## Code Coverage (Optional)
-
-If you want **line/branch coverage** (as opposed to feature coverage), Apex includes an optional helper script that builds with clang’s coverage instrumentation and generates an `llvm-cov` report.
-
-```bash
-cd /path/to/apex
-./test_coverage.sh
-```
-
-Notes:
-- Requires **clang** and LLVM tools (`llvm-cov` + `llvm-profdata`). On macOS, Xcode’s `xcrun` can provide these.
-- Output goes to `build-coverage/` and does not affect the normal `build/` directory.
-- Useful flags:
-  - `./test_coverage.sh --no-html` (print summary only)
-  - `./test_coverage.sh --no-open` (don’t auto-open the HTML report)
-
-### Test Categories
-
-1. **Basic Markdown** (5 tests)
-   - Headers, emphasis, lists
-
-2. **GFM Features** (5 tests)
-   - Strikethrough, task lists, tables
-
-3. **Metadata** (4 tests)
-   - YAML, MMD, Pandoc formats
-   - Variable replacement (`[%key]`)
-
-4. **Wiki Links** (15 tests)
-   - Basic links, display text, sections
-   - Space modes: dash, none, underscore, space
-   - Extension handling (with/without leading dot)
-   - Combinations of space modes and extensions
-
-5. **Math Support** (4 tests)
-   - Inline and display math
-   - False positive prevention
-
-6. **Critic Markup** (3 tests)
-   - Addition, deletion, highlight
-
-7. **Processor Modes** (4 tests)
-   - CommonMark, GFM, MMD, Unified
-
-8. **File Includes** (16 tests) ✨ NEW
-   - Marked: `<<[md]`, `<<(code)`, `<<{html}`
-   - MMD: `{{file}}`
-   - iA Writer: `/filename`
-   - CSV/TSV to table
-
-9. **IAL** (5 tests) ✨ NEW
-   - ID and class attributes
-   - Multiple classes
-
-10. **Definition Lists** (11 tests) ✨ NEW
-    - Basic syntax
-    - Multiple definitions
-
-11. **Advanced Tables** (6 tests)
-    - Captions, rowspan, colspan
-
-12. **Callouts** (10 tests) ✨ NEW
-    - Bear/Obsidian/Xcode syntax
-    - All callout types
-    - Collapsible callouts
-
-13. **TOC Generation** (14 tests) ✨ NEW
-    - Multiple marker formats
-    - Depth control
-    - Nested structure
-
-14. **Terminal Output** (12 tests)
-    - ANSI terminal and terminal256 format
-    - List markers (bullet and ordered, default list_marker styling)
-    - terminal_width option (library)
-    - Optional script: `./tests/terminal_width_test.sh` checks CLI `--width` wrapping
-
-15. **HTML Markdown Attributes** (9 tests) ✨ NEW
-    - markdown="1", "block", "span", "0"
-    - Nested HTML parsing
-
-16. **Abbreviations** (4 tests) ✨ NEW
-    - Definition syntax (partial support)
-
-17. **Emoji** (10 tests) ✨ NEW
-    - 350+ GitHub emoji
-    - Unknown emoji handling
-
-18. **Special Markers** (7 tests) ✨ NEW
-    - Page breaks, pauses
-    - End-of-block markers
-
-19. **Advanced Footnotes** (3 tests) ✨ NEW
-    - Basic and inline footnotes
-    - Markdown in footnotes
-
-## Test Fixtures
-
-Test files are located in `tests/fixtures/includes/`:
-
-- `simple.md` - Markdown content for includes
-- `code.py` - Python code file
-- `raw.html` - Raw HTML content
-- `data.csv` - CSV data
-- `data.tsv` - Tab-separated data
-- `image.png` - Image file (for type detection)
-
-## Adding New Tests
-
-1. Add test function to `test_runner.c`
-
-Use `assert_contains(html, expected, "Test name")` for validation
-
-3. Add test function call in `main()`
-4. Rebuild: `cmake --build build`
-5. Run: `./build/apex_test_runner`
-
-## Known Limitations
-
-Some advanced features work but have limited test coverage:
-
-- **Advanced Tables**: Rowspan/colspan rendering needs
-
-  custom renderer
-
-- **Critic Markup**: Some edge cases with
-
-  substitution/comment syntax
-
-See `docs/TEST_COVERAGE.md` for detailed analysis.
-
-## License
-
-Copyright &copy; 2026 Brett Terpstra
-
-[MIT License](LICENSE)

data/ext/apex_ext/apex_src/tests/benchmark.sh

@@ -1,113 +0,0 @@
-#!/bin/bash
-# Comprehensive Apex Performance Benchmark
-
-# Get script directory and ensure we're in the right place
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
-cd "$PROJECT_ROOT" || exit 1
-
-APEX="$PROJECT_ROOT/build/apex"
-TEST_FILE="$PROJECT_ROOT/tests/fixtures/comprehensive_test.md"
-ITERATIONS=50
-
-# Verify files exist
-if [ ! -f "$APEX" ]; then
-	echo "ERROR: Apex binary not found at $APEX"
-	echo "Please build the project first: make"
-	exit 1
-fi
-
-if [ ! -f "$TEST_FILE" ]; then
-	echo "ERROR: Test file not found at $TEST_FILE"
-	exit 1
-fi
-
-echo "# Apex Markdown Processor - Performance Benchmark"
-echo ""
-echo "## Test Document"
-echo ""
-LINES=$(wc -l <"$TEST_FILE")
-WORDS=$(wc -w <"$TEST_FILE")
-BYTES=$(wc -c <"$TEST_FILE")
-
-echo "- **File:** \`$TEST_FILE\`"
-echo "- **Lines:** $LINES"
-echo "- **Words:** $WORDS"
-echo "- **Size:** $BYTES bytes"
-echo ""
-
-# Function to run benchmark and return results
-benchmark() {
-	local mode="$1"
-	local args="$2"
-	local desc="$3"
-
-	# Warm-up run
-	if ! $APEX $args "$TEST_FILE" >/dev/null 2>&1; then
-		echo "ERROR: Failed to run apex command. Check if binary exists and test file is valid." >&2
-		return 1
-	fi
-
-	# Timed runs
-	local total=0
-	local min=999999
-	local max=0
-
-	for i in $(seq 1 $ITERATIONS); do
-		local start=$(gdate +%s%N 2>/dev/null || echo "$(date +%s)000000000")
-		if ! $APEX $args "$TEST_FILE" >/dev/null 2>&1; then
-			echo "ERROR: Failed on iteration $i" >&2
-			return 1
-		fi
-		local end=$(gdate +%s%N 2>/dev/null || echo "$(date +%s)000000000")
-		local elapsed=$(((end - start) / 1000000))
-
-		# Sanity check - elapsed should be positive
-		if [ $elapsed -lt 0 ]; then
-			echo "WARNING: Negative elapsed time on iteration $i, skipping" >&2
-			continue
-		fi
-
-		total=$((total + elapsed))
-		[ $elapsed -lt $min ] && min=$elapsed
-		[ $elapsed -gt $max ] && max=$elapsed
-	done
-
-	local avg=$((total / ITERATIONS))
-	local throughput="0"
-	if [ $avg -gt 0 ]; then
-		throughput=$(echo "scale=2; $WORDS / ($avg / 1000)" | bc 2>/dev/null || echo "0")
-	fi
-
-	# Output as table row
-	printf "| %s | %d | %d | %d | %d | %.2f |\n" "$desc" "$ITERATIONS" "$avg" "$min" "$max" "$throughput"
-}
-
-# Run benchmarks
-echo "## Output Modes"
-echo ""
-echo "| Mode | Iterations | Average (ms) | Min (ms) | Max (ms) | Throughput (words/sec) |"
-echo "|------|------------|--------------|---------|---------|------------------------|"
-
-benchmark "fragment" "" "Fragment Mode (default HTML output)"
-benchmark "pretty" "--pretty" "Pretty-Print Mode (formatted HTML)"
-benchmark "standalone" "--standalone" "Standalone Mode (complete HTML document)"
-benchmark "combined" "--standalone --pretty" "Standalone + Pretty (full features)"
-
-echo ""
-echo "## Mode Comparison"
-echo ""
-echo "| Mode | Iterations | Average (ms) | Min (ms) | Max (ms) | Throughput (words/sec) |"
-echo "|------|------------|--------------|---------|---------|------------------------|"
-
-benchmark "commonmark" "--mode commonmark" "CommonMark Mode (minimal, spec-compliant)"
-benchmark "gfm" "--mode gfm" "GFM Mode (GitHub Flavored Markdown)"
-benchmark "mmd" "--mode mmd" "MultiMarkdown Mode (metadata, footnotes, tables)"
-benchmark "kramdown" "--mode kramdown" "Kramdown Mode (attributes, definition lists)"
-benchmark "unified" "--mode unified" "Unified Mode (all features enabled)"
-benchmark "default" "" "Default Mode (unified, all features)"
-
-echo ""
-echo "---"
-echo ""
-echo "*Benchmark Complete*"

data/ext/apex_ext/apex_src/tests/benchmark_comparison.sh

@@ -1,166 +0,0 @@
-#!/bin/bash
-# Comparative benchmark: Apex vs other Markdown processors
-
-set -e
-
-# Get script directory and ensure we're in the right place
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
-cd "$PROJECT_ROOT" || exit 1
-
-ITERATIONS=20
-TEST_FILES=(
-	"$PROJECT_ROOT/tests/fixtures/comprehensive_test.md"
-)
-
-# Add larger test files if they exist
-[ -f "$PROJECT_ROOT/tests/fixtures/large_doc.md" ] && TEST_FILES+=("$PROJECT_ROOT/tests/fixtures/large_doc.md")
-
-echo "# Markdown Processor Comparison Benchmark"
-echo ""
-
-# Check which tools are available
-TOOLS=()
-APEX_BIN="$PROJECT_ROOT/build/apex"
-[ -f "$APEX_BIN" ] && TOOLS+=("apex:$APEX_BIN")
-command -v cmark-gfm >/dev/null && TOOLS+=("cmark-gfm:cmark-gfm -e table -e strikethrough -e autolink")
-command -v cmark >/dev/null && TOOLS+=("cmark:cmark")
-command -v pandoc >/dev/null && TOOLS+=("pandoc:pandoc -f markdown -t html")
-command -v multimarkdown >/dev/null && TOOLS+=("multimarkdown:multimarkdown")
-command -v kramdown >/dev/null && TOOLS+=("kramdown:kramdown")
-command -v marked >/dev/null && TOOLS+=("marked:marked")
-
-echo "## Available Tools"
-echo ""
-echo "Found ${#TOOLS[@]} tools:"
-for tool in "${TOOLS[@]}"; do
-	echo "- ${tool%%:*}"
-done
-echo ""
-
-# Function to benchmark a single tool
-benchmark_tool() {
-	local name="$1"
-	local cmd="$2"
-	local file="$3"
-	local iterations="$4"
-
-	# Warm-up
-	eval "$cmd \"$file\"" >/dev/null 2>&1 || return 1
-
-	# Timed runs using hyperfine if available, else manual timing
-	if command -v hyperfine >/dev/null 2>&1; then
-		result=$(hyperfine --warmup 3 --runs "$iterations" --export-json /dev/stdout \
-			"$cmd \"$file\"" 2>/dev/null | tail -n +5 | jq -r '.results[0].mean * 1000' 2>/dev/null)
-		echo "${result:-N/A}"
-	else
-		local total=0
-		for i in $(seq 1 $iterations); do
-			local start=$(python3 -c 'import time; print(int(time.time() * 1000))')
-			eval "$cmd \"$file\"" >/dev/null 2>&1
-			local end=$(python3 -c 'import time; print(int(time.time() * 1000))')
-			total=$((total + end - start))
-		done
-		echo "$((total / iterations))"
-	fi
-}
-
-# Run benchmarks for each file
-for file in "${TEST_FILES[@]}"; do
-	if [ ! -f "$file" ]; then
-		echo "⚠️  File not found: $file" >&2
-		continue
-	fi
-
-	size=$(wc -c <"$file" | tr -d ' ')
-	lines=$(wc -l <"$file" | tr -d ' ')
-
-	echo "## Processor Comparison"
-	echo ""
-	echo "**File:** \`$file\` ($size bytes, $lines lines)"
-	echo ""
-	echo "| Processor | Time (ms) | Relative |"
-	echo "|-----------|-----------|----------|"
-
-	baseline=""
-	for tool in "${TOOLS[@]}"; do
-		name="${tool%%:*}"
-		cmd="${tool#*:}"
-
-		result=$(benchmark_tool "$name" "$cmd" "$file" "$ITERATIONS" 2>/dev/null)
-
-		if [ -n "$result" ] && [ "$result" != "N/A" ]; then
-			if [ -z "$baseline" ]; then
-				baseline="$result"
-				relative="1.00x"
-			else
-				relative=$(echo "scale=2; $result / $baseline" | bc 2>/dev/null || echo "N/A")
-				relative="${relative}x"
-			fi
-			printf "| %s | %.2f | %s |\n" "$name" "$result" "$relative"
-		else
-			printf "| %s | failed | - |\n" "$name"
-		fi
-	done
-	echo ""
-done
-
-# Apex mode comparison
-echo "## Apex Mode Comparison"
-echo ""
-echo "**Test File:** \`${TEST_FILES[0]}\`"
-echo ""
-echo "| Mode | Time (ms) | Relative |"
-echo "|------|-----------|----------|"
-
-mode_baseline=""
-for mode in "commonmark" "gfm" "mmd" "kramdown" "unified" "default"; do
-	if [ "$mode" = "default" ]; then
-		cmd="$APEX_BIN"
-		display="default (unified)"
-	else
-		cmd="$APEX_BIN --mode $mode"
-		display="$mode"
-	fi
-
-	result=$(benchmark_tool "apex-$mode" "$cmd" "${TEST_FILES[0]}" "$ITERATIONS" 2>/dev/null)
-
-	if [ -n "$result" ] && [ "$result" != "N/A" ]; then
-		if [ -z "$mode_baseline" ]; then
-			mode_baseline="$result"
-			relative="1.00x"
-		else
-			relative=$(echo "scale=2; $result / $mode_baseline" | bc 2>/dev/null || echo "N/A")
-			relative="${relative}x"
-		fi
-		printf "| %s | %.2f | %s |\n" "$display" "$result" "$relative"
-	else
-		printf "| %s | failed | - |\n" "$display"
-	fi
-done
-
-echo ""
-echo "## Apex Feature Overhead"
-echo ""
-echo "| Features | Time (ms) |"
-echo "|----------|-----------|"
-
-base=$(benchmark_tool "base" "$APEX_BIN --mode commonmark --no-ids" "${TEST_FILES[0]}" "$ITERATIONS")
-printf "| CommonMark (minimal) | %.2f |\n" "$base"
-
-with_tables=$(benchmark_tool "tables" "$APEX_BIN --mode gfm" "${TEST_FILES[0]}" "$ITERATIONS")
-printf "| + GFM tables/strikethrough | %.2f |\n" "$with_tables"
-
-with_all=$(benchmark_tool "all" "$APEX_BIN" "${TEST_FILES[0]}" "$ITERATIONS")
-printf "| + All Apex features | %.2f |\n" "$with_all"
-
-with_pretty=$(benchmark_tool "pretty" "$APEX_BIN --pretty" "${TEST_FILES[0]}" "$ITERATIONS")
-printf "| + Pretty printing | %.2f |\n" "$with_pretty"
-
-with_standalone=$(benchmark_tool "standalone" "$APEX_BIN --standalone --pretty" "${TEST_FILES[0]}" "$ITERATIONS")
-printf "| + Standalone document | %.2f |\n" "$with_standalone"
-
-echo ""
-echo "---"
-echo ""
-echo "*Benchmark Complete*"