apex-ruby 1.0.30 → 1.0.31

data/ext/apex_ext/apex_src/docs/INDEX_IMPLEMENTATION_ANALYSIS.md

@@ -1,382 +0,0 @@
-# Index Implementation Analysis
-
-This document analyzes the implementation of index syntax
-support in Apex, covering mmark syntax, TextIndex syntax,
-and implementation considerations.
-
-## mmark Index Syntax
-
-### Syntax Overview
-
-mmark uses parentheses-based syntax for index entries:
-
-- **Standard index entry**: `(!item)` - Creates an index
-
-  entry for "item"
-
-- **Sub-item entry**: `(!item, subitem)` - Creates a
-
-  sub-entry under "item"
-
-- **Primary entry**: `(!!item, subitem)` - Marks an entry as
-
-  primary (bold in output)
-
-- **Section-level index**: An index entry can apply to an
-
-  entire section by placing it as the first paragraph after
-  a section heading
-
-### Example Usage
-
-```markdown
-# Test Document
-
-This document contains protocols (!Protocol) and implementations (!Implementation).
-
-## HTTP
-
-HTTP (!HTTP) is a protocol (!Protocol) used for web communication.
-
-### HTTP/1.1
-
-HTTP/1.1 (!HTTP, HTTP/1.1) is a common version.
-
-### HTTP/2
-
-HTTP/2 (!HTTP, HTTP/2) introduced multiplexing.
-
-## Security
-
-Security (!Security) is important. We discuss encryption (!Encryption).
-
-### Encryption Methods
-
-Symmetric encryption (!Encryption, Symmetric) uses the same key.
-
-Asymmetric encryption (!Encryption, Asymmetric) uses different keys.
-
-## Primary Index Entry
-
-This section discusses the primary topic (!!Primary Topic, Sub Topic).
-
-## Section-Level Index
-
-(!Section Index Entry)
-
-This entire section should be indexed as "Section Index Entry".
-
-```
-
-### mmark Output
-
-When processed with `mmark -html`, the syntax generates:
-
-**Inline markers**: Each index entry is converted to a `<span class="index" id="idxref:N"></span>` marker in the HTML
-
-**Index section**: At the end of the document, mmark automatically generates an `<h1 id="index-section">Index</h1>` section with a structured index:
-
-   - Alphabetically organized by first letter
-   - Hierarchical structure for sub-items
-
-Links back to document locations using `idxref:N` anchors
-
-   - Primary entries are marked (typically bold)
-
-### Implementation Notes
-
-- The `-index=false` flag suppresses index generation
-- Index entries are processed during inline parsing
-- The index is generated as a post-processing step after the
-
-  document is parsed
-
-## TextIndex Syntax
-
-### Syntax Overview
-
-TextIndex uses curly braces with a caret for index entries:
-
-- **Basic syntax**: `{^}` - Marks the preceding word or
-
-  phrase for indexing
-
-- **With explicit term**: `[term]{^}` - Indexes "term"
-
-  explicitly
-
-- **With parameters**: `{^param1 param2}` - Supports various
-
-  parameters for sorting, cross-references, etc.
-
-### Example Usage
-
-```markdown
-# Test Document
-
-Most mechanical keyboard firmware{^} supports the use of [key combinations]{^}.
-
-## Protocols
-
-HTTP{^} is a protocol{^} used for web communication.
-
-### HTTP/1.1
-
-HTTP/1.1{^} is a common version of the protocol{^}.
-
-### HTTP/2
-
-HTTP/2{^} introduced multiplexing.
-
-## Security
-
-Security{^} is important. We discuss encryption{^} and authentication{^}.
-
-### Encryption Methods
-
-Symmetric encryption{^} uses the same key for encryption and decryption.
-
-Asymmetric encryption{^} uses different keys.
-
-```
-
-### TextIndex Features
-
-From the Python implementation, TextIndex supports:
-
-1. **Concordance files**: External files that automatically
-
-   mark terms for indexing
-
-2. **Cross-references**: Link related terms within the index
-3. **Aliases**: Provide alternative terms or synonyms
-4. **Plural handling**: Manage singular and plural forms
-5. **Section mode**: Index entire sections
-6. **Sorting options**: Control how entries are sorted
-7. **Emphasis handling**: Mark entries with emphasis
-
-### TextIndex Pattern
-
-The regex pattern used by TextIndex is:
-```regex
-(?:(?<!\\)\[([^\]<>]+)(?<!\\)\]|([^\s\[\]\{\}<>]++))*(?<!>)\{\^([^\}<\n]*)\}(?!<)
-
-```
-
-This matches:
-
-- Optional `[term]` or a word/phrase
-- Followed by `{^}` with optional parameters inside
-
-## Implementation Difficulty Assessment
-
-### Complexity: Medium
-
-Implementing index support in Apex would be **moderately
-complex**, similar to the citations extension but
-potentially simpler in some ways.
-
-### Similar Existing Features
-
-Apex already has several features that demonstrate similar
-patterns:
-
-1. **Citations Extension** (`src/extensions/citations.c`):
-
-Processes inline syntax patterns (`[@key]`, `[#key]`, etc.)
-
-   - Collects citations during parsing
-   - Generates bibliography at the end
-   - Similar two-phase approach: collect during parse,
-
-     generate during render
-
-2. **Table of Contents** (`src/extensions/toc.c`):
-   - Collects headers from AST
-   - Generates TOC HTML
-   - Inserts at `<!--TOC-->` markers or end of document
-   - Similar post-processing approach
-
-3. **Wiki Links** (`src/extensions/wiki_links.c`):
-   - Processes inline syntax `[[Page Name]]`
-   - Uses inline parser matching
-   - Converts to HTML links
-
-### Implementation Approach
-
-#### Phase 1: Inline Parsing
-
-1. **Create index extension** (`src/extensions/index.c`):
-   - Register as inline syntax extension
-   - Match patterns:
-
-mmark: `(!item)`, `(!item, subitem)`, `(!!item, subitem)`
-
-     - TextIndex: `{^}`, `[term]{^}`, `{^params}`
-
-Store index entries in a registry (similar to citations)
-
-2. **Index entry structure**:
-   ```c
-   typedef struct apex_index_entry {
-       char *item;              /* Main index term */
-       char *subitem;           /* Sub-item (optional) */
-       bool primary;            /* Primary entry flag (mmark) */
-       int position;            /* Position in document */
-       char *anchor_id;         /* Generated anchor ID */
-       struct apex_index_entry *next;
-   } apex_index_entry;
-
-   ```
-
-3. **Processing**:
-   - During inline parsing, detect index syntax
-   - Extract terms and parameters
-   - Create index entries
-   - Replace syntax with invisible markers or spans (like
-
-     mmark does)
-
-#### Phase 2: Index Generation
-
-1. **Collect all entries** from the registry
-2. **Sort alphabetically** by item, then subitem
-3. **Group by first letter** (optional, like mmark)
-4. **Generate HTML**:
-   - Create hierarchical structure (items with sub-items)
-   - Link to document positions using anchor IDs
-   - Mark primary entries appropriately
-5. **Insert at end** or at `<!--INDEX-->` marker
-
-### Code Structure
-
-```
-src/extensions/
-  index.c          - Main implementation
-  index.h          - Header with structures and functions
-
-```
-
-Key functions needed:
-
-- `create_index_extension()` - Register extension
-- `apex_process_index_entries()` - Parse and collect entries
-- `apex_generate_index_html()` - Generate index HTML
-- `apex_insert_index()` - Insert index into document
-
-### Challenges
-
-1. **Syntax conflicts**:
-
-`(!item)` could conflict with other parentheses-based syntax
-
-   - `{^}` could conflict with other brace syntax
-   - Need careful pattern matching to avoid false positives
-
-2. **Position tracking**:
-   - Need to track document positions for linking
-   - Similar to how citations track positions
-
-3. **Sorting and grouping**:
-   - Alphabetical sorting with proper handling of special
-
-     characters
-
-   - Grouping by first letter (optional feature)
-
-4. **Section-level indexing**:
-
-Detecting when an index entry applies to an entire section
-
-   - Requires understanding of document structure
-
-5. **Dual syntax support**:
-   - Supporting both mmark and TextIndex syntax
-   - May need configuration option to choose syntax or
-
-     support both
-
-### Estimated Effort
-
-- **Basic mmark syntax**: 2-3 days
-- **TextIndex syntax**: 1-2 days additional
-- **Section-level indexing**: 1 day
-- **Testing and refinement**: 1-2 days
-
-**Total**: ~5-8 days of development time
-
-## Other Markdown Flavors with Index Support
-
-### MultiMarkdown
-
-MultiMarkdown has native support for index creation using
-the same syntax as mmark:
-
-- `(!item)` for standard entries
-- `(!item, subitem)` for sub-entries
-- `(!!item, subitem)` for primary entries
-
-This makes sense since mmark is based on MultiMarkdown.
-
-### Pandoc
-
-Pandoc does **not** have native index support, but:
-
-- Supports extensive customization through filters
-- Can implement indexing using Lua filters
-- Requires additional scripting or external tools
-
-### Kramdown
-
-Kramdown does **not** provide built-in support for indexes.
-Users would need:
-
-- Custom solutions
-- External tools to generate indexes
-
-### Summary
-
-Among major Markdown processors:
-
-- **MultiMarkdown/mmark**: Native support with `(!item)`
-
-  syntax
-
-- **Pandoc**: No native support, but extensible via filters
-- **Kramdown**: No native support
-- **TextIndex**: Standalone tool with `{^}` syntax
-
-## Recommendations
-
-1. **Start with mmark syntax**: It's more established and
-
-   similar to MultiMarkdown (which Apex already supports)
-
-2. **Add TextIndex syntax as optional**: Can be enabled via
-
-   flag or mode
-
-3. **Follow citations pattern**: Use similar architecture to
-
-   citations extension
-
-**Make index generation optional**: Support `--no-index` flag (like mmark's `-index=false`)
-
-5. **Support index marker**: Allow `<!--INDEX-->` marker for
-
-   custom placement
-
-6. **Consider configuration options**:
-   - Alphabetical grouping (by first letter)
-   - Index placement (end vs. marker)
-   - Primary entry styling
-
-## Test Files
-
-Test documents have been created:
-
-- `test_index_mmark.md` - mmark syntax examples
-- `test_index_textindex.md` - TextIndex syntax examples
-
-These can be used for testing once implementation is
-complete.

data/ext/apex_ext/apex_src/docs/INTEGRATION_EXAMPLE.m

@@ -1,144 +0,0 @@
-/**
- * Example code snippets for integrating Apex into Marked
- *
- * These snippets show how to add Apex processor support to existing Marked code
- */
-
-#import "NSString+Apex.h"
-
-// ============================================================================
-// EXAMPLE 1: Adding Apex to MKConductorTransformer.m runProcessor method
-// ============================================================================
-
-// Around line 216, add this case after kramdown:
-
-  } else if ([processor isEqualToString:@"kramdown"]) {
-    useCustom = NO;
-    [defaults setBool:NO forKey:@"isMultiMarkdownDefault"];
-    [defaults setValue:@"Kramdown" forKey:@"defaultProcessor"];
-  } else if ([processor isEqualToString:@"apex"]) {
-    useCustom = NO;
-    [defaults setBool:NO forKey:@"isMultiMarkdownDefault"];
-    [defaults setValue:@"Apex" forKey:@"defaultProcessor"];
-  } else if ([processor isEqualToString:@"custom"]) {
-    useCustom = YES;
-  }
-
-// Around line 232, add this conversion case:
-
-  } else if ([processor isEqualToString:@"Kramdown"]) {
-    result = [NSString convertWithKramdown:text];
-  } else if ([processor isEqualToString:@"Apex"]) {
-    result = [NSString convertWithApex:text];
-  } else if ([processor isEqualToString:@"MultiMarkdown"]) {
-    result = [NSString convertWithMultiMarkdown:text];
-  }
-
-// ============================================================================
-// EXAMPLE 2: Adding Apex to NSString_MultiMarkdown.m processMultiMarkdown
-// ============================================================================
-
-// In the processor selection code around line 3878:
-
-  } else if ([processor isEqualToString:@"Kramdown"]) {
-    DDLogInfo(@"Starting Kramdown conversion");
-    out = [NSString convertWithKramdown:safeInputString];
-  } else if ([processor isEqualToString:@"Apex"]) {
-    DDLogInfo(@"Starting Apex conversion");
-    out = [NSString convertWithApex:safeInputString];
-  } else if ([processor isEqualToString:@"MultiMarkdown"]) {
-    DDLogInfo(@"Starting MultiMarkdown conversion");
-    out = [self convertWithMultiMarkdown:safeInputString];
-  }
-
-// Also add Apex handling in custom processor fallback around line 3780:
-
-  } else if ([outputString.uppercaseString isEqualToString:@"KRAMDOWN"]) {
-    DDLogInfo(@"Custom processor returned KRAMDOWN directive");
-    [defaults setBool:NO forKey:@"isMultiMarkdownDefault"];
-    [defaults setValue:@"Kramdown" forKey:@"defaultProcessor"];
-  } else if ([outputString.uppercaseString isEqualToString:@"APEX"]) {
-    DDLogInfo(@"Custom processor returned APEX directive");
-    [defaults setBool:NO forKey:@"isMultiMarkdownDefault"];
-    [defaults setValue:@"Apex" forKey:@"defaultProcessor"];
-  } else {
-    // ... existing code
-  }
-
-// ============================================================================
-// EXAMPLE 3: Using Apex with specific mode
-// ============================================================================
-
-// You can call Apex with a specific processor mode:
-NSString *html;
-
-// Use GFM mode
-html = [NSString convertWithApex:markdown mode:@"gfm"];
-
-// Use MultiMarkdown mode
-html = [NSString convertWithApex:markdown mode:@"multimarkdown"];
-
-// Use unified mode (all features)
-html = [NSString convertWithApex:markdown mode:@"unified"];
-
-// ============================================================================
-// EXAMPLE 4: Adding to Preferences UI
-// ============================================================================
-
-// In AppPrefsWindowController.m or wherever processor dropdown is populated
-// Add "Apex (Unified)" to the list of processors:
-
-NSArray *processors = @[
-    @"MultiMarkdown",
-    @"Discount (GFM)",
-    @"CommonMark",
-    @"Kramdown",
-    @"Apex"  // Add this
-];
-
-// ============================================================================
-// EXAMPLE 5: Using Apex from Custom Processor Rules
-// ============================================================================
-
-// Users can create a Custom Processor Rule that returns "APEX" to use Apex:
-
-// In a shell script custom processor:
-#!/bin/bash
-if [[ "$MARKED_PATH" == *.wiki ]]; then
-    echo "APEX"
-else
-    echo "NOCUSTOM"
-fi
-
-// ============================================================================
-// EXAMPLE 6: Direct C API usage (if needed for performance)
-// ============================================================================
-
-#include <apex/apex.h>
-
-// Get default options
-apex_options options = apex_options_default();
-
-// Or get mode-specific options
-apex_options gfm_options = apex_options_for_mode(APEX_MODE_GFM);
-
-// Convert markdown
-const char *markdown = "# Hello\n\nWorld";
-char *html = apex_markdown_to_html(markdown, strlen(markdown), &options);
-
-// Use html...
-
-// Clean up
-apex_free_string(html);
-
-// ============================================================================
-// Notes
-// ============================================================================
-
-/*
- * Performance: Apex should be comparable to or faster than existing processors
- * Memory: Uses cmark-gfm's efficient arena allocator
- * Thread Safety: Create separate apex_options for each thread
- * Error Handling: Returns empty string on error, never NULL
- */
-