RubyGems - apex-ruby - Versions diffs - 1.0.30 → 1.0.32 - Mend

apex-ruby 1.0.30 → 1.0.32

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (490) hide show

data/ext/apex_ext/apex_src/docs/CITATIONS_IMPLEMENTATION_PLAN.md DELETED Viewed

@@ -1,749 +0,0 @@
-# Apex Citations and Bibliography Implementation Plan
-## Executive Summary
-This document provides a comprehensive plan for implementing
-citations and bibliographies in Apex, based on analysis of
-how mmark, MultiMarkdown, and Pandoc handle citations. It
-addresses key questions about bibliography formats, external
-files, metadata integration, and syntax modes.
-## Analysis of Existing Tools
-### Pandoc Output Analysis
-**Citation Format:**
-Uses `<span class="citation" data-cites="key1 key2">(Formatted Citation)</span>`
-Supports multiple citations in one span: `data-cites="doe99 smith2000 smith2004"`
-- Author-date style: `(Doe 1999; Jane Smith 2000, 2004)`
-- Author-in-text: `John Smith (2004)` (no parentheses)
-- Author-suppressed: `(2004)` (no author name)
-**Bibliography Format:**
-Uses `<div id="refs" class="references csl-bib-body hanging-indent">`
-- Each entry: `<div id="ref-key" class="csl-entry">`
-- Formatted with CSL (Citation Style Language)
-- Includes proper typography (em dashes, quotes, italics)
-**Key Observations:**
-Pandoc uses CSL (Citation Style Language) for all formatting
-- Citations are processed through citeproc
-- Bibliography is automatically generated from cited items
-- Supports BibTeX, CSL JSON, and other formats as input
-### mmark Output Analysis
-**Citation Format:**
-- Uses XML2RFC format: `<xref target="RFC2535"></xref>`
-For HTML output, converts to simple links or formatted text
-- Special handling for RFC/BCP/STD/I-D/W3C documents
-- Auto-generates references for these document types
-**Bibliography Format:**
-- Requires `{backmatter}` marker to generate bibliography
-- Uses XML2RFC `<reference>` elements
-- Auto-fetches RFC/I-D metadata from citation index
-**Key Observations:**
-- mmark is specialized for RFC documents
-- Uses external citation index for RFC/I-D metadata
-- Bibliography only appears if `{backmatter}` is present
-- Less flexible than Pandoc for general academic citations
-### MultiMarkdown Output Analysis
-**Citation Format:**
-Uses numeric references: `<a href="#Doe:2006" id="Doe:2006:1">[1]</a>`
-- Simple numbered citations
-- Links to bibliography entries
-**Bibliography Format:**
-Uses `<div id="bibliography">` with entries like `<div id="Doe:2006">`
-- Simple format, no CSL styling
-- References defined inline or in separate section
-**Key Observations:**
-- Simpler than Pandoc
-- No CSL support
-- Numeric citations only
-- Good for basic use cases
-## Recommended Approach for Apex
-### Bibliography Format: CSL JSON (Primary)
-**Why CSL JSON?**
-1. **Native Format**: CSL JSON is the native format for
-   citation processing
-2. **Flexibility**: Supports all citation styles via CSL
-   stylesheets
-**Standard**: Widely used (Pandoc, Zotero, Mendeley, etc.)
-4. **Extensibility**: Easy to add new fields and entry types
-5. **Processing**: Can be directly consumed by citeproc
-   processors
-**Format Priority:**
-1. **CSL JSON** (`.json`) - Primary format, native support
-2. **BibTeX** (`.bib`) - Most common, convert to CSL JSON
-   internally
-**CSL YAML** (`.yaml`) - Human-readable, convert to CSL JSON
-4. **RIS** (`.ris`) - Lower priority, can add later
-### Bibliography Storage: External Files (Recommended)
-**Yes, external files should be supported** for the
-following reasons:
-1. **Reusability**: Same bibliography can be used across
-   multiple documents
-2. **Management**: Reference managers (Zotero, Mendeley)
-   export to `.bib` or `.json`
-3. **Standard Practice**: Pandoc, Quarto, and other tools
-   use external files
-4. **Scalability**: Large bibliographies are easier to
-   manage in separate files
-**Implementation Options:**
-#### Option 1: External Files Only (Recommended)
-- Specify via metadata: `bibliography: references.bib`
-- Specify via CLI: `--bibliography references.bib`
-Support multiple files: `bibliography: [refs1.bib, refs2.bib]`
-- Auto-detect format from extension
-#### Option 2: Hybrid (External + Inline)
-- Support external files (primary)
-- Support inline YAML `references:` field (for small
-  bibliographies)
-- Support MultiMarkdown reference definitions (for
-  compatibility)
-- Merge all sources (inline takes precedence)
-**Recommendation**: Implement Option 2 (Hybrid) for maximum
-flexibility.
-### Metadata Integration
-**YAML Front Matter:**
-```yaml
----
-bibliography: references.bib
-# or multiple files:
-bibliography:
-  - refs1.bib
-  - refs2.bib
-csl: apa.csl  # Citation style file
-suppress-bibliography: false
-link-citations: true
-show-tooltips: false
-nocite: |
-  @item1, @item2
-# or include all:
-nocite: '@*'
-references:  # Inline references (CSL JSON format)
-  - id: doe99
-    type: article-journal
-    author:
-      - family: Doe
-        given: John
-    issued:
-      date-parts: [[1999]]
-    title: Article Title
-    container-title: Journal Name
-    volume: 1
-    page: 1-10
----
-```
-**MultiMarkdown Metadata:**
-```
-Bibliography: references.bib
-CSL: apa.csl
-Suppress Bibliography: false
-Link Citations: true
-```
-**Pandoc Title Block:**
-```
-% Bibliography: references.bib
-% CSL: apa.csl
-```
-### Command-Line Flags
-```bash
-# Single bibliography file
-apex input.md --bibliography refs.bib
-# Multiple bibliography files
-apex input.md --bibliography refs1.bib --bibliography refs2.bib
-# Citation style
-apex input.md --bibliography refs.bib --csl apa.csl
-# Suppress bibliography
-apex input.md --bibliography refs.bib --no-bibliography
-# Link citations to bibliography
-apex input.md --bibliography refs.bib --link-citations
-# Show tooltips on citations
-apex input.md --bibliography refs.bib --show-tooltips
-```
-**Precedence Order:**
-1. Command-line flags (highest priority)
-2. Document metadata
-3. Default values (lowest priority)
-### Syntax Modes
-#### Mode-Specific Behavior
-**CommonMark Mode:**
-- Citations: **Disabled** (pure CommonMark, no extensions)
-- Syntax: None
-**GFM Mode:**
-- Citations: **Disabled** (GFM doesn't support citations)
-- Syntax: None
-**MultiMarkdown Mode:**
-- Citations: **Enabled**
-- Syntax: MultiMarkdown syntax only (`[#key]`)
-- Bibliography: From reference definitions or external file
-- Output: Numeric citations (like MultiMarkdown)
-**Kramdown Mode:**
-- Citations: **Disabled** (Kramdown uses different citation
-  system)
-- Syntax: None
-- *Note: Could add Kramdown citation syntax in future*
-**Unified Mode:**
-- Citations: **Enabled**
-- Syntax: **All syntaxes** (Pandoc, MultiMarkdown, mmark)
-- Auto-detection: Flexible parsing
-- Bibliography: All sources (external files, inline YAML,
-  MMD definitions)
-#### Syntax Detection in Unified Mode
-**Strategy: Pattern-Based Detection**
-1. **Pandoc Syntax** (`[@key]` or `@key`):
-   - Pattern: `[@` followed by alphanumeric key
-   - Most common, try first
-Supports: prefixes, locators, suffixes, author suppression
-2. **MultiMarkdown Syntax** (`[#key]`):
-   - Pattern: `[#` followed by citation key
-   - Distinctive `#` marker
-   - Supports: locators before citation
-3. **mmark Syntax** (`[@RFC1234]`):
-   - Pattern: `[@` followed by RFC/BCP/STD/I-D/W3C prefix
-   - Special case: auto-generate references
-Supports: modifiers (`!`, `?`, `-`), combined references
-**Detection Algorithm:**
-```
-For each potential citation:
-  1. Check for mmark pattern: [@RFC...] or [@BCP...] etc.
-     → If match: Process as mmark citation
-  2. Check for MultiMarkdown pattern: [#key]
-     → If match: Process as MMD citation
-  3. Check for Pandoc pattern: [@key] or @key
-     → If match: Process as Pandoc citation
-  4. If no match: Leave as-is (not a citation)
-```
-**Edge Cases:**
-- `[@RFC1234]` could be Pandoc or mmark → Prefer mmark if
-  matches RFC pattern
-- `[#key]` is unambiguous → Always MultiMarkdown
-- `[@key]` without RFC prefix → Pandoc syntax
-## Implementation Architecture
-### Processing Pipeline
-```
-1. Preprocessing Phase
-   ├─ Extract metadata (YAML, MMD, Pandoc)
-   ├─ Load bibliography files (from metadata or CLI)
-   ├─ Parse bibliography (BibTeX → CSL JSON, CSL JSON, CSL YAML)
-   └─ Build citation index (map keys to entries)
-2. Parsing Phase
-   ├─ Scan for citations (mode-aware)
-   ├─ Extract citation data (key, locator, prefix, suffix)
-   ├─ Match citations to bibliography entries
-   └─ Store citation registry
-3. Rendering Phase
-   ├─ Process citations through CSL (or simple formatter)
-   ├─ Generate citation HTML
-   ├─ Collect cited items for bibliography
-   └─ Format bibliography entries
-4. Post-processing Phase
-   ├─ Insert bibliography at marker or end
-   ├─ Add links between citations and bibliography
-   ├─ Add tooltips (if enabled)
-   └─ Final HTML output
-```
-### File Structure
-```
-src/extensions/
-├── citations.h              # Citation parsing header
-├── citations.c              # Citation parsing implementation
-├── bibliography.h            # Bibliography handling header
-├── bibliography.c           # Bibliography parsing/formatting
-├── bibtex_parser.h          # BibTeX parser header
-├── bibtex_parser.c          # BibTeX parser implementation
-└── csl_formatter.h          # CSL formatter header (if implementing minimal CSL)
-└── csl_formatter.c          # CSL formatter implementation (if implementing minimal CSL)
-```
-### Core Data Structures
-```c
-/* Citation structure */
-typedef struct {
-    char *key;                    // Citation key (e.g., "doe99")
-    char *prefix;                 // Prefix text (e.g., "see")
-    char *locator;                // Locator (e.g., "pp. 33-35")
-    char *suffix;                 // Suffix text (e.g., "and passim")
-    bool author_suppressed;       // -@key syntax
-    bool author_in_text;          // @key syntax (no brackets)
-    int syntax_type;              // PANDOC, MMD, MMARK
-    int position;                 // Position in document
-} apex_citation;
-/* Bibliography entry (CSL JSON format) */
-typedef struct {
-    char *id;                     // Citation key
-    char *type;                   // Entry type (article-journal, book, etc.)
-    cJSON *data;                  // Full CSL JSON object (using cJSON)
-} apex_bibliography_entry;
-/* Citation registry */
-typedef struct {
-    apex_citation **citations;   // Array of citations
-    size_t count;                 // Number of citations
-    apex_bibliography_entry **entries;  // Bibliography entries
-    size_t entry_count;           // Number of entries
-    cJSON *csl_style;             // CSL style (if using)
-} apex_citation_registry;
-```
-## Bibliography Format Details
-### CSL JSON Structure
-```json
-[
-  {
-    "id": "doe99",
-    "type": "article-journal",
-    "author": [
-      {
-        "family": "Doe",
-        "given": "John"
-      }
-    ],
-    "issued": {
-      "date-parts": [[1999]]
-    },
-    "title": "Article Title",
-    "container-title": "Journal Name",
-    "volume": "1",
-    "page": "1-10"
-  }
-]
-```
-### BibTeX to CSL JSON Conversion
-**Common BibTeX Entry Types:**
-- `@article` → `article-journal`
-- `@book` → `book`
-- `@inbook` → `chapter`
-- `@incollection` → `chapter`
-- `@inproceedings` → `paper-conference`
-- `@phdthesis` → `thesis`
-- `@mastersthesis` → `thesis`
-- `@techreport` → `report`
-**Field Mapping:**
-- `author` → `author` (parse names)
-- `title` → `title`
-- `journal` → `container-title`
-- `year` → `issued.date-parts`
-- `pages` → `page`
-- `publisher` → `publisher`
-- `volume` → `volume`
-- `number` → `issue`
-### CSL YAML Structure
-```yaml
-- id: doe99
-  type: article-journal
-  author:
-    - family: Doe
-      given: John
-  issued:
-    date-parts:
-      - - 1999
-  title: Article Title
-  container-title: Journal Name
-  volume: 1
-  page: 1-10
-```
-## Output Format
-### Citation HTML (Pandoc-style)
-**Author-Date Style:**
-```html
-<span class="citation" data-cites="doe99">(Doe 1999)</span>
-```
-**Multiple Citations:**
-```html
-<span class="citation" data-cites="doe99 smith2000 smith2004">
-  (Doe 1999; Jane Smith 2000, 2004)
-</span>
-```
-**With Locator:**
-```html
-<span class="citation" data-cites="doe99">Doe (1999, 33–35)</span>
-```
-**Author-Suppressed:**
-```html
-<span class="citation" data-cites="smith04">(2004)</span>
-```
-**With Links:**
-```html
-<a href="#ref-doe99" class="citation" data-cites="doe99">
-  (Doe 1999)
-</a>
-```
-**With Tooltips:**
-```html
-<a href="#ref-doe99" class="citation" data-cites="doe99"
-   title="Doe, John. 1999. 'Article Title.' Journal Name 1: 1-10.">
-  (Doe 1999)
-</a>
-```
-### Bibliography HTML
-```html
-<div id="refs" class="references csl-bib-body hanging-indent" role="doc-bibliography">
-  <div id="ref-doe99" class="csl-entry" role="doc-biblioentry">
-    Doe, John. 1999. <span>"Article Title."</span> <em>Journal Name</em> 1: 1–10.
-  </div>
-  <div id="ref-smith2000" class="csl-entry" role="doc-biblioentry">
-    Smith, Jane. 2000. <em>Book Title</em>. Publisher.
-  </div>
-</div>
-```
-### Bibliography Insertion Points
-1. **`<!-- REFERENCES -->` marker** (if present)
-2. **`{backmatter}` marker** (mmark style, if present)
-3. **`<div id="refs">`** (if present)
-4. **End of document** (default)
-## Dependencies
-### Required Libraries
-1. **JSON Parser**: `cJSON` (lightweight C JSON library)
-   - For parsing CSL JSON and BibTeX (after conversion)
-   - License: MIT
-   - Small footprint
-2. **BibTeX Parser**: Implement minimal parser in C
-   - Parse BibTeX entries
-   - Convert to CSL JSON
-   - Handle common entry types and fields
-3. **YAML Parser**: Reuse existing Apex YAML parser
-   - Already used for metadata
-   - Can parse CSL YAML format
-### CSL Processing Options
-**Option 1: Minimal CSL Implementation (Recommended for
-MVP)**
-- Implement basic author-date formatting
-- Support common entry types (article, book, chapter)
-- Simple formatting rules
-- **Pros**: No external dependencies, full control
-- **Cons**: Limited style support, more work
-**Option 2: Use citeproc-rs via FFI**
-- Rust library with C bindings
-- Full CSL support
-- **Pros**: Complete CSL support, well-tested
-- **Cons**: Requires Rust toolchain, FFI complexity
-**Option 3: Use citeproc-js via Node.js**
-- JavaScript library
-- **Pros**: Full CSL support
-- **Cons**: Requires Node.js, complex integration
-**Recommendation**: Start with Option 1 (minimal
-implementation) for MVP, consider Option 2 for full CSL
-support later.
-## Implementation Phases
-### Phase 1: Basic Citation Parsing (MVP)
-- [ ] Parse Pandoc syntax `[@key]` (simple keys only)
-- [ ] Parse MultiMarkdown syntax `[#key]`
-- [ ] Parse mmark syntax `[@RFC1234]` (basic)
-- [ ] Store citations in registry
-- [ ] Basic HTML output (show key or simple format)
-- [ ] Mode-aware syntax detection
-### Phase 2: Bibliography Loading
-- [ ] Load CSL JSON files
-- [ ] Parse CSL JSON to internal format
-- [ ] Load BibTeX files
-- [ ] Convert BibTeX to CSL JSON
-- [ ] Load CSL YAML files
-- [ ] Parse MultiMarkdown reference definitions
-- [ ] Merge multiple bibliography sources
-- [ ] Match citations to bibliography entries
-### Phase 3: Basic Citation Formatting
-- [ ] Implement minimal author-date formatter
-- [ ] Format citations with author names and years
-- [ ] Handle multiple citations
-- [ ] Generate proper HTML output
-- [ ] Add CSS classes
-### Phase 4: Bibliography Generation
-- [ ] Collect cited items
-- [ ] Sort bibliography entries
-- [ ] Format bibliography entries
-- [ ] Generate bibliography HTML
-- [ ] Insert bibliography at marker or end
-### Phase 5: Advanced Features
-- [ ] Support locators (pages, chapters, etc.)
-- [ ] Support prefixes and suffixes
-- [ ] Support author suppression (`-@key`)
-- [ ] Support author-in-text (`@key`)
-- [ ] Add citation links
-- [ ] Add tooltips
-- [ ] Support `nocite` metadata
-### Phase 6: Full CSL Support (Future)
-- [ ] Integrate full CSL processor
-- [ ] Support multiple citation styles
-- [ ] Load CSL style files
-- [ ] Support numeric and note styles
-## Testing Strategy
-### Unit Tests
-1. **Citation Parsing**:
-   - Test each syntax (Pandoc, MMD, mmark)
-   - Test edge cases (nested brackets, special characters)
-   - Test mode detection in unified mode
-2. **Bibliography Parsing**:
-   - Test each format (BibTeX, CSL JSON, CSL YAML)
-   - Test multiple files
-   - Test inline references
-   - Test error handling (missing files, invalid format)
-3. **Citation Formatting**:
-   - Test author-date formatting
-   - Test multiple citations
-   - Test with/without links
-   - Test with/without tooltips
-4. **Bibliography Generation**:
-   - Test sorting
-   - Test formatting
-   - Test insertion points
-   - Test suppression
-### Integration Tests
-1. **End-to-end**:
-   - Full document with citations
-   - Multiple bibliography sources
-   - Different citation styles
-   - Different output formats
-2. **Mode Compatibility**:
-Test in each mode (commonmark, gfm, mmd, kramdown, unified)
-   - Test syntax detection in unified mode
-   - Test feature flags
-## Configuration
-### API Options
-```c
-typedef struct {
-    // ... existing options ...
-    // Citation options
-    bool enable_citations;
-    char **bibliography_files;      // NULL-terminated array
-    const char *csl_file;            // CSL style file
-    bool suppress_bibliography;
-    bool link_citations;
-    bool show_tooltips;
-    const char *nocite;              // Comma-separated keys, or "*" for all
-} apex_options;
-```
-## Open Questions
-1. **CSL Processor**: Start with minimal implementation or
-   use existing library?
-   - **Recommendation**: Start minimal, add full CSL later
-2. **BibTeX Parser**: Implement or use library?
-   - **Recommendation**: Implement minimal parser
-3. **Performance**: How to handle large bibliographies
-   (1000+ entries)?
-   - **Recommendation**: Use hash table for lookup, cache
-     parsed bibliographies
-4. **Validation**: How strict should citation validation be?
-   - **Recommendation**: Warn on missing citations, don't
-     error (like Pandoc)
-5. **Error Handling**: How to handle missing citations?
-**Recommendation**: Show citation key in output with warning
-6. **Backwards Compatibility**: How to ensure existing
-   documents still work?
-   - **Recommendation**: Citations only processed if
-     bibliography is provided
-## References
-[Pandoc Citations](https://pandoc.org/demo/example33/8.20-citation-syntax.html)
-[MultiMarkdown Citations](https://fletcher.github.io/MultiMarkdown-6/Documentation/bibliography.html)
-- [mmark Citations](https://mmark.miek.nl/post/citations/)
-- [CSL Project](https://citationstyles.org/)
-[Citation Style Language](https://docs.citationstyles.org/en/stable/)
-- [cJSON Library](https://github.com/DaveGamble/cJSON)