RubyGems - apex-ruby - Versions diffs - 1.0.17 → 1.0.20 - Mend

apex-ruby 1.0.17 → 1.0.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (207) hide show

data/ext/apex_ext/apex_src/docs/plans/2025-02-27-man-page-output-implementation.md ADDED Viewed

@@ -0,0 +1,351 @@
+# Man Page (roff / man-html) Output Implementation Plan
+> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
+**Goal:** Add two new Apex output formats: roff (UNIX man page source) and styled man-page HTML, so that `apex -t man` and `apex -t man-html` produce output similar to Pandoc's `-t man` and [md2man](https://github.com/sunaku/md2man)-style HTML (e.g. [EXAMPLE](https://sunaku.github.io/md2man/man/man0/EXAMPLE.html)).
+**Architecture:** Follow the existing AST-renderer pattern used by terminal and markdown: add `APEX_OUTPUT_MAN` and `APEX_OUTPUT_MAN_HTML` to the enum; implement a new module `src/ast_man.c` with `apex_cmark_to_man_roff()` and `apex_cmark_to_man_html()` that walk the cmark AST and emit roff macros (e.g. `.TH`, `.SH`, `.B`, `.I`) or a self-contained styled HTML document. Man page metadata (name, section, date) is taken from document metadata (e.g. `man_title`, `man_section`, `man_date`) or sensible defaults. No external tools (Pandoc/md2man) are invoked; rendering is in-process.
+**Tech Stack:** C, cmark-gfm AST, existing apex_options and CLI; roff (man(7)) and HTML output as plain strings.
+---
+## Task 1: Add output format enum values and API placeholder
+**Files:**
+- Modify: `include/apex/apex.h` (around the `apex_output_format_t` enum, ~lines 43-54)
+- Create: `include/apex/ast_man.h`
+- Modify: `src/apex.c` (includes and one conditional that returns early for man formats with a stub)
+- Modify: `cli/main.c` (help string and `-t`/`--to` parsing)
+**Step 1: Extend the output format enum**
+In `include/apex/apex.h`, add after `APEX_OUTPUT_TERMINAL256`:
+```c
+    APEX_OUTPUT_TERMINAL256 = 9,     /* ANSI terminal output (256-color) */
+    APEX_OUTPUT_MAN = 10,            /* roff (man page source) */
+    APEX_OUTPUT_MAN_HTML = 11        /* styled HTML man page */
+} apex_output_format_t;
+```
+**Step 2: Add public API header**
+Create `include/apex/ast_man.h`:
+```c
+#ifndef APEX_AST_MAN_H
+#define APEX_AST_MAN_H
+#include "apex/apex.h"
+#include "cmark-gfm.h"
+#ifdef __cplusplus
+extern "C" {
+#endif
+char *apex_cmark_to_man_roff(cmark_node *document, const apex_options *options);
+char *apex_cmark_to_man_html(cmark_node *document, const apex_options *options);
+#ifdef __cplusplus
+}
+#endif
+#endif
+```
+**Step 3: Add stub implementation and wire in apex.c**
+Create `src/ast_man.c` with two functions that return a minimal valid result (e.g. roff: `.TH stub 1 "" ""` and newline; HTML: `<!DOCTYPE html><html><body><p>stub</p></body></html>`). In `src/apex.c`: add `#include "apex/ast_man.h"`. After the terminal block (around 5156), add:
+```c
+    if (options->output_format == APEX_OUTPUT_MAN) {
+        char *roff = apex_cmark_to_man_roff(document, options);
+        return roff;
+    }
+    if (options->output_format == APEX_OUTPUT_MAN_HTML) {
+        char *man_html = apex_cmark_to_man_html(document, options);
+        return man_html;
+    }
+```
+**Step 4: Wire CLI**
+In `cli/main.c`: in the `-t`/`--to` parsing block, add branches for `"man"` -> `APEX_OUTPUT_MAN` and `"man-html"` -> `APEX_OUTPUT_MAN_HTML`. Update the help string for `-t, --to FORMAT` to include `man` and `man-html`.
+**Step 5: Add to AST filter target format**
+In `src/apex.c`, in the block that sets `target_format` for AST filters (around 5029), add:
+```c
+} else if (options->output_format == APEX_OUTPUT_MAN ||
+           options->output_format == APEX_OUTPUT_MAN_HTML) {
+    target_format = "man";
+}
+```
+**Step 6: Register ast_man.c in build**
+In `CMakeLists.txt`, add `src/ast_man.c` to the list of source files that include `ast_terminal.c` (around line 113).
+**Step 7: Build and smoke test**
+Run: `cd build && cmake .. && make apex_cli`
+Run: `echo '# Hi' | ./apex -t man`
+Expected: output starts with `.TH` and contains a stub or minimal section.
+Run: `echo '# Hi' | ./apex -t man-html`
+Expected: HTML document containing stub or minimal content.
+**Step 8: Commit**
+```bash
+git add include/apex/apex.h include/apex/ast_man.h src/ast_man.c src/apex.c cli/main.c CMakeLists.txt
+git commit -m "feat: add man and man-html output format enum and stub renderers"
+```
+---
+## Task 2: Implement roff serializer (ast_man.c roff path)
+**Files:**
+- Modify: `src/ast_man.c`
+**Step 1: Implement buffer and escape helpers**
+Add a simple growable buffer (similar to `ast_terminal.c`'s `terminal_buffer`) and a function to escape roff special characters (backslash, `\.` for literal dot at start of line). Expose a function that appends roff-safe text (escape `\`, `'`, and leading `.`).
+**Step 2: Emit .TH from metadata or defaults**
+Parse document metadata (or options) for `man_title`, `man_section`, `man_date`, `man_source` (e.g. from YAML front matter). If missing, use: title from first H1 or "Document", section "1", date from build or "1 January 1970", source "". Emit `.TH title section date source` at the start.
+**Step 3: Block-level roff: headings, paragraphs, lists, code blocks**
+Walk document children. For each node type:
+- `CMARK_NODE_HEADING`: emit `.SH` with heading text (level 1 = `.SH`, deeper levels can be `.SS` or `.SH` with combined text).
+- `CMARK_NODE_PARAGRAPH`: emit paragraph text with `.PP` or `.IP` as needed; inline formatting (bold/italic/code) via `.B`, `.I`, `.BR`, `.IR` etc.
+- `CMARK_NODE_LIST`: bullet list with `.IP \\(bu` or `.IP \\fB*\\fP`; ordered list with `.IP` and number.
+- `CMARK_NODE_CODE_BLOCK`: `.PP` then verbatim block (escape and use `.nf`/`.fi` or inline with escaped newlines).
+- `CMARK_NODE_BLOCK_QUOTE`: indent with `.RS`/`.RE` and recurse.
+- `CMARK_NODE_TABLE`: optional; simple table as roff `TS`/`TE` or as list of rows.
+**Step 4: Inline formatting**
+For inline nodes: strong -> `.B`, em -> `.I`, code -> `\\fB`/`\\fP` or similar; links can be `.I text` and optional URL in parens. Escape special chars in all text.
+**Step 5: Verify with man(1)**
+Run: `echo -e '# NAME\n\nfoo - short desc\n\n# SYNOPSIS\n\n**foo** [*options*]' | ./apex -t man > /tmp/foo.1 && man -l /tmp/foo.1`
+Expected: man viewer shows a minimal man page without errors.
+**Step 6: Commit**
+```bash
+git add src/ast_man.c
+git commit -m "feat: implement roff (man page) serializer in ast_man.c"
+```
+---
+## Task 3: Implement man-html serializer (styled HTML)
+**Files:**
+- Modify: `src/ast_man.c`
+**Step 1: Emit HTML document shell**
+In `apex_cmark_to_man_html()`, emit `<!DOCTYPE html>`, `<html>`, `<head>` with `<meta charset="utf-8">`, `<title>` from man_title (or first H1), and a `<style>` block. Reuse or mirror the structure of [md2man EXAMPLE](https://sunaku.github.io/md2man/man/man0/EXAMPLE.html): section headings, monospace for options/code, list styling.
+**Step 2: Add minimal CSS for man-page look**
+Include inline CSS that styles: `.man-title`, `.man-section` (e.g. NAME, SYNOPSIS, OPTIONS), `.man-option` (bold/italic for option names and args), code blocks and lists so the result is readable and similar to the md2man example.
+**Step 3: Walk AST and emit HTML sections**
+Map block types to semantic HTML: H1 -> name/title; H2/H3 -> section headers (e.g. `<h2 id="name">NAME</h2>`); paragraphs, lists, code blocks -> `<p>`, `<ul>`/`<ol>`, `<pre><code>`; inline bold/italic/code -> `<strong>`, `<em>`, `<code>`. Escape HTML entities in text. Links: `<a href="...">`.
+**Step 4: Sanity check**
+Run: `echo -e '# NAME\n\nfoo - short desc\n\n# SYNOPSIS\n\n**foo** [*options*]' | ./apex -t man-html > /tmp/foo.html && open /tmp/foo.html` (or open in browser).
+Expected: styled page with NAME and SYNOPSIS sections, no raw AST or crashes.
+**Step 5: Commit**
+```bash
+git add src/ast_man.c
+git commit -m "feat: implement styled man-html output in ast_man.c"
+```
+---
+## Task 4: Document metadata for man pages
+**Files:**
+- Modify: `src/ast_man.c` (read metadata from document or options)
+- Modify: `src/extensions/metadata.c` or document in README how metadata keys map to man header
+**Step 1: Define metadata keys**
+Support in document metadata (YAML front matter or metadata block): `man_title` (or `title`), `man_section` (default "1"), `man_date` (default or from doc), `man_source` (optional). In `apex_cmark_to_man_roff` and `apex_cmark_to_man_html`, obtain these from the document's metadata table if available (see how apex exposes metadata to renderers), otherwise fall back to first H1 and defaults.
+**Step 2: Use metadata in both roff and HTML**
+Ensure `.TH` and the HTML `<title>` / header use the same metadata source. If apex does not yet pass metadata to custom renderers, add an optional `apex_metadata_item*` or similar to the options struct for this pipeline, or read from the document's first-child metadata node if the AST carries it.
+**Step 3: Commit**
+```bash
+git add src/ast_man.c [and any metadata changes]
+git commit -m "feat: man output uses document metadata for title, section, date"
+```
+---
+## Task 5: Tests and documentation
+**Files:**
+- Create or modify: `tests/` (existing test runner if any; or add a small script that runs `apex -t man` and `apex -t man-html` on a fixture and checks for expected substrings)
+- Modify: `README.md` and `man/apex.1.md` to document `-t man` and `-t man-html`
+**Step 1: Add fixture and test script**
+Create `tests/fixtures/man-basic.md` with a minimal man-style doc (NAME, SYNOPSIS, one option). Create or extend a test script that: runs `apex -t man -s < tests/fixtures/man-basic.md`, greps for `.TH` and `.SH`; runs `apex -t man-html -s < tests/fixtures/man-basic.md`, greps for `<!DOCTYPE html>` and `<h2` or "NAME". Exit 0 if found, 1 otherwise.
+**Step 2: Run tests**
+Run the test script; fix any failures.
+**Step 3: Update README and man page**
+In README, add `man` and `man-html` to the list of output formats and mention that document metadata can set man page title/section/date. In `man/apex.1.md`, add **man** and **man-html** to the OPTIONS description of `-t, --to FORMAT`.
+**Step 4: Commit**
+```bash
+git add tests/ README.md man/apex.1.md
+git commit -m "docs and tests: man and man-html output"
+```
+---
+## Definition-list option blocks (`: ` / `:: `) for man output
+**Context:** Man pages often use “term + description” pairs (e.g. `**--option**` then `:: Description.`). The definition-list extension should create AST definition lists when the block parser sees a line starting with `: ` or `:: ` after a paragraph, so that man output can render them as `.TP` (term) + body.
+**Done so far:**
+1. **blocks.c (cmark):** In `check_open_blocks`, when the open block is a paragraph and the line starts with `: ` or `:: ` (optional second `:`, then space/tab), `goto done` so the paragraph does not match and the container backs up to the document; `open_new_blocks` then runs and the definition_list extension can open a block.
+2. **apex.c:** Definition-list preprocessing is skipped when `output_format` is `APEX_OUTPUT_MAN` or `APEX_OUTPUT_MAN_HTML` so the block parser sees raw `: ` / `:: `.
+3. **definition_list.c (extension):**
+   - **Segfault fix:** No longer unlink the previous paragraph in `open_block`; only insert the def_list before it. The parser’s finalize path removes the paragraph; unlinking caused use-after-free.
+   - **can_contain:** def_list and term created with `cmark_node_new_with_mem_and_ext` so `cmark_node_can_contain_type(def_list, PARAGRAPH)` allows the description paragraph under the list.
+   - **Term content:** Create a new paragraph under the term and copy `prev->content` into it (with `prev->start_line` / `start_column`), ensure trailing newline; prefer `parser->current` as `prev` when it’s the interrupted paragraph.
+   - **Postprocess:** If the def_list’s first child is a DEFINITION_TERM with a paragraph that has no “real” content (only softbreak/linebreak), and the next sibling of the def_list is a paragraph with inlines, move that paragraph’s inlines into the term’s paragraph and remove the sibling.
+**Current behaviour:** For input `**--foo**` then `:: Enable foo.`:
+- **HTML:** `<dt></dt>`, `<dd><p>Enable foo.</p></dd>`, then `<p><strong>–foo</strong></p>` (term empty; “term” appears as a following paragraph).
+- **Man:** `.TP` then “Enable foo.”, then `.PP` and bold `--foo` (same structure).
+So the term paragraph stays empty (copy from `prev` isn’t populating it and/or the postprocess move isn’t applied).
+---
+### Debug steps (definition-list term empty)
+Use these to see why the term has no content and why the postprocess fix may not run.
+**1. Confirm `prev` and its content in `open_block`**
+In `src/extensions/definition_list.c`, in `open_block`, immediately after resolving `prev` (and before creating `def_list`), add:
+```c
+/* DEBUG: remove after finding why term is empty */
+if (getenv("APEX_DEBUG_DEFLIST")) {
+    fprintf(stderr, "[deflist open_block] prev=%p type=%u content.size=%zu parser->current=%p\n",
+            (void*)prev, (unsigned)cmark_node_get_type(prev),
+            (size_t)prev->content.size, (void*)parser->current);
+    if (prev->content.size > 0 && prev->content.size <= 200)
+        fprintf(stderr, "[deflist open_block] content: %.*s\n", (int)prev->content.size, prev->content.ptr);
+}
+```
+Run (env var must apply to **apex**, not the left side of the pipe; debug is also appended to `/tmp/apex_deflist_debug.log`):
+```bash
+printf '%s\n' '**--foo**' ':: Enable foo.' | ./build/apex -t html -s 2>&1 | head -5
+printf '%s\n' '**--foo**' ':: Enable foo.' | APEX_DEBUG_DEFLIST=1 ./build/apex -t html -s 2>&1 | head -20
+# or: cat /tmp/apex_deflist_debug.log
+```
+- If `content.size == 0`, the paragraph has no content when we copy; trace back to when that paragraph receives the first line (e.g. whether `add_line` is called for it before we run).
+- If `content.size > 0`, the copy path is correct; then either the copied buffer isn’t used by the inline parser (e.g. wrong node or buffer cleared later) or the postprocess is supposed to fix it but doesn’t.
+**2. Confirm postprocess sees the right structure**
+In `postprocess`, at the start of the `while (cur)` loop, when `cur` is a DEFINITION_LIST, add:
+```c
+if (getenv("APEX_DEBUG_DEFLIST")) {
+    cmark_node *term = cmark_node_first_child(cur);
+    cmark_node *term_para = term ? cmark_node_first_child(term) : NULL;
+    cmark_node *next_sib = cmark_node_next(cur);
+    fprintf(stderr, "[deflist postprocess] def_list=%p term=%p term_para=%p term_para.first_child=%p next_sib=%p next_sib.type=%u next_sib.first_child=%p\n",
+            (void*)cur, (void*)term, (void*)term_para,
+            term_para ? (void*)cmark_node_first_child(term_para) : NULL,
+            (void*)next_sib,
+            next_sib ? (unsigned)cmark_node_get_type(next_sib) : 0,
+            next_sib ? (void*)cmark_node_first_child(next_sib) : NULL);
+}
+```
+Run with the same `APEX_DEBUG_DEFLIST=1` command.
+- If `next_sib` is NULL, the document order is not `[def_list, paragraph]` (e.g. paragraph before def_list or root is not the document).
+- If `next_sib.first_child` is NULL, the following paragraph has no inlines when postprocess runs (e.g. postprocess runs before `process_inlines` in the pipeline—unlikely—or that node isn’t the one that got the text).
+- If `term_para.first_child` is non-NULL and not just softbreak/linebreak, the term was already filled (e.g. copy worked) and we intentionally don’t move; then the bug is in rendering (e.g. HTML/man not reading that node).
+**3. Confirm root and document order**
+In `postprocess`, before the `while (cur)` loop:
+```c
+if (getenv("APEX_DEBUG_DEFLIST")) {
+    fprintf(stderr, "[deflist postprocess] root=%p root.type=%u first_child=%p\n",
+            (void*)root, (unsigned)cmark_node_get_type(root), (void*)cmark_node_first_child(root));
+    for (cmark_node *n = cmark_node_first_child(root); n; n = cmark_node_next(n))
+        fprintf(stderr, "  block=%p type=%u\n", (void*)n, (unsigned)cmark_node_get_type(n));
+}
+```
+This verifies that root is the document and whether the first block is the def_list and the second the paragraph.
+**4. Optional: assert in open_block**
+After copying into `term_para`, assert that something was copied (only for local debugging):
+```c
+assert(term_para->content.size > 0 && "def_list term: prev had no content to copy");
+```
+If this fires, the fix is to ensure the interrupted paragraph’s content is available (e.g. use a different source than `prev->content` or fill the term in a later pass).
+**5. Next steps depending on outcome**
+- **prev->content.size == 0 in open_block:** Inspect when the first line is added to the paragraph (e.g. in `add_text_to_container` / `add_line`) and whether `open_block` is invoked before that for the second line. If the parser finalizes or clears the paragraph before we run, consider filling the term in postprocess only (e.g. always move the following paragraph’s inlines into the term when the term is empty), and ensure postprocess runs after `process_inlines`.
+- **postprocess next_sib NULL or wrong order:** Handle “paragraph then def_list” or recurse into a wrapper root so we always see def_list and its next sibling.
+- **postprocess runs but term_para already has content:** Copy is working; then inspect HTML/man renderer for DEFINITION_TERM (e.g. whether they descend into the term’s paragraph and render inlines).
+Remove all `APEX_DEBUG_DEFLIST` branches and asserts before committing.
+---
+## Execution Handoff
+Plan complete and saved to `docs/plans/2025-02-27-man-page-output-implementation.md`.
+**Two execution options:**
+1. **Subagent-Driven (this session)** – I dispatch a fresh subagent per task, review between tasks, fast iteration.
+2. **Parallel Session (separate)** – Open a new session with executing-plans and run through the plan task-by-task with checkpoints.
+Which approach do you want?