markdown_composer 0.7.0

data/docs/compose/md_composer_compose_include.md

@@ -0,0 +1,136 @@
+---
+title: Markdown Composer Compose Include
+type: guide
+status: current
+updated: 2026-06-01
+description: Public guide to Include syntax and examples in Markdown Composer compose rows.
+---
+
+# Markdown Composer Compose Include
+
+### References
+
+| What | Where |
+|------|-------|
+| Compose anatomy | [./md_composer_compose_anatomy.md](./md_composer_compose_anatomy.md) |
+| Select | [./md_composer_compose_select.md](./md_composer_compose_select.md) |
+| Unit tokens | [../reference/md_composer_unit_tokens.md](../reference/md_composer_unit_tokens.md) |
+| Take modifiers | [../reference/md_composer_take.md](../reference/md_composer_take.md) |
+| Where conditions | [../reference/md_composer_where.md](../reference/md_composer_where.md) |
+| Nested syntax | [../reference/md_composer_nested.md](../reference/md_composer_nested.md) |
+| Structured data example | [../examples/md_composer_example_structured_data.md](../examples/md_composer_example_structured_data.md) |
+
+---
+
+## 1. Purpose
+
+`include` chooses what to keep from each selected unit.
+
+```ruby
+{
+  "select" => "heading_2_section",
+  "include" => "heading_title, paragraph[first:1]"
+}
+```
+
+If omitted, `include` defaults to `all`.
+
+## 2. Syntax Anatomy
+
+`include` is an include expression. It runs inside each selected unit.
+
+```text
+include = all | content | include_list
+
+include_list = include_item, include_item, ...
+
+include_item = include_unit [take] [where condition] [except exclude_unit]
+             | data_path("path.expression")
+             | nested_include
+
+nested_include = unit_token { include_item; include_item; ... }
+```
+
+For every include-capable token and alias, see [Unit Tokens](../reference/md_composer_unit_tokens.md).
+For shared bracketed take syntax inside include items, see [Take Modifiers](../reference/md_composer_take.md).
+For shared `where` syntax inside include items, see [Where Conditions](../reference/md_composer_where.md).
+For nested include projection, recursion, and separators, see [Nested Syntax](../reference/md_composer_nested.md).
+
+Include parts:
+
+| Part | Example | Meaning |
+|------|---------|---------|
+| `all` | `all` | Keep the whole selected unit. |
+| `content` | `content` | Keep the selected unit body without its heading/title wrapper. |
+| `include_unit` | `paragraph`, `heading_title`, `code_block` | Keep only matching child units. |
+| `[take]` | `paragraph[first:2]` | Keep positions inside each selected unit. |
+| `where condition` | `paragraph where text:contains("Note")` | Filter child units. |
+| `except` | `paragraph except link` | Keep the include unit while excluding nested matches. |
+| `data_path(...)` | `data_path("metadata.title")` | Extract structured YAML/JSON values from `data_block`. |
+
+Examples:
+
+| Goal | Include |
+|------|---------|
+| Whole selected unit | `all` |
+| Body without section heading | `content` |
+| Only heading title | `heading_title` |
+| First paragraph inside each selected section | `paragraph[first:1]` |
+| Keep paragraphs except links | `paragraph except link` |
+| Extract data value | `data_path("title")` |
+
+## 3. Include Runs Per Selected Unit
+
+This row selects two H2 sections and keeps the first paragraph from each section:
+
+```ruby
+{
+  "select" => "heading_2_section[position:1,2]",
+  "include" => "paragraph[first:1]",
+  "action" => "set"
+}
+```
+
+It does not keep the first paragraph across the whole document. It keeps the first paragraph inside each selected section.
+
+## 4. Nested Include
+
+Nested include is useful when a selection should keep a specific shape.
+
+```ruby
+{
+  "select" => "heading_2_section { heading_title; paragraph[first:1] }",
+  "action" => "set"
+}
+```
+
+The nested include after `{ ... }` is part of the selector string and is normalized as include rules for each selected section.
+
+## 5. Data Path Include
+
+`data_path(...)` is only valid when the selected unit is `data_block`.
+
+```ruby
+{
+  "select" => "data_block where format:equals(\"yaml\")",
+  "include" => "data_path(\"metadata.title\")",
+  "action" => "set"
+}
+```
+
+`data_path(...)` creates derived nodes:
+
+| Derived type | Produced when |
+|--------------|---------------|
+| `data_record` | The extracted value is an object/hash. |
+| `data_value` | The extracted value is a scalar or array value. |
+
+These derived node types matter for metadata and future structured-output targeting. In normal Markdown output, they are serialized back into Markdown text.
+
+## 6. Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| Using `data_path(...)` outside `data_block`. | Select `data_block` first. |
+| Expecting `include` to choose source sections. | Use `select` for source units; use `include` for parts inside those units. |
+| Forgetting `heading_title` when rebuilding summaries. | Include it explicitly with body parts. |

data/docs/compose/md_composer_compose_select.md

@@ -0,0 +1,198 @@
+---
+title: Markdown Composer Compose Select
+type: guide
+status: current
+updated: 2026-06-01
+description: Public guide to Select syntax, unit tokens, take operators, and where conditions.
+---
+
+# Markdown Composer Compose Select
+
+### References
+
+| What | Where |
+|------|-------|
+| Compose anatomy | [./md_composer_compose_anatomy.md](./md_composer_compose_anatomy.md) |
+| Include | [./md_composer_compose_include.md](./md_composer_compose_include.md) |
+| Targets | [./md_composer_compose_targets.md](./md_composer_compose_targets.md) |
+| Unit tokens | [../reference/md_composer_unit_tokens.md](../reference/md_composer_unit_tokens.md) |
+| Take modifiers | [../reference/md_composer_take.md](../reference/md_composer_take.md) |
+| Where conditions | [../reference/md_composer_where.md](../reference/md_composer_where.md) |
+| Nested syntax | [../reference/md_composer_nested.md](../reference/md_composer_nested.md) |
+
+---
+
+## 1. Purpose
+
+`select` chooses source units before `include` decides what to keep from each unit.
+
+```ruby
+{ "select" => "heading_2_section[first:1]" }
+```
+
+If omitted, `select` defaults to `all`.
+
+## 2. Syntax Anatomy
+
+`select` is a selector expression. It can be plain, filtered, narrowed by take, or paired with a nested include block.
+
+Readable shape:
+
+```text
+heading_2_section[position:2,3] where title:contains("API") { heading_title; paragraph[first:1] }
+|                |             |     |                     | |                          |
+|                |             |     |                     | └── nested include rules
+|                |             |     |                     └──── include block for each selected unit
+|                |             |     └────────────────────────── where condition
+|                |             └──────────────────────────────── take modifier
+└─────────────────────────────────────────────────────────────── unit token
+```
+
+Read it as:
+
+```text
+Select H2 sections, keep positions 2 and 3 after filtering, require the title to contain "API", then keep only the heading title and first paragraph from each selected section.
+```
+
+Simple selectors can stop at any earlier part:
+
+```text
+paragraph
+|       |
+└───────└── unit token only
+
+paragraph[first:2]
+|        |       |
+|        └───────└── take modifier
+└────────────────── unit token
+
+paragraph where text:contains("warning")
+|         |     |
+|         |     └── where condition
+|         └──────── where keyword
+└────────────────── unit token
+```
+
+### 2.1. Formal Grammar
+
+```text
+select = all | selector
+
+selector = unit_token [take] [where condition] [nested_include]
+
+unit_token = heading_2_section | paragraph | table | code_block | ...
+
+take = [first:n] | [last:n] | [position:n,n] | [range:n..n] | [odd] | [even] | [every:n]
+
+nested_include = { include_item; include_item; ... }
+```
+
+For every supported unit token and alias, see [Unit Tokens](../reference/md_composer_unit_tokens.md).
+For every take operator, see [Take Modifiers](../reference/md_composer_take.md).
+For every where field, predicate, and group, see [Where Conditions](../reference/md_composer_where.md).
+For nested include projection, see [Nested Syntax](../reference/md_composer_nested.md).
+
+The order matters:
+
+```text
+unit token first, take second, where third, nested include last
+```
+
+### 2.2. Selector Parts
+
+| Part | Example | Purpose |
+|------|---------|---------|
+| [`unit_token`](../reference/md_composer_unit_tokens.md) | `heading_2_section` | Chooses the kind of source unit. |
+| [`[take]`](../reference/md_composer_take.md) | `[position:2,3]` | Narrows by position after unit matching. |
+| [`where condition`](../reference/md_composer_where.md) | `where title:contains("API")` | Filters matches by metadata or text. |
+| [`{ nested include }`](../reference/md_composer_nested.md) | `{ heading_title; paragraph[first:1] }` | Keeps a shaped subset from each selected unit. |
+
+### 2.3. Examples
+
+| Goal | Select | Anatomy used |
+|------|--------|--------------|
+| All content | `all` | special selector |
+| All paragraphs | `paragraph` | unit token |
+| First H2 section | `heading_2_section[first:1]` | unit token + take |
+| H2 section with title | `heading_2_section where title:equals("API")` | unit token + where |
+| Paragraphs containing text | `paragraph where text:contains("warning")` | unit token + where |
+| H2 sections 2 through 4 | `heading_2_section[range:2..4]` | unit token + take |
+| H2 sections with nested include | `heading_2_section { heading_title; paragraph[first:1] }` | unit token + nested include |
+| H2 sections with take, where, and include | `heading_2_section[position:2,3] where title:contains("API") { heading_title; paragraph[first:1] }` | full selector |
+
+## 3. Section Versus Node Tokens
+
+| Token | Selects |
+|-------|---------|
+| `heading_2` / `h2` | Only H2 heading nodes. |
+| `heading_2_section` / `h2_section` | H2 heading plus content until the next H2 or higher heading. |
+| `paragraph` / `p` | Paragraph nodes. |
+| `section` | Heading-defined sections at any level. |
+| `heading` | Heading nodes at any level. |
+
+If you want a heading and its body, use a section token.
+
+## 4. Take Operators
+
+Take operators live inside brackets.
+
+For full syntax anatomy, advanced operators, target/scope take, and field-take, see [Take Modifiers](../reference/md_composer_take.md).
+
+| Goal | Select |
+|------|--------|
+| First two matches | `paragraph[first:2]` |
+| Last match | `heading_2_section[last:1]` |
+| Positions | `heading_2_section[position:2,3]` |
+| Range | `heading_2_section[range:2..4]` |
+| Odd positions | `list_item[odd]` |
+| Every third paragraph | `paragraph[every:3]` |
+| Exclude positions | `heading_2_section[except:1,3]` |
+
+Positions are 1-based. Negative positions count from the end in structured take values.
+
+## 5. Where Conditions
+
+Where conditions are shared selector syntax. They work in `select`, `include`, `target`, and transform `scope`.
+
+In `select`, where conditions filter matches after the unit token is selected and before take is applied.
+
+```ruby
+"heading_2_section where title:contains(\"Install\")"
+```
+
+For the full syntax anatomy, field list, predicates, groups, field-take, and diagnostics, see [Where Conditions](../reference/md_composer_where.md).
+
+Common fields:
+
+| Field | Example |
+|-------|---------|
+| `title` | `heading_2_section where title:equals("API")` |
+| `text` | `paragraph where text:contains("warning")` |
+| `position` | `section where position:range(2..4)` |
+| `language` | `code_block where language:equals("ruby")` |
+| `links` | `section where links:exists` |
+| `empty` | `paragraph where empty:equals(false)` |
+
+Groups combine conditions:
+
+```ruby
+"heading_2_section where any(title:contains(\"API\"); title:contains(\"CLI\"))"
+```
+
+## 6. Data Blocks
+
+Markdown Composer can select structured YAML or JSON blocks:
+
+```ruby
+"data_block where format:equals(\"json\")"
+```
+
+Use `include: 'data_path("...")'` to extract values from selected data blocks. `data_record` and `data_value` are produced by `data_path(...)`; they are not normal `select` tokens.
+
+## 7. Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| Using `h2` when you want body content. | Use `h2_section`. |
+| Putting `where` before take. | Use `unit[take] where condition`. |
+| Treating `data_value` as a normal selector. | Select `data_block`, then include `data_path(...)`. |

data/docs/compose/md_composer_compose_sources.md

@@ -0,0 +1,161 @@
+---
+title: Markdown Composer Compose Sources
+type: guide
+status: current
+updated: 2026-06-01
+description: Public guide to Source values in Markdown Composer compose rows.
+---
+
+# Markdown Composer Compose Sources
+
+### References
+
+| What | Where |
+|------|-------|
+| Compose anatomy | [./md_composer_compose_anatomy.md](./md_composer_compose_anatomy.md) |
+| Source examples | [../examples/md_composer_example_multi_row_compose.md](../examples/md_composer_example_multi_row_compose.md) |
+| Capabilities | [../reference/md_composer_reference_capabilities.md](../reference/md_composer_reference_capabilities.md) |
+
+---
+
+## 1. Purpose
+
+`source` tells a compose row where to read content from.
+
+```ruby
+{ "source" => "current", "select" => "heading_2_section[first:1]" }
+```
+
+If omitted, `source` defaults to `current`.
+
+## 2. Syntax Anatomy
+
+`source` can be omitted, a source token string, a structured source reference, or an inline source object.
+
+```text
+source = omitted
+       | source_token
+       | source_reference
+       | inline_source
+
+source_token = current | explicit | inherited | previous | buffer | inline
+
+source_reference = {
+  type: source_token,
+  key: source_key
+}
+
+inline_source = {
+  type: inline,
+  key: source_key,
+  markdown: markdown_text
+}
+```
+
+Common forms:
+
+| Form | Example | Meaning |
+|------|---------|---------|
+| omitted | no `source` field | Use `current`. |
+| string token | `"source" => "buffer"` | Use a named source behavior. |
+| structured reference | `"source" => { "type" => "explicit", "key" => "guide" }` | Resolve a keyed source. |
+| inline object | `"source" => { "type" => "inline", "markdown" => "..." }` | Embed small source content in the plan. |
+
+## 3. Source Tokens
+
+| Source | Meaning | When to use |
+|--------|---------|-------------|
+| `current` | The main source supplied by the caller. | Most one-document recipes. |
+| `explicit` | A keyed source selected by the plan or caller. | Multi-source composition. |
+| `inherited` | A caller-resolved inherited source. | Applications that provide inheritance. |
+| `previous` | The effective source from the previous row. | Repeating the same source without restating it. |
+| `buffer` | The current composition buffer. | Moving, modifying, removing, or transforming content already composed. |
+| `inline` | Content embedded in the config. | Small generated snippets or tests. |
+
+## 4. Current Source
+
+```ruby
+sources = [
+  { "key" => "current", "type" => "current", "markdown" => "# Guide\n\n## Intro\n\nHello.\n" }
+]
+
+config = {
+  "compose" => [
+    { "source" => "current", "select" => "heading_2_section[first:1]", "action" => "set" }
+  ]
+}
+```
+
+If there is exactly one current source, you can omit `source`.
+
+## 5. Explicit Source
+
+Use explicit source references when a plan reads from more than one input.
+
+```ruby
+sources = [
+  { "key" => "guide", "type" => "explicit", "markdown" => "# Guide\n\n## API\n\nUse it.\n" },
+  { "key" => "faq", "type" => "explicit", "markdown" => "# FAQ\n\n## Install\n\nBundle it.\n" }
+]
+
+config = {
+  "compose" => [
+    { "source" => { "type" => "explicit", "key" => "guide" }, "select" => "heading_2_section", "action" => "set" },
+    { "source" => { "type" => "explicit", "key" => "faq" }, "select" => "heading_2_section", "action" => "append" }
+  ]
+}
+```
+
+## 6. Previous Source
+
+`previous` uses the effective source reference from the prior row. On the first row, it falls back to `current`.
+
+```ruby
+{
+  "compose" => [
+    { "source" => "current", "select" => "heading_2_section[first:1]", "action" => "set" },
+    { "source" => "previous", "select" => "heading_2_section[last:1]", "action" => "append" }
+  ]
+}
+```
+
+## 7. Buffer Source
+
+Use `buffer` when the row should read from composed output rather than the original source.
+
+```ruby
+{
+  "source" => "buffer",
+  "select" => "heading_2_section where title:contains(\"Draft\")",
+  "action" => "remove_buffer_target",
+  "target" => "heading_2_section where title:contains(\"Draft\")"
+}
+```
+
+`buffer` is only useful after a previous row has created buffer content, unless the caller passes an initial buffer option.
+
+## 8. Inline Source
+
+Inline source embeds content directly in the plan.
+
+```ruby
+{
+  "source" => {
+    "type" => "inline",
+    "key" => "notice",
+    "markdown" => "## Notice\n\nGenerated content.\n"
+  },
+  "select" => "heading_2_section",
+  "action" => "append"
+}
+```
+
+Use inline sources for small generated snippets, not large documents.
+
+## 9. Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| Using `buffer` in the first row. | Use `current`, or provide `initial_buffer`. |
+| Using `previous` when the prior row used `buffer`. | Be explicit if the distinction matters. |
+| Forgetting a key for multi-source explicit plans. | Pass `key`, `id`, or `slug` in the source reference. |

data/docs/compose/md_composer_compose_targets.md

@@ -0,0 +1,194 @@
+---
+title: Markdown Composer Compose Targets
+type: guide
+status: current
+updated: 2026-06-01
+description: Public guide to target syntax, placement, selector targets, and structured-output target caveats.
+---
+
+# Markdown Composer Compose Targets
+
+### References
+
+| What | Where |
+|------|-------|
+| Compose anatomy | [./md_composer_compose_anatomy.md](./md_composer_compose_anatomy.md) |
+| Actions | [./md_composer_compose_actions.md](./md_composer_compose_actions.md) |
+| Buffer | [./md_composer_compose_buffer.md](./md_composer_compose_buffer.md) |
+| Unit tokens | [../reference/md_composer_unit_tokens.md](../reference/md_composer_unit_tokens.md) |
+| Take modifiers | [../reference/md_composer_take.md](../reference/md_composer_take.md) |
+| Where conditions | [../reference/md_composer_where.md](../reference/md_composer_where.md) |
+
+---
+
+## 1. Purpose
+
+`target` addresses the current composition buffer. It does not address the original source unless the row explicitly uses `source: "buffer"`.
+
+```ruby
+{
+  "action" => "replace",
+  "target" => "heading_2_section where title:equals(\"Old\")"
+}
+```
+
+## 2. Syntax Anatomy
+
+`target` has three layers: a special buffer position, a selector that matches the current buffer, or a placement expression built from selectors.
+
+```text
+target = special_target | target_selector | placement_target
+
+special_target = whole output | output | start | end | in_place
+
+target_selector = unit_token [take] [where condition]
+
+placement_target = before target_selector
+                 | after target_selector
+                 | between target_selector and target_selector
+```
+
+For every target-capable token and alias, see [Unit Tokens](../reference/md_composer_unit_tokens.md).
+For shared target-selector take syntax, see [Take Modifiers](../reference/md_composer_take.md).
+For shared target-selector `where` syntax, see [Where Conditions](../reference/md_composer_where.md).
+
+| Layer | Example | Meaning |
+|-------|---------|---------|
+| `special_target` | `end` | A built-in buffer location or whole-buffer operation. |
+| `target_selector` | `heading_2_section[first:1]` | A selector matched against the current composition buffer. |
+| `placement_target` | `before heading_2 where text:contains("API")` | A placement instruction anchored to one or two buffer selectors. |
+
+Target selector anatomy:
+
+```text
+unit_token[take] where condition
+```
+
+| Part | Example | Notes |
+|------|---------|-------|
+| `unit_token` | `heading_2_section`, `paragraph`, `table` | Uses the same token registry as `select` and transform `scope`. |
+| `[take]` | `[first:1]`, `[position:2,3]`, `[range:2..4]` | Optional. Positions are 1-based. |
+| `where condition` | `where title:equals("API")` | Optional. Filters buffer matches. |
+
+Common target forms:
+
+| Form | Example | Common actions |
+|------|---------|----------------|
+| `whole output` | `target: "whole output"` | `set`, `replace`, `modify` |
+| `start` | `target: "start"` | `copy`, `move`, `modify` |
+| `end` | `target: "end"` | `copy`, `move`, `modify` |
+| selector | `target: "heading_2_section[first:1]"` | `replace`, `remove_buffer_target`, `transform_buffer_target` |
+| before | `target: "before heading_2 where text:contains(\"API\")"` | `insert_before`, `copy`, `move`, `modify` |
+| after | `target: "after heading_2 where text:contains(\"API\")"` | `insert_after`, `copy`, `move`, `modify` |
+| between | `target: "between heading_2[first:1] and heading_2[last:1]"` | `insert_between`, `modify` |
+| `in_place` | `target: "in_place"` | `modify` with `source: "buffer"` only |
+
+## 3. Selector Targets
+
+Selector targets reuse select syntax, but they match against the current buffer.
+
+```ruby
+{
+  "action" => "replace",
+  "target" => "paragraph where text:contains(\"TODO\")"
+}
+```
+
+The target can use take and where:
+
+```ruby
+"heading_2_section[position:2] where title:contains(\"Draft\")"
+```
+
+That means this row replaces the first matching section already in the buffer, not the first matching section in the original source:
+
+```ruby
+{
+  "select" => "heading_2_section where title:equals(\"New API\")",
+  "action" => "replace",
+  "target" => "heading_2_section where title:equals(\"Old API\")"
+}
+```
+
+## 4. Between Targets
+
+Between targets need a start anchor and an end anchor.
+
+```ruby
+{
+  "action" => "insert_between",
+  "target" => "between heading_2 where text:contains(\"Start\") and heading_2 where text:contains(\"End\")"
+}
+```
+
+The start anchor must resolve before the end anchor. Otherwise validation/execution reports `target.order_invalid`.
+
+## 5. In Place Target
+
+`in_place` rewrites selected buffer content inline.
+
+```ruby
+{
+  "source" => "buffer",
+  "select" => "paragraph where text:contains(\"Draft\")",
+  "action" => "modify",
+  "target" => "in_place",
+  "transforms" => [
+    { "transform" => "replace_text", "mode" => "literal", "options" => "from:Draft; to:Final" }
+  ]
+}
+```
+
+Rules:
+
+| Rule | Meaning |
+|------|---------|
+| Source must be `buffer`. | The row must select from the existing buffer. |
+| Action must be `modify`. | Other actions reject `in_place`. |
+| Selected content must be line-addressable. | Markdown Composer maps transformed fragments back to original buffer line ranges. |
+
+## 6. Target-Supported Unit Tokens
+
+Capabilities mark some unit tokens as valid for target matching. The common practical target tokens are the same node/section tokens used by selection:
+
+| Token family | Example target |
+|--------------|----------------|
+| Sections | `heading_2_section where title:equals("API")` |
+| Headings | `heading_2 where text:contains("API")` |
+| Paragraphs | `paragraph where text:contains("TODO")` |
+| Lists | `list`, `list_item`, `ordered_list`, `unordered_list` |
+| Tables | `table`, `table_row`, `table_cell` |
+| Code and diagrams | `code_block`, `mermaid`, `math_block` |
+| Data blocks | `data_block where format:equals("yaml")` |
+
+`data_record` and `data_value` are different. Capabilities mark them as advanced targets and output-only scopes because they are derived structured-output node types produced by `data_path(...)`. In the standalone Markdown buffer, `data_path(...)` values are serialized back to Markdown text, and normal target matching reparses that Markdown. Treat `data_record` and `data_value` as metadata/deferred structured-output target tokens unless a host adapter preserves them as matchable buffer nodes.
+
+Practical structured-data recipe:
+
+```ruby
+{
+  "select" => "data_block where format:equals(\"yaml\")",
+  "include" => "data_path(\"fields[type=dropdown].label\")",
+  "action" => "set"
+}
+```
+
+Then target the serialized Markdown text if needed:
+
+```ruby
+{
+  "source" => "buffer",
+  "select" => "all",
+  "action" => "remove_buffer_target",
+  "target" => "paragraph where text:contains(\"Old label\")"
+}
+```
+
+## 7. Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| Expecting target to match the original source. | Target matches the current buffer. |
+| Using `data_record` directly as a normal target in standalone Markdown output. | Select `data_block`, include `data_path(...)`, then target the serialized output or use a host adapter that preserves structured nodes. |
+| Using `start` or `end` with `remove_buffer_target`. | Use a selector target. |
+| Using `in_place` without `source: "buffer"`. | Select from the buffer explicitly. |