markdown_composer 0.7.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 1efddd31cae13aa57937ff72966d3d45553318b7f6e723153b09f818dc7fb554
+  data.tar.gz: 67639e63dc038422d7226bf54b3535aad398049c3bd4f08076af7383fbf192b4
+SHA512:
+  metadata.gz: daaa025bb16b154e341938a4b357b85ab9dc4e23eba85a22d6f114996a3add3ee0bd855b307b9654a85b37df9e9b9609f17bd104ba56dceb4cf3fa25bc32f1c4
+  data.tar.gz: c37dc0643e032a97bd04d7b468941be08bc8d88ae0c348b45e0023f92406ce433a36a6dc7e03cc172fe6f3e27fad970e93862f40657e398a85efc56c0c709d0a

data/CHANGELOG.md

@@ -0,0 +1,23 @@
+# Changelog
+
+## 0.7.0 - 2026-05-14
+
+- Moved test fixtures into the gem so the public test suite is self-contained.
+- Added `bundle exec rake test` as the standard test entry point.
+- Added GitHub Actions CI for Ruby 3.1, 3.2, 3.3, and 3.4.
+- Updated public gem metadata for the standalone `SocIt2Em/markdown_composer` repository.
+- Raised declared Ruby support to `>= 3.1` so the public support claim matches the CI matrix.
+- Documented the validation boundary between the core gem and host frameworks such as Rails.
+- Documented regex policy expectations for hosts exposing Composer configs to untrusted users.
+- Added focused edge-case tests that raise gem library line coverage above 95%.
+
+## 0.6.5.1 - 2026-05-14
+
+- Added the standalone `markdown_composer` gem package.
+- Added public APIs for composition, normalization, validation, capabilities, YAML/JSON import/export, row import/export, and the Ruby plan DSL.
+- Added Markdown-first source indexing, selection, include, take, where, action, target, transform, data-path, and diagnostics support.
+- Added HTML output support and best-effort HTML input conversion.
+- Enabled regex matching, regex text replacement, deterministic random take, Mermaid/math/comment tokens, and HTML link attribute transforms.
+- Kept raw HTML, sanitise transforms, host adapter transforms, and target-order behavior behind host policy or roadmap boundaries.
+- Added fixture-backed UAT, release-readiness coverage, and coupling tests for app independence.
+- Kept public capability metadata host-generic so GUI builders do not depend on application-specific document models.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SocIt2Em
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,278 @@
+# Markdown Composer
+
+Markdown Composer is a headless Ruby gem for selecting, composing, validating, and transforming Markdown-first document fragments.
+
+It is framework-independent. It does not depend on Rails, an ORM, an authorization system, a template engine, a component framework, or a UI. Host applications pass plain Markdown/HTML sources into the gem and own source lookup, permissions, sanitization, asset handling, and presentation.
+
+## Requirements
+
+- Ruby `>= 3.1`
+- `commonmarker >= 0.23.10, < 3`
+- `nokogiri >= 1.13.10, < 2`
+
+Development dependencies are limited to test and benchmark tooling.
+
+## Installation
+
+From RubyGems:
+
+```ruby
+# Gemfile
+gem "markdown_composer"
+```
+
+Then run:
+
+```bash
+bundle install
+```
+
+For a standalone Ruby script:
+
+```bash
+gem install markdown_composer
+```
+
+From a Git repository before a RubyGems release:
+
+```ruby
+# Gemfile
+gem "markdown_composer", git: "https://github.com/SocIt2Em/markdown_composer.git"
+```
+
+If installing from source before a RubyGems release, point Bundler at the standalone repository:
+
+```ruby
+# Gemfile
+gem "markdown_composer", git: "https://github.com/SocIt2Em/markdown_composer.git"
+```
+
+## Quick Start
+
+```ruby
+require "markdown_composer"
+
+sources = MarkdownComposer.source_list do
+  current File.read("source_current.md")
+  explicit :guide, File.read("guide.md")
+end
+
+plan = MarkdownComposer.plan do
+  from :current
+  select "h2_section[first:1]"
+  include "all"
+  set
+
+  from({ type: "explicit", key: "guide" })
+  select "h2_section"
+  include "all"
+  append
+end
+
+result = MarkdownComposer.compose(sources: sources, plan: plan)
+
+puts result.markdown
+```
+
+`MarkdownComposer.compose` always receives source content through the `sources:` keyword. Plan-row `inline` sources are the exception: they embed small source content directly in the plan.
+
+## Sources
+
+Runtime sources are plain hashes or values produced by `MarkdownComposer.source_list`.
+
+```ruby
+sources = MarkdownComposer.source_list do
+  current current_markdown
+  explicit :guide, guide_markdown
+  explicit :faq, faq_markdown
+  inherited :site_intro, inherited_markdown
+end
+```
+
+This returns the same shape as:
+
+```ruby
+[
+  { "key" => "current", "type" => "current", "markdown" => current_markdown },
+  { "key" => "guide", "type" => "explicit", "markdown" => guide_markdown },
+  { "key" => "faq", "type" => "explicit", "markdown" => faq_markdown },
+  { "key" => "site_intro", "type" => "inherited", "markdown" => inherited_markdown }
+]
+```
+
+Supported runtime source types:
+
+| Type | Purpose |
+| --- | --- |
+| `current` | Main document supplied by the caller. |
+| `explicit` | Named source selected by key. |
+| `inherited` | Named source resolved by the host application. |
+
+Plan-only source references such as `previous`, `buffer`, and `inline` are not runtime source records.
+
+## Plans
+
+Plans can be authored with the Ruby DSL:
+
+```ruby
+plan = MarkdownComposer.plan do
+  output :markdown
+
+  from :current
+  select 'heading_2_section where none(title:equals("Table of Contents"))'
+  include "heading_title"
+  set
+
+  from :buffer
+  select 'heading_2 where text:starts_with("1.1")'
+  remove_buffer_target
+end
+```
+
+Structured config is also supported for storage, import, and export:
+
+```ruby
+config = {
+  "output" => "markdown",
+  "compose" => [
+    {
+      "source" => "current",
+      "select" => "h2_section[first:1]",
+      "include" => "all",
+      "action" => "set"
+    }
+  ]
+}
+
+result = MarkdownComposer.compose(sources: sources, config: config)
+```
+
+YAML and JSON helpers are available:
+
+```ruby
+plan = MarkdownComposer.parse_yaml(yaml)
+yaml = MarkdownComposer.to_yaml(plan)
+
+plan = MarkdownComposer.parse_json(json)
+json = MarkdownComposer.to_json(plan)
+```
+
+Row-style hashes are accepted for spreadsheet or GUI workflows:
+
+```ruby
+plan = MarkdownComposer.parse_rows([
+  { "Source" => "current", "Select" => "h2_section[first:1]", "Include" => "all", "Action" => "set" }
+])
+```
+
+## Public API
+
+Main entry points:
+
+```ruby
+MarkdownComposer.compose(sources:, config: nil, plan: nil, options: {})
+MarkdownComposer.validate(config:, sources: [], options: {})
+MarkdownComposer.normalise(config_or_rows)
+MarkdownComposer.normalize(config_or_rows)
+MarkdownComposer.capabilities(options: {})
+MarkdownComposer.plan(&block)
+MarkdownComposer.source_list(&block)
+MarkdownComposer.parse_yaml(yaml_string)
+MarkdownComposer.parse_json(json_string)
+MarkdownComposer.parse_rows(row_hashes)
+MarkdownComposer.to_yaml(plan)
+MarkdownComposer.to_json(plan)
+MarkdownComposer.to_rows(plan)
+```
+
+`validate` returns `valid`, the normalized plan, diagnostics, and errors. `capabilities` returns JSON-compatible metadata for building host UI controls.
+
+## Actions And Transforms
+
+Compose actions place selected content into the composition buffer:
+
+- `set`, `append`, `prepend`
+- `insert_before`, `insert_after`, `insert_between`
+- `replace`, `copy`, `move`
+- `modify`
+- `remove_buffer_target`, `transform_buffer_target`
+
+Transforms run after composition or inside `modify`/`transform_buffer_target` rows. Supported transform families include heading numbering, heading levels, text replacement, link handling, content insertion/removal, dedupe, ordering, sanitise, and adapter-owned transforms.
+
+Use `MarkdownComposer.capabilities` as the source of truth for enabled actions, tokens, transform modes, required options, aliases, support status, and policy gates.
+
+## Framework Integration
+
+Markdown Composer does not ship Rails, ViewComponent, React, Vue, or other framework-specific integrations.
+
+Host applications are responsible for:
+
+- resolving documents into plain Markdown/HTML source hashes;
+- enforcing authorization, tenant, visibility, and workflow rules;
+- preparing source Markdown when templates or variables are involved;
+- sanitizing output and resolving assets;
+- rendering previews and mapping diagnostics into host UI.
+
+Rails, Hotwire, React, Vue, CLIs, and scripts should wrap the same headless API.
+
+## Validation And Diagnostics
+
+Validation covers portable Composer configuration: source references, selectors, include rules, targets, actions, transforms, output formats, diagnostics, and policy-gated features.
+
+Example diagnostic:
+
+```ruby
+{
+  code: "transform.option_missing",
+  message: "Missing required option from",
+  path: "transform[0].options.from",
+  severity: :error,
+  details: nil
+}
+```
+
+Current diagnostic families include:
+
+- `action.*`
+- `json.*`
+- `output.*`
+- `source.*`
+- `take.*`
+- `target.*`
+- `token.*`
+- `transform.*`
+- `where.*`
+- `yaml.*`
+
+## Format Support
+
+Markdown is the source-fidelity format. HTML input is accepted through a best-effort HTML-to-Markdown path and may lose fidelity for structures that do not map cleanly to Markdown.
+
+| From | To | Status | Notes |
+| --- | --- | --- | --- |
+| Markdown | Markdown | Supported | Source-fidelity selection and transforms. |
+| Markdown | HTML | Supported | Rendered via CommonMarker when available, fallback renderer otherwise. |
+| HTML | Markdown | Best effort | Uses Nokogiri-backed conversion. |
+| HTML | HTML | Best effort | Converts through Markdown, then renders back to HTML. |
+
+Host-specific sanitization, custom attributes, and asset policies should live outside the gem.
+
+## Development
+
+Run the test suite:
+
+```bash
+bundle exec rake test
+```
+
+Build the gem package:
+
+```bash
+gem build markdown_composer.gemspec
+```
+
+The release test suite covers public helper APIs, diagnostics, transform capability coverage, Markdown/HTML conversion fixtures, syntax checks, and coupling guards.
+
+## License
+
+Markdown Composer is released under the MIT License. See `LICENSE.txt`.

data/ROADMAP.md

@@ -0,0 +1,80 @@
+# Roadmap
+
+This roadmap tracks planned improvements for Markdown Composer after the `0.7.0` standalone gem release. It is intentionally focused on the gem's portable contract: syntax, normalization, validation, diagnostics, composition, and framework-independent helper APIs.
+
+## 1. Canonical Field Syntax Printer
+
+**Status:** Planned
+
+Add a helper that converts normalized selector/config hashes back into canonical field syntax strings.
+
+Example target API:
+
+```ruby
+MarkdownComposer.to_field_syntax(selector_hash)
+# => 'heading_2_section where none(title:equals("Table of Contents"))'
+```
+
+This belongs in the gem because the gem owns the field syntax contract.
+
+Expected behavior:
+
+- Produce deterministic, valid field syntax from normalized hashes.
+- Prefer canonical formatting over preserving original spacing or quote style.
+- Support imports, YAML-to-field workflows, examples, migrations, and normalized presets.
+- Include focused tests for selectors, `where` clauses, predicates, grouping, and invalid input diagnostics.
+
+Non-goal:
+
+- Restoring the exact user-entered text after normalization. Typos, spacing, quote choices, aliases, and partially invalid syntax cannot be reconstructed from a normalized hash.
+
+## 2. Extension Registry API
+
+**Status:** Planned
+
+Add a supported extension API for adding Composer vocabulary and behavior without changing the core row syntax.
+
+Target extension areas:
+
+- Unit tokens, such as custom blocks, callouts, figures, or inline elements.
+- `where` fields, predicates, and groups.
+- Take operators, such as custom windowing or proximity rules.
+- Sources, targets, transforms, and field interpolation providers.
+
+Expected behavior:
+
+- Extensions register metadata for validation, diagnostics, and GUI capability discovery.
+- Runtime behavior is registered explicitly through handlers or callables.
+- Unknown or disabled extension tokens fail validation instead of being accepted silently.
+- Host-specific behavior stays in host apps or separate extension gems, not in the core gem.
+
+## 3. Authored Field Preservation Guidance
+
+**Status:** Planned
+
+Document and support the expected editing workflow for host applications:
+
+- Preserve authored field strings when exact edit-screen fidelity matters.
+- Normalize field strings, YAML, JSON, and row input for validation and execution.
+- Use future canonical pretty-printer helpers when generated syntax is acceptable.
+
+For Rails and other host frameworks, this means the application should keep the user's authored field string while editing, then pass normalized data to the gem for validation and composition.
+
+Expected outcome:
+
+- Clear README examples showing authored string preservation alongside normalized configs.
+- Validation guidance explaining what the gem can verify and what the host framework must verify.
+- Tests proving canonical output is stable without implying exact round-trip text restoration.
+
+## 4. Maintenance And Bug Fixes
+
+**Status:** Ongoing
+
+Maintain the gem through focused bug fixes, compatibility updates, and regression tests.
+
+Priorities:
+
+- Fix parser, normalization, validation, diagnostics, composition, and transform bugs.
+- Add regression tests for confirmed bugs.
+- Keep supported Ruby versions and runtime dependencies current.
+- Avoid breaking stored configs or public APIs without a planned migration note.

data/docs/_md_composer_architecture.md

@@ -0,0 +1,50 @@
+---
+title: Markdown Composer Architecture
+type: architecture
+status: current
+updated: 2026-06-01
+description: Public architecture overview for Markdown Composer internals and host boundaries.
+---
+
+# Markdown Composer Architecture
+
+### References
+
+| What | Where |
+|------|-------|
+| Concepts | [./_md_composer_concepts.md](./_md_composer_concepts.md) |
+| API reference | [./reference/md_composer_reference_api.md](./reference/md_composer_reference_api.md) |
+| Registries reference | [./reference/md_composer_reference_registries.md](./reference/md_composer_reference_registries.md) |
+
+---
+
+## 1. Execution Pipeline
+
+Markdown Composer runs in five broad stages:
+
+```text
+Parse config -> Validate -> Index sources -> Execute compose rows -> Run transforms
+```
+
+The result contains Markdown, optional HTML, diagnostics, errors, and optional stage snapshots.
+
+## 2. Source Indexing
+
+Each source is converted into a document index. The index exposes nodes and sections so selectors can match headings, sections, paragraphs, lists, tables, code blocks, data blocks, and other units.
+
+HTML input is converted to Markdown as a best-effort/lossy source format. Markdown input is the source-fidelity format.
+
+## 3. Composition Buffer
+
+Compose actions write to a `CompositionBuffer`. The buffer is Markdown text plus origin metadata. Target matching runs against a fresh index of the current buffer.
+
+This detail matters: derived `data_record` and `data_value` nodes can be produced from `data_path(...)`, but ordinary target matching reparses the serialized Markdown buffer. Treat those tokens as structured-output metadata unless a host adapter materializes them as matchable buffer nodes.
+
+## 4. Transforms
+
+Transforms run through `TransformRunner` against the buffer. Top-level transforms come from `config["transform"]`. Row-level transforms are used by `modify` and `transform_buffer_target`.
+
+## 5. Host Boundary
+
+The gem owns parsing, validation, indexing, composition, transforms, and capabilities metadata. The host owns source lookup, UI, persistence, authorization, preview shell, and any policy choices for adapter/policy-gated features.
+

data/docs/_md_composer_cheatsheet.md

@@ -0,0 +1,72 @@
+---
+title: Markdown Composer Cheatsheet
+type: cheatsheet
+status: current
+updated: 2026-06-01
+description: Public quick-reference for Markdown Composer syntax and docs navigation.
+---
+
+# Markdown Composer Cheatsheet
+
+### References
+
+| What | Where |
+|------|-------|
+| Compose anatomy | [./compose/md_composer_compose_anatomy.md](./compose/md_composer_compose_anatomy.md) |
+| Transform anatomy | [./transform/md_composer_transform_anatomy.md](./transform/md_composer_transform_anatomy.md) |
+| Unit tokens | [./reference/md_composer_unit_tokens.md](./reference/md_composer_unit_tokens.md) |
+| Take modifiers | [./reference/md_composer_take.md](./reference/md_composer_take.md) |
+| Where conditions | [./reference/md_composer_where.md](./reference/md_composer_where.md) |
+| Examples | [./examples/md_composer_examples_readme.md](./examples/md_composer_examples_readme.md) |
+
+---
+
+## 1. Compose Row
+
+```text
+Source -> Select -> Include -> Action -> Target
+```
+
+| Field | Common values |
+|-------|---------------|
+| Source | `current`, `explicit`, `previous`, `buffer`, `inline` |
+| Select | `all`, `heading_2_section[first:1]`, `paragraph where text:contains("x")` |
+| Include | `all`, `content`, `heading_title`, `paragraph[first:2]`, `data_path("title")` |
+| Action | `set`, `append`, `prepend`, `replace`, `modify`, `remove_buffer_target`, `transform_buffer_target` |
+| Target | blank, `whole output`, `start`, `end`, selector, `before ...`, `after ...`, `between ... and ...`, `in_place` |
+
+## 2. Transform Row
+
+```text
+Scope -> Transform -> Mode -> Options
+```
+
+| Field | Common values |
+|-------|---------------|
+| Scope | `output`, `heading_2_section`, `paragraph where text:contains("x")` |
+| Transform | `replace_text`, `heading_numbers`, `heading_levels`, `links`, `remove_empty` |
+| Mode | transform-specific, such as `literal`, `rebuild`, `promote`, `rewrite_url` |
+| Options | hash or string, such as `{ "from" => "Draft", "to" => "Final" }` or `from:"Draft"; to:"Final"` |
+
+## 3. Select Syntax
+
+| Need | Syntax |
+|------|--------|
+| First H2 section | `heading_2_section[first:1]` |
+| H2 section by title | `heading_2_section where title:equals("API")` |
+| Paragraphs containing text | `paragraph where text:contains("warning")` |
+| Positions 2 and 3 | `heading_2_section[position:2,3]` |
+| Range | `heading_2_section[range:2..4]` |
+| Nested include | `heading_2_section { heading_title; paragraph[first:1] }` |
+
+## 4. High-Value Docs
+
+| Need | Read |
+|------|------|
+| How compose rows work | [Compose anatomy](./compose/md_composer_compose_anatomy.md) |
+| Which unit tokens exist | [Unit tokens](./reference/md_composer_unit_tokens.md) |
+| How `[first:1]`, `[position:2]`, and ranges work | [Take modifiers](./reference/md_composer_take.md) |
+| How `where title:contains(...)` works | [Where conditions](./reference/md_composer_where.md) |
+| How `modify` works | [Actions](./compose/md_composer_compose_actions.md) and [modify example](./examples/md_composer_example_modify.md) |
+| How targets work | [Targets](./compose/md_composer_compose_targets.md) |
+| How transform options work | [Options](./transform/md_composer_transform_options.md) |

data/docs/_md_composer_concepts.md

@@ -0,0 +1,64 @@
+---
+title: Markdown Composer Concepts
+type: guide
+status: current
+updated: 2026-06-01
+description: Human-facing concepts for Markdown Composer plans, rows, buffers, selectors, and transforms.
+---
+
+# Markdown Composer Concepts
+
+### References
+
+| What | Where |
+|------|-------|
+| Compose anatomy | [./compose/md_composer_compose_anatomy.md](./compose/md_composer_compose_anatomy.md) |
+| Buffer guide | [./compose/md_composer_compose_buffer.md](./compose/md_composer_compose_buffer.md) |
+| Transform anatomy | [./transform/md_composer_transform_anatomy.md](./transform/md_composer_transform_anatomy.md) |
+
+---
+
+## 1. Plans
+
+A plan is the normalized instruction set Markdown Composer executes. Public configs usually use:
+
+```ruby
+{
+  "compose" => [ ... ],
+  "transform" => [ ... ],
+  "output" => "markdown"
+}
+```
+
+`compose` builds the buffer. `transform` reshapes the buffer. `output` is `markdown` or `html`.
+
+## 2. Compose Rows
+
+Compose rows answer: what content should be selected, what part of it should be kept, and where should it go?
+
+```text
+Source -> Select -> Include -> Action -> Target
+```
+
+The most common row is: select a section from the current source, include all of it, and `set` the buffer to that content.
+
+## 3. The Buffer
+
+The buffer is the working Markdown document being built. Early rows create it. Later rows append, prepend, replace, move, remove, or transform content already in it.
+
+Use `source: "buffer"` when a row should select from the current buffer instead of from the original source.
+
+## 4. Transform Rows
+
+Transform rows answer: what part of the composed buffer should change, what transform should run, which mode should it use, and what options does it need?
+
+```text
+Scope -> Transform -> Mode -> Options
+```
+
+Use top-level transforms for whole-output cleanup. Use row-level transforms with `modify` or `transform_buffer_target` when the change belongs to a specific compose action.
+
+## 5. Diagnostics
+
+Markdown Composer prefers diagnostics over silent guessing. A config can be syntactically valid but still produce warnings, such as empty selections or empty destructive targets.
+

data/docs/_md_composer_dev_guide.md

@@ -0,0 +1,55 @@
+---
+title: Markdown Composer Dev Guide
+type: dev_guide
+status: current
+updated: 2026-06-01
+description: Public developer guide for testing, validating, and maintaining Markdown Composer docs and examples.
+---
+
+# Markdown Composer Dev Guide
+
+### References
+
+| What | Where |
+|------|-------|
+| API reference | [./reference/md_composer_reference_api.md](./reference/md_composer_reference_api.md) |
+| Capabilities reference | [./reference/md_composer_reference_capabilities.md](./reference/md_composer_reference_capabilities.md) |
+| AI source map | [./ai/md_composer_ai_source_map.md](./ai/md_composer_ai_source_map.md) |
+
+---
+
+## 1. Source Of Truth
+
+For runtime behavior, trust gem code and tests first. For prose, this public `docs/` tree is the reader-facing set. The `docs/ai/` files map back to the audited canonical source material used to create these docs.
+
+## 2. Validate Examples
+
+Use focused Ruby probes for every runnable snippet:
+
+```bash
+bundle exec ruby -Ilib -rmarkdown_composer -e 'puts MarkdownComposer::VERSION'
+```
+
+For broad safety:
+
+```bash
+bundle exec rake test
+```
+
+## 3. Writing Rules
+
+Every public guide should include:
+
+| Requirement | Reason |
+|-------------|--------|
+| Syntax anatomy | Readers need to see the shape before memorizing values. |
+| Valid forms | Avoids guesswork between strings, hashes, YAML, and Ruby. |
+| Runnable example | Keeps docs grounded. |
+| Expected output | Makes behavior inspectable. |
+| Common mistakes | Turns diagnostics into learning. |
+| Related docs | Keeps pages focused without hiding depth. |
+
+## 4. Public Boundary
+
+Keep core gem docs framework-independent. Hosts can provide source pickers, persistence, policy gates, and preview UI, but Markdown Composer itself remains headless.
+