maquina_remend 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 3ae80dab37d601a1b8570d73e4d29e2f10296205ae349cdace4bfe2f8d17cb53
+  data.tar.gz: b97c54a013f0c2dc60616bc1af16ecd3a66786dacf162535c47ed2ec8976ea93
+SHA512:
+  metadata.gz: fc6b594c251f168ac266659d123d3fddf3a43c10b3b7470a076b3496c75ee9477f86da915634e9aa32b9964fe338b4d0d8039e7398328db1ed3ab15af03d384b
+  data.tar.gz: 3fa25bb6cb8cb511338fafbd7261115a52a4f8203a7c91ef2ea293bb3bf595b43ae2b4c2e79078749d429e59a7baff824b4a7cb4d7931a22be46aeb99349d667

data/.rdoc_options

@@ -0,0 +1,16 @@
+# RDoc configuration. Comments in this gem are written in Markdown, not RDoc
+# markup, so `markup` has to be set here as well as in the gemspec's
+# rdoc_options: `gem rdoc` reads this file, rubygems.org's doc build reads the
+# gemspec, and a bare `rdoc` in a checkout reads only this one.
+---
+markup: markdown
+title: maquina_remend
+main_page: README.md
+exclude:
+  - "/sdd/"
+  - "CLAUDE\\.md$"
+  - "/test/"
+  - "remend-patterns\\.md$"
+  - "Gemfile"
+  - "Rakefile$"
+  - "\\.gemspec$"

data/CHANGELOG.md

@@ -0,0 +1,31 @@
+# Changelog
+
+## 0.1.0 — 2026-09-07
+
+First working version. `MaquinaRemend.call` repairs the tail of a markdown
+buffer that is still arriving, so a renderer never sees a broken document
+mid-stream.
+
+- Fourteen completion cases: emphasis, strong, inline code, links and images,
+  block math, setext headings, HTML tags, strikethrough, a dangling escape.
+- Application tags — `<thinking>`, `<answer>`, `<tool_call>`, `<citation>`,
+  `<scratchpad>` — closed while the model is still inside them, so the answer
+  that follows a blank line does not escape the tag. `app_tags:` takes the list
+  of names, or `false`.
+- Nine guard cases, including comparison operators and single tildes that must
+  not be mistaken for markup.
+- A scanner that tracks fence, inline-code and math state, so a repair never
+  fires inside a code block.
+- Per-handler options and a custom handler contract, with `Context` exposing
+  the state a handler needs to decide for itself.
+- Idempotent, and byte-identical on well-formed input. Both are asserted over
+  the whole corpus, at every truncation point of every document.
+- Application-meaning tags a model leaves open — `<thinking>`, `<answer>`,
+  `<tool_call>` and any name you configure — are closed, so a renderer is never
+  handed a document whose AST and DOM disagree about where the tag ends.
+- Zero runtime dependencies and no Rails, asserted in a bare
+  `ruby --disable-gems` process.
+
+Verified against 24 real agent transcripts (14,434 stream chunks), which is how
+the dangling-escape case was found: a trailing backslash escapes whatever the
+next handler appends, so the repair grew on every pass.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mario Alberto Chávez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,169 @@
+# maquina_remend
+
+Repairs the tail of a markdown buffer that is still being streamed.
+
+A model emits markdown a token at a time. Rendered naively, the tail of the
+buffer is broken for most of the message's life. The model has typed `**bol`
+and the reader sees two asterisks; it has typed `[the guide](https://exa` and
+the reader sees a link to the wrong host. `maquina_remend` completes that tail,
+so the renderer always receives a well-formed document.
+
+```ruby
+MaquinaRemend.call("**unclosed bold")      # => "**unclosed bold**"
+MaquinaRemend.call("`code")                # => "`code`"
+MaquinaRemend.call("[label](https://exa")  # => "[label](#)"
+MaquinaRemend.call("<thinking>\nstill reasoning")
+# => "<thinking>\nstill reasoning</thinking>"
+```
+
+It is a pure function. It is idempotent. A document that is already well formed
+comes back byte for byte. **Zero runtime dependencies, and no Rails** — it runs
+in a plain Ruby process with nothing else loaded.
+
+## Install
+
+```sh
+bundle add maquina_remend
+```
+
+or, in a Gemfile:
+
+```ruby
+gem "maquina_remend"
+```
+
+Ruby >= 3.1.
+
+## Usage
+
+```ruby
+require "maquina_remend"
+
+MaquinaRemend.call(markdown, **options) # => String
+```
+
+Call it on every frame of the stream, on the whole buffer so far, and render
+the result. Nothing is remembered between calls.
+
+```ruby
+buffer = +""
+
+stream.each do |chunk|
+  buffer << chunk
+  render MaquinaRemend.call(buffer)
+end
+```
+
+`nil` and `""` come back exactly as they went in. An unrecognised option raises
+`MaquinaRemend::UnknownOption` rather than being ignored, so a typo in a host's
+configuration surfaces at once.
+
+## Options
+
+Every repair is individually disableable, and one is off by default.
+
+| Option | Default | Broken input | Repaired to |
+|---|---|---|---|
+| `bold:` | `true` | `**unclosed bold` | `**unclosed bold**` |
+| `italic:` | `true` | `*unclosed` / `_unclosed` | `*unclosed*` / `_unclosed_` |
+| `bold_italic:` | `true` | `***both` | `***both***` |
+| `inline_code:` | `true` | `` `code `` | `` `code` `` |
+| `strikethrough:` | `true` | `~~struck` | `~~struck~~` |
+| `links:` | `true` | `[label` | `[label]()` |
+| `images:` | `true` | `![alt` | `![alt]()` |
+| `block_math:` | `true` | `$$x = 1` | `$$x = 1$$` |
+| `inline_math:` | **`false`** | `$x = 1` | `$x = 1$` |
+| `setext_headings:` | `true` | `Title\n=` | `Title\n=====` |
+| `comparison_operators:` | `true` | `- a > b` | `- a \> b` |
+| `html_tags:` | `true` | `text <div cla` | `text` |
+| `single_tilde:` | `true` | `20~25°C` | `20\~25°C` |
+| `dangling_escape:` | `true` | `media ecuación \` | `media ecuación` and a space |
+| `app_tags:` | five names | `<thinking>\nstill reasoning` | `<thinking>\nstill reasoning</thinking>` |
+| `link_mode:` | `:protocol` | `[label](http` | `[label](#)` |
+| `handlers:` | `[]` | — | your own handlers, run last |
+
+Pass `false` to switch a repair off:
+
+```ruby
+MaquinaRemend.call("**unclosed bold", bold: false)  # => "**unclosed bold"
+MaquinaRemend.call("![alt", images: false)          # => "![alt"
+```
+
+**`inline_math:` is off because a bare dollar sign is currency far more often
+than it is mathematics.** Turn it on only if you know your corpus.
+
+**`app_tags:` is a list, not a boolean.** It names the tags a model emits to
+carry application meaning — `thinking`, `answer`, `tool_call`, `citation`,
+`scratchpad` by default — and closes the ones the model has left open. Pass
+your own names to match the host's tag registry, or `false` to switch it off:
+
+```ruby
+MaquinaRemend.call("<plan>\nstep one", app_tags: %w[plan])
+# => "<plan>\nstep one</plan>"
+```
+
+Every repair, with the input it fires on and the input it refuses to touch, is
+in **[docs/repairs.md](docs/repairs.md)**. The reasoning behind the `app_tags:`
+list is under [Application tags](docs/repairs.md#application-tags).
+
+## What it will not do
+
+Guards matter more than completions. A false repair corrupts a message that was
+never broken, and there is no frame later in the stream that undoes it.
+
+```ruby
+MaquinaRemend.call("```\n**not bold")      # => "```\n**not bold"
+MaquinaRemend.call("`a * b`")              # => "`a * b`"
+MaquinaRemend.call("$$a_1 + b_2$$")        # => "$$a_1 + b_2$$"
+MaquinaRemend.call("some_var_name")        # => "some_var_name"
+MaquinaRemend.call("costs $5 and $10")     # => "costs $5 and $10"
+```
+
+Nothing is completed inside a fenced code block, inside an inline code span, or
+inside math, because in those places broken-looking markup is the content. The
+full list is in [docs/repairs.md](docs/repairs.md#guards).
+
+## Documentation
+
+* **[docs/repairs.md](docs/repairs.md)** — every repair, one section each, with
+  the input it fires on and the guards that hold it back.
+* **[docs/handlers.md](docs/handlers.md)** — writing a handler of your own: the
+  contract, the `Context` object, ordering, and a worked example.
+* **[docs/streaming.md](docs/streaming.md)** — using it in a stream, what
+  idempotence and prefix safety buy a caller, and what the gem deliberately
+  does not do.
+
+API documentation for every class is on
+[rubydoc.info](https://rubydoc.info/gems/maquina_remend).
+
+## Guarantees
+
+Three properties, asserted over a corpus of whole documents at every truncation
+point rather than over hand-picked inputs:
+
+1. **Idempotence** — `call(call(x)) == call(x)`.
+2. **No-op on well-formed input** — a complete document comes back
+   byte-identical.
+3. **Prefix safety** — every truncation point of a document is repairable
+   without raising.
+
+A fourth is measured rather than proved: an 8KB buffer is repaired in under a
+millisecond.
+
+Run them with `bundle exec rake test`. The prefix-safety property also accepts a
+local corpus of real transcripts:
+
+```sh
+MAQUINA_REMEND_CORPUS=~/some/transcripts bundle exec rake test
+```
+
+That corpus is deliberately not committed: a transcript contains whatever the
+session contained.
+
+## maquina
+
+Part of [maquina](https://maquina.app) — open source for Ruby and Ruby AI.
+
+## License
+
+MIT, © Mario Alberto Chávez. See [LICENSE.txt](LICENSE.txt).

data/docs/handlers.md

@@ -0,0 +1,223 @@
+# Custom handlers
+
+The built-in repairs are listed in [repairs.md](repairs.md). When a host needs
+one of its own — a template placeholder, a shortcode, a marker the model emits
+that no markdown parser knows about — it passes a handler.
+
+```ruby
+MaquinaRemend.call(markdown, handlers: [MyHandler.new])
+```
+
+## The contract
+
+A handler is **any object that responds to `#call(text, context)` and returns a
+String.** There is no class to inherit from and no registration step.
+
+```ruby
+class MyHandler
+  def call(text, context) # context: MaquinaRemend::Context
+    text
+  end
+end
+```
+
+* **`text`** is the buffer as the built-ins left it — already repaired.
+* **`context`** is a `MaquinaRemend::Context` describing where the tail sits.
+* **Return a String.** Returning the argument unchanged is the normal outcome;
+  most passes over most buffers repair nothing.
+* **Handlers run in the order given**, each seeing the previous one's output.
+* **Stay idempotent, and stay byte-identical on well-formed input.** Those two
+  properties belong to the library, not just to the built-ins. A handler that
+  appends unconditionally breaks both: the text it appended dangles on the next
+  pass and gets appended to again.
+
+## The `Context` object
+
+```ruby
+context = MaquinaRemend.context("~~~ruby\ndef call")
+context.in_code_fence?  # => true
+context.open_fence_info # => "ruby"
+context.in_inline_code? # => false
+context.in_math?        # => false
+
+MaquinaRemend.context("a `code").in_inline_code?  # => true
+MaquinaRemend.context("$$x = 1").in_math?         # => true
+```
+
+Three predicates and two readers, and nothing else:
+
+| Member | Answers |
+|---|---|
+| `#in_code_fence?` | is the tail inside an unclosed fenced code block? |
+| `#in_inline_code?` | is an unclosed backtick span open in this paragraph? |
+| `#in_math?` | is an unclosed `$$` block-math span open? |
+| `#open_fence_info` | the open fence's info string, or `nil` |
+| `#options` | the resolved option hash the pipeline was built with |
+
+The three predicates are the three places where broken-looking markup is the
+content rather than a defect. A repair that needs more state than they expose
+means the scanner is missing a state, not that `Context` should grow.
+
+`#open_fence_info` is `nil` when no fence is open, and also when the open fence
+carried no info string:
+
+```ruby
+MaquinaRemend.context("~~~ruby\ndef call").open_fence_info # => "ruby"
+MaquinaRemend.context("~~~\nplain").open_fence_info        # => nil
+MaquinaRemend.context("plain text").open_fence_info        # => nil
+```
+
+`MaquinaRemend.context` is public for callers that want the reading without the
+repair — a renderer deciding whether to run a syntax highlighter over a block
+whose closing fence has not streamed in yet:
+
+```ruby
+context = MaquinaRemend.context(buffer)
+highlight(buffer, language: context.open_fence_info) if context.in_code_fence?
+```
+
+Its `#options` is empty, because it did no repair and therefore had no options.
+
+## Ordering
+
+**Custom handlers run last, after every built-in.** Two consequences.
+
+First, the text a handler receives already carries the built-ins' completions.
+An `upcase` handler sees the closer the emphasis repair appended:
+
+```ruby
+shouty = Object.new
+def shouty.call(text, _context) = text.upcase
+
+MaquinaRemend.call("**bold", handlers: [shouty])  # => "**BOLD**"
+```
+
+If your handler anchors on the end of the buffer, remember that the end of the
+buffer may no longer be where the model stopped typing.
+
+Second, **a custom handler still runs inside a fenced code block.** The
+built-ins all return early there; a custom handler does not, because it is given
+the context and trusted to decide — which it could not do if it never ran. Ask
+for yourself:
+
+```ruby
+return text if context.in_code_fence?
+```
+
+## A worked example
+
+A host renders emoji shortcodes, `:smile:`. Mid-stream the model has typed
+`:smi`, and the reader sees a stray colon and three letters that will change
+under them a frame later. Drop the partial shortcode until it is whole.
+
+```ruby
+class PartialShortcode
+  TRAILING = /(?<=\A|\s):[a-z][a-z0-9_+-]*\z/
+
+  def call(text, context)
+    return text if context.in_code_fence?
+
+    text.sub(TRAILING, "").rstrip
+  end
+end
+
+handler = PartialShortcode.new
+```
+
+It fires on a shortcode that is still arriving:
+
+```ruby
+MaquinaRemend.call("Nice work :smi", handlers: [handler])
+# => "Nice work"
+```
+
+It leaves a finished one alone — the pattern cannot match a trailing `:`, so a
+complete document passes through untouched:
+
+```ruby
+MaquinaRemend.call("Nice work :smile:", handlers: [handler])
+# => "Nice work :smile:"
+MaquinaRemend.call("Nice work", handlers: [handler])
+# => "Nice work"
+```
+
+It leaves a fence alone, because it asked:
+
+```ruby
+MaquinaRemend.call("```\nNice work :smi", handlers: [handler])
+# => "```\nNice work :smi"
+```
+
+The lookbehind is what keeps it off a colon that was never a shortcode:
+
+```ruby
+MaquinaRemend.call("ratio 3:2", handlers: [handler])
+# => "ratio 3:2"
+```
+
+And it is idempotent: running it over its own output changes nothing, in every
+case above.
+
+The one case it does not catch is the ordering rule from the previous section.
+The emphasis repair has already appended `**`, so the shortcode is no longer at
+the end of the buffer and the anchor misses:
+
+```ruby
+MaquinaRemend.call("**Nice work :smi", handlers: [handler])
+# => "**Nice work :smi**"
+```
+
+That is the trade the ordering makes, and it is the right way round: a custom
+handler that ran *before* the built-ins would be repairing a buffer the
+built-ins were about to rewrite underneath it. A handler that must see the raw
+tail should match on the paragraph rather than on `\z`.
+
+## Reusing a validated pipeline
+
+`MaquinaRemend.call` builds a `Pipeline` per call and throws it away. Options
+are validated in the constructor, so building one yourself is worth it when the
+same options are used for every frame of a long stream:
+
+```ruby
+pipeline = MaquinaRemend::Pipeline.new(inline_math: true)
+pipeline.call("$x = 1")  # => "$x = 1$"
+
+deltas.map { |buffer| pipeline.call(buffer) }
+```
+
+It holds no state between buffers. Validation happening in the constructor also
+means a bad option is a startup error for the host rather than a surprise
+halfway through a stream:
+
+```ruby
+MaquinaRemend::Pipeline.new(bolder: true)
+# MaquinaRemend::UnknownOption: unknown option(s): bolder
+```
+
+## Where a custom handler runs in the order
+
+The built-ins run in a fixed order, and then yours:
+
+```ruby
+MaquinaRemend::Pipeline::BUILT_INS.map(&:name)
+# => ["MaquinaRemend::Handlers::DanglingEscape",
+#     "MaquinaRemend::Handlers::HtmlTags",
+#     "MaquinaRemend::Handlers::InlineCode",
+#     "MaquinaRemend::Handlers::Links",
+#     "MaquinaRemend::Handlers::Math",
+#     "MaquinaRemend::Handlers::Emphasis",
+#     "MaquinaRemend::Handlers::Strikethrough",
+#     "MaquinaRemend::Handlers::SetextHeading",
+#     "MaquinaRemend::Handlers::ComparisonOperators",
+#     "MaquinaRemend::Handlers::SingleTilde",
+#     "MaquinaRemend::Handlers::AppTags"]
+```
+
+The order is load-bearing. A dangling backslash is dropped first, because it
+would escape whatever any later handler appends. Links run before emphasis,
+because an emphasis closer appended into a half-typed `[label(` lands inside
+the destination. Application tags run last, because their closer has to sit
+outside every other completion.
+
+The buffer is rescanned after any handler that changed it, so no handler reads
+a view of the text that an earlier one has already invalidated.