maquina_stream 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b102e4961a1ab8c838ac27577f9358003a64c31491569600600e5782f1d0b9e0
+  data.tar.gz: 0c4c875bd048b17c0533fa86dd0c739d32344a2c376d26034b3c45896d056ff5
+SHA512:
+  metadata.gz: ab81d7757223c030d6748fc87abe5124f528497a1562994947c16a1e2ec4a4992e504144beacb83d951b7ebfd36d8d33d43bb6ef542d34e825963c9133f96c5c
+  data.tar.gz: 81e39ef8e44f0437cbcd6d83cd8c2fd78d61aa3a945cca8294b663dda8c6869b30f6534f491ae8d57104696f00e7cb4546dbb2ca8b851eb1cae8b64da407c57b

data/.rdoc_options

@@ -0,0 +1,30 @@
+---
+# RDoc reads this file from the gem root. Comments in lib/ and app/ are written
+# in Markdown, not RDoc markup — see the `markup` key below.
+markup: markdown
+title: maquina_stream — server-rendered streaming markdown for Rails
+main_page: README.md
+# Everything that documents how the gem was BUILT rather than how it is USED.
+# sdd/ holds the phase specs, test/ includes the whole dummy host application,
+# and docs/{plan,design,remend-patterns,spike-sourcepos}.md are the same four
+# files the gemspec does not package: none of them documents the gem to
+# somebody who installs it.
+#
+# app/javascript is excluded because the JavaScript is documented as a DOM and
+# controller contract in docs/javascript.md, which is a better description of
+# it than RDoc's JavaScript parser produces.
+#
+# These are matched against ABSOLUTE paths, so each pattern is anchored to a
+# path boundary rather than to the start of the string.
+exclude:
+  - "(?:\\A|/)test/"
+  - "(?:\\A|/)sdd/"
+  - "(?:\\A|/)bin/"
+  - "(?:\\A|/)tmp/"
+  - "(?:\\A|/)pkg/"
+  - "(?:\\A|/)doc/"
+  - "(?:\\A|/)\\.github/"
+  - "(?:\\A|/)app/javascript/"
+  - "(?:\\A|/)docs/(?:plan|design|remend-patterns|spike-sourcepos)\\.md\\z"
+  - "(?:\\A|/)CLAUDE\\.md\\z"
+  - "(?:\\A|/)Gemfile(?:\\.lock)?\\z"

data/CHANGELOG.md

@@ -0,0 +1,38 @@
+# Changelog
+
+## Unreleased — 0.1.0
+
+First working version. The engine renders a streaming markdown buffer to HTML on
+the server and broadcasts it over Turbo Streams; only rendered HTML reaches the
+browser.
+
+- **Render pipeline** — `maquina_remend` → commonmarker with sourcepos →
+  a single Nokogiri post-pass → sanitizer. Pure: it runs outside a Rails request
+  with no stubbing, asserted in a subprocess with no application booted.
+- **Sealing and broadcast** — blocks seal behind a configurable lag, ids are
+  index-derived, and a sealed block is never re-broadcast. The seal pointer
+  stops at an unresolved link reference definition, because a definition
+  rewrites links arbitrarily far above it.
+- **Repair** — a windowed manifest of block digests, four triggers, and a silent
+  morph of only the blocks that differ. Deltas are an optimization; correctness
+  lives here.
+- **Interaction** — copy, download, table export, fullscreen, link safety and
+  autoscroll, every control individually disableable and inert while streaming.
+- **Client-deferred renderers** — diagrams and math render in the browser from
+  one JSON payload per leaf node, with split ownership across morphs.
+- **Registries** — elements, custom tags and fences, the last with server,
+  passthrough and client strategies.
+- **Text direction per block**, decided by the first strong character. Fences
+  carrying bidi control characters are pinned, against Trojan Source.
+- **History** — sealed messages are frozen and fragment-cached by buffer digest;
+  1942x on a warm cache.
+- **Generators** — `maquina_stream:install` wires the engine into a host (the
+  initializer, the mount, the Turbo and engine importmap pins, the Stimulus
+  registration, the stylesheets) and `maquina_stream:streamable Message` makes
+  one model streamable (the migration, generated from the `Streamable` contract
+  itself, and the macro). Idempotent, non-destructive, and loud about anything
+  they cannot do. Both seams stay the host's: `authorize` is generated as a stub
+  that denies.
+- Spanish default locale, English secondary. No Node build step anywhere.
+
+Not yet released: `maquina_remend` is still a path reference.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mario Alberto Chávez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,163 @@
+# maquina_stream
+
+Server-rendered streaming markdown for Rails, over Turbo, with a repair path.
+
+A model writes markdown a token at a time. `maquina_stream` renders it to HTML
+on the server, broadcasts small patches as it grows, and reconciles the browser
+with the truth when frames go missing — which they do, because Action Cable
+offers no delivery guarantee.
+
+**Only rendered HTML reaches the browser.** The client never parses markdown.
+
+## Install
+
+```ruby
+gem "maquina_stream"
+```
+
+Rails 8, Ruby 3.3+. Depends on `maquina_remend`, `commonmarker`, `nokogiri` and
+`rouge`. `maquina_components` is optional — without it, components render plain
+Tailwind fallbacks. Turbo is required: repair applies Turbo Stream morphs, and
+with `window.Turbo` undefined every repair fails silently.
+
+```sh
+bin/rails generate maquina_stream:install
+bin/rails generate maquina_stream:streamable Message
+bin/rails db:migrate
+```
+
+The first wires the engine into the app — the initializer, the mount, the
+importmap pins, the Stimulus registration, the stylesheets. The second is per
+model, because a host may have several: the migration carrying the contract
+columns, and the `include` plus macro in the model. Both are idempotent and
+neither overwrites a file; a host missing an ingredient is told what to paste
+rather than left with a silent no-op.
+
+## Two seams the generators will not guess
+
+**`authorize`**, in `config/initializers/maquina_stream.rb`. It is generated as
+a stub that denies everything, which is the safe failure and a broken feature
+both — the browser can never repair a message until you replace it. Guessing a
+host's authorization is how an engine leaks other people's messages:
+
+```ruby
+c.authorize = ->(record, request) { record.conversation.member?(request.session[:user_id]) }
+```
+
+**`stream_for:`**, in the model. Who may subscribe to a stream is your question,
+not the engine's:
+
+```ruby
+class Message < ApplicationRecord
+  include MaquinaStream::Streamable
+
+  maquina_stream buffer: :content,
+    stream_for: ->(record) { [:conversation, record.conversation_id, :messages] }
+end
+```
+
+## Streaming a message
+
+Feed the broadcaster and seal exactly once:
+
+```ruby
+broadcaster = MaquinaStream::Broadcaster.new(message)
+chat.ask(prompt) { |chunk| broadcaster.append(chunk.content.to_s) }
+broadcaster.seal!
+```
+
+Render it, live or from history. The engine appends block HTML into
+`#ms-msg-<sid>` and never creates that element — the wrapper is yours:
+
+```erb
+<div id="ms-msg-<%= message.maquina_stream_id %>"
+     data-controller="ms-repair ms-reveal"
+     data-ms-repair-manifest-url-value="<%= maquina_stream.manifest_path(sid: message.maquina_stream_id) %>"
+     data-ms-repair-blocks-url-value="<%= maquina_stream.blocks_path(sid: message.maquina_stream_id) %>"
+     <%= "data-ms-streaming" if message.maquina_stream_open? %>><%= MaquinaStream.render(message) %></div>
+```
+
+That is the whole integration. [Getting started](docs/getting-started.md) walks
+the same path with a working end-to-end run, and lists every step the
+generators take for hosts that would rather do them by hand.
+
+## What you get
+
+Markdown in:
+
+````markdown
+## Status
+
+All **good**. See [docs](https://example.com).
+
+```ruby
+puts 1
+```
+````
+
+HTML out — sanitized, block-addressed, and highlighted by CSS class so a theme
+switch needs no re-render (attributes elided for width):
+
+```html
+<h2 id="ms-42-b0" data-ms-element="h2" data-ms-block data-ms-block-digest="fa59d158d6fc403e">Status…</h2>
+<p id="ms-42-b1" data-ms-element="p" data-ms-block data-ms-block-digest="c39b9098233d53d5">All <strong>good</strong>. See <a href="https://example.com" rel="noopener noreferrer">docs</a>.</p>
+<div id="ms-42-b2" data-ms-code data-ms-code-lang="ruby" data-controller="ms-code" data-ms-block …>
+  <pre><code><span class="nb">puts</span> <span class="mi">1</span></code></pre>
+  <pre hidden data-ms-code-source>puts 1</pre>
+</div>
+```
+
+Plus: copy and download on code blocks and tables, a link-safety dialog, a
+word-level reveal animation, autoscroll that gets out of the reader's way,
+client-rendered diagrams and math, and markdown export.
+
+## Agents
+
+This engine is built for agent output, and a tool call is its own `Streamable`
+record rather than a block inside the assistant's message — so each step of a
+run seals, repairs and exports on its own, and two steps can be open at once,
+which is what a parallel tool call is.
+
+[Nexo](https://maquina.app/documentation/nexo/) is the agent harness it is built
+alongside: `Agent#prompt` reports tool activity through a block, and each
+`:tool_call` opens a record. `Message.stream_agent_run` in
+`test/dummy/app/models/message.rb` is the whole bridge, and `/harness/agent`
+runs it against a real model. See [streaming.md](docs/streaming.md#streaming-an-agent-run).
+
+Nothing here depends on Nexo. Bare `ruby_llm`, or any client that hands you text
+as it arrives, works the same way — `/harness/chat` is that version.
+
+## Documentation
+
+| Document | What it covers |
+|---|---|
+| [getting-started.md](docs/getting-started.md) | Install, the model, the migration, the view, the JavaScript, one message end to end |
+| [configuration.md](docs/configuration.md) | Every option, its default, and how to choose |
+| [streaming.md](docs/streaming.md) | The `Streamable` contract, the broadcaster, sealing, agent runs, history |
+| [repair.md](docs/repair.md) | Why repair exists, the two routes, `find_stream` and `authorize` |
+| [registries.md](docs/registries.md) | Fences, custom tags, element overrides |
+| [javascript.md](docs/javascript.md) | The Stimulus controllers, importmap setup, controls, events |
+| [security.md](docs/security.md) | The sanitizer's guarantees and limits, URL hardening |
+| [deferred-renderers.md](docs/deferred-renderers.md) | Diagrams, math, writing your own |
+
+`test/dummy` is a working host: the `Streamable` model, both seams, all three
+fence strategies, the tag registry, and live pages at `/harness/chat` and
+`/harness/agent` that talk to a real model.
+
+## What it deliberately does not do
+
+- **Deltas do not converge on their own.** Only the open tail is patched; a
+  block that changes after it stops being the tail is fixed by the repair path.
+- **Sealed blocks are never re-broadcast.** That is what keeps bandwidth
+  tracking drift instead of message length.
+- **An open fence is never highlighted**, and a client-deferred fence emits no
+  payload until it closes. Expect no syntax colours and no diagram until the
+  closing ``` arrives.
+
+## maquina
+
+Part of [maquina](https://maquina.app) — open source for Ruby and Ruby AI.
+
+## License
+
+MIT, © Mario Alberto Chávez. See [LICENSE.txt](LICENSE.txt).

data/app/assets/stylesheets/maquina_stream/components/attachment.css

@@ -0,0 +1,35 @@
+/* Vendored fallback for the `attachment` component. Loaded only while the
+ * vendored partial is the one rendering.
+ *
+ * Layout is Tailwind, in the partial. What lives here is what a utility class
+ * cannot express: the variant shapes, truncation of a long filename, and the
+ * hidden state the download control takes when its image fails to load.
+ */
+[data-component="attachment"] {
+  --attachment-radius: var(--surface-radius, 0.5rem);
+
+  border-radius: var(--attachment-radius);
+}
+
+[data-component="attachment"][data-variant="inline"] {
+  --attachment-radius: 9999px;
+}
+
+[data-component="attachment"] [data-attachment-part="filename"],
+[data-component="attachment"] [data-attachment-part="details"] {
+  overflow: hidden;
+  text-overflow: ellipsis;
+  white-space: nowrap;
+}
+
+[data-component="attachment"] [data-attachment-part="thumbnail"] {
+  object-fit: cover;
+  background-color: var(--attachment-thumbnail-bg, rgb(0 0 0 / 0.04));
+}
+
+/* `ms-attachment` sets this when the thumbnail errors: a blob that will not
+ * load must not offer a download that cannot work.
+ */
+[data-component="attachment"] [data-attachment-part="download"][hidden] {
+  display: none;
+}

data/app/assets/stylesheets/maquina_stream/components/code_block.css

@@ -0,0 +1,30 @@
+/* Vendored fallback for the `code-block` component.
+ *
+ * Loaded ONLY while maquina_stream renders the vendored partial. The day
+ * maquina_components ships `code-block`, the resolver stops returning our
+ * partial, MaquinaStream::Components.stylesheets stops listing this file, and
+ * the same selectors are not defined twice.
+ *
+ * Colour lives in the two Rouge theme stylesheets, never inline, so a dark
+ * mode switch is a stylesheet swap and not a re-render.
+ */
+[data-component="code-block"] {
+  --code-block-radius: var(--surface-radius, 0.5rem);
+
+  position: relative;
+  border-radius: var(--code-block-radius);
+}
+
+[data-component="code-block"] [data-code-block-part="pre"] {
+  margin: 0;
+  overflow-x: auto;
+}
+
+[data-component="code-block"] [data-code-block-part="code"] {
+  font-family: var(--font-mono, ui-monospace, SFMono-Regular, Menlo, monospace);
+  tab-size: 2;
+}
+
+[data-component="code-block"] script[data-ms-code-source] {
+  display: none;
+}

data/app/assets/stylesheets/maquina_stream/components/shimmer.css

@@ -0,0 +1,31 @@
+/* Engine-owned and permanent: the one skeleton for open blocks and
+ * unrendered deferred payloads. Pure CSS, zero JS.
+ */
+[data-component="shimmer"] [data-shimmer-part="line"] {
+  position: relative;
+  overflow: hidden;
+}
+
+[data-component="shimmer"] [data-shimmer-part="line"]::after {
+  content: "";
+  position: absolute;
+  inset: 0;
+  transform: translateX(-100%);
+  background-image: linear-gradient(
+    90deg,
+    rgb(255 255 255 / 0) 0,
+    rgb(255 255 255 / 0.35) 50%,
+    rgb(255 255 255 / 0) 100%
+  );
+  animation: ms-shimmer 1.4s infinite;
+}
+
+@keyframes ms-shimmer {
+  100% { transform: translateX(100%); }
+}
+
+@media (prefers-reduced-motion: reduce) {
+  [data-component="shimmer"] [data-shimmer-part="line"]::after {
+    animation: none;
+  }
+}

data/app/assets/stylesheets/maquina_stream/components/snippet.css

@@ -0,0 +1,22 @@
+/* Vendored fallback for the `snippet` component. Loaded only while the
+ * vendored partial is the one rendering.
+ */
+[data-component="snippet"] {
+  --snippet-radius: var(--surface-radius, 0.375rem);
+
+  border-radius: var(--snippet-radius);
+}
+
+[data-component="snippet"] [data-snippet-part="command"] {
+  font-family: var(--font-mono, ui-monospace, SFMono-Regular, Menlo, monospace);
+  white-space: pre;
+  overflow-x: auto;
+}
+
+[data-component="snippet"] [data-snippet-part="prompt"] {
+  user-select: none;
+}
+
+[data-component="snippet"] script[data-ms-code-source] {
+  display: none;
+}

data/app/assets/stylesheets/maquina_stream/components/source_citation.css

@@ -0,0 +1,14 @@
+/* Engine-owned and permanent: the reference implementation of
+ * `register_tag :source`.
+ */
+[data-component="source-citation"] {
+  --source-citation-radius: var(--surface-radius, 0.25rem);
+
+  border-radius: var(--source-citation-radius);
+  line-height: 1.2;
+  vertical-align: baseline;
+}
+
+[data-component="source-citation"] [data-source-citation-part="index"] {
+  font-variant-numeric: tabular-nums;
+}

data/app/assets/stylesheets/maquina_stream/components/suggestion.css

@@ -0,0 +1,32 @@
+/* Vendored fallback for the `suggestion` component. Loaded only while the
+ * vendored partial is the one rendering.
+ *
+ * No JavaScript and no client state: a chip is a link or a form submission, so
+ * everything here is shape and focus.
+ */
+[data-component="suggestion"] {
+  display: flex;
+  flex-wrap: wrap;
+  gap: var(--suggestion-gap, 0.5rem);
+}
+
+[data-component="suggestion"] [data-suggestion-part="chip"] {
+  --suggestion-radius: 9999px;
+
+  border-radius: var(--suggestion-radius);
+  cursor: pointer;
+  white-space: nowrap;
+  max-width: 100%;
+  overflow: hidden;
+  text-overflow: ellipsis;
+}
+
+/* button_to wraps its button in a form; the chip must still sit in the row. */
+[data-component="suggestion"] form {
+  display: contents;
+}
+
+[data-component="suggestion"] [data-suggestion-part="chip"]:focus-visible {
+  outline: var(--focus-ring-width, 2px) solid var(--focus-ring-color, currentColor);
+  outline-offset: 2px;
+}

data/app/assets/stylesheets/maquina_stream/reveal.css

@@ -0,0 +1,48 @@
+/* maquina_stream — streaming reveal.
+ *
+ * Load once in the host layout, next to the theme:
+ *
+ *   <%= stylesheet_link_tag "maquina_stream/reveal" %>
+ *
+ * `ms-reveal` wraps the text that just arrived in one
+ * `<span data-ms-revealing>` and unwraps it again on `animationend`. This file
+ * is therefore one keyframe and one rule: the controller decides *what*
+ * animates, the stylesheet decides only how it looks.
+ *
+ * There is no mask here on purpose. A gradient mask placed from a character
+ * fraction hides a horizontal band, and on a block wrapped over several lines
+ * that band crosses lines the reader has already read — they flash out and back
+ * in on every frame. Animating the new text itself needs no mapping from
+ * characters to pixels at all. See app/javascript/maquina_stream/controllers/
+ * ms_reveal_controller.js.
+ */
+
+@keyframes ms-reveal-in {
+  from {
+    opacity: 0;
+    filter: blur(4px);
+  }
+
+  to {
+    opacity: 1;
+    filter: blur(0);
+  }
+}
+
+[data-ms-revealing] {
+  animation: ms-reveal-in var(--ms-reveal-duration, 320ms) ease-out both;
+}
+
+/* No animation, and the text is fully present the moment it arrives. The
+ * controller answers the same query and stops wrapping entirely — with
+ * `animation: none` no `animationend` fires, so a span wrapped anyway would
+ * never be unwrapped. This rule is the second half of that: it also covers a
+ * span left mid-flight by the preference changing while a message streams.
+ */
+@media (prefers-reduced-motion: reduce) {
+  [data-ms-revealing] {
+    animation: none !important;
+    opacity: 1 !important;
+    filter: none !important;
+  }
+}