maquina_stream 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. checksums.yaml +7 -0
  2. data/.rdoc_options +30 -0
  3. data/CHANGELOG.md +38 -0
  4. data/LICENSE.txt +21 -0
  5. data/README.md +163 -0
  6. data/app/assets/stylesheets/maquina_stream/components/attachment.css +35 -0
  7. data/app/assets/stylesheets/maquina_stream/components/code_block.css +30 -0
  8. data/app/assets/stylesheets/maquina_stream/components/shimmer.css +31 -0
  9. data/app/assets/stylesheets/maquina_stream/components/snippet.css +22 -0
  10. data/app/assets/stylesheets/maquina_stream/components/source_citation.css +14 -0
  11. data/app/assets/stylesheets/maquina_stream/components/suggestion.css +32 -0
  12. data/app/assets/stylesheets/maquina_stream/reveal.css +48 -0
  13. data/app/assets/stylesheets/maquina_stream/themes/dark.css +235 -0
  14. data/app/assets/stylesheets/maquina_stream/themes/light.css +117 -0
  15. data/app/controllers/maquina_stream/application_controller.rb +21 -0
  16. data/app/controllers/maquina_stream/blocks_controller.rb +41 -0
  17. data/app/controllers/maquina_stream/manifests_controller.rb +15 -0
  18. data/app/helpers/maquina_stream/components_helper.rb +80 -0
  19. data/app/javascript/maquina_stream/controllers/application_controller.js +169 -0
  20. data/app/javascript/maquina_stream/controllers/ms_autoscroll_controller.js +110 -0
  21. data/app/javascript/maquina_stream/controllers/ms_code_controller.js +96 -0
  22. data/app/javascript/maquina_stream/controllers/ms_deferred_controller.js +223 -0
  23. data/app/javascript/maquina_stream/controllers/ms_diagram_controller.js +40 -0
  24. data/app/javascript/maquina_stream/controllers/ms_link_safety_controller.js +196 -0
  25. data/app/javascript/maquina_stream/controllers/ms_math_controller.js +32 -0
  26. data/app/javascript/maquina_stream/controllers/ms_repair_controller.js +167 -0
  27. data/app/javascript/maquina_stream/controllers/ms_reveal_controller.js +320 -0
  28. data/app/javascript/maquina_stream/controllers/ms_table_controller.js +183 -0
  29. data/app/javascript/maquina_stream/index.js +59 -0
  30. data/app/views/maquina_stream/components/_attachment.html.erb +139 -0
  31. data/app/views/maquina_stream/components/_code_block.html.erb +74 -0
  32. data/app/views/maquina_stream/components/_shimmer.html.erb +36 -0
  33. data/app/views/maquina_stream/components/_snippet.html.erb +50 -0
  34. data/app/views/maquina_stream/components/_source_citation.html.erb +42 -0
  35. data/app/views/maquina_stream/components/_suggestion.html.erb +73 -0
  36. data/config/importmap.rb +10 -0
  37. data/config/locales/en.yml +79 -0
  38. data/config/locales/es.yml +82 -0
  39. data/config/routes.rb +11 -0
  40. data/docs/configuration.md +219 -0
  41. data/docs/deferred-renderers.md +184 -0
  42. data/docs/getting-started.md +356 -0
  43. data/docs/javascript.md +298 -0
  44. data/docs/registries.md +283 -0
  45. data/docs/repair.md +162 -0
  46. data/docs/security.md +247 -0
  47. data/docs/streaming.md +308 -0
  48. data/lib/generators/maquina_stream/install/USAGE +26 -0
  49. data/lib/generators/maquina_stream/install/install_generator.rb +199 -0
  50. data/lib/generators/maquina_stream/install/templates/initializer.rb.tt +121 -0
  51. data/lib/generators/maquina_stream/streamable/USAGE +28 -0
  52. data/lib/generators/maquina_stream/streamable/streamable_generator.rb +187 -0
  53. data/lib/generators/maquina_stream/streamable/templates/migration.rb.tt +21 -0
  54. data/lib/generators/maquina_stream/streamable/templates/model.rb.tt +4 -0
  55. data/lib/maquina_stream/block.rb +99 -0
  56. data/lib/maquina_stream/broadcaster.rb +233 -0
  57. data/lib/maquina_stream/component_cache.rb +0 -0
  58. data/lib/maquina_stream/components/contract.rb +184 -0
  59. data/lib/maquina_stream/components.rb +135 -0
  60. data/lib/maquina_stream/configuration.rb +240 -0
  61. data/lib/maquina_stream/document.rb +296 -0
  62. data/lib/maquina_stream/engine.rb +46 -0
  63. data/lib/maquina_stream/errors.rb +17 -0
  64. data/lib/maquina_stream/export.rb +66 -0
  65. data/lib/maquina_stream/frame.rb +73 -0
  66. data/lib/maquina_stream/manifest.rb +137 -0
  67. data/lib/maquina_stream/registries.rb +116 -0
  68. data/lib/maquina_stream/renderer/fence.rb +115 -0
  69. data/lib/maquina_stream/renderer/post_pass.rb +363 -0
  70. data/lib/maquina_stream/renderer/tag_blocks.rb +276 -0
  71. data/lib/maquina_stream/renderer/view_context.rb +72 -0
  72. data/lib/maquina_stream/renderer.rb +128 -0
  73. data/lib/maquina_stream/sanitizer.rb +392 -0
  74. data/lib/maquina_stream/streamable.rb +281 -0
  75. data/lib/maquina_stream/text_direction.rb +56 -0
  76. data/lib/maquina_stream/themes.rb +84 -0
  77. data/lib/maquina_stream/version.rb +5 -0
  78. data/lib/maquina_stream.rb +175 -0
  79. metadata +204 -0
@@ -0,0 +1,283 @@
1
+ # Registries
2
+
3
+ Three registries let you change what the pipeline renders without touching the
4
+ engine. All three are process-global and are read during the render post-pass,
5
+ so register from an initializer:
6
+
7
+ ```ruby
8
+ # config/initializers/maquina_stream.rb
9
+ MaquinaStream.register_fence "ruby", strategy: :server
10
+ MaquinaStream.register_tag :source, attributes: %w[id href title], partial: "…"
11
+ MaquinaStream.register_element :h2, partial: "headings/h2"
12
+ ```
13
+
14
+ Whatever a registration renders still goes through the sanitizer, which is the
15
+ last pass and has no exceptions. A registry cannot inject raw HTML into a
16
+ document. See [security.md](security.md).
17
+
18
+ `test/dummy/config/initializers/maquina_stream.rb` registers all of these, and
19
+ `test/maquina_stream/registries_example_test.rb` drives that registration
20
+ through the full pipeline. Copy from there.
21
+
22
+ ## Fences
23
+
24
+ ```ruby
25
+ MaquinaStream.register_fence "ruby", strategy: :server
26
+ MaquinaStream.register_fence "text", strategy: :passthrough
27
+ MaquinaStream.register_fence "mermaid", strategy: :client,
28
+ controller: "ms-diagram",
29
+ payload: ->(source, info) { {source: source, info: info} }
30
+ ```
31
+
32
+ | Strategy | What it does | When to use it |
33
+ |---|---|---|
34
+ | `:server` | Rouge highlights the source once the fence closes, into CSS classes. The default for every unregistered language. | Anything Rouge has a lexer for. |
35
+ | `:client` | Nothing is rendered server-side. The block carries a `payload:` as a data attribute and a Stimulus `controller:` draws it in the browser. | Diagrams, math — anything whose renderer is a JavaScript library the server has no business running. |
36
+ | `:passthrough` | The source is emitted as escaped text and nothing else happens to it. | A language Rouge would mangle, or one whose highlighting is not worth the CPU. |
37
+
38
+ Two rules the pipeline enforces rather than trusts:
39
+
40
+ - **An open fence is never highlighted.** The work would be thrown away on the
41
+ next frame, and it is the difference between a 500-line fence fitting in the
42
+ frame budget and not. Do not expect syntax colours until the closing ``` lands.
43
+ - **A `:client` fence emits no payload until it closes.** Handing the client
44
+ half a diagram to draw produces an error state for text that was merely still
45
+ arriving. Until then it renders a `shimmer` skeleton.
46
+
47
+ ### `:server`, closed
48
+
49
+ ````markdown
50
+ ```ruby
51
+ puts 1
52
+ ```
53
+ ````
54
+
55
+ ```html
56
+ <div data-ms-code data-ms-code-lang="ruby" data-controller="ms-code" data-component="code-block" …>
57
+ <div data-code-block-part="header">
58
+ <span data-code-block-part="lang">ruby</span>
59
+ <span data-code-block-part="controls">
60
+ <button data-code-block-part="copy" data-ms-control data-action="ms-code#copy">Copiar</button>
61
+ <button data-code-block-part="download" data-ms-control data-action="ms-code#download">Descargar</button>
62
+ </span>
63
+ </div>
64
+ <pre data-code-block-part="pre"><code><span class="nb">puts</span> <span class="mi">1</span></code></pre>
65
+ <pre hidden data-ms-code-source>puts 1
66
+ </pre>
67
+ </div>
68
+ ```
69
+
70
+ The `<pre hidden data-ms-code-source>` carrier is what the copy and download
71
+ buttons read, so a copy hands back the raw source rather than Rouge's span
72
+ markup. It is a `<pre hidden>` and not a script tag on purpose: the sanitizer
73
+ drops every script element, and cannot tell our carrier from an imitation of it.
74
+
75
+ The same fence while still open renders the shell and the carrier, with no
76
+ highlighting spans and no control buttons.
77
+
78
+ ### `:client`, closed
79
+
80
+ ````markdown
81
+ ```mermaid
82
+ graph TD; A-->B;
83
+ ```
84
+ ````
85
+
86
+ ```html
87
+ <div data-controller="ms-diagram"
88
+ data-ms-diagram-payload-value='{"source":"graph TD; A--\u003eB;\n","info":"mermaid"}'>
89
+ <div data-ms-diagram-target="output" data-turbo-permanent>…shimmer…</div>
90
+ </div>
91
+ ```
92
+
93
+ Split ownership: the payload attribute is server state and belongs to morph; the
94
+ output element is client state, is `data-turbo-permanent`, and belongs to the
95
+ controller. See [deferred-renderers.md](deferred-renderers.md).
96
+
97
+ While the fence is open there is no payload and no controller at all — only the
98
+ shimmer.
99
+
100
+ ### `:passthrough`
101
+
102
+ ````markdown
103
+ ```text
104
+ as is
105
+ ```
106
+ ````
107
+
108
+ ```html
109
+ <pre data-ms-element="pre"><code class="language-text">as is
110
+ </code></pre>
111
+ ```
112
+
113
+ ## Custom tags
114
+
115
+ Models emit XML-ish tags that mean something to your application rather than to
116
+ markdown: `<source>`, `<citation>`, `<thinking>`. Register one and the post-pass
117
+ replaces its node with your partial.
118
+
119
+ The reference case is a model citing its sources:
120
+
121
+ ```ruby
122
+ MaquinaStream.register_tag :source,
123
+ attributes: %w[id href title],
124
+ partial: "maquina_stream/components/source_citation",
125
+ literal_content: false
126
+ ```
127
+
128
+ | Option | Meaning |
129
+ |---|---|
130
+ | `attributes:` | The attribute names the tag may carry. Anything else on it is dropped before the partial is called. |
131
+ | `partial:` | The partial that renders it. Registered attribute names arrive as locals, so they must match the locals it declares — `href`, not `url`. |
132
+ | `literal_content:` | `true` passes the tag's body as text; `false` renders it as markdown. |
133
+
134
+ ```markdown
135
+ Según la fuente <source id="3" href="https://example.com/a" title="Un artículo" onclick="alert(1)"></source>.
136
+ ```
137
+
138
+ ```html
139
+ <p data-ms-element="p">Según la fuente <span data-component="source-citation" data-ms-source-id="3" …>
140
+ <a data-source-citation-part="link" href="https://example.com/a" rel="noopener noreferrer">Un artículo</a>
141
+ </span>.</p>
142
+ ```
143
+
144
+ `onclick` never reaches the partial at all — it is gone before the sanitizer is
145
+ even asked, because the registration did not list it.
146
+
147
+ ### An unregistered tag does not survive; its content does
148
+
149
+ The sanitizer *unwraps* an element it does not know: the tag goes, its children
150
+ stay.
151
+
152
+ ```markdown
153
+ text <danger id="1">content</danger> more
154
+ ```
155
+
156
+ ```html
157
+ <p data-ms-element="p">text content more</p>
158
+ ```
159
+
160
+ Text stranded at the top level this way is wrapped into a `<p>` before the
161
+ document is split, so it lands in a block of its own with an id and a digest,
162
+ and morphs and repairs like any other block. Its id comes from the first free
163
+ index at or after its position, so ids stay unique but are not always in
164
+ ascending order.
165
+
166
+ Mid-stream this is stable. `maquina_remend` removes only a tag whose `>` has not
167
+ arrived yet (`"text <thinki"` → `"text"`), the sanitizer unwraps the tag whether
168
+ it closed or not, and closing the tag changes no block above it — so nothing
169
+ flickers and no sealed block is rewritten.
170
+
171
+ ### Block-level tags
172
+
173
+ A registered tag can also wrap several paragraphs. The partial receives the
174
+ whole thing as its `content`, rendered as markdown:
175
+
176
+ ```ruby
177
+ MaquinaStream.register_tag :thinking, attributes: [], partial: "tags/reasoning"
178
+ ```
179
+
180
+ ```erb
181
+ <%# app/views/tags/_reasoning.html.erb %>
182
+ <%# locals: (content: "") %>
183
+ <section data-ms-reasoning><%= content.to_s.html_safe %></section>
184
+ ```
185
+
186
+ ```markdown
187
+ <thinking>
188
+ One.
189
+
190
+ Two.
191
+ </thinking>
192
+
193
+ After.
194
+ ```
195
+
196
+ ```html
197
+ <section data-ms-reasoning id="ms-7-b0" data-ms-block data-ms-block-digest="434332299dedf830">
198
+ <p data-ms-element="p">One.</p>
199
+ <p data-ms-element="p">Two.</p>
200
+ </section>
201
+ <p id="ms-7-b1" data-ms-element="p" data-ms-block data-ms-block-digest="e44dfc18531833b6">After.</p>
202
+ ```
203
+
204
+ Two things to know before you reach for this:
205
+
206
+ 1. **The whole tag is one block.** Four paragraphs of reasoning are one id, one
207
+ digest and one unit of repair — not four. If you want them to seal and
208
+ repair separately, leave the tag unregistered and style the blocks instead.
209
+ 2. **That block cannot seal until the tag closes**, so everything inside it is
210
+ re-sent on every frame. A four-paragraph body costs about 30% more bytes on
211
+ the wire than the same text unregistered; a twenty-paragraph body costs 3.5x,
212
+ and the single block is 88% of it. Keep block-level tags short, or accept the
213
+ cost at the tail.
214
+ 3. **Only registered names are treated this way.** With an empty registry the
215
+ buffer is untouched, and a `<thinking>` inside a fence or a backtick span
216
+ stays text:
217
+
218
+ ````markdown
219
+ ```text
220
+ <thinking>x</thinking>
221
+ ```
222
+ ````
223
+
224
+ ```html
225
+ <pre data-ms-element="pre"><code class="language-text">&lt;thinking&gt;x&lt;/thinking&gt;
226
+ </code></pre>
227
+ ```
228
+
229
+ ### Where block-level handling applies
230
+
231
+ Only to names you passed to `register_tag`, and only where the opening tag
232
+ begins a line under four columns of indent — the shape that starts an HTML
233
+ block. An inline `<citation>…</citation>` in the middle of a sentence is left
234
+ exactly as the model wrote it, because it already renders correctly.
235
+
236
+ It never applies inside a fenced code block, an inline code span, an indented
237
+ code block, an HTML comment, `<script>`, `<pre>`, `<style>`, `<textarea>`, CDATA
238
+ or a processing instruction — that text is content, not markup. A tag broken
239
+ across a newline, one whose `>` has not arrived, an opener with no closer, a
240
+ closer with no opener, and `<thinking/>` are all left alone.
241
+
242
+ One limitation follows from the indent rule: **a block-level registered tag
243
+ inside a list item is not supported.** Put it at the top level.
244
+
245
+ ## Element overrides
246
+
247
+ Replace the markup the renderer produces for one element with a partial of your
248
+ own:
249
+
250
+ ```ruby
251
+ MaquinaStream.register_element :h2, partial: "headings/h2"
252
+ ```
253
+
254
+ The partial receives `content:` (the element's inner HTML) and `node:` (the
255
+ Nokogiri node), so it must declare both:
256
+
257
+ ```erb
258
+ <%# app/views/headings/_h2.html.erb %>
259
+ <%# locals: (content: "", node: nil) %>
260
+ <h2 class="section-heading"><span aria-hidden="true">§</span> <%= content.to_s.html_safe %></h2>
261
+ ```
262
+
263
+ ```markdown
264
+ ## Thinking
265
+ ```
266
+
267
+ ```html
268
+ <h2 class="section-heading"><span aria-hidden="true">§</span> Thinking<a href="#thinking" class="anchor" …></a></h2>
269
+ ```
270
+
271
+ Registering the same element twice replaces the first registration; the last one
272
+ in wins. Elements you do not override carry `data-ms-element="<tag>"` as a
273
+ styling hook, which is usually enough — reach for an override when you need
274
+ different structure, not different CSS.
275
+
276
+ ## Resetting
277
+
278
+ ```ruby
279
+ MaquinaStream.reset_registries!
280
+ ```
281
+
282
+ For tests. Called in production it loses every registration your initializer
283
+ made.
data/docs/repair.md ADDED
@@ -0,0 +1,162 @@
1
+ # Repair
2
+
3
+ ## What it is for
4
+
5
+ Action Cable gives no delivery guarantee, no ordering guarantee and no gap
6
+ detection. A frame can simply not arrive: a flaky connection, a backgrounded
7
+ tab, a reconnect in the middle of a message. Deltas are an optimization, and
8
+ they do not converge on their own — only the open tail is patched, so a block
9
+ that changes after it has stopped being the tail is never re-sent.
10
+
11
+ The repair path is where correctness lives. The browser periodically asks the
12
+ server what the message currently is, compares that against its own DOM, and
13
+ asks for the blocks that differ.
14
+
15
+ If you fix a correctness bug inside the delta path, you have fixed it in the
16
+ wrong place.
17
+
18
+ ## What you must implement
19
+
20
+ Three things, and then it runs on its own.
21
+
22
+ ### 1. Mount the engine
23
+
24
+ ```ruby
25
+ # config/routes.rb
26
+ mount MaquinaStream::Engine => "/maquina_stream"
27
+ ```
28
+
29
+ ### 2. Configure the two seams
30
+
31
+ ```ruby
32
+ # config/initializers/maquina_stream.rb
33
+ MaquinaStream.configure do |c|
34
+ c.find_stream = ->(sid) { Message.find_by(id: sid) }
35
+ c.authorize = ->(record, request) { record.conversation.readable_by?(request) }
36
+ end
37
+ ```
38
+
39
+ `find_stream` receives the `:sid` from the URL — that is `#maquina_stream_id`,
40
+ which defaults to `to_param` — and returns a record or `nil`. Return `nil` and
41
+ the engine answers `404`. Leave it unset and every engine request raises
42
+ `MaquinaStream::ConfigurationError`: the engine does not go looking for a model
43
+ on its own.
44
+
45
+ `authorize` receives the record and the `ActionDispatch::Request`. Anything
46
+ falsy answers `403`. **Leaving it unset denies everything**; there is no
47
+ permissive default, because an engine that guesses is an engine that leaks.
48
+
49
+ Scope the lookup rather than relying on `authorize` alone where you can — the
50
+ same advice as any Rails controller:
51
+
52
+ ```ruby
53
+ c.find_stream = ->(sid) { Current.user.messages.find_by(id: sid) }
54
+ ```
55
+
56
+ ### 3. Put the repair URLs on the message element
57
+
58
+ ```erb
59
+ <div id="ms-msg-<%= message.maquina_stream_id %>"
60
+ data-controller="ms-repair"
61
+ data-ms-repair-manifest-url-value="<%= maquina_stream.manifest_path(sid: message.maquina_stream_id) %>"
62
+ data-ms-repair-blocks-url-value="<%= maquina_stream.blocks_path(sid: message.maquina_stream_id) %>"
63
+ data-ms-repair-interval-value="4000"><%= MaquinaStream.render(message) %></div>
64
+ ```
65
+
66
+ `maquina_stream.` is the mounted engine's route proxy, so the paths follow
67
+ wherever you mounted it. `data-ms-repair-interval-value` defaults to 4000ms;
68
+ `0` turns the keyframe timer off and leaves the other three triggers.
69
+
70
+ That is all. The blocks the engine renders already carry the `id`,
71
+ `data-ms-block` and `data-ms-block-digest` the controller diffs on.
72
+
73
+ ## The two routes
74
+
75
+ ### `GET /maquina_stream/:sid/manifest`
76
+
77
+ What the browser is told the message currently *is*, in a bounded number of
78
+ bytes. Not HTML.
79
+
80
+ ```json
81
+ {"seq":7,"cutoff":0,"rollup":"e3b0c44298fc1c14",
82
+ "blocks":[["ms-42-b0","999bd5bd2841c435"],["ms-42-b1","6c7c33d9dcefa12c"]]}
83
+ ```
84
+
85
+ | Key | Meaning |
86
+ |---|---|
87
+ | `seq` | the sequence number this manifest describes |
88
+ | `cutoff` | how many sealed blocks fall behind the window |
89
+ | `rollup` | one digest covering every block behind the cutoff |
90
+ | `blocks` | `[[id, digest], …]` for the blocks inside the window |
91
+
92
+ Only sealed blocks are listed — an open block is about to change, so there is
93
+ nothing to reconcile it against. With the default `seal_lag` of 2, the last two
94
+ blocks of a message are absent from the manifest by design.
95
+
96
+ The manifest is windowed rather than complete because listing every sealed block
97
+ made it track message length almost exactly: 1.8KB for a 2KB message, 88KB for a
98
+ 100KB one, sent every keyframe. The last `manifest_window` sealed blocks go in
99
+ full, and one rollup digest covers everything older. A client whose rollup
100
+ matches knows its history is intact and only has to consider the window; a
101
+ client whose rollup differs asks for the whole thing with `?full=1`, which is
102
+ rare and no more expensive than the cold page load it resembles.
103
+
104
+ ### `GET /maquina_stream/:sid/blocks?ids[]=…`
105
+
106
+ The blocks the client asked for, as morphing Turbo Stream actions.
107
+
108
+ ```html
109
+ <turbo-stream method="morph" action="replace" target="ms-42-b1">
110
+ <template><p id="ms-42-b1" data-ms-element="p" data-ms-block data-ms-block-digest="6c7c33d9dcefa12c">Dos</p></template>
111
+ </turbo-stream>
112
+ ```
113
+
114
+ `method="morph"` is what makes the repair silent: idiomorph patches the existing
115
+ node in place, so client state inside it survives and no animation fires for a
116
+ replacement that never happens.
117
+
118
+ `ids` comes from the client, so it is filtered against the document rather than
119
+ trusted — only ids the message actually has are served, and only the ones
120
+ requested. A request for every block is a legitimate cold repair, so the count
121
+ is not capped, but each id is matched, never interpolated.
122
+
123
+ Asking to repair nothing is a normal answer to a manifest that already agreed,
124
+ and returns an empty stream rather than an error.
125
+
126
+ ## What the browser does with them
127
+
128
+ `ms-repair` fetches on four triggers:
129
+
130
+ 1. the final seal, always;
131
+ 2. a gap in the sequence — every stream action carries `data-ms-seq`;
132
+ 3. a reconnect, or the tab becoming visible again;
133
+ 4. the periodic keyframe.
134
+
135
+ It compares the manifest against its own DOM, requests only the blocks whose
136
+ digests differ, and morphs them in. Repair therefore costs what has drifted, not
137
+ what the message weighs.
138
+
139
+ Before the morph it dispatches `ms:suppress` on the message element and
140
+ `ms:resume` after it, so the reveal animation unwraps whatever was mid-flight
141
+ and re-baselines afterwards — a repair never re-reveals text the reader has
142
+ already read.
143
+
144
+ It dispatches `ms:repaired` (with `{reason, blocks}`) and `ms:repair-failed`
145
+ (with `{reason, error}`) if you want to observe it. See
146
+ [javascript.md](javascript.md).
147
+
148
+ ## Responses you should expect
149
+
150
+ | Situation | Response |
151
+ |---|---|
152
+ | `find_stream` returns `nil` | `404` |
153
+ | `authorize` returns falsy | `403` |
154
+ | `authorize` not configured | `403` |
155
+ | `find_stream` not configured | `MaquinaStream::ConfigurationError` |
156
+
157
+ ## Transport independence
158
+
159
+ The repair path is plain `GET`, so a client that lost frames recovers over HTTP
160
+ no matter how those frames were delivered. Frames, sequence numbers, the
161
+ manifest and these two routes are transport-independent by construction — which
162
+ is what makes `config.transport` a seam worth having rather than a setting.
data/docs/security.md ADDED
@@ -0,0 +1,247 @@
1
+ # Security
2
+
3
+ Everything this engine renders is model output, and model output is
4
+ prompt-injectable by definition. A document that reached the sanitizer having
5
+ "already been checked" has not been checked — it has been parsed.
6
+
7
+ ## The one rule for a host
8
+
9
+ **Never assign model output as raw HTML.**
10
+
11
+ The engine's own output is `html_safe` because it has been through the
12
+ sanitizer. Anything else — a buffer, a payload, a tool result, a value you read
13
+ back out of rendered markup in JavaScript — is not. Do not `html_safe` it, do
14
+ not `innerHTML` it, do not hand it to a template that will.
15
+
16
+ ```erb
17
+ <%= MaquinaStream.render(message) %> <%# sanitized on the way out %>
18
+ <%= raw message.content %> <%# never %>
19
+ ```
20
+
21
+ ## The sanitizer
22
+
23
+ ```ruby
24
+ MaquinaStream::Sanitizer.call(html, config: MaquinaStream.config) # => String
25
+ ```
26
+
27
+ It is the last pass in the render chain — `maquina_remend → CommonMarker →
28
+ Nokogiri post-pass → Sanitizer` — and the only one that assumes the document is
29
+ hostile. It runs unconditionally: no mode, no flag, no fast path skips it. It is
30
+ a pure function, with no Rails and no request.
31
+
32
+ Nokogiri's **HTML5** parser builds the tree, because a sanitizer that parses
33
+ differently from the engine that will render the output is a mutation-XSS bug
34
+ waiting for its input.
35
+
36
+ ```
37
+ in : <a href="javascript:alert(1)">clic</a>
38
+ out: <a>clic</a>
39
+
40
+ in : <img src="x" onerror="alert(1)">
41
+ out: <img src="x">
42
+
43
+ in : <script>alert(1)</script>hola
44
+ out: hola
45
+ ```
46
+
47
+ ### How it decides
48
+
49
+ 1. An element on the drop list — or in the SVG or MathML namespace — is removed
50
+ **with its subtree**.
51
+ 2. An element not on the allowlist is **unwrapped**: the element goes, its
52
+ sanitized children stay. Text survives; markup does not.
53
+ 3. On a surviving element, each attribute must be named by the allowlist.
54
+ Everything else is dropped, not escaped and kept.
55
+ 4. `href` and `src` are re-parsed and re-checked. A rejected `href` leaves the
56
+ link with its text; a rejected `src` removes the `<img>` entirely, because a
57
+ broken rectangle carrying an attacker-chosen `alt` is worse than nothing.
58
+ 5. Comments, processing instructions and doctypes are removed. CDATA becomes
59
+ text.
60
+
61
+ ### What survives
62
+
63
+ **Elements** — rendered markdown plus the post-pass wrappers: headings, `p`,
64
+ `div`, `span`, `br`, `hr`, lists (`ul ol li dl dt dd`), tables
65
+ (`table thead tbody tfoot tr td th caption colgroup col`),
66
+ `pre code kbd samp var`, `blockquote figure figcaption details summary section
67
+ article aside`, `a`, `img`, the inline set (`em strong b i u s del ins mark
68
+ small sub sup q abbr dfn cite time wbr`), and `input` — the last only as the
69
+ tasklist checkbox, forced `disabled`.
70
+
71
+ **Attributes**
72
+
73
+ | Scope | Allowed |
74
+ |---|---|
75
+ | Global | `id class title lang dir role translate`, plus `aria-*` |
76
+ | `a` | `href target rel hreflang type` |
77
+ | `img` | `src alt width height loading decoding` |
78
+ | Lists and tables | `start reversed type value colspan rowspan align valign headers scope abbr span` |
79
+ | `time`, `details` | `datetime`, `open` |
80
+ | The engine's | every `data-ms-*`, plus `data-component data-variant data-size data-slot data-state data-side data-orientation data-controller data-action data-turbo-permanent data-turbo-temporary` |
81
+
82
+ `data-controller` and `data-action` are filtered **by value**: only `ms-`
83
+ identifiers survive, and only action descriptors naming one. A third-party
84
+ controller's `data-<name>-*-value` attributes are dropped with it, so an
85
+ injected controller arrives with no configuration:
86
+
87
+ ```
88
+ in : <div data-controller="evil" data-evil-url-value="x">hi</div>
89
+ out: <div>hi</div>
90
+ ```
91
+
92
+ Anchors keep `target` normalized to `_blank`/`_self` and always gain
93
+ `rel="noopener noreferrer"`.
94
+
95
+ ### What is dropped
96
+
97
+ - Every `on*` attribute, whatever its case, and whatever the parser made of a
98
+ name split across a newline.
99
+ - `srcdoc`, `formaction`, `style`, `action`, `http-equiv`, `background`, `ping`,
100
+ `srcset`, `usemap`, `name`, `contenteditable`, `accesskey`.
101
+ - Every namespaced attribute — `xlink:href`, `xml:base` — without exception.
102
+ - `script style svg math template noscript iframe object embed form button
103
+ select textarea base link meta …`, each with its subtree.
104
+
105
+ That last one includes any `<script type="text/plain">` carrier: the sanitizer
106
+ cannot tell yours from an imitation of it, and Nokogiri's HTML5 serializer
107
+ writes script children unescaped, so a fence containing
108
+ `</script><img onerror=…>` would break out on the next parse. **A code block
109
+ that needs its raw source in the DOM must carry it on a non-script element** —
110
+ which is why the engine's carrier is a `<pre hidden data-ms-code-source>`.
111
+
112
+ ### URL hardening
113
+
114
+ Applied to `href` and `src`, in order:
115
+
116
+ 1. Control characters and Unicode whitespace are stripped first, so the scheme
117
+ tested is the scheme a browser would act on (`java\tscript:`,
118
+ `java\nscript:`).
119
+ 2. `//host`, `\\host`, `/\host` — protocol-relative in every spelling — are
120
+ rejected.
121
+ 3. The scheme is read from the raw value **and** from a decoded copy (HTML
122
+ entities, then percent-encoding). `java&#115;cript:`, `&amp;#106;avascript:`,
123
+ `%6Aavascript:` and `JaVaScRiPt:` all resolve to a dangerous scheme and are
124
+ rejected.
125
+ 4. The scheme must appear in `config.allowed_protocols` (default `http https
126
+ mailto`).
127
+ 5. A relative URL is resolved against `config.default_origin` when one is set,
128
+ and the result must still be on an allowed protocol. With no origin set it is
129
+ left as written. A bare `#fragment` is always kept.
130
+ 6. The final URL must start with one of `config.allowed_link_prefixes` /
131
+ `allowed_image_prefixes`. `["*"]`, the default, means any.
132
+
133
+ `data:` is special-cased: image positions only, only when
134
+ `config.allow_data_images`, and only base64 raster types.
135
+
136
+ ```
137
+ allow_data_images = true <img src="data:image/png;base64,iVBORw0KGgo="> → kept
138
+ allow_data_images = false <img src="data:image/png;base64,iVBORw0KGgo="> → removed
139
+ either <img src="data:image/svg+xml;base64,…"> → removed
140
+ ```
141
+
142
+ `data:image/svg+xml` is a scriptable document wearing an image's MIME type and
143
+ never passes, whatever you configure.
144
+
145
+ Tightening the two prefix lists is the main lever you have over what a model may
146
+ link to. See [configuration.md](configuration.md).
147
+
148
+ ## The markdown normalisation pass
149
+
150
+ One pass runs between `maquina_remend` and the markdown parser, and it can only
151
+ insert newlines: around the opening and closing tags of tags you registered with
152
+ `register_tag`, so a block-level `<thinking>` parses as its own HTML block
153
+ instead of leaving a stray end tag inside a paragraph. Without it, the parser
154
+ never closes the element and the rest of the message is handed to your partial.
155
+
156
+ It is deliberately narrow, because it changes how model-written text is parsed:
157
+
158
+ - The allowlist is your `register_tag` registry. An unregistered tag is never
159
+ matched and never moved.
160
+ - Fenced blocks and inline code win — matching runs over masked text, so a
161
+ registered name inside them is content.
162
+ - CommonMark's raw HTML regions — comments, `<script>`, `<pre>`, `<style>`,
163
+ `<textarea>`, CDATA, processing instructions — are masked too. A blank line
164
+ inserted inside one of those would end it early and publish what it hid.
165
+ - Indented code is excluded by an indent rule.
166
+ - A match must be a complete, well-formed tag on one line, parsed with
167
+ CommonMark's attribute grammar, so a `>` inside a quoted value cannot make an
168
+ insertion land mid-tag.
169
+
170
+ Because it only inserts newlines, it can promote model text from raw HTML into
171
+ markdown, or split a paragraph — but it cannot introduce an element or an
172
+ attribute the allowlist does not name. The sanitizer runs unconditionally,
173
+ last, over the parsed HTML. The whole XSS corpus is additionally run through the
174
+ renderer with tags registered.
175
+
176
+ ## Why the client sanitizes again
177
+
178
+ The server pass proves one thing: the HTML **the engine** serializes is safe. It
179
+ cannot speak for HTML the browser builds afterwards.
180
+
181
+ - Client-deferred renderers (`ms-diagram`, `ms-math`) receive a JSON payload and
182
+ produce new DOM from it. That payload is attacker-influenced text which passed
183
+ through the sanitizer as an attribute *value*, never as markup — nothing has
184
+ sanitized its output until the controller does. So `ms-deferred` applies its
185
+ own allowlist to whatever the library returns.
186
+ - `data-controller` is an allowlisted attribute. The sanitizer restricts it to
187
+ the `ms-` namespace, but cannot tell a controller the post-pass emitted from
188
+ one an injected fragment asked for. Every `ms-` controller therefore treats its
189
+ own values as untrusted input rather than as server intent.
190
+ - Serialize-and-reparse is where mutation XSS lives. Two parsers, two rounds of
191
+ entity decoding and one namespace boundary are enough to turn inert text into
192
+ markup.
193
+
194
+ Server-side escaping protects the attribute boundary. It does not protect
195
+ whatever the client does with the value inside it.
196
+
197
+ ## Renderer posture
198
+
199
+ | Renderer | Setting | What it buys |
200
+ |---|---|---|
201
+ | `ms-diagram` (Mermaid) | `securityLevel: "strict"` | disables click handlers and inline HTML in diagram source |
202
+ | `ms-math` (KaTeX) | `trust: false` | refuses `\htmlClass`, `\includegraphics` and `\href`, all of which take attacker-controlled strings into the DOM |
203
+ | both | output allowlist | the library is third-party; its output is scrubbed before it reaches the DOM |
204
+
205
+ ## Limits
206
+
207
+ - **It sanitizes HTML, not meaning.** A model that writes a plausible phishing
208
+ link to an allowed host produces a link the sanitizer will happily keep. That
209
+ is what `allowed_link_prefixes` and the link-safety dialog are for.
210
+ - **A registered partial is yours to get right.** Registry output still goes
211
+ through the sanitizer, so it cannot introduce an element or attribute the
212
+ allowlist does not name — but a partial that renders an attacker-supplied
213
+ string into an allowed attribute is a hole the allowlist cannot see.
214
+ - **Your own controllers are outside its reach.** A controller of yours that
215
+ reads a value out of rendered markup and assigns it as HTML has undone the
216
+ whole chain.
217
+ - **The broadcast target is yours.** The sanitizer has nothing to say about who
218
+ may subscribe to a Turbo stream.
219
+
220
+ ## Extending the allowlist
221
+
222
+ Everything lives in constants at the top of `lib/maquina_stream/sanitizer.rb`:
223
+
224
+ - a new element → `ALLOWED_ELEMENTS`, or `DROP_WITH_CONTENT` if its contents
225
+ must not survive it;
226
+ - a new attribute → `GLOBAL_ATTRIBUTES`, or the element's entry in
227
+ `ELEMENT_ATTRIBUTES`;
228
+ - a new hook of your own → name it `data-ms-*` and it is already allowed;
229
+ - a new URL-bearing attribute → give it a branch in `scrub_attribute_value` that
230
+ runs it through `safe_url`, never a bare entry in the allowlist. An unhardened
231
+ URL attribute is the whole bug.
232
+
233
+ Every change comes with a corpus file. `test/fixtures/xss/*.txt` holds one
234
+ attack per file:
235
+
236
+ ```
237
+ --- input
238
+ <a href="javascript:alert(1)">clic</a>
239
+ --- note
240
+ javascript: link. The anchor text survives; the href must be dropped.
241
+ ```
242
+
243
+ The suite turns each file into its own test case at load time, so adding a file
244
+ adds a test and there is no way to add an attack that is quietly not run. Every
245
+ entry is asserted to produce no script element, no `on*` attribute, no forbidden
246
+ attribute and no dangerous URL scheme, and to be a fixed point under a second
247
+ sanitization pass.