maquina_stream 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.rdoc_options +30 -0
- data/CHANGELOG.md +38 -0
- data/LICENSE.txt +21 -0
- data/README.md +163 -0
- data/app/assets/stylesheets/maquina_stream/components/attachment.css +35 -0
- data/app/assets/stylesheets/maquina_stream/components/code_block.css +30 -0
- data/app/assets/stylesheets/maquina_stream/components/shimmer.css +31 -0
- data/app/assets/stylesheets/maquina_stream/components/snippet.css +22 -0
- data/app/assets/stylesheets/maquina_stream/components/source_citation.css +14 -0
- data/app/assets/stylesheets/maquina_stream/components/suggestion.css +32 -0
- data/app/assets/stylesheets/maquina_stream/reveal.css +48 -0
- data/app/assets/stylesheets/maquina_stream/themes/dark.css +235 -0
- data/app/assets/stylesheets/maquina_stream/themes/light.css +117 -0
- data/app/controllers/maquina_stream/application_controller.rb +21 -0
- data/app/controllers/maquina_stream/blocks_controller.rb +41 -0
- data/app/controllers/maquina_stream/manifests_controller.rb +15 -0
- data/app/helpers/maquina_stream/components_helper.rb +80 -0
- data/app/javascript/maquina_stream/controllers/application_controller.js +169 -0
- data/app/javascript/maquina_stream/controllers/ms_autoscroll_controller.js +110 -0
- data/app/javascript/maquina_stream/controllers/ms_code_controller.js +96 -0
- data/app/javascript/maquina_stream/controllers/ms_deferred_controller.js +223 -0
- data/app/javascript/maquina_stream/controllers/ms_diagram_controller.js +40 -0
- data/app/javascript/maquina_stream/controllers/ms_link_safety_controller.js +196 -0
- data/app/javascript/maquina_stream/controllers/ms_math_controller.js +32 -0
- data/app/javascript/maquina_stream/controllers/ms_repair_controller.js +167 -0
- data/app/javascript/maquina_stream/controllers/ms_reveal_controller.js +320 -0
- data/app/javascript/maquina_stream/controllers/ms_table_controller.js +183 -0
- data/app/javascript/maquina_stream/index.js +59 -0
- data/app/views/maquina_stream/components/_attachment.html.erb +139 -0
- data/app/views/maquina_stream/components/_code_block.html.erb +74 -0
- data/app/views/maquina_stream/components/_shimmer.html.erb +36 -0
- data/app/views/maquina_stream/components/_snippet.html.erb +50 -0
- data/app/views/maquina_stream/components/_source_citation.html.erb +42 -0
- data/app/views/maquina_stream/components/_suggestion.html.erb +73 -0
- data/config/importmap.rb +10 -0
- data/config/locales/en.yml +79 -0
- data/config/locales/es.yml +82 -0
- data/config/routes.rb +11 -0
- data/docs/configuration.md +219 -0
- data/docs/deferred-renderers.md +184 -0
- data/docs/getting-started.md +356 -0
- data/docs/javascript.md +298 -0
- data/docs/registries.md +283 -0
- data/docs/repair.md +162 -0
- data/docs/security.md +247 -0
- data/docs/streaming.md +308 -0
- data/lib/generators/maquina_stream/install/USAGE +26 -0
- data/lib/generators/maquina_stream/install/install_generator.rb +199 -0
- data/lib/generators/maquina_stream/install/templates/initializer.rb.tt +121 -0
- data/lib/generators/maquina_stream/streamable/USAGE +28 -0
- data/lib/generators/maquina_stream/streamable/streamable_generator.rb +187 -0
- data/lib/generators/maquina_stream/streamable/templates/migration.rb.tt +21 -0
- data/lib/generators/maquina_stream/streamable/templates/model.rb.tt +4 -0
- data/lib/maquina_stream/block.rb +99 -0
- data/lib/maquina_stream/broadcaster.rb +233 -0
- data/lib/maquina_stream/component_cache.rb +0 -0
- data/lib/maquina_stream/components/contract.rb +184 -0
- data/lib/maquina_stream/components.rb +135 -0
- data/lib/maquina_stream/configuration.rb +240 -0
- data/lib/maquina_stream/document.rb +296 -0
- data/lib/maquina_stream/engine.rb +46 -0
- data/lib/maquina_stream/errors.rb +17 -0
- data/lib/maquina_stream/export.rb +66 -0
- data/lib/maquina_stream/frame.rb +73 -0
- data/lib/maquina_stream/manifest.rb +137 -0
- data/lib/maquina_stream/registries.rb +116 -0
- data/lib/maquina_stream/renderer/fence.rb +115 -0
- data/lib/maquina_stream/renderer/post_pass.rb +363 -0
- data/lib/maquina_stream/renderer/tag_blocks.rb +276 -0
- data/lib/maquina_stream/renderer/view_context.rb +72 -0
- data/lib/maquina_stream/renderer.rb +128 -0
- data/lib/maquina_stream/sanitizer.rb +392 -0
- data/lib/maquina_stream/streamable.rb +281 -0
- data/lib/maquina_stream/text_direction.rb +56 -0
- data/lib/maquina_stream/themes.rb +84 -0
- data/lib/maquina_stream/version.rb +5 -0
- data/lib/maquina_stream.rb +175 -0
- metadata +204 -0
data/docs/registries.md
ADDED
|
@@ -0,0 +1,283 @@
|
|
|
1
|
+
# Registries
|
|
2
|
+
|
|
3
|
+
Three registries let you change what the pipeline renders without touching the
|
|
4
|
+
engine. All three are process-global and are read during the render post-pass,
|
|
5
|
+
so register from an initializer:
|
|
6
|
+
|
|
7
|
+
```ruby
|
|
8
|
+
# config/initializers/maquina_stream.rb
|
|
9
|
+
MaquinaStream.register_fence "ruby", strategy: :server
|
|
10
|
+
MaquinaStream.register_tag :source, attributes: %w[id href title], partial: "…"
|
|
11
|
+
MaquinaStream.register_element :h2, partial: "headings/h2"
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Whatever a registration renders still goes through the sanitizer, which is the
|
|
15
|
+
last pass and has no exceptions. A registry cannot inject raw HTML into a
|
|
16
|
+
document. See [security.md](security.md).
|
|
17
|
+
|
|
18
|
+
`test/dummy/config/initializers/maquina_stream.rb` registers all of these, and
|
|
19
|
+
`test/maquina_stream/registries_example_test.rb` drives that registration
|
|
20
|
+
through the full pipeline. Copy from there.
|
|
21
|
+
|
|
22
|
+
## Fences
|
|
23
|
+
|
|
24
|
+
```ruby
|
|
25
|
+
MaquinaStream.register_fence "ruby", strategy: :server
|
|
26
|
+
MaquinaStream.register_fence "text", strategy: :passthrough
|
|
27
|
+
MaquinaStream.register_fence "mermaid", strategy: :client,
|
|
28
|
+
controller: "ms-diagram",
|
|
29
|
+
payload: ->(source, info) { {source: source, info: info} }
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
| Strategy | What it does | When to use it |
|
|
33
|
+
|---|---|---|
|
|
34
|
+
| `:server` | Rouge highlights the source once the fence closes, into CSS classes. The default for every unregistered language. | Anything Rouge has a lexer for. |
|
|
35
|
+
| `:client` | Nothing is rendered server-side. The block carries a `payload:` as a data attribute and a Stimulus `controller:` draws it in the browser. | Diagrams, math — anything whose renderer is a JavaScript library the server has no business running. |
|
|
36
|
+
| `:passthrough` | The source is emitted as escaped text and nothing else happens to it. | A language Rouge would mangle, or one whose highlighting is not worth the CPU. |
|
|
37
|
+
|
|
38
|
+
Two rules the pipeline enforces rather than trusts:
|
|
39
|
+
|
|
40
|
+
- **An open fence is never highlighted.** The work would be thrown away on the
|
|
41
|
+
next frame, and it is the difference between a 500-line fence fitting in the
|
|
42
|
+
frame budget and not. Do not expect syntax colours until the closing ``` lands.
|
|
43
|
+
- **A `:client` fence emits no payload until it closes.** Handing the client
|
|
44
|
+
half a diagram to draw produces an error state for text that was merely still
|
|
45
|
+
arriving. Until then it renders a `shimmer` skeleton.
|
|
46
|
+
|
|
47
|
+
### `:server`, closed
|
|
48
|
+
|
|
49
|
+
````markdown
|
|
50
|
+
```ruby
|
|
51
|
+
puts 1
|
|
52
|
+
```
|
|
53
|
+
````
|
|
54
|
+
|
|
55
|
+
```html
|
|
56
|
+
<div data-ms-code data-ms-code-lang="ruby" data-controller="ms-code" data-component="code-block" …>
|
|
57
|
+
<div data-code-block-part="header">
|
|
58
|
+
<span data-code-block-part="lang">ruby</span>
|
|
59
|
+
<span data-code-block-part="controls">
|
|
60
|
+
<button data-code-block-part="copy" data-ms-control data-action="ms-code#copy">Copiar</button>
|
|
61
|
+
<button data-code-block-part="download" data-ms-control data-action="ms-code#download">Descargar</button>
|
|
62
|
+
</span>
|
|
63
|
+
</div>
|
|
64
|
+
<pre data-code-block-part="pre"><code><span class="nb">puts</span> <span class="mi">1</span></code></pre>
|
|
65
|
+
<pre hidden data-ms-code-source>puts 1
|
|
66
|
+
</pre>
|
|
67
|
+
</div>
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
The `<pre hidden data-ms-code-source>` carrier is what the copy and download
|
|
71
|
+
buttons read, so a copy hands back the raw source rather than Rouge's span
|
|
72
|
+
markup. It is a `<pre hidden>` and not a script tag on purpose: the sanitizer
|
|
73
|
+
drops every script element, and cannot tell our carrier from an imitation of it.
|
|
74
|
+
|
|
75
|
+
The same fence while still open renders the shell and the carrier, with no
|
|
76
|
+
highlighting spans and no control buttons.
|
|
77
|
+
|
|
78
|
+
### `:client`, closed
|
|
79
|
+
|
|
80
|
+
````markdown
|
|
81
|
+
```mermaid
|
|
82
|
+
graph TD; A-->B;
|
|
83
|
+
```
|
|
84
|
+
````
|
|
85
|
+
|
|
86
|
+
```html
|
|
87
|
+
<div data-controller="ms-diagram"
|
|
88
|
+
data-ms-diagram-payload-value='{"source":"graph TD; A--\u003eB;\n","info":"mermaid"}'>
|
|
89
|
+
<div data-ms-diagram-target="output" data-turbo-permanent>…shimmer…</div>
|
|
90
|
+
</div>
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Split ownership: the payload attribute is server state and belongs to morph; the
|
|
94
|
+
output element is client state, is `data-turbo-permanent`, and belongs to the
|
|
95
|
+
controller. See [deferred-renderers.md](deferred-renderers.md).
|
|
96
|
+
|
|
97
|
+
While the fence is open there is no payload and no controller at all — only the
|
|
98
|
+
shimmer.
|
|
99
|
+
|
|
100
|
+
### `:passthrough`
|
|
101
|
+
|
|
102
|
+
````markdown
|
|
103
|
+
```text
|
|
104
|
+
as is
|
|
105
|
+
```
|
|
106
|
+
````
|
|
107
|
+
|
|
108
|
+
```html
|
|
109
|
+
<pre data-ms-element="pre"><code class="language-text">as is
|
|
110
|
+
</code></pre>
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
## Custom tags
|
|
114
|
+
|
|
115
|
+
Models emit XML-ish tags that mean something to your application rather than to
|
|
116
|
+
markdown: `<source>`, `<citation>`, `<thinking>`. Register one and the post-pass
|
|
117
|
+
replaces its node with your partial.
|
|
118
|
+
|
|
119
|
+
The reference case is a model citing its sources:
|
|
120
|
+
|
|
121
|
+
```ruby
|
|
122
|
+
MaquinaStream.register_tag :source,
|
|
123
|
+
attributes: %w[id href title],
|
|
124
|
+
partial: "maquina_stream/components/source_citation",
|
|
125
|
+
literal_content: false
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
| Option | Meaning |
|
|
129
|
+
|---|---|
|
|
130
|
+
| `attributes:` | The attribute names the tag may carry. Anything else on it is dropped before the partial is called. |
|
|
131
|
+
| `partial:` | The partial that renders it. Registered attribute names arrive as locals, so they must match the locals it declares — `href`, not `url`. |
|
|
132
|
+
| `literal_content:` | `true` passes the tag's body as text; `false` renders it as markdown. |
|
|
133
|
+
|
|
134
|
+
```markdown
|
|
135
|
+
Según la fuente <source id="3" href="https://example.com/a" title="Un artículo" onclick="alert(1)"></source>.
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
```html
|
|
139
|
+
<p data-ms-element="p">Según la fuente <span data-component="source-citation" data-ms-source-id="3" …>
|
|
140
|
+
<a data-source-citation-part="link" href="https://example.com/a" rel="noopener noreferrer">Un artículo</a>
|
|
141
|
+
</span>.</p>
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`onclick` never reaches the partial at all — it is gone before the sanitizer is
|
|
145
|
+
even asked, because the registration did not list it.
|
|
146
|
+
|
|
147
|
+
### An unregistered tag does not survive; its content does
|
|
148
|
+
|
|
149
|
+
The sanitizer *unwraps* an element it does not know: the tag goes, its children
|
|
150
|
+
stay.
|
|
151
|
+
|
|
152
|
+
```markdown
|
|
153
|
+
text <danger id="1">content</danger> more
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
```html
|
|
157
|
+
<p data-ms-element="p">text content more</p>
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Text stranded at the top level this way is wrapped into a `<p>` before the
|
|
161
|
+
document is split, so it lands in a block of its own with an id and a digest,
|
|
162
|
+
and morphs and repairs like any other block. Its id comes from the first free
|
|
163
|
+
index at or after its position, so ids stay unique but are not always in
|
|
164
|
+
ascending order.
|
|
165
|
+
|
|
166
|
+
Mid-stream this is stable. `maquina_remend` removes only a tag whose `>` has not
|
|
167
|
+
arrived yet (`"text <thinki"` → `"text"`), the sanitizer unwraps the tag whether
|
|
168
|
+
it closed or not, and closing the tag changes no block above it — so nothing
|
|
169
|
+
flickers and no sealed block is rewritten.
|
|
170
|
+
|
|
171
|
+
### Block-level tags
|
|
172
|
+
|
|
173
|
+
A registered tag can also wrap several paragraphs. The partial receives the
|
|
174
|
+
whole thing as its `content`, rendered as markdown:
|
|
175
|
+
|
|
176
|
+
```ruby
|
|
177
|
+
MaquinaStream.register_tag :thinking, attributes: [], partial: "tags/reasoning"
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
```erb
|
|
181
|
+
<%# app/views/tags/_reasoning.html.erb %>
|
|
182
|
+
<%# locals: (content: "") %>
|
|
183
|
+
<section data-ms-reasoning><%= content.to_s.html_safe %></section>
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
```markdown
|
|
187
|
+
<thinking>
|
|
188
|
+
One.
|
|
189
|
+
|
|
190
|
+
Two.
|
|
191
|
+
</thinking>
|
|
192
|
+
|
|
193
|
+
After.
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
```html
|
|
197
|
+
<section data-ms-reasoning id="ms-7-b0" data-ms-block data-ms-block-digest="434332299dedf830">
|
|
198
|
+
<p data-ms-element="p">One.</p>
|
|
199
|
+
<p data-ms-element="p">Two.</p>
|
|
200
|
+
</section>
|
|
201
|
+
<p id="ms-7-b1" data-ms-element="p" data-ms-block data-ms-block-digest="e44dfc18531833b6">After.</p>
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
Two things to know before you reach for this:
|
|
205
|
+
|
|
206
|
+
1. **The whole tag is one block.** Four paragraphs of reasoning are one id, one
|
|
207
|
+
digest and one unit of repair — not four. If you want them to seal and
|
|
208
|
+
repair separately, leave the tag unregistered and style the blocks instead.
|
|
209
|
+
2. **That block cannot seal until the tag closes**, so everything inside it is
|
|
210
|
+
re-sent on every frame. A four-paragraph body costs about 30% more bytes on
|
|
211
|
+
the wire than the same text unregistered; a twenty-paragraph body costs 3.5x,
|
|
212
|
+
and the single block is 88% of it. Keep block-level tags short, or accept the
|
|
213
|
+
cost at the tail.
|
|
214
|
+
3. **Only registered names are treated this way.** With an empty registry the
|
|
215
|
+
buffer is untouched, and a `<thinking>` inside a fence or a backtick span
|
|
216
|
+
stays text:
|
|
217
|
+
|
|
218
|
+
````markdown
|
|
219
|
+
```text
|
|
220
|
+
<thinking>x</thinking>
|
|
221
|
+
```
|
|
222
|
+
````
|
|
223
|
+
|
|
224
|
+
```html
|
|
225
|
+
<pre data-ms-element="pre"><code class="language-text"><thinking>x</thinking>
|
|
226
|
+
</code></pre>
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
### Where block-level handling applies
|
|
230
|
+
|
|
231
|
+
Only to names you passed to `register_tag`, and only where the opening tag
|
|
232
|
+
begins a line under four columns of indent — the shape that starts an HTML
|
|
233
|
+
block. An inline `<citation>…</citation>` in the middle of a sentence is left
|
|
234
|
+
exactly as the model wrote it, because it already renders correctly.
|
|
235
|
+
|
|
236
|
+
It never applies inside a fenced code block, an inline code span, an indented
|
|
237
|
+
code block, an HTML comment, `<script>`, `<pre>`, `<style>`, `<textarea>`, CDATA
|
|
238
|
+
or a processing instruction — that text is content, not markup. A tag broken
|
|
239
|
+
across a newline, one whose `>` has not arrived, an opener with no closer, a
|
|
240
|
+
closer with no opener, and `<thinking/>` are all left alone.
|
|
241
|
+
|
|
242
|
+
One limitation follows from the indent rule: **a block-level registered tag
|
|
243
|
+
inside a list item is not supported.** Put it at the top level.
|
|
244
|
+
|
|
245
|
+
## Element overrides
|
|
246
|
+
|
|
247
|
+
Replace the markup the renderer produces for one element with a partial of your
|
|
248
|
+
own:
|
|
249
|
+
|
|
250
|
+
```ruby
|
|
251
|
+
MaquinaStream.register_element :h2, partial: "headings/h2"
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
The partial receives `content:` (the element's inner HTML) and `node:` (the
|
|
255
|
+
Nokogiri node), so it must declare both:
|
|
256
|
+
|
|
257
|
+
```erb
|
|
258
|
+
<%# app/views/headings/_h2.html.erb %>
|
|
259
|
+
<%# locals: (content: "", node: nil) %>
|
|
260
|
+
<h2 class="section-heading"><span aria-hidden="true">§</span> <%= content.to_s.html_safe %></h2>
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
```markdown
|
|
264
|
+
## Thinking
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
```html
|
|
268
|
+
<h2 class="section-heading"><span aria-hidden="true">§</span> Thinking<a href="#thinking" class="anchor" …></a></h2>
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
Registering the same element twice replaces the first registration; the last one
|
|
272
|
+
in wins. Elements you do not override carry `data-ms-element="<tag>"` as a
|
|
273
|
+
styling hook, which is usually enough — reach for an override when you need
|
|
274
|
+
different structure, not different CSS.
|
|
275
|
+
|
|
276
|
+
## Resetting
|
|
277
|
+
|
|
278
|
+
```ruby
|
|
279
|
+
MaquinaStream.reset_registries!
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
For tests. Called in production it loses every registration your initializer
|
|
283
|
+
made.
|
data/docs/repair.md
ADDED
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
# Repair
|
|
2
|
+
|
|
3
|
+
## What it is for
|
|
4
|
+
|
|
5
|
+
Action Cable gives no delivery guarantee, no ordering guarantee and no gap
|
|
6
|
+
detection. A frame can simply not arrive: a flaky connection, a backgrounded
|
|
7
|
+
tab, a reconnect in the middle of a message. Deltas are an optimization, and
|
|
8
|
+
they do not converge on their own — only the open tail is patched, so a block
|
|
9
|
+
that changes after it has stopped being the tail is never re-sent.
|
|
10
|
+
|
|
11
|
+
The repair path is where correctness lives. The browser periodically asks the
|
|
12
|
+
server what the message currently is, compares that against its own DOM, and
|
|
13
|
+
asks for the blocks that differ.
|
|
14
|
+
|
|
15
|
+
If you fix a correctness bug inside the delta path, you have fixed it in the
|
|
16
|
+
wrong place.
|
|
17
|
+
|
|
18
|
+
## What you must implement
|
|
19
|
+
|
|
20
|
+
Three things, and then it runs on its own.
|
|
21
|
+
|
|
22
|
+
### 1. Mount the engine
|
|
23
|
+
|
|
24
|
+
```ruby
|
|
25
|
+
# config/routes.rb
|
|
26
|
+
mount MaquinaStream::Engine => "/maquina_stream"
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
### 2. Configure the two seams
|
|
30
|
+
|
|
31
|
+
```ruby
|
|
32
|
+
# config/initializers/maquina_stream.rb
|
|
33
|
+
MaquinaStream.configure do |c|
|
|
34
|
+
c.find_stream = ->(sid) { Message.find_by(id: sid) }
|
|
35
|
+
c.authorize = ->(record, request) { record.conversation.readable_by?(request) }
|
|
36
|
+
end
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
`find_stream` receives the `:sid` from the URL — that is `#maquina_stream_id`,
|
|
40
|
+
which defaults to `to_param` — and returns a record or `nil`. Return `nil` and
|
|
41
|
+
the engine answers `404`. Leave it unset and every engine request raises
|
|
42
|
+
`MaquinaStream::ConfigurationError`: the engine does not go looking for a model
|
|
43
|
+
on its own.
|
|
44
|
+
|
|
45
|
+
`authorize` receives the record and the `ActionDispatch::Request`. Anything
|
|
46
|
+
falsy answers `403`. **Leaving it unset denies everything**; there is no
|
|
47
|
+
permissive default, because an engine that guesses is an engine that leaks.
|
|
48
|
+
|
|
49
|
+
Scope the lookup rather than relying on `authorize` alone where you can — the
|
|
50
|
+
same advice as any Rails controller:
|
|
51
|
+
|
|
52
|
+
```ruby
|
|
53
|
+
c.find_stream = ->(sid) { Current.user.messages.find_by(id: sid) }
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### 3. Put the repair URLs on the message element
|
|
57
|
+
|
|
58
|
+
```erb
|
|
59
|
+
<div id="ms-msg-<%= message.maquina_stream_id %>"
|
|
60
|
+
data-controller="ms-repair"
|
|
61
|
+
data-ms-repair-manifest-url-value="<%= maquina_stream.manifest_path(sid: message.maquina_stream_id) %>"
|
|
62
|
+
data-ms-repair-blocks-url-value="<%= maquina_stream.blocks_path(sid: message.maquina_stream_id) %>"
|
|
63
|
+
data-ms-repair-interval-value="4000"><%= MaquinaStream.render(message) %></div>
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
`maquina_stream.` is the mounted engine's route proxy, so the paths follow
|
|
67
|
+
wherever you mounted it. `data-ms-repair-interval-value` defaults to 4000ms;
|
|
68
|
+
`0` turns the keyframe timer off and leaves the other three triggers.
|
|
69
|
+
|
|
70
|
+
That is all. The blocks the engine renders already carry the `id`,
|
|
71
|
+
`data-ms-block` and `data-ms-block-digest` the controller diffs on.
|
|
72
|
+
|
|
73
|
+
## The two routes
|
|
74
|
+
|
|
75
|
+
### `GET /maquina_stream/:sid/manifest`
|
|
76
|
+
|
|
77
|
+
What the browser is told the message currently *is*, in a bounded number of
|
|
78
|
+
bytes. Not HTML.
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{"seq":7,"cutoff":0,"rollup":"e3b0c44298fc1c14",
|
|
82
|
+
"blocks":[["ms-42-b0","999bd5bd2841c435"],["ms-42-b1","6c7c33d9dcefa12c"]]}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
| Key | Meaning |
|
|
86
|
+
|---|---|
|
|
87
|
+
| `seq` | the sequence number this manifest describes |
|
|
88
|
+
| `cutoff` | how many sealed blocks fall behind the window |
|
|
89
|
+
| `rollup` | one digest covering every block behind the cutoff |
|
|
90
|
+
| `blocks` | `[[id, digest], …]` for the blocks inside the window |
|
|
91
|
+
|
|
92
|
+
Only sealed blocks are listed — an open block is about to change, so there is
|
|
93
|
+
nothing to reconcile it against. With the default `seal_lag` of 2, the last two
|
|
94
|
+
blocks of a message are absent from the manifest by design.
|
|
95
|
+
|
|
96
|
+
The manifest is windowed rather than complete because listing every sealed block
|
|
97
|
+
made it track message length almost exactly: 1.8KB for a 2KB message, 88KB for a
|
|
98
|
+
100KB one, sent every keyframe. The last `manifest_window` sealed blocks go in
|
|
99
|
+
full, and one rollup digest covers everything older. A client whose rollup
|
|
100
|
+
matches knows its history is intact and only has to consider the window; a
|
|
101
|
+
client whose rollup differs asks for the whole thing with `?full=1`, which is
|
|
102
|
+
rare and no more expensive than the cold page load it resembles.
|
|
103
|
+
|
|
104
|
+
### `GET /maquina_stream/:sid/blocks?ids[]=…`
|
|
105
|
+
|
|
106
|
+
The blocks the client asked for, as morphing Turbo Stream actions.
|
|
107
|
+
|
|
108
|
+
```html
|
|
109
|
+
<turbo-stream method="morph" action="replace" target="ms-42-b1">
|
|
110
|
+
<template><p id="ms-42-b1" data-ms-element="p" data-ms-block data-ms-block-digest="6c7c33d9dcefa12c">Dos</p></template>
|
|
111
|
+
</turbo-stream>
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
`method="morph"` is what makes the repair silent: idiomorph patches the existing
|
|
115
|
+
node in place, so client state inside it survives and no animation fires for a
|
|
116
|
+
replacement that never happens.
|
|
117
|
+
|
|
118
|
+
`ids` comes from the client, so it is filtered against the document rather than
|
|
119
|
+
trusted — only ids the message actually has are served, and only the ones
|
|
120
|
+
requested. A request for every block is a legitimate cold repair, so the count
|
|
121
|
+
is not capped, but each id is matched, never interpolated.
|
|
122
|
+
|
|
123
|
+
Asking to repair nothing is a normal answer to a manifest that already agreed,
|
|
124
|
+
and returns an empty stream rather than an error.
|
|
125
|
+
|
|
126
|
+
## What the browser does with them
|
|
127
|
+
|
|
128
|
+
`ms-repair` fetches on four triggers:
|
|
129
|
+
|
|
130
|
+
1. the final seal, always;
|
|
131
|
+
2. a gap in the sequence — every stream action carries `data-ms-seq`;
|
|
132
|
+
3. a reconnect, or the tab becoming visible again;
|
|
133
|
+
4. the periodic keyframe.
|
|
134
|
+
|
|
135
|
+
It compares the manifest against its own DOM, requests only the blocks whose
|
|
136
|
+
digests differ, and morphs them in. Repair therefore costs what has drifted, not
|
|
137
|
+
what the message weighs.
|
|
138
|
+
|
|
139
|
+
Before the morph it dispatches `ms:suppress` on the message element and
|
|
140
|
+
`ms:resume` after it, so the reveal animation unwraps whatever was mid-flight
|
|
141
|
+
and re-baselines afterwards — a repair never re-reveals text the reader has
|
|
142
|
+
already read.
|
|
143
|
+
|
|
144
|
+
It dispatches `ms:repaired` (with `{reason, blocks}`) and `ms:repair-failed`
|
|
145
|
+
(with `{reason, error}`) if you want to observe it. See
|
|
146
|
+
[javascript.md](javascript.md).
|
|
147
|
+
|
|
148
|
+
## Responses you should expect
|
|
149
|
+
|
|
150
|
+
| Situation | Response |
|
|
151
|
+
|---|---|
|
|
152
|
+
| `find_stream` returns `nil` | `404` |
|
|
153
|
+
| `authorize` returns falsy | `403` |
|
|
154
|
+
| `authorize` not configured | `403` |
|
|
155
|
+
| `find_stream` not configured | `MaquinaStream::ConfigurationError` |
|
|
156
|
+
|
|
157
|
+
## Transport independence
|
|
158
|
+
|
|
159
|
+
The repair path is plain `GET`, so a client that lost frames recovers over HTTP
|
|
160
|
+
no matter how those frames were delivered. Frames, sequence numbers, the
|
|
161
|
+
manifest and these two routes are transport-independent by construction — which
|
|
162
|
+
is what makes `config.transport` a seam worth having rather than a setting.
|
data/docs/security.md
ADDED
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
# Security
|
|
2
|
+
|
|
3
|
+
Everything this engine renders is model output, and model output is
|
|
4
|
+
prompt-injectable by definition. A document that reached the sanitizer having
|
|
5
|
+
"already been checked" has not been checked — it has been parsed.
|
|
6
|
+
|
|
7
|
+
## The one rule for a host
|
|
8
|
+
|
|
9
|
+
**Never assign model output as raw HTML.**
|
|
10
|
+
|
|
11
|
+
The engine's own output is `html_safe` because it has been through the
|
|
12
|
+
sanitizer. Anything else — a buffer, a payload, a tool result, a value you read
|
|
13
|
+
back out of rendered markup in JavaScript — is not. Do not `html_safe` it, do
|
|
14
|
+
not `innerHTML` it, do not hand it to a template that will.
|
|
15
|
+
|
|
16
|
+
```erb
|
|
17
|
+
<%= MaquinaStream.render(message) %> <%# sanitized on the way out %>
|
|
18
|
+
<%= raw message.content %> <%# never %>
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## The sanitizer
|
|
22
|
+
|
|
23
|
+
```ruby
|
|
24
|
+
MaquinaStream::Sanitizer.call(html, config: MaquinaStream.config) # => String
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
It is the last pass in the render chain — `maquina_remend → CommonMarker →
|
|
28
|
+
Nokogiri post-pass → Sanitizer` — and the only one that assumes the document is
|
|
29
|
+
hostile. It runs unconditionally: no mode, no flag, no fast path skips it. It is
|
|
30
|
+
a pure function, with no Rails and no request.
|
|
31
|
+
|
|
32
|
+
Nokogiri's **HTML5** parser builds the tree, because a sanitizer that parses
|
|
33
|
+
differently from the engine that will render the output is a mutation-XSS bug
|
|
34
|
+
waiting for its input.
|
|
35
|
+
|
|
36
|
+
```
|
|
37
|
+
in : <a href="javascript:alert(1)">clic</a>
|
|
38
|
+
out: <a>clic</a>
|
|
39
|
+
|
|
40
|
+
in : <img src="x" onerror="alert(1)">
|
|
41
|
+
out: <img src="x">
|
|
42
|
+
|
|
43
|
+
in : <script>alert(1)</script>hola
|
|
44
|
+
out: hola
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### How it decides
|
|
48
|
+
|
|
49
|
+
1. An element on the drop list — or in the SVG or MathML namespace — is removed
|
|
50
|
+
**with its subtree**.
|
|
51
|
+
2. An element not on the allowlist is **unwrapped**: the element goes, its
|
|
52
|
+
sanitized children stay. Text survives; markup does not.
|
|
53
|
+
3. On a surviving element, each attribute must be named by the allowlist.
|
|
54
|
+
Everything else is dropped, not escaped and kept.
|
|
55
|
+
4. `href` and `src` are re-parsed and re-checked. A rejected `href` leaves the
|
|
56
|
+
link with its text; a rejected `src` removes the `<img>` entirely, because a
|
|
57
|
+
broken rectangle carrying an attacker-chosen `alt` is worse than nothing.
|
|
58
|
+
5. Comments, processing instructions and doctypes are removed. CDATA becomes
|
|
59
|
+
text.
|
|
60
|
+
|
|
61
|
+
### What survives
|
|
62
|
+
|
|
63
|
+
**Elements** — rendered markdown plus the post-pass wrappers: headings, `p`,
|
|
64
|
+
`div`, `span`, `br`, `hr`, lists (`ul ol li dl dt dd`), tables
|
|
65
|
+
(`table thead tbody tfoot tr td th caption colgroup col`),
|
|
66
|
+
`pre code kbd samp var`, `blockquote figure figcaption details summary section
|
|
67
|
+
article aside`, `a`, `img`, the inline set (`em strong b i u s del ins mark
|
|
68
|
+
small sub sup q abbr dfn cite time wbr`), and `input` — the last only as the
|
|
69
|
+
tasklist checkbox, forced `disabled`.
|
|
70
|
+
|
|
71
|
+
**Attributes**
|
|
72
|
+
|
|
73
|
+
| Scope | Allowed |
|
|
74
|
+
|---|---|
|
|
75
|
+
| Global | `id class title lang dir role translate`, plus `aria-*` |
|
|
76
|
+
| `a` | `href target rel hreflang type` |
|
|
77
|
+
| `img` | `src alt width height loading decoding` |
|
|
78
|
+
| Lists and tables | `start reversed type value colspan rowspan align valign headers scope abbr span` |
|
|
79
|
+
| `time`, `details` | `datetime`, `open` |
|
|
80
|
+
| The engine's | every `data-ms-*`, plus `data-component data-variant data-size data-slot data-state data-side data-orientation data-controller data-action data-turbo-permanent data-turbo-temporary` |
|
|
81
|
+
|
|
82
|
+
`data-controller` and `data-action` are filtered **by value**: only `ms-`
|
|
83
|
+
identifiers survive, and only action descriptors naming one. A third-party
|
|
84
|
+
controller's `data-<name>-*-value` attributes are dropped with it, so an
|
|
85
|
+
injected controller arrives with no configuration:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
in : <div data-controller="evil" data-evil-url-value="x">hi</div>
|
|
89
|
+
out: <div>hi</div>
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Anchors keep `target` normalized to `_blank`/`_self` and always gain
|
|
93
|
+
`rel="noopener noreferrer"`.
|
|
94
|
+
|
|
95
|
+
### What is dropped
|
|
96
|
+
|
|
97
|
+
- Every `on*` attribute, whatever its case, and whatever the parser made of a
|
|
98
|
+
name split across a newline.
|
|
99
|
+
- `srcdoc`, `formaction`, `style`, `action`, `http-equiv`, `background`, `ping`,
|
|
100
|
+
`srcset`, `usemap`, `name`, `contenteditable`, `accesskey`.
|
|
101
|
+
- Every namespaced attribute — `xlink:href`, `xml:base` — without exception.
|
|
102
|
+
- `script style svg math template noscript iframe object embed form button
|
|
103
|
+
select textarea base link meta …`, each with its subtree.
|
|
104
|
+
|
|
105
|
+
That last one includes any `<script type="text/plain">` carrier: the sanitizer
|
|
106
|
+
cannot tell yours from an imitation of it, and Nokogiri's HTML5 serializer
|
|
107
|
+
writes script children unescaped, so a fence containing
|
|
108
|
+
`</script><img onerror=…>` would break out on the next parse. **A code block
|
|
109
|
+
that needs its raw source in the DOM must carry it on a non-script element** —
|
|
110
|
+
which is why the engine's carrier is a `<pre hidden data-ms-code-source>`.
|
|
111
|
+
|
|
112
|
+
### URL hardening
|
|
113
|
+
|
|
114
|
+
Applied to `href` and `src`, in order:
|
|
115
|
+
|
|
116
|
+
1. Control characters and Unicode whitespace are stripped first, so the scheme
|
|
117
|
+
tested is the scheme a browser would act on (`java\tscript:`,
|
|
118
|
+
`java\nscript:`).
|
|
119
|
+
2. `//host`, `\\host`, `/\host` — protocol-relative in every spelling — are
|
|
120
|
+
rejected.
|
|
121
|
+
3. The scheme is read from the raw value **and** from a decoded copy (HTML
|
|
122
|
+
entities, then percent-encoding). `javascript:`, `&#106;avascript:`,
|
|
123
|
+
`%6Aavascript:` and `JaVaScRiPt:` all resolve to a dangerous scheme and are
|
|
124
|
+
rejected.
|
|
125
|
+
4. The scheme must appear in `config.allowed_protocols` (default `http https
|
|
126
|
+
mailto`).
|
|
127
|
+
5. A relative URL is resolved against `config.default_origin` when one is set,
|
|
128
|
+
and the result must still be on an allowed protocol. With no origin set it is
|
|
129
|
+
left as written. A bare `#fragment` is always kept.
|
|
130
|
+
6. The final URL must start with one of `config.allowed_link_prefixes` /
|
|
131
|
+
`allowed_image_prefixes`. `["*"]`, the default, means any.
|
|
132
|
+
|
|
133
|
+
`data:` is special-cased: image positions only, only when
|
|
134
|
+
`config.allow_data_images`, and only base64 raster types.
|
|
135
|
+
|
|
136
|
+
```
|
|
137
|
+
allow_data_images = true <img src="data:image/png;base64,iVBORw0KGgo="> → kept
|
|
138
|
+
allow_data_images = false <img src="data:image/png;base64,iVBORw0KGgo="> → removed
|
|
139
|
+
either <img src="data:image/svg+xml;base64,…"> → removed
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
`data:image/svg+xml` is a scriptable document wearing an image's MIME type and
|
|
143
|
+
never passes, whatever you configure.
|
|
144
|
+
|
|
145
|
+
Tightening the two prefix lists is the main lever you have over what a model may
|
|
146
|
+
link to. See [configuration.md](configuration.md).
|
|
147
|
+
|
|
148
|
+
## The markdown normalisation pass
|
|
149
|
+
|
|
150
|
+
One pass runs between `maquina_remend` and the markdown parser, and it can only
|
|
151
|
+
insert newlines: around the opening and closing tags of tags you registered with
|
|
152
|
+
`register_tag`, so a block-level `<thinking>` parses as its own HTML block
|
|
153
|
+
instead of leaving a stray end tag inside a paragraph. Without it, the parser
|
|
154
|
+
never closes the element and the rest of the message is handed to your partial.
|
|
155
|
+
|
|
156
|
+
It is deliberately narrow, because it changes how model-written text is parsed:
|
|
157
|
+
|
|
158
|
+
- The allowlist is your `register_tag` registry. An unregistered tag is never
|
|
159
|
+
matched and never moved.
|
|
160
|
+
- Fenced blocks and inline code win — matching runs over masked text, so a
|
|
161
|
+
registered name inside them is content.
|
|
162
|
+
- CommonMark's raw HTML regions — comments, `<script>`, `<pre>`, `<style>`,
|
|
163
|
+
`<textarea>`, CDATA, processing instructions — are masked too. A blank line
|
|
164
|
+
inserted inside one of those would end it early and publish what it hid.
|
|
165
|
+
- Indented code is excluded by an indent rule.
|
|
166
|
+
- A match must be a complete, well-formed tag on one line, parsed with
|
|
167
|
+
CommonMark's attribute grammar, so a `>` inside a quoted value cannot make an
|
|
168
|
+
insertion land mid-tag.
|
|
169
|
+
|
|
170
|
+
Because it only inserts newlines, it can promote model text from raw HTML into
|
|
171
|
+
markdown, or split a paragraph — but it cannot introduce an element or an
|
|
172
|
+
attribute the allowlist does not name. The sanitizer runs unconditionally,
|
|
173
|
+
last, over the parsed HTML. The whole XSS corpus is additionally run through the
|
|
174
|
+
renderer with tags registered.
|
|
175
|
+
|
|
176
|
+
## Why the client sanitizes again
|
|
177
|
+
|
|
178
|
+
The server pass proves one thing: the HTML **the engine** serializes is safe. It
|
|
179
|
+
cannot speak for HTML the browser builds afterwards.
|
|
180
|
+
|
|
181
|
+
- Client-deferred renderers (`ms-diagram`, `ms-math`) receive a JSON payload and
|
|
182
|
+
produce new DOM from it. That payload is attacker-influenced text which passed
|
|
183
|
+
through the sanitizer as an attribute *value*, never as markup — nothing has
|
|
184
|
+
sanitized its output until the controller does. So `ms-deferred` applies its
|
|
185
|
+
own allowlist to whatever the library returns.
|
|
186
|
+
- `data-controller` is an allowlisted attribute. The sanitizer restricts it to
|
|
187
|
+
the `ms-` namespace, but cannot tell a controller the post-pass emitted from
|
|
188
|
+
one an injected fragment asked for. Every `ms-` controller therefore treats its
|
|
189
|
+
own values as untrusted input rather than as server intent.
|
|
190
|
+
- Serialize-and-reparse is where mutation XSS lives. Two parsers, two rounds of
|
|
191
|
+
entity decoding and one namespace boundary are enough to turn inert text into
|
|
192
|
+
markup.
|
|
193
|
+
|
|
194
|
+
Server-side escaping protects the attribute boundary. It does not protect
|
|
195
|
+
whatever the client does with the value inside it.
|
|
196
|
+
|
|
197
|
+
## Renderer posture
|
|
198
|
+
|
|
199
|
+
| Renderer | Setting | What it buys |
|
|
200
|
+
|---|---|---|
|
|
201
|
+
| `ms-diagram` (Mermaid) | `securityLevel: "strict"` | disables click handlers and inline HTML in diagram source |
|
|
202
|
+
| `ms-math` (KaTeX) | `trust: false` | refuses `\htmlClass`, `\includegraphics` and `\href`, all of which take attacker-controlled strings into the DOM |
|
|
203
|
+
| both | output allowlist | the library is third-party; its output is scrubbed before it reaches the DOM |
|
|
204
|
+
|
|
205
|
+
## Limits
|
|
206
|
+
|
|
207
|
+
- **It sanitizes HTML, not meaning.** A model that writes a plausible phishing
|
|
208
|
+
link to an allowed host produces a link the sanitizer will happily keep. That
|
|
209
|
+
is what `allowed_link_prefixes` and the link-safety dialog are for.
|
|
210
|
+
- **A registered partial is yours to get right.** Registry output still goes
|
|
211
|
+
through the sanitizer, so it cannot introduce an element or attribute the
|
|
212
|
+
allowlist does not name — but a partial that renders an attacker-supplied
|
|
213
|
+
string into an allowed attribute is a hole the allowlist cannot see.
|
|
214
|
+
- **Your own controllers are outside its reach.** A controller of yours that
|
|
215
|
+
reads a value out of rendered markup and assigns it as HTML has undone the
|
|
216
|
+
whole chain.
|
|
217
|
+
- **The broadcast target is yours.** The sanitizer has nothing to say about who
|
|
218
|
+
may subscribe to a Turbo stream.
|
|
219
|
+
|
|
220
|
+
## Extending the allowlist
|
|
221
|
+
|
|
222
|
+
Everything lives in constants at the top of `lib/maquina_stream/sanitizer.rb`:
|
|
223
|
+
|
|
224
|
+
- a new element → `ALLOWED_ELEMENTS`, or `DROP_WITH_CONTENT` if its contents
|
|
225
|
+
must not survive it;
|
|
226
|
+
- a new attribute → `GLOBAL_ATTRIBUTES`, or the element's entry in
|
|
227
|
+
`ELEMENT_ATTRIBUTES`;
|
|
228
|
+
- a new hook of your own → name it `data-ms-*` and it is already allowed;
|
|
229
|
+
- a new URL-bearing attribute → give it a branch in `scrub_attribute_value` that
|
|
230
|
+
runs it through `safe_url`, never a bare entry in the allowlist. An unhardened
|
|
231
|
+
URL attribute is the whole bug.
|
|
232
|
+
|
|
233
|
+
Every change comes with a corpus file. `test/fixtures/xss/*.txt` holds one
|
|
234
|
+
attack per file:
|
|
235
|
+
|
|
236
|
+
```
|
|
237
|
+
--- input
|
|
238
|
+
<a href="javascript:alert(1)">clic</a>
|
|
239
|
+
--- note
|
|
240
|
+
javascript: link. The anchor text survives; the href must be dropped.
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
The suite turns each file into its own test case at load time, so adding a file
|
|
244
|
+
adds a test and there is no way to add an attack that is quietly not run. Every
|
|
245
|
+
entry is asserted to produce no script element, no `on*` attribute, no forbidden
|
|
246
|
+
attribute and no dangerous URL scheme, and to be a fixed point under a second
|
|
247
|
+
sanitization pass.
|