@mathzetta/qblur 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +63 -0
- package/LICENSE +1 -1
- package/README.md +34 -22
- package/dist/cpu-worker.js +2 -2
- package/dist/qblur-apply.js +2 -2
- package/dist/qblur-element.js +2 -2
- package/dist/qblur.core.js +2 -2
- package/dist/qblur.external.global.min.js +2 -2
- package/dist/qblur.external.js +2 -2
- package/dist/qblur.global.min.js +2 -2
- package/dist/qblur.js +2 -2
- package/docs/filters.md +92 -89
- package/docs/library.md +52 -55
- package/package.json +9 -3
- package/src/apply-filter.js +8 -7
- package/src/apply.d.ts +12 -11
- package/src/cpu-helpers.js +9 -7
- package/src/cpu-runner.js +26 -12
- package/src/cpu.js +65 -32
- package/src/element.d.ts +1 -1
- package/src/element.js +3 -3
- package/src/format.js +36 -27
- package/src/graphics.js +12 -12
- package/src/index.d.ts +11 -11
- package/src/library.js +29 -29
- package/src/png.js +21 -17
- package/src/runtime.js +25 -6
- package/src/save.js +7 -7
- package/src/strokes.js +15 -7
package/docs/filters.md
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Writing image filters
|
|
2
2
|
|
|
3
|
-
A filter is a bit of
|
|
4
|
-
|
|
5
|
-
out the new
|
|
3
|
+
A filter is a bit of math that runs on every pixel of a picture. It reads the pixel — its
|
|
4
|
+
color `r` `g` `b`, its brightness `gray`, where it is (`x` `y`, or `px` `py` in pixels) — and works
|
|
5
|
+
out the new color. Type one into the *Add a new filter* box on the filters page (`filters.html`, where filters
|
|
6
|
+
are written; the editor puts them on your pictures) and press *Add*:
|
|
6
7
|
|
|
7
8
|
| Write | You get |
|
|
8
9
|
| --- | --- |
|
|
@@ -15,36 +16,38 @@ out the new colour. Type one into the editor's *Add a new filter* box and press
|
|
|
15
16
|
| `gaussian(3)` | a soft blur |
|
|
16
17
|
|
|
17
18
|
Then give it knobs: with the step's *Edit* on, *+ slider* adds a line like
|
|
18
|
-
`strength := [[0..1 step 0.01 = 0.5]]` — a slider the
|
|
19
|
+
`strength := [[0..1 step 0.01 = 0.5]]` — a slider the math can use as `strength` (*Edit* also shows
|
|
19
20
|
the step's name and author, and lets you write notes). Steps run top to bottom, each on the result of the one
|
|
20
21
|
above, so filters chain; *Copy link* puts the whole thing in a link anyone can open, change and
|
|
21
|
-
keep (see [Sharing filters](#sharing-filters)).
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
22
|
+
keep (see [Sharing filters](#sharing-filters)). *Use it in the editor* takes the filter to the editor, where a
|
|
23
|
+
step shows just its knobs and notes beside a large picture (its *Edit this filter* brings it back here). Everything
|
|
24
|
+
runs in your browser — nothing is uploaded. The rest of this guide is everything an expression can use; the
|
|
25
|
+
*Examples* (under the steps, on both pages) each explain one idea, and *From the library* has dozens of filters to
|
|
26
|
+
open and read.
|
|
27
|
+
|
|
28
|
+
**Saving the result.** In the editor, *Download* (at the top) draws the filter at the picture's own size. The
|
|
29
|
+
*Download* tab's options choose the format — PNG, JPEG or WebP, or *auto*, which picks the smallest sensible one (a result
|
|
30
|
+
with few colors becomes a small indexed PNG; one with transparency stays lossless; a photo goes to
|
|
28
31
|
whichever of JPEG and WebP is smaller) — the size, a crop you drag on the result, and for moving
|
|
29
32
|
pictures *Save as GIF* (at the preview size; a GIF plays through once with its own delays, anything
|
|
30
33
|
else gets 3 seconds) or *Record* (video). Effects measured in pixels — blur radii, `px` — stay the
|
|
31
34
|
same number of pixels, so they look finer on a big picture.
|
|
32
35
|
|
|
33
36
|
<!-- dev -->
|
|
34
|
-
**For developers.** MathZetta turns the
|
|
37
|
+
**For developers.** MathZetta turns the math into GLSL, and `filterShader()` wraps it in a complete
|
|
35
38
|
fragment shader, so a filter runs on the GPU — fast enough for video:
|
|
36
39
|
|
|
37
40
|
```js
|
|
38
41
|
import { filterShader, filterVertexShader } from 'mathzetta/glsl';
|
|
39
42
|
|
|
40
43
|
filterShader('(1 - r, 1 - g, 1 - b)') // invert
|
|
41
|
-
filterShader('gray') //
|
|
44
|
+
filterShader('gray') // grayscale
|
|
42
45
|
filterShader({ r: 'g', g: 'r' }) // swap red and green; b and a unchanged
|
|
43
46
|
filterShader('if gray > 0.5 then 1 else 0') // threshold
|
|
44
47
|
filterShader('pixel(px + 8 sin(py / 10 + t / 5), py)') // a wave that moves with t
|
|
45
48
|
```
|
|
46
49
|
|
|
47
|
-
`demo/
|
|
50
|
+
`demo/editor.html` is the editor; `LIBRARY` (exported by the package) is the built-in catalog it
|
|
48
51
|
shows. [qblur on your own pages](library.md) is how to use the package.
|
|
49
52
|
<!-- /dev -->
|
|
50
53
|
|
|
@@ -68,15 +71,15 @@ And everything GLSL has: `clamp(v, 0, 1)`, `mix(a, b, 0.5)`, `smoothstep(0, 1, x
|
|
|
68
71
|
`step(0.5, x)`, `fract(x)`, `length((x, y))`, `distance(…)`, `floor`, `abs`,
|
|
69
72
|
`min`, `max`, `sin`, `cos`, `exp`, `log`, `sqrt`, `pow`, `atan2(y, x)`, `mod`.
|
|
70
73
|
|
|
71
|
-
## Results: numbers,
|
|
74
|
+
## Results: numbers, colors, true/false
|
|
72
75
|
|
|
73
76
|
The shader accepts whatever the expression works out to:
|
|
74
77
|
|
|
75
|
-
- a **number** is
|
|
76
|
-
- a **tuple** `(r, g, b)` or `(r, g, b, a)` is the
|
|
78
|
+
- a **number** is gray (or, for a channel, that channel's value);
|
|
79
|
+
- a **tuple** `(r, g, b)` or `(r, g, b, a)` is the color;
|
|
77
80
|
- a **comparison** is 1 or 0 — `r > 0.5` is a mask you can multiply by.
|
|
78
81
|
|
|
79
|
-
Give one expression for the whole
|
|
82
|
+
Give one expression for the whole color, or an object with any of
|
|
80
83
|
`r`, `g`, `b`, `a`, `rgb`, `rgba`. Channels you don't mention keep their
|
|
81
84
|
value. In a per-channel expression, `r g b a` are always the *original*
|
|
82
85
|
pixel, so `{ r: 'g', g: 'r' }` swaps rather than copies.
|
|
@@ -117,7 +120,7 @@ rgb * (1 - 1.6 ((x - 0.5)^2 + (y - 0.5)^2)) vignette (no t)
|
|
|
117
120
|
For time in seconds instead, set `t` to `performance.now() / 1000` and
|
|
118
121
|
write `sin(2π t)` for one cycle a second.
|
|
119
122
|
|
|
120
|
-
##
|
|
123
|
+
## Neighboring pixels
|
|
121
124
|
|
|
122
125
|
`pixel(px, py)` reads any pixel, so blurs, edges and distortions are
|
|
123
126
|
expressions too:
|
|
@@ -141,7 +144,7 @@ still picture moves:
|
|
|
141
144
|
|
|
142
145
|
| Function | Result |
|
|
143
146
|
| --- | --- |
|
|
144
|
-
| `zoom(s)` `zoom(s, cx, cy)` | the picture `s` times larger, about its
|
|
147
|
+
| `zoom(s)` `zoom(s, cx, cy)` | the picture `s` times larger, about its center or the point (cx, cy) in 0–1 |
|
|
145
148
|
| `shift(dx, dy)` | moved right and down by that many pixels, wrapping around |
|
|
146
149
|
| `rotate(a)` `rotate(a, cx, cy)` | turned clockwise by `a` radians (`rotate(10°)` works) |
|
|
147
150
|
| `crop(x0, y0, x1, y1)` | that part of the picture, in 0–1 fractions, filling the frame |
|
|
@@ -159,9 +162,9 @@ tile(2) four copies
|
|
|
159
162
|
Each is one transform about the frame; to combine them, chain steps
|
|
160
163
|
(`Zoom` then `Spin`). Where a transform leaves the frame — zoomed out,
|
|
161
164
|
rotated corners — the picture is **transparent** when the transform is
|
|
162
|
-
the whole
|
|
165
|
+
the whole color (`zoom(0.5)` alone: the page, or a PNG's transparency,
|
|
163
166
|
shows through; a later step sees `a` = 0 there and can fill it:
|
|
164
|
-
`mix(background, rgb, a)`). Inside a bigger expression it is a
|
|
167
|
+
`mix(background, rgb, a)`). Inside a bigger expression it is a color,
|
|
165
168
|
black there, and `alpha(zoom(0.5))` is 1 inside the picture and 0
|
|
166
169
|
outside, so `mix(rgb, zoom(0.5), alpha(zoom(0.5)))` lays the small copy
|
|
167
170
|
over the original. `shift` and `tile` wrap instead.
|
|
@@ -180,13 +183,13 @@ A named picture is laid over the frame according to its own size:
|
|
|
180
183
|
- `place` — `cover` (fills the frame keeping the aspect ratio, the
|
|
181
184
|
default), `fit` (shows all of it), `stretch` (ignores the ratio), or
|
|
182
185
|
`tile` (repeats it at its own size).
|
|
183
|
-
- `zoom`, `x`, `y`, `angle` — then scaled,
|
|
186
|
+
- `zoom`, `x`, `y`, `angle` — then scaled, centered at (x, y) in 0–1 of
|
|
184
187
|
the frame, and turned. Outside the picture it is transparent, so
|
|
185
188
|
`mix(rgb, inset, alpha(inset))` composites it — a picture in picture.
|
|
186
189
|
|
|
187
190
|
Whoever uses the filter can move it: `"with": { "inset": { "x": 0.2,
|
|
188
191
|
"zoom": 0.5 } }` in a chain step (the demo shows these as knobs beside
|
|
189
|
-
each picture), and `zoom(inset, 1 + t / 500)` in the
|
|
192
|
+
each picture), and `zoom(inset, 1 + t / 500)` in the math animates on
|
|
190
193
|
top of that. The `source` picture is the frame's size, so it has no
|
|
191
194
|
placement.
|
|
192
195
|
|
|
@@ -194,7 +197,7 @@ placement.
|
|
|
194
197
|
|
|
195
198
|
A filter can draw text, rectangles and ovals. Each is named in
|
|
196
199
|
`graphics`, drawn by the browser into a transparent picture the size of
|
|
197
|
-
the frame, and read by the
|
|
200
|
+
the frame, and read by the math like any picture:
|
|
198
201
|
|
|
199
202
|
```json
|
|
200
203
|
{ "name": "Caption",
|
|
@@ -202,10 +205,10 @@ the frame, and read by the maths like any picture:
|
|
|
202
205
|
"color": "mix(rgb, title, alpha(title))" }
|
|
203
206
|
```
|
|
204
207
|
|
|
205
|
-
- `title` is the graphic's
|
|
206
|
-
is — so the
|
|
208
|
+
- `title` is the graphic's color at this pixel, `alpha(title)` where it
|
|
209
|
+
is — so the math decides how it lands: `mix(rgb, title, alpha(title))`
|
|
207
210
|
lays it on; `mix(rgb, mix((1, 0.35, 0.3), (1, 0.85, 0.25), x), alpha(title))`
|
|
208
|
-
|
|
211
|
+
colors the letters with a gradient; `mix(rgb, gaussian(5) * 0.7, alpha(panel))`
|
|
209
212
|
frosts the picture behind a panel (a shape with its own filter);
|
|
210
213
|
`mix(rgb * 0.3, rgb, alpha(spot))` darkens all but an oval.
|
|
211
214
|
- **Animated** through the same geometry as any picture, with `t`:
|
|
@@ -216,9 +219,9 @@ the frame, and read by the maths like any picture:
|
|
|
216
219
|
| Setting | For | Meaning |
|
|
217
220
|
| --- | --- | --- |
|
|
218
221
|
| `type` | all | `text` (the default), `rect` or `ellipse` |
|
|
219
|
-
| `x`, `y` | all | the
|
|
222
|
+
| `x`, `y` | all | the center, 0–1 across and down the frame |
|
|
220
223
|
| `angle` | all | turned clockwise, radians |
|
|
221
|
-
| `color` | all | any CSS
|
|
224
|
+
| `color` | all | any CSS color: `#ffcc00`, `rgba(0,0,0,0.5)`, `white` |
|
|
222
225
|
| `shadow` | all | `{ "color", "blur", "dx", "dy" }` drawn into the graphic (sizes as fractions of the text size, or the frame height for shapes), or `null` |
|
|
223
226
|
| `shadows` | all | a list of up to 8 shadow layers, back to front, each drawn into a picture of its own (see below) |
|
|
224
227
|
| `outline` | all | `{ "color", "width" }`, or `null` |
|
|
@@ -230,10 +233,10 @@ the frame, and read by the maths like any picture:
|
|
|
230
233
|
|
|
231
234
|
### Shadow layers, each with its own filter
|
|
232
235
|
|
|
233
|
-
`shadow` is drawn into the graphic, so whatever the
|
|
236
|
+
`shadow` is drawn into the graphic, so whatever the math does to the
|
|
234
237
|
text it does to that shadow too. `shadows` keeps them apart: each layer
|
|
235
238
|
is drawn into a picture of its own — `title_shadow1`, `title_shadow2`…,
|
|
236
|
-
back to front — and the
|
|
239
|
+
back to front — and the math lays each on as it likes:
|
|
237
240
|
|
|
238
241
|
```json
|
|
239
242
|
{ "graphics": { "title": { "text": "LAYERS", "shadow": null,
|
|
@@ -242,13 +245,13 @@ back to front — and the maths lays each on as it likes:
|
|
|
242
245
|
```
|
|
243
246
|
|
|
244
247
|
Here the first layer darkens the picture under a soft glow that breathes
|
|
245
|
-
with `t`, and the second is a hard offset shadow
|
|
246
|
-
whatever
|
|
248
|
+
with `t`, and the second is a hard offset shadow colored by a gradient
|
|
249
|
+
whatever color it was cast in ("Layered shadow" in the library). The
|
|
247
250
|
plain way to lay them all on is `mix(mix(mix(rgb, title_shadow1,
|
|
248
251
|
alpha(title_shadow1)), title_shadow2, alpha(title_shadow2)), title,
|
|
249
252
|
alpha(title))`; the editor writes that for you when you add a layer to
|
|
250
|
-
a step whose
|
|
251
|
-
own
|
|
253
|
+
a step whose math is still the plain one, and points out a layer your
|
|
254
|
+
own math doesn't use yet.
|
|
252
255
|
|
|
253
256
|
### Fonts
|
|
254
257
|
|
|
@@ -267,9 +270,9 @@ a graphic with `with`, merging settings: `{ "filter": …Caption…, "with":
|
|
|
267
270
|
graphics are not shared between steps: two steps whose text is called
|
|
268
271
|
`title` draw two texts (the compiler binds them as `title`, `title~2`…).
|
|
269
272
|
|
|
270
|
-
In the editor, *+ text*, *+ rectangle* and *+ oval* add a step with an
|
|
273
|
+
In the editor's *Select & draw* tab, *+ text*, *+ rectangle* and *+ oval* add a step with an
|
|
271
274
|
editable expression; each graphic gets controls for its words, font,
|
|
272
|
-
size, place, angle,
|
|
275
|
+
size, place, angle, color, shadow and outline, and can be dragged on
|
|
273
276
|
the picture. A font file loaded from the computer works there but can't
|
|
274
277
|
travel in a link; a font URL can (the browser of whoever opens the link
|
|
275
278
|
fetches it from that address).
|
|
@@ -290,7 +293,7 @@ A brush is a graphic of type `strokes`: what the user paints on the
|
|
|
290
293
|
picture, recorded point by point (`[x, y, milliseconds, pressure]`, x and
|
|
291
294
|
y as fractions of the frame) and drawn again for every frame — only the
|
|
292
295
|
paint laid down by frame `t` is there, so the painting animates. Its
|
|
293
|
-
picture carries data rather than
|
|
296
|
+
picture carries data rather than color:
|
|
294
297
|
|
|
295
298
|
| Channel | Meaning |
|
|
296
299
|
| --- | --- |
|
|
@@ -299,15 +302,15 @@ picture carries data rather than colour:
|
|
|
299
302
|
| `green(brush)` | passes ÷ 4: how many strokes have gone over it (`4 green(brush)` counts them) |
|
|
300
303
|
| `blue(brush)` | progress: how far through the whole drawing it was painted, 0 → 1 |
|
|
301
304
|
|
|
302
|
-
So the
|
|
305
|
+
So the math decides what the paint does, and how it changes:
|
|
303
306
|
|
|
304
307
|
```
|
|
305
|
-
mix(rgb, ink, alpha(brush)) ink in a
|
|
308
|
+
mix(rgb, ink, alpha(brush)) ink in a color (a knob)
|
|
306
309
|
mix(rgb, gaussian(2 + 3 min(4 green(brush), 3)), alpha(brush)) a blur, stronger with every pass
|
|
307
310
|
rgb (1 - 0.22 min(4 green(brush), 4) alpha(brush)) darker with every pass, like a wash
|
|
308
311
|
mix(rgb, mix((1, 0.9, 0.5), (0.1, 0.07, 0.06), red(brush)), alpha(brush)) glowing as it goes down, then cooling
|
|
309
312
|
mix(rgb, fromhsv(blue(brush), 0.85, 1), alpha(brush)) a rainbow along the drawing
|
|
310
|
-
mix((gray, gray, gray), rgb, alpha(brush)) the picture's
|
|
313
|
+
mix((gray, gray, gray), rgb, alpha(brush)) the picture's color painted back in
|
|
311
314
|
```
|
|
312
315
|
|
|
313
316
|
| Setting | Meaning |
|
|
@@ -323,7 +326,7 @@ mix((gray, gray, gray), rgb, alpha(brush)) the pictur
|
|
|
323
326
|
| `clip` | `none`, `inside` or `outside`: paint only inside (or outside) the user's selection |
|
|
324
327
|
|
|
325
328
|
A stroke counts once wherever it goes, however often it crosses itself,
|
|
326
|
-
so `green` counts separate passes. In the editor, "+ brush" adds a step
|
|
329
|
+
so `green` counts separate passes. In the editor (*Select & draw*), "+ brush" adds a step
|
|
327
330
|
and starts drawing (the picture shows the painting finished while you
|
|
328
331
|
draw; "replay" plays it); each brush's panel chooses what the paint does,
|
|
329
332
|
the brush, the timing and where it may paint, and "outline the
|
|
@@ -334,7 +337,7 @@ takes `{ t, mask }`.
|
|
|
334
337
|
|
|
335
338
|
## Several pictures: masks and blending
|
|
336
339
|
|
|
337
|
-
A filter can ask for more pictures by name, and the
|
|
340
|
+
A filter can ask for more pictures by name, and the math treats each
|
|
338
341
|
like the main one:
|
|
339
342
|
|
|
340
343
|
```json
|
|
@@ -343,8 +346,8 @@ like the main one:
|
|
|
343
346
|
"color": "mix(source, rgb, luma(mask))" }
|
|
344
347
|
```
|
|
345
348
|
|
|
346
|
-
- `mask` is that picture's
|
|
347
|
-
`mask(px, py)` is any pixel of it, also a
|
|
349
|
+
- `mask` is that picture's color at this pixel (like `rgb`);
|
|
350
|
+
`mask(px, py)` is any pixel of it, also a color; `red(mask)` …
|
|
348
351
|
`alpha(mask)` and `luma(mask)` read one channel.
|
|
349
352
|
- `source` is always available: the picture the chain started from,
|
|
350
353
|
before any step. So "Through a mask" as the last step shows everything
|
|
@@ -371,11 +374,11 @@ outside — 1 everywhere when nothing is selected, so a filter that uses
|
|
|
371
374
|
it works unchanged. `mix(source, rgb, selected)` as a last step keeps
|
|
372
375
|
the chain's result only inside the selection ("Only inside the
|
|
373
376
|
selection" in the library); `selection(px, py)` reads the mask elsewhere.
|
|
374
|
-
It is a mask, so a
|
|
377
|
+
It is a mask, so a color space leaves it alone. The runner takes it as
|
|
375
378
|
a picture named `selection` (white = in): `runner.setImage('selection',
|
|
376
379
|
maskCanvas)`.
|
|
377
380
|
|
|
378
|
-
The editor makes one
|
|
381
|
+
The editor's *Select & draw* tab makes one. *Trace the edge* is intelligent scissors
|
|
379
382
|
(livewire): click a few points around the object and the line between
|
|
380
383
|
them follows the edge — every pixel has a cost, low where the brightness
|
|
381
384
|
changes sharply, and the line is the cheapest path (Dijkstra) between
|
|
@@ -409,24 +412,24 @@ object. The model and its runtime (ONNX Runtime Web) are
|
|
|
409
412
|
served by the site itself — no CDN, no Hugging Face — load only when
|
|
410
413
|
asked and are then cached by the browser; the rest of the editor never
|
|
411
414
|
depends on them. Anything not among the 80 kinds is for the edge tracer,
|
|
412
|
-
the
|
|
415
|
+
the color fill or the refine brush.
|
|
413
416
|
|
|
414
417
|
The *refine* brush fixes any selection, however it was made. Paint with
|
|
415
418
|
*inside* over a part that was missed, or with *outside* over a part that
|
|
416
419
|
leaked; the boundary then settles on the nearest real edge near the
|
|
417
420
|
stroke. Under the hood, the stroke and the rim of a band round it are
|
|
418
421
|
seeds, and each side grows through the band with a cost that is tiny in
|
|
419
|
-
even
|
|
422
|
+
even color and steep across an edge (measured over three pixels, so a
|
|
420
423
|
soft edge counts in full) — whichever side reaches a pixel first claims it
|
|
421
424
|
(a seeded region competition, Dijkstra again); nothing outside the band
|
|
422
425
|
moves. With nothing selected, an inside stroke makes a selection from
|
|
423
426
|
scratch: the whole picture is in play and its border is "outside", so the
|
|
424
|
-
stroke grows to the object's edges. The size slider sets the brush. *Fill a
|
|
425
|
-
knows several
|
|
427
|
+
stroke grows to the object's edges. The size slider sets the brush. *Fill a color* is a magic wand that
|
|
428
|
+
knows several colors: each click adds that pixel's color to a set,
|
|
426
429
|
and the fill grows from every click through any pixel within the
|
|
427
|
-
tolerance of *any*
|
|
428
|
-
"connected only" off takes those
|
|
429
|
-
click a swatch to drop a
|
|
430
|
+
tolerance of *any* color in the set — a striped shirt is two clicks;
|
|
431
|
+
"connected only" off takes those colors anywhere in the picture;
|
|
432
|
+
click a swatch to drop a color, and the tolerance slider refills live. Feather softens the edge,
|
|
430
433
|
invert swaps inside and out; the selection is drawn at any export size.
|
|
431
434
|
It belongs to the picture, not the filter, so share links don't carry it.
|
|
432
435
|
|
|
@@ -471,7 +474,7 @@ forward and backward a frame at a time:
|
|
|
471
474
|
disagree with the rest are pulled back to the group's movement, and
|
|
472
475
|
each may stray only a few pixels from the group's overall move, scale
|
|
473
476
|
and turn, so the outline can't wander off;
|
|
474
|
-
- the keyframe teaches it the object's
|
|
477
|
+
- the keyframe teaches it the object's colors against a band just
|
|
475
478
|
outside, so each frame has a map of how object-like every pixel is;
|
|
476
479
|
the outline between the points follows the edge of that map (and some
|
|
477
480
|
plain brightness edges) — not every leaf behind the object;
|
|
@@ -480,7 +483,7 @@ forward and backward a frame at a time:
|
|
|
480
483
|
jumped. Between two keyframes, each frame keeps the direction that
|
|
481
484
|
scored higher.
|
|
482
485
|
|
|
483
|
-
|
|
486
|
+
In the editor's *Select & draw* tab, under *Follow the selection through the clip*, the timeline shows confidence (green sure, red not):
|
|
484
487
|
click a frame to go there, fix it ("edit this frame" puts the outline into
|
|
485
488
|
the edge tracer to drag its points, or select again), and use it as another
|
|
486
489
|
keyframe. "Fix red frames with AI" runs the chosen model on each unsure frame
|
|
@@ -500,7 +503,7 @@ The runner takes them with `runner.setImage('mask', image)`; a picture
|
|
|
500
503
|
not given is white, so a missing mask lets everything through. Pictures
|
|
501
504
|
are stretched to the canvas, so size them like the main picture.
|
|
502
505
|
|
|
503
|
-
##
|
|
506
|
+
## Color spaces
|
|
504
507
|
|
|
505
508
|
The frame holds sRGB — the numbers a screen shows. A filter can work in
|
|
506
509
|
another space instead:
|
|
@@ -520,28 +523,28 @@ on the way in only, when the expression writes sRGB itself.
|
|
|
520
523
|
| `srgb` | what the picture holds (the default) |
|
|
521
524
|
| `linear` | light rather than gamma-encoded values: blurs and mixes in it behave like a lens, bright spots stay bright |
|
|
522
525
|
| `hsv`, `hsl` | hue (0–1 around the wheel), saturation, value / lightness |
|
|
523
|
-
| `lab` | L 0–1, then a and b
|
|
524
|
-
| `oklab` | L 0–1, then a and b
|
|
525
|
-
| `ycbcr` | luma, then Cb and Cr
|
|
526
|
+
| `lab` | L 0–1, then a and b centered on 0.5 (0.5 ± value/256) |
|
|
527
|
+
| `oklab` | L 0–1, then a and b centered on 0.5 (0.5 + value) — perceptually even |
|
|
528
|
+
| `ycbcr` | luma, then Cb and Cr centered on 0.5 |
|
|
526
529
|
|
|
527
530
|
Every space keeps its values in 0–1 so they survive the 8-bit texture
|
|
528
531
|
between passes — but 8 bits of Lab or OKLab is coarse for dark, saturated
|
|
529
|
-
|
|
530
|
-
half-float textures for a chain with a
|
|
532
|
+
colors (up to ~20/255 off after the round trip), so the editor uses
|
|
533
|
+
half-float textures for a chain with a color space where the GPU
|
|
531
534
|
allows (`createFilterRunner(gl, c, { precision: 'half' })`,
|
|
532
535
|
`createCpuRunner(c, { precision: 'float' })`). A step's opacity blends
|
|
533
536
|
the result over its input in sRGB. The conversions are also functions you can use anywhere —
|
|
534
537
|
`tohsv(c)`, `fromhsv(c)`, `tolinear`, `fromlinear`, `tohsl`, `tolab`,
|
|
535
538
|
`tooklab`, `toycbcr` and their inverses, sRGB on the other side —
|
|
536
539
|
so a single expression can do it: `fromhsv(fract(red(tohsv(rgb)) + t /
|
|
537
|
-
300), green(tohsv(rgb)), blue(tohsv(rgb)))` cycles the hue.
|
|
538
|
-
|
|
540
|
+
300), green(tohsv(rgb)), blue(tohsv(rgb)))` cycles the hue. On the filters
|
|
541
|
+
page, a step's space chooser sits beside its expression box (and beside *Add a new filter*'s).
|
|
539
542
|
|
|
540
|
-
## Blur, edges and other
|
|
543
|
+
## Blur, edges and other neighborhood functions
|
|
541
544
|
|
|
542
545
|
The classic image-processing operations are functions you can use in
|
|
543
546
|
any expression, so they combine with everything else and with each
|
|
544
|
-
other.
|
|
547
|
+
other. Color-valued ones return a color like `rgb` (alpha is left
|
|
545
548
|
alone); the edge detectors work on brightness and return a number.
|
|
546
549
|
|
|
547
550
|
| Function | Result |
|
|
@@ -550,10 +553,10 @@ alone); the edge detectors work on brightness and return a number.
|
|
|
550
553
|
| `gaussian(sigma)` | Gaussian blur, weights falling off with `sigma` |
|
|
551
554
|
| `blurx(r)` `blury(r)` `gaussianx(s)` `gaussiany(s)` | the same along one axis — chain an x then a y step for a wide, fast blur |
|
|
552
555
|
| `sharpen(amount)` | the 3×3 sharpen kernel, scaled by `amount` |
|
|
553
|
-
| `dilate(radius)` `erode(radius)` | the brightest / darkest
|
|
556
|
+
| `dilate(radius)` `erode(radius)` | the brightest / darkest color within ±radius: light or dark areas grow |
|
|
554
557
|
| `convolve(k1, …, k9)` | your own 3×3 kernel, row by row |
|
|
555
558
|
| `sobel()` `sobelx()` `sobely()` | Sobel edge strength (and its horizontal / vertical parts) |
|
|
556
|
-
| `laplacian()` | the 4-
|
|
559
|
+
| `laplacian()` | the 4-neighbor Laplacian of brightness |
|
|
557
560
|
| `emboss()` | brightness relief lit from the top-left, around 0.5 |
|
|
558
561
|
|
|
559
562
|
```
|
|
@@ -561,14 +564,14 @@ gaussian(3) a blur
|
|
|
561
564
|
rgb + 1.5 (rgb - gaussian(2)) unsharp mask
|
|
562
565
|
rgb + 0.8 max(gaussian(3) - 0.6, 0) bloom: bright parts glow
|
|
563
566
|
if sobel() > 0.3 then 1 else 0 an edge map
|
|
564
|
-
rgb (1 - 3 sobel()) ink outline over the
|
|
567
|
+
rgb (1 - 3 sobel()) ink outline over the color
|
|
565
568
|
convolve(0, -1, 0, -1, 5, -1, 0, -1, 0) sharpen, as a kernel
|
|
566
569
|
dilate(2) - erode(2) morphological gradient: outlines of shapes
|
|
567
570
|
```
|
|
568
571
|
|
|
569
572
|
Every one of them takes expressions, so a parameter can drive it
|
|
570
573
|
(`gaussian(sigma)`), and `t` can too (`gaussian(2 + 2 sin(t / 20))`).
|
|
571
|
-
In a per-channel expression a
|
|
574
|
+
In a per-channel expression a color result gives that channel:
|
|
572
575
|
`{ "r": "gaussian(4)" }` blurs only red. The library in the demo has
|
|
573
576
|
each of these as a filter with knobs, and chains built from them
|
|
574
577
|
(Cartoon = Gaussian blur → Posterize → Outline).
|
|
@@ -576,11 +579,11 @@ each of these as a filter with knobs, and chains built from them
|
|
|
576
579
|
Two limits, because a shader loop needs fixed bounds: the square
|
|
577
580
|
functions reach ±8 pixels, so `gaussian(sigma)` inside an expression is
|
|
578
581
|
truncated above `sigma ≈ 2.7`; the one-axis functions reach ±24. A step
|
|
579
|
-
whose whole
|
|
582
|
+
whose whole color is just `gaussian(sigma)` or `blur(radius)` is
|
|
580
583
|
computed separably for you — an x pass then a y pass, 2n taps instead
|
|
581
584
|
of n², reaching ±24 — so the plain blur is the fast, wide one; write
|
|
582
585
|
`gaussianx`/`gaussiany` yourself only when you want one axis.
|
|
583
|
-
Each call samples the
|
|
586
|
+
Each call samples the neighborhood again, so `gaussian(3)` used twice
|
|
584
587
|
in one expression costs twice — put it in its own step if that matters.
|
|
585
588
|
"Canny edges" in the library is a chain of these: a Gaussian, then
|
|
586
589
|
gradient strength and direction (`atan2(sobely(), sobelx())` into four
|
|
@@ -592,7 +595,7 @@ expressions, so it runs on the CPU too.
|
|
|
592
595
|
## Running it
|
|
593
596
|
|
|
594
597
|
The editor runs a filter on the GPU (WebGL) when the browser has it, and on the CPU otherwise —
|
|
595
|
-
*Advanced* → *runs on* lets you choose. Both give the same pixels; the GPU is much faster, so big
|
|
598
|
+
in the editor, the *Download* tab's *Advanced* → *runs on* lets you choose. Both give the same pixels; the GPU is much faster, so big
|
|
596
599
|
pictures and video want it.
|
|
597
600
|
|
|
598
601
|
<!-- dev -->
|
|
@@ -618,7 +621,7 @@ language, types and sampling as the shader (bilinear `pixel()`, edges
|
|
|
618
621
|
clamped, 8-bit rounding between passes), so both give the same pixels:
|
|
619
622
|
`npm run check -- --cpu` compares them on every library filter (all within
|
|
620
623
|
1/255 apart from a handful of pixels exactly on an edge; `random()` can't
|
|
621
|
-
match — GPUs compute the hash's `sin` in lower precision).
|
|
624
|
+
match — GPUs compute the hash's `sin` in lower precision). Neighborhood
|
|
622
625
|
functions with an argument that is the same for every pixel (`blur(radius)`)
|
|
623
626
|
are computed once per pass, separably; one that changes per pixel
|
|
624
627
|
(`blur(x * 8)`) is computed pixel by pixel and is slow. Rough single-thread
|
|
@@ -628,7 +631,7 @@ workers is 3–5× faster on the heavy ones.
|
|
|
628
631
|
|
|
629
632
|
The generated code is compiled only after MathZetta's `checkGeneratedJs`
|
|
630
633
|
passes it (no names but its own, no strings, only its own arrays indexed),
|
|
631
|
-
so a filter from a share link can read the picture and write
|
|
634
|
+
so a filter from a share link can read the picture and write colors,
|
|
632
635
|
nothing else. Pages whose Content-Security-Policy forbids `'unsafe-eval'`
|
|
633
636
|
can't use the CPU runner.
|
|
634
637
|
|
|
@@ -673,9 +676,9 @@ to a canvas, upload it, draw, and read the canvas back (`toBlob` or
|
|
|
673
676
|
|
|
674
677
|
## Sharing filters
|
|
675
678
|
|
|
676
|
-
A filter is a small JSON document: the
|
|
677
|
-
knobs it exposes. Since it is only
|
|
678
|
-
anyone — it can read the picture and write
|
|
679
|
+
A filter is a small JSON document: the math, a name and author, and the
|
|
680
|
+
knobs it exposes. Since it is only math, it is safe to accept from
|
|
681
|
+
anyone — it can read the picture and write colors, nothing else.
|
|
679
682
|
|
|
680
683
|
```json
|
|
681
684
|
{
|
|
@@ -688,19 +691,19 @@ anyone — it can read the picture and write colours, nothing else.
|
|
|
688
691
|
}
|
|
689
692
|
```
|
|
690
693
|
|
|
691
|
-
- `color` is one expression for the
|
|
694
|
+
- `color` is one expression for the color, or an object with any of
|
|
692
695
|
`r`, `g`, `b`, `a`, `rgb`, `rgba` — exactly as `filterShader()` takes.
|
|
693
696
|
- `params` are the knobs. Each is a number (`"cutoff": 0.5` means
|
|
694
697
|
0–1; larger defaults get 0–2×) or `{ "default", "min", "max", "step",
|
|
695
|
-
"label" }`, or a
|
|
698
|
+
"label" }`, or a color: `"tint": [1, 0.85, 0.7]` (three numbers, 0–1).
|
|
696
699
|
A parameter is a value, never a function, so `strength (x + 1)` is a
|
|
697
700
|
product.
|
|
698
701
|
A number knob is a slider, unless it has `"options": [2, 4, 8]` (a menu
|
|
699
702
|
of those values; the default is the first unless given) or
|
|
700
|
-
`"input": "box"` (a box to type a value in — any
|
|
703
|
+
`"input": "box"` (a box to type a value in — any math that works out
|
|
701
704
|
to a number, such as `1/3`).
|
|
702
705
|
- `notes` is the filter's page, shown in its step in the editor: a list of
|
|
703
|
-
Markdown texts (
|
|
706
|
+
Markdown texts (math in `$…$` is drawn by Inline Math, and `{{cutoff}}`
|
|
704
707
|
inside it is a knob's value) and `{ "param": "cutoff" }` entries that
|
|
705
708
|
place a knob's input among them; knobs it doesn't place come first.
|
|
706
709
|
Notes never change the picture.
|
|
@@ -711,11 +714,11 @@ anyone — it can read the picture and write colours, nothing else.
|
|
|
711
714
|
- `name`, `author`, `description` are free text; `format` says which
|
|
712
715
|
version of this format the document follows.
|
|
713
716
|
|
|
714
|
-
|
|
715
|
-
you can edit (shown while the step's *Edit* is on): `cutoff := [[0..1 step 0.01 = 0.5]]` (a slider: from, to,
|
|
717
|
+
On the filters page each knob is written as an Inline Math variable, one line
|
|
718
|
+
you can edit (shown while the step's *Edit* is on; the editor shows only the knob): `cutoff := [[0..1 step 0.01 = 0.5]]` (a slider: from, to,
|
|
716
719
|
its step, where it starts), `radius := [[3]]` (a box), `levels := [[2 | 4 | 8]]`
|
|
717
|
-
(a menu) or `tint := #ff8800` (a
|
|
718
|
-
in the
|
|
720
|
+
(a menu) or `tint := #ff8800` (a color). Renaming a knob there renames it
|
|
721
|
+
in the math too.
|
|
719
722
|
|
|
720
723
|
### Combining filters
|
|
721
724
|
|
|
@@ -832,7 +835,7 @@ only run what has a numeric meaning per pixel:
|
|
|
832
835
|
- **Names:** `x y r g b a t w h px py rgb gray unchanged` are taken, as
|
|
833
836
|
are GLSL's own words (`in`, `out`, `float`, `main`, …). Greek letters are
|
|
834
837
|
spelled out (`θ` → `theta`), primes become `_prime`. `f`, `g` and `h`
|
|
835
|
-
followed by a bracket are read as function calls, as in
|
|
838
|
+
followed by a bracket are read as function calls, as in math: write
|
|
836
839
|
`g * (1 - r)`, not `g(1 - r)`.
|
|
837
840
|
|
|
838
841
|
MathZetta's [how it works](https://github.com/triplelog/mathzetta/blob/main/packages/mathzetta/docs/how-it-works.md)
|