@woosh/meep-engine 3.9.0 → 3.10.0

package/src/avif/native/DECISIONS.md

@@ -0,0 +1,537 @@
+# Native AVIF — decision log
+
+Autonomous decisions taken while building `src/avif/native/`, with the reasoning that produced
+them. Newest entries at the bottom of each section. A decision recorded here is settled; revisiting
+one means adding a superseding entry, not editing the old one.
+
+---
+
+## Scope
+
+### D1 — Still images only; inter prediction is out of scope
+AVIF still images are AV1 key frames. Inter prediction (motion vector prediction, compound modes,
+OBMC, warped motion, masked compound, reference frame state) is roughly the same volume of code
+again as everything else in the decoder, and no AVIF still image can use it. Animated AVIF (`avis`)
+is rejected with an explicit error rather than half-decoded.
+
+**Consequence:** meep cannot decode animated AVIF through this path. It does not today either — the
+existing call sites decode single images.
+
+### D2 — Decoder implements every tool; encoder uses a subset
+The decoder must handle whatever an arbitrary encoder emitted, so it implements deblocking, CDEF,
+loop restoration, superres, film grain, palette, intra block copy and quantizer matrices. The
+encoder signals the in-loop filters off and does not use palette or IBC. These are quality tools;
+omitting them costs compression efficiency, not conformance, and each one omitted is a large body
+of search code that would ship untested.
+
+### D3 — Large-scale tile (Annex D) and the decoder model (Annex E) are not implemented
+Neither is reachable from an AVIF still image. Decoder-model fields in the sequence header are
+parsed (they must be, to find the fields after them) and discarded.
+
+---
+
+## Data and tables
+
+### D4 — Normative tables are generated, never hand-written
+All 156 constant tables come from the AOMedia AV1 specification's markdown sources via
+`tools/generate_tables.mjs`. A single mistyped digit in a CDF desynchronises the arithmetic decoder
+for the rest of the tile, and no test short of a full oracle comparison would catch it. Generation
+makes the tables auditable against their source and regenerable.
+
+Provenance and licensing: [NOTICE.md](NOTICE.md).
+
+### D5 — Large tables are base64-packed, small ones are literals
+Tables above 4096 entries (the coefficient CDFs, the quantizer matrices) are emitted as base64
+strings decoded once at module load into typed arrays. Below that they are plain typed-array
+literals, which stay readable and greppable. The threshold trades source size against
+debuggability; the quantizer matrices alone are ~130 KB of `uint8` and would dominate the module
+otherwise.
+
+### D6 — CDFs are stored in specification form
+The spec stores a CDF for `N` symbols as `N+1` values: `N-1` cumulative probabilities, then
+`1 << 15`, then an update counter. Reference implementations store the complement (`32768 - p`) to
+save an operation in their inner loop. We keep the spec form so the code reads as a direct
+transliteration of §8.3 and the tables can be diffed against the specification text by eye. The
+inner-loop cost is one subtraction.
+
+---
+
+## Representation
+
+### D7 — All samples are `Uint16Array`, regardless of bit depth
+8-bit content costs 2× the memory and forgoes any 8-bit-specific fast path. The alternative is
+templating every sample-touching function over `Uint8Array`/`Uint16Array`, which in JavaScript
+means either duplicated source or megamorphic call sites that deoptimise the whole pixel pipeline.
+One uniform path is worth the constant factor; a specialised 8-bit path can be added later behind
+the same tests if profiling justifies it.
+
+### D8 — Coefficients and residuals are `Int32Array`
+The spec's intermediate precision for the larger transforms exceeds 16 bits, and `Int32Array` maps
+onto JavaScript's native 32-bit integer semantics (`|0`, `>>`) without surprises.
+
+### D9 — Mode info is stored as parallel typed-array planes
+Per-4×4 state (mode, tx size, skip, segment id, palette size, …) is stored one typed array per
+field, indexed `mi_row * mi_stride + mi_col`. An array of objects would allocate per block and make
+the neighbour lookups — which are the hottest reads in the tile decoder — pointer chases.
+
+---
+
+## Container
+
+### D10 — Read with a `DataView` cursor, write with `BinaryBuffer`
+`core/binary/BinaryBuffer.js` is a growable stream, which is exactly right for writing and carries
+the big-endian accessors ISOBMFF needs. For reading it is the wrong shape: box parsing wants
+zero-copy random access into a buffer that already exists, and the growth machinery is dead weight.
+A thin `DataView` cursor covers reading in a few dozen lines. Using the right tool in each direction
+is not duplication.
+
+### D11 — `iloc` is written with fixed-width 32-bit offsets and a two-pass patch
+Item offsets in `iloc` point into `mdat`, whose position depends on the size of the metadata that
+precedes it — including `iloc` itself. Writing offsets as fixed-width fields makes the metadata size
+invariant to the offset values, so a single measure-then-patch pass converges immediately instead of
+iterating. 32-bit offsets cap a written file at 4 GiB, which is far beyond anything meep produces;
+the writer throws if a payload would exceed it rather than silently truncating.
+
+### D12 — Written files use construction method 0 (file offsets)
+Method 1 (`idat`) and method 2 (item references) are supported on read because real files use them,
+but writing always uses plain file offsets into `mdat`. It is what every decoder handles best and it
+keeps the writer's layout logic linear.
+
+---
+
+## Colour and HDR
+
+### D13 — The existing HDR convention is preserved exactly
+`encoder_worker.js` encodes engine-linear float16 as PQ at 12 bits, 4:2:2, signalling BT.709
+primaries / PQ transfer / BT.709 matrix, with engine linear scaled by `PQ_SDR_WHITE_LINEAR` before
+`linear_to_PQ`. Alpha is mapped linearly to the full 12-bit range. Assets already written by meep
+depend on this, so the native encoder reproduces it as its HDR default and the native decoder reads
+it back to identical values.
+
+Note on the existing convention: BT.709 primaries with PQ transfer is unusual — PQ content is
+normally BT.2020 — but changing it would make already-written assets decode with wrong chromaticity.
+The encoder therefore keeps BT.709 as the default and exposes primaries as an option.
+
+### D14 — Decoded HDR is returned in engine normalisation
+`avif_to_sampler2d` returns linear float16 where `1.0` is SDR reference white (203 nits), matching
+`PQ_SDR_WHITE_LINEAR`'s convention and the rest of the engine. The raw PQ-normalised values are
+available from `decode_avif` for callers that want them.
+
+### D15 — Colour conversion is float, and its tolerances are stated
+The codec layer is bit-exact (I4). The colour layer is not and cannot be: transfer functions are
+transcendental. Each conversion function documents its expected round-trip error, and colour specs
+assert against those tolerances rather than exact equality.
+
+---
+
+## API
+
+### D16 — The public API is synchronous with no initialisation step
+No `init()`, no module instantiation, no promise. This is the main ergonomic gain over the WASM
+path and it falls directly out of being pure JS. Callers needing responsiveness drive it from a
+worker, exactly as `threaded_image_encoder.js` already does.
+
+### D17 — A step-wise encoder driver is offered alongside the one-shot call
+`encode_avif` runs to completion synchronously. For callers that must stay responsive without
+threads, the encoder also exposes a driver that advances one superblock row per call, so the work
+can be spread across frames. Same code path, different pump.
+
+---
+
+## Testing
+
+### D18 — The WASM reference is used offline to mint fixtures, not called from specs
+The libavif WASM glue is built for the web only: it asserts on the node environment and needs
+`window`/`self`/`document` shims plus temporarily removing `globalThis.process` to load. Doing that
+inside vitest is hostile to the runner. `tools/generate_fixtures.mjs` performs the shimming once,
+offline, and commits small `.avif` files with pixel-dump sidecars. The committed suite stays pure,
+deterministic and portable.
+
+### D19 — Fixture generation deliberately sweeps encoder settings
+Default libaom settings exercise a narrow slice of the format. Fixtures are minted across bit
+depths, subsamplings, lossless, tiling, CDEF/loop-restoration on and off, film grain and superres,
+so the decode paths that only appear under unusual settings are covered rather than assumed.
+
+---
+
+## Implementation choices made while building
+
+### D20 — `BitReader` and `BitWriter` are classes with methods, not structs plus free functions
+The rest of the codec follows C2 (structs carry fields, free functions carry behaviour). Stream
+cursors are the exception, because a cursor's entire purpose is the coupling of a position to a
+buffer, and because meep already spells that shape as a class in
+`core/binary/BinaryBuffer.js`. Matching the existing precedent beats internal consistency with a
+rule written for the pixel pipeline.
+
+### D21 — Item property structs live together in one file
+`ItemProperty.js` holds all fifteen property structs. They are a family, each is a handful of
+fields, and splitting them across fifteen files would make the set harder to read without making
+anything more testable. Reading and writing stay in separate files, so the decoder tree can be
+imported without the writer.
+
+### D22 — Tables are generated from the specification's markdown, not from a reference decoder
+`tools/generate_tables.mjs` parses the published `av1-spec` markdown. That source is normative and
+stores CDFs in specification form already, so no conversion is needed and the generated numbers can
+be diffed against the specification text by eye. A reference implementation would have meant
+translating from its own internal representation, which is one more place to be wrong.
+
+### D23 — The generator recovers from a defect in the published markdown
+`Split_Tx_Size` in `10.additional.tables.md` is missing the comma between its thirteenth and
+fourteenth entries. Rather than carrying a patch list against the source, the initialiser parser
+falls back to treating whitespace as a separator when a token will not evaluate as a whole but each
+of its words will, and reports every recovery it makes. `av1/tables/tables.spec.js` pins the
+resulting values so the recovery cannot silently change.
+
+### D24 — Generated tables are validated structurally, not by transcription
+`tables.spec.js` checks that every one of the 96 default CDF tables is a valid cumulative
+distribution — strictly increasing, terminating at `1 << 15`, with a zeroed adaptation counter —
+that every scan table is a permutation of its positions, that the transform and block geometry
+tables agree with each other, and that the cosine lookup matches its closed form. These are
+properties a mis-parse would violate, which is a far stronger check than spot-checking values
+against the text.
+
+### D25 — The decoder tolerates non-zero byte-alignment padding; the reader does not
+The specification says the padding `byte_alignment()` skips "shall be equal to 0". libaom does not
+comply: it rounds a frame header up to a whole byte by advancing its write cursor, so inside an
+`OBU_FRAME` those bits are whatever was already in its output buffer. `BitReader.byte_alignment`
+keeps the strict check, because our own writer must satisfy it; the decoder calls
+`skip_to_byte_boundary` instead. Refusing such files would reject a large share of the AVIFs that
+exist over bits nothing reads.
+
+### D26 — Fixture properties are measured from the encoded file, never assumed
+libavif silently overrides a requested identity matrix to BT.601 for anything but a lossless
+encode. Fixtures generated with `matrixCoefficients: 0` at quality 90 are therefore *not* an
+identity oracle, and comparing their planes to the reference RGBA as though they were produced a
+convincing but entirely wrong failure. `tools/generate_fixtures.mjs` now parses each file it mints
+and records what it actually contains — matrix, subsampling, and which post-processing filters the
+frame header enables — and `exact` is derived from that.
+
+Monochrome is the other exact shape, and the more useful one: with a single plane every output
+channel is the luma, whatever the signalled matrix, so a monochrome fixture is an exact oracle for
+the luma path at *any* quality. That is what makes lossy coefficient coding testable at all.
+
+## D27. Fixtures are synthesised where the reference encoder will not cooperate
+
+CDEF and superres had no oracle: the libavif build here emits `enable_cdef = 0` in every sequence
+header it writes, and never asks for superres. Lossy chroma had none either, because libavif
+overrides identity matrix coefficients for anything that is not lossless, so every exact fixture was
+monochrome or lossless.
+
+Rather than ship three untested filters, `tools/synthesise_fixtures.mjs` rewrites the headers of an
+already-encoded stream and leaves the tile data untouched. That is sound for exactly three changes:
+
+- **Matrix coefficients** decide how the planes are interpreted, not how they are coded. Retargeting
+  a lossy 4:4:4 stream to the identity matrix makes the reference decoder hand back the planes
+  themselves, which turns it into an exact oracle for chroma.
+- **CDEF parameters** live in the uncompressed frame header. With `cdef_bits` at zero the
+  per-superblock `cdef_idx` is a zero-bit read, so the tile data is bit-identical.
+- **Superres** signals the *upscaled* width and derives the coded width from it. Choosing an
+  upscaled width that derives back to the width the encoder used leaves the coded frame — and so the
+  tile data — exactly as it was.
+
+This is not a self-consistency check. The reference decoder still produces the expected samples, and
+it has never seen this decoder's output. What it cannot cover is a bug that lives in the tile data
+for those configurations, which is why the encoder phase will cross-check the other direction.
+
+## D28. Intra block copy is refused rather than half-supported
+
+`allow_intrabc` predicts a block from elsewhere in the same frame with a block vector, which needs
+the motion vector machinery this decoder otherwise has no use for. Screen-content AVIF that uses it
+is rare, and a decoder that silently mis-predicted would be worse than one that says so. The stream
+is refused by name; if a real file turns up that needs it, the vector reading is a contained
+addition to the tile decoder.
+
+## D29. The forward transform is measured from the inverse, not transliterated
+
+AV1 specifies only the *inverse* transform. Every encoder invents a forward one, and the only
+property that matters is that the normative inverse turns its output back into the picture.
+
+Rather than transliterate libaom's integer butterflies — several hundred lines, and correct only if
+every one of them is — `forward_transform_2d.js` probes the specification's own inverse with unit
+impulses and inverts the resulting matrix. It is a dozen lines, it gives the ADST for free, and it
+is exactly as correct as the inverse it was measured from.
+
+The measured map is very slightly non-linear, because the integer butterfly rounds internally, so
+the round trip is exact to within one step rather than exactly. Quantisation swallows that. At
+quantizer index zero, where nothing may be lost, the lossless path uses the Walsh-Hadamard pair,
+which is exactly invertible — and is tested to be.
+
+## D30. The encoder mirrors a subset of the decoder's syntax, and shares its contexts
+
+An encoder has to make the same decisions a decoder makes, in the same order, from the same state.
+Where that state is *derived* — coefficient contexts, partition CDF selection, the intra edge filter
+type, the block decoded flags — the derivation is now in a module both sides import, rather than
+written twice. A context that drifted by one would desynchronise the arithmetic coder several blocks
+later, somewhere that looks nothing like the mistake.
+
+The traversal itself is written twice, because the encoder interleaves decisions with coding and
+inverting the decoder's control flow to serve both would be worse than a mirror. What keeps the two
+honest is that the encoder reconstructs through the decoder's own `reconstruct`, so its picture of
+what it has coded is the decoder's picture by construction, and that every test decodes what the
+encoder wrote.
+
+The syntax the encoder emits is deliberately narrow: one tile group, no palette, no filter intra, no
+segmentation, no delta quantisers, no deblocking, largest-transform mode, blocks from 8x8 to 32x32,
+five intra modes. Everything it does emit, the decoder reads — and so does libavif.
+
+## D31. Compression is within about 1.5x of libavif, and that is a separate piece of work
+
+At matched quality this encoder produces files around 1.4 to 1.6 times the size of libavif's. The
+gap is decisions, not correctness: no rate-distortion search over modes, transform types or
+partitions, no trellis over coefficient levels, no rectangular partitions, and a fixed dead zone
+rather than a tuned one. Each of those is an addition that changes no interface and breaks no test —
+which is why the first version is the one that is provably correct rather than the one that is
+nearly as small.
+
+## D32. Two normalisations, one boundary, and each layer says which it speaks
+
+There are two defensible meanings for a linear `1.0` here and the module needs both, so the question
+is not which one wins but where they meet.
+
+**The colour layer keeps a hundred nits.** `transfer_to_linear` and `linear_to_transfer` implement
+H.273 table 3, and the SDR curves in it — BT.709, sRGB, gamma 2.2, gamma 2.8 — are defined against a
+hundred nit reference display. Signal `1.0` decoding to linear `1.0` is those curves' own fixed
+point, not a convention this module picks. Renaming `LINEAR_REFERENCE_NITS` to `SDR_WHITE_NITS`
+would land sRGB white at 0.4926 and turn a statement about a standard into a statement about meep,
+so it stays at 100 and says why in its own doc comment.
+
+**The renderer-facing pair speaks the renderer's white.** meep's `1.0` is SDR reference white, 203
+nits per BT.2408, which is what `PQ_constants.js` documents and what every sampler in the engine
+counts in. `sampler2d_to_avif` and `avif_to_sampler2d` are the boundary, and each names its white to
+the codec through `linear_reference_nits` — an option on `encode_avif` and on `convert_to_rgba` that
+says what a linear `1.0` on *the caller's* side stands for. Both pass `SDR_WHITE_NITS`, so the pair
+is an exact inverse in renderer units and visibly so: the two calls read the same. It is tested in
+`api/sampler2d_round_trip.spec.js`, which also pins the file itself down to 203 nits so a later
+"simplification" cannot make the two ends agree on the wrong white.
+
+Naming the white rather than multiplying at the call site is what buys the symmetry. It also keeps
+the scaling in the one loop that already touches every sample, so the decode side costs a multiply
+rather than a second pass through half float.
+
+### What this changes, and for whom
+
+Encoding is byte-for-byte what it was: the factor moved from `sampler2d_to_avif`'s pixel loop into
+`encode_avif`'s, and the file still carries renderer `1.0` as 203 nits. Decoding is the fix, and it
+is a real behaviour change — `avif_to_sampler2d` now returns 2.03 times less than it did.
+
+That 2.03 was not a convention anybody chose. It predates this module: the WebAssembly pair had it
+too, because the old `encoder_worker.js` multiplied by `PQ_SDR_WHITE_LINEAR` before `linear_to_PQ`
+while libavif's "PQ to linear" normalises to a hundred nits. The native codec reproduced it
+deliberately so that swapping the codec changed no picture; this is the follow-up that was owed.
+
+Nothing downstream was compensating for it. `decode_image_source_hdr` and `deserialize_scene` pass
+the sampler straight through; `texture_write_to_gpu`'s note about Uint16 samplers is about half
+float *bit patterns*, not about normalisation, and its premultiply is unaffected; there is no
+exposure or scale anywhere in `src/shade` carrying a reciprocal. Auto-exposure would have absorbed
+some of it on an environment map, which is probably why it survived — but an albedo map has no such
+cover, and 2.03 there is simply blown out.
+
+**No migration.** Scenes on disk are correct as written: their PQ AVIFs say 203 nits and mean it,
+and it was only the reader that misread them. Rewriting the files would preserve the bug rather than
+fix it. The visible change is a one-off darkening of HDR image sources back to the brightness they
+were authored at, and content tuned by eye against the old reader will want re-checking. The
+compounding is the argument that little was: a scene saved and reloaded got 2.03 times brighter each
+round, so nothing can have been held to the old behaviour for long without blowing out.
+
+## D33. Orientation is applied on request; a gain map falls back to its base picture
+
+Two things a real file carries that the coded picture does not:
+
+- **`irot` and `imir`** say the picture is stored one way round and meant to be seen another, which
+  is what a camera writes rather than rotating the samples. libavif hands them to its caller instead
+  of applying them, so applying them unconditionally would rotate twice for any caller that already
+  honours orientation. `decode_avif` takes `apply_transformations` and defaults it off, which keeps
+  the behaviour of the decoder being replaced and makes the capability available to anything that
+  wants it. `clap`, the clean aperture, is left alone: it is a crop expressed as four rationals, it
+  is rare, and cropping by a misread fraction is worse than not cropping.
+- **A `tmap` primary item** is a base picture plus a gain map — increasingly what a phone writes for
+  HDR. Reading the gain map is a piece of work in its own right; falling back to the base picture is
+  what the format expects of a decoder that does not, and it produces a correct picture rather than
+  a broken one. That fallback is here; the gain map is not.
+
+## D34. The compression gap is closed to about 1.08x, by searching rather than by guessing
+
+Supersedes D31, and the clauses of D2 and D30 that describe what the encoder does not do.
+
+D31 recorded the gap as "around 1.4 to 1.6 times libavif at matched quality". That number was
+measured by encoding both at the same quality setting and comparing two sizes, which understates it:
+two encoders' quality scales are not the same scale, and at quality 75 this encoder was 1.4 times
+the size *and* 2.7 dB worse. Swept properly — size against fidelity, compared where the curves
+overlap — the gap on photographic content was nearer **2.2 times**. `tools/measure_encoder.mjs` is
+that measurement, and it takes real photographs on the command line since none can be committed here.
+
+It now stands at about **1.08 times** when the file is allowed to ask the reader for everything,
+and at **1.13 times** as this codec ships — which asks for as little as possible, for the reasons in
+D36. At the thorough setting two of the eight pictures come out smaller than libavif's. What closed it, in the order it was closed and with what each was worth:
+
+| | rate for the same fidelity |
+|---|---|
+| where D31 left it | +112% |
+| rate-distortion mode decision, over all thirteen intra modes | +79% |
+| rate-distortion partition decision | +38% |
+| transform type search | +33% |
+| trailing coefficients trimmed against their cost — *withdrawn, see D35* | +31.7% |
+| a deblocking level chosen after coding | +29.5% |
+| a directional filter strength chosen after coding | +24.0% |
+| blocks cut in half as well as in quarters | +21.1% |
+| the trim withdrawn, and the search stripped of work it did not need | +20.8% |
+| a transform size chosen per block rather than always the largest | +18.5% |
+| chroma predicted from the luma already coded | +9.7% |
+| blocks down to four by four, sharing their chroma in pairs | +8.6% |
+| a loop restoration filter fitted to the finished picture | +7.9% |
+
+The four T-shaped partitions and the two quarterings are implemented and are *not* on by default:
+they are worth 1.2 and 0.6 points respectively and cost two thirds of the encoder's time between
+them, so the balanced setting offers the two halves and the thorough setting offers all eight.
+
+### The one idea the rest are applications of
+
+An encoder cannot choose on rate without knowing the rate, and the only thing that knows the rate is
+the syntax. So a `SymbolWriter` has a costing mode: `reset_costing` puts it into one where writing a
+symbol adds up what writing it would have cost and emits nothing. `write_coefficients`, and every
+other piece of syntax the encoder emits, takes a writer and does not know which kind it has. Every
+rate any decision here is made on is the rate of exactly the symbols that decision would emit,
+through exactly the contexts they would meet, because it is the same function that emits them. A
+cost model written alongside the syntax would be a second implementation of it, and would drift.
+
+It was two classes before it was a mode. One writer with a branch measured faster than two writers
+with a shared interface, because a call site that sees one class stays monomorphic and a call site
+that sees two does not — which is a fact about how JavaScript is run rather than about codecs, and
+it is worth the mode.
+
+The partition decision needs more than that: it has to price answers that have not been made, and a
+block's price depends on the reconstruction and the contexts the block before it left. So the
+search *makes* each answer — predicts, quantises, reconstructs, advances the contexts — measures it,
+and puts the state back. `Av1BlockSnapshot` is what "puts the state back" means. Each superblock is
+therefore coded twice, once to decide and once to write, and the two agree because the second starts
+from exactly the state the first did and replays what the first recorded.
+
+Loop restoration then made the *frame* work that way as well. Its filters can only be fitted to the
+finished picture, and the format codes them inside the tile — so the decisions of a whole frame are
+kept and the tile is written a second time from them. That is why the record of what the search
+chose is frame-wide rather than a superblock at a time.
+
+### What is measured rather than derived
+
+Three constants that could have been guessed and are not:
+
+- **Lambda is 0.0016 of the squared quantiser step.** Encode at two quantisers, divide the change in
+  squared error over the coded planes by the change in coded bits. It comes out within a sixth of
+  that across synthetic and photographic pictures and three decades of quantiser, because it is a
+  property of the format's quantiser tables rather than of any picture.
+- **A sixty-fourth of the coefficient domain's squared error is the sample domain's.** The
+  specification's inverse transform is orthonormal up to a scale, and the scale is the same for
+  every size and type this encoder emits. That is what lets a coefficient be priced without running
+  the inverse transform to find out what it was worth.
+- **Ranking by squared error beats ranking by SATD** at every equal-time point, which is the
+  opposite of the usual advice. A Hadamard transform does rank better — one priced mode with it is
+  worth about two without — but it costs more per candidate than it saves in trials. Measured, and
+  then removed.
+
+### What this costs, and where the decision about that lives
+
+Encoding is an order of magnitude slower than it was: 260 ms/MP became about 6 s/MP at the balanced
+setting, or eleven times the WebAssembly libavif build at speed 6 measured beside it. That is what
+searching costs in JavaScript, and it is a knob rather than a fact — `effort` on `encode_avif`
+selects between +15.7% at about 2.5 s/MP and +2.4% at about 25 s/MP, and the eight settings behind
+it are on the encode context for anything finer.
+
+Two thirds of what is left is the forward transform, the coefficient syntax, and the *inverse*
+transform the search runs to reconstruct what it is pricing. None of those is waste; each trial
+needs all three. The transform has since been folded along the symmetry of its own basis, which took
+`n * n` multiplies to a little over `n * n / 3` — and moved the whole encoder by four per cent,
+because the work is spread across twenty functions rather than concentrated in one.
+
+Decoding is where the *reader* pays, and what it pays is not the same axis. See D36: it is
+`decode_effort`, it defaults to asking for as little as possible, and the numbers are there.
+
+### What is still not done
+
+The self-guided restoration filter, which fits noise rather than ringing and would cost a three-way
+symbol on every unit whether either filter wins or not. Palette mode and intra block copy, which are
+screen-content tools this decoder does not implement either (see D28). Angle deltas on directional
+modes, and filter-intra. Each is worth low single digits on photographs and none of them changes an
+interface.
+
+The trellis is not on that list. It was built, measured, and removed: see D35.
+
+## D35. A trellis over coefficient levels measured worse than rounding, and is not there
+
+Quantisation rounds at 0.45 of a step rather than 0.5, which is `ec.rounding`, and that is the whole
+of what this encoder does about coefficient levels. It is not for want of trying the other thing.
+
+The obvious next move from a rate-distortion mode decision is a rate-distortion *coefficient*
+decision: walk the transform block's levels, and for each one ask whether coding it is worth what it
+costs. The cheap form of it — trim the trailing coefficients, since dropping a tail also moves the
+end-of-block position and saves the symbols that would have carried it — was implemented, measured
+at the time as worth 1.3 points, and shipped.
+
+Measured again, more carefully, it was worth **less than nothing**: +20.7% without it against +21.4%
+with, and five times as many calls into the coefficient syntax to reach that. Two things had hidden
+it. The first measurement priced a trial by what the quantiser said it lost rather than by
+reconstructing it and subtracting the source, and the two differ by eight per cent on a partition
+decision — enough to reverse a result this size. And the trim had been landing before the
+rate-distortion partition search, which then made a *different* set of partitions, so the two
+measurements were not of the same encoder.
+
+A full trellis — every level, not just the tail, with the contexts each choice leaves for the next —
+would be a better answer than the trim was. It would also be several times the coefficient syntax's
+current cost, which is already a fifth of the encoder, to chase something the 0.45 rounding is
+already collecting most of. libaom's own trellis is off below its speed 4 for the same reason.
+
+The number that matters is the one in the table in D34: the row for the trim reads +31.7%, and the
+row for withdrawing it reads +20.8%. It was not a wash. It was a loss both ways.
+
+## D36. What the file asks of the reader is a separate dial, and it defaults to asking for little
+
+AV1's three in-loop filters are the only coding tools whose cost lands on the decoder rather than
+the encoder. They are chosen after the picture is coded, they change no block's decisions, and each
+one is a pass over the whole frame that whoever opens the file has to run. Measured over eight
+pictures against libavif at speed 6, at the balanced encode setting, with decode times on one
+megapixel of the same picture:
+
+| | rate | decode |
+|---|---|---|
+| no in-loop filters at all | +15.7% | 100 ms |
+| deblocking — `DECODE_FAST` | +13.1% | 115 ms |
+| and the directional filter — `DECODE_BALANCED` | +8.6% | 294 ms |
+| and loop restoration — `DECODE_THOROUGH` | +7.9% | 350 ms |
+
+Deblocking is nearly free and worth two and a half points, so it is in all three. The directional
+filter is 4.5 points for **172 ms a megapixel** — its direction search runs over every 8x8 block of
+the frame — and restoration is another 0.7 points for 56 ms.
+
+### Why the default is the cheap end
+
+This codec exists to move textures into a renderer. A texture is decoded on the machine that will
+draw with it, once per load, competing for CPU with everything else that machine is doing at the
+moment it is needed. A megapixel texture five per cent larger costs a few kilobytes of transfer —
+tens of milliseconds on a slow connection, and nothing at all after the first time, because it is
+cached. The filters cost a fifth of a second of that machine's CPU on *every* read. The two are two
+orders of magnitude apart, and they point the same way for anything decoded more often than it is
+transferred.
+
+Reverse it for anything whose bytes are the scarce thing — an image served to many readers over a
+network someone pays for by the gigabyte, where a fifth of a second on each of their machines is not
+your cost and five per cent of every transfer is. `decode_effort` on `encode_avif` is that reversal,
+and `DECODE_THOROUGH` is the smallest file this encoder makes.
+
+### Why it is not `effort`
+
+Encode effort spends this machine's time; decode effort spends the reader's. They are independent —
+a slow, careful search can produce a file that is cheap to read, and a hasty one can produce a file
+that is expensive to read — so conflating them would mean neither could be chosen. They were
+conflated once, and it showed: the fast encode setting turned the directional filter *on*, which is
+the opposite of what anything asking for speed wants.
+
+The finer knobs stay where they were. `cdef_trials` and `restoration_trials` are how hard the
+*encoder* looks once a filter is in play — twelve directional strengths rather than three is worth
+0.8 points for a fifth more encode time — so they belong to effort, and zero on either still turns
+its filter off outright.
+
+### What it does not change
+
+Fidelity at a given quality setting. What the filters buy is rate at matched fidelity: the same
+picture in fewer bytes, not a better picture. And nothing about conformance — `verify_encoder`
+carries a case at each of the three settings, and libavif decodes all sixteen to the same samples
+this decoder produces.

package/src/avif/native/NOTICE.md

@@ -0,0 +1,61 @@
+# Native AVIF — third-party notices and provenance
+
+## Normative constant tables
+
+The constant tables under [`av1/tables/`](av1/tables/) are normative values from the
+
+> **AV1 Bitstream & Decoding Process Specification**, version 1.0.0 with Errata 1,
+> The Alliance for Open Media.
+> <https://aomediacodec.github.io/av1-spec/>
+
+They were generated from the specification's published markdown sources
+(<https://github.com/AOMediaCodec/av1-spec>) by a script that is no longer in the tree — see
+[Where the development tools went](#where-the-development-tools-went) — which carry the following
+notice:
+
+> Copyright (c) 2017–2021, Alliance for Open Media. All rights reserved.
+>
+> This Bitstream Specification is licensed under the Alliance for Open Media Patent License 1.0
+> and the accompanying BSD 2-Clause license for the reference software. Redistribution and use in
+> source and binary forms, with or without modification, are permitted provided that the above
+> copyright notice and this permission notice appear in all copies.
+
+The tables are the bitstream format itself — the values that any conformant implementation must
+use. They are reproduced here for that purpose and for no other.
+
+## Decoding and encoding processes
+
+The processes in `av1/` are written from the specification's pseudocode. Section references appear
+in the JSDoc of each function that implements a normative process, in the form
+`@see AV1 spec 7.11.2`, so any function can be checked against its source.
+
+No source code from libaom, dav1d, libgav1 or libavif is copied into this module.
+
+## Test oracle
+
+The fixtures under `__fixtures__/` were minted by a development-only script that drove libavif's
+WebAssembly builds. Those builds are libavif with libaom and are covered by their own licences, as
+is their output — which is what those fixtures are. Nothing in the shipped `native/` code path ever
+imported them, and no code from them is reproduced here.
+
+## Where the development tools went
+
+Neither the reference builds nor the scripts that drove them are in the tree any more. They were
+development artefacts: they shipped to nobody, and the things they produced — the constant tables,
+the fixtures — are committed and read directly. One command brings all of them back:
+
+    git checkout ba1e2de5f -- src/avif/codec src/avif/native/tools
+
+That commit is the last one carrying both. What needs them is re-deriving something rather than
+running anything: regenerating `av1/tables/*.js` from the specification, minting a fixture,
+re-checking that libavif decodes what this encoder writes, or measuring this encoder's rate against
+libavif's. The test suite needs none of it.
+
+`DECISIONS.md` still refers to those scripts by name, and should: it is the record of how the
+fixtures were minted and how the interop claims were checked. Both remain true.
+
+## AV1 patent licence
+
+Use of the AV1 format is governed by the Alliance for Open Media Patent License 1.0
+(<https://aomedia.org/license/patent-license/>). This implementation makes no change to the
+licensing position that already applied to meep's use of AVIF through the WebAssembly builds.

package/src/avif/native/api/AvifDecoder.d.ts

@@ -0,0 +1,67 @@
+/**
+ * Somewhere to assemble a grid of tiles, or to hold a frame that outlives the context that decoded
+ * it.
+ *
+ * @author Alex Goldring
+ * @copyright Company Named Limited (c) 2026
+ */
+export class PlaneStore {
+    /** @type {Uint16Array[]} */
+    planes: Uint16Array[];
+    /** @type {Int32Array} */
+    stride: Int32Array;
+    /** @type {number} */
+    width: number;
+    /** @type {number} */
+    height: number;
+    /**
+     * @param {number} width
+     * @param {number} height
+     * @param {number} subsampling_x
+     * @param {number} subsampling_y
+     * @param {number} num_planes
+     */
+    resize(width: number, height: number, subsampling_x: number, subsampling_y: number, num_planes: number): void;
+}
+/**
+ * Everything a decode needs beyond the file itself, allocated once and reused.
+ *
+ * Colour and alpha get a frame context each. They could share one, but then decoding the alpha
+ * would overwrite the picture, and every caller would have to copy the picture out first — which is
+ * exactly the allocation this is here to avoid.
+ *
+ * @author Alex Goldring
+ * @copyright Company Named Limited (c) 2026
+ */
+export class AvifDecoder {
+    /** @type {Av1FrameContext} */
+    colour_frame: Av1FrameContext;
+    /** @type {Av1FrameContext} */
+    alpha_frame: Av1FrameContext;
+    /** @type {PlaneStore} */
+    colour_store: PlaneStore;
+    /** @type {PlaneStore} */
+    alpha_store: PlaneStore;
+    /** @type {YuvImage} the picture */
+    image: YuvImage;
+    /** @type {YuvImage} the alpha plane, which AVIF codes as an image in its own right */
+    alpha_image: YuvImage;
+    /** @type {ColourTransform} */
+    transform: ColourTransform;
+    /** @type {Float32Array} one row of Cb, at full width */
+    cb_row: Float32Array;
+    /** @type {Float32Array} */
+    cr_row: Float32Array;
+    /** @type {Float32Array} one row of interleaved R, G, B */
+    rgb_row: Float32Array;
+    /** @type {number} */
+    row_width: number;
+    /**
+     * @param {number} width
+     */
+    resize_rows(width: number): void;
+}
+import { Av1FrameContext } from "../av1/decode/Av1FrameContext.js";
+import { YuvImage } from "../color/YuvImage.js";
+import { ColourTransform } from "../color/ColourTransform.js";
+//# sourceMappingURL=AvifDecoder.d.ts.map