zamlpy 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,149 @@
1
+ # Changelog
2
+
3
+ All notable changes to `zaml` are documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [Unreleased]
9
+
10
+ ## [0.3.0] - 2026-10-22
11
+
12
+ ### Added
13
+
14
+ - **JavaScript package** (`zamljs` on npm): the same parser, compiled to
15
+ WebAssembly, for browsers, Node.js, Bun and Deno. The package is
16
+ ESM-only TypeScript with no runtime dependencies and ships its own
17
+ types. `parse` returns plain JavaScript values, with integers that do
18
+ not fit a `number` kept exact as `bigint` and `!!binary` values kept
19
+ as `Uint8Array`. `parseCbor` and `parseJson` return the raw encoded
20
+ document, and a `ZamlError` carries the line and column of a parse
21
+ failure.
22
+ - **Wasmer command** (`niranjanaryan/zaml`): the parser as a WASI
23
+ command that reads YAML on standard input and writes JSON or CBOR to
24
+ standard output (`--format json|cbor`).
25
+ - **Shared core** in `core/`: the parts of the parser that do not depend
26
+ on the host. The WebAssembly builds use it as a module, and the
27
+ Erlang NIF and the CPython extension now share its scalar
28
+ classification instead of carrying their own copy of the rules.
29
+ - A release workflow that publishes the WebAssembly modules to GitHub
30
+ releases, the JavaScript package to npm and the command to the Wasmer
31
+ registry.
32
+
33
+ ### Changed
34
+
35
+ - The WebAssembly core no longer builds a tree of values before it
36
+ encodes a document: the CBOR and the JSON encoders write the
37
+ document while the events of the parse arrive. The bytes do not
38
+ change, but a 14 MB document needs 70 MB of memory instead of 500 MB,
39
+ and the parse is 27% faster as JSON and 18% faster as CBOR.
40
+
41
+ ### Fixed
42
+
43
+ - The `strcmp` and `strncmp` of the freestanding WebAssembly build no
44
+ longer overflow when the compared strings have different lengths,
45
+ which crashed the parser on tagged scalars.
46
+ - The WebAssembly parser no longer fails on an empty document.
47
+ - The CPython extension no longer leaves a pending exception behind
48
+ when an explicitly tagged integer cannot be converted, which turned
49
+ `!!int abc` into a `SystemError` instead of the string `abc`.
50
+ - The unit tests of the shared core run again. They were imported but
51
+ never pulled into a test block, so the test binary had no tests in
52
+ it at all.
53
+
54
+ ### Known issues
55
+
56
+ - The CPython extension does not build on Windows. The C importer of
57
+ Zig cannot translate the headers of the Windows SDK yet
58
+ ([translate-c#307](https://codeberg.org/ziglang/translate-c/issues/307)),
59
+ so the import of `Python.h` fails. The other hosts are not affected.
60
+ - Zig 0.17 does not build the CPython extension yet, because its C
61
+ importer fails on the headers of the C libraries. The extension needs
62
+ Zig 0.16.x.
63
+
64
+ ## [0.2.0] - 2026-10-09
65
+
66
+ ### Python extension
67
+
68
+ - The in-repo Python extension is now a real, `pip install`-able CPython
69
+ extension built with Zig 0.16 + libyaml, compatible with CPython
70
+ 3.10–3.14 (Python 3.12's packed `PyObject` refcount is handled
71
+ adaptively). The Python package version tracks the Elixir release at
72
+ `0.2.0`.
73
+ - The Python distribution is published on PyPI as **`zamlpy`** (the `zaml`
74
+ project on PyPI is an unrelated third-party stub and cannot be reused):
75
+ `pip install zamlpy` and `import zamlpy`.
76
+ - `zamlpy.load(str_or_bytes)` returns native Python objects: maps, lists,
77
+ strings, arbitrary-precision `int`, `float`, `bool`, and `None`, resolved
78
+ with the YAML 1.2 core schema. Explicit `!!str` / `!!int` / `!!float` /
79
+ `!!bool` / `!!null` tags are honored, and anchors/aliases resolve to the
80
+ aliased object. Only the first document in a stream is returned;
81
+ malformed input raises `ValueError`.
82
+ - Packaging modernized (PEP 517/621): `pyproject.toml` now carries
83
+ `[build-system]` and full `[project]` metadata (single-sourced
84
+ `version`, `requires-python`, classifiers, URLs, SPDX license) and
85
+ `setup.py` is reduced to the Zig `Extension` + custom `build_ext`.
86
+
87
+ ### Changed
88
+
89
+ - **Performance.** Maps are now built with a single
90
+ [`enif_make_map_from_arrays`](https://www.erlang.org/doc/man/erl_nif.html)
91
+ call per map instead of a per-pair `enif_make_map_put`, eliminating the
92
+ O(n^2) rehashing on wide maps. String scalars are now returned as
93
+ sub-binaries of the input (`enif_make_sub_binary`) rather than copied
94
+ into freshly allocated binaries, so the parse no longer re-copies every
95
+ string value. Together these take the 100k-key nested benchmark from
96
+ ~0.65 s to ~0.53 s and shave the 1M-key flat benchmark to ~0.84 s,
97
+ keeping the lead over `fast_yaml` at ~3.0–3.5×.
98
+
99
+ The sub-binary path is guarded: we only sub-binary when the scalar
100
+ pointer provably lies inside the input buffer, falling back to a copy
101
+ otherwise, so the change stays correct across libyaml versions that may
102
+ write quoted/block/folded scalars into their own buffer.
103
+
104
+ ### Fixed
105
+
106
+ - **Anchors and aliases.** `&name` is now registered for scalars and for
107
+ mapping/sequence starts, and `*name` resolves to the aliased term. Anchor
108
+ names are copied into a parse-scoped arena because libyaml frees the event
109
+ buffer after each event. Previously any document using an anchor returned
110
+ `:parse_error`.
111
+
112
+ ## [0.1.0] - 2026-09-05
113
+
114
+ ### Added
115
+
116
+ - **Elixir NIF**: a Zig NIF wrapping [libyaml](https://pyyaml.org/wiki/LibYaml) for
117
+ fast YAML 1.2 parsing in Elixir.
118
+ - `Zaml.load/1` parses a YAML binary and returns the corresponding Elixir term
119
+ (maps, lists, integers, floats, booleans, `:nil`).
120
+ - YAML 1.2 *core* schema scalar resolution for ints, floats, bools, and null.
121
+ - Respects explicit `!!str` / `!!int` / `!!float` / `!!bool` / `!!null` tags.
122
+ - Anchors and aliases (`&name` / `*name`).
123
+ - Returns `:error` for non-binary input, `:parse_error` for malformed YAML,
124
+ and `:nil` for an empty document.
125
+ - NIF is compiled at `mix compile` via [`elixir_make`](https://hex.pm/packages/elixir_make)
126
+ and the project `Makefile` (also invokable as Mix task `ZamlNif.Build`).
127
+ - `mix zaml_nif.bench PATH` Mix task for head-to-head benchmarking against
128
+ `fast_yaml`.
129
+ - `Makefile` driving the `zig build-lib` invocation directly, with
130
+ `pkg-config`/Homebrew/`/usr` libyaml auto-detection.
131
+ - `mix.exs` auto-detects `libyaml` on common paths and exports
132
+ `CFLAGS`/`CPPFLAGS`/`LDFLAGS` so rebar3-compiled deps (e.g. `fast_yaml` in
133
+ the bench task) find it.
134
+ - ExUnit test suite covering basic scalars, nested maps and lists, explicit
135
+ tags, anchors, empty input, and parse-error handling.
136
+ - Cross-language benchmark suite (`benchmark/`) comparing the NIF against
137
+ `fast_yaml`, PyYAML, and `ruamel.yaml`.
138
+ - `benchmark/RESULTS.md` with the measured numbers on a 1M-line flat mapping
139
+ and a 100k-key nested mapping.
140
+
141
+ ### Notes
142
+
143
+ - Merge keys (`<<: *alias`) are not yet expanded.
144
+ - Multi-document input returns only the first document.
145
+
146
+ [Unreleased]: https://github.com/niranjanaryan/zaml/compare/v0.3.0...HEAD
147
+ [0.3.0]: https://github.com/niranjanaryan/zaml/releases/tag/v0.3.0
148
+ [0.2.0]: https://github.com/niranjanaryan/zaml/releases/tag/v0.2.0
149
+ [0.1.0]: https://github.com/niranjanaryan/zaml/releases/tag/v0.1.0
zamlpy-0.3.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Niranjan Aryan
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,6 @@
1
+ include builder.py
2
+ include zamlmodule.zig
3
+ recursive-include core *.zig
4
+ include README.md
5
+ include LICENSE
6
+ include CHANGELOG.md
zamlpy-0.3.0/PKG-INFO ADDED
@@ -0,0 +1,481 @@
1
+ Metadata-Version: 2.4
2
+ Name: zamlpy
3
+ Version: 0.3.0
4
+ Summary: Fast YAML 1.2 parser for Python, built with Zig + libyaml
5
+ Author: Niranjan Aryan
6
+ Maintainer: Adam Serafini, Niranjan Aryan
7
+ License-Expression: MIT
8
+ Project-URL: Homepage, https://github.com/niranjanaryan/zaml
9
+ Project-URL: Repository, https://github.com/niranjanaryan/zaml
10
+ Project-URL: Issues, https://github.com/niranjanaryan/zaml/issues
11
+ Project-URL: Documentation, https://hexdocs.pm/zaml
12
+ Keywords: yaml,parser,zig,libyaml,native
13
+ Classifier: Development Status :: 4 - Beta
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Programming Language :: Python :: 3.14
21
+ Classifier: Programming Language :: Python :: Implementation :: CPython
22
+ Classifier: Topic :: Software Development :: Libraries
23
+ Requires-Python: >=3.10
24
+ Description-Content-Type: text/markdown
25
+ License-File: LICENSE
26
+ Dynamic: license-file
27
+
28
+ # 🚀 zaml
29
+
30
+ A **fast YAML 1.2 parser for Elixir**, powered by a Zig NIF that wraps
31
+ [libyaml](https://pyyaml.org/wiki/LibYAML). Heavily inspired by
32
+ [`kubkon/zig-yaml`](https://github.com/kubkon/zig-yaml), with a matching
33
+ Python extension that lives in this same repository (see
34
+ [Python extension](#python-extension) below). The same parser is also
35
+ available for JavaScript (see
36
+ [JavaScript package](#javascript-package) below) and as a command that
37
+ turns YAML into JSON or CBOR (see
38
+ [Command line](#command-line) below).
39
+
40
+ ```elixir
41
+ iex> Zaml.load("foo: 1\nbar: [a, b, 3.14]\n")
42
+ %{"bar" => ["a", "b", 3.14], "foo" => 1}
43
+
44
+ iex> Zaml.load("""
45
+ ...> name: zaml
46
+ ...> tags: [yaml, elixir, nif, zig]
47
+ ...> ratio: 3.14
48
+ ...> flag: true
49
+ ...> empty: ~
50
+ ...> """)
51
+ %{"name" => "zaml", "tags" => ["yaml", "elixir", "nif", "zig"],
52
+ "ratio" => 3.14, "flag" => true, "empty" => nil}
53
+ ```
54
+
55
+ ## Why?
56
+
57
+ The most popular Elixir YAML libraries — [`yaml_elixir`](https://hex.pm/packages/yaml_elixir)
58
+ and its underlying [`yamerl`](https://hex.pm/packages/yamerl) — are
59
+ pure-Erlang ports. They're correct but slow. The existing native
60
+ option on Hex is [`fast_yaml`](https://hex.pm/packages/fast_yaml)
61
+ (processone), which is the obvious thing to compare against because it
62
+ also wraps `libyaml`.
63
+
64
+ `Zaml` is `fast_yaml` but with a thinner event-to-term bridge: the
65
+ libyaml event stream is consumed in Zig and turned into Erlang terms
66
+ in a single pass, building maps directly instead of going through
67
+ intermediate proplists. That gives a **~3–3.5× speedup over `fast_yaml`**
68
+ across flat and nested mappings, and **36–46× over the pure-Erlang
69
+ parsers**. See [benchmarks](#benchmarks) below.
70
+
71
+ The newer [`glazer`](https://hex.pm/packages/glazer) NIF is the one
72
+ library that can beat `Zaml` — it's faster on nested data (~1.3×) and
73
+ slower on a single flat 1M-key map (~2.1×). It gets there with a
74
+ hand-rolled C++ YAML parser built with profile-guided optimisation,
75
+ whereas `Zaml` deliberately stays on top of battle-tested `libyaml`.
76
+
77
+ The toolchain is also notably lighter: `Zaml` is built with the
78
+ single `zig` CLI invocation you see in the `Makefile`, with no
79
+ `./configure`, no `rebar3` port-compiler, and no autoconf.
80
+
81
+ ## Benchmarks
82
+
83
+ Two fixtures, both parsed to native Erlang terms, on an Apple M3 Pro.
84
+ Full report at [`benchmark/RESULTS.md`](benchmark/RESULTS.md).
85
+
86
+ ### 1M-line flat mapping (16 MB)
87
+
88
+ | Parser | Avg (s) | Relative |
89
+ | ----------------------------------- | -------:| --------:|
90
+ | **Zaml (NIF, libyaml + Zig)** | **0.84**| **1.00×**|
91
+ | `glazer 1.1.6` (C++ NIF, PGO) | 1.73 | 2.06× |
92
+ | `fast_yaml 1.0.40` (rebar3 NIF) | 2.54 | 3.03× |
93
+ | `yamerl 0.10.0` (pure Erlang) | 30.27 | 36.1× |
94
+ | `yaml_elixir 2.12.2` (yamerl) | 30.95 | 36.9× |
95
+
96
+ ### 100k-key nested mapping (15 MB)
97
+
98
+ | Parser | Avg (s) | Relative |
99
+ | ----------------------------------- | -------:| --------:|
100
+ | **Zaml (NIF, libyaml + Zig)** | **0.53**| **1.00×**|
101
+ | `glazer 1.1.6` (C++ NIF, PGO) | 0.41 | 0.77× |
102
+ | `fast_yaml 1.0.40` (rebar3 NIF) | 1.86 | 3.50× |
103
+ | `yamerl 0.10.0` (pure Erlang) | 23.88 | 45.0× |
104
+ | `yaml_elixir 2.12.2` (yamerl) | 24.40 | 45.9× |
105
+
106
+ Both `Zaml` and `fast_yaml` wrap the same `libyaml` C parser, so the
107
+ ~3× gap there is purely the event→term bridge. `glazer` is a
108
+ different design (a hand-rolled C++ parser, not libyaml): it wins on
109
+ nested data and loses on the wide flat map, so there's no single
110
+ "fastest" library across shapes.
111
+
112
+ ## Installation
113
+
114
+ Add `zaml` to your `mix.exs`:
115
+
116
+ ```elixir
117
+ def deps do
118
+ [
119
+ {:zaml, "~> 0.3.0"}
120
+ ]
121
+ end
122
+ ```
123
+
124
+ ### Build prerequisites
125
+
126
+ The NIF is built at compile time from source. You need:
127
+
128
+ - **Zig 0.16.x** (`brew install zig`, `apt install zig`, `asdf install zig latest`, …).
129
+ Zig 0.17 does not work yet: its C importer fails on the headers of the
130
+ C libraries.
131
+ - **libyaml** development headers
132
+ - macOS: `brew install libyaml`
133
+ - Debian/Ubuntu: `apt install libyaml-dev`
134
+ - Alpine: `apk add yaml-dev`
135
+ - Fedora: `dnf install libyaml-devel`
136
+ - An Erlang/OTP installation (any reasonably recent version, 24+)
137
+
138
+ The compiler auto-detects `libyaml` via `pkg-config` first, then falls
139
+ back to well-known locations. Override with `YAML_INCLUDE_DIR` and
140
+ `YAML_LIB_DIR` env vars or `make` variables if it lives somewhere
141
+ unusual.
142
+
143
+ ```bash
144
+ mix deps.get
145
+ mix compile # builds the NIF via elixir_make / Makefile
146
+ ```
147
+
148
+ ## Usage
149
+
150
+ ```elixir
151
+ Zaml.load(yaml_string) :: term
152
+ ```
153
+
154
+ Returns the corresponding Elixir term for the **first YAML document** in
155
+ the input. Scalar type detection follows the YAML 1.2 *core* schema:
156
+
157
+ | YAML scalar | Elixir term |
158
+ | ------------------- | ----------------- |
159
+ | `42`, `-3`, `+1_000` | integer (`42`, `-3`, `1000`) |
160
+ | `3.14`, `1.0e-3` | float (`3.14`, `0.001`) |
161
+ | `true`, `yes`, `on` | atom `true` (or `:true` quoted) |
162
+ | `false`, `no`, `off` | atom `false` |
163
+ | `null`, `~`, empty | atom `nil` |
164
+ | everything else | binary (string) |
165
+ | `!!str 42` | binary `"42"` (explicit tag respected) |
166
+ | `!!int "42"` | integer `42` (explicit tag respected) |
167
+
168
+ Sequences become lists, mappings become maps with binary keys.
169
+
170
+ Returns `:error` for non-binary input, `:parse_error` for malformed
171
+ YAML, and `:nil` for an empty document.
172
+
173
+ ### Anchor / alias example
174
+
175
+ ```elixir
176
+ iex> Zaml.load("""
177
+ ...> defaults: &d {a: 1, b: 2}
178
+ ...> prod: *d
179
+ ...> """)
180
+ %{"defaults" => %{"a" => 1, "b" => 2}, "prod" => %{"a" => 1, "b" => 2}}
181
+ ```
182
+
183
+ > **Note:** YAML merge keys (`<<: *alias`) are not yet supported.
184
+
185
+ ### Using the alternatives
186
+
187
+ For reference, here's how the same parse is done with each library the
188
+ benchmarks compare against:
189
+
190
+ ```elixir
191
+ # zaml — returns the first document as a term
192
+ Zaml.load(yaml)
193
+
194
+ # fast_yaml — returns {:ok, [doc]}; :maps gives maps instead of proplists
195
+ :fast_yaml.decode(yaml, [:sane_scalars, :maps])
196
+
197
+ # glazer — hand-rolled C++ NIF; use_nil maps null -> nil (default is :null)
198
+ :glazer_yaml.decode(yaml, [:use_nil])
199
+ # ...or the Elixir wrapper:
200
+ Glazer.YAML.decode!(yaml)
201
+
202
+ # yaml_elixir — the most widely used wrapper, over pure-Erlang yamerl
203
+ YamlElixir.read_from_string(yaml) # => {:ok, term}
204
+
205
+ # yamerl directly (raw records; yaml_elixir adds the Elixir mapper)
206
+ :yamerl_constr.string(yaml, detailed_constr: true, str_node_as_binary: true)
207
+ ```
208
+
209
+ Two other libraries in the space aren't in the parse benchmarks:
210
+ [`ymlr`](https://hex.pm/packages/ymlr) is an **encoder** (`Ymlr.document!/1`),
211
+ not a parser, and [`yamleam`](https://hex.pm/packages/yamleam) is a
212
+ pure-Gleam parser, so it isn't reachable as a plain Mix dependency.
213
+
214
+ ## Running tests
215
+
216
+ ```bash
217
+ mix test
218
+ ```
219
+
220
+ The test suite covers basic scalars, nested maps and lists, explicit
221
+ tags, anchors, empty input, and parse-error handling.
222
+
223
+ ## Running benchmarks
224
+
225
+ The repo ships a Mix task that compares the Elixir NIF against
226
+ `fast_yaml`, `glazer`, `yaml_elixir`, and `yamerl` on the same fixture
227
+ (any parser whose dependency isn't installed is skipped).
228
+
229
+ ```bash
230
+ # 1. Build the Elixir NIF + the benchmark deps
231
+ # (glazer needs a C++23 compiler and rebar3; `brew install rebar3`)
232
+ mix deps.get
233
+ mix compile
234
+
235
+ # 2. Generate the 100k-key nested fixture
236
+ python3 benchmark/gen_big_yaml.py 100000 benchmark/big.yml
237
+
238
+ # 3. Generate the 1M-line flat fixture
239
+ python3 -c "
240
+ import yaml
241
+ yaml.dump({f'a{i}': f'b{i+1}' for i in range(1_000_000)},
242
+ Dumper=yaml.CSafeDumper, default_flow_style=False,
243
+ open('benchmark/big_1m.yml', 'w'))
244
+ "
245
+
246
+ # 4. Run the head-to-head
247
+ mix zaml_nif.bench benchmark/big.yml # Zaml parse ~0.5 s (all parsers run)
248
+ mix zaml_nif.bench benchmark/big_1m.yml # Zaml parse ~0.8 s (all parsers run)
249
+ ```
250
+
251
+ A separate cross-language script (`benchmark/run_benchmark.py`) also
252
+ compares against PyYAML and ruamel.yaml if you want the Python
253
+ numbers.
254
+
255
+ ## Project layout
256
+
257
+ ```
258
+ .
259
+ ├── lib/
260
+ │ ├── zaml.ex # public Elixir API
261
+ │ └── mix/tasks/
262
+ │ ├── zaml_nif.build.ex # manual NIF build (`mix zaml_nif.build`)
263
+ │ └── zaml_nif.bench.ex # `mix zaml_nif.bench` benchmark task
264
+ ├── native/
265
+ │ └── zaml_nif.zig # the NIF (uses libyaml via @cImport)
266
+ ├── core/ # the host independent core
267
+ │ ├── scalar.zig # how a scalar and a tag resolve
268
+ │ ├── stream.zig # what the streaming encoders share
269
+ │ ├── parse.zig, parse_c.zig # the event stream, native and WASM
270
+ │ ├── cbor.zig, json.zig # the output formats
271
+ │ ├── base64.zig, emit.zig, event.zig
272
+ │ ├── value.zig # the value tree, the reference of the tests
273
+ │ └── native.zig # what the native hosts share
274
+ ├── wasm/
275
+ │ ├── main.zig # the WebAssembly module (browser)
276
+ │ ├── wasi_main.zig # the WASI command
277
+ │ ├── alloc.zig, c_abi.zig # the freestanding heap and the C ABI
278
+ │ └── shim/ # the C bridge to the vendored libyaml
279
+ ├── packages/
280
+ │ ├── zamljs/ # the npm package
281
+ │ └── wasmer/ # the Wasmer package
282
+ ├── vendor/libyaml/ # the libyaml sources the WASM builds use
283
+ ├── build.zig # builds the WebAssembly targets
284
+ ├── priv/ # compiled zaml.so lands here
285
+ ├── test/ # ExUnit tests
286
+ ├── benchmark/
287
+ │ ├── benchmark.exs # Elixir side
288
+ │ ├── run_benchmark.py # Python driver
289
+ │ ├── gen_big_yaml.py # fixture generator
290
+ │ ├── big.yml # generated 100k-key fixture
291
+ │ └── RESULTS.md # last benchmark run
292
+ ├── Makefile # `make` builds the NIF directly
293
+ └── mix.exs
294
+ ```
295
+
296
+ ## Limitations / status
297
+
298
+ - YAML merge keys (`<<: *alias`) are not yet expanded.
299
+ - Multi-document input returns only the first document.
300
+ - Not yet battle-tested on very large or adversarial input. If you find
301
+ a crash, please open an issue with the input.
302
+
303
+ ## Credits
304
+
305
+ - [`kubkon/zig-yaml`](https://github.com/kubkon/zig-yaml) — the original
306
+ pure-Zig YAML parser that inspired this project.
307
+ - [`yaml/libyaml`](https://github.com/yaml/libyaml) — the production
308
+ YAML 1.1 parser that both the Elixir NIF and the Python extension wrap.
309
+
310
+ ---
311
+
312
+ ## Python extension
313
+
314
+ The original proof-of-concept has grown into a real, `pip install`-able
315
+ CPython extension (`zamlpy` 0.3.0, Python 3.10+):
316
+ `zamlpy.load(str_or_bytes)` parses a YAML document with libyaml and returns
317
+ native Python objects. Like the Elixir NIF, it is built in pure Zig by
318
+ importing `Python.h` directly — no `clang` or setuptools C glue. It began
319
+ as the subject of a [PyCon DE 2022 talk][talk] on *Speeding Up Python with
320
+ Zig*.
321
+
322
+ [talk]: https://2022.pycon.de/program/DFWSQR/
323
+
324
+ Scalars are resolved with the YAML 1.2 core schema (`int`, arbitrary-
325
+ precision `int`, `float`, `bool`, `null`), `!!str` / `!!int` / `!!float` /
326
+ `!!bool` / `!!null` tags are honored, and anchors/aliases are supported.
327
+ Only the first document in a stream is returned; malformed input raises
328
+ `ValueError`.
329
+
330
+ ### Prerequisites
331
+
332
+ - **Python 3.10+** (CPython)
333
+ - **Zig 0.16.x**
334
+ - **libyaml** (`brew install libyaml`, `apt install libyaml-dev`,
335
+ `dnf install libyaml-devel`)
336
+
337
+ `builder.py` auto-detects libyaml under `/opt/homebrew`, `/usr/local`,
338
+ and `/usr`. Override with `YAML_INCLUDE_DIR` / `YAML_LIB_DIR` if needed.
339
+
340
+ ### Building the Python extension
341
+
342
+ The extension has no Python runtime dependencies; it links libyaml and is
343
+ built from source by `pip` (via the custom `builder.py` backend) on
344
+ install.
345
+
346
+ ```bash
347
+ python -m venv .venv
348
+ source .venv/bin/activate
349
+ pip install . # or: pip install -e . for a development checkout
350
+ python test.py
351
+ ```
352
+
353
+ > The PyPI `zaml` project (owned by a third party) is an unrelated no-op
354
+ > stub. This project's Python distribution is published as `zamlpy`.
355
+
356
+ ### Python benchmark
357
+
358
+ ```bash
359
+ cd benchmark
360
+ python run_benchmark.py
361
+ ```
362
+
363
+ On an Apple M3 Pro (Python 3.14), parsing a 15 MB YAML document:
364
+
365
+ ```text
366
+ zamlpy 0.51 s 1.0×
367
+ PyYAML CSafeLoader 9.16 s 17.8×
368
+ PyYAML SafeLoader 33.85 s 65.9×
369
+ ruamel.yaml 48.40 s 94.2×
370
+ ```
371
+
372
+ ### Cross-platform notes (Python extension)
373
+
374
+ - **macOS:** `brew install zig libyaml`.
375
+ - **Linux:** tested via Docker (`fedora` base image, `libyaml-devel` via
376
+ the distro package manager).
377
+ - **Windows:** the extension does not build yet. The C importer of Zig
378
+ cannot translate the headers of the Windows SDK, so the `Python.h`
379
+ import fails (see
380
+ [translate-c#307](https://codeberg.org/ziglang/translate-c/issues/307)).
381
+ The other hosts (Elixir, JavaScript, Wasmer) are not affected.
382
+
383
+ ---
384
+
385
+ ## JavaScript package
386
+
387
+ The parser is also available for JavaScript and TypeScript as `zamljs` on
388
+ npm. The package ships a WebAssembly module built from the same libyaml
389
+ sources as the native builds, so the rules are the same everywhere.
390
+
391
+ ```console
392
+ npm install zamljs
393
+ ```
394
+
395
+ ```javascript
396
+ import { parse } from 'zamljs';
397
+
398
+ parse('foo: 1\nbar: [a, b, 3.14]\n');
399
+ // { foo: 1, bar: [ 'a', 'b', 3.14 ] }
400
+ ```
401
+
402
+ The package is ESM-only TypeScript with no runtime dependencies and it
403
+ ships its own types. It works in browsers, Node.js, Bun and Deno: the
404
+ `browser` and `node` export conditions pick the right loader, so a
405
+ bundler gets the browser build and Node reads the module from the disk.
406
+
407
+ Nothing is lost on the way to JavaScript: an integer that does not fit a
408
+ `number` comes back as a `bigint`, and a `!!binary` value stays a
409
+ `Uint8Array`. A mapping whose keys are not all strings becomes a `Map`.
410
+ A document that cannot be parsed throws a `ZamlError` that carries the
411
+ line and column of the problem.
412
+
413
+ `parseCbor` and `parseJson` return the raw encoded document instead of a
414
+ value, which is what the command below writes.
415
+
416
+ Building the package from a checkout:
417
+
418
+ ```bash
419
+ zig build wasm # builds the WebAssembly module
420
+ cd packages/zamljs
421
+ npm install
422
+ npm run build # compiles the TypeScript, copies the module
423
+ npm test
424
+ ```
425
+
426
+ ---
427
+
428
+ ## Command line
429
+
430
+ The parser is also a command that turns YAML into JSON or CBOR, built for
431
+ WASI and published on the [Wasmer registry](https://wasmer.io) as
432
+ [`niranjanaryan/zaml`](https://wasmer.io/packages/niranjanaryan/zaml).
433
+
434
+ ```console
435
+ wasmer install niranjanaryan/zaml
436
+ ```
437
+
438
+ ```console
439
+ $ echo 'foo: 1
440
+ bar: [a, b, 3.14]' | wasmer run niranjanaryan/zaml
441
+ {"foo":1,"bar":["a","b",3.14]}
442
+ ```
443
+
444
+ The output is JSON by default. CBOR keeps the exact value of every
445
+ integer, which JSON cannot:
446
+
447
+ ```console
448
+ $ echo 'big: 123456789012345678901234567890' | wasmer run niranjanaryan/zaml -- --format cbor
449
+ ```
450
+
451
+ A document that cannot be parsed is reported on standard error, and the
452
+ command exits with 1:
453
+
454
+ ```console
455
+ $ echo 'a: [1, 2' | wasmer run niranjanaryan/zaml
456
+ zaml: line 2, column 1: did not find expected ',' or ']'
457
+ ```
458
+
459
+ Building it from a checkout:
460
+
461
+ ```bash
462
+ zig build wasm-wasi # writes zig-out/dist/wasm/zaml.wasi.wasm
463
+ ```
464
+
465
+ ---
466
+
467
+ ## Building the WebAssembly targets
468
+
469
+ Both WebAssembly targets are built by the build script of the repository
470
+ and need nothing but the Zig toolchain — the libyaml sources they use
471
+ are vendored in `vendor/libyaml/`.
472
+
473
+ ```bash
474
+ zig build test # the tests of the shared core
475
+ zig build wasm # zig-out/dist/wasm/zaml.wasm
476
+ zig build wasm-wasi # zig-out/dist/wasm/zaml.wasi.wasm
477
+ ```
478
+
479
+ The browser module has no imports at all: it grows its own memory and
480
+ carries the handful of C functions libyaml needs. The WASI command uses
481
+ the C library of the runtime instead.