yeptris 0.1.10.0 → 0.1.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cec65c1f8da9f6256a42fdf00f6c03c637376c0a0e4d0305f4b540f31f43cc1e
4
- data.tar.gz: fb4d51fe7dad0b08c1364b32572b4abd92fa737fab79066665c9ce0820f3c6b9
3
+ metadata.gz: 9fa74d905a2d19dbeaf70381760c3a629738808e507d0ef9f33c9e266b756854
4
+ data.tar.gz: 2d3fb1c11b0d2dbb88c5bb75e837f41abe971992c1940a8c5be191b0168411b8
5
5
  SHA512:
6
- metadata.gz: 2c361ed6318a55ebdaeaacc56130e5972d699e065dcb7ed0453e84660f9c8f7054d7db1b55ab382bf438484b5a3cb42c201711517cd4feee7d08a90fd0b2fc17
7
- data.tar.gz: 9553bc1afea94e8188dd8d28d8b7186c697865cfc5757fb1e42ab1fc6196fe83ba9a5834af922e65acd5ddca861c0f6b6dddfeec9d14fc2eecc8f19aa46559fe
6
+ metadata.gz: e621ced9cf2707ada102a8633f7f75f597f63488f2a66da4efdb0dd3e292ef0719bbba509db321ca9e5cfd38f455a732a90cb3d1709b79f4ae7cab5ef0dd48da
7
+ data.tar.gz: 9d5efd07ce2e2f498320133776e3b0ed08e1fec4977a7e5cd1b607616630377bc29f7d5708e00c058c2f7d97022a6b668d6e3865c33da17a7d9c8ba71d445f2e
data/README.adoc CHANGED
@@ -7,8 +7,9 @@ compilation at install.
7
7
 
8
8
  == Install (development)
9
9
 
10
- The native library is vendored in the platform gems (TODO.impl/15
11
- phase D). For development against a local build:
10
+ The gem loads a shared `libyeptris` from `YEPTRIS_LIB_PATH`, a
11
+ vendored `lib/platform/<tag>/` copy, or the system paths. For
12
+ development against a local build:
12
13
 
13
14
  ....
14
15
  # the sibling C checkout: ~/src/leptris/yeptris
@@ -32,7 +33,9 @@ require "yeptris"
32
33
  Yeptris::YAML.load("name: yeptris\nrating: 10\n")
33
34
  # => {"name" => "yeptris", "rating" => 10}
34
35
 
35
- Yeptris::YAML.dump("name" => "yeptris", tags: [:yaml, :fast])
36
+ # brace mixed hashes on Ruby 3.x: a trailing symbol key otherwise
37
+ # splits the literal into keywords
38
+ Yeptris::YAML.dump({"name" => "yeptris", "tags" => [:yaml, :fast]})
36
39
 
37
40
  doc = Yeptris::Document.parse(config_yaml)
38
41
  doc.root["server"]["port"].to_i
@@ -49,11 +52,50 @@ Handles are document-scoped: `Document#free` releases everything
49
52
  (one C call), a GC finalizer backs it up, and any use after free
50
53
  raises `Yeptris::FreedError` — never a segfault.
51
54
 
52
- == Roadmap
55
+ == Native materializer (opt-in): faster than JSON.parse
53
56
 
54
- * Phase B: recorder-driven Ruby materialization (bulk event records,
55
- the FFI tax paid per chunk, not per node) + the `Yeptris::Psych`
56
- drop-in namespace and the ported Psych suite.
57
- * Phase C: `.tml` corpora, readonly mode, YAMLTree dumps.
58
- * Phase D: vendored precompiled platform gems, lockstep versioning
59
- with the C library, symbol audit task.
57
+ For JSON-shaped input, the binding can beat Ruby's own `JSON.parse`:
58
+
59
+ ....
60
+ cd ext/yeptris_native
61
+ YEPTRIS_LIB_PATH=/path/to/libyeptris.dylib ruby extconf.rb && make
62
+ cp native.bundle ../../lib/yeptris/ # or .so on Linux
63
+ ....
64
+
65
+ `require "yeptris"` picks it up automatically (a missing build falls
66
+ back to the FFI Marshal ladder silently). Measured on the 152 KB /
67
+ 29.4k-value JSON corpus (mean of 300, Ruby 3.4):
68
+
69
+ ....
70
+ JSON.parse mean 1.597 ms
71
+ Yeptris::YAML.load mean 1.145 ms (0.72x — faster than JSON.parse)
72
+ Psych.load mean 40.60 ms (35x slower than yeptris)
73
+ ....
74
+
75
+ The extension is a fused RFC 8259 → `VALUE` parser (one pass, no
76
+ intermediate records): libyeptris scan kernels tokenize, the Ruby C
77
+ API allocates, repeated keys/tokens share one frozen `String`, and GC
78
+ is paused for the duration. YAML inputs keep the FFI ladder so
79
+ timestamps, aliases, and Psych's scalar quirks resolve through the
80
+ same path `Psych.load` uses.
81
+
82
+ == Shipped beyond the original plan
83
+
84
+ * The columnar value drain (bulk, pre-converted typed columns — the
85
+ load fast path) and the bulk DOM builder (one `document_build`
86
+ call per dump).
87
+ * **Ruby Marshal 4.8 emission** (libyeptris >= 0.1.11): the C side
88
+ converts value records into Marshal bytes; one `Marshal.load`
89
+ materializes the whole graph. ~10× faster than the columnar walk
90
+ on JSON-shaped input, ~5× on YAML, ~50× on the per-node DOM walk
91
+ (`Node#to_ruby` becomes bulk). Alias identity preserved through
92
+ object links; merge keys and timestamps return UNSUPPORTED for the
93
+ record-walk fallback.
94
+ * `Yeptris::Psych` — the drop-in namespace with the ported Psych
95
+ suite (143 specs), `.tml` corpora conformance, and the Encodable
96
+ object protocol (`include Yeptris::Psych::Encodable` +
97
+ `encode_with`/`init_with` — no `respond_to?`, no ivar reflection;
98
+ the library never reaches into an object's internals).
99
+ * Lockstep versioning with the C library
100
+ (`{c-semver}.{gem-patch}`); releases published from the C repo's
101
+ workflow through the RubyGems trusted publisher.
@@ -0,0 +1,39 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "mkmf"
4
+
5
+ lib_path = ENV["YEPTRIS_LIB_PATH"]
6
+ candidates = []
7
+ if lib_path
8
+ candidates << File.dirname(lib_path)
9
+ candidates << lib_path if File.directory?(lib_path)
10
+ end
11
+ candidates << File.expand_path("../../../../yeptris/build-validate/src", __dir__)
12
+ candidates << File.expand_path("../../../../yeptris/build/src", __dir__)
13
+ candidates << File.expand_path("../../../yeptris/build-validate/src", __dir__)
14
+ candidates << File.expand_path("../../../yeptris/build/src", __dir__)
15
+
16
+ src_root = [
17
+ File.expand_path("../../../../yeptris/src", __dir__),
18
+ File.expand_path("../../../yeptris/src", __dir__),
19
+ ].find { |d| d && File.directory?(File.join(d, "include")) }
20
+ abort "yeptris sources not found" unless src_root
21
+
22
+ $INCFLAGS << " -I#{src_root}/include -I#{src_root}/yeptris"
23
+ %w[build-validate/generated build/generated].each do |g|
24
+ d = File.join(File.dirname(src_root), g)
25
+ $INCFLAGS << " -I#{d}" if File.directory?(d)
26
+ end
27
+
28
+ libdir = candidates.find do |d|
29
+ d && File.directory?(d) && Dir[File.join(d, "libyeptris*.{dylib,so,dll}")].any?
30
+ end
31
+ abort "libyeptris not found (set YEPTRIS_LIB_PATH)" unless libdir
32
+
33
+ $LIBPATH << libdir
34
+ $LDFLAGS << " -Wl,-rpath,#{libdir}" if RUBY_PLATFORM =~ /linux|darwin/
35
+ have_library("yeptris", "yep_json_string") or abort "yep_json_string not exported — rebuild libyeptris"
36
+ have_library("yeptris", "yeptris_visit_json") or abort "yeptris_visit_json missing"
37
+
38
+ $CFLAGS << " -O3 -fvisibility=hidden" unless RUBY_PLATFORM =~ /mswin|mingw/
39
+ create_makefile("yeptris/native")
@@ -0,0 +1,203 @@
1
+ /* json_ruby.c — fused RFC 8259 → Ruby VALUE (TODO.restructure/22). */
2
+ #include <ruby.h>
3
+ #include <ruby/encoding.h>
4
+ #include <stdlib.h>
5
+ #include <string.h>
6
+ #include "parse/numbers.h"
7
+ #include "parse/scalars.h"
8
+ #include "scan/json.h"
9
+
10
+ #define YEP_JR_MAX 1000
11
+ #define YEP_KC 256
12
+
13
+ typedef struct {
14
+ uint64_t h;
15
+ uint32_t len;
16
+ VALUE v;
17
+ } kcent;
18
+
19
+ typedef struct {
20
+ const char* p;
21
+ size_t len;
22
+ size_t i;
23
+ char* scratch;
24
+ size_t scratch_cap;
25
+ int depth;
26
+ int err;
27
+ rb_encoding* enc;
28
+ kcent kc[YEP_KC];
29
+ } jr;
30
+
31
+ static VALUE jr_value(jr* j);
32
+
33
+ static void jr_ws(jr* j) {
34
+ while (j->i < j->len) {
35
+ unsigned char c = (unsigned char)j->p[j->i];
36
+ if (c == ' ' || c == '\t' || c == '\n' || c == '\r') j->i++;
37
+ else break;
38
+ }
39
+ }
40
+
41
+ static uint64_t jr_hash(const char* sp, long sl) {
42
+ /* FNV-1a 64 — cheap, good enough for short keys */
43
+ uint64_t h = 14695981039346656037ull;
44
+ for (long i = 0; i < sl; i++) {
45
+ h ^= (unsigned char)sp[i];
46
+ h *= 1099511628211ull;
47
+ }
48
+ h ^= (uint64_t)sl;
49
+ return h;
50
+ }
51
+
52
+ static VALUE jr_cached(jr* j, const char* sp, long sl) {
53
+ uint64_t h = jr_hash(sp, sl);
54
+ uint32_t slot = (uint32_t)(h & (YEP_KC - 1));
55
+ kcent* e = &j->kc[slot];
56
+ if (e->v != 0 && e->h == h && e->len == (uint32_t)sl &&
57
+ (long)RSTRING_LEN(e->v) == sl &&
58
+ memcmp(RSTRING_PTR(e->v), sp, (size_t)sl) == 0) {
59
+ return e->v;
60
+ }
61
+ VALUE s = rb_enc_str_new(sp, sl, j->enc);
62
+ rb_str_freeze(s);
63
+ e->h = h;
64
+ e->len = (uint32_t)sl;
65
+ e->v = s;
66
+ return s;
67
+ }
68
+
69
+ static VALUE jr_str(jr* j, int as_key) {
70
+ size_t start = j->i, close = 0;
71
+ int has_esc = 0;
72
+ if (!yep_json_string(j->p, j->len, &j->i, &close, &has_esc)) { j->err = -2; return Qnil; }
73
+ const char* sp; long sl;
74
+ if (has_esc) {
75
+ uint32_t span = (uint32_t)(close - start - 1);
76
+ if (span + 1 > j->scratch_cap) {
77
+ size_t cap = j->scratch_cap ? j->scratch_cap : 64;
78
+ while (cap < span + 1) cap *= 2;
79
+ char* ns = realloc(j->scratch, cap);
80
+ if (!ns) { j->err = -1; return Qnil; }
81
+ j->scratch = ns; j->scratch_cap = cap;
82
+ }
83
+ sl = (long)yep_finish_double_into(j->p, (uint32_t)(start + 1), (uint32_t)close, j->scratch, span);
84
+ sp = j->scratch;
85
+ } else {
86
+ sp = j->p + start + 1;
87
+ sl = (long)(close - start - 1);
88
+ }
89
+ if (as_key || sl <= 2) return jr_cached(j, sp, sl);
90
+ return rb_enc_str_new(sp, sl, j->enc);
91
+ }
92
+
93
+ static VALUE jr_num(jr* j) {
94
+ size_t start = j->i;
95
+ if (!yep_json_number(j->p, j->len, &j->i)) { j->err = -2; return Qnil; }
96
+ const char* s = j->p + start;
97
+ uint32_t n = (uint32_t)(j->i - start);
98
+ int is_float = 0, neg = 0;
99
+ uint32_t k = 0;
100
+ if (s[0] == '-') { neg = 1; k = 1; }
101
+ for (; k < n; k++) {
102
+ char c = s[k];
103
+ if (c == '.' || c == 'e' || c == 'E') { is_float = 1; break; }
104
+ }
105
+ if (!is_float && n - (uint32_t)neg <= 18) {
106
+ int64_t v = 0;
107
+ for (k = (uint32_t)neg; k < n; k++) v = v * 10 + (s[k] - '0');
108
+ if (neg) v = -v;
109
+ return LL2NUM(v);
110
+ }
111
+ if (is_float) {
112
+ double d = 0.0;
113
+ if (yep_num_f64(s, n, &d) != 0) { j->err = -2; return Qnil; }
114
+ return DBL2NUM(d);
115
+ }
116
+ int64_t v = 0;
117
+ if (yep_num_i64(s, n, &v) != 0) {
118
+ double d = 0.0;
119
+ if (yep_num_f64(s, n, &d) != 0) { j->err = -2; return Qnil; }
120
+ return DBL2NUM(d);
121
+ }
122
+ return LL2NUM(v);
123
+ }
124
+
125
+ static VALUE jr_object(jr* j) {
126
+ j->i++; j->depth++;
127
+ VALUE h = rb_hash_new_capa(8);
128
+ jr_ws(j);
129
+ if (j->i < j->len && j->p[j->i] == '}') { j->i++; j->depth--; return h; }
130
+ for (;;) {
131
+ jr_ws(j);
132
+ if (j->i >= j->len || j->p[j->i] != '"') { j->err = -2; return Qnil; }
133
+ VALUE key = jr_str(j, 1);
134
+ if (j->err) return Qnil;
135
+ jr_ws(j);
136
+ if (j->i >= j->len || j->p[j->i] != ':') { j->err = -2; return Qnil; }
137
+ j->i++;
138
+ VALUE val = jr_value(j);
139
+ if (j->err) return Qnil;
140
+ rb_hash_aset(h, key, val);
141
+ jr_ws(j);
142
+ if (j->i >= j->len) { j->err = -2; return Qnil; }
143
+ if (j->p[j->i] == ',') { j->i++; continue; }
144
+ if (j->p[j->i] == '}') { j->i++; j->depth--; return h; }
145
+ j->err = -2; return Qnil;
146
+ }
147
+ }
148
+
149
+ static VALUE jr_array(jr* j) {
150
+ j->i++; j->depth++;
151
+ VALUE a = rb_ary_new_capa(8);
152
+ jr_ws(j);
153
+ if (j->i < j->len && j->p[j->i] == ']') { j->i++; j->depth--; return a; }
154
+ for (;;) {
155
+ VALUE v = jr_value(j);
156
+ if (j->err) return Qnil;
157
+ rb_ary_push(a, v);
158
+ jr_ws(j);
159
+ if (j->i >= j->len) { j->err = -2; return Qnil; }
160
+ if (j->p[j->i] == ',') { j->i++; continue; }
161
+ if (j->p[j->i] == ']') { j->i++; j->depth--; return a; }
162
+ j->err = -2; return Qnil;
163
+ }
164
+ }
165
+
166
+ static VALUE jr_value(jr* j) {
167
+ if (j->depth >= YEP_JR_MAX) { j->err = -2; return Qnil; }
168
+ jr_ws(j);
169
+ if (j->i >= j->len) { j->err = -2; return Qnil; }
170
+ char c = j->p[j->i];
171
+ if (c == '{') return jr_object(j);
172
+ if (c == '[') return jr_array(j);
173
+ if (c == '"') return jr_str(j, 0);
174
+ if (c == 't') {
175
+ if (!yep_json_literal(j->p, j->len, &j->i, "true")) { j->err = -2; return Qnil; }
176
+ return Qtrue;
177
+ }
178
+ if (c == 'f') {
179
+ if (!yep_json_literal(j->p, j->len, &j->i, "false")) { j->err = -2; return Qnil; }
180
+ return Qfalse;
181
+ }
182
+ if (c == 'n') {
183
+ if (!yep_json_literal(j->p, j->len, &j->i, "null")) { j->err = -2; return Qnil; }
184
+ return Qnil;
185
+ }
186
+ if (c == '-' || (c >= '0' && c <= '9')) return jr_num(j);
187
+ j->err = -2; return Qnil;
188
+ }
189
+
190
+ VALUE yep_rb_parse_json(const char* p, size_t len) {
191
+ jr j;
192
+ memset(&j, 0, sizeof(j));
193
+ j.p = p; j.len = len; j.enc = rb_utf8_encoding();
194
+ VALUE already = rb_gc_disable();
195
+ VALUE v = jr_value(&j);
196
+ jr_ws(&j);
197
+ if (j.i != len && j.err == 0) j.err = -2;
198
+ free(j.scratch);
199
+ if (already == Qfalse) rb_gc_enable();
200
+ if (j.err == -1) rb_raise(rb_eNoMemError, "yeptris native json");
201
+ if (j.err != 0) rb_raise(rb_path2class("Yeptris::ParseError"), "native json parse failed");
202
+ return v;
203
+ }
@@ -0,0 +1,333 @@
1
+ /* yeptris_native.c — Ruby C-API materializer over libyeptris visit
2
+ * (TODO.restructure/22). One Ruby→C call builds the whole object
3
+ * graph via rb_hash_new / rb_ary_push / rb_str_new_len — the same
4
+ * shape as JSON.parse, so the binding can beat it on the fused JSON
5
+ * path. The extension is optional: LoadError falls back to the FFI
6
+ * Marshal ladder. */
7
+
8
+ #include <ruby.h>
9
+ #include <ruby/encoding.h>
10
+
11
+ #include <stdlib.h>
12
+ #include <string.h>
13
+
14
+ #include <yeptris/api.h>
15
+ #include <yeptris/error.h>
16
+ #include <yeptris/resolve.h>
17
+ #include <yeptris/visit.h>
18
+
19
+ #define YEP_RB_MAX_DEPTH 1024
20
+
21
+ static rb_encoding* utf8_enc;
22
+
23
+ typedef struct {
24
+ VALUE stack[YEP_RB_MAX_DEPTH];
25
+ VALUE keys[YEP_RB_MAX_DEPTH]; /* pending map key at this depth */
26
+ int is_map[YEP_RB_MAX_DEPTH];
27
+ int sp;
28
+ VALUE root;
29
+ VALUE anchors; /* Hash name=>object for YAML identity */
30
+ VALUE pending_anchor; /* String name awaiting the next value */
31
+ int failed;
32
+ } rb_ctx;
33
+
34
+ static void rb_fail(rb_ctx* c) {
35
+ c->failed = 1;
36
+ }
37
+
38
+ static int rb_push_value(rb_ctx* c, VALUE v) {
39
+ if (c->failed) {
40
+ return -1;
41
+ }
42
+ /* bind pending anchor */
43
+ if (!NIL_P(c->pending_anchor) && !NIL_P(c->anchors)) {
44
+ rb_hash_aset(c->anchors, c->pending_anchor, v);
45
+ c->pending_anchor = Qnil;
46
+ }
47
+ if (c->sp == 0) {
48
+ c->root = v;
49
+ return 0;
50
+ }
51
+ VALUE parent = c->stack[c->sp - 1];
52
+ if (c->is_map[c->sp - 1]) {
53
+ VALUE key = c->keys[c->sp - 1];
54
+ if (NIL_P(key)) {
55
+ rb_fail(c);
56
+ return -1;
57
+ }
58
+ rb_hash_aset(parent, key, v);
59
+ c->keys[c->sp - 1] = Qnil;
60
+ } else {
61
+ rb_ary_push(parent, v);
62
+ }
63
+ return 0;
64
+ }
65
+
66
+ static int on_null(void* ctx) {
67
+ return rb_push_value((rb_ctx*)ctx, Qnil);
68
+ }
69
+
70
+ static int on_bool(void* ctx, int truthy) {
71
+ return rb_push_value((rb_ctx*)ctx, truthy ? Qtrue : Qfalse);
72
+ }
73
+
74
+ static int on_int(void* ctx, int64_t v) {
75
+ return rb_push_value((rb_ctx*)ctx, LL2NUM(v));
76
+ }
77
+
78
+ static int on_float(void* ctx, double v) {
79
+ return rb_push_value((rb_ctx*)ctx, DBL2NUM(v));
80
+ }
81
+
82
+ static rb_encoding* utf8_enc;
83
+
84
+ static VALUE rb_utf8_str(const char* p, size_t len) {
85
+ return rb_enc_str_new(p, (long)len, utf8_enc);
86
+ }
87
+
88
+ /* One-shot interned string (no intermediate alloc) — keys and the
89
+ * short repeated values ("a"/"b"/…) share one VALUE. */
90
+ static VALUE rb_utf8_interned(const char* p, size_t len) {
91
+ return rb_enc_interned_str(p, (long)len, utf8_enc);
92
+ }
93
+
94
+ static int on_string(void* ctx, const char* p, size_t len) {
95
+ rb_ctx* c = (rb_ctx*)ctx;
96
+ /* short strings are almost always repeated tokens in JSON corpora;
97
+ * intern them. Longer payloads stay unique. */
98
+ VALUE s = (len <= 16) ? rb_utf8_interned(p, len) : rb_utf8_str(p, len);
99
+ return rb_push_value(c, s);
100
+ }
101
+
102
+ static int on_key(void* ctx, const char* p, size_t len) {
103
+ rb_ctx* c = (rb_ctx*)ctx;
104
+ if (c->sp == 0 || !c->is_map[c->sp - 1]) {
105
+ rb_fail(c);
106
+ return -1;
107
+ }
108
+ c->keys[c->sp - 1] = rb_utf8_interned(p, len);
109
+ return 0;
110
+ }
111
+
112
+ static int on_seq_start(void* ctx) {
113
+ rb_ctx* c = (rb_ctx*)ctx;
114
+ if (c->failed || c->sp >= YEP_RB_MAX_DEPTH) {
115
+ rb_fail(c);
116
+ return -1;
117
+ }
118
+ VALUE a = rb_ary_new();
119
+ /* bind anchor to the container before placing it */
120
+ if (!NIL_P(c->pending_anchor) && !NIL_P(c->anchors)) {
121
+ rb_hash_aset(c->anchors, c->pending_anchor, a);
122
+ c->pending_anchor = Qnil;
123
+ }
124
+ if (c->sp == 0) {
125
+ c->root = a;
126
+ } else {
127
+ VALUE parent = c->stack[c->sp - 1];
128
+ if (c->is_map[c->sp - 1]) {
129
+ VALUE key = c->keys[c->sp - 1];
130
+ if (NIL_P(key)) {
131
+ rb_fail(c);
132
+ return -1;
133
+ }
134
+ rb_hash_aset(parent, key, a);
135
+ c->keys[c->sp - 1] = Qnil;
136
+ } else {
137
+ rb_ary_push(parent, a);
138
+ }
139
+ }
140
+ c->stack[c->sp] = a;
141
+ c->is_map[c->sp] = 0;
142
+ c->keys[c->sp] = Qnil;
143
+ c->sp++;
144
+ return 0;
145
+ }
146
+
147
+ static int on_seq_end(void* ctx) {
148
+ rb_ctx* c = (rb_ctx*)ctx;
149
+ if (c->sp <= 0) {
150
+ rb_fail(c);
151
+ return -1;
152
+ }
153
+ c->sp--;
154
+ return 0;
155
+ }
156
+
157
+ static int on_map_start(void* ctx) {
158
+ rb_ctx* c = (rb_ctx*)ctx;
159
+ if (c->failed || c->sp >= YEP_RB_MAX_DEPTH) {
160
+ rb_fail(c);
161
+ return -1;
162
+ }
163
+ VALUE h = rb_hash_new();
164
+ if (!NIL_P(c->pending_anchor) && !NIL_P(c->anchors)) {
165
+ rb_hash_aset(c->anchors, c->pending_anchor, h);
166
+ c->pending_anchor = Qnil;
167
+ }
168
+ if (c->sp == 0) {
169
+ c->root = h;
170
+ } else {
171
+ VALUE parent = c->stack[c->sp - 1];
172
+ if (c->is_map[c->sp - 1]) {
173
+ VALUE key = c->keys[c->sp - 1];
174
+ if (NIL_P(key)) {
175
+ rb_fail(c);
176
+ return -1;
177
+ }
178
+ rb_hash_aset(parent, key, h);
179
+ c->keys[c->sp - 1] = Qnil;
180
+ } else {
181
+ rb_ary_push(parent, h);
182
+ }
183
+ }
184
+ c->stack[c->sp] = h;
185
+ c->is_map[c->sp] = 1;
186
+ c->keys[c->sp] = Qnil;
187
+ c->sp++;
188
+ return 0;
189
+ }
190
+
191
+ static int on_map_end(void* ctx) {
192
+ rb_ctx* c = (rb_ctx*)ctx;
193
+ if (c->sp <= 0) {
194
+ rb_fail(c);
195
+ return -1;
196
+ }
197
+ c->sp--;
198
+ return 0;
199
+ }
200
+
201
+ static int on_anchor(void* ctx, const char* name, size_t len) {
202
+ rb_ctx* c = (rb_ctx*)ctx;
203
+ c->pending_anchor = rb_utf8_str(name, len);
204
+ return 0;
205
+ }
206
+
207
+ static int on_alias(void* ctx, const char* name, size_t len) {
208
+ rb_ctx* c = (rb_ctx*)ctx;
209
+ VALUE key = rb_utf8_str(name, len);
210
+ VALUE v = rb_hash_lookup2(c->anchors, key, Qundef);
211
+ if (v == Qundef) {
212
+ v = Qnil;
213
+ }
214
+ return rb_push_value(c, v);
215
+ }
216
+
217
+ static int on_doc(void* ctx) {
218
+ /* multi-doc: for load_all we'd collect; single-load takes first.
219
+ * reset root so subsequent docs replace — load_stream uses a
220
+ * different entry that accumulates. */
221
+ (void)ctx;
222
+ return 0;
223
+ }
224
+
225
+ static const YeptrisVisitVTable k_vt = {
226
+ on_null, on_bool, on_int, on_float, on_string,
227
+ on_seq_start, on_seq_end, on_map_start, on_map_end,
228
+ on_key, on_doc, on_anchor, on_alias,
229
+ };
230
+
231
+ /* fused JSON→Ruby (json_ruby.c) — no vtable, beats JSON.parse */
232
+ VALUE yep_rb_parse_json(const char* p, size_t len);
233
+
234
+ static VALUE ctx_result(rb_ctx* c, YeptrisStatus st) {
235
+ if (st != YEPTRIS_OK || c->failed) {
236
+ if (st == YEPTRIS_ERROR_PARSE) {
237
+ rb_raise(rb_path2class("Yeptris::ParseError"), "native parse failed");
238
+ }
239
+ if (st == YEPTRIS_ERROR_MEMORY) {
240
+ rb_raise(rb_eNoMemError, "yeptris native");
241
+ }
242
+ rb_raise(rb_path2class("Yeptris::Error"), "native materialize failed (%d)", (int)st);
243
+ }
244
+ return c->root;
245
+ }
246
+
247
+ static VALUE native_load_json(VALUE self, VALUE input) {
248
+ (void)self;
249
+ StringValue(input);
250
+ return yep_rb_parse_json(RSTRING_PTR(input), (size_t)RSTRING_LEN(input));
251
+ }
252
+
253
+ static VALUE native_load(VALUE self, VALUE input, VALUE schema) {
254
+ (void)self;
255
+ StringValue(input);
256
+ int sch = YEPTRIS_SCHEMA_11_COMPAT;
257
+ if (!NIL_P(schema)) {
258
+ Check_Type(schema, T_SYMBOL);
259
+ if (rb_sym2id(schema) == rb_intern("core_12")) {
260
+ sch = YEPTRIS_SCHEMA_12_CORE;
261
+ }
262
+ }
263
+ rb_ctx c;
264
+ memset(&c, 0, sizeof(c));
265
+ c.root = Qnil;
266
+ c.pending_anchor = Qnil;
267
+ c.anchors = rb_hash_new();
268
+ VALUE already = rb_gc_disable();
269
+ YeptrisStatus st = yeptris_visit(RSTRING_PTR(input), (size_t)RSTRING_LEN(input),
270
+ (YeptrisSchema)sch, &k_vt, &c);
271
+ if (already == Qfalse) {
272
+ rb_gc_enable();
273
+ }
274
+ return ctx_result(&c, st);
275
+ }
276
+
277
+ /* load_stream: accumulate documents into an Array. */
278
+ typedef struct {
279
+ rb_ctx inner;
280
+ VALUE docs;
281
+ int in_doc;
282
+ } rb_stream_ctx;
283
+
284
+ static int stream_on_doc(void* ctx) {
285
+ rb_stream_ctx* s = (rb_stream_ctx*)ctx;
286
+ if (s->in_doc && !NIL_P(s->inner.root)) {
287
+ rb_ary_push(s->docs, s->inner.root);
288
+ }
289
+ s->inner.root = Qnil;
290
+ s->inner.sp = 0;
291
+ s->in_doc = 1;
292
+ return 0;
293
+ }
294
+
295
+ static VALUE native_load_stream(VALUE self, VALUE input, VALUE schema) {
296
+ (void)self;
297
+ StringValue(input);
298
+ int sch = YEPTRIS_SCHEMA_11_COMPAT;
299
+ if (!NIL_P(schema) && rb_sym2id(schema) == rb_intern("core_12")) {
300
+ sch = YEPTRIS_SCHEMA_12_CORE;
301
+ }
302
+ rb_stream_ctx s;
303
+ memset(&s, 0, sizeof(s));
304
+ s.inner.root = Qnil;
305
+ s.inner.pending_anchor = Qnil;
306
+ s.inner.anchors = rb_hash_new();
307
+ s.docs = rb_ary_new();
308
+ YeptrisVisitVTable vt = k_vt;
309
+ vt.on_doc = stream_on_doc;
310
+ /* trick: the ctx for scalar callbacks is &s.inner, but on_doc needs
311
+ * &s. Use a unified ctx — rebind all callbacks to take stream ctx
312
+ * by making inner the first field (already is). on_doc uses outer;
313
+ * others use inner via same pointer since inner is first field. */
314
+ YeptrisStatus st = yeptris_visit(RSTRING_PTR(input), (size_t)RSTRING_LEN(input),
315
+ (YeptrisSchema)sch, &vt, &s);
316
+ if (st == YEPTRIS_OK && !s.inner.failed) {
317
+ if (!NIL_P(s.inner.root) || s.in_doc) {
318
+ rb_ary_push(s.docs, s.inner.root);
319
+ }
320
+ return s.docs;
321
+ }
322
+ return ctx_result(&s.inner, st == YEPTRIS_OK ? YEPTRIS_ERROR_INTERNAL : st);
323
+ }
324
+
325
+ RUBY_FUNC_EXPORTED void Init_native(void) {
326
+ utf8_enc = rb_utf8_encoding();
327
+ VALUE mYep = rb_define_module("Yeptris");
328
+ VALUE mNat = rb_define_module_under(mYep, "Native");
329
+ rb_define_singleton_method(mNat, "load_json", native_load_json, 1);
330
+ rb_define_singleton_method(mNat, "load", native_load, 2);
331
+ rb_define_singleton_method(mNat, "load_stream", native_load_stream, 2);
332
+ rb_define_const(mNat, "AVAILABLE", Qtrue);
333
+ }
data/lib/yeptris/ffi.rb CHANGED
@@ -136,6 +136,25 @@ module Yeptris
136
136
  :bools, :pointer, :arena, :pointer
137
137
  end
138
138
 
139
+ # Marshal 4.8 emission (TODO.restructure/21): the C side converts
140
+ # value records directly into Ruby's wire format; the binding
141
+ # materializes the whole graph with one core-C Marshal.load call
142
+ # instead of walking per-value records in pure Ruby. Same feature-
143
+ # detect discipline as the columnar drain (older libraries fall
144
+ # back to the record walk).
145
+ MARSHAL = begin
146
+ MARSHAL_ALL_DOCS = 0
147
+ MARSHAL_FIRST_DOC = 1
148
+ attach_function :yeptris_marshal,
149
+ %i[pointer size_t int int pointer pointer], :int
150
+ attach_function :yeptris_marshal_node,
151
+ %i[yeptris_node pointer pointer], :int
152
+ attach_function :yeptris_marshal_free, [:pointer], :void
153
+ true
154
+ rescue ::FFI::NotFoundError
155
+ false
156
+ end
157
+
139
158
  # bulk build (TODO.impl/15 phase D): one call raises a document
140
159
  BUILD_SCALAR = 1
141
160
  BUILD_SEQ = 2
@@ -229,6 +248,9 @@ module Yeptris
229
248
  ERROR_MEMORY = 2
230
249
  ERROR_DEPTH = 3
231
250
  ERROR_ENCODING = 4
251
+ ERROR_IO = 5
232
252
  ERROR_ARG = 6
253
+ ERROR_UNSUPPORTED = 7
254
+ ERROR_INTERNAL = 8
233
255
  end
234
256
  end
data/lib/yeptris/node.rb CHANGED
@@ -243,14 +243,48 @@ class Yeptris::Node
243
243
  end
244
244
 
245
245
  def to_ruby(memo = nil)
246
- # readonly documents memoize per node: repeated materialization
247
- # of a shared subtree returns the SAME object at zero walk cost
246
+ # readonly documents memoize per node: a second materialization
247
+ # returns the SAME object (the readonly cache is the ground truth
248
+ # for alias identity across calls).
248
249
  if @document.readonly?
249
250
  rm = @document.readonly_memo
250
251
  cached = rm[node_id]
251
252
  return cached if cached
252
-
253
- return rm[node_id] = to_ruby_walk({})
253
+ end
254
+ # Marshal fast path (TODO.restructure/21): one C call turns the
255
+ # whole subtree into Ruby objects, preserving alias identity via
256
+ # the object's own object-link table. Only at the TOP-LEVEL entry
257
+ # (memo nil): inside a recursive walk a partial marshal would
258
+ # materialize outside the shared memo and break alias identity.
259
+ # Falls back to the per-node FFI walk on constructs the format
260
+ # cannot express (merge keys, timestamps) and on older builds.
261
+ if Yeptris::FFI::MARSHAL && memo.nil?
262
+ result =
263
+ begin
264
+ out_p = ::FFI::MemoryPointer.new(:pointer)
265
+ olen_p = ::FFI::MemoryPointer.new(:size_t)
266
+ st = Yeptris::FFI.yeptris_marshal_node(@c_ptr, out_p, olen_p)
267
+ if st == Yeptris::FFI::ERROR_UNSUPPORTED
268
+ nil
269
+ elsif st != Yeptris::FFI::OK
270
+ raise Yeptris::ParseError, Yeptris::FFI.last_error_message
271
+ else
272
+ buf = out_p.read_pointer
273
+ len = olen_p.read_uint64
274
+ bytes = buf.read_bytes(len)
275
+ bytes.force_encoding(Encoding::ASCII_8BIT)
276
+ ::Marshal.load(bytes)
277
+ end
278
+ ensure
279
+ Yeptris::FFI.yeptris_marshal_free(out_p.read_pointer) if out_p
280
+ end
281
+ unless result.nil?
282
+ @document.readonly_memo[node_id] = result if @document.readonly?
283
+ return result
284
+ end
285
+ end
286
+ if @document.readonly?
287
+ return @document.readonly_memo[node_id] = to_ruby_walk({})
254
288
  end
255
289
  to_ruby_walk(memo || {})
256
290
  end
@@ -0,0 +1,14 @@
1
+ # frozen_string_literal: true
2
+
3
+ # TODO.restructure/23 — the typed opt-in marker for arbitrary-object
4
+ # dump/load. Classes include this module and implement
5
+ # encode_with(coder) / init_with(coder) (Psych's coder protocol).
6
+ # The visitors call those methods directly; the library never uses
7
+ # respond_to?, never reads or writes instance variables from outside
8
+ # the object's public surface (the encapsulation law).
9
+ module Yeptris
10
+ module Psych
11
+ module Encodable
12
+ end
13
+ end
14
+ end
@@ -9,7 +9,7 @@ module Yeptris
9
9
  module Visitors
10
10
  # The dump-side visitor (Psych's YAMLTree core): an arbitrary
11
11
  # Ruby object graph becomes DOM nodes — anchors for shared
12
- # objects, aliases for repeats, encode_with support, and the
12
+ # objects, aliases for repeats, the Encodable protocol, and the
13
13
  # !ruby/... tags Psych's own emitter produces.
14
14
  class YAMLTree
15
15
  def initialize
@@ -33,19 +33,24 @@ module Yeptris
33
33
 
34
34
  def visit(obj)
35
35
  case obj
36
- when nil, true, false, Integer, Float, String then scalar(obj)
37
- when Symbol then scalar(":#{obj}")
38
- when Date, Time then scalar(obj) # timestamp text, never ivars
39
- when Hash then visit_hash(obj)
40
- when Array then visit_array(obj)
41
- when Struct then visit_struct(obj)
42
- when Set then visit_set(obj)
36
+ when nil, true, false, ::Integer, ::Float, ::String then scalar(obj)
37
+ when ::Symbol then scalar(":#{obj}")
38
+ when ::Date, ::Time then scalar(obj) # timestamp text, never ivars
39
+ when ::Hash then visit_hash(obj)
40
+ when ::Array then visit_array(obj)
41
+ when ::Struct then visit_struct(obj)
42
+ when ::Set then visit_set(obj)
43
+ when Encodable then visit_encode_with(obj)
43
44
  else
44
- if obj.respond_to?(:encode_with)
45
- visit_encode_with(obj)
46
- else
47
- visit_object(obj)
45
+ if obj.instance_of?(::Object)
46
+ # A BARE Object has no declared state to lose — the
47
+ # empty !ruby/object form is correct by construction
48
+ # (an Object with ivars set from outside its class is
49
+ # outside the encapsulation contract; use a real class
50
+ # with Encodable for stateful objects).
51
+ return visit_bare_object(obj)
48
52
  end
53
+ refuse_dump(obj)
49
54
  end
50
55
  end
51
56
 
@@ -79,33 +84,43 @@ module Yeptris
79
84
  obj.each_pair { |_k, v| @refs[v.object_id] += 1; count_refs(v, seen) }
80
85
  when Set
81
86
  obj.each { |v| @refs[v.object_id] += 1; count_refs(v, seen) }
82
- when String, Integer, Float, Symbol, Date, Time, true, false, nil, Numeric
87
+ when ::String, ::Integer, ::Float, ::Symbol, ::Date, ::Time, true, false, nil, ::Numeric
83
88
  # immutables: identity anchors are meaningless
84
89
  else
85
- if obj.respond_to?(:encode_with)
86
- # coder contents are opaque here; the object itself may
87
- # repeat count it from the caller side only
88
- else
89
- obj.instance_variables.each do |iv|
90
- v = obj.instance_variable_get(iv)
91
- @refs[v.object_id] += 1
92
- count_refs(v, seen)
90
+ unless obj.instance_of?(::Object)
91
+ refuse_dump(obj) unless obj.is_a?(Encodable)
92
+ # Encodable: descend through the PUBLIC protocol so
93
+ # shared objects inside coder contents still get
94
+ # anchors — encode_with must be pure (it runs once more
95
+ # at dump).
96
+ coder = ::Yeptris::Psych::CoderShim.new(nil)
97
+ obj.encode_with(coder)
98
+ case coder.type
99
+ when :seq
100
+ coder.seq.each { |v| @refs[v.object_id] += 1; count_refs(v, seen) }
101
+ when :map
102
+ coder.each { |_k, v| @refs[v.object_id] += 1; count_refs(v, seen) }
93
103
  end
94
104
  end
95
105
  end
96
106
  end
97
107
 
108
+ def refuse_dump(obj)
109
+ raise ::Yeptris::DumpError,
110
+ "cannot dump #{obj.class}: include Yeptris::Psych::Encodable and implement encode_with(coder)"
111
+ end
112
+
98
113
  def scalar(obj, tag: nil)
99
114
  # strings route through the builder's plain-safety helper
100
115
  # (one quoting rule, DRY); other scalars are their own text
101
116
  text =
102
- if obj.is_a?(Date) || obj.is_a?(Time)
117
+ if obj.is_a?(::Date) || obj.is_a?(::Time)
103
118
  obj.iso8601 # canonical timestamp form, not to_s
104
119
  else
105
120
  obj.nil? ? "null" : obj.to_s
106
121
  end
107
122
  n =
108
- if obj.is_a?(String)
123
+ if obj.is_a?(::String)
109
124
  ::Yeptris::YAML::Builder.build_string(@tree, obj)
110
125
  else
111
126
  @tree.new_scalar(text, :plain)
@@ -165,37 +180,40 @@ module Yeptris
165
180
  return alias_of(obj, name) if state == :alias
166
181
 
167
182
  coder = ::Yeptris::Psych::CoderShim.new(obj.class.name)
168
- obj.encode_with(coder)
169
- node =
170
- case coder.type
171
- when :scalar then scalar(coder.scalar, tag: coder.tag)
172
- when :seq
173
- s = @tree.new_sequence
174
- s.set_tag(coder.tag) if coder.tag
175
- coder.seq.each { |e| s.seq_add(visit(e)) }
176
- s
177
- else
178
- m = @tree.new_mapping
179
- m.set_tag(coder.tag) if coder.tag
180
- coder.each { |k, v| m.map_add(key_text(k), visit(v)) }
181
- m
182
- end
183
- remember(obj, node)
184
- node.set_anchor(name) if name
185
- node
183
+ obj.encode_with(coder) # the typed public surface
184
+ # Wrap coder contents in a __init__ sub-mapping so the load
185
+ # side can dispatch back into obj.init_with(coder) without
186
+ # library-side ivar reflection (the encapsulation law).
187
+ init = @tree.new_mapping
188
+ case coder.type
189
+ when :scalar
190
+ init.map_add("__scalar__", scalar(coder.scalar))
191
+ init.map_add("__tag__", scalar(coder.tag)) if coder.tag
192
+ when :seq
193
+ inner = @tree.new_sequence
194
+ inner.set_tag(coder.tag) if coder.tag
195
+ coder.seq.each { |e| inner.seq_add(visit(e)) }
196
+ init.seq_add(inner)
197
+ else
198
+ coder.each { |k, v| init.map_add(key_text(k), visit(v)) }
199
+ init.set_tag(coder.tag) if coder.tag
200
+ end
201
+ m = @tree.new_mapping
202
+ m.set_anchor(name) if name
203
+ m.set_tag("!ruby/object:#{obj.class.name}")
204
+ m.map_add("__init__", init)
205
+ remember(obj, m)
206
+ m
186
207
  end
187
208
 
188
- def visit_object(obj)
209
+ def visit_bare_object(obj)
189
210
  state, name = anchor_for(obj)
190
211
  return alias_of(obj, name) if state == :alias
191
212
 
192
213
  m = @tree.new_mapping
193
214
  remember(obj, m)
194
215
  m.set_anchor(name) if name
195
- m.set_tag("!ruby/object:#{obj.class.name}")
196
- obj.instance_variables.each do |ivar|
197
- m.map_add(ivar.to_s, visit(obj.instance_variable_get(ivar)))
198
- end
216
+ m.set_tag("!ruby/object")
199
217
  m
200
218
  end
201
219
 
@@ -322,30 +340,31 @@ module Yeptris
322
340
  end
323
341
 
324
342
  def revive_object(klass, node)
325
- obj = klass ? klass.allocate : ::Object.new
343
+ # bare !ruby/object (no class): Psych.dump(Object.new)'s form.
344
+ # An Object has no declared state — allocation is the whole
345
+ # revival (no reflection, no protocol needed).
346
+ return ::Object.allocate if klass.nil?
347
+
348
+ raise ::Yeptris::DumpError, "#{klass} is not Yeptris::Psych::Encodable: implement init_with(coder)" unless klass <= ::Yeptris::Psych::Encodable
349
+
350
+ obj = klass.allocate
326
351
  anchors[node.anchor] = obj if node.anchor
352
+ # The class opts in via init_with(coder); the encoder wrote
353
+ # `coder["__init__"] = the mapping` so we can pass it back
354
+ # without library-side ivar reflection.
327
355
  node.children.each_slice(2) do |k, v|
328
356
  key = k.to_ruby.to_s
329
- if key == "__init__"
330
- init_with(obj, v)
331
- else
332
- ivar = key.start_with?("@") ? key.to_sym : "@#{key}".to_sym
333
- obj.instance_variable_set(ivar, visit(v))
334
- end
335
- end
336
- obj
337
- end
357
+ next unless key == "__init__"
338
358
 
339
- def init_with(obj, value_node)
340
- return unless obj.respond_to?(:init_with)
341
-
342
- coder = ::Yeptris::Psych::CoderShim.new
343
- if value_node.is_a?(::Yeptris::Psych::Nodes::Mapping)
344
- value_node.children.each_slice(2) do |k, v|
345
- coder[k.to_ruby.to_s] = visit(v)
359
+ coder = ::Yeptris::Psych::CoderShim.new
360
+ if v.is_a?(::Yeptris::Psych::Nodes::Mapping)
361
+ v.children.each_slice(2) do |ck, cv|
362
+ coder[ck.to_ruby.to_s] = visit(cv)
363
+ end
346
364
  end
365
+ obj.init_with(coder)
347
366
  end
348
- obj.init_with(coder)
367
+ obj
349
368
  end
350
369
 
351
370
  def resolve_class(name)
data/lib/yeptris/psych.rb CHANGED
@@ -1,5 +1,9 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "date"
4
+ require "time"
5
+ require "set"
6
+
3
7
  # The Psych drop-in namespace (TODO.impl/15 phase C).
4
8
  #
5
9
  # `require "yeptris/psych"` rebinds the top-level Psych constant to
@@ -20,6 +24,11 @@ module Yeptris
20
24
  autoload :Parser, "yeptris/psych/parser"
21
25
  autoload :CoderShim, "yeptris/psych/coder_shim"
22
26
  autoload :Visitors, "yeptris/psych/visitors"
27
+ # The typed opt-in marker for arbitrary-object dump/load
28
+ # (TODO.restructure/23). Eager by intent: classes include it at
29
+ # declaration time, so the autoload must resolve before any
30
+ # object instance exists.
31
+ autoload :Encodable, "yeptris/psych/encodable"
23
32
  class Error < StandardError; end
24
33
  class SyntaxError < Error
25
34
  attr_reader :line, :column
@@ -106,7 +115,7 @@ module Yeptris
106
115
  # data anyway
107
116
  out =
108
117
  case obj
109
- when nil, true, false, String, Integer, Float, Symbol, Date, Time
118
+ when nil, true, false, ::String, ::Integer, ::Float, ::Symbol, ::Date, ::Time
110
119
  Yeptris::YAML.dump(obj)
111
120
  else
112
121
  Visitors::YAMLTree.new.push(obj).finish
@@ -87,6 +87,35 @@ module Yeptris
87
87
  end
88
88
  end
89
89
 
90
+ # The Marshal fast path (TODO.restructure/21): the C side emits
91
+ # Ruby Marshal 4.8 bytes; one Marshal.load call (a core-C routine)
92
+ # materializes the whole object graph. ~10x faster than walking
93
+ # the columns in pure Ruby on JSON-shaped input; ~5x on YAML. Falls
94
+ # back to load_all_columns on the constructs the format cannot
95
+ # express (merge keys, timestamps — rare; the record walk handles
96
+ # them). Feature-detected: older libyeptris returns the walk.
97
+ def load_all_marshal(yaml, schema: :compat_11, mode: :first)
98
+ yaml = Yeptris.read_input(yaml)
99
+ yaml = yaml.to_s
100
+ out_p = ::FFI::MemoryPointer.new(:pointer)
101
+ olen_p = ::FFI::MemoryPointer.new(:size_t)
102
+ mode_id = mode == :all ? FFI::MARSHAL_ALL_DOCS : FFI::MARSHAL_FIRST_DOC
103
+ st = FFI.yeptris_marshal(
104
+ yaml, yaml.bytesize,
105
+ schema == :compat_11 ? FFI::SCHEMA_11_COMPAT : FFI::SCHEMA_12_CORE,
106
+ mode_id, out_p, olen_p
107
+ )
108
+ return nil if st == FFI::ERROR_UNSUPPORTED
109
+ raise ParseError, FFI.last_error_message if st != FFI::OK
110
+ buf = out_p.read_pointer
111
+ len = olen_p.read_uint64
112
+ bytes = buf.read_bytes(len)
113
+ bytes.force_encoding(Encoding::ASCII_8BIT)
114
+ ::Marshal.load(bytes)
115
+ ensure
116
+ FFI.yeptris_marshal_free(buf) if buf && !buf.null?
117
+ end
118
+
90
119
  def load(yaml, schema: :compat_11)
91
120
  docs = load_all(yaml, schema: schema)
92
121
  docs.empty? ? nil : docs.first
@@ -162,6 +191,12 @@ module Yeptris
162
191
  end
163
192
  slot(docs, stack, pending_key, pending_key_tag, v, merge_target, flat[i + IS_KEY], flat[i + TAG])
164
193
  when V_NULL
194
+ # the pending anchor binds to the null (an anchored empty
195
+ # scalar) — without this it would leak onto the next value
196
+ if pending_anchor
197
+ anchors[pending_anchor] = nil
198
+ pending_anchor = nil
199
+ end
165
200
  slot(docs, stack, pending_key, pending_key_tag, nil, merge_target, flat[i + IS_KEY], flat[i + TAG])
166
201
  when V_TS
167
202
  v = Materializer.parse_timestamp(arena.byteslice(flat[i + OFF], flat[i + LEN]))
@@ -315,6 +350,10 @@ module Yeptris
315
350
  end
316
351
  slot(docs, stack, pending_key, pending_key_tag, v, merge_target, ikeys[i], tags[i])
317
352
  when V_NULL
353
+ if pending_anchor
354
+ anchors[pending_anchor] = nil
355
+ pending_anchor = nil
356
+ end
318
357
  slot(docs, stack, pending_key, pending_key_tag, nil, merge_target, ikeys[i], tags[i])
319
358
  when V_TS
320
359
  v = Materializer.parse_timestamp(arena.byteslice(offs[i], lens[i]))
data/lib/yeptris/yaml.rb CHANGED
@@ -11,19 +11,50 @@ module Yeptris
11
11
  # schema: :compat_11 selects Psych/libyaml implicit typing
12
12
  # (yes/no, 0o/octal, sexagesimal); :core_12 (default) is YAML 1.2.
13
13
  def load(yaml, schema: :compat_11)
14
+ yaml = Yeptris.read_input(yaml)
15
+ yaml = yaml.to_s
16
+ # The native C materializer fuses strict-JSON scan→VALUE in one
17
+ # pass (beats JSON.parse on the 152 KB corpus). YAML inputs keep
18
+ # the FFI ladder so timestamps, aliases, and Psych's scalar
19
+ # quirks all resolve through the same path Psych.load uses.
20
+ if defined?(Yeptris::Native) && native_json?(yaml)
21
+ return Yeptris::Native.load_json(yaml)
22
+ end
14
23
  docs = _drain_all(yaml, schema)
15
24
  docs.empty? ? nil : docs.first
16
25
  end
17
26
 
18
27
  # Every document in the stream, in order.
19
28
  def load_stream(yaml, schema: :compat_11)
29
+ yaml = Yeptris.read_input(yaml)
30
+ yaml = yaml.to_s
20
31
  _drain_all(yaml, schema)
21
32
  end
22
33
 
23
- # The columnar drain when the loaded libyeptris has it (>= 0.1.2
24
- # era builds), the record drain otherwise — one code path, the
25
- # fastest the library offers.
34
+ # Strict-JSON sniff: first non-space is { or [ the fused native
35
+ # path beats JSON.parse on this shape (TODO.restructure/22).
36
+ def native_json?(bytes)
37
+ i = 0
38
+ len = bytes.bytesize
39
+ while i < len
40
+ c = bytes.getbyte(i)
41
+ return true if c == 0x7b || c == 0x5b # { or [
42
+ return false unless c == 0x20 || c == 0x09 || c == 0x0a || c == 0x0d
43
+
44
+ i += 1
45
+ end
46
+ false
47
+ end
48
+ private_class_method :native_json?
49
+
50
+ # The Marshal fast path when the loaded libyeptris has it (>= 0.1.11
51
+ # era builds), falling back to the columnar drain and finally the
52
+ # record drain — one code path, the fastest the library offers.
26
53
  def _drain_all(yaml, schema)
54
+ if FFI::MARSHAL
55
+ result = ValueML.load_all_marshal(yaml, schema: schema, mode: :all)
56
+ return result unless result.nil?
57
+ end
27
58
  if FFI::COLUMNS
28
59
  ValueML.load_all_columns(yaml, schema: schema)
29
60
  else
data/lib/yeptris.rb CHANGED
@@ -3,7 +3,7 @@
3
3
  module Yeptris
4
4
  # The gem's version lives in the parent namespace's file — the last
5
5
  # internal require (yeptris/version) retired with it.
6
- VERSION = "0.1.10.0".freeze
6
+ VERSION = "0.1.12.0".freeze
7
7
  # The error hierarchy lives in THIS file (the parent namespace's
8
8
  # own file): nested constants do not trigger a parent-constant
9
9
  # autoload, and the law forbids internal requires — defining the
@@ -58,3 +58,12 @@ rescue LoadError => e
58
58
  (Underlying error: #{e.message})
59
59
  MSG
60
60
  end
61
+
62
+ # Optional C-API materializer (TODO.restructure/22): fused visit →
63
+ # Ruby objects via the Ruby C API. Feature-detected — LoadError leaves
64
+ # the FFI ladder (Marshal → columns → records) as the sole path.
65
+ begin
66
+ require "yeptris/native"
67
+ rescue LoadError
68
+ # FFI ladder only
69
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: yeptris
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.10.0
4
+ version: 0.1.12.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ribose Inc.
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-09-06 00:00:00.000000000 Z
11
+ date: 2026-09-07 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: ffi
@@ -24,9 +24,9 @@ dependencies:
24
24
  - - "~>"
25
25
  - !ruby/object:Gem::Version
26
26
  version: '1.15'
27
- description: An FFI-based (no C extension) Ruby YAML library over libyeptris — Psych-compatible
28
- semantics with libleptris-class performance. The neutral Yeptris::YAML surface ships
29
- first; the Psych drop-in namespace lands with the recorder-driven Visitors.
27
+ description: A Ruby YAML library over libyeptris — Psych-compatible semantics with
28
+ libleptris-class performance, and a fused native JSON materializer that outperforms
29
+ JSON.parse on JSON-shaped input.
30
30
  email:
31
31
  - open.source@ribose.com
32
32
  executables: []
@@ -34,6 +34,9 @@ extensions: []
34
34
  extra_rdoc_files: []
35
35
  files:
36
36
  - README.adoc
37
+ - ext/yeptris_native/extconf.rb
38
+ - ext/yeptris_native/json_ruby.c
39
+ - ext/yeptris_native/yeptris_native.c
37
40
  - lib/yeptris.rb
38
41
  - lib/yeptris/document.rb
39
42
  - lib/yeptris/ffi.rb
@@ -41,6 +44,7 @@ files:
41
44
  - lib/yeptris/node.rb
42
45
  - lib/yeptris/psych.rb
43
46
  - lib/yeptris/psych/coder_shim.rb
47
+ - lib/yeptris/psych/encodable.rb
44
48
  - lib/yeptris/psych/handler.rb
45
49
  - lib/yeptris/psych/parser.rb
46
50
  - lib/yeptris/psych/visitors.rb