nosj 0.1.0-x86_64-linux → 0.2.0-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0a0fd640bbecb91b12ed17a8ee5edc5b87befe85b1fcdcd124c58624255b3d02
-  data.tar.gz: 6cc111ee1c0a1d55640325d48e5b71fad155f18d218211a9fcaf2f3f2a3a54b7
+  metadata.gz: 46cefa7803d25d779048e2c450ea1c5b5daa0c42508277432ef99df7918c783c
+  data.tar.gz: b11bcb71b9334785acbc7c3f1c48570f212e77646cb3c613db7cc76261895250
 SHA512:
-  metadata.gz: 6dcc90744851a53f59a088e2ea4429ece7df99e2b72cbe81d049818641beb500b6847b9a3d9f703094c6d2f9aec669c89ab3ae652d47d29a8f3cefe0ee195ba9
-  data.tar.gz: 1f50198d497a46a8abbd281934ce3a04e5a704a1cbec221e2cf1dc0d570de817e447c4eb580c61e288960d5c797ba8f3f7b5e9215e85f90b439ae02e9dce57db
+  metadata.gz: 1789804de8bff93994f515aba7d8fe5ac0f527e865bb8d917c8ed7d9a2cd08e7a3cddb2b0194653697eb62aafa466820c97b01c15a9afde9530a4f8b98bdedd5
+  data.tar.gz: 5aab59746b4856fb2afb505d18da3c4d9b2c3f0f02e115c402a17e94c5669b911443899cbf0958771606ae95789764ced92b091a1d5917cb186c620570811978

data/CHANGELOG.md

@@ -1,4 +1,25 @@
-## [0.1.0] - 2026-07-15
+## [0.2.0] - 2026-07-16
+
+- File APIs. `NOSJ.load_file(path, opts)` parses a file directly
+  (~1.3× faster than `parse(File.read(path))`—no file-sized Ruby
+  String is created), and `NOSJ.write_file(path, obj, opts)` generates
+  straight to disk, returning the byte count like `File.write`.
+  `NOSJ.load_lazy_file(path, opts)` wraps a file as a lazy document
+  over a read-only memory map, and `NOSJ.at_pointer_file` /
+  `NOSJ.dig_file` pull single values out of a file without reading the
+  rest into Ruby. Missing files raise the usual `Errno` exceptions.
+- `NOSJ.lazy`: lazy documents. Wrap a document once, then read only
+  what you need: `doc["users"][3]["name"]` parses just that path, `#dig`
+  and `#at_pointer` resolve whole paths, and `#keys`, `#size`, and
+  `#each` inspect a node without parsing its values. Containers come
+  back lazy, scalars come back as plain Ruby values, and repeated
+  reads are cached. `#value` (also `#to_h` / `#to_a`) materializes a
+  subtree under the usual parse options (`symbolize_names`, `freeze`,
+  ...). Pass a frozen string and creating the view is practically
+  free, even on megabyte documents. Malformed content raises on first
+  read, not at wrap time.
+
+## [0.1.0] - 2026-07-16
 
 Initial release.
 

data/README.md

@@ -10,10 +10,12 @@
 - It is **faster** than gem json and every
 third-party parser, including Oj, RapidJSON, FastJsonparser, Yajl. 1.0–1.8× faster than the bundled json gem, 1.3–11× faster than Oj, and up to 17×
 faster than Yajl—[see Benchmarks](#benchmarks).
+- It has **lazy documents**: `NOSJ.lazy` wraps a document and parses a value only when you touch it—repeated access costs nanoseconds, and everything you never read is never parsed.
+- It has a **partial parsing mode**: JSON Pointer lookups that pull single values out of big documents in microseconds, skipping everything else.
+- It has **file APIs**: parse, generate, dig, and lazy-wrap files directly—no throwaway file-sized Ruby String, and the partial modes memory-map the file so unread pages never even leave the disk.
 - It comes **precompiled** (platform gems built with per-platform optimizations,
 nothing to compile on install).
-- It has a **partial parsing mode**: JSON Pointer lookups that pull single values out of big documents in microseconds, skipping everything else.
-- Same API and option names as gem json.
+- Otherwise, same API and option names as gem json.
 
 **And there's more**: validate documents without building a single Ruby object, resolve whole batches of paths in one pass, and accelerate an entire application with a one-line drop-in.
 
@@ -83,17 +85,23 @@ NOSJ.generate(obj)                       # indent, space, object_nl, ...,
 NOSJ.pretty_generate(obj)                # ascii_only, script_safe, strict
 ```
 
-**Validation without parsing.** `NOSJ.valid?` runs the full
-parser—tokenizers, string decode, number validation—into a null sink
-and allocates no Ruby objects at all. It is 2-4× faster than
-`NOSJ.parse`, which already leads every parser above:
+**Lazy documents.** `NOSJ.lazy` wraps a document in a lazy view: read
+a field and only that path is parsed—containers stay lazy, scalars
+arrive as plain Ruby values, and repeated reads are cached:
 
 ```ruby
-NOSJ.valid?('{"a":1}')                #=> true
-NOSJ.valid?('{"a":}')                 #=> false
-NOSJ.valid?(src, max_nesting: false)  # same options as parse
+doc = NOSJ.lazy(json)
+doc["users"][3]["name"]   # parses only this path
+doc.dig("meta", "count")  # a whole path in one fused resolution
+doc["users"].size         # counted without materializing anything
+doc["users"][3].value     # materialize one subtree (parse options apply)
 ```
 
+Pass a frozen string and creating the view is practically free—
+nanoseconds, even on a megabyte document. Malformed content raises
+when it is first read, not at wrap time.
+
+
 **Partial parsing.** Pull values out of a document without
 materializing the rest—skipped content is stepped over at SIMD block
 speed, so a lookup costs what it skips, not what the document weighs:
@@ -114,11 +122,38 @@ at the far end of a 570 KB document costs ~71µs, still 13× faster
 than parse-then-dig. Misses return nil; matched subtrees materialize
 with the same options as `parse` (`symbolize_names:`, `freeze:`).
 
+**Files.** Every mode has a file-native form, so a document never
+round-trips through a throwaway Ruby String:
+
+```ruby
+NOSJ.load_file("config.json")                 # 1.3× File.read + parse
+NOSJ.write_file("out.json", obj)              # generate straight to disk
+NOSJ.dig_file("huge.json", "users", 3, "name")   # never reads the rest
+NOSJ.at_pointer_file("huge.json", "/meta/count")
+doc = NOSJ.load_lazy_file("huge.json")        # lazy view over a memory map
+```
+
+The partial and lazy forms memory-map the file, so pages you never
+read are never loaded from disk. Missing files raise the usual
+`Errno` exceptions. Measured numbers live in
+[Benchmarks → File APIs](#file-apis).
+
+**Validation without parsing.** `NOSJ.valid?` runs the full
+parser—tokenizers, string decode, number validation—into a null sink
+and allocates no Ruby objects at all. It is 2-4× faster than
+`NOSJ.parse`, which already leads every parser above:
+
+```ruby
+NOSJ.valid?('{"a":1}')                #=> true
+NOSJ.valid?('{"a":}')                 #=> false
+NOSJ.valid?(src, max_nesting: false)  # same options as parse
+```
+
 ## Benchmarks
 
 Every installed JSON gem, benchmark-ips: AWS EC2 c7a.2xlarge (AMD EPYC 9R14, Zen 4), Ruby 4.0.6 + YJIT, json 2.21.1, Oj 3.17.4, RapidJSON 0.4.0, FastJsonparser 0.6.0, Yajl 1.4.3, PGO build, 2026-07-16. `×N` = times slower than nosj. 
 
-Parse:
+### Parse
 
 | file | nosj (i/s) | json | Oj | FastJsonparser | RapidJSON | Yajl |
 |---|---:|---:|---:|---:|---:|---:|
@@ -136,7 +171,7 @@ Parse:
 | tolstoy | **8.9k** | ×1.79 | ×1.96 | ×2.29 | ×2.10 | ×17.31 |
 | twitter | **1.1k** | ×1.09 | ×1.83 | ×2.25 | ×2.61 | ×5.40 |
 
-Generate:
+### Generate
 
 | file | nosj (i/s) | json | Oj | RapidJSON | Yajl |
 |---|---:|---:|---:|---:|---:|
@@ -157,6 +192,23 @@ Generate:
 \* canada-generate is a statistical tie with the json gem (within
 measurement error).
 
+### File APIs
+
+twitter.json (570 KB) from a warm page cache, medians of 7 alternating
+rounds against the plain-Ruby composition on the same parser (Apple
+Silicon dev box, Ruby 4.0.6 + YJIT, PGO build, 2026-07-16):
+
+| operation | µs/op | vs the Ruby way |
+|---|---:|---|
+| `NOSJ.load_file` (parse the whole file) | 948 | ×1.33 vs `NOSJ.parse(File.read(path))` |
+| `NOSJ.dig_file` (one deep field) | 246 | ×5.2 vs read + parse + dig |
+| `NOSJ.load_lazy_file` + one field | 257 | ×5.0 vs read + parse + dig |
+
+`NOSJ.write_file` measures at parity with
+`File.write(NOSJ.generate(obj))` on this box—file-write timings swing
+too much for an honest multiplier; what it saves is the intermediate
+file-sized Ruby String.
+
 Reproduce with `rake bench` (the parity-gated comparison, after a PGO retrain—the shipping configuration) or `rake bench:ips` (the multi-gem shoot-out).
 
 ## Switching from the json gem

data/lib/nosj/lazy.rb

@@ -0,0 +1,165 @@
+# frozen_string_literal: true
+
+module NOSJ
+  # A lazy view of one JSON container inside a document created with
+  # {NOSJ.lazy}. Nothing is parsed until it is touched: indexing walks
+  # the raw bytes to the requested child (skipping siblings at SIMD
+  # block speed), returns another Lazy node for containers, and
+  # materializes plain Ruby values for scalars. Resolved children are
+  # cached, so repeated access is free and object identity is stable.
+  #
+  # Validation is as-you-go: skipped content is bracket-balance
+  # checked and resolved targets fully validated, so a malformed region
+  # raises when an access first walks it, not at {NOSJ.lazy} time.
+  #
+  # Nodes hold their own stable copy of the document (shared across the
+  # whole node tree), so mutating the source string later is harmless.
+  #
+  # @example Pull two fields out of a big document
+  #   doc = NOSJ.lazy(huge_json)
+  #   doc["users"][3]["name"]     # only this path is parsed
+  #   doc.dig("meta", "count")
+  #
+  # @example Materialize a subtree
+  #   doc["users"][3].value       #=> {"name" => ..., ...}
+  class Lazy
+    include Enumerable
+
+    # Resolves one child. String and Symbol keys index objects; Integer
+    # indices index arrays (negative indices resolve to +nil+, as in
+    # {NOSJ.dig}). Containers come back as further Lazy nodes, scalars
+    # as plain Ruby values, misses as +nil+. Results are cached on the
+    # node.
+    #
+    # @param token [String, Symbol, Integer]
+    # @return [NOSJ::Lazy, Object, nil]
+    def [](token)
+      cache = (@children ||= {})
+      cache.fetch(token) { cache[token] = __get(token) }
+    end
+
+    # Hash#dig-shaped access, fused into a single resolution: the whole
+    # path resolves in one walk of this node's bytes, no intermediate
+    # nodes. Semantics match {NOSJ.dig}: misses, negative indices, and
+    # steps into scalars resolve to +nil+. Not cached.
+    #
+    # @param path [Array<String, Symbol, Integer>]
+    # @return [NOSJ::Lazy, Object, nil]
+    def dig(first, *rest)
+      __dig([first, *rest])
+    end
+
+    # Resolves an RFC 6901 JSON Pointer within this node's subtree,
+    # with the same lazy/materialize behavior as {#[]}. Not cached.
+    #
+    # @param pointer [String] e.g. <tt>"/users/3/name"</tt>
+    # @return [NOSJ::Lazy, Object, nil]
+    def at_pointer(pointer)
+      __at_pointer(pointer)
+    end
+
+    # Materializes this node's whole subtree as plain Ruby values,
+    # under the options given to {NOSJ.lazy} (+symbolize_names+,
+    # +freeze+, ...).
+    #
+    # @return [Hash, Array]
+    def value
+      __materialize
+    end
+    alias_method :materialize, :value
+
+    # @return [Hash] the materialized subtree
+    # @raise [TypeError] on an array node
+    def to_h
+      raise TypeError, "to_h on a JSON array" unless object?
+
+      __materialize
+    end
+
+    # @return [Array] the materialized subtree
+    # @raise [TypeError] on an object node
+    def to_a
+      raise TypeError, "to_a on a JSON object" unless array?
+
+      __materialize
+    end
+
+    # @return [Boolean] whether this node is a JSON object
+    def object?
+      __kind == :object
+    end
+
+    # @return [Boolean] whether this node is a JSON array
+    def array?
+      __kind == :array
+    end
+
+    # The object's keys (always Strings), read in one walk without
+    # materializing any values.
+    #
+    # @return [Array<String>]
+    # @raise [TypeError] on an array node
+    def keys
+      __keys
+    end
+
+    # Entry count (object pairs or array elements), read in one walk
+    # without materializing anything.
+    #
+    # @return [Integer]
+    def size
+      __size
+    end
+    alias_method :length, :size
+    alias_method :count, :size
+
+    # @return [Boolean]
+    def empty?
+      size.zero?
+    end
+
+    # Iterates direct children, resolved in one walk: object nodes
+    # yield +[key, child]+ pairs (like Hash), array nodes yield
+    # children. Containers arrive as Lazy nodes, scalars as values.
+    #
+    # @return [Enumerator] when no block is given
+    def each(&block)
+      return enum_for(:each) unless block
+
+      if object?
+        __children.each { |pair| yield pair }
+      else
+        __children.each { |child| yield child }
+      end
+      self
+    end
+
+    # @return [String] kind and span size, without touching content
+    def inspect
+      "#<NOSJ::Lazy #{__kind} (#{__byte_size} bytes)>"
+    end
+    alias_method :to_s, :inspect
+
+    private :__get, :__dig, :__at_pointer, :__materialize, :__kind,
+      :__byte_size, :__keys, :__size, :__children
+  end
+
+  # Wraps a JSON document for lazy, on-demand access: returns a
+  # {NOSJ::Lazy} node for a container root, or the materialized value
+  # for a scalar root. The document bytes are copied once; no parsing
+  # happens beyond locating the root value.
+  #
+  # @example
+  #   doc = NOSJ.lazy('{"users":[{"name":"ada"},{"name":"grace"}]}')
+  #   doc["users"][1]["name"]  #=> "grace" — the rest is never parsed
+  #
+  # @param source [String] the JSON document (UTF-8 or US-ASCII)
+  # @param opts [Hash, nil] {NOSJ.parse} options applied whenever a
+  #   value materializes: +symbolize_names+, +freeze+, +max_nesting+,
+  #   +allow_nan+, +allow_trailing_comma+
+  # @return [NOSJ::Lazy, Object]
+  # @raise [RuntimeError] when the document root is malformed
+  def self.lazy(source, opts = nil)
+    lazy_native(source, opts)
+  end
+end

data/lib/nosj/version.rb

@@ -2,5 +2,5 @@
 
 module NOSJ
   # The gem version.
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

data/lib/nosj.rb

@@ -2,13 +2,15 @@
 
 require_relative "nosj/version"
 require_relative "nosj/native"
+require_relative "nosj/lazy"
 
 # nosj is the evil twin of the +json+ gem: the same API, output bytes,
 # option names, and error messages, backed by the Rust
 # {https://github.com/yaroslav/nosj nosj} crate with SIMD-accelerated
 # parsing and generation. Beyond the +json+ gem's surface it adds
-# zero-allocation validation ({.valid?}) and partial parsing
-# ({.dig}, {.at_pointer}, and their batch forms).
+# zero-allocation validation ({.valid?}), partial parsing
+# ({.dig}, {.at_pointer}, and their batch forms), and lazy documents
+# ({.lazy}).
 #
 # Options arrive as a positional Hash (the +json+ gem's own calling
 # convention); an explicit +**kwargs+ would allocate per call.
@@ -190,4 +192,85 @@ module NOSJ
   def self.at_pointers(source, pointers, opts = nil)
     at_pointers_native(source, pointers, opts)
   end
+
+  # Parses a JSON file, like +JSON.load_file+—except the file is read
+  # natively into a reused buffer, so no file-sized Ruby String is ever
+  # created (or garbage-collected).
+  #
+  # @example
+  #   NOSJ.load_file("config.json", symbolize_names: true)
+  #
+  # @param path [String] the file to parse (UTF-8)
+  # @param opts [Hash, nil] the same options as {.parse}
+  # @return [Object] the parsed value tree
+  # @raise [SystemCallError] +Errno::ENOENT+ and friends, like File.read
+  # @raise [RuntimeError] when the document is malformed or not UTF-8
+  def self.load_file(path, opts = nil)
+    load_file_native(path, opts)
+  end
+
+  # Generates +obj+ as JSON and writes it to +path+, streaming the
+  # generator's buffer straight to disk—no intermediate Ruby String.
+  #
+  # @example
+  #   NOSJ.write_file("out.json", {"a" => [1, true]})   #=> 14
+  #   NOSJ.write_file("pretty.json", obj, indent: "  ", object_nl: "\n")
+  #
+  # @param path [String] the file to (over)write
+  # @param obj [Object] the value tree to generate
+  # @param opts [Hash, nil] the same options as {.generate}
+  # @return [Integer] the number of bytes written, like File.write
+  # @raise [SystemCallError] +Errno::ENOENT+ and friends, like File.write
+  # @raise [GeneratorError] like {.generate}
+  def self.write_file(path, obj, opts = nil)
+    write_file_native(path, obj, opts)
+  end
+
+  # Wraps a JSON file as a lazy document ({.lazy} for files): the file
+  # is memory-mapped read-only, so beyond one sequential UTF-8 check,
+  # pages you never read are never loaded from disk. The mapping lives
+  # as long as any node on it; the file must not be modified while it
+  # is in use.
+  #
+  # @example
+  #   doc = NOSJ.load_lazy_file("huge.json")
+  #   doc["users"][3]["name"]   # touches only these pages
+  #
+  # @param path [String] the file to wrap (UTF-8)
+  # @param opts [Hash, nil] {.parse} options applied on materialization
+  # @return [NOSJ::Lazy, Object]
+  # @raise [SystemCallError] +Errno::ENOENT+ and friends
+  # @raise [RuntimeError] when the file is not UTF-8 or the root is malformed
+  def self.load_lazy_file(path, opts = nil)
+    load_lazy_file_native(path, opts)
+  end
+
+  # {.at_pointer} against a file: memory-maps it, resolves the pointer,
+  # materializes only the matched subtree, and never reads the rest
+  # into Ruby.
+  #
+  # @example
+  #   NOSJ.at_pointer_file("huge.json", "/users/3/name")
+  #
+  # @param path [String] the file to query (UTF-8)
+  # @param pointer [String] an RFC 6901 JSON Pointer
+  # @param opts [Hash, nil] materialization options
+  # @return [Object, nil] nil when the pointer misses
+  # @raise [ArgumentError] for malformed pointers
+  def self.at_pointer_file(path, pointer, opts = nil)
+    at_pointer_file_native(path, pointer, opts)
+  end
+
+  # {.dig} against a file: the Hash#dig-shaped counterpart of
+  # {.at_pointer_file}. Negative indices resolve to nil.
+  #
+  # @example
+  #   NOSJ.dig_file("huge.json", "users", 3, "name")
+  #
+  # @param path [String] the file to query (UTF-8)
+  # @param path_elements [Array<String, Symbol, Integer>]
+  # @return [Object, nil]
+  def self.dig_file(path, *path_elements)
+    dig_file_native(path, path_elements)
+  end
 end

data/sig/nosj.rbs

@@ -47,6 +47,60 @@ module NOSJ
 
   def self.at_pointers: (String source, Array[String] pointers, ?opts opts) -> Array[value]
 
+  def self.lazy: (String source, ?opts opts) -> (Lazy | value)
+
+  def self.load_file: (String path, ?opts opts) -> value
+
+  def self.write_file: (String path, untyped obj, ?opts opts) -> Integer
+
+  def self.load_lazy_file: (String path, ?opts opts) -> (Lazy | value)
+
+  def self.at_pointer_file: (String path, String pointer, ?opts opts) -> value
+
+  def self.dig_file: (String path, *path_element path_elements) -> value
+
+  # A lazy view of one JSON container: children resolve on first
+  # access (containers as further Lazy nodes, scalars as values,
+  # misses as nil) and are cached on the node.
+  class Lazy
+    include Enumerable[untyped]
+
+    def []: (path_element token) -> (Lazy | value)
+
+    def dig: (path_element first, *path_element rest) -> (Lazy | value)
+
+    def at_pointer: (String pointer) -> (Lazy | value)
+
+    def value: () -> value
+
+    alias materialize value
+
+    def to_h: () -> Hash[String | Symbol, value]
+
+    def to_a: () -> Array[value]
+
+    def object?: () -> bool
+
+    def array?: () -> bool
+
+    def keys: () -> Array[String]
+
+    def size: () -> Integer
+
+    alias length size
+
+    alias count size
+
+    def empty?: () -> bool
+
+    def each: () { (untyped) -> void } -> self
+            | () -> Enumerator[untyped, self]
+
+    def inspect: () -> String
+
+    alias to_s inspect
+  end
+
   # Defined by `require "nosj/multi_json"`. The runtime superclass is
   # multi_json's Adapter (whose namespace differs across multi_json
   # versions), so it is not declared here.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nosj
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: x86_64-linux
 authors:
 - Yaroslav Markin
@@ -11,7 +11,9 @@ date: 1980-01-02 00:00:00.000000000 Z
 dependencies: []
 description: 'gem nosj is an extremely fast, json-gem-compatible JSON parser and generator
   for Ruby: Rust and SIMD via the first-party nosj crate, precompiled platform gems
-  with per-platform PGO, partial parsing (JSON Pointer, single and batch), allocation-free
+  with per-platform PGO, partial parsing (JSON Pointer, single and batch), lazy documents
+  that parse a value only when you touch it, file APIs that parse, generate, and query
+  files natively (memory-mapped, so unread pages never leave the disk), allocation-free
   validation, and a one-line JSON module drop-in.'
 email:
 - yaroslav@markin.net
@@ -28,6 +30,7 @@ files:
 - lib/nosj/3.4/nosj.so
 - lib/nosj/4.0/nosj.so
 - lib/nosj/json.rb
+- lib/nosj/lazy.rb
 - lib/nosj/multi_json.rb
 - lib/nosj/native.rb
 - lib/nosj/version.rb