senml 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7ef841f3fb72ceb0281f1d3436b14f35e9962ffa9561b2491f946740816f755b
+  data.tar.gz: e2af948829e3b3a4a64691f5f2cf0f9d4d5fa824be914bc87520291f212b732d
+SHA512:
+  metadata.gz: fe3bd198b38e2c030089840be2a1c8b5550c69942c93fc9be4594e17605ed67c7e47465c7dcb42e44a9e4f0c6d718cb8944a639438e96c57dec786a7213b282f
+  data.tar.gz: d1e18dc6e68b9c7aa704df707b954a5018af81e635784a0034cf3b6d464eef548e8a3c692bd3403203c53d3fd45c00cb13b5aba83f7d2e9b42508e2802c0469b

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.1.0 - 2026-10-12
+
+Initial release.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Yudai Takada
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,114 @@
+# SenML
+
+Read, validate, write, and resolve [RFC 8428](https://www.rfc-editor.org/rfc/rfc8428) sensor measurements in Ruby. Supports SenML JSON, incremental SenSML JSON, record fragments, and optional CBOR.
+
+## Installation
+
+```ruby
+gem "senml"
+# Optional on CRuby 3.3–4.0 and TruffleRuby:
+gem "cbor", "~> 0.5"
+```
+
+Requires Ruby 3.3 or later and JSON 2.19.4 or later, below version 4. JSON works on CRuby, JRuby, and TruffleRuby. The optional `cbor` native extension is tested on CRuby 3.3–4.0 and TruffleRuby; JRuby and Ruby 4.1 development builds run JSON features only. There are no dependencies on other measurement or JSON-format gems.
+
+## Read and resolve
+
+```ruby
+require "senml"
+
+pack = SenML::Pack.build(base_name: "room:", base_time: 1_700_000_000, base_unit: "Cel") do |p|
+  p.add(n: "temperature", v: 21.5, t: 2)
+  p.add(n: "temperature", v: 21.8, t: 1)
+end
+
+parsed = SenML.parse(pack.to_json)
+parsed.first.name                         # => "room:temperature"
+parsed.first.time                         # => 1700000002
+parsed.first["t"]                         # => 2 (the original field)
+resolved = parsed.resolve(now: Time.at(1_700_000_000))
+resolved.map(&:value)                     # => [21.8, 21.5]
+resolved.first.time_at                    # => a UTC Time
+```
+
+`name`, `unit`, `time`, `value`, and `sum` apply the current base fields. `[]` and `to_h` preserve original fields, including unknown noncritical extensions. `Record#data` lazily decodes unpadded base64url `vd` into a binary String. String and boolean measurements do not receive numeric base values. A record may have a value, a sum, or both; an inherited base value or base sum can supply a missing measurement field.
+
+Resolution removes base fields, expands names, units, values and sums, converts relative times using `now`, and stably sorts by time. Version 10 is implicit; legacy positive versions up to 10 are retained when resolving. Future versions and mixed versions are rejected. A relative `Record#time_at` needs an explicit `now:`. Integers retain Ruby precision; nonintegral numbers use binary64 Float, and generated JSON preserves finite Float values across parsing.
+
+## Streams and fragments
+
+```ruby
+File.open("measurements.json", "rb") do |io|
+  SenML::StreamReader.new(io).each { |record| puts record.name }
+end
+
+selected = pack.select_fragment("#rec=1,2-*")
+```
+
+The stream reader yields each complete object without retaining the whole array. It carries immutable base fields and version state between records. Repeated records reuse inherited base strings; name-prefix validation is cached until the base changes. `allow_incomplete: true` also accepts a SenSML stream that ends after a complete record without a closing array bracket; an unfinished object still fails. This reader handles JSON streams; CBOR parsing consumes a complete input String.
+
+Fragments use one-based record indices and support single records, closed ranges, open ranges, and comma-separated combinations. Selection keeps original order, removes duplicates, and carries each selected record's effective base context, emitting inherited base fields only at the first selected record and when they change. Out-of-range and reversed ranges fail.
+
+## Validation and limits
+
+Strict parsing checks field types, value counts, resolved names, registered units, versions, and canonical unpadded base64url. Duplicate JSON keys and uppercase numeric exponents fail. `strict: false` preserves recoverable semantic problems as `pack.warnings`; field types, unknown critical labels ending in `_`, and version problems always fail. Writing and resolving perform strict validation again.
+
+All APIs that construct packs or readers accept these positive integer limits:
+
+| Option | Default | Applies to |
+| --- | ---: | --- |
+| `max_bytes` | 16,777,216 | Complete inputs/outputs and the JSON representation of native or decoded packs |
+| `max_records` | 100,000 | Packs and streams |
+| `max_record_bytes` | 1,048,576 | Each record, including stream buffers |
+| `max_data_bytes` | 786,432 | Decoded binary measurement size |
+| `max_depth` | 32 | Nested values; hard maximum 128 |
+| `max_warnings` | 1,000 | Retained lenient diagnostics in packs and streams |
+
+Native array construction and CBOR decoding also check the cumulative JSON representation size against `max_bytes`; the decoded representation may be larger than its CBOR input. Streams use record limits instead of a total byte limit. Exceeding `max_warnings` raises `LimitError` before yielding the next record; `reader.warnings` retains the diagnostics collected so far. Resolution checks `max_bytes` incrementally as base fields expand, before retaining an oversized resolved pack. Binary size is checked before decoding. `SenML::ParseError` reports malformed input or field types, `ValidationError` reports semantic problems with a `pointer`, `UnsupportedVersionError` identifies unknown versions, and `LimitError` reports exceeded bounds. A conversion that cannot preserve supported semantics raises `ConversionError`.
+
+## Optional CBOR
+
+```ruby
+require "senml/cbor"
+bytes = pack.to_cbor
+SenML.parse_cbor(bytes)
+# Accept a complete SenSML indefinite-length CBOR array:
+SenML.parse_cbor(stream_bytes, streaming: true)
+```
+
+CBOR maps the RFC's integer labels and binary data values, preserves JSON-compatible extension fields, and supports decimal-fraction tag 4 with exponents from -400 to 400. A preflight scan checks lengths, nesting, duplicate map keys, and allowed types before the native decoder runs. Unknown integer labels, non-JSON binary extension values, other tags, indefinite strings, and lossy conversion of large integers fail explicitly. JSON nonintegral values use Float; conversion of arbitrary decimal fractions can round to binary64.
+
+## Scope and reference data
+
+| Supported | Outside this release |
+| --- | --- |
+| SenML and incremental SenSML JSON | XML and EXI |
+| Base resolution and record fragments | Unit conversion and CoAP transport |
+| Optional CBOR on the runtimes listed above | Incremental CBOR decoding |
+| RFC 8428 version rules and registered labels | FETCH/PATCH and RFC 9100 feature versions |
+
+The implementation supports RFC 8428 sections 4–6 and 9: SenML/SenSML JSON, base resolution, optional CBOR, and record fragments. It recognizes the IANA unit identifiers and registered labels checked on 2026-10-11, including `vlo` and the `ct`/`bct` labels from [RFC 9193](https://www.rfc-editor.org/rfc/rfc9193). Content-format strings are preserved opaquely, and `bct` resolves onto binary measurements. Unknown base labels stop resolution and fragment selection because their inheritance rules are unknown, including when resolving a later record that inherits the unsupported base.
+
+XML, EXI, unit conversion, CoAP transport, FETCH/PATCH variants, and feature-version negotiation from RFC 9100 are outside this release. The library makes no network requests when parsing or resolving measurements.
+
+Only independently authored identifier/type metadata is shipped in `data/`. RFC text, examples, grammars, IANA source files, and external implementation code are never committed or packaged. Development scripts fetch checksum-pinned references into ignored `tmp/` directories, retain notices, and extract test fixtures there. See [IANA terms](https://www.iana.org/help/licensing-terms), [IETF terms](https://trustee.ietf.org/license-info), and `script/sources.json` for sources and checksums.
+
+The RFC's explicitly unfinished section 5.1.2 stream is closed after its final complete measurement for whole-pack tests. The first section 5.1.7 thermostat example starts with a base-only record, conflicting with section 4.2's value-or-sum requirement; strict parsing rejects the original, and its base name is moved into the first measurement for the roundtrip fixture. Original excerpts and corrections remain together in ignored fixture storage. [Erratum 6728](https://www.rfc-editor.org/errata/eid6728), held for document update, concerns an XML/CDDL comma and does not change the supported encodings.
+
+## Development
+
+```sh
+bundle install
+bundle exec rake test
+COVERAGE=1 bundle exec rake test
+bundle exec rubocop
+bundle exec rbs -I sig validate
+bundle exec ruby script/interop.rb  # Go and unzip required
+bundle exec ruby script/benchmark.rb
+```
+
+Tests cover all 14 RFC JSON examples, the full CBOR hex example, all five fragment examples, MUST violations, malformed and random CBOR, random numeric roundtrips, and a generated 100,000-record stream. CI compares normalization with pinned `farshidtz/senml` v2.0.0 (MIT), retaining its license in temporary storage. The IANA checker fails on unexpected registry changes; review updated sources and checksums before running `bundle exec ruby script/update_registry.rb --write`.
+
+## License
+
+MIT. See [LICENSE.txt](LICENSE.txt).

data/SECURITY.md

@@ -0,0 +1,11 @@
+# Security
+
+Report vulnerabilities privately through [GitHub security advisories](https://github.com/ydah/senml/security/advisories/new).
+
+Parsing does not perform network requests or execute measurement content. Inputs have configurable byte, record, depth, and binary-data limits. Streaming applies per-record limits before retaining further bytes. CBOR is optional and uses a native extension after a bounded structural preflight scan; keep that dependency up to date.
+
+Unknown critical labels and unsupported or mixed format versions fail even in lenient mode. Noncritical extension data is retained without interpretation. Applications must decide which sensor names, units, content formats, values, and timestamps they trust. Conversion and resolution do not authenticate measurements.
+
+Use bounds appropriate to the application's workload. Parsing a complete pack and resolving it require memory proportional to the pack; use the JSON stream reader when whole-pack sorting is unnecessary. A stream can yield valid records before a later malformed record raises an error, so callers needing atomic ingestion must provide their own transaction boundary.
+
+Lenient diagnostics stop with `LimitError` at `max_warnings` (default 1,000). Inherited base strings are immutable and shared; resolution stops before expanded output exceeds `max_bytes`. Unknown base context also prevents resolution of later records.

data/data/labels.yml

@@ -0,0 +1,19 @@
+# Independently authored implementation metadata.
+bn: {cbor: -2, type: string, accessor: base_name, base: true, section: '4.1'}
+bt: {cbor: -3, type: number, accessor: base_time, base: true, section: '4.1'}
+bu: {cbor: -4, type: string, accessor: base_unit, base: true, section: '4.1'}
+bv: {cbor: -5, type: number, accessor: base_value, base: true, section: '4.1'}
+bs: {cbor: -6, type: number, accessor: base_sum, base: true, section: '4.1'}
+bver: {cbor: -1, type: integer, accessor: base_version, base: true, section: '4.1'}
+n: {cbor: 0, type: string, accessor: name, section: '4.2'}
+u: {cbor: 1, type: string, accessor: unit, section: '4.2'}
+v: {cbor: 2, type: number, accessor: numeric_value, section: '4.2'}
+vs: {cbor: 3, type: string, accessor: string_value, section: '4.2'}
+vb: {cbor: 4, type: boolean, accessor: boolean_value, section: '4.2'}
+vd: {cbor: 8, type: string, accessor: data_value, section: '4.2'}
+s: {cbor: 5, type: number, accessor: sum, section: '4.2'}
+t: {cbor: 6, type: number, accessor: time, section: '4.2'}
+ut: {cbor: 7, type: number, accessor: update_time, section: '4.2'}
+vlo: {type: string, accessor: object_link, section: 'OMA LwM2M 1.1 7.4.5'}
+bct: {type: string, accessor: base_content_format, base: true, section: 'RFC9193 4'}
+ct: {type: string, accessor: content_format, section: 'RFC9193 3'}

data/data/units.yml

@@ -0,0 +1,68 @@
+---
+- m
+- kg
+- g
+- s
+- A
+- K
+- cd
+- mol
+- Hz
+- rad
+- sr
+- "N"
+- Pa
+- J
+- W
+- C
+- V
+- F
+- Ohm
+- S
+- Wb
+- T
+- H
+- Cel
+- lm
+- lx
+- Bq
+- Gy
+- Sv
+- kat
+- m2
+- m3
+- l
+- m/s
+- m/s2
+- m3/s
+- l/s
+- W/m2
+- cd/m2
+- bit
+- bit/s
+- lat
+- lon
+- pH
+- dB
+- dBW
+- Bspl
+- count
+- "/"
+- "%"
+- "%RH"
+- "%EL"
+- EL
+- 1/s
+- 1/min
+- beat/min
+- beats
+- S/m
+- B
+- VA
+- VAs
+- var
+- vars
+- J/m
+- kg/m3
+- deg
+- NTU

data/lib/senml/cbor.rb

@@ -0,0 +1,189 @@
+# frozen_string_literal: true
+
+require_relative "../senml"
+require "cbor"
+module SenML
+  module CBORCodec
+    KEYS = LABELS.to_h { |label, spec| [spec.fetch("cbor", label), label] }.freeze
+    # Bound nesting and allocations, and reject duplicate keys before the native decoder loses them.
+    class Guard
+      def initialize(bytes, limits, streaming)
+        @bytes = bytes
+        @limits = limits
+        @streaming = streaming
+        @offset = 0
+      end
+
+      def check
+        raise ParseError, "CBOR pack must be an array" unless @bytes.getbyte(0)&.div(32) == 4
+        raise ParseError, "SenML CBOR requires a definite-length array" if @bytes.getbyte(0) == 159 && !@streaming
+
+        scan(0)
+        raise ParseError, "trailing CBOR data" unless @offset == @bytes.bytesize
+      end
+
+      private
+
+      def take(size)
+        raise ParseError, "truncated CBOR" if size > @bytes.bytesize - @offset
+
+        result = @bytes.byteslice(@offset, size)
+        @offset += size
+        result
+      end
+
+      def scan(depth)
+        raise LimitError, "CBOR exceeds max_depth" if depth > @limits[:max_depth]
+
+        initial = take(1).getbyte(0)
+        major = initial >> 5
+        info = initial & 31
+        size = case info
+               when 0..23 then info
+               when 24 then take(1).unpack1("C")
+               when 25 then take(2).unpack1("n")
+               when 26 then take(4).unpack1("N")
+               when 27 then take(8).unpack1("Q>")
+               when 31 then nil
+               else raise ParseError, "reserved CBOR argument"
+               end
+        case major
+        when 0, 1 then raise ParseError, "indefinite CBOR integer" unless size
+        when 2, 3
+          raise ParseError, "CBOR strings must have definite length" unless size
+          raise LimitError, "CBOR string exceeds record bound" if size > @limits[:max_record_bytes]
+
+          take(size)
+        when 4, 5
+          raise LimitError, "CBOR container exceeds input size" if size && size > @bytes.bytesize
+          raise LimitError, "CBOR pack exceeds max_records" if depth.zero? && size && size > @limits[:max_records]
+
+          count = 0
+          keys = {}
+          until size ? count == size : @bytes.getbyte(@offset) == 255
+            start = @offset
+            if major == 5
+              kind = @bytes.getbyte(@offset)&.div(32)
+              raise ParseError, "CBOR map keys must be integers or text" unless [0, 1, 3].include?(kind)
+
+              scan(depth + 1)
+              key = CBOR.decode(@bytes.byteslice(start, @offset - start))
+              raise ParseError, "duplicate CBOR map key" if keys.key?(key)
+
+              keys[key] = true
+            end
+            scan(depth + 1)
+            count += 1
+            raise LimitError, "CBOR record exceeds max_record_bytes" if depth.zero? && @offset - start > @limits[:max_record_bytes]
+            raise LimitError, "CBOR pack exceeds max_records" if depth.zero? && count > @limits[:max_records]
+          end
+          take(1) unless size
+        when 6
+          raise ParseError, "unsupported CBOR tag" unless size == 4
+
+          scan(depth + 1)
+        when 7
+          raise ParseError, "unsupported CBOR simple value" unless [20, 21, 22, 25, 26, 27].include?(info)
+        end
+      end
+    end
+
+    def self.decode(bytes, limits, streaming)
+      Guard.new(bytes, limits, streaming).check
+      raw = CBOR.decode(bytes)
+      raw.map do |record|
+        raise ParseError, "CBOR record must be a map" unless record.is_a?(Hash)
+
+        mapped = record.to_h do |key, value|
+          label = KEYS[key]
+          raise ParseError, "unknown integer CBOR label" if key.is_a?(Integer) && !label
+
+          label ||= key
+          raise ParseError, "CBOR labels must be text or defined integers" unless label.is_a?(String)
+
+          if label == "vd"
+            raise ParseError, "CBOR data value must be a byte string" unless value.is_a?(String) && value.encoding == Encoding::BINARY
+
+            value = [value].pack("m0").tr("+/", "-_").delete("=")
+          else
+            value = json_value(value)
+          end
+          [label, value]
+        end
+        raise ParseError, "duplicate translated CBOR label" unless mapped.size == record.size
+
+        mapped
+      end
+    rescue CBOR::UnpackError => e
+      raise ParseError, e.message
+    end
+
+    def self.json_value(value)
+      case value
+      when CBOR::Tagged
+        raise ParseError, "invalid decimal fraction" unless value.tag == 4 && value.value.is_a?(Array) && value.value.length == 2 && value.value.all?(Integer)
+
+        exponent, mantissa = value.value
+        raise ParseError, "decimal fraction exponent outside supported range" unless exponent.between?(-400, 400)
+
+        (mantissa * (10.to_r**exponent)).to_f
+      when Hash then value.transform_values { |v| json_value(v) }
+      when Array then value.map { |v| json_value(v) }
+      when String
+        raise ConversionError, "unknown binary extension cannot be represented as JSON" if value.encoding == Encoding::BINARY
+
+        value
+      else value
+      end
+    end
+
+    def self.encode(records, limits)
+      values = records.map do |record|
+        record.to_h do |label, value|
+          value = label == "vd" ? Record.new(record, **limits).data : cbor_value(value)
+          [LABELS.fetch(label, {}).fetch("cbor", label), value]
+        end
+      end
+      CBOR.encode(values)
+    rescue CBOR::TypeError, RangeError => e
+      raise ConversionError, e.message
+    end
+
+    def self.cbor_value(value)
+      case value
+      when Hash then value.transform_values { |v| cbor_value(v) }
+      when Array then value.map { |v| cbor_value(v) }
+      when Integer
+        return value if value.between?(-(2**64), (2**64) - 1)
+
+        number = value.to_f
+        raise ConversionError, "integer cannot be represented exactly in supported CBOR numbers" unless number.finite? && number.to_i == value
+
+        number
+      else value
+      end
+    end
+  end
+
+  # Parse a complete binary CBOR pack; load this API with require 'senml/cbor'.
+  # @param bytes [String] CBOR octets
+  # @param streaming [Boolean] permit a SenSML indefinite-length array
+  # @return [Pack]
+  def self.parse_cbor(bytes, strict: true, streaming: false, **options)
+    options = limits(options)
+    raise ParseError, "CBOR input must be a String" unless bytes.is_a?(String)
+    raise LimitError, "CBOR pack exceeds max_bytes" if bytes.bytesize > options[:max_bytes]
+
+    Pack.new(CBORCodec.decode(bytes, options, streaming), strict: strict, **options)
+  end
+
+  class Pack
+    def to_cbor
+      self.class.new(to_h, **@limits)
+      result = CBORCodec.encode(to_h, @limits)
+      raise LimitError, "CBOR output exceeds max_bytes" if result.bytesize > @limits[:max_bytes]
+
+      result
+    end
+  end
+end

data/lib/senml/json_codec.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module SenML
+  module JSONCodec
+    class Number
+      def initialize(source)
+        raise ParseError, "numeric exponent must use lowercase e" if source.include?("E")
+
+        @value = Float(source)
+      end
+      attr_reader :value
+    end
+
+    def self.parse(source, max_depth:)
+      source = source.dup.force_encoding(Encoding::UTF_8)
+      raise ParseError, "JSON must be valid UTF-8" unless source.valid_encoding?
+
+      convert(JSON.parse(source, allow_duplicate_key: false, decimal_class: Number, max_nesting: max_depth))
+    rescue JSON::ParserError, EncodingError => e
+      raise ParseError, e.message
+    end
+
+    def self.convert(value)
+      case value
+      when Hash then value.transform_values { |v| convert(v) }
+      when Array then value.map { |v| convert(v) }
+      when Number then value.value
+      else value
+      end
+    end
+
+    def self.generate(value)
+      case value
+      when Hash then "{#{value.map { |k, v| "#{JSON.generate(k)}:#{generate(v)}" }.join(",")}}"
+      when Array then "[#{value.map { |v| generate(v) }.join(",")}]"
+      when Float
+        raise ValidationError, "nonfinite number" unless value.finite?
+
+        number = format("%.17g", value)
+        number += ".0" unless number.match?(/[.e]/)
+        number
+      else JSON.generate(value)
+      end
+    rescue JSON::GeneratorError, EncodingError => e
+      raise ValidationError, e.message
+    end
+  end
+end

data/lib/senml/labels.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module SenML
+  LABELS = YAML.safe_load_file(File.expand_path("../../data/labels.yml", __dir__)).transform_values do |fields|
+    fields.transform_values(&:freeze).freeze
+  end.freeze
+  BASE_LABELS = LABELS.select { |_, fields| fields["base"] }.keys.freeze
+  VALUE_LABELS = %w[v vs vb vd vlo].freeze
+  module Units
+    SYMBOLS = YAML.safe_load_file(File.expand_path("../../data/units.yml", __dir__)).map(&:freeze).freeze
+    def self.registered?(symbol) = SYMBOLS.include?(symbol)
+  end
+end

data/lib/senml/pack.rb

@@ -0,0 +1,128 @@
+# frozen_string_literal: true
+
+module SenML
+  class Pack
+    include Enumerable
+
+    attr_reader :warnings
+
+    def initialize(records, strict: true, **options)
+      @limits = SenML.limits(options)
+      raise ParseError, "pack must be an array" unless records.is_a?(Array)
+      raise ValidationError, "pack must contain at least one record" if records.empty?
+      raise LimitError, "pack exceeds max_records" if records.length > @limits[:max_records]
+
+      validator = Validator.new(strict: strict, limits: @limits)
+      bytes = 2
+      @records = records.each_with_index.map do |data, index|
+        data = data.to_h if data.is_a?(Record)
+        base = validator.accept(data, index)
+        bytes += validator.record_bytes + (index.zero? ? 0 : 1)
+        raise LimitError, "pack exceeds max_bytes" if bytes > @limits[:max_bytes]
+
+        Record.new(data, base: base, **@limits)
+      end
+      @warnings = validator.warnings
+    end
+
+    def each(&) = @records.each(&)
+    def [](index) = @records[index]
+    def size = @records.size
+    alias length size
+    def to_h = @records.map(&:to_h)
+
+    def to_json(*_args)
+      self.class.new(to_h, **@limits)
+      text = JSONCodec.generate(to_h)
+      raise LimitError, "pack exceeds max_bytes" if text.bytesize > @limits[:max_bytes]
+
+      text
+    end
+
+    # Expand inherited base fields and stably sort absolute measurement times.
+    # @param now [Time, Integer, Float] reference for relative timestamps
+    # @return [Pack] a new pack without inherited base fields
+    def resolve(now: Time.now)
+      reference = now.is_a?(Time) ? now.to_f : now
+      raise ArgumentError, "now must be a finite number or Time" unless reference.is_a?(Numeric) && reference.finite?
+
+      bytes = 2 # JSON array delimiters
+      validator = Validator.new(strict: true, limits: @limits)
+      records = @records.each_with_index.map do |record, index|
+        resolved = record.resolved(now: reference)
+        validator.json_value(resolved, "/#{index}", 0)
+        bytes += JSONCodec.generate(resolved).bytesize + (index.zero? ? 0 : 1)
+        raise LimitError, "resolved pack exceeds max_bytes" if bytes > @limits[:max_bytes]
+
+        [resolved, index]
+      end
+      self.class.new(records.sort_by { |record, index| [record["t"], index] }.map(&:first), **@limits)
+    end
+
+    # Select one-based indices and ranges, preserving inherited base context.
+    # @param fragment [String] RFC 8428 fragment, optionally beginning with '#'
+    # @return [Pack]
+    def select_fragment(fragment)
+      fragment = fragment.delete_prefix("#")
+      raise ValidationError, "invalid record fragment" unless fragment.match?(/\Arec=[1-9]\d*(?:-(?:[1-9]\d*|\*))?(?:,[1-9]\d*(?:-(?:[1-9]\d*|\*))?)*\z/)
+
+      selected = {}
+      fragment.delete_prefix("rec=").split(",").each do |range|
+        start, stop = range.split("-", 2)
+        first = start.to_i
+        last = if stop == "*"
+                 size
+               else
+                 (stop ? stop.to_i : first)
+               end
+        raise ValidationError, "fragment outside pack bounds" unless last.between?(first, size)
+
+        (first..last).each { |index| selected[index - 1] = true }
+      end
+      base = {}
+      emitted_base = {}
+      bytes = 2
+      selected_count = 0
+      records = @records.each_with_index.filter_map do |record, index|
+        data = record.to_h
+        unknown_base = data.keys.find { |key| key.start_with?("b") && !BASE_LABELS.include?(key) }
+        raise ConversionError, "cannot select with unknown base field #{unknown_base}" if unknown_base
+
+        BASE_LABELS.each { |label| base[label] = data[label] if data.key?(label) }
+        next unless selected[index]
+
+        changes = base.reject { |label, value| emitted_base.key?(label) && emitted_base[label] == value }
+        entry = changes.merge(data)
+        bytes += JSONCodec.generate(entry).bytesize + (selected_count.zero? ? 0 : 1)
+        raise LimitError, "selected pack exceeds max_bytes" if bytes > @limits[:max_bytes]
+
+        emitted_base = base.dup
+        selected_count += 1
+        entry
+      end
+      self.class.new(records, **@limits)
+    end
+
+    def self.build(base_name: nil, base_time: nil, base_unit: nil, **)
+      builder = Builder.new({ "bn" => base_name, "bt" => base_time, "bu" => base_unit }.compact)
+      yield builder
+      new(builder.records, **)
+    end
+
+    class Builder
+      attr_reader :records
+
+      def initialize(base)
+        @base = base
+        @records = []
+      end
+
+      def add(**values)
+        data = values.transform_keys(&:to_s)
+        data = @base.merge(data) if @records.empty?
+        @records << data
+        self
+      end
+    end
+  end
+end

data/lib/senml/record.rb

@@ -0,0 +1,79 @@
+# frozen_string_literal: true
+
+module SenML
+  class Record
+    LABELS.each { |label, fields| define_method(fields.fetch("accessor")) { self[label] } }
+    def initialize(data, base: {}, **options)
+      @data = SenML.copy(data)
+      # Validator snapshots contain immutable strings, which can be shared even
+      # when a long base name applies to thousands of short records.
+      @base = base.transform_values { |value| value.frozen? ? value : SenML.copy(value).freeze }.freeze
+      @limits = SenML.limits(options)
+    end
+
+    def [](key) = @data[key.to_s]
+    def to_h = SenML.copy(@data)
+    def name = @base.fetch("bn", "") + @data.fetch("n", "")
+    def unit = @data.fetch("u") { @base["bu"] }&.dup
+    def time = @base.fetch("bt", 0) + @data.fetch("t", 0)
+
+    def value
+      key = VALUE_LABELS.find { |label| @data.key?(label) }
+      return data if key == "vd"
+      return @data[key] if key && key != "v"
+      return nil unless key || @base.key?("bv")
+
+      @base.fetch("bv", 0) + @data.fetch("v", 0)
+    end
+
+    def sum
+      return unless @data.key?("s") || @base.key?("bs")
+
+      @base.fetch("bs", 0) + @data.fetch("s", 0)
+    end
+
+    def time_at(now: nil)
+      seconds = time
+      if seconds < ABSOLUTE_TIME
+        raise ConversionError, "relative time requires explicit now" unless now
+
+        seconds += now.is_a?(Time) ? now.to_r : now
+      end
+      Time.at(seconds).utc
+    end
+
+    # Decode the data value only when requested, after checking its size.
+    # @return [String, nil] binary octets, or nil if vd is absent
+    def data
+      encoded = @data["vd"]
+      return unless encoded
+
+      Validator.new(strict: true, limits: @limits).validate_data(encoded, "/vd")
+      padded = encoded.tr("-_", "+/") + ("=" * ((4 - (encoded.length % 4)) % 4))
+      padded.unpack1("m0")
+    rescue ArgumentError => e
+      raise ConversionError, e.message
+    end
+
+    def resolved(now:)
+      Validator.new(strict: true, limits: @limits).accept(@base.merge(@data), 0)
+      unknown_base = (@base.keys | @data.keys).find { |key| key.start_with?("b") && !BASE_LABELS.include?(key) }
+      raise ConversionError, "cannot resolve unknown base field #{unknown_base}" if unknown_base
+
+      result = @data.except(*BASE_LABELS)
+      result["n"] = name
+      result["u"] = unit if unit
+      seconds = time
+      seconds += now if seconds < ABSOLUTE_TIME
+      raise ConversionError, "resolved time must be absolute and finite" unless seconds.finite? && seconds >= ABSOLUTE_TIME
+
+      result["t"] = seconds
+      result["v"] = @base.fetch("bv", 0) + @data.fetch("v", 0) if @data.key?("v") || (VALUE_LABELS.none? { |key| @data.key?(key) } && @base.key?("bv"))
+      result["s"] = sum if sum
+      result["ct"] ||= @base["bct"] if @data.key?("vd") && @base.key?("bct")
+      version = @base.fetch("bver", FORMAT_VERSION)
+      result["bver"] = version if version != FORMAT_VERSION
+      result
+    end
+  end
+end

data/lib/senml/stream_reader.rb

@@ -0,0 +1,75 @@
+# frozen_string_literal: true
+
+module SenML
+  class StreamReader
+    include Enumerable
+
+    attr_reader :warnings
+
+    def initialize(io, strict: true, allow_incomplete: false, **options)
+      @io = io
+      @strict = strict
+      @allow_incomplete = allow_incomplete
+      @limits = SenML.limits(options)
+      @warnings = []
+    end
+
+    # Yield records immediately; a later malformed record can still raise.
+    # @yieldparam record [Record] measurement with the current base context
+    # @return [StreamReader, Enumerator]
+    def each
+      return enum_for(:each) unless block_given?
+
+      validator = Validator.new(strict: @strict, limits: @limits)
+      @warnings = validator.warnings
+      phase = :start
+      buffer = +"".b
+      depth = 0
+      quoted = false
+      escaped = false
+      index = 0
+      while (chunk = @io.read(4096)) && !chunk.empty?
+        chunk.each_byte do |byte|
+          if depth.positive?
+            buffer << byte
+            raise LimitError, "stream record exceeds max_record_bytes" if buffer.bytesize > @limits[:max_record_bytes]
+
+            if quoted
+              if escaped then escaped = false
+              elsif byte == 92 then escaped = true
+              elsif byte == 34 then quoted = false
+              end
+            elsif byte == 34 then quoted = true
+            elsif [123, 91].include?(byte) then depth += 1
+            elsif [125, 93].include?(byte) then depth -= 1
+            end
+            raise LimitError, "stream record exceeds max_depth" if depth > @limits[:max_depth]
+            next unless depth.zero?
+
+            raw = JSONCodec.parse(buffer, max_depth: @limits[:max_depth])
+            base = validator.accept(raw, index)
+            yield Record.new(raw, base: base, **@limits)
+            index += 1
+            phase = :separator
+          elsif [9, 10, 13, 32].include?(byte)
+            next
+          elsif (phase == :start && byte == 91) || (phase == :separator && byte == 44)
+            phase = :record
+          elsif phase == :record && byte == 123
+            raise LimitError, "stream exceeds max_records" if index >= @limits[:max_records]
+
+            buffer = +"{".b
+            depth = 1
+          elsif phase == :separator && byte == 93
+            phase = :end
+          else
+            raise ParseError, "invalid SenSML array structure"
+          end
+        end
+      end
+      raise ParseError, "incomplete SenSML array" unless phase == :end || (@allow_incomplete && index.positive? && depth.zero?)
+
+      self
+    end
+  end
+end

data/lib/senml/validator.rb

@@ -0,0 +1,118 @@
+# frozen_string_literal: true
+
+module SenML
+  class Validator
+    attr_reader :base, :warnings, :version, :record_bytes
+
+    def initialize(strict:, limits:)
+      @strict = strict
+      @limits = limits
+      @base = {}
+      @warnings = []
+      @version = nil
+    end
+
+    def accept(record, index)
+      pointer = "/#{index}"
+      raise ParseError, "#{pointer}: record must be an object" unless record.is_a?(Hash)
+
+      json_value(record, pointer, 0)
+      @record_bytes = JSONCodec.generate(record).bytesize
+      raise LimitError, "record exceeds max_record_bytes" if @record_bytes > @limits[:max_record_bytes]
+
+      record.each do |label, value|
+        raise ParseError, "record labels must be strings" unless label.is_a?(String)
+
+        if label.end_with?("_") && !LABELS.key?(label)
+          escaped = label.gsub("~", "~0").gsub("/", "~1")
+          raise ValidationError.new("unknown critical field", pointer: "#{pointer}/#{escaped}")
+        end
+
+        spec = LABELS[label]
+        next unless spec
+
+        valid = case spec["type"]
+                when "string" then value.is_a?(String)
+                when "number" then (value.is_a?(Integer) || value.is_a?(Float)) && value.finite?
+                when "integer" then value.is_a?(Integer) && value.positive?
+                when "boolean" then [true, false].include?(value)
+                end
+        raise ParseError, "#{pointer}/#{label}: invalid #{spec["type"]} value" unless valid
+      end
+      BASE_LABELS.each do |key|
+        next unless record.key?(key)
+
+        value = record[key]
+        @base[key] = value.is_a?(String) ? value.dup.freeze : value
+      end
+      # Only the first unknown base name is needed to prevent unsafe resolution;
+      # do not retain arbitrary unknown base payloads across the whole stream.
+      @unknown_base ||= record.keys.find { |key| key.start_with?("b") && !BASE_LABELS.include?(key) }
+      @base[@unknown_base] = nil if @unknown_base
+      @base_name_valid = @base.fetch("bn", "").match?(%r{\A[A-Za-z0-9][A-Za-z0-9:._/-]*\z}) if record.key?("bn")
+      current = @base.fetch("bver", FORMAT_VERSION)
+      raise UnsupportedVersionError.new("unsupported version #{current}", pointer: "#{pointer}/bver") if current > FORMAT_VERSION
+      raise ValidationError.new("all records must use the same version", pointer: "#{pointer}/bver") if @version && @version != current
+
+      @version = current
+      fields = VALUE_LABELS.count { |key| record.key?(key) }
+      effective_value = fields == 1 || (fields.zero? && @base.key?("bv"))
+      effective_sum = record.key?("s") || @base.key?("bs")
+      violation("expected exactly one value, or a sum", pointer) if fields > 1 || (!effective_value && !effective_sum)
+      suffix = record.fetch("n", "")
+      valid_name = if @base.fetch("bn", "").empty?
+                     suffix.match?(%r{\A[A-Za-z0-9][A-Za-z0-9:._/-]*\z})
+                   else
+                     @base_name_valid && suffix.match?(%r{\A[A-Za-z0-9:._/-]*\z})
+                   end
+      violation("invalid resolved name", "#{pointer}/n") unless valid_name
+      %w[bu u].each do |key|
+        violation("unregistered unit", "#{pointer}/#{key}") if record.key?(key) && !Units.registered?(record[key])
+      end
+      violation("update time must be nonnegative", "#{pointer}/ut") if record.key?("ut") && record["ut"].negative?
+      validate_data(record["vd"], "#{pointer}/vd") if record.key?("vd")
+      @base.dup.freeze
+    end
+
+    def validate_data(text, pointer)
+      valid = text.match?(/\A[A-Za-z0-9_-]*\z/) && text.length % 4 != 1
+      if valid && !text.empty? && text.length % 4 != 0
+        value = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_".index(text[-1])
+        valid = value.nobits?(text.length % 4 == 2 ? 15 : 3)
+      end
+      violation("expected canonical unpadded base64url", pointer) unless valid
+      raise LimitError, "binary value exceeds max_data_bytes" if text.bytesize * 3 / 4 > @limits[:max_data_bytes]
+    end
+
+    def json_value(value, pointer, depth)
+      raise LimitError, "value exceeds max_depth" if depth > @limits[:max_depth]
+
+      case value
+      when Hash
+        value.each do |key, child|
+          valid = key.is_a?(String) && key.valid_encoding? && (key.encoding == Encoding::UTF_8 || key.ascii_only?)
+          raise ParseError, "JSON keys must be valid UTF-8 text" unless valid
+
+          json_value(child, "#{pointer}/#{key.gsub("~", "~0").gsub("/", "~1")}", depth + 1)
+        end
+      when Array then value.each_with_index { |child, i| json_value(child, "#{pointer}/#{i}", depth + 1) }
+      when Float then raise ParseError, "nonfinite number" unless value.finite?
+      when String then raise ParseError, "invalid UTF-8 string" unless value.encoding == Encoding::UTF_8 ? value.valid_encoding? : value.ascii_only?
+      when Integer, TrueClass, FalseClass, NilClass
+        nil
+      else raise ParseError, "unsupported JSON value"
+      end
+    end
+
+    private
+
+    def violation(message, pointer)
+      error = ValidationError.new(message, pointer: pointer)
+      raise error if @strict
+
+      raise LimitError, "diagnostics exceed max_warnings" if @warnings.length >= @limits[:max_warnings]
+
+      @warnings << error
+    end
+  end
+end

data/lib/senml/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Senml
+  VERSION = "0.1.0"
+end

data/lib/senml.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+require "json"
+require "yaml"
+require "time"
+require_relative "senml/version"
+
+module SenML
+  VERSION = Senml::VERSION
+  FORMAT_VERSION = 10
+  ABSOLUTE_TIME = 2**28
+  LIMITS = { max_bytes: 16_777_216, max_records: 100_000, max_record_bytes: 1_048_576, max_data_bytes: 786_432, max_depth: 32, max_warnings: 1_000 }.freeze
+  class Error < StandardError; end
+  class ParseError < Error; end
+  class ConversionError < Error; end
+  class LimitError < ParseError; end
+
+  class ValidationError < Error
+    attr_reader :pointer
+
+    def initialize(message, pointer: "")
+      @pointer = pointer
+      super("#{pointer.empty? ? "/" : pointer}: #{message}")
+    end
+  end
+
+  class UnsupportedVersionError < ValidationError; end
+
+  def self.limits(options)
+    raise ArgumentError, "unknown limit option" unless (options.keys - LIMITS.keys).empty?
+
+    values = LIMITS.merge(options)
+    raise ArgumentError, "limits must be positive integers" unless values.values.all? { |v| v.is_a?(Integer) && v.positive? }
+    raise ArgumentError, "max_depth must be at most 128" if values[:max_depth] > 128
+
+    values
+  end
+
+  def self.copy(value, depth = 0)
+    raise LimitError, "data exceeds safe copy depth" if depth > 128
+
+    case value
+    when Hash then value.to_h { |k, v| [k.dup, copy(v, depth + 1)] }
+    when Array then value.map { |v| copy(v, depth + 1) }
+    when String then value.dup
+    else value
+    end
+  end
+
+  # Parse a complete SenML JSON pack without executing or fetching its contents.
+  # @param source [String] UTF-8 JSON array
+  # @param strict [Boolean] raise on recoverable semantic violations
+  # @return [Pack]
+  # @raise [ParseError, ValidationError, LimitError]
+  def self.parse(source, strict: true, **options)
+    options = limits(options)
+    raise ParseError, "JSON input must be a String" unless source.is_a?(String)
+    raise LimitError, "pack exceeds max_bytes" if source.bytesize > options[:max_bytes]
+
+    records = JSONCodec.parse(source, max_depth: options[:max_depth])
+    Pack.new(records, strict: strict, **options)
+  end
+end
+require_relative "senml/labels"
+require_relative "senml/json_codec"
+require_relative "senml/validator"
+require_relative "senml/record"
+require_relative "senml/pack"
+require_relative "senml/stream_reader"

data/sig/senml.rbs

@@ -0,0 +1,86 @@
+module Senml
+  VERSION: String
+end
+module SenML
+  VERSION: String
+  FORMAT_VERSION: Integer
+  ABSOLUTE_TIME: Integer
+  type json = nil | bool | Integer | Float | String | Array[json] | Hash[String, json]
+  type object = Hash[String, json]
+  type number = Integer | Float
+  class Error < StandardError
+  end
+  class ParseError < Error
+  end
+  class ConversionError < Error
+  end
+  class LimitError < ParseError
+  end
+  class ValidationError < Error
+    attr_reader pointer: String
+    def initialize: (String, ?pointer: String) -> void
+  end
+  class UnsupportedVersionError < ValidationError
+  end
+  def self.parse: (String, ?strict: bool, **Integer) -> Pack
+  def self.parse_cbor: (String, ?strict: bool, ?streaming: bool, **Integer) -> Pack
+  module Units
+    SYMBOLS: Array[String]
+    def self.registered?: (String) -> bool
+  end
+  class Record
+    def initialize: (object, ?base: object, **Integer) -> void
+    def []: (String | Symbol) -> json
+    def to_h: () -> object
+    def name: () -> String
+    def unit: () -> String?
+    def time: () -> number
+    def value: () -> (number | String | bool)?
+    def sum: () -> number?
+    def time_at: (?now: Time | number | nil) -> Time
+    def data: () -> String?
+    def resolved: (now: number) -> object
+    def base_name: () -> String?
+    def base_time: () -> number?
+    def base_unit: () -> String?
+    def base_value: () -> number?
+    def base_sum: () -> number?
+    def base_version: () -> Integer?
+    def numeric_value: () -> number?
+    def string_value: () -> String?
+    def boolean_value: () -> bool?
+    def data_value: () -> String?
+    def update_time: () -> number?
+    def object_link: () -> String?
+    def base_content_format: () -> String?
+    def content_format: () -> String?
+  end
+  class Pack
+    include Enumerable[Record]
+    attr_reader warnings: Array[ValidationError]
+    def initialize: (Array[object | Record], ?strict: bool, **Integer) -> void
+    def each: () { (Record) -> void } -> Array[Record]
+            | () -> Enumerator[Record, Array[Record]]
+    def []: (Integer) -> Record?
+    def size: () -> Integer
+    alias length size
+    def to_h: () -> Array[object]
+    def to_json: (*untyped) -> String
+    def to_cbor: () -> String
+    def resolve: (?now: Time | number) -> Pack
+    def select_fragment: (String) -> Pack
+    def self.build: (?base_name: String?, ?base_time: number?, ?base_unit: String?, **untyped) { (Builder) -> void } -> Pack
+    class Builder
+      attr_reader records: Array[object]
+      def initialize: (object) -> void
+      def add: (**json) -> Builder
+    end
+  end
+  class StreamReader
+    include Enumerable[Record]
+    attr_reader warnings: Array[ValidationError]
+    def initialize: (untyped, ?strict: bool, ?allow_incomplete: bool, **Integer) -> void
+    def each: () { (Record) -> void } -> StreamReader
+            | () -> Enumerator[Record, StreamReader]
+  end
+end

metadata

@@ -0,0 +1,81 @@
+--- !ruby/object:Gem::Specification
+name: senml
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Yudai Takada
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: json
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.19.4
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '4'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.19.4
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '4'
+description: RFC 8428 SenML JSON, base-value resolution, incremental SenSML streams,
+  fragments, and optional CBOR.
+email:
+- t.yudai92@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- SECURITY.md
+- data/labels.yml
+- data/units.yml
+- lib/senml.rb
+- lib/senml/cbor.rb
+- lib/senml/json_codec.rb
+- lib/senml/labels.rb
+- lib/senml/pack.rb
+- lib/senml/record.rb
+- lib/senml/stream_reader.rb
+- lib/senml/validator.rb
+- lib/senml/version.rb
+- sig/senml.rbs
+homepage: https://github.com/ydah/senml
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+  source_code_uri: https://github.com/ydah/senml
+  changelog_uri: https://github.com/ydah/senml/blob/main/CHANGELOG.md
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.3.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.16
+specification_version: 4
+summary: Read, validate, and resolve SenML sensor measurements
+test_files: []