senml 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +9 -13
- data/SECURITY.md +1 -1
- data/lib/senml/binary_cbor.rb +285 -0
- data/lib/senml/cbor.rb +17 -90
- data/lib/senml/version.rb +1 -1
- metadata +4 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: be83f76fe8b28e2cf2f65dc7962438cb1d6b47404d4113e2515269340b721979
|
|
4
|
+
data.tar.gz: c6c5adbff865d71650f66d48006a2c6c260ee5ec5a787b55849d0f3d57a508eb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d7102b0ef6fe437b595f777edd5c79639df2c273d679f9c0951eda498a93c1bb2cc85275ab4098375213bbcb9f94345e2ecf82fb3f92749d4cf2f067a731e4fa
|
|
7
|
+
data.tar.gz: 64c42ba085c82b2b969d1cfd425c539a40e0cdcbb8dc6b8b9b39b2d1cde420915af4a1b32b56d14685e3178e7753b3ff1d3824b90c89c8adbc275b9b482081e0
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
|
@@ -1,16 +1,14 @@
|
|
|
1
1
|
# SenML
|
|
2
2
|
|
|
3
|
-
Read, validate, write, and resolve [RFC 8428](https://www.rfc-editor.org/rfc/rfc8428) sensor measurements in Ruby. Supports SenML JSON, incremental SenSML JSON, record fragments, and
|
|
3
|
+
Read, validate, write, and resolve [RFC 8428](https://www.rfc-editor.org/rfc/rfc8428) sensor measurements in Ruby. Supports SenML JSON, incremental SenSML JSON, record fragments, and CBOR.
|
|
4
4
|
|
|
5
5
|
## Installation
|
|
6
6
|
|
|
7
7
|
```ruby
|
|
8
8
|
gem "senml"
|
|
9
|
-
# Optional on CRuby 3.3–4.0 and TruffleRuby:
|
|
10
|
-
gem "cbor", "~> 0.5"
|
|
11
9
|
```
|
|
12
10
|
|
|
13
|
-
Requires Ruby 3.3 or later and JSON 2.19.4 or later, below version 4. JSON
|
|
11
|
+
Requires Ruby 3.3 or later and JSON 2.19.4 or later, below version 4. JSON and the built-in Ruby CBOR codec work on CRuby, JRuby, and TruffleRuby, including CRuby development builds. There are no dependencies on other measurement or JSON-format gems.
|
|
14
12
|
|
|
15
13
|
## Read and resolve
|
|
16
14
|
|
|
@@ -66,7 +64,7 @@ All APIs that construct packs or readers accept these positive integer limits:
|
|
|
66
64
|
|
|
67
65
|
Native array construction and CBOR decoding also check the cumulative JSON representation size against `max_bytes`; the decoded representation may be larger than its CBOR input. Streams use record limits instead of a total byte limit. Exceeding `max_warnings` raises `LimitError` before yielding the next record; `reader.warnings` retains the diagnostics collected so far. Resolution checks `max_bytes` incrementally as base fields expand, before retaining an oversized resolved pack. Binary size is checked before decoding. `SenML::ParseError` reports malformed input or field types, `ValidationError` reports semantic problems with a `pointer`, `UnsupportedVersionError` identifies unknown versions, and `LimitError` reports exceeded bounds. A conversion that cannot preserve supported semantics raises `ConversionError`.
|
|
68
66
|
|
|
69
|
-
##
|
|
67
|
+
## CBOR
|
|
70
68
|
|
|
71
69
|
```ruby
|
|
72
70
|
require "senml/cbor"
|
|
@@ -76,7 +74,7 @@ SenML.parse_cbor(bytes)
|
|
|
76
74
|
SenML.parse_cbor(stream_bytes, streaming: true)
|
|
77
75
|
```
|
|
78
76
|
|
|
79
|
-
CBOR maps the RFC's integer labels and binary data values, preserves JSON-compatible extension fields, and supports decimal-fraction tag 4 with exponents from -400 to 400.
|
|
77
|
+
CBOR maps the RFC's integer labels and binary data values, preserves JSON-compatible extension fields, and supports decimal-fraction tag 4 with exponents from -400 to 400. The decoder checks lengths, nesting, UTF-8, duplicate map keys, and allowed types while reading. No separate CBOR gem is required. Unknown integer labels, non-JSON binary extension values, other tags, indefinite strings, and lossy conversion of large integers fail explicitly. JSON nonintegral values use Float; conversion of arbitrary decimal fractions can round to binary64.
|
|
80
78
|
|
|
81
79
|
## Scope and reference data
|
|
82
80
|
|
|
@@ -84,14 +82,14 @@ CBOR maps the RFC's integer labels and binary data values, preserves JSON-compat
|
|
|
84
82
|
| --- | --- |
|
|
85
83
|
| SenML and incremental SenSML JSON | XML and EXI |
|
|
86
84
|
| Base resolution and record fragments | Unit conversion and CoAP transport |
|
|
87
|
-
|
|
|
85
|
+
| CBOR on the runtimes listed above | Incremental CBOR decoding |
|
|
88
86
|
| RFC 8428 version rules and registered labels | FETCH/PATCH and RFC 9100 feature versions |
|
|
89
87
|
|
|
90
|
-
The implementation supports RFC 8428 sections 4–6 and 9: SenML/SenSML JSON, base resolution,
|
|
88
|
+
The implementation supports RFC 8428 sections 4–6 and 9: SenML/SenSML JSON, base resolution, CBOR, and record fragments. It recognizes the IANA unit identifiers and registered labels checked on 2026-10-11, including `vlo` and the `ct`/`bct` labels from [RFC 9193](https://www.rfc-editor.org/rfc/rfc9193). Content-format strings are preserved opaquely, and `bct` resolves onto binary measurements. Unknown base labels stop resolution and fragment selection because their inheritance rules are unknown, including when resolving a later record that inherits the unsupported base.
|
|
91
89
|
|
|
92
90
|
XML, EXI, unit conversion, CoAP transport, FETCH/PATCH variants, and feature-version negotiation from RFC 9100 are outside this release. The library makes no network requests when parsing or resolving measurements.
|
|
93
91
|
|
|
94
|
-
|
|
92
|
+
The identifier/type metadata in `data/` records the supported protocol fields. RFC text, examples, grammars, IANA source files, and external implementation code are never committed or packaged. Development scripts fetch checksum-pinned references into ignored `tmp/` directories, retain notices, and extract test fixtures there. See [IANA terms](https://www.iana.org/help/licensing-terms), [IETF terms](https://trustee.ietf.org/license-info), and `script/sources.json` for sources and checksums.
|
|
95
93
|
|
|
96
94
|
The RFC's explicitly unfinished section 5.1.2 stream is closed after its final complete measurement for whole-pack tests. The first section 5.1.7 thermostat example starts with a base-only record, conflicting with section 4.2's value-or-sum requirement; strict parsing rejects the original, and its base name is moved into the first measurement for the roundtrip fixture. Original excerpts and corrections remain together in ignored fixture storage. [Erratum 6728](https://www.rfc-editor.org/errata/eid6728), held for document update, concerns an XML/CDDL comma and does not change the supported encodings.
|
|
97
95
|
|
|
@@ -101,14 +99,12 @@ The RFC's explicitly unfinished section 5.1.2 stream is closed after its final c
|
|
|
101
99
|
bundle install
|
|
102
100
|
bundle exec rake test
|
|
103
101
|
COVERAGE=1 bundle exec rake test
|
|
104
|
-
bundle exec rubocop
|
|
105
102
|
bundle exec rbs -I sig validate
|
|
106
|
-
bundle exec ruby script/interop.rb # Go and unzip required
|
|
107
103
|
bundle exec ruby script/benchmark.rb
|
|
108
104
|
```
|
|
109
105
|
|
|
110
|
-
Tests cover all 14 RFC JSON examples, the full CBOR hex example, all five fragment examples, MUST violations, malformed and random CBOR, random numeric roundtrips, and a generated 100,000-record stream.
|
|
106
|
+
Tests cover all 14 RFC JSON examples, the full CBOR hex example, all five fragment examples, MUST violations, malformed and random CBOR, random numeric roundtrips, and a generated 100,000-record stream. Normalization tests use original examples with explicit expected measurements; CBOR tests run on every supported Ruby engine without external codec or language tools. The IANA checker fails on unexpected registry changes; review updated sources and checksums before running `bundle exec ruby script/update_registry.rb --write`.
|
|
111
107
|
|
|
112
108
|
## License
|
|
113
109
|
|
|
114
|
-
MIT
|
|
110
|
+
Project code is MIT licensed; see [LICENSE.txt](LICENSE.txt). Added library dependencies are limited to MIT-licensed libraries. Ruby implementations and standard/default libraries (including JSON) are exceptions and retain their own licenses. Downloaded standards material is test reference data under its source terms and is neither committed nor packaged. These statements do not relicense third-party material.
|
data/SECURITY.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Report vulnerabilities privately through [GitHub security advisories](https://github.com/ydah/senml/security/advisories/new).
|
|
4
4
|
|
|
5
|
-
Parsing does not perform network requests or execute measurement content. Inputs have configurable byte, record, depth, and binary-data limits. Streaming applies per-record limits before retaining further bytes.
|
|
5
|
+
Parsing does not perform network requests or execute measurement content. Inputs have configurable byte, record, depth, and binary-data limits. Streaming applies per-record limits before retaining further bytes. The built-in Ruby CBOR decoder checks structural limits while reading, rejects duplicate map keys, and never evaluates tags as application code.
|
|
6
6
|
|
|
7
7
|
Unknown critical labels and unsupported or mixed format versions fail even in lenient mode. Noncritical extension data is retained without interpretation. Applications must decide which sensor names, units, content formats, values, and timestamps they trust. Conversion and resolution do not authenticate measurements.
|
|
8
8
|
|
|
@@ -0,0 +1,285 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SenML
|
|
4
|
+
# Internal wire codec. RFC 8949 sections 3.1–3.4 define the byte format;
|
|
5
|
+
# application-specific tag meanings belong to the caller.
|
|
6
|
+
module BinaryCBOR
|
|
7
|
+
class Error < StandardError; end
|
|
8
|
+
class LimitError < Error; end
|
|
9
|
+
class UnsupportedError < Error; end
|
|
10
|
+
Tagged = Struct.new(:tag, :value)
|
|
11
|
+
Simple = Struct.new(:value)
|
|
12
|
+
Float32 = Struct.new(:value)
|
|
13
|
+
|
|
14
|
+
def self.decode(bytes, **options)
|
|
15
|
+
Decoder.new(bytes, **options).read
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def self.encode(value, **options)
|
|
19
|
+
Encoder.new(**options).write(value)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
class Decoder
|
|
23
|
+
DEFAULTS = { max_input_size: 16_777_216, max_depth: 32, max_nodes: 1_000_000, max_collection_size: 200_000,
|
|
24
|
+
max_string_size: 16_777_216, max_binary_size: 16_777_216, allowed_tags: nil, allow_indefinite_strings: true,
|
|
25
|
+
max_records: nil, max_record_bytes: nil }.freeze
|
|
26
|
+
|
|
27
|
+
def initialize(bytes, **options)
|
|
28
|
+
@limits = DEFAULTS.merge(options)
|
|
29
|
+
raise Error, "CBOR input must be a String" unless bytes.is_a?(String)
|
|
30
|
+
raise LimitError, "CBOR exceeds max_input_size" if bytes.bytesize > @limits[:max_input_size]
|
|
31
|
+
|
|
32
|
+
@bytes = bytes
|
|
33
|
+
@offset = 0
|
|
34
|
+
@nodes = 0
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def read
|
|
38
|
+
value = read_value(0)
|
|
39
|
+
raise Error, "trailing CBOR input" unless @offset == @bytes.bytesize
|
|
40
|
+
|
|
41
|
+
value
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
private
|
|
45
|
+
|
|
46
|
+
def take(size)
|
|
47
|
+
raise Error, "truncated CBOR input" if size > @bytes.bytesize - @offset
|
|
48
|
+
if @record_start && @limits[:max_record_bytes] && @offset + size - @record_start > @limits[:max_record_bytes]
|
|
49
|
+
raise LimitError, "CBOR record exceeds max_record_bytes"
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
result = @bytes.byteslice(@offset, size)
|
|
53
|
+
@offset += size
|
|
54
|
+
result
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def argument(info)
|
|
58
|
+
return info if info < 24
|
|
59
|
+
return nil if info == 31
|
|
60
|
+
|
|
61
|
+
format = { 24 => [1, "C"], 25 => [2, "n"], 26 => [4, "N"], 27 => [8, "Q>"] }[info]
|
|
62
|
+
raise Error, "reserved CBOR additional information" unless format
|
|
63
|
+
|
|
64
|
+
take(format[0]).unpack1(format[1])
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def read_value(depth)
|
|
68
|
+
@nodes += 1
|
|
69
|
+
raise LimitError, "CBOR exceeds max_nodes" if @nodes > @limits[:max_nodes]
|
|
70
|
+
raise LimitError, "CBOR exceeds max_depth" if depth > @limits[:max_depth]
|
|
71
|
+
|
|
72
|
+
initial = take(1).getbyte(0)
|
|
73
|
+
major, info = initial.divmod(32)
|
|
74
|
+
return read_simple(info) if major == 7
|
|
75
|
+
|
|
76
|
+
size = argument(info)
|
|
77
|
+
raise Error, "indefinite CBOR integer or tag" if size.nil? && [0, 1, 6].include?(major)
|
|
78
|
+
|
|
79
|
+
case major
|
|
80
|
+
when 0 then size
|
|
81
|
+
when 1 then -1 - size
|
|
82
|
+
when 2, 3 then read_string(major, size, depth)
|
|
83
|
+
when 4, 5 then read_collection(major, size, depth)
|
|
84
|
+
when 6
|
|
85
|
+
raise UnsupportedError, "unsupported CBOR tag" if @limits[:allowed_tags] && !@limits[:allowed_tags].include?(size)
|
|
86
|
+
|
|
87
|
+
Tagged.new(size, read_value(depth + 1))
|
|
88
|
+
end
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
def read_string(major, size, depth)
|
|
92
|
+
limit = @limits[:max_string_size]
|
|
93
|
+
limit = [limit, @limits[:max_binary_size]].min if major == 2
|
|
94
|
+
if size
|
|
95
|
+
raise LimitError, "CBOR string exceeds #{major == 2 ? "max_binary_size" : "max_string_size"}" if size > limit
|
|
96
|
+
|
|
97
|
+
value = take(size).dup
|
|
98
|
+
value.force_encoding(major == 2 ? Encoding::BINARY : Encoding::UTF_8)
|
|
99
|
+
raise Error, "invalid CBOR UTF-8 text" unless value.valid_encoding?
|
|
100
|
+
|
|
101
|
+
return value
|
|
102
|
+
end
|
|
103
|
+
raise Error, "CBOR strings must have definite length" unless @limits[:allow_indefinite_strings]
|
|
104
|
+
|
|
105
|
+
result = String.new(encoding: major == 2 ? Encoding::BINARY : Encoding::UTF_8)
|
|
106
|
+
until @bytes.getbyte(@offset) == 255
|
|
107
|
+
initial = @bytes.getbyte(@offset)
|
|
108
|
+
raise Error, "invalid indefinite CBOR string chunk" unless initial && initial.div(32) == major && (initial & 31) < 28
|
|
109
|
+
|
|
110
|
+
chunk = read_value(depth + 1)
|
|
111
|
+
raise LimitError, "CBOR string exceeds byte limit" if result.bytesize + chunk.bytesize > limit
|
|
112
|
+
|
|
113
|
+
result << chunk
|
|
114
|
+
end
|
|
115
|
+
take(1)
|
|
116
|
+
result
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
def read_collection(major, size, depth)
|
|
120
|
+
limit = @limits[:max_collection_size]
|
|
121
|
+
limit = [limit, @limits[:max_records]].min if depth.zero? && @limits[:max_records]
|
|
122
|
+
raise LimitError, "CBOR exceeds collection limit" if size && size > limit
|
|
123
|
+
|
|
124
|
+
result = major == 4 ? [] : {}
|
|
125
|
+
count = 0
|
|
126
|
+
until size ? count == size : @bytes.getbyte(@offset) == 255
|
|
127
|
+
raise LimitError, "CBOR exceeds collection limit" if count >= limit
|
|
128
|
+
|
|
129
|
+
@record_start = @offset if depth.zero? && @limits[:max_record_bytes]
|
|
130
|
+
if major == 4
|
|
131
|
+
result << read_value(depth + 1)
|
|
132
|
+
else
|
|
133
|
+
key = read_value(depth + 1)
|
|
134
|
+
unless key.is_a?(Integer) || (key.is_a?(String) && key.encoding == Encoding::UTF_8)
|
|
135
|
+
raise UnsupportedError, "CBOR map keys must be text or integers"
|
|
136
|
+
end
|
|
137
|
+
raise Error, "duplicate CBOR map key" if result.key?(key)
|
|
138
|
+
|
|
139
|
+
result[key] = read_value(depth + 1)
|
|
140
|
+
end
|
|
141
|
+
@record_start = nil if depth.zero?
|
|
142
|
+
count += 1
|
|
143
|
+
end
|
|
144
|
+
take(1) unless size
|
|
145
|
+
result
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
def read_simple(info)
|
|
149
|
+
case info
|
|
150
|
+
when 20 then false
|
|
151
|
+
when 21 then true
|
|
152
|
+
when 22 then nil
|
|
153
|
+
when 0..19, 23 then Simple.new(info)
|
|
154
|
+
when 24
|
|
155
|
+
value = take(1).getbyte(0)
|
|
156
|
+
raise Error, "reserved CBOR simple value" if value < 32
|
|
157
|
+
|
|
158
|
+
Simple.new(value)
|
|
159
|
+
when 25 then half(take(2).unpack1("n"))
|
|
160
|
+
when 26 then take(4).unpack1("g")
|
|
161
|
+
when 27 then take(8).unpack1("G")
|
|
162
|
+
else raise Error, "reserved CBOR value or unexpected break"
|
|
163
|
+
end
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
def half(bits)
|
|
167
|
+
exponent = (bits >> 10) & 31
|
|
168
|
+
fraction = bits & 1023
|
|
169
|
+
magnitude = if exponent == 31
|
|
170
|
+
fraction.zero? ? Float::INFINITY : Float::NAN
|
|
171
|
+
elsif exponent.zero?
|
|
172
|
+
Math.ldexp(fraction.to_f, -24)
|
|
173
|
+
else
|
|
174
|
+
Math.ldexp((1024 + fraction).to_f, exponent - 25)
|
|
175
|
+
end
|
|
176
|
+
bits[15].zero? ? magnitude : -magnitude
|
|
177
|
+
end
|
|
178
|
+
end
|
|
179
|
+
|
|
180
|
+
class Encoder
|
|
181
|
+
def initialize(max_output_size: 16_777_216, max_depth: 128)
|
|
182
|
+
@max_output_size = max_output_size
|
|
183
|
+
@max_depth = max_depth
|
|
184
|
+
@buffer = String.new(encoding: Encoding::BINARY)
|
|
185
|
+
end
|
|
186
|
+
|
|
187
|
+
def write(value)
|
|
188
|
+
append_value(value, 0)
|
|
189
|
+
@buffer
|
|
190
|
+
end
|
|
191
|
+
|
|
192
|
+
private
|
|
193
|
+
|
|
194
|
+
def append(bytes)
|
|
195
|
+
raise LimitError, "CBOR output exceeds max_input_size" if @buffer.bytesize + bytes.bytesize > @max_output_size
|
|
196
|
+
|
|
197
|
+
@buffer << bytes.b
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
def header(major, number)
|
|
201
|
+
raise Error, "CBOR argument outside unsigned 64-bit range" unless number.is_a?(Integer) && number.between?(0, (2**64) - 1)
|
|
202
|
+
|
|
203
|
+
case number
|
|
204
|
+
when 0...24 then append([(major << 5) | number].pack("C"))
|
|
205
|
+
when 24..255 then append([(major << 5) | 24, number].pack("CC"))
|
|
206
|
+
when 256..65_535 then append([(major << 5) | 25, number].pack("Cn"))
|
|
207
|
+
when 65_536..4_294_967_295 then append([(major << 5) | 26, number].pack("CN"))
|
|
208
|
+
else append([(major << 5) | 27, number].pack("CQ>"))
|
|
209
|
+
end
|
|
210
|
+
end
|
|
211
|
+
|
|
212
|
+
def append_value(value, depth)
|
|
213
|
+
raise LimitError, "CBOR output exceeds max_depth" if depth > @max_depth
|
|
214
|
+
|
|
215
|
+
case value
|
|
216
|
+
when Integer then append_integer(value)
|
|
217
|
+
when Float then append_float(value)
|
|
218
|
+
when Float32 then append("\xfa".b + [value.value].pack("g"))
|
|
219
|
+
when String then append_string(value)
|
|
220
|
+
when Array
|
|
221
|
+
header(4, value.length)
|
|
222
|
+
value.each { |item| append_value(item, depth + 1) }
|
|
223
|
+
when Hash
|
|
224
|
+
header(5, value.length)
|
|
225
|
+
value.each { |key, item| append_value(key, depth + 1); append_value(item, depth + 1) }
|
|
226
|
+
when Tagged
|
|
227
|
+
header(6, value.tag)
|
|
228
|
+
append_value(value.value, depth + 1)
|
|
229
|
+
when Simple
|
|
230
|
+
raise Error, "invalid CBOR simple value" unless value.value.is_a?(Integer) && (value.value.between?(0, 23) || value.value.between?(32, 255))
|
|
231
|
+
|
|
232
|
+
header(7, value.value)
|
|
233
|
+
when TrueClass then append("\xf5".b)
|
|
234
|
+
when FalseClass then append("\xf4".b)
|
|
235
|
+
when NilClass then append("\xf6".b)
|
|
236
|
+
else raise Error, "unsupported CBOR value type"
|
|
237
|
+
end
|
|
238
|
+
end
|
|
239
|
+
|
|
240
|
+
def append_integer(value)
|
|
241
|
+
magnitude = value.negative? ? -1 - value : value
|
|
242
|
+
return header(value.negative? ? 1 : 0, magnitude) if magnitude < 2**64
|
|
243
|
+
|
|
244
|
+
header(6, value.negative? ? 3 : 2)
|
|
245
|
+
hex = magnitude.to_s(16)
|
|
246
|
+
hex = "0#{hex}" if hex.length.odd?
|
|
247
|
+
append_string([hex].pack("H*"))
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
def append_float(value)
|
|
251
|
+
half = half_bits(value)
|
|
252
|
+
return append("\xf9".b + [half].pack("n")) if half
|
|
253
|
+
|
|
254
|
+
single = [value].pack("g")
|
|
255
|
+
append(single.unpack1("g") == value ? "\xfa".b + single : "\xfb".b + [value].pack("G"))
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
def half_bits(value)
|
|
259
|
+
return 0x7e00 if value.nan?
|
|
260
|
+
|
|
261
|
+
sign = value.negative? || (value.zero? && (1.0 / value).negative?) ? 0x8000 : 0
|
|
262
|
+
return sign | 0x7c00 if value.infinite?
|
|
263
|
+
return sign if value.zero?
|
|
264
|
+
|
|
265
|
+
significand, exponent = Math.frexp(value.abs)
|
|
266
|
+
if exponent < -13
|
|
267
|
+
fraction = Math.ldexp(value.abs, 24)
|
|
268
|
+
return sign | fraction.to_i if fraction == fraction.to_i
|
|
269
|
+
elsif exponent <= 16
|
|
270
|
+
fraction = significand * 2048 - 1024
|
|
271
|
+
return sign | ((exponent + 14) << 10) | fraction.to_i if fraction == fraction.to_i
|
|
272
|
+
end
|
|
273
|
+
nil
|
|
274
|
+
end
|
|
275
|
+
|
|
276
|
+
def append_string(value)
|
|
277
|
+
binary = value.encoding == Encoding::BINARY
|
|
278
|
+
raise Error, "invalid CBOR UTF-8 text" unless binary || (value.valid_encoding? && (value.encoding == Encoding::UTF_8 || value.ascii_only?))
|
|
279
|
+
|
|
280
|
+
header(binary ? 2 : 3, value.bytesize)
|
|
281
|
+
append(value)
|
|
282
|
+
end
|
|
283
|
+
end
|
|
284
|
+
end
|
|
285
|
+
end
|
data/lib/senml/cbor.rb
CHANGED
|
@@ -1,96 +1,18 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "../senml"
|
|
4
|
-
|
|
4
|
+
require_relative "binary_cbor"
|
|
5
5
|
module SenML
|
|
6
6
|
module CBORCodec
|
|
7
7
|
KEYS = LABELS.to_h { |label, spec| [spec.fetch("cbor", label), label] }.freeze
|
|
8
|
-
# Bound nesting and allocations, and reject duplicate keys before the native decoder loses them.
|
|
9
|
-
class Guard
|
|
10
|
-
def initialize(bytes, limits, streaming)
|
|
11
|
-
@bytes = bytes
|
|
12
|
-
@limits = limits
|
|
13
|
-
@streaming = streaming
|
|
14
|
-
@offset = 0
|
|
15
|
-
end
|
|
16
|
-
|
|
17
|
-
def check
|
|
18
|
-
raise ParseError, "CBOR pack must be an array" unless @bytes.getbyte(0)&.div(32) == 4
|
|
19
|
-
raise ParseError, "SenML CBOR requires a definite-length array" if @bytes.getbyte(0) == 159 && !@streaming
|
|
20
|
-
|
|
21
|
-
scan(0)
|
|
22
|
-
raise ParseError, "trailing CBOR data" unless @offset == @bytes.bytesize
|
|
23
|
-
end
|
|
24
|
-
|
|
25
|
-
private
|
|
26
|
-
|
|
27
|
-
def take(size)
|
|
28
|
-
raise ParseError, "truncated CBOR" if size > @bytes.bytesize - @offset
|
|
29
|
-
|
|
30
|
-
result = @bytes.byteslice(@offset, size)
|
|
31
|
-
@offset += size
|
|
32
|
-
result
|
|
33
|
-
end
|
|
34
|
-
|
|
35
|
-
def scan(depth)
|
|
36
|
-
raise LimitError, "CBOR exceeds max_depth" if depth > @limits[:max_depth]
|
|
37
|
-
|
|
38
|
-
initial = take(1).getbyte(0)
|
|
39
|
-
major = initial >> 5
|
|
40
|
-
info = initial & 31
|
|
41
|
-
size = case info
|
|
42
|
-
when 0..23 then info
|
|
43
|
-
when 24 then take(1).unpack1("C")
|
|
44
|
-
when 25 then take(2).unpack1("n")
|
|
45
|
-
when 26 then take(4).unpack1("N")
|
|
46
|
-
when 27 then take(8).unpack1("Q>")
|
|
47
|
-
when 31 then nil
|
|
48
|
-
else raise ParseError, "reserved CBOR argument"
|
|
49
|
-
end
|
|
50
|
-
case major
|
|
51
|
-
when 0, 1 then raise ParseError, "indefinite CBOR integer" unless size
|
|
52
|
-
when 2, 3
|
|
53
|
-
raise ParseError, "CBOR strings must have definite length" unless size
|
|
54
|
-
raise LimitError, "CBOR string exceeds record bound" if size > @limits[:max_record_bytes]
|
|
55
|
-
|
|
56
|
-
take(size)
|
|
57
|
-
when 4, 5
|
|
58
|
-
raise LimitError, "CBOR container exceeds input size" if size && size > @bytes.bytesize
|
|
59
|
-
raise LimitError, "CBOR pack exceeds max_records" if depth.zero? && size && size > @limits[:max_records]
|
|
60
|
-
|
|
61
|
-
count = 0
|
|
62
|
-
keys = {}
|
|
63
|
-
until size ? count == size : @bytes.getbyte(@offset) == 255
|
|
64
|
-
start = @offset
|
|
65
|
-
if major == 5
|
|
66
|
-
kind = @bytes.getbyte(@offset)&.div(32)
|
|
67
|
-
raise ParseError, "CBOR map keys must be integers or text" unless [0, 1, 3].include?(kind)
|
|
68
|
-
|
|
69
|
-
scan(depth + 1)
|
|
70
|
-
key = CBOR.decode(@bytes.byteslice(start, @offset - start))
|
|
71
|
-
raise ParseError, "duplicate CBOR map key" if keys.key?(key)
|
|
72
|
-
|
|
73
|
-
keys[key] = true
|
|
74
|
-
end
|
|
75
|
-
scan(depth + 1)
|
|
76
|
-
count += 1
|
|
77
|
-
raise LimitError, "CBOR record exceeds max_record_bytes" if depth.zero? && @offset - start > @limits[:max_record_bytes]
|
|
78
|
-
raise LimitError, "CBOR pack exceeds max_records" if depth.zero? && count > @limits[:max_records]
|
|
79
|
-
end
|
|
80
|
-
take(1) unless size
|
|
81
|
-
when 6
|
|
82
|
-
raise ParseError, "unsupported CBOR tag" unless size == 4
|
|
83
|
-
|
|
84
|
-
scan(depth + 1)
|
|
85
|
-
when 7
|
|
86
|
-
raise ParseError, "unsupported CBOR simple value" unless [20, 21, 22, 25, 26, 27].include?(info)
|
|
87
|
-
end
|
|
88
|
-
end
|
|
89
|
-
end
|
|
90
|
-
|
|
91
8
|
def self.decode(bytes, limits, streaming)
|
|
92
|
-
|
|
93
|
-
|
|
9
|
+
raise ParseError, "CBOR pack must be an array" unless bytes.getbyte(0)&.div(32) == 4
|
|
10
|
+
raise ParseError, "SenML CBOR requires a definite-length array" if bytes.getbyte(0) == 159 && !streaming
|
|
11
|
+
|
|
12
|
+
raw = BinaryCBOR.decode(bytes, max_input_size: limits[:max_bytes], max_depth: limits[:max_depth], max_nodes: limits[:max_bytes],
|
|
13
|
+
max_collection_size: limits[:max_bytes], max_string_size: limits[:max_record_bytes],
|
|
14
|
+
max_binary_size: limits[:max_record_bytes], max_records: limits[:max_records],
|
|
15
|
+
max_record_bytes: limits[:max_record_bytes], allowed_tags: [4], allow_indefinite_strings: false)
|
|
94
16
|
raw.map do |record|
|
|
95
17
|
raise ParseError, "CBOR record must be a map" unless record.is_a?(Hash)
|
|
96
18
|
|
|
@@ -114,19 +36,22 @@ module SenML
|
|
|
114
36
|
|
|
115
37
|
mapped
|
|
116
38
|
end
|
|
117
|
-
rescue
|
|
39
|
+
rescue BinaryCBOR::LimitError => e
|
|
40
|
+
raise LimitError, e.message
|
|
41
|
+
rescue BinaryCBOR::Error => e
|
|
118
42
|
raise ParseError, e.message
|
|
119
43
|
end
|
|
120
44
|
|
|
121
45
|
def self.json_value(value)
|
|
122
46
|
case value
|
|
123
|
-
when
|
|
47
|
+
when BinaryCBOR::Tagged
|
|
124
48
|
raise ParseError, "invalid decimal fraction" unless value.tag == 4 && value.value.is_a?(Array) && value.value.length == 2 && value.value.all?(Integer)
|
|
125
49
|
|
|
126
50
|
exponent, mantissa = value.value
|
|
127
51
|
raise ParseError, "decimal fraction exponent outside supported range" unless exponent.between?(-400, 400)
|
|
128
52
|
|
|
129
53
|
(mantissa * (10.to_r**exponent)).to_f
|
|
54
|
+
when BinaryCBOR::Simple then raise ParseError, "unsupported CBOR simple value"
|
|
130
55
|
when Hash then value.transform_values { |v| json_value(v) }
|
|
131
56
|
when Array then value.map { |v| json_value(v) }
|
|
132
57
|
when String
|
|
@@ -144,8 +69,10 @@ module SenML
|
|
|
144
69
|
[LABELS.fetch(label, {}).fetch("cbor", label), value]
|
|
145
70
|
end
|
|
146
71
|
end
|
|
147
|
-
|
|
148
|
-
rescue
|
|
72
|
+
BinaryCBOR.encode(values, max_output_size: limits[:max_bytes])
|
|
73
|
+
rescue BinaryCBOR::LimitError => e
|
|
74
|
+
raise LimitError, e.message
|
|
75
|
+
rescue BinaryCBOR::Error, RangeError => e
|
|
149
76
|
raise ConversionError, e.message
|
|
150
77
|
end
|
|
151
78
|
|
data/lib/senml/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: senml
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Yudai Takada
|
|
@@ -30,7 +30,7 @@ dependencies:
|
|
|
30
30
|
- !ruby/object:Gem::Version
|
|
31
31
|
version: '4'
|
|
32
32
|
description: RFC 8428 SenML JSON, base-value resolution, incremental SenSML streams,
|
|
33
|
-
fragments, and
|
|
33
|
+
fragments, and CBOR.
|
|
34
34
|
email:
|
|
35
35
|
- t.yudai92@gmail.com
|
|
36
36
|
executables: []
|
|
@@ -44,6 +44,7 @@ files:
|
|
|
44
44
|
- data/labels.yml
|
|
45
45
|
- data/units.yml
|
|
46
46
|
- lib/senml.rb
|
|
47
|
+
- lib/senml/binary_cbor.rb
|
|
47
48
|
- lib/senml/cbor.rb
|
|
48
49
|
- lib/senml/json_codec.rb
|
|
49
50
|
- lib/senml/labels.rb
|
|
@@ -75,7 +76,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
75
76
|
- !ruby/object:Gem::Version
|
|
76
77
|
version: '0'
|
|
77
78
|
requirements: []
|
|
78
|
-
rubygems_version:
|
|
79
|
+
rubygems_version: 3.6.9
|
|
79
80
|
specification_version: 4
|
|
80
81
|
summary: Read, validate, and resolve SenML sensor measurements
|
|
81
82
|
test_files: []
|