c0data 0.3.1 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3cf0ff8a402528534f2cd0a46c25d811e0dad7a1875bd0c24993b88c416c0cac
4
- data.tar.gz: a46da5bc31159944a5ebde356e9e0ba24550b1d92ba8bc4a70ad13a083bd1fec
3
+ metadata.gz: 595437d790d9deec04a877718be10fc625ac1c6a1674580072dd99d8ed4376c3
4
+ data.tar.gz: ef2352f78e7f86403acab6b521bb10663245d9ec29bf770c39d498c8faaf3192
5
5
  SHA512:
6
- metadata.gz: 58f26bab23ed8c97f913fe2a0d7fff18d5b2b7ec25dcc9619ce2f946ad459923da2df66c9f5be949f1dea578f3b85f694b3e600c89cc63632e00288a518d645f
7
- data.tar.gz: 7fa188710231af60512f498dda885f69c804ffac04b2e08c6daff868b8cc4a1eb85523157d01f919595a1f5a61f583fcc0455d8dd4a9a649cb3a3e6a3857fad4
6
+ metadata.gz: 16eeeef117dda0f64d51a543d98c4efe2c95ddb46c1a526d7dc04887d8b80cbcd22429a6fb16b8c9b04e41dd00da556a62fd260339e917bf03daad492e905914
7
+ data.tar.gz: 7e70a53a0e71461aa105092d22a6e28893014db178ae1bdc29b40c5552cc5ddf656a2a8a0e628df512ca2b2321968e78de4581abfd6cd98fe0f36e8da27fcebb
data/README.md CHANGED
@@ -14,9 +14,9 @@ shared byteslices into the input buffer.
14
14
  ```ruby
15
15
  require "c0"
16
16
 
17
- # Write: ␖␁shape␟tabular␝users␁␞name␟amount␞Alice␟1502.30␞Bob␟340.00␄
17
+ # Write: ␖␁version␟1␝users␁␞name␟amount␞Alice␟1502.30␞Bob␟340.00␄
18
18
  buf = C0.build do |b|
19
- b.syn(shape: "tabular")
19
+ b.syn(version: "1")
20
20
  b.group("users", %w[name amount]) do # ␝users then the header row ␁␞name␟amount
21
21
  b.record("Alice", "1502.30")
22
22
  b.record("Bob", "340.00")
@@ -26,7 +26,7 @@ end
26
26
 
27
27
  # Read (zero-copy: values are shared byteslices of buf)
28
28
  doc = C0::Document.new(buf)
29
- doc.shape # => "tabular"
29
+ doc.heading.version # => 1
30
30
  doc["users"].headers # => ["name", "amount"]
31
31
  doc["users"].each do |rec|
32
32
  name, amount = rec.values # logical values: text decoded, layout trimmed
@@ -36,16 +36,36 @@ end
36
36
  # Canonical form is hashable for content addressing
37
37
  C0.canonical?(doc.body.raw) # => true
38
38
 
39
- # Streams, containers, pretty form
39
+ # Streams, containers, pretty form, diffs
40
40
  C0::Stream::Reader.new(File.binread("events.c0")) # #torn?, #committed, #blocks, #records
41
41
  C0::Container.new(buf) # several documents, EM tail, #blob(name)
42
42
  C0::Pretty.format(buf, mode: :aligned) # Unicode Control Pictures
43
+ C0::Diff.apply(diff, "greeting.txt" => source) # C0DIFF, validated before anything is written
43
44
  ```
44
45
 
45
46
  Values are byte-transparent and written in their one spelling: plain when
46
47
  they can be, text (`␂…␃`) when they hold a control byte, a line break or edge
47
- whitespace. `C0.validate(buf)` checks a buffer against the grammar once
48
- (raising a `C0::SyntaxError`); the navigators then assume well-formed input.
48
+ whitespace.
49
+
50
+ **Parse is validate.** Constructing a `Document`, `Container`, `Body` or
51
+ `Stream::Reader` applies the whole grammar to the bytes and raises a
52
+ `C0::SyntaxError` (with `#position`) on malformed input; the navigators then
53
+ assume well-formed input. `C0.validate(buf)` is the same check on a whole
54
+ buffer. `C0.canonical?(buf)` checks the canonical constraints and is opt-in:
55
+ a non-canonical document still reads.
56
+
57
+ The model is the **result**: a dead element (`␇␞…`) is parsed and dropped
58
+ with everything it holds (`␇␝g` takes the group's header row, records and
59
+ subsections; `␇␜f` the file's groups). SUB makes the element after it a
60
+ child of the element before, at that code's grain: `␚␝s` is a subsection at
61
+ depth two (`g.depth`), `␞a␟b␚␟c` is two units with `c` a sub-unit of `b`,
62
+ not a unit of the record (`field_count`, `values` and `pairs` see units
63
+ only); `␚␞`, `␚␜` and `␚␁` are reserved and refused. `rec.unit(n)` and
64
+ `rec.unit(key)` give a unit with its `subs`, each a `C0::Unit` with `subs`
65
+ of its own; `rec.units` lists every positional unit so.
66
+ Loud warnings (`doc.warnings`: no SYN, no EOT, an unknown or duplicate
67
+ heading key, a shape other than `diff` or `stream`, a version other than 1)
68
+ never fail a parse.
49
69
 
50
70
  ### Keyed records, nesting, references
51
71
 
@@ -60,7 +80,10 @@ buf = C0.build do |b|
60
80
  b.record("1 Main", "Springfield")
61
81
  end
62
82
  b.ref("tags") # ␅tags
63
- b.ref("tags", "001", "label") # ␅␂tags␟001␟label␃
83
+ b.ref("tags", "001", "label") # ␅␐␝tags␞001␟label␘
84
+ b.ref { b.gs("tags").rs.soh("host").us } # ␅␐␝tags␞␁host␟␘, any pattern
85
+ b.field("10").sub_unit("20") # 10␚␟20: 20 a sub-unit of 10
86
+ b.group("notes", nil, 2) # ␚␝notes: a subsection of server
64
87
  end
65
88
 
66
89
  g = C0::Document.new(buf)["server"]
@@ -68,9 +91,20 @@ g.record(0).keyed? # => true
68
91
  g.record(0)["port"] # => "8080" (last wins)
69
92
  g.record(1).list(1) # => ["Admin", "Editor"]
70
93
  g.record(1).level(2).record(0) # => the nested record
94
+ g.record(1).ref(3).name # => "tags" (a plain group reference)
95
+ g.record(1).ref(4).pattern # => a Body: the level ␝tags␞001␟label parsed
71
96
  g.record(1).ref(4).path # => ["tags", "001", "label"]
97
+ g.record(1).value(6) # => "10" (the unit)
98
+ g.record(1).unit(6).subs.map(&:value) # => ["20"] (its sub-unit)
99
+ g.record(1).field_count # => 7 (a sub-unit is not a unit)
72
100
  ```
73
101
 
102
+ A reference (`rec.ref(i)`) is a group `name` (`␅tags`) or a `pattern`: the
103
+ level `␅␐␝tags␞001␟label␘`, one branch of the target with its values left
104
+ out, parsed as a body. `ref.path` linearises either form; `ref.group?` says
105
+ whether it names a group as a whole. The text form `␅␂…␃` is an error.
106
+ Resolution is not implemented.
107
+
74
108
  The builder also has `file`, `section`, `header`, `field`, `key` (with a
75
109
  block for a nested value), `heading_level`, `note`, `block`, `item`,
76
110
  `etb(payload)` and `em(tail)`, matching the Crystal reference; every method
@@ -79,12 +113,12 @@ returns the builder, so calls chain.
79
113
  ### Primitives
80
114
 
81
115
  Underneath, the builder has one method per code: `fs`, `gs`, `rs`, `us`,
82
- `sub` (C0-DIFF only), `soh`, `dle`, `can`, `bel`, `enq`, `syn`, `eot`,
83
- `etb`, `em`. Each writes its code and, when given, what follows it, so a
84
- chain reads like the bytes, and anything the grammar allows can be written.
85
- The argument is optional where the grammar allows the bare code (`gs`, `rs`,
86
- `us`, `sub`, `soh`, `bel`, `etb`); `enq` takes one name or a path of two or
87
- three:
116
+ `sub`, `soh`, `dle`, `can`, `bel`, `enq`, `syn`, `eot`, `etb`, `em`. Each
117
+ writes its code and, when given, what follows it, so a chain reads like the
118
+ bytes, and anything the grammar allows can be written. The argument is
119
+ optional where the grammar allows the bare code (`gs`, `rs`, `us`, `soh`,
120
+ `bel`, `etb`); `sub` is always bare; `enq` takes a group name, or a block
121
+ that writes a pattern:
88
122
 
89
123
  ```ruby
90
124
  b.rs("Alice").us("1502.30") # ␞Alice␟1502.30
@@ -93,17 +127,78 @@ b.gs("users").soh.rs("name").us("amount") # ␝users␁␞name␟
93
127
  b.rs("cfg").us.dle.rs.soh("b").us("2").can # ␞cfg␟␐␞␁b␟2␘: a nested level
94
128
  b.bel("checked").rs("x") # ␇checked␞x: a note
95
129
  b.bel.gs("old") # ␇␝old: a dead group
96
- b.gs.gs("sub").rs.enq("users").us.enq("users", "1") # ␝␝sub␞␅users␟␅␂users␟1␃
130
+ b.rs("old").sub.us("new") # ␞old␚␟new: new a sub-unit of old
131
+ b.gs("a").rs("1").sub.gs("b").rs("2") # ␝a␞1␚␝b␞2: b a subsection of a
132
+ b.gs("a").gs("b", depth: 2) # ␝a␚␝b: the same, depth spelt
133
+ b.rs.enq("tags").us.enq { b.gs("tags").rs("001") } # ␞␅tags␟␅␐␝tags␞001␘
97
134
  b.syn.soh("version").us("1").rs("a").eot # ␖␁version␟1␞a␄
98
- b.rs("old").sub("new") # ␞old␚new: a C0-DIFF replace-all
99
135
  ```
100
136
 
137
+ `sub` writes SUB alone: the element after the next separator is subordinate
138
+ to the one before it, at that code's grain (`␚␝` a subsection, `␚␟` a
139
+ sub-unit; `␚␞`, `␚␜` and `␚␁` are reserved). `gs(name, depth:)` and
140
+ `sub_unit(value, depth:)` spell the prefix for you. A SUB prefix is
141
+ structure: it is hashed as written, kept by the canonical form and
142
+ round-tripped by the pretty form. A repeated GS is an empty-named group
143
+ (`␝␝b` is group `""` then `b`), as a repeated code is an empty element
144
+ everywhere.
145
+
101
146
  The composed methods above are built on these and cannot write ill-formed
102
147
  output; with the primitives, well-formedness is yours. Values keep their one
103
148
  spelling either way (there is no STX/ETX primitive), and names are still
104
149
  checked. The two layers share the record state and mix freely:
105
150
  `b.rs("a").field("b")` writes `␞a␟b`, `b.record.us("c")` writes `␞␟c`.
106
151
 
152
+ ### C0DIFF
153
+
154
+ A diff is an ordinary document of `shape` `diff`: files of records, each
155
+ record an anchored pattern, with `old␚␟new` as the substitution
156
+ (`DIFF.md` in the spec).
157
+
158
+ ```
159
+ ␖␁shape␟diff
160
+ ␜greeting.txt
161
+ ␞␂Hello ␃␟world␚␟universe␟!
162
+ ␄
163
+ ```
164
+
165
+ ```ruby
166
+ diff = C0::Diff.build do |d|
167
+ d.file("greeting.txt") do
168
+ d.record { |r| r.anchor("Hello "); r.sub("world", "universe"); r.anchor("!") }
169
+ d.assert("Bye") # must be present once; nothing changes
170
+ d.insert_after("!", "\n") # ␞!␟␚␟␂\n␃
171
+ end
172
+ end
173
+ C0::Diff.apply(diff, "greeting.txt" => source) # => { "greeting.txt" => edited }, validated first
174
+ C0::Diff.apply_files(diff, "src") # on disk, atomically (temporary file + rename)
175
+ C0::Diff.parse(diff) # => FileEdits of Patterns of Units
176
+ ```
177
+
178
+ Each record's search text must match once from the previous match to the
179
+ end of the file. There is no replace-all mark: a tool expands one into
180
+ anchored records against the target. Anything the text diff does not use
181
+ (keyed units, levels, references, header rows, GS, dead records, two
182
+ sub-units on one unit, a sub-unit of a sub-unit, an empty search text,
183
+ `␟␚␟`, the same path twice) is refused with a `C0::Diff::Error`.
184
+
185
+ ### Stream logs
186
+
187
+ ```ruby
188
+ r = C0::Stream::Reader.read("events.c0")
189
+ r.torn? # uncommitted tail?
190
+ r.damaged # indexes of committed blocks that are not well-formed
191
+ r.each_record { |rec| ... }
192
+ ```
193
+
194
+ Any error inside a committed block damages the block, and a damaged block
195
+ delivers nothing. By default the reader yields the records before it and
196
+ raises `C0::Stream::DamagedBlockError` with the block's `index` and the
197
+ `resume_offset` to go on from; with
198
+ `Reader.new(buf, continue_past_damage: true)` it skips the block and reads
199
+ on. A damaged block adds nothing to the parse state, so what follows is read
200
+ in the context before it.
201
+
107
202
  ## Install / build
108
203
 
109
204
  ```sh
@@ -113,7 +208,7 @@ gem install c0data # the gem is c0data; in code it is `require "c0"`
113
208
  From a checkout, the C core and the shared conformance vectors are git submodules:
114
209
 
115
210
  ```sh
116
- git submodule update --init # pulls in c0-c (the header) and c0-spec
211
+ git submodule update --init # pulls in c0-c (the headers and c0_diff.c) and c0-spec
117
212
  rake compile # builds the extension
118
213
  ```
119
214
 
@@ -121,16 +216,19 @@ Requires a C compiler and Ruby development headers (MRI).
121
216
 
122
217
  ## Status
123
218
 
124
- Binds the c0-c 1.0 core: validator and canonical check, documents and
125
- containers (SYN heading, EOT, EM tail, warnings), bodies with files, groups,
126
- header rows, positional and keyed records, nested levels, lists and references
127
- (all zero-copy), the canonical builder, ETB stream mode (committed region,
128
- damaged blocks, torn-tail detection) and pretty form (compact, aligned and
219
+ Binds the c0-c 1.0 core: one parser for the whole grammar (parse is
220
+ validate) and the canonical check, documents and containers (SYN heading
221
+ with `shape`, `schema`, `version` and `blob`, EOT, EM tail, warnings),
222
+ bodies with files, groups, header rows, positional and keyed records, nested
223
+ levels, lists and references (plain and pattern), subsections, sub-units
224
+ and dead elements, the canonical builder, ETB stream mode
225
+ (committed region, damaged blocks with resume offsets, torn-tail detection),
226
+ C0DIFF (parse, build, atomic apply) and pretty form (compact, aligned and
129
227
  spaced). Passes every shared conformance vector from
130
- [c0-spec](https://github.com/c0data/c0-spec) (decode, encode, canonical,
131
- invalid, nested, stream).
228
+ [c0-spec](https://github.com/c0data/c0-spec): decode, encode, canonical,
229
+ invalid, nested, stream and diff (277 cases).
132
230
 
133
- Converters (CSV / JSON / C0-DIFF) are not yet wrapped.
231
+ Converters (CSV / JSON) are not yet wrapped.
134
232
 
135
233
  ## Test
136
234