rbjq 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 52c3e4deac2047a2c2d54cc338b64eda7157a3d1670b2af975f439eabe69ca23
4
- data.tar.gz: c88f49272ab46fbf4ec2279166cda8f96aae55b19dc0b750fe23c20b6dbff9a9
3
+ metadata.gz: 0d32e4488a80488433e57f546c331b567398fbf5e78442e8163c21ced9364037
4
+ data.tar.gz: 388c8deb0dd22b169d7ab986da80c8bd6056831216031d7942cd2bc8503b9c1e
5
5
  SHA512:
6
- metadata.gz: 030c8a230a52de382fcd134f2717e6b8831dae68eb99022877f64166bf468a937476bf3e64b644c401591ca9d1648143bbfb64fa31ba53993c1325b162f4dbfb
7
- data.tar.gz: 12057e4fc6ddb714416cb8704e9602f826fdd843eec0c02d4c68e711dc38a7dd11557216a7467d7fda5ed542ff2d5458b34503e9f0c87ec7f5deef1a2e1ce1c1
6
+ metadata.gz: 656ff877d9a9c22f43a1339a2fc318fe5e3c2f4844e4c3b288e112d5ff0032ec218ba27a438ee69950f265dd0d54c75fb50aae7ea51cb497325d651eaaa009bb
7
+ data.tar.gz: 121450e6089c0597186c69ff07b168ee1c2790285f68663f7d9e3d0f1cac32a116d8b43797bf0f9015dcdcfe0fa5ac91e225c396071e289a6ffa064fdf3b624d
data/README.md CHANGED
@@ -1,5 +1,6 @@
1
1
  # rbjq
2
2
 
3
+ [![Gem Version](https://badge.fury.io/rb/rbjq.svg)](https://badge.fury.io/rb/rbjq)
3
4
  [![test](https://github.com/winebarrel/rbjq/actions/workflows/test.yml/badge.svg)](https://github.com/winebarrel/rbjq/actions/workflows/test.yml)
4
5
  [![codecov](https://codecov.io/gh/winebarrel/rbjq/graph/badge.svg)](https://codecov.io/gh/winebarrel/rbjq)
5
6
  [![AI Generated](https://img.shields.io/badge/AI%20Generated-Claude-orange?logo=anthropic)](https://claude.com/product/claude-code)
@@ -27,7 +28,7 @@ Rbjq.query({ 'users' => [{ 'name' => 'a', 'age' => 20 }, { 'name' => 'b', 'age'
27
28
 
28
29
  # Compile once, run many times
29
30
  prog = Rbjq.compile('map(. * 2)')
30
- prog.run([1, 2, 3]) #=> [[2, 4, 6]]
31
+ prog.run([1, 2, 3]) #=> [[2, 4, 6]]
31
32
  prog.each([1, 2]) { |v| p v } # streams outputs
32
33
 
33
34
  # JSON text in (may contain several values), Ruby values out
@@ -76,6 +77,93 @@ Errors:
76
77
  - `Rbjq::JQError`: runtime errors (`#value` holds the error value, e.g. from `error({...})`)
77
78
  - `Rbjq::ParseError` (a `JQError`): invalid JSON
78
79
 
80
+ #### Building filters in Ruby
81
+
82
+ `require 'rbjq/expr'` lets you write a filter as Ruby method chains instead
83
+ of a jq string. It is not loaded by `require 'rbjq'`.
84
+
85
+ ```ruby
86
+ require 'rbjq/expr'
87
+
88
+ r = Rbjq.root
89
+ q = r[:users][].select { _1[:age] > 30 }[:name]
90
+
91
+ Rbjq.query(data, q) # compile / query / query_json accept it
92
+ Rbjq.compile(q).source #=> ".users[] | select(.age > 30).name"
93
+ ```
94
+
95
+ `Rbjq.root` is the program's input. A block or lambda passed to a method is
96
+ a filter, and its parameter is jq's `.` at that point (`_1`, a named
97
+ parameter, or `it` on Ruby 3.4 and later).
98
+
99
+ | jq | Ruby |
100
+ |---|---|
101
+ | `.a`, `.[0]`, `.[]` | `r[:a]`, `r[0]`, `r[]` |
102
+ | `.["a", "b"]` (outputs `.a`, then `.b`) | `r[:a, :b]` (`r.then { [_1[:a, :b]] }` for `[.a, .b]`) |
103
+ | `.[1:3]` | `r[1..2]`, `r[1...3]`, `r.slice(1, 3)` |
104
+ | `.a \| f` | `r[:a].f` |
105
+ | `.a \| [.b, .c]` | `r[:a].then { [_1[:b], _1[:c]] }` |
106
+ | `.a + 1`, `.a > 1`, `-.a` | `r[:a] + 1`, `r[:a] > 1`, `-r[:a]` |
107
+ | `a and b`, `a or b`, `a \| not` | `a.and(b)`, `a.or(b)`, `a.not` or `!a` |
108
+ | `a, b, c` | `a.comma(b, c)` |
109
+ | `.a // 0` | `r[:a].default(0)` |
110
+ | `[.a, 1]`, `{name: .n, (.k): 1}` | `r.then { [_1[:a], 1] }`, `r.then { { name: _1[:n], _1[:k] => 1 } }` |
111
+ | `"hi \(.name)"` | `"hi #{r[:name]}"` |
112
+ | `select(.age > 30)` | `select { _1[:age] > 30 }` |
113
+ | `limit(3; .[])` | `limit(3) { _1[] }` |
114
+ | `until(. > 100; . * 2)` | `until(->(x) { x > 100 }) { _1 * 2 }` |
115
+ | `if c then a elif c2 then b else d end` | `.if(c, a, c2, b, d)` |
116
+ | `f?`, `try f catch g` | `f.try`, `f.try { \|err\| g }` |
117
+ | `f as $x \| g` | `f.as { \|x\| g }` |
118
+ | `f as [$a, $b] \| g`, `f as {k: $v} \| g` | `f.as { \|a, b\| g }`, `f.as(:k) { \|v\| g }` |
119
+ | `reduce .[] as $x (0; . + $x)` | `r[].reduce(0) { \|acc, x\| acc + x }` |
120
+ | `foreach .[] as $x (0; . + $x; [$x, .])` | `r[].foreach(0, ->(acc, x) { [x, acc] }) { \|acc, x\| acc + x }` |
121
+ | `.a = 1`, `.a \|= . + 1` | `r[:a].set(1)`, `r[:a].update { _1 + 1 }` |
122
+ | `.a += 1`, `.a //= 0` | `r[:a].update(:+, 1)`, `r[:a].update(:default, 0)` |
123
+ | anything else | `r[:a].jq('to_entries \| from_entries')` |
124
+
125
+ Objects are Ruby Hashes returned from a block, with Exprs as values (or as
126
+ keys, for computed keys). There is no shorthand like jq's `{a, b}`; write
127
+ each key and value:
128
+
129
+ ```ruby
130
+ r.then { { name: _1[:name], age: _1[:age] } }
131
+ # {name: .name, age: .age} (jq's {name, age})
132
+
133
+ r[:users][].then { |u| { name: u[:name], tags: u[:tags].length } }
134
+ # .users[] | {name: .name, tags: (.tags | length)}
135
+
136
+ r[:users].map { |u| { name: u[:name], adult: u[:age] >= 20 } }
137
+ # .users | map({name: .name, adult: .age >= 20})
138
+
139
+ r[:users][].then { |u| { u[:name] => u[:age] } }
140
+ # .users[] | {(.name): .age}
141
+
142
+ r[:users][].then { |u| { name: u[:name], total: r[:total] } }
143
+ # . as $__root | .users[] | {name: .name, total: $__root.total}
144
+ ```
145
+
146
+ - Other methods call the jq function of that name (builtins, `functions:`,
147
+ `Rbjq.define_function`). Arguments are Ruby values (literals), Exprs, or
148
+ filters (blocks or lambdas); a block is the last argument. Use
149
+ `e.call(:name, ...)` for a function whose name is taken by a method above.
150
+ - An Expr always refers to the `.` it was made from. `Rbjq.root` is the
151
+ outermost input also inside blocks, and a block parameter can be used in
152
+ nested blocks; the generated program binds them to variables:
153
+ `r[:users][].select { _1[:age] > r[:min] }` is
154
+ `. as $__root | .users[] | select(.age > $__root.min)`.
155
+ - A method applies to the whole chain before it: `r[][:a].tonumber.try` is
156
+ `(.[].a | tonumber)?`, which stops at the first error. For
157
+ `.[] | (.a | tonumber)?`, write `r[].then { _1[:a].tonumber.try }`.
158
+ - Use `and`/`or`, not `&&`/`||`: Ruby treats every Expr as true. Put the
159
+ Expr on the left of `==` (`1 == e` is a Ruby comparison).
160
+ - A Ruby Array or String cannot start a chain (`[r[:a]].length` calls
161
+ Ruby's method); build it inside `then`: `r.then { [_1[:a]] }.length`.
162
+ - An Expr in a String (`"#{e}"`) becomes a signed marker that only the
163
+ builder turns back into an interpolation, so do not print or store it.
164
+ - Not supported: `def`, `label`/`break`, `?//`, `$__loc__`, `@format`
165
+ strings, and nested destructuring (use `jq('...')`).
166
+
79
167
  ### Command line
80
168
 
81
169
  ```sh
data/lib/rbjq/expr.rb ADDED
@@ -0,0 +1,508 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Expr is a BasicObject (no is_a?/nil?), so values that may be Exprs are
4
+ # tested with Class#===.
5
+ # rubocop:disable Style/CaseEquality
6
+
7
+ require 'digest'
8
+ require_relative '../rbjq'
9
+ require_relative 'unparser'
10
+
11
+ module Rbjq
12
+ # Builds jq ASTs from Ruby code (see Expr).
13
+ #
14
+ # Inside a block, the block parameter stands for jq's `.` at that point.
15
+ # Such a parameter is stored as a [:ctx, id, label] node. When the block
16
+ # returns, the node becomes `.` where `.` is still the block's input, and
17
+ # `$__label1` (bound by `. as $__label1 | ...` at the top of the block body)
18
+ # where `.` has changed, e.g. inside a nested block. Rbjq.root is the
19
+ # outermost context and is resolved when the program is compiled.
20
+ module ExprBuilder
21
+ ROOT_ID = 0
22
+
23
+ ASSIGN_OPS = {
24
+ :+ => :'+=', :- => :'-=', :* => :'*=', :/ => :'/=', :% => :'%=', :default => :'//='
25
+ }.freeze
26
+
27
+ # Ruby protocol methods that must not become jq calls (implicit
28
+ # conversions, Hash keys, pp, Marshal, ...).
29
+ NOT_JQ = /\A(?:to_(?!entries\z)|pretty_print|marshal_|_dump\z|encode_with\z|init_with\z)|[?!=]\z/
30
+
31
+ MARK_OPEN = "\u{10FFFC}"
32
+ MARK_CLOSE = "\u{10FFFD}"
33
+ MARK_RE = %r{\u{10FFFC}([A-Za-z0-9+/]*=*)\.(\h{64})\u{10FFFD}}
34
+
35
+ @mutex = Mutex.new
36
+ @last_id = ROOT_ID
37
+ @key = Random.urandom(32)
38
+
39
+ class << self
40
+ def root
41
+ Expr.new([:ctx, ROOT_ID, 'root'])
42
+ end
43
+
44
+ def next_id
45
+ @mutex.synchronize { @last_id += 1 }
46
+ end
47
+
48
+ # Ruby value -> AST
49
+ def lift(v)
50
+ case v
51
+ when Expr then v.__ast__
52
+ when Proc then filter(v)
53
+ when nil, true, false then [:literal, v]
54
+ when Integer, Float then [:literal, Value.normalize_number(v)]
55
+ when String then string(v)
56
+ when Symbol then [:literal, v.to_s]
57
+ when Array then array(v)
58
+ when Hash then [:object, v.map { |k, x| [key(k), lift(x)] }]
59
+ when Struct then lift(v.to_h)
60
+ else lift(Program.normalize_value(v))
61
+ end
62
+ end
63
+
64
+ def array(elems)
65
+ return [:array, nil] if elems.empty?
66
+
67
+ [:array, elems.map { |e| lift(e) }.reduce { |a, b| [:comma, a, b] }]
68
+ end
69
+
70
+ def key(k)
71
+ Symbol === k ? [:literal, k.to_s] : lift(k) # k may be an Expr (no is_a?)
72
+ end
73
+
74
+ # A block or lambda used as a filter: its parameter is `.`.
75
+ def filter(pr)
76
+ id = next_id
77
+ label = label_for(pr.parameters.first)
78
+ body = lift(invoke(pr, [Expr.new([:ctx, id, label])]))
79
+ close(body, id, label)
80
+ end
81
+
82
+ def invoke(pr, args)
83
+ if pr.lambda? && pr.arity >= 0
84
+ pr.call(*args.first(pr.arity))
85
+ else
86
+ pr.call(*args)
87
+ end
88
+ end
89
+
90
+ def label_for(param)
91
+ name = param && param[1]&.to_s
92
+ name&.match?(/\A[a-z][a-zA-Z0-9_]*\z/) ? name : 'it'
93
+ end
94
+
95
+ def var_name(param, i)
96
+ name = param && param[1]&.to_s
97
+ name&.match?(/\A[a-zA-Z_][a-zA-Z0-9_]*\z/) ? name : "__v#{i}"
98
+ end
99
+
100
+ # Pattern and variables for the parameters of an `as`/`reduce`/`foreach` block.
101
+ def bind_pattern(params, keys = [])
102
+ if keys.empty?
103
+ n = [params.size, 1].max
104
+ names = Array.new(n) { |i| var_name(params[i], i) }
105
+ pat = n == 1 ? [:pvar, names[0]] : [:parray, names.map { |nm| [:pvar, nm] }]
106
+ else
107
+ names = Array.new(keys.size) { |i| var_name(params[i], i) }
108
+ pat = [:pobject, keys.zip(names).map { |k, nm| [[:literal, k.to_s], nil, [:pvar, nm]] }]
109
+ end
110
+ [pat, names.map { |nm| Expr.new([:var, nm]) }]
111
+ end
112
+
113
+ # (a | b) | c is a | (b | c), which reads better as jq source
114
+ def pipe(l, r)
115
+ l[0] == :pipe ? [:pipe, l[1], pipe(l[2], r)] : [:pipe, l, r]
116
+ end
117
+
118
+ # (a | b).k is a | b.k when the operands (k) do not depend on `.`
119
+ def postfix(target, *operands, &build)
120
+ if target[0] == :pipe && operands.all? { |o| o.nil? || o[0] == :literal }
121
+ [:pipe, target[1], postfix(target[2], *operands, &build)]
122
+ else
123
+ build.call(target)
124
+ end
125
+ end
126
+
127
+ def if_node(list)
128
+ c, t, *rest = list
129
+ e = case rest.size
130
+ when 0 then nil
131
+ when 1 then rest[0]
132
+ else if_node(rest)
133
+ end
134
+ [:if, c, t, e]
135
+ end
136
+
137
+ # Replaces context `id` in ast; binds it to a variable if needed.
138
+ def close(ast, id, label)
139
+ bound = false
140
+ out = resolve(ast, id, true) { bound = true }
141
+ bound ? [:as, [:identity], [[:pctx, id, label]], out] : out
142
+ end
143
+
144
+ # same: whether `.` at this node is still the context's `.`
145
+ def resolve(node, id, same, &on_bind)
146
+ return node unless node.is_a?(Array)
147
+
148
+ r = ->(n, s = same) { resolve(n, id, s, &on_bind) }
149
+ case node[0]
150
+ when :ctx
151
+ return node unless node[1] == id
152
+ return [:identity] if same
153
+
154
+ on_bind.call
155
+ [:ctxvar, id, node[2]]
156
+ when :identity, :literal, :var, :loc, :format, :break, :ctxvar then node
157
+ when :index then [:index, r.call(node[1]), r.call(node[2])]
158
+ when :iterate then [:iterate, r.call(node[1])]
159
+ when :slice then [:slice, r.call(node[1]), r.call(node[2]), r.call(node[3])]
160
+ when :pipe
161
+ l = r.call(node[1])
162
+ l == [:identity] ? r.call(node[2]) : [:pipe, l, r.call(node[2], false)]
163
+ when :comma, :and, :or, :alt then [node[0], r.call(node[1]), r.call(node[2])]
164
+ when :binop then [:binop, node[1], r.call(node[2]), r.call(node[3])]
165
+ when :neg then [:neg, r.call(node[1])]
166
+ when :call then [:call, node[1], node[2].map { |a| r.call(a, false) }]
167
+ when :if then [:if, r.call(node[1]), r.call(node[2]), r.call(node[3])]
168
+ when :try then [:try, r.call(node[1]), r.call(node[2], false)]
169
+ when :as then [:as, r.call(node[1]), node[2], r.call(node[3])]
170
+ when :array then [:array, r.call(node[1])]
171
+ when :object then [:object, node[1].map { |k, v| [r.call(k), r.call(v)] }]
172
+ when :string then [:string, node[1].map { |p| p.is_a?(String) ? p : r.call(p) }, node[2]]
173
+ when :reduce then [:reduce, r.call(node[1]), node[2], r.call(node[3]), r.call(node[4], false)]
174
+ when :foreach then [:foreach, r.call(node[1]), node[2], r.call(node[3]), r.call(node[4], false),
175
+ r.call(node[5], false)]
176
+ when :func_def then [:func_def, node[1], node[2], r.call(node[3], false), r.call(node[4])]
177
+ when :assign then [:assign, node[1], r.call(node[2]), r.call(node[3], node[1] != :'|=' && same)]
178
+ when :label then [:label, node[1], r.call(node[2])]
179
+ else raise ArgumentError, "unknown node #{node[0].inspect}"
180
+ end
181
+ end
182
+
183
+ # The AST of a whole program built from Rbjq.root.
184
+ def finalize(ast)
185
+ name_vars(close(ast, ROOT_ID, 'root'), strict: true)
186
+ end
187
+
188
+ def inspect_source(ast)
189
+ Unparser.unparse(name_vars(close(ast, ROOT_ID, 'root'), strict: false))
190
+ rescue StandardError => e
191
+ "<#{e.class}: #{e.message}>"
192
+ end
193
+
194
+ # Gives the context variables names in order of appearance.
195
+ def name_vars(ast, strict:)
196
+ names = {}
197
+ counts = Hash.new(0)
198
+ name = lambda do |(_, id, label)|
199
+ names[id] ||= id == ROOT_ID ? '__root' : "__#{label}#{counts[label] += 1}"
200
+ end
201
+ walk = lambda do |n|
202
+ next n unless n.is_a?(Array)
203
+
204
+ case n[0]
205
+ when :ctx
206
+ raise CompileError, 'a block parameter is used outside its block' if strict
207
+
208
+ [:identity] # Expr#inspect inside a block: the block's `.`
209
+ when :ctxvar then [:var, name.call(n)]
210
+ when :pctx then [:pvar, name.call(n)]
211
+ else n.map { |c| walk.call(c) }
212
+ end
213
+ end
214
+ walk.call(ast)
215
+ end
216
+
217
+ # ------------------------------------------------ string interpolation
218
+ #
219
+ # Expr#to_s returns a marker holding the AST, signed with a per-process
220
+ # key, so that "#{expr}" in a String can be turned back into jq string
221
+ # interpolation. Markers that fail the check are kept as plain text.
222
+
223
+ def marker(ast)
224
+ payload = [Marshal.dump(ast)].pack('m0')
225
+ "#{MARK_OPEN}#{payload}.#{hmac(payload)}#{MARK_CLOSE}"
226
+ end
227
+
228
+ def string(s)
229
+ return [:literal, s] unless s.encoding == Encoding::UTF_8 && s.valid_encoding? && s.include?(MARK_OPEN)
230
+
231
+ parts = []
232
+ pos = 0
233
+ s.scan(MARK_RE) do
234
+ m = Regexp.last_match
235
+ ast = unmark(m[1], m[2])
236
+ next unless ast
237
+
238
+ parts << s[pos...m.begin(0)]
239
+ parts << ast
240
+ pos = m.end(0)
241
+ end
242
+ parts << s[pos..]
243
+ parts = parts.reject { |p| p.is_a?(String) && p.empty? }
244
+ parts.all?(String) ? [:literal, parts.join] : [:string, parts, nil]
245
+ end
246
+
247
+ def unmark(payload, mac)
248
+ return nil unless hmac(payload) == mac
249
+
250
+ Marshal.load(payload.unpack1('m0')) # rubocop:disable Security/MarshalLoad
251
+ end
252
+
253
+ def hmac(data)
254
+ k = @key.ljust(64, "\0").bytes
255
+ ipad = k.map { |b| b ^ 0x36 }.pack('C*')
256
+ opad = k.map { |b| b ^ 0x5c }.pack('C*')
257
+ Digest::SHA256.hexdigest(opad + Digest::SHA256.digest(ipad + data))
258
+ end
259
+ end
260
+ end
261
+
262
+ # A jq filter built from Ruby code. Start from Rbjq.root (the program's
263
+ # input) and chain methods; pass the result to Rbjq.compile / Rbjq.query.
264
+ #
265
+ # q = Rbjq.root[:users][].select { _1[:age] > 30 }[:name]
266
+ # Rbjq.query(data, q)
267
+ #
268
+ # Methods other than the ones below call the jq function of that name with
269
+ # the receiver as input (`e.length` is `e | length`). Arguments are Ruby
270
+ # values (literals), Exprs, or a block / lambda (a filter whose parameter
271
+ # is `.`); a block is the last argument.
272
+ class Expr < BasicObject
273
+ def initialize(ast)
274
+ @ast = ast
275
+ end
276
+
277
+ def __ast__
278
+ @ast
279
+ end
280
+
281
+ # e[:a] is .a, e[0] is .[0], e[] is .[], e[1..2] is .[1:3],
282
+ # e[:a, :b] is .["a", "b"] (outputs .a, then .b)
283
+ def [](*args)
284
+ case args.size
285
+ when 0 then wrap(ExprBuilder.postfix(@ast) { |t| [:iterate, t] })
286
+ when 1
287
+ k = args[0]
288
+ return range_slice(k) if ::Range === k
289
+
290
+ key = ExprBuilder.key(k)
291
+ wrap(ExprBuilder.postfix(@ast, key) { |t| [:index, t, key] })
292
+ else
293
+ multi_index(args)
294
+ end
295
+ end
296
+
297
+ # .[from:to] (`to` is exclusive; nil leaves a side open)
298
+ def slice(from, to = nil)
299
+ from = ::NilClass === from ? nil : ExprBuilder.lift(from)
300
+ to = ::NilClass === to ? nil : ExprBuilder.lift(to)
301
+ wrap(ExprBuilder.postfix(@ast, from, to) { |t| [:slice, t, from, to] })
302
+ end
303
+
304
+ %i[+ - * / % == != < <= > >=].each do |op|
305
+ define_method(op) { |other| wrap([:binop, op, @ast, ExprBuilder.lift(other)]) }
306
+ end
307
+
308
+ def -@
309
+ wrap([:neg, @ast])
310
+ end
311
+
312
+ # !e is e | not (BasicObject#! would make it false)
313
+ def !
314
+ wrap(ExprBuilder.pipe(@ast, [:call, 'not', []]))
315
+ end
316
+
317
+ # 1 + e, 1 < e
318
+ def coerce(other)
319
+ [Expr.new(ExprBuilder.lift(other)), self]
320
+ end
321
+
322
+ def and(other)
323
+ wrap([:and, @ast, ExprBuilder.lift(other)])
324
+ end
325
+
326
+ def or(other)
327
+ wrap([:or, @ast, ExprBuilder.lift(other)])
328
+ end
329
+
330
+ # e, a, b
331
+ def comma(*others)
332
+ wrap(others.reduce(@ast) { |a, o| [:comma, a, ExprBuilder.lift(o)] })
333
+ end
334
+
335
+ # e // other
336
+ def default(other)
337
+ wrap([:alt, @ast, ExprBuilder.lift(other)])
338
+ end
339
+
340
+ # e | <block>
341
+ def then(&blk)
342
+ ::Kernel.raise ::ArgumentError, 'then requires a block' unless blk
343
+
344
+ wrap(ExprBuilder.pipe(@ast, ExprBuilder.filter(blk)))
345
+ end
346
+
347
+ # e | if c then a elif c2 then b else d end: if(c, a, c2, b, d)
348
+ def if(*args)
349
+ ::Kernel.raise ::ArgumentError, 'if requires a condition and a value' if args.size < 2
350
+
351
+ wrap(ExprBuilder.pipe(@ast, ExprBuilder.if_node(args.map { |a| ExprBuilder.lift(a) })))
352
+ end
353
+
354
+ # e? / try e catch <block>
355
+ def try(&blk)
356
+ wrap([:try, @ast, blk && ExprBuilder.filter(blk)])
357
+ end
358
+
359
+ # e as $x | <block>; as(:a, :b) { |a, b| } is e as {a: $a, b: $b} | ...
360
+ def as(*keys, &blk)
361
+ ::Kernel.raise ::ArgumentError, 'as requires a block' unless blk
362
+
363
+ pat, vars = ExprBuilder.bind_pattern(blk.parameters, keys)
364
+ wrap([:as, @ast, [pat], ExprBuilder.lift(ExprBuilder.invoke(blk, vars))])
365
+ end
366
+
367
+ # reduce e as $x (init; <block>): reduce(init) { |acc, x| ... }
368
+ def reduce(init, &blk)
369
+ ::Kernel.raise ::ArgumentError, 'reduce requires a block' unless blk
370
+
371
+ pat, _, update = reduce_parts(blk)
372
+ wrap([:reduce, @ast, [pat], ExprBuilder.lift(init), update])
373
+ end
374
+
375
+ # foreach e as $x (init; <block>; extract): foreach(init, ->(acc, x) { }) { |acc, x| ... }
376
+ def foreach(init, extract = nil, &blk)
377
+ ::Kernel.raise ::ArgumentError, 'foreach requires a block' unless blk
378
+
379
+ pat, vars, update = reduce_parts(blk)
380
+ ex = if ::Proc === extract
381
+ id = ExprBuilder.next_id
382
+ label = ExprBuilder.label_for(extract.parameters.first)
383
+ body = ExprBuilder.invoke(extract, [Expr.new([:ctx, id, label]), *vars])
384
+ ExprBuilder.close(ExprBuilder.lift(body), id, label)
385
+ elsif !(::NilClass === extract)
386
+ ExprBuilder.lift(extract)
387
+ end
388
+ wrap([:foreach, @ast, [pat], ExprBuilder.lift(init), update, ex])
389
+ end
390
+
391
+ # e = v
392
+ def set(value)
393
+ wrap([:assign, :'=', @ast, ExprBuilder.lift(value)])
394
+ end
395
+
396
+ # e |= <block>; update(:+, v) is e += v; update(:default, v) is e //= v
397
+ def update(*args, &blk)
398
+ if args.empty? && blk
399
+ wrap([:assign, :'|=', @ast, ExprBuilder.filter(blk)])
400
+ elsif args.size == 2 && !blk
401
+ op = ExprBuilder::ASSIGN_OPS.fetch(args[0]) do
402
+ ::Kernel.raise ::ArgumentError, "unknown update operator: #{args[0].inspect}"
403
+ end
404
+ wrap([:assign, op, @ast, ExprBuilder.lift(args[1])])
405
+ else
406
+ ::Kernel.raise ::ArgumentError, 'update takes a block, or an operator and a value'
407
+ end
408
+ end
409
+
410
+ # e | <jq source>
411
+ def jq(source)
412
+ wrap(ExprBuilder.pipe(@ast, Parser.parse(source)))
413
+ end
414
+
415
+ # Calls a jq function whose name is taken by a method of Expr.
416
+ def call(name, *args, &blk)
417
+ args = args.map { |a| ExprBuilder.lift(a) }
418
+ args << ExprBuilder.filter(blk) if blk
419
+ wrap(ExprBuilder.pipe(@ast, [:call, name.to_s, args]))
420
+ end
421
+
422
+ def to_s
423
+ ExprBuilder.marker(@ast)
424
+ end
425
+
426
+ def inspect
427
+ "#<Rbjq::Expr #{ExprBuilder.inspect_source(@ast)}>"
428
+ end
429
+
430
+ def hash
431
+ __id__.hash
432
+ end
433
+
434
+ def eql?(other)
435
+ equal?(other)
436
+ end
437
+
438
+ private
439
+
440
+ def wrap(ast)
441
+ Expr.new(ast)
442
+ end
443
+
444
+ # Numbers are rejected: e[1, 2] reads as start and length in Ruby.
445
+ def multi_index(keys)
446
+ if keys.any? { |k| ::Numeric === k || ::Range === k }
447
+ ::Kernel.raise ::ArgumentError, "keys must be Symbols, Strings or Exprs: #{keys.inspect}"
448
+ end
449
+
450
+ key = keys.map { |k| ExprBuilder.key(k) }.reduce { |a, b| [:comma, a, b] }
451
+ wrap([:index, @ast, key])
452
+ end
453
+
454
+ def range_slice(range)
455
+ from = range.begin
456
+ to = range.end
457
+ [from, to].each do |v|
458
+ unless v.nil? || v.is_a?(::Integer)
459
+ ::Kernel.raise ::ArgumentError,
460
+ "slice bounds must be Integers: #{range.inspect}"
461
+ end
462
+ end
463
+ to = if to.nil? then nil
464
+ elsif range.exclude_end? then to
465
+ elsif to == -1 then nil
466
+ else to + 1
467
+ end
468
+ from &&= [:literal, from]
469
+ to &&= [:literal, to]
470
+ wrap(ExprBuilder.postfix(@ast, from, to) { |t| [:slice, t, from, to] })
471
+ end
472
+
473
+ def reduce_parts(blk)
474
+ params = blk.parameters
475
+ id = ExprBuilder.next_id
476
+ label = ExprBuilder.label_for(params[0])
477
+ pat, vars = ExprBuilder.bind_pattern(params.drop(1))
478
+ body = ExprBuilder.invoke(blk, [Expr.new([:ctx, id, label]), *vars])
479
+ [pat, vars, ExprBuilder.close(ExprBuilder.lift(body), id, label)]
480
+ end
481
+
482
+ def method_missing(name, *, &)
483
+ if ExprBuilder::NOT_JQ.match?(name.to_s)
484
+ ::Kernel.raise ::NoMethodError.new("undefined method '#{name}' for an instance of Rbjq::Expr", name,
485
+ receiver: self)
486
+ end
487
+
488
+ call(name, *, &)
489
+ end
490
+
491
+ def respond_to_missing?(name, _include_private = false)
492
+ !ExprBuilder::NOT_JQ.match?(name.to_s)
493
+ end
494
+ end
495
+
496
+ class << self
497
+ # The program's input, the starting point for building a filter in Ruby
498
+ # (see Rbjq::Expr). It always means the outermost input, also inside
499
+ # blocks.
500
+ #
501
+ # require 'rbjq/expr'
502
+ # Rbjq.query(data, Rbjq.root[:users][].select { _1[:age] > 30 }[:name])
503
+ def root
504
+ ExprBuilder.root
505
+ end
506
+ end
507
+ end
508
+ # rubocop:enable Style/CaseEquality
data/lib/rbjq/program.rb CHANGED
@@ -45,9 +45,21 @@ module Rbjq
45
45
  # (see Rbjq::Functions); they take precedence over functions defined
46
46
  # with Rbjq.define_function, which are captured at this point
47
47
  # options: :stderr (IO for debug/stderr), :filename_proc
48
+ # source: jq source text, or an Rbjq::Expr (then #source is the jq
49
+ # source generated from it)
48
50
  def initialize(source, vars: {}, functions: {}, **options)
49
- @source = source
50
- @ast = Parser.parse(source)
51
+ if defined?(Expr) && Expr === source # rubocop:disable Style/CaseEquality
52
+ @ast = ExprBuilder.finalize(source.__ast__)
53
+ @source = Unparser.unparse(@ast)
54
+ else
55
+ unless source.is_a?(String)
56
+ raise ArgumentError, "filter must be a String or an Rbjq::Expr, not #{source.class} " \
57
+ '(to build one from Ruby values, use Rbjq.root.then { ... } from rbjq/expr)'
58
+ end
59
+
60
+ @source = source
61
+ @ast = Parser.parse(source)
62
+ end
51
63
  @vars = vars.each_with_object({}) { |(k, v), h| h[k.to_s.delete_prefix('$')] = self.class.normalize_value(v) }
52
64
  @functions = Functions.global.merge(Functions.build_all(functions))
53
65
  Checker.new(self.class.known_functions | @functions.keys).check(@ast, @vars.keys)
@@ -0,0 +1,218 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Rbjq
4
+ # Turns an AST back into jq source. Parsing the output gives an equivalent
5
+ # program (numbers may parse as negation, e.g. `-1` as `-(1)`).
6
+ module Unparser
7
+ # Precedence levels, lowest to highest (see Parser).
8
+ PIPE = 0
9
+ COMMA = 1
10
+ ALT = 2
11
+ ASSIGN = 3
12
+ OR = 4
13
+ AND = 5
14
+ COMPARE = 6
15
+ ADD = 7
16
+ MUL = 8
17
+ UNARY = 9
18
+ POSTFIX = 10
19
+
20
+ IDENT = /\A[a-zA-Z_][a-zA-Z_0-9]*\z/
21
+
22
+ module_function
23
+
24
+ def unparse(node)
25
+ str(node, PIPE)
26
+ end
27
+
28
+ def str(node, min)
29
+ s, prec = gen(node)
30
+ prec < min ? "(#{s})" : s
31
+ end
32
+
33
+ # Returns [source, precedence].
34
+ def gen(node)
35
+ case node[0]
36
+ when :identity then ['.', POSTFIX]
37
+ when :literal then literal(node[1])
38
+ when :index then index(node)
39
+ when :iterate then ["#{target(node[1])}[]", POSTFIX]
40
+ when :slice then slice(node)
41
+ when :pipe then ["#{str(node[1], COMMA)} | #{str(node[2], PIPE)}", PIPE]
42
+ when :comma then ["#{str(node[1], COMMA)}, #{str(node[2], ALT)}", COMMA]
43
+ when :alt then ["#{str(node[1], ASSIGN)} // #{str(node[2], ALT)}", ALT]
44
+ when :assign then ["#{str(node[2], OR)} #{node[1]} #{str(node[3], ALT)}", ASSIGN]
45
+ when :or then ["#{str(node[1], OR)} or #{str(node[2], AND)}", OR]
46
+ when :and then ["#{str(node[1], AND)} and #{str(node[2], COMPARE)}", AND]
47
+ when :binop then binop(node)
48
+ when :neg then ["-#{str(node[1], POSTFIX)}", UNARY]
49
+ when :call then call(node)
50
+ when :var then ["$#{node[1]}", POSTFIX]
51
+ when :loc then ['$__loc__', POSTFIX]
52
+ when :format then ["@#{node[1]}", POSTFIX]
53
+ when :string then [string(node[1], node[2]), POSTFIX]
54
+ when :array then ["[#{str(node[1], PIPE) if node[1]}]", POSTFIX]
55
+ when :object then [object(node[1]), POSTFIX]
56
+ when :if then [if_chain(node), POSTFIX]
57
+ when :try then try(node)
58
+ when :as then ["#{str(node[1], ALT)} as #{patterns(node[2])} | #{str(node[3], PIPE)}", PIPE]
59
+ when :reduce then reduce(node)
60
+ when :foreach then foreach(node)
61
+ when :func_def then func_def(node)
62
+ when :label then ["label $#{node[1]} | #{str(node[2], PIPE)}", PIPE]
63
+ when :break then ["break $#{node[1]}", POSTFIX]
64
+ else raise ArgumentError, "unknown node #{node[0].inspect}"
65
+ end
66
+ end
67
+
68
+ def literal(v)
69
+ case v
70
+ when String then [Encoder.dump(v), POSTFIX]
71
+ when Float
72
+ if v.nan? then ['nan', POSTFIX]
73
+ elsif v.infinite? then [v.positive? ? 'infinite' : '-infinite', UNARY]
74
+ else number(Encoder.format_number(v))
75
+ end
76
+ when Integer then number(Encoder.format_number(v))
77
+ else [Encoder.dump(v), POSTFIX]
78
+ end
79
+ end
80
+
81
+ def number(s)
82
+ [s, s.start_with?('-') ? UNARY : POSTFIX]
83
+ end
84
+
85
+ # The term before a postfix (`.a`, `[0]`, `[]`, ...); `.` itself is omitted.
86
+ def target(node)
87
+ node == [:identity] ? '.' : str(node, POSTFIX)
88
+ end
89
+
90
+ def index(node)
91
+ _, t, key = node
92
+ if key[0] == :literal && key[1].is_a?(String)
93
+ name = key[1]
94
+ suffix = field?(name) ? name : Encoder.dump(name)
95
+ return [t == [:identity] ? ".#{suffix}" : "#{str(t, POSTFIX)}.#{suffix}", POSTFIX]
96
+ end
97
+
98
+ ["#{target(t)}[#{str(key, PIPE)}]", POSTFIX]
99
+ end
100
+
101
+ def field?(name)
102
+ IDENT.match?(name) && !Lexer::KEYWORDS.include?(name)
103
+ end
104
+
105
+ def slice(node)
106
+ _, t, from, to = node
107
+ ["#{target(t)}[#{str(from, PIPE) if from}:#{str(to, PIPE) if to}]", POSTFIX]
108
+ end
109
+
110
+ def binop(node)
111
+ _, op, l, r = node
112
+ case op
113
+ when :+, :- then ["#{str(l, ADD)} #{op} #{str(r, MUL)}", ADD]
114
+ when :*, :/, :%
115
+ # `-a * b` parses as `-(a * b)`, so a negation on the left needs parens
116
+ ls = l[0] == :neg || negative_literal?(l) ? "(#{str(l, PIPE)})" : str(l, MUL)
117
+ ["#{ls} #{op} #{str(r, UNARY)}", MUL]
118
+ else ["#{str(l, ADD)} #{op} #{str(r, ADD)}", COMPARE]
119
+ end
120
+ end
121
+
122
+ def negative_literal?(node)
123
+ v = node[1]
124
+ node[0] == :literal && v.is_a?(Numeric) && (v.negative? || (v.zero? && (1.0 / v).negative?))
125
+ end
126
+
127
+ def call(node)
128
+ _, name, args = node
129
+ return [name, POSTFIX] if args.empty?
130
+
131
+ ["#{name}(#{args.map { |a| str(a, PIPE) }.join('; ')})", POSTFIX]
132
+ end
133
+
134
+ def string(parts, format)
135
+ body = parts.map do |p|
136
+ p.is_a?(String) ? Encoder.dump(p)[1...-1] : "\\(#{str(p, PIPE)})"
137
+ end.join
138
+ format ? "@#{format} \"#{body}\"" : "\"#{body}\""
139
+ end
140
+
141
+ def object(entries)
142
+ return '{}' if entries.empty?
143
+
144
+ body = entries.map do |k, v|
145
+ "#{object_key(k)}: #{str(v, ALT)}"
146
+ end
147
+ "{#{body.join(', ')}}"
148
+ end
149
+
150
+ def object_key(k)
151
+ if k[0] == :literal && k[1].is_a?(String)
152
+ field?(k[1]) ? k[1] : Encoder.dump(k[1])
153
+ elsif k[0] == :string && k[2].nil?
154
+ string(k[1], nil)
155
+ elsif k[0] == :var
156
+ "$#{k[1]}"
157
+ else
158
+ "(#{str(k, PIPE)})"
159
+ end
160
+ end
161
+
162
+ def if_chain(node)
163
+ _, c, t, e = node
164
+ s = "if #{str(c, PIPE)} then #{str(t, PIPE)}"
165
+ while e && e[0] == :if
166
+ s << " elif #{str(e[1], PIPE)} then #{str(e[2], PIPE)}"
167
+ e = e[3]
168
+ end
169
+ s << " else #{str(e, PIPE)}" if e
170
+ s << ' end'
171
+ end
172
+
173
+ def try(node)
174
+ _, body, handler = node
175
+ return ["#{str(body, POSTFIX)}?", POSTFIX] if handler.nil?
176
+
177
+ ["try #{str(body, POSTFIX)} catch #{str(handler, POSTFIX)}", UNARY]
178
+ end
179
+
180
+ def reduce(node)
181
+ _, src, pats, init, update = node
182
+ ["reduce #{str(src, ALT)} as #{patterns(pats)} (#{str(init, PIPE)}; #{str(update, PIPE)})", POSTFIX]
183
+ end
184
+
185
+ def foreach(node)
186
+ _, src, pats, init, update, extract = node
187
+ s = "foreach #{str(src, ALT)} as #{patterns(pats)} (#{str(init, PIPE)}; #{str(update, PIPE)}"
188
+ s += "; #{str(extract, PIPE)}" if extract
189
+ ["#{s})", POSTFIX]
190
+ end
191
+
192
+ def func_def(node)
193
+ _, name, params, body, rest = node
194
+ ps = params.empty? ? '' : "(#{params.join('; ')})"
195
+ ["def #{name}#{ps}: #{str(body, PIPE)}; #{str(rest, PIPE)}", PIPE]
196
+ end
197
+
198
+ def patterns(pats)
199
+ pats.map { |p| pattern(p) }.join(' ?// ')
200
+ end
201
+
202
+ def pattern(p)
203
+ case p[0]
204
+ when :pvar then "$#{p[1]}"
205
+ when :parray then "[#{p[1].map { |e| pattern(e) }.join(', ')}]"
206
+ when :pobject
207
+ entries = p[1].map do |key, var, sub|
208
+ if var
209
+ sub ? "$#{var}: #{pattern(sub)}" : "$#{var}"
210
+ else
211
+ "#{object_key(key)}: #{pattern(sub)}"
212
+ end
213
+ end
214
+ "{#{entries.join(', ')}}"
215
+ end
216
+ end
217
+ end
218
+ end
data/lib/rbjq/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Rbjq
4
- VERSION = '1.0.0'
4
+ VERSION = '1.1.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rbjq
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0
4
+ version: 1.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - winebarrel
@@ -28,6 +28,7 @@ files:
28
28
  - lib/rbjq/checker.rb
29
29
  - lib/rbjq/cli.rb
30
30
  - lib/rbjq/env.rb
31
+ - lib/rbjq/expr.rb
31
32
  - lib/rbjq/formats.rb
32
33
  - lib/rbjq/functions.rb
33
34
  - lib/rbjq/interpreter.rb
@@ -37,6 +38,7 @@ files:
37
38
  - lib/rbjq/parser.rb
38
39
  - lib/rbjq/prelude.rb
39
40
  - lib/rbjq/program.rb
41
+ - lib/rbjq/unparser.rb
40
42
  - lib/rbjq/value.rb
41
43
  - lib/rbjq/version.rb
42
44
  homepage: https://github.com/winebarrel/rbjq