json_patterns 0.1.1 → 0.1.2
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +357 -0
- metadata +41 -23
data/README.md
ADDED
@@ -0,0 +1,357 @@
|
|
1
|
+
# JSON Patterns
|
2
|
+
|
3
|
+
The JSON Patterns Ruby Gem is a domain-specific language (DSL) library for validating
|
4
|
+
the structure of JSON documents, using patterns that resemble JSON. Just as regexps
|
5
|
+
look a lot like a string, with special symbols to match patterns, a JSON pattern looks
|
6
|
+
like a JSON value, with special keywords to allow for abstract pattern matching.
|
7
|
+
|
8
|
+
It supports the use of alternate object patterns, which can be distinguished by a tag.
|
9
|
+
This allows for a JSON structure to mimic subtyping.
|
10
|
+
|
11
|
+
## Contents
|
12
|
+
|
13
|
+
This gem contains:
|
14
|
+
|
15
|
+
* A family of classes descending from Pattern that represents JSON patterns.
|
16
|
+
* A series of helper functions that provide special keywords to make pattern
|
17
|
+
creation easy.
|
18
|
+
* A family of classes descending from `Validation` or `ObjectMemberValidation`,
|
19
|
+
that do the work of validating your JSON structures.
|
20
|
+
|
21
|
+
## Installation
|
22
|
+
|
23
|
+
gem install json_patterns
|
24
|
+
|
25
|
+
You will also need a JSON parser, such as [yajl-ruby][]. This gem works only in
|
26
|
+
Ruby 1.9 and higher.
|
27
|
+
|
28
|
+
[yajl-ruby]: https://github.com/brianmario/yajl-ruby
|
29
|
+
|
30
|
+
## Usage
|
31
|
+
|
32
|
+
1. `require 'json_patterns'`
|
33
|
+
2. Create a pattern.
|
34
|
+
3. Call `Validation.new_from_pattern` on the pattern to generate a `Validation`.
|
35
|
+
4. Obtain a JSON string, and use a JSON parser such as [yajl-ruby][] to parse it
|
36
|
+
into a nested Ruby data structure.
|
37
|
+
5. Call the `validate_from_root` method on the `Validation` object, passing it
|
38
|
+
the parsed JSON. You'll get an array of `ValidationFailure` objects. If it's
|
39
|
+
empty, the validation was successful.
|
40
|
+
|
41
|
+
Example:
|
42
|
+
|
43
|
+
require 'yajl'
|
44
|
+
require 'json_patterns'
|
45
|
+
|
46
|
+
pattern = { users: array_of({ id: Integer, name: String }) }
|
47
|
+
validation = Validation.new_from_pattern(pattern)
|
48
|
+
|
49
|
+
json = File.new('users.json', 'r')
|
50
|
+
parser = Yajl::Parser.new
|
51
|
+
hash = parser.parse(json)
|
52
|
+
|
53
|
+
errors = validation.validate(hash)
|
54
|
+
print errors
|
55
|
+
|
56
|
+
## Pattern Syntax
|
57
|
+
|
58
|
+
The simplest patterns are JSON literals:
|
59
|
+
|
60
|
+
* A boolean value, e.g. `true`
|
61
|
+
* A number, e.g. `5` or `-3.26`
|
62
|
+
* A string, e.g. `"foo"`
|
63
|
+
* The Ruby value `nil` represents a JSON `null`
|
64
|
+
|
65
|
+
You can match values of a particular type, by using either a Ruby class or the JSON
|
66
|
+
name as a symbol:
|
67
|
+
|
68
|
+
* `Boolean` or `:boolean` matches a `true` or `false`
|
69
|
+
* `TrueClass` matches `true`, `FalseClass` matches `false`
|
70
|
+
* `String` or `:string` matches a string
|
71
|
+
* `Integer` or `:integer` matches an integer
|
72
|
+
* `Float`, `Numeric` or `:float` matches a floating-point number
|
73
|
+
* `NilClass` matches a JSON `null`
|
74
|
+
* `Array` or `:array` matches any array
|
75
|
+
* `Hash` or `:object` matches a JSON object
|
76
|
+
|
77
|
+
You can match some special kinds of strings, too:
|
78
|
+
|
79
|
+
* `Email` matches an email address
|
80
|
+
* `URL` matches an `http`, `https` or `ftp` URL
|
81
|
+
* A `Regexp` such as `/a.*b/` matches strings that match the `Regexp`
|
82
|
+
|
83
|
+
There's a special pattern, `__`, that matches any value.
|
84
|
+
|
85
|
+
You can match arrays, but only when every element is expected to have a different value.
|
86
|
+
These are called *uniform arrays*.
|
87
|
+
|
88
|
+
array_of(String)
|
89
|
+
|
90
|
+
will match a JSON array where all elements are expected to be a string.
|
91
|
+
|
92
|
+
You can match JSON objects using Ruby hashes. Please note the terminology differences:
|
93
|
+
**hashes** in Ruby are called **objects** in JSON. **Key-value pairs** in Ruby are called
|
94
|
+
**object members** in JSON, and Ruby's **keys** are called **names** in JSON. All class
|
95
|
+
names and error messages use the JSON terminology.
|
96
|
+
|
97
|
+
So, a hash:
|
98
|
+
|
99
|
+
{ id: Integer, name: String }
|
100
|
+
|
101
|
+
will match a JSON object with names `id` and `name`, with corresponding integer and string
|
102
|
+
values, respectively. For example,
|
103
|
+
|
104
|
+
{ "id": 5, "name": "Bob" }
|
105
|
+
|
106
|
+
Analogous to the ability to match uniform arrays, you can match *uniform objects.* These
|
107
|
+
are objects that may have an arbitrary number of names, but each value must match the
|
108
|
+
same pattern. For example,
|
109
|
+
|
110
|
+
{ many => String }
|
111
|
+
|
112
|
+
will match any object where all the values are strings. The `many` keyword is a special
|
113
|
+
helper function provided by the gem.
|
114
|
+
|
115
|
+
Optional names can be matched in an object using the `optional` keyword:
|
116
|
+
|
117
|
+
{ id: Integer, name: String, optional => { address: String } }
|
118
|
+
|
119
|
+
will match objects with an `id` and a `name`, and optionally an `address` with a string
|
120
|
+
value. If multiple members are listed in an `optional` clause, they must occur together.
|
121
|
+
So,
|
122
|
+
|
123
|
+
{ id: Integer, optional => { name: String, address: String } }
|
124
|
+
|
125
|
+
would match objects with an `id` only, or objects with names `id`, `name` and `address`,
|
126
|
+
but `name` cannot appear with out `address`, or vice versa.
|
127
|
+
|
128
|
+
### Alternate patterns (disjunctions)
|
129
|
+
|
130
|
+
It is possible to match one of a set of alternate patterns. There are two syntaxes for
|
131
|
+
this, depending on the context. If you are trying to match one of a set of values,
|
132
|
+
pass a list of patterns to the `one_of` function. E.g.,
|
133
|
+
|
134
|
+
one_of('circle', 'square', 'triangle')
|
135
|
+
|
136
|
+
will match any of the three strings. Note that the values passed to `one_of` need not
|
137
|
+
all be of the same type.
|
138
|
+
|
139
|
+
In an object member context, use the keyword `one_of`, with an array value listing the
|
140
|
+
{ id: Integer, one_of => [ { name: String }, { age: Integer } ] }
|
141
|
+
|
142
|
+
This will match an object with an name `id` of type integer, and *either* the name
|
143
|
+
`name` with a string value, or the name `age` with an integer value. Once again,
|
144
|
+
the internal hashes may contain multiple keys. You can think of the curly braces
|
145
|
+
in this case as mere syntax: they delineate a pattern of object members, not a
|
146
|
+
nested object. You can think of the `one_of` keyword as removing the top level of
|
147
|
+
curly braces inside the array.
|
148
|
+
|
149
|
+
A common use case for alternate object patterns is to indicate different "types"
|
150
|
+
that can be considered subtypes of some base type. In Ruby, you might use subclasses
|
151
|
+
to represent alternate forms. For example, if you were representing HTML form
|
152
|
+
element tags, you might have a base `Tag` class, with subclasses such as
|
153
|
+
`InputTag`, `TextAreaTag`, or `SelectTag`. Each of these would in turn have
|
154
|
+
different attributes, though they would share some attributes in common.
|
155
|
+
|
156
|
+
In JSON, you might represent an `<input>` tag with a pattern like this (deliberately
|
157
|
+
oversimplified):
|
158
|
+
|
159
|
+
{ tag: 'input', type: one_of('checkbox', 'text'), value: String }
|
160
|
+
|
161
|
+
and a `<textarea>` tag with a pattern like this:
|
162
|
+
|
163
|
+
{ tag: 'textarea', rows: Integer, cols: Integer }
|
164
|
+
|
165
|
+
To handle either type of tag, you could use a `one_of` pattern. To simplify the
|
166
|
+
code, we can use Ruby variables to hold on to each pattern:
|
167
|
+
|
168
|
+
inputTag = { tag: 'input', type: one_of('checkbox', 'text'), value: String }
|
169
|
+
textAreaTag = { tag: 'textarea', rows: Integer, cols: Integer }
|
170
|
+
tag = one_of(inputTag, textAreaTag)
|
171
|
+
|
172
|
+
So one can think of `tag` as a base class, and `inputTag` and `textAreaTag` as
|
173
|
+
subclasses. They are distinguished by the value of the `tag` object member.
|
174
|
+
|
175
|
+
The Validation objects are smart enough to look at the first object members
|
176
|
+
(key-value pairs) of alternate patterns, and use them to distinguish the
|
177
|
+
appropriate case to follow. Please note that they must be distinguished by
|
178
|
+
the first name (key) or value encountered in the pattern. There is a deliberate
|
179
|
+
ordering to the object members in the patterns, but no ordering in the matched
|
180
|
+
JSON objects. If the cases are not distinguished by the first object member
|
181
|
+
of the pattern, a ValidationAmbiguity error will occur.
|
182
|
+
|
183
|
+
### Cyclic patterns
|
184
|
+
|
185
|
+
Sometimes it is desirable to match nested patterns that repeat. This can
|
186
|
+
be done using the `cyclic` function:
|
187
|
+
|
188
|
+
cyclic { |person| { name: String, email: Email, friends: array_of(person) } }
|
189
|
+
|
190
|
+
This matches an object with three names: `name`, `email` and `friends`, where
|
191
|
+
`friends` points to an array of objects with `name`, `email`, and `friends`,
|
192
|
+
and so on. If you create a cyclic structure, you are responsible for ensuring that
|
193
|
+
it is *well-founded,* that is, it should not require an infinite-sized JSON
|
194
|
+
structure, as all JSON structures are finite. This pattern is well-founded,
|
195
|
+
because at any level in the hierarchy, the `friends` list may be empty.
|
196
|
+
|
197
|
+
Note that we are making use of Ruby's block syntax to create a local variable
|
198
|
+
`person` which can be used to refer to the top level of the cyclic structure.
|
199
|
+
Here is another well-founded example:
|
200
|
+
|
201
|
+
tree = cyclic { |tree| one_of(Integer, { left: tree, right: tree }) }
|
202
|
+
|
203
|
+
This time, we show storing the result in a variable. Please note that due to
|
204
|
+
Ruby scoping rules, the `tree` variable on the outside is distinct from
|
205
|
+
the other `tree` variables inside the call to `cyclic`, which all refer
|
206
|
+
to the same thing. This pattern would match JSON such as:
|
207
|
+
|
208
|
+
{ "left": 4, "right": { "left": 2, "right": 5 } }
|
209
|
+
|
210
|
+
It's also possible to create cyclic patterns by manipulating structures
|
211
|
+
using assignment:
|
212
|
+
|
213
|
+
branch = {left: nil, right: nil}
|
214
|
+
tree = one_of(nil, branch)
|
215
|
+
branch[:left] = tree
|
216
|
+
branch[:right] = tree
|
217
|
+
|
218
|
+
This pattern will work as the one declared using `cyclic` above. All
|
219
|
+
structural sharing and cycles are preserved in the the transformation
|
220
|
+
from a `Pattern` to a `Validation`.
|
221
|
+
|
222
|
+
### The members keyword
|
223
|
+
|
224
|
+
You may have noticed that, whenever we use a special keyword within a hash,
|
225
|
+
the value, that is represented as a hash, is treated as a list of object
|
226
|
+
members, rather than as an object. There is a special keyword called
|
227
|
+
`members`, which does nothing but apply this transformation. It is useful
|
228
|
+
if you have a pattern that represents an object, and want to flatten
|
229
|
+
it into a list of object members.
|
230
|
+
|
231
|
+
For example, if you have an object pattern:
|
232
|
+
|
233
|
+
address = { street: String, city: String }
|
234
|
+
|
235
|
+
You could use it as a nested object:
|
236
|
+
|
237
|
+
user = { name: String, address: address }
|
238
|
+
|
239
|
+
which would match JSON like this:
|
240
|
+
|
241
|
+
{ "name": "Bob", "address": { "street": "10 Forbes Ave", "city": "New York" } }
|
242
|
+
|
243
|
+
Using the `members` keyword, you can flatten the pattern:
|
244
|
+
|
245
|
+
user = { name: String, members => address }
|
246
|
+
|
247
|
+
This will match JSON such as:
|
248
|
+
|
249
|
+
{ "name": "Bob", "street": "10 Forbes Ave", "city": "New York" }
|
250
|
+
|
251
|
+
Note that the same pattern can be used in both ways, in different places. In
|
252
|
+
addition, the same flattening occurs if a pattern is used along with the
|
253
|
+
`one_of` or `optional` keywords.
|
254
|
+
|
255
|
+
## Errors
|
256
|
+
|
257
|
+
Errors are descended from the class `ValidationError`, which consists of subclasses
|
258
|
+
`ValidationUnexpected` and `ValidationAmbiguity`. Each of them has a `path`
|
259
|
+
attribute, which is an array containing the path through the JSON where the error
|
260
|
+
was found. Each element in the array is either a string (object name) or integer
|
261
|
+
(array index).
|
262
|
+
|
263
|
+
A `ValidationUnexpected` has `found` and `expected` attributes, describing what was
|
264
|
+
found and what was expected, respectively. The description is minimal for indicating
|
265
|
+
what went wrong. For example, if a string was expected but an array was found, then
|
266
|
+
`found` would indicate "array", rather than representing the entire contents of the
|
267
|
+
array. If alternate values were expected, they will be listed in an array, e.g.:
|
268
|
+
|
269
|
+
expected: ["square", "circle", "triangle"]
|
270
|
+
found: "dodecahedron"
|
271
|
+
|
272
|
+
If a certain name was expected, but not found, all the names found in the object will
|
273
|
+
be listed. For example,
|
274
|
+
|
275
|
+
expected: 'name: "street"'
|
276
|
+
found: 'names: "name", "city"'
|
277
|
+
|
278
|
+
A `ValidationAmbiguity` has `found` and `overlapping_patterns` attributes. The
|
279
|
+
`found` attribute describes what was found, and `overlapping_patterns` is an array
|
280
|
+
of minimal textual descriptions of the alternate patterns that matched what was
|
281
|
+
found.
|
282
|
+
|
283
|
+
All errors have a `to_s` method for convenient textual representation.
|
284
|
+
|
285
|
+
== Questions and Answers
|
286
|
+
|
287
|
+
**Q:** Why only Ruby 1.9? Can you support 1.8?
|
288
|
+
|
289
|
+
**A:** There are several reasons for using Ruby 1.9, but the most important
|
290
|
+
one is that there is a guaranteed ordering of key-value pairs in hashes. This
|
291
|
+
is required for the patterns to be ordered. The JSON-like key syntax of following
|
292
|
+
the key with a colon instead of using a rocket is nice, too.
|
293
|
+
|
294
|
+
**Q:** I'd like to match arrays of a fixed length, with different patterns at each
|
295
|
+
index. Is that possible?
|
296
|
+
|
297
|
+
**A:** Not at the moment. I may add this in a future version. The syntax would
|
298
|
+
simply be the Ruby array syntax. I just haven't had a use for this yet.
|
299
|
+
|
300
|
+
**Q:** How about checking the length of a string?
|
301
|
+
|
302
|
+
**A:** You can use a Regexp. `/.{8,8}` will match 8-character strings.
|
303
|
+
|
304
|
+
**Q:** How about checking a numeric range?
|
305
|
+
|
306
|
+
**A:** Not possible in this version.
|
307
|
+
|
308
|
+
**Q:** I want to use arbitrary functions to validate values.
|
309
|
+
|
310
|
+
**A:** Not possible at the moment, but I might consider adding this. One thing
|
311
|
+
I don't like about it is that it's not serializable.
|
312
|
+
|
313
|
+
**Q:** Your `one_of` keyword is like an `or`. How about a sort of `and` operation
|
314
|
+
on patterns?
|
315
|
+
|
316
|
+
**A:** Yes, I'm considering creating an `all_of` for that. It's too bad you can't
|
317
|
+
overload the `&&` and `||` operators in Ruby.
|
318
|
+
|
319
|
+
**Q:** How about using regexes for keys?
|
320
|
+
|
321
|
+
**A:** This creates too much ambiguity. The number of cases that must be tried
|
322
|
+
is larger, and the error messages become more confusing. Better to be clear
|
323
|
+
with your structure definition.
|
324
|
+
|
325
|
+
Instead I have special cased the `/.*/` regex using the `many` keyword. With
|
326
|
+
this there is no ambiguity about which names should be matched. All are matched.
|
327
|
+
|
328
|
+
**Q:** What is this "name" thing? Do you mean keys?
|
329
|
+
|
330
|
+
**A:** See the [JSON specification](http://www.json.org/). They really do call them
|
331
|
+
object members, and name/value pairs. Just trying to stick with the spec. Sorry
|
332
|
+
for the confusion.
|
333
|
+
|
334
|
+
**Q:** But you can't do mutually recursive patterns with `cyclic`!
|
335
|
+
|
336
|
+
**A:** OK, smartypants. I didn't have a use for mutually recursive patterns.
|
337
|
+
But it is possible to write them, by manipulating the patterns after you
|
338
|
+
create them, to point at each other. The magic `cyclic` keyword isn't really
|
339
|
+
required. The Pattern -> Validation transformation preserves all sharing
|
340
|
+
and cyclic structures. Exercise left for the daring reader.
|
341
|
+
|
342
|
+
**Q:** If there's an ambiguity error, it can be hard to find where in the pattern
|
343
|
+
the problem is occuring.
|
344
|
+
|
345
|
+
**A:** At some point I may implement pattern paths. They won't be aware of the
|
346
|
+
Ruby variables you've used, but it could help.
|
347
|
+
|
348
|
+
## Acknowledgements
|
349
|
+
|
350
|
+
Thanks to [Janrain](http://janrain.com), my employer, for permitting me to open source
|
351
|
+
this code under my own name.
|
352
|
+
|
353
|
+
## Support
|
354
|
+
|
355
|
+
Contact lyle@kopnicky.com for help with this gem.
|
356
|
+
|
357
|
+
Copyright (c) 2012 Lyle Kopnicky under the BSD2 license.
|
metadata
CHANGED
@@ -1,48 +1,66 @@
|
|
1
|
-
--- !ruby/object:Gem::Specification
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
2
|
name: json_patterns
|
3
|
-
version: !ruby/object:Gem::Version
|
4
|
-
|
5
|
-
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
prerelease: false
|
5
|
+
segments:
|
6
|
+
- 0
|
7
|
+
- 1
|
8
|
+
- 2
|
9
|
+
version: 0.1.2
|
6
10
|
platform: ruby
|
7
|
-
authors:
|
11
|
+
authors:
|
8
12
|
- Lyle Kopnicky
|
9
13
|
autorequire:
|
10
14
|
bindir: bin
|
11
15
|
cert_chain: []
|
12
|
-
|
16
|
+
|
17
|
+
date: 2012-10-11 00:00:00 -07:00
|
18
|
+
default_executable:
|
13
19
|
dependencies: []
|
14
|
-
|
15
|
-
|
20
|
+
|
21
|
+
description: Validate patterns in JSON using a domain-specific language that looks as much as possible like the JSON you are trying to validate.
|
16
22
|
email: lyle@kopnicky.com
|
17
23
|
executables: []
|
24
|
+
|
18
25
|
extensions: []
|
26
|
+
|
19
27
|
extra_rdoc_files: []
|
20
|
-
|
28
|
+
|
29
|
+
files:
|
30
|
+
- README.md
|
21
31
|
- lib/json_patterns.rb
|
22
32
|
- test/test_json_patterns.rb
|
33
|
+
has_rdoc: true
|
23
34
|
homepage: http://github.com/lylek/json-patterns-ruby
|
24
35
|
licenses: []
|
36
|
+
|
25
37
|
post_install_message:
|
26
38
|
rdoc_options: []
|
27
|
-
|
39
|
+
|
40
|
+
require_paths:
|
28
41
|
- lib
|
29
|
-
required_ruby_version: !ruby/object:Gem::Requirement
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
42
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
43
|
+
requirements:
|
44
|
+
- - ">="
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
segments:
|
47
|
+
- 1
|
48
|
+
- 9
|
49
|
+
- 0
|
34
50
|
version: 1.9.0
|
35
|
-
required_rubygems_version: !ruby/object:Gem::Requirement
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
51
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
52
|
+
requirements:
|
53
|
+
- - ">="
|
54
|
+
- !ruby/object:Gem::Version
|
55
|
+
segments:
|
56
|
+
- 0
|
57
|
+
version: "0"
|
41
58
|
requirements: []
|
59
|
+
|
42
60
|
rubyforge_project:
|
43
|
-
rubygems_version: 1.
|
61
|
+
rubygems_version: 1.3.6
|
44
62
|
signing_key:
|
45
63
|
specification_version: 3
|
46
64
|
summary: A DSL of patterns to validate JSON structure
|
47
|
-
test_files:
|
65
|
+
test_files:
|
48
66
|
- test/test_json_patterns.rb
|