corvus_json_schema 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Cargo.lock +13 -12
- data/README.md +12 -1
- data/THIRD-PARTY-NOTICES.txt +96 -0
- data/VERSIONHISTORY.md +33 -0
- data/ext/corvus_json_schema/Cargo.toml +2 -2
- data/ext/corvus_json_schema/clippy.toml +3 -4
- data/ext/corvus_json_schema/src/lib.rs +70 -20
- data/ext/corvus_json_schema/vendor/corvus-json-schema/Cargo.toml +2 -2
- data/ext/corvus_json_schema/vendor/corvus-json-schema/README.md +68 -4
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/compiler.rs +91 -97
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/document.rs +2 -2
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/eval/plan/fused.rs +13 -9
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/eval/plan.rs +12 -11
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/eval.rs +105 -73
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/formats.rs +102 -7
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/lib.rs +165 -30
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/loader.rs +19 -21
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/numbers.rs +1 -1
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/options.rs +211 -4
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/pattern.rs +635 -586
- data/ext/corvus_json_schema/vendor/corvus-json-schema/src/uri.rs +4 -4
- data/lib/corvus_json_schema/version.rb +1 -1
- data/lib/corvus_json_schema.rb +18 -3
- metadata +3 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7e2347892c4bd0c1e41ef9b14747b260c5938d47f1a644f1fdbe0c3992f822a1
|
|
4
|
+
data.tar.gz: 78e9eda166cac2e81b48f87927299d2c092f45b1269179c58ed65495e9cf278f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b5d369d4df79459748f2eeaf93a1586986ab37d8d112e3095ca22650c0144ca110b6f0d85559c0f4cd82b704ba1bcee3117813785fa796d120e643f500228253
|
|
7
|
+
data.tar.gz: 9fca29ee3cdf761a2caeebf80a6f8c83f788957d346c0d66fdd1d7af2735c439b6b9a0f7161e5e570a07aef2f91075f2d9077aa0785fd3ecaa194a4642695de2
|
data/Cargo.lock
CHANGED
|
@@ -63,16 +63,26 @@ dependencies = [
|
|
|
63
63
|
|
|
64
64
|
[[package]]
|
|
65
65
|
name = "corvus-json-schema"
|
|
66
|
-
version = "0.
|
|
66
|
+
version = "0.2.0"
|
|
67
67
|
dependencies = [
|
|
68
|
+
"corvus-regex-ecma262",
|
|
68
69
|
"regex",
|
|
69
|
-
"regress",
|
|
70
70
|
"serde_json",
|
|
71
71
|
]
|
|
72
72
|
|
|
73
|
+
[[package]]
|
|
74
|
+
name = "corvus-regex-ecma262"
|
|
75
|
+
version = "0.2.7"
|
|
76
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
77
|
+
checksum = "be72de82781404398482a431a88c1d03b82d474a5fad3003cac18f5893fc71ee"
|
|
78
|
+
dependencies = [
|
|
79
|
+
"aho-corasick",
|
|
80
|
+
"memchr",
|
|
81
|
+
]
|
|
82
|
+
|
|
73
83
|
[[package]]
|
|
74
84
|
name = "corvus_json_schema"
|
|
75
|
-
version = "0.
|
|
85
|
+
version = "0.2.0"
|
|
76
86
|
dependencies = [
|
|
77
87
|
"corvus-json-schema",
|
|
78
88
|
"magnus",
|
|
@@ -273,15 +283,6 @@ version = "0.8.11"
|
|
|
273
283
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
274
284
|
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
|
|
275
285
|
|
|
276
|
-
[[package]]
|
|
277
|
-
name = "regress"
|
|
278
|
-
version = "0.12.0"
|
|
279
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
280
|
-
checksum = "32eef8b209c3c1c15dbad02c1f30f9539f00dc7253e0cbcdaae442a50a09d7c1"
|
|
281
|
-
dependencies = [
|
|
282
|
-
"memchr",
|
|
283
|
-
]
|
|
284
|
-
|
|
285
286
|
[[package]]
|
|
286
287
|
name = "rustc-hash"
|
|
287
288
|
version = "2.1.3"
|
data/README.md
CHANGED
|
@@ -53,6 +53,9 @@ Its options:
|
|
|
53
53
|
| `base_uri` | `nil` | The schema's base URI, when it has no `$id`. |
|
|
54
54
|
| `entry_point` | `nil` | A reference to the subschema to validate against, such as `"#/$defs/item"`. |
|
|
55
55
|
| `max_depth` | `128` | The deepest the evaluator recurses in place before it raises `DepthError`. |
|
|
56
|
+
| `pattern_match_limit` | `1.0` | The seconds one match of a `pattern` or `patternProperties` regular expression against one string may take before the validation raises `PatternMatchLimitError`. `nil` or `0` turns the limit off, for a caller who trusts both the schema and the instances. |
|
|
57
|
+
| `regex_edition` | `:dialect` | The edition of ECMA-262 `pattern`, `patternProperties` and the `regex` format are read with: `:dialect` (ECMAScript 2020 for 2019-09 and 2020-12, which name it, today's for the drafts before them), `:current` or `:es2020`. Under ECMAScript 2020 a modifier group such as `(?i:...)` is not valid. |
|
|
58
|
+
| `regex_unicode_version` | `:default` | The version of Unicode patterns are read with: `:default` (the version of the V8 the answers are held to, 17 today), `:latest`, `13`, `17` or `18`. |
|
|
56
59
|
|
|
57
60
|
A `Validator` is immutable and can be shared between threads.
|
|
58
61
|
|
|
@@ -65,7 +68,10 @@ A `Validator` is immutable and can be shared between threads.
|
|
|
65
68
|
schema location.
|
|
66
69
|
|
|
67
70
|
Errors are subclasses of `CorvusJsonSchema::Error`: `CompilationError` (an invalid schema, or a reference that cannot
|
|
68
|
-
be resolved), `
|
|
71
|
+
be resolved), `PatternTooLargeError` (a `CompilationError` for a regular expression over a limit of size: 1,048,576
|
|
72
|
+
bytes, 1,000 levels of groups, or about 16,000 terms), `DepthError`, `PatternMatchLimitError` (a match of a regular
|
|
73
|
+
expression reached its limit, so the instance is neither valid nor not valid) and `InvalidJsonError` (for
|
|
74
|
+
`valid_json?`). An error raised by a format or resolver callable
|
|
69
75
|
propagates.
|
|
70
76
|
|
|
71
77
|
## Values
|
|
@@ -109,3 +115,8 @@ repository's submodule (or `JSON_SCHEMA_TEST_SUITE`).
|
|
|
109
115
|
## License
|
|
110
116
|
|
|
111
117
|
Apache-2.0
|
|
118
|
+
|
|
119
|
+
The gem is built on the `corvus-json-schema` Rust crate, which matches patterns with the `corvus-regex-ecma262` crate,
|
|
120
|
+
whose Unicode tables are derived from the Unicode Character Database, which is under the Unicode License V3.
|
|
121
|
+
[THIRD-PARTY-NOTICES.txt](THIRD-PARTY-NOTICES.txt) has that notice, and the one of the `regress` crate, from whose
|
|
122
|
+
tables those were written. The file is in the source gem and in every native gem, beside `LICENSE`.
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
corvus_json_schema uses third-party libraries or other resources that may be
|
|
2
|
+
distributed under licenses different than the corvus_json_schema software.
|
|
3
|
+
|
|
4
|
+
In the event that we accidentally failed to list a required notice, please
|
|
5
|
+
bring it to our attention. Post an issue or email us:
|
|
6
|
+
|
|
7
|
+
licensing@endjin.com
|
|
8
|
+
|
|
9
|
+
The attached notices are provided for information only.
|
|
10
|
+
|
|
11
|
+
License notice for the Unicode Character Database
|
|
12
|
+
-------------------------------
|
|
13
|
+
|
|
14
|
+
The gem is built on the corvus-json-schema Rust crate. The source gem holds
|
|
15
|
+
the crate's sources (ext/corvus_json_schema/vendor/corvus-json-schema), and a
|
|
16
|
+
native gem holds the extension compiled from them. That crate matches patterns
|
|
17
|
+
with the corvus-regex-ecma262 crate, which is compiled in with it.
|
|
18
|
+
corvus-regex-ecma262 holds tables derived from the data files of the Unicode
|
|
19
|
+
Character Database, versions 13.0.0, 17.0.0 and 18.0.0
|
|
20
|
+
(https://www.unicode.org/Public/), and from its emoji sequence files: the
|
|
21
|
+
General_Category values, the binary properties, the Script and
|
|
22
|
+
Script_Extensions values, the simple case folding, the simple uppercase
|
|
23
|
+
mapping and the properties of strings. The data files are licensed under the
|
|
24
|
+
Unicode License V3 (SPDX-License-Identifier: Unicode-3.0), as
|
|
25
|
+
https://www.unicode.org/license.txt gives it:
|
|
26
|
+
|
|
27
|
+
UNICODE LICENSE V3
|
|
28
|
+
|
|
29
|
+
COPYRIGHT AND PERMISSION NOTICE
|
|
30
|
+
|
|
31
|
+
Copyright © 1991-2026 Unicode, Inc.
|
|
32
|
+
|
|
33
|
+
NOTICE TO USER: Carefully read the following legal agreement. BY
|
|
34
|
+
DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
|
|
35
|
+
SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
|
|
36
|
+
TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
|
|
37
|
+
DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.
|
|
38
|
+
|
|
39
|
+
Permission is hereby granted, free of charge, to any person obtaining a
|
|
40
|
+
copy of data files and any associated documentation (the "Data Files") or
|
|
41
|
+
software and any associated documentation (the "Software") to deal in the
|
|
42
|
+
Data Files or Software without restriction, including without limitation
|
|
43
|
+
the rights to use, copy, modify, merge, publish, distribute, and/or sell
|
|
44
|
+
copies of the Data Files or Software, and to permit persons to whom the
|
|
45
|
+
Data Files or Software are furnished to do so, provided that either (a)
|
|
46
|
+
this copyright and permission notice appear with all copies of the Data
|
|
47
|
+
Files or Software, or (b) this copyright and permission notice appear in
|
|
48
|
+
associated Documentation.
|
|
49
|
+
|
|
50
|
+
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
|
|
51
|
+
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
52
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
|
|
53
|
+
THIRD PARTY RIGHTS.
|
|
54
|
+
|
|
55
|
+
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
|
|
56
|
+
BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
|
|
57
|
+
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
|
|
58
|
+
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
|
|
59
|
+
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
|
|
60
|
+
FILES OR SOFTWARE.
|
|
61
|
+
|
|
62
|
+
Except as contained in this notice, the name of a copyright holder shall
|
|
63
|
+
not be used in advertising or otherwise to promote the sale, use or other
|
|
64
|
+
dealings in these Data Files or Software without prior written
|
|
65
|
+
authorization of the copyright holder.
|
|
66
|
+
|
|
67
|
+
License notice for https://github.com/ridiculousfish/regress
|
|
68
|
+
-------------------------------
|
|
69
|
+
|
|
70
|
+
The tables of the corvus-regex-ecma262 crate were written, through the tables
|
|
71
|
+
of the Corvus Go module, from the property tables of regress (its gen-unicode
|
|
72
|
+
tool), which are generated from the Unicode Character Database. regress is
|
|
73
|
+
licensed under the MIT license or the Apache License, Version 2.0, at the
|
|
74
|
+
user's option. Neither the gem nor the crates it is built on depend on it.
|
|
75
|
+
|
|
76
|
+
MIT License
|
|
77
|
+
|
|
78
|
+
Copyright (c) 2020 ridiculous_fish
|
|
79
|
+
|
|
80
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
81
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
82
|
+
in the Software without restriction, including without limitation the rights
|
|
83
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
84
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
85
|
+
furnished to do so, subject to the following conditions:
|
|
86
|
+
|
|
87
|
+
The above copyright notice and this permission notice shall be included in all
|
|
88
|
+
copies or substantial portions of the Software.
|
|
89
|
+
|
|
90
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
91
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
92
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
93
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
94
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
95
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
96
|
+
SOFTWARE.
|
data/VERSIONHISTORY.md
CHANGED
|
@@ -2,6 +2,39 @@
|
|
|
2
2
|
|
|
3
3
|
The version history of the `corvus_json_schema` Ruby gem. It is versioned independently of the Corvus NuGet packages and of the [corvus-json-schema](../../src-rs/corvus-json-schema/VERSIONHISTORY.md) Rust crate it is built on.
|
|
4
4
|
|
|
5
|
+
## V0.2.0
|
|
6
|
+
|
|
7
|
+
V0.2.0 takes version 0.2.0 of the `corvus-json-schema` crate, which matches patterns with the [`corvus-regex-ecma262`](https://crates.io/crates/corvus-regex-ecma262) crate and reads them in the grammar of the dialect. It is a minor version because under 2019-09 and 2020-12 a pattern that uses what ECMAScript added after 2020 is no longer valid. Everything else that was there is unchanged, and there are two new options.
|
|
8
|
+
|
|
9
|
+
- **The grammar of the dialect.** JSON Schema 2019-09 and 2020-12 name the grammar of ECMAScript 2020, and patterns under them are now read with it: a modifier group such as `(?i:...)`, or a group name used by two groups, is not valid, and `CorvusJsonSchema.compile` raises `CompilationError` for a schema with one in a `pattern`. A property escape that the tables of 2020 do not have (`\p{Script=Yezidi}`) is not valid with the `u` flag, so the pattern is read without it, where `\p` is the letter `p`, as a pattern with `\&` is. The `regex` format reads only the grammar of the flag. Drafts 4 to 7 read today's grammar, as before.
|
|
10
|
+
- **`regex_edition:`** (`:dialect`, the default, `:current` or `:es2020`) chooses the edition for every dialect, and **`regex_unicode_version:`** (`:default`, the default, `:latest`, `13`, `17` or `18`) the version of Unicode. Another value raises `ArgumentError`.
|
|
11
|
+
- **The engine.** Every pattern that no dedicated matcher of the crate takes is matched by `corvus-regex-ecma262`, whose answers are V8's for the whole of the ECMA-262 regular expression test data. The limits are those of 0.1.4. Over the jsonschema-benchmark corpora the crate validates in 0.98 of the time of 0.1.7 once warm, and takes 1.06 of it for the first validation and 1.05 to compile, since the engine builds a pattern's automaton on its first match.
|
|
12
|
+
- **Notices.** `THIRD-PARTY-NOTICES.txt` names the tables of `corvus-regex-ecma262`, which the native gems now hold in place of the crate's own. The source gem no longer carries a copy of the crate's notice, which the crate no longer has.
|
|
13
|
+
|
|
14
|
+
## V0.1.4
|
|
15
|
+
|
|
16
|
+
V0.1.4 takes version 0.1.7 of the `corvus-json-schema` crate. That fixes wrong results for several forms of pattern and a validation that did not end, replaces the crate's regular expression engine, and gives every match of a pattern a limit of time. The functions and types that were there are unchanged, and there are additions for the limit. Versions 0.1.0 to 0.1.3 are affected.
|
|
17
|
+
|
|
18
|
+
### The regular expression engine
|
|
19
|
+
|
|
20
|
+
The crate no longer uses the `regress` crate for the patterns with a lookaround, a backreference or a modifier group. It matches them with an engine of its own, a port of the ECMA-262 engine of the Corvus Go module, and matches every pattern as it is written. The crate's [version history](../../src-rs/corvus-json-schema/VERSIONHISTORY.md) lists the patterns that were answered wrongly before (a repetition inside a repetition that lost its count or did not end, a backreference from inside its group, two groups with one name, case-insensitive matching without the `u` flag, a pattern that is valid only without the `u` flag and a character beyond the Basic Multilingual Plane, `\B` beside a character of several bytes, and others), and what was measured: V8's answer for all 4,268,313 texts of the ECMA-262 regular expression test data that can be asked.
|
|
21
|
+
|
|
22
|
+
### Limits on regular expressions
|
|
23
|
+
|
|
24
|
+
A schema and an instance may both come from outside, so a pattern is now bounded in what it costs. Nothing that was there has changed. There is one new option and there are two new errors.
|
|
25
|
+
|
|
26
|
+
- **A match has a limit of one second.** `pattern_match_limit`, a keyword of `CorvusJsonSchema.compile`, is the seconds one match of a `pattern` or `patternProperties` regular expression against one string may take. The default is `CorvusJsonSchema::DEFAULT_PATTERN_MATCH_LIMIT`, `1.0`. `nil` or `0` turns the limit off for a caller who trusts both the schema and the instances. The limit is on each match, not on the validation.
|
|
27
|
+
- **`CorvusJsonSchema::PatternMatchLimitError`** is raised by `valid?`, `valid_json?` and `evaluate` when a match reached the limit, or when its backtracking held 1,048,576 entries. The instance is then neither valid nor not valid. The message names the pattern and the limit. `compile` can raise it too, because a pattern of `patternProperties` is matched against the property names the schema declares.
|
|
28
|
+
- **`CorvusJsonSchema::PatternTooLargeError`** is raised by `compile` for a schema with a regular expression over a limit of size: 1,048,576 bytes of pattern, 1,000 levels of groups, or 1,048,576 counted for it as it is read (a pattern of plain characters can be 16,384 characters long). It is a `CompilationError`, so code that rescues those still does. A regular expression that is not valid raises `CompilationError` itself, as before.
|
|
29
|
+
|
|
30
|
+
### Bug fixes
|
|
31
|
+
|
|
32
|
+
- **A separated list with a `$` inside it.** The crate matches a pattern of the form `^item(separator item)*$`, such as `^([a-z]+)(\.[a-z]+)*$`, without a regular expression engine. It dropped a `$` at the end of the first item or of the repeated group, so `^x(,a$)*$` was matched as `^x(,a)*$`. A `$` holds only at the end of the string, so the group of `^x(,a$)*$` can match once at most. `CorvusJsonSchema.compile({ "pattern" => '^x(,a$)*$' }).valid?("x,a,a")` returned `true`, where the instance is not valid. `^x$(,a)*$` took `"x,a"`, and `^(a,$)*b$` took `"a,b"`. `valid?`, `valid_json?` and `evaluate` were all affected, and so was a pattern used for `patternProperties` or `propertyNames`. Such a pattern is now matched by a regular expression engine. An escaped `\$` is a literal character and is matched as before. No other form of pattern was affected. The extension's crate requirement names 0.1.7, so the copy of the crate vendored at build time cannot be an older one.
|
|
33
|
+
|
|
34
|
+
### Notices
|
|
35
|
+
|
|
36
|
+
- **The notice of the Unicode data is in the gem.** The Unicode tables of the `corvus-json-schema` crate are derived from the Unicode Character Database, which is under the Unicode License V3. That license asks that its notice goes with every copy of the data. `THIRD-PARTY-NOTICES.txt` has it, with the notice of the `regress` crate that the tables were first read from, and is now in the source gem and in every native gem. Versions 0.1.0 to 0.1.3 carried no such notice.
|
|
37
|
+
|
|
5
38
|
## V0.1.3
|
|
6
39
|
|
|
7
40
|
V0.1.3 fixes `gem install corvus_json_schema` on Alpine Linux and other musl systems, where RubyGems before 4.0 could install the glibc gem, which does not load there. There are no API changes, and the extension is the one in 0.1.2. Versions 0.1.0 to 0.1.2 are affected.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[package]
|
|
2
2
|
name = "corvus_json_schema"
|
|
3
|
-
version = "0.
|
|
3
|
+
version = "0.2.0"
|
|
4
4
|
edition = "2024"
|
|
5
5
|
rust-version = "1.89"
|
|
6
6
|
description = "The native extension of the corvus_json_schema gem: the corvus-json-schema crate, reading Ruby values in place."
|
|
@@ -12,7 +12,7 @@ crate-type = ["cdylib"]
|
|
|
12
12
|
|
|
13
13
|
[dependencies]
|
|
14
14
|
# The crate, copied in by `rake vendor` (from src-rs/corvus-json-schema) so that the source gem builds on its own.
|
|
15
|
-
corvus = { package = "corvus-json-schema", path = "vendor/corvus-json-schema", version = "0.
|
|
15
|
+
corvus = { package = "corvus-json-schema", path = "vendor/corvus-json-schema", version = "0.2.0" }
|
|
16
16
|
magnus = { version = "0.9.1", features = ["rb-sys"] }
|
|
17
17
|
rb-sys = "0.9.130"
|
|
18
18
|
serde_json = { version = "1.0.151", features = ["preserve_order", "float_roundtrip"] }
|
|
@@ -1,4 +1,3 @@
|
|
|
1
|
-
# Clippy suggests idioms by the minimum Rust version
|
|
2
|
-
#
|
|
3
|
-
|
|
4
|
-
msrv = "1.85"
|
|
1
|
+
# Clippy suggests idioms by the minimum Rust version, which is the one declared in Cargo.toml (rust-version). Keep the
|
|
2
|
+
# two the same: clippy warns when they differ, and uses this one.
|
|
3
|
+
msrv = "1.89"
|
|
@@ -4,7 +4,8 @@
|
|
|
4
4
|
use std::cell::{Cell, RefCell};
|
|
5
5
|
|
|
6
6
|
use corvus::{
|
|
7
|
-
ArrayView, CompileOptions, Dialect, Instance, JsonSchemaResultsCollector, Kind, ObjectView,
|
|
7
|
+
ArrayView, CompileOptions, Dialect, Instance, JsonSchemaResultsCollector, Kind, ObjectView, RegexEdition,
|
|
8
|
+
RegexUnicodeVersion, ResultsLevel, View,
|
|
8
9
|
};
|
|
9
10
|
use magnus::prelude::*;
|
|
10
11
|
use magnus::rb_sys::AsRawValue;
|
|
@@ -401,8 +402,14 @@ struct Validator {
|
|
|
401
402
|
ruby_formats: bool,
|
|
402
403
|
}
|
|
403
404
|
|
|
404
|
-
|
|
405
|
-
|
|
405
|
+
/// The exception for an evaluation that gave no answer: it recursed in place beyond the maximum depth, or a match
|
|
406
|
+
/// of a regular expression reached its limit.
|
|
407
|
+
fn evaluation_error(ruby: &Ruby, e: corvus::SchemaEvaluationError) -> Error {
|
|
408
|
+
match e {
|
|
409
|
+
corvus::SchemaEvaluationError::InvalidJson(e) => raise(ruby, "InvalidJsonError", e.to_string()),
|
|
410
|
+
corvus::SchemaEvaluationError::PatternMatchLimit(e) => raise(ruby, "PatternMatchLimitError", e.to_string()),
|
|
411
|
+
other => raise(ruby, "DepthError", other.to_string()),
|
|
412
|
+
}
|
|
406
413
|
}
|
|
407
414
|
|
|
408
415
|
impl Validator {
|
|
@@ -411,15 +418,15 @@ impl Validator {
|
|
|
411
418
|
fn check(&self, ruby: &Ruby, value: Value) -> Result<bool, Error> {
|
|
412
419
|
if !self.ruby_formats {
|
|
413
420
|
let ctx = Context::new();
|
|
414
|
-
let result = self.inner.
|
|
421
|
+
let result = self.inner.try_validate_instance(RbInstance::root(value, &ctx));
|
|
415
422
|
if !ctx.fallback.get() {
|
|
416
|
-
return result.map_err(|e|
|
|
423
|
+
return result.map_err(|e| evaluation_error(ruby, e));
|
|
417
424
|
}
|
|
418
425
|
}
|
|
419
426
|
let json = to_json(ruby, value, 0)?;
|
|
420
|
-
let result = self.inner.
|
|
427
|
+
let result = self.inner.try_validate(&json);
|
|
421
428
|
take_callback_error()?;
|
|
422
|
-
result.map_err(|e|
|
|
429
|
+
result.map_err(|e| evaluation_error(ruby, e))
|
|
423
430
|
}
|
|
424
431
|
|
|
425
432
|
fn valid(ruby: &Ruby, rb_self: &Self, value: Value) -> Result<bool, Error> {
|
|
@@ -435,13 +442,9 @@ impl Validator {
|
|
|
435
442
|
} else {
|
|
436
443
|
str_of(text).ok_or_else(|| raise(ruby, "InvalidJsonError", "the JSON text is not valid UTF-8"))?
|
|
437
444
|
};
|
|
438
|
-
let result = rb_self.inner.
|
|
445
|
+
let result = rb_self.inner.try_validate_json(json);
|
|
439
446
|
take_callback_error()?;
|
|
440
|
-
|
|
441
|
-
Ok(valid) => Ok(valid),
|
|
442
|
-
Err(corvus::JsonValidationError::InvalidJson(e)) => Err(raise(ruby, "InvalidJsonError", e.to_string())),
|
|
443
|
-
Err(corvus::JsonValidationError::DepthExceeded(e)) => Err(depth_error(ruby, e)),
|
|
444
|
-
}
|
|
447
|
+
result.map_err(|e| evaluation_error(ruby, e))
|
|
445
448
|
}
|
|
446
449
|
|
|
447
450
|
/// Evaluates into the collector (replacing its results), reporting every keyword at its level.
|
|
@@ -449,18 +452,18 @@ impl Validator {
|
|
|
449
452
|
let mut c = JsonSchemaResultsCollector::new(collector.level);
|
|
450
453
|
if !rb_self.ruby_formats {
|
|
451
454
|
let ctx = Context::new();
|
|
452
|
-
let result = rb_self.inner.
|
|
455
|
+
let result = rb_self.inner.try_evaluate_instance(RbInstance::root(value, &ctx), &mut c);
|
|
453
456
|
if !ctx.fallback.get() {
|
|
454
457
|
*collector.inner.borrow_mut() = c;
|
|
455
|
-
return result.map_err(|e|
|
|
458
|
+
return result.map_err(|e| evaluation_error(ruby, e));
|
|
456
459
|
}
|
|
457
460
|
c = JsonSchemaResultsCollector::new(collector.level);
|
|
458
461
|
}
|
|
459
462
|
let json = to_json(ruby, value, 0)?;
|
|
460
|
-
let result = rb_self.inner.
|
|
463
|
+
let result = rb_self.inner.try_evaluate(&json, &mut c);
|
|
461
464
|
take_callback_error()?;
|
|
462
465
|
*collector.inner.borrow_mut() = c;
|
|
463
|
-
result.map_err(|e|
|
|
466
|
+
result.map_err(|e| evaluation_error(ruby, e))
|
|
464
467
|
}
|
|
465
468
|
}
|
|
466
469
|
|
|
@@ -478,6 +481,9 @@ fn compile(
|
|
|
478
481
|
base_uri: Option<String>,
|
|
479
482
|
entry_point: Option<String>,
|
|
480
483
|
max_depth: u32,
|
|
484
|
+
pattern_match_limit: Option<f64>,
|
|
485
|
+
regex_edition: String,
|
|
486
|
+
regex_unicode_version: String,
|
|
481
487
|
) -> Result<Validator, Error> {
|
|
482
488
|
let schema: Json = match RString::from_value(schema) {
|
|
483
489
|
Some(text) => serde_json::from_str(&text.to_string()?)
|
|
@@ -499,6 +505,41 @@ fn compile(
|
|
|
499
505
|
base_uri,
|
|
500
506
|
entry_point,
|
|
501
507
|
max_depth,
|
|
508
|
+
// In seconds. nil, zero or less turns the limit off.
|
|
509
|
+
pattern_match_limit: match pattern_match_limit {
|
|
510
|
+
Some(seconds) if seconds.is_nan() => {
|
|
511
|
+
return Err(Error::new(ruby.exception_arg_error(), "pattern_match_limit is not a number of seconds"));
|
|
512
|
+
}
|
|
513
|
+
Some(seconds) if seconds > 0.0 => {
|
|
514
|
+
Some(std::time::Duration::try_from_secs_f64(seconds).unwrap_or(std::time::Duration::MAX))
|
|
515
|
+
.filter(|limit| !limit.is_zero())
|
|
516
|
+
}
|
|
517
|
+
_ => None,
|
|
518
|
+
},
|
|
519
|
+
regex_edition: match regex_edition.as_str() {
|
|
520
|
+
"dialect" => None,
|
|
521
|
+
"current" => Some(RegexEdition::Current),
|
|
522
|
+
"es2020" => Some(RegexEdition::Es2020),
|
|
523
|
+
e => {
|
|
524
|
+
return Err(Error::new(
|
|
525
|
+
ruby.exception_arg_error(),
|
|
526
|
+
format!("regex_edition is :dialect, :current or :es2020, not {e:?}"),
|
|
527
|
+
));
|
|
528
|
+
}
|
|
529
|
+
},
|
|
530
|
+
regex_unicode_version: match regex_unicode_version.as_str() {
|
|
531
|
+
"default" => RegexUnicodeVersion::Default,
|
|
532
|
+
"latest" => RegexUnicodeVersion::Latest,
|
|
533
|
+
"13" => RegexUnicodeVersion::V13,
|
|
534
|
+
"17" => RegexUnicodeVersion::V17,
|
|
535
|
+
"18" => RegexUnicodeVersion::V18,
|
|
536
|
+
v => {
|
|
537
|
+
return Err(Error::new(
|
|
538
|
+
ruby.exception_arg_error(),
|
|
539
|
+
format!("regex_unicode_version is :default, :latest, 13, 17 or 18, not {v:?}"),
|
|
540
|
+
));
|
|
541
|
+
}
|
|
542
|
+
},
|
|
502
543
|
..CompileOptions::default()
|
|
503
544
|
};
|
|
504
545
|
let mut ruby_formats = false;
|
|
@@ -560,7 +601,12 @@ fn compile(
|
|
|
560
601
|
Some(r) => format!("{} ({r})", e.message()),
|
|
561
602
|
None => e.message().to_owned(),
|
|
562
603
|
};
|
|
563
|
-
|
|
604
|
+
let class = match e.kind() {
|
|
605
|
+
corvus::SchemaCompilationErrorKind::PatternTooLarge(_) => "PatternTooLargeError",
|
|
606
|
+
corvus::SchemaCompilationErrorKind::PatternMatchLimit(_) => "PatternMatchLimitError",
|
|
607
|
+
_ => "CompilationError",
|
|
608
|
+
};
|
|
609
|
+
Err(raise(ruby, class, message))
|
|
564
610
|
}
|
|
565
611
|
}
|
|
566
612
|
}
|
|
@@ -621,12 +667,16 @@ impl Collector {
|
|
|
621
667
|
fn init(ruby: &Ruby) -> Result<(), Error> {
|
|
622
668
|
let module = ruby.define_module("CorvusJsonSchema")?;
|
|
623
669
|
let error = module.define_error("Error", ruby.exception_standard_error())?;
|
|
624
|
-
module.define_error("CompilationError", error)?;
|
|
670
|
+
let compilation = module.define_error("CompilationError", error)?;
|
|
671
|
+
// A regular expression over a limit of size. It is a CompilationError, so code that rescues those still does.
|
|
672
|
+
module.define_error("PatternTooLargeError", compilation)?;
|
|
625
673
|
module.define_error("DepthError", error)?;
|
|
674
|
+
// A match of a regular expression that reached its limit: the instance is neither valid nor not valid.
|
|
675
|
+
module.define_error("PatternMatchLimitError", error)?;
|
|
626
676
|
module.define_error("InvalidJsonError", error)?;
|
|
627
677
|
|
|
628
678
|
let native = module.define_module("Native")?;
|
|
629
|
-
native.define_singleton_method("compile", function!(compile,
|
|
679
|
+
native.define_singleton_method("compile", function!(compile, 13))?;
|
|
630
680
|
native.define_singleton_method("crate_version", function!(|| corvus::VERSION, 0))?;
|
|
631
681
|
|
|
632
682
|
let validator = module.define_class("Validator", ruby.class_object())?;
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[package]
|
|
2
2
|
name = "corvus-json-schema"
|
|
3
|
-
version = "0.
|
|
3
|
+
version = "0.2.0"
|
|
4
4
|
edition = "2024"
|
|
5
5
|
rust-version = "1.89"
|
|
6
6
|
description = "A high-performance JSON Schema evaluator (draft 4 to 2020-12) with results collection and annotations, ported from the Corvus.Text.Json V5 runtime evaluator."
|
|
@@ -17,8 +17,8 @@ categories = ["encoding", "parser-implementations"]
|
|
|
17
17
|
include = ["src/**", "Cargo.toml", "README.md", "LICENSE"]
|
|
18
18
|
|
|
19
19
|
[dependencies]
|
|
20
|
+
corvus-regex-ecma262 = "0.2.7"
|
|
20
21
|
regex = "1.13.1"
|
|
21
|
-
regress = "0.12.0"
|
|
22
22
|
serde_json = { version = "1.0.151", features = ["preserve_order", "float_roundtrip"] }
|
|
23
23
|
|
|
24
24
|
[profile.release]
|
|
@@ -70,10 +70,63 @@ this way, through CPython's C API.
|
|
|
70
70
|
|
|
71
71
|
- **Numbers:** compared exactly. Integers are compared as integers and doubles against integers without rounding.
|
|
72
72
|
`multipleOf` is decided on decimal forms, so `0.0075` is a multiple of `0.0001`.
|
|
73
|
-
- **Patterns:** use ECMA-262 semantics with the `u` flag.
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
73
|
+
- **Patterns:** use ECMA-262 semantics. A pattern is compiled with the `u` flag. One that is not valid with it (it has
|
|
74
|
+
`\&` or `\-`, say) is compiled without it, as many validators accept such patterns, and is then matched as
|
|
75
|
+
ECMA-262 says for it: by UTF-16 code unit, so a character beyond the Basic Multilingual Plane is two. The `regex`
|
|
76
|
+
format accepts only the grammar of the flag, and asks the grammar alone, without the engine's limits of size. Common shapes (class sequences, a few literals and alternatives, line
|
|
77
|
+
lengths) get dedicated matchers, as in the C# evaluator, where each was measured faster than the engine over the
|
|
78
|
+
strings of the jsonschema-benchmark corpora. Every other pattern is matched by
|
|
79
|
+
[`corvus-regex-ecma262`](https://crates.io/crates/corvus-regex-ecma262), an ECMA-262 engine. The crate depends
|
|
80
|
+
on it, `regex` (for the format checks) and `serde_json`.
|
|
81
|
+
|
|
82
|
+
**What is exact.** Everything. A pattern is matched as it is written, whichever matcher has it, and none is
|
|
83
|
+
refused for what an engine cannot do. The engine's answers are V8's for the whole of the
|
|
84
|
+
[ECMA-262 regular expression test data](https://github.com/corvus-dotnet/ecma262-regex-test-data): lookaheads and
|
|
85
|
+
lookbehinds, backreferences into and out of repetitions and lookarounds, named groups, a name shared by groups in
|
|
86
|
+
separate alternatives, modifier groups, repetitions inside repetitions with any count, and case-insensitive
|
|
87
|
+
matching in both grammars.
|
|
88
|
+
|
|
89
|
+
**The grammar of the dialect.** JSON Schema 2019-09 and 2020-12 name the grammar of ECMAScript 2020, so under them a
|
|
90
|
+
pattern is read with `RegexEdition::Es2020`: what later editions added (modifier groups such as `(?i:...)`, a group
|
|
91
|
+
name used by two groups) is not valid, and a schema that uses it is refused. A property escape that is not in the
|
|
92
|
+
tables of 2020 is not valid with the `u` flag, so the pattern is read without it, where `\p` is the letter `p`. Drafts
|
|
93
|
+
4 to 7 read today's grammar. `CompileOptions::regex_edition` chooses another edition and
|
|
94
|
+
`CompileOptions::regex_unicode_version` the version of Unicode (by default the version of the V8 the engine's answers
|
|
95
|
+
are held to, 17 today), for `pattern`, `patternProperties` and the `regex` format alike.
|
|
96
|
+
|
|
97
|
+
**Limits.** A schema and an instance may both come from outside, so a pattern is bounded in what it costs.
|
|
98
|
+
|
|
99
|
+
- *A match has a limit of time.* `CompileOptions::pattern_match_limit` is the time one match against one string may
|
|
100
|
+
take. The default is one second (`DEFAULT_PATTERN_MATCH_LIMIT`), and `None` or zero turns it off for a caller who
|
|
101
|
+
trusts both the schema and the instances. A match that reaches it is abandoned, and so is one whose backtracking
|
|
102
|
+
runs out of the memory the engine allows it. The validation then has no answer: the `try_` functions
|
|
103
|
+
(`try_validate`, `try_validate_json`, `try_evaluate` and the rest) return
|
|
104
|
+
`SchemaEvaluationError::PatternMatchLimit` with the pattern and the limit.
|
|
105
|
+
- *The functions without the new error.* `is_valid` returns a `bool` and fails closed: it returns `false` for a
|
|
106
|
+
match that reached its limit, as it does for the maximum depth, and never `true`. `validate`,
|
|
107
|
+
`validate_instance`, `validate_json` and the `evaluate` functions report it as the one error their type has for
|
|
108
|
+
a validation that was abandoned (`SchemaEvaluationDepthError`, named when the depth was the one limit). Its
|
|
109
|
+
message says the validation was abandoned at a limit, the maximum depth or the limit of a match, and that the
|
|
110
|
+
`try_` functions say which. At a limit the instance is not known to be not valid, whatever `is_valid` returned.
|
|
111
|
+
- *A pattern has limits of size.* A pattern longer than 1,048,576 bytes, nested more than 1,000 groups deep, or
|
|
112
|
+
counted as more than 1,048,576 as it is read (64 for a term, 32 for an alternative, 8 for a range of a class) is
|
|
113
|
+
refused when the schema is compiled. The message says it is too large, not that it is not valid, and
|
|
114
|
+
`SchemaCompilationError::kind` is `PatternTooLarge`. A pattern of plain characters can be 16,384 characters long.
|
|
115
|
+
These are the engine's limits.
|
|
116
|
+
- *What the crate keeps is bounded.* The cache of compiled patterns of the process holds 1,024 patterns and 64 MiB
|
|
117
|
+
at most.
|
|
118
|
+
- *The format checks take time proportional to the string.* They read strings of the instance and are not under
|
|
119
|
+
the limit of a match. Dates, times, UUIDs, IP addresses and host names are read by scanners that pass over the
|
|
120
|
+
string once, and the others are regular expressions on the `regex` crate, which is linear. `regex` reads the
|
|
121
|
+
string with the engine's grammar-only reader, without the limits of size. `idn-hostname`, and the domain of an `idn-email`, read nothing of
|
|
122
|
+
a string over 1,012 bytes: an internationalized host name has 253 characters at most, and its check decodes and
|
|
123
|
+
encodes Punycode, which takes time that grows with the square of a label.
|
|
124
|
+
- *What is not bounded.* The limit is for each match, not for a validation. The matchers that read a string once
|
|
125
|
+
are not stopped. With the limit turned off, nothing bounds a match: a group read back inside a repetition, as in
|
|
126
|
+
`^(?:(a+)\1*)+$`, takes time that doubles with each character of some strings.
|
|
127
|
+
|
|
128
|
+
**Lone surrogates.** A JSON string with the escape of half of a surrogate pair (`"\ud800"`) is not a Rust string.
|
|
129
|
+
`validate_json` and `serde_json` both refuse the document as not valid JSON.
|
|
77
130
|
- **Formats:** are annotations unless asserted by the vocabulary or by `CompileOptions::assert_format`. The format
|
|
78
131
|
validators follow the C# implementations (RFC 3339 dates and times, IDN hostnames, IRIs, and so on).
|
|
79
132
|
|
|
@@ -137,3 +190,14 @@ This runs:
|
|
|
137
190
|
- **`tests/results.rs`:** the results-collection expectations shared with the C# and TypeScript evaluators.
|
|
138
191
|
- **`tests/metaschemas.rs`:** checks that `src/metaschemas.rs` matches `src/Corvus.Text.Json/metaschema`.
|
|
139
192
|
Regenerate it with `UPDATE_METASCHEMAS=1 cargo test --test metaschemas`.
|
|
193
|
+
- **The tests of `src/pattern.rs`:** every dedicated matcher agrees with the engine on strings made to probe it,
|
|
194
|
+
and the cache of compiled patterns stays within its bounds.
|
|
195
|
+
- **`tests/pattern_anchors.rs` and `tests/pattern_class.rs`:** patterns that an engine or a matcher answered wrongly
|
|
196
|
+
at some point, with V8's answers.
|
|
197
|
+
- **`tests/pattern_limits.rs`:** the limit of a match and the limits of size, through the public functions.
|
|
198
|
+
- **`tests/allocations.rs`:** validating JSON text allocates nothing in the steady state, with patterns the engine
|
|
199
|
+
matches and with the limit on or off.
|
|
200
|
+
|
|
201
|
+
## License
|
|
202
|
+
|
|
203
|
+
Apache 2.0.
|