protovalidate 0.1.0.beta1 → 0.1.0.beta3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitattributes +12 -0
- data/CHANGELOG.md +10 -0
- data/README.md +22 -2
- data/Rakefile +5 -0
- data/ext/protovalidate/src/lib.rs +35 -2
- data/ext/protovalidate/sys/build.rs +81 -1
- data/ext/protovalidate/sys/shim/pv_shim.cc +55 -21
- data/ext/protovalidate/sys/shim/pv_shim.h +18 -0
- data/ext/protovalidate/sys/src/lib.rs +38 -0
- data/lib/protovalidate/validator.rb +78 -17
- data/lib/protovalidate/version.rb +1 -1
- data/lib/protovalidate.rb +19 -0
- data/sig/protovalidate/native.rbs +7 -0
- data/sig/protovalidate.rbs +22 -3
- metadata +3 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d67bc65a7ddf2e634a3ae981ae3a514da0bd11e19e07f6ad29be593670117646
|
|
4
|
+
data.tar.gz: 1037f9d79262c4426840e64a4688619d5a1f3dcb8c19d6f6059b9fd842947207
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cde29cd8ed0e2c74f77cbf2b18c8c6c8ee1d3caaa98f07cd505c642717db122b302aa4901cb2be72a6ef952880fe1d581416d250b4ed048e9502938956e16a80
|
|
7
|
+
data.tar.gz: d2b45c9e369b02e0d0649960e8524564de93e6edb3efa9a2b7a78bcf1e5a524e55342a32c75b4e9830abdbe26ee5dda4ec6f1b39694d42665ffdbfd2d2312b1f
|
data/.gitattributes
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
ext/protovalidate/sys/third_party/** linguist-vendored
|
|
2
|
+
|
|
3
|
+
# script/extract-native-sources
|
|
4
|
+
ext/protovalidate/sys/gen/** linguist-generated
|
|
5
|
+
ext/protovalidate/sys/filelists/*.txt linguist-generated
|
|
6
|
+
|
|
7
|
+
# rake proto:generate
|
|
8
|
+
lib/buf/validate/validate_pb.rb linguist-generated
|
|
9
|
+
spec/gen/** linguist-generated
|
|
10
|
+
|
|
11
|
+
# Compiled descriptor set of spec/proto
|
|
12
|
+
spec/fixtures/spec_messages.fdset linguist-generated
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.0.beta3 (2026-09-22)
|
|
4
|
+
|
|
5
|
+
- `Protovalidate.register` and `Validator#register` compile the rules of message types ahead of their first validation, so the compilation, which blocks concurrent validations, can run at boot instead of inside a request. `register_all` does the same for every loaded message class.
|
|
6
|
+
- Validating an already seen message type no longer takes a lock or allocates its type name on the Ruby side.
|
|
7
|
+
|
|
8
|
+
## 0.1.0.beta2 (2026-09-21)
|
|
9
|
+
|
|
10
|
+
- Precompiled gems for `x86_64-linux-musl`, `aarch64-linux-musl`, `arm-linux-gnu` and `x64-mingw-ucrt`.
|
|
11
|
+
- Precompiled Linux glibc gems now require glibc 2.34 or newer (was 2.28); older hosts fall back to the source gem.
|
|
12
|
+
|
|
3
13
|
## 0.1.0.beta1 (2026-09-20)
|
|
4
14
|
|
|
5
15
|
- Initial release: `Protovalidate.validate` / `collect_violations` backed by protovalidate-cc 1.2.0 (protovalidate 1.2.2).
|
data/README.md
CHANGED
|
@@ -14,8 +14,11 @@ Precompiled gems are published for:
|
|
|
14
14
|
|
|
15
15
|
| Platform | Requirements |
|
|
16
16
|
|---|---|
|
|
17
|
-
| `x86_64-linux-gnu`, `aarch64-linux-gnu` | glibc 2.
|
|
17
|
+
| `x86_64-linux-gnu`, `aarch64-linux-gnu` | glibc 2.34 or newer |
|
|
18
|
+
| `arm-linux-gnu` (ARMv7, hard float) | glibc 2.31 or newer |
|
|
19
|
+
| `x86_64-linux-musl`, `aarch64-linux-musl` | musl libc, e.g. Alpine Linux |
|
|
18
20
|
| `arm64-darwin` | macOS 11 or newer |
|
|
21
|
+
| `x64-mingw-ucrt` | RubyInstaller (UCRT) Ruby |
|
|
19
22
|
|
|
20
23
|
Each covers Ruby 3.3, 3.4 and 4.0. On other platforms or Ruby versions the source gem is built at install time, which needs a Rust toolchain (1.85+), a C++17 compiler and a few minutes: it compiles protovalidate-cc, cel-cpp, abseil, protobuf, re2 and the ANTLR runtime.
|
|
21
24
|
|
|
@@ -74,6 +77,23 @@ All inherit from `Protovalidate::Error` except `ArgumentError`.
|
|
|
74
77
|
|
|
75
78
|
A `Validator` compiles the rules of each message type once and caches them, so reuse one instance. Validating from several threads at once is supported and releases the GVL while protovalidate-cc runs. Ractors are not supported.
|
|
76
79
|
|
|
80
|
+
The rules of a message type are compiled the first time a message of that type is validated, and protovalidate-cc holds an exclusive lock while it compiles, so every concurrent validation on the same validator waits until the compilation finishes. To keep that off the request path, register the message types your application validates at boot, for example from a Rails initializer or before the server forks workers:
|
|
81
|
+
|
|
82
|
+
```ruby
|
|
83
|
+
Protovalidate.register(Example::User, Example::Order)
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
`Protovalidate.register_all` registers every message class loaded so far instead of a hand-maintained list, so call it once all generated code has been loaded. In a Rails application, `config.after_initialize` callbacks run in the `finisher_hook` initializer, after the `eager_load!` initializer, so with `config.eager_load` enabled every generated class under an autoload path is already loaded by then. Generated code outside the autoload paths, such as under `lib/`, must be required before the callback runs.
|
|
87
|
+
|
|
88
|
+
```ruby
|
|
89
|
+
# config/initializers/protovalidate.rb
|
|
90
|
+
Rails.application.config.after_initialize do
|
|
91
|
+
Protovalidate.register_all if Rails.application.config.eager_load
|
|
92
|
+
end
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Types that were not registered still compile on first use, so registration is an optimization rather than a requirement. Both methods raise `Protovalidate::CompilationError` for a rule that does not compile, so a bad rule fails at boot rather than on the first request that reaches it. `register_all` skips message types the engine cannot see, which happens when google-protobuf ships a newer copy of a file compiled into the extension, such as `google/protobuf/descriptor.proto`; those types cannot be validated either way. Once a type is registered, validating it takes no lock on the Ruby side, and the shared validator returned by `Protovalidate.validator` is created once and then read without locking.
|
|
96
|
+
|
|
77
97
|
## Development
|
|
78
98
|
|
|
79
99
|
```bash
|
|
@@ -86,7 +106,7 @@ bundle exec rake steep yard:check rubocop
|
|
|
86
106
|
|
|
87
107
|
The native code lives in `ext/protovalidate`: `src/lib.rs` binds `Protovalidate::Native::Engine` with [magnus](https://github.com/matsadler/magnus), and `sys/` compiles protovalidate-cc and its dependencies from git submodules using file lists derived from protovalidate-cc's Bazel build. To move to a new protovalidate-cc release, edit `ext/protovalidate/sys/versions.json`, run `script/extract-native-sources` (needs Bazel), update `PROTOVALIDATE_VERSION` in `lib/protovalidate/version.rb` if the specification version moved, and run `bundle exec rake proto:generate`.
|
|
88
108
|
|
|
89
|
-
Precompiled gems are assembled by `.github/workflows/native.yml`: Linux builds run inside
|
|
109
|
+
Precompiled gems are assembled by `.github/workflows/native.yml`: glibc Linux builds run inside manylinux_2_34 containers, macOS builds on macos-15 and Windows builds on windows-2025. musl and 32-bit ARM builds are cross-compiled with `rake cross compile` inside [rb-sys-dock](https://github.com/oxidize-rb/rb-sys) images, with newer cross compilers swapped in. `rake 'gem:native[PLATFORM]'` packages one gem per platform.
|
|
90
110
|
|
|
91
111
|
## License
|
|
92
112
|
|
data/Rakefile
CHANGED
|
@@ -14,6 +14,11 @@ Rake::ExtensionTask.new("protovalidate_native") do |ext|
|
|
|
14
14
|
ext.ext_dir = "ext/protovalidate"
|
|
15
15
|
ext.lib_dir = "lib/protovalidate"
|
|
16
16
|
ext.source_pattern = "{Cargo.toml,Cargo.lock,build.rs,src/**/*.rs,sys/Cargo.toml,sys/build.rs,sys/src/**/*.rs,sys/shim/*,sys/filelists/*.txt}"
|
|
17
|
+
# Platforms without a native CI runner are cross-compiled inside rb-sys-dock
|
|
18
|
+
# images (see .github/workflows/native.yml). Given several RUBY_CC_VERSIONs,
|
|
19
|
+
# `rake cross compile` stages each under tmp/<platform>/stage/lib/protovalidate/<X.Y>/.
|
|
20
|
+
ext.cross_compile = true
|
|
21
|
+
ext.cross_platform = %w[x86_64-linux-musl aarch64-linux-musl arm-linux-gnueabihf]
|
|
17
22
|
end
|
|
18
23
|
|
|
19
24
|
RSpec::Core::RakeTask.new(:spec)
|
|
@@ -6,8 +6,8 @@
|
|
|
6
6
|
use magnus::Module as _;
|
|
7
7
|
use magnus::Object as _;
|
|
8
8
|
|
|
9
|
-
/// The engine, shared between Ruby threads. `validate`
|
|
10
|
-
/// `add_file`
|
|
9
|
+
/// The engine, shared between Ruby threads. `validate` and `message_type?`
|
|
10
|
+
/// take the read lock; `add_file` and `compile` take the write lock.
|
|
11
11
|
#[magnus::wrap(class = "Protovalidate::Native::Engine", free_immediately)]
|
|
12
12
|
struct Engine {
|
|
13
13
|
inner: std::sync::RwLock<protovalidate_cc_sys::Engine>,
|
|
@@ -39,6 +39,37 @@ impl Engine {
|
|
|
39
39
|
.map_err(|error| crate::to_ruby_error(ruby, error))
|
|
40
40
|
}
|
|
41
41
|
|
|
42
|
+
/// Whether the engine knows the named message type.
|
|
43
|
+
fn message_type(rb_self: &Self, type_name: String) -> bool {
|
|
44
|
+
let engine = rb_self
|
|
45
|
+
.inner
|
|
46
|
+
.read()
|
|
47
|
+
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
48
|
+
engine.has_message_type(&type_name)
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
/// Compiles the rules of the named type ahead of its first validation.
|
|
52
|
+
///
|
|
53
|
+
/// Compilation inserts into the factory's `flat_hash_map` of rules, and a
|
|
54
|
+
/// rehash there moves every entry. A concurrent `Validate` holds a pointer
|
|
55
|
+
/// into that map after releasing the factory's reader lock, so compiling
|
|
56
|
+
/// under the shared read lock could leave it reading freed memory. The
|
|
57
|
+
/// write lock keeps validations out until the insert is done.
|
|
58
|
+
fn compile(
|
|
59
|
+
ruby: &magnus::Ruby,
|
|
60
|
+
rb_self: &Self,
|
|
61
|
+
type_name: String,
|
|
62
|
+
) -> Result<(), magnus::Error> {
|
|
63
|
+
let result = crate::gvl::without_gvl(|| {
|
|
64
|
+
let engine = rb_self
|
|
65
|
+
.inner
|
|
66
|
+
.write()
|
|
67
|
+
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
68
|
+
engine.compile(&type_name)
|
|
69
|
+
});
|
|
70
|
+
result.map_err(|error| crate::to_ruby_error(ruby, error))
|
|
71
|
+
}
|
|
72
|
+
|
|
42
73
|
/// Validates a serialized message of the named type, returning serialized
|
|
43
74
|
/// violations or `nil` when it is valid.
|
|
44
75
|
fn validate(
|
|
@@ -156,6 +187,8 @@ fn init(ruby: &magnus::Ruby) -> Result<(), magnus::Error> {
|
|
|
156
187
|
let class = module.define_class("Engine", ruby.class_object())?;
|
|
157
188
|
class.define_singleton_method("new", magnus::function!(Engine::new, 0))?;
|
|
158
189
|
class.define_method("add_file", magnus::method!(Engine::add_file, 1))?;
|
|
190
|
+
class.define_method("message_type?", magnus::method!(Engine::message_type, 1))?;
|
|
191
|
+
class.define_method("compile", magnus::method!(Engine::compile, 1))?;
|
|
159
192
|
class.define_method("validate", magnus::method!(Engine::validate, 3))?;
|
|
160
193
|
Ok(())
|
|
161
194
|
}
|
|
@@ -101,6 +101,62 @@ fn read_filelist(path: &Path) -> Vec<String> {
|
|
|
101
101
|
.collect()
|
|
102
102
|
}
|
|
103
103
|
|
|
104
|
+
/// Whether the target is Windows with the GNU toolchain (RubyInstaller's).
|
|
105
|
+
fn target_is_mingw() -> bool {
|
|
106
|
+
env::var("CARGO_CFG_TARGET_OS").as_deref() == Ok("windows")
|
|
107
|
+
&& env::var("CARGO_CFG_TARGET_ENV").as_deref() == Ok("gnu")
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
/// Whether libstdc++ is linked into the extension rather than loaded at
|
|
111
|
+
/// runtime. musl distributions do not install libstdc++ by default, and on
|
|
112
|
+
/// Windows the MinGW runtime DLLs are not on the PATH Ruby loads from.
|
|
113
|
+
fn link_libstdcxx_statically() -> bool {
|
|
114
|
+
env::var("CARGO_CFG_TARGET_ENV").as_deref() == Ok("musl") || target_is_mingw()
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
/// cel-cpp's typeinfo.cc takes `_WIN32` to mean MSVC and demangles with
|
|
118
|
+
/// `type_info::raw_name`, which only MSVC has. MinGW gets a copy that tests
|
|
119
|
+
/// `_MSC_VER` instead and so takes the Itanium ABI path GCC implements.
|
|
120
|
+
fn patch_for_mingw(path: PathBuf) -> PathBuf {
|
|
121
|
+
if !target_is_mingw() || !path.ends_with("third_party/cel-cpp/common/typeinfo.cc") {
|
|
122
|
+
return path;
|
|
123
|
+
}
|
|
124
|
+
let source =
|
|
125
|
+
std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{}: {e}", path.display()));
|
|
126
|
+
assert_eq!(
|
|
127
|
+
source.matches("#ifdef _WIN32").count(),
|
|
128
|
+
2,
|
|
129
|
+
"{}: the _WIN32 guards moved; revisit patch_for_mingw",
|
|
130
|
+
path.display(),
|
|
131
|
+
);
|
|
132
|
+
let patched =
|
|
133
|
+
PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR")).join("cel_typeinfo_mingw.cc");
|
|
134
|
+
std::fs::write(&patched, source.replace("#ifdef _WIN32", "#ifdef _MSC_VER"))
|
|
135
|
+
.unwrap_or_else(|e| panic!("{}: {e}", patched.display()));
|
|
136
|
+
patched
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
/// Lets rustc find an archive that ships with the C++ compiler; rustc
|
|
140
|
+
/// resolves `static=` libraries itself rather than asking the compiler.
|
|
141
|
+
fn add_archive_search_path(compiler: &cc::Tool, archive: &str) {
|
|
142
|
+
let output = compiler
|
|
143
|
+
.to_command()
|
|
144
|
+
.arg(format!("-print-file-name={archive}"))
|
|
145
|
+
.output()
|
|
146
|
+
.expect("run the C++ compiler");
|
|
147
|
+
let path = PathBuf::from(String::from_utf8(output.stdout).expect("utf-8").trim());
|
|
148
|
+
// An archive the compiler does not know comes back as the bare name.
|
|
149
|
+
assert!(path.is_absolute(), "the C++ compiler has no {archive}");
|
|
150
|
+
let dir = path.parent().expect("archive directory");
|
|
151
|
+
println!("cargo::rustc-link-search=native={}", dir.display());
|
|
152
|
+
}
|
|
153
|
+
|
|
154
|
+
/// Links an archive that ships with the C++ compiler into the extension.
|
|
155
|
+
fn link_static_archive(compiler: &cc::Tool, name: &str) {
|
|
156
|
+
add_archive_search_path(compiler, &format!("lib{name}.a"));
|
|
157
|
+
println!("cargo::rustc-link-lib=static={name}");
|
|
158
|
+
}
|
|
159
|
+
|
|
104
160
|
/// One static library built from the C++ sources.
|
|
105
161
|
struct CxxLib {
|
|
106
162
|
name: String,
|
|
@@ -129,13 +185,16 @@ impl CxxLib {
|
|
|
129
185
|
build.flag_if_supported("-fvisibility=hidden");
|
|
130
186
|
build.flag_if_supported("-fvisibility-inlines-hidden");
|
|
131
187
|
}
|
|
188
|
+
if link_libstdcxx_statically() {
|
|
189
|
+
build.cpp_link_stdlib_static(true);
|
|
190
|
+
}
|
|
132
191
|
for dir in includes {
|
|
133
192
|
build.include(dir);
|
|
134
193
|
}
|
|
135
194
|
|
|
136
195
|
let root = manifest_dir();
|
|
137
196
|
for file in read_filelist(&root.join(format!("filelists/{name}.txt"))) {
|
|
138
|
-
build.file(root.join(file));
|
|
197
|
+
build.file(patch_for_mingw(root.join(file)));
|
|
139
198
|
}
|
|
140
199
|
Self {
|
|
141
200
|
name: name.to_owned(),
|
|
@@ -212,6 +271,27 @@ fn main() {
|
|
|
212
271
|
antlr4_lib.compile();
|
|
213
272
|
absl_lib.compile();
|
|
214
273
|
|
|
274
|
+
let compiler = absl_lib.build.get_compiler();
|
|
275
|
+
if link_libstdcxx_statically() {
|
|
276
|
+
add_archive_search_path(&compiler, "libstdc++.a");
|
|
277
|
+
// MinGW's libstdc++ implements threads on winpthreads, whose DLL is
|
|
278
|
+
// no more on Ruby's PATH than libstdc++'s.
|
|
279
|
+
if target_is_mingw() {
|
|
280
|
+
link_static_archive(&compiler, "winpthread");
|
|
281
|
+
}
|
|
282
|
+
}
|
|
283
|
+
// 32-bit ARM has no 64-bit atomic instructions, so GCC calls out to
|
|
284
|
+
// libatomic, which distributions do not always install.
|
|
285
|
+
if env::var("CARGO_CFG_TARGET_ARCH").as_deref() == Ok("arm") {
|
|
286
|
+
link_static_archive(&compiler, "atomic");
|
|
287
|
+
}
|
|
288
|
+
|
|
289
|
+
// abseil's sysinfo.cc reads the CPU frequency from the registry; bazel
|
|
290
|
+
// links advapi32 through abseil's linkopts.
|
|
291
|
+
if env::var("CARGO_CFG_TARGET_OS").as_deref() == Ok("windows") {
|
|
292
|
+
println!("cargo::rustc-link-lib=advapi32");
|
|
293
|
+
}
|
|
294
|
+
|
|
215
295
|
if env::var("CARGO_CFG_TARGET_OS").as_deref() == Ok("macos") {
|
|
216
296
|
println!("cargo::rustc-link-lib=framework=CoreFoundation");
|
|
217
297
|
}
|
|
@@ -154,6 +154,43 @@ struct pv_engine {
|
|
|
154
154
|
ABSL_GUARDED_BY(compiled_mutex);
|
|
155
155
|
};
|
|
156
156
|
|
|
157
|
+
namespace {
|
|
158
|
+
|
|
159
|
+
const google::protobuf::Descriptor* FindMessageType(pv_engine* engine,
|
|
160
|
+
absl::string_view name,
|
|
161
|
+
char** error) {
|
|
162
|
+
const google::protobuf::Descriptor* descriptor =
|
|
163
|
+
engine->pool.FindMessageTypeByName(name);
|
|
164
|
+
if (descriptor == nullptr) {
|
|
165
|
+
SetError(error, absl::StrCat("unknown message type: ", name));
|
|
166
|
+
}
|
|
167
|
+
return descriptor;
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
// Compiles the rules reachable from `descriptor` unless a previous call
|
|
171
|
+
// already did. Compilation holds protovalidate-cc's own writer lock, so
|
|
172
|
+
// concurrent validations wait for it; only the first call per type pays.
|
|
173
|
+
int EnsureCompiled(pv_engine* engine,
|
|
174
|
+
const google::protobuf::Descriptor* descriptor,
|
|
175
|
+
char** error) {
|
|
176
|
+
{
|
|
177
|
+
absl::ReaderMutexLock lock(&engine->compiled_mutex);
|
|
178
|
+
if (engine->compiled.contains(descriptor)) return PV_OK;
|
|
179
|
+
}
|
|
180
|
+
absl::flat_hash_set<const google::protobuf::Descriptor*> seen;
|
|
181
|
+
absl::Status compiled =
|
|
182
|
+
CompileRules(*engine->validator_factory, descriptor, seen);
|
|
183
|
+
if (!compiled.ok()) {
|
|
184
|
+
SetError(error, std::string(compiled.message()));
|
|
185
|
+
return PV_ERR_COMPILATION;
|
|
186
|
+
}
|
|
187
|
+
absl::WriterMutexLock lock(&engine->compiled_mutex);
|
|
188
|
+
engine->compiled.insert(descriptor);
|
|
189
|
+
return PV_OK;
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
} // namespace
|
|
193
|
+
|
|
157
194
|
extern "C" {
|
|
158
195
|
|
|
159
196
|
pv_engine* pv_engine_new(char** error) {
|
|
@@ -192,33 +229,30 @@ int pv_engine_add_file(pv_engine* engine, const uint8_t* file_descriptor_proto,
|
|
|
192
229
|
return PV_OK;
|
|
193
230
|
}
|
|
194
231
|
|
|
232
|
+
int pv_engine_has_message_type(pv_engine* engine, const char* type_name,
|
|
233
|
+
size_t type_name_len) {
|
|
234
|
+
return engine->pool.FindMessageTypeByName(
|
|
235
|
+
absl::string_view(type_name, type_name_len)) != nullptr;
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
int pv_engine_compile(pv_engine* engine, const char* type_name,
|
|
239
|
+
size_t type_name_len, char** error) {
|
|
240
|
+
const google::protobuf::Descriptor* descriptor =
|
|
241
|
+
FindMessageType(engine, absl::string_view(type_name, type_name_len), error);
|
|
242
|
+
if (descriptor == nullptr) return PV_ERR_ARGUMENT;
|
|
243
|
+
return EnsureCompiled(engine, descriptor, error);
|
|
244
|
+
}
|
|
245
|
+
|
|
195
246
|
int pv_engine_validate(pv_engine* engine, const char* type_name,
|
|
196
247
|
size_t type_name_len, const uint8_t* payload,
|
|
197
248
|
size_t payload_len, int fail_fast, uint8_t** out,
|
|
198
249
|
size_t* out_len, char** error) {
|
|
199
250
|
absl::string_view name(type_name, type_name_len);
|
|
200
251
|
const google::protobuf::Descriptor* descriptor =
|
|
201
|
-
engine
|
|
202
|
-
if (descriptor == nullptr)
|
|
203
|
-
|
|
204
|
-
return
|
|
205
|
-
}
|
|
206
|
-
|
|
207
|
-
bool known;
|
|
208
|
-
{
|
|
209
|
-
absl::ReaderMutexLock lock(&engine->compiled_mutex);
|
|
210
|
-
known = engine->compiled.contains(descriptor);
|
|
211
|
-
}
|
|
212
|
-
if (!known) {
|
|
213
|
-
absl::flat_hash_set<const google::protobuf::Descriptor*> seen;
|
|
214
|
-
absl::Status compiled =
|
|
215
|
-
CompileRules(*engine->validator_factory, descriptor, seen);
|
|
216
|
-
if (!compiled.ok()) {
|
|
217
|
-
SetError(error, std::string(compiled.message()));
|
|
218
|
-
return PV_ERR_COMPILATION;
|
|
219
|
-
}
|
|
220
|
-
absl::WriterMutexLock lock(&engine->compiled_mutex);
|
|
221
|
-
engine->compiled.insert(descriptor);
|
|
252
|
+
FindMessageType(engine, name, error);
|
|
253
|
+
if (descriptor == nullptr) return PV_ERR_ARGUMENT;
|
|
254
|
+
if (int code = EnsureCompiled(engine, descriptor, error); code != PV_OK) {
|
|
255
|
+
return code;
|
|
222
256
|
}
|
|
223
257
|
|
|
224
258
|
google::protobuf::Arena arena;
|
|
@@ -60,6 +60,24 @@ void pv_engine_free(pv_engine* engine);
|
|
|
60
60
|
int pv_engine_add_file(pv_engine* engine, const uint8_t* file_descriptor_proto,
|
|
61
61
|
size_t len, char** error);
|
|
62
62
|
|
|
63
|
+
// Returns 1 when the pool knows the fully-qualified message type named by
|
|
64
|
+
// `type_name` and `type_name_len`, 0 otherwise. A type can be missing after
|
|
65
|
+
// its file was added when the file is one compiled into this extension and
|
|
66
|
+
// the caller's copy is newer.
|
|
67
|
+
int pv_engine_has_message_type(pv_engine* engine, const char* type_name,
|
|
68
|
+
size_t type_name_len);
|
|
69
|
+
|
|
70
|
+
// Compiles the rules of the fully-qualified message type named by `type_name`
|
|
71
|
+
// and `type_name_len`, and of every message type reachable from its fields,
|
|
72
|
+
// so that a later pv_engine_validate finds them ready. pv_engine_validate
|
|
73
|
+
// compiles on demand as well; this only moves the work earlier. The name need
|
|
74
|
+
// not be NUL-terminated.
|
|
75
|
+
//
|
|
76
|
+
// Returns PV_OK, PV_ERR_ARGUMENT for an unknown type, or PV_ERR_COMPILATION
|
|
77
|
+
// with a malloc'd message in *error.
|
|
78
|
+
int pv_engine_compile(pv_engine* engine, const char* type_name,
|
|
79
|
+
size_t type_name_len, char** error);
|
|
80
|
+
|
|
63
81
|
// Validates `payload`, a serialized message of the fully-qualified type named
|
|
64
82
|
// by `type_name` and `type_name_len`. The name need not be NUL-terminated:
|
|
65
83
|
// the pool lookup takes a string view, so callers pass their own string
|
|
@@ -28,6 +28,17 @@ unsafe extern "C" {
|
|
|
28
28
|
len: usize,
|
|
29
29
|
error: *mut *mut c_char,
|
|
30
30
|
) -> c_int;
|
|
31
|
+
fn pv_engine_has_message_type(
|
|
32
|
+
engine: *mut PvEngine,
|
|
33
|
+
type_name: *const c_char,
|
|
34
|
+
type_name_len: usize,
|
|
35
|
+
) -> c_int;
|
|
36
|
+
fn pv_engine_compile(
|
|
37
|
+
engine: *mut PvEngine,
|
|
38
|
+
type_name: *const c_char,
|
|
39
|
+
type_name_len: usize,
|
|
40
|
+
error: *mut *mut c_char,
|
|
41
|
+
) -> c_int;
|
|
31
42
|
fn pv_engine_validate(
|
|
32
43
|
engine: *mut PvEngine,
|
|
33
44
|
type_name: *const c_char,
|
|
@@ -105,6 +116,33 @@ impl Engine {
|
|
|
105
116
|
}
|
|
106
117
|
}
|
|
107
118
|
|
|
119
|
+
/// Whether the engine's pool knows the message type `type_name`.
|
|
120
|
+
pub fn has_message_type(&self, type_name: &str) -> bool {
|
|
121
|
+
unsafe {
|
|
122
|
+
pv_engine_has_message_type(self.0, type_name.as_ptr().cast::<c_char>(), type_name.len())
|
|
123
|
+
!= 0
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
/// Compiles the rules of `type_name` and of every message type reachable
|
|
128
|
+
/// from it, ahead of the first `validate`.
|
|
129
|
+
pub fn compile(&self, type_name: &str) -> Result<(), PvError> {
|
|
130
|
+
let mut error: *mut c_char = ptr::null_mut();
|
|
131
|
+
let code = unsafe {
|
|
132
|
+
pv_engine_compile(
|
|
133
|
+
self.0,
|
|
134
|
+
type_name.as_ptr().cast::<c_char>(),
|
|
135
|
+
type_name.len(),
|
|
136
|
+
&raw mut error,
|
|
137
|
+
)
|
|
138
|
+
};
|
|
139
|
+
if code == PV_OK {
|
|
140
|
+
Ok(())
|
|
141
|
+
} else {
|
|
142
|
+
Err(unsafe { status_error(code, error) })
|
|
143
|
+
}
|
|
144
|
+
}
|
|
145
|
+
|
|
108
146
|
/// Validates a serialized message against the rules of `type_name`,
|
|
109
147
|
/// returning the serialized `buf.validate.Violations`, or `None` when the
|
|
110
148
|
/// message is valid.
|
|
@@ -9,17 +9,57 @@ module Protovalidate
|
|
|
9
9
|
# type once and caches them, so reuse one instance rather than creating one
|
|
10
10
|
# per validation. Instances may be shared between threads; Ractors are not
|
|
11
11
|
# supported.
|
|
12
|
+
#
|
|
13
|
+
# The rules of a message type are compiled the first time it is validated,
|
|
14
|
+
# and concurrent validations wait while that happens. {#register} moves the
|
|
15
|
+
# compilation to a time of your choosing, such as application boot.
|
|
12
16
|
class Validator
|
|
17
|
+
# Every generated message class is a direct subclass of this private
|
|
18
|
+
# google-protobuf class, so its subclasses are the loaded message classes.
|
|
19
|
+
MESSAGE_BASE = Google::Protobuf.const_get(:AbstractMessage)
|
|
20
|
+
private_constant :MESSAGE_BASE
|
|
21
|
+
|
|
13
22
|
# @param descriptor_pool [Google::Protobuf::DescriptorPool] pool the
|
|
14
23
|
# validated messages' schema files and their imports are looked up in
|
|
15
24
|
def initialize(descriptor_pool: Google::Protobuf::DescriptorPool.generated_pool)
|
|
16
25
|
@descriptor_pool = descriptor_pool
|
|
17
26
|
@engine = Native::Engine.new
|
|
18
27
|
@registered_files = Set.new
|
|
19
|
-
|
|
28
|
+
types = {} #: Hash[message_class, String]
|
|
29
|
+
@types = types.compare_by_identity.freeze
|
|
20
30
|
@registration = Mutex.new
|
|
21
31
|
end
|
|
22
32
|
|
|
33
|
+
# Compiles the rules of message classes ahead of their first validation.
|
|
34
|
+
#
|
|
35
|
+
# Classes not registered here are still compiled on first use, so this is
|
|
36
|
+
# an optimization, not a requirement.
|
|
37
|
+
#
|
|
38
|
+
# @param message_classes [Array<Class>] message classes generated by google-protobuf
|
|
39
|
+
# @return [void]
|
|
40
|
+
# @raise [CompilationError] when the rules of a message type cannot be compiled
|
|
41
|
+
# @raise [ArgumentError] when an import of a message's file is missing from the pool
|
|
42
|
+
# @raise [TypeError] when a class is not a Protobuf message class
|
|
43
|
+
def register(*message_classes)
|
|
44
|
+
message_classes.each { |klass| type_name_of(klass) }
|
|
45
|
+
nil
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# Registers every message class loaded so far whose type is defined in
|
|
49
|
+
# this validator's descriptor pool. Call it once all generated code has
|
|
50
|
+
# been loaded, such as after eager loading.
|
|
51
|
+
#
|
|
52
|
+
# Types the engine cannot see are skipped: google-protobuf may ship a
|
|
53
|
+
# newer copy of a file compiled into the extension, such as
|
|
54
|
+
# descriptor.proto, and the types it added cannot be validated anyway.
|
|
55
|
+
#
|
|
56
|
+
# @return [void]
|
|
57
|
+
# @raise (see #register)
|
|
58
|
+
def register_all
|
|
59
|
+
loaded_message_classes.each { |klass| register_class(klass, skip_unknown: true) }
|
|
60
|
+
nil
|
|
61
|
+
end
|
|
62
|
+
|
|
23
63
|
# Validates a message, raising when it is invalid.
|
|
24
64
|
#
|
|
25
65
|
# @param message [Google::Protobuf::MessageExts]
|
|
@@ -46,9 +86,8 @@ module Protovalidate
|
|
|
46
86
|
# @raise [EvaluationError] when a rule fails during evaluation
|
|
47
87
|
# @raise [TypeError] when `message` is not a Protobuf message
|
|
48
88
|
def collect_violations(message, fail_fast: false)
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
serialized = @engine.validate(descriptor.name, message.class.encode(message), fail_fast)
|
|
89
|
+
type_name = type_name_of(message.class)
|
|
90
|
+
serialized = @engine.validate(type_name, message.class.encode(message), fail_fast)
|
|
52
91
|
return [] if serialized.nil?
|
|
53
92
|
|
|
54
93
|
Buf::Validate::Violations.decode(serialized).violations.map do |proto|
|
|
@@ -56,24 +95,46 @@ module Protovalidate
|
|
|
56
95
|
end
|
|
57
96
|
end
|
|
58
97
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
end
|
|
64
|
-
|
|
65
|
-
klass.descriptor
|
|
98
|
+
# The full name of a message class's type, registering and compiling it on
|
|
99
|
+
# first sight. Registered classes resolve with one lookup and no lock.
|
|
100
|
+
private def type_name_of(klass)
|
|
101
|
+
@types[klass] || register_class(klass)
|
|
66
102
|
end
|
|
67
103
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
104
|
+
# @types is replaced with a frozen copy rather than mutated, so a reader
|
|
105
|
+
# outside the lock never observes a Hash being written to.
|
|
106
|
+
private def register_class(klass, skip_unknown: false)
|
|
107
|
+
descriptor = descriptor_of(klass)
|
|
71
108
|
@registration.synchronize do
|
|
72
|
-
|
|
109
|
+
@types[klass] || begin
|
|
110
|
+
register_file(descriptor.file_descriptor)
|
|
111
|
+
name = descriptor.name.freeze
|
|
112
|
+
next if skip_unknown && !@engine.message_type?(name)
|
|
113
|
+
|
|
114
|
+
@engine.compile(name)
|
|
115
|
+
types = @types.dup
|
|
116
|
+
types[klass] = name
|
|
117
|
+
@types = types.freeze
|
|
118
|
+
name
|
|
119
|
+
end
|
|
120
|
+
end
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
# Classes generated from other pools are skipped, since their files are
|
|
124
|
+
# not resolvable against this one.
|
|
125
|
+
private def loaded_message_classes
|
|
126
|
+
MESSAGE_BASE.subclasses.select do |klass|
|
|
127
|
+
descriptor = klass.descriptor
|
|
128
|
+
@descriptor_pool.lookup(descriptor.name).equal?(descriptor)
|
|
129
|
+
end
|
|
130
|
+
end
|
|
73
131
|
|
|
74
|
-
|
|
75
|
-
|
|
132
|
+
private def descriptor_of(klass)
|
|
133
|
+
unless klass.is_a?(Google::Protobuf::MessageExts::ClassMethods)
|
|
134
|
+
raise TypeError, "expected a Google::Protobuf message class, got #{klass.inspect}"
|
|
76
135
|
end
|
|
136
|
+
|
|
137
|
+
klass.descriptor
|
|
77
138
|
end
|
|
78
139
|
|
|
79
140
|
# The engine resolves imports eagerly, so they are added before the importer.
|
data/lib/protovalidate.rb
CHANGED
|
@@ -32,6 +32,25 @@ module Protovalidate
|
|
|
32
32
|
validator.validate(message, fail_fast:)
|
|
33
33
|
end
|
|
34
34
|
|
|
35
|
+
# Compiles the rules of message classes ahead of their first validation with
|
|
36
|
+
# the shared validator, for example at application boot.
|
|
37
|
+
#
|
|
38
|
+
# @param (see Validator#register)
|
|
39
|
+
# @return [void]
|
|
40
|
+
# @raise (see Validator#register)
|
|
41
|
+
def self.register(*message_classes)
|
|
42
|
+
validator.register(*message_classes)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# Registers every loaded message class of the generated pool with the shared
|
|
46
|
+
# validator. Call it once all generated code has been loaded.
|
|
47
|
+
#
|
|
48
|
+
# @return [void]
|
|
49
|
+
# @raise (see Validator#register)
|
|
50
|
+
def self.register_all
|
|
51
|
+
validator.register_all
|
|
52
|
+
end
|
|
53
|
+
|
|
35
54
|
# Collects the violations of a message with the shared validator.
|
|
36
55
|
#
|
|
37
56
|
# @param (see Validator#collect_violations)
|
|
@@ -6,6 +6,13 @@ module Protovalidate
|
|
|
6
6
|
# Adds a serialized google.protobuf.FileDescriptorProto to the pool.
|
|
7
7
|
def add_file: (String file) -> void
|
|
8
8
|
|
|
9
|
+
# Whether the engine's pool knows the named message type.
|
|
10
|
+
def message_type?: (String type_name) -> bool
|
|
11
|
+
|
|
12
|
+
# Compiles the rules of the named type and of every type reachable from
|
|
13
|
+
# it, ahead of the first validate.
|
|
14
|
+
def compile: (String type_name) -> void
|
|
15
|
+
|
|
9
16
|
# Validates a serialized message of the named type; returns serialized
|
|
10
17
|
# buf.validate.Violations, or nil when the message is valid.
|
|
11
18
|
def validate: (String type_name, String payload, bool fail_fast) -> String?
|
data/sig/protovalidate.rbs
CHANGED
|
@@ -11,11 +11,18 @@ module Protovalidate
|
|
|
11
11
|
# A Protobuf message instance produced by google-protobuf.
|
|
12
12
|
type message = Google::Protobuf::MessageExts
|
|
13
13
|
|
|
14
|
+
# A Protobuf message class generated by google-protobuf.
|
|
15
|
+
type message_class = Google::Protobuf::MessageExts::ClassMethods
|
|
16
|
+
|
|
14
17
|
# The validator shared by the module-level functions.
|
|
15
18
|
def self.validator: () -> Validator
|
|
16
19
|
|
|
17
20
|
def self.validate: (message message, ?fail_fast: bool) -> void
|
|
18
21
|
|
|
22
|
+
def self.register: (*message_class message_classes) -> void
|
|
23
|
+
|
|
24
|
+
def self.register_all: () -> void
|
|
25
|
+
|
|
19
26
|
def self.collect_violations: (message message, ?fail_fast: bool) -> Array[Violation]
|
|
20
27
|
|
|
21
28
|
class Error < StandardError
|
|
@@ -36,21 +43,33 @@ module Protovalidate
|
|
|
36
43
|
end
|
|
37
44
|
|
|
38
45
|
class Validator
|
|
46
|
+
# google-protobuf's private base class of generated messages, reached with const_get.
|
|
47
|
+
MESSAGE_BASE: untyped
|
|
48
|
+
|
|
39
49
|
@descriptor_pool: Google::Protobuf::DescriptorPool
|
|
40
50
|
@engine: Native::Engine
|
|
41
51
|
@registered_files: Set[String]
|
|
42
|
-
|
|
52
|
+
# Message class to its full type name, replaced as a whole on registration.
|
|
53
|
+
@types: Hash[message_class, String]
|
|
43
54
|
@registration: Mutex
|
|
44
55
|
|
|
45
56
|
def initialize: (?descriptor_pool: Google::Protobuf::DescriptorPool) -> void
|
|
46
57
|
|
|
58
|
+
def register: (*message_class message_classes) -> void
|
|
59
|
+
|
|
60
|
+
def register_all: () -> void
|
|
61
|
+
|
|
47
62
|
def validate: (message message, ?fail_fast: bool) -> void
|
|
48
63
|
|
|
49
64
|
def collect_violations: (message message, ?fail_fast: bool) -> Array[Violation]
|
|
50
65
|
|
|
51
|
-
private def
|
|
66
|
+
private def loaded_message_classes: () -> Array[message_class]
|
|
67
|
+
|
|
68
|
+
private def type_name_of: (message_class klass) -> String
|
|
69
|
+
|
|
70
|
+
private def register_class: (message_class klass, ?skip_unknown: bool) -> String?
|
|
52
71
|
|
|
53
|
-
private def
|
|
72
|
+
private def descriptor_of: (message_class klass) -> Google::Protobuf::Descriptor
|
|
54
73
|
|
|
55
74
|
private def register_file: (Google::Protobuf::FileDescriptor file) -> void
|
|
56
75
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: protovalidate
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.0.
|
|
4
|
+
version: 0.1.0.beta3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Sorah Fukumori
|
|
@@ -39,7 +39,7 @@ dependencies:
|
|
|
39
39
|
version: 0.9.130
|
|
40
40
|
description: Ruby implementation of protovalidate. Validates Protobuf messages against
|
|
41
41
|
the buf.validate rules in their schema, evaluated with CEL by protovalidate-cc through
|
|
42
|
-
a native extension. Precompiled gems are published for Linux and
|
|
42
|
+
a native extension. Precompiled gems are published for Linux, macOS and Windows.
|
|
43
43
|
email:
|
|
44
44
|
- sorah@ivry.jp
|
|
45
45
|
executables: []
|
|
@@ -47,6 +47,7 @@ extensions:
|
|
|
47
47
|
- ext/protovalidate/extconf.rb
|
|
48
48
|
extra_rdoc_files: []
|
|
49
49
|
files:
|
|
50
|
+
- ".gitattributes"
|
|
50
51
|
- ".yardopts"
|
|
51
52
|
- CHANGELOG.md
|
|
52
53
|
- LICENSE.txt
|