protobufable 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +30 -0
- data/DESIGN.md +298 -0
- data/LICENSE.txt +21 -0
- data/README.md +229 -0
- data/docs/columns.md +34 -0
- data/docs/connect.md +55 -0
- data/docs/errors.md +190 -0
- data/docs/request-bodies.md +42 -0
- data/docs/responses.md +49 -0
- data/docs/validation.md +121 -0
- data/lib/protobufable/alba_binding.rb +149 -0
- data/lib/protobufable/bad_request.rb +60 -0
- data/lib/protobufable/column_validatable.rb +56 -0
- data/lib/protobufable/connect/problem_rescuable.rb +92 -0
- data/lib/protobufable/connect.rb +20 -0
- data/lib/protobufable/field_path.rb +65 -0
- data/lib/protobufable/json_type.rb +114 -0
- data/lib/protobufable/problem/information.rb +75 -0
- data/lib/protobufable/problem/retry_info.rb +62 -0
- data/lib/protobufable/problem/typeable.rb +259 -0
- data/lib/protobufable/problem.rb +27 -0
- data/lib/protobufable/problem_details_pb.rb +17 -0
- data/lib/protobufable/problem_pb.rb +15 -0
- data/lib/protobufable/protovalidate.rb +15 -0
- data/lib/protobufable/railtie.rb +26 -0
- data/lib/protobufable/renderers.rb +63 -0
- data/lib/protobufable/request_parseable.rb +188 -0
- data/lib/protobufable/request_validatable.rb +111 -0
- data/lib/protobufable/version.rb +10 -0
- data/lib/protobufable.rb +40 -0
- data/lib/tapioca/dsl/compilers/protobufable_request_parseable.rb +69 -0
- data/proto/protobufable/problem.proto +36 -0
- data/proto/protobufable/problem_details.proto +31 -0
- data/sig/generated/protobufable/alba_binding.rbs +101 -0
- data/sig/generated/protobufable/bad_request.rbs +37 -0
- data/sig/generated/protobufable/column_validatable.rbs +23 -0
- data/sig/generated/protobufable/connect/problem_rescuable.rbs +49 -0
- data/sig/generated/protobufable/connect.rbs +7 -0
- data/sig/generated/protobufable/field_path.rbs +33 -0
- data/sig/generated/protobufable/json_type.rbs +72 -0
- data/sig/generated/protobufable/problem/information.rbs +58 -0
- data/sig/generated/protobufable/problem/retry_info.rbs +44 -0
- data/sig/generated/protobufable/problem/typeable.rbs +166 -0
- data/sig/generated/protobufable/problem.rbs +8 -0
- data/sig/generated/protobufable/protovalidate.rbs +2 -0
- data/sig/generated/protobufable/railtie.rbs +8 -0
- data/sig/generated/protobufable/renderers.rbs +22 -0
- data/sig/generated/protobufable/request_parseable.rbs +131 -0
- data/sig/generated/protobufable/request_validatable.rbs +71 -0
- data/sig/generated/protobufable/version.rbs +6 -0
- data/sig/generated/protobufable.rbs +24 -0
- data/sig/manual/alba_binding.rbs +29 -0
- data/sig/manual/connect/problem_rescuable.rbs +13 -0
- data/sig/manual/connect_rpc_rails.rbs +9 -0
- data/sig/manual/problem/typeable.rbs +28 -0
- data/sig/manual/problem_details_pb.rbs +12 -0
- data/sig/manual/problem_pb.rbs +8 -0
- data/sig/manual/protovalidate.rbs +38 -0
- data/sig/manual/request_parseable.rbs +17 -0
- metadata +316 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 5f133f23141bc2f8aef5ad8d1d0dd0f88dd58a51843d468d3b242cf0ec7a6d83
|
|
4
|
+
data.tar.gz: f030966b166069c87390a9d962904fe363ab36a5a9b9531c66deb4e77c3a67cb
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 6b3ea92d4a634aa17b4a67b2f612e6d5172c1de26184fb0f5462222f07553ed97091aaa21994f7f8e3c55ec49149aaf74597068e732b97e9f29c2d9af99180b1
|
|
7
|
+
data.tar.gz: 4895b5014905f7e2e353f31fca6f8b76000560cf474602b0c2f8e3bafcc3efaedb74d15b5e39aa661c6bd6148f628093c6aa96a1469a9d70378dc8e96c5c222a
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0 - 2026-10-13
|
|
4
|
+
|
|
5
|
+
Initial release.
|
|
6
|
+
|
|
7
|
+
- `Protobufable::RequestParseable` decodes a JSON request body into the protobuf message an
|
|
8
|
+
action declares with `protobuf_body`. A request without a JSON body raises
|
|
9
|
+
`ActionDispatch::Http::Parameters::ParseError` (a 400).
|
|
10
|
+
- A tapioca DSL compiler types `RequestParseable#message` on each controller declaring
|
|
11
|
+
`protobuf_body`: the declared class, or the union of the classes its actions declare.
|
|
12
|
+
- `Protobufable::Renderers` answers with a message as ProtoJSON or binary protobuf.
|
|
13
|
+
- `Protobufable::JsonType` stores a message in a `json` or `jsonb` column, and with
|
|
14
|
+
`preserve_proto_fieldnames:` writes the `.proto` field names instead of their lowerCamelCase
|
|
15
|
+
JSON names.
|
|
16
|
+
- `Protobufable::Problem::Typeable` derives an RFC 9457 type and status from a protobuf enum, as
|
|
17
|
+
a catalogue layer over the problem gem. The gem ships `protobufable.ProblemTypeOptions`, the
|
|
18
|
+
message an enum value is annotated with; the extension carrying it is declared by the
|
|
19
|
+
application.
|
|
20
|
+
- `Protobufable::Problem::Information` publishes protobuf messages in a problem document, and
|
|
21
|
+
`Protobufable::Problem::RetryInfo` publishes a retry interval as a `google.rpc.RetryInfo`.
|
|
22
|
+
- `Protobufable::Connect::ProblemRescuable` (`require "protobufable/connect"`) answers a problem
|
|
23
|
+
raised in a connect_rpc_rails controller as a Connect error carrying the document as a
|
|
24
|
+
`protobufable.ProblemDetails`, shipped in `protobufable/problem_details.proto`.
|
|
25
|
+
- `Protobufable::RequestValidatable` and `Protobufable::ColumnValidatable` enforce
|
|
26
|
+
`buf.validate` rules on requests and on stored columns, with `Protobufable::FieldPath` and
|
|
27
|
+
`Protobufable::BadRequest` reporting which fields failed. The railtie lists
|
|
28
|
+
`Protobufable::RequestValidatable::InvalidMessage` in `rescue_responses` as `:bad_request`.
|
|
29
|
+
- `Protobufable::AlbaBinding` binds an Alba resource to a message and checks their field parity
|
|
30
|
+
at load time.
|
data/DESIGN.md
ADDED
|
@@ -0,0 +1,298 @@
|
|
|
1
|
+
# Design
|
|
2
|
+
|
|
3
|
+
Why this library is shaped the way it is. [README.md](README.md) and [docs/](docs/) cover using
|
|
4
|
+
it. Extracted from a production API, where most of these decisions were paid for once already.
|
|
5
|
+
|
|
6
|
+
## What belongs here
|
|
7
|
+
|
|
8
|
+
Three gems divide this space, and the boundaries are deliberate.
|
|
9
|
+
|
|
10
|
+
[problem](https://github.com/sorah/problem) owns RFC 9457: the document, the declaration on the
|
|
11
|
+
error class, the renderer, the exceptions app. protobufable adds a catalogue backed by a
|
|
12
|
+
protobuf enum and a way to carry protobuf messages in the document. It reimplements none of the
|
|
13
|
+
rest, and `Problem::Details`, `Problem::Rescuable` and the `problem+json` renderer are not
|
|
14
|
+
wrapped here.
|
|
15
|
+
|
|
16
|
+
[connect_rpc_rails](https://github.com/ivry-inc/connect_rpc_rails) owns the Connect protocol.
|
|
17
|
+
Connect defines its own error body, and the code determines the HTTP status, so an RFC 9457
|
|
18
|
+
document cannot be the response; the two catalogues are bridged rather than duplicated, through
|
|
19
|
+
`ConnectRpcRails::Error.code_for_http_status` and a `render_connect_error` call. That bridge
|
|
20
|
+
lives here, as an optional `require`, because what crosses it is the document as a protobuf
|
|
21
|
+
message, which is this gem's to build. The protocol itself stays on that side.
|
|
22
|
+
|
|
23
|
+
protobufable owns what is left: the schema being the schema, in requests, columns, responses
|
|
24
|
+
and the error catalogue.
|
|
25
|
+
|
|
26
|
+
## Parsing through the body parser
|
|
27
|
+
|
|
28
|
+
`Protobufable::RequestParseable` replaces the `:json` entry in `ActionDispatch::Request`'s
|
|
29
|
+
parser table. The body is then decoded exactly once, by `decode_json`, into the class the action
|
|
30
|
+
declared, and a wrong type is a 400 before any application code runs.
|
|
31
|
+
|
|
32
|
+
`connect_rpc_rails` solves the same problem the other way, decoding in `process_action` and
|
|
33
|
+
assigning `request.request_parameters`. That suits a protocol that owns the whole request; it
|
|
34
|
+
does not suit a REST endpoint that should keep behaving like a Rails endpoint, where `params`
|
|
35
|
+
must stay filled so an existing `params[:name]` reader keeps working and an endpoint can be
|
|
36
|
+
moved over one at a time.
|
|
37
|
+
|
|
38
|
+
### Declarations on the class, not in a table
|
|
39
|
+
|
|
40
|
+
The original kept a process-wide hash keyed by `[controller_path, action]`, filled from
|
|
41
|
+
`method_added`. That table outlives a reload, keeps an entry for a renamed action, cannot
|
|
42
|
+
describe an anonymous controller, and collides when two controllers name an action the same.
|
|
43
|
+
Here the declarations are a `class_attribute`, and the parser reaches them through
|
|
44
|
+
`ActionDispatch::Request#controller_class`, which the request already resolves for routing.
|
|
45
|
+
|
|
46
|
+
### The message wrapper
|
|
47
|
+
|
|
48
|
+
Rails' parameter filtering works on hashes and cannot see inside a protobuf message. A message
|
|
49
|
+
left plainly in `params` would therefore be rendered whole by the log formatter, every field
|
|
50
|
+
included. The wrapper answers `[FILTERED]` to `as_json` and exposes nothing to `inspect`.
|
|
51
|
+
|
|
52
|
+
## Two encoding settings, pointing opposite ways
|
|
53
|
+
|
|
54
|
+
`emit_defaults` is off for responses and on for stored columns, on purpose.
|
|
55
|
+
|
|
56
|
+
A response wants to be small, and wants an absent `optional` field to stay distinguishable from
|
|
57
|
+
one set to its default, which is the answer to the usual objection that proto3 cannot express
|
|
58
|
+
absence. A stored document wants the opposite: a shape that does not depend on which fields
|
|
59
|
+
happened to be set, so that comparing a freshly loaded document against the message in memory
|
|
60
|
+
means something, and so that anything reading the column in SQL does not have to tell absent
|
|
61
|
+
from default.
|
|
62
|
+
|
|
63
|
+
`preserve_proto_fieldnames` is on everywhere. ProtoJSON defaults to lowerCamelCase; snake_case
|
|
64
|
+
matches the rest of a Rails API and the OpenAPI generated with `naming=proto`. Both settings are
|
|
65
|
+
one-way doors for clients, which is why neither is left to the default.
|
|
66
|
+
|
|
67
|
+
## A type, not a coder
|
|
68
|
+
|
|
69
|
+
`Protobufable::JsonType` is an `ActiveModel::Type::Value` reporting `:json`. A `serialize` coder
|
|
70
|
+
over a `text` column would encode and decode just as well and get none of the rest: Rails and
|
|
71
|
+
the adapter would not know the column is JSON, casting from a Hash would not happen, and dirty
|
|
72
|
+
tracking would compare encoded text. `changed_in_place?` compares decoded messages, because a
|
|
73
|
+
`jsonb` column hands back a canonicalized document whose key order need not match what
|
|
74
|
+
`encode_json` produced, which would otherwise read as a change on every load.
|
|
75
|
+
|
|
76
|
+
## The problem catalogue
|
|
77
|
+
|
|
78
|
+
`Problem::Detailable` documents an extension contract for exactly this: prepend a module to the
|
|
79
|
+
error class's singleton overriding `type` and `status`, and call `super` for a class the
|
|
80
|
+
catalogue does not cover. Its DESIGN.md lists five clauses that make it work, each of which
|
|
81
|
+
fails silently rather than loudly, so `spec/protobufable/problem/typeable_derivation_spec.rb`
|
|
82
|
+
pins them from this side too.
|
|
83
|
+
|
|
84
|
+
### Overriding a method, not configuring an object
|
|
85
|
+
|
|
86
|
+
Which enum, and which annotation, are overridable class methods:
|
|
87
|
+
|
|
88
|
+
```ruby
|
|
89
|
+
def self.problem_type_enum = "myapp.api.ProblemType"
|
|
90
|
+
def self.problem_type_annotation = "myapp.api.problem_type"
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
A configuration object would bless one catalogue shape, add global mutable state, and not
|
|
94
|
+
compose with inheritance. A method is inherited, so a base class declares it once and a second
|
|
95
|
+
catalogue in the same application is simply another base class. Nothing is registered anywhere,
|
|
96
|
+
and the catalogue is built on first use and memoized on the class that asked, so a reloaded or
|
|
97
|
+
anonymous class cannot read one built for a previous one.
|
|
98
|
+
|
|
99
|
+
Neither method has a default. Since no annotation is shipped (below), there is nothing to guess
|
|
100
|
+
at, and the raise names the method to override.
|
|
101
|
+
|
|
102
|
+
### The enum stays off the wire
|
|
103
|
+
|
|
104
|
+
RFC 9457 requires `type` to be a URI reference, so the enum value cannot be it. That constraint
|
|
105
|
+
turns out to be useful: the enum can then be as fine-grained as the server needs for monitoring
|
|
106
|
+
while the response only distinguishes what a caller is allowed to distinguish, which is what
|
|
107
|
+
lets several values publish one identifier.
|
|
108
|
+
|
|
109
|
+
Nothing stops an application publishing the value anyway, as an extension member —
|
|
110
|
+
[docs/errors.md](docs/errors.md#publishing-the-enum-value) shows the mixin, and it is four lines. It is not shipped as a module because it is not a
|
|
111
|
+
default anyone should reach for without deciding: an API whose catalogue hides nothing loses
|
|
112
|
+
nothing by publishing the value, and an API with even one deliberately shared identifier hands
|
|
113
|
+
that distinction straight back. That is a judgement about a particular catalogue, not something
|
|
114
|
+
a gem can make.
|
|
115
|
+
|
|
116
|
+
### Titles come from the identifier
|
|
117
|
+
|
|
118
|
+
The original keyed titles by the enum value name, which meant two values publishing one
|
|
119
|
+
identifier had two locale entries that a spec had to check were still saying the same thing.
|
|
120
|
+
`Problem::I18nable` already keys titles by the published identifier, so sharing an identifier
|
|
121
|
+
shares a title structurally. The spec, and the duplicated entries, are gone.
|
|
122
|
+
|
|
123
|
+
### A literal declaration still wins
|
|
124
|
+
|
|
125
|
+
`type` and `status` fall through to the literal DSL when the class set one, which is the rule
|
|
126
|
+
`Problem::I18nable` follows for titles. A codebase can then move onto a catalogue gradually
|
|
127
|
+
instead of all at once.
|
|
128
|
+
|
|
129
|
+
## What a schema gem can and cannot ship
|
|
130
|
+
|
|
131
|
+
**The catalogue can never be shipped.** The enum is the list of problems one API can return: it
|
|
132
|
+
is the application's own vocabulary, and a gem that shipped one would be shipping a guess at
|
|
133
|
+
somebody else's API. Nor can anything referencing it — a message with a `ProblemType` field, a
|
|
134
|
+
`ProblemDetails` carrying one — which is the same guess one level removed. So the enum is named
|
|
135
|
+
by an overridable method and read through the descriptor pool, never through a generated
|
|
136
|
+
constant.
|
|
137
|
+
|
|
138
|
+
**The option message can, and does.** `protobufable.ProblemTypeOptions` says what a problem type
|
|
139
|
+
publishes — an identifier and a status — in the same shape for every application, and references
|
|
140
|
+
nothing. `proto/protobufable/problem.proto` holds exactly that, ships in the gem, and generates
|
|
141
|
+
`lib/protobufable/problem_pb.rb`.
|
|
142
|
+
|
|
143
|
+
**The document message can too.** `protobufable.ProblemDetails` carries the published `type`,
|
|
144
|
+
never the enum value, so it is RFC 9457's shape and nobody's catalogue. It sits in its own file,
|
|
145
|
+
`problem_details.proto`, because an application referencing it from its schema has to vendor
|
|
146
|
+
it, and should not have to vendor the option message along with it.
|
|
147
|
+
|
|
148
|
+
**The extension carrying it cannot ship yet**, and it is the only piece waiting on anything. An
|
|
149
|
+
extension's field number is a global namespace registered in
|
|
150
|
+
[protocolbuffers/protobuf docs/options.md](https://github.com/protocolbuffers/protobuf/blob/main/docs/options.md);
|
|
151
|
+
protobufable claims **1376**, provisionally, and does not hold it. A number that moved after
|
|
152
|
+
release would break every `.proto` that imported the extension, and one taken from the
|
|
153
|
+
internal-use range invites exactly the collision the registry exists to prevent. An application
|
|
154
|
+
therefore declares the `extend` itself, with a number of its own, over the message the gem
|
|
155
|
+
supplies:
|
|
156
|
+
|
|
157
|
+
```proto
|
|
158
|
+
import "google/protobuf/descriptor.proto";
|
|
159
|
+
import "protobufable/problem.proto";
|
|
160
|
+
|
|
161
|
+
extend google.protobuf.EnumValueOptions {
|
|
162
|
+
optional protobufable.ProblemTypeOptions problem_type = 50001;
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
That is one line of schema rather than a copied message definition, and the shape stays the same
|
|
167
|
+
everywhere, which is what makes `problem_type_annotation` enough to point the catalogue at it.
|
|
168
|
+
|
|
169
|
+
### One sub-message, not one number per property
|
|
170
|
+
|
|
171
|
+
A catalogue could spend two extension numbers, one for the identifier and one for the status, to
|
|
172
|
+
say one thing about a value. Carrying both in one message spends one:
|
|
173
|
+
|
|
174
|
+
```proto
|
|
175
|
+
optional protobufable.ProblemTypeOptions problem_type = 50001;
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
A third property later is then a field on the message rather than another number, and a number
|
|
179
|
+
is the scarce thing. It is also why the message is worth shipping at all: were it two scalars,
|
|
180
|
+
there would be nothing generic left to ship.
|
|
181
|
+
|
|
182
|
+
The split layout is still readable, through `problem_uri_annotation` and
|
|
183
|
+
`problem_status_annotation`. An application whose enum is already annotated that way should not
|
|
184
|
+
have to rewrite its schema to be read from here, so those two are a supported extension point
|
|
185
|
+
rather than a hidden one — just not the shape a new catalogue should choose.
|
|
186
|
+
|
|
187
|
+
## Problems over Connect
|
|
188
|
+
|
|
189
|
+
### Built by the controller
|
|
190
|
+
|
|
191
|
+
The message the document travels as is built by `problem_message_for`, a controller method
|
|
192
|
+
beside `Problem::Rescuable`'s own `problem_for`. An application that declared its own copy of
|
|
193
|
+
the message, perhaps under its own package before this gem existed, overrides that one method,
|
|
194
|
+
and nothing is registered or configured anywhere.
|
|
195
|
+
|
|
196
|
+
### One detail, not one per message
|
|
197
|
+
|
|
198
|
+
The document goes out as a single Connect detail with `information` inside it, rather than
|
|
199
|
+
with each `information` message beside it as its own detail. A caller then reads one thing, in
|
|
200
|
+
the same shape it reads over REST, and a detail list does not grow a second copy of what the
|
|
201
|
+
document already carries.
|
|
202
|
+
|
|
203
|
+
### Only what the controller raises
|
|
204
|
+
|
|
205
|
+
`Connect::ProblemRescuable` is a `rescue_from`, so it sees what the controller raises and nothing
|
|
206
|
+
earlier. An exception escaping before dispatch reaches connect_rpc_rails' exceptions app, which
|
|
207
|
+
answers from `rescue_responses`. That keeps the exceptions app free of any knowledge of
|
|
208
|
+
problems, at the cost of a problem raised in middleware going out as whatever status its class
|
|
209
|
+
is registered with.
|
|
210
|
+
|
|
211
|
+
### Registered twice
|
|
212
|
+
|
|
213
|
+
`Problem::Rescuable` registers its handler when it is first included. A controller whose parent
|
|
214
|
+
already included it would keep that registration, which precedes the `rescue_responses`
|
|
215
|
+
handlers `ConnectRpcRails::Controller` installs, so a problem class an application also listed
|
|
216
|
+
in `rescue_responses` would be answered by the transport without its document.
|
|
217
|
+
`Connect::ProblemRescuable` registers the handler again to follow them.
|
|
218
|
+
|
|
219
|
+
## Constants and files
|
|
220
|
+
|
|
221
|
+
The problem integration lives under `Protobufable::Problem` and the Connect one under
|
|
222
|
+
`Protobufable::Connect`, one file per constant: `lib/protobufable/problem/typeable.rb` defines
|
|
223
|
+
`Protobufable::Problem::Typeable`. Inside `Protobufable` a bare `Problem` resolves to
|
|
224
|
+
`Protobufable::Problem`, so the problem gem's own constants are always written
|
|
225
|
+
`::Problem::Detailable`. A missing prefix fails with a `NameError` rather than reaching the wrong
|
|
226
|
+
constant, because the namespace defines none of the problem gem's names.
|
|
227
|
+
|
|
228
|
+
The protovalidate integration stays flat (`Protobufable::RequestValidatable`): it is four
|
|
229
|
+
constants, and a `Protobufable::Protovalidate` namespace would shadow the protovalidate gem the
|
|
230
|
+
same way for little to group.
|
|
231
|
+
|
|
232
|
+
Each mixin carries the layer it attaches to in its name — `RequestParseable` and
|
|
233
|
+
`RequestValidatable` are controller concerns, `ColumnValidatable` is an ActiveRecord one — so a
|
|
234
|
+
reader knows where it belongs without opening the file.
|
|
235
|
+
|
|
236
|
+
## Callback ordering
|
|
237
|
+
|
|
238
|
+
Rails appends a re-registered `before_action` to the end of the chain. `RequestValidatable`
|
|
239
|
+
therefore re-registers its callback when an action binds its `protobuf_body`, which moves it
|
|
240
|
+
behind the authentication and authorization callbacks the controller declares above the action.
|
|
241
|
+
Without that, an invalid body would be answered before an unauthenticated one, and the endpoint
|
|
242
|
+
would become an oracle for what a valid body looks like. A controller that skipped the callback
|
|
243
|
+
has no entry to move and is left alone.
|
|
244
|
+
|
|
245
|
+
## Field paths
|
|
246
|
+
|
|
247
|
+
A protobuf field has two published names: `preferred_language` in the binary encoding and in
|
|
248
|
+
ProtoJSON with `preserve_proto_fieldnames`, `preferredLanguage` without it. A field path that
|
|
249
|
+
names the wrong one is a path the client cannot follow. `Protobufable::FieldPath` resolves each
|
|
250
|
+
segment against the descriptor rather than interpolating the name the violation carried;
|
|
251
|
+
`Protovalidate::Violation#field_path` always uses the proto name, which is why this exists.
|
|
252
|
+
|
|
253
|
+
## googleapis descriptors
|
|
254
|
+
|
|
255
|
+
`Problem::RetryInfo` and `BadRequest` look their message classes up in the generated pool rather than
|
|
256
|
+
requiring them. `google.rpc.*` belongs to the application that generates googleapis alongside
|
|
257
|
+
its own protos, and a second copy of those descriptors in one process is a boot failure, not a
|
|
258
|
+
warning. The error names what to generate.
|
|
259
|
+
|
|
260
|
+
## Alba internals
|
|
261
|
+
|
|
262
|
+
The `with_protobuf` parity check reads Alba's `@_attributes` and `@_traits`, which are private
|
|
263
|
+
to Alba. There is no public way to ask a resource what it declares, and a trait is a block
|
|
264
|
+
rather than a list, so seeing what it adds means evaluating it against a throwaway subclass. An
|
|
265
|
+
Alba release can rename either ivar and the check would silently stop checking, so
|
|
266
|
+
`spec/protobufable/alba_binding_spec.rb` asserts both names against the installed version.
|
|
267
|
+
|
|
268
|
+
## Types
|
|
269
|
+
|
|
270
|
+
Signatures are `rbs-inline` annotations generated into `sig/generated`, checked in and verified
|
|
271
|
+
in CI, plus hand-written `sig/manual` for what RBS cannot infer: `class_attribute` accessors,
|
|
272
|
+
and the self-type interfaces that say what a concern's includer supplies.
|
|
273
|
+
|
|
274
|
+
`rbs/` is a local RBS collection source carrying google-protobuf's 4.x surface. The gem ships no
|
|
275
|
+
signatures, and ruby/gem_rbs_collection's entry stops at 3.22, whose non-generic `RepeatedField`
|
|
276
|
+
contradicts the signatures protovalidate ships. It has to be a collection source rather than a
|
|
277
|
+
`sig/manual` file, because a gem's own signatures are loaded as library RBS and cannot reference
|
|
278
|
+
a project signature.
|
|
279
|
+
|
|
280
|
+
connect_rpc_rails' own signatures are ignored in `rbs_collection.yaml`: they stub
|
|
281
|
+
`Rails::Railtie.initializer`, as problem's do, and the two library declarations conflict. The
|
|
282
|
+
little of it the Connect integration calls is declared in `sig/manual/connect_rpc_rails.rbs`.
|
|
283
|
+
|
|
284
|
+
`protoc --rbs_out` is not used. It emits `RepeatedField` with two type parameters, which agrees
|
|
285
|
+
with neither the collection nor protovalidate, and the Steepfile checks `lib` only, so the spec
|
|
286
|
+
fixtures need no signatures.
|
|
287
|
+
|
|
288
|
+
## Deliberately out of scope
|
|
289
|
+
|
|
290
|
+
- **RFC 9457 itself.** [problem](https://github.com/sorah/problem) owns it.
|
|
291
|
+
- **Connect and gRPC.** [connect_rpc_rails](https://github.com/ivry-inc/connect_rpc_rails) owns
|
|
292
|
+
the protocol. Only the error bridge is here, and it replaces one method of
|
|
293
|
+
`Problem::Rescuable`.
|
|
294
|
+
- **OpenAPI generation.** Post-processing a gnostic document is a build step over `.proto`
|
|
295
|
+
sources, not a library that runs in a request.
|
|
296
|
+
- **Opaque page tokens.** A protobuf-backed cursor is a small thing to write and a large thing
|
|
297
|
+
to make general.
|
|
298
|
+
- **`application/problem+xml`.**
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Sorah Fukumori
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,229 @@
|
|
|
1
|
+
# protobufable: Protocol Buffers as the schema of record for a Rails API
|
|
2
|
+
|
|
3
|
+
`protobufable` makes generated protobuf message classes the shape of a Rails application's
|
|
4
|
+
data, on the way in and on the way out. A JSON request body is decoded straight into the
|
|
5
|
+
message its action declares, so a wrong type is a 400 before the action runs and the action
|
|
6
|
+
reads a typed object instead of `params`. A `jsonb` column holds a message whose shape is
|
|
7
|
+
declared in a `.proto`. Errors take their type and status from a protobuf enum. Responses are
|
|
8
|
+
encoded by the schema's own encoder.
|
|
9
|
+
|
|
10
|
+
Installing the gem switches on only the `proto_json` and `protobuf` renderers and the request
|
|
11
|
+
body parser, which leaves a request alone unless its action declares `protobuf_body`. Every other
|
|
12
|
+
piece is a concern you include into the classes that want it, and it acts on those classes only;
|
|
13
|
+
each section below starts from that setup. The gem's only hard dependencies are
|
|
14
|
+
`google-protobuf` and Rails; the integrations with [problem](https://github.com/sorah/problem),
|
|
15
|
+
[protovalidate](https://github.com/sorah/protovalidate-rb),
|
|
16
|
+
[alba](https://github.com/okuramasafumi/alba) and
|
|
17
|
+
[connect_rpc_rails](https://github.com/ivry-inc/connect_rpc_rails) are separate `require`s,
|
|
18
|
+
and none of those gems is a dependency.
|
|
19
|
+
|
|
20
|
+
```ruby
|
|
21
|
+
class WidgetsController < ApplicationController
|
|
22
|
+
include Protobufable::RequestParseable
|
|
23
|
+
|
|
24
|
+
protobuf_body Api::CreateWidgetRequest
|
|
25
|
+
def create
|
|
26
|
+
widget = Widget.create!(name: message.name)
|
|
27
|
+
render proto_json: Api::CreateWidgetResponse.new(widget: WidgetResource.new(widget).as_protobuf)
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
```console
|
|
33
|
+
$ curl -sS -X POST -H 'Content-Type: application/json' -d '{"name": 42}' https://api.example.com/v1/widgets
|
|
34
|
+
HTTP/1.1 400 Bad Request
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Features
|
|
38
|
+
|
|
39
|
+
### Request bodies
|
|
40
|
+
|
|
41
|
+
`protobuf_body` declares the message an action accepts. The JSON body is decoded into it before
|
|
42
|
+
the action runs, so a wrong type or an unknown field is a 400 and the action reads a typed
|
|
43
|
+
`message` instead of `params`. See [docs/request-bodies.md](docs/request-bodies.md).
|
|
44
|
+
|
|
45
|
+
```ruby
|
|
46
|
+
class WidgetsController < ApplicationController
|
|
47
|
+
include Protobufable::RequestParseable
|
|
48
|
+
|
|
49
|
+
protobuf_body Api::CreateWidgetRequest
|
|
50
|
+
def create
|
|
51
|
+
Widget.create!(name: message.name, tags: message.tags.to_a)
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### Responses
|
|
57
|
+
|
|
58
|
+
`render proto_json:` and `render protobuf:` encode a message with the schema's own encoder, in
|
|
59
|
+
any controller. An Alba resource bound to a message fails at load time when the two disagree,
|
|
60
|
+
rather than silently sending an empty field. See [docs/responses.md](docs/responses.md).
|
|
61
|
+
|
|
62
|
+
```ruby
|
|
63
|
+
require "protobufable/alba_binding"
|
|
64
|
+
|
|
65
|
+
class WidgetResource < ApplicationResource
|
|
66
|
+
include Protobufable::AlbaBinding
|
|
67
|
+
|
|
68
|
+
with_protobuf Api::Widget do # raises at load time unless it matches Api::Widget
|
|
69
|
+
attributes :id, :name
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
render proto_json: Api::GetWidgetResponse.new(widget: WidgetResource.new(widget).as_protobuf)
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Messages in a column
|
|
77
|
+
|
|
78
|
+
`Protobufable::JsonType` keeps a message in a `json` or `jsonb` column, declared per attribute.
|
|
79
|
+
It is an `ActiveModel` type, not a `serialize` coder, so dirty tracking and casting behave. See
|
|
80
|
+
[docs/columns.md](docs/columns.md).
|
|
81
|
+
|
|
82
|
+
```ruby
|
|
83
|
+
class Widget < ApplicationRecord
|
|
84
|
+
attribute :settings, Protobufable::JsonType.new(Internal::WidgetSettings),
|
|
85
|
+
default: -> { Internal::WidgetSettings.new }
|
|
86
|
+
end
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
### Errors
|
|
90
|
+
|
|
91
|
+
An error catalogue in the schema, built on [problem](https://github.com/sorah/problem): `type`
|
|
92
|
+
and `status` come from an enum value's annotation, so two errors needing different statuses are
|
|
93
|
+
different types, and a typo in the catalogue fails on boot. See [docs/errors.md](docs/errors.md).
|
|
94
|
+
|
|
95
|
+
```proto
|
|
96
|
+
enum ProblemType {
|
|
97
|
+
PROBLEM_TYPE_UNSPECIFIED = 0;
|
|
98
|
+
PROBLEM_TYPE_NOT_FOUND = 1 [(problem_type) = {uri: "not-found", http_status: 404}];
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
```ruby
|
|
103
|
+
require "protobufable/problem"
|
|
104
|
+
|
|
105
|
+
class ApiError < StandardError
|
|
106
|
+
include Protobufable::Problem::Typeable
|
|
107
|
+
|
|
108
|
+
def self.problem_type_enum = "myapp.api.ProblemType"
|
|
109
|
+
def self.problem_type_annotation = "myapp.api.problem_type"
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
class NotFound < ApiError
|
|
113
|
+
problem_type :PROBLEM_TYPE_NOT_FOUND
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
class ApplicationController < ActionController::API
|
|
117
|
+
include Problem::Rescuable # from problem; renders them as application/problem+json
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
raise NotFound.new(detail: "no widget w_123") # 404, "type": "not-found"
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
### Value validation
|
|
124
|
+
|
|
125
|
+
`buf.validate` rules run on request bodies and on stored columns, each opted into separately. A
|
|
126
|
+
request breaking a rule is a 400 before the action runs. See
|
|
127
|
+
[docs/validation.md](docs/validation.md), which also covers reporting the failed fields with field
|
|
128
|
+
paths in the spelling the caller sent.
|
|
129
|
+
|
|
130
|
+
```proto
|
|
131
|
+
message CreateWidgetRequest {
|
|
132
|
+
string name = 1 [(buf.validate.field).string = {min_len: 1, max_len: 100}];
|
|
133
|
+
}
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
```ruby
|
|
137
|
+
require "protobufable/protovalidate"
|
|
138
|
+
|
|
139
|
+
class WidgetsController < ApplicationController
|
|
140
|
+
include Protobufable::RequestValidatable # brings RequestParseable
|
|
141
|
+
|
|
142
|
+
protobuf_body Api::CreateWidgetRequest
|
|
143
|
+
def create = ...
|
|
144
|
+
end
|
|
145
|
+
|
|
146
|
+
class ApplicationRecord < ActiveRecord::Base
|
|
147
|
+
include Protobufable::ColumnValidatable
|
|
148
|
+
end
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### Connect RPC
|
|
152
|
+
|
|
153
|
+
A problem raised in a Connect RPC goes out as a Connect error carrying the same document, so a
|
|
154
|
+
caller dispatches on the same `type` over REST and Connect. See [docs/connect.md](docs/connect.md).
|
|
155
|
+
|
|
156
|
+
```ruby
|
|
157
|
+
require "protobufable/connect"
|
|
158
|
+
|
|
159
|
+
class WidgetsController < ActionController::API
|
|
160
|
+
include ConnectRpcRails::Controller
|
|
161
|
+
include Protobufable::Connect::ProblemRescuable # after ConnectRpcRails::Controller
|
|
162
|
+
end
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
```json
|
|
166
|
+
{"code": "not_found", "message": "no widget w_123",
|
|
167
|
+
"details": [{"type": "protobufable.ProblemDetails", "value": "..."}]}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### Typed
|
|
171
|
+
|
|
172
|
+
RBS signatures ship with the gem, and the suite type checks under Steep.
|
|
173
|
+
|
|
174
|
+
## Requirements
|
|
175
|
+
|
|
176
|
+
- Ruby 3.4 or later
|
|
177
|
+
- Rails 7.1 or later
|
|
178
|
+
- `google-protobuf` 4.26 or later
|
|
179
|
+
|
|
180
|
+
Optional dependencies are not installed for you. Add the ones for the features you use:
|
|
181
|
+
|
|
182
|
+
- [problem](https://github.com/sorah/problem): [Errors](docs/errors.md) and [Connect RPC](docs/connect.md)
|
|
183
|
+
- [protovalidate](https://github.com/sorah/protovalidate-rb): [Value validation](docs/validation.md)
|
|
184
|
+
- [alba](https://github.com/okuramasafumi/alba): [bound Alba resources](docs/responses.md#bound-alba-resources)
|
|
185
|
+
- [connect_rpc_rails](https://github.com/ivry-inc/connect_rpc_rails): [Connect RPC](docs/connect.md)
|
|
186
|
+
|
|
187
|
+
## Installation
|
|
188
|
+
|
|
189
|
+
```
|
|
190
|
+
bundle add protobufable
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Rails wires itself up through a railtie, which registers the renderers, installs the request
|
|
194
|
+
body parser, and lists a broken validation rule as a 400 in `rescue_responses`. Outside Rails,
|
|
195
|
+
call `Protobufable.install!` at boot.
|
|
196
|
+
|
|
197
|
+
## Caveats
|
|
198
|
+
|
|
199
|
+
- **`protovalidate` is a prerelease** (`0.1.0.beta3` at the time of writing).
|
|
200
|
+
- Editing a `.proto` does not hot-reload. `google-protobuf` raises on a duplicate descriptor
|
|
201
|
+
definition, so the development loop is to restart on regeneration.
|
|
202
|
+
|
|
203
|
+
Caveats specific to a feature are on its page, such as vendoring the shipped `.proto` files in
|
|
204
|
+
[docs/errors.md](docs/errors.md#caveats).
|
|
205
|
+
|
|
206
|
+
## Development
|
|
207
|
+
|
|
208
|
+
```
|
|
209
|
+
bundle install
|
|
210
|
+
bundle exec rake # specs, then rbs and steep
|
|
211
|
+
bundle exec yard # docs
|
|
212
|
+
hk check --all # rubocop, actionlint, zizmor
|
|
213
|
+
mise run protoc # regenerate the spec fixtures
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
## See also
|
|
217
|
+
|
|
218
|
+
- [DESIGN.md](DESIGN.md) for why the library is shaped this way.
|
|
219
|
+
- [problem](https://github.com/sorah/problem) for RFC 9457 itself, and
|
|
220
|
+
[connect_rpc_rails](https://github.com/ivry-inc/connect_rpc_rails) for the Connect protocol.
|
|
221
|
+
|
|
222
|
+
## Contributing
|
|
223
|
+
|
|
224
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/sorah/protobufable.
|
|
225
|
+
|
|
226
|
+
## License
|
|
227
|
+
|
|
228
|
+
The gem is available as open source under the terms of the
|
|
229
|
+
[MIT License](https://opensource.org/licenses/MIT). Copyright (c) 2026 Sorah Fukumori.
|
data/docs/columns.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# Messages in a column
|
|
2
|
+
|
|
3
|
+
```ruby
|
|
4
|
+
class Widget < ApplicationRecord
|
|
5
|
+
attribute :settings, Protobufable::JsonType.new(Internal::WidgetSettings),
|
|
6
|
+
default: -> { Internal::WidgetSettings.new }
|
|
7
|
+
end
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
The column is `json` or `jsonb`, and its shape is declared in a `.proto` rather than left to
|
|
11
|
+
whatever a Hash happened to hold.
|
|
12
|
+
|
|
13
|
+
| method | in | out |
|
|
14
|
+
|---|---|---|
|
|
15
|
+
| `deserialize` | the stored document | the message, or `nil` |
|
|
16
|
+
| `serialize` | a message | the document to store, or `nil` |
|
|
17
|
+
| `cast` | a message, String, Hash or `nil` | the message |
|
|
18
|
+
| `changed_in_place?` | the loaded document and the held message | whether they differ |
|
|
19
|
+
| `type` | — | `:json` |
|
|
20
|
+
|
|
21
|
+
Pass `nil_as_default: true` for a nullable column whose Ruby value is semantically non-null: a
|
|
22
|
+
null column then reads back as an empty message, while `nil` is still written as `NULL`.
|
|
23
|
+
|
|
24
|
+
Pass `preserve_proto_fieldnames: true` to write the `.proto` field names (`part_name`) rather
|
|
25
|
+
than their lowerCamelCase JSON names (`partName`), for a column read in SQL or by tools that use
|
|
26
|
+
the `.proto` spelling. Reads accept either, so switching an existing column needs no migration.
|
|
27
|
+
|
|
28
|
+
Reads tolerate unknown fields, so a row written by a newer schema stays readable mid-deploy.
|
|
29
|
+
Writes emit defaults, so the stored shape does not depend on which fields were set — which is
|
|
30
|
+
what makes the `changed_in_place?` comparison meaningful, and spares anything reading the column
|
|
31
|
+
in SQL from telling absent from default.
|
|
32
|
+
|
|
33
|
+
To run `buf.validate` rules over the column before save, see
|
|
34
|
+
[Value validation](validation.md#stored-columns).
|
data/docs/connect.md
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Connect RPC
|
|
2
|
+
|
|
3
|
+
[connect_rpc_rails](https://github.com/ivry-inc/connect_rpc_rails) serves Connect RPCs from
|
|
4
|
+
ActionController. Connect defines its own error body, and the Connect code decides the HTTP
|
|
5
|
+
status, so a problem cannot be answered as `application/problem+json` there.
|
|
6
|
+
`Protobufable::Connect::ProblemRescuable` answers it as a Connect error instead:
|
|
7
|
+
|
|
8
|
+
```ruby
|
|
9
|
+
require "protobufable/connect"
|
|
10
|
+
|
|
11
|
+
class WidgetsController < ActionController::API
|
|
12
|
+
include ConnectRpcRails::Controller
|
|
13
|
+
include Protobufable::Connect::ProblemRescuable
|
|
14
|
+
|
|
15
|
+
connect_service "myapp.api.WidgetsService"
|
|
16
|
+
|
|
17
|
+
def get_widget
|
|
18
|
+
widget = Widget.find_by(id: connect_request.id)
|
|
19
|
+
raise Errors::NotFound.new(detail: "no widget #{connect_request.id}") unless widget
|
|
20
|
+
|
|
21
|
+
Api::GetWidgetResponse.new(widget: WidgetResource.new(widget).as_protobuf)
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
```json
|
|
27
|
+
{"code": "not_found", "message": "no widget w_123",
|
|
28
|
+
"details": [{"type": "protobufable.ProblemDetails", "value": "..."}]}
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
The code is read off the problem's status with `ConnectRpcRails::Error.code_for_http_status`,
|
|
32
|
+
and the message is the detail, or the title when there is none. The document travels as the
|
|
33
|
+
detail, `information` included, so a caller reads the same `type` it would over REST. The HTTP
|
|
34
|
+
status is the code's: a 422 problem goes out as 400 `invalid_argument`, while the document's
|
|
35
|
+
`status` still reads 422.
|
|
36
|
+
|
|
37
|
+
Include it after `ConnectRpcRails::Controller`. It is `Problem::Rescuable` with only the
|
|
38
|
+
rendering step replaced, so reporting, headers, `problem_for` and `around_problem_render`
|
|
39
|
+
behave as they do over REST.
|
|
40
|
+
|
|
41
|
+
The message is `protobufable.ProblemDetails`, shipped in `protobufable/problem_details.proto`.
|
|
42
|
+
`information` carries the error's `problem_information` messages, packed as they are; other
|
|
43
|
+
extension members have no slot in the message and are dropped. To reference the message from
|
|
44
|
+
your own schema, for an OpenAPI document say, vendor that file (see
|
|
45
|
+
[Errors: Caveats](errors.md#caveats)). To pack a message of your own instead, override
|
|
46
|
+
`problem_message_for` in the controller:
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
private def problem_message_for(problem, error)
|
|
50
|
+
Api::ProblemDetails.new(type: problem.type, title: problem.title, status: problem.status)
|
|
51
|
+
end
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Only a problem raised inside the controller is answered this way. An exception escaping before
|
|
55
|
+
dispatch reaches the exceptions app, which connect_rpc_rails answers from `rescue_responses`.
|