restless-sdk 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CONFORMANCE.md +13 -8
- data/LICENSE +18 -0
- data/README.md +66 -6
- data/exe/restless-conformance +1 -1
- data/install.md +35 -9
- data/lib/restless/capture.rb +86 -5
- data/lib/restless/client.rb +29 -1
- data/lib/restless/conformance.rb +43 -1
- data/lib/restless/injection.rb +31 -16
- data/lib/restless/rack.rb +54 -8
- data/lib/restless/sampling.rb +225 -0
- data/lib/restless/settings.rb +7 -4
- data/lib/restless/uploader.rb +53 -1
- data/lib/restless/version.rb +2 -2
- data/lib/restless.rb +2 -1
- data/spec/VECTORS_VERSION +1 -1
- data/spec/vectors/debug-injection.json +219 -0
- data/spec/vectors/fingerprint.json +1 -1
- data/spec/vectors/har.json +1 -1
- data/spec/vectors/mask.json +1 -1
- data/spec/vectors/recovery-slug.json +1 -1
- data/spec/vectors/redact.json +1 -1
- data/spec/vectors/request-id.json +1 -1
- data/spec/vectors/sampling.json +738 -0
- metadata +10 -10
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f2ab526956a3600f1166a74828a06c9b01b5a0ce2a17808bdba530f6ea344ee9
|
|
4
|
+
data.tar.gz: 4e5776ac3709c2f1e8b5d0a710d45335927a331eddb0cd9b027a0bd8305c42ee
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ce0a15152665dba1b04f64377e6b6dc2e773f128b48ae6fdcddedc0f35fc34df3c40716470857f16c35b8409793b03af07e2bda387af9a24ba8636bcc419149c
|
|
7
|
+
data.tar.gz: 395c589c7b35aa121e9612fd0142e32df3c2f0dc75cc3eeb8607912b73d279ef046a3478f4d63e7a548d813ca4a33ae1212a18bef2a7b60f3bdb0ba88d306a55
|
data/CONFORMANCE.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
| | |
|
|
4
4
|
|---|---|
|
|
5
|
-
| **Spec version** | 1.
|
|
5
|
+
| **Spec version** | 1.1.0 |
|
|
6
6
|
| **Level** | L2 (core + batching, caches, injection, safety) |
|
|
7
7
|
| **Reference** | `restlesshq/node` (`@restlessai/sdk`) |
|
|
8
8
|
| **Driver** | `ruby exe/restless-conformance` |
|
|
@@ -12,7 +12,7 @@ Declared in `lib/restless/version.rb` (META-001).
|
|
|
12
12
|
## Verifying
|
|
13
13
|
|
|
14
14
|
The harness and vectors live in the reference SDK, so the commands below
|
|
15
|
-
assume it is checked out as a sibling (`../node
|
|
15
|
+
assume it is checked out as a sibling (`../node`), which is how
|
|
16
16
|
`setup.sh` in the install repo arranges things. The vectors in `spec/` here
|
|
17
17
|
are a pinned copy, so `ruby -Ilib -Itest test/all.rb` alone works without it.
|
|
18
18
|
|
|
@@ -22,18 +22,18 @@ are a pinned copy, so `ruby -Ilib -Itest test/all.rb` alone works without it.
|
|
|
22
22
|
ruby -Ilib -Itest test/all.rb
|
|
23
23
|
|
|
24
24
|
# the shared cross-language harness
|
|
25
|
-
node ../node
|
|
25
|
+
node ../node/spec/harness/run-vectors.mjs -- ruby exe/restless-conformance
|
|
26
26
|
|
|
27
27
|
# differential fuzz against the reference implementation
|
|
28
|
-
node ../node
|
|
29
|
-
--ref "node ../node
|
|
28
|
+
node ../node/spec/harness/fuzz.mjs \
|
|
29
|
+
--ref "node ../node/spec/driver/.build/node.js" \
|
|
30
30
|
--test "ruby exe/restless-conformance" \
|
|
31
31
|
--iterations 8000 --seed 24301
|
|
32
32
|
```
|
|
33
33
|
|
|
34
|
-
Current status: **
|
|
35
|
-
divergence across
|
|
36
|
-
99991, 777).
|
|
34
|
+
Current status: **252 vectors, 244 passed, 0 failed, 8 skipped.** Zero
|
|
35
|
+
divergence across 35,926 fuzz comparisons on five seeds (24301, 1, 4242,
|
|
36
|
+
99991, 777), including the four sampling ops.
|
|
37
37
|
|
|
38
38
|
The 8 skips are cases outside this implementation's dialect, not gaps:
|
|
39
39
|
|
|
@@ -71,6 +71,11 @@ the reference. They are the reason this SDK is byte-compatible.
|
|
|
71
71
|
| FP-042 | `project_relative` splits on `/` and scans BACKWARDS from the second-to-last segment, so the LAST project dir wins and a deployment root named `/app` (Docker `WORKDIR`, Heroku) cannot survive into the key. `split("/", -1)`: without the negative limit Ruby drops the trailing empty field, so `/proj/src/` would miss `src` here and match it in JavaScript. |
|
|
72
72
|
| FP-043 | Ruby's `Exception#backtrace` is innermost-FIRST, like v8 and Go and unlike a Python traceback, so the walk goes forwards. Verified empirically in `test/test_stack_frames.rb` rather than assumed. |
|
|
73
73
|
| FP-044 | Frames are `path:line:in \`method'` (`'method'` from Ruby 3.4); both quote styles parse. `block (N levels) in foo` is normalized to `block in foo`, since the nesting count behaves like a line number. Skips are matched by FILE PATH: the SDK's own directory (resolved from `__dir__`), `RbConfig`'s stdlib dirs, `Gem.path`, any `/gems/` segment, and `<internal:` frames. Never by module or class name - a name check would also skip a customer's own Restless-flavoured code and would fail to skip this gem when vendored under another constant. |
|
|
74
|
+
| SAMPLE-002 | `(r.to_f * 10000 + 0.5).floor`, never `round`. JSON hands over `1` as an Integer and `0.5` as a Float, so the value goes through `to_f` first; a quantized rate is a Float, which is why a rate of 0 serializes as `0.0` (the same JSON number). `true` is not `Numeric` in Ruby, so it needs no special case to be invalid. |
|
|
75
|
+
| SAMPLE-010 | Keys are trimmed of U+0020 only (`sub(/\A +/, "")`), never `strip`, which would also eat a trailing tab or NUL and turn the invalid key `"GET /x\t"` into a valid one. The method is uppercased with `upcase(:ascii)`. |
|
|
76
|
+
| SAMPLE-032 | The decision is the first thing `finish` does. Rack returns status and headers together before the server writes anything, so unlike Node there is no `writeHead` to intercept: none of the SDK's headers exist yet, even for a streaming body or a hijack. A drop returns the app's response triple untouched, body unread. |
|
|
77
|
+
| SAMPLE-040 | The rate token is `format("0.%04d", k)` from the integer step count. `String#sort` compares bytes, which is the order the contract asks for. |
|
|
78
|
+
| Section 14 | `sampling:` accepts symbol or string keys (`default:`, `"routes"`) and symbol modes (`:off`). Neither can come from the settings file or the driver, so the vectors are unaffected. |
|
|
74
79
|
| INJECT-005 | `Text.ws_trim`, not `String#strip`. Ruby's strip removes NUL and leaves NBSP; the contract wants exactly the PRIM-002 set. |
|
|
75
80
|
| BATCH-003 | The environment indicator is `RESTLESS_ENV`, then `RACK_ENV`, then `RAILS_ENV`. Ruby has three spellings in the wild and no single winner. |
|
|
76
81
|
| BATCH-008 | Test detection keys on those same variables being `test`, plus `RSpec::Core`, `Minitest::Test` and `Test::Unit::TestCase` being defined. |
|
data/LICENSE
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
Copyright © 2026 Restless
|
|
2
|
+
|
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
|
4
|
+
this software and associated documentation files (the “Software”), to deal in
|
|
5
|
+
the Software without restriction, including without limitation the rights to
|
|
6
|
+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
|
7
|
+
the Software, and to permit persons to whom the Software is furnished to do so,
|
|
8
|
+
subject to the following conditions:
|
|
9
|
+
|
|
10
|
+
The above copyright notice and this permission notice shall be included in all
|
|
11
|
+
copies or substantial portions of the Software.
|
|
12
|
+
|
|
13
|
+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
14
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
|
15
|
+
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
|
16
|
+
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
|
17
|
+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, OUT OF OR IN CONNECTION WITH THE
|
|
18
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
CHANGED
|
@@ -1,3 +1,18 @@
|
|
|
1
|
+
<a href="https://restless.ai">
|
|
2
|
+
<picture>
|
|
3
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/restless-init-dark.svg">
|
|
4
|
+
<img width="100%" src="docs/restless-init.svg" alt="Restless">
|
|
5
|
+
</picture>
|
|
6
|
+
</a>
|
|
7
|
+
|
|
8
|
+
Run in your codebase to get started:
|
|
9
|
+
|
|
10
|
+
```sh
|
|
11
|
+
npx restless init
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
This scans your project, figures out your framework, generates an OpenAPI spec, and automatically wires the SDK into your server.
|
|
15
|
+
|
|
1
16
|
# restless-sdk
|
|
2
17
|
|
|
3
18
|
Capture your API traffic and send it to [Restless](https://restless.ai).
|
|
@@ -5,7 +20,9 @@ Capture your API traffic and send it to [Restless](https://restless.ai).
|
|
|
5
20
|
One Rack middleware covers **Rails**, **Sinatra**, **Hanami**, **Grape**,
|
|
6
21
|
**Roda** and anything else that speaks Rack. Ruby 2.6+. No dependencies.
|
|
7
22
|
|
|
8
|
-
##
|
|
23
|
+
## Manual installation
|
|
24
|
+
|
|
25
|
+
If you'd rather wire it up by hand:
|
|
9
26
|
|
|
10
27
|
```sh
|
|
11
28
|
gem install restless-sdk
|
|
@@ -68,8 +85,8 @@ and body your app produced rather than an inner layer's.
|
|
|
68
85
|
and friends are redacted before anything leaves your process. Bodies with
|
|
69
86
|
nothing to redact are passed through byte for byte, so your payloads are
|
|
70
87
|
never re-serialized on the way out.
|
|
71
|
-
- **Error triage.**
|
|
72
|
-
and a `debug` block in the JSON body. If someone attaches a "next steps"
|
|
88
|
+
- **Error triage.** Every response gets `x-log-url` and `x-debug` headers,
|
|
89
|
+
and 4xx/5xx responses also get a `debug` block in the JSON body. If someone attaches a "next steps"
|
|
73
90
|
message to an error in the dashboard, the SDK injects it as
|
|
74
91
|
`debug.recovery` - read synchronously from an in-process cache, never
|
|
75
92
|
blocking the response on a network call. A 5xx that groups by its throw
|
|
@@ -79,6 +96,50 @@ and body your app produced rather than an inner layer's.
|
|
|
79
96
|
request before your handler runs, or
|
|
80
97
|
`{ block: { status: 402, message: "Payment required" } }` to pick the
|
|
81
98
|
status.
|
|
99
|
+
- **Sampling.** Send less for noisy endpoints without losing your counts.
|
|
100
|
+
See below.
|
|
101
|
+
|
|
102
|
+
## Sampling
|
|
103
|
+
|
|
104
|
+
Each endpoint is `"on"` (the default), `"off"`, `"errors"` (only status
|
|
105
|
+
>= 400), or a rate such as `0.1`: that fraction of successful requests is
|
|
106
|
+
logged in full and the rest without bodies, so counts, latency and
|
|
107
|
+
per-customer usage stay exact. Errors are never sampled.
|
|
108
|
+
|
|
109
|
+
```ruby
|
|
110
|
+
CLIENT = Restless.new(ENV["RESTLESS_KEY"], sampling: {
|
|
111
|
+
default: "on",
|
|
112
|
+
routes: {
|
|
113
|
+
"GET /health" => "off",
|
|
114
|
+
"GET /status" => "errors",
|
|
115
|
+
"GET /pets" => 0.1,
|
|
116
|
+
"* /pets/{id}" => 0.5,
|
|
117
|
+
},
|
|
118
|
+
})
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
| mode | successes (< 400) | errors (>= 400) |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| `"on"` (default) | full log | full log |
|
|
124
|
+
| a rate `0` to `1` | that fraction in full, the rest without bodies | full log |
|
|
125
|
+
| `"errors"` | nothing sent | full log |
|
|
126
|
+
| `"off"` | nothing sent | nothing sent |
|
|
127
|
+
|
|
128
|
+
Keys are a method (or `*` for any) and the route pattern your framework
|
|
129
|
+
matched (see [Route patterns](#route-patterns)); `{id}` and `:id` are the
|
|
130
|
+
same, and `HEAD` does not fall back to `GET`. A request with no route
|
|
131
|
+
pattern uses `default`. Symbols work for the modes (`:off`, `:errors`).
|
|
132
|
+
|
|
133
|
+
The same `sampling` block can live under an API entry in
|
|
134
|
+
`.restless/settings.json`, and the option in code wins per route. A request
|
|
135
|
+
that sends nothing (an `"off"` route, or a success on an `"errors"` route)
|
|
136
|
+
also gets no `x-request-id` header and no debug headers, since there is no
|
|
137
|
+
log for them to point at. An invalid value never raises: it falls back to
|
|
138
|
+
`"on"` and says so under `DEBUG=restless`.
|
|
139
|
+
|
|
140
|
+
With sampling configured, the SDK tells Restless the configuration once, on
|
|
141
|
+
the first request it serves (never at boot), so the dashboard can show an
|
|
142
|
+
`"off"` endpoint as off rather than as idle.
|
|
82
143
|
|
|
83
144
|
## The setup callback
|
|
84
145
|
|
|
@@ -138,11 +199,10 @@ re-raised untouched, leaving your own error handling completely unaffected.
|
|
|
138
199
|
|
|
139
200
|
## Conformance
|
|
140
201
|
|
|
141
|
-
This SDK implements version 1.
|
|
202
|
+
This SDK implements version 1.1.0 of the Restless SDK Contract at level L2,
|
|
142
203
|
and is verified against the shared cross-language conformance vectors and a
|
|
143
204
|
differential fuzzer run against the reference implementation. See
|
|
144
205
|
[CONFORMANCE.md](./CONFORMANCE.md).
|
|
145
206
|
|
|
146
207
|
## License
|
|
147
|
-
|
|
148
|
-
ISC
|
|
208
|
+
MIT
|
data/exe/restless-conformance
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env ruby
|
|
2
2
|
# frozen_string_literal: true
|
|
3
3
|
|
|
4
|
-
# The Ruby conformance driver. See
|
|
4
|
+
# The Ruby conformance driver. See sdk/node/spec/driver/PROTOCOL.md.
|
|
5
5
|
#
|
|
6
6
|
# Dev-only: it is a thin shell over Restless::Conformance, which
|
|
7
7
|
# test/test_vectors.rb also uses, so the vectors cannot describe behaviour the
|
data/install.md
CHANGED
|
@@ -191,29 +191,30 @@ api_key: CLIENT.mask(request.header("Authorization") || "anonymous")
|
|
|
191
191
|
|
|
192
192
|
## 7. `.restless/settings.json`
|
|
193
193
|
|
|
194
|
-
Read at startup, walking up from the working directory. Created and owned by the `
|
|
194
|
+
Read at startup, walking up from the working directory. Created and owned by the `restless` CLI (`npx restless init`). Every Restless SDK reads the same file with the same camelCase keys, so a polyglot repo needs only one.
|
|
195
195
|
|
|
196
196
|
```json
|
|
197
197
|
{
|
|
198
198
|
"version": 1,
|
|
199
|
-
"projectId": "<team/workspace uuid>",
|
|
200
199
|
"apis": [
|
|
201
200
|
{
|
|
202
201
|
"id": "<api uuid>",
|
|
203
202
|
"name": "Public API",
|
|
204
203
|
"rootDir": ".",
|
|
204
|
+
"projectId": "<restless project uuid>",
|
|
205
205
|
"oasFile": ".restless/openapi.yaml",
|
|
206
206
|
"framework": "rails",
|
|
207
207
|
"language": "ruby",
|
|
208
208
|
"baseUrl": "https://api.example.com",
|
|
209
209
|
"requestIdPrefix": "PUB",
|
|
210
|
-
"redact": { "headers": ["x-company-auth"], "bodyKeys": ["ssh_private_key"] }
|
|
210
|
+
"redact": { "headers": ["x-company-auth"], "bodyKeys": ["ssh_private_key"] },
|
|
211
|
+
"sampling": { "default": "on", "routes": { "GET /health": "off", "GET /pets": 0.1 } }
|
|
211
212
|
}
|
|
212
213
|
]
|
|
213
214
|
}
|
|
214
215
|
```
|
|
215
216
|
|
|
216
|
-
The SDK reads `requestIdPrefix` and `
|
|
217
|
+
The SDK reads `requestIdPrefix`, `redact` and `sampling` (§8a, merged with the `sampling:` option, code winning per route) from the matching entry. The rest, `projectId` included - which is per-API and never top-level - belongs to the CLI, not the runtime. Pick one when several are defined:
|
|
217
218
|
|
|
218
219
|
```ruby
|
|
219
220
|
Restless.new(ENV["RESTLESS_KEY"], api: "Public API")
|
|
@@ -243,11 +244,36 @@ Sentinel format, which the dashboard pattern-matches:
|
|
|
243
244
|
|
|
244
245
|
Bodies are capped at **256 KiB** (UTF-8 bytes) and truncated with `[...TRUNCATED: original N bytes]`.
|
|
245
246
|
|
|
247
|
+
## 8a. Sampling (off by default)
|
|
248
|
+
|
|
249
|
+
Without a `sampling` block every request is logged in full. With one, each endpoint gets a mode:
|
|
250
|
+
|
|
251
|
+
| mode | successes (< 400) | errors (>= 400) |
|
|
252
|
+
|---|---|---|
|
|
253
|
+
| `"on"` (default) | full log | full log |
|
|
254
|
+
| a rate `0` to `1` | that fraction in full, the rest without bodies | full log |
|
|
255
|
+
| `"errors"` | nothing sent | full log |
|
|
256
|
+
| `"off"` | nothing sent | nothing sent |
|
|
257
|
+
|
|
258
|
+
```ruby
|
|
259
|
+
Restless.new(ENV["RESTLESS_KEY"], sampling: {
|
|
260
|
+
routes: { "GET /health" => "off", "GET /pets" => 0.1, "* /internal/{job}" => "errors" },
|
|
261
|
+
})
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
Rules an installer should know:
|
|
265
|
+
|
|
266
|
+
- Keys are `"<METHOD> <route pattern>"`, with `*` for any method. Use the pattern the framework matched (`/pets/{id}` or `/pets/:id`, both work; §4 "Route patterns"), never a concrete URL like `/pets/42`. `HEAD` does not fall back to `GET`. A request with no route pattern uses `default`.
|
|
267
|
+
- Modes are the strings `"on"`, `"off"`, `"errors"` (symbols work in code), or a number from 0 to 1. Anything else falls back to `"on"`; it never raises.
|
|
268
|
+
- Only suggest `"off"` or `"errors"` for endpoints the user agrees are noise (health checks, readiness probes, polling). Never sample auth or payment endpoints without asking.
|
|
269
|
+
- A request that sends nothing also gets no `x-request-id` header, so do not use an `"off"` route to verify the install (§13).
|
|
270
|
+
|
|
246
271
|
## 9. Request IDs and response injection
|
|
247
272
|
|
|
248
273
|
- Request IDs are v4 UUIDs, never time-based, so they leak no ordering.
|
|
249
|
-
-
|
|
250
|
-
- On status
|
|
274
|
+
- The SDK sets exactly one id header, always its own: `x-request-id` by default, or `x-restless-id` when the incoming request already carried an `x-request-id` (we never stomp or reuse an existing chain). With no key resolved the value is the literal `missing-key`.
|
|
275
|
+
- On **every** status the SDK adds `x-log-url` and `x-debug` headers; on status **>= 400** it also merges a `debug` key into a JSON body. There is no user-configurable hook for this. The one exception is a request that sampling drops (§8a): it gets no id header and nothing injected.
|
|
276
|
+
- `x-log-url` points at your project's public docs host, which the server tells the SDK on each upload. Until the first upload round-trips it is omitted rather than guessed: a URL that 404s is worse than no URL. The ingest host is never used for it.
|
|
251
277
|
|
|
252
278
|
## 10. Blocking
|
|
253
279
|
|
|
@@ -260,7 +286,7 @@ CLIENT.setup do |request|
|
|
|
260
286
|
end
|
|
261
287
|
```
|
|
262
288
|
|
|
263
|
-
The handler never runs. The response body is JSON `{"error": "<message>"}`. `enrich` is not called for a blocked request.
|
|
289
|
+
The handler never runs. The response body is JSON `{"error": "<message>"}`. `enrich` is not called for a blocked request. Sampling (§8a) treats a blocked request like any other response with its status: a 403 or 429 is logged in full, except on an `"off"` route, where it is not logged and gets no id header.
|
|
264
290
|
|
|
265
291
|
## 11. Environment variables and batching
|
|
266
292
|
|
|
@@ -295,5 +321,5 @@ Batching is fixed: 10 requests per batch, a 5000 ms flush interval, a 1000-entry
|
|
|
295
321
|
1. `restless-sdk` appears in the `Gemfile`, and `bundle list` finds it.
|
|
296
322
|
2. `CLIENT.rack` is mounted in `config.ru` or `config/application.rb`, as far out as possible.
|
|
297
323
|
3. A `CLIENT.setup` block exists and reads its header via `request.header(...)`.
|
|
298
|
-
4. `.restless/settings.json` exists (created by `npx
|
|
299
|
-
5. Starting the server and curling any endpoint returns an `x-
|
|
324
|
+
4. `.restless/settings.json` exists (created by `npx restless init`).
|
|
325
|
+
5. Starting the server and curling any endpoint returns an `x-request-id` response header carrying a fresh id - `<prefix>-<uuid>` when the API has a `requestIdPrefix` in `.restless/settings.json` (the CLI sets one on every project), otherwise a bare UUID. If your curl sends its own `x-request-id`, look for `x-restless-id` instead. Plenty of stacks set `x-request-id` themselves, so the unambiguous proof it came through our SDK is the `x-debug` header, which rides every captured response. A value of `missing-key` means the server is up but never loaded `RESTLESS_KEY`; restart it.
|
data/lib/restless/capture.rb
CHANGED
|
@@ -5,6 +5,7 @@ require "json"
|
|
|
5
5
|
require_relative "caches"
|
|
6
6
|
require_relative "fingerprint"
|
|
7
7
|
require_relative "redact"
|
|
8
|
+
require_relative "sampling"
|
|
8
9
|
require_relative "uploader"
|
|
9
10
|
|
|
10
11
|
module Restless
|
|
@@ -20,12 +21,22 @@ module Restless
|
|
|
20
21
|
|
|
21
22
|
attr_reader :uploader, :enrich_cache, :recovery_cache
|
|
22
23
|
|
|
24
|
+
# `sampling` is a Sampling::Config, already merged and validated. Nil, or
|
|
25
|
+
# the default config, means no sampling at all.
|
|
23
26
|
def initialize(api_key:, base_url:, request_id_prefix: nil, redact: nil,
|
|
24
|
-
transport: nil)
|
|
27
|
+
sampling: nil, transport: nil)
|
|
25
28
|
@redact = redact || {}
|
|
29
|
+
# Only kept for a non-default config (SAMPLE-041). Without one, `sample`
|
|
30
|
+
# is a constant :full and nothing about sampling touches the wire.
|
|
31
|
+
@sampling = sampling && !sampling.default? ? sampling : nil
|
|
32
|
+
@sampling_hash = @sampling && Sampling.digest(@sampling)
|
|
33
|
+
@sampling_report = :unsent
|
|
34
|
+
@sampling_mutex = Mutex.new
|
|
35
|
+
# Test hook: anything responding to `call` with a Float in [0, 1).
|
|
36
|
+
@random = -> { Random.rand }
|
|
26
37
|
@enrich_cache = EnrichCache.new
|
|
27
38
|
@recovery_cache = RecoveryCache.new
|
|
28
|
-
@
|
|
39
|
+
@portal_url = nil
|
|
29
40
|
@docs_mutex = Mutex.new
|
|
30
41
|
@callback = nil
|
|
31
42
|
@uploader = Uploader.new(
|
|
@@ -33,18 +44,42 @@ module Restless
|
|
|
33
44
|
base_url: base_url,
|
|
34
45
|
request_id_prefix: request_id_prefix,
|
|
35
46
|
transport: transport,
|
|
47
|
+
sampling_hash: @sampling_hash,
|
|
36
48
|
on_response: method(:handle_server_response)
|
|
37
49
|
)
|
|
38
50
|
end
|
|
39
51
|
|
|
52
|
+
attr_writer :random
|
|
53
|
+
|
|
54
|
+
# The sampling outcome for one request (CONTRACT.md SAMPLE-030): :full,
|
|
55
|
+
# :stub or :drop. The middleware calls this exactly once per request, when
|
|
56
|
+
# the status is final and before any SDK header is added (SAMPLE-032),
|
|
57
|
+
# and acts on the answer.
|
|
58
|
+
#
|
|
59
|
+
# Never raises. Anything unexpected sends the request in full
|
|
60
|
+
# (SAMPLE-035): sampling may only ever remove data, never break capture.
|
|
61
|
+
def sample(method, route_pattern, status)
|
|
62
|
+
return :full if @sampling.nil?
|
|
63
|
+
|
|
64
|
+
report_sampling(false)
|
|
65
|
+
mode = Sampling.resolve_mode(@sampling, method, route_pattern)
|
|
66
|
+
Sampling.decide(mode, status.to_i, @random.call)
|
|
67
|
+
rescue StandardError => e
|
|
68
|
+
Env.debug_log("sampling failed, sending in full: #{e.class}: #{e.message}")
|
|
69
|
+
:full
|
|
70
|
+
end
|
|
71
|
+
|
|
40
72
|
def callback=(callback)
|
|
41
73
|
@callback = callback
|
|
42
74
|
end
|
|
43
75
|
|
|
44
76
|
# INJECT-006. The latest server-resolved docs origin, or nil when no batch
|
|
45
77
|
# has round-tripped yet.
|
|
46
|
-
|
|
47
|
-
|
|
78
|
+
# INJECT-006. The server-published portal origin every injected URL is
|
|
79
|
+
# built on. Nil before the first upload round-trip, and then nothing is
|
|
80
|
+
# emitted rather than a guess.
|
|
81
|
+
def portal_url
|
|
82
|
+
@docs_mutex.synchronize { @portal_url }
|
|
48
83
|
end
|
|
49
84
|
|
|
50
85
|
def flush
|
|
@@ -53,6 +88,9 @@ module Restless
|
|
|
53
88
|
|
|
54
89
|
# WIRE-020..023, CACHE-006, CACHE-011..013.
|
|
55
90
|
def handle_server_response(body, batch_fingerprints)
|
|
91
|
+
# WIRE-025, SAMPLE-053.
|
|
92
|
+
report_sampling(true) if body["needsSamplingConfig"] == true
|
|
93
|
+
|
|
56
94
|
needs = body["needsEnrichment"]
|
|
57
95
|
if needs.is_a?(Array)
|
|
58
96
|
needs.each { |key| @enrich_cache.invalidate(key) if key.is_a?(String) }
|
|
@@ -61,7 +99,9 @@ module Restless
|
|
|
61
99
|
docs = body["docsUrl"]
|
|
62
100
|
if docs.is_a?(String) && !docs.empty?
|
|
63
101
|
# Origin only; strip trailing slashes so the server can be lax.
|
|
64
|
-
|
|
102
|
+
# The wire key stays `docsUrl`: every already-deployed SDK reads it,
|
|
103
|
+
# so renaming would strand them all with no portal origin (WIRE-023).
|
|
104
|
+
@docs_mutex.synchronize { @portal_url = docs.sub(%r{/+\z}, "") }
|
|
65
105
|
end
|
|
66
106
|
|
|
67
107
|
messages = body["recoveryMessages"].is_a?(Hash) ? body["recoveryMessages"] : {}
|
|
@@ -207,6 +247,13 @@ module Restless
|
|
|
207
247
|
def record(captured, stack_frame: nil)
|
|
208
248
|
request = captured["request"] || {}
|
|
209
249
|
response = captured["response"] || {}
|
|
250
|
+
if captured["sampled"]
|
|
251
|
+
# A stub ships as if no body had been captured (SAMPLE-034). Dropping
|
|
252
|
+
# them here, before redaction, skips the parse work for exactly the
|
|
253
|
+
# requests the customer asked to send less of.
|
|
254
|
+
request = request.merge("body" => nil)
|
|
255
|
+
response = response.merge("body" => nil)
|
|
256
|
+
end
|
|
210
257
|
request_headers = request["headers"] || {}
|
|
211
258
|
response_headers = response["headers"] || {}
|
|
212
259
|
|
|
@@ -246,6 +293,40 @@ module Restless
|
|
|
246
293
|
|
|
247
294
|
private
|
|
248
295
|
|
|
296
|
+
# Tell the ingest which config produced this process's uploads, so an
|
|
297
|
+
# "off" endpoint does not read as a dead one (SAMPLE-050..053). Once per
|
|
298
|
+
# process, on the first decision; again only when the server asks, and
|
|
299
|
+
# never while a report is already in flight.
|
|
300
|
+
def report_sampling(resend)
|
|
301
|
+
return if @sampling.nil?
|
|
302
|
+
|
|
303
|
+
send_now = @sampling_mutex.synchronize do
|
|
304
|
+
if @sampling_report == :inflight || (@sampling_report == :sent && !resend)
|
|
305
|
+
false
|
|
306
|
+
else
|
|
307
|
+
@sampling_report = :inflight
|
|
308
|
+
true
|
|
309
|
+
end
|
|
310
|
+
end
|
|
311
|
+
return unless send_now
|
|
312
|
+
|
|
313
|
+
report = {
|
|
314
|
+
"hash" => @sampling_hash,
|
|
315
|
+
"default" => @sampling.default,
|
|
316
|
+
"routes" => Sampling.sorted_routes(@sampling),
|
|
317
|
+
"creator" => { "name" => SDK_NAME, "version" => VERSION }
|
|
318
|
+
}
|
|
319
|
+
# Fire and forget on a background thread, and a failure is not retried
|
|
320
|
+
# (SAMPLE-051): the `x-restless-sampling` header on the next upload is
|
|
321
|
+
# how the server asks again if it never got this.
|
|
322
|
+
@uploader.send_sampling_report_async(report) do
|
|
323
|
+
@sampling_mutex.synchronize { @sampling_report = :sent }
|
|
324
|
+
end
|
|
325
|
+
rescue StandardError => e
|
|
326
|
+
@sampling_mutex.synchronize { @sampling_report = :sent }
|
|
327
|
+
Env.debug_log("sampling report failed: #{e.class}: #{e.message}")
|
|
328
|
+
end
|
|
329
|
+
|
|
249
330
|
# Accept both `:api_key`/`"apiKey"` spellings so the callback reads
|
|
250
331
|
# naturally in Ruby without losing the documented wire names (SETUP-001).
|
|
251
332
|
def normalize_setup(raw)
|
data/lib/restless/client.rb
CHANGED
|
@@ -4,6 +4,7 @@ require_relative "capture"
|
|
|
4
4
|
require_relative "env"
|
|
5
5
|
require_relative "mask"
|
|
6
6
|
require_relative "request_id"
|
|
7
|
+
require_relative "sampling"
|
|
7
8
|
require_relative "settings"
|
|
8
9
|
require_relative "version"
|
|
9
10
|
|
|
@@ -22,10 +23,25 @@ module Restless
|
|
|
22
23
|
# them (REDACT-014). Both the settings file and this option are additive
|
|
23
24
|
# on top of the defaults (REDACT-015).
|
|
24
25
|
#
|
|
26
|
+
# `sampling` sends less for high-traffic endpoints (CONTRACT.md section
|
|
27
|
+
# 12.1). Each endpoint is "on" (the default), "off", "errors" (only
|
|
28
|
+
# status >= 400) or a rate in [0, 1]: that fraction of successful
|
|
29
|
+
# requests in full and the rest without bodies. Errors are never sampled.
|
|
30
|
+
#
|
|
31
|
+
# sampling: {
|
|
32
|
+
# default: "on",
|
|
33
|
+
# routes: { "GET /health" => "off", "GET /pets" => 0.1, "* /pets/{id}" => "errors" }
|
|
34
|
+
# }
|
|
35
|
+
#
|
|
36
|
+
# Symbols work too (`default: :off`). Merged with `sampling` in
|
|
37
|
+
# `.restless/settings.json`, this option winning per route (SAMPLE-014).
|
|
38
|
+
# An invalid value never raises: it falls back to "on" and says so under
|
|
39
|
+
# DEBUG=restless.
|
|
40
|
+
#
|
|
25
41
|
# `transport` is an internal test hook -- anything responding to
|
|
26
42
|
# `call(url, headers, body) -> [status, body]` -- and is not public API.
|
|
27
43
|
def initialize(api_key = nil, base_url: nil, api: nil, redact: nil,
|
|
28
|
-
transport: nil)
|
|
44
|
+
sampling: nil, transport: nil)
|
|
29
45
|
resolved_key = Env.resolve_api_key(api_key)
|
|
30
46
|
|
|
31
47
|
# CONFIG-002. Construction still succeeds and capture still runs; only
|
|
@@ -41,11 +57,23 @@ module Restless
|
|
|
41
57
|
entry = Settings.resolve_api(Settings.load, api)
|
|
42
58
|
settings_redact = entry && entry[:redact]
|
|
43
59
|
|
|
60
|
+
sampling_config = begin
|
|
61
|
+
config, warnings = Sampling.normalize(entry && entry[:sampling], sampling)
|
|
62
|
+
warnings.each { |w| Env.debug_log(w) }
|
|
63
|
+
config
|
|
64
|
+
rescue StandardError => e
|
|
65
|
+
# SAMPLE-003: a bad config never raises. Without one, every request
|
|
66
|
+
# is sent in full.
|
|
67
|
+
Env.debug_log("sampling config ignored: #{e.class}: #{e.message}")
|
|
68
|
+
nil
|
|
69
|
+
end
|
|
70
|
+
|
|
44
71
|
@engine = CaptureEngine.new(
|
|
45
72
|
api_key: resolved_key,
|
|
46
73
|
base_url: Env.resolve_base_url(base_url),
|
|
47
74
|
request_id_prefix: entry && entry[:request_id_prefix],
|
|
48
75
|
redact: merge_redact(settings_redact, redact),
|
|
76
|
+
sampling: sampling_config,
|
|
49
77
|
transport: transport
|
|
50
78
|
)
|
|
51
79
|
end
|
data/lib/restless/conformance.rb
CHANGED
|
@@ -7,6 +7,7 @@ require_relative "stack_frames"
|
|
|
7
7
|
require_relative "har"
|
|
8
8
|
require_relative "request_id"
|
|
9
9
|
require_relative "injection"
|
|
10
|
+
require_relative "sampling"
|
|
10
11
|
|
|
11
12
|
module Restless
|
|
12
13
|
# The shared operation table behind the conformance driver.
|
|
@@ -16,7 +17,7 @@ module Restless
|
|
|
16
17
|
# through this one file, so it is impossible for the vectors to describe
|
|
17
18
|
# behaviour the driver does not exhibit.
|
|
18
19
|
#
|
|
19
|
-
# See
|
|
20
|
+
# See sdk/node/spec/driver/PROTOCOL.md. Internal: never part of the public
|
|
20
21
|
# API, never shipped behaviour a customer depends on.
|
|
21
22
|
module Conformance
|
|
22
23
|
# An input this implementation's language cannot represent or parse. The
|
|
@@ -79,16 +80,57 @@ module Restless
|
|
|
79
80
|
# --- injection (section 10) ---
|
|
80
81
|
when "recoverySlug"
|
|
81
82
|
Injection.recovery_slug(str_or_nil(input["method"]), str_or_nil(input["path"]))
|
|
83
|
+
when "debugInjection"
|
|
84
|
+
debug_injection(input)
|
|
82
85
|
|
|
83
86
|
# --- HAR (section 7) ---
|
|
84
87
|
when "harEntry"
|
|
85
88
|
Har.to_har_entry(hash(input["captured"]))
|
|
86
89
|
|
|
90
|
+
# --- sampling (section 12.1) ---
|
|
91
|
+
# Routes come back as sorted [key, mode] pairs so the comparison pins
|
|
92
|
+
# the normalized keys without depending on hash order. Warnings are
|
|
93
|
+
# debug output, not contract surface, so they are not emitted.
|
|
94
|
+
when "samplingConfig"
|
|
95
|
+
config = sampling_config(input)
|
|
96
|
+
{ "default" => config.default, "routes" => config.routes.sort_by { |key, _| key } }
|
|
97
|
+
when "samplingMode"
|
|
98
|
+
Sampling.resolve_mode(sampling_config(input), str(input["method"]),
|
|
99
|
+
str_or_nil(input["route"]))
|
|
100
|
+
when "samplingDecide"
|
|
101
|
+
status = input["status"].is_a?(Numeric) ? input["status"] : 0
|
|
102
|
+
u = input["u"].is_a?(Numeric) ? input["u"] : 0
|
|
103
|
+
Sampling.decide(input["mode"], status, u).to_s
|
|
104
|
+
when "samplingHash"
|
|
105
|
+
config = sampling_config(input)
|
|
106
|
+
{ "canonical" => Sampling.canonical(config), "hash" => Sampling.digest(config) }
|
|
107
|
+
|
|
87
108
|
else
|
|
88
109
|
raise UnknownOp, "unknown op: #{op}"
|
|
89
110
|
end
|
|
90
111
|
end
|
|
91
112
|
|
|
113
|
+
# The observable surface only: the headers, and the `debug` object the
|
|
114
|
+
# adapter would merge (nil when there is nothing to merge).
|
|
115
|
+
def debug_injection(input)
|
|
116
|
+
built = Injection.build(
|
|
117
|
+
status: input["status"].to_i,
|
|
118
|
+
request_id: str(input["requestId"]),
|
|
119
|
+
prefix: str_or_nil(input["prefix"]),
|
|
120
|
+
recovery: str_or_nil(input["recovery"]),
|
|
121
|
+
method: str_or_nil(input["method"]),
|
|
122
|
+
path: str_or_nil(input["path"]),
|
|
123
|
+
portal_url: str_or_nil(input["portalUrl"])
|
|
124
|
+
)
|
|
125
|
+
{ "headers" => built[:headers], "debug" => built[:debug] }
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
# `settings` and `code` are passed verbatim, possibly invalid, and every
|
|
129
|
+
# op that takes them merges and normalizes them first (SAMPLE-014).
|
|
130
|
+
def sampling_config(input)
|
|
131
|
+
Sampling.normalize(input["settings"], input["code"]).first
|
|
132
|
+
end
|
|
133
|
+
|
|
92
134
|
def fingerprint(input)
|
|
93
135
|
stack = stack_text(input["stackTrace"])
|
|
94
136
|
if !stack.empty? && v8_dialect?(stack)
|
data/lib/restless/injection.rb
CHANGED
|
@@ -31,19 +31,37 @@ module Restless
|
|
|
31
31
|
flat.empty? ? m : "#{m}-#{flat}"
|
|
32
32
|
end
|
|
33
33
|
|
|
34
|
+
# INJECT-002. The debug response headers, which ship on every status.
|
|
35
|
+
#
|
|
36
|
+
# `x-log-url` is omitted with no portal origin; `x-debug` carries no URL,
|
|
37
|
+
# so it always ships.
|
|
38
|
+
def headers(request_id:, prefix: nil, portal_url: nil)
|
|
39
|
+
display = RequestId.format_request_id(request_id, prefix)
|
|
40
|
+
out = { "x-debug" => "npx api debug #{display}" }
|
|
41
|
+
out["x-log-url"] = "#{portal_url}/logs/#{request_id}" unless portal_url.nil? || portal_url.empty?
|
|
42
|
+
out
|
|
43
|
+
end
|
|
44
|
+
|
|
34
45
|
# INJECT-001..004, INJECT-006. Returns the headers to set plus the `debug`
|
|
35
|
-
# object to merge into a JSON body
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
46
|
+
# object to merge into a JSON body (nil when there is nothing to merge).
|
|
47
|
+
#
|
|
48
|
+
# `portal_url` is the project's public portal origin, published by the
|
|
49
|
+
# server. It is NOT the ingest base URL, which serves `/v1/*` and would
|
|
50
|
+
# 404 both paths, and there is deliberately no fallback to it: with no
|
|
51
|
+
# portal origin we emit `x-debug` alone. A caller cannot tell a broken URL
|
|
52
|
+
# from a missing one, and one fetched 404 teaches an agent to stop
|
|
53
|
+
# following the link.
|
|
54
|
+
def build(status:, request_id:, prefix: nil, recovery: nil,
|
|
55
|
+
method: nil, path: nil, portal_url: nil)
|
|
56
|
+
hdrs = headers(request_id: request_id, prefix: prefix, portal_url: portal_url)
|
|
39
57
|
|
|
40
|
-
|
|
41
|
-
#
|
|
42
|
-
#
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
log_url = "
|
|
46
|
-
debug_cmd = "
|
|
58
|
+
# INJECT-001. The body object is 4xx/5xx only: a successful body is the
|
|
59
|
+
# caller's data, not ours to reshape. With no portal origin there is no
|
|
60
|
+
# URL to put in one either (INJECT-006).
|
|
61
|
+
return { headers: hdrs, debug: nil } if status < 400 || portal_url.nil? || portal_url.empty?
|
|
62
|
+
|
|
63
|
+
log_url = hdrs["x-log-url"]
|
|
64
|
+
debug_cmd = hdrs["x-debug"]
|
|
47
65
|
|
|
48
66
|
# Per-request "dig-in" URL the calling agent (often an AI) can fetch for
|
|
49
67
|
# concrete next steps. Deliberately LEGIBLE: it ends in `<slug>.md` so it
|
|
@@ -52,7 +70,7 @@ module Restless
|
|
|
52
70
|
# can correlate the follow-up without any new tracking token.
|
|
53
71
|
slug = recovery_slug(method, path)
|
|
54
72
|
dig_in = "For the accepted parameters and next steps, " \
|
|
55
|
-
"fetch #{
|
|
73
|
+
"fetch #{portal_url}/p/#{request_id}/#{slug}.md"
|
|
56
74
|
# INJECT-004: a cached recovery message precedes the dig-in line,
|
|
57
75
|
# separated by a blank line.
|
|
58
76
|
recovery_text =
|
|
@@ -63,10 +81,7 @@ module Restless
|
|
|
63
81
|
end
|
|
64
82
|
|
|
65
83
|
{
|
|
66
|
-
headers:
|
|
67
|
-
"x-log-url" => log_url, # INJECT-002
|
|
68
|
-
"x-debug" => debug_cmd
|
|
69
|
-
},
|
|
84
|
+
headers: hdrs,
|
|
70
85
|
debug: {
|
|
71
86
|
"log" => log_url,
|
|
72
87
|
"cli" => debug_cmd,
|