camada 0.1.5 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +10 -4
- data/lib/camada/beacon_js.rb +3 -3
- data/lib/camada/constants.rb +3 -0
- data/lib/camada/engine.rb +28 -15
- data/lib/camada/rack.rb +15 -0
- data/lib/camada/snapshot/match.rb +8 -6
- data/lib/camada/snapshot/parse.rb +515 -24
- data/lib/camada/version.rb +3 -2
- data/lib/camada.rb +2 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6ff3ec1e73a6da51f20af06916fb4438fe50fc9fa87a9023445e90561c1a7cf8
|
|
4
|
+
data.tar.gz: d9d10163bdb349345ccb3a65666966624640fef0352d4727f848485abc0ea734
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 42beca1c8a2d5299814aa3b09963e21963e0e25344238e70cf087d19d484feb7cd91dbaae8930ec0c5236eb77c6906c5f9de1b5cd265704798061a99fae40040
|
|
7
|
+
data.tar.gz: 79148a9dd5427db583f916520b64726a6baede851368a2677a55e846ed6f4ed83500ec020544b6ba90c263ee189d42acd8c11a52f8f1a2a713d908d7d867476f
|
data/README.md
CHANGED
|
@@ -94,8 +94,9 @@ use Camada::Rack, engine
|
|
|
94
94
|
A block answers `403 Forbidden` with `x-block-reason`, `x-block-version` and, when a rule
|
|
95
95
|
decided, `x-block-rule`; its event ships with `blk` (and `rl`). A `warn` rule passes and stamps
|
|
96
96
|
`wrn`; a `skip` rule passes with nothing stamped. Cold (no snapshot yet) passes: fail open.
|
|
97
|
-
4. Challenge: a `challenge` verdict gets the self-contained proof-of-work page
|
|
98
|
-
|
|
97
|
+
4. Challenge: a `challenge` verdict gets the self-contained proof-of-work page on a page load. Any other
|
|
98
|
+
request passes to your app signalled: `x-camada-challenge: 1`, `cache-control: no-store` and the CDN cache headers (replacing
|
|
99
|
+
yours) and `chs: 1` on its event, which ships whatever `sample` and `exclude` say; `POST /__camada/challenge` verifies the solution, sets `_cch` (bound to the
|
|
99
100
|
ip, one hour) and 302s back. A request whose ip cannot be resolved is never challenged. The page
|
|
100
101
|
carries its own `Content-Security-Policy` (its inline solver and style by hash, nothing else)
|
|
101
102
|
and replaces any policy your app set on it (Rails' `content_security_policy`,
|
|
@@ -161,7 +162,8 @@ event name is free-form; the analyst's app-context rules read this vocabulary:
|
|
|
161
162
|
|
|
162
163
|
A route you gate yourself: `Camada.serve_challenge(env)` returns a Rack triple
|
|
163
164
|
`[status, headers, [body]]` to answer with (Sinatra: `halt(*answer)`) until the browser holds a
|
|
164
|
-
valid `_cch`, then `nil`.
|
|
165
|
+
valid `_cch`, then `nil`. A non-page (non-HTML) request also gets `nil` and is signalled
|
|
166
|
+
(`x-camada-challenge: 1`, `no-store`, `chs: 1`).
|
|
165
167
|
|
|
166
168
|
## What this tap can see
|
|
167
169
|
|
|
@@ -175,7 +177,11 @@ at this position covers ip, path, user-agent and header conditions — ASN, coun
|
|
|
175
177
|
fail open in-app. `matches` patterns are JS regexes read by Ruby's Onigmo: named groups are
|
|
176
178
|
native, `[^]` and `\cX` are translated, `^`/`$` become `\A`/`\z` (Ruby's are line anchors,
|
|
177
179
|
JS's are not), `\d`/`\w`/`\b` are ASCII as in JS; a spelling Ruby still rejects never matches
|
|
178
|
-
here (and never raises), while it does at the edge.
|
|
180
|
+
here (and never raises), while it does at the edge. An astral character (an emoji, say) is one
|
|
181
|
+
character here and two UTF-16 units at the edge, so a pattern that counts characters around one
|
|
182
|
+
(`^.$`, `.{2}`, `[^a]`, `\S`) or puts one in a class or before a quantifier can disagree; `.*`
|
|
183
|
+
and a bare emoji agree. A repeated capture group whose body can match empty (as in `((x?[ab]*)+?[ab]){2}`) can
|
|
184
|
+
match differently here than at the edge: that is an Onigmo engine difference.
|
|
179
185
|
|
|
180
186
|
## Deploying it
|
|
181
187
|
|
data/lib/camada/beacon_js.rb
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
# dist/auto.global.js) vendored as a String literal: served at /_cam/b.js with no runtime file
|
|
5
5
|
# read. spec/beacon_js_spec.rb pins it to the sibling build byte for byte.
|
|
6
6
|
module Camada
|
|
7
|
-
BEACON_VERSION = "0.
|
|
8
|
-
BEACON_SHA256 = "
|
|
9
|
-
BEACON_JS = "\"use strict\";(()=>{var k=\"0.
|
|
7
|
+
BEACON_VERSION = "0.4.0"
|
|
8
|
+
BEACON_SHA256 = "59233fa412e6f2f1b09a851e3ccf8021c8dd550476004bf1e280bf1511d7bf7e"
|
|
9
|
+
BEACON_JS = "\"use strict\";(()=>{var k=\"0.4.0\";var _=\"@camada/browser\";var b=`${_}/${k}`;function t(n){try{return n()}catch{return null}}function x(n){let e=2166136261;for(let i=0;i<n.length;i++)e^=n.charCodeAt(i),e=Math.imul(e,16777619);return(e>>>0).toString(16)}function M(){let n=document.createElement(\"canvas\");n.width=240,n.height=60;let e=n.getContext(\"2d\");return e?(e.textBaseline=\"alphabetic\",e.fillStyle=\"#f60\",e.fillRect(120,5,60,20),e.fillStyle=\"#069\",e.font=\"15px Arial\",e.fillText(\"Sfp\\u2603 1.0\",2,15),e.fillStyle=\"rgba(102,204,0,0.7)\",e.font=\"17px Times\",e.fillText(\"Sfp\\u2603 1.0\",4,45),x(n.toDataURL())):null}function R(){let n=document.createElement(\"canvas\"),e=n.getContext(\"webgl\")||n.getContext(\"experimental-webgl\");if(!e)return null;let i=e.getExtension(\"WEBGL_debug_renderer_info\");return{v:i?e.getParameter(i.UNMASKED_VENDOR_WEBGL):e.getParameter(e.VENDOR),r:i?e.getParameter(i.UNMASKED_RENDERER_WEBGL):e.getParameter(e.RENDERER)}}function S(){return x([Math.tan(-1e300),Math.sinh(1),Math.expm1(1),Math.cbrt(100),Math.log1p(10),Math.atanh(.5)].join(\"|\"))}var N=/^\\$?cdc_|^__webdriver|^__selenium|^__nightmare|^__playwright|^callPhantom/,L=/^\\$?cdc_|^__webdriver|^__selenium|^__nightmare|^__playwright|^callPhantom|^_phantom/;function j(){let n=document,e=navigator,i=window,o=0;for(let s in n)N.test(s)&&o++;for(let s in i)L.test(s)&&o++;return{wd:e.webdriver===!0?1:0,cdc:o,chrome:typeof i.chrome<\"u\"?1:0,plugins:e.plugins?e.plugins.length:-1,langs:e.languages?e.languages.length:-1,outer:window.outerWidth===0&&window.outerHeight===0?1:0,perm:t(()=>Notification.permission),pdf:typeof e.pdfViewerEnabled==\"boolean\"?e.pdfViewerEnabled?1:0:-1,err:t(()=>{try{null.x}catch(s){let c=s.stack;return c?c.split(`\n`).length:0}})}}function T({endpoint:n,rid:e}){let i=e||null,o=performance.now(),s=0,c=0,m=0,f=0,h=0,u=null;function l(){u===null&&(u=Math.round(performance.now()-o))}addEventListener(\"mousemove\",()=>{s++,l()},{passive:!0}),addEventListener(\"scroll\",()=>{c++,l()},{passive:!0}),addEventListener(\"keydown\",()=>{m++,l()},{passive:!0}),addEventListener(\"touchstart\",()=>{f++,l()},{passive:!0}),addEventListener(\"click\",()=>{h++,l()},{passive:!0});function w(){let r=navigator,a=t(()=>performance.getEntriesByType(\"navigation\")[0]),D=t(()=>{let p=performance.getEntriesByType(\"paint\");return p.length?Math.round(p[p.length-1].startTime):null}),d=t(()=>r.connection)||{};return{sdk:b,rid:i,ts:Date.now(),scr:t(()=>[screen.width,screen.height,screen.availWidth,screen.availHeight,screen.colorDepth]),win:t(()=>[window.innerWidth,window.innerHeight,window.outerWidth,window.outerHeight]),dpr:t(()=>window.devicePixelRatio),tz:t(()=>Intl.DateTimeFormat().resolvedOptions().timeZone),tzo:t(()=>new Date().getTimezoneOffset()),lang:t(()=>r.language),langs:t(()=>r.languages?Array.prototype.slice.call(r.languages,0,5):null),plat:t(()=>r.platform),cores:t(()=>r.hardwareConcurrency),mem:t(()=>r.deviceMemory),touch:t(()=>r.maxTouchPoints),ua:t(()=>r.userAgent),dnt:t(()=>r.doNotTrack),cookies:t(()=>r.cookieEnabled?1:0),conn:t(()=>[d.effectiveType,d.rtt,d.downlink,d.saveData?1:0]),gl:t(R),cv:t(M),mf:t(S),auto:t(j),vis:t(()=>document.visibilityState),focus:t(()=>document.hasFocus()?1:0),timing:a?[Math.round(a.domContentLoadedEventEnd),Math.round(a.loadEventEnd),Math.round(a.responseEnd-a.requestStart)]:null,paint:D,input:{moves:s,scrolls:c,keys:m,touches:f,clicks:h,first:u,at:Math.round(performance.now()-o)}}}function v(r){let a=JSON.stringify(r?Object.assign(w(),{hi:r}):w());navigator.sendBeacon&&navigator.sendBeacon(n,new Blob([a],{type:\"application/json\"}))||fetch(n,{method:\"POST\",body:a,keepalive:!0,headers:{\"content-type\":\"application/json\"}}).catch(()=>{})}let g=navigator.userAgentData,E=g&&g.getHighEntropyValues?g.getHighEntropyValues([\"architecture\",\"bitness\",\"model\",\"platformVersion\",\"fullVersionList\"]).catch(()=>null):Promise.resolve(null);setTimeout(()=>{E.then(v)},2e3),addEventListener(\"pagehide\",()=>{E.then(r=>{v(r)})},{once:!0})}function O(n){let e=new URL(n,\"http://_\"),i=e.pathname.split(\"/\");return i[i.length-1]=\"fp\",{endpoint:i.join(\"/\"),rid:e.searchParams.get(\"r\")}}(function(){try{let n=document.currentScript,e=n&&n.src;if(!e)return;let{endpoint:i,rid:o}=O(e);T({endpoint:i,rid:o})}catch{}})();})();\n"
|
|
10
10
|
end
|
data/lib/camada/constants.rb
CHANGED
|
@@ -19,6 +19,9 @@ module Camada
|
|
|
19
19
|
FP_MAX = 32 * 1024 # matches the server's /fp cap: never accept what ingest will 413
|
|
20
20
|
CHALLENGE_PATH = "/__camada/challenge"
|
|
21
21
|
BODY_MAX = 4 * 1024 # the verify form is ~120 bytes; anything larger is not ours
|
|
22
|
+
# Row keys only the SDK can attest (TLS/network facts, outcome, session, track()). A beacon body is the
|
|
23
|
+
# visitor's, so these never ride a sig:1 row; the analyst joins them from the beacon's request row.
|
|
24
|
+
REQUEST_ROW_KEYS = %w[tlsx ja4 asn cc sid ns st dur rt blk rl wrn chs ch et uid].freeze
|
|
22
25
|
|
|
23
26
|
SESSION_COOKIE = "_sfp" # same cookie as the edge collector: sid/ns comparable across taps
|
|
24
27
|
SESSION_MAX_AGE = 2_592_000 # 30 days
|
data/lib/camada/engine.rb
CHANGED
|
@@ -83,6 +83,8 @@ module Camada
|
|
|
83
83
|
# done. Everything runs inside the fail-open envelope: a camada bug must never 5xx the customer,
|
|
84
84
|
# and CAMADA_DISABLED=1 bypasses the SDK entirely.
|
|
85
85
|
class Engine
|
|
86
|
+
NO_STORE_HEADERS = %w[cache-control cdn-cache-control surrogate-control cloudflare-cdn-cache-control
|
|
87
|
+
vercel-cdn-cache-control netlify-cdn-cache-control].freeze
|
|
86
88
|
CHALLENGE_HEADERS = { "cache-control" => "no-store", "x-camada-challenge" => "1" }.freeze
|
|
87
89
|
|
|
88
90
|
attr_reader :env, :snap, :queue, :kit, :script_path, :fp_path, :challenge_path, :challenge_on
|
|
@@ -148,8 +150,9 @@ module Camada
|
|
|
148
150
|
end
|
|
149
151
|
|
|
150
152
|
# Serve the challenge for this request on demand — for a route the app wants to gate itself.
|
|
151
|
-
# nil when the client already holds a valid _cch (render your own page),
|
|
152
|
-
# cannot be identified (fail open)
|
|
153
|
+
# nil when the client already holds a valid _cch (render your own page), when the client
|
|
154
|
+
# cannot be identified (fail open), or when it is not a page load: that request is signalled
|
|
155
|
+
# instead (`x-camada-challenge: 1`, `chs: 1`) and the app renders.
|
|
153
156
|
def serve_challenge(ctx)
|
|
154
157
|
return nil if disabled? || @kit.nil? || ctx.nil?
|
|
155
158
|
|
|
@@ -157,6 +160,11 @@ module Camada
|
|
|
157
160
|
ip = ctx["ip"]
|
|
158
161
|
return nil if req.nil? || ip.nil? || ip.empty? || challenge_passed?(req, ip)
|
|
159
162
|
|
|
163
|
+
unless Challenge.wants_html?(req.header("accept"), req.header("sec-fetch-dest"))
|
|
164
|
+
ctx["signalled"] = true
|
|
165
|
+
return nil
|
|
166
|
+
end
|
|
167
|
+
|
|
160
168
|
ctx["challenged"] = true
|
|
161
169
|
serve_challenge_answer(req, ip, ctx["sid"])
|
|
162
170
|
rescue StandardError => e
|
|
@@ -230,11 +238,18 @@ module Camada
|
|
|
230
238
|
# A challenge needs a resolved client IP: the nonce and the _cch cookie are bound to it,
|
|
231
239
|
# so without one a single solve would mint a cookie every unidentified client could
|
|
232
240
|
# present. No ip -> no challenge (fail open), the same stance ip rules take.
|
|
241
|
+
signalled = false
|
|
233
242
|
if @challenge_on && ip && !ip.empty?
|
|
234
243
|
# The verify endpoint answers first: a challenged client must be able to reach it.
|
|
235
244
|
return verify(req, body, ip) if req.method == "POST" && req.path == @challenge_path
|
|
245
|
+
|
|
236
246
|
if v.challenge && !challenge_passed?(req, ip)
|
|
237
|
-
|
|
247
|
+
# Only a page load can be challenged; anything else passes, signalled (camada-all-7cgs).
|
|
248
|
+
if Challenge.wants_html?(req.header("accept"), req.header("sec-fetch-dest"))
|
|
249
|
+
return serve_challenge_answer(req, ip, Camada.cookie_value(req.header("cookie"), SESSION_COOKIE))
|
|
250
|
+
end
|
|
251
|
+
|
|
252
|
+
signalled = true
|
|
238
253
|
end
|
|
239
254
|
end
|
|
240
255
|
|
|
@@ -256,6 +271,7 @@ module Camada
|
|
|
256
271
|
set_cookie += "; Secure" if secure?(req)
|
|
257
272
|
end
|
|
258
273
|
ctx = { "rid" => rid, "sid" => sid, "ip" => ip, "_req" => req, "_engine" => self }
|
|
274
|
+
ctx["signalled"] = true if signalled
|
|
259
275
|
|
|
260
276
|
cfg = @snap.config || {}
|
|
261
277
|
excluded = (cfg["exclude"] || []).any? { |x| req.path.start_with?(x.to_s) }
|
|
@@ -266,9 +282,12 @@ module Camada
|
|
|
266
282
|
on_finish = lambda do |status|
|
|
267
283
|
# serve_challenge may have answered from inside the app, and it already shipped the
|
|
268
284
|
# `blk: "challenge"` row — one request, one event.
|
|
269
|
-
next if ctx["challenged"]
|
|
285
|
+
next if ctx["challenged"]
|
|
286
|
+
# a signalled row ships whatever sample and exclude say, like the 403 row it replaces
|
|
287
|
+
next if !ctx["signalled"] && (excluded || !sampled)
|
|
270
288
|
|
|
271
289
|
ev = event(req, rid, sid, new_session, ip)
|
|
290
|
+
ev["chs"] = 1 if ctx["signalled"] # challenge signalled (contract §A3)
|
|
272
291
|
ev["ts"] = ts0
|
|
273
292
|
ev["st"] = status
|
|
274
293
|
ev["dur"] = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).to_i
|
|
@@ -299,7 +318,8 @@ module Camada
|
|
|
299
318
|
parsed = Camada.parse_json(body)
|
|
300
319
|
return answer unless parsed.is_a?(Hash)
|
|
301
320
|
|
|
302
|
-
|
|
321
|
+
# spread first: ip, tap and rts (the relay clock) are the server's word
|
|
322
|
+
@queue.push(parsed.except(*REQUEST_ROW_KEYS).merge!("sig" => 1, "ip" => ip, "tap" => TAP, "rts" => now_ms))
|
|
303
323
|
answer
|
|
304
324
|
end
|
|
305
325
|
|
|
@@ -318,17 +338,10 @@ module Camada
|
|
|
318
338
|
Answer.new(status: 403, headers: headers, body: html)
|
|
319
339
|
end
|
|
320
340
|
|
|
321
|
-
# 403 + the proof-of-work page
|
|
322
|
-
#
|
|
341
|
+
# 403 + the proof-of-work page, plus the `blk: "challenge"` event — a served challenge is
|
|
342
|
+
# reported like a block (contract §D2). Callers pass page loads only.
|
|
323
343
|
def serve_challenge_answer(req, ip, sid)
|
|
324
|
-
|
|
325
|
-
answer =
|
|
326
|
-
if Challenge.wants_html?(req.header("accept"), req.header("sec-fetch-dest"))
|
|
327
|
-
page(ip, to)
|
|
328
|
-
else
|
|
329
|
-
Answer.new(status: 403, headers: CHALLENGE_HEADERS.merge("content-type" => "application/json"),
|
|
330
|
-
body: '{"error":"challenge_required"}')
|
|
331
|
-
end
|
|
344
|
+
answer = page(ip, Challenge.safe_return_to(req.path + req.query))
|
|
332
345
|
begin
|
|
333
346
|
ev = event(req, SecureRandom.uuid, sid, false, ip)
|
|
334
347
|
ev["st"] = 403
|
data/lib/camada/rack.rb
CHANGED
|
@@ -168,6 +168,7 @@ module Camada
|
|
|
168
168
|
raise
|
|
169
169
|
end
|
|
170
170
|
headers = stamp(status, headers, passed)
|
|
171
|
+
headers = signal(headers) if passed.ctx && passed.ctx["signalled"]
|
|
171
172
|
headers = own_challenge_csp(headers) if passed.ctx && passed.ctx["challenged"]
|
|
172
173
|
finish = passed.on_finish
|
|
173
174
|
return [status, headers, body] if finish.nil?
|
|
@@ -197,6 +198,20 @@ module Camada
|
|
|
197
198
|
headers
|
|
198
199
|
end
|
|
199
200
|
|
|
201
|
+
# A request that passed under a challenge (handle() or serve_challenge() marked the ctx) carries
|
|
202
|
+
# `x-camada-challenge: 1`, and `no-store` in place of the app's cache-control and CDN cache headers: the signal
|
|
203
|
+
# is per client. CDNs read their own cache header ahead of cache-control, so all of these end up no-store.
|
|
204
|
+
def signal(headers)
|
|
205
|
+
headers = headers.to_h
|
|
206
|
+
headers.delete_if { |k, _| Engine::NO_STORE_HEADERS.include?(k.to_s.downcase) }
|
|
207
|
+
Engine::NO_STORE_HEADERS.each { |h| headers[h] = "no-store" }
|
|
208
|
+
headers["x-camada-challenge"] = "1"
|
|
209
|
+
headers
|
|
210
|
+
rescue StandardError => e
|
|
211
|
+
Guarded.log_rate_limited(e)
|
|
212
|
+
headers
|
|
213
|
+
end
|
|
214
|
+
|
|
200
215
|
# A challenge page the app returned (serve_challenge) keeps its own CSP: a policy the app or its middleware put on
|
|
201
216
|
# that response (Rails' content_security_policy, secure_headers) is dropped and the page's is set, or the solver
|
|
202
217
|
# would be blocked (camada-all-y38o). Any other response comes back unchanged.
|
|
@@ -60,7 +60,12 @@ module Camada
|
|
|
60
60
|
@snap = snap
|
|
61
61
|
end
|
|
62
62
|
|
|
63
|
-
|
|
63
|
+
# The whole decision (rules, allow, block, challenge) runs in one request regex budget.
|
|
64
|
+
def match(i) = Snapshot.with_regex_budget { decide(i) }
|
|
65
|
+
|
|
66
|
+
private
|
|
67
|
+
|
|
68
|
+
def decide(i)
|
|
64
69
|
s = @snap
|
|
65
70
|
ip = i.ip || ""
|
|
66
71
|
n4 = -1
|
|
@@ -76,9 +81,8 @@ module Camada
|
|
|
76
81
|
unless s.rules.empty?
|
|
77
82
|
r = RuleRequest.new(n4: n4, ip6: w, asn: i.asn, country: i.country, tlsx: i.tlsx,
|
|
78
83
|
paths: forms, ua: i.ua, header: i.header)
|
|
79
|
-
s.rules.
|
|
80
|
-
|
|
81
|
-
end
|
|
84
|
+
hit = s.rules.find { |rule| rule.conds.all? { |cond| cond.call(r) } } # the order IS the precedence (§A4): first match wins
|
|
85
|
+
return Snapshot.rule_result(hit, s.version) if hit
|
|
82
86
|
end
|
|
83
87
|
reason = side(s.allow, i, n4, w, forms, false) # an exemption: every canonical spelling must agree
|
|
84
88
|
return MatchResult.new(allowed: true, reason: reason, version: s.version) if reason
|
|
@@ -92,8 +96,6 @@ module Camada
|
|
|
92
96
|
MatchResult.new(version: s.version)
|
|
93
97
|
end
|
|
94
98
|
|
|
95
|
-
private
|
|
96
|
-
|
|
97
99
|
def blocked4?(n)
|
|
98
100
|
s = @snap
|
|
99
101
|
b = n >> 8
|
|
@@ -134,46 +134,429 @@ module Camada
|
|
|
134
134
|
cmp_words(r, o, w) <= 0 && cmp_words(r, o + 4, w) >= 0 # rubocop:disable Style/ComparableBetween -- two different comparisons
|
|
135
135
|
end
|
|
136
136
|
|
|
137
|
+
# regex-limit v1: the longest one customer regex may run, and what one request's rule walk may spend on all of them.
|
|
138
|
+
MATCH_BUDGET_MS = 50
|
|
139
|
+
REQUEST_REGEX_BUDGET_MS = 100
|
|
140
|
+
MATCH_TIMEOUT = Regexp.respond_to?(:timeout) ? MATCH_BUDGET_MS / 1000.0 : nil
|
|
141
|
+
|
|
142
|
+
# Runs the block with a fresh per-request regex budget (seconds left, in request-scoped state), reset in ensure.
|
|
143
|
+
def self.with_regex_budget
|
|
144
|
+
Thread.current[:camada_regex_budget] = REQUEST_REGEX_BUDGET_MS / 1000.0
|
|
145
|
+
yield
|
|
146
|
+
ensure
|
|
147
|
+
Thread.current[:camada_regex_budget] = nil
|
|
148
|
+
end
|
|
149
|
+
|
|
137
150
|
# A pattern this runtime rejects never matches, and never throws (fail open). Patterns are
|
|
138
151
|
# authored as JS regexes (the analyst validates them with `new RegExp`), so the JS spellings
|
|
139
152
|
# Onigmo reads differently are translated first — see js_to_onigmo; \d \w \b are ASCII in
|
|
140
|
-
# Ruby by default, as JS reads them.
|
|
153
|
+
# Ruby by default, as JS reads them. An astral char is one code point here and two UTF-16 units
|
|
154
|
+
# in JS, so `.`, `[^a]` and `\S` count it once (an accepted gap, camada-all-vuvo).
|
|
155
|
+
# A quantifier right after a raw astral char repeats the whole char here but only its low unit in JS,
|
|
156
|
+
# so js_to_onigmo answers as V8 does: the char when the count allows 1, else `(?!)` (camada-all-5lra, see astral_count).
|
|
157
|
+
# Where a lookbehind ends up holding `(?!)`, Onigmo cannot compile it, so compile_regex returns nil and the rule never matches.
|
|
158
|
+
# V8's zero-low-unit reading is not emulated.
|
|
159
|
+
# Onigmo also diverges from JS on a
|
|
160
|
+
# repeated capture group whose body can match empty, even with no backreference: `((x?[ab]*)+?[ab]){2}` on 'aa'
|
|
161
|
+
# matches in JS, not here (an accepted gap, camada-all-mx0s). Under IGNORECASE Onigmo also applies
|
|
162
|
+
# multi-char case folds JS /i (one char at a time) does not: `\A(?:ss)\z` matches "ß", `[ß]` and `[a\S]` match "SS", `ff` matches
|
|
163
|
+
# "ff", and an ASCII-only pattern does the same once Ruby recompiles it for a non-ASCII value. Extra matches only, so a rule hits a
|
|
164
|
+
# value JS would not and never misses one; splitting every literal run would slow string search (an accepted gap, camada-all-wdii).
|
|
165
|
+
# Inside a lookbehind js_to_onigmo splits adjacent literals (camada-all-ccjo) so it stays fixed-length, which stops ss folding to ß,
|
|
166
|
+
# and reads a multi-fold char (ß, ff, İ) case-sensitively, as JS matches it: Onigmo there reads it as its fold alone (ß as "ss",
|
|
167
|
+
# never "ß"; Ruby 3.3.13 and 3.4.12 do the same for ff and İ, camada-all-ozix).
|
|
168
|
+
# A backreference JS and Onigmo read differently is rewritten or, when it cannot be, fails open (scan_groups).
|
|
169
|
+
# A `(?` or `(*` group JS lacks (`(?i)`, `(?>`, `(*FAIL)`, ...) fails open too: JS rejects it or V8 mis-evaluates it (camada-all-lsl8).
|
|
170
|
+
# Only a UTF-8 or ASCII pattern is walked: snapshot patterns arrive as UTF-8 JSON, and the binary walk
|
|
171
|
+
# misreads other encodings.
|
|
141
172
|
def self.compile_regex(pattern, options = 0)
|
|
142
|
-
|
|
173
|
+
return nil unless pattern.is_a?(String) && (pattern.encoding == Encoding::UTF_8 || pattern.ascii_only?)
|
|
174
|
+
|
|
175
|
+
bin = pattern.b # the one binary copy both walks read (O(1) indexing; the syntax is ASCII)
|
|
176
|
+
scan = scan_groups(bin)
|
|
177
|
+
return nil if scan[1]
|
|
178
|
+
|
|
179
|
+
source = js_to_onigmo(bin, scan).force_encoding(pattern.encoding)
|
|
180
|
+
verbose = $VERBOSE
|
|
181
|
+
begin
|
|
182
|
+
$VERBOSE = nil # `[a\S]` expands to a class with a duplicated range; Onigmo warns under -w, matching is right
|
|
183
|
+
# Ruby >= 3.2: a per-regex match bound (regex-limit v1). Never the global Regexp.timeout, that is the host app's.
|
|
184
|
+
# Ruby 3.1 has no match-time bound and rejects the keyword, so it compiles without it.
|
|
185
|
+
MATCH_TIMEOUT ? Regexp.new(source, options, timeout: MATCH_TIMEOUT) : Regexp.new(source, options)
|
|
186
|
+
ensure
|
|
187
|
+
$VERBOSE = verbose
|
|
188
|
+
end
|
|
143
189
|
rescue RegexpError, TypeError, ArgumentError, EncodingError
|
|
144
190
|
nil
|
|
145
191
|
end
|
|
146
192
|
|
|
147
|
-
#
|
|
193
|
+
# One escape/class-aware walk: [the capture groups of a JS pattern in number order (the name, or
|
|
194
|
+
# nil when unnamed), whether the pattern fails open, {reference position => [end position, :empty or :cond]}].
|
|
195
|
+
# JS reads a backreference to a group that has not captured as empty and clears captures each
|
|
196
|
+
# iteration; Onigmo fails it and keeps the last capture. A reference to a group not closed before
|
|
197
|
+
# it (forward or self) is always empty in JS: kind :empty, written `(?:)`. One to a group that may not have
|
|
198
|
+
# captured (it or an enclosing group is followed by a min-0 quantifier or sits in a negative lookaround, or
|
|
199
|
+
# it lies in an alternation branch: a `|` in an enclosing group or at top level) becomes the conditional
|
|
200
|
+
# `(?(N)\N)` (kind :cond; unset reads as empty), unless the group or an ancestor repeats (an empty last iteration
|
|
201
|
+
# makes Onigmo overwrite the capture, JS rejects that iteration and keeps it and the per-iteration reset
|
|
202
|
+
# diverges) or the group sits inside a positive lookahead (a lookahead keeps its first successful path, and JS tries the
|
|
203
|
+
# paths in another order and rejects an empty optional iteration): fail open. So does a plain reference (the group
|
|
204
|
+
# closed before it, always captured) to a group that sits in, or is, a repeated body that can match empty
|
|
205
|
+
# (`(b*)+x\1`): the walk tracks per frame whether its body must consume input (solid) or can match empty (nullable).
|
|
206
|
+
# A reference inside that repeat reads its own iteration's capture and stays compiled. So does a backward reference to a
|
|
207
|
+
# name two groups share (JS allows that in separate
|
|
208
|
+
# branches; Onigmo reads another group). A backreference inside a lookbehind, or to a group with a lookbehind
|
|
209
|
+
# ancestor, fails open too: JS matches a lookbehind right to left ("closed before the reference" is backwards
|
|
210
|
+
# there; a quantified group keeps its leftmost repetition). Beside named groups, a `\k<name>` naming no group
|
|
211
|
+
# is a JS SyntaxError: fail open (Onigmo aborts the process on it).
|
|
212
|
+
# So does a plain reference to a group in a positive lookahead that holds a quantified frame with optional
|
|
213
|
+
# iterations and a body that can match empty (JS rejects the empty iteration, so the lookahead's first success differs).
|
|
214
|
+
# Walks a binary copy (O(1) indexing; the syntax is ASCII), so the names come back binary.
|
|
215
|
+
# After the walk one forward pass over the frames (a parent precedes its children) settles, per
|
|
216
|
+
# frame, what its ancestors imply, so each reference costs O(1) however deeply its group nests.
|
|
217
|
+
def self.scan_groups(pattern)
|
|
218
|
+
pattern = binary(pattern)
|
|
219
|
+
return [[], false, {}] unless pattern.match?(/\\[1-9k]/) # no reference: nothing reads the names or can diverge
|
|
220
|
+
|
|
221
|
+
cur = { parent: nil }
|
|
222
|
+
frames = [cur] # every group, capturing or not, in `(` order
|
|
223
|
+
groups = [] # capture groups in number order
|
|
224
|
+
refs = [] # [position, number or name]
|
|
225
|
+
in_class = false
|
|
226
|
+
i = 0
|
|
227
|
+
while i < pattern.length
|
|
228
|
+
ch = pattern[i]
|
|
229
|
+
if ch == "\\"
|
|
230
|
+
if in_class
|
|
231
|
+
refs << [i, :bad, cur, i + 2] if pattern[i + 1] == "k" # `[\k]` is a SyntaxError beside named groups
|
|
232
|
+
elsif (m = pattern.match(/\G(?:k<([^<>]*)>|([1-9][0-9]*)|(k))/, i + 1))
|
|
233
|
+
refs << [i, m[3] ? :bad : m[1] || m[2].to_i, cur, m.end(0)]
|
|
234
|
+
i = m.end(0) - 2 # the reference is no atom: skip its name or digits (the loop's step and the one below)
|
|
235
|
+
elsif (m = ESC_ATOM.match(pattern, i))
|
|
236
|
+
cur[:solid] = true unless min_zero?(pattern, m.end(0))
|
|
237
|
+
i = m.end(0) - 2
|
|
238
|
+
end
|
|
239
|
+
i += 1
|
|
240
|
+
elsif in_class
|
|
241
|
+
if ch == "]"
|
|
242
|
+
in_class = false
|
|
243
|
+
cur[:solid] = true unless min_zero?(pattern, i + 1)
|
|
244
|
+
end
|
|
245
|
+
elsif ch == "["
|
|
246
|
+
in_class = true
|
|
247
|
+
elsif ch == "("
|
|
248
|
+
cur = { parent: cur, neg: pattern.match?(/\G\(\?<?!/, i), lb: pattern.match?(/\G\(\?<[=!]/, i), la: pattern.match?(/\G\(\?=/, i) }
|
|
249
|
+
frames << cur
|
|
250
|
+
if pattern[i + 1] != "?"
|
|
251
|
+
groups << cur
|
|
252
|
+
elsif (m = pattern.match(/\G\(\?<([^=!<>][^<>]*)>/, i))
|
|
253
|
+
groups << cur.merge!(name: m[1])
|
|
254
|
+
end
|
|
255
|
+
if (m = pattern.match(/\G\(\?(?:<[=!]|<[^<>]*>|[A-Za-z-]*:|[=!])/, i))
|
|
256
|
+
i = m.end(0) - 1 # the group's prefix is no atom
|
|
257
|
+
end
|
|
258
|
+
elsif ch == ")" && cur[:parent]
|
|
259
|
+
cur[:end] = i
|
|
260
|
+
b = pattern.match(JS_BRACE, i + 1)
|
|
261
|
+
cur[:zero] = min_zero?(pattern, i + 1)
|
|
262
|
+
cur[:nullable] ||= !cur[:solid] || cur[:lb] || cur[:la] || cur[:neg] # a lookaround consumes nothing
|
|
263
|
+
cur[:rep] = pattern.match?(/\G[+*]/, i + 1) || (b && (b[2] ? b[3].empty? || over_one?(b[3]) : over_one?(b[1]))) # repeats
|
|
264
|
+
cur[:holds] ||= (cur[:zero] || cur[:rep]) && cur[:nullable] # it or a frame inside it: an optional nullable repeat
|
|
265
|
+
cur[:parent][:holds] ||= cur[:holds]
|
|
266
|
+
closed = cur
|
|
267
|
+
cur = cur[:parent]
|
|
268
|
+
cur[:solid] = true unless closed[:nullable] || closed[:zero] # the branch now holds an atom that must consume
|
|
269
|
+
elsif ch == "|"
|
|
270
|
+
cur[:alt] = true
|
|
271
|
+
cur[:nullable] ||= !cur[:solid]
|
|
272
|
+
cur[:solid] = false
|
|
273
|
+
elsif ch == "{" && (b = pattern.match(JS_BRACE, i))
|
|
274
|
+
i = b.end(0) - 1 # a quantifier, not an atom
|
|
275
|
+
elsif ch.ord >= 0xC0 || (ch.ord < 0x80 && !"^$?*+".include?(ch))
|
|
276
|
+
e = i + 1
|
|
277
|
+
e += 1 while e < pattern.length && pattern.getbyte(e).between?(0x80, 0xBF)
|
|
278
|
+
cur[:solid] = true unless min_zero?(pattern, e)
|
|
279
|
+
i = e - 1
|
|
280
|
+
end
|
|
281
|
+
i += 1
|
|
282
|
+
end
|
|
283
|
+
return [[], false, {}] if refs.empty? # an escaped backslash or a class `[\1]` is no reference
|
|
284
|
+
|
|
285
|
+
names = groups.map { |g| g[:name] }
|
|
286
|
+
index = names.each_with_index.reverse_each.to_h # name => first group number - 1
|
|
287
|
+
named = names.any?
|
|
288
|
+
frames.drop(1).each do |f|
|
|
289
|
+
p = f[:parent]
|
|
290
|
+
f[:la_div] = (f[:la] && f[:holds]) || p[:la_div]
|
|
291
|
+
f[:in_lb] = f[:lb] || p[:in_lb]
|
|
292
|
+
f[:unset] = f[:zero] || f[:neg] || p[:alt] || p[:unset] # may not have captured when a reference runs
|
|
293
|
+
f[:rep_above] = p[:rep] || p[:rep_above] # a strict ancestor repeats
|
|
294
|
+
f[:in_la] = f[:la] || p[:in_la] # inside a positive lookahead
|
|
295
|
+
f[:rn] = f[:rep] && f[:nullable] ? f : p[:rn] # innermost repeating frame (or ancestor) whose body can match empty
|
|
296
|
+
end
|
|
297
|
+
dups = names.compact.tally.select { |_, c| c > 1 }
|
|
298
|
+
plan = {}
|
|
299
|
+
diverges = refs.any? do |pos, ref, home, stop|
|
|
300
|
+
next named if ref == :bad # a `\k` not followed by `<name>`: a SyntaxError beside named groups
|
|
301
|
+
|
|
302
|
+
k = if ref.is_a?(String)
|
|
303
|
+
index[ref]
|
|
304
|
+
else
|
|
305
|
+
ref <= groups.length ? ref - 1 : nil
|
|
306
|
+
end
|
|
307
|
+
next named && ref.is_a?(String) unless k # a name with no group beside named groups: a JS SyntaxError
|
|
308
|
+
|
|
309
|
+
g = groups[k]
|
|
310
|
+
next true if home[:in_lb] || g[:in_lb]
|
|
311
|
+
|
|
312
|
+
unless g[:end] && g[:end] < pos
|
|
313
|
+
plan[pos] = [stop, :empty] # never captured yet: empty, and a quantifier after it keeps its atom
|
|
314
|
+
next false
|
|
315
|
+
end
|
|
316
|
+
next true if g[:name] && dups[g[:name]] # JS allows a name in two branches; Onigmo reads another group
|
|
317
|
+
# a repeat that can match empty: Onigmo's last empty iteration overwrites the capture, JS keeps it
|
|
318
|
+
next true if (rn = g[:rn]) && rn[:end] < pos
|
|
319
|
+
|
|
320
|
+
next true if g[:la_div]
|
|
321
|
+
next false unless g[:unset]
|
|
322
|
+
next true if g[:rep_above] || g[:rep] || g[:in_la]
|
|
323
|
+
|
|
324
|
+
plan[pos] = [stop, :cond]
|
|
325
|
+
false
|
|
326
|
+
end
|
|
327
|
+
[names, diverges, plan]
|
|
328
|
+
end
|
|
329
|
+
|
|
330
|
+
# Whether the text at j starts a min-0 quantifier (`?`, `*`, `{0}`, `{0,}`, `{0,m}`).
|
|
331
|
+
def self.min_zero?(pattern, j)
|
|
332
|
+
c = pattern.getbyte(j)
|
|
333
|
+
return true if c == 0x3F || c == 0x2A # ? *
|
|
334
|
+
return false unless c == 0x7B # {
|
|
335
|
+
|
|
336
|
+
b = pattern.match(JS_BRACE, j)
|
|
337
|
+
!b.nil? && b[1].delete("0").empty?
|
|
338
|
+
end
|
|
339
|
+
|
|
340
|
+
JS_BRACE = /\G\{([0-9]+)(?:(,)([0-9]*))?\}/
|
|
341
|
+
JS_QUANT = /\G(?:([?*+])|#{JS_BRACE.source.delete_prefix('\\G')})\??/ # a JS quantifier with its lazy `?`
|
|
342
|
+
ONIGMO_REPEAT_MAX = 100_000 # Onigmo rejects a repeat number above this ("too big number for repeat range")
|
|
343
|
+
JS_WS = "\\t-\\r\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff" # JS \s (Ruby's is ASCII)
|
|
344
|
+
JS_SPACE = { "s" => "[#{JS_WS}]", "S" => "[^#{JS_WS}]" }.freeze
|
|
345
|
+
ANY_CHAR = '[\x00-\x7f]|[\xc0-\xff][\x80-\xbf]*|[\s\S]' # one char of a binary UTF-8 string, whole
|
|
346
|
+
JS_ATOM = /\G(?:\\(?:c[A-Za-z0-9_]|x\h{2}|u\h{4}|[0-3][0-7]{0,2}|[4-7][0-7]?|(?!c)(?:#{ANY_CHAR}))|#{ANY_CHAR})/n # one class atom
|
|
347
|
+
# One escape that must consume input (nil where it may match empty: \b \B, \c without a letter, a trailing \).
|
|
348
|
+
# An unsure escape is no match: calling it nullable only fails open, splitting an atom would not.
|
|
349
|
+
ESC_ATOM = /\G\\(?:c[A-Za-z]|x\h{2}|u\h{4}|0[0-7]{0,2}|(?![bBc])(?:#{ANY_CHAR}))/n
|
|
350
|
+
JS_DOT = "[^\\n\\r\\u2028\\u2029]" # JS . (Onigmo's excludes only \n)
|
|
351
|
+
NON_JS_GROUP = /\G\((?:\*|\?(?![:=!<]))/ # a `(?` or `(*` opening none of the JS forms `(?:` `(?=` `(?!` `(?<`
|
|
352
|
+
JS_OPENER = /\G\((?:\?(?:[:=!]|(<)[=!]))?/ # an unnamed group's opener; the capture is set for a lookbehind
|
|
353
|
+
JS_GROUP = /\G\(\?<(?![=!])([^<>]*)>/n # a named group's opener: a JS name holds no `<` or `>`
|
|
354
|
+
JS_NAME = /\A[\p{ID_Start}$_][\p{ID_Continue}$\u200c\u200d]*\z/ # a JS group name
|
|
355
|
+
JS_ESCAPE = /\G(?:[bdDfnrsStvwW]|x\h{2}|u\h{4})/ # the letter escapes JS reads without `u`
|
|
356
|
+
|
|
357
|
+
# The JS-only spellings a tenant is likely to author: `[^]` (any char) -> `(?m:.)`, `\cX` ->
|
|
148
358
|
# the control character, and the anchors: JS `^`/`$` bind the whole input while Ruby's are
|
|
149
|
-
# line anchors, so outside a class they become `\A`/`\z
|
|
150
|
-
#
|
|
151
|
-
|
|
152
|
-
|
|
359
|
+
# line anchors, so outside a class they become `\A`/`\z`; a `{` that is no JS quantifier
|
|
360
|
+
# (`{,3}`, `{ 1}`) -> `\{` (JS reads it as text, Onigmo as a bound), and `x{n}?` -> `x{n}` (JS
|
|
361
|
+
# reads it as exact-n, Onigmo as optional). `\<letter>` outside JS's escape set is the bare
|
|
362
|
+
# letter, as JS reads it without `u` (`\p{L}` is "p{L}", `\A` is "A"). `.` -> JS_DOT, `[]` ->
|
|
363
|
+
# `(?!)` (never matches; a `]` right after `[`/`[^` closes the class, a stray `]` -> `\]`), and
|
|
364
|
+
# `\s`/`\S` -> the JS whitespace set; inside a class they nest as a class
|
|
365
|
+
# (`[a\s]` -> `[a[WS]]`: one atom, no ReDoS), a `-` beside one stays a literal dash, as in JS,
|
|
366
|
+
# and so does a `-` right after the atom that follows it.
|
|
367
|
+
# Under IGNORECASE Onigmo folds several chars at once (`ss` ~ ß, and a positive class holding
|
|
368
|
+
# `\S` gains such folds), which makes a lookbehind variable-length and uncompilable; JS folds one
|
|
369
|
+
# char at a time. So inside a lookbehind adjacent literal chars are split with `(?:)` (`ab` -> `a(?:)b`; a
|
|
370
|
+
# lone literal stays a plain string node) and a positive
|
|
371
|
+
# class with `\S` (or `\W`/`\D`) becomes `(?:[^WS]|[rest])`, and a closed one-char class counts as a literal for
|
|
372
|
+
# that split (camada-all-ccjo, camada-all-fe1r). A char whose fold is several chars (ß, ff, İ; raw, `\u00df` or `\ß`)
|
|
373
|
+
# becomes `(?-i:ß)` there, pulled out of a positive class like `\S` unless a `-` is beside it (camada-all-ozix):
|
|
374
|
+
# JS /i matches it only to itself, and Onigmo would read it as its fold alone (version-dependent).
|
|
375
|
+
# In a class `\c` is a control char only before a letter, digit or `_` (ClassControlLetter); before
|
|
376
|
+
# anything else (`[\c*]`) it is a literal `\` then `c` (camada-all-mqcb).
|
|
377
|
+
# A `{m,n}` bound above Onigmo's 100000 is clamped: m to 100000, and an n above it drops (`{m,}`,
|
|
378
|
+
# exact for any input shorter than n), where JS accepts it (camada-all-yh2t).
|
|
379
|
+
# A literal `[` or `&` in a class is escaped (no nesting, POSIX class or `&&`, as in JS).
|
|
380
|
+
# `(?<name>` and `\k<name>` are copied verbatim once the name fits JS's grammar (JS_NAME); a name
|
|
381
|
+
# used twice is kept only when the twins sit in different alternatives of one disjunction (JS
|
|
382
|
+
# throws otherwise), and then no backreference may exist (camada-all-7xyl). Anything else Ruby
|
|
383
|
+
# rejects still fails open.
|
|
384
|
+
# Raises RegexpError on a group JS lacks (see compile_regex).
|
|
385
|
+
# `scan` is the scan_groups result (default: scan the pattern). Walks a binary copy (O(1) indexing
|
|
386
|
+
# on a non-ASCII pattern) and returns the pattern's encoding.
|
|
387
|
+
def self.js_to_onigmo(pattern, scan = nil)
|
|
388
|
+
enc = pattern.encoding
|
|
389
|
+
pattern = binary(pattern)
|
|
390
|
+
out = String.new(encoding: Encoding::BINARY)
|
|
153
391
|
i = 0
|
|
154
392
|
n = pattern.length
|
|
155
393
|
in_class = false
|
|
394
|
+
pull_end = solo_end = -1 # where the last in-class \\s/\\S ended; where the atom after its dash ended
|
|
395
|
+
group_names, _, plan = scan || scan_groups(pattern)
|
|
396
|
+
named = group_names.any?
|
|
397
|
+
opens = [] # the unclosed `(` indices outside a class
|
|
398
|
+
bars = [-1] # per open group, the root first: the index of the last `|` in it or an ancestor
|
|
399
|
+
lbs = [false] # per open group, the root first: inside a lookbehind (camada-all-ccjo)
|
|
400
|
+
lit_end = -1 # the pattern index right after the last literal emitted inside a lookbehind
|
|
401
|
+
class_start = nil # the out index of an open positive class's `[` inside a lookbehind
|
|
402
|
+
class_alts = [] # the `\S`/`\W`/`\D` and multi-fold chars that class held
|
|
403
|
+
last_named = {} # name -> the index of its last opener
|
|
156
404
|
while i < n
|
|
157
405
|
ch = pattern[i]
|
|
406
|
+
# in a lookbehind (positive class too, unless a range end) a multi-fold char becomes `(?-i:c)` (camada-all-ozix)
|
|
407
|
+
if lbs.last && (!in_class || class_start) && (w = multi_fold(pattern, i)) &&
|
|
408
|
+
!(in_class && (pattern[i - 1] == "-" || pattern[i + w[1]] == "-"))
|
|
409
|
+
in_class ? class_alts << "(?-i:#{w[0]})" : out << "(?-i:#{w[0]})"
|
|
410
|
+
i += w[1]
|
|
411
|
+
next
|
|
412
|
+
end
|
|
158
413
|
if ch == "\\" && i + 1 < n
|
|
159
|
-
if
|
|
160
|
-
|
|
414
|
+
if !in_class && (r = plan[i])
|
|
415
|
+
if r[1] == :empty
|
|
416
|
+
out << "(?:)"
|
|
417
|
+
else
|
|
418
|
+
ref = pattern[(i + 1)...r[0]]
|
|
419
|
+
nm = ref.start_with?("k") ? ref[2..-2] : (named && group_names[ref.to_i - 1])
|
|
420
|
+
out << (nm ? "(?(<#{nm}>)\\k<#{nm}>)" : "(?(#{ref})\\#{ref})")
|
|
421
|
+
end
|
|
422
|
+
i = r[0]
|
|
423
|
+
next
|
|
424
|
+
end
|
|
425
|
+
l = pattern[i + 1]
|
|
426
|
+
if in_class && class_start && (l == "S" || l == "W" || l == "D")
|
|
427
|
+
class_alts << (l == "S" ? JS_SPACE["S"] : "\\#{l}") # spliced in at the `]`: a positive class holding one gains multi-char folds
|
|
428
|
+
pull_end = i + 2
|
|
429
|
+
i += 2
|
|
430
|
+
next
|
|
431
|
+
end
|
|
432
|
+
if l == "s" || l == "S"
|
|
433
|
+
out << JS_SPACE[l] # in a class too: Onigmo nests classes, one atom
|
|
434
|
+
pull_end = i + 2 if in_class
|
|
435
|
+
i += 2
|
|
436
|
+
next
|
|
437
|
+
end
|
|
438
|
+
if l == "c" && pattern[i + 2]&.match?(in_class ? /[A-Za-z0-9_]/ : /[A-Za-z]/) # JS Annex B: `[\c1]` too
|
|
439
|
+
out << Regexp.escape((pattern[i + 2].ord & 0x1F).chr)
|
|
161
440
|
i += 3
|
|
162
441
|
next
|
|
163
442
|
end
|
|
164
|
-
|
|
443
|
+
if named && !in_class && (m = pattern.match(/\G[1-9][0-9]*/, i + 1)) && (m[0].to_i <= group_names.length)
|
|
444
|
+
nm = group_names[m[0].to_i - 1] # Onigmo rejects a numbered backreference beside named groups
|
|
445
|
+
if nm
|
|
446
|
+
out << "\\k<#{nm}>"
|
|
447
|
+
i += 1 + m[0].length
|
|
448
|
+
next
|
|
449
|
+
end
|
|
450
|
+
end
|
|
451
|
+
if named && !in_class && l == "k" && (m = pattern.match(/\G\\k<[^<>]*>/n, i))
|
|
452
|
+
out << m[0] # a `$` in the name is no anchor; scan_groups failed open on a name no group has
|
|
453
|
+
i += m[0].length
|
|
454
|
+
next
|
|
455
|
+
end
|
|
456
|
+
if !in_class && l.ord >= 0xF0 && pattern.match?(JS_QUANT, i + 5) # `\😀?` reads as `😀?` (camada-all-5lra)
|
|
457
|
+
i += 1
|
|
458
|
+
next
|
|
459
|
+
end
|
|
460
|
+
kept = !l.match?(/[A-Za-z]/) || pattern.match?(JS_ESCAPE, i + 1) || (l == "B" && !in_class) ||
|
|
461
|
+
(l == "k" && named)
|
|
462
|
+
l = "\\\\c" if l == "c" # JS reads `\c` before a non-letter as text
|
|
463
|
+
if kept
|
|
464
|
+
e = (pattern.match(/\G\\\d+/n, i) || pattern.match(JS_ATOM, i))[0] # the whole escape: a lookbehind must not split `\x41`
|
|
465
|
+
out << e
|
|
466
|
+
i += e.length
|
|
467
|
+
next
|
|
468
|
+
end
|
|
469
|
+
if !in_class && lbs.last # a bare letter is a literal (2 long): split it from a literal just before
|
|
470
|
+
out << "(?:)" if lit_end == i
|
|
471
|
+
lit_end = i + 2
|
|
472
|
+
end
|
|
473
|
+
out << l
|
|
165
474
|
i += 2
|
|
166
475
|
next
|
|
167
476
|
end
|
|
168
477
|
if in_class
|
|
169
|
-
|
|
478
|
+
if ch == "]"
|
|
479
|
+
in_class = false
|
|
480
|
+
if class_alts.any?
|
|
481
|
+
rest = out.slice!(class_start..)[1..]
|
|
482
|
+
rest = "\\#{rest}" if rest.start_with?("^") # `[^..]` would negate the spliced class
|
|
483
|
+
alts = class_alts.join("|")
|
|
484
|
+
alts = "(?:#{alts}" + (rest.empty? ? ")" : "|[#{rest}])") if class_alts.length > 1 || !rest.empty?
|
|
485
|
+
out << alts
|
|
486
|
+
class_alts = []
|
|
487
|
+
else
|
|
488
|
+
out << "]"
|
|
489
|
+
end
|
|
490
|
+
lit_end = i + 1 if class_start # Onigmo merges a one-char class into the literals beside it
|
|
491
|
+
class_start = nil
|
|
492
|
+
i += 1
|
|
493
|
+
next
|
|
494
|
+
end
|
|
495
|
+
if ch == "-" && (pull_end == i || solo_end == i || pattern.match?(class_start ? /\G-\\[sSWD]/ : /\G-\\[sS]/, i))
|
|
496
|
+
if pull_end == i # JS reads the atom after that dash as standalone: a dash right after it is text too
|
|
497
|
+
m = pattern.match(JS_ATOM, i + 1) # nil when the pattern ends on the dash
|
|
498
|
+
solo_end = m ? i + 1 + m[0].length : pattern.length
|
|
499
|
+
end
|
|
500
|
+
ch = "\\-"
|
|
501
|
+
end
|
|
502
|
+
ch = "\\#{ch}" if ch == "[" || ch == "&" # JS: `[[:alpha:]]` is {[ : a l p h} then `]`
|
|
503
|
+
elsif ch == "(" && pattern.match?(NON_JS_GROUP, i)
|
|
504
|
+
raise RegexpError, "non-JS group" # compile_regex fails open
|
|
505
|
+
elsif ch == "(" && (m = pattern.match(JS_GROUP, i))
|
|
506
|
+
nm = m[1].dup.force_encoding(Encoding::UTF_8)
|
|
507
|
+
raise RegexpError, "invalid group name" unless nm.valid_encoding? && nm.match?(JS_NAME)
|
|
508
|
+
|
|
509
|
+
# the twin is exclusive when a group open since before it has seen a `|` since (transitive:
|
|
510
|
+
# checking the last twin is enough)
|
|
511
|
+
if (prev = last_named[nm]) && (named || bars[opens.bsearch_index { |o| o >= prev } || opens.length] < prev)
|
|
512
|
+
raise RegexpError, "duplicate group name"
|
|
513
|
+
end
|
|
514
|
+
|
|
515
|
+
last_named[nm] = i
|
|
516
|
+
opens << i
|
|
517
|
+
bars << bars.last
|
|
518
|
+
lbs << lbs.last
|
|
519
|
+
out << m[0]
|
|
520
|
+
i += m[0].length
|
|
521
|
+
next
|
|
522
|
+
elsif ch == "("
|
|
523
|
+
m = pattern.match(JS_OPENER, i)
|
|
524
|
+
opens << i
|
|
525
|
+
bars << bars.last
|
|
526
|
+
lbs << (lbs.last || !m[1].nil?)
|
|
527
|
+
out << m[0]
|
|
528
|
+
i += m[0].length
|
|
529
|
+
next
|
|
530
|
+
elsif ch == "|"
|
|
531
|
+
bars[-1] = i
|
|
532
|
+
elsif ch == ")" && opens.any?
|
|
533
|
+
opens.pop
|
|
534
|
+
bars.pop
|
|
535
|
+
lbs.pop
|
|
536
|
+
elsif ch == "."
|
|
537
|
+
out << JS_DOT
|
|
538
|
+
i += 1
|
|
539
|
+
next
|
|
540
|
+
elsif ch == "]"
|
|
541
|
+
out << "\\]"
|
|
542
|
+
i += 1
|
|
543
|
+
next
|
|
170
544
|
elsif ch == "["
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
545
|
+
neg = pattern[i + 1] == "^"
|
|
546
|
+
j = i + (neg ? 2 : 1)
|
|
547
|
+
if pattern[j] == "]"
|
|
548
|
+
out << (neg ? "(?m:.)" : "(?!)")
|
|
549
|
+
i = j + 1
|
|
174
550
|
next
|
|
175
551
|
end
|
|
552
|
+
if !neg && lbs.last
|
|
553
|
+
out << "(?:)" if lit_end == i
|
|
554
|
+
class_start = out.length
|
|
555
|
+
end
|
|
556
|
+
out << pattern[i...j]
|
|
557
|
+
i = j
|
|
176
558
|
in_class = true
|
|
559
|
+
next
|
|
177
560
|
elsif ch == "^"
|
|
178
561
|
out << "\\A"
|
|
179
562
|
i += 1
|
|
@@ -182,23 +565,121 @@ module Camada
|
|
|
182
565
|
out << "\\z"
|
|
183
566
|
i += 1
|
|
184
567
|
next
|
|
568
|
+
elsif ch == "{"
|
|
569
|
+
m = pattern.match(JS_BRACE, i)
|
|
570
|
+
out << (m ? brace_quantifier(m) : "\\{")
|
|
571
|
+
i += m ? m[0].length : 1
|
|
572
|
+
i += 1 if m && !m[0].include?(",") && pattern[i] == "?"
|
|
573
|
+
next
|
|
574
|
+
end
|
|
575
|
+
if !in_class && ch.ord >= 0xF0 && (q = pattern.match(JS_QUANT, i + 4)) && !(count = astral_count(q)).nil?
|
|
576
|
+
if count && lbs.last # the char alone is a literal
|
|
577
|
+
out << "(?:)" if lit_end == i
|
|
578
|
+
lit_end = q.end(0)
|
|
579
|
+
end
|
|
580
|
+
# `(?!)` inside a lookbehind is not compilable by Onigmo: compile_regex returns nil and the rule never matches
|
|
581
|
+
out << (count ? pattern[i, 4] : "(?!)")
|
|
582
|
+
i = q.end(0)
|
|
583
|
+
next
|
|
584
|
+
end
|
|
585
|
+
# a literal in a lookbehind directly after another gets `(?:)` (never inside a UTF-8 char): Onigmo folds `ss` to ß under /i
|
|
586
|
+
if !in_class && lbs.last && !"()|*+?".include?(ch)
|
|
587
|
+
if ch.ord.between?(0x80, 0xBF)
|
|
588
|
+
lit_end = i + 1 if lit_end == i
|
|
589
|
+
else
|
|
590
|
+
out << "(?:)" if lit_end == i
|
|
591
|
+
lit_end = i + 1
|
|
592
|
+
end
|
|
185
593
|
end
|
|
186
594
|
out << ch
|
|
187
595
|
i += 1
|
|
188
596
|
end
|
|
189
|
-
out
|
|
597
|
+
out.force_encoding(enc)
|
|
598
|
+
end
|
|
599
|
+
|
|
600
|
+
# V8 repeats only the low unit of an astral char: true when the quantifier allows one, false when not, nil when m > n (left as written).
|
|
601
|
+
# V8's zero-low-unit reading (the high unit alone: `x😀{0}` on "x😀", a trailing `x😀?` on "x😁") is not emulated: that branch
|
|
602
|
+
# never matches, as the whole pattern never matches in the linear engines (camada-all-k938).
|
|
603
|
+
def self.astral_count(match)
|
|
604
|
+
sym, lo, comma, hi = match.captures
|
|
605
|
+
lo, hi = { "?" => %w[0 1], "*" => ["0", nil], "+" => ["1", nil] }[sym] || [lo, comma ? hi : lo]
|
|
606
|
+
lo, hi = canonical_bounds(lo, hi)
|
|
607
|
+
return nil unless lo # m > n
|
|
608
|
+
|
|
609
|
+
!over_one?(lo) && hi != "0"
|
|
610
|
+
end
|
|
611
|
+
|
|
612
|
+
# [lo, hi] without leading zeros (hi "" is no upper bound), or nil when m > n: a JS SyntaxError, left as written
|
|
613
|
+
def self.canonical_bounds(lo, hi)
|
|
614
|
+
lo, hi = [lo, hi.to_s].map { |d| d.sub(/\A0+(?=.)/, "") }
|
|
615
|
+
[lo, hi] unless hi != "" && ([lo.length, lo] <=> [hi.length, hi]).positive?
|
|
616
|
+
end
|
|
617
|
+
|
|
618
|
+
def self.over_one?(digits)
|
|
619
|
+
digits = digits.sub(/\A0+/, "") # a digit-string test: to_i on a huge one is slow
|
|
620
|
+
digits.length > 1 || digits > "1"
|
|
621
|
+
end
|
|
622
|
+
|
|
623
|
+
def self.binary(str)
|
|
624
|
+
str.encoding == Encoding::BINARY ? str : str.b
|
|
625
|
+
end
|
|
626
|
+
|
|
627
|
+
# [its text, its length] when the char at i (raw UTF-8, `\uXXXX` or an identity escape `\ß`) has a case fold of
|
|
628
|
+
# several chars (ß, İ, ff: 104 of them), else nil. JS /i without `u` matches each of them only to itself.
|
|
629
|
+
def self.multi_fold(pattern, i)
|
|
630
|
+
if pattern[i] == "\\"
|
|
631
|
+
return (w = multi_fold(pattern, i + 1)) && [w[0], w[1] + 1] if pattern.getbyte(i + 1).to_i >= 0x80
|
|
632
|
+
|
|
633
|
+
m = pattern.match(/\G\\u(\h{4})/n, i)
|
|
634
|
+
return nil unless m && !(cp = m[1].hex).between?(0xD800, 0xDFFF)
|
|
635
|
+
|
|
636
|
+
c = cp.chr(Encoding::UTF_8)
|
|
637
|
+
len = 6
|
|
638
|
+
else
|
|
639
|
+
b = pattern.getbyte(i)
|
|
640
|
+
return nil unless b.between?(0xC2, 0xEF) # a 2- or 3-byte lead: every such char is in the BMP
|
|
641
|
+
|
|
642
|
+
len = b < 0xE0 ? 2 : 3
|
|
643
|
+
c = pattern.byteslice(i, len).force_encoding(Encoding::UTF_8)
|
|
644
|
+
end
|
|
645
|
+
[pattern.byteslice(i, len), len] if c.valid_encoding? && c.downcase(:fold).length > 1
|
|
646
|
+
end
|
|
647
|
+
|
|
648
|
+
def self.brace_quantifier(match)
|
|
649
|
+
lo, hi = canonical_bounds(match[1], match[3])
|
|
650
|
+
return match[0] unless lo # m > n is a JS SyntaxError: leave it for Onigmo to reject
|
|
651
|
+
|
|
652
|
+
lo = ONIGMO_REPEAT_MAX.to_s if over_repeat_max?(lo)
|
|
653
|
+
hi = "" if over_repeat_max?(hi) # an upper bound past it never binds a real input
|
|
654
|
+
"{#{lo}#{match[2]}#{hi}}"
|
|
655
|
+
end
|
|
656
|
+
|
|
657
|
+
def self.over_repeat_max?(digits)
|
|
658
|
+
digits = digits.to_s # nil: `{m}` has no upper bound
|
|
659
|
+
digits.length > 6 || digits.to_i > ONIGMO_REPEAT_MAX # length first: to_i on a huge digit string is slow
|
|
190
660
|
end
|
|
191
661
|
|
|
192
662
|
# A regex test that never raises: a value in an encoding the pattern cannot read (a binary
|
|
193
663
|
# PATH_INFO with high bytes, say) reads as "no match" rather than an exception on the request path.
|
|
194
664
|
#
|
|
195
|
-
# `on_error` is what a value the engine cannot read answers.
|
|
196
|
-
#
|
|
665
|
+
# `on_error` is what a value the engine cannot read answers. A caller with no side passes false;
|
|
666
|
+
# a `matches` condition (a custom rule or a meta `pathsRegex` entry) passes `deny` (regex-cap-sdk v1: an unevaluable value is a
|
|
197
667
|
# hit for block/challenge/warn and a miss for skip, the same as an over-cap one, camada-all-cwb4).
|
|
668
|
+
#
|
|
669
|
+
# Inside a match decision the request's regex budget is spent first: once it is gone the regex does not
|
|
670
|
+
# run and answers `on_error`, like an over-limit run (Regexp::TimeoutError is a StandardError).
|
|
198
671
|
def self.regex_hit?(rx, value, on_error = false)
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
672
|
+
left = Thread.current[:camada_regex_budget]
|
|
673
|
+
return on_error if left && left <= 0
|
|
674
|
+
|
|
675
|
+
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC) if left
|
|
676
|
+
begin
|
|
677
|
+
rx.match?(utf8_view(value))
|
|
678
|
+
rescue StandardError
|
|
679
|
+
on_error
|
|
680
|
+
ensure
|
|
681
|
+
Thread.current[:camada_regex_budget] = left - (Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) if left
|
|
682
|
+
end
|
|
202
683
|
end
|
|
203
684
|
|
|
204
685
|
# A value tagged ASCII-8BIT (Rack) whose bytes are valid UTF-8 is read as UTF-8, so a non-ASCII
|
|
@@ -339,17 +820,23 @@ module Camada
|
|
|
339
820
|
false
|
|
340
821
|
end
|
|
341
822
|
|
|
342
|
-
# One path form against a side's (or the block side's) exact, prefix and regex
|
|
823
|
+
# One path form against a side's (or the block side's) exact, prefix and regex predicates.
|
|
343
824
|
def self.path_in?(st, path, regexes = [])
|
|
344
825
|
return true if st.paths_exact.include?(path)
|
|
345
826
|
return true if !st.paths_prefix.empty? && prefix_hit?(st.paths_prefix, path)
|
|
346
827
|
|
|
347
|
-
regexes.any? { |
|
|
828
|
+
regexes.any? { |pred| pred.call(path) }
|
|
348
829
|
end
|
|
349
830
|
|
|
350
831
|
# Does the block hold for this request's path? deny = any spelling; an exemption = both canonical forms.
|
|
351
832
|
def self.path_hit?(forms, deny, &pred)
|
|
352
|
-
deny
|
|
833
|
+
if deny
|
|
834
|
+
# a form equal to one already tried is not matched again, so a repeat never spends the request's regex budget twice
|
|
835
|
+
return pred.call(forms[0]) ||
|
|
836
|
+
(forms[1] != forms[0] && pred.call(forms[1])) ||
|
|
837
|
+
(forms[2] != forms[0] && forms[2] != forms[1] && pred.call(forms[2]))
|
|
838
|
+
end
|
|
839
|
+
pred.call(forms[1]) && (forms[2] == forms[1] || pred.call(forms[2]))
|
|
353
840
|
end
|
|
354
841
|
|
|
355
842
|
# One path condition -> a predicate over a single path form. A `matches` form over MAX_MATCH_INPUT
|
|
@@ -389,6 +876,10 @@ module Camada
|
|
|
389
876
|
def self.compile_cond(c, sets, deny = true)
|
|
390
877
|
f = c["f"].to_s
|
|
391
878
|
op = c["op"].to_s
|
|
879
|
+
# camada-all-zpbv: a stored `matches` value that is "" or not a String never matches (no to_s
|
|
880
|
+
# coercion into a regex; "" would match everything). First, so it also beats the over-cap fail-closed.
|
|
881
|
+
return ->(_r) { false } if op == "matches" && (!c["v"].is_a?(String) || c["v"].empty?)
|
|
882
|
+
|
|
392
883
|
if f == "path" # every path op reads the canonical forms (see "paths" above)
|
|
393
884
|
raw = c["v"]
|
|
394
885
|
pred = path_pred(op, raw.is_a?(Array) ? raw.map(&:to_s) : [raw.to_s], deny)
|
|
@@ -531,7 +1022,7 @@ module Camada
|
|
|
531
1022
|
tls: (meta["tls"] || []).to_set,
|
|
532
1023
|
paths_exact: (meta["pathsExact"] || []).to_set { |p| canon_path(p.to_s) },
|
|
533
1024
|
paths_prefix: (meta["pathsPrefix"] || []).to_set { |p| dir_key(p.to_s) },
|
|
534
|
-
paths_regex: (meta["pathsRegex"] || []).
|
|
1025
|
+
paths_regex: (meta["pathsRegex"] || []).map { |p| path_pred("matches", [p], true) },
|
|
535
1026
|
allow: range_set(sec.fetch(10, EMPTY), sec.fetch(11, EMPTY), meta["allow"]),
|
|
536
1027
|
challenge: range_set(sec.fetch(12, EMPTY), sec.fetch(13, EMPTY), meta["challenge"]),
|
|
537
1028
|
rules: compile_rules(meta, rule4, rule6)
|
data/lib/camada/version.rb
CHANGED
|
@@ -4,7 +4,8 @@ module Camada
|
|
|
4
4
|
# The SDK's wire identity: x-camada-sdk: @camada/ruby/<version>. This literal is the single
|
|
5
5
|
# source: camada.gemspec reads it at build, and the sibling drift guards (camada-backend,
|
|
6
6
|
# camada-web, camada-mkt) parse this file with /^\s*VERSION = "([^"]+)"/m the way they parse
|
|
7
|
-
# a package.json. Plain X.Y.Z only
|
|
8
|
-
|
|
7
|
+
# a package.json. Plain X.Y.Z only, and @camada/ruby must stay in the analyst's SDK_NAMES:
|
|
8
|
+
# its sdkId drops anything else.
|
|
9
|
+
VERSION = "0.2.1"
|
|
9
10
|
SDK_ID = "@camada/ruby/#{VERSION}".freeze
|
|
10
11
|
end
|
data/lib/camada.rb
CHANGED
|
@@ -83,7 +83,8 @@ module Camada
|
|
|
83
83
|
end
|
|
84
84
|
|
|
85
85
|
# A Rack triple [status, headers, [body]] to answer with (Sinatra: `halt(*answer)`) until the
|
|
86
|
-
# browser holds a valid _cch, then nil.
|
|
86
|
+
# browser holds a valid _cch, then nil. A non-page (non-HTML) request also gets nil and is
|
|
87
|
+
# signalled (x-camada-challenge: 1, no-store, chs: 1).
|
|
87
88
|
def serve_challenge(env)
|
|
88
89
|
ctx = ctx_of(env)
|
|
89
90
|
engine_for(ctx).serve_challenge(ctx)&.to_rack
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: camada
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1
|
|
4
|
+
version: 0.2.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- camada
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-26 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: 'camada SDK for Ruby: inline enforcement of your snapshot (ordered custom
|
|
14
14
|
rules, then allow, block, challenge), first-party beacon and proof-of-work challenge,
|