constile_gate 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 51879f686a8d88a87ab113a5b4b2aaf8747dd8e2092deccd9368c1c714ed75a7
4
+ data.tar.gz: d902ec0beb8a223e0770269d0397b439f168c8a189971d04b408cda82f3bb226
5
+ SHA512:
6
+ metadata.gz: 93d2b9c2dd1d10276877dc9932d678477b6b3d1a948811eb620e591ad4fc0ae9a4f7be91bf01749fe5fc114ee6326773c714d4b3d8d88be409657b965b3bbffc
7
+ data.tar.gz: 2a2a7ec02af0b62dc27cccaa1e67c23b35bf66392d9cbb72394245b513e767d34215563f669bd30f2efef0407dfe0d1af95e0e607222570544172eda4abfa0d3
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 CCC2C Inc.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,226 @@
1
+ # Constile gate for Ruby
2
+
3
+ The Constile gate runs in front of your site. When an AI agent fetches
4
+ one of your pages for a person, the gate serves the Constile offer
5
+ instead of the article. People, search engines, and crawlers get your pages as
6
+ usual.
7
+
8
+ This gem is the gate as Rack middleware. It works in Rails, Sinatra, and other
9
+ Rack apps on Ruby 2.7 or later, and it uses only Ruby's standard library.
10
+
11
+ ## Before you begin
12
+
13
+ You need:
14
+
15
+ - A Rack app on Ruby 2.7 or later.
16
+ - Your site ID, site secret, and the network URL. Find them on your site page in
17
+ the publisher dashboard.
18
+
19
+ The gate answers at the root of your domain, at `/.well-known/constile` and
20
+ `/sponsored/`. Your app must serve the whole domain, not a subpath such as
21
+ `/blog`.
22
+
23
+ ## Install the gem
24
+
25
+ Add it to your `Gemfile`, then run `bundle install`:
26
+
27
+ ```ruby
28
+ gem "constile_gate", "~> 0.6"
29
+ ```
30
+
31
+ If your app doesn't use Bundler, run `gem install constile_gate` on each server
32
+ instead.
33
+
34
+ To use the copy from your site page instead, download
35
+ `constile_gate-0.6.0.gem`, unpack it into your app with
36
+ `gem unpack constile_gate-0.6.0.gem --target vendor/gems`, point your
37
+ `Gemfile` at it with
38
+ `gem "constile_gate", path: "vendor/gems/constile_gate-0.6.0"`, and commit
39
+ `vendor/gems/constile_gate-0.6.0` so your deploys get it too.
40
+
41
+ ## Set your site's settings
42
+
43
+ The gate reads its settings from environment variables:
44
+
45
+ | Variable | Value |
46
+ |---|---|
47
+ | `CONSTILE_SITE_ID` | Your site ID. |
48
+ | `CONSTILE_SITE_SECRET` | Your site secret. Keep it out of source control. |
49
+ | `CONSTILE_NETWORK_URL` | The network URL. |
50
+
51
+ If any of the three is missing, the gate passes every request to your app and
52
+ logs one warning when your app starts.
53
+
54
+ You can also pass each setting as an option, which wins over its variable:
55
+
56
+ | Option | Variable | Default | What it does |
57
+ |---|---|---|---|
58
+ | `site_id` | `CONSTILE_SITE_ID` | | Your site ID. |
59
+ | `site_secret` | `CONSTILE_SITE_SECRET` | | Your site secret. |
60
+ | `network_url` | `CONSTILE_NETWORK_URL` | | The network URL. |
61
+ | `timeout` | `CONSTILE_TIMEOUT` | `5` | Seconds the gate waits for each call to the network, from connecting to the last byte. |
62
+ | `config_retry` | `CONSTILE_CONFIG_RETRY` | `30` | Seconds the gate leaves the network alone after it fails to give your site's configuration. |
63
+ | `trusted_proxies` | `CONSTILE_TRUSTED_PROXIES` | none | Addresses or ranges of your proxies, such as `10.0.0.0/8`. See [Behind a proxy](#behind-a-proxy-or-load-balancer). |
64
+ | `enabled` | | `true` | Set it to `false` to turn the gate off without a warning. |
65
+ | | `CONSTILE_DISABLED` | | Set it to `1` to turn the gate off: every request goes to your app unchanged, and nothing goes to the network. Read on every request. |
66
+ | `platform` | | `"rack"`, or `"rails"` in Rails | The name your site page and the install check show. |
67
+
68
+ ## Rails
69
+
70
+ The gem adds the gate as the first middleware in your app, in front of
71
+ everything else. After you install the gem and set your settings, there's
72
+ nothing more to add.
73
+
74
+ To confirm that the gate is first, run:
75
+
76
+ ```sh
77
+ bin/rails middleware
78
+ ```
79
+
80
+ The first line is `use ConstileGate::Rack`.
81
+
82
+ To set options, use `config.constile_gate` in `config/application.rb` or in a file
83
+ under `config/environments/`:
84
+
85
+ ```ruby
86
+ # config/environments/production.rb
87
+ config.constile_gate.trusted_proxies = ["10.0.0.0/8"]
88
+ ```
89
+
90
+ To turn the gate off in development and test, add this to
91
+ `config/environments/development.rb` and `config/environments/test.rb`:
92
+
93
+ ```ruby
94
+ config.constile_gate.enabled = false
95
+ ```
96
+
97
+ ## Sinatra
98
+
99
+ Add the gate in `config.ru`, in front of your app:
100
+
101
+ ```ruby
102
+ # config.ru
103
+ require "constile_gate"
104
+ require_relative "app"
105
+
106
+ use ConstileGate::Rack, platform: "sinatra"
107
+ run App
108
+ ```
109
+
110
+ If your app has no `config.ru`, add `use ConstileGate::Rack, platform: "sinatra"` to
111
+ the app itself. It works the same way, but it runs after Sinatra's own
112
+ middleware, such as sessions, so `config.ru` is the better place.
113
+
114
+ ## Any other Rack app
115
+
116
+ Add the gate first in `config.ru`, before any other middleware, so it sees each
117
+ request as it arrived:
118
+
119
+ ```ruby
120
+ # config.ru
121
+ require "constile_gate"
122
+
123
+ use ConstileGate::Rack
124
+ run MyApp
125
+ ```
126
+
127
+ For another Rack framework, add `ConstileGate::Rack` as the first middleware the way
128
+ that framework adds Rack middleware.
129
+
130
+ ## Paywalled content
131
+
132
+ Gate 0.6.0 and later can fetch a paywalled article with the release
133
+ instructions the network sends, so an agent gets the full text and the offer
134
+ fits it. The network gives the gate a page address on your site's own host
135
+ and, optionally, headers to send: `Cookie`, `Authorization`,
136
+ `Constile-Paywall-Key`, or a header starting with `X-`. The gate ignores an
137
+ address on another host and drops other headers. It strips `Set-Cookie` from
138
+ what it releases and never logs the headers. To set up a method, open your
139
+ site's page in the dashboard and choose a content API, a paywall rule, or a
140
+ Constile login. A paywalled page gets offers only after the method works and
141
+ the gate is 0.6.0 or later.
142
+
143
+ ## Behind a proxy or load balancer
144
+
145
+ The gate sends the network the address of an agent that claims to be a
146
+ registered agent, so the network can check it against the agent's published
147
+ addresses. The gate reads that address from `REMOTE_ADDR`. It never believes
148
+ `X-Forwarded-For`, because any client can send one.
149
+
150
+ If your app runs behind a proxy or load balancer, `REMOTE_ADDR` is the proxy's
151
+ address. To read the visitor's address, list your proxies in `trusted_proxies`.
152
+ The gate then reads `X-Forwarded-For` from right to left and takes the first
153
+ address that isn't one of your proxies.
154
+
155
+ ```ruby
156
+ use ConstileGate::Rack, trusted_proxies: ["10.0.0.0/8", "192.0.2.10"]
157
+ ```
158
+
159
+ From one of your trusted proxies, the gate also reads the page's scheme and host
160
+ from `X-Forwarded-Proto` and `X-Forwarded-Host`. From any other client, it
161
+ ignores them. If your proxy terminates TLS, list it in `trusted_proxies`, so the
162
+ gate knows the page is `https`.
163
+
164
+ ## Caches in front of your app
165
+
166
+ A cache in front of your app, such as a CDN or Varnish, can serve a cached page
167
+ to an agent without asking your app. That agent then gets the article without an
168
+ offer. To avoid that, configure the cache to send requests to your app when the
169
+ `User-Agent` names an AI agent, or to vary on `User-Agent`. The gate marks each
170
+ offer `Cache-Control: no-store`, so a cache never serves an offer to a person.
171
+
172
+ ## Check the install
173
+
174
+ To check the install, open `https://your-site.example/.well-known/constile`. The
175
+ gate answers with your site ID, `"gate_version": "constile-gate 0.6.0"`, and the
176
+ platform. The install check on your site page reads the same file.
177
+
178
+ ## How the gate behaves
179
+
180
+ - **Agents.** For a registered agent's request to an eligible page, the gate gets
181
+ the page from your app, sends its title, description, and keywords to the
182
+ network, and serves the offer in its place.
183
+ - **Everyone else.** Every other response goes back from your app untouched. The
184
+ gate reads a response only for an agent's request.
185
+ - **Settings from the network.** The gate gets your site's configuration from the
186
+ network and keeps it for as long as the network says, from 10 seconds to an
187
+ hour. Apart from that, people's requests never reach the network.
188
+ - **Outages.** If the network can't be reached, the gate serves your page as if
189
+ the gate weren't there. Each call to the network takes one `timeout` at most,
190
+ from connecting to the last byte, with no retries. On Ruby 2.7, a DNS lookup
191
+ that hangs can take longer, because Ruby 2.7 can't interrupt the system
192
+ resolver. When a call for your site's configuration fails, the gate
193
+ passes every request through without asking the network for `config_retry`
194
+ seconds, then asks again. The `/sponsored/` callbacks don't fall back to your
195
+ page: they release content only against a recorded commitment, so they answer
196
+ with an error instead.
197
+ - **Errors.** If the gate itself fails, it logs the error and serves your app's
198
+ page, as it does in an outage. The `/sponsored/` routes and the seller file
199
+ answer 503 instead. Errors your app raises go on as they would without the
200
+ gate.
201
+ - **Character sets.** The gate reads a page's title and keywords in the charset
202
+ its `Content-Type` names, or UTF-8 when it names none, and passes a page
203
+ through byte for byte.
204
+ - **Released articles.** When the network grants an agent access, the gate gets
205
+ the article from your app with a fresh request that carries none of the
206
+ agent's headers, cookies, or session, and drops any cookies your app sets on
207
+ it.
208
+
209
+ ### Limits
210
+
211
+ - Under Rack 3, a page your app streams (a body that has neither `to_ary` nor
212
+ `to_path`) gets no offer, because Rack 3 doesn't let middleware read it. The
213
+ page goes to the agent as usual. Under Rack 2, the gate reads streamed pages.
214
+ - For an agent's request, the gate asks your app for the page without
215
+ compression, so `Rack::Deflater` doesn't get in the way. If your app
216
+ compresses the page anyway, the gate reads gzip and deflate. A page compressed
217
+ another way, such as Brotli, still gets an offer, but the network sees only
218
+ its path, not its title or keywords.
219
+
220
+ ## Support
221
+
222
+ Write to <hello@constile.app>, or see the [setup guides](https://constile.app/setup).
223
+
224
+ ## License
225
+
226
+ MIT. See LICENSE.
@@ -0,0 +1,21 @@
1
+ require_relative "lib/constile_gate/version"
2
+
3
+ Gem::Specification.new do |spec|
4
+ spec.name = "constile_gate"
5
+ spec.version = ConstileGate::VERSION
6
+ spec.authors = ["CCC2C Inc."]
7
+ spec.email = ["hello@constile.app"]
8
+ spec.summary = "The Constile gate as Rack middleware, for Rails, Sinatra, and any Rack app."
9
+ spec.description = "Serves the Constile offer to AI agents that fetch a page for a person, " \
10
+ "forwards the offer's callbacks to the Constile network, and passes everyone else through untouched."
11
+ spec.license = "MIT"
12
+ spec.homepage = "https://constile.app"
13
+ spec.metadata = {
14
+ "documentation_uri" => "https://constile.app/setup/rack",
15
+ "rubygems_mfa_required" => "true",
16
+ }
17
+ spec.required_ruby_version = ">= 2.7"
18
+ # The gemspec ships too, so an unpacked copy works as a Bundler path gem.
19
+ spec.files = Dir.chdir(__dir__) { Dir["lib/**/*.rb", "README.md", "LICENSE", "constile_gate.gemspec"].sort }
20
+ spec.require_paths = ["lib"]
21
+ end
@@ -0,0 +1,50 @@
1
+ require "constile_gate/js"
2
+ require "constile_gate/url"
3
+
4
+ module ConstileGate
5
+ # Who is asking for a page. A port of classifyClient and signatureAgentOrigin in
6
+ # packages/core/src/agents.ts. Registry entries are the network's config agents:
7
+ # Hashes with string keys (id, token, kind, tier, signature_agent).
8
+ module Agents
9
+ # The reference's /bot\b|crawler|spider|headless|scraper/, with an ASCII word boundary.
10
+ GENERIC_BOT = /bot(?![A-Za-z0-9_])|crawler|spider|headless|scraper/.freeze
11
+ QUOTED_HTTPS = %r{"(https://[^"]+)"}.freeze
12
+
13
+ module_function
14
+
15
+ # The origin named in a Signature-Agent header, in either the dictionary or the quoted-string form.
16
+ def signature_agent_origin(header)
17
+ m = QUOTED_HTTPS.match(header || "")
18
+ return nil unless m
19
+
20
+ URL.parse(m[1]).origin
21
+ rescue ArgumentError, RangeError
22
+ nil
23
+ end
24
+
25
+ # Returns { kind: "agent" | "crawler" | "search" | "browser" | "unknown_client", agent: entry or nil }.
26
+ def classify_client(headers, registry)
27
+ ua = JS.downcase(headers.get("user-agent") || "")
28
+ # Longest token first, so "claude-searchbot" wins over a shorter token it contains. Ties keep registry order.
29
+ by_length = registry.each_with_index.sort_by { |a, i| [-JS.utf16_length(a["token"].to_s), i] }.map(&:first)
30
+ hit = by_length.find { |a| JS.truthy?(a["token"]) && ua.include?(JS.downcase(a["token"])) }
31
+ if hit
32
+ return { kind: "agent", agent: hit } if hit["kind"] == "user_agent"
33
+
34
+ return { kind: hit["kind"], agent: hit }
35
+ end
36
+ # A signed request names its agent in Signature-Agent. The network checks the signature;
37
+ # an agent recognized only this way gets no offer unless the signature verifies.
38
+ origin = headers.has?("signature") ? signature_agent_origin(headers.get("signature-agent")) : nil
39
+ signer = origin && registry.find do |a|
40
+ a["kind"] == "user_agent" && JS.truthy?(a["signature_agent"]) && signature_agent_origin(%("#{a["signature_agent"]}")) == origin
41
+ end
42
+ return { kind: "agent", agent: signer } if signer
43
+ return { kind: "unknown_client", agent: nil } if ua.empty?
44
+ # Never show an offer to a person: any browser-looking client without a bot marker is a browser.
45
+ return { kind: "browser", agent: nil } if ua.start_with?("mozilla/") && !ua.match?(GENERIC_BOT)
46
+
47
+ { kind: "unknown_client", agent: nil }
48
+ end
49
+ end
50
+ end