mcp_toolkit 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f6ccc255fce267fe8c0caafe02f29e8a5db48e9b410494ef463bf9520047cbe2
-  data.tar.gz: feb8b4e46f7e7054abef0bf99ad8d633ebb30d1fb36624b350313d35e83824f9
+  metadata.gz: e99039b1f2109b3da40c5d271338fbefc565bce7fb472b36947caf322d0ab3f8
+  data.tar.gz: db79d656f89ce7566e5437956a8f0feac0ba9f4a906896078db3cee4879b512b
 SHA512:
-  metadata.gz: 50cc58bd25162c2ecf70b97b90b1de329a8b3858494dc719699d8cf609607d3c6f8abacef5c1ed1f0e0eea8b7dc9760c0a4752a816433e65ee661147404a53d8
-  data.tar.gz: 72474820f84e31bf3601d5ae26e6e153069accf7f51d9c3bc4e11428be2a068139f1ecc9656b5e4fc95bb40275f999a35e24e60bae5c0462dcdf03098e40a030
+  metadata.gz: '0804f99b21ac1cf54aa5d379957b1a3bec5af4a2aa3630bb2c0210d3b594cb98cf56dc3af4cc3693e1d872b2855ade1f1f2dc97a03760f56d16af85994f56547'
+  data.tar.gz: 07626c80512de4d475cd1cd4b7fbe31a210718dbc8a5d2b81a97009900f8cc1df91b79cae903bfc1b0769a049ab224232cccadfd2b86f61c50ce83e5e08766f5

data/CHANGELOG.md

@@ -1,3 +1,201 @@
+## [0.4.0] - 2026-07-06
+
+### Added
+
+- **Authority dispatch path** — a hand-rolled JSON-RPC front-end for a first-party
+  server that authenticates tokens locally and serves its OWN tools (and, as a
+  gateway, aggregates + proxies upstreams) WITHOUT the official `mcp` SDK in the
+  request path. It coexists with the SDK-backed satellite path
+  (`McpToolkit::Server.build`) by design — the gem now carries two dispatch
+  front-ends, each for its role. The satellite path is unchanged.
+  - `McpToolkit::Protocol` — JSON-RPC constants + envelope helpers
+    (`SUPPORTED_VERSIONS`, `LATEST_VERSION`, `JSONRPC_VERSION`, `ErrorCodes`,
+    `Error` + subclasses with `#code`/`#data`/`#to_h`, `success_response` /
+    `error_response`). The byte contract of a first-party endpoint's error
+    envelope + version negotiation.
+  - `McpToolkit::Dispatcher` — `new(context:, config:)` + `#handle_request(request)`.
+    Dispatches `initialize` / `initialized` / `tools/list` / `tools/call` / `ping`
+    and the custom `notifications/<app>/tools/list_changed` cache-bust. `tools/list`
+    merges the host's own tool definitions with the gateway's namespaced upstream
+    tools; `tools/call` routes a namespaced name to `Gateway::Proxy` (translating
+    `UnknownUpstream` → method-not-found and relaying an upstream JSON-RPC error
+    verbatim) or a host tool (scope-gated) otherwise. Server identity + negotiated
+    versions come from config; NO SDK touchpoint.
+  - `McpToolkit::Authority::ControllerMethods` — the authority transport as an
+    includable concern. Every billing/tenancy step is an overridable hook
+    (`mcp_authenticate!`, `mcp_rate_limit!`, `mcp_track_usage`, `mcp_flush_usage`,
+    `mcp_resolve_account`, `mcp_session_data`, `mcp_dispatch`, `mcp_health_payload`,
+    `mcp_config`), each defaulting to a config callable so a PURE host needs no
+    subclass. The per-request loop RE-RESOLVES the account for every JSON-RPC call
+    — including each element of a batch — so a mixed-account batch still meters one
+    usage event per call against the right account (the batch is never delegated to
+    a bulk handler that couldn't re-resolve per element).
+  - `McpToolkit::Authority::ServerController` — a base controller (concern wired in,
+    lazily-parented) a host subclasses when its hooks touch app models (the
+    recommended path).
+  - `McpToolkit::Authority::Context` — the per-request context threaded into the
+    dispatcher + tools: `account`, `principal`, `bearer_token`, and a derived
+    `superuser?` (duck-typed off the principal).
+  - `McpToolkit::Tools::AuthorityBase` — an optional base for a host's own tools
+    (class DSL `tool_name` / `description` / `input_schema` /
+    `required_permissions_scope` / `definition`; `.call(context:, **arguments)`
+    entry; context accessors; `ensure_resource_accessible!`; ArgumentError →
+    InvalidParams / StandardError → InternalError mapping). Host tools plug in
+    through the api-agnostic `config.tool_provider` seam — the gem never references
+    a host's API layer, serializers, or resource catalog.
+- **`config.tool_provider`** — the api-agnostic tool seam. Duck-typed:
+  `provider.tool_definitions(context) -> [{ name:, description:, inputSchema: }]`
+  (context lets the host hide superuser-only tools) and `provider.find(name) -> a
+  tool object` (responding to `#required_permissions_scope` + `#call(context:,
+  **arguments)`). The dispatcher enforces the per-tool scope gate CENTRALLY.
+- **Registry-backed authority tools** — the authority-path counterpart to the
+  satellite's SDK tools, so a first-party server can serve the SAME four generic
+  read tools (`resources` / `resource_schema` / `get` / `list`) over
+  `config.registry` through the hand-rolled dispatcher, reusing the existing
+  `ListExecutor` / `GetExecutor` / `ResourceSchema` / `Serialization` /
+  `FieldSelection` / `Filtering` UNCHANGED.
+  - `McpToolkit::Authority::RegistryToolProvider.new(config:)` — a `tool_provider`
+    serving the four generic tools; `find(name)` returns a tool instance, and each
+    tool declares NO static scope (the per-resource scope is enforced dynamically
+    at call time). The satellite SDK tool path (`McpToolkit::Tools::*`,
+    `McpToolkit::Server`) is untouched — this is added alongside it.
+  - `McpToolkit::Authority::Tools::{Resources,ResourceSchema,Get,List}` — the four
+    thin tools. Each resolves the `resource` argument against the registry
+    (InvalidParams for unknown), gates a `superusers_only?` resource against
+    `context.superuser?` (REFUSE in get/list/resource_schema, HIDE in resources),
+    gates the resource's `required_scope_for` against the principal, and (get/list)
+    requires a resolved `context.account`. Returns a raw Hash for the dispatcher to
+    wrap — distinct by design from the satellite tools' `MCP::Tool::Response`.
+  - `McpToolkit::Authority::CompositeToolProvider.new(*providers)` — composes
+    several providers (e.g. the RegistryToolProvider + a host's bespoke tools)
+    behind one `config.tool_provider`: `tool_definitions` concatenates in order,
+    `find` returns the first match.
+  - `config.generic_tool_name_prefix` (default `""`) — namespaces the four generic
+    Registry-backed tools. When set (e.g. `"foo_"`) the provider advertises and
+    resolves them as `foo_resources` / `foo_resource_schema` / `foo_get` /
+    `foo_list`, letting a host keep stable, namespaced tool names for existing
+    clients; the empty default keeps the bare base names.
+- **`McpToolkit::Resource` generic seams** (all api-agnostic) — `superusers_only!`
+  / `superusers_only?` (authority tools honor it), `note(text)` + reader (surfaced
+  by `resource_schema`), and `filter(name, type:, description:, &applier)` +
+  `custom_filters` — a resource-specific filter whose block narrows the scoped
+  relation from a TOP-LEVEL request param, so a host can express a relational
+  filter the generic equality/operator allowlist can't. `ListExecutor` applies the
+  matching custom filters BEFORE the allowlist filters (its only change).
+- **`McpToolkit::ResourceSchema` enrichment** — each attribute now advertises the
+  filter `operators` it accepts (derived from `Filtering::OPERATORS_BY_TYPE`), and
+  the resource `note` is passed through, so a client can discover exactly which
+  `{ op:, value: }` conditions `list` will accept.
+- **Server-vs-gateway identity split** — `config.gateway_client_name` /
+  `gateway_client_version` (each defaulting to `server_name` / `server_version`).
+  `Gateway::Client`'s handshake `clientInfo` now reads the GATEWAY identity, so an
+  authority can advertise its own `server_name` to its callers while keeping its
+  upstream handshake byte-identical.
+- **`config.supported_protocol_versions`** (default
+  `McpToolkit::Protocol::SUPPORTED_VERSIONS`) — the version set the authority
+  dispatcher negotiates.
+- **`config.rate_limiter` / `usage_recorder` / `usage_flusher`** — the authority
+  transport's billing hooks as config callables (all default `nil` / no-op).
+- **Built-in rate limiting** — `McpToolkit::RateLimiter`, a fixed-window
+  per-principal counter backed by `config.cache_store`, plus
+  `config.rate_limit_max_requests` (Integer, default `nil` = OFF) and
+  `config.rate_limit_window` (seconds, default `3600`). When a cap is set, the
+  authority transport's `mcp_rate_limit!` counts each request, sets the
+  `X-RateLimit-Limit` / `X-RateLimit-Remaining` / `X-RateLimit-Reset` headers on
+  every capped response, and over the limit renders a JSON-RPC error (code
+  `-32029`) at HTTP `429` with a `Retry-After` header. Two new overridable hooks —
+  `mcp_rate_limit_max_requests` (default `config.rate_limit_max_requests`) and
+  `mcp_rate_limit_key` (default `mcp_principal.id`) — let a host keep the cap in
+  its own constant/model or bucket the counter differently. `config.rate_limiter`
+  remains as an escape hatch that fully replaces the built-in when set. A pure
+  host that sets no cap is unaffected.
+- **`config.superuser_resolver`** — an optional `->(principal) -> Boolean` making
+  superuser a first-class, OPTIONAL gem concept. `Authority::Context#superuser?`
+  calls it when set, else falls back to duck-typing `principal.superuser?` (false
+  when the principal doesn't respond to it). Together with the existing
+  `superusers_only!` resource flag and the authority tools' gating, this
+  formalizes superuser gating; the default (no resolver, no superuser-aware
+  principal) is "no superusers".
+- **Lazy `parent_controller` (Constraint B)** — the engine's `ServerController` /
+  `TokensController` and the authority `ServerController` are no longer eager-
+  loadable files; they are built from the CURRENT config by
+  `McpToolkit.build_engine_controllers!`, triggered lazily via `const_missing` and
+  reset on each reload by the engine's `config.to_prepare`. The parent is therefore
+  read only at build time — after the host's initializers/to_prepare — so a host's
+  whole MCP initializer can live in `to_prepare`. `TokensController#introspect`
+  behavior is preserved exactly.
+
+### Fixed
+
+- **Authority boundary returns JSON-RPC errors for bad input, not a 500** — a
+  malformed JSON body now maps to a JSON-RPC parse error (`-32700`) via a
+  `respond_to?`-guarded `rescue_from` (fires even from the session before_action),
+  and a non-object request or batch element maps to `invalid_request` instead of
+  raising a `NoMethodError` in the per-call loop.
+- **`initialize` advertises `instructions`** — the authority dispatcher now
+  includes `config.server_instructions` in the `initialize` result when set
+  (omitted when nil), matching the SDK-backed satellite server and the documented
+  contract.
+- **Gateway tool-list cache is contract-enforced, not assumption-based** — the
+  per-upstream list cache is keyed by upstream only, which is safe only when every
+  upstream's `tools/list` is caller-independent. That is now an explicit
+  registration contract: an upstream that filters its list by caller privilege
+  registers `public_tool_list: false` and is pulled live per request (never
+  cached), so a privileged caller's list can't leak to an unprivileged one.
+
+### Removed
+
+- The engine's `app/controllers/mcp_toolkit/{server_controller,tokens_controller}.rb`
+  files, replaced by the lazy builder above (their routes + behavior are unchanged).
+
+- **Gateway / upstream layer** (`McpToolkit::Gateway::*`) — the generic,
+  SDK-independent machinery a central app uses to aggregate *other* MCP servers
+  and proxy calls to them, previously an app-only concern. All app-specific values
+  (upstream URLs, account-selector meta key, logger, timeouts) are injected via
+  `McpToolkit::Configuration`; nothing in the layer names a deployment.
+  - `McpToolkit::Gateway::UpstreamRegistry` — a PER-CONFIG registry of upstream
+    servers (`Upstream = Data.define(:key, :url, :public_tool_list)` with
+    `#name_for`), exposed as `config.upstreams` and reset with a fresh config (test
+    isolation for free). API: `#register(key:, url:, public_tool_list: true)` (blank
+    url ignored), `#reset!`, `#all`, `#find`, `#split_tool_name`. Config sugar:
+    `config.register_upstream(key:, url:, public_tool_list: true)`.
+  - `McpToolkit::Gateway::Client` — a minimal Streamable-HTTP MCP client
+    (`#tools_list`, `#tools_call`) with single-shot session-loss recovery (HTTP
+    404 / JSON-RPC `-32001`), SSE `data:` unwrapping, and content negotiation. Its
+    `Client::Error` (< `McpToolkit::Error`) carries `jsonrpc_error` / `http_status`
+    and references NO transport/protocol-error class — the consumer maps it. The
+    handshake `clientInfo` and protocol version come from config
+    (`DEFAULT_PROTOCOL_VERSION` falls back to the wrapped `mcp` SDK's latest).
+  - `McpToolkit::Gateway::Aggregator` — namespaces + caches (`config.cache_store`,
+    `config.upstream_list_ttl`) each upstream's tool list, pulled CONCURRENTLY via
+    concurrent-ruby (wrapped in `Rails.application.executor` when a booted Rails app
+    is present, plain futures otherwise). Only a non-empty pull is cached; a stale
+    empty is a miss (poisoned-cache self-heal); a failing upstream degrades (omit +
+    log) rather than breaking the list.
+  - `McpToolkit::Gateway::Proxy` — proxies a namespaced call, forwarding the
+    resolved `account_id` as `_meta[config.account_meta_key]`. An unknown key raises
+    `McpToolkit::Gateway::UnknownUpstream`; an upstream failure raises
+    `McpToolkit::Gateway::UpstreamCallError` (carrying `jsonrpc_error` /
+    `http_status`). Neither is mapped to a protocol-error class here.
+- **Authority introspection endpoint** — `McpToolkit::TokensController#introspect`,
+  drawn by the engine at `POST /mcp/tokens/introspect`, so a central app answers
+  introspection with no controller of its own. Its parent class is configurable via
+  `parent_controller` (like `ServerController`). The route is drawn ONLY when
+  `auth_role` is `:authority`: introspection is the provider side of the protocol,
+  so a satellite (the default role) that mounts the engine gets no such route
+  rather than one it should never answer. The controller also fails safe as defence
+  in depth — with no `token_authenticator` it answers `{ valid: false }`.
+- **`McpToolkit::Session#data`** — an opaque payload attachable at
+  `create!(data:)` and round-tripped through `find`, so an authority can bind a
+  session to a token id (letting a revoked token kill an in-flight session). The
+  gem does not interpret it; legacy rows default to `{}`.
+- `McpToolkit::Configuration` gains `upstreams` (a `Gateway::UpstreamRegistry`),
+  `register_upstream`, `upstream_timeout` (default `10`), `upstream_list_ttl`
+  (default `900`), and `logger` (default `nil`; all gateway/session call sites
+  guard with `logger&.`).
+- `concurrent-ruby` is now a direct dependency (already transitive via
+  activesupport) — the aggregator's parallel upstream pulls require it.
+
 ## [0.3.0] - 2026-07-03
 
 ### Added
@@ -119,5 +317,6 @@ hand-rolled controller wiring.
 
 ### Notes
 
-- The gateway / upstream-aggregation layer (`Mcp::Upstreams*`) is intentionally
-  out of scope (core-only).
+- The gateway / upstream-aggregation layer was intentionally out of scope for this
+  initial extraction. It was later extracted into the gem in 0.4.0 (see above), as
+  `McpToolkit::Gateway::*` — fully config-injected and app-agnostic.