connectors 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +16 -0
  3. data/CONNECTORS_FRAMEWORK.md +54 -99
  4. data/MCP_CLIENT.md +5 -5
  5. data/README.md +5 -3
  6. data/app/connectors/gmail/api.rb +6 -14
  7. data/app/connectors/gmail/connector.rb +11 -24
  8. data/app/connectors/gmail/mime_builder.rb +0 -3
  9. data/app/connectors/gmail/mime_parser.rb +1 -2
  10. data/app/connectors/gmail/polling.rb +7 -14
  11. data/app/connectors/resend/connector.rb +2 -4
  12. data/app/controllers/connectors/actions_controller.rb +0 -6
  13. data/app/controllers/connectors/credentials_controller.rb +10 -23
  14. data/app/controllers/connectors/grants_controller.rb +6 -11
  15. data/app/controllers/connectors/types_controller.rb +18 -45
  16. data/app/controllers/connectors/webhooks_controller.rb +2 -4
  17. data/app/jobs/connectors/deliver_webhook_job.rb +2 -4
  18. data/app/models/connectors/credential_share.rb +3 -10
  19. data/app/models/connectors/grant.rb +2 -4
  20. data/config/routes.rb +6 -13
  21. data/db/migrate/20260518210324_create_connectors_grants.rb +4 -11
  22. data/db/migrate/20260521140000_create_connectors_credential_shares.rb +2 -8
  23. data/docs/releasing.md +4 -4
  24. data/lib/connectors/action.rb +4 -7
  25. data/lib/connectors/action_builder.rb +5 -12
  26. data/lib/connectors/action_runner.rb +5 -11
  27. data/lib/connectors/auth_injection.rb +7 -18
  28. data/lib/connectors/client_builder.rb +4 -10
  29. data/lib/connectors/configuration.rb +9 -39
  30. data/lib/connectors/connector.rb +31 -83
  31. data/lib/connectors/credential_schema.rb +12 -29
  32. data/lib/connectors/credential_tester.rb +3 -4
  33. data/lib/connectors/credential_type_registry.rb +9 -20
  34. data/lib/connectors/credential_types/http_auth.rb +18 -55
  35. data/lib/connectors/engine.rb +3 -6
  36. data/lib/connectors/errors.rb +2 -5
  37. data/lib/connectors/middleware/authenticate_generic.rb +4 -14
  38. data/lib/connectors/middleware/pre_authentication.rb +2 -13
  39. data/lib/connectors/oauth/authorize_url.rb +3 -9
  40. data/lib/connectors/oauth/client_credentials.rb +0 -4
  41. data/lib/connectors/oauth/pkce.rb +3 -9
  42. data/lib/connectors/oauth/revoke.rb +2 -6
  43. data/lib/connectors/oauth1.rb +4 -8
  44. data/lib/connectors/permission_check.rb +2 -4
  45. data/lib/connectors/poll_runner.rb +1 -5
  46. data/lib/connectors/pre_authentication_helpers.rb +4 -9
  47. data/lib/connectors/version.rb +1 -1
  48. data/lib/connectors/webhook_context.rb +8 -18
  49. data/lib/connectors/webhook_lifecycle.rb +1 -8
  50. data/lib/connectors/webhook_methods.rb +2 -6
  51. data/openapi.yaml +16 -16
  52. metadata +21 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: dad1a8edb9dac56e6c94e54f0f926422488dfede45c390c974296cc98bdc7e4a
4
- data.tar.gz: ee1826f96e2b7b9475f0030404bd307eb1a1b89ccb3b732bc04d5f1172a0c51e
3
+ metadata.gz: 151e674f0e09fb93928d10f222909aef9d2886ae28bf5f7c3690517807ec277e
4
+ data.tar.gz: a738a4ff2cdf3c265b51ad58571349687f1a5ea8dada2b0976e68fa76ad3f12a
5
5
  SHA512:
6
- metadata.gz: c096165959caa7f1ae30a904054175a37d2ff048025fadf78a57cd9dd11c4b78adbf5e9b610e9678b2b7825726b02e36728b6ffd2cd917ba713b7e62d40a45d5
7
- data.tar.gz: f6263fcfa7aeee54d01864c10c60c6aa9cc70080f5d4a9f85e66988a4787bf0864bb916b30e12763c9bae994ff030a846a21cf3e1a90206c0ee3330c1ff83437
6
+ metadata.gz: cea593fcb524afaeeb6e2ba04604bea8baf2461d0ce730314b937d21f5d310fa6af4cd3d54bf58de2c0ab2cfce71d2cb75cecfa7edf4c550092bda9a1c5d1a1e
7
+ data.tar.gz: 739c3a4ae3fde6598d5aacf6357c443fd9dbcd4504a7f9b5bbd204ef407053d05f2b4732a7e2cac992c0987c4df78bfd5721b5bf731eed71455402581055d12a
data/CHANGELOG.md CHANGED
@@ -4,6 +4,22 @@ Changes are recorded here before release. Published packages are listed on [Ruby
4
4
 
5
5
  ## Unreleased
6
6
 
7
+ ## 0.1.1 — 2026-09-22
8
+
9
+ ### Fixed
10
+
11
+ - Constrain JSON to 2.x for Rails 8.1 decoder compatibility; JSON 3 raises when loading encrypted credentials.
12
+ - Update the development lockfile to Rails 8.1.3.1, Faraday 2.14.4 and OAuth2 2.0.25, and update CI actions.
13
+
14
+ ### Documentation
15
+
16
+ - Describe connector contracts directly across guides, API metadata, source comments and test names.
17
+ - Remove obsolete demo smoke scripts from the public source tree; supported development and package checks are documented in the contributing guide.
18
+
19
+ ### Upgrading
20
+
21
+ - Run `bundle update connectors` to receive the JSON compatibility constraint. No database migrations or application API changes are required for this patch.
22
+
7
23
  ## 0.1.0 — 2026-09-22
8
24
 
9
25
  ### Added
@@ -1,15 +1,15 @@
1
1
  # Connectors Framework Reference
2
2
 
3
- Developer reference for the `connectors` Rails engine. Historical n8n source references describe design inspiration; they are not a claim of current n8n parity. For installation and supported environments, start with the [README](README.md).
3
+ Developer reference for the `connectors` Rails engine. For installation and supported environments, start with the [README](README.md).
4
4
 
5
- Current behavior is verified by the repository test suite. [Architecture](docs/architecture.md), [connector authoring](docs/adding-connectors.md) and [MCP support](MCP_CLIENT.md) cover the newer action and remote-tool paths.
5
+ Current behavior is verified by the repository test suite. [Architecture](docs/architecture.md), [connector authoring](docs/adding-connectors.md) and [MCP support](MCP_CLIENT.md) describe action execution and remote tools.
6
6
 
7
7
  ---
8
8
 
9
9
  ## 1. Architecture
10
10
 
11
11
  ```
12
- flow-api (host Rails app)
12
+ Host Rails application
13
13
  └─ mounts Connectors::Engine at <host-chosen prefix> (e.g. /api/v1/connectors)
14
14
  ├─ Configuration block (host wires owner / OAuth secrets / vault)
15
15
  ├─ Registry (loaded at boot from app/connectors/*/)
@@ -31,7 +31,7 @@ flow-api (host Rails app)
31
31
 
32
32
  **Mount-path independence.** The host picks where to mount the engine in `config/routes.rb`. The engine introspects the actual mount path at boot (`Connectors::Engine.mount_path`) and every URL builder (OAuth callback, default webhook URL, types-endpoint `redirect_uri` field) is wired through it. No `mount_path` configuration in initializers, no string concatenation traps — the host's routes file is the single source of truth. See §1.1 below.
33
33
 
34
- **No expression engine.** Phase 1 ships a tiny `{{$credentials.<field>}}` substitutor (`Connectors::AuthInjection`) scoped to credential templates. The connectors engine stays workflow-independent anything richer belongs in the automations engine.
34
+ **Credential templates.** `Connectors::AuthInjection` substitutes `{{$credentials.<field>}}` lookups in authentication properties. It does not evaluate general expressions or workflow logic.
35
35
 
36
36
  ### 1.1 Mount-path introspection
37
37
 
@@ -59,7 +59,7 @@ Concrete consequences:
59
59
  | Default `hook_url` for `POST /grants/:id/webhook_subscribe` | hardcoded `<host>/connectors/...` | matches the mount, whatever it is |
60
60
  | `GET /types/:name` `connector.redirect_uri` / `authorize_url` / `authorize_json_url` | hardcoded | matches the mount |
61
61
 
62
- The host can remount under any prefix (`/api/v2/connectors`, `/_/integrations`, etc.) and everything Just Works.
62
+ For example, a mount under `/api/v2/connectors` produces callbacks and webhook URLs under that prefix.
63
63
 
64
64
  ---
65
65
 
@@ -82,7 +82,7 @@ Registers the connector. **Required** — every connector class calls this first
82
82
  | Arg | Type | Meaning |
83
83
  |---|---|---|
84
84
  | `key:` | symbol | machine identifier, unique per app (`:slack`) |
85
- | `auth:` | symbol | legacy auth scheme name (`:api_key`, `:oauth2`). Kept for back-compat with pre-Phase-1 connectors. New code should rely on `authenticate` (§3.3). |
85
+ | `auth:` | symbol | auth scheme name (`:api_key`, `:oauth2`). A declared `authenticate` block takes precedence for request injection (§3.3). |
86
86
  | `base_url:` | string | every `client.get`/`post`/etc. is relative to this |
87
87
  | `display_name:` | string | UI label (defaults to `key.humanize`) |
88
88
  | `icon:` | string | URL or asset path |
@@ -95,7 +95,7 @@ Declares the credential form. Inner block uses `field` (see §4).
95
95
 
96
96
  ### 3.3 `authenticate type: :generic, properties: { ... }`
97
97
 
98
- Declarative auth injection. Mirrors n8n's `IAuthenticateGeneric` (`packages/workflow/src/interfaces.ts:269-288`). `properties` accepts any subset of:
98
+ Declarative authentication injection into outgoing requests. `properties` accepts any subset of:
99
99
 
100
100
  ```ruby
101
101
  authenticate type: :generic, properties: {
@@ -155,7 +155,7 @@ oauth1 request_token_url: "https://api.twitter.com/oauth/request_token",
155
155
  signature_method: "HMAC-SHA1" # or HMAC-SHA256 / HMAC-SHA512
156
156
  ```
157
157
 
158
- Engine signs request-token + access-token legs inline (no `oauth-1.0a` gem). Consumer key/secret come from `Connectors.configuration.oauth_credentials_for(connector_key)` — same slot as OAuth2's client id/secret.
158
+ The engine signs request-token and access-token exchanges. Consumer key/secret come from `Connectors.configuration.oauth_credentials_for(connector_key)` — same slot as OAuth2's client id/secret.
159
159
 
160
160
  ### 3.7 `revoke_token_url "..."` / `revoke_token { |grant| ... }`
161
161
 
@@ -173,7 +173,7 @@ end
173
173
 
174
174
  ### 3.8 `test_request method:, url:, headers:, query:, expect_status:, rules:`
175
175
 
176
- Declarative "Test connection". Mirrors n8n's `ICredentialTestRequest` (`interfaces.ts:340-348`).
176
+ Connection test performed through the connector client, using its authentication and middleware.
177
177
 
178
178
  ```ruby
179
179
  test_request method: :get, url: "auth.test",
@@ -223,7 +223,7 @@ The controller calls `webhook_style` to know whether to expect `:app_level` (sin
223
223
 
224
224
  ### 3.12 `webhook_methods(:group) do ... end`
225
225
 
226
- Subscription lifecycle. n8n parity: `INodeType.webhookMethods.default.{checkExists, create, delete}`.
226
+ Subscription lifecycle with `check_exists`, `create` and `delete` callbacks.
227
227
 
228
228
  ```ruby
229
229
  webhook_methods do # group :default (implicit)
@@ -278,7 +278,7 @@ Manual fire: `POST <mount>/grants/:id/poll` with optional `instance_key`. Return
278
278
 
279
279
  ### 3.14 `generic_auth!`, `supported_nodes :...`, `http_request_node ...`
280
280
 
281
- n8n credential-visibility flags (`interfaces.ts:379-381`):
281
+ Credential visibility and HTTP-request picker metadata:
282
282
 
283
283
  ```ruby
284
284
  generic_auth! # eligible for the HTTP-Request node picker
@@ -311,7 +311,7 @@ end
311
311
 
312
312
  ```ruby
313
313
  def refresh!; end # called by AutoRefresh middleware on 401
314
- def poll; end # called by polling scheduler (workflow-roadmap)
314
+ def poll; end # called by PollJob; PollRunner uses the separate polling DSL
315
315
  def handle_webhook(ctx); end # called by DeliverWebhookJob; ctx = WebhookContext
316
316
  def validate_credentials!; end # auto: runs CredentialSchema.validate! against grant
317
317
  ```
@@ -320,7 +320,7 @@ def validate_credentials!; end # auto: runs CredentialSchema.validate! against
320
320
 
321
321
  ## 4. CredentialSchema Field Reference
322
322
 
323
- `Connectors::CredentialSchema::Field` mirrors n8n's `INodeProperties` (`interfaces.ts:1773-1812`):
323
+ `Connectors::CredentialSchema::Field` describes an input and its rendering metadata:
324
324
 
325
325
  ```ruby
326
326
  field :api_key,
@@ -338,23 +338,26 @@ field :api_key,
338
338
  options: [{ name: "Header", value: "header" }]
339
339
  ```
340
340
 
341
- `type_options` flags propagate to the editor verbatim:
342
- - `password: true` — masked input + redacted in serializer responses
343
- - `expirable: true` — UI marks the field as "will rotate"
344
- - `redactJsonLeaves: true` — JSON leaf values masked in error messages / logs
345
- - `resolvable_field: true` — field accepts `{{ }}` expressions
341
+ `type_options` flags are serialized as form metadata for host renderers:
342
+
343
+ - `password: true` — suggests a masked input; also set by `secret: true`
344
+ - `expirable: true` — indicates a value that may rotate
345
+ - `redactJsonLeaves: true` — suggests masking JSON leaf values
346
+ - `resolvable_field: true` — indicates a host-resolvable input
347
+
348
+ These flags do not implement token rotation, log filtering or expression evaluation. Credential response access is enforced separately by the sharing policy; owner responses can include decrypted secrets.
346
349
 
347
350
  Inheritance via `extends`:
348
351
 
349
352
  ```ruby
350
353
  credentials do
351
- extends :oauth2 # inherit OAuth2Api base fields
354
+ extends :oauth2 # inherit OAuth2 base fields
352
355
  field :authorization_url, type: "hidden", # lock provider endpoint
353
356
  default: "https://slack.com/oauth/v2/authorize"
354
357
  end
355
358
  ```
356
359
 
357
- Children override parents by redeclaring with the same field name. Most-specific wins. Same precedence rule n8n's credential-walker uses (`packages/cli/src/credential-types.ts:26-37`).
360
+ Children override parents by redeclaring with the same field name. Most-specific wins.
358
361
 
359
362
  ### Schema-level DSL
360
363
 
@@ -364,7 +367,7 @@ Inside `credentials do ... end`:
364
367
  - `authenticate type: :generic, properties: {...}` — schema-level injection (inherited by every connector that `extends` this schema)
365
368
  - `generic_auth!` — mark eligible for HTTP-Request node
366
369
  - `display_name "Bearer Auth"` — UI label for the credential type itself
367
- - `documentation_url "httprequest"` — n8n docs slug
370
+ - `documentation_url "https://provider.example/docs/auth"` — documentation link
368
371
 
369
372
  ---
370
373
 
@@ -384,7 +387,7 @@ Connectors.configure do |c|
384
387
  }
385
388
 
386
389
  # ---- Optional: webhook dispatch hook -----------------------------------
387
- c.on_webhook = ->(event) { Inbox::IngestJob.perform_later(event.id) }
390
+ c.on_webhook = ->(event) { ProcessWebhookJob.perform_later(event.id) }
388
391
 
389
392
  # ---- Optional: sharing principals (host middleware supplies trusted owner) ----------------------------
390
393
  c.principal_resolver = ->(ctrl) {
@@ -392,7 +395,7 @@ Connectors.configure do |c|
392
395
  owner ? [[owner.class.name, owner.id]] : []
393
396
  }
394
397
 
395
- # ---- Optional: external secrets manager (Phase 10) ---------------------
398
+ # ---- Optional: external secrets manager ---------------------
396
399
  c.secrets_resolver = ->(grant) { vault.read(grant.external_ref) }
397
400
  c.secrets_managed_fields_for = ->(key) { { slack: %w[client_secret] }.fetch(key.to_sym, []) }
398
401
  end
@@ -400,6 +403,8 @@ end
400
403
 
401
404
  The Rack owner entry in this example must be populated by trusted host authentication middleware; engine controllers do not inherit host controller callbacks. Slack/Twitter configuration here illustrates custom OAuth connectors, not shipped profiles.
402
405
 
406
+ `ProcessWebhookJob` and `vault` are examples supplied by the host.
407
+
403
408
  All callables are invoked with `Connectors.configuration.<name>.call(...)` — the engine never invokes host constants directly.
404
409
 
405
410
  ---
@@ -408,16 +413,16 @@ All callables are invoked with `Connectors.configuration.<name>.call(...)` — t
408
413
 
409
414
  Registered on engine boot in `Connectors::CredentialTypeRegistry`. Connectors `extends` any of these.
410
415
 
411
- | Key | n8n source | Engine status |
412
- |---|---|---|
413
- | `:oauth2` | `OAuth2Api.credentials.ts:1-238` | all 3 grant types + JWE form fields |
414
- | `:oauth1_api` | `OAuth1Api.credentials.ts:1-72` | HMAC-SHA1/256/512 |
415
- | `:http_basic_auth` | `HttpBasicAuth.credentials.ts` | Basic header via `auth:` shortcut |
416
- | `:http_bearer_auth` | `HttpBearerAuth.credentials.ts` | Authorization: Bearer |
417
- | `:http_header_auth` | `HttpHeaderAuth.credentials.ts` | ✅ user-named header (templated key) |
418
- | `:http_query_auth` | `HttpQueryAuth.credentials.ts` | user-named query param |
419
- | `:http_digest_auth` | `HttpDigestAuth.credentials.ts` | ⚠️ schema only (Faraday-Digest gem TBD) |
420
- | `:http_custom_auth` | `HttpCustomAuth.credentials.ts` | ⚠️ schema only (runtime in Custom-API-Call node) |
416
+ | Key | Support |
417
+ |---|---|
418
+ | `:oauth2` | Authorization code, client credentials and PKCE flows. JWE fields are form metadata only; no JWE decryption. |
419
+ | `:oauth1_api` | HMAC-SHA1, HMAC-SHA256 and HMAC-SHA512 signing. |
420
+ | `:http_basic_auth` | Basic authentication through the `auth:` shortcut. |
421
+ | `:http_bearer_auth` | Authorization bearer token. |
422
+ | `:http_header_auth` | Credential-defined header name and value. |
423
+ | `:http_query_auth` | Credential-defined query parameter name and value. |
424
+ | `:http_digest_auth` | Form schema only; no digest challenge handling. |
425
+ | `:http_custom_auth` | Form schema only; no runtime injection of custom JSON. |
421
426
 
422
427
  ---
423
428
 
@@ -437,7 +442,7 @@ One row = one authorized account. Columns:
437
442
  | `expires_at`, `last_used_at` | datetime | |
438
443
  | `external_account_id` | string | denormalized provider-side id for webhook routing |
439
444
  | `static_data` | jsonb | per-group scratch (webhook + polling cursor state) |
440
- | `external_ref`, `is_managed` | string + bool | external secrets manager (Phase 10) |
445
+ | `external_ref`, `is_managed` | string + bool | external secrets manager |
441
446
 
442
447
  Methods:
443
448
 
@@ -539,9 +544,9 @@ All persist a `WebhookEvent` and enqueue `DeliverWebhookJob` → calls `Connecto
539
544
 
540
545
  ---
541
546
 
542
- ## 9. `WebhookContext` (Phase 7)
547
+ ## 9. `WebhookContext`
543
548
 
544
- Passed to `Connector#handle_webhook`. Mirrors n8n's `IWebhookFunctions` (`interfaces.ts:1327-1350`).
549
+ Passed to `Connector#handle_webhook` with the captured request data and persisted event.
545
550
 
546
551
  ```ruby
547
552
  def handle_webhook(ctx)
@@ -632,7 +637,7 @@ POST /credentials/:id/revoke → revoke_token block OR RFC 7009 POST to revok
632
637
 
633
638
  ---
634
639
 
635
- ## 12. Webhook Subscription Lifecycle (Phase 6)
640
+ ## 12. Webhook Subscription Lifecycle
636
641
 
637
642
  ```
638
643
  POST /grants/:id/webhook_subscribe (optional: webhook_name, hook_url)
@@ -652,7 +657,7 @@ Default `hook_url` is computed from `host_base_url` + `connector_key` + grant_id
652
657
 
653
658
  ---
654
659
 
655
- ## 13. Polling (Phase 8)
660
+ ## 13. Polling
656
661
 
657
662
  ```
658
663
  POST /grants/:id/poll
@@ -666,7 +671,7 @@ The diagram shows the legacy call without an instance key. For independent consu
666
671
 
667
672
  ---
668
673
 
669
- ## 14. Sharing (Phase 9)
674
+ ## 14. Sharing
670
675
 
671
676
  Role hierarchy: `viewer (0) < editor (1) < owner (2)`. HTTP credential access uses `GrantAccess#find_visible_grant!(id, min_role:)` and `GrantPolicy`; MCP services enforce the same roles through `MCP::Access`. Direct REST Ruby callers must authorize their selected grant.
672
677
 
@@ -678,7 +683,7 @@ Index returns the union. Show/update/destroy escalate the minimum role required.
678
683
 
679
684
  ---
680
685
 
681
- ## 15. External Secrets (Phase 10)
686
+ ## 15. External Secrets
682
687
 
683
688
  ```
684
689
  grant.is_managed? + grant.external_ref
@@ -732,68 +737,18 @@ end
732
737
  ```
733
738
 
734
739
  Patterns used throughout `spec/connectors/`:
735
- - **Anonymous classes** for per-spec test connectors → registry cleanup in `after`
740
+ - **Anonymous classes** for per-spec test connectors → registry isolation in the RSpec setup
736
741
  - **`WebMock`** for outbound HTTP
737
742
  - **`stub_request(...)` with `.with(headers: ...)`** verifies the auth-injecting middleware did its job
738
743
  - **`Connectors::OAuth::State.decode(token)`** to peek inside the encrypted state for round-trip testing
739
744
 
740
745
  ---
741
746
 
742
- ## 18. Reference Index
743
-
744
- Every n8n source citation used across the engine:
745
-
746
- ```
747
- packages/workflow/src/interfaces.ts
748
- :197-208 IRequestOptionsSimplifiedAuth (authenticate properties shape)
749
- :269-288 IAuthenticateGeneric (declarative auth shape)
750
- :340-348 ICredentialTestRequest (test_request)
751
- :350-354 ICredentialHttpRequestNode (http_request_node)
752
- :356-382 ICredentialType (envelope: name, displayName, etc.)
753
- :374-377 preAuthentication (hook)
754
- :379-381 genericAuth/supportedNodes/httpRequestNode
755
- :382 __overwrittenProperties (managed fields)
756
- :1257-1274 getWorkflowStaticData (static_data parity)
757
- :1327-1350 IWebhookFunctions (WebhookContext)
758
- :1561-1584 NodePropertyTypes (field type enum)
759
- :1730-1771 IDisplayOptions + _cnd (show/hide)
760
- :1773-1812 INodeProperties (field model)
761
- :2017 webhookMethods (declaration site)
762
- :2060 polling (poll() method)
763
- :2091-2095 webhookMethods.default (checkExists/create/delete)
764
- :2600 IWebhookDescription (named-group routing)
765
-
766
- packages/cli/src/credential-types.ts:26-37 extends walker
767
-
768
- packages/cli/src/credentials/credentials.controller.ts:67-411 CRUD endpoints
769
- :100-111 GET /new (unique default name)
770
- :141-152 POST /test
771
-
772
- packages/cli/src/controllers/oauth/oauth1-credential.controller.ts:43-101 OAuth1 callback
773
- packages/cli/src/controllers/oauth/oauth2-credential.controller.ts:26-37 authorize.json
774
-
775
- packages/cli/src/oauth/oauth.service.ts:511-525, 578-586, 765-825 OAuth2 grant types + PKCE
776
- :601-700 OAuth1 generate auth uri
777
- :528, :690 redirect URI computation
778
-
779
- packages/cli/src/services/frontend.service.ts:681-705 __overwrittenProperties population
780
- :707-711 __skipManagedCreation
781
-
782
- packages/cli/src/webhooks/webhook.service.ts:418 IWebhookFunctions consumption
783
-
784
- packages/nodes-base/credentials/
785
- OAuth2Api.credentials.ts:1-238 canonical OAuth2 (incl. JWE fields :208-237)
786
- OAuth1Api.credentials.ts:1-72 canonical OAuth1
787
- HttpBasicAuth.credentials.ts basic
788
- HttpBearerAuth.credentials.ts bearer
789
- HttpHeaderAuth.credentials.ts user-named header
790
- HttpQueryAuth.credentials.ts user-named query
791
- HttpDigestAuth.credentials.ts digest
792
- HttpCustomAuth.credentials.ts custom JSON
793
- CrowdStrikeOAuth2Api.credentials.ts:62-76 preAuthentication reference
794
- SlackApi.credentials.ts:56-71 test_request reference
795
-
796
- packages/nodes-base/nodes/
797
- Postmark/PostmarkTrigger.node.ts:114-247 webhookMethods reference
798
- Google/Gmail/GmailTrigger.node.ts:65, 281-553 polling reference
799
- ```
747
+ ## 18. Further Reading
748
+
749
+ - [Installation and host setup](README.md)
750
+ - [Architecture and execution boundaries](docs/architecture.md)
751
+ - [Adding a connector](docs/adding-connectors.md)
752
+ - [Remote MCP connections](MCP_CLIENT.md)
753
+ - [HTTP API contract](openapi.yaml)
754
+ - [Development and testing](CONTRIBUTING.md)
data/MCP_CLIENT.md CHANGED
@@ -2,11 +2,11 @@
2
2
 
3
3
  The engine consumes remote MCP **2026-07-28** servers through Streamable HTTP. It supports public servers, bearer tokens, custom authentication headers, and user OAuth with discovery, PKCE, registration, refresh and scope upgrades. Tools remain scoped to their grant; they are not added to the global connector registry.
4
4
 
5
- The implementation uses public helpers from `mcp` **1.6.0**, `json_schemer` **2.5**, and `event_stream_parser` **1.0**. Its durable OAuth coordinator and HTTP transport are owned by this engine. The SDK's synchronous OAuth flow and high-level tool projection did not meet the reviewed requirements. No SDK private methods are patched or invoked. Server result types are validated against the vendored, versioned official schema; its source and license are in [protocol/README.md](lib/connectors/mcp/protocol/README.md).
5
+ The implementation uses public helpers from `mcp` **1.6.0**, `json_schemer` **2.5**, and `event_stream_parser` **1.0**. The engine implements durable OAuth coordination and HTTP transport, using only public SDK interfaces. Server result types are validated against the vendored, versioned official schema; its source and license are in [protocol/README.md](lib/connectors/mcp/protocol/README.md).
6
6
 
7
7
  ## Installation and host setup
8
8
 
9
- Run `bundle install` and the host's migrations. The new migration creates encrypted, expiring authorization and interaction records, with PostgreSQL foreign keys and one-time claims. Existing connector tables and credentials are preserved.
9
+ Run `bundle install` and the host's migrations. Migrations create encrypted, expiring authorization and interaction records, with PostgreSQL foreign keys and one-time claims.
10
10
 
11
11
  Follow the [host installation guide](README.md#installation), including encryption and trusted authentication middleware. Configure the engine's existing owner/principal resolvers. These remain the authentication boundary; callers must never supply their own trusted principal list through request parameters.
12
12
 
@@ -74,9 +74,9 @@ The named connector fixes the official endpoint to `https://mcp.clickup.com/mcp`
74
74
 
75
75
  `GET /connectors/types/clickup` exposes `connector.mcp` metadata with endpoint, authentication mode, protocol version and grant-scoped URL templates. Replace `:id` with the created credential ID. `connector.redirect_uri` uses the same callback resolver as authorization. The generic MCP type exposes the same workflow metadata, with a user-configured endpoint and authentication mode.
76
76
 
77
- **Evidence and validation (2026-09-22):** [ClickUp's official documentation](https://developer.clickup.com/docs/connect-an-ai-assistant-to-clickups-mcp-server) specifies the endpoint and OAuth-only authentication. Its live 401 challenge points to [protected-resource metadata](https://mcp.clickup.com/.well-known/oauth-protected-resource/mcp), advertising `read` and `write` scopes. Its [authorization-server metadata](https://mcp.clickup.com/.well-known/oauth-authorization-server) advertises S256 PKCE, dynamic registration, public-client token authentication and required callback issuer identification. It advertises `authorization_code` without `refresh_token`; registration requests respect that metadata. Endpoints and scopes are discovered at runtime, not copied into provider-specific OAuth code.
77
+ **Provider discovery:** [ClickUp's official documentation](https://developer.clickup.com/docs/connect-an-ai-assistant-to-clickups-mcp-server) specifies the endpoint and OAuth-only authentication. Its live 401 challenge points to [protected-resource metadata](https://mcp.clickup.com/.well-known/oauth-protected-resource/mcp), advertising `read` and `write` scopes. Its [authorization-server metadata](https://mcp.clickup.com/.well-known/oauth-authorization-server) advertises S256 PKCE, dynamic registration, public-client token authentication and required callback issuer identification. It advertises `authorization_code` without `refresh_token`; registration requests respect that metadata. Endpoints and scopes are discovered at runtime, not copied into provider-specific OAuth code.
78
78
 
79
- Request specs use captured public metadata and simulated registration/token/tool responses. The synthetic tool in those specs is explicitly a fixture. Live verification has covered public discovery and a demo owner completing consent with encrypted credential storage. Authenticated tool discovery/invocation against that Workspace has not been verified by the automated fixture suite.
79
+ Request specs use captured public metadata and simulated registration, token and tool responses. Synthetic tools are fixtures, not a fixed catalog of ClickUp capabilities. Automated tests do not authorize or invoke tools against a live ClickUp account.
80
80
 
81
81
  ## Discover, invoke and resume
82
82
 
@@ -158,7 +158,7 @@ This release targets **2026-07-28 remote tools**. It does not advertise historic
158
158
 
159
159
  ## Validation
160
160
 
161
- Validated on 2026-09-22 after packaging and documentation review: **543 examples, 0 failures** (random seed `17450`, frozen lockfile), **200 Ruby files with no RuboCop offenses**, and a strict gem build plus isolated installation/boot check. This includes the existing connector regression suite.
161
+ The [contributing guide](CONTRIBUTING.md) describes the test environment and package verification. CI runs RSpec, RuboCop and an isolated installation/boot check.
162
162
 
163
163
  RSpec covers protocol/schema contracts, authentication/registration, access isolation, secret serialization, real PostgreSQL concurrency, durable resume, real chunked SSE and idle cancellation. Interoperability runs against:
164
164
 
data/README.md CHANGED
@@ -9,23 +9,25 @@ A mountable Rails engine for connecting accounts to provider APIs and remote MCP
9
9
  | Component | Requirement | Validation baseline |
10
10
  | --- | --- | --- |
11
11
  | Ruby | 3.2 or later | Ruby 3.4.8 locally and in the CI configuration |
12
- | Rails | 8.1.3 or later in the 8.1 series | Rails 8.1.3 |
12
+ | Rails | 8.1.3 or later in the 8.1 series | Rails 8.1.3.1 |
13
13
  | Database | PostgreSQL 13+; migrations use UUIDs, JSONB and partial indexes | CI targets PostgreSQL 16 |
14
14
  | Identity | Persisted owner model with UUID primary keys; sharing principals also use UUIDs | Dummy `Owner` model in tests |
15
15
  | Encryption | Host-configured Active Record Encryption keys and stable Rails `secret_key_base` | Synthetic keys in tests |
16
16
 
17
17
  The declared Ruby minimum matches Rails' requirement; it is not a claim that every Ruby/OS combination has been tested. SQLite, MySQL and integer owner IDs are not supported by the shipped migrations. Install the host's PostgreSQL adapter (`pg`) in its Gemfile.
18
18
 
19
+ JSON is constrained to 2.x because Rails 8.1's decoder is incompatible with JSON 3's argument signature.
20
+
19
21
  ## Installation
20
22
 
21
23
  For a published release, add to your host's Gemfile:
22
24
 
23
25
  ```ruby
24
- gem "connectors", "~> 0.1.0"
26
+ gem "connectors", "~> 0.1.1"
25
27
  gem "pg"
26
28
  ```
27
29
 
28
- Then run `bundle install`. Before the first publication, or to test unreleased changes, use `gem "connectors", git: "https://github.com/jackhelio/connectors.git", ref: "<reviewed-commit-sha>"` instead, replacing the ref with an actual reviewed commit. Local development can use `gem "connectors", path: "../connectors"`. See [release checks](docs/releasing.md).
30
+ Then run `bundle install`. To test unreleased changes, use `gem "connectors", git: "https://github.com/jackhelio/connectors.git", ref: "<reviewed-commit-sha>"` instead, replacing the ref with an actual reviewed commit. Local development can use `gem "connectors", path: "../connectors"`. See [release checks](docs/releasing.md).
29
31
 
30
32
  Mount the engine in `config/routes.rb`:
31
33
 
@@ -1,10 +1,6 @@
1
1
  module Gmail
2
- # Shared HTTP layer for every Gmail action. Mirrors n8n's
3
- # `googleApiRequest` / `googleApiRequestAllItems` in
4
- # `packages/nodes-base/nodes/Google/Gmail/GenericFunctions.ts` —
5
- # one entry point that normalizes Google-specific errors into our
6
- # `Connectors::ApiError` hierarchy with user-friendly messages,
7
- # plus an auto-paginating variant for `*.list` endpoints.
2
+ # Shared HTTP layer for Gmail actions. Normalizes provider errors into
3
+ # the Connectors::ApiError hierarchy and paginates list endpoints.
8
4
  module Api
9
5
  module_function
10
6
 
@@ -29,8 +25,7 @@ module Gmail
29
25
  raise translate(e, resource: resource)
30
26
  end
31
27
 
32
- # Walks `nextPageToken` until exhausted; concatenates the array at
33
- # `property_name`. Matches n8n's signature so the porting is mechanical.
28
+ # Walks `nextPageToken` until exhausted and concatenates `property_name` arrays.
34
29
  def request_all(client, property_name, method, path, body: nil, query: nil, resource: "resource", page_size: 100)
35
30
  items = []
36
31
  q = (query || {}).dup
@@ -47,9 +42,8 @@ module Gmail
47
42
  items
48
43
  end
49
44
 
50
- # n8n's `googleApiRequest` rescue ladder translated to our types.
51
- # When a translation matches, we raise a new ApiError with the
52
- # user-meaningful message; otherwise the original passes through.
45
+ # Translate recognized provider errors into actionable messages.
46
+ # Unrecognized errors pass through unchanged.
53
47
  def translate(error, resource:)
54
48
  status = error.respond_to?(:status) ? error.status.to_i : 0
55
49
  body = error.respond_to?(:body) ? error.body : nil
@@ -73,9 +67,7 @@ module Gmail
73
67
  status: status, body: body
74
68
  )
75
69
  when 409
76
- # n8n parity: any 409 on the label resource is treated as a
77
- # name collision — Gmail's API doesn't return 409 for any other
78
- # reason on /users/me/labels.
70
+ # Treat a label conflict as a name collision.
79
71
  if resource.to_s == "label"
80
72
  raise Connectors::ApiError.new("Label name already exists", status: status, body: body)
81
73
  end
@@ -4,16 +4,9 @@ require_dependency Connectors::Engine.root.join("app/connectors/gmail/mime_parse
4
4
  require_dependency Connectors::Engine.root.join("app/connectors/gmail/polling.rb").to_s
5
5
 
6
6
  module Gmail
7
- # Gmail OAuth2 connector. Talks to gmail.googleapis.com using a Bearer
8
- # access_token. Auth is the standard Google OAuth2 redirect dance with
9
- # PKCE (Google enforces it for confidential clients) and refresh_token
10
- # rotation. Per Jackson's call: the scope set declared here is a
11
- # SUGGESTION — the host can override per-grant by passing `?scope=...`
12
- # to /authorize, so multi-tenant apps can ask for narrower / wider sets.
13
- #
14
- # n8n parity: packages/nodes-base/credentials/GmailOAuth2.credentials.ts
15
- # + packages/nodes-base/nodes/Google/Gmail/v2/GmailV2.node.ts. Action
16
- # surface mirrors n8n's Message / Label / Thread / Draft resources.
7
+ # Gmail OAuth2 connector with PKCE and refresh-token support.
8
+ # The declared scopes are defaults; hosts can override them with the
9
+ # `scope` parameter when starting authorization.
17
10
  class Connector < Connectors::Connector
18
11
  DEFAULT_SCOPES = [
19
12
  "openid",
@@ -94,13 +87,8 @@ module Gmail
94
87
 
95
88
  test_request method: :get, url: "users/me/profile"
96
89
 
97
- # =========================================================================
98
- # POLLING TRIGGER Gmail-as-trigger via users.messages.list?q=after:<ts>
99
- # n8n parity: packages/nodes-base/nodes/Google/Gmail/GmailTrigger.node.ts
100
- # Cursor: { last_checked_at: <unix>, possible_duplicates: [<id>, ...] }
101
- # Filters supplied by the workflow author at runtime (passed through
102
- # `grant.static_data["polling"]["filters"]` if set, otherwise nil).
103
- # =========================================================================
90
+ # Polls users.messages.list using a timestamp cursor and duplicate IDs.
91
+ # Filters come from the polling state supplied by PollRunner.
104
92
  polling do |grant, sd|
105
93
  Gmail::Polling.new(grant, sd).run
106
94
  end
@@ -144,7 +132,7 @@ module Gmail
144
132
 
145
133
  action :reply_to_message,
146
134
  display_name: "Reply to Email",
147
- description: "Reply to an existing message, preserving thread + In-Reply-To/References headers (mirrors n8n's `Message > Reply`)." do
135
+ description: "Reply to an existing message, preserving thread + In-Reply-To/References headers." do
148
136
  field :message_id, type: "string", display_name: "Message ID", required: true,
149
137
  description: "The Gmail id of the message you're replying to."
150
138
  field :html, type: "string", display_name: "HTML Body",
@@ -560,8 +548,8 @@ module Gmail
560
548
 
561
549
  # =========================================================================
562
550
  # API METHODS — thin wrappers around Gmail REST endpoints. Actions
563
- # delegate to these; downstream automations and curl users can call
564
- # them directly via `grant.connector.<method>`.
551
+ # delegate to these methods; authorized Ruby callers can use
552
+ # `grant.connector.<method>` directly.
565
553
  # =========================================================================
566
554
 
567
555
  def send_message(to:, subject:, html: nil, text: nil,
@@ -590,9 +578,8 @@ module Gmail
590
578
  normalize_message(response)
591
579
  end
592
580
 
593
- # n8n parity: packages/nodes-base/nodes/Google/Gmail/utils/replyToEmail.ts
594
- # Fetches the parent message's headers (Message-ID, Subject, From, To,
595
- # Reply-To), assembles a properly threaded reply via MimeBuilder, sends.
581
+ # Fetches the parent message's headers and sends a threaded reply
582
+ # using MimeBuilder.
596
583
  def reply_to_message(message_id:, html: nil, text: nil,
597
584
  cc: nil, bcc: nil, sender_name: nil,
598
585
  reply_to_sender_only: false,
@@ -615,7 +602,7 @@ module Gmail
615
602
  subject = headers["subject"].to_s
616
603
  reply_subject = subject.start_with?(/re:\s/i) ? subject : "Re: #{subject}"
617
604
 
618
- # Build the To: list same precedence rules as n8n's replyToEmail.
605
+ # Apply sender/recipient flags, prefer Reply-To over From, and exclude self.
619
606
  profile = Api.request(client, :get, "users/me/profile", resource: "profile")
620
607
  my_email = profile["emailAddress"].to_s
621
608
 
@@ -19,9 +19,6 @@ module Gmail
19
19
  # - Subject + From/To/Cc display-name phrases: RFC 2047 encoded-word
20
20
  # - Attachment filenames: RFC 2231 continuation (filename*=UTF-8''…)
21
21
  # - Bodies: base64 transfer-encoded UTF-8
22
- #
23
- # n8n parity: their Gmail node uses nodemailer's mail-composer for the
24
- # same job. We hand-roll because Gmail only needs the simple subset.
25
22
  class MimeBuilder
26
23
  CRLF = "\r\n".freeze
27
24
 
@@ -2,8 +2,7 @@ require "base64"
2
2
 
3
3
  module Gmail
4
4
  # Parses Google's `messages.get?format=full` payload tree into a clean,
5
- # downstream-friendly shape. Mirrors the work `parseRawEmail` does in
6
- # n8n's `Gmail/GenericFunctions.ts` (which delegates to mailparser).
5
+ # normalized message envelope.
7
6
  #
8
7
  # Google returns:
9
8
  # { payload: {
@@ -1,10 +1,9 @@
1
1
  module Gmail
2
2
  # Polling trigger logic. Invoked once per scheduler tick by
3
- # `Connectors::PollRunner.run(grant)`, which threads in the per-grant
4
- # scratch hash stored at `grant.static_data["polling"]`.
3
+ # `Connectors::PollRunner.run`, which supplies isolated state when an
4
+ # instance_key is provided, or per-grant polling state otherwise.
5
5
  #
6
- # n8n parity: `packages/nodes-base/nodes/Google/Gmail/GmailTrigger.node.ts`.
7
- # Key invariants we copy 1:1:
6
+ # Cursor and filtering behavior:
8
7
  #
9
8
  # * Cursor = `last_checked_at` (unix seconds). First-ever poll bootstraps
10
9
  # the cursor to `now` and emits nothing.
@@ -19,9 +18,7 @@ module Gmail
19
18
  # SENT and INBOX when the user is the recipient; we want the inbound
20
19
  # copy, not the outbound).
21
20
  #
22
- # Filters are passed via `grant.static_data["polling"]["filters"]` so the
23
- # workflow author can configure them per-trigger when the automations
24
- # engine hooks this up:
21
+ # Filters are read from the supplied polling state under "filters":
25
22
  #
26
23
  # {
27
24
  # "q" => "from:boss has:attachment", # raw Gmail search
@@ -94,9 +91,7 @@ module Gmail
94
91
 
95
92
  private
96
93
 
97
- # Gmail search syntax. n8n's prepareQuery is the reference; the order
98
- # of clauses doesn't matter to Gmail, but keep the same join scheme so
99
- # diffs against n8n are easy to read.
94
+ # Combine the cursor boundary and configured Gmail search filters.
100
95
  def build_query
101
96
  qs = {}
102
97
  qs["labelIds[]"] = Array(@filters["label_ids"]) if @filters["label_ids"]
@@ -110,12 +105,10 @@ module Gmail
110
105
  status = @filters["read_status"].to_s
111
106
  q_parts << "is:#{status}" if status == "unread" || status == "read"
112
107
 
113
- # Boundary-inclusive `after:` same behavior n8n leans on; we
114
- # de-dupe via possible_duplicates rather than narrowing the window.
108
+ # Include the cursor boundary and suppress previously emitted IDs.
115
109
  q_parts << "after:#{@sd['last_checked_at']}"
116
110
 
117
- # `-in:scheduled` matches n8n's v1.4+ guard (scheduled-send drafts
118
- # appear in users.messages.list but aren't real inbound mail yet).
111
+ # Exclude scheduled messages from inbound polling.
119
112
  q_parts << "-in:scheduled"
120
113
 
121
114
  qs[:q] = q_parts.join(" ")