@tiledesk/tiledesk-server 3.3.5 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/CHANGELOG.md +17 -0
  2. package/app.js +7 -0
  3. package/docs/activities-client.md +27 -0
  4. package/docs/mcp-oauth2-it.md +279 -0
  5. package/docs/mcp-oauth2.md +279 -0
  6. package/docs/superpowers/plans/2026-11-10-mcp-oauth2-server-it.md +155 -0
  7. package/docs/superpowers/plans/2026-11-10-mcp-oauth2-server.md +155 -0
  8. package/docs/superpowers/plans/2026-11-10-mcp-oauth2-tybot-it.md +63 -0
  9. package/docs/superpowers/plans/2026-11-10-mcp-oauth2-tybot.md +63 -0
  10. package/docs/superpowers/specs/2026-11-10-mcp-oauth2-design-it.md +207 -0
  11. package/docs/superpowers/specs/2026-11-10-mcp-oauth2-design.md +207 -0
  12. package/jobs.js +15 -0
  13. package/models/mcpOAuthConnection.js +131 -0
  14. package/package.json +1 -1
  15. package/pubmodules/activities/activityMessageUtil.js +137 -0
  16. package/pubmodules/activities/models/activity.js +2 -0
  17. package/pubmodules/activities/routes/activity.js +30 -1
  18. package/pubmodules/activities/test/execution-filter.test.js +133 -0
  19. package/pubmodules/activities/test/execution-messages.test.js +158 -0
  20. package/pubmodules/executions-publisher/index.js +457 -0
  21. package/pubmodules/executions-publisher/test/audit-consumer.test.js +384 -0
  22. package/pubmodules/executions-publisher/test/executions-publisher.test.js +204 -0
  23. package/pubmodules/pubModulesManager.js +12 -0
  24. package/routes/executions-refire.js +148 -0
  25. package/routes/mcp.js +11 -8
  26. package/routes/mcpOAuth.js +133 -0
  27. package/routes/mcpOAuthCallback.js +75 -0
  28. package/routes/request.js +5 -0
  29. package/routes/webhook.js +120 -54
  30. package/services/mcpOAuthDiscovery.js +248 -0
  31. package/services/mcpOAuthGoogleScopes.js +55 -0
  32. package/services/mcpOAuthService.js +732 -0
  33. package/services/mcpService.js +111 -15
  34. package/services/requestService.js +4 -2
  35. package/services/webhookService.js +4 -2
  36. package/test/executions-refire.test.js +362 -0
  37. package/test/mcpOAuthDiscovery.test.js +331 -0
  38. package/test/mcpOAuthGoogleScopes.test.js +25 -0
  39. package/test/mcpOAuthRoute.test.js +455 -0
  40. package/test/mcpOAuthService.test.js +693 -0
  41. package/test/mcpOAuthUtils.test.js +348 -0
  42. package/test/mcpServiceOAuth.test.js +267 -0
  43. package/test/requestRoute.js +57 -0
  44. package/test/webhookConversationRoute.js +51 -0
  45. package/utils/logRedactor.js +4 -2
  46. package/utils/mcpOAuthCrypto.js +69 -0
  47. package/utils/mcpOAuthPkce.js +20 -0
  48. package/utils/outboundUrlGuard.js +131 -0
@@ -0,0 +1,63 @@
1
+ # MCP OAuth2 — Piano di implementazione tiledesk-chatbot (tybot)
2
+
3
+ > **Per gli agenti:** SUB-SKILL RICHIESTA: usare superpowers:subagent-driven-development (consigliata) o superpowers:executing-plans per implementare questo piano task per task. Gli step usano la sintassi checkbox (`- [ ]`) per il tracciamento.
4
+
5
+ **Obiettivo:** Quando un'action AI Prompt usa un server MCP la cui voce nell'integration ha `oauth.connection_id`, tybot recupera l'access token da tiledesk-server e lo invia come `Authorization: Bearer …` al backend LLM insieme agli altri header MCP.
6
+
7
+ **Architettura:** Nuovo `services/McpOAuthTokenService.js` (`GET /:id_project/mcp/oauth/connections/:id/token` con il JWT del chatbot, breve cache in-process) → `DirAiPrompt` risolve i token dopo `enrichServersFromIntegration` e li passa ad `arrayToObject`, che imposta `headers.Authorization` per ultimo.
8
+
9
+ **Stack:** Node, `utils/HttpUtils`, mocha + l'harness di test esistente di AI Prompt.
10
+
11
+ **Spec:** `docs/superpowers/specs/2026-11-10-mcp-oauth2-design-it.md` (§7 endpoint del token, §8 cosa salva il server nell'integration, §11 errori).
12
+
13
+ ## Vincoli globali
14
+
15
+ - Branch: nuovo `feature/mcp-oauth2` da `main` in tiledesk-chatbot. Un commit per task. Mai fare push.
16
+ - Dipende dall'endpoint del token sul server, già implementato in tiledesk-server (commit `496a71cb`, verificato end to end: spec §14). I token si recuperano sempre dal server: tybot non legge mai `mcp_oauth_connections` dal suo database, anche se `IntegrationService` legge le integration da lì (i token sono cifrati e il refresh sta sul server).
17
+ - Richiesta: `GET API_ENDPOINT + "/" + id_project + "/mcp/oauth/connections/" + connection_id + "/token"`, header `Authorization: 'JWT ' + this.token` (come `IntegrationService.getIntegrationFromApi`). Body della risposta 200 `{ access_token, token_type, expires_at }`.
18
+ - Cache: `Map` in-process con chiave `id_project:connection_id`, voce valida fino a `expires_at - 60 s`; nessuna cache quando `expires_at` è null; mai nei log.
19
+ - Errore (409 `needs_reauth`, 404, 5xx, rete): l'action fallisce come quando manca l'integration MCP — `flowError` = `"MCP server <name> needs to be reconnected"` per il 409, `"MCP server <name> OAuth token unavailable"` negli altri casi; poi `falseIntent` se impostato, altrimenti `callback()`.
20
+ - L'`Authorization` da OAuth sovrascrive un `Authorization` proveniente da `customHeaders` o da variabili del flow.
21
+ - Test: sottoinsieme di `npm test` per `tybotRoute/test/conversation-ai_prompt_test.js`, `tybotRoute/test/dir_ai_prompt_headers_test.js` e il nuovo file; il server è mockato dallo stub HTTP già usato in quei test.
22
+
23
+ ## Punti di attenzione per la review
24
+
25
+ 1. Un server senza `oauth` → nessuna richiesta del token (nessuna regressione, nessuna latenza aggiunta).
26
+ 2. Il token non compare mai nell'output di `winston`/`this.logger` (`json.servers` è loggato in debug: oscurare lì `headers.Authorization`).
27
+ 3. I server nativi non hanno mai `oauth`: il server lo rifiuta sulle voci native (400, spec §8). Una voce nativa non deve chiamare il service del token e mantiene invariati gli header delle variabili del flow.
28
+
29
+ ---
30
+
31
+ ### Task 1: service del token
32
+
33
+ **File:**
34
+ - Creare: `tybotRoute/services/McpOAuthTokenService.js`
35
+ - Test: `tybotRoute/test/mcp_oauth_token_service_test.js`
36
+
37
+ **Interfacce:**
38
+ - `getToken(id_project, connection_id, token) → Promise<{ access_token, token_type, expires_at }>`; rifiuta con `err.status` (409/404/5xx, 502 su errore di rete) ed `err.code` preso dal body.
39
+ - `clearCache()` per i test. Campi d'istanza `httpUtils`, `now` sostituibili come in `IntegrationService`.
40
+
41
+ - [ ] **Step 1: Test che falliscono.** 200 → restituito e messo in cache (seconda chiamata senza HTTP); la cache scade 60 s prima di `expires_at` (stub di `now`); `expires_at: null` → nessuna cache; 409 → rifiuta con `status 409, code 'needs_reauth'`; errore di rete → 502.
42
+ - [ ] **Step 2: Eseguire** → fallisce.
43
+ - [ ] **Step 3: Implementare.**
44
+ - [ ] **Step 4: Eseguire** → passa.
45
+ - [ ] **Step 5: Commit** — `feat(ai-prompt): MCP OAuth token service`.
46
+
47
+ ### Task 2: iniezione del bearer in DirAiPrompt
48
+
49
+ **File:**
50
+ - Modificare: `tybotRoute/tiledeskChatbotPlugs/directives/DirAiPrompt.js`
51
+ - Test: `tybotRoute/test/dir_ai_prompt_headers_test.js` (nuovi casi), `tybotRoute/test/conversation-ai_prompt_test.js` (un caso end-to-end)
52
+
53
+ **Interfacce:**
54
+ - `resolveOAuthTokens(servers, mcp_integration) → Promise<{ tokens: { [serverName]: 'Bearer …' }, error?: string }>`: per ogni server dell'action, `getIntegrationServer`; se `integrationServer.oauth?.connection_id` → `McpOAuthTokenService.getToken(this.projectId, id, this.token)`. Le richieste vanno in parallelo; il primo errore valorizza `error`.
55
+ - `arrayToObject(arr, mcp_integration, flowVariables, oauthTokens = {})`: dopo `mergeHeadersWithVariables`, `if (oauthTokens[server.name]) cleanServer.headers.Authorization = oauthTokens[server.name]`.
56
+ - Chiamato nel ramo `action.servers` subito dopo `resolveNativeServerUrls`, prima di costruire `flowVariables`; il percorso d'errore ricalca `nativeUrlError`.
57
+ - La riga `winston.debug("DirAiPrompt json.servers: ", …)` logga una copia con `headers.Authorization` sostituito da `'[REDACTED]'`.
58
+
59
+ - [ ] **Step 1: Test che falliscono.** Server dell'integration con `oauth: { connection_id: 'c1' }` → `json.servers.<name>.headers.Authorization === 'Bearer t1'`; `Authorization: 'x'` nei customHeaders → sovrascritto; server senza `oauth` → service del token non chiamato (punto di review 1); server nativo senza `oauth` → service del token non chiamato e header delle variabili del flow invariati (punto di review 3); service del token 409 → `flowError` "needs to be reconnected" e viene eseguito il false intent; il log di debug non contiene `t1` (punto di review 2).
60
+ - [ ] **Step 2: Eseguire** → fallisce.
61
+ - [ ] **Step 3: Implementare.**
62
+ - [ ] **Step 4: Eseguire** i due file di test AI Prompt esistenti più quello del Task 1 → passa.
63
+ - [ ] **Step 5: Commit** — `feat(ai-prompt): send MCP OAuth bearer from the server token endpoint`.
@@ -0,0 +1,63 @@
1
+ # MCP OAuth2 — tiledesk-chatbot (tybot) Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** When an AI Prompt action uses an MCP server whose integration entry has `oauth.connection_id`, tybot fetches the access token from tiledesk-server and sends it as `Authorization: Bearer …` to the LLM backend with the other MCP headers.
6
+
7
+ **Architecture:** New `services/McpOAuthTokenService.js` (`GET /:id_project/mcp/oauth/connections/:id/token` with the chatbot JWT, short in-process cache) → `DirAiPrompt` resolves tokens after `enrichServersFromIntegration` and passes them to `arrayToObject`, which sets `headers.Authorization` last.
8
+
9
+ **Tech Stack:** Node, `utils/HttpUtils`, mocha + the existing AI Prompt test harness.
10
+
11
+ **Spec:** `docs/superpowers/specs/2026-11-10-mcp-oauth2-design.md` (§7 token endpoint, §8 what the server stores in the integration, §11 errors).
12
+
13
+ ## Global Constraints
14
+
15
+ - Branch: new `feature/mcp-oauth2` from `main` in tiledesk-chatbot. One commit per task. Never push.
16
+ - Depends on the server token endpoint, already implemented in tiledesk-server (commit `496a71cb`, verified end to end: spec §14). Tokens are always fetched from the server: tybot never reads `mcp_oauth_connections` from its database, even though `IntegrationService` reads integrations from it (tokens are encrypted and refresh lives on the server).
17
+ - Request: `GET API_ENDPOINT + "/" + id_project + "/mcp/oauth/connections/" + connection_id + "/token"`, header `Authorization: 'JWT ' + this.token` (same as `IntegrationService.getIntegrationFromApi`). 200 body `{ access_token, token_type, expires_at }`.
18
+ - Cache: in-process `Map` keyed `id_project:connection_id`, entry valid until `expires_at - 60 s`; no cache when `expires_at` is null; never logged.
19
+ - Failure (409 `needs_reauth`, 404, 5xx, network): the action fails like a missing MCP integration — `flowError` = `"MCP server <name> needs to be reconnected"` for 409, `"MCP server <name> OAuth token unavailable"` otherwise; then `falseIntent` if set, else `callback()`.
20
+ - `Authorization` from OAuth overrides an `Authorization` coming from `customHeaders` or flow variables.
21
+ - Tests: `npm test` subset for `tybotRoute/test/conversation-ai_prompt_test.js`, `tybotRoute/test/dir_ai_prompt_headers_test.js` and the new file; the server is mocked by the existing HTTP stub used in those tests.
22
+
23
+ ## Review Focus
24
+
25
+ 1. A server without `oauth` → no token request at all (no regression, no added latency).
26
+ 2. The token never appears in `winston`/`this.logger` output (`json.servers` is logged at debug: redact `headers.Authorization` there).
27
+ 3. Native servers never carry `oauth`: the server rejects it on native entries (400, spec §8). A native entry must not call the token service and keeps its flow-variable headers unchanged.
28
+
29
+ ---
30
+
31
+ ### Task 1: token service
32
+
33
+ **Files:**
34
+ - Create: `tybotRoute/services/McpOAuthTokenService.js`
35
+ - Test: `tybotRoute/test/mcp_oauth_token_service_test.js`
36
+
37
+ **Interfaces:**
38
+ - `getToken(id_project, connection_id, token) → Promise<{ access_token, token_type, expires_at }>`; rejects with `err.status` (409/404/5xx, 502 on network) and `err.code` from the body.
39
+ - `clearCache()` for tests. Instance fields `httpUtils`, `now` substitutable like `IntegrationService`.
40
+
41
+ - [ ] **Step 1: Failing tests.** 200 → returned and cached (second call no HTTP); cache expiry 60 s before `expires_at` (stub `now`); `expires_at: null` → no cache; 409 → rejects `status 409, code 'needs_reauth'`; network error → 502.
42
+ - [ ] **Step 2: Run** → fail.
43
+ - [ ] **Step 3: Implement.**
44
+ - [ ] **Step 4: Run** → pass.
45
+ - [ ] **Step 5: Commit** — `feat(ai-prompt): MCP OAuth token service`.
46
+
47
+ ### Task 2: DirAiPrompt bearer injection
48
+
49
+ **Files:**
50
+ - Modify: `tybotRoute/tiledeskChatbotPlugs/directives/DirAiPrompt.js`
51
+ - Test: `tybotRoute/test/dir_ai_prompt_headers_test.js` (new cases), `tybotRoute/test/conversation-ai_prompt_test.js` (one end-to-end case)
52
+
53
+ **Interfaces:**
54
+ - `resolveOAuthTokens(servers, mcp_integration) → Promise<{ tokens: { [serverName]: 'Bearer …' }, error?: string }>`: for each action server, `getIntegrationServer`; if `integrationServer.oauth?.connection_id` → `McpOAuthTokenService.getToken(this.projectId, id, this.token)`. Requests run in parallel; first failure sets `error`.
55
+ - `arrayToObject(arr, mcp_integration, flowVariables, oauthTokens = {})`: after `mergeHeadersWithVariables`, `if (oauthTokens[server.name]) cleanServer.headers.Authorization = oauthTokens[server.name]`.
56
+ - Called in the `action.servers` branch right after `resolveNativeServerUrls`, before building `flowVariables`; the error path mirrors `nativeUrlError`.
57
+ - The `winston.debug("DirAiPrompt json.servers: ", …)` line logs a copy with `headers.Authorization` replaced by `'[REDACTED]'`.
58
+
59
+ - [ ] **Step 1: Failing tests.** Integration server with `oauth: { connection_id: 'c1' }` → `json.servers.<name>.headers.Authorization === 'Bearer t1'`; customHeaders `Authorization: 'x'` → overridden; server without `oauth` → token service not called (Review Focus 1); native server without `oauth` → token service not called and flow-variable headers unchanged (Review Focus 3); token service 409 → `flowError` "needs to be reconnected" and the false intent runs; debug log does not contain `t1` (Review Focus 2).
60
+ - [ ] **Step 2: Run** → fail.
61
+ - [ ] **Step 3: Implement.**
62
+ - [ ] **Step 4: Run** the two existing AI Prompt test files plus Task 1's → pass.
63
+ - [ ] **Step 5: Commit** — `feat(ai-prompt): send MCP OAuth bearer from the server token endpoint`.
@@ -0,0 +1,207 @@
1
+ # MCP OAuth2 — Design
2
+
3
+ Data: 2026-11-10 · Stato: lato server implementato (commit f4fe91fe..3224546e) e verificato end to end contro il server di riferimento (§14); restano tybot, UI e job dello scheduler
4
+ Repo: tiledesk-server, tiledesk-chatbot (tybot). La UI di dashboard / design-studio e tiledesk-scheduler-service sono follow-up (§12).
5
+ Branch: `features-2611/oauth2-mcp` (tiledesk-server).
6
+ Server MCP di riferimento: [modelcontextprotocol/example-remote-server](https://github.com/modelcontextprotocol/example-remote-server) ([guida OAuth](https://github.com/modelcontextprotocol/example-remote-server/blob/main/docs/oauth-implementation.md)), specifica MCP Authorization `2025-06-18`.
7
+
8
+ ## 1. Obiettivo
9
+
10
+ Un admin di progetto può collegare un server MCP protetto da OAuth2. tiledesk-server fa da client OAuth: esegue il flusso authorization code + PKCE, salva i token cifrati, li rinfresca e fornisce al chatbot engine un access token valido ogni volta che un'action AI Prompt chiama quel server MCP. Tre modi per configurare il client OAuth:
11
+
12
+ - **mcp** — la specifica MCP Authorization: discovery a partire dall'URL del server MCP, Dynamic Client Registration. L'admin inserisce solo l'URL MCP.
13
+ - **google** — preset con gli endpoint di Google e un unico client OAuth Google di Tiledesk da env.
14
+ - **manual** — l'admin inserisce endpoint di autorizzazione e token, client id/secret e scope (qualsiasi provider OAuth2 senza discovery).
15
+
16
+ ## 2. Decisioni (utente, 2026-11-10)
17
+
18
+ 1. I token stanno nel database. Il chatbot chiama un endpoint del server che restituisce l'access token letto dal DB. Uno scheduler, in seguito, chiamerà un'API del server che rinfresca e salva i token nel DB.
19
+ 2. Strategia di refresh per ora: **lazy** — l'endpoint del token rinfresca quando l'access token sta per scadere. L'API di refresh per lo scheduler si crea ora; la registrazione del job nello scheduler è un follow-up.
20
+ 3. Google: un unico client OAuth Google Cloud di Tiledesk (`MCP_OAUTH_GOOGLE_CLIENT_ID` / `_SECRET`). Nessun client per progetto.
21
+ 4. Scope di questo giro: fasi server 1–3 (store + flusso + token a runtime) più l'API di refresh, e la modifica a tybot che usa l'endpoint del token.
22
+ 5. Il codice del client OAuth è scritto con axios. Nessuna dipendenza da `@modelcontextprotocol/sdk`.
23
+
24
+ ## 3. Stato attuale (verificato 2026-11-10)
25
+
26
+ - `services/mcpService.js` gestisce l'autenticazione tramite `customHeaders` o il vecchio `auth` (bearer / api-key / basic). `buildServerConfig` rifiuta un oggetto `oauth`: *"OAuth2 authentication is not supported yet"*.
27
+ - I server MCP sono salvati nell'integration `mcp` (`models/integrations.js`, `value.servers[]`). I server custom sono identificati da `url`, quelli nativi da `nativeId` (`findServerIndex`).
28
+ - `GET /:projectid/integration*` è in cache mongoose per `cacheUtil.longTTL` (`project:<id>:integrations`) e restituisce `customHeaders[].value` non mascherati (`maskIntegrationSecrets` maschera solo `apikey`, `api_key`, `token`, `secret`, `client_secret`, `password`). Quindi un token non deve essere scritto nell'integration.
29
+ - `DirAiPrompt.js` di tybot legge `/integration/name/mcp` con il token del bot a ogni esecuzione, trasforma i `customHeaders` in `headers` (`arrayToObject`, `enrichServersFromIntegration`) e li inoltra al backend LLM.
30
+ - `/:projectid/mcp` è montato con `passport basic|jwt`, `validtoken`, `roleChecker.hasRoleOrTypes('admin', ['bot','subscription'])`.
31
+ - Gli endpoint interni protetti da token seguono `/webhook/scheduled-start`: header `x-auth-token`, `crypto.timingSafeEqual` contro un token da env, 503 se non configurato.
32
+ - `TdCache` ha `setNX(key, value, ttlSeconds)`, `setJSON`, `getJSON`, `del`.
33
+ - `utils/logRedactor.js` oscura le chiavi `^(…|token|…|authorization|x-auth-token|…)$` (regex ancorata: `access_token` e `refresh_token` **non** sono coperti) e le stringhe `Bearer …`.
34
+ - `routes/urlPreview.js` ha solo un controllo degli IP privati basato su regex dell'hostname (nessuna risoluzione DNS).
35
+ - Disponibili Node 18.19, axios, nock, mocha + chai-http.
36
+
37
+ ## 4. Modello dati (tiledesk-server)
38
+
39
+ Nuova collection `mcp_oauth_connections` (`models/mcpOAuthConnection.js`):
40
+
41
+ ```
42
+ {
43
+ id_project: String (index),
44
+ mode: 'mcp' | 'google' | 'manual',
45
+ server_url: String, // URL del server MCP = risorsa OAuth (RFC 8707)
46
+ name: String, // nome visualizzato
47
+ status: 'pending' | 'active' | 'needs_reauth' | 'revoked',
48
+ // client OAuth
49
+ client_id: String,
50
+ client_secret_enc: String, // cifrato, assente per client pubblici / google (env)
51
+ client_registered_at: Date, // solo DCR
52
+ // endpoint (scoperti, preset o manuali)
53
+ resource: String, // modalità mcp: `resource` dei protected resource metadata (può essere l'origin, non server_url)
54
+ issuer: String,
55
+ authorization_endpoint: String,
56
+ token_endpoint: String,
57
+ revocation_endpoint: String,
58
+ registration_endpoint: String,
59
+ token_endpoint_auth_method: 'client_secret_basic' | 'client_secret_post' | 'none',
60
+ scopes: [String],
61
+ // token
62
+ access_token_enc: String,
63
+ refresh_token_enc: String,
64
+ token_type: String, // 'Bearer'
65
+ expires_at: Date, // null quando il provider non invia expires_in
66
+ granted_scopes: [String],
67
+ last_refresh_at: Date,
68
+ last_error: { code: String, message: String, at: Date },
69
+ created_by: String, // id dell'utente che ha avviato il flusso
70
+ createdAt, updatedAt
71
+ }
72
+ ```
73
+
74
+ Indice univoco `{ id_project: 1, server_url: 1 }`: una connessione per server MCP per progetto.
75
+
76
+ La voce del server nell'integration riceve solo un riferimento (nessun segreto):
77
+
78
+ ```
79
+ oauth: { connection_id: String, mode: 'mcp'|'google'|'manual' }
80
+ ```
81
+
82
+ Cifratura: AES-256-GCM, chiave `MCP_OAUTH_ENCRYPTION_KEY` (32 byte, base64). Formato salvato `v1:<iv b64>:<tag b64>:<ciphertext b64>`. Mai restituito da nessuna API tranne l'endpoint del token (§7), che restituisce solo l'access token.
83
+
84
+ ## 5. Configurazione
85
+
86
+ | Env | Scopo |
87
+ |---|---|
88
+ | `MCP_OAUTH_ENCRYPTION_KEY` | Obbligatoria. Senza, ogni rotta OAuth risponde 503 `"MCP OAuth is not available on this installation"`. |
89
+ | `MCP_OAUTH_REDIRECT_BASE_URL` | URL base pubblico del server; redirect URI = `<base>/oauth2/mcp/callback`. Se assente si usa `API_URL`. |
90
+ | `MCP_OAUTH_RETURN_ORIGINS` | Origin separate da virgola ammesse come `return_url` (dashboard / design-studio). |
91
+ | `MCP_OAUTH_GOOGLE_CLIENT_ID`, `MCP_OAUTH_GOOGLE_CLIENT_SECRET` | Preset Google. Senza, `mode: 'google'` risponde 503. |
92
+ | `MCP_OAUTH_REFRESH_TOKEN` | `x-auth-token` dell'API di refresh per lo scheduler. Senza, l'API risponde 503. |
93
+ | `MCP_OAUTH_ALLOW_INSECURE_URLS` | `true` solo in dev/test: consente `http://` e host privati/loopback (serve per il server di riferimento su `localhost:3232`). |
94
+
95
+ ## 6. Flusso di autorizzazione
96
+
97
+ 1. **Avvio** — `POST /:projectid/mcp/oauth/authorize` (admin), body `{ server_url, name?, mode, scopes?, return_url?, manual?: { authorization_endpoint, token_endpoint, revocation_endpoint?, client_id, client_secret?, token_endpoint_auth_method? } }`.
98
+ - `mcp`: discovery (§6.1), poi DCR se per quel server non c'è un client salvato (§6.2).
99
+ - `google`: endpoint `https://accounts.google.com/o/oauth2/v2/auth`, `https://oauth2.googleapis.com/token`, `https://oauth2.googleapis.com/revoke`; parametri extra `access_type=offline`, `prompt=consent`, `include_granted_scopes=true`; `scopes` obbligatori.
100
+ - `manual`: endpoint e client dal body (client secret cifrato).
101
+ - Crea o aggiorna la connessione (`status: 'pending'` se nuova; una connessione `active` resta utilizzabile finché il nuovo flusso non si completa).
102
+ - Genera `state` (32 byte casuali, base64url) e `code_verifier` (64 caratteri), `code_challenge = base64url(sha256(verifier))`, metodo `S256`.
103
+ - Salva `mcp_oauth:state:<state>` → `{ id_project, connection_id, code_verifier, return_url, user_id }` su Redis, TTL 600 s.
104
+ - Restituisce `{ authorization_url, connection_id }`. Parametri dell'URL: `response_type=code`, `client_id`, `redirect_uri`, `code_challenge`, `code_challenge_method=S256`, `state`, `scope` (separati da spazio, se presenti), `resource=<server_url>` (modalità mcp).
105
+ 2. **Callback** — `GET /oauth2/mcp/callback?code&state` (pubblica, montata prima dei router `/:projectid`).
106
+ - Legge e **cancella** lo state (monouso). Mancante/scaduto → pagina 400.
107
+ - Parametro `error` dal provider → `last_error` sulla connessione, redirect a `return_url?mcp_oauth=error&error=<code>`.
108
+ - Scambia il code sul token endpoint (`grant_type=authorization_code`, `code`, `redirect_uri`, `code_verifier`, `resource` in modalità mcp, autenticazione del client secondo `token_endpoint_auth_method`).
109
+ - Salva i token cifrati, `expires_at = now + expires_in`, `status: 'active'`, azzera `last_error`.
110
+ - Redirect a `return_url?mcp_oauth=success&connection_id=<id>`; senza `return_url`, una pagina HTML minima "Connesso, puoi chiudere questa finestra".
111
+ 3. **Uso** — l'admin salva poi il server MCP con `oauth.connection_id` (`POST /:projectid/mcp/servers`); `mcpService` usa il token per connect / discovery dei tool (§8).
112
+
113
+ ### 6.1 Discovery (modalità `mcp`)
114
+
115
+ 1. `POST <server_url>` con una richiesta `initialize` senza autenticazione. Ci si aspetta 401 con `WWW-Authenticate: Bearer resource_metadata="<url>"`.
116
+ 2. Protected Resource Metadata (RFC 9728): l'URL `resource_metadata`, altrimenti `<origin>/.well-known/oauth-protected-resource<path>`, altrimenti `<origin>/.well-known/oauth-protected-resource`. Si prende `authorization_servers[0]` e `scopes_supported` come scope di default se non ne vengono indicati.
117
+ 3. Authorization Server Metadata: `<issuer>/.well-known/oauth-authorization-server` (RFC 8414, con inserimento del path se l'issuer ha un path), altrimenti `/.well-known/openid-configuration`. Richiede `authorization_endpoint`, `token_endpoint`; `code_challenge_methods_supported`, se presente, deve includere `S256`.
118
+ 4. Se il server risponde 200 all'initialize non autenticato, la discovery fallisce con 400 `"This MCP server does not require OAuth"`.
119
+
120
+ ### 6.2 Dynamic Client Registration (modalità `mcp`)
121
+
122
+ - `POST registration_endpoint` con `{ client_name: 'Tiledesk', redirect_uris: [<redirect uri>], grant_types: ['authorization_code','refresh_token'], response_types: ['code'], token_endpoint_auth_method: 'client_secret_post' }`.
123
+ - Si salvano `client_id`, `client_secret` (cifrato), il `token_endpoint_auth_method` restituito, `client_registered_at`.
124
+ - Nessun `registration_endpoint` → 400 `"This MCP server does not support dynamic client registration; use mode 'manual'"`.
125
+ - Il client registrato viene riusato dai flussi successivi della stessa connessione. Una risposta `invalid_client` dal token endpoint elimina il client salvato, così il prossimo authorize si registra di nuovo (il server di riferimento fa scadere i client dopo 30 giorni).
126
+
127
+ ## 7. Token a runtime e refresh
128
+
129
+ `oauthTokenService.getAccessToken(connection)`:
130
+
131
+ - `status !== 'active'` → errore `needs_reauth` (409).
132
+ - `expires_at` assente o a più di `REFRESH_WINDOW_S = 300` s → decifra e restituisce.
133
+ - Altrimenti refresh:
134
+ - Lock `mcp_oauth:refresh_lock:<id>` con `setNX`, TTL 30 s.
135
+ - Lock non acquisito → interroga il DB ogni 250 ms per al massimo 5 s cercando un `last_refresh_at` più recente; poi restituisce il token se ancora valido, altrimenti 503.
136
+ - Lock acquisito → ricarica la connessione, `grant_type=refresh_token` (+ `resource` in modalità mcp). Salva il nuovo access token; salva il nuovo refresh token **solo se ne arriva uno** (Google non li ruota mai, il server di riferimento sì). Rilascia il lock.
137
+ - `invalid_grant` → `status: 'needs_reauth'`, `last_error`, 409. `invalid_client` → elimina anche il client DCR (§6.2). Errore di rete/5xx → `last_error`, 503, stato invariato.
138
+ - Nessun refresh token e access token scaduto → `needs_reauth`.
139
+
140
+ Endpoint:
141
+
142
+ - `GET /:projectid/mcp/oauth/connections/:id/token` — **solo chiamanti bot o subscription** (un JWT admin riceve 403, così i token non arrivano mai alla dashboard). 200 `{ access_token, token_type, expires_at }`.
143
+ - `POST /:projectid/mcp/oauth/connections/:id/refresh` — admin; forza un refresh, restituisce la connessione pubblica.
144
+ - `POST /webhook/mcp-oauth/refresh` — scheduler; header `x-auth-token: $MCP_OAUTH_REFRESH_TOKEN`. Body `{ connection_id }` ne rinfresca una e risponde 200 `{ success, connection_id, expires_at }` (404 se non esiste, 409 `needs_reauth`); `{ within_seconds }` (default 900, max 86400) rinfresca ogni connessione `active` con refresh token e `expires_at` entro quella finestra, in sequenza, isolando ogni errore. 200 `{ success, refreshed, failed, needs_reauth }`; 400 se `within_seconds` non è un intero tra 60 e 86400. 401 token errato, 503 non configurato.
145
+
146
+ ## 8. Modifiche a mcpService
147
+
148
+ - `serverConfig` riceve `oauth: { connection_id }`. `applyRequestHeaders` diventa compatibile con l'asincrono tramite un nuovo `resolveRequestHeaders(serverConfig)`: quando `oauth.connection_id` è valorizzato carica la connessione (stesso `id_project`), ottiene il token e imposta `Authorization: Bearer <token>`, sovrascrivendo un eventuale `Authorization` dei `customHeaders`.
149
+ - `sendJSONRPCRequest`: un 401 da un server con OAuth provoca un refresh forzato e un solo nuovo tentativo.
150
+ - `buildServerConfig` accetta `oauth.connection_id` (rifiuta la vecchia forma di `oauth` con il messaggio attuale). `normalizeMcpServerEntry` copia `oauth: { connection_id, mode }` nella voce salvata dopo aver verificato che la connessione appartenga al progetto.
151
+ - `getCacheKey` include l'id della connessione, così le sessioni non vengono condivise tra connessioni.
152
+ - Deciso durante lo sviluppo: il bearer viene inviato solo se il `server_url` della connessione coincide con l'URL chiamato (normalizzato), altrimenti 400. Senza questo controllo un admin potrebbe collegare la connessione del server A al server B e consegnare a B il token di A.
153
+ - Deciso durante lo sviluppo: le voci native con `oauth` rispondono 400. L'URL dei server nativi non viene mai esposto, quindi l'admin non può lanciare `authorize` su di essi; per supportarli `authorize` deve accettare un `nativeId` (§12).
154
+ - Gli errori OAuth mantengono il loro status attraverso `sendJSONRPCRequest` e `routes/mcp.js` (404, 409, 503) invece del 500 generico.
155
+
156
+ ## 9. API di gestione delle connessioni (admin)
157
+
158
+ - `GET /:projectid/mcp/oauth/connections` — forma pubblica: `{ _id, mode, server_url, name, status, scopes, granted_scopes, expires_at, last_refresh_at, last_error, issuer, createdAt, updatedAt }`. Nessun segreto, nessun `client_id` per google.
159
+ - `GET /:projectid/mcp/oauth/connections/:id` — stessa forma.
160
+ - `DELETE /:projectid/mcp/oauth/connections/:id` — revoca best-effort (refresh token, poi access token) su `revocation_endpoint`, cancella la connessione, rimuove `oauth` dalle voci dell'integration che la referenziano (le voci restano, senza autenticazione) ed emette `integration.update`. 200 `{ success: true, revoked: Boolean }`.
161
+
162
+ ## 10. Sicurezza
163
+
164
+ - PKCE S256 sempre; `state` monouso, TTL 10 minuti, legato a progetto e utente.
165
+ - L'origin di `return_url` deve essere in `MCP_OAUTH_RETURN_ORIGINS`, altrimenti 400 all'authorize.
166
+ - Guard sugli URL in uscita (`utils/outboundUrlGuard.js`) per ogni URL influenzato dall'utente (URL del server, metadati, endpoint da discovery o manuali): solo `https:`, gli indirizzi risolti via DNS non devono essere loopback, privati, link-local, CGNAT, ULA o non specificati; `maxRedirects: 0`; timeout 10 s; body delle risposte ≤ 1 MB. `MCP_OAUTH_ALLOW_INSECURE_URLS=true` lo disattiva.
167
+ - Segreti cifrati a riposo; `logRedactor` aggiunge `access_token`, `refresh_token`, `code_verifier`, `code`, `client_secret`; `maskIntegrationSecrets` invariato (nessun segreto nell'integration).
168
+ - Endpoint del token raggiungibile solo con credenziali bot/subscription dello stesso progetto.
169
+
170
+ ## 11. Errori (riepilogo)
171
+
172
+ | Caso | Stato |
173
+ |---|---|
174
+ | Env della feature mancante | 503 |
175
+ | Body errato / discovery o DCR non supportati / return_url non ammesso | 400 |
176
+ | Connessione di un altro progetto o inesistente | 404 |
177
+ | Admin che chiama l'endpoint del token | 403 |
178
+ | Connessione `needs_reauth` / `revoked` / `pending` sul token | 409 `{ success: false, error: <messaggio>, code: 'needs_reauth' }` |
179
+ | Connessione OAuth usata per un altro URL di server | 400 |
180
+ | Provider irraggiungibile o 5xx | 503 (502 su authorize / callback) |
181
+
182
+ ## 12. Follow-up (non in questo giro)
183
+
184
+ - **Job dello scheduler**: un cron globale (ogni 10–15 minuti) in tiledesk-scheduler-service che chiama `POST /webhook/mcp-oauth/refresh` con `{ within_seconds: 900 }`. Serve un nuovo target (o un target HTTP generico) nello scheduler; contratto non verificato (repo non accessibile da qui). Mantiene vivi i refresh token dei provider con scadenza per inattività breve (il server di riferimento: 7 giorni) e segnala presto i `needs_reauth`.
185
+ - **UI** (dashboard / design-studio): pulsante "Connetti" che apre `authorization_url` in un popup, legge `mcp_oauth=success|error` sul `return_url`, mostra lo stato e "Riconnetti".
186
+ - **Verifica Google**: gli scope sensibili/ristretti (Gmail, Drive) richiedono la verifica dell'app da parte di Google (ristretti: assessment CASA); con stato di pubblicazione "Testing" i refresh token scadono dopo 7 giorni.
187
+ - Server MCP nativi con OAuth: `authorize` che accetta un `nativeId` (l'URL resta nascosto), poi voci del catalogo con modalità `oauth` e scope di default.
188
+
189
+ ## 13. Test
190
+
191
+ - Unitari: round-trip della cifratura e rilevamento delle manomissioni; challenge PKCE; costruzione degli URL di discovery (con e senza path); guard sugli URL.
192
+ - Test delle rotte con nock per discovery, DCR, token, refresh, revoca; Redis per state/lock.
193
+ - Concorrenza: due richieste del token in parallelo su un token in scadenza → esattamente una chiamata di refresh.
194
+ - End-to-end manuale contro il server di riferimento (`npm run dev:internal`, porta 3232) con `MCP_OAUTH_ALLOW_INSECURE_URLS=true`.
195
+
196
+ ## 14. Verifica (2026-11-10)
197
+
198
+ - Automatica: 128 test mocha (`test/mcpOAuthUtils`, `mcpOAuthDiscovery`, `mcpOAuthService`, `mcpOAuthRoute`, `mcpServiceOAuth`, `nativemcp`), eseguiti con Node 22 e con i moduli enterprise tolti da `node_modules`. Provider e server MCP sono simulati con nock; Mongo e Redis sono reali (locali).
199
+ - End to end manuale contro il server di riferimento (`npm run dev:internal`, porta 3232, auth in modalità interna) e tiledesk-server sulla porta 3000 con `MCP_OAUTH_ALLOW_INSECURE_URLS=true` e un database dedicato. Niente di simulato; la pagina di consenso dell'identity provider fittizio del server di riferimento è stata guidata con curl. Risultati:
200
+ - `authorize` in modalità `mcp`: discovery, registrazione dinamica (`token_endpoint_auth_method: none`, nessun secret) e parametro `resource` `http://localhost:3232/`.
201
+ - Callback: 302 al `return_url` con `mcp_oauth=success`; la connessione è `active` e la forma pubblica non contiene segreti.
202
+ - `POST /mcp/connect` e `/mcp/servers` con `oauth.connection_id`: 9 tool; la voce salvata contiene `{ connection_id, mode: 'mcp' }`.
203
+ - `/token`: admin 403, JWT del bot 200. Con `expires_at` spostato nel passato la chiamata successiva ha fatto il refresh e restituito un nuovo access token, con cui i tool sono stati elencati. Il server di riferimento ruota i refresh token e tre refresh di seguito sono andati a buon fine.
204
+ - Webhook dello scheduler: token errato 401; `{ connection_id }` 200; `{ within_seconds: 900 }` con una connessione in scadenza → `refreshed: 1`.
205
+ - A riposo: `access_token_enc` / `refresh_token_enc` nel formato `v1:`, il token in chiaro assente dal documento.
206
+ - `DELETE`: `revoked: true`, la voce dell'integration resta senza `oauth`, le chiamate successive con il vecchio `connection_id` rispondono 404.
207
+ - Non verificato: Google con un client reale (servono credenziali Google Cloud), il percorso 401 e nuovo tentativo contro un server reale (coperto solo con nock), il lato scheduler.
@@ -0,0 +1,207 @@
1
+ # MCP OAuth2 — Design
2
+
3
+ Date: 2026-11-10 · Status: server side implemented (commits f4fe91fe..3224546e) and verified end to end against the reference server (§14); tybot, UI and scheduler job pending
4
+ Repos: tiledesk-server, tiledesk-chatbot (tybot). Dashboard / design-studio UI and tiledesk-scheduler-service are follow-ups (§12).
5
+ Branch: `features-2611/oauth2-mcp` (tiledesk-server).
6
+ Reference MCP server: [modelcontextprotocol/example-remote-server](https://github.com/modelcontextprotocol/example-remote-server) ([OAuth guide](https://github.com/modelcontextprotocol/example-remote-server/blob/main/docs/oauth-implementation.md)), MCP Authorization spec `2025-06-18`.
7
+
8
+ ## 1. Goal
9
+
10
+ A project admin can connect an MCP server that is protected by OAuth2. tiledesk-server acts as the OAuth client: it runs the authorization code + PKCE flow, stores the tokens encrypted, refreshes them, and gives the chatbot engine a valid access token whenever an AI Prompt action calls that MCP server. Three ways to configure the OAuth client:
11
+
12
+ - **mcp** — the MCP Authorization spec: discovery from the MCP server URL, Dynamic Client Registration. The admin types only the MCP URL.
13
+ - **google** — preset with Google's endpoints and one Tiledesk-owned Google OAuth client from env.
14
+ - **manual** — the admin enters authorization/token endpoints, client id/secret and scopes (any OAuth2 provider without discovery).
15
+
16
+ ## 2. Decisions (user, 2026-11-10)
17
+
18
+ 1. Tokens live in the database. The chatbot calls a server endpoint that returns the access token read from the DB. A scheduler, later, calls a server API that refreshes and saves tokens in the DB.
19
+ 2. Refresh strategy for now: **lazy** — the token endpoint refreshes when the access token is about to expire. The scheduler-facing refresh API is built now; registering the scheduler job is a follow-up.
20
+ 3. Google: one Tiledesk Google Cloud OAuth client (`MCP_OAUTH_GOOGLE_CLIENT_ID` / `_SECRET`). No per-project client.
21
+ 4. Scope of this round: server phases 1–3 (store + flow + runtime token) plus the refresh API, and the tybot change that consumes the token endpoint.
22
+ 5. OAuth client code is written with axios. No `@modelcontextprotocol/sdk` dependency.
23
+
24
+ ## 3. Current state (verified 2026-11-10)
25
+
26
+ - `services/mcpService.js` handles auth via `customHeaders` or legacy `auth` (bearer / api-key / basic). `buildServerConfig` rejects an `oauth` object: *"OAuth2 authentication is not supported yet"*.
27
+ - MCP servers are stored in the `mcp` integration (`models/integrations.js`, `value.servers[]`). Custom servers are keyed by `url`, native ones by `nativeId` (`findServerIndex`).
28
+ - `GET /:projectid/integration*` is mongoose-cached for `cacheUtil.longTTL` (`project:<id>:integrations`) and returns `customHeaders[].value` unmasked (`maskIntegrationSecrets` masks only `apikey`, `api_key`, `token`, `secret`, `client_secret`, `password`). So a token must not be written into the integration.
29
+ - tybot `DirAiPrompt.js` reads `/integration/name/mcp` with the bot token on every run, turns `customHeaders` into `headers` (`arrayToObject`, `enrichServersFromIntegration`) and forwards them to the LLM backend.
30
+ - `/:projectid/mcp` is mounted with `passport basic|jwt`, `validtoken`, `roleChecker.hasRoleOrTypes('admin', ['bot','subscription'])`.
31
+ - Internal token-protected endpoints follow `/webhook/scheduled-start`: header `x-auth-token`, `crypto.timingSafeEqual` against an env token, 503 when not configured.
32
+ - `TdCache` has `setNX(key, value, ttlSeconds)`, `setJSON`, `getJSON`, `del`.
33
+ - `utils/logRedactor.js` redacts keys `^(…|token|…|authorization|x-auth-token|…)$` (anchored: `access_token`, `refresh_token` are **not** covered) and `Bearer …` strings.
34
+ - `routes/urlPreview.js` has a hostname-regex private-IP check only (no DNS resolution).
35
+ - Node 18.19, axios, nock, mocha + chai-http available.
36
+
37
+ ## 4. Data model (tiledesk-server)
38
+
39
+ New collection `mcp_oauth_connections` (`models/mcpOAuthConnection.js`):
40
+
41
+ ```
42
+ {
43
+ id_project: String (index),
44
+ mode: 'mcp' | 'google' | 'manual',
45
+ server_url: String, // MCP server URL = OAuth resource (RFC 8707)
46
+ name: String, // display name
47
+ status: 'pending' | 'active' | 'needs_reauth' | 'revoked',
48
+ // OAuth client
49
+ client_id: String,
50
+ client_secret_enc: String, // encrypted, absent for public clients / google (env)
51
+ client_registered_at: Date, // DCR only
52
+ // endpoints (discovered, preset or manual)
53
+ resource: String, // mcp mode: `resource` of the protected resource metadata (may be the origin, not server_url)
54
+ issuer: String,
55
+ authorization_endpoint: String,
56
+ token_endpoint: String,
57
+ revocation_endpoint: String,
58
+ registration_endpoint: String,
59
+ token_endpoint_auth_method: 'client_secret_basic' | 'client_secret_post' | 'none',
60
+ scopes: [String],
61
+ // tokens
62
+ access_token_enc: String,
63
+ refresh_token_enc: String,
64
+ token_type: String, // 'Bearer'
65
+ expires_at: Date, // null when the provider sends no expires_in
66
+ granted_scopes: [String],
67
+ last_refresh_at: Date,
68
+ last_error: { code: String, message: String, at: Date },
69
+ created_by: String, // user id that started the flow
70
+ createdAt, updatedAt
71
+ }
72
+ ```
73
+
74
+ Index `{ id_project: 1, server_url: 1 }` unique: one connection per MCP server per project.
75
+
76
+ The integration server entry gains a reference only (no secrets):
77
+
78
+ ```
79
+ oauth: { connection_id: String, mode: 'mcp'|'google'|'manual' }
80
+ ```
81
+
82
+ Encryption: AES-256-GCM, key `MCP_OAUTH_ENCRYPTION_KEY` (32 bytes, base64). Stored format `v1:<iv b64>:<tag b64>:<ciphertext b64>`. Never returned by any API except the token endpoint (§7), which returns only the access token.
83
+
84
+ ## 5. Configuration
85
+
86
+ | Env | Purpose |
87
+ |---|---|
88
+ | `MCP_OAUTH_ENCRYPTION_KEY` | Required. Without it every OAuth route answers 503 `"MCP OAuth is not available on this installation"`. |
89
+ | `MCP_OAUTH_REDIRECT_BASE_URL` | Public base URL of the server; redirect URI = `<base>/oauth2/mcp/callback`. Falls back to `API_URL`. |
90
+ | `MCP_OAUTH_RETURN_ORIGINS` | Comma-separated origins allowed as `return_url` (dashboard / design-studio). |
91
+ | `MCP_OAUTH_GOOGLE_CLIENT_ID`, `MCP_OAUTH_GOOGLE_CLIENT_SECRET` | Google preset. Without them, `mode: 'google'` answers 503. |
92
+ | `MCP_OAUTH_REFRESH_TOKEN` | `x-auth-token` of the scheduler refresh API. Without it the API answers 503. |
93
+ | `MCP_OAUTH_ALLOW_INSECURE_URLS` | `true` only in dev/test: allows `http://` and private/loopback hosts (needed for the reference server on `localhost:3232`). |
94
+
95
+ ## 6. Authorization flow
96
+
97
+ 1. **Start** — `POST /:projectid/mcp/oauth/authorize` (admin), body `{ server_url, name?, mode, scopes?, return_url?, manual?: { authorization_endpoint, token_endpoint, revocation_endpoint?, client_id, client_secret?, token_endpoint_auth_method? } }`.
98
+ - `mcp`: discovery (§6.1), then DCR if no stored client for that server (§6.2).
99
+ - `google`: endpoints `https://accounts.google.com/o/oauth2/v2/auth`, `https://oauth2.googleapis.com/token`, `https://oauth2.googleapis.com/revoke`; extra params `access_type=offline`, `prompt=consent`, `include_granted_scopes=true`; `scopes` required.
100
+ - `manual`: endpoints and client from the body (client secret encrypted).
101
+ - Upserts the connection (`status: 'pending'` when new; an `active` connection stays usable until the new flow completes).
102
+ - Generates `state` (32 random bytes, base64url) and `code_verifier` (64 chars), `code_challenge = base64url(sha256(verifier))`, method `S256`.
103
+ - Stores `mcp_oauth:state:<state>` → `{ id_project, connection_id, code_verifier, return_url, user_id }` in Redis, TTL 600 s.
104
+ - Returns `{ authorization_url, connection_id }`. URL params: `response_type=code`, `client_id`, `redirect_uri`, `code_challenge`, `code_challenge_method=S256`, `state`, `scope` (space-joined, when any), `resource=<server_url>` (mcp mode).
105
+ 2. **Callback** — `GET /oauth2/mcp/callback?code&state` (public, mounted before the `/:projectid` routers).
106
+ - Reads and **deletes** the state (single use). Missing/expired → 400 page.
107
+ - `error` param from the provider → connection `last_error`, redirect `return_url?mcp_oauth=error&error=<code>`.
108
+ - Exchanges the code at the token endpoint (`grant_type=authorization_code`, `code`, `redirect_uri`, `code_verifier`, `resource` in mcp mode, client auth per `token_endpoint_auth_method`).
109
+ - Saves tokens encrypted, `expires_at = now + expires_in`, `status: 'active'`, clears `last_error`.
110
+ - Redirects to `return_url?mcp_oauth=success&connection_id=<id>`; without `return_url`, a minimal HTML page "Connected, you can close this window".
111
+ 3. **Use** — the admin then saves the MCP server with `oauth.connection_id` (`POST /:projectid/mcp/servers`); `mcpService` uses the token for connect / tools discovery (§8).
112
+
113
+ ### 6.1 Discovery (mode `mcp`)
114
+
115
+ 1. `POST <server_url>` with an `initialize` request and no auth. Expect 401 with `WWW-Authenticate: Bearer resource_metadata="<url>"`.
116
+ 2. Protected Resource Metadata (RFC 9728): the `resource_metadata` URL, else `<origin>/.well-known/oauth-protected-resource<path>`, else `<origin>/.well-known/oauth-protected-resource`. Take `authorization_servers[0]`, and `scopes_supported` as default scopes when none are given.
117
+ 3. Authorization Server Metadata: `<issuer>/.well-known/oauth-authorization-server` (RFC 8414, with path insertion when the issuer has a path), else `/.well-known/openid-configuration`. Requires `authorization_endpoint`, `token_endpoint`; `code_challenge_methods_supported` must include `S256` when present.
118
+ 4. If the server answers 200 to the unauthenticated initialize, discovery fails with 400 `"This MCP server does not require OAuth"`.
119
+
120
+ ### 6.2 Dynamic Client Registration (mode `mcp`)
121
+
122
+ - `POST registration_endpoint` with `{ client_name: 'Tiledesk', redirect_uris: [<redirect uri>], grant_types: ['authorization_code','refresh_token'], response_types: ['code'], token_endpoint_auth_method: 'client_secret_post' }`.
123
+ - Store `client_id`, `client_secret` (encrypted), the returned `token_endpoint_auth_method`, `client_registered_at`.
124
+ - No `registration_endpoint` → 400 `"This MCP server does not support dynamic client registration; use mode 'manual'"`.
125
+ - The registered client is reused by later flows of the same connection. A token endpoint answer `invalid_client` drops the stored client so the next authorize registers again (the reference server expires clients after 30 days).
126
+
127
+ ## 7. Runtime token and refresh
128
+
129
+ `oauthTokenService.getAccessToken(connection)`:
130
+
131
+ - `status !== 'active'` → error `needs_reauth` (409).
132
+ - `expires_at` absent or more than `REFRESH_WINDOW_S = 300` s away → decrypt and return.
133
+ - Otherwise refresh:
134
+ - Lock `mcp_oauth:refresh_lock:<id>` with `setNX`, TTL 30 s.
135
+ - Not acquired → poll the DB every 250 ms for up to 5 s for a newer `last_refresh_at`; then return the token if still valid, else 503.
136
+ - Acquired → reload the connection, `grant_type=refresh_token` (+ `resource` in mcp mode). Save the new access token; save the new refresh token **only when one is returned** (Google never rotates, the reference server does). Release the lock.
137
+ - `invalid_grant` → `status: 'needs_reauth'`, `last_error`, 409. `invalid_client` → also drop the DCR client (§6.2). Network/5xx → `last_error`, 503, status unchanged.
138
+ - No refresh token and the access token expired → `needs_reauth`.
139
+
140
+ Endpoints:
141
+
142
+ - `GET /:projectid/mcp/oauth/connections/:id/token` — **bot or subscription callers only** (an admin JWT gets 403, so tokens never reach the dashboard). 200 `{ access_token, token_type, expires_at }`.
143
+ - `POST /:projectid/mcp/oauth/connections/:id/refresh` — admin; forces a refresh, returns the public connection.
144
+ - `POST /webhook/mcp-oauth/refresh` — scheduler; header `x-auth-token: $MCP_OAUTH_REFRESH_TOKEN`. Body `{ connection_id }` refreshes one and answers 200 `{ success, connection_id, expires_at }` (404 unknown, 409 `needs_reauth`); `{ within_seconds }` (default 900, max 86400) refreshes every `active` connection with a refresh token and `expires_at` within that window, sequentially, each failure isolated. 200 `{ success, refreshed, failed, needs_reauth }`; 400 when `within_seconds` is not an integer in 60..86400. 401 bad token, 503 not configured.
145
+
146
+ ## 8. mcpService changes
147
+
148
+ - `serverConfig` gains `oauth: { connection_id }`. `applyRequestHeaders` becomes async-aware through a new `resolveRequestHeaders(serverConfig)`: when `oauth.connection_id` is set it loads the connection (same `id_project`), gets the token and sets `Authorization: Bearer <token>`, overriding any `Authorization` from `customHeaders`.
149
+ - `sendJSONRPCRequest`: a 401 from an OAuth-backed server triggers one forced refresh and one retry.
150
+ - `buildServerConfig` accepts `oauth.connection_id` (rejects the old `oauth` shape with the current message). `normalizeMcpServerEntry` copies `oauth: { connection_id, mode }` into the stored entry after checking the connection belongs to the project.
151
+ - `getCacheKey` includes the connection id so sessions are not shared across connections.
152
+ - Decided during implementation: the bearer is sent only when the connection's `server_url` equals the requested URL (normalized), else 400. Without it an admin could point a connection of server A at server B and hand B the token of A.
153
+ - Decided during implementation: native entries with `oauth` answer 400. Native URLs are never exposed, so the admin cannot run `authorize` on them; supporting them needs `authorize` to accept a `nativeId` (§12).
154
+ - OAuth errors keep their status through `sendJSONRPCRequest` and `routes/mcp.js` (404, 409, 503) instead of the generic 500.
155
+
156
+ ## 9. Connection management API (admin)
157
+
158
+ - `GET /:projectid/mcp/oauth/connections` — public shape: `{ _id, mode, server_url, name, status, scopes, granted_scopes, expires_at, last_refresh_at, last_error, issuer, createdAt, updatedAt }`. No secrets, no `client_id` for google.
159
+ - `GET /:projectid/mcp/oauth/connections/:id` — same shape.
160
+ - `DELETE /:projectid/mcp/oauth/connections/:id` — best-effort revocation (refresh token, then access token) at `revocation_endpoint`, delete the connection, remove `oauth` from integration entries that reference it (they stay, unauthenticated) and emit `integration.update`. 200 `{ success: true, revoked: Boolean }`.
161
+
162
+ ## 10. Security
163
+
164
+ - PKCE S256 always; `state` single-use, 10-min TTL, bound to project and user.
165
+ - `return_url` origin must be in `MCP_OAUTH_RETURN_ORIGINS`, else 400 at authorize.
166
+ - Outbound URL guard (`utils/outboundUrlGuard.js`) for every user-influenced URL (server URL, metadata, endpoints from discovery or manual): `https:` only, DNS-resolved addresses must not be loopback, private, link-local, CGNAT, ULA or unspecified; `maxRedirects: 0`; 10 s timeout; response bodies ≤ 1 MB. `MCP_OAUTH_ALLOW_INSECURE_URLS=true` disables it.
167
+ - Secrets encrypted at rest; `logRedactor` gains `access_token`, `refresh_token`, `code_verifier`, `code`, `client_secret`; `maskIntegrationSecrets` unchanged (no secrets in the integration).
168
+ - Token endpoint reachable only with bot/subscription credentials of the same project.
169
+
170
+ ## 11. Errors (summary)
171
+
172
+ | Case | Status |
173
+ |---|---|
174
+ | Feature env missing | 503 |
175
+ | Bad body / discovery or DCR not supported / return_url not allowed | 400 |
176
+ | Connection of another project or missing | 404 |
177
+ | Admin calling the token endpoint | 403 |
178
+ | Connection `needs_reauth` / `revoked` / `pending` on token | 409 `{ success: false, error: <message>, code: 'needs_reauth' }` |
179
+ | OAuth connection used for another server URL | 400 |
180
+ | Provider unreachable or 5xx | 503 (502 at authorize / callback) |
181
+
182
+ ## 12. Follow-ups (not in this round)
183
+
184
+ - **Scheduler job**: one global cron (every 10–15 min) in tiledesk-scheduler-service calling `POST /webhook/mcp-oauth/refresh` with `{ within_seconds: 900 }`. Needs a new target (or a generic HTTP target) in the scheduler; contract not verified (repo not accessible from here). Keeps refresh tokens alive for providers with short idle expiry (the reference server: 7 days) and flags `needs_reauth` early.
185
+ - **UI** (dashboard / design-studio): "Connect" button opening `authorization_url` in a popup, reading `mcp_oauth=success|error` on `return_url`, showing status and "Reconnect".
186
+ - **Google verification**: sensitive/restricted scopes (Gmail, Drive) need Google app verification (restricted: CASA assessment); in "Testing" publishing status refresh tokens expire after 7 days.
187
+ - Native MCP servers with OAuth: `authorize` accepting a `nativeId` (the URL stays hidden), then catalog entries with a default `oauth` mode/scopes.
188
+
189
+ ## 13. Testing
190
+
191
+ - Unit: crypto round-trip and tamper detection; PKCE challenge; discovery URL construction (with and without path); URL guard.
192
+ - Route tests with nock for discovery, DCR, token, refresh, revoke; Redis for state/lock.
193
+ - Concurrency: two parallel token requests on an expiring token → exactly one refresh call.
194
+ - Manual end-to-end against the reference server (`npm run dev:internal`, port 3232) with `MCP_OAUTH_ALLOW_INSECURE_URLS=true`.
195
+
196
+ ## 14. Verification (2026-11-10)
197
+
198
+ - Automated: 128 mocha tests (`test/mcpOAuthUtils`, `mcpOAuthDiscovery`, `mcpOAuthService`, `mcpOAuthRoute`, `mcpServiceOAuth`, `nativemcp`), run on Node 22 with the enterprise modules removed from `node_modules`. Providers and MCP servers are mocked with nock; Mongo and Redis are real (local).
199
+ - Manual end to end against the reference server (`npm run dev:internal`, port 3232, internal auth mode) and tiledesk-server on port 3000 with `MCP_OAUTH_ALLOW_INSECURE_URLS=true` and a dedicated database. Nothing mocked; the consent page of the reference server's mock identity provider was driven with curl. Results:
200
+ - `authorize` in mode `mcp`: discovery, dynamic registration (`token_endpoint_auth_method: none`, no secret) and the `resource` parameter `http://localhost:3232/`.
201
+ - Callback: 302 to `return_url` with `mcp_oauth=success`; the connection is `active` and the public shape has no secrets.
202
+ - `POST /mcp/connect` and `/mcp/servers` with `oauth.connection_id`: 9 tools; the saved entry holds `{ connection_id, mode: 'mcp' }`.
203
+ - `/token`: admin 403, bot JWT 200. With `expires_at` moved to the past the next call refreshed and returned a new access token, and tools were listed with it. The reference server rotates refresh tokens and three refreshes in a row succeeded.
204
+ - Scheduler webhook: wrong token 401; `{ connection_id }` 200; `{ within_seconds: 900 }` with one connection due → `refreshed: 1`.
205
+ - At rest: `access_token_enc` / `refresh_token_enc` in `v1:` format, the plaintext token absent from the document.
206
+ - `DELETE`: `revoked: true`, the integration entry kept without `oauth`, later calls with the old `connection_id` answer 404.
207
+ - Not verified: Google with a real client (needs Google Cloud credentials), the 401-then-retry path against a live server (covered only by nock), the scheduler side.
package/jobs.js CHANGED
@@ -120,6 +120,19 @@ function startJobListeners(tdCache) {
120
120
  jobsManager.listenMultiWorker(require('@tiledesk/tiledesk-multi-worker'));
121
121
  }
122
122
 
123
+ /**
124
+ * Conversations closed here (the scheduler's unresponsive-bot close, say) emit request.close in this process only:
125
+ * without the publisher their executions never end. Publish only: the audit consumer runs on the API pods.
126
+ * Off (no listeners) when EXECUTIONS_AMQP_URL is unset.
127
+ */
128
+ function startExecutionsPublisher() {
129
+ try {
130
+ require('./pubmodules/executions-publisher').listen({ audit: false });
131
+ } catch (err) {
132
+ winston.info("Jobs error initializing executionsPublisher module", err);
133
+ }
134
+ }
135
+
123
136
  function startAnalyticsPublisher() {
124
137
  try {
125
138
  require('./pubmodules/analytics-publisher').listen();
@@ -150,6 +163,8 @@ async function main() {
150
163
 
151
164
  startAnalyticsPublisher();
152
165
 
166
+ startExecutionsPublisher();
167
+
153
168
  winston.info("Jobs started");
154
169
 
155
170
  // Keeps the process alive.