guniweb-sap-mcp 0.2.1 → 0.3.0

package/README.md

@@ -36,7 +36,7 @@ graph LR
 ### Tool Visibility (v1.5)
 - **3-Tier Tool Registry** -- Core / OData / IDoc tiers with on-demand activation
 - **`sap_enable_tools`** -- LLMs activate additional tool tiers at runtime
-- **`--read-only` Mode** -- Hide all write/destructive tools
+- **Read-only by default (since 0.2.2)** -- write tools require `--allow-write`
 - **IDoc Auto-Disable** -- IDoc tools hidden when no IDoc config present
 - **MCP Annotations** -- readOnlyHint, destructiveHint, idempotentHint on all 17 tools
 
@@ -67,7 +67,7 @@ graph TB
 - **Three Transport Modes** -- stdio, Streamable HTTP, SSE
 - **API Key Protection** -- Secure HTTP transport with Bearer token
 - **Docker Ready** -- Docker Compose for n8n + SAP MCP sidecar
-- **918 Tests** -- Unit, integration, E2E with CI/CD via GitHub Actions
+- **1267 Tests** -- Unit, integration, E2E with CI/CD via GitHub Actions
 
 ## Quick Start
 
@@ -84,7 +84,9 @@ export SAP_BASE_URL=https://your-sap-system.example.com
 export SAP_AUTH_TYPE=basic
 export SAP_USERNAME=your-user
 export SAP_PASSWORD=your-password
-export SAP_CLIENT=100
+export SAP_CLIENT=100   # do not omit: without it a client-dependent Gateway
+                        # answers 401 "Anmeldung fehlgeschlagen", which looks
+                        # like a wrong password but is not one
 ```
 
 ### 3. Run
@@ -98,9 +100,138 @@ guniweb-sap-mcp --transport http --port 8808
 
 # With tool tier control
 guniweb-sap-mcp --transport http --port 8808 --tiers core,odata
-guniweb-sap-mcp --transport http --port 8808 --read-only
+guniweb-sap-mcp --transport http --port 8808 --allow-write   # writes are OFF by default
+
+# Production: authenticate the MCP endpoint AND the inbound IDoc webhook
+guniweb-sap-mcp --transport http --port 8808 --api-key <mcp-key> --webhook-secret <idoc-secret>
+```
+
+#### Production hardening
+
+| Variable / Flag               | Default        | Description                                                                                     |
+|-------------------------------|----------------|-------------------------------------------------------------------------------------------------|
+| `--tool-timeout` / `SAP_MCP_TOOL_TIMEOUT` | `30` (seconds) | Time budget for a **whole tool call**, however many SAP requests it makes. On expiry the client gets a valid MCP response with `truncated: true` and a plain-language reason — never a hanging connection. Paginated reads return the pages fetched so far. `0` disables the budget (logged as a warning at startup) |
+| `SAP_HTTP_TIMEOUT_MS`         | `30000`        | Timeout for a **single** outbound SAP request (ms). Hanging gateways fail fast with a structured `TIMEOUT` error instead of stalling the n8n agent. Capped by whatever is left of the tool budget above |
+| `--cache-dir` / `SAP_CACHE_DIR` | _(unset = in-memory only)_ | Persist the catalog and metadata caches to disk so a restart does not refetch them. Without it every restart re-reads the full catalog — on a system with 1222 services that is the cold start that breaks client timeouts. n8n evicts its own client entry on a transport error, so restarts are not rare |
+| `SAP_CA_CERT` | _(unset)_ | Path to the root CA that signs your SAP certificate — the documented way past a corporate PKI. Mirrored into `NODE_EXTRA_CA_CERTS`. Prefer this over `SAP_TLS_VERIFY=false`, which disables the check rather than solving it |
+| `--concurrency` / `SAP_CONCURRENCY` | `5` | Upper bound on simultaneous SAP requests where several are unavoidable |
+| `--allow-write` / `SAP_MCP_ALLOW_WRITE` | _(unset = read-only)_ | **Since 0.2.2 the server is read-only by default.** `sap_create`, `sap_update`, `sap_delete`, `sap_function` and `sap_idoc_send` are not registered at all until you set this — a tool an agent cannot see is a tool it cannot call by accident. `--read-only` is still accepted and now simply describes the default |
+| `SAP_JWT_ISSUER` | _(unset = no verification)_ | Issuer whose JWKS is used to verify incoming user tokens. Without it tokens are only decoded, and the resulting identity is marked `verified: false` — safe to forward to SAP (SAP verifies itself), but not safe to base a local decision on. With it, a token failing verification is rejected with `401` rather than silently downgraded to the technical user |
+| `SAP_WEBHOOK_SECRET` / `--webhook-secret` | _(unset = open + startup warning)_ | Shared secret for the inbound IDoc webhook. Accepts `Authorization: Bearer <secret>` or HTTP Basic auth whose **password** equals the secret (matches SAP SM59 HTTP destinations) |
+| `SAP_WEBHOOK_HOST` / `--webhook-host` | all interfaces | Bind address of the standalone IDoc webhook (stdio mode). Set `127.0.0.1` behind a reverse proxy |
+| `SAP_WEBHOOK_MAX_BODY_BYTES`  | `10485760` (10 MB) | Reject larger IDoc payloads with `413` before they are buffered in memory                    |
+
+In HTTP transport mode the `/idoc` route is mounted on the MCP port: with `--webhook-secret` it authenticates independently of `--api-key` (so SAP's Basic auth works while n8n uses the Bearer key); without it, it falls under `--api-key` if set — and is **open** if neither is configured (the server logs a warning at startup).
+
+Both timeouts must stay **below the smallest client limit** in your setup. Measured against a production S/4HANA behind Cloudflare: Cloudflare aborts at 100 s (`524`), n8n's MCP client transport at 300 s. A budget above the smallest limit never takes effect, because the client gives up first.
+
+#### Tracing a slow tool call
+
+Every tool call logs one line on completion, carrying a correlation id that also appears on each SAP request it triggered:
+
+```json
+{"correlationId":"a3f19c8b2d04","tool":"sap_discover_services","outcome":"ok","durationMs":842,"sapCalls":1,"resultBytes":18422}
 ```
 
+`grep a3f19c8b2d04` then shows the full story of that one call — including how many SAP requests it produced. `outcome` is one of `ok`, `error`, `timeout` (budget expired) or `aborted` (client disconnected).
+
+**`level: 50` means something is actually wrong.** Expected non-2xx responses — the `404`s from the catalog path probing and from the V4 catalog attempt on systems that have none — are logged at `info` with `"expected": true`, not at `error`. Measured on a production system: over 96 hours of runtime that false alarm was the *only* error-level entry in the log. Alerting on `level: 50` is therefore meaningful; before, it produced nothing but noise.
+
+#### When something breaks
+
+`test-connection` walks the whole chain and reports the **first** failing stage — DNS, TLS (including the certificate chain), authentication, client (Mandant), catalog. Stages after the failure are reported as `skipped`, never as passed. The same five stages are logged once at startup, so the container log shows the state of the connection without calling anything.
+
+Every error response carries a classification, the SAP error code where one exists, a plain-language message and the next action — no tool answers with `"Unknown error"`. Three cases that used to cost a separate diagnosis each:
+
+| What SAP returns | What the server tells you |
+|---|---|
+| `403` with `/IWFND/MED/170` | The service is not registered — activate it in `/IWFND/MAINT_SERVICE`. **Not** an authorization problem, despite the 403 |
+| `401` with an HTML logon page | Is `sap-client` set? Without it a client-dependent Gateway rejects the logon in a way that looks like a wrong password |
+| `UNABLE_TO_GET_ISSUER_CERT_LOCALLY` | The certificate chain is incomplete; supply the root CA via `SAP_CA_CERT`. The message names the issuing CA |
+
+If `SAP_TLS_VERIFY=false` is in effect, **every tool result** carries a warning about it — not just the startup log. A disabled check that nobody sees after the first day tends to stay for years; on the production system that prompted this work it was set, and nobody noticed.
+
+#### Streamable HTTP is stateless
+
+`--transport http` serves MCP in **stateless** mode: every request is self-contained and the server neither issues nor expects an `mcp-session-id` header. There is no server-side session to resume, so a client may reconnect at any time without losing state.
+
+Clients that *require* a session id — they read `mcp-session-id` from the initialize response and send it back on subsequent requests — will fail against this server. The n8n MCP Client, Claude Desktop, and the MCP Inspector all work without it.
+
+The legacy `--transport sse` mode is different: it keeps a per-connection session and expects a `?sessionId=` query parameter on `/messages`. Prefer `http` for new setups.
+
+#### Several SAP systems: `destinations.json`
+
+The `SAP_*` variables describe **one** SAP system. To run one server against several — a production system next to a test client, or several customers' systems behind one n8n — put the connections into a `destinations.json` and point the server at it:
+
+```bash
+guniweb-sap-mcp --transport http --port 8808 --destinations /config/destinations.json
+# or: SAP_MCP_DESTINATIONS=/config/destinations.json
+```
+
+```json
+{
+  "version": 1,
+  "destinations": {
+    "default": {
+      "baseUrl": "https://s4prod.example.com",
+      "authType": "basic",
+      "username": "MCP_USER",
+      "password": "${S4PROD_PASSWORD}",
+      "sapClient": "100"
+    },
+    "s4test": {
+      "baseUrl": "https://s4test.example.com",
+      "authType": "basic",
+      "username": "MCP_USER",
+      "password": "${S4TEST_PASSWORD}",
+      "sapClient": "200"
+    }
+  }
+}
+```
+
+- **One entry = one connection.** Every entry accepts exactly the fields the `SAP_*` variables accept (`authType` and its credentials, `sapClient`, `baseUrl`) — all seven authentication types work per destination.
+- **Secrets stay out of the file.** Any string may contain `${VARIABLE}`; it is resolved from the environment at load time (Docker secrets, `.env`). A placeholder that cannot be resolved makes the file invalid — an empty password would otherwise surface as a misleading `401` from SAP.
+- **Hot reload.** The file is watched; a change is picked up on the next request, no restart. `kill -HUP <pid>` forces a reload. An invalid file **never** replaces the running configuration — the error is logged and the last valid one stays in force.
+- **Backwards compatible.** Without `--destinations` nothing changes: the `SAP_*` variables are the single destination `default`. With the file, `SAP_*` (if set) is added as `default` unless the file defines one — so an installation can migrate without a gap.
+- **Which destination answers?** Without a token: the one named `default` (or the only one). With a token: the token's destination — see below.
+
+**Tokens: one Bearer value per n8n credential selects the system.** The n8n MCP Client sends exactly one `Authorization` header, so a token does two jobs at once — it authenticates the request and picks the destination. SAP passwords never leave the server; n8n only holds the token.
+
+```json
+{
+  "version": 1,
+  "destinations": { "s4prod": { "...": "..." }, "s4test": { "...": "..." } },
+  "tokens": [
+    { "hash": "sha256:9f2c…", "destination": "s4prod", "label": "n8n prod workflows" },
+    { "hash": "sha256:1a7e…", "destination": "s4test", "label": "n8n dev", "revoked": false }
+  ]
+}
+```
+
+- The file holds only the **SHA-256 hash**; the plain token (`gsm_<destination>_<random>`) is printed **once** by `tokens issue` and cannot be recovered afterwards. Put it into an n8n **Bearer Auth** credential on the MCP Client node. As soon as at least one token exists, requests without a valid token get `401` — `--api-key` keeps working alongside and serves the `default` destination.
+- Lookup is constant-time over all entries; `"revoked": true` disables a token on the next request (hot reload, no restart). After 10 failed attempts within a minute the client address gets `429` with `Retry-After` — also for a valid token — as a brake on guessing (a reverse proxy with a real rate limit is still the right place for that). Tokens are never written to the log; rejections are logged with address and reason.
+**Managing the file — no hand-editing needed.** The same binary carries the admin commands; every command validates the result exactly like the server does before writing (never leaves an unloadable file), writes atomically, and a running server picks the change up on the next request:
+
+```bash
+export SAP_MCP_DESTINATIONS=/config/destinations.json      # or --destinations <path> on each call
+
+guniweb-sap-mcp destinations add s4prod --base-url https://s4prod.example.com --auth-type basic \
+    --username MCP_USER --password '${S4PROD_PASSWORD}' --sap-client 100
+guniweb-sap-mcp destinations add legacy --from-env         # copy the current SAP_* environment (migration)
+guniweb-sap-mcp destinations list                          # names, URLs, auth type, client, token count — never secrets
+
+guniweb-sap-mcp tokens issue s4prod --label "n8n prod" --read-only --tiers core,odata
+#   → prints the plain token ONCE on stdout (everything else goes to stderr, so TOKEN=$(…) works)
+guniweb-sap-mcp tokens list                                # label, destination, hash prefix, policy, status
+guniweb-sap-mcp tokens revoke "n8n prod"                   # by label or ≥ 8 hex chars of the hash
+guniweb-sap-mcp destinations remove s4test [--force]       # refuses while tokens point at it, unless --force
+```
+
+`--set field=value` covers any auth type (`--auth-type oauth2 --set clientId=… --set clientSecret='${BTP_SECRET}' --set tokenServiceUrl=…`); `${ENV_VAR}` values are checked against the current environment (`--allow-missing-env` to skip, e.g. when the variable only exists inside the container). In Docker, run the commands inside the container so they see the same file and environment: `docker compose exec sap-mcp guniweb-sap-mcp tokens issue s4prod`.
+
+- **Per-token permissions** — `"policy": { "readOnly": true, "tiers": ["core", "odata"] }`. A policy only ever *restricts*: `readOnly: true` hides the write tools for that token even when the server runs with `--allow-write` (`readOnly: false` cannot open a read-only server), and `tiers` is a ceiling — tiers outside it are invisible for that token and `sap_enable_tools` cannot switch them on. `tools/list` is filtered per request, so an agent on a read-only token never sees `sap_create` in the first place.
+
 ### 4. Use with n8n
 
 1. Add an **AI Agent** node with an **MCP Client Tool** sub-node
@@ -165,16 +296,16 @@ In addition to the variables documented in [Quick Start](#quick-start):
 ### Core Tier (always active)
 | Tool | Description |
 |------|-------------|
-| `test-connection` | Test connectivity to SAP (pings catalog service) |
-| `sap_list_services` | List entity sets in an OData service |
+| `test-connection` | Stage-by-stage connection diagnosis (DNS → TLS → auth → client → catalog); names the first failing stage |
+| `sap_list_services` | List entity sets in an OData service (paged, default 50) |
 | `sap_discover_services` | Discover all services with category filtering and search |
-| `sap_get_metadata` | Inspect entity types, properties, keys (with search filter) |
+| `sap_get_metadata` | Inspect entity types, properties, keys, and what SAP allows per entity set (creatable/updatable/deletable) |
 
 ### OData Tier (active by default)
 | Tool | Description |
 |------|-------------|
 | `sap_read` | Read a single entity or collection |
-| `sap_query` | Query with OData $filter, $select, $expand, pagination |
+| `sap_query` | Query with OData $filter, $select, $expand, paging (default `top` 50) |
 | `sap_nl_query` | Natural language query with structured filters |
 | `sap_create` | Create entities (supports deep insert + draft services) |
 | `sap_update` | Update entities (automatic ETag handling) |