ldap-rest 1.1.0 → 1.2.1

package/docs/usage/configuration.md

@@ -39,17 +39,20 @@ All LDAP-Rest configuration options.
     - [core/auth/trustedProxy](#coreauthtrustedproxy)
   - [Twake Integration Plugins](#twake-integration-plugins)
     - [core/twake/james](#coretwakejames)
+    - [core/twake/migrationProxy](#coretwakemigrationproxy)
     - [core/twake/calendar](#coretwakecalendar)
     - [core/twake/applicativeAccounts](#coretwakeapplicativeaccounts)
     - [core/twake/appAccountsConsistency](#coretwakeappaccountsconsistency)
   - [Utility Plugins](#utility-plugins)
     - [core/static](#corestatic)
     - [core/weblogs](#coreweblogs)
+    - [core/metrics](#coremetrics)
     - [core/storage](#corestorage)
     - [core/configApi](#coreconfigapi)
 - [Configuration File](#configuration-file)
 - [LDAP Failover](#ldap-failover)
 - [Log Levels (`--log-level`)](#log-levels---log-level)
+- [Logger (`--logger`)](#logger---logger)
 
 ## Environment Variables
 
@@ -148,15 +151,18 @@ to keep the default.
 
 ## General Options
 
-| CLI              | Plural           | Env               | Default          | Description                                 |
-| ---------------- | ---------------- | ----------------- | ---------------- | ------------------------------------------- |
-| `--port`         |                  | `DM_PORT`         | `8081`           | Listen port                                 |
-| `--plugin`       | `--plugins`      | `DM_PLUGINS`      | `[]`             | Plugins to load                             |
-| `--log-level`    |                  | `DM_LOG_LEVEL`    | `notice`         | Log level: error, warn, notice, info, debug |
-| `--logger`       |                  | `DM_LOGGER`       | `console`        | Logger type                                 |
-| `--api-prefix`   |                  | `DM_API_PREFIX`   | `/api`           | API URL prefix                              |
-| `--mail-domain`  | `--mail-domains` | `DM_MAIL_DOMAIN`  | `[]`             | Mail domains                                |
-| `--schemas-path` |                  | `DM_SCHEMAS_PATH` | `static/schemas` | Path to JSON schemas                        |
+| CLI                  | Plural           | Env                   | Default          | Description                                                                                       |
+| -------------------- | ---------------- | --------------------- | ---------------- | ------------------------------------------------------------------------------------------------- |
+| `--port`             |                  | `DM_PORT`             | `8081`           | Listen port                                                                                       |
+| `--shutdown-timeout` |                  | `DM_SHUTDOWN_TIMEOUT` | `20000`          | How long a stop may take, in ms, see [shutdown](operations.md#shutdown)                           |
+| `--plugin`           | `--plugins`      | `DM_PLUGINS`          | `[]`             | Plugins to load                                                                                   |
+| `--log-level`        |                  | `DM_LOG_LEVEL`        | `notice`         | Log level: error, warn, notice, info, debug                                                       |
+| `--logger`           |                  | `DM_LOGGER`           | `console`        | `console` or `file`, see [Logger](#logger---logger)                                               |
+| `--log-file`         |                  | `DM_LOG_FILE`         | `error.log`      | File `--logger file` writes to                                                                    |
+| `--debug`            |                  | `DM_DEBUG`            | `false`          | Name the failure in the 5xx answers of the error middleware and `serverError`; not for production |
+| `--api-prefix`       |                  | `DM_API_PREFIX`       | `/api`           | API URL prefix                                                                                    |
+| `--mail-domain`      | `--mail-domains` | `DM_MAIL_DOMAIN`      | `[]`             | Mail domains                                                                                      |
+| `--schemas-path`     |                  | `DM_SCHEMAS_PATH`     | `static/schemas` | Path to JSON schemas                                                                              |
 
 A schema file named by an option may extend another one, shipped or not, and
 hold only the differences:
@@ -167,8 +173,8 @@ hold only the differences:
 | CLI                          | Plural           | Env                      | Default                            | Description                                 |
 | ---------------------------- | ---------------- | ------------------------ | ---------------------------------- | ------------------------------------------- |
 | `--ldap-url`                 | `--ldap-urls`    | `DM_LDAP_URL`            | `ldap://localhost`                 | LDAP server URL(s)                          |
-| `--ldap-dn`                  |                  | `DM_LDAP_DN`             | `cn=admin,dc=example,dc=com`       | Bind DN                                     |
-| `--ldap-pwd`                 |                  | `DM_LDAP_PWD`            | `admin`                            | Password                                    |
+| `--ldap-dn`                  |                  | `DM_LDAP_DN`             |                                    | Bind DN (required)                          |
+| `--ldap-pwd`                 |                  | `DM_LDAP_PWD`            |                                    | Password (required)                         |
 | `--ldap-base`                |                  | `DM_LDAP_BASE`           |                                    | Base DN for searches (required)             |
 | `--ldap-user-main-attribute` |                  | `DM_LDAP_USER_ATTRIBUTE` | `uid`                              | User identifier attribute                   |
 | `--ldap-cache-max`           |                  | `DM_LDAP_CACHE_MAX`      | `1000`                             | Max cache entries                           |
@@ -493,6 +499,19 @@ whichever of the two names it selects. A value other than `req.user` or
 
 The wait before each retry grows with the attempt, so the total wait is `--james-retry-delay` × n(n−1)/2 for n attempts: 10 s with the defaults, 190 s with 20 attempts.
 
+#### `core/twake/migrationProxy`
+
+Keeps the migrated-users list of the Twake Mail migration proxy in step with the accounts, [details](plugins/integrations/mail-migration.md).
+
+| CLI                                | Plural                       | Env                                 | Default | Description                                                      |
+| ---------------------------------- | ---------------------------- | ----------------------------------- | ------- | ---------------------------------------------------------------- |
+| `--migration-proxy-url`            |                              | `DM_MIGRATION_PROXY_URL`            |         | Webadmin URL of the proxy (required)                             |
+| `--migration-proxy-token`          |                              | `DM_MIGRATION_PROXY_TOKEN`          |         | Bearer token                                                     |
+| `--migration-proxy-branch`         | `--migration-proxy-branches` | `DM_MIGRATION_PROXY_BRANCHES`       | `[]`    | Branches whose entries are followed, `;`-separated (required)    |
+| `--migration-proxy-concurrency`    |                              | `DM_MIGRATION_PROXY_CONCURRENCY`    | `10`    | Calls to the proxy at once                                       |
+| `--migration-proxy-retry-attempts` |                              | `DM_MIGRATION_PROXY_RETRY_ATTEMPTS` | `5`     | Attempts in all of a call the proxy answers 500, 502, 503 or 504 |
+| `--migration-proxy-retry-delay`    |                              | `DM_MIGRATION_PROXY_RETRY_DELAY`    | `1000`  | Base retry delay (ms), growing with the attempt                  |
+
 #### `core/twake/calendar`
 
 Formerly `core/twake/calendarResources`, still accepted as a deprecated alias.
@@ -624,8 +643,8 @@ Extends `core/ldap/groups`, whose options it reads too.
 
 | CLI                                           | Env                                            | Default            | Description                                           |
 | --------------------------------------------- | ---------------------------------------------- | ------------------ | ----------------------------------------------------- |
-| `--twake-group-base`                          | `DM_TWAKE_GROUP_BASE`                          |                    | Group branch of an organization, with `{org}`         |
-| `--twake-group-user-base`                     | `DM_TWAKE_GROUP_USER_BASE`                     |                    | User branch of an organization, with `{org}`          |
+| `--twake-group-base`                          | `DM_TWAKE_GROUP_BASE`                          |                    | Group branch of an organization, holding `{org}` once |
+| `--twake-group-user-base`                     | `DM_TWAKE_GROUP_USER_BASE`                     |                    | User branch of an organization, holding `{org}` once  |
 | `--twake-group-organization-dn`               | `DM_TWAKE_GROUP_ORGANIZATION_DN`               |                    | Organization entry, with `{org}` (empty: not checked) |
 | `--twake-group-organization-status-attribute` | `DM_TWAKE_GROUP_ORGANIZATION_STATUS_ATTRIBUTE` | `twakeOrgStatus`   | Organization status attribute                         |
 | `--twake-group-organization-deleted-value`    | `DM_TWAKE_GROUP_ORGANIZATION_DELETED_VALUE`    | `deleted`          | Status of a deleted organization                      |
@@ -644,17 +663,17 @@ organization entry, and `core/ldap/onChange`; with `--rabbitmq-url`, it
 requires `core/rabbitmq` and reads the organization domain from
 `--twake-instance-organization-domain-attribute`.
 
-| CLI                                    | Env                                     | Default                                | Description                                        |
-| -------------------------------------- | --------------------------------------- | -------------------------------------- | -------------------------------------------------- |
-| `--twake-space-base`                   | `DM_TWAKE_SPACE_BASE`                   |                                        | Space branch of an organization, with `{org}`      |
-| `--twake-space-class`                  | `DM_TWAKE_SPACE_CLASS`                  | `top,twakeSpace`                       | Object classes of a space                          |
-| `--twake-space-display-name-attribute` | `DM_TWAKE_SPACE_DISPLAY_NAME_ATTRIBUTE` | `twakeDisplayName`                     | Space name attribute                               |
-| `--twake-space-admin-attribute`        | `DM_TWAKE_SPACE_ADMIN_ATTRIBUTE`        | `twakeSpaceAdmin`                      | DNs holding the admin role                         |
-| `--twake-space-editor-attribute`       | `DM_TWAKE_SPACE_EDITOR_ATTRIBUTE`       | `twakeSpaceEditor`                     | DNs holding the editor role                        |
-| `--twake-space-viewer-attribute`       | `DM_TWAKE_SPACE_VIEWER_ATTRIBUTE`       | `twakeSpaceViewer`                     | DNs holding the viewer role                        |
-| `--twake-space-user-role-attribute`    | `DM_TWAKE_SPACE_USER_ROLE_ATTRIBUTE`    |                                        | User attribute holding `<space id>:<role>`, if set |
-| `--twake-space-exchange`               | `DM_TWAKE_SPACE_EXCHANGE`               | `space`                                | Topic exchange of the space events                 |
-| `--twake-space-sync-queue`             | `DM_TWAKE_SPACE_SYNC_QUEUE`             | `twake.space.sync.requested.ldap-rest` | Queue of the space sync requests                   |
+| CLI                                    | Env                                     | Default                                | Description                                           |
+| -------------------------------------- | --------------------------------------- | -------------------------------------- | ----------------------------------------------------- |
+| `--twake-space-base`                   | `DM_TWAKE_SPACE_BASE`                   |                                        | Space branch of an organization, holding `{org}` once |
+| `--twake-space-class`                  | `DM_TWAKE_SPACE_CLASS`                  | `top,twakeSpace`                       | Object classes of a space                             |
+| `--twake-space-display-name-attribute` | `DM_TWAKE_SPACE_DISPLAY_NAME_ATTRIBUTE` | `twakeDisplayName`                     | Space name attribute                                  |
+| `--twake-space-admin-attribute`        | `DM_TWAKE_SPACE_ADMIN_ATTRIBUTE`        | `twakeSpaceAdmin`                      | DNs holding the admin role                            |
+| `--twake-space-editor-attribute`       | `DM_TWAKE_SPACE_EDITOR_ATTRIBUTE`       | `twakeSpaceEditor`                     | DNs holding the editor role                           |
+| `--twake-space-viewer-attribute`       | `DM_TWAKE_SPACE_VIEWER_ATTRIBUTE`       | `twakeSpaceViewer`                     | DNs holding the viewer role                           |
+| `--twake-space-user-role-attribute`    | `DM_TWAKE_SPACE_USER_ROLE_ATTRIBUTE`    |                                        | User attribute holding `<space id>:<role>`, if set    |
+| `--twake-space-exchange`               | `DM_TWAKE_SPACE_EXCHANGE`               | `space`                                | Topic exchange of the space events                    |
+| `--twake-space-sync-queue`             | `DM_TWAKE_SPACE_SYNC_QUEUE`             | `twake.space.sync.requested.ldap-rest` | Queue of the space sync requests                      |
 
 See [spaces](plugins/integrations/spaces.md).
 
@@ -675,6 +694,15 @@ See [security headers](plugins/utilities/static.md#security-headers).
 
 Web access logging plugin. No configuration options - just add the plugin to enable access logs.
 
+#### `core/metrics`
+
+Prometheus metrics on a port of their own, see
+[metrics](operations.md#metrics).
+
+| CLI              | Env               | Default | Description                                                |
+| ---------------- | ----------------- | ------- | ---------------------------------------------------------- |
+| `--metrics-port` | `DM_METRICS_PORT` | `9464`  | Port `GET /metrics` is served on; keep it off the internet |
+
 #### `core/storage`
 
 Keyed storage with a deadline, for the plugins that need to keep something —
@@ -743,3 +771,13 @@ The system will:
 | `debug`  | Everything, including debug output           |
 
 The `notice` level is ideal for production as it shows web access logs without flooding with general info messages.
+
+## Logger _(`--logger`)_
+
+| Logger    | Output                                                                                               |
+| --------- | ---------------------------------------------------------------------------------------------------- |
+| `console` | One JSON object per line on standard output (errors and warnings on standard error)                  |
+| `file`    | Text lines appended to `--log-file` (`DM_LOG_FILE`), `error.log` in the working directory by default |
+
+An empty value is `console`. Any other value, or `file` with an empty
+`--log-file`, stops the server at startup.

package/docs/usage/operations.md

@@ -0,0 +1,193 @@
+# Operations
+
+## Health and readiness
+
+Two routes answer outside the API prefix, without authentication, and are
+never cached:
+
+| Route          | Answers                                                       |
+| -------------- | ------------------------------------------------------------- |
+| `GET /healthz` | `200 {"status":"ok"}` as long as the process serves           |
+| `GET /readyz`  | `200` when the server can serve, `503` when it cannot (below) |
+
+`/readyz` answers a JSON body:
+
+| `status`      | Code | Meaning                                                                                                |
+| ------------- | ---- | ------------------------------------------------------------------------------------------------------ |
+| `starting`    | 503  | The plugins are still loading (a host serving before `ready`; the shipped binary opens the port after) |
+| `stopping`    | 503  | The server was told to stop and finishes what is in flight; a new connection is refused                |
+| `unavailable` | 503  | The directory does not answer a read of its root DSE in 2 s (`ldap: failed`)                           |
+| `ok`          | 200  | Everything answers                                                                                     |
+| `degraded`    | 200  | The directory answers, a plugin reports a resource of its own down                                     |
+
+```json
+{ "status": "degraded", "ldap": "ok", "plugins": { "rabbitmq": "degraded" } }
+```
+
+`plugins` lists the plugins that report their health; `failed` means the
+report threw or did not answer in time. Only the directory takes a replica
+out of rotation: a broken outbound service is visible in the body, and the
+server keeps serving what does not need it.
+
+The directory and the plugins are asked at once, so the answer takes 2 s at
+most. The probe reads the root DSE, which any bound account may: a wrong
+`--ldap-base` is not what it reports. The directory is probed on a connection of its own, bound as the
+server, outside the connection pool and the search cache. One answer serves
+5 seconds whatever the number of probes; the probes arriving meanwhile
+share it.
+
+Both routes are served before the authentication and the rate limiting,
+and `/readyz` names the loaded plugins that report: on an ingress exposing
+the server to the internet, keep the two paths to the cluster.
+
+The Helm chart probes both routes over HTTP (`livenessProbe` and
+`readinessProbe` in `values.yaml`).
+
+### What the shipped plugins report
+
+| Plugin           | `degraded` when                                                                                                                                                                                             |
+| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `storage`        | the backend refuses a `SELECT 1` or a `PING`, or the `file` backend's directory is not a directory this user may write in; the `ldap` backend has no answer of its own, the directory check above covers it |
+| `rabbitmq`       | a broker is configured and no connection to it stands: until the first publish or subscription opens one, and while the client reconnects                                                                   |
+| `james`          | James answers its `GET /healthcheck` with 503, one of its own checks unhealthy, or not within 1.5 s; a James whose checks are only degraded answers 200, so `ok` here                                       |
+| `migrationProxy` | the migration proxy answers its `GET /healthcheck` with an error, or not within 1.5 s                                                                                                                       |
+
+A check that leaves the probe to time out reads as `failed`.
+
+### For plugin authors
+
+A plugin reports its own resources with an optional `health()` method
+returning `ok` or `degraded`, called at most once every 5 seconds; it must
+answer within 2 s and never throw for an ordinary outage. The probe stops
+waiting at 2 s but cancels nothing: a plugin bounds its own call, as
+`james` does with its 1.5 s.
+
+## Request id
+
+Every response the API sends carries an `X-Request-Id` header: an
+identifier the server generates for the request, the same one its writes
+hand to the plugins and its log lines carry. An `X-Request-Id` the client sends is not taken: a
+caller could otherwise make the writes of two requests look like one.
+
+A proxy or an ingress generating an identifier of its own does not pass
+it on either: log the answer's `X-Request-Id` there to match its lines with
+the server's.
+
+Every line logged while a request is served carries `requestId`, the lines
+of the hooks it ran included, as a field of the JSON console output and as
+`[<id>]` after the level in the log file. The line of `core/weblogs` carries
+it too. A line without one was written outside any request: at startup, by
+a scheduled task, by a message consumer, by a connection the server keeps.
+Work shared between requests, a schema load or a readiness answer, logs
+under the request that started it.
+
+A line a hook writes carries `plugin`, the plugin that registered the hook,
+and `op`, the hook's name, the `Hook error` line of a hook that threw
+included: in the JSON console output as fields, in the log file as
+`[<plugin> <op>]` after the request id. A line naming another `plugin`
+itself keeps it, with no `op` from the hook, and reads `[<plugin>]` in the
+file wherever it is written. These are the labels of
+`ldap_rest_hook_failures_total`.
+
+A line carrying an `error`, the reason of a failure, has it as a field of
+the JSON console output and in parentheses after the message in the log
+file; the file keeps no field other than these.
+
+## Metrics
+
+`core/metrics` serves Prometheus metrics on `GET /metrics`, on
+`--metrics-port` (9464): a port of its own, outside the authentication,
+the rate limiting and the request id, to keep to the cluster or the host.
+
+```bash
+--plugin core/metrics --metrics-port 9464
+```
+
+| Metric                                        | Type      | Labels                      | Meaning                                                                                                                                                                                                                                                                                                         |
+| --------------------------------------------- | --------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `ldap_rest_http_request_duration_seconds`     | histogram | `method`, `route`, `status` | Time to answer, by route template (`/api/v1/users/:id`; `none` unmatched or answered by a middleware, `core/static` included)                                                                                                                                                                                   |
+| `ldap_rest_ldap_operation_duration_seconds`   | histogram | `operation`, `result`       | Time the directory takes: `bind`, `search`, `add`, `modify`, `rename`, `delete`; `ok`, the LDAP result code of a refusal (`32`: no such entry), `error` when it did not answer                                                                                                                                  |
+| `ldap_rest_ldap_pool_wait_seconds`            | histogram |                             | Time to get a pooled connection, opening one included                                                                                                                                                                                                                                                           |
+| `ldap_rest_ldap_pool_connections`             | gauge     | `state`                     | Pooled connections, `busy` or `idle`                                                                                                                                                                                                                                                                            |
+| `ldap_rest_ldap_pool_waiting`                 | gauge     |                             | Operations waiting for a connection, the pool being full                                                                                                                                                                                                                                                        |
+| `ldap_rest_ldap_pool_max`                     | gauge     |                             | `--ldap-pool-size`                                                                                                                                                                                                                                                                                              |
+| `ldap_rest_ldap_search_cache_total`           | counter   | `result`                    | Cacheable searches, `hit` or `miss`; nothing when the cache is off                                                                                                                                                                                                                                              |
+| `ldap_rest_hook_failures_total`               | counter   | `hook`, `plugin`            | Notification hooks (`*done`, `onLdap*`) that threw, which the server logs and goes on                                                                                                                                                                                                                           |
+| `ldap_rest_outbound_request_duration_seconds` | histogram | `target`, `status`          | Time a service the server calls takes to answer: the calling plugin (`james`, `migrationProxy`, `calendar` or its old name `calendarResources`, `drive`, `cozyProvision`, `clouderyProvision`, `crowdsec`) or the instance provider (`cloudery`, `cozy-stack`); the HTTP status, `error` when it did not answer |
+| `ldap_rest_unhandled_rejections_total`        | counter   |                             | Promise rejections nothing handled; counted by the binary, a host skipping `exitOnCrash()` leaves it at zero                                                                                                                                                                                                    |
+| `process_*`, `nodejs_*`                       |           |                             | The process: CPU, memory, event loop lag, handles, GC                                                                                                                                                                                                                                                           |
+
+A search the caller pages is timed page by page, what the caller does
+between two pages left out; the readiness ping is not timed. The route
+label is the template a route was declared with, never the URL: a label
+per URL would grow without bound; under a router a plugin mounted, the
+mount reads `*`. An outbound call is timed from the moment it is sent,
+after the plugin's own queue, to the answer's headers, each attempt of a
+retried call on its own; a plugin's health check that asks its service,
+James's does, is a call like another. A plugin calling a service through
+`this.fetch` has its calls counted under its name; the OpenID Connect
+client's calls to its provider are made by its library, and not timed.
+
+The instruments come from `prom-client` 15, kept while the server supports
+Node 20, which its successor `@prometheus-io/client` dropped: the
+deprecation notice `npm install` prints is expected.
+
+The port listens on every interface. What to alert on: `/readyz` for a
+replica out of rotation, `result="error"` in
+`ldap_rest_ldap_operation_duration_seconds` (the directory not answering;
+a refusal is a result code), a
+`ldap_rest_ldap_pool_waiting` that stays above zero (raise
+`--ldap-pool-size`), `ldap_rest_hook_failures_total` rising (a
+synchronization that fails only in the log otherwise), `status="error"`
+or the 5xx share in `ldap_rest_outbound_request_duration_seconds` by
+`target` (a service the server depends on failing), the 5xx share of the
+requests.
+
+The Helm chart exposes the port on the Service with `metrics.enabled`, and
+renders a `ServiceMonitor` for the Prometheus operator with
+`metrics.serviceMonitor.enabled`; `core/metrics` still has to be in
+`DM_PLUGINS`.
+
+## Shutdown
+
+On `SIGTERM` or `SIGINT`, the binary stops in order:
+
+1. the listening socket closes: a new connection is refused, and
+   `/readyz` answers 503 `stopping` on the connections still open;
+2. idle keep-alive connections close at once; the others close with the
+   response they carry, sent with `Connection: close`, or are cut when the
+   drain is spent: `--shutdown-timeout` (20 s) less the 5 s kept for the
+   steps below, 1 s at least;
+3. every plugin's `close()` runs, at once: the broker connection, the
+   storage backend;
+4. the directory connections are closed;
+5. the signal ends the process, with its usual code (143 on `SIGTERM`).
+
+The whole takes `--shutdown-timeout` at most: a `close()` that does not
+settle in what remains of the budget is logged and left behind, by
+`shutdown()` itself, so a host gets the same bound, and the directory
+connections are closed anyway. The drain and the closes get 1 s each at
+least, so a budget under 2 s is exceeded. Kubernetes takes the pod out
+of its endpoints as it sends the signal, and the proxies follow a moment
+later: a request routed meanwhile is refused. A `preStop` hook sleeping a
+few seconds, before the signal, closes that window; the chart's `lifecycle`
+value takes one. The sleep and `--shutdown-timeout` together must stay
+under `terminationGracePeriodSeconds`, 30 s by default, which the chart
+leaves to Kubernetes: 5 s and 20 s fit.
+
+A host assembling the server in its own process calls `shutdown()` itself,
+`shutdownOnSignals()` when it wants the same behaviour on a signal, and
+`exitOnCrash()` for the policy below; without the latter, Node's own action
+on an uncaught exception applies.
+
+An exception nothing caught leaves the process in a state nobody knows: it
+is logged, the shutdown above runs, and the process exits with 1, within
+`--shutdown-timeout` whatever hangs, for its supervisor to start it again.
+A promise rejection nothing handled is logged and counted, and the process
+keeps serving.
+
+### For plugin authors
+
+A plugin holding a connection, a timer or a queue implements `close()`,
+called once after the server stopped serving; it must settle, a failure is
+logged and does not hold the others.

package/docs/usage/plugins/README.md

@@ -57,6 +57,7 @@ Connect to external systems:
 | ---------------------------------------------------- | ------------------------------------ |
 | [james-mail](integrations/james-mail.md)             | Apache James synchronization         |
 | [james-mailboxes](integrations/james-mailboxes.md)   | Apache James team mailboxes          |
+| [mail-migration](integrations/mail-migration.md)     | Twake Mail migration proxy           |
 | [calendar](integrations/calendar.md)                 | Twake Calendar resources and users   |
 | [app-accounts](integrations/app-accounts.md)         | Applicative accounts API             |
 | [lifecycle-events](integrations/lifecycle-events.md) | Account lifecycle events to RabbitMQ |
@@ -70,6 +71,7 @@ Connect to external systems:
 | ------------------------------- | ------------------------------------------------------------------- |
 | [static](utilities/static.md)   | Static file server                                                  |
 | [weblogs](utilities/weblogs.md) | HTTP logging                                                        |
+| [metrics](utilities/metrics.md) | Prometheus metrics on a port of their own                           |
 | [storage](utilities/storage.md) | Keyed storage with a deadline, in LDAP, files, PostgreSQL or Valkey |
 
 ## Plugin Dependencies

package/docs/usage/plugins/auth/authz-per-branch.md

@@ -536,7 +536,8 @@ This plugin only handles **authorization**. Combine with an authentication plugi
 
 ```json
 {
-  "error": "check logs"
+  "error": "Internal Server Error",
+  "message": "An error occurred"
 }
 ```
 

package/docs/usage/plugins/integrations/README.md

@@ -4,10 +4,11 @@ Plugins for connecting LDAP-Rest to external systems.
 
 ## Apache James
 
-| Plugin                                | Description                     |
-| ------------------------------------- | ------------------------------- |
-| [james-mail](james-mail.md)           | User and domain synchronization |
-| [james-mailboxes](james-mailboxes.md) | Team mailbox management         |
+| Plugin                                | Description                      |
+| ------------------------------------- | -------------------------------- |
+| [james-mail](james-mail.md)           | User and domain synchronization  |
+| [james-mailboxes](james-mailboxes.md) | Team mailbox management          |
+| [mail-migration](mail-migration.md)   | Migrated-users list of the proxy |
 
 ## Twake
 
@@ -45,6 +46,9 @@ core/twake/james
 core/twake/calendar
   └─ requires: core/ldap/onChange
 
+core/twake/migrationProxy
+  └─ requires: core/ldap/onChange
+
 core/twake/lifecycleEvents
   ├─ requires: core/ldap/onChange
   └─ requires: core/rabbitmq

package/docs/usage/plugins/integrations/groups.md

@@ -137,8 +137,8 @@ for it.
   describes department groups and requires `twakeDepartmentLink`, which these
   groups do not have. Groups created through SCIM are written directly and
   are not checked against this schema.
-- `--twake-group-base`, `--twake-group-user-base`: required, each with
-  `{org}`.
+- `--twake-group-base`, `--twake-group-user-base`: required, each holding
+  `{org}` once.
 - `--twake-group-organization-dn`: the organization entry. Empty means no
   organization is checked.
 - `--twake-group-organization-status-attribute` (default `twakeOrgStatus`),

package/docs/usage/plugins/integrations/mail-migration.md

@@ -0,0 +1,102 @@
+# Mail migration proxy
+
+`core/twake/migrationProxy` keeps the migrated-users list of the
+[Twake Mail migration proxy](https://github.com/linagora/tmail-backend/blob/master/docs/modules/ROOT/pages/migration-proxy/objective.adoc)
+in step with the directory, while a mail platform is moved to Twake Mail.
+
+The proxy sits in front of the old and the new mail backends. It relays
+each IMAP (and, when enabled, submission) connection to the backend hosting
+the authenticated user, and routes each inbound recipient to the backend
+hosting it. An address on its migrated list lives on the new backend; any
+other address lives on the old one. An account created during the migration
+has no mailbox on the old system, so it has to be on the list before its
+first login and its first mail: this plugin puts it there, and keeps the
+addresses of a migrated account on the list as they change.
+
+Load it for the duration of the migration, next to `core/twake/james`, and
+drop it once the old backend is gone. It requires `core/ldap/onChange`.
+
+## Scope
+
+Only the entries under `--migration-proxy-branch` are followed, the
+branches holding the accounts whose addresses the proxy routes: users, and
+the groups whose address is a team mailbox or a mailing list when those are
+created during the migration, as inbound mail for them is routed by
+recipient too. Keep out the branches whose entries carry an address that is
+not their own, the applicative accounts of
+[app-accounts](app-accounts.md) for instance: they would put their owner on
+the list or take it off.
+
+The addresses of an entry are every value of `--mail-attribute` and of
+`--alias-attribute`, the first mail being the one that says where the
+account lives. The proxy folds usernames to lower case, so a change of case
+is no change.
+
+## Behaviour
+
+Through the `onLdapEntryChange` hook, after the write has answered, one
+entry at a time:
+
+- An entry that gains its mail, when created or later, is put on the list
+  with all its addresses: `PUT /migratedUsers/{address}`.
+- An entry that keeps its mail and whose addresses change asks the proxy
+  about its first mail (`HEAD /migratedUsers/{mail}`). When the account is
+  migrated, the addresses it gained are put on the list, then the ones it
+  lost are taken off, once every addition holds; a user the proxy does not
+  know is on the old system, and its change is left to the migration, which
+  puts the new addresses on the list when it moves the mailbox.
+- An entry deleted, or losing its mail, is taken off the list with all its
+  addresses: `DELETE /migratedUsers/{address}`. A tombstone or a trashed
+  entry keeps its address, so it stays on the list until it is erased.
+
+The plugin cannot tell a new account from an existing one whose mail was
+cleared and set again, or from one created anew over its tombstone: those
+are put on the list too. Marking the existing accounts migrated is the
+migration's job, through the proxy's webadmin or its `migration:switch`
+exchange.
+
+### Failures
+
+A call the proxy answers with 500, 502, 503 or 504 is made again,
+`--migration-proxy-retry-attempts` times in all, after
+`--migration-proxy-retry-delay` milliseconds times the number of attempts
+made. A refusal, such as 400 for an address the proxy does not take as a
+username or 401 for a wrong token, and a call the proxy does not answer at
+all, connection refused or reset, are not. Each failed attempt logs a warning, the last failure an
+error naming the address, the call and the entry. Every call is counted in
+the [outbound metrics](../../operations.md#metrics), under the target
+`migrationProxy`.
+
+A failed `PUT` leaves the account routed to the old backend, which has no
+mailbox for it: watch those errors, and put the address on the list by hand
+through the proxy's webadmin. When the proxy has not said whether a renamed
+account is migrated, nothing is changed and the error is logged.
+
+## Where an account lives
+
+`GET /api/v1/users/:user/mail-migration` answers `{ "migrated": true }`
+when the proxy routes the user to the new backend, `{ "migrated": false }`
+otherwise, from the proxy's `HEAD`. The user is found by `uid` across the
+directory, and answered only when the caller may read the entry: another
+one gets 404, as a user that is not there. A user without mail gets 400,
+and a proxy that does not answer 502.
+
+## Readiness
+
+The plugin asks the proxy's `GET /healthcheck` for `/readyz`, and reports
+`degraded` when it answers an error or not within 1.5 s, see
+[what the shipped plugins report](../../operations.md#what-the-shipped-plugins-report).
+
+## Configuration
+
+```bash
+--plugin core/ldap/onChange \
+--plugin core/twake/james \
+--plugin core/twake/migrationProxy \
+--migration-proxy-url http://migration-proxy:8000 \
+--migration-proxy-token "$MIGRATION_PROXY_TOKEN" \
+--migration-proxy-branch "ou=users,dc=example,dc=com"
+```
+
+The options are listed in
+[configuration](../../configuration.md#coretwakemigrationproxy).

package/docs/usage/plugins/integrations/scim.md

@@ -309,6 +309,16 @@ mapping contains an entry with `scim: "department"`; otherwise the server return
 `400 invalidFilter`. Same for PATCH: operations on unmapped paths produce
 `400 invalidPath`.
 
+A User or Group body is checked against RFC 7643 before anything is written
+and before any hook reads it: `userName`, `displayName` and `id` are
+strings; a simple attribute takes a string, a number or a boolean, or a list
+of them; a complex attribute takes an object whose sub-attributes are scalars
+(or, for a reference such as `manager`, an object whose `value` is one); a
+multi-valued complex attribute such as `emails` takes a list of objects whose
+`value` is a scalar. Anything else is answered
+`400 invalidValue`, as is a PATCH operation that is not an object or whose
+`op` is not a string.
+
 A PATCH `add`, or a `remove` naming values, changes only the values it names:
 values the caller's read did not show, such as group members an
 `ldapsearchfilter` hides, stay. Two PATCHes changing different values of
@@ -369,7 +379,8 @@ via `escapeLdapFilter()` — no LDAP injection is possible.
 | `a eq "x" and b eq "y"`          | `(&(a=x)(b=y))`                           |
 | `not (a eq "x")`                 | `(!(a=x))`                                |
 
-Unknown SCIM attributes raise `400 invalidFilter`.
+Unknown SCIM attributes raise `400 invalidFilter`, as does a filter nested
+more than 32 levels deep (a parenthesis and a `not (` each count one).
 
 ## Pagination
 
@@ -734,10 +745,12 @@ Notable fields:
 - `schemaUrls` — direct download URLs of the schema files SCIM reads
   (`User.json` and `Group.json` under `--schemas-path`, with the
   `default-mapping.json` template beside them when it is there), when a
-  `core/static` plugin serves them, under its own `--static-name` (editor
+  `core/static` plugin serves them merged, through its schema routes, under
+  its own `--static-name` (editor
   UIs typically fetch them to render attribute pickers). Among plugins
   serving different paths, the one registered as `static` is preferred; a
-  path an earlier plugin already serves is left to that one. Absent when
+  path an earlier plugin already serves, lies under or holds, is left to
+  that one. Absent when
   none serves them: read it with a guard.
 - `resourceTypes[].mapping` — the live SCIM↔LDAP mapping used server-side, so
   UI builders can render filter helpers / attribute selectors without second

package/docs/usage/plugins/integrations/spaces.md

@@ -249,7 +249,7 @@ A space reads, strongest role first:
 The plugin loads [groups](groups.md), and reads its options for the
 organization's users, groups and organization entry.
 
-- `--twake-space-base`: required, with `{org}`.
+- `--twake-space-base`: required, holding `{org}` once.
 - `--twake-space-class` (default `top,twakeSpace`): the classes of a space
   created.
 - `--twake-space-display-name-attribute` (default `twakeDisplayName`).

package/docs/usage/plugins/ldap/bulk-import.md

@@ -174,7 +174,9 @@ mail: jdoe@company.com
 
 ### Fixed Attributes
 
-Attributes marked as `"fixed": true` in the schema are automatically added and should NOT be included in the CSV:
+Attributes marked as `"fixed": true` in the schema are automatically added. A
+row with a value under such a column fails, whatever the case of the column
+name; an empty cell is ignored as any other:
 
 **Schema**:
 

package/docs/usage/plugins/ldap/flat-generic.md

@@ -74,7 +74,9 @@ Each schema file must include an `entity` metadata section and attribute definit
 
 - **name**: Internal name used for hooks (e.g., `ldap{name}add`)
 - **mainAttribute**: DN attribute (e.g., `uid`, `cn`)
-- **objectClass**: LDAP objectClass values
+- **objectClass**: the classes of every entry created, whatever `objectClass`
+  the request carried; the schema's `objectClass` attribute only describes
+  them
 - **singularName**: Used in API paths and messages (e.g., "user")
 - **pluralName**: Used in API paths (e.g., "users")
 - **base**: LDAP base DN, supports config placeholders like `{ldap_base}`
@@ -138,7 +140,8 @@ never what it holds.
 
 Fixed attributes:
 
-- Auto-provisioned with default value on creation
+- Auto-provisioned with default value on creation; `objectClass` takes the
+  classes of the entity metadata instead
 - Cannot be modified after creation
 - Cannot be deleted
 

package/docs/usage/plugins/ldap/on-change.md

@@ -28,7 +28,8 @@ side:
 says who made the write, and through which door:
 
 - `actor`: the caller's `userName`, or its `user` when it has none;
-- `requestId`: generated, the same for every write of one request;
+- `requestId`: the answer's `X-Request-Id`, the same for every write of one
+  request, see [operations](../../operations.md#request-id);
 - `source`: `scim` for the SCIM API, `rest` otherwise.
 
 A write no request is behind, such as a scheduled task's, gets `{}`. The LDAP