ldap-rest 0.16.2 → 0.17.1

package/docs/client-development/schemas/README.md

@@ -13,6 +13,7 @@ LDAP-Rest uses JSON schemas to define LDAP entity structure, validation rules, a
 - [Semantic Roles](#semantic-roles)
 - [Validation Rules](#validation-rules)
 - [Predefined Schemas](#predefined-schemas)
+- [Extending a Schema](#extending-a-schema)
 
 ---
 
@@ -1055,6 +1056,91 @@ npx ldap-rest \
 
 ---
 
+## Extending a Schema
+
+To change a few points of a schema, shipped or not, write a file holding only
+the differences and naming the schema it `extends`. The deployment keeps the
+later fixes of the base schema, which a copy would miss:
+
+```json
+{
+  "extends": "/app/node_modules/ldap-rest/static/schemas/twake/users.json",
+  "attributes": { "personalTitle": null }
+}
+```
+
+Point the option at this file as at any schema:
+`DM_LDAP_FLAT_SCHEMA=/etc/ldap-rest/schemas/users.json`.
+
+### Naming the Base
+
+`extends` takes:
+
+- `ldap-rest:<path>`: a schema shipped with LDAP-Rest, `<path>` being relative
+  to its `static/schemas` directory, wherever the package is installed and
+  whatever `--schemas-path` says: `"extends": "ldap-rest:twake/users.json"`
+- an absolute path
+- any other path, relative to the directory of the extending file:
+  `"extends": "../base/users.json"`
+
+The base may extend another file in turn. A loop (`a.json` extends `b.json`,
+which extends `a.json`) is an error.
+
+### Merge Rules
+
+The extending file is applied on its base:
+
+- objects are merged key by key, at every depth; the extending file wins
+- `null` deletes the key
+- arrays and other values replace those of the base: an `objectClass` list
+  must be written whole
+- `extends` itself is not part of the result
+
+Removing an attribute and changing only the French label of another:
+
+```json
+{
+  "extends": "ldap-rest:twake/users.json",
+  "attributes": {
+    "personalTitle": null,
+    "cn": { "label": { "fr": "Nom complet" } }
+  }
+}
+```
+
+An attribute added by the extending file comes after the inherited ones, and
+so does its field in the forms built from the schema.
+
+The `__ldap_base__` style placeholders are replaced in each file of the chain
+before the merge: inherited values get them, and `extends` may hold one.
+`{ldap_base}` in `entity.base` is resolved on the merged schema.
+
+### Where It Applies
+
+Every schema file is read this way: `--ldap-flat-schema`, `--group-schema`,
+`--organization-schema`, `--bulk-import-schemas`, the SCIM schemas of
+`--schemas-path`, and the schemas `core/static` serves. A file whose base is
+missing or invalid is not loaded, as an unreadable file; `core/static` answers
+it with a 500.
+
+### Served Schemas
+
+`core/static` serves the merged schema. The configuration API gives each
+entity its merged schema inline (`schema`) and, with `core/static`, a
+`schemaUrl` made of the file path from its first `schemas` directory:
+`/etc/ldap-rest/schemas/users.json` gets `/static/schemas/users.json`.
+`core/static` serves a configured schema file at that URL even when it is
+outside `--static-path`, and before a shipped schema of the same URL:
+`/etc/ldap-rest/schemas/twake/users.json` is what
+`/static/schemas/twake/users.json` returns. An option left at its default
+does not count: without `--group-schema`, a `--static-path` of your own
+still serves its `schemas/twake/groups.json`. If two configured files get
+the same URL, the first one is served there and a warning names the other.
+A file outside any `schemas` directory has no `schemaUrl`; its schema is
+still inline.
+
+---
+
 ## Best Practices
 
 ### 1. Always Define ObjectClass

package/docs/plugin-development/dependencies.md

@@ -18,6 +18,12 @@ dependencies = {
 };
 ```
 
+A dependency is loaded when the plugin registers, unless a plugin of that
+name already is. A module requested again under the same name and without
+overrides — a dependency also given to `--plugin`, or an entry listed twice —
+is loaded once: the second request waits for the first. Only an entry with
+overrides builds a second instance, and it needs a name of its own.
+
 ### Hook Dependencies
 
 **Hook Providers**: Plugins that implement hooks (via `hooks: Hooks = { ... }`)

package/docs/presentations/en.md

@@ -148,7 +148,7 @@ James WebAdmin: POST /users/alice@.../rename/alice.smith@...
 ## 2. Quota Update
 
 ```
-LDAP: mailQuota = 1000000000 → 5000000000 (1GB → 5GB)
+LDAP: mailQuotaSize = 1000000000 → 5000000000 (1GB → 5GB)
   ↓ onChange detects the change
   ↓ Hook onLdapQuotaChange triggered
   ↓
@@ -584,7 +584,7 @@ npx ldap-rest \
   --james-webadmin-url http://james:8000 \
   --james-webadmin-token "admin-token" \
   --mail-attribute mail \
-  --quota-attribute mailQuota \
+  --quota-attribute mailQuotaSize \
   --alias-attribute mailAlternateAddress
 ```
 

package/docs/presentations/fr.md

@@ -148,7 +148,7 @@ James WebAdmin: POST /users/alice@.../rename/alice.smith@...
 ## 2. Mise à jour de quota
 
 ```
-LDAP: mailQuota = 1000000000 → 5000000000 (1GB → 5GB)
+LDAP: mailQuotaSize = 1000000000 → 5000000000 (1GB → 5GB)
   ↓ onChange détecte le changement
   ↓ Hook onLdapQuotaChange déclenché
   ↓
@@ -584,7 +584,7 @@ npx ldap-rest \
   --james-webadmin-url http://james:8000 \
   --james-webadmin-token "admin-token" \
   --mail-attribute mail \
-  --quota-attribute mailQuota \
+  --quota-attribute mailQuotaSize \
   --alias-attribute mailAlternateAddress
 ```
 

package/docs/usage/configuration.md

@@ -156,6 +156,10 @@ to keep the default.
 | `--mail-domain`  | `--mail-domains` | `DM_MAIL_DOMAIN`  | `[]`             | Mail domains                                |
 | `--schemas-path` |                  | `DM_SCHEMAS_PATH` | `static/schemas` | Path to JSON schemas                        |
 
+A schema file named by an option may extend another one, shipped or not, and
+hold only the differences:
+[extending a schema](../client-development/schemas/README.md#extending-a-schema).
+
 ## LDAP Connection
 
 | CLI                          | Plural           | Env                      | Default                            | Description                                 |
@@ -192,7 +196,7 @@ at `0` otherwise.
 | CLI                        | Env                         | Default                 | Description            |
 | -------------------------- | --------------------------- | ----------------------- | ---------------------- |
 | `--mail-attribute`         | `DM_MAIL_ATTRIBUTE`         | `mail`                  | Email attribute        |
-| `--quota-attribute`        | `DM_QUOTA_ATTRIBUTE`        | `mailQuota`             | Quota attribute        |
+| `--quota-attribute`        | `DM_QUOTA_ATTRIBUTE`        | `mailQuotaSize`         | Quota attribute        |
 | `--delegation-attribute`   | `DM_DELEGATION_ATTRIBUTE`   | `twakeDelegatedUsers`   | Delegation attribute   |
 | `--alias-attribute`        | `DM_ALIAS_ATTRIBUTE`        | `mailAlternateAddress`  | Email alias attribute  |
 | `--forward-attribute`      | `DM_FORWARD_ATTRIBUTE`      | `mailForwardingAddress` | Forward attribute      |
@@ -455,10 +459,14 @@ whichever of the two names it selects. A value other than `req.user` or
 | `--james-signature-template`     |                                 | `DM_JAMES_SIGNATURE_TEMPLATE`     |                         | Email signature template    |
 | `--james-concurrency`            |                                 | `DM_JAMES_CONCURRENCY`            | `10`                    | James API concurrency       |
 | `--james-init-delay`             |                                 | `DM_JAMES_INIT_DELAY`             | `1000`                  | Init delay (ms)             |
+| `--james-retry-attempts`         |                                 | `DM_JAMES_RETRY_ATTEMPTS`         | `5`                     | Attempts per user call      |
+| `--james-retry-delay`            |                                 | `DM_JAMES_RETRY_DELAY`            | `1000`                  | Base retry delay (ms)       |
 | `--james-mailing-list-branch`    | `--james-mailing-list-branches` | `DM_JAMES_MAILING_LIST_BRANCHES`  | `[]`                    | Mailing list branches       |
 | `--james-mailbox-type-attribute` |                                 | `DM_JAMES_MAILBOX_TYPE_ATTRIBUTE` | `twakeMailboxType`      | Mailbox type attribute      |
 | `--ldap-concurrency`             |                                 | `DM_LDAP_CONCURRENCY`             | `10`                    | LDAP operations concurrency |
 
+The wait before each retry grows with the attempt, so the total wait is `--james-retry-delay` × n(n−1)/2 for n attempts: 10 s with the defaults, 190 s with 20 attempts.
+
 #### `core/twake/calendar`
 
 Formerly `core/twake/calendarResources`, still accepted as a deprecated alias.

package/docs/usage/plugins/integrations/james-mail.md

@@ -20,7 +20,7 @@ The `twake/james` plugin automatically synchronizes email address and quota chan
 --plugin twake/james \
 --ldap-flat-schema ./static/schemas/twake/users.json \
 --mail-attribute mail \
---quota-attribute mailQuota \
+--quota-attribute mailQuotaSize \
 --james-webadmin-url http://james:8000 \
 --james-webadmin-token "your-admin-token"
 ```
@@ -37,9 +37,11 @@ DM_JAMES_WEBADMIN_TOKEN="your-admin-token"
 - `--james-webadmin-url`: James WebAdmin API base URL (required)
 - `--james-webadmin-token`: Bearer token for James WebAdmin authentication (optional)
 - `--mail-attribute`: LDAP attribute for email (default: `mail`)
-- `--quota-attribute`: LDAP attribute for quota (default: `mailQuota`)
+- `--quota-attribute`: LDAP attribute for quota (default: `mailQuotaSize`)
 - `--alias-attribute`: LDAP attribute for email aliases (default: `mailAlternateAddress`)
 - `--james-mailbox-type-attribute`: LDAP attribute for mailbox type (default: `twakeMailboxType`)
+- `--james-retry-attempts`: attempts in all of a call about a user James may not see yet (default: `5`), see [Replication lag](#replication-lag)
+- `--james-retry-delay`: base delay between those attempts, in ms (default: `1000`); the total wait is delay × n(n−1)/2 for n attempts
 
 ## How It Works
 
@@ -130,7 +132,7 @@ When a user's quota changes:
    PUT /api/v1/ldap/users/jdoe
    {
      "replace": {
-       "mailQuota": "5000000000"
+       "mailQuotaSize": "5000000000"
      }
    }
    ```
@@ -646,7 +648,7 @@ Alternative authentication methods (Basic Auth, JWT) can be configured via rever
 --plugin twake/james \
 --ldap-flat-schema ./schemas/twake/users.json \
 --mail-attribute mail \
---quota-attribute mailQuota \
+--quota-attribute mailQuotaSize \
 --james-webadmin-url http://james:8000
 ```
 
@@ -663,7 +665,7 @@ Alternative authentication methods (Basic Auth, JWT) can be configured via rever
 --ldap-flat-schema ./schemas/twake/users.json \
 --ldap-flat-schema ./schemas/twake/positions.json \
 --mail-attribute mail \
---quota-attribute mailQuota \
+--quota-attribute mailQuotaSize \
 --james-webadmin-url http://james:8000 \
 --james-webadmin-token "james-admin-token" \
 --static-path ./static \
@@ -707,6 +709,34 @@ The plugin logs all operations:
 
 ## Error Handling
 
+### Replication lag
+
+James reads its users from LDAP, and may read them from a replica that has
+not received the latest write yet: for up to a few seconds after a user is
+created or renamed, James does not know the new address. A call about that
+address fails meanwhile, with 400 (rename) or 404 (quota, alias, identity).
+
+The calls that need James to know the user are therefore retried on 400 and
+404: the rename, the quota, an alias or a forward added, a delegation added,
+and the identity. Each failed attempt logs a warning
+(`"result": "retry"`, with the attempt number); the error is logged when the
+last attempt fails. With the defaults, 5 attempts, the waits are 1, 2, 3 and
+4 seconds: `--james-retry-delay` times the number of attempts made, so
+`--james-retry-delay` × n(n−1)/2 in all for n attempts: 10 seconds with the
+defaults, 190 seconds with 20 attempts. `--james-retry-attempts 1` disables
+the retries. Other
+failures, such as 401 or 500, are not retried, nor are deletions.
+
+The group calls are not retried: James does not look the members up in its
+users repository there. Address groups (`/address/groups`) accept any valid
+address, and the member routes of a TMail team mailbox do not check the user;
+their only 404 is a missing team mailbox, which is avoided by creating it
+before adding the members.
+
+The retries run in the background, after the LDAP write has answered; a
+call waiting for its next attempt does not take a slot of
+`--james-concurrency`.
+
 ### Non-Existent User
 
 If mail account doesn't exist in James:
@@ -770,7 +800,7 @@ curl http://james:8000/users/john.doe@company.com
 curl -X PUT http://localhost:8081/api/v1/ldap/users/jdoe \
   -H "Authorization: Bearer admin-token" \
   -H "Content-Type: application/json" \
-  -d '{"replace": {"mailQuota": "5000000000"}}'
+  -d '{"replace": {"mailQuotaSize": "5000000000"}}'
 
 # Verify in James
 curl http://james:8000/quota/users/jdoe@company.com/size
@@ -780,12 +810,12 @@ curl http://james:8000/quota/users/jdoe@company.com/size
 
 Quota values are in **bytes**:
 
-| Size      | Bytes       | Example                      |
-| --------- | ----------- | ---------------------------- |
-| 1 GB      | 1000000000  | `"mailQuota": "1000000000"`  |
-| 5 GB      | 5000000000  | `"mailQuota": "5000000000"`  |
-| 10 GB     | 10000000000 | `"mailQuota": "10000000000"` |
-| Unlimited | -1          | `"mailQuota": "-1"`          |
+| Size      | Bytes       | Example                          |
+| --------- | ----------- | -------------------------------- |
+| 1 GB      | 1000000000  | `"mailQuotaSize": "1000000000"`  |
+| 5 GB      | 5000000000  | `"mailQuotaSize": "5000000000"`  |
+| 10 GB     | 10000000000 | `"mailQuotaSize": "10000000000"` |
+| Unlimited | -1          | `"mailQuotaSize": "-1"`          |
 
 ## Synchronization Scenarios
 
@@ -875,7 +905,7 @@ James deletes the entire address group.
 
 1. **One-way sync**: LDAP → James only. James changes don't sync back to LDAP.
 2. **No account creation**: Plugin doesn't create James accounts, only updates existing ones.
-3. **No user deletion**: Plugin doesn't delete James accounts when LDAP users are deleted.
+3. **No user deletion**: deleting an LDAP user leaves its James data (mailboxes, quota, aliases) as it is; purging it is left to James.
 4. **Synchronous**: James API calls block LDAP response. Slow James = slow LDAP.
 5. **No send restrictions**: James doesn't enforce list type restrictions (open/member/owner-only). These are metadata in LDAP only.
 
@@ -941,7 +971,7 @@ hooks: {
 
    ```bash
    --mail-attribute mail
-   --quota-attribute mailQuota
+   --quota-attribute mailQuotaSize
    ```
 
 3. Enable debug logging:

package/docs/usage/plugins/ldap/on-change.md

@@ -8,7 +8,7 @@ LDAP-Rest, and hooks for the attributes other plugins follow.
 ```bash
 --plugin core/ldap/onChange \
 --mail-attribute mail \
---quota-attribute mailQuota
+--quota-attribute mailQuotaSize
 ```
 
 ## Hooks
@@ -46,6 +46,12 @@ The other hooks are derived from it:
 | `onLdapDriveQuotaChange`  | `--drive-quota-attribute` changed                                                       | `(dn, oldQuota, newQuota)`             |
 | `onLdapDisplayNameChange` | the name from `--display-name-attribute`, else `cn`, else `givenName` and `sn`, changed | `(dn, oldName, newName)`               |
 
+A delete fires `onLdapChange` and `onLdapMailChange`, whose `newMail` is
+`null`, and none of the others: the entry has nothing left to apply, and the
+services fed by these hooks keep the data of a deleted user. The hooks given a
+`mail` fire only while the entry has one: a write removing the mail attribute
+does not fire them.
+
 `changes` maps each attribute that changed to `[oldValues, newValues]`, the
 full values on each side, `null` where the attribute is absent:
 

package/docs/usage/upgrading.md

@@ -4,6 +4,20 @@ What to check before deploying, newest first. Only releases that need a
 decision or a configuration change appear here; see the
 [CHANGELOG](../../CHANGELOG.md) for everything else.
 
+## To 0.17.0
+
+### The quota attribute defaults to `mailQuotaSize`
+
+**Who is affected:** a deployment that loads `twake/james` or
+`core/ldap/onChange`, or runs `bin/sync-james.mjs`, without setting
+`--quota-attribute` (`DM_QUOTA_ATTRIBUTE`).
+
+The default is now `mailQuotaSize`, the attribute the Twake schemas define,
+instead of `mailQuota`. A directory that holds the quota in `mailQuota` must
+set `DM_QUOTA_ATTRIBUTE=mailQuota` (or `--quota-attribute mailQuota`):
+without it, quota changes are no longer sent to the mail server. A deployment
+that already sets the option is unchanged.
+
 ## To 0.16.0
 
 ### Space events need the organization domain attribute

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ldap-rest",
-  "version": "0.16.2",
+  "version": "0.17.1",
   "description": "Light directory manager",
   "keywords": [
     "Identity-Manager"

package/static/schemas/twake/users.json

@@ -171,16 +171,6 @@
         "fr": "Autres boîtes"
       }
     },
-    "mailQuota": {
-      "type": "number",
-      "required": false,
-      "role": "emailQuota",
-      "group": "Mailbox Settings",
-      "label": {
-        "en": "Mail quota",
-        "fr": "Quota"
-      }
-    },
     "mailQuotaSize": {
       "type": "number",
       "required": false,