ldap-rest 0.8.2 → 0.10.0

package/docs/usage/plugins/ldap/organizations.md

@@ -174,6 +174,43 @@ curl "http://localhost:8081/api/v1/ldap/organizations/ou%3DIT%2Co%3Dgov%2Cc%3Dmu
 ]
 ```
 
+**When the answer is partial**
+
+Two things bound what comes back, and both are announced in the list rather
+than left for the client to guess:
+
+- The attached entries are capped at `--ldap-organization-max-subnodes`
+  (default 50). Child organizations are not capped.
+- The directory has a size limit of its own, and a branch holding more
+  entries than it will list in one answer makes it refuse rather than
+  truncate. The endpoint then asks again for a bounded number and returns
+  what it gets, writing a `warn` line naming the node.
+
+Either way the list ends with a row that is not an entry:
+
+```json
+{
+  "dn": "more-ou=IT,dc=example,dc=com",
+  "cn": ["... 412 more elements"],
+  "objectClass": ["moreIndicator"],
+  "_isMoreIndicator": "true",
+  "_totalCount": "462",
+  "_displayedCount": "50"
+}
+```
+
+Its DN answers nothing — `more-<the organization's DN>` for the attached
+entries, `more-organizations-<the organization's DN>` for the child
+organizations — and a client should drop any row carrying
+`_isMoreIndicator` before treating the rest as entries. `_totalCount` is
+present only when the directory listed everything and the cap did the
+trimming; when the directory itself refused, nothing knows the total.
+
+A refusal is not emptiness: a node whose children cannot be listed answers
+its first fifty and says there are more, and any other directory failure is
+raised rather than returned as `[]`. Raise the size limit for the account
+ldap-rest binds as (`olcLimits`, or `olcSizeLimit`) to see a whole branch.
+
 ### Create Organization
 
 ```http

package/docs/usage/plugins/utilities/storage.md

@@ -0,0 +1,94 @@
+# Storage Plugin
+
+Keyed storage for whoever needs to keep something with a deadline.
+
+## Overview
+
+`core/storage` gives other plugins a key, a value and an instant past which
+the value stops being returned. It knows nothing about why a record exists:
+Back-Channel Logout tombstones were the first use, a refresh-token lock and a
+shared counter are the ones in sight.
+
+One plugin, one backend, named by `--storage-backend`. Consumers ask this one
+for its store and work through a single interface; where the records actually
+live is a deployment decision, not theirs.
+
+Having exactly one is the point. The alternative — each consumer loading the
+backend it likes — puts the same record in two places, and makes whichever
+plugin was registered first decide what happens when one of them is unwell.
+Loading a second instance is refused rather than left to registration order,
+which is invisible in a configuration.
+
+## Configuration
+
+```bash
+--plugin core/storage --storage-backend ldap \
+  --storage-ldap-base ou=Records,dc=example,dc=com
+```
+
+| Option                        | Environment                    | Default              | Description                                               |
+| ----------------------------- | ------------------------------ | -------------------- | --------------------------------------------------------- |
+| `--storage-backend`           | `DM_STORAGE_BACKEND`           | _(none)_             | `ldap` or `file`. Empty means the plugin refuses to start |
+| `--storage-sweep-interval`    | `DM_STORAGE_SWEEP_INTERVAL`    | `600`                | Seconds between two passes dropping expired records       |
+| `--storage-ldap-base`         | `DM_STORAGE_LDAP_BASE`         | _(none)_             | Branch the `ldap` backend writes to                       |
+| `--storage-ldap-object-class` | `DM_STORAGE_LDAP_OBJECT_CLASS` | `applicationProcess` | Object class of the entries it writes                     |
+| `--storage-file-directory`    | `DM_STORAGE_FILE_DIRECTORY`    | _(none)_             | Directory the `file` backend writes to                    |
+
+`core/storage` is in the priority list, so it is registered before the
+plugins that consume it whatever order they are written in. A consumer that
+declares it as a dependency — `core/bcl` does — also loads it on its own when
+the configuration does not name it.
+
+## Backends
+
+### `ldap`
+
+One entry per record under `--storage-ldap-base`. The branch holds records
+and nothing else, so it belongs outside the branches the directory serves —
+the plugin does not create it, and will not write into a branch that is not
+there.
+
+Suited to a deployment that already replicates its directory and wants the
+records to follow, and to one where no other store is available.
+
+### `file`
+
+One file per record under `--storage-file-directory`. A write goes to a
+temporary file and is renamed into place, so a reader never sees half a
+record; the directory and its files are created `0700` and `0600`.
+
+The directory is the plugin's own: nothing else should be writing there, and
+the sweeper removes what it finds expired. It is local to one process, so it
+suits a single instance, not a cluster sharing one verdict.
+
+## Expiry
+
+Two rules the backends do not get to reinterpret:
+
+- **Expiry is enforced on read.** A record past its deadline is answered as
+  absent whatever the sweeper has done, so a sweeper running late is a
+  storage cost, never a wrong answer.
+- **The deadline belongs to the consumer.** How long a record lives is the
+  policy of whoever wrote it. The store is told an instant and respects it.
+
+Keys are namespaced per consumer, so two of them cannot collide in one branch
+or one directory.
+
+## For Plugin Authors
+
+```ts
+const storage = this.requirePlugin<Storage>('storage');
+await storage.store.set('myplugin', key, value, Date.now() + 3600_000);
+const kept = await storage.store.get('myplugin', key); // null once expired
+await storage.store.delete('myplugin', key);
+```
+
+`requirePlugin` warns and answers `null` when storage is absent, which is
+right for a feature that can be skipped. A plugin making a **security**
+decision must not read "no storage" as "nothing to check": it refuses to load
+instead, as [Back-Channel Logout](../auth/back-channel-logout.md) does.
+
+## See Also
+
+- [Back-Channel Logout](../auth/back-channel-logout.md) — the first consumer
+- [OpenID Connect](../auth/oidc.md)

package/docs/usage/troubleshooting.md

@@ -89,15 +89,7 @@ DM_LOG_LEVEL=debug ldap-rest
 
 ## Testing
 
-Run the test suite:
-
-```bash
-# All tests
-source ~/.test-env && npm run test:dev
-
-# Single file
-source ~/.test-env && npm run test:one test/path/to/file.test.ts
-```
+See [Tests and checks](../../CONTRIBUTING.md#tests-and-checks).
 
 ## Support
 

package/docs/usage/upgrading.md

@@ -4,6 +4,347 @@ What to check before deploying, newest first. Only releases that need a
 decision or a configuration change appear here; see the
 [CHANGELOG](../../CHANGELOG.md) for everything else.
 
+## To 0.10.0
+
+### LDAP base is now required
+
+**Who is affected:** a deployment that starts `ldap-rest` without
+`--ldap-base` (or `DM_LDAP_BASE`).
+
+The server refuses to start without it. It used to take the second RDN of
+`--ldap-dn` — `dc=example` for `cn=admin,dc=example,dc=com` — so subtree
+searches failed or found nothing. Set `--ldap-base` to the DN under which
+the entries are searched.
+
+### Group rules now apply
+
+**Who is affected:** a deployment whose `--authz-per-branch-config` declares
+a `groups` section.
+
+That section granted nothing: permissions came from `users` and `default`
+alone. A member of a configured group now also gets what its rule grants.
+**Check the `groups` rules before upgrading.**
+
+### An identity naming several entries is refused
+
+**Who is affected:** a directory where the same `uid` appears twice — one
+without a `unique` overlay, or one where a homonym was left in the trash
+while the uid was reused.
+
+`authzLinid1` refuses every request of such a caller with a `403`.
+`authzPerBranch` keeps the uid's `users` rules and withholds the `groups`
+rules. Both log a warning naming the identity and the number of entries it
+matched.
+
+**Check your directory before upgrading:**
+
+```bash
+ldapsearch -x -b "<base>" "(uid=<uid>)" dn
+```
+
+must answer one entry. Where it answers two, remove the duplicate — the
+trash included — or restore the uniqueness the `unique` overlay enforces.
+
+### `onLdapChange` gives full values
+
+**Who is affected:** plugins listening to `onLdapChange` that read the values
+of an `add`, or of a `delete` naming values, as the values added or removed.
+
+Each attribute now maps to all its values before and after the write. Adding
+`c` to `member: [a, b]` gave `[null, [c]]` and gives `[[a, b], [a, b, c]]`.
+Compute what was added or removed as the difference of the two sides, as the
+plugins shipped with LDAP-Rest do. A `replace` with the current value no
+longer fires the hook.
+
+## To 0.9.0
+
+### Two authorization plugins judging the same requests no longer start
+
+**Who is affected:** anyone loading two of `core/auth/authzPerBranch`,
+`core/auth/authzLinid1` and `core/auth/authzDynamic` together — including
+`authzPerBranch` beside `authzDynamic`, where the branch plugin judged the
+tokens too.
+
+Every plugin judging LDAP operations registers the same hooks, and all of
+them run: the first refusal wins. Loaded together, they composed as an AND
+nobody had decided, and the 403 did not say whose refusal it was. The server
+now refuses to start, naming both plugins. Two ways out:
+
+- **Scope each plugin to its own callers** with `authz_for`, the instance
+  names of the authentication plugins whose requests it judges — so that no
+  request is judged by both:
+
+  ```bash
+  --plugin core/auth/authzDynamic
+  --plugin 'core/auth/openidconnect:oidc:{"auth_path_prefix":"/api/admin"}'
+  --plugin 'core/auth/authzPerBranch:authzPerBranch:{"authz_for":["oidc"]}'
+  ```
+
+- **Keep the AND**, if it was meant, with `--authz-combine`. Behaviour is
+  then what it was, and the startup log says the two compose.
+
+`authzPerRoute` is not concerned: it registers no LDAP hook, and route plus
+branch stays a combination that starts as before.
+
+**Hosts assembling the server themselves** — `registerPlugin` after `ready`
+rather than `--plugin` — get the same refusal, out of the `registerPlugin`
+call that brings the second plugin. A plugin arriving is judged against what
+is already there: register the authentication plugins an `authz_for` names,
+and the plugins `--authz-scope-source` names, before the plugin naming them.
+
+A lone `authzPerBranch` is not refused, but mind its default: unset,
+`--authz-per-branch-config` grants read everywhere and write nowhere to
+**every** authenticated identity, whatever authenticated it. The startup
+line `authzPerBranch judges … for every authenticated request (…)` names the
+authenticators it judges; one it was not written for needs an `authz_for`.
+
+`GET /v1/authz/scope` changes in two ways. It answers with a plugin that
+judges the caller, lists every plugin judging them under `sources`, and says
+which one answered under `source` — `--authz-scope-source` chooses when
+several can. And when the only plugins judging the caller cannot describe a
+scope — `authzPerRoute`, `authzDynamic` — it answers `described: false` with
+no entity instead of `unrestricted: true` with `create: true` everywhere. A
+client that enumerates the entities it may create offers nothing there
+rather than everything. An anonymous caller is answered `401` as soon as any
+authorization plugin is loaded, where it used to be told `unrestricted`.
+
+### A second word after an option taking a list is refused
+
+**Who is affected:** anyone whose command line puts two values after one
+`--plugin`, `--authz-for`, `--mail-domain` or other list option.
+
+`--authz-for oidc authToken` read as `["oidc"]`: the second word was dropped
+in silence, and for `authz_for` that is a population smaller than the
+command line says — requests of `authToken` that nothing judges. The server
+now refuses to start on it, naming the option. Repeat the option for each
+value (`--authz-for oidc --authz-for authToken`), or use the plural form
+where there is one (`--plugins "a b"`), or the environment variable, which
+splits on spaces and commas.
+
+### `getLogger()` may return `undefined`, and its type says so
+
+**Who is affected:** anyone building a plugin or a host in TypeScript
+against the package's `expressformatedresponses` export and calling
+`getLogger()`.
+
+The logger is installed by the `DM` constructor, and before that there is
+none. The type used to say `winston.Logger`, so a reader without a guard
+compiled and threw `Cannot read properties of undefined` at runtime — from
+inside an error path, which is how #182 and #199 happened. It now says
+`winston.Logger | undefined`, and such a reader stops compiling:
+
+```ts
+getLogger().info('…'); // no longer compiles
+getLogger()?.info('…'); // logs when there is a logger
+```
+
+A plugin holds `this.logger`, which is always set; `getLogger()` is for a
+helper with no server in hand. `setLogger` accepts `undefined` as well, so a
+test restoring the state it found no longer needs a cast.
+
+### OpenID Connect honours `auth_path_prefix`
+
+**Who is affected:** anyone running `core/auth/openidconnect`, and in
+particular anyone who gave it an `auth_path_prefix`.
+
+The plugin did not go through the authentication dispatcher: it mounted its
+own middleware in `api()`, which kept its registration order, and it never
+read `auth_path_prefix`. Three consequences, all of them now gone:
+
+- **Scoping did nothing.** An instance scoped to `/api/admin` guarded the
+  whole server. It now guards `/api/admin` — and **the paths it used to
+  guard by accident are no longer guarded by it**. Check that another
+  authentication plugin covers them, or drop the prefix to keep guarding
+  everything. The startup line naming unguarded routes works again and is
+  the fastest way to tell.
+- **Authorization could run before the identity.** With
+  `core/auth/authzPerRoute` loaded, a named OIDC instance landed in the
+  parallel batch and mounted after it, so every rule was judged with no
+  `req.user` — and an unidentified request is passed, not refused. Rules
+  that read as enforced were inert.
+- **Two authentications composed as an AND.** A valid `Authorization:
+Bearer` answered by a catch-all token plugin was then redirected to the
+  provider anyway. Only the plugin whose claim is most specific runs now.
+
+`/login`, `/logout`, `/callback` and `/backchannel-logout` are claimed by
+the plugin whatever its scope, so a catch-all authentication no longer
+answers the provider's redirect with a 401.
+
+Unauthenticated requests are unchanged: a browser is redirected, an API
+client receives 401.
+
+### `core/auth/llng` refuses a request the handler named nobody for
+
+**Who is affected:** anyone whose LemonLDAP::NG configuration has a `skip`
+rule on a path this plugin guards.
+
+`skip` makes the handler return immediately — no session read, no
+`Lm-Remote-User` header — and the plugin published `req.user = undefined`.
+Every authorization plugin skips its check when a request carries no
+identity, so the rule meant "authenticated, and scoped by nobody" rather
+than "no authentication on this path". Such a request now answers 401.
+
+Serve those paths from a virtual host this plugin does not guard, or scope
+the plugin with `auth_path_prefix` so they fall outside it.
+
+### `authzDynamic` no longer steps aside for another authenticator
+
+**Who is affected:** anyone loading `core/auth/authzDynamic` beside another
+authentication plugin on the same paths — `core/auth/token`,
+`core/auth/openidconnect`, `core/auth/llng`.
+
+The plugin returned early as soon as a request already carried a `req.user`,
+so the token ACLs were applied to nothing. With `core/auth/token` registered
+first, its static tokens reached the whole directory as unscoped
+administrators; registered the other way round, the same configuration
+demanded both credentials. Which one a deployment got was plugin import
+order.
+
+The ACLs now apply to the token a request carries, whoever else identified
+it, and a request carrying none is refused. **A deployment relying on the
+old behaviour will start answering 401** — its callers authenticate with the
+other plugin and have no dynamic token.
+
+`--authz-dynamic-bypass` names what may pass without a token:
+
+```
+--authz-dynamic-bypass any-authenticated   # what the plugin did before 0.9.0
+--authz-dynamic-bypass ops-admin           # one identity, as another plugin publishes it
+--authz-dynamic-bypass trusted-proxy       # what core/auth/trustedProxy vouched for
+```
+
+`any-authenticated` restores the previous behaviour in one line. The
+difference is that it is now written down, logged per request with the
+reason, and announced at startup when this plugin shares its paths with
+another authenticator.
+
+`trusted-proxy` is the composition the old code's comment described and
+never performed: `core/auth/trustedProxy` sets `req.trustedProxy` and
+`req.proxyAuthUser`, never `req.user`, so that pair never took the early
+return — while every authenticator that does set it, did.
+
+### An identity that does not resolve is refused
+
+**Who is affected:** anyone running an authorization plugin that resolves
+identities against the directory — `core/auth/authzLinid1` — behind an
+authenticator that publishes something other than the directory's
+`ldap_user_main_attribute`.
+
+Every authorization hook skipped its check when `resolveUser` answered
+null: a `warn`, then the read, the write or the delete went ahead. With
+`authzLinid1`, whose `resolveUser` searches
+`(<ldap_user_main_attribute>=<identity>)`, an authenticator publishing
+anything else resolved to nothing on every request — `core/auth/openidconnect`
+publishes the OIDC `sub`, `core/auth/llng` whatever `whatToTrace` traces — so
+anyone the identity provider admitted read, wrote and deleted across the
+whole tree, behind a line a `notice` production log does not show.
+
+An authenticated identity that does not resolve is now refused, with the same
+403 as any other refusal. **Check your logs before upgrading**: a deployment
+in this state looks healthy, and the requests that were passing will start
+answering 403.
+
+```
+grep 'could not be resolved' <your log>
+```
+
+If that matches, the caller's identity and the plugin's model disagree, and
+the upgrade turns an open door into a refusal — which may be a service
+outage where the door was being used. `--authz-unresolved-user allow`
+restores the old behaviour while you fix the pairing; it keeps the warning,
+and it keeps the door open.
+
+An **anonymous** request is unchanged: no identity at all is still skipped,
+and the route's own authentication is what answers it.
+
+Both answers are cached now, negatives included, for the plugin's own cache
+TTL. An identity added to the directory after being refused waits out that
+TTL.
+
+### `/subnodes/search` caps what it returns
+
+**Who is affected:** a client reading the whole answer of
+`/api/v1/ldap/organizations/<dn>/subnodes/search` and expecting every match.
+
+It returned every attached entry it found, which is the one shape a directory
+with a size limit cannot answer: past that limit the server refuses the
+search rather than shortening it. It now caps them at
+`--ldap-organization-max-subnodes` (default 50) and ends the list with the
+`moreIndicator` row `/subnodes` already used, so the two endpoints answer the
+same shape.
+
+A client that treats every row as an entry will show that row as one. Drop
+whatever carries `_isMoreIndicator` — see
+[the endpoint's notes](plugins/ldap/organizations.md#get-organization-subnodes).
+
+The same row can now also appear among the **child organizations**, which
+were never capped and still are not: it says the directory refused to list
+them all, and it carries no `_totalCount`, nothing having counted them.
+
+### `/subnodes/search` is authorized, and lists the children by page
+
+**Who is affected:** anyone whose callers reach that route with a token that
+does not hold the whole directory, and anyone running against Active
+Directory.
+
+Neither of its searches carried the request, and an authorization plugin
+skips its check when there is none — the same gap the flat routes had until
+0.8.2. A caller now sees what its branch grants it, where it used to see
+everything the directory held; a caller outside its branch is refused. This
+is a fix, but it changes what an existing client receives.
+
+The child organizations of both routes are also searched by page now. On
+OpenLDAP that changes nothing — the server's size limit bounds a paged
+search as it does an unpaged one. On **Active Directory** it does: an
+unpaged search answered at most `MaxPageSize` entries (1000 by default) and
+said nothing about the rest, so a node with more children than that now
+returns all of them, and a client holding the answer in memory receives a
+larger one than before.
+
+### Back-Channel Logout keeps its marks in `core/storage`
+
+`core/bcl/ldap` and `core/bcl/file` are gone, and with them
+`--bcl-ldap-base`, `--bcl-ldap-object-class`, `--bcl-file-directory` and
+`--bcl-sweep-interval`. One plugin, `core/bcl`, keeps what a logout killed in
+`core/storage`, so a deployment chooses where records live once and every
+consumer of that store follows. Sweeping is now `--storage-sweep-interval`;
+`--bcl-retention` stays, because how long a mark is kept is Back-Channel
+Logout's policy rather than the store's. See
+[Storage](plugins/utilities/storage.md) and
+[Back-Channel Logout](plugins/auth/back-channel-logout.md).
+
+**Marks written before this are not read.** The key is namespaced now, and
+the LDAP branch is the store's rather than the plugin's. Back-Channel Logout
+shipped in no release — 0.8.2 predates it — so this can only affect an
+instance running a build from `master`. If yours is one, everything logged out
+within the retention window stops counting once you upgrade, and the old
+`ou=BclTombstones` branch, or the old `--bcl-file-directory`, is left behind
+for you to remove.
+
+### Judging an account by what it is attached to
+
+`--authz-filter-attached-entries` changes what a branch grant means, so it is
+off by default and nothing moves until you set it.
+
+Every account of a directory lives in the same `ou=users`, so a branch check
+on its parent says the same thing about all of them: either every
+administrator reaches every account, or none does. With the option on, an
+account is judged by the organization its `twakeDepartmentLink` names —
+listings drop the accounts attached elsewhere, and a write is refused on an
+account outside the branch you hold. Entries attached to nothing, which is
+what an organization or a nomenclature value is, stay readable by every
+administrator.
+
+**Turn it on only where a branch is a department.** Where a branch is a
+tenant, the option also stops refusing a read on the branch it targets — the
+organization tree and the reference data become readable across branches, and
+what keeps one customer out of another's data is then the per-entry filter
+alone.
+
+It is honoured by `core/auth/authzPerBranch` and `core/auth/authzLinid1`.
+`core/auth/authzDynamic` registers its own hooks and never sees the filter:
+setting the option there does nothing at all.
+
 ## To 0.8.0
 
 ### Node 20 is the floor

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ldap-rest",
-  "version": "0.8.2",
+  "version": "0.10.0",
   "description": "Light directory manager",
   "keywords": [
     "Identity-Manager"
@@ -92,6 +92,10 @@
       "types": "./dist/src/plugins/auth/trustedProxy.d.ts",
       "import": "./dist/plugins/auth/trustedProxy.js"
     },
+    "./plugin-bcl": {
+      "types": "./dist/src/plugins/bcl.d.ts",
+      "import": "./dist/plugins/bcl.js"
+    },
     "./plugin-configapi": {
       "types": "./dist/src/plugins/configApi.d.ts",
       "import": "./dist/plugins/configApi.js"
@@ -212,6 +216,18 @@
       "types": "./dist/src/plugins/static.d.ts",
       "import": "./dist/plugins/static.js"
     },
+    "./plugin-storage": {
+      "types": "./dist/src/plugins/storage.d.ts",
+      "import": "./dist/plugins/storage.js"
+    },
+    "./plugin-storage-file": {
+      "types": "./dist/src/plugins/storage/file.d.ts",
+      "import": "./dist/plugins/storage/file.js"
+    },
+    "./plugin-storage-ldap": {
+      "types": "./dist/src/plugins/storage/ldap.d.ts",
+      "import": "./dist/plugins/storage/ldap.js"
+    },
     "./plugin-twake-appaccountsapi": {
       "types": "./dist/src/plugins/twake/appAccountsApi.d.ts",
       "import": "./dist/plugins/twake/appAccountsApi.js"