matrix-nodejs-homeserver 0.2.0 → 0.3.2

package/README.md

@@ -1,11 +1,16 @@
 # matrix-nodejs-homeserver
 
+[![Load test: SQLite](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fk18k%2Fmatrix-nodejs-homeserver%2Floadtest-results%2Fbadges%2Fsqlite.json)](https://github.com/k18k/matrix-nodejs-homeserver/actions/workflows/loadtest.yml)
+[![Load test: Postgres](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fk18k%2Fmatrix-nodejs-homeserver%2Floadtest-results%2Fbadges%2Fpg-nocluster.json)](https://github.com/k18k/matrix-nodejs-homeserver/actions/workflows/loadtest.yml)
+[![Load test: Postgres cluster, 1 node](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fk18k%2Fmatrix-nodejs-homeserver%2Floadtest-results%2Fbadges%2Fpg-cluster-1.json)](https://github.com/k18k/matrix-nodejs-homeserver/actions/workflows/loadtest.yml)
+[![Load test: Postgres cluster, 5 nodes](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fk18k%2Fmatrix-nodejs-homeserver%2Floadtest-results%2Fbadges%2Fpg-cluster-5.json)](https://github.com/k18k/matrix-nodejs-homeserver/actions/workflows/loadtest.yml)
+
 A [Matrix](https://matrix.org) homeserver as an importable Node.js library. It isn't
 tied to any HTTP framework, database, auth system or deployment shape. You can use
 it for a demo on SQLite, embed it in a monolith next to your existing users, or run
 it as a horizontally scaled chat backend that replaces Slack or Teams.
 
-- Node.js **26+**. TypeScript with only erasable syntax (`erasableSyntaxOnly`); one runtime dependency, [zod](https://zod.dev), for request validation.
+- Node.js **22+** (tested on 22, 24 and 26; development uses 26). TypeScript with only erasable syntax (`erasableSyntaxOnly`), and no runtime dependencies ([zod](https://zod.dev), used for request validation, is bundled).
 - Client-Server API, Server-Server API (federation), end-to-end encryption, sliding sync, VoIP/MatrixRTC and push
 - Runs on `node:http`, Express, Fastify, Hono or any fetch runtime
 - Stores data in SQLite, PostgreSQL, MySQL, memory or your own adapter
@@ -75,12 +80,12 @@ By default a homeserver assumes it is the only process using its database. To ru
 several instances behind a load balancer, give them a shared **cluster adapter**:
 
 ```ts
-import { postgresCluster, redisCluster } from "matrix-nodejs-homeserver";
+import { postgresCluster } from "matrix-nodejs-homeserver";
 
 createHomeserver({
   homeserver: "example.com",
   databaseAdapter: postgresAdapter(pool),
-  cluster: postgresCluster(pool),          // or: redisCluster({ client: new Redis(), subscriber: new Redis() })
+  cluster: postgresCluster(pool),
 });
 ```
 
@@ -95,8 +100,7 @@ The cluster adapter provides:
 - **Shared ephemeral state:** UIA sessions, SSO logins and transaction deduplication.
 
 [examples/multi-node.ts](examples/multi-node.ts) is meant to run unchanged on several
-machines behind a load balancer, all sharing one Postgres database (and optionally Redis
-for coordination). It also shows a media adapter backed by that database, since local
+machines behind a load balancer, all sharing one Postgres database, which also coordinates them. It also shows a media adapter backed by that database, since local
 disks differ between machines.
 
 The database adapter has to support concurrent access from several processes, so
@@ -130,6 +134,9 @@ createHomeserver({
     clientId, clientSecret, mapUser: (claims) => ({ localpart: claims.email.split("@")[0] }) }] },
   publicBaseUrl: "https://matrix.example.com",
   ```
+  After signing in, users confirm the client the login token goes to, on a page
+  that names it. Clients you trust skip that page:
+  `sso: { providers, clientAllowlist: ["https://app.element.io/"] }`.
 - **Delegated OAuth 2.0** (the Matrix OAuth API, e.g. with
   matrix-authentication-service, Keycloak or Authentik). The provider owns the
   accounts, and the homeserver validates its access tokens by introspection:
@@ -139,8 +146,10 @@ createHomeserver({
 
 ## Federation
 
-Federation is on by default and needs no configuration beyond a reachable port
-8448, or `.well-known/matrix/server` delegation. The implementation includes:
+Federation is **off by default**, so a server only joins the Matrix network on purpose.
+Turn it on with `federation: { enabled: true }`. It then needs no configuration beyond a
+reachable port 8448, or `.well-known/matrix/server` delegation (`federationAddress`). The
+implementation includes:
 
 - Server discovery (`.well-known` and SRV) and signed requests
 - A persistent outbound queue with retries
@@ -155,7 +164,8 @@ Federation is on by default and needs no configuration beyond a reachable port
 federation: { enabled: true, allowedServers: ["partner.org"], blockedServers: ["spam.example"] },
 ```
 
-Set `federation: { enabled: false }` for a private deployment.
+Without federation, local users can only talk to each other: federation endpoints answer
+`M_UNRECOGNIZED`, and remote users, rooms and media cannot be reached.
 
 ## VoIP and MatrixRTC
 
@@ -302,9 +312,14 @@ await hs.admin.createUser("alice", { password: "…", admin: true });
 await hs.admin.createRegistrationToken({ usesAllowed: 10 });
 await hs.admin.deactivateUser("@spammer:example.com");
 await hs.admin.deleteRoom(roomId, { purge: true });
-await hs.admin.sendServerNotice("@alice:example.com", { msgtype: "m.text", body: "Maintenance tonight" });
+await hs.admin.sendServerNotice("@alice:example.com", { msgtype: M.TEXT, body: "Maintenance tonight" });
 ```
 
+`M` (exported from the package) holds the Matrix identifiers as one readonly object:
+`M.ROOM.MEMBER` is `"m.room.member"`, `M.LOGIN.PASSWORD` is `"m.login.password"`.
+`UNSTABLE` does the same for unstable and vendor identifiers:
+`UNSTABLE.ORG.MATRIX.CROSS_SIGNING_RESET` is `"org.matrix.cross_signing_reset"`.
+
 The same operations are available over HTTP at Synapse-compatible `/_synapse/admin/…`
 paths (server admins only), so existing tools keep working.
 
@@ -319,7 +334,8 @@ paths (server admins only), so existing tools keep working.
 | `audit` | `{ enabled, database, sink, actions, retentionDays }`. Disabled by default. |
 | `publicBaseUrl` | Advertised via `/.well-known/matrix/client`. Required for SSO. |
 | `oauth`, `sso` | Delegated OAuth or upstream OIDC SSO. |
-| `federation` | `{ enabled, verifyTls, allowedServers, blockedServers }` |
+| `federation` | `{ enabled, verifyTls, allowedServers, blockedServers, fasterJoins }`. Off unless `enabled: true`. |
+| `allowPrivateAddresses` | Lets requests to addresses that users or remote servers choose (federation, push gateways, identity servers, URL previews, remote media) reach private addresses: `true`, or ranges such as `["127.0.0.1/32"]`. Blocked by default. |
 | `rtc` | `{ turn, transports }` |
 | `delayedEvents` | `{ maxDelayMs, maxPerUser }` |
 | `rateLimits` | `false`, or `{ limiter, rules }`. In-memory token buckets by default. Plug in a shared limiter when running a cluster. |
@@ -363,12 +379,26 @@ npm run build                    # tsdown → dist/ (what the published package
 node examples/node-http.ts       # development runs the TypeScript sources directly (type stripping)
 ```
 
+Endpoints are procedures (`src/http/procedure.ts`): a route declares what it needs and a
+handler, and the procedure checks the request in a fixed order (authentication, rate limit,
+body, then the response) before and after the handler runs:
+
+```ts
+r.client("POST", "/rooms/:roomId/invite", procedure({
+  auth: "user",                 // ctx.session (or "optional", "none")
+  rateLimit: "invite",
+  input: body({ user_id: lenient(z.string()), reason: lenient(z.string()) }), // ctx.input
+  handler: async (ctx) => { /* ctx.hs, ctx.session, ctx.input, ctx.params */ },
+}));
+```
+
 Inside `src`, modules import each other through the package's
 [subpath imports](https://nodejs.org/api/packages.html#subpath-imports): `#core/errors.ts` is
 `src/core/errors.ts`. The build turns them back into relative imports.
 
 Node does not strip types from files inside `node_modules`, so the package ships `dist/`,
-built by tsdown: one `.js` and one `.d.ts` per source module. `npm pack`, `npm publish` and
+built by tsdown: one `.js` and one `.d.ts` per source module, with zod bundled under
+`dist/vendor/`. `npm pack`, `npm publish` and
 installs from GitHub build it through the `prepare` script. Recent npm versions ask you to
 approve install scripts of git dependencies (`npm install-scripts approve`).
 
@@ -429,6 +459,7 @@ Live status, updated after every Complement run (full or targeted) with
 
 | Run | Top-level | Subtests | Note |
 |---|---|---|---|
+| 9 | 214 / 214 | 851 / 851 | zod, procedures, M/UNSTABLE, SSO confirmation, outbound address policy, admin guard (ec5aa79) |
 | 8 | 214 / 214 | 851 / 851 | Complement runs in a Go container (no submodule, no local Go); account locking/suspension, policy servers, 3PIDs, legacy sync endpoints |
 | 7 | 214 / 214 | 851 / 851 | All Complement tests pass (previous full run: 211/214). Fallback keys stored apart from one-time keys; user directory public-room rules; MSC4311 create event required from v12 |
 | 6 | 189 / 214 | 803 / 851 | Partial-state joins (MSC3902): all subtests pass |
@@ -449,6 +480,53 @@ Live status, updated after every Complement run (full or targeted) with
 CI runs the whole suite with the same script. `complement/passing.txt` lists the tests
 (and subtests) that are known to pass, and CI fails if any of them regress.
 
+### Load testing
+
+Tuning and investigation runs are local; a [workflow](.github/workflows/loadtest.yml) runs the same scenarios on main for the badges. [loadtest/](loadtest/) holds a k6 scenario and a
+homeserver that reports, every 10 s, requests per second, memory, event-loop delay and
+(with Postgres) how many queries wait for a pool connection and how long locks are held.
+
+The scenario: `ONLINE_USERS` each hold a long-polling `/sync` open in one of `ROOMS` rooms,
+while `MESSAGES_PER_SECOND` messages are sent in total. Every user registers and joins
+through the client API. It measures each request type and the time from sending a message to
+its arrival in other users' `/sync` (`message_delivery`).
+
+```sh
+# Postgres with N homeserver nodes behind nginx (round-robin, no sticky sessions)
+ONLINE_USERS=1000 MESSAGES_PER_SECOND=50 ROOMS=25 RAMP=30s DURATION=30s \
+  docker compose -p matrix-loadtest -f loadtest/docker-compose.yml up --build \
+  --scale homeserver=5 --abort-on-container-exit --exit-code-from k6 k6
+# CLUSTER=0: one process on Postgres without a cluster adapter
+
+# SQLite, one process, no Docker (needs k6 installed)
+DATA_DIR=/tmp/lt node loadtest/server.ts &
+BASE_URL=http://localhost:8008 ONLINE_USERS=1000 MESSAGES_PER_SECOND=50 ROOMS=25 k6 run loadtest/k6/matrix.js
+```
+
+<!-- loadtest-results:start -->
+The badges at the top come from the [Load test workflow](https://github.com/k18k/matrix-nodejs-homeserver/actions/workflows/loadtest.yml) on main: the same
+scenario on one GitHub runner (4 CPUs), with k6 on the same machine. Each run's job summary has
+the full table.
+
+Latest results: 1000 users online in 25 rooms (40 per room), 50 messages/s, 30 s ramp-up then
+30 s at full load, on one developer machine (WSL2, Docker Desktop). Times are p50 / p95.
+
+| Setup | req/s | send | /sync | join | delivery | messages delivered | failed |
+|---|---|---|---|---|---|---|---|
+| SQLite, 1 process (before #28–#35) | ~800 | 46 ms / 0.69 s | 0.50 s / 2.4 s | 61 ms / 0.15 s | 0.58 s / 0.78 s | 5,816 | 0% |
+| Postgres, no cluster | 404 | 49 ms / 71 ms | 2.6 s / 3.6 s | 18 ms / 27 ms | 4.6 s / 6.5 s | 56,297 | 0% |
+| Postgres, cluster, 1 node | 314 | 81 ms / 124 ms | 2.8 s / 4.4 s | 50 ms / 70 ms | 5.2 s / 7.5 s | 52,071 | 0% |
+| Postgres, cluster, 5 nodes | 840 | 118 ms / 197 ms | 0.90 s / 1.4 s | 86 ms / 151 ms | 1.6 s / 2.3 s | 58,298 | 0% |
+
+What the load tests found and fixed so far: presence written on every `/sync` (#26), one commit
+per push notification (#27), a missing index (#28), Postgres cluster locks that could deadlock
+the pool (#29), stream writes serialized per instance (#30), `/sync` reading every change on the
+server (#31) and every joined room (#33), lock handover by polling (#32). Then `/sync` reads were batched
+(#34) and a process builds at most four sync responses at once (#35), so sends no longer queue
+behind the syncs a message wakes. A Postgres node now builds ~330 syncs/s here; delivery time is
+syncs waiting for their turn, and more nodes shorten it.
+<!-- loadtest-results:end -->
+
 ## License
 
 [PolyForm Noncommercial 1.0.0](LICENSE). Free to use, modify and share for any noncommercial

package/dist/adapters/cluster.d.ts

@@ -19,37 +19,12 @@ export interface PgClusterPoolLike {
   connect(): Promise<PgClientLike>;
 }
 /**
- * Cluster adapter backed by PostgreSQL (`pg` Pool): sequences, advisory locks,
+ * Cluster adapter backed by PostgreSQL (`pg` Pool): sequences, leased locks,
  * LISTEN/NOTIFY pub/sub and an expiring key-value table. Lets several homeserver
  * instances share one database.
  */
 export declare function postgresCluster(pool: PgClusterPoolLike, options?: {
   tablePrefix?: string;
 }): ClusterAdapter;
-/** Minimal ioredis-compatible client. */
-export interface RedisLike {
-  incr(key: string): Promise<number>;
-  get(key: string): Promise<string | null>;
-  set(key: string, value: string, ...args: (string | number)[]): Promise<string | null>;
-  del(...keys: string[]): Promise<number>;
-  eval(script: string, numKeys: number, ...args: (string | number)[]): Promise<unknown>;
-  publish(channel: string, message: string): Promise<number>;
-  scan(cursor: string | number, ...args: (string | number)[]): Promise<[string, string[]]>;
-  mget(...keys: string[]): Promise<(string | null)[]>;
-}
-/** A Redis connection in subscriber mode (a separate connection, as Redis requires). */
-export interface RedisSubscriberLike {
-  subscribe(...channels: string[]): Promise<unknown>;
-  on(event: "message", listener: (channel: string, message: string) => void): unknown;
-}
-/**
- * Cluster adapter backed by Redis (ioredis API). Pass two connections: one for
- * commands and one dedicated to subscriptions.
- */
-export declare function redisCluster(opts: {
-  client: RedisLike;
-  subscriber: RedisSubscriberLike;
-  prefix?: string;
-}): ClusterAdapter;
 //#endregion
 export { type ClusterAdapter, localCluster };

package/dist/adapters/cluster.js

@@ -16,9 +16,13 @@ function localQueue() {
 	};
 }
 const PG_CHANNEL = "matrix_cluster";
+/** Internal pub/sub channel announcing released locks. */
+const RELEASED = "__lock_released";
+/** How long a lock lease lasts without renewal (it is renewed every third of that while held). */
+const LOCK_TTL_MS = 3e4;
 const MAX_NOTIFY_PAYLOAD = 7e3;
 /**
-* Cluster adapter backed by PostgreSQL (`pg` Pool): sequences, advisory locks,
+* Cluster adapter backed by PostgreSQL (`pg` Pool): sequences, leased locks,
 * LISTEN/NOTIFY pub/sub and an expiring key-value table. Lets several homeserver
 * instances share one database.
 */
@@ -27,6 +31,20 @@ function postgresCluster(pool, options = {}) {
 	const kv = `"${options.tablePrefix ?? ""}cluster_kv"`;
 	const handlers = /* @__PURE__ */ new Map();
 	const queue = localQueue();
+	const waiting = /* @__PURE__ */ new Map();
+	const releasedOrAfter = (lockKey, ms) => new Promise((resolve) => {
+		let set = waiting.get(lockKey);
+		if (!set) waiting.set(lockKey, set = /* @__PURE__ */ new Set());
+		const wake = () => {
+			clearTimeout(timer);
+			set.delete(wake);
+			if (set.size === 0 && waiting.get(lockKey) === set) waiting.delete(lockKey);
+			resolve();
+		};
+		const timer = setTimeout(wake, ms);
+		set.add(wake);
+	});
+	handlers.set(RELEASED, /* @__PURE__ */ new Set([(lockKey) => [...waiting.get(lockKey) ?? []].forEach((wake) => wake())]));
 	let listener = null;
 	let cleanup = null;
 	const ready = (async () => {
@@ -80,16 +98,38 @@ function postgresCluster(pool, options = {}) {
 		lock(key, fn) {
 			return queue(key, async () => {
 				await ready;
-				const client = await pool.connect();
+				const lockKey = `lock:${key}`;
+				const token = randomUUID();
+				const acquire = async () => {
+					const now = Date.now();
+					return ((await pool.query(`INSERT INTO ${kv} (key, value, expires_at) VALUES ($1, $2, $3)
+             ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, expires_at = EXCLUDED.expires_at
+             WHERE ${kv}.expires_at IS NOT NULL AND ${kv}.expires_at <= $4 RETURNING key`, [
+						lockKey,
+						token,
+						now + LOCK_TTL_MS,
+						now
+					])).rowCount ?? 0) > 0;
+				};
+				for (let delay = 5; !await acquire(); delay = Math.min(delay * 2, 200)) await releasedOrAfter(lockKey, delay);
+				const renew = setInterval(() => void pool.query(`UPDATE ${kv} SET expires_at = $3 WHERE key = $1 AND value = $2`, [
+					lockKey,
+					token,
+					Date.now() + LOCK_TTL_MS
+				]).catch(() => {}), LOCK_TTL_MS / 3);
 				try {
-					await client.query("SELECT pg_advisory_lock(hashtextextended($1, 0))", [key]);
-					try {
-						return await fn();
-					} finally {
-						await client.query("SELECT pg_advisory_unlock(hashtextextended($1, 0))", [key]);
-					}
+					return await fn();
 				} finally {
-					client.release();
+					clearInterval(renew);
+					await pool.query(`WITH released AS (DELETE FROM ${kv} WHERE key = $1 AND value = $2 RETURNING key) SELECT pg_notify($3, $4) FROM released`, [
+						lockKey,
+						token,
+						PG_CHANNEL,
+						JSON.stringify({
+							channel: RELEASED,
+							message: lockKey
+						})
+					]).catch(() => {});
 				}
 			});
 		},
@@ -152,98 +192,5 @@ function postgresCluster(pool, options = {}) {
 		}
 	};
 }
-const UNLOCK = `if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end`;
-const EXTEND = `if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("pexpire", KEYS[1], ARGV[2]) else return 0 end`;
-/**
-* Cluster adapter backed by Redis (ioredis API). Pass two connections: one for
-* commands and one dedicated to subscriptions.
-*/
-function redisCluster(opts) {
-	const { client, subscriber } = opts;
-	const prefix = opts.prefix ?? "matrix:";
-	const channel = `${prefix}cluster`;
-	const handlers = /* @__PURE__ */ new Map();
-	const queue = localQueue();
-	const ready = subscriber.subscribe(channel);
-	subscriber.on("message", (ch, raw) => {
-		if (ch !== channel) return;
-		try {
-			const { channel: name, message } = JSON.parse(raw);
-			for (const h of handlers.get(name) ?? []) h(message);
-		} catch {}
-	});
-	const k = (key) => `${prefix}${key}`;
-	return {
-		async nextId(name) {
-			return Number(await client.incr(k(`seq:${name}`)));
-		},
-		async currentId(name) {
-			return Number(await client.get(k(`seq:${name}`)) ?? 0);
-		},
-		async advance(name, value) {
-			await client.eval(`local v = tonumber(redis.call("get", KEYS[1]) or "0") if v < tonumber(ARGV[1]) then redis.call("set", KEYS[1], ARGV[1]) end return 1`, 1, k(`seq:${name}`), value);
-		},
-		lock(key, fn) {
-			return queue(key, async () => {
-				const lockKey = k(`lock:${key}`);
-				const token = randomUUID();
-				const ttl = 3e4;
-				for (let delay = 5;; delay = Math.min(delay * 2, 200)) {
-					if (await client.set(lockKey, token, "PX", ttl, "NX") === "OK") break;
-					await new Promise((r) => setTimeout(r, delay));
-				}
-				const renew = setInterval(() => void client.eval(EXTEND, 1, lockKey, token, ttl).catch(() => {}), ttl / 3);
-				try {
-					return await fn();
-				} finally {
-					clearInterval(renew);
-					await client.eval(UNLOCK, 1, lockKey, token).catch(() => {});
-				}
-			});
-		},
-		async publish(name, message) {
-			await ready;
-			await client.publish(channel, JSON.stringify({
-				channel: name,
-				message
-			}));
-		},
-		subscribe(name, handler) {
-			let set = handlers.get(name);
-			if (!set) handlers.set(name, set = /* @__PURE__ */ new Set());
-			set.add(handler);
-		},
-		async get(key) {
-			const v = await client.get(k(`kv:${key}`));
-			return v === null ? null : JSON.parse(v);
-		},
-		async set(key, value, ttlMs) {
-			if (ttlMs) await client.set(k(`kv:${key}`), JSON.stringify(value), "PX", ttlMs);
-			else await client.set(k(`kv:${key}`), JSON.stringify(value));
-		},
-		async setIfAbsent(key, value, ttlMs) {
-			return (ttlMs ? await client.set(k(`kv:${key}`), JSON.stringify(value), "PX", ttlMs, "NX") : await client.set(k(`kv:${key}`), JSON.stringify(value), "NX")) === "OK";
-		},
-		async delete(key) {
-			await client.del(k(`kv:${key}`));
-		},
-		async entries(keyPrefix) {
-			const out = [];
-			let cursor = "0";
-			const pattern = `${k(`kv:${keyPrefix}`).replace(/[*?[\]]/g, (c) => `\\${c}`)}*`;
-			do {
-				const [next, keys] = await client.scan(cursor, "MATCH", pattern, "COUNT", 200);
-				cursor = next;
-				if (keys.length) {
-					const values = await client.mget(...keys);
-					keys.forEach((key, i) => {
-						if (values[i] !== null) out.push([key.slice(k("kv:").length), JSON.parse(values[i])]);
-					});
-				}
-			} while (cursor !== "0");
-			return out;
-		}
-	};
-}
 //#endregion
-export { localCluster, postgresCluster, redisCluster };
+export { localCluster, postgresCluster };