tickcli-js 0.1.4 → 0.1.6

package/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.1.6
+
+Blue/green hand-over of one namespace. `hub.exportScope(scope)` moves one namespace's live rooms to another process without touching the others: pending changes go out, open actions fail ("server restarting"), profile saves still loading land first, saves are written and in the store before it resolves, the rooms are exported (`exportRooms(scope)` now takes an optional namespace) and closed here (save leases given up), and its sockets hear 1012. Joins to it meanwhile are refused with 1012. The other process passes the snapshot to `importRooms` and the players carry on in the same match (`kept`). A router in front can re-join each page over its open socket; a page on its own reconnects as after a restart. `hub.scopeList()`: namespaces with live rooms and their rooms, seats and online players. `createRemoteServer({ onSnapshot })`: a restart of a remote room server keeps its matches.
+
+## 0.1.5
+
+Per-customer shares on a shared server. A deployment over its `kbPerSec` cap no longer only waits at the door: all its open rooms send less often (half, down to an eighth) until it is back under 80% of the cap, so one busy customer cannot take the whole uplink; other deployments keep their full rate and nobody is disconnected. New deployment cap `joinsPerMinute`: joins a minute across the deployment (reconnects count), on top of the per-address limit; over it the page hears 1013 busy and tries again. Hub: `scopeCaps` option (`{ outBytes, joinsPerMinute }` per namespace), `remote.scopeCaps(scope)`, `scopeStats(scope).slowed`, `stats().live.slowedScopes`, refusal reasons `scope_budget` and `scope_joins`.
+
 ## 0.1.4
 
 Idle pages send less. A visible page with no key, pointer, touch or gamepad input for 1 min asks the hub for a quarter of the update frames (nothing is lost, changes are merged into each frame); the first input brings the full rate back. In a 4-player test room the idle reader's download fell from 4.4 to 1.2 KB/s. Hidden pages already got no frames.

package/README.md

@@ -248,17 +248,17 @@ import { createRemoteServer } from "tickcli-js/remote";
 
 const deployments = {
   // "http://localhost:*" lets the author test the same page from a laptop; drop it whenever you like
-  dep_ab12: { id: "dep_ab12", origins: ["https://mygame.example", "https://*.portal.example", "http://localhost:*"], caps: { rooms: 20, players: 200, kbPerSec: 2048 } },
+  dep_ab12: { id: "dep_ab12", origins: ["https://mygame.example", "https://*.portal.example", "http://localhost:*"], caps: { rooms: 20, players: 200, kbPerSec: 2048, joinsPerMinute: 600 } },
 };
 const app = createRemoteServer({ deployment: (id) => deployments[id], secret: process.env.TICKCLI_SECRET, saves: "./saves" });
 await app.listen(8080);   // behind https: the pages use wss://
 ```
 
-Or in your own server: `createRemote({ hub, deployment })` gives `upgrade(req, socket, head)` and `request(req, res)` (pass `roomsPerScope: (s) => remote.roomsPerScope(s)` to the hub for room caps).
+Or in your own server: `createRemote({ hub, deployment })` gives `upgrade(req, socket, head)` and `request(req, res)` (pass `roomsPerScope: (s) => remote.roomsPerScope(s)` and `scopeCaps: (s) => remote.scopeCaps(s)` to the hub for the room, data and joins caps).
 
 - A deployment is a public id plus the sites allowed to use it: `https://site`, one wildcard level `https://*.example`, any port `:*`. The server checks the page's `Origin` when the socket opens: browsers send it and page scripts cannot change it, so another site cannot use your deployment. There is one kind of deployment: https sites, and http only for this machine (`localhost`, `127.0.0.1`, `[::1]`) when listed, e.g. `http://localhost:*` to test from a laptop. Pages with no site (`null`, `file://`) are refused.
 - Guests: the page keeps a random 128-bit key in its own `localStorage` and sends it as the socket's first message (no cookie, so it works from any site). The same key is the same seat; other players see a short id made from it, never the key.
-- Each deployment is its own namespace (rooms and saves never mix). `caps`: open rooms, connected sockets, KB a second going out (over it, new sockets are told the game is busy and retry; open rooms carry on). `revoked: true` refuses new sockets; `remote.revoke(id)` also closes its open rooms.
+- Each deployment is its own namespace (rooms and saves never mix). `caps`: open rooms, connected sockets, KB a second going out (over it, new sockets are told the game is busy and retry, and its open rooms send less often, down to an eighth, until it is back under 80%; other deployments keep their full rate), joins a minute across the deployment (reconnects count: leave room for every player joining again after a restart). `revoked: true` refuses new sockets; `remote.revoke(id)` also closes its open rooms.
 - A refused page stops and shows why ("This site is not allowed for deployment dep_ab12.").
 - Not a guard: a script outside a browser can send any `Origin`. The caps, the per-address limits and revoking the id are what stop a misused deployment.
 

package/dist/hub.d.ts

@@ -367,6 +367,8 @@ export interface LiveStats {
     /** over the data cap or a late loop now; rooms slowed by their own budget */
     pressure: boolean;
     slowedRooms: number;
+    /** namespaces slowed because they are over their own outBytes cap (ScopeCaps) */
+    slowedScopes: number;
     /** the 5 rooms sending the most */
     top: {
         players: number;
@@ -421,6 +423,14 @@ export interface RoomsSnapshot {
     at: number;
     rooms: SnapRoom[];
 }
+/** One namespace's own share of the server (a host app's per-customer caps, tickcli-js/remote). */
+export interface ScopeCaps {
+    /** data going out to its rooms, bytes a second (moving average over all its rooms): over it, all its rooms send half as often (up to 8 times
+     *  less) until the faster rate would be under 80% of it again. Other namespaces keep their full rate. */
+    outBytes?: number;
+    /** joins a minute across the namespace (reconnects count), on top of the per-address limit; over it: 1013 busy, the page tries again */
+    joinsPerMinute?: number;
+}
 export interface HubStats {
     rooms: number;
     players: number;
@@ -441,6 +451,10 @@ export declare class RoomHub {
     private perAddr;
     private addrHits;
     private joins;
+    /** per namespace with caps: its slow-down factor and moving average out (bytes a second) */
+    private scopeSlow;
+    /** per namespace with a joins cap: joins in this minute and the last one (a sliding estimate, no list per join) */
+    private scopeJoins;
     private refused;
     private timer;
     private sub;
@@ -478,6 +492,8 @@ export declare class RoomHub {
     pressure: boolean;
     /** world save leases: "scope|slot|who" -> the room key that may write it */
     private leases;
+    /** namespaces being handed to another process now (exportScope): joins to them are refused with 1012 */
+    private moving;
     /** shutdown() hands the live rooms to this (a restart writes them to disk; importRooms() in the next process takes them back) */
     private readonly onSnapshot;
     /** the room list (rooms {}) is answered; off: the answer is {off: true} */
@@ -486,6 +502,8 @@ export declare class RoomHub {
     private lists;
     /** rooms allowed in one namespace when it differs from limits.maxRoomsPerScope (a host app's per-customer cap, tickcli-js/remote) */
     private readonly roomsPerScope;
+    /** a namespace's own data and joins caps (tickcli-js/remote: a deployment's caps); undefined = only the server-wide limits */
+    private readonly scopeCaps;
     constructor(opts: {
         secret: string;
         now?: () => number;
@@ -494,6 +512,7 @@ export declare class RoomHub {
         onSnapshot?: (snap: RoomsSnapshot) => Promise<void> | void;
         listRooms?: boolean;
         roomsPerScope?: (scope: string) => number | undefined;
+        scopeCaps?: (scope: string) => ScopeCaps | undefined;
     });
     /** A short public id for a seat in one namespace; the seat key itself (a user id or guest pass id) is never shown to others. */
     publicId(scope: string, seat: string): string;
@@ -596,9 +615,15 @@ export declare class RoomHub {
     /** A room over its data budget sends half as often (up to 8 times less), its host is told once (error too_much_data); it speeds up again when
      *  the faster rate would be under 80% of the budget. */
     private budget;
-    /** How many ticks apart a room sends its state: 1 (50 ms), 2 in rooms over 8 players, times its own budget factor, times 2 under server pressure
-     *  for rooms over 4 players. */
+    /** How many ticks apart a room sends its state: 1 (50 ms), 2 in rooms over 8 players, times its own budget factor, times its namespace's factor
+     *  (over its outBytes cap), times 2 under server pressure for rooms over 4 players. */
     private every;
+    private scopeF;
+    /** Once a second: each namespace with an outBytes cap over its share slows all its rooms (twice as slow per second over, up to 8 times), and
+     *  speeds up one step when the faster rate would be under 80% of the cap. A busy customer then costs its own players smoothness, not everyone's. */
+    private scopeBudgets;
+    /** One more join in a namespace with a joinsPerMinute cap, counted over a sliding minute (this minute plus the share of the last one still in it). */
+    private scopeJoinAllowed;
     /** Once a second: the rates over the last second (server and per room). */
     private meters;
     /** Everything but the room sends; `sendAll` false when subTick sends rooms on their own phase. */
@@ -614,8 +639,21 @@ export declare class RoomHub {
     /** Server shutdown: pending changes and saves are written, the live rooms go to onSnapshot (the next process takes them back), then every player
      *  hears 1012 and reconnects. Without onSnapshot a restart ends every match. */
     shutdown(): Promise<void>;
-    /** The live rooms as plain JSON: everything a new process needs to carry on (sockets, timers and open actions are not kept). */
-    exportRooms(): RoomsSnapshot;
+    /** Blue/green hand-over of one namespace to another process: its pending changes go out, open actions fail ("server restarting"), saves are
+     *  written, its rooms are exported, its sockets hear 1012 (a router in front re-joins them on the other process over the same page socket; a
+     *  page on its own reconnects) and its rooms close here, giving up their save leases. Joins to it are refused with 1012 meanwhile. Resolves
+     *  once every save is in the store, so the other process may take the snapshot (importRooms) and write the same saves. */
+    exportScope(scope: string): Promise<RoomsSnapshot>;
+    /** Namespaces with live rooms here and their size (a router's hand-over list, a dashboard). */
+    scopeList(): {
+        scope: string;
+        rooms: number;
+        players: number;
+        online: number;
+    }[];
+    /** The live rooms (of one namespace when given) as plain JSON: everything a new process needs to carry on (sockets, timers and open actions
+     *  are not kept). */
+    exportRooms(scope?: string): RoomsSnapshot;
     /** Take back rooms from exportRooms() after a restart: every seat offline (held 60 s for its player to come back), no host until one is back, a new
      *  term, save leases taken again. A snapshot older than maxAgeMs or of another format is ignored. Returns how many rooms came back. */
     importRooms(snap: unknown, o?: {
@@ -629,11 +667,13 @@ export declare class RoomHub {
     /** Close every room of one namespace (a public link stopped). */
     closeScope(scope: string, reason?: string): void;
     /** One namespace's use right now: rooms, seats, connected players and bytes out a second (over the last second). For per-customer caps. */
+    /** `slowed`: how many times less often its rooms send because the namespace is over its own outBytes cap (1 = full rate). */
     scopeStats(scope: string): {
         rooms: number;
         players: number;
         online: number;
         bytesOut: number;
+        slowed: number;
     };
     stats(): HubStats;
 }

package/dist/hub.js

@@ -350,6 +350,10 @@ export class RoomHub {
     perAddr = new Map();
     addrHits = new Map();
     joins = new Map();
+    /** per namespace with caps: its slow-down factor and moving average out (bytes a second) */
+    scopeSlow = new Map();
+    /** per namespace with a joins cap: joins in this minute and the last one (a sliding estimate, no list per join) */
+    scopeJoins = new Map();
     refused = {};
     timer = null;
     sub = 0;
@@ -381,6 +385,8 @@ export class RoomHub {
     pressure = false;
     /** world save leases: "scope|slot|who" -> the room key that may write it */
     leases = new Map();
+    /** namespaces being handed to another process now (exportScope): joins to them are refused with 1012 */
+    moving = new Set();
     /** shutdown() hands the live rooms to this (a restart writes them to disk; importRooms() in the next process takes them back) */
     onSnapshot;
     /** the room list (rooms {}) is answered; off: the answer is {off: true} */
@@ -389,8 +395,11 @@ export class RoomHub {
     lists = new Map();
     /** rooms allowed in one namespace when it differs from limits.maxRoomsPerScope (a host app's per-customer cap, tickcli-js/remote) */
     roomsPerScope;
+    /** a namespace's own data and joins caps (tickcli-js/remote: a deployment's caps); undefined = only the server-wide limits */
+    scopeCaps;
     constructor(opts) {
         this.roomsPerScope = opts.roomsPerScope ?? null;
+        this.scopeCaps = opts.scopeCaps ?? null;
         this.saves = opts.saves ?? null;
         this.listRooms = opts.listRooms !== false;
         this.onSnapshot = opts.onSnapshot ?? null;
@@ -575,6 +584,8 @@ export class RoomHub {
         const L = this.limits, now = this.now();
         if (s.room)
             return this.bad(s, "already in a room");
+        if (this.moving.has(s.scope))
+            return s.end(CLOSE.restart, "moving to another server");
         const spec = parseSpec(m.spec, L.maxPlayers);
         if (!spec)
             return this.bad(s, "bad multiplayer option");
@@ -589,6 +600,10 @@ export class RoomHub {
             for (const [k, v] of this.joins)
                 if (!v.length || now - v[v.length - 1] > 60_000)
                     this.joins.delete(k);
+        if (!this.scopeJoinAllowed(s.scope, now)) {
+            this.refuse("scope_joins");
+            return s.end(CLOSE.busy, "this game is busy right now, trying again");
+        }
         const who = s.who;
         if (who.guest && typeof m.name === "string") {
             const n = cleanName(m.name);
@@ -1439,7 +1454,7 @@ export class RoomHub {
                     continue;
                 const h = r.host?.session;
                 rooms.push({ _r: r, code: r.code, host: r.host?.name ?? null, players: on, away, max: r.spec.max, started: r.started, match: r.match,
-                    ping: h?.rtt ?? 0, slowed: r.slowF > 1 || (this.pressure && r.seats.size > 4), full: taken >= r.spec.max, spec: r.specKey, created: r.created });
+                    ping: h?.rtt ?? 0, slowed: r.slowF > 1 || (this.pressure && r.seats.size > 4) || this.scopeF(r.scope) > 1, full: taken >= r.spec.max, spec: r.specKey, created: r.created });
             }
             rooms.sort((a, b) => Number(a.full) - Number(b.full) || Number(a.started) - Number(b.started) || b.players - a.players || b.created - a.created);
             if (this.lists.size > 1000)
@@ -1483,9 +1498,87 @@ export class RoomHub {
             r.outAvg *= 2;
         }
     }
-    /** How many ticks apart a room sends its state: 1 (50 ms), 2 in rooms over 8 players, times its own budget factor, times 2 under server pressure
-     *  for rooms over 4 players. */
-    every(r) { return (r.seats.size > 8 ? 2 : 1) * r.slowF * (this.pressure && r.seats.size > 4 ? 2 : 1); }
+    /** How many ticks apart a room sends its state: 1 (50 ms), 2 in rooms over 8 players, times its own budget factor, times its namespace's factor
+     *  (over its outBytes cap), times 2 under server pressure for rooms over 4 players. */
+    every(r) { return (r.seats.size > 8 ? 2 : 1) * r.slowF * this.scopeF(r.scope) * (this.pressure && r.seats.size > 4 ? 2 : 1); }
+    scopeF(scope) { return this.scopeSlow.get(scope)?.f ?? 1; }
+    /** Once a second: each namespace with an outBytes cap over its share slows all its rooms (twice as slow per second over, up to 8 times), and
+     *  speeds up one step when the faster rate would be under 80% of the cap. A busy customer then costs its own players smoothness, not everyone's. */
+    scopeBudgets() {
+        if (!this.scopeCaps)
+            return;
+        const out = new Map();
+        for (const r of this.rooms.values())
+            out.set(r.scope, (out.get(r.scope) || 0) + r.outRate);
+        for (const k of this.scopeSlow.keys())
+            if (!out.has(k))
+                this.scopeSlow.delete(k);
+        for (const [scope, rate] of out) {
+            let cap;
+            try {
+                cap = this.scopeCaps(scope)?.outBytes;
+            }
+            catch {
+                cap = undefined;
+            }
+            if (!cap || cap <= 0) {
+                this.scopeSlow.delete(scope);
+                continue;
+            }
+            let x = this.scopeSlow.get(scope);
+            if (!x) {
+                if (rate <= cap)
+                    continue;
+                x = { f: 1, avg: 0 };
+                this.scopeSlow.set(scope, x);
+            }
+            x.avg = x.avg ? x.avg * 0.7 + rate * 0.3 : rate;
+            if (x.avg > cap && x.f < 8) {
+                x.f *= 2;
+                x.avg /= 2;
+                this.refuse("scope_budget");
+            }
+            else if (x.f > 1 && x.avg * 2 < cap * 0.8) {
+                x.f /= 2;
+                x.avg *= 2;
+            }
+            if (x.f === 1 && x.avg <= cap)
+                this.scopeSlow.delete(scope);
+        }
+    }
+    /** One more join in a namespace with a joinsPerMinute cap, counted over a sliding minute (this minute plus the share of the last one still in it). */
+    scopeJoinAllowed(scope, now) {
+        if (!this.scopeCaps)
+            return true;
+        let cap;
+        try {
+            cap = this.scopeCaps(scope)?.joinsPerMinute;
+        }
+        catch {
+            cap = undefined;
+        }
+        if (!cap || cap <= 0)
+            return true;
+        let w = this.scopeJoins.get(scope);
+        if (!w || now - w.at >= 120_000) {
+            w = { at: now, n: 0, prev: 0 };
+            this.scopeJoins.set(scope, w);
+        }
+        else if (now - w.at >= 60_000) {
+            w.prev = w.n;
+            w.n = 0;
+            w.at += 60_000;
+        }
+        if (this.scopeJoins.size > 10_000)
+            for (const [k, v] of this.scopeJoins)
+                if (now - v.at >= 120_000)
+                    this.scopeJoins.delete(k);
+        const used = w.n + w.prev * Math.max(0, 1 - (now - w.at) / 60_000);
+        if (used >= cap)
+            return false;
+        w.n++;
+        return true;
+    }
     /** Once a second: the rates over the last second (server and per room). */
     meters(now) {
         if (now - this.mark.at < 1000)
@@ -1501,6 +1594,7 @@ export class RoomHub {
             r.outMark = r.outBytes;
             this.budget(r);
         }
+        this.scopeBudgets();
         // the whole server: over 80% of its data cap, or the event loop running late (p99 over 30 ms for 10 s, from the host process), bigger rooms
         // send at half rate and new rooms wait; back to normal under 60% and a loop on time
         const mbit = (this.rates.bytesOut * 8) / 1e6, cap = this.limits.maxOutMbit;
@@ -1703,11 +1797,70 @@ export class RoomHub {
         if (snap)
             await this.onSnapshot(snap);
     }
-    /** The live rooms as plain JSON: everything a new process needs to carry on (sockets, timers and open actions are not kept). */
-    exportRooms() {
+    /** Blue/green hand-over of one namespace to another process: its pending changes go out, open actions fail ("server restarting"), saves are
+     *  written, its rooms are exported, its sockets hear 1012 (a router in front re-joins them on the other process over the same page socket; a
+     *  page on its own reconnects) and its rooms close here, giving up their save leases. Joins to it are refused with 1012 meanwhile. Resolves
+     *  once every save is in the store, so the other process may take the snapshot (importRooms) and write the same saves. */
+    async exportScope(scope) {
+        this.moving.add(scope);
+        try {
+            // a profile save waits for its profile to load: let those land (2 s at most) so the write below carries them
+            const loads = [];
+            for (const r of this.rooms.values())
+                if (r.scope === scope)
+                    for (const x of r.seats.values())
+                        if (x.profLoad)
+                            loads.push(x.profLoad.catch(() => null));
+            if (loads.length)
+                await Promise.race([Promise.all(loads), new Promise((r) => setTimeout(r, 2000).unref?.())]);
+            const mine = [...this.rooms.values()].filter((r) => r.scope === scope);
+            for (const room of mine) {
+                this.flush(room);
+                for (const [aid, a] of room.actions) {
+                    a.from.session?.send({ t: "result", id: a.ref, ok: false, error: "server restarting" });
+                    room.actions.delete(aid);
+                }
+                this.saveWorld(room, true);
+                for (const x of room.seats.values())
+                    this.flushProfile(room, x, true);
+            }
+            const snap = this.exportRooms(scope);
+            for (const s of [...this.sessions])
+                if (s.scope === scope)
+                    s.end(CLOSE.restart, "moving to another server");
+            for (const room of mine)
+                this.closeRoom(room);
+            await this.settle();
+            return snap;
+        }
+        finally {
+            this.moving.delete(scope);
+        }
+    }
+    /** Namespaces with live rooms here and their size (a router's hand-over list, a dashboard). */
+    scopeList() {
+        const by = new Map();
+        for (const r of this.rooms.values()) {
+            let x = by.get(r.scope);
+            if (!x) {
+                x = { scope: r.scope, rooms: 0, players: 0, online: 0 };
+                by.set(r.scope, x);
+            }
+            x.rooms++;
+            for (const z of r.seats.values()) {
+                x.players++;
+                if (z.session)
+                    x.online++;
+            }
+        }
+        return [...by.values()];
+    }
+    /** The live rooms (of one namespace when given) as plain JSON: everything a new process needs to carry on (sockets, timers and open actions
+     *  are not kept). */
+    exportRooms(scope) {
         const rooms = [];
         for (const r of this.rooms.values()) {
-            if (r.seats.size === 0)
+            if (r.seats.size === 0 || (scope !== undefined && r.scope !== scope))
                 continue;
             rooms.push({
                 key: r.key, scope: r.scope, game: r.game, spec: r.spec, code: r.code, priv: r.priv, slot: r.slot, owner: r.owner, world: r.world, lease: r.lease, saveBusy: r.saveBusy,
@@ -1793,6 +1946,7 @@ export class RoomHub {
         }
     }
     /** One namespace's use right now: rooms, seats, connected players and bytes out a second (over the last second). For per-customer caps. */
+    /** `slowed`: how many times less often its rooms send because the namespace is over its own outBytes cap (1 = full rate). */
     scopeStats(scope) {
         let rooms = 0, players = 0, online = 0, bytesOut = 0;
         for (const r of this.rooms.values()) {
@@ -1806,7 +1960,7 @@ export class RoomHub {
                     online++;
             }
         }
-        return { rooms, players, online, bytesOut };
+        return { rooms, players, online, bytesOut, slowed: this.scopeF(scope) };
     }
     stats() {
         let players = 0, online = 0, started = 0, slow = 0;
@@ -1830,6 +1984,7 @@ export class RoomHub {
             ...this.rates, tickMs: { p50: pct(0.5), p99: pct(0.99), max: n ? Math.round(sorted[n - 1] * 100) / 100 : 0 }, slow, started,
             savesOk: this.counts.saves_ok || 0, savesFailed: this.refused.save_failed || 0, loadsFailed: this.refused.load_failed || 0, worldWritesPerMin: this.worldWrites.length,
             loop: this.loop, top, pressure: this.pressure, slowedRooms: [...this.rooms.values()].filter((r) => r.slowF > 1).length,
+            slowedScopes: [...this.scopeSlow.values()].filter((x) => x.f > 1).length,
         };
         return { rooms: this.rooms.size, players, online, connections: this.sessions.size, pending: this.pending, refused: { ...this.refused }, live };
     }