toiljs 0.0.85 → 0.0.87

package/docs/database/capacity.md

@@ -0,0 +1,350 @@
+# Capacity (limited stock without overselling)
+
+A `Capacity` collection models a **finite resource that must never be oversold**:
+concert tickets, seats, limited-drop stock, rate grants. It uses a two-phase
+`reserve` then `confirm`/`cancel` pattern so two buyers can never grab the same
+last unit.
+
+## What and why
+
+Some resources are strictly limited. There are 100 tickets, and selling 101 is a
+real, expensive mistake. The classic bug is a race: two requests both read "1
+left", both decide they can sell it, and both sell it. Now you have oversold.
+
+A `Capacity` collection prevents that. Instead of "read, then decrement" (two
+steps a race can slip between), it does an atomic **hold**: the check that enough
+is available and the decrement happen as one indivisible step, at one place. And
+it adds a second phase so a shopper who starts a checkout but never pays does not
+lock up stock forever.
+
+The lifecycle has three moves:
+
+- **`reserve`**: hold some units for a short time. The units come out of
+  "available" immediately, and you get back a **reservation id**. If there is not
+  enough available, the reserve fails (and nothing is oversold).
+- **`confirm`**: the shopper paid. Turn the hold into a permanent sale.
+- **`cancel`**: the shopper backed out. Release the hold back to available.
+
+If a hold is neither confirmed nor cancelled within its time limit (its **TTL**,
+time to live), it **auto-releases**. So an abandoned checkout heals itself.
+
+```mermaid
+sequenceDiagram
+    participant Buyer
+    participant App as Your route
+    participant Cap as Capacity ledger
+    Buyer->>App: start checkout
+    App->>Cap: reserve(key, 2, ttl)
+    Cap-->>App: reservationId (available drops by 2)
+    Note over Cap: 2 units are held, not yet sold
+    alt payment succeeds
+        Buyer->>App: pay
+        App->>Cap: confirm(key, reservationId)
+        Cap-->>App: true (hold becomes a permanent sale)
+    else buyer cancels
+        Buyer->>App: cancel
+        App->>Cap: cancel(key, reservationId)
+        Cap-->>App: true (2 units returned to available)
+    else buyer disappears
+        Note over Cap: TTL passes, hold auto-releases (2 units returned)
+    end
+```
+
+Use `Capacity` whenever selling one too many is unacceptable: tickets, seats,
+inventory for a limited drop, a pool of licences, or any "N and no more" resource.
+
+## Why not just a counter?
+
+A [Counter](./counters.md) can only `add` and `get`. To sell a ticket with a
+counter you would read the remaining count, check it is above zero, and then
+subtract one. Those are separate steps. Two requests can both read "1 left"
+before either subtracts, and both proceed: **oversold**. A counter has no way to
+atomically check-and-decrement, and no notion of a temporary hold that expires.
+
+`Capacity` fixes both problems:
+
+- The availability check and the hold are **one atomic step**, so contenders are
+  serialized: the first reserve takes the unit, the second sees zero available
+  and fails.
+- The two-phase **reserve then confirm/cancel** means an unpaid, abandoned
+  checkout does not permanently consume stock: its hold expires on the TTL.
+
+If a small over- or under-count is harmless (likes, page views, an
+approximate inventory display), a counter is simpler and cheaper. If overselling
+by one is a real problem, use `Capacity`.
+
+## The type
+
+```ts
+Capacity<K>
+```
+
+`Capacity` has a single type parameter, the **key** (`K`): it picks *which*
+resource. There is no value type, because the value is a private ledger the
+platform manages for you (it tracks the total, the confirmed sales, and the live
+holds). For one ticketed event, the key is the event id.
+
+Declare it as a `@collection` field on a `@database` class:
+
+```ts
+@data
+class ShowKey {
+  show: string = '';
+  constructor(show: string = '') { this.show = show; }
+}
+
+@database
+class TicketsDb {
+  @collection static seats: Capacity<ShowKey>;
+}
+```
+
+## Operations
+
+Five operations, matching the toilscript API exactly. Exact signatures:
+
+| Operation | Signature | What it does |
+| --- | --- | --- |
+| `available` | `available(key: K): i64` | How many units can be reserved right now. |
+| `reserve` | `reserve(key: K, amount: i64, ttlMs: i64): u64` | Hold `amount` units for `ttlMs` milliseconds. Returns a reservation id (> 0), or `0` if there was not enough. |
+| `confirm` | `confirm(key: K, reservationId: u64): bool` | Turn a hold into a permanent sale. Returns whether the id was valid. |
+| `cancel` | `cancel(key: K, reservationId: u64): bool` | Release a hold back to available. Returns whether it was released. |
+| `setTotal` | `setTotal(key: K, total: i64): void` | Set the ceiling (seed or restock). Background task only. |
+
+The core accounting the platform maintains for you:
+
+```txt
+available = total - confirmed - held
+```
+
+`total` is the ceiling you set. `confirmed` is units permanently sold. `held` is
+units currently reserved but not yet confirmed. `available` never goes below zero.
+
+### `available`
+
+Reads how many units could be reserved right now. It is a keyed read, legal from
+any handler including a `@get`.
+
+```ts
+const left = TicketsDb.seats.available(new ShowKey('jazz-night'));
+```
+
+### `reserve`
+
+Holds `amount` units for `ttlMs` milliseconds and returns a reservation id. This
+is the operation that prevents overselling.
+
+```ts
+const id = TicketsDb.seats.reserve(new ShowKey('jazz-night'), 2, 120_000); // hold 2 for 2 minutes
+if (id == 0) {
+  // Not enough available. This is a normal outcome, NOT an error: no oversell.
+  return Response.conflict('sold out');
+}
+// id > 0: you now hold 2 units for up to 2 minutes. Keep the id.
+```
+
+Two things to know:
+
+- A return of `0` means "not enough available". It is the safe, expected answer
+  when the resource is (nearly) sold out. Always check for it.
+- The hold auto-releases after `ttlMs` if you do not confirm or cancel it. Choose
+  a TTL that covers a realistic checkout (a couple of minutes), not hours.
+
+`reserve` is a **write**, so call it from an action handler (`@post`, `@put`,
+`@patch`, `@del`), not from a `@get`.
+
+### `confirm`
+
+Turns a hold into a permanent sale. Call it once payment (or whatever finalizes
+the deal) succeeds.
+
+```ts
+const ok = TicketsDb.seats.confirm(new ShowKey('jazz-night'), id);
+// ok === true  -> the id was valid; those units are now permanently sold
+// ok === false -> the id was unknown (never reserved, or already expired)
+```
+
+A confirmed sale is **permanent**: it can never be cancelled and its TTL never
+reclaims it. Confirm is safe to call more than once for the same id (it stays
+confirmed).
+
+### `cancel`
+
+Releases a hold back to available. Call it when the shopper backs out before
+paying.
+
+```ts
+const released = TicketsDb.seats.cancel(new ShowKey('jazz-night'), id);
+// released === true  -> the hold was released, units are available again
+// released === false -> the id was unknown, already expired, OR already confirmed
+```
+
+You **cannot** cancel a confirmed sale (a sale is final). `cancel` returns `false`
+in that case.
+
+### `setTotal`
+
+Sets the ceiling: the total number of units. You call it to **seed** a new
+resource ("this show has 100 seats") and to **restock** ("we opened the balcony,
+now 150"). Existing holds and confirmed sales are untouched; only the ceiling
+moves, and `available` reflects the new total.
+
+`setTotal` is a **privileged** operation: it may only run from a background task
+(a `@job`), never from a request handler. Seeding and restocking are
+administrative actions, so they live off the request path. See
+[background tasks](../background/README.md).
+
+```ts
+@database
+class TicketsDb {
+  @collection static seats: Capacity<ShowKey>;
+
+  // A background task seeds/restocks the ceiling. This runs off the request path.
+  @job
+  openSeats(): void {
+    TicketsDb.seats.setTotal(new ShowKey('jazz-night'), 100);
+  }
+}
+```
+
+Note that lowering the total below what is already sold plus held does not
+cancel anything (confirmed sales are permanent); `available` simply floors at
+zero until holds clear.
+
+## The lifecycle in words
+
+1. **Seed.** A background `@job` calls `setTotal(key, 100)`. Now `available` is
+   100.
+2. **Reserve.** A buyer starts checkout. Your `@post` calls `reserve(key, 1,
+   ttl)` and gets an id. `available` drops to 99. The unit is *held*, not sold.
+3. **Finalize, one of:**
+   - **Confirm.** Payment succeeds; your `@post` calls `confirm(key, id)`. The
+     unit is now permanently sold. `available` stays 99.
+   - **Cancel.** The buyer backs out; your `@post` calls `cancel(key, id)`. The
+     unit returns to available; `available` goes back to 100.
+   - **Expire.** The buyer vanishes. After `ttl` passes, the hold auto-releases;
+     `available` goes back to 100 on its own.
+
+At no point can the confirmed count exceed the total. That is the guarantee.
+
+## Worked example: selling limited tickets
+
+```ts
+import { ShowKey } from '../models/ShowKey';
+import { ReserveRequest } from '../models/ReserveRequest';
+import { FinalizeRequest } from '../models/FinalizeRequest';
+import { ReserveResult } from '../models/ReserveResult';
+
+@database
+class TicketsDb {
+  @collection static seats: Capacity<ShowKey>;
+
+  // Background task: seed the show's capacity (and restock if you reopen it).
+  @job
+  openSeats(): void {
+    TicketsDb.seats.setTotal(new ShowKey('jazz-night'), 100);
+  }
+}
+
+@rest('tickets')
+class Tickets {
+  // How many seats are left (a keyed read, legal in a GET).
+  @get('/jazz-night/available')
+  public left(): i64 {
+    return TicketsDb.seats.available(new ShowKey('jazz-night'));
+  }
+
+  // Start checkout: hold the seats. Returns a reservation id, or 0 if sold out.
+  @post('/jazz-night/reserve')
+  public reserve(input: ReserveRequest): ReserveResult {
+    const id = TicketsDb.seats.reserve(new ShowKey('jazz-night'), input.count, 120_000);
+    // id === 0 means not enough available. No oversell; tell the buyer honestly.
+    return new ReserveResult(id, id != 0);
+  }
+
+  // Payment succeeded: make the hold a permanent sale.
+  @post('/jazz-night/confirm')
+  public confirm(input: FinalizeRequest): bool {
+    return TicketsDb.seats.confirm(new ShowKey('jazz-night'), input.reservationId);
+  }
+
+  // Buyer backed out before paying: release the hold.
+  @post('/jazz-night/cancel')
+  public cancel(input: FinalizeRequest): bool {
+    return TicketsDb.seats.cancel(new ShowKey('jazz-night'), input.reservationId);
+  }
+}
+```
+
+The models:
+
+```ts
+@data
+export class ReserveRequest {
+  count: i64 = 1;
+}
+
+@data
+export class FinalizeRequest {
+  reservationId: u64 = 0;
+}
+
+@data
+export class ReserveResult {
+  reservationId: u64 = 0;
+  ok: bool = false;
+  constructor(reservationId: u64 = 0, ok: bool = false) {
+    this.reservationId = reservationId;
+    this.ok = ok;
+  }
+}
+```
+
+Even if a thousand people hit `reserve` at the same instant for the last few
+seats, the ledger serializes them: the first buyers get ids, the rest get `0`.
+The total is never exceeded.
+
+## Consistency
+
+Unlike most ToilDB families, `Capacity` is **strongly consistent**, and that is
+the whole point. Every reserve/confirm/cancel for a key is routed to that key's
+one home location and applied there in order, at a single serialization point. So
+even though ToilDB spans many regions, there is exactly one place that decides
+whether a reserve succeeds. That is what makes "never oversell" a hard guarantee
+rather than a best effort. `available` reflects that home ledger.
+
+## Gotchas
+
+- **Check for `0` from `reserve`.** A `0` is "not enough available", the normal
+  sold-out answer, not an error. Do not treat it as a failure to retry blindly.
+- **Always finalize a reservation.** After a successful `reserve`, you should
+  `confirm` it (paid) or `cancel` it (abandoned). If you do neither, the hold
+  sits until its TTL expires, temporarily reducing availability.
+- **Pick a sensible TTL.** Too short and a slow-but-real checkout loses its hold
+  mid-payment. Too long and abandoned carts starve real buyers. A couple of
+  minutes is typical for a checkout.
+- **Confirmed sales are permanent.** You cannot `cancel` a confirmed sale, and
+  its TTL never reclaims it. If you support refunds, model that as your own
+  restock (raise `setTotal`), not as a cancel.
+- **`reserve` is not automatically deduplicated.** Each call to `reserve` creates
+  a new, distinct hold. If a client might retry the same reserve, avoid holding
+  twice: reserve once, keep the returned id, and drive `confirm`/`cancel` off
+  that id.
+- **`setTotal` is background-only.** You cannot set the total from a route. Seed
+  and restock from a `@job`.
+- **There is a per-key cap on live holds.** A key can carry only so many
+  simultaneous unconfirmed holds; a flood of holds beyond that is rejected as
+  "not enough available" (and abandoned holds clear on their TTL). Normal traffic
+  never hits this; it is a guardrail against a hold flood.
+
+## Related
+
+- [Counters](./counters.md): a running total when overselling is not a concern
+  (and why it cannot guarantee a limit).
+- [Documents](./documents.md): store the order/booking record that a confirmed
+  sale produces.
+- [background tasks](../background/README.md): where `setTotal` (seeding/restock)
+  runs.
+- [Data types (`@data`)](../concepts/types.md): how the capacity key is stored.
+- [Decorators](../concepts/decorators.md): which handler kinds may reserve,
+  confirm, and cancel.

package/docs/database/counters.md

@@ -0,0 +1,174 @@
+# Counters
+
+The **Counter** family is a distributed running total that many callers can increase at the same time, from anywhere in the world, without ever losing a count. It is how you build likes, view counts, and inventory tallies correctly.
+
+## What and why
+
+A **Counter collection** maps a `@data` key to a single whole number. You do just two things with it: read the current total (`get`) and add to it (`add`). There is deliberately **no "set to this value"** operation, and that missing operation is the entire point of the family, as the next section explains.
+
+Reach for a Counter whenever the thing you are storing is a number that many requests bump concurrently:
+
+- likes or upvotes on a post
+- page views or play counts
+- a stock or quantity tally
+- any "increase this by N" metric
+
+Declare one by typing a `@collection` as `Counter<Key>`. Notice there is no value type: the value is always a number the database manages for you.
+
+```ts
+@data
+class PostId {
+  id: string = '';
+  constructor(id: string = '') { this.id = id; }
+}
+
+@database
+class AppDb {
+  @collection static likes: Counter<PostId>;
+  @collection static views: Counter<PostId>;
+}
+```
+
+## The operations
+
+`K` is the key type. The value is always a 64-bit signed integer (`i64`).
+
+| Operation | Signature | Returns | Use it to |
+| --- | --- | --- | --- |
+| `get` | `get(key: K): i64` | the current total (`0` if nothing has been added) | read the count |
+| `add` | `add(key: K, delta: i64): void` | nothing | change the count by `delta` (may be negative) |
+
+`get` is a read (works in any function). `add` is a write, so it needs an **Action** (a `@post` route or an `@action`); see [Setup](./setup.md#how-access-is-gated-query-action-and-friends).
+
+### `get`
+
+`get` returns the current total. A counter that has never been touched reads as `0`, not `null`: there is no "absent" state to handle.
+
+```ts
+const likeCount: i64 = AppDb.likes.get(new PostId('p_42'));
+```
+
+### `add`
+
+`add` changes the total by a delta. The delta can be positive to increase or negative to decrease:
+
+```ts
+AppDb.likes.add(new PostId('p_42'), 1);   // one more like
+AppDb.likes.add(new PostId('p_42'), -1);  // undo a like
+AppDb.stock.add(new SkuId('sku_9'), -3);  // sold three
+```
+
+`add` saturates at the `i64` limits: it will not wrap around from a huge positive to a negative if you somehow overflow. Note that `add` returns nothing, not the new total. If you need the total right after adding, call `get` (bearing in mind the consistency note below).
+
+## Why a Counter, and not a number in a Documents record?
+
+This is the question the family exists to answer, so it is worth walking through.
+
+Suppose you stored the like count as a field in a Documents record and incremented it the obvious way: read the record, add one, write it back. Now two servers on opposite sides of the world each get a like at the same moment:
+
+```mermaid
+sequenceDiagram
+    participant Tokyo as Server (Tokyo)
+    participant Paris as Server (Paris)
+    participant DB as Documents record
+
+    Note over DB: likes = 10
+    Tokyo->>DB: read likes -> 10
+    Paris->>DB: read likes -> 10
+    Tokyo->>DB: write likes = 11
+    Paris->>DB: write likes = 11
+    Note over DB: likes = 11 (one like LOST)
+```
+
+Both read `10`, both wrote `11`, and one like vanished. This is a **lost update**, and it happens whenever two callers do read-modify-write on the same value at once. It is not a rare edge case on a busy post; it is the normal case.
+
+A Counter avoids this entirely because you never send a final value, you send a **delta**. Each server just says "add 1." The database merges the two deltas into "add 1, add 1 = add 2," so the total goes from 10 to 12 and nothing is lost. Deltas are **commutative** (the order they arrive in does not matter) and **additive**, so concurrent increments from anywhere in the world always combine correctly. That property is what "conflict-free" means: there is no conflict to resolve, because two adds are never at odds.
+
+This is also why there is no `set` operation. A `set` would reintroduce the exact race above (two callers overwriting each other). By offering only `add`, the Counter family makes the lost-update bug impossible to write.
+
+**Use a Counter when** the number is bumped concurrently by many requests. **Use a Documents field instead when** the number is only ever changed by a single logical owner in a controlled way (for example, a value you always overwrite with a freshly computed absolute figure, not an increment), where you would use `patch` to store the whole new value.
+
+## Worked example: page views and likes
+
+Here is a small feature that counts views on every read and likes on demand, then reports both.
+
+```ts
+import { Response, RouteContext } from 'toiljs/server/runtime';
+
+@data
+class PostId {
+  id: string = '';
+  constructor(id: string = '') { this.id = id; }
+}
+
+@data
+class Stats {
+  views: i64 = 0;
+  likes: i64 = 0;
+}
+
+@database
+class AppDb {
+  @collection static views: Counter<PostId>;
+  @collection static likes: Counter<PostId>;
+}
+
+@rest('posts')
+class Posts {
+  // POST /posts/:id/view  -> count a view, return the running totals (Action)
+  @post('/:id/view')
+  public view(ctx: RouteContext): Stats {
+    const key = new PostId(ctx.param('id'));
+    AppDb.views.add(key, 1);
+    const s = new Stats();
+    s.views = AppDb.views.get(key);
+    s.likes = AppDb.likes.get(key);
+    return s;
+  }
+
+  // POST /posts/:id/like  -> add a like (Action)
+  @post('/:id/like')
+  public like(ctx: RouteContext): Stats {
+    const key = new PostId(ctx.param('id'));
+    AppDb.likes.add(key, 1);
+    const s = new Stats();
+    s.views = AppDb.views.get(key);
+    s.likes = AppDb.likes.get(key);
+    return s;
+  }
+
+  // GET /posts/:id/stats  -> just read the totals (Query: read-only)
+  @get('/:id/stats')
+  public stats(ctx: RouteContext): Stats {
+    const key = new PostId(ctx.param('id'));
+    const s = new Stats();
+    s.views = AppDb.views.get(key);
+    s.likes = AppDb.likes.get(key);
+    return s;
+  }
+}
+```
+
+Counting a view lives in a `@post` because it is a write (a counter `add`), even though it feels like part of a read. The `@get` stats route only reads, so it runs as a Query.
+
+## Consistency notes
+
+- **Counters are eventually consistent.** `get` returns the total known to the copy nearest you, which may briefly trail increments applied moments ago in other regions. The copies converge quickly, so the count catches up on its own.
+- **No increment is ever lost.** That is the strong guarantee: every `add` from anywhere is eventually included in the total, because deltas merge. The total may be a little behind, but it is never wrong in the "lost a like" sense.
+- **Monotonic when you only add positives.** If every `add` is positive, the total only ever goes up as it converges. Mixing in negative `add`s (undoing a like, selling stock) is fine and correct; it simply means the total is not monotonic.
+- Because of the lag, do not treat the value `get` returns immediately after an `add` as a globally final figure. It is a fast, close-enough total, which is exactly right for likes and views.
+
+## Gotchas
+
+- **There is no `set`.** To move a counter to a specific number, add the difference (`add(key, target - get(key))`), but be aware that read-then-add reintroduces a race if two callers do it at once. Prefer to think in deltas.
+- **`add` does not return the new total.** Call `get` after if you need it, remembering it may lag concurrent remote adds.
+- **A never-touched counter reads `0`, not `null`.** There is no "does this counter exist" check; every key answers `0` until something is added.
+- **Counters hold a number, not a record.** If you need per-key fields alongside the count, keep the record in [Documents](./documents.md) and the tally in a Counter, keyed the same way (the example above effectively does this with two counters).
+
+## Related
+
+- [ToilDB overview](./README.md): the seven families and how to choose.
+- [Setup](./setup.md): declaring the collection and which function kinds may write.
+- [Documents](./documents.md): when the number is part of a record you overwrite wholesale.
+- [Events](./events.md): when you need the individual events, not just a total.
+- [Data types (`@data`)](../backend/data.md): the counter's key type.