@venizia/ignis-docs 0.2.0 → 0.2.1-1

package/content/extensions/helpers/uid/index.md

@@ -1,272 +1,259 @@
+---
+title: UID
+description: Two ID generators - time-sortable Snowflake IDs, and short random IDs a human can read back
+difficulty: beginner
+---
+
 # UID
 
-Snowflake ID generator with Base62 encoding for unique, time-sortable distributed identifiers.
+Two generators, and they answer different questions.
+
+| Helper | Reads back as | Pick it when |
+|---|---|---|
+| `SnowflakeUidHelper` | Timestamp, worker ID, sequence | The ID is a primary key and you want insert order for free |
+| `OpaqueUidHelper` | Nothing | The ID leaves your system - a human reads it, or a stranger sees it |
 
-## Quick Reference
+A Snowflake ID is transparent by design. `parseId()` gives back the exact millisecond it was minted and the worker that minted it. That is what you want in a log, and what you do not want printed on an invoice a customer keeps.
 
-| Item | Value |
-|------|-------|
-| **Package** | `@venizia/ignis-helpers` |
-| **Class** | `SnowflakeUidHelper` |
-| **Extends** | `BaseHelper` |
-| **Runtimes** | Both |
+An opaque ID gives nothing back. You trade the ordering for that, and for a short ID a human can read out loud.
 
-#### Import Paths
+## In one example
 
 ```typescript
-import { SnowflakeUidHelper, SnowflakeConfig } from '@venizia/ignis-helpers';
+import { SnowflakeUidHelper } from '@venizia/ignis-helpers';
 
-// Types
-import type { IIdGeneratorOptions, ISnowflakeParsedId } from '@venizia/ignis-helpers';
+const generator = new SnowflakeUidHelper(); // workerId: 199, epoch: 2025-01-01 UTC
+const id = generator.nextId();
+// => e.g. "9du1sJXO88"
 ```
 
-#### Snowflake ID Structure (70 bits)
-
-| Component | Bits | Range | Purpose |
-|-----------|------|-------|---------|
-| Timestamp | 48 | ~8,919 years | Time since epoch |
-| Worker ID | 10 | 0--1023 | Unique worker identifier |
-| Sequence | 12 | 0--4095 | IDs per millisecond |
-
-#### Key Specifications
-
-| Spec | Value |
-|------|-------|
-| Base62 Output | 10--12 characters |
-| Throughput | 4,096,000 IDs/second/worker |
-| Max Workers | 1024 |
-| Lifespan | Until ~10,944 AD |
-| Default Epoch | 2025-01-01 00:00:00 UTC |
+Use one `SnowflakeUidHelper` instance per worker process. `nextId()` returns a URL-safe Base62 string, and it sorts in the same order the IDs were generated.
 
-## Creating an Instance
+## How it works
 
-`SnowflakeUidHelper` extends `BaseHelper`, providing scoped logging via `LoggerFactory`.
+- **The ID packs three fields into 70 bits.** `nextSnowflake()` shifts and OR's together a timestamp, a worker ID, and a sequence - see the bit layout below.
+- **Sorting the ID sorts by generation time.** This holds for the raw `bigint`, and for the Base62 string.
+- **The sequence resets every millisecond, per worker.** It increments within the millisecond and wraps at 4096.
+- **A wrap forces a busy-wait for the next millisecond.** That caps throughput at 4,096,000 IDs per second per worker.
+- **A small backward clock drift busy-waits.** Up to 100ms (`MAX_CLOCK_BACKWARD_MS`), the generator waits for the clock to catch up, and logs a warning.
+- **A larger drift throws instead.** IGNIS refuses to risk generating a duplicate ID.
+- **`encodeBase62` / `decodeBase62` are a bigint <-> string codec.** They don't know about the Snowflake layout.
+- **`parseId()` layers on top of the codec.** It calls `decodeBase62()`, then the three `extract*` calls, using the same instance's `epoch` and bit-shift constants.
+- **An expiry warning logs once the epoch nears its 48-bit limit.** The timestamp field overflows about 8,919 years after the configured epoch. A warning starts logging 10 years before that.
 
-```typescript
-import { SnowflakeUidHelper } from '@venizia/ignis-helpers';
+**Snowflake layout (70 bits)**
 
-// Use defaults (workerId: 199, epoch: 2025-01-01 00:00:00 UTC)
-const generator = new SnowflakeUidHelper();
-
-// Or with custom values
-const customGenerator = new SnowflakeUidHelper({
-  workerId: 123,
-  epoch: BigInt(1735689600000),
-});
-```
+| Component | Bits | Range | Purpose |
+|-----------|------|-------|---------|
+| Timestamp | 48 | ~8,919 years | Milliseconds since `epoch` |
+| Worker ID | 10 | 0-1023 | Set per instance, must be unique across a deployment |
+| Sequence | 12 | 0-4095 | IDs generated within the same millisecond |
 
-#### `IIdGeneratorOptions`
+**`IIdGeneratorOptions`**
 
 | Option | Type | Default | Description |
 |--------|------|---------|-------------|
-| `workerId` | `number` | `199` | Worker ID (0--1023). Must be unique per instance in distributed deployments |
-| `epoch` | `bigint` | `BigInt(1735689600000)` | Custom epoch timestamp in milliseconds since Unix epoch. Must be positive and in the past |
+| `workerId` | `number` | `199` | Integer, `0`-`1023` |
+| `epoch` | `bigint` | `BigInt(1735689600000)` (2025-01-01 00:00:00 UTC) | Positive, must be in the past |
 
-> [!TIP]
-> Use unique worker IDs in distributed systems to prevent ID collisions, and keep the epoch consistent across all instances to ensure proper ID ordering.
+## Common tasks
 
-#### `SnowflakeConfig` Constants
+### Generate an ID
+
+`nextId()` is the common case - a Base62 string, 10-12 characters. `nextSnowflake()` returns the raw `bigint`, for arithmetic or bit-level access.
 
 ```typescript
-import { SnowflakeConfig } from '@venizia/ignis-helpers';
-
-SnowflakeConfig.DEFAULT_EPOCH      // BigInt(1735689600000) - 2025-01-01 00:00:00 UTC
-SnowflakeConfig.MAX_WORKER_ID      // 1023n
-SnowflakeConfig.MAX_SEQUENCE       // 4095n
-SnowflakeConfig.TIMESTAMP_SHIFT    // 22n
-SnowflakeConfig.WORKER_ID_SHIFT    // 12n
-
-// Bit widths
-SnowflakeConfig.TIMESTAMP_BITS     // 48n
-SnowflakeConfig.WORKER_ID_BITS     // 10n
-SnowflakeConfig.SEQUENCE_BITS      // 12n
-
-// Safety thresholds
-SnowflakeConfig.MAX_CLOCK_BACKWARD_MS  // 100n (busy-wait tolerance)
-SnowflakeConfig.MAX_TIMESTAMP_MS       // (1n << 48n) - 1n (~8,919 years)
-SnowflakeConfig.WARNING_THRESHOLD_MS   // ~8,909 years (warns 10 years before expiry)
+const id = generator.nextId();               // "9du1sJXO88"
+const raw = generator.nextSnowflake();        // 130546360012247045n
 ```
 
-## Usage
-
-### Generating IDs
-
-Generate unique Snowflake IDs as either compact Base62 strings or raw bigints.
+### Parse an ID back into its components
 
 ```typescript
-// Generate Base62 encoded ID (recommended for most use cases)
-const id = generator.nextId();
-// => e.g., "9du1sJXO88"
-
-// Generate raw Snowflake ID as bigint
-const snowflakeId = generator.nextSnowflake();
-// => e.g., 130546360012247045n
+const parsed = generator.parseId('9du1sJXO88');
+// => { raw: 130546360012247045n, timestamp: Date, workerId: 199, sequence: 0 }
 ```
 
-> [!TIP]
-> Use `nextId()` for most cases -- it returns a compact Base62 string (10--12 chars) that works well as database primary keys and URL-safe identifiers. Use `nextSnowflake()` only when you need the raw bigint for arithmetic or bit-level operations.
+### Extract a single component from a raw ID
 
-#### Clock Drift Handling
+```typescript
+const timestamp = generator.extractTimestamp(raw);
+const workerId = generator.extractWorkerId(raw);
+const sequence = generator.extractSequence(raw);
+```
 
-When `nextSnowflake()` detects the system clock has moved backward:
-- Drifts up to 100ms are handled automatically by busy-waiting until the clock catches up.
-- Drifts exceeding 100ms throw an error to protect ID uniqueness.
-- A warning is logged whenever any backward clock movement is detected.
+### Run one generator per worker in a distributed deployment
 
-#### Sequence Exhaustion
+Give each process a unique `workerId`. Keep `epoch` identical across every process, so ID ordering stays meaningful.
 
-If more than 4096 IDs are requested in a single millisecond from the same worker, the generator busy-waits until the next millisecond before continuing. This is transparent to the caller.
+```typescript
+const generator = new SnowflakeUidHelper({ workerId: Number(process.env.WORKER_ID) });
+```
 
-### Base62 Encoding and Decoding
+### Decode Base62 defensively
 
-Encode any bigint to a compact, URL-safe Base62 string using the alphabet `0-9A-Za-z`, or decode back to the original bigint.
+`decodeBase62()` throws on any character outside `0-9A-Za-z` - including a leading space or a URL-decoded `+`.
 
 ```typescript
-// Encode
-const encoded = generator.encodeBase62(130546360012247045n);
-// => "9du1sJXO88"
-
-// Decode
-const decoded = generator.decodeBase62('9du1sJXO88');
-// => 130546360012247045n
+try {
+  const raw = generator.decodeBase62(userSuppliedId);
+} catch (error) {
+  // not a valid Base62 ID
+}
 ```
 
-> [!WARNING]
-> `decodeBase62()` throws if the input contains characters outside the Base62 alphabet (`0-9`, `A-Z`, `a-z`). Watch for URL-encoded strings or accidental whitespace.
-
-### Parsing IDs
+## OpaqueUidHelper
 
-Extract the embedded timestamp, worker ID, and sequence from a generated ID.
+```typescript
+import { OpaqueUidHelper } from '@venizia/ignis-helpers';
 
-#### Parse a Base62 ID
+const generator = new OpaqueUidHelper({
+  prefix: { enable: true, value: 'INV' },
+  delimiter: { enable: true, value: '-' },
+  length: 6,
+});
 
-```typescript
-const parsed = generator.parseId('9du1sJXO88');
-// => {
-//   raw: 130546360012247045n,
-//   timestamp: Date,
-//   workerId: 199,
-//   sequence: 0
-// }
+generator.nextId();
+// => e.g. "INV-7K2MQ9"
 ```
 
-#### `ISnowflakeParsedId`
+`length` counts the body only. The prefix and the delimiter sit in front of it, so that ID is 10 characters.
 
-```typescript
-interface ISnowflakeParsedId {
-  raw: bigint;
-  timestamp: Date;
-  workerId: number;
-  sequence: number;
-}
-```
+### Options
 
-#### Extract Individual Components
+| Option | Type | Default | Meaning |
+|---|---|---|---|
+| `prefix` | `{ enable, value? }` | disabled | Leads the ID. `value` survives being disabled, so you can toggle it back on |
+| `delimiter` | `{ enable, value? }` | disabled, `_` | Separates prefix from body. Requires a prefix |
+| `length` | `number` | `6` | Characters in the body |
+| `caseForm` | `'upper' \| 'lower' \| 'mixed'` | `'upper'` | Folds the alphabet before sampling |
+| `alphabet` | `string` | `UidAlphabets.CROCKFORD` | The characters to draw from |
+| `exclude` | `string` | `''` | Removed from `alphabet` |
 
-```typescript
-const snowflakeId = generator.nextSnowflake();
+Four alphabets ship with it:
 
-// Extract timestamp
-const timestamp = generator.extractTimestamp(snowflakeId);
-// => Date object
+| Alphabet | Size | Drops | Use for |
+|---|---|---|---|
+| `CROCKFORD` | 32 | Lowercase, `I` `L` `O` `U` | IDs a human reads, types, or says aloud |
+| `BASE58` | 58 | `0` `O` `I` `l` | IDs in a URL or a log |
+| `NO_VOWEL` | 50 | `BASE58` plus every vowel | Customer-facing IDs - no word can form |
+| `BASE62` | 62 | Nothing | Machines only |
 
-// Extract worker ID
-const workerId = generator.extractWorkerId(snowflakeId);
-// => 199
+`getAlphabet()` returns the set actually in use, after the case fold and every exclusion. That is what `length` is measured against, so read it before calculating how many IDs a length affords you.
 
-// Extract sequence
-const sequence = generator.extractSequence(snowflakeId);
-// => 0-4095
+### Six characters needs a unique index
 
-// Get current instance's worker ID
-const currentWorkerId = generator.getWorkerId();
-// => 199
-```
+Six Crockford characters is 2^30 combinations, the size of an airline record locator. That works for airlines because of three things, not because 2^30 is large:
 
-## API Summary
+1. **Scoped.** A record locator is unique per carrier, not worldwide.
+2. **Recycled.** The airline reuses it once the trip is over.
+3. **Retried.** The reservation system regenerates on conflict.
 
-| Method | Signature | Description |
-|--------|-----------|-------------|
-| `nextId` | `nextId(): string` | Generate a Base62-encoded Snowflake ID (10--12 chars) |
-| `nextSnowflake` | `nextSnowflake(): bigint` | Generate a raw 70-bit Snowflake ID |
-| `encodeBase62` | `encodeBase62(num: bigint): string` | Encode a bigint to Base62 string |
-| `decodeBase62` | `decodeBase62(str: string): bigint` | Decode a Base62 string to bigint |
-| `parseId` | `parseId(base62Id: string): ISnowflakeParsedId` | Parse a Base62 ID into its components |
-| `extractTimestamp` | `extractTimestamp(id: bigint): Date` | Extract the timestamp from a raw Snowflake ID |
-| `extractWorkerId` | `extractWorkerId(id: bigint): number` | Extract the worker ID from a raw Snowflake ID |
-| `extractSequence` | `extractSequence(id: bigint): number` | Extract the sequence number from a raw Snowflake ID |
-| `getWorkerId` | `getWorkerId(): number` | Get the current instance's worker ID |
+> [!WARNING]
+> At the default length, a 1% chance of at least one collision arrives at roughly 4,600 IDs in one space, and 50% at roughly 38,000.
 
-## Troubleshooting
+A `prefix` PARTITIONS the space. `INV-7K2MQ9` can never collide with `CUS-7K2MQ9`, so ten entity types get ten separate spaces instead of sharing one. It does not make an ID unique inside its own space - within `INV`, the numbers above still hold.
 
-### "Worker ID must be between 0 and 1023"
+So do both. Give each entity type its own `prefix`, and put the column behind a unique index.
 
-**Cause:** The `workerId` option passed to the constructor is negative or exceeds 1023 (10-bit maximum).
+### Reject an ID you already hold
 
-**Fix:** Ensure the worker ID is within the valid range (0--1023):
+`nextId()` takes a callback. Return `true` to accept the ID, `false` to draw another.
 
 ```typescript
-// Wrong
-new SnowflakeUidHelper({ workerId: 2000 });
+const issued = new Set<string>();
+const generator = new OpaqueUidHelper({ length: 8 });
 
-// Correct
-new SnowflakeUidHelper({ workerId: 1 });
+const id = generator.nextId({
+  isAvailable: candidate => !issued.has(candidate),
+});
+issued.add(id);
 ```
 
-### "Epoch must be a positive number"
+Every argument is optional, including the options object itself - `nextId()` alone is the common case.
 
-**Cause:** The `epoch` option is zero or negative.
+**`nextId()` options**
 
-**Fix:** Provide a valid epoch as a positive bigint representing milliseconds since Unix epoch:
+| Option | Type | Default | Meaning |
+|---|---|---|---|
+| `prefix` | `string` | The configured prefix | Replaces it for this call |
+| `isAvailable` | `(id: string) => ValueOrPromise<boolean>` | none | `true` accepts, `false` draws again |
+| `maxAttempts` | `number` | `10` | Draws before it throws. Requires `isAvailable` |
+
+The callback sees the finished ID, prefix and delimiter included - the string the column stores.
+
+**The callback's return type decides `nextId`'s.** A synchronous check keeps the call synchronous, so no promise is allocated. An asynchronous one makes it a promise:
 
 ```typescript
-// Wrong
-new SnowflakeUidHelper({ epoch: BigInt(0) });
+const id = generator.nextId({ isAvailable: candidate => !issued.has(candidate) });
+// string
 
-// Correct
-new SnowflakeUidHelper({ epoch: BigInt(1735689600000) });
+const id = await generator.nextId({
+  isAvailable: async candidate => !(await invoiceRepository.exists(candidate)),
+});
+// Promise<string>
 ```
 
-### "Epoch cannot be in the future"
+> [!WARNING]
+> An asynchronous check does not make an ID safe. Against a database it is a read and then a write with a gap between them, and another request can take the ID inside that gap. `isAvailable` lowers the collision rate; the unique index is what removes it.
 
-**Cause:** The `epoch` option is set to a timestamp later than the current time. A future epoch would produce negative timestamp offsets in generated IDs.
+`maxAttempts` bounds the redraws. Repeated rejection means the space is full, and the helper throws with the length and alphabet size rather than looping.
 
-**Fix:** Use a past date as the epoch. The default (`2025-01-01 00:00:00 UTC`) is recommended unless you have a specific reason to change it.
+### Regenerate on conflict, never check first
 
-### "Clock moved backward by Xms. Refusing to generate ID."
+Insert and catch the violation. Checking whether an ID exists and then inserting it leaves a window where another request takes the same ID between the two statements.
 
-**Cause:** The system clock moved backward by more than 100ms, typically caused by NTP time synchronization. Small drifts (up to 100ms) are handled by busy-waiting, but larger jumps are rejected to protect ID uniqueness.
+```typescript
+import { executeWithRetry, OpaqueUidHelper } from '@venizia/ignis-helpers';
 
-**Fix:** Ensure your system clock is stable. If running in containers, verify the host clock is not being adjusted aggressively. There is no code-level workaround -- the error protects against duplicate IDs.
+const generator = new OpaqueUidHelper({
+  prefix: { enable: true, value: 'INV' },
+  delimiter: { enable: true, value: '-' },
+});
 
-### "Invalid Base62 character: X"
+const invoice = await executeWithRetry({
+  operation: 'createInvoice',
+  maxAttempts: 5,
+  // A fresh ID per attempt - retrying with the same one would collide forever.
+  execution: () => invoiceRepository.create({ data: { id: generator.nextId(), total } }),
+  shouldRetry: context => isUniqueViolation(context.error),
+});
+```
 
-**Cause:** `decodeBase62()` encountered a character outside the Base62 alphabet (`0-9`, `A-Z`, `a-z`).
+Five attempts is generous. At the default length a second collision on the same insert has a probability in the millionths.
 
-**Fix:** Ensure the input string only contains valid Base62 characters:
+Or raise `length` - every extra character multiplies the space by 32.
 
-```typescript
-// Wrong
-generator.decodeBase62('9du1sJ+O88');   // '+' is not Base62
-generator.decodeBase62(' 9du1sJXO88');  // leading space
+### What the constructor refuses
 
-// Correct
-generator.decodeBase62('9du1sJXO88');
-```
+Each of these fails loudly at construction, because each one fails silently at runtime:
+
+| Configuration | Why it is refused |
+|---|---|
+| `delimiter` enabled, `prefix` disabled | The delimiter would lead the ID and separate nothing |
+| A delimiter character that is in the alphabet | The ID could not be split back into prefix and body |
+| A `prefix` that disagrees with `caseForm` | A lowercase prefix on an uppercase ID defeats the reason for choosing one case |
+| `caseForm` folding a two-case alphabet | Uppercasing `BASE58` brings `I` and `O` back - the pair it drops so `1` and `0` stay readable |
+| An alphabet under 16 characters | `exclude` has eaten it |
+| `maxAttempts` passed without `isAvailable` | Nothing rejects a draw, so the retry it configures cannot happen |
+
+### Randomness
+
+`OpaqueUidHelper` draws from `crypto.getRandomValues`, and rejects samples that fall outside the alphabet rather than wrapping them. A `byte % 58` would favour the first 24 characters by about 1.5%, forever, and no caller could see it.
 
-## See Also
+Unlike `crypto.randomUUID`, `getRandomValues` works outside a secure context. The generator runs on a plain-http origin and inside a browser Worker.
 
-- **Related Concepts:**
-  - [Models](/guides/core-concepts/persistent/models) - Using UIDs as primary keys
-  - [Services](/guides/core-concepts/services) - Generating unique IDs in services
+## See also
 
-- **Other Helpers:**
-  - [Helpers Index](../index) - All available helpers
-  - [Crypto Helper](../crypto/) - For cryptographic random values
+- [Models](/guides/core-concepts/persistent/models) - using UIDs as primary keys
+- [Services](/guides/core-concepts/services) - generating unique IDs in services
+- [Helpers Overview](/extensions/helpers/) - all available helpers
+- [Crypto Helper](/extensions/helpers/crypto/) - cryptographic random values
+- [Snowflake ID](https://en.wikipedia.org/wiki/Snowflake_ID) - the algorithm this helper implements
 
-- **External Resources:**
-  - [Snowflake ID](https://en.wikipedia.org/wiki/Snowflake_ID) - Snowflake algorithm explained
-  - [Base62 Encoding](https://en.wikipedia.org/wiki/Base62) - Base62 encoding overview
+**Files:**
 
-- **Best Practices:**
-  - [Data Modeling](/best-practices/data-modeling) - ID generation strategies
+- [`packages/helpers/src/modules/uid/helper.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/uid/helper.ts) - `SnowflakeUidHelper`, `SnowflakeConfig`
+- [`packages/helpers/src/modules/uid/opaque.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/uid/opaque.ts) - `OpaqueUidHelper`
+- [`packages/helpers/src/modules/uid/common/constants.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/uid/common/constants.ts) - `UidAlphabets`, `UidCaseForms`
+- [`packages/helpers/src/modules/uid/index.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/uid/index.ts) - module barrel