@jimhoyd/urlcode 0.4.0-alpha.1 → 0.4.0-alpha.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (191) hide show
  1. package/.claude/skills/urlcode-authoring/SKILL.md +24 -8
  2. package/.claude/skills/urlcode-operations/SKILL.md +10 -16
  3. package/.claude-plugin/marketplace.json +1 -1
  4. package/README.md +75 -62
  5. package/ROADMAP.md +61 -16
  6. package/SECURITY.md +23 -8
  7. package/dist/BUILD-MANIFEST.json +38 -43
  8. package/dist/agents-guide.js +18 -18
  9. package/dist/build-static.js +134 -0
  10. package/dist/capabilities.js +85 -31
  11. package/dist/capability-query.js +0 -1
  12. package/dist/cli.js +23 -31
  13. package/dist/compliance-rules/baseline.js +2 -10
  14. package/dist/compliance-rules/privacy.js +5 -16
  15. package/dist/compliance-rules/shared.js +0 -2
  16. package/dist/compliance.js +6 -8
  17. package/dist/config.js +15 -8
  18. package/dist/context.js +9 -10
  19. package/dist/examples.js +2 -2
  20. package/dist/explain-cli.js +3 -3
  21. package/dist/explain.js +5 -6
  22. package/dist/extensions.js +61 -2
  23. package/dist/function-sources.js +34 -2
  24. package/dist/function-worker.js +3 -1
  25. package/dist/functions.js +84 -13
  26. package/dist/guest-api.js +29 -3
  27. package/dist/index.js +3 -6
  28. package/dist/manifest.js +5 -7
  29. package/dist/mcp.js +1 -1
  30. package/dist/observability.js +1 -21
  31. package/dist/policies/cache.js +8 -3
  32. package/dist/policy.js +28 -9
  33. package/dist/project-tests.js +3 -3
  34. package/dist/readiness.js +32 -9
  35. package/dist/router.js +5 -7
  36. package/dist/runtime.js +77 -58
  37. package/dist/sandbox.js +48 -0
  38. package/dist/scaffold.js +0 -0
  39. package/dist/scripts/operational-drills.js +12 -54
  40. package/dist/server.js +3 -29
  41. package/dist/tooling.js +1 -1
  42. package/dist/trusted-functions.js +210 -0
  43. package/dist/types/build-static.d.ts +43 -0
  44. package/dist/types/capabilities.d.ts +14 -5
  45. package/dist/types/compliance-rules/shared.d.ts +0 -2
  46. package/dist/types/compliance.d.ts +0 -3
  47. package/dist/types/config.d.ts +2 -1
  48. package/dist/types/context.d.ts +2 -1
  49. package/dist/types/examples.d.ts +1 -1
  50. package/dist/types/extensions.d.ts +55 -0
  51. package/dist/types/function-sources.d.ts +4 -0
  52. package/dist/types/functions.d.ts +48 -5
  53. package/dist/types/guest-api.d.ts +1 -0
  54. package/dist/types/index.d.ts +3 -6
  55. package/dist/types/manifest.d.ts +1 -3
  56. package/dist/types/observability.d.ts +1 -14
  57. package/dist/types/project-tests.d.ts +1 -2
  58. package/dist/types/readiness.d.ts +10 -3
  59. package/dist/types/router.d.ts +2 -1
  60. package/dist/types/runtime.d.ts +0 -27
  61. package/dist/types/sandbox.d.ts +12 -0
  62. package/dist/types/scaffold.d.ts +0 -2
  63. package/dist/types/server.d.ts +1 -4
  64. package/dist/types/tooling.d.ts +3 -3
  65. package/dist/types/trusted-functions.d.ts +29 -0
  66. package/dist/types/types.d.ts +14 -7
  67. package/dist/types/verify-deployment.d.ts +2 -2
  68. package/dist/types.js +17 -4
  69. package/dist/typescript-authoring.js +55 -17
  70. package/dist/verify-deployment.js +3 -3
  71. package/docs/AI-AUTHORING.md +104 -9
  72. package/docs/AWS.md +4 -4
  73. package/docs/BEST-PRACTICES.md +3 -2
  74. package/docs/CAPABILITIES.md +29 -16
  75. package/docs/CAPACITY.md +128 -31
  76. package/docs/CLOUDFLARE.md +1 -2
  77. package/docs/COMPLIANCE.md +6 -9
  78. package/docs/DEPLOYMENT-CHECKS.md +1 -1
  79. package/docs/EXTENSIONS.md +183 -11
  80. package/docs/FRAMEWORK.md +35 -19
  81. package/docs/FUNCTION-SECURITY.md +129 -32
  82. package/docs/INSTALL.md +0 -5
  83. package/docs/LOAD-TESTING.md +4 -4
  84. package/docs/MIDDLEWARE.md +27 -16
  85. package/docs/MONITORING.md +2 -19
  86. package/docs/NEXT-PHASE-PLAN.md +12 -4
  87. package/docs/NEXT-STEPS.md +259 -40
  88. package/docs/OBSERVABILITY.md +7 -16
  89. package/docs/OPEN-DECISIONS.md +212 -0
  90. package/docs/OPERATIONAL-PROOF.md +26 -30
  91. package/docs/OPERATIONS.md +23 -32
  92. package/docs/POLICIES.md +15 -4
  93. package/docs/PRERENDER.md +2 -2
  94. package/docs/PROJECT-DIRECTION.md +34 -9
  95. package/docs/README.md +13 -11
  96. package/docs/RELEASE-READINESS.md +49 -29
  97. package/docs/RELEASE-SECURITY.md +89 -3
  98. package/docs/RESILIENCE.md +16 -15
  99. package/docs/ROUTING.md +8 -10
  100. package/docs/SANDBOX-REVIEW.md +19 -6
  101. package/docs/SCAFFOLDING.md +0 -2
  102. package/docs/SECURITY-AUDIT.md +40 -0
  103. package/docs/SPECIFICATION.md +60 -26
  104. package/docs/SPIKE-AI-FRAMEWORK-BENCHMARK.md +287 -0
  105. package/docs/SPIKE-BUSINESS-SUITE.md +8 -0
  106. package/docs/SPIKE-CORE-LAYERING.md +337 -0
  107. package/docs/SPIKE-DEFAULT-TRUST-MODEL.md +209 -0
  108. package/docs/SPIKE-EXTENSION-MODEL.md +2 -2
  109. package/docs/SPIKE-EXTENSIONS.md +5 -0
  110. package/docs/SPIKE-LAMBDA-COMPILE.md +4 -2
  111. package/docs/SPIKE-MONOREPO.md +322 -0
  112. package/docs/STARTERS.md +1 -1
  113. package/docs/STATIC.md +94 -0
  114. package/docs/TOOLING.md +13 -9
  115. package/docs/TUNNELS.md +0 -3
  116. package/docs/TYPESCRIPT-AUTHORING.md +27 -12
  117. package/docs/TYPESCRIPT.md +25 -4
  118. package/docs/USABILITY-REVIEW.md +7 -1
  119. package/docs/VERCEL.md +4 -5
  120. package/docs/VERSION-ALIGNMENT.md +205 -0
  121. package/docs/YAML-GUIDE.md +0 -3
  122. package/docs/YAML-REFERENCE.md +16 -6
  123. package/docs/policies/cache.md +13 -0
  124. package/examples/assets/example.yaml +1 -1
  125. package/examples/aws/example.yaml +1 -1
  126. package/examples/cloudflare/example.yaml +1 -1
  127. package/examples/conditions/example.yaml +1 -1
  128. package/examples/cookbook/example.yaml +1 -1
  129. package/examples/cookbook/middleware/bucket.mjs +12 -2
  130. package/examples/cookbook/middleware/locale.mjs +7 -3
  131. package/examples/egress/example.yaml +1 -1
  132. package/examples/extensions/example.yaml +1 -1
  133. package/examples/prerender/README.md +2 -2
  134. package/examples/prerender/example.yaml +1 -1
  135. package/examples/provider-conformance/example.yaml +1 -1
  136. package/examples/vercel/example.yaml +1 -1
  137. package/llms-full.txt +594 -219
  138. package/llms.txt +23 -12
  139. package/package.json +10 -3
  140. package/packaging/claude-plugin/.claude-plugin/plugin.json +1 -1
  141. package/packaging/claude-plugin/skills/urlcode-authoring/SKILL.md +24 -8
  142. package/packaging/claude-plugin/skills/urlcode-operations/SKILL.md +10 -16
  143. package/recipes/authenticated-json-api/recipe.yaml +1 -1
  144. package/recipes/contact-form/recipe.yaml +1 -1
  145. package/recipes/cors-api/recipe.yaml +1 -1
  146. package/recipes/health-page/recipe.yaml +1 -1
  147. package/recipes/json-api/recipe.yaml +1 -1
  148. package/recipes/middleware/middleware/bucket.mjs +12 -2
  149. package/recipes/middleware/middleware/locale.mjs +7 -3
  150. package/recipes/middleware/recipe.yaml +1 -1
  151. package/recipes/protected-download/recipe.yaml +1 -1
  152. package/recipes/redirect/recipe.yaml +1 -1
  153. package/recipes/static-plus-api/recipe.yaml +1 -1
  154. package/recipes/typescript/README.md +2 -1
  155. package/recipes/typescript/recipe.yaml +1 -1
  156. package/recipes/webhook-receiver/README.md +5 -1
  157. package/recipes/webhook-receiver/recipe.yaml +2 -1
  158. package/recipes/webhook-receiver/urlcode.yaml +7 -0
  159. package/schemas/recipe.schema.json +4 -3
  160. package/schemas/urlcode.schema.json +14 -41
  161. package/skills/urlcode/SKILL.md +29 -8
  162. package/starters/default/AGENTS.md +18 -18
  163. package/starters/default/urlcode.yaml +0 -1
  164. package/dist/link-api.js +0 -136
  165. package/dist/link-cli.js +0 -141
  166. package/dist/link-events.js +0 -76
  167. package/dist/link-records.js +0 -31
  168. package/dist/link-store-worker.js +0 -150
  169. package/dist/link-store.js +0 -250
  170. package/dist/management-policy.js +0 -40
  171. package/dist/sqlite-version.js +0 -6
  172. package/dist/types/link-api.d.ts +0 -30
  173. package/dist/types/link-cli.d.ts +0 -37
  174. package/dist/types/link-events.d.ts +0 -27
  175. package/dist/types/link-records.d.ts +0 -11
  176. package/dist/types/link-store-worker.d.ts +0 -1
  177. package/dist/types/link-store.d.ts +0 -130
  178. package/dist/types/management-policy.d.ts +0 -8
  179. package/dist/types/sqlite-version.d.ts +0 -1
  180. package/docs/DYNAMIC-LINKS.md +0 -61
  181. package/docs/MANAGEMENT-SECURITY.md +0 -82
  182. package/docs/links/cli.md +0 -110
  183. package/docs/links/limits.md +0 -175
  184. package/docs/links/management-api.md +0 -80
  185. package/docs/links/pools.md +0 -75
  186. package/docs/links/setup.md +0 -135
  187. package/docs/yaml/links.md +0 -30
  188. package/examples/live-links/README.md +0 -11
  189. package/examples/live-links/example.yaml +0 -21
  190. package/examples/live-links/tests/requests.json +0 -6
  191. package/examples/live-links/urlcode.yaml +0 -16
@@ -1,130 +0,0 @@
1
- import type { LinkRecord } from './link-records.ts';
2
- import type { LogFn } from './types.ts';
3
- /** A stored link as the worker returns it. */
4
- export interface LinkRow extends LinkRecord {
5
- collection: string;
6
- code: string;
7
- version: number;
8
- }
9
- export interface ListOptions {
10
- limit?: number | undefined;
11
- after?: string | undefined;
12
- }
13
- export interface AuditIdentity {
14
- actor?: string | undefined;
15
- requestId?: string | undefined;
16
- }
17
- export interface ExportHeader {
18
- format: string;
19
- schemaVersion: number;
20
- applicationId: number;
21
- collection: string | null;
22
- revision: number;
23
- records: number;
24
- generatedAt: string;
25
- }
26
- export interface ExportPageOptions {
27
- afterCollection?: string | undefined;
28
- afterCode?: string | undefined;
29
- limit?: number | undefined;
30
- }
31
- export interface ExportOptions {
32
- collection?: string | undefined;
33
- pageSize?: number | undefined;
34
- deadlineMs?: number | undefined;
35
- }
36
- export interface ExportHandlers {
37
- onHeader?: ((header: ExportHeader) => unknown) | undefined;
38
- onRecords?: ((records: LinkRow[]) => unknown) | undefined;
39
- }
40
- export interface ExportSummary extends ExportHeader {
41
- exported: number;
42
- }
43
- export interface PoolStats {
44
- connections: number;
45
- healthyConnections: number;
46
- limit: number;
47
- inFlight: number;
48
- completed: number;
49
- failed: number;
50
- rejected: number;
51
- durationMs: number;
52
- }
53
- export interface LinkStoreStats {
54
- closed: boolean;
55
- exporting: boolean;
56
- read: PoolStats;
57
- write: PoolStats;
58
- }
59
- export interface LinkStoreWorkerData {
60
- file: string;
61
- readOnly: boolean;
62
- }
63
- export type LinkStoreOperation = 'get' | 'list' | 'create' | 'update' | 'delete' | 'exportBegin' | 'exportPage' | 'exportEnd' | 'close';
64
- export interface LinkStoreArgs extends ListOptions, ExportPageOptions {
65
- collection?: string | undefined;
66
- code?: unknown;
67
- data?: unknown;
68
- expectedVersion?: unknown;
69
- audit?: AuditIdentity | undefined;
70
- }
71
- export interface LinkStoreCommand {
72
- id: number;
73
- operation: LinkStoreOperation;
74
- args: LinkStoreArgs;
75
- }
76
- export type LinkStoreReply = {
77
- ready: true;
78
- } | {
79
- failed: true;
80
- } | {
81
- id: number;
82
- value: unknown;
83
- } | {
84
- id: number;
85
- error: {
86
- status: number;
87
- message: string;
88
- };
89
- };
90
- /** One worker-backed SQLite connection. */
91
- export interface LinkConnection {
92
- readonly healthy: boolean;
93
- get(collection: string, code: string): Promise<LinkRow | null>;
94
- list(collection: string, options?: ListOptions): Promise<LinkRow[]>;
95
- exportBegin(options?: {
96
- collection?: string | undefined;
97
- }): Promise<ExportHeader>;
98
- exportPage(options?: ExportPageOptions): Promise<LinkRow[]>;
99
- exportEnd(): Promise<boolean>;
100
- create(collection: string, data: unknown, code?: unknown, audit?: AuditIdentity | undefined): Promise<LinkRow>;
101
- update(collection: string, code: string, data: unknown, expectedVersion: unknown, audit?: AuditIdentity | undefined): Promise<LinkRow>;
102
- delete(collection: string, code: string, expectedVersion: unknown, audit?: AuditIdentity | undefined): Promise<boolean>;
103
- close(): Promise<void>;
104
- }
105
- /** The pooled store openLinkStore returns. */
106
- export interface LinkStore {
107
- readonly readHealthy: boolean;
108
- readonly writeHealthy: boolean;
109
- readonly healthy: boolean;
110
- readonly atomicAudit: true;
111
- stats(): LinkStoreStats;
112
- exportSnapshot(options?: ExportOptions, handlers?: ExportHandlers): Promise<ExportSummary>;
113
- get(collection: string, code: string): Promise<LinkRow | null>;
114
- list(collection: string, options?: ListOptions): Promise<LinkRow[]>;
115
- create(collection: string, data: unknown, code?: unknown, audit?: AuditIdentity | undefined): Promise<LinkRow>;
116
- update(collection: string, code: string, data: unknown, expectedVersion: unknown, audit?: AuditIdentity | undefined): Promise<LinkRow>;
117
- delete(collection: string, code: string, expectedVersion: unknown, audit?: AuditIdentity | undefined): Promise<boolean>;
118
- close(): Promise<void>;
119
- }
120
- export interface LinkStoreOptions {
121
- file?: string | undefined;
122
- project?: string | undefined;
123
- readOnly?: boolean | undefined;
124
- readers?: number | undefined;
125
- maxReads?: number | undefined;
126
- maxWrites?: number | undefined;
127
- log?: LogFn | undefined;
128
- }
129
- export declare function outsideProject(file: unknown, project: string): Promise<string>;
130
- export declare function openLinkStore({ file, project, readOnly, readers, maxReads, maxWrites, log }?: LinkStoreOptions): Promise<LinkStore>;
@@ -1,8 +0,0 @@
1
- /** A credential from the operator's management policy file, once it authenticated a request. */
2
- export interface ManagementPrincipal {
3
- id: string;
4
- collections: string[];
5
- actions: string[];
6
- }
7
- export type ManagementAuthorizer = (token: string) => Promise<ManagementPrincipal | undefined>;
8
- export declare function managementPolicy(file: unknown, project: string): Promise<ManagementAuthorizer>;
@@ -1 +0,0 @@
1
- export declare function supportsConcurrentWal(version: unknown): boolean;
@@ -1,61 +0,0 @@
1
- # Dynamic short links without reloads
2
-
3
- Define a stable route once and create, update and delete short-code records
4
- while `serve` keeps running, from optional SQLite storage on one host. Each
5
- section below lives on its own page; the headings here keep existing links working.
6
-
7
- ## Node build requirement
8
-
9
- SQLite 3.51.3+ (or 3.50.7 / 3.44.6) bundled with Node; `urlcode doctor` reports it. Read [setup](links/setup.md#node-build-requirement).
10
-
11
- ## Behavior in YAML, data outside Git
12
-
13
- The `link` handler and the operator store binding. Read [setup](links/setup.md#behavior-in-yaml-data-outside-git).
14
-
15
- ## Run the included example
16
-
17
- `links init`, `links create` and `serve --link-store` against a private data directory. Read [setup](links/setup.md#run-the-included-example).
18
-
19
- ## Update, disable, expire, list and delete
20
-
21
- `links get/list/update/delete` and the optimistic version rules. Read [cli](links/cli.md#update-disable-expire-list-and-delete).
22
-
23
- ## Consistent operator export and restore
24
-
25
- `links export` holds one read transaction; `links import` refuses occupied collections. Read [cli](links/cli.md#consistent-operator-export-and-restore).
26
-
27
- ## A separate authenticated management API
28
-
29
- `links api`: bearer token, endpoints, status codes and scope. Read [management-api](links/management-api.md#a-separate-authenticated-management-api).
30
-
31
- ## Persistence, bounds and recovery
32
-
33
- Pools, deadlines, worker replacement with backoff, record caps and backups. Read [limits](links/limits.md#persistence-bounds-and-recovery).
34
-
35
- ## Middleware, sandbox and tests
36
-
37
- What guest code can and cannot see; fixtures for `test` and `audit`. Read [setup](links/setup.md#middleware-sandbox-and-tests).
38
-
39
- ## Opt-in completed-redirect events
40
-
41
- The `linkEvents` observer: outcomes, redaction, bounded delivery. Read [limits](links/limits.md#opt-in-completed-redirect-events).
42
-
43
- ## Shutdown and management defaults
44
-
45
- Loopback defaults, drain on close, unknown outcomes after a timeout. Read [pools](links/pools.md#shutdown-and-management-defaults).
46
-
47
- ## Explicit project opt-in
48
-
49
- `dynamicLinks: true` in the entry file only; what enabling changes. Read [setup](links/setup.md#explicit-project-opt-in).
50
-
51
- ## Separate reader and writer pools
52
-
53
- `--link-readers`, `--link-read-limit`, `--link-write-limit` and `stats()`. Read [pools](links/pools.md#separate-reader-and-writer-pools).
54
-
55
- ## Management HTTP and audit safeguards
56
-
57
- Admission, socket timeout and `management_request` events. Read [management-api](links/management-api.md#management-http-and-audit-safeguards).
58
-
59
- ## Management hardening baseline
60
-
61
- Loopback only, `--auth-file` credentials, durable audit rows. Read [management-api](links/management-api.md#management-hardening-baseline).
@@ -1,82 +0,0 @@
1
- # Private management and durable mutation audit
2
-
3
- Management binds only `127.0.0.1` or `::1`. Use an authenticated private tunnel
4
- (e.g. SSH/VPN with loopback forwarding); never publish its port through a public
5
- proxy or container port mapping. This is operator management, not an end-user
6
- account service. Browser Origin requests remain rejected.
7
-
8
- ## Individual credentials
9
-
10
- Use `urlcode links api --project ./my-links --store /operator/links.sqlite
11
- --auth-file /operator/management.json --host 127.0.0.1 --port 3001` (one line).
12
- The policy is operator-owned, outside the application, at most 64 KiB and mode
13
- 600 on POSIX. Protect its parent directory and apply equivalent Windows ACLs.
14
- It is not YAML and must never be placed in the application or Git.
15
-
16
- Generate each token with `node -e "console.log(require('node:crypto').randomBytes(32).toString('base64url'))"`
17
- on a trusted operator machine. Deliver it through your secret manager. Hash the
18
- raw token using SHA-256 without a newline; the server expects `Authorization:
19
- Bearer <token>`. Tokens must contain 43–256 base64url characters. Never use a
20
- human password as a token. This synthetic policy illustrates the shape; replace
21
- the hash with the real token hash and choose a short operational expiry:
22
-
23
- ```json
24
- {
25
- "version": 1,
26
- "credentials": [{
27
- "id": "operator-alice",
28
- "sha256": "0000000000000000000000000000000000000000000000000000000000000000",
29
- "expires": "2026-10-01T00:00:00Z",
30
- "collections": ["links"],
31
- "actions": ["get", "list", "create", "update", "delete"]
32
- }]
33
- }
34
- ```
35
-
36
- At most 128 credentials; unique IDs and hashes; explicit collection and action
37
- allowlists with no wildcards. Readers should receive only `get` and `list`.
38
- An expired, removed or `"revoked": true` credential gets 401; a valid credential
39
- outside its scope gets 403. The entire policy is validated on every authenticated
40
- request. Malformed/unreadable policy fails closed with 503, without retaining an
41
- old permissive copy. Write a replacement file with mode 600 and atomically rename
42
- it over the policy. No restart is needed. Requests already authorized may finish;
43
- revocation does not cancel an in-flight transaction. Credential administration is
44
- an operator filesystem action, never exposed through this HTTP API.
45
-
46
- Legacy `--token-file` remains available for local compatibility, with full access
47
- to that listener's collection and actor `legacy-shared`. Prefer `--auth-file` for
48
- attribution, expiry and revocation. They are mutually exclusive. These bearer
49
- credentials do not establish verified human identity, MFA, SSO or session login.
50
- Host administrators and custom embedding code remain trusted operators.
51
-
52
- ## Audit durability and recovery
53
-
54
- The built-in SQLite writer creates an additive `urlcode_link_audit` table when
55
- opening a store. Every successful create/update/delete (including CLI and embedded
56
- store calls) records revision, UTC timestamp, actor, request ID, collection,
57
- action and SHA-256 of the short code in the **same transaction** as the mutation.
58
- The default local CLI actor is `local-operator`. Scoped HTTP management requires
59
- an adapter explicitly declaring atomic audit support; the built-in store provides it.
60
-
61
- An audit insert failure rolls back both mutation and revision. Conditional-write
62
- conflicts and rejected requests produce no successful-mutation row. HTTP request
63
- logs still describe failed/aborted requests and remain best effort. A client timeout
64
- may follow a committed transaction: reconcile the revision and request ID before
65
- retrying. A lost HTTP reply is not evidence that the transaction was rolled back.
66
-
67
- Inspect the journal with an operator-only SQLite client, for example:
68
-
69
- ```sql
70
- SELECT revision, timestamp, actor, request_id, collection, action, code_sha256
71
- FROM urlcode_link_audit WHERE revision > 0 ORDER BY revision LIMIT 100;
72
- ```
73
-
74
- No token, URL, request body or raw short code is stored in the journal. Code hashes
75
- are correlatable and guessable for short codes; restrict access accordingly.
76
- This is a durable local transaction journal, **not tamper-evident external storage**.
77
- An administrator with database access can change it. Export ordered revisions to
78
- your protected collector/backup system and monitor lag, gaps and disk/WAL growth.
79
- There is no automatic pruning: define retention and verify archival before any
80
- operator purge. Keep audit records in backups. User_version remains 1 because
81
- this table is additive; older readers work, but older writers do not emit audit
82
- rows. Never roll a management writer back to an unaudited revision.
package/docs/links/cli.md DELETED
@@ -1,110 +0,0 @@
1
- # Dynamic links: CLI: create, update, export and restore
2
-
3
- Part of [dynamic links](../DYNAMIC-LINKS.md), which indexes every page.
4
-
5
- ## Update, disable, expire, list and delete
6
-
7
- ```sh
8
- urlcode links get --store /absolute/links.sqlite --code demo
9
- urlcode links list --store /absolute/links.sqlite --limit 100
10
- urlcode links list --store /absolute/links.sqlite --limit 100 --after last-code
11
- urlcode links update --store /absolute/links.sqlite --code demo \
12
- --if-version 1 --destination https://example.com/new --enabled false
13
- urlcode links delete --store /absolute/links.sqlite --code demo --if-version 2
14
- ```
15
-
16
- Replace illustrative versions with the actual returned `version`. Update is a
17
- **full replacement**: omitted status becomes 302, enabled becomes true, and expiry
18
- becomes null. Use `--status 307` or `--expires 2030-01-01T00:00:00Z` as needed.
19
- Allowed status codes match ordinary redirects. Disabled/expired records remain
20
- stored and reserve the code until explicitly deleted.
21
-
22
- Create is insert-only: an existing code returns conflict, never an overwrite.
23
- Updates/deletes require a matching positive version. Every mutation uses a
24
- transaction; revisions advance globally within that database, including deletion,
25
- so a stale edit cannot silently affect a deleted-and-recreated code. Competing
26
- writers receive a conflict. Read the latest record before making a new decision;
27
- do not blindly retry a stale update. Listing is ordered by code, at most 100 per
28
- page, within one collection. Pagination is not a snapshot across concurrent writes.
29
-
30
- CLI commands print record data intentionally; treat output as operational data.
31
- Do not store secrets in destinations or capture output into public logs.
32
- `urlcode add` remains the separate command for adding a Git/YAML-defined redirect.
33
-
34
- ## Consistent operator export and restore
35
-
36
- Listing pages one after another is not a snapshot: inserts, updates and deletes
37
- between pages can produce a logically inconsistent copy. `links export` instead
38
- holds one SQLite read transaction for the whole export, so every record it writes
39
- comes from a single point in time.
40
-
41
- ```sh
42
- umask 077
43
- urlcode links export --store /absolute/links.sqlite > /absolute/backups/links-export.ndjson
44
- urlcode links import --store /absolute/restored.sqlite --input /absolute/backups/links-export.ndjson
45
- ```
46
-
47
- Export is an operator command on the operator's own database. It is not reachable
48
- from the public redirect server, from route YAML, from guest function code or from
49
- the management HTTP API, and it grants guest code no storage capability. The
50
- output is operational data: write it somewhere only operators can read, keep it out
51
- of the project, Git and build artifacts, and treat it like the database itself.
52
-
53
- **Consistency contract.** The export reflects the database exactly as of the moment
54
- the snapshot is pinned, which is the first read after the transaction opens. Writers
55
- are never blocked and keep committing; none of their later commits appear in the
56
- export, and no record appears twice or is skipped. The header line carries
57
- `format`, `schemaVersion`, `applicationId`, the store `revision` at that instant and
58
- `generatedAt`, so a restored copy can be identified and ordered against others. This
59
- is a consistent logical copy, not a point-in-time recovery system: it has no
60
- continuous log and cannot reconstruct a moment between two exports.
61
-
62
- **Contents.** Every record in the store, or in one `--collection`, including
63
- enabled, disabled and expired records, with `collection`, `code`, `url`, `status`,
64
- `enabled`, `expires` and `version`. Disabled and expired records are exported
65
- because they still reserve their codes. The output is NDJSON: a
66
- `link-export-begin` header line, one `{"record": {...}}` line per record ordered by
67
- collection and code, and a `link-export-complete` line carrying the record count
68
- and a SHA-256 digest over every preceding line. A stream without that final line is
69
- truncated, and `links import` rejects it.
70
-
71
- **Restore semantics.** `links import` replays the records into the target store and
72
- refuses to touch a collection that already holds records, so a restore never
73
- overwrites live data. It verifies the format, schema version, store identity, record
74
- count and digest before it commits the last record. Codes, destinations, statuses,
75
- enabled flags and expiries are restored exactly. **Versions are not.** The target
76
- assigns its own revisions, which is why the report sets `versionsReassigned: true`:
77
- management ETags taken against the exported database are stale after a restore, so
78
- discard them and re-read records before the next conditional write, exactly as after
79
- restoring an older database file.
80
-
81
- **Audit journal.** The export carries records only. The mutation audit journal stays
82
- in the source database and is not part of an export, so a store restored from one
83
- starts a fresh journal covering only mutations made after the restore. Keep the
84
- journal by backing up the database file itself, as described under *Persistence,
85
- bounds and recovery*; that file backup, not the export, is the archival copy of who
86
- changed what. See [management security](../MANAGEMENT-SECURITY.md) for retention.
87
-
88
- **Bounds.** One export runs at a time per store and a second is rejected with 409.
89
- It pins exactly one reader connection and holds that reader's admission for its whole
90
- life, so it can never exceed the pool's read budget or starve redirects of every
91
- reader — run it against a management store or size `--link-readers` accordingly.
92
- Pages are at most 100 records (`--page-size`), each page carries the usual
93
- five-second operation deadline, and the export as a whole has a 60-second default
94
- deadline after which it fails and releases the reader. A consumer that fails or a
95
- process that stops ends the read transaction rather than leaving it open. Because
96
- the transaction pins a WAL read mark, a long export delays WAL checkpointing: keep
97
- exports short and do not leave one running against a busy store.
98
-
99
- Embedders call the same mechanism directly:
100
-
101
- ```js
102
- await store.exportSnapshot({collection: 'links', pageSize: 100, deadlineMs: 60000}, {
103
- onHeader: header => sink.write(header),
104
- onRecords: records => sink.write(records),
105
- });
106
- ```
107
-
108
- `onRecords` is awaited, so a slow sink applies backpressure to the export instead of
109
- buffering the store in memory; anything it throws aborts the export and releases the
110
- reader. `stats()` reports `exporting` while one is in flight.
@@ -1,175 +0,0 @@
1
- # Dynamic links: Persistence, bounds, recovery and events
2
-
3
- Part of [dynamic links](../DYNAMIC-LINKS.md), which indexes every page.
4
-
5
- ## Persistence, bounds and recovery
6
-
7
- SQLite operations use separate reader and writer pools, outside the HTTP event
8
- loop and function workers. Default: two read-only worker connections plus one
9
- writer for writable stores; public serving opens readers only. Reads and writes
10
- have independent 32-operation admission limits and 5-second deadlines including
11
- waiting. Lock wait is one second. Excess work returns 503. Failed connections
12
- are excluded from selection and readiness degrades; surviving readers can still
13
- serve requests. Plain YAML redirects remain independent.
14
-
15
- Startup and established-worker failures recover differently.
16
-
17
- * **Startup failure.** A connection that never reports ready is terminated and its
18
- error is returned to the caller: `openLinkStore` rejects and activation fails
19
- closed. Nothing is retried behind the operator's back, so a `serve` or `links api`
20
- process that cannot open the store does not start, and a reload that cannot open
21
- it keeps the last-good runtime.
22
- * **Established-worker failure.** A connection that had been serving and then
23
- errors, exits or misses an operation deadline is replaced automatically.
24
- In-flight operations on it reject with 503, the connection is marked unhealthy
25
- and excluded from selection, and a replacement worker is launched after an
26
- exponential backoff from 250 ms up to 30 seconds. Each attempt emits a
27
- `link_store_worker` event with `status: "restarting"`, the attempt number and the
28
- delay; a replacement that serves an operation resets the backoff, and one that
29
- starts but dies on every operation keeps backing off instead of spinning. Close
30
- cancels a pending replacement.
31
-
32
- While a pool member is down the pool is degraded, not off: `readHealthy`,
33
- `writeHealthy` and `healthy` report false and readiness degrades, but surviving
34
- readers keep answering and a recovered writer resumes accepting mutations with no
35
- operator action. Records live on disk, so a replaced connection loses no committed
36
- data.
37
-
38
- Restart the process when recovery cannot help: an unsuitable Node/SQLite build,
39
- missing or invalid revision metadata, an incompatible schema, a store file that was
40
- replaced, moved or symlinked under a running connection, or a host-level fault such
41
- as a full or read-only disk. Those fail activation rather than reconnecting, and
42
- the replacement worker will keep failing until the underlying cause is fixed.
43
-
44
- Automatic connection recovery does not make writes idempotent. A mutation whose
45
- reply was lost to a worker failure or deadline may still have committed, and the
46
- records it touched carry versions that advance globally. Callers must therefore
47
- re-read the record and decide again instead of blindly retrying a write; the
48
- optimistic-version rules under *Update, disable, expire, list and delete* apply
49
- unchanged. Recovery behavior is covered by the `acknowledged writes survive abrupt
50
- writer exit and pagination retains records`, `a blocked writer does not occupy read
51
- connections and recovers after lock release` and `stores with missing revision
52
- metadata fail activation` cases in `test/links.test.ts`.
53
-
54
- The initial store has a 100,000-record cap across collections and an 8,192-byte
55
- normalized destination limit. WAL + FULL synchronous commits provide transactional
56
- persistence subject to the disk/filesystem's guarantees. The format has an
57
- application identifier and schema version; incompatible databases fail activation.
58
- Use trusted local storage and a protected parent directory. Do not replace,
59
- symlink or move an open database or its WAL/SHM files. The database and token must
60
- be outside the project; keep them outside public directories, Git and artifacts.
61
-
62
- An acknowledged mutation is committed. If a caller loses the response or receives
63
- a timeout, the write may nevertheless have committed: inspect state before retry.
64
- For retryable creation, choose a stable code and resolve conflicts; automatic
65
- code generation cannot give exactly-once semantics after a lost response.
66
-
67
- `links export` gives a consistent logical copy of the records while the store keeps
68
- serving; it does not replace a file backup, which is what preserves the audit
69
- journal and the exact record versions. For offline backups, stop management writers and all readers, then copy the
70
- database together with any remaining WAL file as one consistent stopped set,
71
- preserving their matching basenames. Restore into a separate private directory
72
- while no connection is open. Do not discard a WAL just because the app stopped.
73
- For online backups, use SQLite-aware tooling rather than copying only the live
74
- main file. SQLite's [WAL documentation](https://www.sqlite.org/wal.html) explains why
75
- committed state may still be in the WAL. Test restores on a separate closed store.
76
- Restoring an older database also restores older record versions: discard old
77
- management ETags and re-read records after restore. This is not a replication or
78
- point-in-time recovery system. Define retention, RPO/RTO and disk limits yourself.
79
-
80
- Multiple same-host processes can share the local file; a distributed deployment
81
- needs another adapter. The trusted embedding API accepts
82
- `linkStores: {links: adapter}` where `get(collection, code)` resolves to null or a
83
- record with url/status/enabled/expires. The caller owns adapter shutdown and must
84
- provide bounded operations, validation and consistency. Optional `healthy=false`
85
- makes readiness fail. `openLinkStore` provides the built-in implementation plus
86
- create/update/delete/list/exportSnapshot/close methods. Adapter code is operator code, never
87
- loaded from route YAML. No remote provider adapter ships in this release.
88
-
89
- ## Opt-in completed-redirect events
90
-
91
- Default request logs stay minimal: they carry status and timing, and with
92
- `--request-log detailed` the method and the configured route pattern. They never
93
- carry a short code or a request target. Counting store lookups is not a substitute
94
- either, because a lookup cannot tell a completed redirect from a HEAD probe, an
95
- error or a client that disconnected.
96
-
97
- A trusted operator embedding the runtime can instead enable a post-response
98
- observer. It is explicitly enabled in operator code, off by default, and there is no
99
- `serve` flag and no YAML setting for it: route YAML cannot name a callback, and no
100
- untrusted code is ever loaded as one.
101
-
102
- ```js
103
- import {startServer} from '@jimhoyd/urlcode';
104
-
105
- await startServer({
106
- project: './links',
107
- linkStore: {collection: 'links', file: '/absolute/links.sqlite'},
108
- linkEvents: {
109
- observe: event => collector.record(event), // operator code, awaited off the request path
110
- includeCode: false, // set true to disclose the short code to this collector
111
- maxQueue: 256, // 1–4096 events; excess is dropped and counted
112
- timeoutMs: 1000, // 1–10000 ms budget per observer call
113
- },
114
- });
115
- ```
116
-
117
- In TypeScript the collector's argument is `LinkEvent` and the option block is
118
- `LinkObserverOptions`, both exported from `urlcode` beside `LinkStore`,
119
- `LinkRow`, `LinkStoreOptions`, `LinkApi` and `LinkApiOptions`; the declarations
120
- ship with the package:
121
-
122
- ```ts
123
- import { startServer, type LinkEvent, type LinkObserverOptions } from '@jimhoyd/urlcode';
124
-
125
- const linkEvents: LinkObserverOptions = {
126
- observe: (event: LinkEvent) => collector.record(event), // event.code is null for an invalid code
127
- includeCode: false,
128
- };
129
- await startServer({ project: './links', linkStore: { collection: 'links', file: '/absolute/links.sqlite' }, linkEvents });
130
- ```
131
-
132
- Each event is `{event: 'link_request', requestId, collection, route, method, status,
133
- outcome, durationMs}`, plus `code` only when `includeCode` is true. `route` is the
134
- configured route pattern, never the request target. Nothing else from the request is
135
- included: no token, destination URL, query string, headers, body, cookie or client
136
- IP address, and no stored record. Only stored-link routes produce events; a plain
137
- YAML redirect never does. Disclosing a short code identifies a link, so treat a
138
- collector that receives one as holding operational data and keep it off public logs.
139
-
140
- | `outcome` | Meaning |
141
- |---|---|
142
- | `completed` | The redirect response finished. With `method: "GET"` this is the closest thing to a click; `HEAD` is a probe, not a click |
143
- | `aborted` | A redirect was produced but the response never finished, because the peer disconnected |
144
- | `missing` | No record for that code |
145
- | `disabled` | The record exists but is disabled |
146
- | `expired` | The record exists but its expiry has passed |
147
- | `invalid_code` | The code failed route input validation |
148
- | `invalid_record` | The stored record failed validation |
149
- | `unavailable` | The store was unavailable or over its admission budget |
150
-
151
- Nothing here is a human click count. Bots, prefetchers, proxies and repeat requests
152
- all produce `completed` events, the runtime does not deduplicate, and browser and
153
- proxy caches mean a real navigation may produce no request at all. Do the
154
- interpretation in your own collector.
155
-
156
- **The observer cannot affect a redirect.** Events are enqueued after the response
157
- finishes or the connection closes, never before, so an observer cannot delay,
158
- rewrite or fail a redirect. Delivery is sequential and bounded: at most `maxQueue`
159
- events are held, each call gets `timeoutMs` and a slow or hung collector is
160
- abandoned rather than allowed to pin the queue, and a call that throws is counted
161
- instead of propagated. Drops and failures are counted, reported through
162
- `link_observer` events on the normal log and readable at any time through
163
- `app.linkEventStats()` as `{queued, delivered, dropped, failed, timedOut, closed}`.
164
- An overloaded collector loses events, by design, instead of growing memory.
165
-
166
- Shutdown closes the observer after the server's connections are gone, drains what
167
- was already accepted within one bounded deadline, drops the rest and emits a final
168
- `link_observer` event with `status: "closed"` and those totals. Events are
169
- best-effort operational signals, not durable analytics or an audit record: the
170
- durable, atomic record of mutations remains the store's audit journal.
171
-
172
- Tests cover GET and HEAD, completed and aborted responses, missing, disabled and
173
- expired records, default redaction and opt-in code disclosure, a failing collector,
174
- a hung collector hitting its budget, queue overflow with counted drops, and drain on
175
- shutdown.
@@ -1,80 +0,0 @@
1
- # Dynamic links: The management API
2
-
3
- Part of [dynamic links](../DYNAMIC-LINKS.md), which indexes every page.
4
-
5
- ## A separate authenticated management API
6
-
7
- For a web product, your trusted backend calls the management API after applying
8
- its own user authentication, authorization, quotas and abuse rules. The API is an
9
- operator interface, not a public anonymous link-creation endpoint. Never put its
10
- shared token in browser JavaScript. It is not available on the public route server.
11
-
12
- Generate a private token file outside the application (POSIX-compatible Node example):
13
-
14
- ```sh
15
- node --input-type=module -e 'import {writeFileSync} from "node:fs"; import {randomBytes} from "node:crypto"; writeFileSync(process.argv[1],randomBytes(32).toString("base64url"),{mode:0o600,flag:"wx"})' /absolute/link-admin.token
16
- urlcode links api --store /absolute/links.sqlite --collection links \
17
- --token-file /absolute/link-admin.token --host 127.0.0.1 --port 3001
18
- ```
19
-
20
- Use a securely generated token, at least 43 base64url characters. File permissions
21
- must exclude group/other access on POSIX; protect Windows files with operator ACLs.
22
- The API reads the token at startup; rotate by replacing it and restarting this
23
- management process. This does not require restarting public resolution.
24
-
25
- Every request needs `Authorization: Bearer <token>`. Authenticate before body
26
- processing. Use application/json for POST/PUT, with no compression; max body is
27
- 16 KiB. Browser Origin-bearing requests are rejected. No CORS, cookies, user
28
- accounts, per-user scopes, JWT system or automatic rate limiter is provided.
29
- Keep it on a private network/loopback behind authenticated TLS ingress as needed.
30
- If changing `--host`, protect it before exposing it; no built-in HTTPS exists.
31
-
32
- | Request | Result |
33
- |---|---|
34
- | `POST /v1/links` with `{ "code": "demo", "url": "https://example.com/demo" }` | 201 with record and ETag; code optional |
35
- | `GET /v1/links/demo` | 200 with record and ETag, or 404 |
36
- | `GET /v1/links?limit=100&after=demo` | `{items, nextAfter}`; nextAfter null on a short page |
37
- | `PUT /v1/links/demo`, `If-Match: "VERSION"`, complete record fields | Replace and return new record/ETag |
38
- | `DELETE /v1/links/demo`, `If-Match: "VERSION"` | 204 |
39
-
40
- POST accepts optional `code` plus `url`, optional `status`, `enabled`, `expires`.
41
- PUT accepts the same record fields except code, which is immutable. `expires`
42
- may be null. Responses include collection, code, url, status, enabled, expires,
43
- version. Missing precondition: 428; stale version or duplicate code: 409;
44
- invalid input: 400; authentication failure: 401; unsupported media: 415;
45
- record limit: 507; store failure/capacity: 503. Error messages omit credentials,
46
- submitted URLs and SQL details. A full page can return a cursor even when the
47
- next page will be empty. No bulk mutation API is implemented, and the management
48
- API exposes no export: sequential list pages are not a snapshot, so use the
49
- operator `links export` command above for a consistent copy.
50
-
51
- A token authorizes its configured collection, not all collections. There are no
52
- per-end-user permissions: those belong to your backend. The API can also list
53
- inactive records for management. The public `/r/{code}` route never exposes
54
- management JSON, token files or a mutation endpoint.
55
-
56
- ## Management HTTP and audit safeguards
57
-
58
- The private API admits up to 32 in-flight HTTP requests through response finish
59
- or disconnect, returning 503 on overload. A 10-second socket inactivity timeout
60
- closes stalled peers; this is not a total response deadline. The embedding API
61
- accepts `maxInFlightRequests` (1–64) and `socketTimeoutMs` (100–60,000). Existing
62
- connection/body/header and database admission limits still apply. Noncanonical
63
- dot-segment/backslash path normalization is rejected.
64
-
65
- JSON `management_request` events go to stdout by default and include timestamp,
66
- request ID, collection, action, authentication result, status and finish/abort
67
- outcome. No token, code, destination, URL, headers or body is logged. Embedders
68
- can supply `log(event)`; failed sinks cannot crash request handling. These are
69
- best-effort operational events, not durable per-actor audit records. Aborted
70
- mutations may have committed: re-read state before retrying. See the
71
- [security review](../SECURITY-AUDIT.md) for remaining controls.
72
-
73
- ## Management hardening baseline
74
-
75
- Management is now restricted to literal loopback addresses. Prefer `--auth-file`
76
- for individual expiring, revocable credentials with collection/action scopes.
77
- Every successful built-in store mutation has an atomic, durable SQLite audit row;
78
- HTTP request logs remain best effort. See [management security](../MANAGEMENT-SECURITY.md)
79
- for policy examples, compatibility, archival and rollback requirements, and
80
- [operational proof](../OPERATIONAL-PROOF.md) for executable recovery drills.