@pikku/kysely 0.13.38 → 0.13.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/bin/pikku-kysely-pure.js +0 -0
- package/package.json +15 -9
- package/CHANGELOG.md +0 -2367
- package/knowledge/decisions/a-console-role-cannot-take-a-system-roles-name.md +0 -25
- package/knowledge/decisions/a-credential-with-no-user-is-one-row-per-name.md +0 -22
- package/knowledge/decisions/a-lease-is-taken-by-one-guarded-update.md +0 -30
- package/knowledge/decisions/a-virtual-users-schedule-starts-disabled.md +0 -23
- package/knowledge/decisions/classification-driven-encryption-happens-in-a-plugin.md +0 -71
- package/knowledge/decisions/grants-reference-declared-scopes-and-users-by-foreign-key.md +0 -29
- package/knowledge/decisions/index.md +0 -21
- package/knowledge/decisions/keks-are-scoped-by-purpose-as-well-as-version.md +0 -45
- package/knowledge/decisions/leases-are-judged-by-the-database-clock.md +0 -42
- package/knowledge/decisions/the-plugin-is-given-a-resolver-never-a-key.md +0 -45
- package/knowledge/decisions/the-secrets-audit-table-is-declared-even-when-auditing-is-off.md +0 -22
- package/knowledge/decisions/workflows-own-the-lease-table-too.md +0 -22
- package/knowledge/index.md +0 -19
- package/knowledge/questions/do-the-kysely-stores-check-who-calls-them.md +0 -27
- package/knowledge/questions/does-a-trigger-source-row-hold-only-provider-identifiers.md +0 -26
- package/knowledge/questions/index.md +0 -17
- package/knowledge/questions/should-pikku-user-sessions-expire-on-their-own.md +0 -21
- package/knowledge/questions/should-secret-and-credential-reads-be-audited-by-default.md +0 -32
- package/run-tests.sh +0 -61
|
@@ -1,25 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: A console role cannot take a system role's name
|
|
4
|
-
description: createRole refuses any name the store holds as a system role, and asks the pikkuRoles table rather than the declared list, so one name never answers to two roles
|
|
5
|
-
tags: [roles, scopes, kysely, system-roles]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# A console role cannot take a system role's name
|
|
9
|
-
|
|
10
|
-
`KyselyScopeService.createRole` in `packages/services/kysely/src/kysely-scope-service.ts`
|
|
11
|
-
throws `SystemRoleShadowedError` when `isSystemRole(role.name)` is true
|
|
12
|
-
(`kysely-scope-service.ts:167-170`). `isSystemRole` reads the `system` flag from the
|
|
13
|
-
`pikkuRoles` table for that name (`kysely-scope-service.ts:221-228`).
|
|
14
|
-
|
|
15
|
-
The method's doc comment gives the reason for asking the store rather than the generated
|
|
16
|
-
declaration list: the store is where a shadow would collide, and a role whose declaration
|
|
17
|
-
was deleted stays immutable until it is pruned (`kysely-scope-service.ts:214-220`).
|
|
18
|
-
|
|
19
|
-
The schema comment says the same refusal exists because "two rows answering to one name
|
|
20
|
-
would make 'does Susan hold `buyer`?' depend on which one the store returned"
|
|
21
|
-
(`packages/services/kysely/src/schema/scope.schema.ts:39-43`).
|
|
22
|
-
|
|
23
|
-
**What this rules out:** a console role sharing a system role's name, and checking the
|
|
24
|
-
name against the code's declaration list instead of the store. `deleteRole` and
|
|
25
|
-
`setRoleScopes` apply the same `isSystemRole` guard (`kysely-scope-service.ts:191-201`).
|
|
@@ -1,22 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: A credential with no user is one row per name
|
|
4
|
-
description: The database enforces one app-wide credential per name by indexing COALESCE(user_id, ''), and the service maps a missing user to IS NULL
|
|
5
|
-
tags: [credentials, schema, kysely, uniqueness]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# A credential with no user is one row per name
|
|
9
|
-
|
|
10
|
-
`credentialSchema` in `packages/services/kysely/src/schema/credential.schema.ts` creates
|
|
11
|
-
a unique index `credentials_name_user_id_unique` on `(name, COALESCE(user_id, ''))`
|
|
12
|
-
(`credential.schema.ts:33-36`). Its comment says `user_id` is nullable and that two rows
|
|
13
|
-
with a null user must still collide on name, which is why the rule is an expression
|
|
14
|
-
index (`credential.schema.ts:7-9`).
|
|
15
|
-
|
|
16
|
-
`KyselyCredentialService.whereUserId` turns a call with no user into `userId IS NULL`
|
|
17
|
-
(`packages/services/kysely/src/kysely-credential-service.ts:118-122`), so `get`, `set`,
|
|
18
|
-
`has` and `delete` without a user all address that single row. `getUsersWithCredential`
|
|
19
|
-
and `getAllUsers` leave the null-user row out (`kysely-credential-service.ts:233-253`).
|
|
20
|
-
|
|
21
|
-
**What this rules out:** two app-wide rows for the same credential name, and treating a
|
|
22
|
-
null user as a distinct owner from every other null user.
|
|
@@ -1,30 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: A lease is taken by one guarded UPDATE
|
|
4
|
-
description: Acquire inserts the key's row already lapsed, takes it with a guarded UPDATE, and reads the row back, because that is the one shape that behaves the same on SQLite, Postgres and MySQL
|
|
5
|
-
tags: [leases, kysely, mysql]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# A lease is taken by one guarded UPDATE
|
|
9
|
-
|
|
10
|
-
`acquire` is three statements, in an order that looks roundabout:
|
|
11
|
-
|
|
12
|
-
1. insert the key's row with `token = 0, expires_at = 0` — already lapsed —
|
|
13
|
-
ignoring a conflict;
|
|
14
|
-
2. `UPDATE` it to this holder where it is this holder's or has lapsed, raising
|
|
15
|
-
`token` unless this holder is re-acquiring a live lease;
|
|
16
|
-
3. read the row back and answer whether this holder now has it.
|
|
17
|
-
|
|
18
|
-
The natural shape, `INSERT … ON CONFLICT DO UPDATE … WHERE`, does not exist on
|
|
19
|
-
MySQL, whose `ON DUPLICATE KEY UPDATE` has no guard. Inserting the row lapsed
|
|
20
|
-
means there is a single path to holding a key — the guarded update — whether the
|
|
21
|
-
key is new or not.
|
|
22
|
-
|
|
23
|
-
Two MySQL behaviours shape the rest. `SET` is evaluated left to right against
|
|
24
|
-
values already assigned in the same statement, so `token` is assigned before
|
|
25
|
-
`holder`: the re-acquire test has to see the previous holder. And MySQL reports
|
|
26
|
-
rows _changed_, not rows _matched_, so a holder re-acquiring with an unchanged
|
|
27
|
-
row would look like a loss; the row is read back instead of trusting the count.
|
|
28
|
-
|
|
29
|
-
**What this rules out:** a dialect-specific upsert per database, and deciding
|
|
30
|
-
the outcome from `numUpdatedRows`.
|
|
@@ -1,23 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: A virtual user's schedule starts disabled
|
|
4
|
-
description: A virtual-user schedule is written with enabled off unless the caller says otherwise, and only enabled rows come due, so unattended runs need an explicit enable
|
|
5
|
-
tags: [virtual-users, schedules, kysely, defaults]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# A virtual user's schedule starts disabled
|
|
9
|
-
|
|
10
|
-
The `virtualUserSchedule` table gives `enabled` a default of 0 and marks it not null
|
|
11
|
-
(`packages/services/kysely/src/schema/virtual-user-schedule.schema.ts:24-27`). The
|
|
12
|
-
schema comment says "Off by default, so creating the table spends nothing"
|
|
13
|
-
(`virtual-user-schedule.schema.ts:24-26`).
|
|
14
|
-
|
|
15
|
-
`KyselyVirtualUserScheduleStore.set` resolves the flag as
|
|
16
|
-
`schedule.enabled ?? existing?.enabled ?? false`
|
|
17
|
-
(`packages/services/kysely/src/kysely-virtual-user-schedule-store.ts:55`). `due()` reads
|
|
18
|
-
only rows with `enabled = 1` (`kysely-virtual-user-schedule-store.ts:131`). The store
|
|
19
|
-
comment says enabling a schedule "is a deliberate act" (`kysely-virtual-user-schedule-store.ts:70-71`).
|
|
20
|
-
|
|
21
|
-
**What this rules out:** a virtual user that runs on its own from the moment its schedule row
|
|
22
|
-
is created. Who may write the enabled flag is a separate question, recorded in the Kysely
|
|
23
|
-
questions.
|
|
@@ -1,71 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: Classification-driven encryption happens in a Kysely plugin
|
|
4
|
-
description: The existing column classification manifest drives a transparent query plugin, rather than encrypting the whole database file or asking call sites to encrypt
|
|
5
|
-
tags: [security, encryption, kysely, classification]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Classification-driven encryption happens in a Kysely plugin
|
|
9
|
-
|
|
10
|
-
Encrypting a local database at rest has two obvious shapes and both were
|
|
11
|
-
rejected. Encrypting the **whole file** on shutdown and decrypting on startup is
|
|
12
|
-
the simplest to describe and the easiest to lose data with: a crash, a battery
|
|
13
|
-
death, or two processes racing the same file leaves a half-written database with
|
|
14
|
-
no recovery story. Encrypting **every column** keeps the file safe but destroys
|
|
15
|
-
the database — you cannot `WHERE`, `ORDER BY`, or join on ciphertext, so the app
|
|
16
|
-
either reads whole tables into memory or stops using SQL.
|
|
17
|
-
|
|
18
|
-
The middle answer already exists in the repo. `classification.gen.ts` is a
|
|
19
|
-
per-column manifest of `private` / `secret` / `public` with a `form` of `plain`,
|
|
20
|
-
`hashed`, `wrapped` or `sealed`. Only `wrapped` and `sealed` columns are
|
|
21
|
-
encrypted; everything else stays queryable, which is most of the schema. The
|
|
22
|
-
manifest is generated from the hand-authored `db/annotations.ts` by
|
|
23
|
-
`pikku db migrate`, so the decision about a column is made once, in one place,
|
|
24
|
-
and the query layer follows it.
|
|
25
|
-
|
|
26
|
-
Enforcement belongs in a **Kysely plugin**, not at call sites. A call site that
|
|
27
|
-
forgets to encrypt writes plaintext into a column the manifest says is secret,
|
|
28
|
-
and nothing catches it — the row looks fine. A plugin sits in the query path
|
|
29
|
-
every write goes through, so "encrypted" becomes a property of the column rather
|
|
30
|
-
than a property of the developer's memory.
|
|
31
|
-
|
|
32
|
-
Transparency turned out to be **asymmetric**, and the implementation says so.
|
|
33
|
-
Reads are genuinely transparent: `transformResult` is async, so ciphertext is
|
|
34
|
-
decrypted on the way out and application code is unchanged. Writes are not.
|
|
35
|
-
`transformQuery` is synchronous — it returns a `RootOperationNode`, not a
|
|
36
|
-
promise — and WebCrypto is async, so the plugin cannot encrypt on the way in.
|
|
37
|
-
What it does instead is **refuse**: plaintext heading for a `wrapped` or
|
|
38
|
-
`sealed` column throws, and values are produced by
|
|
39
|
-
`ClassificationCrypto.encryptColumn()`. The security property is intact — a
|
|
40
|
-
forgotten call site is a loud error rather than a silent plaintext row — but
|
|
41
|
-
"transparent" describes only half of it.
|
|
42
|
-
|
|
43
|
-
Two alternatives were rejected to get there. Node's synchronous `crypto` would
|
|
44
|
-
fork crypto away from core and break every non-Node runtime. Proxying the query
|
|
45
|
-
builder is the kind of workaround the repo bans. A **`Driver` wrapper**
|
|
46
|
-
encrypting compiled-query parameters *would* deliver full write transparency and
|
|
47
|
-
reuses the manifest walk and resolver verbatim, so it stays open — but it needs
|
|
48
|
-
a parameter-index-to-column mapping recovered after compilation, and that
|
|
49
|
-
mapping goes fragile exactly where inserts get interesting (multi-row,
|
|
50
|
-
`onConflict().doUpdateSet()`). A mapping that is subtly wrong encrypts the wrong
|
|
51
|
-
column, which is worse than the problem it solves. Not worth it until the write
|
|
52
|
-
path is the thing actually hurting.
|
|
53
|
-
|
|
54
|
-
That plugin lives here, in `@pikku/kysely`, and not in a SQLite-specific
|
|
55
|
-
package. Nothing about wrapping a column value is dialect-specific, and the same
|
|
56
|
-
plugin is wanted against Postgres — cloud has the same classified columns and
|
|
57
|
-
the same reason to protect them. Putting it beside the dialect would mean
|
|
58
|
-
writing it twice.
|
|
59
|
-
|
|
60
|
-
The `hashed` form must never be encrypted, and the existing `ColumnForm` JSDoc
|
|
61
|
-
already says why: a token hash *is* the lookup key, so encrypting it makes the
|
|
62
|
-
lookup impossible.
|
|
63
|
-
|
|
64
|
-
The cost was measured on an M-series Mac rather than estimated: envelope
|
|
65
|
-
encryption runs about **38µs per value** to write and **33µs** to read, which
|
|
66
|
-
over 500 rows batched with `Promise.all` is roughly **9ms**. Key derivation is
|
|
67
|
-
the expensive step at **~48ms**, and it happens once, at unlock.
|
|
68
|
-
|
|
69
|
-
**What this rules out:** SQLCipher or whole-file encryption, encrypting every
|
|
70
|
-
column, and any design where an application function calls the crypto helpers
|
|
71
|
-
directly.
|
|
@@ -1,29 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: Grants reference declared scopes and users by foreign key
|
|
4
|
-
description: The scope tables tie role and user grants to pikkuScopes and to Better Auth's user table with foreign keys, so the database refuses a grant of an undeclared scope
|
|
5
|
-
tags: [scopes, roles, schema, kysely, foreign-keys]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Grants reference declared scopes and users by foreign key
|
|
9
|
-
|
|
10
|
-
`scopeSchema` in `packages/services/kysely/src/schema/scope.schema.ts` sets these
|
|
11
|
-
references:
|
|
12
|
-
|
|
13
|
-
- `pikkuRoleScopes.scope` references `pikkuScopes.name` (`scope.schema.ts:63-72`).
|
|
14
|
-
- `pikkuUserScope.scope` references `pikkuScopes.name` (`scope.schema.ts:101-113`).
|
|
15
|
-
- `pikkuUserRole.role` references `pikkuRoles.name` (`scope.schema.ts:80-82`).
|
|
16
|
-
- `pikkuUserRole.userId` and `pikkuUserScope.userId` reference `user.id`, which Better
|
|
17
|
-
Auth owns (`scope.schema.ts:77-79`, `scope.schema.ts:98-100`). The schema declares that
|
|
18
|
-
requirement at `scope.schema.ts:23`.
|
|
19
|
-
|
|
20
|
-
The file's header comment explains the choice: a grant is made to a user, so the user
|
|
21
|
-
table must exist first (`scope.schema.ts:7-11`). The `KyselyScopeService` class comment
|
|
22
|
-
states that `pikku_role_scopes` FKs into `pikku_scopes`, "so the database itself refuses
|
|
23
|
-
to grant a scope that was never declared" (`packages/services/kysely/src/kysely-scope-service.ts:24-25`).
|
|
24
|
-
|
|
25
|
-
`syncScopes` marks scopes that are no longer declared rather than deleting them
|
|
26
|
-
(`kysely-scope-service.ts:41-46`), so a foreign key never loses a grant during a deploy.
|
|
27
|
-
|
|
28
|
-
**What this rules out:** storing a grant for a scope name that was never declared, and
|
|
29
|
-
storing a user grant that has no user row.
|
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: overview
|
|
3
|
-
title: Decisions
|
|
4
|
-
description: How encrypted columns are wrapped, and which key wraps them
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
# Decisions
|
|
8
|
-
|
|
9
|
-
<!-- pikku:knowledge-index -->
|
|
10
|
-
- [A console role cannot take a system role's name](a-console-role-cannot-take-a-system-roles-name.md) — createRole refuses any name the store holds as a system role, and asks the pikkuRoles table rather than the declared list, so one name never answers to two roles
|
|
11
|
-
- [A credential with no user is one row per name](a-credential-with-no-user-is-one-row-per-name.md) — The database enforces one app-wide credential per name by indexing COALESCE(user_id, ''), and the service maps a missing user to IS NULL
|
|
12
|
-
- [A lease is taken by one guarded UPDATE](a-lease-is-taken-by-one-guarded-update.md) — Acquire inserts the key's row already lapsed, takes it with a guarded UPDATE, and reads the row back, because that is the one shape that behaves the same on SQLite, Postgres and MySQL
|
|
13
|
-
- [A virtual user's schedule starts disabled](a-virtual-users-schedule-starts-disabled.md) — A virtual-user schedule is written with enabled off unless the caller says otherwise, and only enabled rows come due, so unattended runs need an explicit enable
|
|
14
|
-
- [Classification-driven encryption happens in a Kysely plugin](classification-driven-encryption-happens-in-a-plugin.md) — The existing column classification manifest drives a transparent query plugin, rather than encrypting the whole database file or asking call sites to encrypt
|
|
15
|
-
- [Grants reference declared scopes and users by foreign key](grants-reference-declared-scopes-and-users-by-foreign-key.md) — The scope tables tie role and user grants to pikkuScopes and to Better Auth's user table with foreign keys, so the database refuses a grant of an undeclared scope
|
|
16
|
-
- [KEKs are scoped by purpose as well as version](keks-are-scoped-by-purpose-as-well-as-version.md) — Rows carry a keyId, and the plugin resolves KEKs through a seam rather than holding one; v1 ships a single key, but scoping never becomes a migration
|
|
17
|
-
- [Leases are judged by the database's clock](leases-are-judged-by-the-database-clock.md) — A lease and a step lease are written and compared in SQL against the server's own clock, stored as epoch milliseconds, so no worker's clock decides who holds anything
|
|
18
|
-
- [The plugin is given a resolver, never a key](the-plugin-is-given-a-resolver-never-a-key.md) — Assembly order for classified columns — a plain Kysely handle, then a plugin that asks a `KEKResolver` per operation rather than being handed a key it holds for the life of the process
|
|
19
|
-
- [The secrets audit table is declared even when auditing is off](the-secrets-audit-table-is-declared-even-when-auditing-is-off.md) — secretsAudit is created by the secret migration unconditionally, because a migration cannot depend on runtime config and an unused table costs nothing
|
|
20
|
-
- [Workflows own the lease table too](workflows-own-the-lease-table-too.md) — `leaseSchema` is owned by `workflowService` as well as `leaseService`, because the engine takes the run lease without any function ever naming leaseService
|
|
21
|
-
<!-- /pikku:knowledge-index -->
|
|
@@ -1,45 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: KEKs are scoped by purpose as well as version
|
|
4
|
-
description: Rows carry a keyId, and the plugin resolves KEKs through a seam rather than holding one; v1 ships a single key, but scoping never becomes a migration
|
|
5
|
-
tags: [security, encryption, kysely, keys]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# KEKs are scoped by purpose as well as version
|
|
9
|
-
|
|
10
|
-
`KyselySecretService` already versions its key-encrypting key: each row stores a
|
|
11
|
-
`keyVersion` beside its `ciphertext` and `wrappedDek`, per-version salts live in
|
|
12
|
-
their own table, and `rotateKEK` walks rows through `envelopeRewrap`. Version is
|
|
13
|
-
the *when* of a key. It is not the *what* — one KEK per version means every
|
|
14
|
-
encrypted column in the database is protected by the same secret, so unlocking
|
|
15
|
-
anything unlocks everything.
|
|
16
|
-
|
|
17
|
-
That is acceptable for the first release and wrong as a permanent shape. A
|
|
18
|
-
desktop app has one user and one passphrase; a multi-tenant deployment does not,
|
|
19
|
-
and "the passphrase that opens my notes also opens my credentials" is a
|
|
20
|
-
distinction people expect to be able to draw. So the design keeps scoping
|
|
21
|
-
possible without paying for it now.
|
|
22
|
-
|
|
23
|
-
Two things carry that. Encrypted rows store a **`keyId`** alongside `keyVersion`,
|
|
24
|
-
so a row records which key protects it rather than inheriting the answer from
|
|
25
|
-
context — retrofitting that column later would mean rewriting every encrypted
|
|
26
|
-
row in every deployment. And the plugin does not hold a KEK; it calls a
|
|
27
|
-
**resolver** that maps a `keyId` to one. In v1 the resolver returns the same key
|
|
28
|
-
for every id. Scoping by tenant, or by purpose, replaces the resolver.
|
|
29
|
-
|
|
30
|
-
Separate purposes derive from the same key material rather than requiring
|
|
31
|
-
separate passphrases. `expandKeyMaterial(keyMaterial, info, salt)` is the HKDF
|
|
32
|
-
helper already used for `REMOTE_SESSION_INFO`, and a distinct `info` string per
|
|
33
|
-
purpose yields independent keys from one unlock — which is what makes multiple
|
|
34
|
-
KEKs practical for a single-user app at all.
|
|
35
|
-
|
|
36
|
-
The envelope survives all of this unchanged. Core's decision that
|
|
37
|
-
[secrets use a per-secret DEK wrapped by a KEK](../../../../core/knowledge/decisions/security/core-secrets-use-a-per-secret-dek-wrapped-by-a-kek.md)
|
|
38
|
-
already rules out collapsing it, and scoping is the argument *for* the envelope
|
|
39
|
-
rather than against it: rescoping or rotating a key rewraps DEKs, touching a
|
|
40
|
-
small constant per row, while a design that encrypted values directly under the
|
|
41
|
-
KEK would have to decrypt and re-encrypt every value.
|
|
42
|
-
|
|
43
|
-
**What this rules out:** a single process-wide KEK with no row-level record of
|
|
44
|
-
which key applies, a separate passphrase per purpose, and deferring the `keyId`
|
|
45
|
-
column to a later migration.
|
|
@@ -1,42 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: Leases are judged by the database's clock
|
|
4
|
-
description: A lease and a step lease are written and compared in SQL against the server's own clock, stored as epoch milliseconds, so no worker's clock decides who holds anything
|
|
5
|
-
tags: [leases, workflows, kysely, clocks]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Leases are judged by the database's clock
|
|
9
|
-
|
|
10
|
-
A lease is only as good as the clock that decides it has lapsed. When each
|
|
11
|
-
worker computed `new Date(Date.now() + ttl)` and compared against its own
|
|
12
|
-
`Date.now()`, a worker whose clock ran an hour fast saw every live lease as
|
|
13
|
-
expired and took it — a lease and a step claim both — and wrote leases that
|
|
14
|
-
outlived their ttl by the same hour for everyone else.
|
|
15
|
-
|
|
16
|
-
So both leases are computed in SQL: `expires_at` is set to _now + ttl_ and
|
|
17
|
-
compared against _now_, where _now_ is the database's —
|
|
18
|
-
`clock_timestamp()` on Postgres, `utc_timestamp(6)` on MySQL. There is one clock,
|
|
19
|
-
and every worker reads it. The columns are `bigint` epoch milliseconds rather
|
|
20
|
-
than `timestamp`, because the arithmetic then means the same thing in every
|
|
21
|
-
dialect and no driver gets to reinterpret a time zone.
|
|
22
|
-
|
|
23
|
-
On MySQL the epoch is counted from `utc_timestamp`, never
|
|
24
|
-
`unix_timestamp(now())`. `now()` is local to the session's time zone, and when
|
|
25
|
-
that zone falls back an hour, two real instants share one local time: the
|
|
26
|
-
epoch stood still for an hour, then jumped, so a dead holder's lease lived an
|
|
27
|
-
hour longer and every lease written in the repeated hour lapsed at once.
|
|
28
|
-
|
|
29
|
-
`PgKyselyLeaseService`, `MySQLKyselyLeaseService` and the dialect workflow
|
|
30
|
-
services override `nowMs()`; `KyselyLeaseService` and `KyselyWorkflowService`
|
|
31
|
-
keep the process clock, which is only correct when every process shares a host,
|
|
32
|
-
as they do on SQLite.
|
|
33
|
-
|
|
34
|
-
**What this rules out:** comparing a lease in JavaScript after reading it — the
|
|
35
|
-
claim used to pre-check `isStepLeaseLive` in the app, and that check is exactly
|
|
36
|
-
where a skewed worker decided wrongly.
|
|
37
|
-
|
|
38
|
-
**Testing it:** PGlite runs in-process and takes its clock from the JS
|
|
39
|
-
`Date.now`, so skewing `Date.now` skews the database too and a skew test on
|
|
40
|
-
PGlite proves nothing. The skew assertions live in
|
|
41
|
-
`verifiers/workflows/src/tests/lease-db.assert.ts`, against real Postgres and
|
|
42
|
-
MySQL servers.
|
|
@@ -1,45 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: The plugin is given a resolver, never a key
|
|
4
|
-
description: Assembly order for classified columns — a plain Kysely handle, then a plugin that asks a `KEKResolver` per operation rather than being handed a key it holds for the life of the process
|
|
5
|
-
tags: [security, encryption, kysely, classification]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# The plugin is given a resolver, never a key
|
|
9
|
-
|
|
10
|
-
The pieces look circular the first time you assemble them: the plugin needs a
|
|
11
|
-
key, and whatever produces that key may itself need to read from the database.
|
|
12
|
-
It resolves because the plugin is never handed a key at all.
|
|
13
|
-
|
|
14
|
-
```ts
|
|
15
|
-
const db = new Kysely<DB>({ dialect })
|
|
16
|
-
|
|
17
|
-
const crypto = new ClassificationCrypto({ resolveKEK })
|
|
18
|
-
const classified = db.withPlugin(
|
|
19
|
-
createClassificationPlugin({ manifest, crypto })
|
|
20
|
-
)
|
|
21
|
-
```
|
|
22
|
-
|
|
23
|
-
**The plugin is given a resolver, never a key.** `resolveKEK` is asked _per
|
|
24
|
-
operation_. That is the whole reason a key can arrive after everything around it
|
|
25
|
-
has been constructed — neither the services, the Kysely instance nor the plugin
|
|
26
|
-
are rebuilt when it does — and it is why a key source that later becomes
|
|
27
|
-
unavailable makes the next classified query fail rather than use a stale key.
|
|
28
|
-
|
|
29
|
-
**A resolver that reads from the database reads through the plain handle**, not
|
|
30
|
-
the classified one, and whatever it reads must not itself be classified. A key
|
|
31
|
-
source that needed its own key to be read could never answer.
|
|
32
|
-
|
|
33
|
-
**A resolver may derive a KEK any way it likes**, as long as the same `keyId`
|
|
34
|
-
resolves to the same key: from a passphrase, from a KMS, from an environment
|
|
35
|
-
secret. `ClassificationCrypto` only requires the key and the version it should
|
|
36
|
-
stamp into the envelope. Deriving on every call is the mistake to avoid — key
|
|
37
|
-
stretching is the expensive step, so a resolver caches per `keyId`.
|
|
38
|
-
|
|
39
|
-
**What this rules out:** constructing the Kysely instance with the classification
|
|
40
|
-
plugin already attached before a key source exists, giving a database-backed
|
|
41
|
-
resolver the classified handle, and resolving a key once to cache it inside the
|
|
42
|
-
plugin.
|
|
43
|
-
|
|
44
|
-
Related: [[classification-driven-encryption-happens-in-a-plugin]],
|
|
45
|
-
[[keks-are-scoped-by-purpose-as-well-as-version]].
|
package/knowledge/decisions/the-secrets-audit-table-is-declared-even-when-auditing-is-off.md
DELETED
|
@@ -1,22 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: The secrets audit table is declared even when auditing is off
|
|
4
|
-
description: secretsAudit is created by the secret migration unconditionally, because a migration cannot depend on runtime config and an unused table costs nothing
|
|
5
|
-
tags: [secrets, audit, migrations, kysely]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# The secrets audit table is declared even when auditing is off
|
|
9
|
-
|
|
10
|
-
`secretSchema` in `packages/services/kysely/src/schema/secret.schema.ts` creates the
|
|
11
|
-
`secretsAudit` table as one of its statements (`secret.schema.ts:39-47`), with no
|
|
12
|
-
condition on the audit setting. The comment gives the reason: a migration cannot depend
|
|
13
|
-
on runtime configuration, and an unused table costs nothing, while turning auditing on
|
|
14
|
-
against a database that never got the table fails on the first write
|
|
15
|
-
(`secret.schema.ts:7-11`).
|
|
16
|
-
|
|
17
|
-
Whether audit rows are written is a separate setting. `KyselySecretService` defaults
|
|
18
|
-
`audit` to false (`packages/services/kysely/src/kysely-secret-service.ts:42`), and its
|
|
19
|
-
`logAudit` returns early when auditing is off (`kysely-secret-service.ts:56`).
|
|
20
|
-
|
|
21
|
-
**What this rules out:** generating the audit table only when the service is configured
|
|
22
|
-
with auditing on, which would make the schema depend on runtime configuration.
|
|
@@ -1,22 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: decision
|
|
3
|
-
title: Workflows own the lease table too
|
|
4
|
-
description: `leaseSchema` is owned by `workflowService` as well as `leaseService`, because the engine takes the run lease without any function ever naming leaseService
|
|
5
|
-
tags: [leases, workflows, schema]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Workflows own the lease table too
|
|
9
|
-
|
|
10
|
-
`pikku db generate` writes a schema for a project only when a service in
|
|
11
|
-
`ownedBy` is one the inspector found the project's code reaching. The workflow
|
|
12
|
-
engine takes `workflow-run:<id>` from `leaseService` on every orchestration pass,
|
|
13
|
-
but no function destructures `leaseService` to make that happen, so a workflow
|
|
14
|
-
project was never given `pikku_lease` — and registering a lease service then
|
|
15
|
-
failed at boot against a table nothing had migrated.
|
|
16
|
-
|
|
17
|
-
`leaseSchema` is therefore owned by `workflowService` as well. A workflow project
|
|
18
|
-
gets the table whether or not it registers a lease service yet.
|
|
19
|
-
|
|
20
|
-
**What this rules out:** making `leaseService` a required service of every
|
|
21
|
-
workflow project. That would force the in-memory and single-process apps, which
|
|
22
|
-
have nothing to serialise, to wire a lease they cannot use.
|
package/knowledge/index.md
DELETED
|
@@ -1,19 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: overview
|
|
3
|
-
title: Knowledge
|
|
4
|
-
description: Why encryption in this package is a query-layer concern rather than a call-site one
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
# Knowledge
|
|
8
|
-
|
|
9
|
-
Encrypting data at rest inside a database that still has to answer SQL queries
|
|
10
|
-
is a set of trade-offs, not a feature toggle. What gets encrypted, who does the
|
|
11
|
-
encrypting, and which key protects which rows are three separate questions, and
|
|
12
|
-
answering them independently is what keeps the result usable.
|
|
13
|
-
|
|
14
|
-
These notes record how this package answers them.
|
|
15
|
-
|
|
16
|
-
<!-- pikku:knowledge-index -->
|
|
17
|
-
- [decisions](decisions/index.md) — a rule that was chosen, and what it rules out
|
|
18
|
-
- [questions](questions/index.md) — something asked and not yet answered
|
|
19
|
-
<!-- /pikku:knowledge-index -->
|
|
@@ -1,27 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: question
|
|
3
|
-
title: Do the Kysely stores check who calls them?
|
|
4
|
-
description: The virtual-user run, schedule and audit stores take no identity or scope argument, so their authorisation rests on callers, and the run store's comment says reads are scope-gated somewhere else
|
|
5
|
-
tags: [authorization, virtual-users, audit, kysely]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Do the Kysely stores check who calls them?
|
|
9
|
-
|
|
10
|
-
Three Kysely stores take no identity or scope argument, and the files do not check one.
|
|
11
|
-
|
|
12
|
-
- **Virtual-user run store.** `get(runId)` and `list({ virtualUserId, limit, offset })` run
|
|
13
|
-
queries with no caller check (`packages/services/kysely/src/kysely-virtual-user-run-store.ts:125-151`).
|
|
14
|
-
The class comment says the store is privileged because an adversarial run's findings
|
|
15
|
-
are "working exploits carrying live ids", and that "the scaffolded reads are
|
|
16
|
-
scope-gated" (`kysely-virtual-user-run-store.ts:37-39`). The gate is not in this file.
|
|
17
|
-
- **Virtual-user schedule store.** `set(schedule)` takes no identity, and an enabled
|
|
18
|
-
flag written through it makes a virtual user due (`packages/services/kysely/src/kysely-virtual-user-schedule-store.ts:45-48`,
|
|
19
|
-
`:55`, `:131`). `remove(virtualUserId)` takes none either (`:161-167`).
|
|
20
|
-
- **Audit service.** `query` applies the `userIds`, `types` and `orgId` filters the
|
|
21
|
-
caller passes and runs them (`packages/services/kysely/src/kysely-audit-service.ts:161-182`,
|
|
22
|
-
`:211-236`). Nothing in those lines checks who is asking.
|
|
23
|
-
|
|
24
|
-
Current behaviour: authorisation for these stores is entirely in the callers.
|
|
25
|
-
|
|
26
|
-
Open: which layer is meant to check identity and scope for each store, and is the
|
|
27
|
-
run-store scope gate the comment describes implemented anywhere in the repository?
|
|
@@ -1,26 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: question
|
|
3
|
-
title: Does a trigger-source row hold only provider identifiers?
|
|
4
|
-
description: The trigger-source schema says state holds provider identifiers and that signing secrets go to the credential service, but the store writes whatever state the caller passes as JSON
|
|
5
|
-
tags: [kysely, trigger-source, secrets]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Does a trigger-source row hold only provider identifiers?
|
|
9
|
-
|
|
10
|
-
The doc comment on the table says "`state` holds provider identifiers only: a signing
|
|
11
|
-
secret a provider issues goes to the credential service"
|
|
12
|
-
(`packages/services/kysely/src/schema/trigger-source.schema.ts:4-7`).
|
|
13
|
-
|
|
14
|
-
The table has a `state` text column and a `detail` text column
|
|
15
|
-
(`trigger-source.schema.ts:26-27`). Nothing in the schema restricts what is written to
|
|
16
|
-
them.
|
|
17
|
-
|
|
18
|
-
The store writes the value it is given. `recordTriggerSource` sets
|
|
19
|
-
`state` to `JSON.stringify(result.state)` whenever `result.state` is set
|
|
20
|
-
(`packages/services/kysely/src/kysely-trigger-source-store.ts:102-107`). Its test
|
|
21
|
-
writes `{ id: 'we_1' }` (`kysely-trigger-source-store.test.ts:41`), which is an
|
|
22
|
-
identifier. The test does not check that a secret is refused.
|
|
23
|
-
|
|
24
|
-
This note did not read the code that builds `result.state`, so it cannot say whether
|
|
25
|
-
any caller puts a secret there. The question is whether the comment describes a rule
|
|
26
|
-
the code enforces, or only the convention the callers are expected to follow.
|
|
@@ -1,17 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: overview
|
|
3
|
-
title: Questions
|
|
4
|
-
description: Things asked of the Kysely stores and not yet answered
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
# Questions
|
|
8
|
-
|
|
9
|
-
Something asked and not yet answered. The code makes a choice nobody has stated, so the
|
|
10
|
-
doubt is recorded with the current behaviour, not a guessed rationale.
|
|
11
|
-
|
|
12
|
-
<!-- pikku:knowledge-index -->
|
|
13
|
-
- [Do the Kysely stores check who calls them?](do-the-kysely-stores-check-who-calls-them.md) — The virtual-user run, schedule and audit stores take no identity or scope argument, so their authorisation rests on callers, and the run store's comment says reads are scope-gated somewhere else
|
|
14
|
-
- [Does a trigger-source row hold only provider identifiers?](does-a-trigger-source-row-hold-only-provider-identifiers.md) — The trigger-source schema says state holds provider identifiers and that signing secrets go to the credential service, but the store writes whatever state the caller passes as JSON
|
|
15
|
-
- [Should pikku user sessions expire on their own?](should-pikku-user-sessions-expire-on-their-own.md) — The Kysely session store writes a session with no expiry, reads it back at any age, and its table has no expiry column
|
|
16
|
-
- [Should secret and credential reads be audited by default?](should-secret-and-credential-reads-be-audited-by-default.md) — The Kysely secret and credential stores leave auditing off, leave reads unaudited unless asked, and the secret store never records bulk reads; the generic query audit goes the other way
|
|
17
|
-
<!-- /pikku:knowledge-index -->
|
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: question
|
|
3
|
-
title: Should pikku user sessions expire on their own?
|
|
4
|
-
description: The Kysely session store writes a session with no expiry, reads it back at any age, and its table has no expiry column
|
|
5
|
-
tags: [sessions, expiry, kysely]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Should pikku user sessions expire on their own?
|
|
9
|
-
|
|
10
|
-
`KyselySessionStore` (`packages/services/kysely/src/kysely-session-store.ts`) writes a
|
|
11
|
-
session as JSON with an `updatedAt` stamp and nothing else (`kysely-session-store.ts:35-46`).
|
|
12
|
-
`get` returns the stored session whatever its age, and has no expiry check
|
|
13
|
-
(`kysely-session-store.ts:21-33`). `clear` deletes a user's row (`kysely-session-store.ts:48-53`).
|
|
14
|
-
|
|
15
|
-
The table has `pikkuUserId`, `session`, `createdAt` and `updatedAt`, and no expiry column
|
|
16
|
-
(`packages/services/kysely/src/schema/session.schema.ts:8-18`).
|
|
17
|
-
|
|
18
|
-
Current behaviour: a stored session stays readable until something calls `clear`.
|
|
19
|
-
|
|
20
|
-
Open: should a session expire in this store, or is its lifetime bounded somewhere else? The
|
|
21
|
-
files do not say.
|
|
@@ -1,32 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
type: question
|
|
3
|
-
title: Should secret and credential reads be audited by default?
|
|
4
|
-
description: The Kysely secret and credential stores leave auditing off, leave reads unaudited unless asked, and the secret store never records bulk reads; the generic query audit goes the other way
|
|
5
|
-
tags: [audit, secrets, credentials, kysely]
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Should secret and credential reads be audited by default?
|
|
9
|
-
|
|
10
|
-
`KyselySecretService` (`packages/services/kysely/src/kysely-secret-service.ts`) defaults
|
|
11
|
-
`audit` and `auditReads` to false (`kysely-secret-service.ts:42-43`). `logAudit` returns
|
|
12
|
-
early when auditing is off, and for reads when `auditReads` is off
|
|
13
|
-
(`kysely-secret-service.ts:56-57`). A single `getSecret` records a read
|
|
14
|
-
(`kysely-secret-service.ts:129`). `getSecrets` records nothing, even with auditing on
|
|
15
|
-
(`kysely-secret-service.ts:178-209`). Writes, deletes and rotations are recorded when
|
|
16
|
-
auditing is on (`kysely-secret-service.ts:170`, `:175`, `:237`).
|
|
17
|
-
|
|
18
|
-
`KyselyCredentialService` (`packages/services/kysely/src/kysely-credential-service.ts`)
|
|
19
|
-
has the same defaults (`kysely-credential-service.ts:40-41`) and the same gate
|
|
20
|
-
(`kysely-credential-service.ts:50-56`). `get` records a read (`:140`), and `getAll`
|
|
21
|
-
records one row per credential, still under the same flag (`:207-230`, `:227`).
|
|
22
|
-
|
|
23
|
-
`createAuditedKysely` goes the other way. It skips selects only when `auditReads` is
|
|
24
|
-
explicitly `false` (`packages/services/kysely/src/create-audited-kysely.ts:196-198`), so
|
|
25
|
-
database selects are audited by default.
|
|
26
|
-
|
|
27
|
-
Current behaviour: secret and credential reads leave no record until the app turns
|
|
28
|
-
auditing on, and bulk secret reads never leave one. The code does not say whether that is
|
|
29
|
-
meant.
|
|
30
|
-
|
|
31
|
-
Open: should secret and credential reads be audited by default, and should `getSecrets`
|
|
32
|
-
record a row per key? The files do not say.
|
package/run-tests.sh
DELETED
|
@@ -1,61 +0,0 @@
|
|
|
1
|
-
#!/bin/bash
|
|
2
|
-
|
|
3
|
-
# Enable nullglob to handle cases where no files match the pattern
|
|
4
|
-
shopt -s nullglob
|
|
5
|
-
|
|
6
|
-
# Initialize variables for options
|
|
7
|
-
watch_mode=false
|
|
8
|
-
coverage_mode=false
|
|
9
|
-
|
|
10
|
-
# Parse command-line options
|
|
11
|
-
while [[ $# -gt 0 ]]; do
|
|
12
|
-
case $1 in
|
|
13
|
-
--watch)
|
|
14
|
-
watch_mode=true
|
|
15
|
-
shift
|
|
16
|
-
;;
|
|
17
|
-
--coverage)
|
|
18
|
-
coverage_mode=true
|
|
19
|
-
shift
|
|
20
|
-
;;
|
|
21
|
-
*)
|
|
22
|
-
echo "Unknown option: $1"
|
|
23
|
-
exit 1
|
|
24
|
-
;;
|
|
25
|
-
esac
|
|
26
|
-
done
|
|
27
|
-
|
|
28
|
-
# Define the pattern to match your test files
|
|
29
|
-
pattern="src/*.test.ts"
|
|
30
|
-
|
|
31
|
-
# Expand the pattern into an array of files
|
|
32
|
-
files=($(find src -type f -name "*.test.ts"))
|
|
33
|
-
|
|
34
|
-
# Check if any files matched the pattern
|
|
35
|
-
if [ ${#files[@]} -eq 0 ]; then
|
|
36
|
-
echo "No test files found matching pattern: $pattern"
|
|
37
|
-
if [ "${STRICT_TEST_DISCOVERY}" = "1" ]; then
|
|
38
|
-
exit 1
|
|
39
|
-
fi
|
|
40
|
-
exit 0
|
|
41
|
-
fi
|
|
42
|
-
|
|
43
|
-
# Construct the node command
|
|
44
|
-
#
|
|
45
|
-
# Every other package runs its tests on bun. This one stays on node because
|
|
46
|
-
# better-sqlite3, which every one of these suites opens its fixture database
|
|
47
|
-
# with, aborts the bun process on load.
|
|
48
|
-
node_cmd=(node --import tsx --test)
|
|
49
|
-
|
|
50
|
-
# Append options based on flags
|
|
51
|
-
if [ "$watch_mode" = true ]; then
|
|
52
|
-
node_cmd+=(--watch)
|
|
53
|
-
fi
|
|
54
|
-
|
|
55
|
-
if [ "$coverage_mode" = true ]; then
|
|
56
|
-
node_cmd+=(--test-coverage-include="src/**/*.{ts,js}" --test-coverage-exclude="**/dist/**" --experimental-test-coverage --test-reporter=lcov --test-reporter-destination=lcov.info)
|
|
57
|
-
export PIKKU_TEST_COVERAGE=1
|
|
58
|
-
fi
|
|
59
|
-
|
|
60
|
-
# Execute the node command with the expanded list of files
|
|
61
|
-
"${node_cmd[@]}" "${files[@]}"
|