turbine-orm 0.58.0 → 0.59.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -454,7 +454,7 @@ const db = turbine({
454
454
 
455
455
  Where a pg-style alias exists (`max`, `idleTimeoutMillis`, `connectionTimeoutMillis`), the explicit Turbine field wins when both are set.
456
456
 
457
- > **`planCacheMode` (Postgres only, opt-in).** PostgreSQL may promote a **named** prepared statement to a generic plan from its sixth execution onward, and a generic plan is costed blind to the bound values. On a predicate whose selectivity swings per value (a `tenant_id` equality on a shared table, where one value matches a handful of rows and another matches most of them), the statement can be locked onto a plan chosen for the average value, and it never reverts. `planCacheMode: 'auto' | 'force_custom_plan' | 'force_generic_plan'` pins the backend's choice; `'force_custom_plan'` re-plans every execution and removes the cliff. It is applied as a connection parameter (`options=-c plan_cache_mode=...`) when Turbine opens a connection, so it is in force for that connection's first statement and for every checkout, `$transaction`, stream and pipeline on it, and it cannot race your first query. Leave it unset (the default) and Turbine sends nothing at all. Reach for it when you have measured a statement getting slower after its fifth execution. **Correction to the 0.54 text, which said `findMany` / `findFirst` bind `LIMIT $n` and are "much less exposed":** that was false. PostgreSQL does not deny the planner a limit fraction for a bound limit, it substitutes a default of 10% of the child node's own row estimate (clamped at one row), and an unknown `OFFSET` triggers the same substitution even when the limit is a constant, which a paginated Turbine read always has. Two things also need saying about the sentence that opens this note. The sixth execution is a ceiling, not a trigger: `auto` promotes only when the generic plan's **estimated** cost is not worse than the average custom cost, so many statements are never promoted at all, and `pg_prepared_statements.generic_plans` is how you tell. And the shape that gets promoted unprompted is the one with **no limit**, not the limited one: measured on a skewed join predicate, an unlimited `count()`-shaped statement promoted under the default `auto` and ran a nested loop at 430x the buffers of the custom plan, while the same predicate under `LIMIT $n` was never promoted across eight executions (its substituted row count made the generic plan look more expensive). A limited `findMany` gives the planner two unknowns instead of one, which is not the same as more damage. `implicitPkOrdering` is **off by default in core**, so a default `findMany` emits no `ORDER BY`; switching it on adds an ordering a generic plan can walk the whole table in. Measure with `plan_cache_mode = force_generic_plan` against `force_custom_plan` rather than reasoning about which shapes ought to be safe; the fixtures and numbers are on the [relations page](https://turbineorm.dev/relations) and in the 0.55.0 changelog. **Two 0.56 corrections to the paragraph above.** First, "neither an ORDER BY nor a limit is required" is true, but it read as if ordering did not matter: on a real multi-tenant schema swept table by table, every divergent shape had an `ORDER BY` and every shape without one measured 1.00x, so it is not necessary in general and is still the strongest single predictor in practice. Second, a custom plan is not automatically the better one: on a reproducible fixture where one dense value's rows are packed at the end of the heap, `LIMIT 20` with no ordering reads 4,262 buffers custom against 71 generic (the default `auto` never promotes there, so it produces the 4,262-buffer plan too). Since 0.56 the per-query read arg **`forceCustomPlan: true`** covers the case a connection-wide setting cannot express, custom on one query and `auto` everywhere else, and `turbine doctor` detects the distribution that admits the flip. **0.57 correction:** that read arg reached the core client only. Through `turbine-orm/prisma-compat` it was accepted and silently dropped until 0.57.0, so a compat integration that followed this advice got a no-op; confirm at the wire with `pg_prepared_statements` rather than assuming. 0.57 also adds a third divergence mechanism to `doctor`: an **unindexed** filter column, where the good plan is a sequential scan the generic plan will not choose (measured 250 buffers against 20,074 on a 20,000-row / 247-page fixture). Three scope limits: it does nothing on an **external pool** (Turbine never opens those connections, so set the GUC in the driver's own setup; Turbine-owned string `replicas` on that same client DO get it); a Postgres wire-compatible engine without the setting (CockroachDB, YugabyteDB, pre-12 PostgreSQL) refuses the connection parameter itself; and a **connection pooler** may filter startup parameters (PgBouncer's `ignore_startup_parameters`), where `ALTER ROLE ... SET plan_cache_mode = ...` is the way in. Any value outside the three throws `ValidationError` at construction, and a non-Postgres engine throws `UnsupportedFeatureError` (`TURBINE_E017`).
457
+ > **`planCacheMode` (Postgres only, opt-in).** PostgreSQL may promote a **named** prepared statement to a generic plan from its sixth execution onward, and a generic plan is costed blind to the bound values. On a predicate whose selectivity swings per value (a `tenant_id` equality on a shared table, where one value matches a handful of rows and another matches most of them), the statement can be locked onto a plan chosen for the average value, and it never reverts. `planCacheMode: 'auto' | 'force_custom_plan' | 'force_generic_plan'` pins the backend's choice; `'force_custom_plan'` re-plans every execution and removes the cliff. It is applied as a connection parameter (`options=-c plan_cache_mode=...`) when Turbine opens a connection, so it is in force for that connection's first statement and for every checkout, `$transaction`, stream and pipeline on it, and it cannot race your first query. Leave it unset (the default) and Turbine sends nothing at all. Reach for it when you have measured a statement getting slower after its fifth execution. **Correction to the 0.54 text, which said `findMany` / `findFirst` bind `LIMIT $n` and are "much less exposed":** that was false. PostgreSQL does not deny the planner a limit fraction for a bound limit, it substitutes a default of 10% of the child node's own row estimate (clamped at one row), and an unknown `OFFSET` triggers the same substitution even when the limit is a constant, which a paginated Turbine read always has. Two things also need saying about the sentence that opens this note. The sixth execution is a ceiling, not a trigger: `auto` promotes only when the generic plan's **estimated** cost is not worse than the average custom cost, so many statements are never promoted at all, and `pg_prepared_statements.generic_plans` is how you tell. And the shape that gets promoted unprompted is the one with **no limit**, not the limited one: measured on a skewed join predicate, an unlimited `count()`-shaped statement promoted under the default `auto` and ran a nested loop at 430x the buffers of the custom plan, while the same predicate under `LIMIT $n` was never promoted across eight executions (its substituted row count made the generic plan look more expensive). A limited `findMany` gives the planner two unknowns instead of one, which is not the same as more damage. `implicitPkOrdering` is **off by default in core**, so a default `findMany` emits no `ORDER BY`; switching it on adds an ordering a generic plan can walk the whole table in. Measure with `plan_cache_mode = force_generic_plan` against `force_custom_plan` rather than reasoning about which shapes ought to be safe; the fixtures and numbers are on the [relations page](https://turbineorm.dev/relations) and in the 0.55.0 changelog. **Two 0.56 corrections to the paragraph above.** First, "neither an ORDER BY nor a limit is required" is true, but it read as if ordering did not matter: in a table-by-table sweep of a multi-tenant schema, every divergent shape measured had an `ORDER BY` and every shape without one measured 1.00x, so it is not necessary in general and is still the strongest single predictor in practice. Second, a custom plan is not automatically the better one: on a reproducible fixture where one dense value's rows are packed at the end of the heap, `LIMIT 20` with no ordering reads 4,262 buffers custom against 71 generic (the default `auto` never promotes there, so it produces the 4,262-buffer plan too). Since 0.56 the per-query read arg **`forceCustomPlan: true`** covers the case a connection-wide setting cannot express, custom on one query and `auto` everywhere else, and `turbine doctor` detects the distribution that admits the flip. **0.57 correction:** that read arg reached the core client only. Through `turbine-orm/prisma-compat` it was accepted and silently dropped until 0.57.0, so a compat integration that followed this advice got a no-op; confirm at the wire with `pg_prepared_statements` rather than assuming. 0.57 also adds a third divergence mechanism to `doctor`: an **unindexed** filter column, where the good plan is a sequential scan the generic plan will not choose (measured 250 buffers against 20,074 on a 20,000-row / 247-page fixture). Three scope limits: it does nothing on an **external pool** (Turbine never opens those connections, so set the GUC in the driver's own setup; Turbine-owned string `replicas` on that same client DO get it); a Postgres wire-compatible engine without the setting (CockroachDB, YugabyteDB, pre-12 PostgreSQL) refuses the connection parameter itself; and a **connection pooler** may filter startup parameters (PgBouncer's `ignore_startup_parameters`), where `ALTER ROLE ... SET plan_cache_mode = ...` is the way in. Any value outside the three throws `ValidationError` at construction, and a non-Postgres engine throws `UnsupportedFeatureError` (`TURBINE_E017`).
458
458
 
459
459
  > **`preparedStatements` and connection poolers.** With prepared statements on, Turbine submits queries as `{ name, text, values }` so Postgres caches the parse and plan **per backend connection**. That is a real win against a database you connect to directly, and a hazard behind a transaction-pooling proxy (PgBouncer in `transaction` mode, Supabase's pooler port, some serverless poolers): the named statement is prepared on one backend and your next query may land on another, which fails with `prepared statement "..." does not exist`. Turbine defaults it to `true` only for pools it creates itself and `false` for external pools passed via `pool` / `turbineHttp()`, because serverless drivers are the common case there. If you are pointing a Turbine-owned pool at a transaction pooler, set `preparedStatements: false`. The environment variable `TURBINE_DISABLE_PREPARED=1` turns it off globally without a code change.
460
460
 
@@ -2389,7 +2389,8 @@ async function cmdDoctor(args, config) {
2389
2389
  : { findings: [], notices: [], candidatesConsidered: 0, consideredIndexed: 0, consideredUnindexed: 0 };
2390
2390
  // Statistics can say how bad a flip WOULD be; only the planner can say whether
2391
2391
  // it is reachable. The `unindexed-filter` branch shipped in 0.57 without that
2392
- // question answered and was right 6 times in 13 on a real schema, so every one
2392
+ // question answered, and a measured sample of 13 findings held up only 6 times,
2393
+ // so every one
2393
2394
  // of its findings is now put to a plan-only EXPLAIN. Nothing is executed, and a
2394
2395
  // probe that fails keeps its finding rather than dropping it.
2395
2396
  const flipProbe = divergenceOn && scored.findings.some(plan_flip_probe_js_1.needsFlipProbe)
@@ -6,13 +6,14 @@
6
6
  * `plan-divergence.ts` scores a column from statistics alone and answers "IF the
7
7
  * cached plan flips, how bad is it". Its `unindexed-filter` branch (0.57) shipped
8
8
  * without an answer to the prior question, "CAN it flip at all", and that turned
9
- * out to be the majority case: on a real 118-model schema the branch produced 39
10
- * findings of which a measured sample was right 6 times in 13.
9
+ * out to be the majority case: in validation against a large schema the branch
10
+ * emitted 39 findings, and a measured sample of 13 of them held up only 6 times.
11
11
  *
12
- * Every false positive had one signature: **the generic plan kept the same
13
- * sequential scan the custom plan chose.** There was no flip to be had, so the
12
+ * Every false positive had one signature: **the generic plan was not the ordered
13
+ * index walk the finding claims.** There was no flip to be had, so the
14
14
  * amplification the finding printed described a plan the planner would never
15
- * pick.
15
+ * pick. See {@link verdictFromPlanJson} for the two ways a plan fails to be that
16
+ * walk; 0.58.0 shipped only one of them and 0.59.0 added the other.
16
17
  *
17
18
  * ## Why this is a probe and not another rule
18
19
  *
@@ -54,11 +55,10 @@
54
55
  * ```
55
56
  *
56
57
  * The custom plan is not needed. The finding's whole claim is that a promoted
57
- * generic plan abandons the seq scan for an ordered index walk, so if the generic
58
- * plan IS a seq scan on the target table, the claim is refuted no matter what the
59
- * custom plan does. Asking one question instead of two halves the work and
60
- * removes the need for a representative rare value, which statistics do not
61
- * carry.
58
+ * generic plan performs an ordered index walk, so a generic plan that is NOT that
59
+ * walk refutes it no matter what the custom plan does. Asking one question
60
+ * instead of two halves the work and removes the need for a representative rare
61
+ * value, which statistics do not carry.
62
62
  *
63
63
  * `NULL` is a safe argument precisely because the plan is generic: a generic plan
64
64
  * is built without looking at the value, which is the property the whole check is
@@ -80,11 +80,11 @@ import type { PlanDivergenceFinding, PlanDivergenceReport } from './plan-diverge
80
80
  /**
81
81
  * The planner's answer for one finding.
82
82
  *
83
- * - `'flip-reachable'`, the generic plan is NOT a plain seq scan of the target
83
+ * - `'flip-reachable'`, the generic plan IS an ordered index walk on the target
84
84
  * table, so the divergence the finding describes is one the planner can
85
85
  * actually choose.
86
- * - `'no-flip'`, the generic plan sequentially scans the target table, the same
87
- * access the good plan uses. Nothing to diverge to.
86
+ * - `'no-flip'`, the generic plan is not that walk (a `Sort` bounds it, or the
87
+ * access is a plain seq scan). Nothing to diverge to.
88
88
  * - `'unknown'`, the probe did not produce an answer. The finding is kept.
89
89
  */
90
90
  export type FlipVerdict = 'flip-reachable' | 'no-flip' | 'unknown';
@@ -133,11 +133,49 @@ export declare function buildFlipProbeSql(finding: PlanDivergenceFinding, name:
133
133
  * Exported for unit tests: the plan shapes this has to classify are exactly the
134
134
  * ones that are tedious to produce live.
135
135
  *
136
- * The rule is deliberately narrow. Only a `Seq Scan` ON THE TARGET TABLE refutes
137
- * a finding. A seq scan of some other relation in a more complex plan says
138
- * nothing about this column, and anything that is not a plain sequential scan of
139
- * the target (index scan, bitmap heap scan, index-only scan) leaves the flip
140
- * reachable.
136
+ * ## The question, stated exactly
137
+ *
138
+ * The finding claims the generic plan performs an ORDERED INDEX WALK along the
139
+ * `ORDER BY` column and fetches nearly every tuple before the `LIMIT` fills. So
140
+ * the refutation is not "the plan is a seq scan", it is **"the plan is not that
141
+ * ordered walk"**, and there are two ways for it not to be:
142
+ *
143
+ * 1. A `Sort` lies above the target table's scan. A sort materializes the whole
144
+ * matched set and orders it, so the cost is bounded by how many rows match,
145
+ * not by how far into the heap the ordered walk has to travel. Whatever feeds
146
+ * it (seq scan, bitmap heap scan) the catastrophic shape is absent.
147
+ * 2. The target's own scan node is a `Seq Scan`. Kept as an independent ground
148
+ * rather than folded into the first, so a hypothetical ordered seq scan with
149
+ * no sort still refutes.
150
+ *
151
+ * 0.58.0 shipped only the second ground and therefore missed every LOW-estimate
152
+ * column that has ANY usable index, because those plan as `Limit > Sort > Bitmap
153
+ * Heap Scan`. The case that surfaced it was a column carrying
154
+ * `btree (col) WHERE col IS NOT NULL`: an equality predicate implies not-null, so
155
+ * that partial index is fully usable and the plan never reaches a seq scan.
156
+ * Reproduced, and the fixture is the pair below at the same estimate:
157
+ *
158
+ * ```txt
159
+ * partial index, est 1.9 Limit > Sort > Bitmap Heap Scan <- 0.58 kept this
160
+ * no index, est 1.9 Limit > Sort > Seq Scan <- 0.58 refuted this
161
+ * either, est 500 Limit > Index Scan (no Sort) <- both keep, correctly
162
+ * ```
163
+ *
164
+ * ## Why this is not "exclude partial-index columns"
165
+ *
166
+ * Because a partial index whose predicate is NOT implied by the equality does not
167
+ * serve the query at all, and such a column produces a genuine finding: one was
168
+ * measured at 19,961x. The property that matters is whether the planner COULD
169
+ * use it, which is a proof obligation over predicates, and the plan already
170
+ * carries the answer. Reading the plan is cheaper and cannot drift from the
171
+ * planner's own implication rules.
172
+ *
173
+ * ## The safe direction is KEEP
174
+ *
175
+ * Over-refuting deletes real findings, which is invisible in the report;
176
+ * over-keeping only costs noise. So `Incremental Sort` does NOT refute: it means
177
+ * the index supplies a PREFIX of the ordering and the walk is still partly
178
+ * ordered, which is closer to the catastrophic shape than to the bounded one.
141
179
  */
142
180
  export declare function verdictFromPlanJson(payload: unknown, table: string): FlipVerdict;
143
181
  export interface ProbePlanFlipsOptions {
@@ -7,13 +7,14 @@
7
7
  * `plan-divergence.ts` scores a column from statistics alone and answers "IF the
8
8
  * cached plan flips, how bad is it". Its `unindexed-filter` branch (0.57) shipped
9
9
  * without an answer to the prior question, "CAN it flip at all", and that turned
10
- * out to be the majority case: on a real 118-model schema the branch produced 39
11
- * findings of which a measured sample was right 6 times in 13.
10
+ * out to be the majority case: in validation against a large schema the branch
11
+ * emitted 39 findings, and a measured sample of 13 of them held up only 6 times.
12
12
  *
13
- * Every false positive had one signature: **the generic plan kept the same
14
- * sequential scan the custom plan chose.** There was no flip to be had, so the
13
+ * Every false positive had one signature: **the generic plan was not the ordered
14
+ * index walk the finding claims.** There was no flip to be had, so the
15
15
  * amplification the finding printed described a plan the planner would never
16
- * pick.
16
+ * pick. See {@link verdictFromPlanJson} for the two ways a plan fails to be that
17
+ * walk; 0.58.0 shipped only one of them and 0.59.0 added the other.
17
18
  *
18
19
  * ## Why this is a probe and not another rule
19
20
  *
@@ -55,11 +56,10 @@
55
56
  * ```
56
57
  *
57
58
  * The custom plan is not needed. The finding's whole claim is that a promoted
58
- * generic plan abandons the seq scan for an ordered index walk, so if the generic
59
- * plan IS a seq scan on the target table, the claim is refuted no matter what the
60
- * custom plan does. Asking one question instead of two halves the work and
61
- * removes the need for a representative rare value, which statistics do not
62
- * carry.
59
+ * generic plan performs an ordered index walk, so a generic plan that is NOT that
60
+ * walk refutes it no matter what the custom plan does. Asking one question
61
+ * instead of two halves the work and removes the need for a representative rare
62
+ * value, which statistics do not carry.
63
63
  *
64
64
  * `NULL` is a safe argument precisely because the plan is generic: a generic plan
65
65
  * is built without looking at the value, which is the property the whole check is
@@ -166,39 +166,83 @@ function buildFlipProbeSql(finding, name, searchSchema) {
166
166
  deallocate: `DEALLOCATE ${name}`,
167
167
  };
168
168
  }
169
- function* walkPlan(node) {
170
- yield node;
171
- for (const child of node.Plans ?? [])
172
- yield* walkPlan(child);
173
- }
174
169
  /**
175
170
  * Read a verdict out of one `EXPLAIN (FORMAT JSON)` payload.
176
171
  *
177
172
  * Exported for unit tests: the plan shapes this has to classify are exactly the
178
173
  * ones that are tedious to produce live.
179
174
  *
180
- * The rule is deliberately narrow. Only a `Seq Scan` ON THE TARGET TABLE refutes
181
- * a finding. A seq scan of some other relation in a more complex plan says
182
- * nothing about this column, and anything that is not a plain sequential scan of
183
- * the target (index scan, bitmap heap scan, index-only scan) leaves the flip
184
- * reachable.
175
+ * ## The question, stated exactly
176
+ *
177
+ * The finding claims the generic plan performs an ORDERED INDEX WALK along the
178
+ * `ORDER BY` column and fetches nearly every tuple before the `LIMIT` fills. So
179
+ * the refutation is not "the plan is a seq scan", it is **"the plan is not that
180
+ * ordered walk"**, and there are two ways for it not to be:
181
+ *
182
+ * 1. A `Sort` lies above the target table's scan. A sort materializes the whole
183
+ * matched set and orders it, so the cost is bounded by how many rows match,
184
+ * not by how far into the heap the ordered walk has to travel. Whatever feeds
185
+ * it (seq scan, bitmap heap scan) the catastrophic shape is absent.
186
+ * 2. The target's own scan node is a `Seq Scan`. Kept as an independent ground
187
+ * rather than folded into the first, so a hypothetical ordered seq scan with
188
+ * no sort still refutes.
189
+ *
190
+ * 0.58.0 shipped only the second ground and therefore missed every LOW-estimate
191
+ * column that has ANY usable index, because those plan as `Limit > Sort > Bitmap
192
+ * Heap Scan`. The case that surfaced it was a column carrying
193
+ * `btree (col) WHERE col IS NOT NULL`: an equality predicate implies not-null, so
194
+ * that partial index is fully usable and the plan never reaches a seq scan.
195
+ * Reproduced, and the fixture is the pair below at the same estimate:
196
+ *
197
+ * ```txt
198
+ * partial index, est 1.9 Limit > Sort > Bitmap Heap Scan <- 0.58 kept this
199
+ * no index, est 1.9 Limit > Sort > Seq Scan <- 0.58 refuted this
200
+ * either, est 500 Limit > Index Scan (no Sort) <- both keep, correctly
201
+ * ```
202
+ *
203
+ * ## Why this is not "exclude partial-index columns"
204
+ *
205
+ * Because a partial index whose predicate is NOT implied by the equality does not
206
+ * serve the query at all, and such a column produces a genuine finding: one was
207
+ * measured at 19,961x. The property that matters is whether the planner COULD
208
+ * use it, which is a proof obligation over predicates, and the plan already
209
+ * carries the answer. Reading the plan is cheaper and cannot drift from the
210
+ * planner's own implication rules.
211
+ *
212
+ * ## The safe direction is KEEP
213
+ *
214
+ * Over-refuting deletes real findings, which is invisible in the report;
215
+ * over-keeping only costs noise. So `Incremental Sort` does NOT refute: it means
216
+ * the index supplies a PREFIX of the ordering and the walk is still partly
217
+ * ordered, which is closer to the catastrophic shape than to the bounded one.
185
218
  */
186
219
  function verdictFromPlanJson(payload, table) {
187
220
  const root = Array.isArray(payload) ? payload[0] : undefined;
188
221
  const plan = root?.Plan;
189
222
  if (!plan)
190
223
  return 'unknown';
191
- for (const node of walkPlan(plan)) {
192
- if (node['Relation Name'] !== table)
193
- continue;
224
+ // Walk root-downward, tracking whether a full Sort sits ABOVE the target's
225
+ // scan. Depth matters: a Sort somewhere else in a larger plan says nothing
226
+ // about how this table is reached.
227
+ const search = (node, sortedAbove) => {
194
228
  const type = node['Node Type'];
195
- if (type === undefined)
196
- continue;
197
- return type === 'Seq Scan' ? 'no-flip' : 'flip-reachable';
198
- }
199
- // The target table is not in the plan at all, which should not happen for a
200
- // statement that selects from it. Treated as unknown rather than as a refutation.
201
- return 'unknown';
229
+ if (node['Relation Name'] === table && type !== undefined) {
230
+ if (sortedAbove)
231
+ return 'no-flip';
232
+ return type === 'Seq Scan' ? 'no-flip' : 'flip-reachable';
233
+ }
234
+ // 'Incremental Sort' is deliberately excluded, see the header.
235
+ const nowSorted = sortedAbove || type === 'Sort';
236
+ for (const child of node.Plans ?? []) {
237
+ const found = search(child, nowSorted);
238
+ if (found !== null)
239
+ return found;
240
+ }
241
+ return null;
242
+ };
243
+ // The target table not appearing at all should not happen for a statement that
244
+ // selects from it. Treated as unknown rather than as a refutation.
245
+ return search(plan, false) ?? 'unknown';
202
246
  }
203
247
  /**
204
248
  * Ask the planner, once per candidate finding, whether the flip is reachable.
package/dist/cli/index.js CHANGED
@@ -2339,7 +2339,8 @@ async function cmdDoctor(args, config) {
2339
2339
  : { findings: [], notices: [], candidatesConsidered: 0, consideredIndexed: 0, consideredUnindexed: 0 };
2340
2340
  // Statistics can say how bad a flip WOULD be; only the planner can say whether
2341
2341
  // it is reachable. The `unindexed-filter` branch shipped in 0.57 without that
2342
- // question answered and was right 6 times in 13 on a real schema, so every one
2342
+ // question answered, and a measured sample of 13 findings held up only 6 times,
2343
+ // so every one
2343
2344
  // of its findings is now put to a plan-only EXPLAIN. Nothing is executed, and a
2344
2345
  // probe that fails keeps its finding rather than dropping it.
2345
2346
  const flipProbe = divergenceOn && scored.findings.some(needsFlipProbe)
@@ -6,13 +6,14 @@
6
6
  * `plan-divergence.ts` scores a column from statistics alone and answers "IF the
7
7
  * cached plan flips, how bad is it". Its `unindexed-filter` branch (0.57) shipped
8
8
  * without an answer to the prior question, "CAN it flip at all", and that turned
9
- * out to be the majority case: on a real 118-model schema the branch produced 39
10
- * findings of which a measured sample was right 6 times in 13.
9
+ * out to be the majority case: in validation against a large schema the branch
10
+ * emitted 39 findings, and a measured sample of 13 of them held up only 6 times.
11
11
  *
12
- * Every false positive had one signature: **the generic plan kept the same
13
- * sequential scan the custom plan chose.** There was no flip to be had, so the
12
+ * Every false positive had one signature: **the generic plan was not the ordered
13
+ * index walk the finding claims.** There was no flip to be had, so the
14
14
  * amplification the finding printed described a plan the planner would never
15
- * pick.
15
+ * pick. See {@link verdictFromPlanJson} for the two ways a plan fails to be that
16
+ * walk; 0.58.0 shipped only one of them and 0.59.0 added the other.
16
17
  *
17
18
  * ## Why this is a probe and not another rule
18
19
  *
@@ -54,11 +55,10 @@
54
55
  * ```
55
56
  *
56
57
  * The custom plan is not needed. The finding's whole claim is that a promoted
57
- * generic plan abandons the seq scan for an ordered index walk, so if the generic
58
- * plan IS a seq scan on the target table, the claim is refuted no matter what the
59
- * custom plan does. Asking one question instead of two halves the work and
60
- * removes the need for a representative rare value, which statistics do not
61
- * carry.
58
+ * generic plan performs an ordered index walk, so a generic plan that is NOT that
59
+ * walk refutes it no matter what the custom plan does. Asking one question
60
+ * instead of two halves the work and removes the need for a representative rare
61
+ * value, which statistics do not carry.
62
62
  *
63
63
  * `NULL` is a safe argument precisely because the plan is generic: a generic plan
64
64
  * is built without looking at the value, which is the property the whole check is
@@ -80,11 +80,11 @@ import type { PlanDivergenceFinding, PlanDivergenceReport } from './plan-diverge
80
80
  /**
81
81
  * The planner's answer for one finding.
82
82
  *
83
- * - `'flip-reachable'`, the generic plan is NOT a plain seq scan of the target
83
+ * - `'flip-reachable'`, the generic plan IS an ordered index walk on the target
84
84
  * table, so the divergence the finding describes is one the planner can
85
85
  * actually choose.
86
- * - `'no-flip'`, the generic plan sequentially scans the target table, the same
87
- * access the good plan uses. Nothing to diverge to.
86
+ * - `'no-flip'`, the generic plan is not that walk (a `Sort` bounds it, or the
87
+ * access is a plain seq scan). Nothing to diverge to.
88
88
  * - `'unknown'`, the probe did not produce an answer. The finding is kept.
89
89
  */
90
90
  export type FlipVerdict = 'flip-reachable' | 'no-flip' | 'unknown';
@@ -133,11 +133,49 @@ export declare function buildFlipProbeSql(finding: PlanDivergenceFinding, name:
133
133
  * Exported for unit tests: the plan shapes this has to classify are exactly the
134
134
  * ones that are tedious to produce live.
135
135
  *
136
- * The rule is deliberately narrow. Only a `Seq Scan` ON THE TARGET TABLE refutes
137
- * a finding. A seq scan of some other relation in a more complex plan says
138
- * nothing about this column, and anything that is not a plain sequential scan of
139
- * the target (index scan, bitmap heap scan, index-only scan) leaves the flip
140
- * reachable.
136
+ * ## The question, stated exactly
137
+ *
138
+ * The finding claims the generic plan performs an ORDERED INDEX WALK along the
139
+ * `ORDER BY` column and fetches nearly every tuple before the `LIMIT` fills. So
140
+ * the refutation is not "the plan is a seq scan", it is **"the plan is not that
141
+ * ordered walk"**, and there are two ways for it not to be:
142
+ *
143
+ * 1. A `Sort` lies above the target table's scan. A sort materializes the whole
144
+ * matched set and orders it, so the cost is bounded by how many rows match,
145
+ * not by how far into the heap the ordered walk has to travel. Whatever feeds
146
+ * it (seq scan, bitmap heap scan) the catastrophic shape is absent.
147
+ * 2. The target's own scan node is a `Seq Scan`. Kept as an independent ground
148
+ * rather than folded into the first, so a hypothetical ordered seq scan with
149
+ * no sort still refutes.
150
+ *
151
+ * 0.58.0 shipped only the second ground and therefore missed every LOW-estimate
152
+ * column that has ANY usable index, because those plan as `Limit > Sort > Bitmap
153
+ * Heap Scan`. The case that surfaced it was a column carrying
154
+ * `btree (col) WHERE col IS NOT NULL`: an equality predicate implies not-null, so
155
+ * that partial index is fully usable and the plan never reaches a seq scan.
156
+ * Reproduced, and the fixture is the pair below at the same estimate:
157
+ *
158
+ * ```txt
159
+ * partial index, est 1.9 Limit > Sort > Bitmap Heap Scan <- 0.58 kept this
160
+ * no index, est 1.9 Limit > Sort > Seq Scan <- 0.58 refuted this
161
+ * either, est 500 Limit > Index Scan (no Sort) <- both keep, correctly
162
+ * ```
163
+ *
164
+ * ## Why this is not "exclude partial-index columns"
165
+ *
166
+ * Because a partial index whose predicate is NOT implied by the equality does not
167
+ * serve the query at all, and such a column produces a genuine finding: one was
168
+ * measured at 19,961x. The property that matters is whether the planner COULD
169
+ * use it, which is a proof obligation over predicates, and the plan already
170
+ * carries the answer. Reading the plan is cheaper and cannot drift from the
171
+ * planner's own implication rules.
172
+ *
173
+ * ## The safe direction is KEEP
174
+ *
175
+ * Over-refuting deletes real findings, which is invisible in the report;
176
+ * over-keeping only costs noise. So `Incremental Sort` does NOT refute: it means
177
+ * the index supplies a PREFIX of the ordering and the walk is still partly
178
+ * ordered, which is closer to the catastrophic shape than to the bounded one.
141
179
  */
142
180
  export declare function verdictFromPlanJson(payload: unknown, table: string): FlipVerdict;
143
181
  export interface ProbePlanFlipsOptions {
@@ -6,13 +6,14 @@
6
6
  * `plan-divergence.ts` scores a column from statistics alone and answers "IF the
7
7
  * cached plan flips, how bad is it". Its `unindexed-filter` branch (0.57) shipped
8
8
  * without an answer to the prior question, "CAN it flip at all", and that turned
9
- * out to be the majority case: on a real 118-model schema the branch produced 39
10
- * findings of which a measured sample was right 6 times in 13.
9
+ * out to be the majority case: in validation against a large schema the branch
10
+ * emitted 39 findings, and a measured sample of 13 of them held up only 6 times.
11
11
  *
12
- * Every false positive had one signature: **the generic plan kept the same
13
- * sequential scan the custom plan chose.** There was no flip to be had, so the
12
+ * Every false positive had one signature: **the generic plan was not the ordered
13
+ * index walk the finding claims.** There was no flip to be had, so the
14
14
  * amplification the finding printed described a plan the planner would never
15
- * pick.
15
+ * pick. See {@link verdictFromPlanJson} for the two ways a plan fails to be that
16
+ * walk; 0.58.0 shipped only one of them and 0.59.0 added the other.
16
17
  *
17
18
  * ## Why this is a probe and not another rule
18
19
  *
@@ -54,11 +55,10 @@
54
55
  * ```
55
56
  *
56
57
  * The custom plan is not needed. The finding's whole claim is that a promoted
57
- * generic plan abandons the seq scan for an ordered index walk, so if the generic
58
- * plan IS a seq scan on the target table, the claim is refuted no matter what the
59
- * custom plan does. Asking one question instead of two halves the work and
60
- * removes the need for a representative rare value, which statistics do not
61
- * carry.
58
+ * generic plan performs an ordered index walk, so a generic plan that is NOT that
59
+ * walk refutes it no matter what the custom plan does. Asking one question
60
+ * instead of two halves the work and removes the need for a representative rare
61
+ * value, which statistics do not carry.
62
62
  *
63
63
  * `NULL` is a safe argument precisely because the plan is generic: a generic plan
64
64
  * is built without looking at the value, which is the property the whole check is
@@ -124,39 +124,83 @@ export function buildFlipProbeSql(finding, name, searchSchema) {
124
124
  deallocate: `DEALLOCATE ${name}`,
125
125
  };
126
126
  }
127
- function* walkPlan(node) {
128
- yield node;
129
- for (const child of node.Plans ?? [])
130
- yield* walkPlan(child);
131
- }
132
127
  /**
133
128
  * Read a verdict out of one `EXPLAIN (FORMAT JSON)` payload.
134
129
  *
135
130
  * Exported for unit tests: the plan shapes this has to classify are exactly the
136
131
  * ones that are tedious to produce live.
137
132
  *
138
- * The rule is deliberately narrow. Only a `Seq Scan` ON THE TARGET TABLE refutes
139
- * a finding. A seq scan of some other relation in a more complex plan says
140
- * nothing about this column, and anything that is not a plain sequential scan of
141
- * the target (index scan, bitmap heap scan, index-only scan) leaves the flip
142
- * reachable.
133
+ * ## The question, stated exactly
134
+ *
135
+ * The finding claims the generic plan performs an ORDERED INDEX WALK along the
136
+ * `ORDER BY` column and fetches nearly every tuple before the `LIMIT` fills. So
137
+ * the refutation is not "the plan is a seq scan", it is **"the plan is not that
138
+ * ordered walk"**, and there are two ways for it not to be:
139
+ *
140
+ * 1. A `Sort` lies above the target table's scan. A sort materializes the whole
141
+ * matched set and orders it, so the cost is bounded by how many rows match,
142
+ * not by how far into the heap the ordered walk has to travel. Whatever feeds
143
+ * it (seq scan, bitmap heap scan) the catastrophic shape is absent.
144
+ * 2. The target's own scan node is a `Seq Scan`. Kept as an independent ground
145
+ * rather than folded into the first, so a hypothetical ordered seq scan with
146
+ * no sort still refutes.
147
+ *
148
+ * 0.58.0 shipped only the second ground and therefore missed every LOW-estimate
149
+ * column that has ANY usable index, because those plan as `Limit > Sort > Bitmap
150
+ * Heap Scan`. The case that surfaced it was a column carrying
151
+ * `btree (col) WHERE col IS NOT NULL`: an equality predicate implies not-null, so
152
+ * that partial index is fully usable and the plan never reaches a seq scan.
153
+ * Reproduced, and the fixture is the pair below at the same estimate:
154
+ *
155
+ * ```txt
156
+ * partial index, est 1.9 Limit > Sort > Bitmap Heap Scan <- 0.58 kept this
157
+ * no index, est 1.9 Limit > Sort > Seq Scan <- 0.58 refuted this
158
+ * either, est 500 Limit > Index Scan (no Sort) <- both keep, correctly
159
+ * ```
160
+ *
161
+ * ## Why this is not "exclude partial-index columns"
162
+ *
163
+ * Because a partial index whose predicate is NOT implied by the equality does not
164
+ * serve the query at all, and such a column produces a genuine finding: one was
165
+ * measured at 19,961x. The property that matters is whether the planner COULD
166
+ * use it, which is a proof obligation over predicates, and the plan already
167
+ * carries the answer. Reading the plan is cheaper and cannot drift from the
168
+ * planner's own implication rules.
169
+ *
170
+ * ## The safe direction is KEEP
171
+ *
172
+ * Over-refuting deletes real findings, which is invisible in the report;
173
+ * over-keeping only costs noise. So `Incremental Sort` does NOT refute: it means
174
+ * the index supplies a PREFIX of the ordering and the walk is still partly
175
+ * ordered, which is closer to the catastrophic shape than to the bounded one.
143
176
  */
144
177
  export function verdictFromPlanJson(payload, table) {
145
178
  const root = Array.isArray(payload) ? payload[0] : undefined;
146
179
  const plan = root?.Plan;
147
180
  if (!plan)
148
181
  return 'unknown';
149
- for (const node of walkPlan(plan)) {
150
- if (node['Relation Name'] !== table)
151
- continue;
182
+ // Walk root-downward, tracking whether a full Sort sits ABOVE the target's
183
+ // scan. Depth matters: a Sort somewhere else in a larger plan says nothing
184
+ // about how this table is reached.
185
+ const search = (node, sortedAbove) => {
152
186
  const type = node['Node Type'];
153
- if (type === undefined)
154
- continue;
155
- return type === 'Seq Scan' ? 'no-flip' : 'flip-reachable';
156
- }
157
- // The target table is not in the plan at all, which should not happen for a
158
- // statement that selects from it. Treated as unknown rather than as a refutation.
159
- return 'unknown';
187
+ if (node['Relation Name'] === table && type !== undefined) {
188
+ if (sortedAbove)
189
+ return 'no-flip';
190
+ return type === 'Seq Scan' ? 'no-flip' : 'flip-reachable';
191
+ }
192
+ // 'Incremental Sort' is deliberately excluded, see the header.
193
+ const nowSorted = sortedAbove || type === 'Sort';
194
+ for (const child of node.Plans ?? []) {
195
+ const found = search(child, nowSorted);
196
+ if (found !== null)
197
+ return found;
198
+ }
199
+ return null;
200
+ };
201
+ // The target table not appearing at all should not happen for a statement that
202
+ // selects from it. Treated as unknown rather than as a refutation.
203
+ return search(plan, false) ?? 'unknown';
160
204
  }
161
205
  /**
162
206
  * Ask the planner, once per candidate finding, whether the flip is reachable.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "turbine-orm",
3
- "version": "0.58.0",
3
+ "version": "0.59.1",
4
4
  "description": "Postgres-native TypeScript ORM, runs on Neon, Vercel Postgres, Cloudflare, Supabase. Streaming cursors, typed errors, single-query nested relations. One dependency, no WASM engine",
5
5
  "type": "module",
6
6
  "//exports": "Each subpath declares its types PER CONDITION. A single shared top-level \"types\" resolves to the ESM declarations for `require` too, which is TS1479 (\"is an ES module ... cannot be require()d\") for any CJS consumer on moduleResolution node16/nodenext. The require condition points at dist/cjs, which ships its own {\"type\":\"commonjs\"} package.json, so those declarations are CJS declarations. Gated in CI by publint + @arethetypeswrong/cli + a real .cts consumer typecheck (see the package-types job in ci.yml).",
@@ -126,8 +126,9 @@
126
126
  "lint:fix": "biome check --write src/",
127
127
  "format": "biome format --write src/",
128
128
  "check:error-codes": "tsx scripts/check-error-codes.ts",
129
+ "check:changelog": "node scripts/check-changelog-headings.mjs",
129
130
  "check:package": "publint --strict && attw --pack . --profile node16",
130
- "prepublishOnly": "npm run build && npm run typecheck && npm run lint && npm run test:unit && npm run check:error-codes && npm run size",
131
+ "prepublishOnly": "npm run build && npm run typecheck && npm run lint && npm run test:unit && npm run check:error-codes && npm run check:changelog && npm run size",
131
132
  "prepack": "node scripts/strip-prepare.mjs",
132
133
  "postpack": "node scripts/restore-prepare.mjs",
133
134
  "size": "size-limit",