@bastani/atomic-storage-libsql 0.9.89-alpha.2 → 0.9.90-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/CHANGELOG.md +19 -0
  2. package/libsql.js +398 -46
  3. package/package.json +1 -1
package/CHANGELOG.md CHANGED
@@ -2,6 +2,25 @@
2
2
 
3
3
  ## [Unreleased]
4
4
 
5
+ ## [0.9.90-alpha.1] - 2026-11-19
6
+
7
+ ### Fixed
8
+
9
+ - A `CREATE TRIGGER` sent alone in `exec` now retries after `SQLITE_SCHEMA` while other connections change a libSQL server's schema. Semicolons inside the trigger body no longer make it look like several statements. An `exec` containing additional statements is still never replayed, also when names in the trigger use characters outside ASCII such as combining accents ([#5509](https://github.com/bastani-inc/atomic/issues/5509))
10
+ - Write batches now refresh their schema after taking the write lock. A connection opened before another connection's schema change could otherwise report `no such table` for an existing table, including when a single write retried after `SQLITE_SCHEMA` ([#5536](https://github.com/bastani-inc/atomic/issues/5536))
11
+ - One statement followed by white space outside ASCII after its `;`, such as a byte-order mark or a no-break space, counts as one statement again: it runs again after `SQLITE_SCHEMA` or a busy answer, falls back to a write transaction of its own while other connections keep changing the schema, and a `SELECT` written so runs on the reused stream, as before the trigger change above ([#5676](https://github.com/bastani-inc/atomic/issues/5676))
12
+ - On a libSQL server (sqld) whose schema other connections keep changing, an `exec` of several statements outside a transaction no longer fails now and then with `SQLITE_SCHEMA: database schema has changed`, or with `SQLITE_BUSY`. The server refuses one statement this way without running it, after the statements before it committed, and the adapter used to report the failure without saying which statements ran. Such an `exec` is now sent as its statements, still in one request that runs them in order and stops at the first failure. A statement the server refused this way is sent again on its own, the way a single statement is, and the statements after it follow in another request. A statement that ran is never sent again. If a later request fails before reaching the server, the error is a `StorageTransientError` with an `unknown` outcome rather than `not-applied`, so the whole `exec` is not run again. A text with a `PRAGMA`, transaction control, a temporary table, a statement that returns rows (`SELECT`, `VALUES`, `WITH` or `RETURNING`), `last_insert_rowid()`, `changes()` or a parameter, or one the adapter cannot split exactly, is still sent as one text and not sent again. A text the server cannot read fails with the server's error for the whole text, which names the line and column in it, and an error inside a statement gives the same position as before ([#5652](https://github.com/bastani-inc/atomic/issues/5652))
13
+
14
+ ## [0.9.89] - 2026-11-18
15
+
16
+ ### Fixed
17
+
18
+ - A write that follows an outage on a libSQL server (sqld) or Turso database no longer waits for the adapter's next timed resend of the close of a stream the outage cut while it held the write lock. The adapter sent such a close again only on its backoff timer, 250 ms and then twice as long each time, so an outage longer than a wait left the close for the next one, up to seconds after the server was back, while the next write waited on the server for the lock the stream still held. Now every request sends the kept closes ahead of itself. A write waits for them, so that on a server that answers them the close arrives first and frees the lock; a request made only of plain `SELECT` statements does not wait, and any other statement, including `WITH`, `PRAGMA` and `EXPLAIN` queries, is treated as a possible write and may share the same bounded wait. Writes wait at most 2 seconds in all for closes the server takes but does not answer, then go ahead and wait for those closes no more, though a close kept later is waited for again; a caller's abort ends the wait at once. A close the server refuses because the outage lost the answer to a request it had already run still frees nothing: the lock clears when the server expires the stream. The timer still sends the closes while the database sends nothing else, and only its own unanswered resends lengthen its wait, once each. Against sqld after a 300 ms outage, the next write committed 4–14 ms after the server was back instead of 455–466 ms ([#5114](https://github.com/bastani-inc/atomic/issues/5114)).
19
+ - Fixed a worker host crashing after a libSQL server rolled back a write batch. Batch execution now observes every statement, COMMIT and ROLLBACK result, so later timeout rejections cannot escape the reported storage failure. A batch whose COMMIT never ran is no longer reported as successful ([#4768](https://github.com/bastani-inc/atomic/issues/4768))
20
+ - Fixed `run`, `get` and `all` statements failing with `Storage unavailable` on `ws:` and `wss:` URLs. These statements now use WebSockets instead of attempting HTTP requests ([#4768](https://github.com/bastani-inc/atomic/issues/4768))
21
+ - On a libSQL server (sqld) shared by stores that create and drop their tables, such as several sessions, workflows or teams using one server, a statement no longer fails with `SQLITE_SCHEMA: database schema has changed` when other connections change the schema while the server prepares it. The server ran nothing of such a statement, so it is sent again, as a statement the server answers busy is, for up to 5 seconds. A write outside a transaction (`INSERT`, `UPDATE`, `DELETE`, `REPLACE`, or a `WITH` that writes), or a schema change (`CREATE`, `DROP`, `ALTER`) sent alone in one `exec`, also no longer fails with `SQLITE_SCHEMA` while other connections keep changing the schema for longer than the 5-second wait. Sent again as it was, such a write could meet a changed schema on every try, and on a loaded server only two or three tries fit in the wait. After the first `SQLITE_SCHEMA` answer, such a statement is now sent again inside a write transaction of its own, in one request, with a wait of its own. While the transaction holds the write lock no other connection can change the schema, so the statement runs, and it commits once or not at all. Reads (including `VALUES` and a `WITH` that only selects) and other statements, such as pragmas, are still sent again as they were. A statement inside a transaction, and several statements sent in one `exec`, are not sent again
22
+ - On a libSQL server (sqld) shared by stores that create and drop their tables, a read or write outside a transaction no longer fails now and then with `SQLITE_UNKNOWN: no such table` for a table that exists. sqld gives that answer, without running the statement, when it compiles the statement while other connections change the schema; it happened to a `DELETE` from a table whose rows a trigger also removes from a full-text index, on kept and new streams alike. When the server names a table the database's schema lists, the statement now runs again with growing waits, up to 20 times; a table that does not exist is still reported at once ([#5464](https://github.com/bastani-inc/atomic/issues/5464))
23
+
5
24
  ## [0.9.89-alpha.2] - 2026-11-18
6
25
 
7
26
  ### Fixed
package/libsql.js CHANGED
@@ -1,11 +1,14 @@
1
1
  // @ts-check
2
+ import { statfs } from "node:fs/promises";
2
3
  import { createRequire } from "node:module";
4
+ import { fileURLToPath } from "node:url";
3
5
  import { setFlagsFromString } from "node:v8";
4
6
  import { runInNewContext } from "node:vm";
5
7
  import { createSqliteDatabase, redactSecrets, StorageAdapterLoadError, switchToWriteAheadLog } from "@bastani/atomic/storage";
6
8
 
7
9
  /**
8
10
  * @typedef {{ url: string, authToken?: string, namespace?: string, headers?: Readonly<Record<string, string>> }} LibsqlDatabaseOptions
11
+ * @typedef {[typeof import("@libsql/hrana-client"), typeof import("@libsql/core/config"), typeof import("@libsql/core/uri"), typeof import("@libsql/core/api")]} ServerProtocol
9
12
  */
10
13
 
11
14
  /** Header names an HTTP request can carry: RFC 9110 tokens. */
@@ -662,13 +665,18 @@ async function writeBatch(hrana, api, client, batched) {
662
665
  : error;
663
666
  const version = await client.getVersion();
664
667
  const stream = client.openStream();
665
- const { began, ran, committed, executed } = (() => {
668
+ const { began, refreshed, ran, committed, executed } = (() => {
666
669
  try {
667
670
  const batch = stream.batch(false);
668
671
  const inTransaction = () => hrana.BatchCond.not(hrana.BatchCond.isAutocommit(batch));
669
672
  const begin = batch.step();
670
673
  const began = settle(begin.run("BEGIN IMMEDIATE"));
671
- let last = begin;
674
+ // The connection may have loaded its schema before another connection's DDL committed. BEGIN takes the
675
+ // write lock but does not refresh that schema; read it after the lock, while no other DDL can change it.
676
+ const refresh = batch.step().condition(hrana.BatchCond.ok(begin));
677
+ if (version >= 3) refresh.condition(inTransaction());
678
+ const refreshed = settle(refresh.run(SCHEMA_CHECK));
679
+ let last = refresh;
672
680
  const ran = batched.map(({ sql, params }) => {
673
681
  const step = batch.step().condition(hrana.BatchCond.ok(last));
674
682
  if (version >= 3) step.condition(inTransaction());
@@ -681,7 +689,7 @@ async function writeBatch(hrana, api, client, batched) {
681
689
  if (version >= 3) commit.condition(inTransaction());
682
690
  const committed = settle(commit.run("COMMIT"));
683
691
  void settle(batch.step().condition(hrana.BatchCond.not(hrana.BatchCond.ok(commit))).run("ROLLBACK"));
684
- return { began, ran, committed, executed: batch.execute() };
692
+ return { began, refreshed, ran, committed, executed: batch.execute() };
685
693
  } finally {
686
694
  stream.closeGracefully();
687
695
  }
@@ -693,6 +701,10 @@ async function writeBatch(hrana, api, client, batched) {
693
701
  }
694
702
  const beginning = await began;
695
703
  if (!beginning.ok) throw mapped(beginning.error);
704
+ const refresh = await refreshed;
705
+ if (!refresh.ok) throw mapped(refresh.error);
706
+ if (refresh.value === undefined)
707
+ throw new api.LibsqlError("The batch ended before its schema could be refreshed", "TRANSACTION_CLOSED");
696
708
  /** @type {import("@bastani/atomic/storage").SqliteDriverResult[]} */
697
709
  const results = [];
698
710
  for (const [index, outcome] of (await Promise.all(ran)).entries()) {
@@ -738,9 +750,24 @@ async function writeBatch(hrana, api, client, batched) {
738
750
  * @param {() => Promise<ReusedStream | undefined>} [shared] the stream statements outside a transaction reuse
739
751
  * @param {(batched: readonly import("@bastani/atomic/storage").SqliteStatement[]) => Promise<import("@bastani/atomic/storage").SqliteDriverResult[]>} [locked]
740
752
  * runs statements in one write transaction, where no other connection can change the schema
753
+ * @param {(separate: readonly string[]) => Promise<StoppedStatement | undefined>} [stepped] runs statements in order
754
+ * in one request, each committing on its own (`stepBatch`)
741
755
  * @returns {import("@bastani/atomic/storage").SqliteDriverExecutor}
742
756
  */
743
- function statements(connection, watch, single = watch, shared = async () => undefined, locked) {
757
+ function statements(connection, watch, single = watch, shared = async () => undefined, locked, stepped) {
758
+ /**
759
+ * An `exec` of one statement, which wrote nothing when it fails, as one `execute` did.
760
+ * @param {string} sql
761
+ */
762
+ const one = (sql) =>
763
+ single(
764
+ async () => (await connection()).executeMultiple(sql),
765
+ locked !== undefined && rerunsLocked(sql)
766
+ ? async () => {
767
+ await locked([{ sql, params: [] }]);
768
+ }
769
+ : undefined,
770
+ );
744
771
  return {
745
772
  execute: (sql, params) =>
746
773
  single(
@@ -753,22 +780,177 @@ function statements(connection, watch, single = watch, shared = async () => unde
753
780
  // so sent as it was it could keep meeting a changed schema for as long as other connections keep changing it.
754
781
  locked !== undefined && rerunsLocked(sql) ? async () => (await locked([{ sql, params }]))[0] : undefined,
755
782
  ),
756
- // An `exec` of one statement wrote nothing when it fails, as one `execute` did; one of several statements may
757
- // fail after the statements before it committed.
758
- executeMultiple: (sql) =>
759
- isOneStatement(sql)
760
- ? single(
761
- async () => (await connection()).executeMultiple(sql),
762
- locked !== undefined && rerunsLocked(sql)
763
- ? async () => {
764
- await locked([{ sql, params: [] }]);
765
- }
766
- : undefined,
767
- )
768
- : watch(async () => (await connection()).executeMultiple(sql)),
783
+ // One of several statements may fail after the statements before it committed, so the text is never sent again.
784
+ // On a server, outside a transaction, a text that `separateStatements` splits is sent as its statements, in
785
+ // one request that runs them in order and stops at the first that fails, as the server runs the text. A
786
+ // statement the server refused without running it, with `SQLITE_SCHEMA` while other connections change the
787
+ // schema (#5652) or busy, then runs again on its own as an `exec` of one statement does, and the statements
788
+ // after it go on in a request of their own: a statement that ran is never sent again.
789
+ executeMultiple: async (sql) => {
790
+ if (isOneStatement(sql)) return one(sql);
791
+ const separate = stepped === undefined ? undefined : separateStatements(sql);
792
+ const whole = () => watch(async () => (await connection()).executeMultiple(sql));
793
+ if (stepped === undefined || separate === undefined) return whole();
794
+ for (let next = 0; next < separate.length; ) {
795
+ /** @type {number} */
796
+ let failed = next;
797
+ try {
798
+ /** @type {StoppedStatement | undefined} */
799
+ let stopped;
800
+ try {
801
+ stopped = await watch(() => stepped(separate.slice(next)));
802
+ } catch (error) {
803
+ // The server reads every statement of a request before it runs any, so statements it could not read ran
804
+ // none. It reads the whole text differently: it tells where in the text a syntax error is, and runs the
805
+ // statements before an unfinished last one. So the text is sent as it is, and the caller gets the
806
+ // server's own answer for it.
807
+ if (next === 0 && UNREAD_CODES.has(/** @type {{ readonly code?: unknown }} */ (error)?.code)) return whole();
808
+ throw error;
809
+ }
810
+ if (stopped === undefined) return;
811
+ failed = next + stopped.index;
812
+ if (!isBusy(stopped.error) && !isSchemaChanged(stopped.error)) throw stopped.error;
813
+ await one(separate[failed]);
814
+ next = failed + 1;
815
+ } catch (error) {
816
+ if (failed > 0 && typeof error === "object" && error !== null) partlyApplied.add(error);
817
+ throw error;
818
+ }
819
+ }
820
+ },
769
821
  };
770
822
  }
771
823
 
824
+ /**
825
+ * The statement of a `stepBatch` that did not run, by its index among the statements sent, and the server's answer.
826
+ * @typedef {{ index: number, error: unknown }} StoppedStatement
827
+ */
828
+
829
+ /**
830
+ * Runs `separate`, statements that each commit on their own, in order in one request on a stream of its own, as the
831
+ * server runs an `exec` of their text: the server reads every statement before it runs any, so a text it cannot read
832
+ * runs nothing, and a statement runs only when the one before it succeeded.
833
+ * @param {typeof import("@libsql/hrana-client")} hrana
834
+ * @param {typeof import("@libsql/core/api")} api
835
+ * @param {import("@libsql/hrana-client").Client} client
836
+ * @param {readonly string[]} separate
837
+ * @returns {Promise<StoppedStatement | undefined>} the first statement that did not run, or undefined when all ran
838
+ */
839
+ async function stepBatch(hrana, api, client, separate) {
840
+ /** @param {unknown} error */
841
+ const mapped = (error) =>
842
+ error instanceof hrana.ClientError
843
+ ? new api.LibsqlError(error.message, hranaCode(hrana, error), undefined, undefined, error)
844
+ : error;
845
+ const stream = client.openStream();
846
+ const { ran, executed } = (() => {
847
+ try {
848
+ const batch = stream.batch(false);
849
+ /** @type {import("@libsql/hrana-client").BatchStep | undefined} */
850
+ let last;
851
+ const ran = separate.map((sql) => {
852
+ const step = batch.step();
853
+ if (last !== undefined) step.condition(hrana.BatchCond.ok(last));
854
+ last = step;
855
+ return settle(step.run(sql));
856
+ });
857
+ return { ran, executed: batch.execute() };
858
+ } finally {
859
+ stream.closeGracefully();
860
+ }
861
+ })();
862
+ try {
863
+ await executed;
864
+ } catch (error) {
865
+ throw mapped(error);
866
+ }
867
+ for (const [index, outcome] of (await Promise.all(ran)).entries()) {
868
+ if (!outcome.ok) return { index, error: mapped(outcome.error) };
869
+ if (outcome.value === undefined)
870
+ return {
871
+ index,
872
+ error: new api.LibsqlError("Statement in a batch was not executed", "TRANSACTION_CLOSED"),
873
+ };
874
+ }
875
+ return undefined;
876
+ }
877
+
878
+ /**
879
+ * The codes a server answers a request with when it cannot read its statements, which it reads before it runs any.
880
+ * @type {ReadonlySet<unknown>}
881
+ */
882
+ const UNREAD_CODES = new Set(["SQL_PARSE_ERROR", "SQL_NO_STATEMENT", "SQL_MANY_STATEMENTS"]);
883
+
884
+ /**
885
+ * Failures of an `exec` of several statements after statements of it committed. A transient failure among them is
886
+ * reported with an unknown outcome even when the request that failed never reached the server, so the facade does not
887
+ * run the whole `exec` again.
888
+ * @type {WeakSet<object>}
889
+ */
890
+ const partlyApplied = new WeakSet();
891
+
892
+ /**
893
+ * The keywords that start a statement `separateStatements` may send apart from the ones before it: statements whose
894
+ * effect is in the database, which another connection sees once it committed, and which return no rows.
895
+ */
896
+ const SEPARABLE_KEYWORDS = new Set(["CREATE", "DROP", "ALTER", "INSERT", "UPDATE", "DELETE", "REPLACE"]);
897
+ /**
898
+ * Names, quoted or not, that tie a statement to the connection that ran the statements before it: a temporary table,
899
+ * view or trigger, and the functions that read what that connection did last or load code into it.
900
+ */
901
+ const CONNECTION_NAMES = new Set(["TEMP", "TEMPORARY", "LAST_INSERT_ROWID", "CHANGES", "TOTAL_CHANGES", "LOAD_EXTENSION"]);
902
+ /** The keyword after which a write returns rows. */
903
+ const RETURNING_KEYWORD = "RETURNING";
904
+
905
+ /**
906
+ * The statements of `sql`, an `exec` of several, each as the server reads it on its own, or undefined when they must
907
+ * run on one connection or are not told apart exactly. Each statement starts with one of `SEPARABLE_KEYWORDS`, no
908
+ * name of `CONNECTION_NAMES` and no `RETURNING_KEYWORD` appears in the text, quoted or not, the text's tokens are
909
+ * SQLite's (`statementSpans`), and each statement passes `isOneStatement` on its own. A statement ends at the `;`
910
+ * after it, and a `CREATE TRIGGER` at the first `END` right after a `;`, as in SQLite's grammar. Settings such as a
911
+ * `PRAGMA`, transaction control and temporary tables hold on one connection only, which statements sent again apart
912
+ * from the ones before them may not share, so a text with one of them is not split. Nor is a text with a statement
913
+ * that returns rows: a server describes the columns of each statement sent apart, which together may pass the size it
914
+ * answers in one response, and describes none for the statements of a text. Each statement starts where the server's
915
+ * reading of the text starts it, right after the `;` before it (the first at the text's start), so a failure the
916
+ * server reports at a place in the statement is at the same place as in the text.
917
+ * @param {string} sql
918
+ * @returns {string[] | undefined}
919
+ */
920
+ function separateStatements(sql) {
921
+ const spans = statementSpans(sql);
922
+ if (spans === undefined) return undefined;
923
+ for (const { start, end } of spans) {
924
+ const text = sql.slice(start, end);
925
+ const name = Object.hasOwn(QUOTE_END, text[0]) ? text.slice(1, -1) : text;
926
+ const upper = name.toUpperCase();
927
+ if (CONNECTION_NAMES.has(upper) || upper === RETURNING_KEYWORD) return undefined;
928
+ }
929
+ /** @type {string[]} */
930
+ const separate = [];
931
+ let from = 0;
932
+ for (let index = 0; index < spans.length; ) {
933
+ if (spans[index].token === ";") {
934
+ index++;
935
+ continue;
936
+ }
937
+ if (!SEPARABLE_KEYWORDS.has(spans[index].token)) return undefined;
938
+ let last = index;
939
+ if (spans[index].token === "CREATE" && spans[index + 1]?.token === "TRIGGER") {
940
+ while (last + 1 < spans.length && !(spans[last].token === ";" && spans[last + 1].token === "END")) last++;
941
+ if (last + 1 >= spans.length) return undefined;
942
+ last++;
943
+ if (last + 1 < spans.length && spans[last + 1].token !== ";") return undefined;
944
+ } else while (last + 1 < spans.length && spans[last + 1].token !== ";") last++;
945
+ const statement = sql.slice(from, spans[last].end);
946
+ if (!isOneStatement(statement)) return undefined;
947
+ separate.push(statement);
948
+ from = spans[last + 1]?.end ?? sql.length;
949
+ index = last + 1;
950
+ }
951
+ return separate.length > 1 ? separate : undefined;
952
+ }
953
+
772
954
  /** The keyword of a statement that writes and runs the same inside a write transaction as on its own. */
773
955
  const LOCKED_RERUN_KEYWORD = /(?:INSERT|UPDATE|DELETE|REPLACE|CREATE|DROP|ALTER)\b/iy;
774
956
  /** The keyword of a statement that starts with common table expressions, which may read or write. */
@@ -793,13 +975,125 @@ function rerunsLocked(sql) {
793
975
  }
794
976
 
795
977
  /**
796
- * Whether `sql` is one statement: it has no `;` but at its end. A `;` in a literal or a comment also counts, so
797
- * several statements are never taken for one.
978
+ * Whether `sql` is one statement: it has no `;` but at its end, or it is one `CREATE TRIGGER`, whose body holds a `;`
979
+ * after each of its statements. Otherwise a `;` in a literal or a comment also counts, so several statements are never
980
+ * taken for one. After the last `;`, any white space counts as none, a no-break space or a byte-order mark too (#5676):
981
+ * SQLite skips a byte-order mark and fails on any other such character, and sqld runs such a text as one statement,
982
+ * so none of them can start another statement that runs.
798
983
  * @param {string} sql
799
984
  */
800
985
  function isOneStatement(sql) {
801
986
  const first = sql.indexOf(";");
802
- return first === -1 || /^[\s;]*$/.test(sql.slice(first));
987
+ return first === -1 || /^[\s;]*$/.test(sql.slice(first)) || isOneTrigger(sql);
988
+ }
989
+
990
+ /** The keyword that starts a `CREATE TRIGGER`. */
991
+ const CREATE_KEYWORD = /CREATE\b/iy;
992
+ /** The characters SQLite skips between tokens; it reads any other character, a no-break space too, as part of one. */
993
+ const SQL_SPACE = " \t\n\f\r";
994
+ /** The characters of operators and punctuation, other than `;`, that SQLite reads as tokens that are not names. */
995
+ const SQL_PUNCTUATION = "(),.+-*/%=<>!|&~";
996
+ /** The character that ends a quoted literal or name, by the character that opened it. */
997
+ const QUOTE_END = /** @type {Record<string, string>} */ ({ "'": "'", '"': '"', "`": "`", "[": "]" });
998
+
999
+ /**
1000
+ * Whether the UTF-16 code unit `code` belongs to a keyword or a name that is not quoted, as SQLite reads them: an ASCII
1001
+ * letter or digit, `_`, `$`, or any character outside ASCII, every byte of which is outside ASCII in the UTF-8 SQLite
1002
+ * reads. So a combining accent, a joiner or a no-break space after the letters of a keyword makes a name.
1003
+ * @param {number} code
1004
+ */
1005
+ function isWordCode(code) {
1006
+ return (
1007
+ (code >= 0x30 && code <= 0x39) ||
1008
+ (code >= 0x41 && code <= 0x5a) ||
1009
+ (code >= 0x61 && code <= 0x7a) ||
1010
+ code === 0x5f ||
1011
+ code === 0x24 ||
1012
+ code > 0x7f
1013
+ );
1014
+ }
1015
+
1016
+ /**
1017
+ * The tokens of `sql`, as SQLite and sqld split it, that tell where its statements end: `;`, each keyword or name that
1018
+ * is not quoted, as its ASCII letters upper-cased or as "" when it has another character, and "" for any other token.
1019
+ * Whitespace and comments are left out. Undefined for text whose tokens this does not tell exactly, so that no
1020
+ * statement boundary can hide in it: an open literal, quoted name or block comment, a parameter (`?`, `:`, `@`, `#`, or
1021
+ * `$` where a token starts, which no trigger may hold), a NUL, at which SQLite stops reading, a byte-order mark where a
1022
+ * token starts, which SQLite skips as whitespace and sqld reads as part of a name (inside a token both read it as part
1023
+ * of it), and any other character that is in no token.
1024
+ * @param {string} sql
1025
+ * @returns {string[] | undefined}
1026
+ */
1027
+ function statementTokens(sql) {
1028
+ return statementSpans(sql)?.map(({ token }) => token);
1029
+ }
1030
+
1031
+ /**
1032
+ * The tokens `statementTokens` reads in `sql`, each with where it starts and ends in `sql`.
1033
+ * @param {string} sql
1034
+ * @returns {{ token: string, start: number, end: number }[] | undefined}
1035
+ */
1036
+ function statementSpans(sql) {
1037
+ if (sql.includes("\0")) return undefined;
1038
+ /** @type {{ token: string, start: number, end: number }[]} */
1039
+ const tokens = [];
1040
+ let index = 0;
1041
+ while (index < sql.length) {
1042
+ const character = sql[index];
1043
+ if (SQL_SPACE.includes(character)) index++;
1044
+ else if (sql.startsWith("--", index)) {
1045
+ const end = sql.indexOf("\n", index + 2);
1046
+ index = end === -1 ? sql.length : end + 1;
1047
+ } else if (sql.startsWith("/*", index)) {
1048
+ const end = sql.indexOf("*/", index + 2);
1049
+ if (end === -1) return undefined;
1050
+ index = end + 2;
1051
+ } else if (Object.hasOwn(QUOTE_END, character)) {
1052
+ // A quote doubled inside a literal or name stays in it: the scan goes on from the quote after it, which starts a
1053
+ // token here that is the same token's rest to SQLite. Neither is `;` or a word.
1054
+ const end = sql.indexOf(QUOTE_END[character], index + 1);
1055
+ if (end === -1) return undefined;
1056
+ tokens.push({ token: "", start: index, end: end + 1 });
1057
+ index = end + 1;
1058
+ } else if (character === ";" || SQL_PUNCTUATION.includes(character)) {
1059
+ tokens.push({ token: character === ";" ? ";" : "", start: index, end: index + 1 });
1060
+ index++;
1061
+ } else if (character !== "$" && character !== "\uFEFF" && isWordCode(sql.charCodeAt(index))) {
1062
+ // A number is read as a word: SQLite reads the letters and digits that follow a number's own as part of it.
1063
+ let end = index + 1;
1064
+ while (end < sql.length && isWordCode(sql.charCodeAt(end))) end++;
1065
+ const word = sql.slice(index, end);
1066
+ tokens.push({ token: /^[A-Za-z]+$/.test(word) ? word.toUpperCase() : "", start: index, end });
1067
+ index = end;
1068
+ } else return undefined;
1069
+ }
1070
+ return tokens;
1071
+ }
1072
+
1073
+ /**
1074
+ * Whether `sql` is one `CREATE [TEMP] TRIGGER ... BEGIN ... END` statement, followed by nothing but `;`, whitespace and
1075
+ * comments. Its tokens are SQLite's, or the answer is no. SQLite takes no `;` before a trigger's `BEGIN`, ends every
1076
+ * statement of its body with `;`, and starts none with `END`, so the trigger ends at the first `END` right after a `;`,
1077
+ * whatever `CASE ... END` the body holds. A text that starts so but whose first statement is no such trigger is a
1078
+ * syntax error, which runs nothing.
1079
+ * @param {string} sql
1080
+ */
1081
+ function isOneTrigger(sql) {
1082
+ // Most texts are no trigger, which their first word tells without reading them whole.
1083
+ const start = firstTokenStart(sql);
1084
+ if (start === -1) return false;
1085
+ CREATE_KEYWORD.lastIndex = start;
1086
+ if (!CREATE_KEYWORD.test(sql)) return false;
1087
+ const tokens = statementTokens(sql);
1088
+ if (tokens === undefined || tokens[0] !== "CREATE") return false;
1089
+ let index = tokens[1] === "TEMP" || tokens[1] === "TEMPORARY" ? 2 : 1;
1090
+ if (tokens[index++] !== "TRIGGER") return false;
1091
+ const first = tokens.indexOf(";", index);
1092
+ if (first === -1 || !tokens.slice(index, first).includes("BEGIN")) return false;
1093
+ for (let at = first; at + 1 < tokens.length; at++) {
1094
+ if (tokens[at] === ";" && tokens[at + 1] === "END") return tokens.slice(at + 2).every((token) => token === ";");
1095
+ }
1096
+ return false;
803
1097
  }
804
1098
 
805
1099
  /**
@@ -1134,11 +1428,24 @@ const CONNECTION_LOST = new Set(["ECONNRESET", "EPIPE", "ETIMEDOUT", "UND_ERR_SO
1134
1428
  /**
1135
1429
  * The failure, as one that passes, when the server ended the stream or transaction of a process that stalled, or the
1136
1430
  * connection to the server could not be made or broke, or something between answered HTTP 5xx; read from each error's
1137
- * `code`, `name` and `status` along the cause chain, never from its message.
1431
+ * `code`, `name` and `status` along the cause chain, never from its message. A failure of an `exec` of several
1432
+ * statements after some of them committed (`partlyApplied`) has an unknown outcome.
1138
1433
  * @param {import("@bastani/atomic/storage").StorageFailureCause} failure
1139
1434
  * @returns {import("@bastani/atomic/storage").SqliteDriverTransientFailure | undefined}
1140
1435
  */
1141
1436
  function transientFailure(failure) {
1437
+ const transient = transientCause(failure);
1438
+ return transient !== undefined && typeof failure === "object" && failure !== null && partlyApplied.has(failure)
1439
+ ? { ...transient, outcome: "unknown" }
1440
+ : transient;
1441
+ }
1442
+
1443
+ /**
1444
+ * `transientFailure` of one request.
1445
+ * @param {import("@bastani/atomic/storage").StorageFailureCause} failure
1446
+ * @returns {import("@bastani/atomic/storage").SqliteDriverTransientFailure | undefined}
1447
+ */
1448
+ function transientCause(failure) {
1142
1449
  /** @type {unknown} */
1143
1450
  let current = failure;
1144
1451
  for (let depth = 0; depth < MAX_CAUSE_DEPTH && typeof current === "object" && current !== null; depth++) {
@@ -1160,6 +1467,26 @@ function transientFailure(failure) {
1160
1467
  return undefined;
1161
1468
  }
1162
1469
 
1470
+ /**
1471
+ * The bytes this process may still write on the file system that holds the database file a `file:` URL names (where a
1472
+ * link to it points, and where its write-ahead log goes), or undefined for a database in memory or when the file system
1473
+ * does not say.
1474
+ * @param {string} url
1475
+ * @returns {Promise<number | undefined>}
1476
+ */
1477
+ async function freeBytesOfFile(url) {
1478
+ try {
1479
+ const file = url.split("?")[0] ?? "";
1480
+ const path = /^file:\/\//i.test(file) ? fileURLToPath(file) : decodeURIComponent(file.slice("file:".length));
1481
+ if (path === "" || path === ":memory:") return undefined;
1482
+ // The file itself, not its directory: a link to a database on another file system measures that file system.
1483
+ const { bavail, bsize } = await statfs(path);
1484
+ return Number(bavail) * Number(bsize);
1485
+ } catch {
1486
+ return undefined;
1487
+ }
1488
+ }
1489
+
1163
1490
  /**
1164
1491
  * @param {LibsqlDatabaseOptions} options
1165
1492
  * @returns {Promise<import("@bastani/atomic/storage").SqliteDatabase>}
@@ -1174,7 +1501,7 @@ export async function openLibsqlDatabase({ url, authToken, namespace, headers })
1174
1501
  throw new StorageAdapterLoadError("@libsql/client", /** @type {import("@bastani/atomic/storage").StorageFailureCause} */ (cause));
1175
1502
  }
1176
1503
  const local = /^file:/i.test(url);
1177
- /** @type {[typeof import("@libsql/hrana-client"), typeof import("@libsql/core/config"), typeof import("@libsql/core/uri"), typeof import("@libsql/core/api")] | undefined} */
1504
+ /** @type {ServerProtocol | undefined} */
1178
1505
  let protocol;
1179
1506
  if (!local) {
1180
1507
  try {
@@ -1198,6 +1525,40 @@ export async function openLibsqlDatabase({ url, authToken, namespace, headers })
1198
1525
  throw new StorageAdapterLoadError("libsql", /** @type {import("@bastani/atomic/storage").StorageFailureCause} */ (cause));
1199
1526
  }
1200
1527
  }
1528
+ const redact = redactSecrets([authToken ?? "", ...urlCredentials(url), ...Object.values(headers ?? {})]);
1529
+ const connect = () => createConnection(url, authToken, added, redact, createClient, protocol);
1530
+ const database = connect();
1531
+ // The reader shares the checked route and credentials, but no client, stream or queue. Reads need no write setup.
1532
+ if (!local) database.openReader = async () => connect();
1533
+ try {
1534
+ await database.exec("PRAGMA foreign_keys = ON");
1535
+ // Readers and the one writer of a file share it through the write-ahead log, as in Atomic's own SQLite stores;
1536
+ // in the default rollback journal a read refuses a writer's commit and the other way round. The mode is stored
1537
+ // in the file, so this changes it once and costs nothing afterwards. SQLite does not wait for another
1538
+ // connection's lock while it changes the mode, so the switch waits out the busy timeout itself: several
1539
+ // processes opening one new file at once, or one writing it, would otherwise fail the open with SQLITE_BUSY.
1540
+ if (local) await switchToWriteAheadLog((sql) => database.exec(sql), FILE_BUSY_TIMEOUT_MS);
1541
+ return database;
1542
+ } catch (error) {
1543
+ await database
1544
+ .close()
1545
+ .catch((cleanupCause) =>
1546
+ Object.defineProperty(/** @type {Error} */ (error), "cleanupCause", { value: cleanupCause, configurable: true }),
1547
+ );
1548
+ throw error;
1549
+ }
1550
+ }
1551
+
1552
+ /**
1553
+ * @param {string} url
1554
+ * @param {string | undefined} authToken
1555
+ * @param {[string, string][]} added
1556
+ * @param {(text: string) => string} redact
1557
+ * @param {typeof import("@libsql/client").createClient} createClient
1558
+ * @param {ServerProtocol | undefined} protocol
1559
+ */
1560
+ function createConnection(url, authToken, added, redact, createClient, protocol) {
1561
+ const local = protocol === undefined;
1201
1562
  /** @type {import("@libsql/client").Client | undefined} */
1202
1563
  let client;
1203
1564
  const abandoned = new AbandonedStreams();
@@ -1334,20 +1695,28 @@ export async function openLibsqlDatabase({ url, authToken, namespace, headers })
1334
1695
  );
1335
1696
  return results.map(driverResult);
1336
1697
  };
1337
- const redact = redactSecrets([authToken ?? "", ...urlCredentials(url), ...Object.values(headers ?? {})]);
1338
1698
  /** The server's location as diagnostics show it, without credentials. */
1339
1699
  const redacted = redact(url);
1340
- const database = createSqliteDatabase({
1700
+ return createSqliteDatabase({
1341
1701
  descriptor: { adapter: "libsql", location: url },
1342
1702
  redact,
1343
1703
  transientFailure,
1344
1704
  payloadLimits: local ? LOCAL_PAYLOAD_LIMITS : SERVER_PAYLOAD_LIMITS,
1705
+ // A server's disk is not this process's to see.
1706
+ ...(local ? { freeBytes: () => freeBytesOfFile(url) } : {}),
1345
1707
  // A server's batch the busy lock ended: the facade sends it again within the caller's `lockWaitMs`.
1346
1708
  lockBusy: (failure) => !local && isLockBusy(failure),
1347
1709
  // One statement outside a transaction, run alone or as an `exec` of one statement, that waited out the busy
1348
- // timeout wrote nothing and runs again. Several statements in one `exec` do not: the ones before the failed
1349
- // statement were committed.
1350
- ...statements(connection, watch, waitForLock, shared, local ? undefined : writeTransaction),
1710
+ // timeout wrote nothing and runs again. Several statements in one `exec` are never sent again as a whole: the
1711
+ // ones before the failed statement were committed. On a server, the failed statement alone may be (`statements`).
1712
+ ...statements(
1713
+ connection,
1714
+ watch,
1715
+ waitForLock,
1716
+ shared,
1717
+ local ? undefined : writeTransaction,
1718
+ protocol === undefined ? undefined : (separate) => stepBatch(protocol[0], protocol[3], remote(), separate),
1719
+ ),
1351
1720
  // One request for the whole batch, in a write transaction: the journal sends a batch of appends this way. Only
1352
1721
  // its `BEGIN IMMEDIATE` waits for the write lock, so a batch the busy timeout ended wrote nothing (`lockBusy`). A
1353
1722
  // server's batch counts against this process's writes in flight to that server, unless it has priority.
@@ -1374,8 +1743,8 @@ export async function openLibsqlDatabase({ url, authToken, namespace, headers })
1374
1743
  () => true,
1375
1744
  );
1376
1745
  },
1377
- async beginTransaction() {
1378
- const begin = async () => (await connection()).transaction("write");
1746
+ async beginTransaction(options) {
1747
+ const begin = async () => (await connection()).transaction(options?.readOnly === true ? "read" : "write");
1379
1748
  let transaction = await waitForLock(begin);
1380
1749
  // A server's transaction sends its `BEGIN IMMEDIATE` with its first statement. A server that answers that busy
1381
1750
  // fails the statement and the client closes the transaction, having run nothing of it, so the statement runs
@@ -1430,23 +1799,6 @@ export async function openLibsqlDatabase({ url, authToken, namespace, headers })
1430
1799
  if (local) await releaseClosedConnections();
1431
1800
  },
1432
1801
  });
1433
- try {
1434
- await database.exec("PRAGMA foreign_keys = ON");
1435
- // Readers and the one writer of a file share it through the write-ahead log, as in Atomic's own SQLite stores;
1436
- // in the default rollback journal a read refuses a writer's commit and the other way round. The mode is stored
1437
- // in the file, so this changes it once and costs nothing afterwards. SQLite does not wait for another
1438
- // connection's lock while it changes the mode, so the switch waits out the busy timeout itself: several
1439
- // processes opening one new file at once, or one writing it, would otherwise fail the open with SQLITE_BUSY.
1440
- if (local) await switchToWriteAheadLog((sql) => database.exec(sql), FILE_BUSY_TIMEOUT_MS);
1441
- return database;
1442
- } catch (error) {
1443
- await database
1444
- .close()
1445
- .catch((cleanupCause) =>
1446
- Object.defineProperty(/** @type {Error} */ (error), "cleanupCause", { value: cleanupCause, configurable: true }),
1447
- );
1448
- throw error;
1449
- }
1450
1802
  }
1451
1803
 
1452
1804
  /** @satisfies {import("@bastani/atomic/storage").StorageAdapterModule} */
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@bastani/atomic-storage-libsql",
3
- "version": "0.9.89-alpha.2",
3
+ "version": "0.9.90-alpha.1",
4
4
  "description": "libSQL / Turso storage adapter for Atomic sessions, workflow journals, and Intercom.",
5
5
  "contributors": [
6
6
  "Norin Lavaee",