frostlake 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bb09cf620d3baab58a71403524ea8a8edc880fc52597196c2e6e10afa4507ad8
-  data.tar.gz: 8eefe45f34075e8300cf7964588a0e8b7891da9158c9110c79aa9db5a081280d
+  metadata.gz: 3cf1e48b23f2ced0c5ddea38ec4df19cb621c1ea10138f0a608c47f7781feebc
+  data.tar.gz: 1fe5fbed7c4ac104666ebe75a48ed8d4b27b6f739177ea4989d69cc89ed8c1dd
 SHA512:
-  metadata.gz: df21cb9b7ec8a9c37dbe062d705e6f1cdbebc11640c45bcb8db0f2250ef5b9ad203dee933980a802a455efcf2ba1b798ccf1bfee8559205d9371251192e13d49
-  data.tar.gz: 16752c3e007ff4c046b5b57762039f0db36ef7e24aa3d9b144b144ebee9cd4f96950d79a1a1f56c6b198e5e643231564fbb1406b5805ce5ea568051acfce3c1b
+  metadata.gz: ce4592c00895efb401c086b7c170100820ddb5036928692621227e85f523faf72834bd46c9c53896ec942abc9d2383bf57759f5e00db7a5b8a985feb2fe56b80
+  data.tar.gz: f67792bc1e5beddc690ba2aeff1f14569eedd49eeffafe62b10e635750c99b33ab273f612e497c4f987253153d258f33c67a6b952b653fdbb89ebee66df7a9e4

data/README.md

@@ -24,7 +24,7 @@ passed through intact.
 
 ## Engine version
 
-Requires a Frostlake engine **0.0.7 or newer**. Ask a running server which one it is with
+Requires a Frostlake engine **0.2.0 or newer**. Ask a running server which one it is with
 `SELECT CURRENT_VERSION()` — every release answers it, so the check works against any engine.
 
 The driver versions independently of the engine: it speaks the HTTP protocol, not
@@ -32,7 +32,8 @@ the jar, so this is a floor rather than a lockstep pin.
 
 One behaviour does depend on the engine: a `TIMESTAMP_TZ` column only reports back the
 UTC offset it was given from engine **0.1.0** on. Against an older engine a bound `Time`
-still round-trips, but the offset comes back as `+00:00`.
+still round-trips, but the offset comes back as `+00:00`. Recovering from a lost session
+and releasing the session on close need **0.1.0** too; see [Session lifetime](#session-lifetime).
 
 ## Usage
 
@@ -81,6 +82,13 @@ conn.execute("...")
 conn.commit                  # / conn.rollback
 ```
 
+`begin_transaction` sends `BEGIN` with autocommit off, and the connection leaves
+autocommit only once the engine has opened the transaction. A begin that fails — refused,
+unanswered or unreadable, lost with its session, or never sent because a `USE` queued
+ahead of it was refused — leaves the connection in autocommit, so the statements after it
+commit as they run. `transaction` follows a begin that fails with a best-effort rollback
+as well, in case a `BEGIN` whose answer was lost did open a transaction.
+
 ### Bind values
 
 Parameters are inlined client-side (`?` placeholders); placeholders inside string
@@ -99,6 +107,12 @@ Frostlake's other drivers.
 | `Date` | `'…'::DATE` |
 | `Array` | `[…]` (elements formatted recursively) |
 
+A bound `Time` goes into a `TIMESTAMP_TZ` column as it is. A `TIMESTAMP_NTZ` or
+`TIMESTAMP_LTZ` column refuses it while compiling, as the account refuses any
+`TIMESTAMP_TZ` written into one (`expecting TIMESTAMP_NTZ(9) but got TIMESTAMP_TZ(9)`),
+so cast the bind there: `CAST(? AS TIMESTAMP_NTZ)` keeps the wall clock the `Time` was
+written with, and `CAST(? AS TIMESTAMP_LTZ)` keeps its instant.
+
 ### Result types
 
 Fixed-point `NUMBER` keeps the exact digits the engine sent: `BigDecimal` when the
@@ -108,8 +122,23 @@ arbitrary precision). `FLOAT`/`DOUBLE`/`REAL` are genuine binary floats and stay
 `BINARY` a binary-encoded `String`; `TIME` and semi-structured values keep their
 wire shape as strings.
 
-Each entry in `columns` is a hash of `{ name:, data_type:, scale: }`, carrying the
-engine's own type name.
+A `TIMESTAMP_NTZ` is a wall clock with no zone of its own, so it comes back as a `Time`
+flagged UTC (`utc?` is true) whose fields read back exactly as stored, whatever zone the
+host is in; a column declared `DATETIME`, or `TIMESTAMP` under the default mapping, is
+one. Read in the host's local zone instead, a wall clock that zone skips would move:
+`2024-03-31 01:30` does not exist in London, whose clocks go from 01:00 straight to 02:00
+that night, so it would come back as `02:30 +0100`. The instant such a `Time` names is its
+wall clock read as UTC, the one the engine's own epoch arithmetic
+(`DATE_PART(EPOCH_SECOND, …)`) gives the value, and written back through
+`CAST(? AS TIMESTAMP_NTZ)` it stores the wall clock it was read with. `TIMESTAMP_LTZ` and
+`TIMESTAMP_TZ` carry an offset on the wire and come back as a `Time` at that instant and
+offset.
+
+Each entry in `columns` is a hash of `{ name:, data_type:, scale:, length: }`, carrying
+the engine's own type name. `length` is the width a text or binary column was declared
+with — characters for `VARCHAR(9)`, bytes for `BINARY(5)`, and the maximum (16777216 /
+8388608) for one declared without a width. Every other type reports `nil`: the server
+sends no width for it, and `nil` is that, not a width of `0`.
 
 A result set arrives as one JSON body and is fully materialised — the driver holds
 every row in memory, and the protocol offers no cursor to page through a large
@@ -121,14 +150,33 @@ a bundler setup on Ruby ≥ 3.4 without it in the Gemfile — those cells fall b
 
 ### Several statements at once
 
+A request carries one statement unless the session asks for more, as on the account, so a pack
+sent without asking is refused with `Actual statement count 2 did not match the desired
+statement count 1.` Ask with `ALTER SESSION SET MULTI_STATEMENT_COUNT = n`, or `0` for any
+number.
+
 `execute` returns the first result set. `execute_all` returns every one, in order:
 
 ```ruby
+conn.execute("ALTER SESSION SET MULTI_STATEMENT_COUNT = 0")
 sets = conn.execute_all("SELECT 1 AS a; SELECT 2 AS b;")
 sets.length      # => 2
 sets.last.rows   # => [{ "B" => 2 }]
 ```
 
+A call can declare its own count instead of asking the session, with the
+`multi_statement_count:` keyword on `execute` and `execute_all`:
+
+```ruby
+sets = conn.execute_all("SELECT 1 AS a; SELECT 2 AS b;", [], multi_statement_count: 2)
+```
+
+The count says how many statements that one call carries, `0` for any number. It
+travels with that request and outranks the session's `MULTI_STATEMENT_COUNT` for it,
+but changes no session state — nothing to save and put back, and a connection shared
+between threads is unaffected. Left out, nothing is sent and the session's value
+decides, which is 1 until it is told otherwise.
+
 If any statement in the string fails the whole call raises and no result sets come
 back — not even for the statements before it. The engine discards their effects
 too: an `INSERT` followed by a failing statement leaves nothing behind, whether the
@@ -164,29 +212,61 @@ same goes for `verify_ssl` and `ca_file` on a DSN that is not `https` — they a
 refused however they were spelled, since they would do nothing. The path names one
 database, so `frostlake://host/db/extra` is refused too.
 
-### Idle sessions
+### Session lifetime
+
+A connection is one session on the engine, and the session is where the current
+database and schema, session variables, `ALTER SESSION` settings, temporary tables and
+an open transaction live. The engine ends a session after 30 minutes idle, when it is
+released, or when the server restarts. From engine 0.1.0 on every answer says whether
+the session it ran in is new (`newSession`), and the driver takes the first answer that
+names a session as the sign of which kind of engine it is talking to.
+
+**What is sent.** Every request after the first names the session. To an engine that
+reports `newSession` it also sends `requireSession: true`, so a session the engine no
+longer holds is refused (HTTP 404) instead of being quietly replaced by a fresh one in
+which the statement would run somewhere else. An older engine is sent neither that
+field nor the release below.
+
+**After a lost session.** The refused statement did not run. If the lost session held
+nothing a fresh one lacks, the driver starts a fresh session, puts the DSN's database and
+schema back on it, and sends the statement once more; if that is refused too, it raises.
+If the lost session held an open transaction, or context set up with `USE`,
+`SET`/`UNSET`, `ALTER SESSION`, a temporary object or a `CREATE`/`DROP` of a database or
+schema, running the statement again could put it somewhere its author did not intend, so
+the driver raises `Frostlake::SessionLostError` instead, saying which. Either way the
+connection stays usable: its next statement starts a fresh session on the DSN's database
+and schema.
 
-The engine drops a session after 30 minutes idle and then quietly builds a fresh one
-for the id the driver keeps sending. A connection left sitting therefore loses the
-database and schema it had selected, and **nothing in the reply says so** — the id
-you sent is echoed back either way, and `/api/sessions` reports only a count, so the
-driver cannot ask whether its session survived.
+```ruby
+begin
+  conn.execute("INSERT INTO acc VALUES (2)")
+rescue Frostlake::SessionLostError
+  # The transaction and anything set up on the session are gone; start the unit of
+  # work over. The connection itself is fine.
+end
+```
 
-What it does instead: once a connection has been idle longer than
-`session_idle_limit` (1800 seconds by default, matching the engine), it re-applies
-the database and schema from the DSN before the next statement. It stops doing that
-the moment you run a `USE` of your own, since the DSN no longer describes where you
-are.
+**Close.** `close` sends `DELETE /api/sessions/{id}`, which ends the session and rolls
+back a transaction it left open. It is best effort and bounded (five seconds to connect
+and five to be answered, or the connection's own timeouts when they are shorter), and it
+never raises. Closing again sends nothing. An engine older than 0.1.0 has no such
+endpoint, so it is not asked, and the session lingers until its idle expiry.
+
+**Older engines.** Before 0.1.0 the engine quietly builds a fresh session for the id the
+driver keeps sending, and nothing in the reply says so. Against such an engine a
+connection that has been idle longer than `session_idle_limit` (1800 seconds by default,
+matching the engine) re-applies the database and schema from the DSN before the next
+statement. It stops doing that the moment you run a `USE` of your own, since the DSN no
+longer describes where you are. Everything else a dropped session held — the warehouse,
+the role, session variables, an open transaction — is gone, and no client can restore
+it. An engine that reports `newSession` has no need of the timer, and the driver leaves
+it off there.
 
 ```ruby
 Frostlake.connect(dsn, session_idle_limit: 600)   # re-apply after ten idle minutes
 Frostlake.connect("frostlake://host:18082/DB?session_idle_limit=0")  # never
 ```
 
-Everything else a dropped session held — the warehouse, the role, session variables,
-an open transaction — is gone, and no client can restore it. If a connection may idle
-for long stretches, reconnecting is the dependable answer.
-
 ### Errors
 
 Everything the driver raises is a `Frostlake::Error`, so a single rescue still catches
@@ -195,6 +275,7 @@ the lot. The subclass says which kind it was:
 | Class | Raised when |
 | --- | --- |
 | `Frostlake::ConnectionError` | the server is unreachable, unhealthy, or the request failed |
+| `Frostlake::SessionLostError` | a `ConnectionError`: the engine no longer holds the session, and the transaction or context it held cannot be put back; the statement did not run |
 | `Frostlake::QueryError` | the engine rejected the statement; the message is the engine's |
 | `Frostlake::UsageError` | the driver was misused: bad DSN, closed connection, unbindable value |
 
@@ -218,11 +299,48 @@ rake test        # or: ruby test/test_frostlake.rb
 Without `FROSTLAKE_CLASSPATH` the integration tests skip themselves and only the
 substitution unit tests run.
 
+## Testkit corpus runner
+
+`testkit_runner.rb` replays the engine's testkit corpus — the language-neutral JSON
+suites in `frostlake/engine/src/test/resources/testkit/suites`, format in the `SCHEMA.md`
+beside them — through this driver, and compares every cell as the driver hands it back.
+`FL_CORPUS` names that testkit directory, best as an absolute path. With it set, the test
+suite replays the corpus as one more test, which fails when any case does; without it, or
+with neither `FROSTLAKE_URL` nor `FROSTLAKE_CLASSPATH` to replay against, that test is
+skipped:
+
+```sh
+FL_CORPUS=/path/to/frostlake/engine/src/test/resources/testkit ruby test/test_frostlake.rb
+```
+
+The runner also runs on its own:
+
+```sh
+FL_CORPUS=/path/to/frostlake/engine/src/test/resources/testkit \
+  FROSTLAKE_URL=frostlake://127.0.0.1:18082 ruby testkit_runner.rb
+```
+
+Every case recreates `test_db`, so point it at a scratch server — or leave
+`FROSTLAKE_URL` out and it boots one of its own from `FROSTLAKE_CLASSPATH`, in the test
+suite too. `FROSTLAKE_TESTKIT_FILTER=word1,word2` replays only the suites whose name
+contains one of the words.
+
+Each case runs on a connection of its own, after the reset the corpus prescribes; a case
+skipped for `ruby` or `http` reports SKIP. The report, `results/testkit-ruby.tsv` unless
+`FROSTLAKE_TESTKIT_REPORT` says otherwise, holds one row per case (`suite`, `test`,
+`status`, `failedStep`, `detail`, `ms`). Beside it `missing-apis-ruby.md` lists the checks
+the protocol cannot express — an expected error's code or SQLSTATE — which are recorded
+rather than failed. The run ends with `testkit [ruby]: <P> passed, <F> failed, <S> skipped`
+and exits 1 when any case failed, or when `FL_CORPUS` holds no suites.
+
 ## Protocol
 
-One `POST /api/execute` per statement with `{ sql, sessionId, autoCommit }`; the server
+One `POST /api/execute` per statement with `{ sql, sessionId, autoCommit }` — plus
+`multiStatementCount` when a call declares one, and nothing at all when it does not; the server
 issues the `sessionId` on first contact and the driver echoes it back, so session state
-(current database/schema, transactions) persists across statements. `GET /api/health`
+(current database/schema, transactions) persists across statements. To an engine that
+reports `newSession` the driver adds `requireSession: true`, and `close` sends
+`DELETE /api/sessions/{id}` (see [Session lifetime](#session-lifetime)). `GET /api/health`
 backs `Frostlake.connect`'s reachability check.
 
 ## License

data/lib/frostlake.rb

@@ -32,7 +32,7 @@ rescue LoadError
 end
 
 module Frostlake
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 
   # Every failure the driver raises is a Frostlake::Error, so one rescue still
   # catches the lot; the subclasses only say which kind it was.
@@ -41,6 +41,15 @@ module Frostlake
   # The server could not be reached, or the connection failed mid-statement.
   class ConnectionError < Error; end
 
+  # The engine no longer holds the connection's session: it expired, was
+  # released, or the server restarted. Raised instead of re-running the
+  # statement when the lost session held what a fresh one cannot reproduce —
+  # an open transaction, or context set up with USE, SET, ALTER SESSION or a
+  # temporary object. The statement did not run, and the connection stays
+  # usable: its next statement starts a fresh session on the DSN's database
+  # and schema.
+  class SessionLostError < ConnectionError; end
+
   # The engine rejected a statement. The message is the engine's own.
   class QueryError < Error; end
 
@@ -60,14 +69,34 @@ module Frostlake
   DEFAULT_OPEN_TIMEOUT = 10
   DEFAULT_READ_TIMEOUT = 300
 
-  # The engine reaps a session after 30 minutes idle. Past that we have to
-  # assume ours is gone, because nothing in a response says so.
+  # The engine reaps a session after 30 minutes idle. An engine that predates
+  # newSession says nothing when it does, so past that we have to assume ours
+  # is gone.
   DEFAULT_SESSION_IDLE_LIMIT = 1800
 
+  # How long closing may spend releasing the session, in seconds, to connect
+  # and again to be answered; a connection timeout that is shorter wins.
+  CLOSE_BUDGET = 5
+
   # The engine's binary floating-point types. Every other numeric it reports is
   # fixed-point and keeps its digits.
   APPROXIMATE_TYPES = ["FLOAT", "FLOAT4", "FLOAT8", "DOUBLE", "DOUBLE PRECISION", "REAL"].freeze
 
+  # A character of an unquoted identifier or keyword. $ is one, which is why
+  # A$$B is a name.
+  WORD_CHAR = /[[:alnum:]_$]/.freeze
+
+  # The words that may sit between CREATE, DROP or ALTER and the kind of
+  # object the statement names.
+  OBJECT_MODIFIERS = ["OR", "REPLACE", "TRANSIENT", "TEMPORARY", "TEMP", "VOLATILE", "LOCAL", "GLOBAL",
+                      "SECURE", "IF", "NOT", "EXISTS", "PUBLIC", "PRIVATE", "ICEBERG", "DYNAMIC",
+                      "HYBRID", "EVENT", "RECURSIVE", "MATERIALIZED", "EXTERNAL"].freeze
+
+  # The modifiers that make an object temporary: it lives only as long as the
+  # session that created it.
+  TEMPORARY = ["TEMPORARY", "TEMP", "VOLATILE"].freeze
+  private_constant :WORD_CHAR, :OBJECT_MODIFIERS, :TEMPORARY
+
   # Connects, verifies the server is reachable via GET /api/health, and applies
   # the database and schema from the DSN. Timeouts are in seconds; verify_ssl
   # and ca_file apply to https DSNs. All four may also be given in the DSN
@@ -130,6 +159,11 @@ module Frostlake
   end
 
   class Connection
+    # What round_trip answers when the engine refused the session id as one it
+    # no longer holds. Nothing ran.
+    SESSION_GONE = :session_gone
+    private_constant :SESSION_GONE
+
     def initialize(dsn, open_timeout: nil, read_timeout: nil, verify_ssl: nil, ca_file: nil,
                    session_idle_limit: nil)
       uri = begin
@@ -188,6 +222,15 @@ module Frostlake
       # DSN's defaults are no longer the whole truth about this session.
       @session_touched = false
       @session_id = nil
+      # Whether the engine reports newSession, which arrived together with
+      # requireSession and DELETE /api/sessions: nil until the first answer
+      # that names a session, which settles it either way.
+      @tracks_sessions = nil
+      # What the session holds that a fresh one would not: context a statement
+      # set up (USE, SET, ALTER SESSION, a temporary object, CREATE or DROP of a
+      # database or schema), and an open transaction.
+      @dirty = false
+      @in_transaction = false
       @autocommit = true
       @closed = false
       @pending_use = []
@@ -198,8 +241,8 @@ module Frostlake
         raise UsageError, "the DSN path names one database, got #{uri.path.inspect}"
       end
       schema = query["schema"]
-      @pending_use << "USE DATABASE #{self.class.quote_ident(database)}" unless database.empty?
-      @pending_use << "USE SCHEMA #{self.class.quote_ident(schema)}" if schema
+      @pending_use << "USE DATABASE #{self.class.use_ident(database)}" unless database.empty?
+      @pending_use << "USE SCHEMA #{self.class.use_ident(schema)}" if schema
       # Kept so they can be put back if the session is replaced under us.
       @session_defaults = @pending_use.dup.freeze
     end
@@ -213,9 +256,7 @@ module Frostlake
     # on whatever query happens to run first.
     def use_dsn_defaults
       check_open
-      @lock.synchronize do
-        round_trip(@pending_use.shift) until @pending_use.empty?
-      end
+      @lock.synchronize { apply_pending_use }
       nil
     end
 
@@ -236,31 +277,46 @@ module Frostlake
     # column name and whose row_count is the affected-row count for DML. A
     # multi-statement string answers with its first result set — use
     # execute_all for the rest.
-    def execute(sql, binds = [])
-      execute_all(sql, binds).first
+    def execute(sql, binds = [], multi_statement_count: nil)
+      execute_all(sql, binds, multi_statement_count: multi_statement_count).first
     end
 
     # Executes a statement string and returns every result set it produced, in
     # order. A single statement gives a one-element array.
-    def execute_all(sql, binds = [])
+    #
+    # multi_statement_count says how many statements this call carries, 0 for
+    # any number; the engine refuses a call whose count differs, as the account
+    # does. It rides on this one request and outranks the session's
+    # MULTI_STATEMENT_COUNT for it without changing any session state, so there
+    # is nothing to restore and a connection shared between threads is
+    # unaffected. Left out, nothing is sent and the session's value decides.
+    def execute_all(sql, binds = [], multi_statement_count: nil)
       check_open
+      unless multi_statement_count.nil? ||
+             (multi_statement_count.is_a?(Integer) && !multi_statement_count.negative?)
+        raise UsageError, "multi_statement_count must be a non-negative Integer, " \
+                          "got #{multi_statement_count.inspect}"
+      end
       rendered = binds.empty? ? sql : self.class.substitute(sql, binds)
       # The pending USE statements and the statement itself have to reach the
       # session as one unit: another thread must not slip a query in between,
       # and two threads must not both try to shift the same pending entry.
       @lock.synchronize do
-        restore_session_defaults
-        round_trip(@pending_use.shift) until @pending_use.empty?
-        results = shape_results(round_trip(rendered))
+        results = shape_results(perform(rendered, multi_statement_count))
         @session_touched = true if self.class.selects_session_state?(sql)
         results
       end
     end
 
+    # Opens a transaction: BEGIN, sent with autocommit off. The connection
+    # leaves autocommit only once the engine has opened the transaction, so a
+    # BEGIN that fails — refused, unreadable, never answered, lost with its
+    # session, or never sent because a USE queued ahead of it was refused —
+    # leaves every later statement committing as it runs.
     def begin_transaction
       @lock.synchronize do
+        perform("BEGIN", nil, false)
         @autocommit = false
-        execute("BEGIN")
       end
       nil
     end
@@ -297,9 +353,15 @@ module Frostlake
       raise
     end
 
+    # Closes the connection and releases its session on the engine with one
+    # DELETE /api/sessions/{id}, which also rolls back a transaction left open.
+    # The release is best effort, bounded by CLOSE_BUDGET, and never raises;
+    # an engine that predates it is sent nothing, and keeps the session until
+    # its own idle expiry. Closing again sends nothing.
     def close
       @closed = true
       @lock.synchronize do
+        release_session
         @http.finish if @http.started?
       rescue IOError
         # Already gone; closing is still closing.
@@ -314,14 +376,166 @@ module Frostlake
       raise UsageError, "connection is closed" if @closed
     end
 
-    # The engine reaps a session once it has been idle long enough and then
-    # quietly builds a fresh one for the id we keep sending, losing the database
-    # and schema we selected. Nothing in the reply gives it away — the id we
-    # sent is echoed back either way, and /api/sessions reports only a count —
-    # so past the limit the only safe reading is that the session is new, and
-    # the DSN's defaults go back on. Not once the caller has selected something
-    # themselves: putting our defaults over their choice is its own surprise.
+    # One statement on the session, under the lock: the pending USE statements
+    # first, with the connection's own autocommit, then the statement with
+    # auto_commit, and the session's state tracked from it. Answers the
+    # engine's answer.
+    def perform(sql, multi_statement_count, auto_commit = @autocommit)
+      # Again under the lock: a close that got in first has released the
+      # session, and a statement now would start one nobody releases.
+      check_open
+      restore_session_defaults
+      # Each pending USE is one statement of its own, whatever this call declares.
+      apply_pending_use
+      out = run(sql, multi_statement_count, auto_commit)
+      track_session(sql)
+      out
+    end
+
+    # Runs one statement on the session, or — when the engine no longer holds
+    # it — once more on a fresh one, if nothing the lost session held is lost
+    # with it. Sent again, it carries the autocommit it was first sent with.
+    def run(sql, multi_statement_count, auto_commit = @autocommit)
+      out = round_trip(sql, multi_statement_count, auto_commit)
+      return out unless out == SESSION_GONE
+
+      session_lost
+      apply_pending_use
+      out = round_trip(sql, multi_statement_count, auto_commit)
+      raise SessionLostError, "the engine refused a session it had just started" if out == SESSION_GONE
+
+      out
+    end
+
+    # Sends the pending USE statements, one request each. A session lost or
+    # replaced part-way takes the USEs already sent with it, so the whole of
+    # the DSN's scope goes onto the fresh one: once, since an engine that loses
+    # the session again within the same few requests is keeping none. And a
+    # scope that fails part-way stays pending in full, so no statement runs in
+    # a scope nobody chose.
+    def apply_pending_use
+      restarted = false
+      until @pending_use.empty?
+        queue = @pending_use.dup
+        answer = begin
+          round_trip(queue.first)
+        rescue Error
+          @pending_use.replace(@session_defaults.dup)
+          raise
+        end
+        if answer == SESSION_GONE
+          raise SessionLostError, "the engine refused a session it had just started" if restarted
+
+          restarted = true
+          session_lost
+        elsif @pending_use == queue || restarted
+          @pending_use.replace(queue.drop(1))
+        else
+          # absorb found the session replaced and queued the scope again.
+          restarted = true
+        end
+      end
+    end
+
+    # The engine no longer holds the session: it expired, was released, or the
+    # server restarted, and nothing ran. With a transaction or a moved context
+    # gone with it, re-running would put the statement somewhere its author
+    # did not intend, so that raises; either way the id is dropped and the
+    # DSN's scope queued, so the next statement starts a fresh session there.
+    def session_lost
+      had_transaction = @in_transaction
+      had_context = @dirty
+      forget_session
+      if had_transaction
+        raise SessionLostError,
+              "the engine no longer holds this connection's session (it expired, was released, " \
+              "or the server restarted), so its open transaction is gone; the statement did not run"
+      end
+      return unless had_context
+
+      raise SessionLostError,
+            "the engine no longer holds this connection's session (it expired, was released, " \
+            "or the server restarted), and the context set up on it (USE, SET, ALTER SESSION or " \
+            "a temporary object) went with it, so the statement was not re-run; the next " \
+            "statement starts a fresh session on the connection's scope"
+    end
+
+    # Starts over: no session, nothing held on one, and the DSN's scope queued
+    # for the next statement. A transaction lost with the session takes the
+    # driver's autocommit-off with it.
+    def forget_session
+      @autocommit = true if @in_transaction
+      @session_id = nil
+      @dirty = false
+      @in_transaction = false
+      @pending_use.replace(@session_defaults.dup)
+    end
+
+    # Keeps the connection's picture of its session in step with a statement
+    # that succeeded: whether it left context behind that a fresh session would
+    # not have, and whether a transaction is open.
+    def track_session(sql)
+      self.class.statements(sql).each do |statement|
+        @dirty = true if self.class.touches_session?(statement)
+        case self.class.transaction_effect(statement)
+        when :begins then @in_transaction = true
+        when :ends then @in_transaction = false
+        end
+      end
+    end
+
+    # Learns from an answer which session it ran in, and whether the engine
+    # tracks sessions: an answer naming one carries newSession, or comes from
+    # an engine older than the field, requireSession and DELETE /api/sessions.
+    def absorb(out, sent_id)
+      id = out["sessionId"]
+      return if id.nil?
+
+      if out.key?("newSession")
+        @tracks_sessions = true
+        # The engine ran the statement in a fresh session in place of ours, so
+        # whatever the old one held is gone, and the DSN's scope goes back on
+        # before the next statement.
+        forget_session if out["newSession"] == true && !sent_id.nil?
+      elsif @tracks_sessions.nil?
+        @tracks_sessions = false
+      end
+      @session_id = id
+    end
+
+    # One DELETE /api/sessions/{id}, bounded and never raising. Releasing the
+    # session also rolls back a transaction it left open.
+    def release_session
+      id = @session_id
+      @session_id = nil
+      @in_transaction = false
+      return if id.nil? || !@tracks_sessions
+
+      @http.open_timeout = [@http.open_timeout, CLOSE_BUDGET].min
+      @http.read_timeout = [@http.read_timeout, CLOSE_BUDGET].min
+      @http.write_timeout = [@http.write_timeout, CLOSE_BUDGET].min
+      # Net::HTTP re-sends an idempotent request once after a timeout, which
+      # would double the budget.
+      @http.max_retries = 0
+      path = "/api/sessions/#{URI.encode_www_form_component(id).gsub('+', '%20')}"
+      @http.request(Net::HTTP::Delete.new(path))
+      nil
+    rescue StandardError
+      # Best effort: the engine's idle expiry releases whatever this did not.
+      nil
+    end
+
+    # An engine that predates newSession reaps a session once it has been idle
+    # long enough and then quietly builds a fresh one for the id we keep
+    # sending, losing the database and schema we selected. Nothing in its reply
+    # gives it away — the id we sent is echoed back either way, and
+    # /api/sessions reports only a count — so past the limit the only safe
+    # reading is that the session is new, and the DSN's defaults go back on.
+    # Not once the caller has selected something themselves: putting our
+    # defaults over their choice is its own surprise. A later engine refuses a
+    # session it no longer holds, and run puts the scope back itself.
     def restore_session_defaults
+      return unless @tracks_sessions == false
       return if @session_defaults.empty? || @session_touched
       return if @session_idle_limit.zero? || @last_used_at.nil?
       return if self.class.monotonic_now - @last_used_at < @session_idle_limit
@@ -343,10 +557,24 @@ module Frostlake
       @http.ca_file = authority unless authority.nil?
     end
 
-    def round_trip(sql)
+    # One POST /api/execute: the parsed answer, or SESSION_GONE when the
+    # engine refused the session id as one it no longer holds. auto_commit is
+    # the request's autocommit: the connection's own unless a statement asks
+    # for another, as BEGIN does.
+    def round_trip(sql, multi_statement_count = nil, auto_commit = @autocommit)
       @lock.synchronize do
-        payload = { "sql" => sql, "autoCommit" => @autocommit }
-        payload["sessionId"] = @session_id if @session_id
+        sent_id = @session_id
+        # Resume this session or refuse, rather than have the engine start a
+        # fresh one under the same id where the statement would run without
+        # the context set up earlier. Only to an engine known to take the
+        # field: an older one might refuse a field it does not know.
+        required = !sent_id.nil? && @tracks_sessions == true
+        payload = { "sql" => sql, "autoCommit" => auto_commit }
+        payload["sessionId"] = sent_id if sent_id
+        payload["requireSession"] = true if required
+        # Absent unless this call asked for a count: a request without the field
+        # is the one the server has always been sent, and the session decides.
+        payload["multiStatementCount"] = multi_statement_count unless multi_statement_count.nil?
         request = Net::HTTP::Post.new("/api/execute", "content-type" => "application/json")
         request.body = JSON.generate(payload)
         response = begin
@@ -360,7 +588,12 @@ module Frostlake
         rescue JSON::ParserError
           raise ConnectionError, "HTTP #{response.code} with unreadable body"
         end
-        @session_id = out["sessionId"] if out["sessionId"]
+        # A 404 that names no session is the refusal requireSession asked for.
+        if required && response.code.to_s == "404" && !out["success"] && out["sessionId"].nil?
+          return SESSION_GONE
+        end
+
+        absorb(out, sent_id)
         raise QueryError, out["errorMessage"] || "statement failed" unless out["success"]
 
         @last_used_at = self.class.monotonic_now
@@ -381,7 +614,10 @@ module Frostlake
 
     def shape_result(result_set)
       columns = (result_set["columns"] || []).map do |c|
-        { name: c["name"], data_type: c["dataType"], scale: c["scale"] }
+        # length is a text column's width in characters and a binary column's in
+        # bytes. Every other type sends none, and so does a server that predates
+        # the field: it stays nil rather than becoming a width of 0.
+        { name: c["name"], data_type: c["dataType"], scale: c["scale"], length: c["length"] }
       end
       values = (result_set["rows"] || []).map do |raw|
         cells = []
@@ -410,6 +646,17 @@ module Frostlake
         "\"#{text.gsub('"', '""')}\""
       end
 
+      # A DSN's database or schema, rendered for USE. A plain name means what it
+      # means unquoted in SQL — the upper-case object it folds to — so it is
+      # folded before it is quoted; anything else is quoted exactly as given.
+      # Quoted as given, a lower-case name would ask for a lower-case object,
+      # which USE refuses: it resolves names exactly, as live does.
+      def use_ident(name)
+        text = name.to_s
+        text = text.upcase if text.match?(/\A[A-Za-z_][A-Za-z0-9_$]*\z/)
+        quote_ident(text)
+      end
+
       # Keeps every JSON number exact: the engine serializes fixed-point
       # numerics from BigDecimal, and Float would round the digits away before
       # convert ever sees them.
@@ -425,7 +672,9 @@ module Frostlake
         case (data_type || "").upcase
         when "DATE"
           value.is_a?(String) ? Date.parse(value) : value
-        when "TIMESTAMP", "TIMESTAMP_NTZ", "TIMESTAMP_LTZ", "TIMESTAMP_TZ", "DATETIME"
+        when "TIMESTAMP", "TIMESTAMP_NTZ", "DATETIME"
+          value.is_a?(String) ? wall_clock(value) : value
+        when "TIMESTAMP_LTZ", "TIMESTAMP_TZ"
           value.is_a?(String) ? Time.parse(value) : value
         when "BINARY", "VARBINARY"
           value.is_a?(String) ? decode_hex(value) : value
@@ -434,6 +683,18 @@ module Frostlake
         end
       end
 
+      # A TIMESTAMP_NTZ is a wall clock with no zone, and its text carries none.
+      # Read in the process's local zone, a wall clock that zone skips would be
+      # moved — 01:30 on the night London springs forward would come back as
+      # 02:30 — and what a caller read would depend on the machine. UTC skips
+      # nothing, so the text is read there: every field comes back as written,
+      # on every host, and the instant is the one the engine's epoch arithmetic
+      # gives the value. Text with an offset of its own keeps it, since the
+      # parser takes the first zone it meets.
+      def wall_clock(text)
+        Time.parse("#{text} UTC")
+      end
+
       # The engine renders binary as hex. Anything else is not ours to
       # reinterpret: pack("H*") turns "ZZ" into a byte and pads odd-length input
       # rather than admitting it was handed something else.
@@ -511,6 +772,93 @@ module Frostlake
         /(\A|[;\n])\s*USE\s/i.match?(sql)
       end
 
+      # -- session tracking --------------------------------------------------
+
+      # The request split on its top-level semicolons, blank pieces dropped. A
+      # semicolon inside a literal, a quoted identifier, a $$ body or a comment
+      # does not split: the same constructs substitute steps over. A scripting
+      # block is split along with everything else, which only makes the checks
+      # below more willing to flag a request, the safe direction to be wrong in.
+      def statements(sql)
+        pieces = []
+        start = 0
+        i = 0
+        while i < sql.length
+          past = skip_non_code(sql, i)
+          if past
+            i = past
+          elsif sql[i] == ";"
+            pieces << sql[start...i]
+            start = i + 1
+            i += 1
+          else
+            i += 1
+          end
+        end
+        pieces << sql[start..]
+        pieces.reject { |piece| piece.strip.empty? }
+      end
+
+      # Up to limit leading words of a statement, upper-cased, skipping
+      # whitespace and comments and stopping at the first thing that is not a
+      # word.
+      def leading_words(statement, limit)
+        words = []
+        i = 0
+        while words.length < limit && i < statement.length
+          ch = statement[i]
+          nxt = statement[i + 1]
+          if ch.match?(/\s/)
+            i += 1
+          elsif (ch == "-" && nxt == "-") || (ch == "/" && nxt == "/")
+            i = skip_line(statement, i)
+          elsif ch == "/" && nxt == "*"
+            stop = statement.index("*/", i + 2)
+            i = stop.nil? ? statement.length : stop + 2
+          elsif ch.match?(WORD_CHAR)
+            start = i
+            i += 1 while i < statement.length && statement[i].match?(WORD_CHAR)
+            words << statement[start...i].upcase
+          else
+            break
+          end
+        end
+        words
+      end
+
+      # Whether a statement leaves behind state a fresh session would not have:
+      # a moved scope (USE, or CREATE or DROP of a DATABASE or SCHEMA), a
+      # session variable or setting (SET, UNSET, ALTER SESSION), or a temporary
+      # object. CREATE TABLE and its kind leave the session as it was.
+      def touches_session?(statement)
+        verb, *rest = leading_words(statement, 16)
+        case verb
+        when "USE", "SET", "UNSET"
+          true
+        when "ALTER"
+          rest.drop_while { |word| OBJECT_MODIFIERS.include?(word) }.first == "SESSION"
+        when "CREATE", "DROP"
+          modifiers = rest.take_while { |word| OBJECT_MODIFIERS.include?(word) }
+          return true if %w[DATABASE SCHEMA].include?(rest[modifiers.length])
+
+          verb == "CREATE" && modifiers.any? { |word| TEMPORARY.include?(word) }
+        else
+          false
+        end
+      end
+
+      # :begins, :ends or nil — what a statement does to the session's
+      # transaction. BEGIN on its own (or with TRANSACTION, WORK or NAME) opens
+      # one; BEGIN followed by a statement opens a scripting block instead.
+      def transaction_effect(statement)
+        words = leading_words(statement, 2)
+        return :ends if %w[COMMIT ROLLBACK].include?(words.first)
+        return :begins if words == ["START", "TRANSACTION"]
+        return nil unless words.first == "BEGIN"
+
+        words.length == 1 || %w[TRANSACTION WORK NAME].include?(words[1]) ? :begins : nil
+      end
+
       # An explicit argument wins over the DSN, which wins over the default.
       def boolean_for(name, argument, from_dsn, fallback)
         given = argument.nil? ? from_dsn : argument
@@ -620,6 +968,25 @@ module Frostlake
 
       private
 
+      # Index just past the literal, quoted identifier, $$ body or comment
+      # starting at i, or nil when i is code; the rules substitute follows.
+      def skip_non_code(sql, i)
+        ch = sql[i]
+        nxt = sql[i + 1]
+        if ch == "'"
+          skip_string(sql, i)
+        elsif ch == '"'
+          skip_quoted(sql, i)
+        elsif (ch == "-" && nxt == "-") || (ch == "/" && nxt == "/")
+          skip_line(sql, i)
+        elsif ch == "/" && nxt == "*"
+          stop = sql.index("*/", i + 2)
+          stop.nil? ? sql.length : stop + 2
+        elsif ch == "$" && nxt == "$"
+          skip_dollar_quoted(sql, i)
+        end
+      end
+
       def skip_string(sql, i)
         j = i + 1
         while j < sql.length

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: frostlake
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - MLorek