sequel-notion 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8a7f11227df00a9112d8b47f760985a0290807b89d7224d9e9fce4c93df7d7c9
-  data.tar.gz: 30ce794fb6a0d9fe33ac45790319539f8d0d63b2d0e8df5cce10367a566ad39a
+  metadata.gz: c1fb73a833c0fc96c70528e504873f4765efec4ccddc4737f6eedc8df3723e60
+  data.tar.gz: 0e2feb6f99c68194d3e7964d3ae16c36c63887ca2064a299b2ba46469cf5da8f
 SHA512:
-  metadata.gz: ccc702b83af21366469e911884ba787dc56d6bc94c6e186a7cf495f695c773f9830d4f98a53ed061f6e2f95ef2078be9f4a6554768fec336cece00179d3d363a
-  data.tar.gz: 27003c0414d36f7e0088eddafb1e4dfc1a8dc568bf121bc259685097565e66fb9029def065ac11afeaeca3ccea120b824e0467538191d697a24e36c4d075bb95
+  metadata.gz: d280403f620b4c64ba88a46e2b18963c175464c0545d27026a9dfe05a20b4dd527bf8368d25386a5b4cb409fa4bbaffbc8cb0a6fb3e14c9101a1d33719c0c8cf
+  data.tar.gz: f2113de1c055a39cc0f77755447c520a36cb615e9e5bc9a1652a9d6c0394fd7df5e3da99d3971d4a85a094385626b0fe893a6429212e7dee54ced3b785d0e3de

data/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 0.2.1 — 2026-10-22
+
+### Added
+
+- `DB.upload_file(data, filename:, content_type:)` uploads a file and
+  returns its id; `File.uploaded(id, name:)` writes it to a files
+  property (checked live).
+- `DB.request` is documented as the way to reach endpoints the adapter
+  does not map (blocks, comments), with its retries and errors.
+
+### Changed
+
+- Failed connections (refused, reset, unresolved, connect timeout) are
+  retried like timeouts; page creation and a file upload's send only
+  when the connection never opened.
+
 ## 0.2.0 — 2026-10-10
 
 ### Added

data/README.md

@@ -22,6 +22,7 @@ else raises a `Sequel::Error` rather than be dropped.
 - [Writing](#writing)
 - [Models](#models)
 - [Schema](#schema)
+- [Raw requests](#raw-requests)
 - [Errors, retries and logging](#errors-retries-and-logging)
 - [Known shortfalls](#known-shortfalls)
 - [Checked against the live API](#checked-against-the-live-api)
@@ -339,6 +340,27 @@ writable columns in schema order.
 `update` and `delete` first collect the matching page ids, then send one
 request per page.
 
+### Uploading files
+
+`DB.upload_file` uploads bytes to Notion and returns the file upload's
+id; `File.uploaded` turns that id into a value for a files property:
+
+```ruby
+id = DB.upload_file(File.binread("q.pdf"), filename: "q.pdf",
+                    content_type: "application/pdf")
+DB[:leases].where(id: page).update(Docs: Sequel::Notion::File.uploaded(id))
+```
+
+It takes the file's bytes, as a `String` or anything with `#read` (a
+`String` is the content, never a path: `upload_file("q.pdf", …)`
+uploads the five bytes `q.pdf`), and sends them in one part
+(Notion's `single_part` mode; `multi_part` is not supported). The upload
+reads back as a Notion-hosted file. Notion keeps the upload's file name
+in ASCII only (`reçu été "q".pdf` became `reu_t_q.pdf`);
+`File.uploaded(id, name: "reçu.pdf")` names the file as given. An upload
+that is never attached carries an `expiry_time` an hour away. The id can also go into a file block
+through [raw requests](#raw-requests).
+
 
 ## Models
 
@@ -385,15 +407,47 @@ in Notion; every name of that data source (an alias, its id) sees the
 change.
 
 
+## Raw requests
+
+`DB.request(verb, path, data = nil)` sends one request to any JSON
+endpoint the adapter does not cover (blocks, comments, users), through
+the same connection, retries, errors and logging as every query:
+
+```ruby
+DB.request(:get, "blocks/#{page}/children", page_size: 100)
+DB.request(:patch, "blocks/#{page}/children", children: [block])
+DB.request(:delete, "blocks/#{block_id}")
+```
+
+- `path` is relative to `https://api.notion.com/v1`; the version sent is
+  `Sequel::Notion::API_VERSION`.
+- `data` is the query string for `:get` and `:delete`, and the JSON body
+  for `:post` and `:patch`.
+- It returns the parsed JSON body, a `Hash`; a response that is not
+  typed JSON comes back as its `String`.
+- Errors and retries are those of the next section. Every POST but page
+  creation and a file upload's send is retried, so a POST sent this way
+  must be safe to repeat; so must a PATCH, retried after a timeout.
+- It spends no `max_requests` budget, unless called inside a query that
+  set one.
+- It logs `VERB path` with `data`, so `data` should hold neither secrets
+  nor bulk bytes.
+
+It sends JSON only: a file goes through [`upload_file`](#uploading-files).
+
+
 ## Errors, retries and logging
 
-- A failed request, or a response that is not valid JSON, raises
+- A failed request, or a response typed JSON that does not parse, raises
   `Sequel::DatabaseError`, with Notion's error code and message in the
   text. A 404 raises its subclass `Sequel::Notion::NotFoundError`
   (`where(id:)` turns it into no row).
 - Responses 429, 502, 503 and 504 are retried up to four times, honouring
-  `Retry-After`. Page creation is retried only after a 429, so a timeout
-  cannot create a page twice.
+  `Retry-After`. So are timeouts and failed connections (refused, reset,
+  unresolved). Page creation and a file upload's send are retried only
+  after a 429 or when the connection never opened, so a timeout or a
+  reset cannot create a page twice, nor send a file Notion already holds
+  (it refuses a second send, checked live).
 - Requests go to the database's loggers (`DB.loggers << Logger.new($stdout)`)
   as `POST data_sources/…/query` with their body.
 
@@ -443,7 +497,11 @@ believes Notion accepts. These were also checked against
 - **Rollups:** a `show_original` rollup through 30 relations read in
   full, all 30 values, in a query and by id.
 - **Files:** a Notion-hosted file read from a page and written back
-  unchanged, signed URL included, kept by Notion.
+  unchanged, signed URL included, kept by Notion. `upload_file` with
+  binary bytes and a non-ASCII name, attached through `File.uploaded`
+  with and without `name:`, read back and downloaded byte for byte; a
+  second send to the same upload refused; an unsent upload's
+  `expiry_time` an hour after its creation (2026-10-22).
 - **Pages:** lookups by id with or without dashes; creation with the
   `data_source_id` parent; writing, reading back and clearing every
   writable type; trashing and restoring; a relation of 26 pages read in

data/lib/sequel/adapters/notion.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "json"
+require "net/http"
 
 require "faraday"
 require "faraday/retry"
@@ -8,6 +9,7 @@ require "sequel"
 
 require "sequel/notion/dataset"
 require "sequel/notion/errors"
+require "sequel/notion/file_uploads"
 require "sequel/notion/model_support"
 require "sequel/notion/page_api"
 require "sequel/notion/registry"
@@ -26,15 +28,40 @@ module Sequel
         # faraday-retry honours.
         RETRY_STATUSES = [429, 502, 503, 504].freeze
 
-        # Creating a page is the one call a retry could duplicate: retry it
-        # only when Notion refused it outright (rate limited).
-        RETRY_IF = lambda do |env, _exception|
-            !(env.method == :post && env.url.path.end_with?("/pages")) ||
-                env.status == 429
+        # Exceptions worth another attempt: faraday-retry's timeouts, plus
+        # a failed connection (refused, reset, unresolved), which is not
+        # among them.
+        RETRY_EXCEPTIONS = [*Faraday::Retry::Middleware::DEFAULT_EXCEPTIONS,
+                            Faraday::ConnectionFailed].freeze
+
+        # Causes of a ConnectionFailed raised before the request left: no
+        # connection was ever opened, so nothing reached Notion.
+        UNSENT_ERRORS = [Errno::ECONNREFUSED, Errno::EHOSTUNREACH,
+                         Errno::ENETUNREACH, SocketError,
+                         Net::OpenTimeout].freeze
+
+        # The POSTs a repeat could duplicate or break: creating a page, and
+        # sending a file, which Notion refuses once the upload holds one.
+        UNREPEATABLE = %r{/(?:pages|file_uploads/[^/]+/send)\z}
+
+        # Retry an unrepeatable POST only when Notion refused it outright
+        # (rate limited) or when it was never sent.
+        RETRY_IF = lambda do |env, exception|
+            !(env.method == :post && env.url.path.match?(UNREPEATABLE)) ||
+                env.status == 429 || UNSENT.call(exception)
+        end
+
+        # Did the exception prove nothing was sent? An adapter may raise a
+        # bare timeout, which wraps nothing.
+        UNSENT = lambda do |exception|
+            cause = exception.is_a?(Faraday::Error) &&
+                    exception.wrapped_exception
+            UNSENT_ERRORS.any? { cause.is_a?(it) }
         end
 
         class Database < Sequel::Database
             include PageApi
+            include FileUploads
             include Registry
             include RequestBudget
             include SchemaLookup
@@ -88,16 +115,9 @@ module Sequel
             # HTTP failure re-raised as a Sequel::DatabaseError. Only its
             # 200 spends the query's budget, after the retries.
             def request(verb, path, body = nil)
-                within_budget do
-                    synchronize do |conn|
-                        log_connection_yield("#{verb.upcase} #{path}", conn,
-                                             body && [body]) do
-                            conn.public_send(verb, path, body).body
-                        end
-                    end
+                notion_call(verb, path, body && [body]) do |conn|
+                    conn.public_send(verb, path, body).body
                 end
-            rescue Faraday::Error => e
-                raise database_error(e)
             end
 
             # Where Sequel sends SQL to run: run, <<, truncate, the
@@ -108,6 +128,19 @@ module Sequel
 
             private
 
+            # Run the block with a pooled connection, under the budget,
+            # logged as "VERB path" with +logged+
+            def notion_call(verb, path, logged)
+                within_budget do
+                    synchronize do |conn|
+                        log_connection_yield("#{verb.upcase} #{path}", conn,
+                                             logged) { yield conn }
+                    end
+                end
+            rescue Faraday::Error => e
+                raise database_error(e)
+            end
+
             # A number column is :float, and Sequel's model typecast would
             # turn 5 into 5.0; Notion keeps, and reads back, an Integer
             def typecast_value_float(value)
@@ -138,6 +171,7 @@ module Sequel
                   interval: 0.5,
                   backoff_factor: 2,
                   methods: %i[get patch delete],
+                  exceptions: RETRY_EXCEPTIONS,
                   retry_if: RETRY_IF,
                   retry_statuses: RETRY_STATUSES }
             end

data/lib/sequel/notion/file_uploads.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+require "securerandom"
+
+module Sequel
+    module Notion
+        # Notion's single-part file upload: create the upload, send its
+        # bytes as multipart/form-data, and hand back the id that a files
+        # property (File.uploaded) or a file block attaches.
+        module FileUploads
+            # Upload +data+, the file's bytes (a String, never a path, or
+            # anything with #read), as one part, and return the file
+            # upload's id. Notion keeps an upload for an hour unless it is
+            # attached.
+            #
+            #   id = DB.upload_file(File.binread("q.pdf"),
+            #                       filename: "q.pdf",
+            #                       content_type: "application/pdf")
+            #   DB[:leases].where(id: page).update(Docs: File.uploaded(id))
+            #
+            def upload_file(data, filename:, content_type:)
+                bytes  = data.respond_to?(:read) ? data.read : data.to_str
+                upload = request(:post, "file_uploads",
+                                 mode: "single_part", filename: filename,
+                                 content_type: content_type)
+                send_file_upload(upload.fetch("id"), bytes,
+                                 filename, content_type)
+                upload["id"]
+            end
+
+            private
+
+            # The bytes go out as a String body under their own content
+            # type, which the :json middleware leaves alone; the log names
+            # the file rather than inspect it.
+            def send_file_upload(id, bytes, filename, content_type)
+                path       = "file_uploads/#{id}/send"
+                body, type = multipart_body(bytes, filename, content_type)
+                logged     = "#{filename} (#{bytes.bytesize} bytes)"
+                notion_call(:post, path, [logged]) do |conn|
+                    conn.post(path, body, "Content-Type" => type).body
+                end
+            end
+
+            # One "file" part, as a browser encodes a form: a quote, CR
+            # or LF in the file name is percent-encoded.
+            def multipart_body(bytes, filename, content_type)
+                boundary = "sequel-notion-#{SecureRandom.hex(16)}"
+                name     = filename.gsub(/["\r\n]/) { format("%%%02X", it.ord) }
+                head     = "--#{boundary}\r\n" \
+                           "Content-Disposition: form-data; name=\"file\"; " \
+                           "filename=\"#{name}\"\r\n" \
+                           "Content-Type: #{content_type}\r\n\r\n"
+                body     = head.b + bytes.b + "\r\n--#{boundary}--\r\n".b
+                [body, "multipart/form-data; boundary=#{boundary}"]
+            end
+        end
+    end
+end

data/lib/sequel/notion/notion_file.rb

@@ -151,6 +151,18 @@ module Sequel
                 new(url: url, type: :external, name: name)
             end
 
+            # A file uploaded with Database#upload_file, to write to a
+            # files property; it reads back as a Notion-hosted file
+            #
+            #   File.uploaded(DB.upload_file(io, filename: "q.pdf", …))
+            #
+            def self.uploaded(id, name: nil)
+                raw = { "type" => "file_upload",
+                        "file_upload" => { "id" => id } }
+                raw["name"] = name if name
+                new(url: nil, type: :file_upload, name: name, raw: raw)
+            end
+
             # Wrap multiple URLs into File instances
             #
             #   File.from_urls("https://a.com/1.pdf", "https://b.com/2.pdf")

data/lib/sequel/notion/version.rb

@@ -2,6 +2,6 @@
 
 module Sequel
     module Notion
-        VERSION = "0.2.0"
+        VERSION = "0.2.1"
     end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sequel-notion
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Stéphane D'Alu
@@ -77,6 +77,7 @@ files:
 - lib/sequel/notion/dataset_truncation.rb
 - lib/sequel/notion/discovery.rb
 - lib/sequel/notion/errors.rb
+- lib/sequel/notion/file_uploads.rb
 - lib/sequel/notion/filter_comparison.rb
 - lib/sequel/notion/filter_compiler.rb
 - lib/sequel/notion/filter_constants.rb