sequel-notion 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +207 -0
- data/LICENSE.txt +21 -0
- data/README.md +470 -0
- data/lib/sequel/adapters/notion.rb +159 -0
- data/lib/sequel/notion/dataset.rb +167 -0
- data/lib/sequel/notion/dataset_aggregates.rb +82 -0
- data/lib/sequel/notion/dataset_compounds.rb +89 -0
- data/lib/sequel/notion/dataset_computed.rb +124 -0
- data/lib/sequel/notion/dataset_grouping.rb +145 -0
- data/lib/sequel/notion/dataset_having.rb +99 -0
- data/lib/sequel/notion/dataset_joins.rb +124 -0
- data/lib/sequel/notion/dataset_pages.rb +145 -0
- data/lib/sequel/notion/dataset_selection.rb +51 -0
- data/lib/sequel/notion/dataset_truncation.rb +39 -0
- data/lib/sequel/notion/discovery.rb +76 -0
- data/lib/sequel/notion/errors.rb +9 -0
- data/lib/sequel/notion/filter_comparison.rb +123 -0
- data/lib/sequel/notion/filter_compiler.rb +121 -0
- data/lib/sequel/notion/filter_constants.rb +37 -0
- data/lib/sequel/notion/filter_helpers.rb +162 -0
- data/lib/sequel/notion/filter_like.rb +143 -0
- data/lib/sequel/notion/filter_like_tokenizer.rb +42 -0
- data/lib/sequel/notion/filter_negation.rb +76 -0
- data/lib/sequel/notion/filter_nested.rb +96 -0
- data/lib/sequel/notion/filter_nulls.rb +53 -0
- data/lib/sequel/notion/filter_predicates.rb +160 -0
- data/lib/sequel/notion/filter_shape.rb +75 -0
- data/lib/sequel/notion/filter_tables.rb +111 -0
- data/lib/sequel/notion/group_accumulator.rb +55 -0
- data/lib/sequel/notion/join_output.rb +58 -0
- data/lib/sequel/notion/join_where.rb +111 -0
- data/lib/sequel/notion/model_support.rb +52 -0
- data/lib/sequel/notion/notion_file.rb +228 -0
- data/lib/sequel/notion/page_api.rb +49 -0
- data/lib/sequel/notion/registry.rb +150 -0
- data/lib/sequel/notion/request_budget.rb +64 -0
- data/lib/sequel/notion/schema.rb +74 -0
- data/lib/sequel/notion/schema_lookup.rb +82 -0
- data/lib/sequel/notion/sort_compiler.rb +63 -0
- data/lib/sequel/notion/type_map.rb +125 -0
- data/lib/sequel/notion/type_map_builders.rb +125 -0
- data/lib/sequel/notion/type_map_dates.rb +67 -0
- data/lib/sequel/notion/type_map_extractors.rb +89 -0
- data/lib/sequel/notion/version.rb +7 -0
- data/lib/sequel/notion.rb +5 -0
- metadata +132 -0
data/README.md
ADDED
|
@@ -0,0 +1,470 @@
|
|
|
1
|
+
# sequel-notion
|
|
2
|
+
|
|
3
|
+
A [Sequel](https://sequel.jeremyevans.net/) adapter for Notion. Each Notion
|
|
4
|
+
**data source** (the tables inside a Notion database, API version
|
|
5
|
+
`2026-03-11`) is a Sequel table: you read it with `where`, `order`,
|
|
6
|
+
`limit` and `select`, write it with `insert`, `update` and `delete`, and
|
|
7
|
+
can put a `Sequel::Model` on top of it.
|
|
8
|
+
|
|
9
|
+
Every Sequel call becomes one or more Notion API requests; nothing is
|
|
10
|
+
translated to SQL. What Notion can answer is sent to Notion. What it
|
|
11
|
+
cannot — joins, groups, aggregates, `distinct`, unions — is computed in
|
|
12
|
+
Ruby, only on a dataset that asks for it with `client_side`. Anything
|
|
13
|
+
else raises a `Sequel::Error` rather than be dropped.
|
|
14
|
+
|
|
15
|
+
- [Requirements](#requirements)
|
|
16
|
+
- [Installation](#installation)
|
|
17
|
+
- [Connecting](#connecting)
|
|
18
|
+
- [Naming tables](#naming-tables)
|
|
19
|
+
- [Reading](#reading)
|
|
20
|
+
- [Computed in Ruby](#computed-in-ruby)
|
|
21
|
+
- [Values](#values)
|
|
22
|
+
- [Writing](#writing)
|
|
23
|
+
- [Models](#models)
|
|
24
|
+
- [Schema](#schema)
|
|
25
|
+
- [Errors, retries and logging](#errors-retries-and-logging)
|
|
26
|
+
- [Known shortfalls](#known-shortfalls)
|
|
27
|
+
- [Checked against the live API](#checked-against-the-live-api)
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
## Requirements
|
|
31
|
+
|
|
32
|
+
- Ruby **>= 3.4**
|
|
33
|
+
- A Notion integration token, with the integration shared to the pages
|
|
34
|
+
and databases it should see
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
## Installation
|
|
38
|
+
|
|
39
|
+
```sh
|
|
40
|
+
gem install sequel-notion
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Or from a checkout:
|
|
44
|
+
|
|
45
|
+
```sh
|
|
46
|
+
bundle install
|
|
47
|
+
bundle exec rake test
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
## Connecting
|
|
52
|
+
|
|
53
|
+
```ruby
|
|
54
|
+
require "sequel"
|
|
55
|
+
|
|
56
|
+
DB = Sequel.connect(adapter: :notion, token: ENV["NOTION_TOKEN"])
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
| Option | Meaning |
|
|
60
|
+
| ----------------- | --------------------------------------------------- |
|
|
61
|
+
| `token` | The integration token (required) |
|
|
62
|
+
| `auto_register` | Discover every data source on the first lookup |
|
|
63
|
+
| `faraday_adapter` | Faraday adapter (default `Faraday.default_adapter`) |
|
|
64
|
+
|
|
65
|
+
The test suite passes `faraday_adapter: [:test, stubs]` to answer for
|
|
66
|
+
Notion.
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
## Naming tables
|
|
70
|
+
|
|
71
|
+
A table name is resolved to a data source id in this order:
|
|
72
|
+
|
|
73
|
+
1. a name registered with `register_data_source` or
|
|
74
|
+
`register_all_data_sources`;
|
|
75
|
+
2. a table name that is itself a data source id (32 hex digits, dashes
|
|
76
|
+
optional, either case);
|
|
77
|
+
3. with `auto_register: true`, every data source the token can see,
|
|
78
|
+
discovered once;
|
|
79
|
+
4. a search for a data source whose title normalises to the table
|
|
80
|
+
name. The result is remembered; two matching data sources raise.
|
|
81
|
+
|
|
82
|
+
Titles normalise to lower snake case. Accents are dropped from Latin
|
|
83
|
+
letters only, and other scripts are kept as they are: `"My Tasks"` →
|
|
84
|
+
`:my_tasks`, `"Électricité"` → `:electricite`, `"タスク"` → `:タスク`. A
|
|
85
|
+
title with no letter or digit (`"🚀"`) is registered under its id.
|
|
86
|
+
A data source in the trash is never named by discovery, a search or
|
|
87
|
+
`register_all_data_sources`, though `data_sources` still lists it with
|
|
88
|
+
`in_trash: true`.
|
|
89
|
+
|
|
90
|
+
```ruby
|
|
91
|
+
DB.register_data_source(:tasks, "<data source id>")
|
|
92
|
+
DB.register_all_data_sources(database: "<database id>")
|
|
93
|
+
DB.register_all_data_sources { |title, id| "notion_#{title}" } # by search
|
|
94
|
+
|
|
95
|
+
DB.tables # => [:tasks, ...]
|
|
96
|
+
DB.data_sources(query: "Bills") # => [{id:, name:, parent_database_id:, ...}]
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
A name already bound to a different data source raises an error rather than
|
|
100
|
+
being rebound, and `register_all_data_sources` then registers none of the
|
|
101
|
+
names it found. Discovery is more lenient: a name two data sources share
|
|
102
|
+
is left out of `tables` and raises when it is looked up, until
|
|
103
|
+
`register_data_source` picks one; the other names register as usual, and
|
|
104
|
+
discovery keeps a name already registered.
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
## Reading
|
|
108
|
+
|
|
109
|
+
```ruby
|
|
110
|
+
DB[:tasks].where(Status: "In Progress", Done: false)
|
|
111
|
+
.order(Sequel.desc(:Due))
|
|
112
|
+
.limit(25)
|
|
113
|
+
.all
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Each row has `:id` (the page id), `:in_trash`, and one key per property,
|
|
117
|
+
named as in Notion (`:"Due Date"` for a property with a space). A property
|
|
118
|
+
named `id` or `in_trash` would hide the page's own column, so a data
|
|
119
|
+
source that has one raises; rename the property in Notion. What each
|
|
120
|
+
type reads back as is under [Values](#values).
|
|
121
|
+
|
|
122
|
+
### Filters
|
|
123
|
+
|
|
124
|
+
| Sequel | Notion filter |
|
|
125
|
+
| ------------------------------ | -------------------------------------- |
|
|
126
|
+
| `where(P: v)`, `exclude(P: v)` | `equals`, `does_not_equal` |
|
|
127
|
+
| `where(P: nil)` | `is_empty`; excluded: `is_not_empty` |
|
|
128
|
+
| `where(Done: true)` | checkbox `equals`, or a formula's |
|
|
129
|
+
| `where(:Done)` | the same |
|
|
130
|
+
| `where(P: [a, b])` | `or` of `equals`; `nil` is empty |
|
|
131
|
+
| `where(P: [])` | no page: no request is sent |
|
|
132
|
+
| `exclude(P: [])` | every page: no filter is sent |
|
|
133
|
+
| `<`, `<=`, `>`, `>=` | numbers; dates: `before`, `after`, … |
|
|
134
|
+
| `Sequel.like(:P, "%x%")` | `contains` |
|
|
135
|
+
| `"x%"`, `"%x"`, `"x"` | `starts_with`, `ends_with`, `equals` |
|
|
136
|
+
| `Sequel.like(:P, "%")` | `is_not_empty`; `NOT LIKE`: `is_empty` |
|
|
137
|
+
| multi-select, people, relation | `=` as `contains` |
|
|
138
|
+
| formula | nested by the value's class |
|
|
139
|
+
| rollup giving one value | nested under `number` or `date` |
|
|
140
|
+
| unique ID | its number: `62` or `"TK-62"` |
|
|
141
|
+
| `&`, `\|`, `~` | `and`, `or`, and the inverse operator |
|
|
142
|
+
|
|
143
|
+
A rollup filters when its function gives one value (`sum`, `count`,
|
|
144
|
+
`latest_date`, …). `nil` filters work on formulas and rollups too.
|
|
145
|
+
|
|
146
|
+
Negations follow SQL, where `!=` never matches `NULL`: Notion's
|
|
147
|
+
`does_not_equal` and `does_not_contain` match an empty property, so
|
|
148
|
+
`exclude(Status: "Done")`, `NOT LIKE` and `NOT IN` add `is_not_empty`
|
|
149
|
+
beside them, whatever the property's type (a checkbox or a unique ID is
|
|
150
|
+
never empty and needs none, so `nil` on one raises, alone or in a list).
|
|
151
|
+
Notion nests
|
|
152
|
+
`and`/`or` two levels deep at most: an `and` inside an `and` is merged
|
|
153
|
+
into it, a level too many is distributed (`(a & b) | c` becomes
|
|
154
|
+
`(a | c) & (b | c)`, up to 32 clauses), and a filter still deeper raises.
|
|
155
|
+
|
|
156
|
+
### Ordering
|
|
157
|
+
|
|
158
|
+
`order` maps to Notion sorts. Notion puts empty values last in both
|
|
159
|
+
directions, so `nulls: :first` raises and `nulls: :last` changes nothing.
|
|
160
|
+
Ordering by `:id` or `:in_trash` raises: they are the page's own columns,
|
|
161
|
+
not properties Notion can sort by. A checkbox sorts `false` first (checked
|
|
162
|
+
live), and so do rows computed in Ruby. Pages looked up by id are sorted
|
|
163
|
+
in Ruby, as a query's would be.
|
|
164
|
+
|
|
165
|
+
### Pages by id, and selection
|
|
166
|
+
|
|
167
|
+
`where(id: "…")` or `where(id: [...])` fetches those pages directly,
|
|
168
|
+
including pages in the trash (`:in_trash` says so). An id may be written
|
|
169
|
+
with or without dashes, in either case, and a repeated id gives one row.
|
|
170
|
+
Several id conditions intersect. A missing page, or one from another
|
|
171
|
+
data source, is no row. An `id` condition combined with any other
|
|
172
|
+
condition raises `Sequel::Error`.
|
|
173
|
+
|
|
174
|
+
`select(:Name, Sequel.as(:Due, :due))` keeps only those keys, renamed by
|
|
175
|
+
the alias. Only plain, existing columns can be selected.
|
|
176
|
+
|
|
177
|
+
### Paging
|
|
178
|
+
|
|
179
|
+
Requests are paginated automatically, 100 rows each. `offset` is
|
|
180
|
+
applied to the rows read, so the rows it skips are still fetched, and
|
|
181
|
+
`count` pages through the results; neither needs `client_side`.
|
|
182
|
+
`paged_each` follows Notion's cursor, as Sequel's cursor adapters do: it
|
|
183
|
+
needs no order, sends one request per `rows_per_fetch` rows (at most
|
|
184
|
+
100), and ignores `:strategy`.
|
|
185
|
+
|
|
186
|
+
|
|
187
|
+
## Computed in Ruby
|
|
188
|
+
|
|
189
|
+
Notion computes no aggregate, `distinct`, group, join or combination of
|
|
190
|
+
queries. The adapter works them out in Ruby over every row the query
|
|
191
|
+
returns, at 100 rows per request and about 3 requests a second, so it
|
|
192
|
+
does so only on a dataset that asks for it; otherwise such a query
|
|
193
|
+
raises before sending anything:
|
|
194
|
+
|
|
195
|
+
```ruby
|
|
196
|
+
DB[:tasks].sum(:Hours) # raises: needs client_side
|
|
197
|
+
DB[:tasks].client_side.sum(:Hours) # reads every task
|
|
198
|
+
DB[:tasks].client_side(max_requests: 20) # and at most 20 requests
|
|
199
|
+
.join(:projects, id: :Project).all
|
|
200
|
+
Task.client_side.group_and_count(:Status).all
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
```text
|
|
204
|
+
query ──▸ needs Ruby? ── no ──▸ Notion: where → filter, order →
|
|
205
|
+
(join, group, sorts; offset and limit on the
|
|
206
|
+
distinct, sum, …) pages read
|
|
207
|
+
│ yes
|
|
208
|
+
▾
|
|
209
|
+
client_side? ── no ──▸ Sequel::Error, nothing sent
|
|
210
|
+
│ yes
|
|
211
|
+
▾
|
|
212
|
+
Notion: one query per table, with the where
|
|
213
|
+
│ conditions that test it; max_requests caps
|
|
214
|
+
│ the requests of all of them
|
|
215
|
+
▾
|
|
216
|
+
Ruby: match, group, aggregate or combine the rows,
|
|
217
|
+
then order, offset and limit
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
`max_requests` counts every request of one query that Notion answers
|
|
221
|
+
with a 200, both tables of a join and both sides of a union included; a
|
|
222
|
+
rate-limited attempt the adapter retries, or a request that fails (a
|
|
223
|
+
page gone), counts nothing. The query raises `Sequel::Error` before the
|
|
224
|
+
request past it. A query run on a row inside `each` is a query of its
|
|
225
|
+
own, with its own budget.
|
|
226
|
+
|
|
227
|
+
- **Aggregates.** `sum`, `avg`, `min`, `max` and `count(:col)` skip
|
|
228
|
+
`nil`s and give `nil` over no value.
|
|
229
|
+
- **Distinct.** `distinct` drops repeated rows before `offset` and
|
|
230
|
+
`limit`; `distinct(:P)` keeps the first row of each value, in the
|
|
231
|
+
query's order.
|
|
232
|
+
- **Groups.** `group(:P)` with the columns it groups and `count`, `sum`,
|
|
233
|
+
`avg`, `min` or `max` (`group_and_count`, `select_group`) keeps one
|
|
234
|
+
running value per group; `order`, `offset` and `limit` then apply to
|
|
235
|
+
the groups, empty values last. `having` filters them, on an aggregate
|
|
236
|
+
written out (`having { count.function.* > 1 }`) or on an output's
|
|
237
|
+
name, with SQL's rules for `nil`.
|
|
238
|
+
- **Combined queries.** `union` (with or without `all:`), `intersect` and
|
|
239
|
+
`except` (without `all:`) combine the rows of two queries as SQL does:
|
|
240
|
+
the result takes the first query's column names, the other's values
|
|
241
|
+
matched by position, and queries selecting a different number of
|
|
242
|
+
columns raise. `order`, `offset`, `limit` and a `select` of columns
|
|
243
|
+
apply to the
|
|
244
|
+
result, and a `where`, `group`, `having`, `distinct` or join added to
|
|
245
|
+
it raises.
|
|
246
|
+
- **Joins.** `join` and `left_join` match rows on one equality, and a
|
|
247
|
+
relation matches every page it lists, so a join follows it:
|
|
248
|
+
|
|
249
|
+
```ruby
|
|
250
|
+
DB[:tasks].client_side.join(:projects, id: :Project)
|
|
251
|
+
.where(Sequel[:projects][:Budget] > 1000)
|
|
252
|
+
.select(Sequel[:tasks][:Name].as(:task),
|
|
253
|
+
Sequel[:projects][:Name].as(:project))
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Each `where` condition that tests one table runs in Notion, on that
|
|
257
|
+
table's query. On a left join's joined table it still filters the
|
|
258
|
+
joined rows, as SQL's `WHERE` does: a row whose partners all fail it
|
|
259
|
+
is dropped, and a row with no partner is kept only if an empty page
|
|
260
|
+
passes it (`where(Sequel[:projects][:Budget] => nil)`); that table is
|
|
261
|
+
queried twice, with and without the condition. A column both tables
|
|
262
|
+
have must be qualified. Without a
|
|
263
|
+
`select`, a later table's column wins a shared name, as with SQL
|
|
264
|
+
adapters.
|
|
265
|
+
|
|
266
|
+
|
|
267
|
+
## Values
|
|
268
|
+
|
|
269
|
+
How each Notion type reads back, and what a write takes for it:
|
|
270
|
+
|
|
271
|
+
| Notion type | Reads as | A write takes |
|
|
272
|
+
| ------------------- | ----------------------- | --------------------------- |
|
|
273
|
+
| title, rich_text | plain text | anything (`to_s`) |
|
|
274
|
+
| number | Integer or Float | `Numeric`, decimal `String` |
|
|
275
|
+
| select, status | the option name | the option name |
|
|
276
|
+
| multi_select | an `Array` of names | an `Array` of names, or one |
|
|
277
|
+
| date | ISO start, or a `Range` | `Date`, `Time`, `Range`, … |
|
|
278
|
+
| checkbox | `true` / `false` | `true` / `false` |
|
|
279
|
+
| url, email, phone | a `String` | `to_s` |
|
|
280
|
+
| relation | an `Array` of page ids | page id(s) |
|
|
281
|
+
| people | an `Array` of user ids | user id(s) |
|
|
282
|
+
| files | `Sequel::Notion::File`s | `File`, URL, or an `Array` |
|
|
283
|
+
| formula | its result | read-only |
|
|
284
|
+
| rollup | its value | read-only |
|
|
285
|
+
| unique_id | as shown, `"TK-62"` | read-only |
|
|
286
|
+
| created/edited time | an ISO 8601 `String` | read-only |
|
|
287
|
+
| created/edited by | Notion's user object | read-only |
|
|
288
|
+
|
|
289
|
+
`nil` clears a property: to `[]` for text and lists, `false` for a
|
|
290
|
+
checkbox, `null` otherwise.
|
|
291
|
+
|
|
292
|
+
- **Text** is written in runs of 2000 characters as Notion counts them
|
|
293
|
+
(an emoji is two).
|
|
294
|
+
- **Numbers** given as a `String` must be decimal (`"1e3"`, not `"0x1A"`
|
|
295
|
+
or `"1_000"`), in writes and filters alike; a number other than
|
|
296
|
+
Integer or Float is sent as a Float.
|
|
297
|
+
- **Dates** read back as the ISO 8601 start, or as a `Range` of the two
|
|
298
|
+
strings when the date has an end, which a write takes back as is. A
|
|
299
|
+
write also takes an ISO 8601 `String` or `{start:, end:}`. Notion's
|
|
300
|
+
date ranges include their end, so an exclusive range `d1...d2` ends on
|
|
301
|
+
the day before `d2`; it must then end on a `Date`, and an exclusive
|
|
302
|
+
`Time` range raises. A range needs a start: `d1..` leaves the end
|
|
303
|
+
open, and `..d2` raises, as does a `Hash` with no `:start`. Notion
|
|
304
|
+
keeps a time to the minute: `10:20:30.123` reads back as `10:20:00`
|
|
305
|
+
(checked live). A date filter takes a `Date`, `Time`, `DateTime` or
|
|
306
|
+
ISO 8601 string, and raises on anything else.
|
|
307
|
+
- **Relations and people** read back in full: a page lists at most 25,
|
|
308
|
+
and the rest is fetched from Notion for the columns a query selects.
|
|
309
|
+
- **Files**: an unnamed file is named after the URL's last path segment,
|
|
310
|
+
decoded (`a%20b.pdf` names it `a b.pdf`);
|
|
311
|
+
a file of a type the adapter does not know reads back with its `raw`
|
|
312
|
+
hash and is written back unchanged.
|
|
313
|
+
- **Rollups** read as their value: a number, a date, or an `Array` of
|
|
314
|
+
the rolled-up values, each read as its own type.
|
|
315
|
+
|
|
316
|
+
NaN and Infinity, which JSON cannot carry, raise `Sequel::Error` in a
|
|
317
|
+
write or a filter, and so does an external file URL that does not parse
|
|
318
|
+
(a space in it, for instance).
|
|
319
|
+
|
|
320
|
+
|
|
321
|
+
## Writing
|
|
322
|
+
|
|
323
|
+
```ruby
|
|
324
|
+
id = DB[:tasks].insert(Name: "Write the adapter", Status: "Todo",
|
|
325
|
+
Tags: %w[ruby notion], Due: Date.today)
|
|
326
|
+
DB[:tasks].where(Status: "Todo").update(Status: "Done")
|
|
327
|
+
DB[:tasks].where(id: id).delete # moves the page to the trash
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
Values are encoded according to the property's Notion type, read from
|
|
331
|
+
the data source (see [Values](#values)). Writing a computed property
|
|
332
|
+
(formula, rollup, created/edited time or by, unique ID, button,
|
|
333
|
+
verification) or an unknown property raises `Sequel::Error`. `insert`
|
|
334
|
+
ignores `:id`; `update` ignores `:id` and turns `:in_trash` into
|
|
335
|
+
trashing or restoring the page, so `where(id: id).update(in_trash: false)`
|
|
336
|
+
restores a trashed page. A positional `insert(["a", 2])` fills the
|
|
337
|
+
writable columns in schema order.
|
|
338
|
+
|
|
339
|
+
`update` and `delete` first collect the matching page ids, then send one
|
|
340
|
+
request per page.
|
|
341
|
+
|
|
342
|
+
|
|
343
|
+
## Models
|
|
344
|
+
|
|
345
|
+
```ruby
|
|
346
|
+
class Task < Sequel::Model(DB[:tasks])
|
|
347
|
+
end
|
|
348
|
+
|
|
349
|
+
task = Task.create(Name: "Ship it", Status: "Todo")
|
|
350
|
+
task.update(Status: "Done")
|
|
351
|
+
Task[task.id].delete
|
|
352
|
+
```
|
|
353
|
+
|
|
354
|
+
The primary key is `:id`. A `save` of a loaded record sends only the
|
|
355
|
+
columns that changed, as `update` and `save_changes` do: a row reads
|
|
356
|
+
back rich text as plain text, and writing the whole row back would
|
|
357
|
+
make the loss permanent. Computed properties are marked `generated` in
|
|
358
|
+
the schema, for Sequel's `skip_saving_columns` plugin. Date columns are
|
|
359
|
+
not typecast, so a `Time` or a `Range` reaches Notion as given.
|
|
360
|
+
|
|
361
|
+
Notion cannot sort by page id, so a model adds no primary key order:
|
|
362
|
+
`Task.paged_each` streams in Notion's order, and `Task.last` needs an
|
|
363
|
+
explicit one, or raises Sequel's `No order specified`. A unique ID
|
|
364
|
+
property gives one in creation order, `Task.order(:ID).last`. A
|
|
365
|
+
created-time property does too, `Task.order(:Created).last`, but only to
|
|
366
|
+
the minute: Notion stores created and edited times without seconds, so
|
|
367
|
+
pages created in the same minute tie. Sequel's
|
|
368
|
+
`paged_operations` plugin pages by primary key ranges, so it raises on a
|
|
369
|
+
Notion model. `Task.client_side` opens what is
|
|
370
|
+
[computed in Ruby](#computed-in-ruby) to a model.
|
|
371
|
+
|
|
372
|
+
Notion has no transactions: `DB.transaction` runs its block, swallows
|
|
373
|
+
`Sequel::Rollback` (re-raised with `rollback: :reraise`), and rolls nothing
|
|
374
|
+
back. `rollback: :always` raises, since every write would be kept.
|
|
375
|
+
|
|
376
|
+
|
|
377
|
+
## Schema
|
|
378
|
+
|
|
379
|
+
`DB.schema(:tasks)` lists `:id`, `:in_trash` and each property, with
|
|
380
|
+
`:db_type` set to its Notion type; computed properties are marked
|
|
381
|
+
`generated: true`. It is cached per data source. `DB.table_exists?`
|
|
382
|
+
answers by resolving the name and fetching the data source. Call
|
|
383
|
+
`DB.refresh_schema!(:tasks)` after changing the data source's properties
|
|
384
|
+
in Notion; every name of that data source (an alias, its id) sees the
|
|
385
|
+
change.
|
|
386
|
+
|
|
387
|
+
|
|
388
|
+
## Errors, retries and logging
|
|
389
|
+
|
|
390
|
+
- A failed request, or a response that is not valid JSON, raises
|
|
391
|
+
`Sequel::DatabaseError`, with Notion's error code and message in the
|
|
392
|
+
text. A 404 raises its subclass `Sequel::Notion::NotFoundError`
|
|
393
|
+
(`where(id:)` turns it into no row).
|
|
394
|
+
- Responses 429, 502, 503 and 504 are retried up to four times, honouring
|
|
395
|
+
`Retry-After`. Page creation is retried only after a 429, so a timeout
|
|
396
|
+
cannot create a page twice.
|
|
397
|
+
- Requests go to the database's loggers (`DB.loggers << Logger.new($stdout)`)
|
|
398
|
+
as `POST data_sources/…/query` with their body.
|
|
399
|
+
|
|
400
|
+
|
|
401
|
+
## Known shortfalls
|
|
402
|
+
|
|
403
|
+
- No raw SQL (`with_sql`, `run`, `truncate`, schema changes): there is
|
|
404
|
+
no SQL to run it, and each raises `Sequel::Error`. No locks
|
|
405
|
+
(`for_update`): Notion has none.
|
|
406
|
+
- Joins are inner or left, on one equality; right, full and cross joins,
|
|
407
|
+
and a `where` condition testing two tables, raise.
|
|
408
|
+
- What is [computed in Ruby](#computed-in-ruby) reads every
|
|
409
|
+
row its queries return, and so do `offset` and `count`.
|
|
410
|
+
- Filters compare a property with a value, never with another property or
|
|
411
|
+
an expression.
|
|
412
|
+
- `LIKE` patterns are limited to the shapes in the filter table. A `_`
|
|
413
|
+
wildcard or a `%` in the middle raises. Notion's own case rules apply to
|
|
414
|
+
both `LIKE` and `ILIKE`. On a multi-select, people or relation property,
|
|
415
|
+
Notion matches whole values only, so a pattern with any `%` raises.
|
|
416
|
+
- A rollup that keeps every value (`show_original`, `show_unique`) cannot
|
|
417
|
+
be filtered: Notion's `any`/`every`/`none` have no SQL reading.
|
|
418
|
+
- Mentions inside a title or rich text are cut at 25 by Notion, unflagged,
|
|
419
|
+
and are not completed.
|
|
420
|
+
- Notion's rate limit is 3 requests per second on most plans, so a large
|
|
421
|
+
`update` or `delete` is slow. The retry on a 429 is tested against
|
|
422
|
+
stubs only: bursts of 30 and 90 parallel queries drew no 429 from
|
|
423
|
+
Notion (2026-10-09).
|
|
424
|
+
|
|
425
|
+
|
|
426
|
+
## Checked against the live API
|
|
427
|
+
|
|
428
|
+
The suite stubs Notion, so it proves the payloads match what the adapter
|
|
429
|
+
believes Notion accepts. These were also checked against
|
|
430
|
+
`api.notion.com` (2026-10-09):
|
|
431
|
+
|
|
432
|
+
- **Filters:** title, url and email through the `rich_text` key; created
|
|
433
|
+
and edited times through the `date` key; negations excluding empty
|
|
434
|
+
values, formula negations excluding empty results, a rollup negation
|
|
435
|
+
guarded against an empty average, `NOT IN` on dates and created
|
|
436
|
+
times, and filters merged
|
|
437
|
+
or distributed to two levels; unique ID filters and sorts; filters and
|
|
438
|
+
sorts on a sum rollup and on a `latest_date` rollup; `nil` filters on
|
|
439
|
+
string and number formulas and on both rollups.
|
|
440
|
+
- **Dates:** a time written to a date property kept to the minute; an
|
|
441
|
+
open range (`d..`) and `{start:}` both clearing a range's end.
|
|
442
|
+
- **Checkboxes:** sorted `false` first, `true` first descending.
|
|
443
|
+
- **Rollups:** a `show_original` rollup through 30 relations read in
|
|
444
|
+
full, all 30 values, in a query and by id.
|
|
445
|
+
- **Files:** a Notion-hosted file read from a page and written back
|
|
446
|
+
unchanged, signed URL included, kept by Notion.
|
|
447
|
+
- **Pages:** lookups by id with or without dashes; creation with the
|
|
448
|
+
`data_source_id` parent; writing, reading back and clearing every
|
|
449
|
+
writable type; trashing and restoring; a relation of 26 pages read in
|
|
450
|
+
full; a date range read back as a `Range` and written back; rollups of
|
|
451
|
+
a number, dates and titles read as values.
|
|
452
|
+
- **Models:** create, update and destroy; a `save` of a loaded record
|
|
453
|
+
keeping a date range's end and its relations; `order` on a created-time
|
|
454
|
+
property, whose values Notion stores to the minute.
|
|
455
|
+
- **Reading:** `paged_each` following the cursor with a `page_size` below
|
|
456
|
+
100.
|
|
457
|
+
- **Computed in Ruby:** refused without `client_side` and no request
|
|
458
|
+
sent; aggregates, `distinct`, `DISTINCT ON`, `group`, `having`,
|
|
459
|
+
`union`, `intersect` and `except` over rows with and without values,
|
|
460
|
+
and a `select` on their result;
|
|
461
|
+
inner and left joins through a relation, with a `where` per side, a
|
|
462
|
+
sum and a group over them; a left join's `where` on its joined table,
|
|
463
|
+
equal to a value and empty; `max_requests` stopping a join.
|
|
464
|
+
- **Discovery:** search keeps listing a trashed data source, flagged
|
|
465
|
+
`in_trash`.
|
|
466
|
+
|
|
467
|
+
|
|
468
|
+
## License
|
|
469
|
+
|
|
470
|
+
MIT — see `LICENSE.txt`.
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
require "faraday"
|
|
6
|
+
require "faraday/retry"
|
|
7
|
+
require "sequel"
|
|
8
|
+
|
|
9
|
+
require "sequel/notion/dataset"
|
|
10
|
+
require "sequel/notion/errors"
|
|
11
|
+
require "sequel/notion/model_support"
|
|
12
|
+
require "sequel/notion/page_api"
|
|
13
|
+
require "sequel/notion/registry"
|
|
14
|
+
require "sequel/notion/request_budget"
|
|
15
|
+
require "sequel/notion/schema"
|
|
16
|
+
require "sequel/notion/schema_lookup"
|
|
17
|
+
require "sequel/notion/type_map"
|
|
18
|
+
require "sequel/notion/version"
|
|
19
|
+
|
|
20
|
+
module Sequel
|
|
21
|
+
module Notion
|
|
22
|
+
API_BASE = "https://api.notion.com/v1"
|
|
23
|
+
API_VERSION = "2026-03-11"
|
|
24
|
+
|
|
25
|
+
# Statuses worth another attempt; 429 carries Retry-After, which
|
|
26
|
+
# faraday-retry honours.
|
|
27
|
+
RETRY_STATUSES = [429, 502, 503, 504].freeze
|
|
28
|
+
|
|
29
|
+
# Creating a page is the one call a retry could duplicate: retry it
|
|
30
|
+
# only when Notion refused it outright (rate limited).
|
|
31
|
+
RETRY_IF = lambda do |env, _exception|
|
|
32
|
+
!(env.method == :post && env.url.path.end_with?("/pages")) ||
|
|
33
|
+
env.status == 429
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
class Database < Sequel::Database
|
|
37
|
+
include PageApi
|
|
38
|
+
include Registry
|
|
39
|
+
include RequestBudget
|
|
40
|
+
include SchemaLookup
|
|
41
|
+
|
|
42
|
+
set_adapter_scheme :notion
|
|
43
|
+
|
|
44
|
+
def initialize(...)
|
|
45
|
+
super
|
|
46
|
+
@data_source_cache = {}
|
|
47
|
+
@data_source_epoch = Hash.new(0)
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
# ----------------------------------------------------------
|
|
51
|
+
# Connection — one Faraday client per pooled connection
|
|
52
|
+
# ----------------------------------------------------------
|
|
53
|
+
|
|
54
|
+
def connect(_server)
|
|
55
|
+
token = opts[:token]
|
|
56
|
+
raise Error, "Missing Notion token" unless token
|
|
57
|
+
|
|
58
|
+
Faraday.new(url: API_BASE) { build_stack(it, token) }
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def disconnect_connection(_conn) = nil
|
|
62
|
+
|
|
63
|
+
def dataset_class_default = Notion::Dataset
|
|
64
|
+
|
|
65
|
+
# Notion has no transactions: run the block as is, so that
|
|
66
|
+
# Sequel::Model (which wraps saves in one) works.
|
|
67
|
+
def transaction(opts = OPTS)
|
|
68
|
+
refuse_rollback_always!(opts)
|
|
69
|
+
synchronize { yield it }
|
|
70
|
+
rescue Rollback
|
|
71
|
+
raise if opts[:rollback] == :reraise
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
def table_exists?(name)
|
|
75
|
+
ds_id = data_source_id_for(name)
|
|
76
|
+
!ds_id.nil? && !data_source(ds_id).nil?
|
|
77
|
+
rescue NotFoundError
|
|
78
|
+
false
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def in_transaction?(_opts = OPTS) = false
|
|
82
|
+
|
|
83
|
+
def supports_savepoints? = false
|
|
84
|
+
def supports_schema_parsing? = true
|
|
85
|
+
def supports_transaction_isolation_levels? = false
|
|
86
|
+
|
|
87
|
+
# One Notion request: logged through Sequel's loggers, and any
|
|
88
|
+
# HTTP failure re-raised as a Sequel::DatabaseError. Only its
|
|
89
|
+
# 200 spends the query's budget, after the retries.
|
|
90
|
+
def request(verb, path, body = nil)
|
|
91
|
+
within_budget do
|
|
92
|
+
synchronize do |conn|
|
|
93
|
+
log_connection_yield("#{verb.upcase} #{path}", conn,
|
|
94
|
+
body && [body]) do
|
|
95
|
+
conn.public_send(verb, path, body).body
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
rescue Faraday::Error => e
|
|
100
|
+
raise database_error(e)
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
# Where Sequel sends SQL to run: run, <<, truncate, the
|
|
104
|
+
# with_sql_* writes and every schema change
|
|
105
|
+
def execute(sql, _opts = OPTS)
|
|
106
|
+
raise Error, "Notion takes no SQL: #{sql}"
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
private
|
|
110
|
+
|
|
111
|
+
# A number column is :float, and Sequel's model typecast would
|
|
112
|
+
# turn 5 into 5.0; Notion keeps, and reads back, an Integer
|
|
113
|
+
def typecast_value_float(value)
|
|
114
|
+
value.is_a?(Integer) ? value : super
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def refuse_rollback_always!(opts)
|
|
118
|
+
return unless opts[:rollback] == :always
|
|
119
|
+
|
|
120
|
+
raise Error, "Notion has no transactions to roll back"
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def build_stack(conn, token)
|
|
124
|
+
conn.headers["Authorization"] = "Bearer #{token}"
|
|
125
|
+
conn.headers["Notion-Version"] = API_VERSION
|
|
126
|
+
conn.request :json
|
|
127
|
+
# raise_error must wrap retry, so retry sees the raw
|
|
128
|
+
# status before it is turned into an exception.
|
|
129
|
+
conn.response :raise_error
|
|
130
|
+
conn.request :retry, retry_options
|
|
131
|
+
conn.response :json, content_type: /\bjson$/
|
|
132
|
+
conn.adapter(*Array(opts[:faraday_adapter] ||
|
|
133
|
+
Faraday.default_adapter))
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
def retry_options
|
|
137
|
+
{ max: 4,
|
|
138
|
+
interval: 0.5,
|
|
139
|
+
backoff_factor: 2,
|
|
140
|
+
methods: %i[get patch delete],
|
|
141
|
+
retry_if: RETRY_IF,
|
|
142
|
+
retry_statuses: RETRY_STATUSES }
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
def database_error(error)
|
|
146
|
+
body = error.response_body
|
|
147
|
+
status = error.response_status
|
|
148
|
+
detail = if body.is_a?(Hash)
|
|
149
|
+
"#{body["code"]}: #{body["message"]}"
|
|
150
|
+
else
|
|
151
|
+
error.message
|
|
152
|
+
end
|
|
153
|
+
klass = status == 404 ? NotFoundError : DatabaseError
|
|
154
|
+
klass.new("Notion #{status || "request"} #{detail}")
|
|
155
|
+
.tap { it.wrapped_exception = error }
|
|
156
|
+
end
|
|
157
|
+
end
|
|
158
|
+
end
|
|
159
|
+
end
|