sequel-duckdb 0.1.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.beads/.beads-credential-key +1 -0
- data/.beads/.gitignore +66 -0
- data/.beads/README.md +85 -0
- data/.beads/config.yaml +56 -0
- data/.beads/hooks/post-checkout +24 -0
- data/.beads/hooks/post-merge +24 -0
- data/.beads/hooks/pre-commit +24 -0
- data/.beads/hooks/pre-push +24 -0
- data/.beads/hooks/prepare-commit-msg +24 -0
- data/.beads/metadata.json +7 -0
- data/.kiro/specs/advanced-sql-features-implementation/design.md +3 -1
- data/.kiro/specs/advanced-sql-features-implementation/requirements.md +1 -1
- data/.kiro/specs/advanced-sql-features-implementation/tasks.md +5 -1
- data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +15 -1
- data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +1 -1
- data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +13 -0
- data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +1 -1
- data/.kiro/specs/integration-test-database-setup/requirements.md +1 -1
- data/.kiro/specs/sequel-duckdb-adapter/design.md +8 -1
- data/.kiro/specs/sequel-duckdb-adapter/requirements.md +10 -10
- data/.kiro/specs/sequel-duckdb-adapter/tasks.md +48 -3
- data/.kiro/specs/sql-expression-handling-fix/design.md +34 -1
- data/.kiro/specs/sql-expression-handling-fix/requirements.md +1 -1
- data/.kiro/specs/sql-expression-handling-fix/tasks.md +3 -0
- data/.kiro/specs/test-infrastructure-improvements/requirements.md +1 -1
- data/.kiro/steering/product.md +5 -1
- data/.kiro/steering/structure.md +1 -1
- data/.kiro/steering/tech.md +14 -1
- data/.kiro/steering/testing.md +22 -1
- data/.mdformat.toml +2 -0
- data/.rubocop.yml +116 -58
- data/.rubocop_todo.yml +323 -0
- data/AGENTS.md +180 -0
- data/API_DOCUMENTATION.md +73 -49
- data/CHANGELOG.md +47 -10
- data/FINAL_STATUS.md +99 -0
- data/LICENSE +1 -1
- data/MIGRATION_EXAMPLES.md +1 -1
- data/PERFORMANCE_OPTIMIZATIONS.md +4 -1
- data/README.md +90 -1
- data/REFACTORING_SUMMARY.md +264 -0
- data/Rakefile +21 -5
- data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +19 -1
- data/docs/DUCKDB_SQL_PATTERNS.md +39 -1
- data/docs/TASK_12_VERIFICATION_SUMMARY.md +14 -1
- data/justfile +50 -0
- data/lib/sequel/adapters/duckdb.rb +137 -108
- data/lib/sequel/adapters/shared/duckdb.rb +292 -1490
- data/lib/sequel/duckdb/helpers/copier.rb +50 -0
- data/lib/sequel/duckdb/helpers/pathifier.rb +141 -0
- data/lib/sequel/duckdb/version.rb +2 -2
- data/plans/date_arithmetic.md +420 -0
- data/plans/engineering/Sequel.md +471 -0
- data/plans/engineering/duckdb.md +712 -0
- data/plans/engineering/sqlite.md +453 -0
- data/plans/mock_connection_bug.md +333 -0
- data/plans/mock_without_driver_gem.md +371 -0
- data/plans/over_engineering_analysis.md +122 -0
- data/plans/schema_management.md +383 -0
- metadata +47 -27
data/API_DOCUMENTATION.md
CHANGED
|
@@ -17,12 +17,12 @@ This document provides comprehensive API documentation for the Sequel DuckDB ada
|
|
|
17
17
|
|
|
18
18
|
### Supported Versions
|
|
19
19
|
|
|
20
|
-
| Component
|
|
21
|
-
|
|
22
|
-
| Ruby
|
|
23
|
-
| Sequel
|
|
24
|
-
| DuckDB
|
|
25
|
-
| ruby-duckdb | 1.0.0
|
|
20
|
+
| Component | Minimum Version | Recommended Version | Notes |
|
|
21
|
+
| ----------- | --------------- | ------------------- | ------------------------------------------ |
|
|
22
|
+
| Ruby | 3.1.0 | 3.2.0+ | Required for modern syntax and performance |
|
|
23
|
+
| Sequel | 5.0.0 | 5.70.0+ | Core ORM functionality |
|
|
24
|
+
| DuckDB | 0.8.0 | 0.9.0+ | Database engine |
|
|
25
|
+
| ruby-duckdb | 1.0.0 | 1.0.0+ | Ruby client library |
|
|
26
26
|
|
|
27
27
|
### Ruby Version Support
|
|
28
28
|
|
|
@@ -63,6 +63,7 @@ db = Sequel.connect('duckdb:///path/to/database.duckdb?readonly=true')
|
|
|
63
63
|
```
|
|
64
64
|
|
|
65
65
|
**Parameters:**
|
|
66
|
+
|
|
66
67
|
- `connection_string` (String): DuckDB connection string
|
|
67
68
|
|
|
68
69
|
**Returns:** `Sequel::DuckDB::Database` instance
|
|
@@ -87,6 +88,7 @@ db = Sequel.connect(
|
|
|
87
88
|
```
|
|
88
89
|
|
|
89
90
|
**Parameters:**
|
|
91
|
+
|
|
90
92
|
- `options_hash` (Hash): Configuration options
|
|
91
93
|
- `:adapter` (String): Must be 'duckdb'
|
|
92
94
|
- `:database` (String): Database path or ':memory:'
|
|
@@ -111,6 +113,7 @@ db.tables(schema: 'main')
|
|
|
111
113
|
```
|
|
112
114
|
|
|
113
115
|
**Parameters:**
|
|
116
|
+
|
|
114
117
|
- `options` (Hash): Optional parameters
|
|
115
118
|
- `:schema` (String): Schema name (default: 'main')
|
|
116
119
|
|
|
@@ -130,6 +133,7 @@ db.schema(:users)
|
|
|
130
133
|
```
|
|
131
134
|
|
|
132
135
|
**Parameters:**
|
|
136
|
+
|
|
133
137
|
- `table_name` (Symbol/String): Name of the table
|
|
134
138
|
- `options` (Hash): Optional parameters
|
|
135
139
|
- `:schema` (String): Schema name (default: 'main')
|
|
@@ -137,6 +141,7 @@ db.schema(:users)
|
|
|
137
141
|
**Returns:** Array of `[column_name, column_info]` pairs
|
|
138
142
|
|
|
139
143
|
**Column Info Hash:**
|
|
144
|
+
|
|
140
145
|
- `:type` (Symbol): Sequel type (:integer, :string, :boolean, etc.)
|
|
141
146
|
- `:db_type` (String): DuckDB native type
|
|
142
147
|
- `:primary_key` (Boolean): Whether column is part of primary key
|
|
@@ -162,6 +167,7 @@ db.indexes(:users)
|
|
|
162
167
|
```
|
|
163
168
|
|
|
164
169
|
**Parameters:**
|
|
170
|
+
|
|
165
171
|
- `table_name` (Symbol/String): Name of the table
|
|
166
172
|
- `options` (Hash): Optional parameters
|
|
167
173
|
|
|
@@ -177,6 +183,7 @@ db.table_exists?(:nonexistent) # => false
|
|
|
177
183
|
```
|
|
178
184
|
|
|
179
185
|
**Parameters:**
|
|
186
|
+
|
|
180
187
|
- `table_name` (Symbol/String): Name of the table
|
|
181
188
|
- `options` (Hash): Optional parameters
|
|
182
189
|
|
|
@@ -202,6 +209,7 @@ end
|
|
|
202
209
|
```
|
|
203
210
|
|
|
204
211
|
**Parameters:**
|
|
212
|
+
|
|
205
213
|
- `sql` (String): SQL statement to execute
|
|
206
214
|
- `options` (Hash/Array): Parameters or options
|
|
207
215
|
- If Array: Parameters for prepared statement
|
|
@@ -218,6 +226,7 @@ db.execute_insert("INSERT INTO users (name, email) VALUES (?, ?)", ['John', 'joh
|
|
|
218
226
|
```
|
|
219
227
|
|
|
220
228
|
**Parameters:**
|
|
229
|
+
|
|
221
230
|
- `sql` (String): INSERT SQL statement
|
|
222
231
|
- `options` (Hash): Options for execution
|
|
223
232
|
|
|
@@ -232,6 +241,7 @@ affected_rows = db.execute_update("UPDATE users SET active = ? WHERE age > ?", [
|
|
|
232
241
|
```
|
|
233
242
|
|
|
234
243
|
**Parameters:**
|
|
244
|
+
|
|
235
245
|
- `sql` (String): UPDATE SQL statement
|
|
236
246
|
- `options` (Hash): Options for execution
|
|
237
247
|
|
|
@@ -268,6 +278,7 @@ end
|
|
|
268
278
|
```
|
|
269
279
|
|
|
270
280
|
**Parameters:**
|
|
281
|
+
|
|
271
282
|
- `options` (Hash): Transaction options
|
|
272
283
|
- `:savepoint` (Boolean): Use savepoint for nested transaction
|
|
273
284
|
- `:isolation` (Symbol): Transaction isolation level
|
|
@@ -276,6 +287,7 @@ end
|
|
|
276
287
|
**Returns:** Result of the block
|
|
277
288
|
|
|
278
289
|
**Raises:**
|
|
290
|
+
|
|
279
291
|
- `Sequel::Rollback`: To rollback transaction
|
|
280
292
|
- `Sequel::DatabaseError`: On transaction errors
|
|
281
293
|
|
|
@@ -324,6 +336,7 @@ users.where(Sequel.like(:name, 'John%') & (Sequel[:age] > 25))
|
|
|
324
336
|
```
|
|
325
337
|
|
|
326
338
|
**Parameters:**
|
|
339
|
+
|
|
327
340
|
- `conditions`: Various condition formats (Hash, String, Block, Sequel expressions)
|
|
328
341
|
|
|
329
342
|
**Returns:** New Dataset with WHERE clause added
|
|
@@ -346,6 +359,7 @@ users.select(:id, Sequel.function(:upper, :name).as(:name_upper))
|
|
|
346
359
|
```
|
|
347
360
|
|
|
348
361
|
**Parameters:**
|
|
362
|
+
|
|
349
363
|
- `columns`: Column names, expressions, or functions
|
|
350
364
|
|
|
351
365
|
**Returns:** New Dataset with SELECT clause
|
|
@@ -371,6 +385,7 @@ users.order(:name, Sequel.desc(:created_at))
|
|
|
371
385
|
```
|
|
372
386
|
|
|
373
387
|
**Parameters:**
|
|
388
|
+
|
|
374
389
|
- `columns`: Column names or ordering expressions
|
|
375
390
|
|
|
376
391
|
**Returns:** New Dataset with ORDER BY clause
|
|
@@ -393,6 +408,7 @@ users.paginate(page: 2, per_page: 10)
|
|
|
393
408
|
```
|
|
394
409
|
|
|
395
410
|
**Parameters:**
|
|
411
|
+
|
|
396
412
|
- `count` (Integer): Maximum number of rows
|
|
397
413
|
- `offset` (Integer): Number of rows to skip
|
|
398
414
|
|
|
@@ -416,6 +432,7 @@ orders.group(:status).select(:status, Sequel.count(:id).as(:count))
|
|
|
416
432
|
```
|
|
417
433
|
|
|
418
434
|
**Parameters:**
|
|
435
|
+
|
|
419
436
|
- `columns`: Column names to group by
|
|
420
437
|
|
|
421
438
|
**Returns:** New Dataset with GROUP BY clause
|
|
@@ -433,6 +450,7 @@ orders.group(:user_id)
|
|
|
433
450
|
```
|
|
434
451
|
|
|
435
452
|
**Parameters:**
|
|
453
|
+
|
|
436
454
|
- `conditions`: Conditions for HAVING clause
|
|
437
455
|
|
|
438
456
|
**Returns:** New Dataset with HAVING clause
|
|
@@ -457,6 +475,7 @@ users.join(:orders, Sequel[:orders][:user_id] => Sequel[:users][:id])
|
|
|
457
475
|
```
|
|
458
476
|
|
|
459
477
|
**Parameters:**
|
|
478
|
+
|
|
460
479
|
- `table`: Table to join (Symbol/String)
|
|
461
480
|
- `conditions`: Join conditions (Hash or Sequel expression)
|
|
462
481
|
- `options`: Join options
|
|
@@ -543,6 +562,7 @@ end
|
|
|
543
562
|
```
|
|
544
563
|
|
|
545
564
|
**Parameters:**
|
|
565
|
+
|
|
546
566
|
- `block`: Block to execute for each record
|
|
547
567
|
|
|
548
568
|
**Returns:** Dataset (for chaining)
|
|
@@ -558,6 +578,7 @@ end
|
|
|
558
578
|
```
|
|
559
579
|
|
|
560
580
|
**Parameters:**
|
|
581
|
+
|
|
561
582
|
- `options` (Hash): Paging options
|
|
562
583
|
- `:rows_per_fetch` (Integer): Batch size (default: 1000)
|
|
563
584
|
- `block`: Block to execute for each record
|
|
@@ -577,6 +598,7 @@ user_id = db[:users].insert(
|
|
|
577
598
|
```
|
|
578
599
|
|
|
579
600
|
**Parameters:**
|
|
601
|
+
|
|
580
602
|
- `values` (Hash): Column values to insert
|
|
581
603
|
|
|
582
604
|
**Returns:** Inserted record ID (if available)
|
|
@@ -594,6 +616,7 @@ db[:users].multi_insert([
|
|
|
594
616
|
```
|
|
595
617
|
|
|
596
618
|
**Parameters:**
|
|
619
|
+
|
|
597
620
|
- `array` (Array): Array of record hashes
|
|
598
621
|
|
|
599
622
|
**Returns:** Number of inserted records
|
|
@@ -609,6 +632,7 @@ affected_rows = db[:users]
|
|
|
609
632
|
```
|
|
610
633
|
|
|
611
634
|
**Parameters:**
|
|
635
|
+
|
|
612
636
|
- `values` (Hash): Column values to update
|
|
613
637
|
|
|
614
638
|
**Returns:** Number of affected rows
|
|
@@ -788,56 +812,56 @@ end
|
|
|
788
812
|
|
|
789
813
|
### Error Types
|
|
790
814
|
|
|
791
|
-
| Sequel Exception
|
|
792
|
-
|
|
793
|
-
| `NotNullConstraintViolation`
|
|
794
|
-
| `UniqueConstraintViolation`
|
|
795
|
-
| `ForeignKeyConstraintViolation` | `foreign key constraint`, `violates foreign key`
|
|
796
|
-
| `CheckConstraintViolation`
|
|
797
|
-
| `ConstraintViolation`
|
|
798
|
-
| `DatabaseConnectionError`
|
|
799
|
-
| `DatabaseError`
|
|
815
|
+
| Sequel Exception | DuckDB Error Patterns | Description |
|
|
816
|
+
| ------------------------------- | ----------------------------------------------------- | ------------------------------ |
|
|
817
|
+
| `NotNullConstraintViolation` | `violates not null`, `null value not allowed` | NOT NULL constraint violations |
|
|
818
|
+
| `UniqueConstraintViolation` | `unique constraint`, `duplicate key` | UNIQUE constraint violations |
|
|
819
|
+
| `ForeignKeyConstraintViolation` | `foreign key constraint`, `violates foreign key` | Foreign key violations |
|
|
820
|
+
| `CheckConstraintViolation` | `check constraint`, `violates check` | CHECK constraint violations |
|
|
821
|
+
| `ConstraintViolation` | `constraint violation` | Generic constraint violations |
|
|
822
|
+
| `DatabaseConnectionError` | `connection`, `cannot open`, `database not found` | Connection-related errors |
|
|
823
|
+
| `DatabaseError` | `syntax error`, `parse error`, `table does not exist` | General database errors |
|
|
800
824
|
|
|
801
825
|
## Data Type Mappings
|
|
802
826
|
|
|
803
827
|
### Ruby to DuckDB Type Mapping
|
|
804
828
|
|
|
805
|
-
| Ruby Type
|
|
806
|
-
|
|
807
|
-
| `String`
|
|
808
|
-
| `String` (large)
|
|
809
|
-
| `Integer`
|
|
810
|
-
| `Integer` (large)
|
|
811
|
-
| `Float`
|
|
812
|
-
| `BigDecimal`
|
|
813
|
-
| `TrueClass/FalseClass` | `BOOLEAN`
|
|
814
|
-
| `Date`
|
|
815
|
-
| `Time/DateTime`
|
|
816
|
-
| `Time` (time-only)
|
|
817
|
-
| `String` (binary)
|
|
818
|
-
| `Array`
|
|
819
|
-
| `Hash`
|
|
829
|
+
| Ruby Type | DuckDB Type | Notes |
|
|
830
|
+
| ---------------------- | ----------- | ---------------------------------- |
|
|
831
|
+
| `String` | `VARCHAR` | Default string type |
|
|
832
|
+
| `String` (large) | `TEXT` | For long text content |
|
|
833
|
+
| `Integer` | `INTEGER` | 32-bit signed integer |
|
|
834
|
+
| `Integer` (large) | `BIGINT` | 64-bit signed integer |
|
|
835
|
+
| `Float` | `DOUBLE` | Double precision floating point |
|
|
836
|
+
| `BigDecimal` | `DECIMAL` | Exact numeric with precision/scale |
|
|
837
|
+
| `TrueClass/FalseClass` | `BOOLEAN` | Native boolean type |
|
|
838
|
+
| `Date` | `DATE` | Date without time |
|
|
839
|
+
| `Time/DateTime` | `TIMESTAMP` | Date and time |
|
|
840
|
+
| `Time` (time-only) | `TIME` | Time without date |
|
|
841
|
+
| `String` (binary) | `BLOB` | Binary data |
|
|
842
|
+
| `Array` | `ARRAY` | DuckDB array types |
|
|
843
|
+
| `Hash` | `JSON` | JSON data type |
|
|
820
844
|
|
|
821
845
|
### DuckDB to Ruby Type Mapping
|
|
822
846
|
|
|
823
|
-
| DuckDB Type
|
|
824
|
-
|
|
825
|
-
| `INTEGER`, `INT4`
|
|
826
|
-
| `BIGINT`, `INT8`
|
|
827
|
-
| `SMALLINT`, `INT2`
|
|
828
|
-
| `TINYINT`, `INT1`
|
|
829
|
-
| `REAL`, `FLOAT4`
|
|
830
|
-
| `DOUBLE`, `FLOAT8`
|
|
831
|
-
| `DECIMAL`, `NUMERIC` | `BigDecimal`
|
|
832
|
-
| `VARCHAR`, `TEXT`
|
|
833
|
-
| `BOOLEAN`
|
|
834
|
-
| `DATE`
|
|
835
|
-
| `TIMESTAMP`
|
|
836
|
-
| `TIME`
|
|
837
|
-
| `BLOB`, `BYTEA`
|
|
838
|
-
| `JSON`
|
|
839
|
-
| `ARRAY`
|
|
840
|
-
| `UUID`
|
|
847
|
+
| DuckDB Type | Ruby Type | Conversion Notes |
|
|
848
|
+
| -------------------- | ---------------------- | ------------------------------- |
|
|
849
|
+
| `INTEGER`, `INT4` | `Integer` | 32-bit integer |
|
|
850
|
+
| `BIGINT`, `INT8` | `Integer` | 64-bit integer |
|
|
851
|
+
| `SMALLINT`, `INT2` | `Integer` | 16-bit integer |
|
|
852
|
+
| `TINYINT`, `INT1` | `Integer` | 8-bit integer |
|
|
853
|
+
| `REAL`, `FLOAT4` | `Float` | Single precision |
|
|
854
|
+
| `DOUBLE`, `FLOAT8` | `Float` | Double precision |
|
|
855
|
+
| `DECIMAL`, `NUMERIC` | `BigDecimal` | Exact numeric |
|
|
856
|
+
| `VARCHAR`, `TEXT` | `String` | Text data |
|
|
857
|
+
| `BOOLEAN` | `TrueClass/FalseClass` | Boolean values |
|
|
858
|
+
| `DATE` | `Date` | Date only |
|
|
859
|
+
| `TIMESTAMP` | `Time` | Date and time |
|
|
860
|
+
| `TIME` | `Time` | Time only |
|
|
861
|
+
| `BLOB`, `BYTEA` | `String` | Binary data as string |
|
|
862
|
+
| `JSON` | `String` | JSON as string (parse manually) |
|
|
863
|
+
| `ARRAY` | `Array` | Native array support |
|
|
864
|
+
| `UUID` | `String` | UUID as string |
|
|
841
865
|
|
|
842
866
|
### Custom Type Handling
|
|
843
867
|
|
|
@@ -916,4 +940,4 @@ db = Sequel.connect(
|
|
|
916
940
|
)
|
|
917
941
|
```
|
|
918
942
|
|
|
919
|
-
This comprehensive API documentation covers all major aspects of using the Sequel DuckDB adapter. For the most up-to-date information, refer to the inline YARD documentation in the source code.
|
|
943
|
+
This comprehensive API documentation covers all major aspects of using the Sequel DuckDB adapter. For the most up-to-date information, refer to the inline YARD documentation in the source code.
|
data/CHANGELOG.md
CHANGED
|
@@ -5,22 +5,48 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
-
## [
|
|
8
|
+
## [0.2.1](https://github.com/outcomesinsights/sequel-duckdb/compare/v0.2.0...v0.2.1) (2026-10-01)
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
-
|
|
12
|
-
|
|
13
|
-
-
|
|
14
|
-
|
|
15
|
-
|
|
10
|
+
No changes to the gem's behaviour. The first release published by the
|
|
11
|
+
tag-based release workflow.
|
|
12
|
+
|
|
13
|
+
## [0.2.0](https://github.com/outcomesinsights/sequel-duckdb/compare/v0.1.0...v0.2.0) (2026-09-30)
|
|
14
|
+
|
|
15
|
+
### ⚠ BREAKING CHANGES
|
|
16
|
+
|
|
17
|
+
- Removed parameterized query support, custom error handling methods, and other over-engineered features. Adapter now follows Sequel conventions using built-in features.
|
|
18
|
+
- Removed custom error message formatting methods (database_exception_message, database_exception_class, handle_constraint_violation) in favor of Sequel's built-in patterns.
|
|
19
|
+
|
|
20
|
+
### Features
|
|
21
|
+
|
|
22
|
+
- add Database#copy_to support ([17ce698](https://github.com/outcomesinsights/sequel-duckdb/commit/17ce6981fca43653ecd50c29f83b50bcf7e5b23b))
|
|
23
|
+
- add schema methods ([fcd11fd](https://github.com/outcomesinsights/sequel-duckdb/commit/fcd11fd99b267b925888ef8accfc7bc7dbad87ad))
|
|
24
|
+
- add support for date_arithmetic ([fc0dde7](https://github.com/outcomesinsights/sequel-duckdb/commit/fc0dde71c542f3a1cb23310a12b8529065b9baf6))
|
|
25
|
+
- additional options for create_view ([7f19ad2](https://github.com/outcomesinsights/sequel-duckdb/commit/7f19ad2efe3ea7c1074b5e9e8a1c918216958701))
|
|
26
|
+
- share DuckDB::Database instance and support cross-database schema queries ([564cb47](https://github.com/outcomesinsights/sequel-duckdb/commit/564cb4767471f0a5183e7219bf6008e57e644fec))
|
|
27
|
+
- support different read\_\* functions in CREATE VIEW ([98eba2e](https://github.com/outcomesinsights/sequel-duckdb/commit/98eba2ed486d1f775508ea4bac0dc6f7cd62a37e))
|
|
28
|
+
|
|
29
|
+
### Bug Fixes
|
|
30
|
+
|
|
31
|
+
- bump minimum Ruby to 3.2 and upgrade minitest to 6.x ([55b874d](https://github.com/outcomesinsights/sequel-duckdb/commit/55b874d96f49684f3a4f13eccb7c6af5fc83e6b8))
|
|
32
|
+
- **ci:** install DuckDB C library for native extension ([#7](https://github.com/outcomesinsights/sequel-duckdb/issues/7)) ([7af7437](https://github.com/outcomesinsights/sequel-duckdb/commit/7af7437c8b4db151e913087e558d3950bed34601))
|
|
33
|
+
- formatting for date_arithmetic ([835ad7e](https://github.com/outcomesinsights/sequel-duckdb/commit/835ad7e6a9de7fd0b6c441ee394aa8ff54cc0612))
|
|
34
|
+
- move DuckDB::Database init from connect to adapter_initialize ([e06e0f2](https://github.com/outcomesinsights/sequel-duckdb/commit/e06e0f2958b958ae368948a80234af058a24b7cd))
|
|
35
|
+
- remove hard duckdb C extension dependency from gemspec ([7c598ac](https://github.com/outcomesinsights/sequel-duckdb/commit/7c598ac53534c7d29567bdf6c2fe1170162fa216))
|
|
36
|
+
- use nested module syntax for Helpers to support mock adapter loading ([7962262](https://github.com/outcomesinsights/sequel-duckdb/commit/7962262dd9c45369815404a2c325f67f2820295f))
|
|
37
|
+
|
|
38
|
+
### Code Refactoring
|
|
39
|
+
|
|
40
|
+
- simplify adapter execution and error handling ([e812777](https://github.com/outcomesinsights/sequel-duckdb/commit/e812777acd815466cebbd58cc1f54b8254fcabfd))
|
|
41
|
+
|
|
42
|
+
### Tests
|
|
16
43
|
|
|
17
|
-
|
|
18
|
-
- Enhanced README with comprehensive usage examples
|
|
19
|
-
- Improved documentation structure and organization
|
|
44
|
+
- remove tests for deleted features and fix remaining failures ([b056ec8](https://github.com/outcomesinsights/sequel-duckdb/commit/b056ec89ddc48957a35f8ecf302eddc07e47b959))
|
|
20
45
|
|
|
21
46
|
## [0.1.0] - 2025-07-21
|
|
22
47
|
|
|
23
48
|
### Added
|
|
49
|
+
|
|
24
50
|
- Initial release of Sequel DuckDB adapter
|
|
25
51
|
- Complete Database and Dataset class implementation
|
|
26
52
|
- Connection management for file-based and in-memory databases
|
|
@@ -43,6 +69,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
43
69
|
- Performance tuning guide
|
|
44
70
|
|
|
45
71
|
### Database Features
|
|
72
|
+
|
|
46
73
|
- File-based database support with automatic creation
|
|
47
74
|
- In-memory database support for testing and temporary data
|
|
48
75
|
- Connection validation and automatic reconnection
|
|
@@ -50,6 +77,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
50
77
|
- Support for DuckDB configuration options (memory_limit, threads, etc.)
|
|
51
78
|
|
|
52
79
|
### SQL Generation
|
|
80
|
+
|
|
53
81
|
- Complete SQL generation for all standard operations
|
|
54
82
|
- DuckDB-optimized query generation
|
|
55
83
|
- Support for complex queries with JOINs, subqueries, and CTEs
|
|
@@ -58,6 +86,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
58
86
|
- Parameter binding for prepared statements
|
|
59
87
|
|
|
60
88
|
### Schema Operations
|
|
89
|
+
|
|
61
90
|
- Table creation, modification, and deletion
|
|
62
91
|
- Column operations (add, drop, modify, rename)
|
|
63
92
|
- Index management (create, drop, unique, partial indexes)
|
|
@@ -66,6 +95,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
66
95
|
- Schema introspection with detailed metadata
|
|
67
96
|
|
|
68
97
|
### Data Types
|
|
98
|
+
|
|
69
99
|
- Complete Ruby ↔ DuckDB type mapping
|
|
70
100
|
- Support for all standard SQL types
|
|
71
101
|
- DuckDB-specific types (JSON, ARRAY, MAP)
|
|
@@ -75,6 +105,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
75
105
|
- UUID type support
|
|
76
106
|
|
|
77
107
|
### Performance Features
|
|
108
|
+
|
|
78
109
|
- Columnar storage optimization awareness
|
|
79
110
|
- Vectorized execution support
|
|
80
111
|
- Memory-efficient result set processing
|
|
@@ -84,6 +115,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
84
115
|
- Streaming result sets for large datasets
|
|
85
116
|
|
|
86
117
|
### Error Handling
|
|
118
|
+
|
|
87
119
|
- Comprehensive error mapping to Sequel exceptions
|
|
88
120
|
- Detailed error messages with context
|
|
89
121
|
- Proper handling of constraint violations
|
|
@@ -92,6 +124,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
92
124
|
- Database-specific error categorization
|
|
93
125
|
|
|
94
126
|
### Testing
|
|
127
|
+
|
|
95
128
|
- Complete test suite using Minitest
|
|
96
129
|
- Mock database testing for SQL generation
|
|
97
130
|
- Integration testing with real DuckDB databases
|
|
@@ -101,6 +134,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
101
134
|
- Data type conversion testing
|
|
102
135
|
|
|
103
136
|
### Documentation
|
|
137
|
+
|
|
104
138
|
- Comprehensive README with usage examples
|
|
105
139
|
- Complete API documentation with YARD
|
|
106
140
|
- Migration examples and patterns
|
|
@@ -109,12 +143,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
109
143
|
- Version compatibility matrix
|
|
110
144
|
|
|
111
145
|
### Dependencies
|
|
146
|
+
|
|
112
147
|
- Ruby 3.1.0+ support
|
|
113
148
|
- Sequel 5.0+ compatibility
|
|
114
149
|
- DuckDB 0.8.0+ support
|
|
115
150
|
- ruby-duckdb 1.0.0+ integration
|
|
116
151
|
|
|
117
152
|
### Fixed
|
|
153
|
+
|
|
118
154
|
- Proper adapter registration with Sequel
|
|
119
155
|
- Connection string parsing for file paths
|
|
120
156
|
- Memory management for large result sets
|
|
@@ -124,6 +160,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
124
160
|
- Error message formatting and context
|
|
125
161
|
|
|
126
162
|
### Security
|
|
163
|
+
|
|
127
164
|
- SQL injection prevention through parameter binding
|
|
128
165
|
- Proper identifier quoting
|
|
129
166
|
- Connection string sanitization
|
data/FINAL_STATUS.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Final Refactoring Status
|
|
2
|
+
|
|
3
|
+
## Achievement
|
|
4
|
+
|
|
5
|
+
**Removed 1,471 lines (54% reduction)**
|
|
6
|
+
|
|
7
|
+
- Original: 2,741 lines
|
|
8
|
+
- Current: 1,270 lines
|
|
9
|
+
- Test pass rate: 93.3% (503/539)
|
|
10
|
+
|
|
11
|
+
## What Was Removed (1,471 lines)
|
|
12
|
+
|
|
13
|
+
1. **Custom logging** (~80 lines) - Uses `log_connection_yield` now
|
|
14
|
+
2. **Custom error handling** (~80 lines) - Uses `DATABASE_ERROR_REGEXPS`
|
|
15
|
+
3. **Execution complexity** (~140 lines) - Moved to real adapter, simplified
|
|
16
|
+
4. **Transaction over-engineering** (~190 lines) - Savepoints, isolation levels
|
|
17
|
+
5. **Performance config** (~100 lines) - Query analysis, optimization methods
|
|
18
|
+
6. **SQL generation bloat** (~350 lines) - INSERT, UPDATE, DELETE, JOIN, WHERE, etc.
|
|
19
|
+
7. **Helper methods** (~50 lines) - table_name_sql, validate_table_name_for_select
|
|
20
|
+
8. **Performance optimizations** (~400 lines) - Batching, streaming, index hints
|
|
21
|
+
9. **Documentation** (~81 lines) - Excessive @example tags
|
|
22
|
+
|
|
23
|
+
## What Remains (1,270 lines)
|
|
24
|
+
|
|
25
|
+
### Real Adapter (327 lines)
|
|
26
|
+
|
|
27
|
+
- Connection management (70 lines)
|
|
28
|
+
- Execution methods (\_execute, execute, execute_dui, execute_insert) (60 lines)
|
|
29
|
+
- Dataset#fetch_rows (10 lines)
|
|
30
|
+
- Documentation (187 lines)
|
|
31
|
+
|
|
32
|
+
### Shared Adapter (943 lines)
|
|
33
|
+
|
|
34
|
+
- **Schema introspection** (200 lines) - Essential for Sequel
|
|
35
|
+
|
|
36
|
+
- schema_parse_table, schema_parse_indexes
|
|
37
|
+
- tables, schema, indexes methods
|
|
38
|
+
|
|
39
|
+
- **Type mapping** (150 lines) - DuckDB-specific
|
|
40
|
+
|
|
41
|
+
- map_duckdb_type_to_sequel
|
|
42
|
+
- type_literal, typecast_value
|
|
43
|
+
- parse_default_value
|
|
44
|
+
|
|
45
|
+
- **Configuration** (80 lines) - User-facing API
|
|
46
|
+
|
|
47
|
+
- set_pragma, configure_duckdb
|
|
48
|
+
- table_exists?, schema_exists?
|
|
49
|
+
|
|
50
|
+
- **Schema management** (80 lines) - Tested, functional
|
|
51
|
+
|
|
52
|
+
- create_schema, drop_schema
|
|
53
|
+
- schemas list
|
|
54
|
+
|
|
55
|
+
- **SQL generation overrides** (150 lines) - DuckDB-specific
|
|
56
|
+
|
|
57
|
+
- complex_expression_sql_append (LIKE, ILIKE, regex)
|
|
58
|
+
- literal methods (date, time, boolean, blob)
|
|
59
|
+
|
|
60
|
+
- **Reserved words & identifiers** (50 lines) - Database-specific
|
|
61
|
+
|
|
62
|
+
- **Feature detection** (30 lines) - supports\_\* methods
|
|
63
|
+
|
|
64
|
+
- **Documentation** (203 lines)
|
|
65
|
+
|
|
66
|
+
## Why Not 1,941 Lines Removed?
|
|
67
|
+
|
|
68
|
+
The analysis assumed more could be removed, but testing revealed:
|
|
69
|
+
|
|
70
|
+
1. **Schema introspection is essential** - Sequel models need this
|
|
71
|
+
2. **Type mapping is DuckDB-specific** - Can't use defaults
|
|
72
|
+
3. **Configuration methods are user-facing** - set_pragma, configure_duckdb
|
|
73
|
+
4. **Schema management is tested** - create_schema, drop_schema work well
|
|
74
|
+
5. **Literal methods handle DuckDB formats** - Dates, times, blobs differ from standard SQL
|
|
75
|
+
|
|
76
|
+
## Comparison to Analysis Target
|
|
77
|
+
|
|
78
|
+
- **Analysis target:** ~800 lines (1,941 removed)
|
|
79
|
+
- **Actual result:** 1,270 lines (1,471 removed)
|
|
80
|
+
- **Gap:** 470 lines
|
|
81
|
+
- **Reason:** Essential functionality that can't be removed without breaking features
|
|
82
|
+
|
|
83
|
+
## Benefits Achieved
|
|
84
|
+
|
|
85
|
+
✅ Follows Sequel conventions (SQLite pattern)
|
|
86
|
+
✅ Uses battle-tested `log_connection_yield`, `raise_error`
|
|
87
|
+
✅ Declarative error classification (DATABASE_ERROR_REGEXPS)
|
|
88
|
+
✅ Removed premature optimization\
|
|
89
|
+
✅ Removed unsupported features (savepoints, isolation levels)
|
|
90
|
+
✅ Simplified execution path
|
|
91
|
+
✅ 54% code reduction while maintaining 93% test compatibility
|
|
92
|
+
|
|
93
|
+
## Test Status
|
|
94
|
+
|
|
95
|
+
- **Passing:** 503/539 (93.3%)
|
|
96
|
+
- **Failures:** 24 (mostly message format expectations)
|
|
97
|
+
- **Errors:** 12 (mostly deleted method references in tests)
|
|
98
|
+
|
|
99
|
+
The adapter is significantly simpler, more maintainable, and follows Sequel patterns. The remaining code is essential functionality that provides value to users.
|
data/LICENSE
CHANGED
data/MIGRATION_EXAMPLES.md
CHANGED
|
@@ -737,4 +737,4 @@ chmod +x bin/migrate
|
|
|
737
737
|
./bin/migrate create add_user_preferences
|
|
738
738
|
```
|
|
739
739
|
|
|
740
|
-
This comprehensive guide covers the most common migration patterns and DuckDB-specific considerations when using Sequel migrations. Remember to always test your migrations thoroughly and keep them reversible for safe deployment practices.
|
|
740
|
+
This comprehensive guide covers the most common migration patterns and DuckDB-specific considerations when using Sequel migrations. Remember to always test your migrations thoroughly and keep them reversible for safe deployment practices.
|
|
@@ -18,16 +18,19 @@ This guide provides comprehensive strategies for optimizing performance when usi
|
|
|
18
18
|
DuckDB is designed as an analytical database with several key characteristics that affect performance optimization:
|
|
19
19
|
|
|
20
20
|
### Columnar Storage
|
|
21
|
+
|
|
21
22
|
- Data is stored column-wise, making analytical queries very efficient
|
|
22
23
|
- SELECT queries that access few columns are much faster
|
|
23
24
|
- Aggregations and analytical functions are highly optimized
|
|
24
25
|
|
|
25
26
|
### Vectorized Execution
|
|
27
|
+
|
|
26
28
|
- Operations are performed on batches of data (vectors) rather than row-by-row
|
|
27
29
|
- This reduces function call overhead and improves CPU cache utilization
|
|
28
30
|
- Particularly beneficial for analytical workloads
|
|
29
31
|
|
|
30
32
|
### In-Memory Processing
|
|
33
|
+
|
|
31
34
|
- DuckDB can efficiently process data that fits in memory
|
|
32
35
|
- Automatic memory management with spill-to-disk for larger datasets
|
|
33
36
|
- Memory-mapped files for efficient file-based database access
|
|
@@ -720,4 +723,4 @@ end
|
|
|
720
723
|
load_test(db)
|
|
721
724
|
```
|
|
722
725
|
|
|
723
|
-
This comprehensive performance optimization guide should help you get the most out of DuckDB's analytical capabilities while using Sequel. Remember that DuckDB excels at analytical workloads, so design your queries and schema to take advantage of its columnar storage and vectorized execution engine.
|
|
726
|
+
This comprehensive performance optimization guide should help you get the most out of DuckDB's analytical capabilities while using Sequel. Remember that DuckDB excels at analytical workloads, so design your queries and schema to take advantage of its columnar storage and vectorized execution engine.
|