sequel-duckdb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. checksums.yaml +7 -0
  2. data/.kiro/specs/advanced-sql-features-implementation/design.md +24 -0
  3. data/.kiro/specs/advanced-sql-features-implementation/requirements.md +43 -0
  4. data/.kiro/specs/advanced-sql-features-implementation/tasks.md +24 -0
  5. data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +258 -0
  6. data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +84 -0
  7. data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +94 -0
  8. data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +32 -0
  9. data/.kiro/specs/integration-test-database-setup/design.md +0 -0
  10. data/.kiro/specs/integration-test-database-setup/requirements.md +117 -0
  11. data/.kiro/specs/sequel-duckdb-adapter/design.md +542 -0
  12. data/.kiro/specs/sequel-duckdb-adapter/requirements.md +202 -0
  13. data/.kiro/specs/sequel-duckdb-adapter/tasks.md +247 -0
  14. data/.kiro/specs/sql-expression-handling-fix/design.md +298 -0
  15. data/.kiro/specs/sql-expression-handling-fix/requirements.md +86 -0
  16. data/.kiro/specs/sql-expression-handling-fix/tasks.md +22 -0
  17. data/.kiro/specs/test-infrastructure-improvements/requirements.md +106 -0
  18. data/.kiro/steering/product.md +22 -0
  19. data/.kiro/steering/structure.md +88 -0
  20. data/.kiro/steering/tech.md +124 -0
  21. data/.kiro/steering/testing.md +192 -0
  22. data/.rubocop.yml +103 -0
  23. data/.yardopts +8 -0
  24. data/API_DOCUMENTATION.md +919 -0
  25. data/CHANGELOG.md +131 -0
  26. data/LICENSE +21 -0
  27. data/MIGRATION_EXAMPLES.md +740 -0
  28. data/PERFORMANCE_OPTIMIZATIONS.md +723 -0
  29. data/README.md +692 -0
  30. data/Rakefile +27 -0
  31. data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +164 -0
  32. data/docs/DUCKDB_SQL_PATTERNS.md +410 -0
  33. data/docs/TASK_12_VERIFICATION_SUMMARY.md +122 -0
  34. data/lib/sequel/adapters/duckdb.rb +256 -0
  35. data/lib/sequel/adapters/shared/duckdb.rb +2349 -0
  36. data/lib/sequel/duckdb/version.rb +16 -0
  37. data/lib/sequel/duckdb.rb +43 -0
  38. data/sig/sequel/duckdb.rbs +6 -0
  39. metadata +235 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: eb4c8565280c6828b7fd8122ca87de1867da66e4cc5c7f4b640a0b34ee1880ac
4
+ data.tar.gz: c415c11256156a1b21e6b5f02268e2f22a786f3ce240e17596d6ba0e61f5a460
5
+ SHA512:
6
+ metadata.gz: d282d7015f97c66fa0a9581552be9ff189b07dbf7678dca94a8990070d42f66335f2359056ca9fbeed4e8b1a89da1afc03884a0ba8994ecda23fc9e1282d5048
7
+ data.tar.gz: 237ea21f9f3a6d2d148b85e9f4f0338864efe2055b426c5a169ff2d2e3775f54f91957a10e061869066c414921abfc1fd0ae15c375090573e9bd1aa38575b7fd
@@ -0,0 +1,24 @@
1
+ # Design Document: Advanced SQL Features Implementation
2
+
3
+ ## Overview
4
+
5
+ Most advanced SQL features are already implemented. This focuses on verification, testing, and minor enhancements.
6
+
7
+ **Status:**
8
+ - JOIN USING, recursive CTEs, set operations, error handling: ✅ Implemented
9
+ - Window functions: ✅ Basic support exists
10
+ - DuckDB configuration: ✅ Basic PRAGMA support exists
11
+
12
+ **Remaining Work:**
13
+ - Test window functions with actual DuckDB
14
+ - Test advanced expressions (arrays, JSON)
15
+ - Add user-friendly configuration methods
16
+ - Integration testing
17
+
18
+ ## Implementation Approach
19
+
20
+ 1. **Test existing functionality** - Verify implemented features work correctly
21
+ 2. **Add convenience methods** - User-friendly configuration interface
22
+ 3. **Integration testing** - Test with actual DuckDB databases
23
+
24
+ This is primarily testing and verification, not new feature implementation.
@@ -0,0 +1,43 @@
1
+ # Requirements Document
2
+
3
+ ## Introduction
4
+
5
+ Most advanced SQL features are already implemented. This focuses on remaining verification and testing needs.
6
+
7
+ ## Requirements
8
+
9
+ ### Requirement 1: Window Function Verification
10
+
11
+ **User Story:** As a developer, I want to verify window functions work correctly with DuckDB.
12
+
13
+ #### Acceptance Criteria
14
+
15
+ 1. WHEN I use window functions THEN they SHALL generate correct SQL and execute properly
16
+ 2. WHEN I use LAG/LEAD functions THEN they SHALL support offset and default parameters
17
+
18
+ ### Requirement 2: Advanced Expression Testing
19
+
20
+ **User Story:** As a developer, I want to test advanced DuckDB expressions work correctly.
21
+
22
+ #### Acceptance Criteria
23
+
24
+ 1. WHEN I use array syntax THEN it SHALL use DuckDB's `[1, 2, 3]` format
25
+ 2. WHEN I use JSON functions THEN they SHALL work with DuckDB's JSON support
26
+
27
+ ### Requirement 3: Configuration Interface Enhancement
28
+
29
+ **User Story:** As a developer, I want user-friendly configuration methods.
30
+
31
+ #### Acceptance Criteria
32
+
33
+ 1. WHEN I use `db.set_pragma(key, value)` THEN it SHALL execute PRAGMA statements
34
+ 2. WHEN I use `db.configure_duckdb(options)` THEN it SHALL apply multiple settings
35
+
36
+ ### Requirement 4: Integration Testing
37
+
38
+ **User Story:** As a developer, I want comprehensive integration tests.
39
+
40
+ #### Acceptance Criteria
41
+
42
+ 1. WHEN I run tests THEN all advanced features SHALL work with actual DuckDB databases
43
+ 2. WHEN errors occur THEN they SHALL be properly mapped to Sequel exceptions
@@ -0,0 +1,24 @@
1
+ # Implementation Plan
2
+
3
+ Most advanced SQL features are already implemented. Remaining tasks based on current code analysis:
4
+
5
+ - [x] 1. Test window functions
6
+ - ~~Add tests to `test/dataset_test.rb` for ROW_NUMBER, RANK, DENSE_RANK~~ ✅ Already implemented
7
+ - ~~Test LAG/LEAD functions with offset and default parameters~~ ✅ Already implemented
8
+ - _Requirements: 1.1, 1.2_
9
+
10
+ - [x] 2. Test advanced expressions
11
+ - Add tests to `test/sql_test.rb` for DuckDB array syntax `[1, 2, 3]`
12
+ - Test JSON functions like `json_extract`
13
+ - _Requirements: 2.1, 2.2_
14
+
15
+ - [x] 3. Add configuration convenience methods
16
+ - Add `set_pragma(key, value)` method to `DatabaseMethods` (user-friendly wrapper)
17
+ - Add `configure_duckdb(options)` method for batch configuration
18
+ - Add tests to `test/database_test.rb`
19
+ - _Requirements: 3.1, 3.2_
20
+
21
+ - [x] 4. Integration testing
22
+ - ~~Add integration tests to existing test files using actual DuckDB databases~~ ✅ Already implemented
23
+ - ~~Test that all advanced features work together correctly~~ ✅ Already implemented
24
+ - _Requirements: 4.1, 4.2_
@@ -0,0 +1,258 @@
1
+ # Design Document: DuckDB SQL Syntax Compatibility
2
+
3
+ ## Overview
4
+
5
+ This design addresses SQL generation issues in the sequel-duckdb adapter where the adapter is generating non-standard SQL syntax that doesn't match Sequel's established conventions. The issues include unwanted ESCAPE clauses in LIKE statements, missing parentheses in complex expressions, incorrect qualified column reference format, and improper subquery column references.
6
+
7
+ The solution is to fix the adapter's SQL generation methods to produce clean, standard SQL that follows Sequel's patterns while being fully compatible with DuckDB. This ensures consistent SQL generation that matches Sequel's conventions across all operations including LIKE clauses, complex expressions, qualified identifiers, regex operations, and subquery column references.
8
+
9
+ ## Architecture
10
+
11
+ ### Design Philosophy
12
+
13
+ 1. **Fix Root Causes**: Address SQL generation issues in the adapter rather than working around them in tests
14
+ 2. **Standard SQL Generation**: Generate clean, standard SQL that follows Sequel's established patterns
15
+ 3. **Minimal Adapter Changes**: Make targeted fixes to specific SQL generation methods
16
+
17
+ ### Current Structure (Targeted Fixes)
18
+
19
+ ```
20
+ lib/sequel/adapters/
21
+ ├── duckdb.rb # Main adapter (unchanged)
22
+ └── shared/
23
+ └── duckdb.rb # DatasetMethods (fix SQL generation methods)
24
+
25
+ test/
26
+ ├── sql_test.rb # Fix dataset creation issues
27
+ ├── dataset_test.rb # Tests should pass with fixed adapter
28
+ └── spec_helper.rb # No changes needed
29
+ ```
30
+
31
+ ## Components and Interfaces
32
+
33
+ ### 1. LIKE Clause Generation Fix
34
+
35
+ **Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
36
+
37
+ Override LIKE handling to prevent unwanted ESCAPE clauses:
38
+
39
+ ```ruby
40
+ def complex_expression_sql_append(sql, op, args)
41
+ case op
42
+ when :LIKE
43
+ # Generate clean LIKE without ESCAPE clause
44
+ sql << "("
45
+ literal_append(sql, args[0])
46
+ sql << " LIKE "
47
+ literal_append(sql, args[1])
48
+ sql << ")"
49
+ when :ILIKE
50
+ # Convert ILIKE to UPPER() LIKE UPPER() with proper parentheses
51
+ sql << "(UPPER("
52
+ literal_append(sql, args[0])
53
+ sql << ") LIKE UPPER("
54
+ literal_append(sql, args[1])
55
+ sql << "))"
56
+ when :~
57
+ # Regex with proper parentheses
58
+ sql << "("
59
+ literal_append(sql, args[0])
60
+ sql << " ~ "
61
+ literal_append(sql, args[1])
62
+ sql << ")"
63
+ else
64
+ super
65
+ end
66
+ end
67
+ ```
68
+
69
+ ### 2. Table Alias Generation Fix
70
+
71
+ **Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
72
+
73
+ Override alias handling to use standard AS syntax:
74
+
75
+ ```ruby
76
+ def table_alias_sql_append(sql, table, alias_name)
77
+ sql << table.to_s
78
+ sql << " AS "
79
+ sql << alias_name.to_s
80
+ end
81
+ ```
82
+
83
+ ### 3. Qualified Column Reference Fix
84
+
85
+ **Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
86
+
87
+ Override qualified identifier handling to use dot notation:
88
+
89
+ ```ruby
90
+ def qualified_identifier_sql_append(sql, table, column)
91
+ sql << table.to_s
92
+ sql << "."
93
+ sql << column.to_s
94
+ end
95
+ ```
96
+
97
+ ### 4. Subquery Column Reference Fix
98
+
99
+ **Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
100
+
101
+ Ensure subqueries properly reference outer query columns using dot notation:
102
+
103
+ ```ruby
104
+ def subquery_sql_append(sql, ds)
105
+ # Ensure subqueries use proper column references
106
+ sql << "("
107
+ subquery_sql_append_sql(sql, ds.sql)
108
+ sql << ")"
109
+ end
110
+
111
+ def exists_sql_append(sql, ds)
112
+ # EXISTS subqueries with proper column references
113
+ sql << "EXISTS ("
114
+ subquery_sql_append_sql(sql, ds.sql)
115
+ sql << ")"
116
+ end
117
+ ```
118
+
119
+ ### 5. SQL Test Infrastructure Fix
120
+
121
+ **Location**: `test/sql_test.rb`
122
+
123
+ Fix dataset creation issues in SQL tests:
124
+
125
+ ```ruby
126
+ class SqlTest < SequelDuckDBTest::TestCase
127
+ def setup
128
+ super
129
+ # Use proper mock dataset creation instead of subclasses
130
+ @dataset = mock_dataset(:users)
131
+ end
132
+
133
+ def test_like_clause_generation
134
+ dataset = @dataset.where(Sequel.like(:name, "%John%"))
135
+ expected_sql = "SELECT * FROM users WHERE (name LIKE '%John%')"
136
+ assert_sql expected_sql, dataset
137
+ end
138
+
139
+ def test_ilike_clause_generation
140
+ dataset = @dataset.where(Sequel.ilike(:name, "%john%"))
141
+ expected_sql = "SELECT * FROM users WHERE (UPPER(name) LIKE UPPER('%john%'))"
142
+ assert_sql expected_sql, dataset
143
+ end
144
+
145
+ def test_subquery_column_references
146
+ subquery = @dataset.select(:id).where(Sequel.qualify(:users, :active) => true)
147
+ dataset = @dataset.where(id: subquery)
148
+ expected_sql = "SELECT * FROM users WHERE (id IN (SELECT id FROM users WHERE (users.active = true)))"
149
+ assert_sql expected_sql, dataset
150
+ end
151
+ end
152
+ ```
153
+
154
+ ## Data Models
155
+
156
+ No new data models needed. The existing adapter structure handles SQL generation correctly.
157
+
158
+ ## Error Handling
159
+
160
+ Use existing Sequel error handling patterns. No special error handling needed for syntax variations since they are all valid SQL.
161
+
162
+ ## Testing Strategy
163
+
164
+ ### 1. SQL Generation Unit Tests
165
+ - Test all SQL generation methods using Sequel's mock database functionality
166
+ - Verify exact SQL syntax matches expected Sequel patterns
167
+ - Test LIKE clauses generate clean SQL without ESCAPE clauses
168
+ - Test complex expressions have proper parentheses
169
+ - Test qualified column references use dot notation
170
+ - Test regex expressions are properly formatted and parenthesized
171
+ - Test subquery column references use standard SQL format
172
+
173
+ ### 2. Integration Tests
174
+ - Ensure actual database operations work correctly with generated SQL
175
+ - Verify functional correctness alongside syntactic correctness
176
+ - Test correlated subqueries with proper column references
177
+ - Test complex queries with multiple SQL generation features
178
+
179
+ ### 3. Test Infrastructure Consistency
180
+ - Fix SQL test infrastructure to use proper dataset creation
181
+ - Ensure tests expect standard SQL syntax consistently
182
+ - Maintain comprehensive test coverage for all SQL generation patterns
183
+ - Use Test-Driven Development approach for all fixes
184
+
185
+ ## Design Decisions and Rationales
186
+
187
+ ### 1. Fix Adapter SQL Generation
188
+ **Decision**: Fix the root cause SQL generation issues in the adapter
189
+ **Rationale**: The adapter is generating non-standard SQL that doesn't follow Sequel conventions. Tests are correct to expect standard SQL.
190
+
191
+ ### 2. Targeted Method Overrides
192
+ **Decision**: Override specific SQL generation methods in DatasetMethods
193
+ **Rationale**: Surgical fixes to specific issues without disrupting the overall adapter architecture.
194
+
195
+ ### 3. Standard SQL Compliance
196
+ **Decision**: Generate SQL that follows standard SQL and Sequel conventions
197
+ **Rationale**: Ensures compatibility with existing Sequel patterns and makes the adapter more predictable.
198
+
199
+ ### 4. Maintain Test Coverage
200
+ **Decision**: Keep all existing tests, fix the adapter to make them pass
201
+ **Rationale**: Tests are validating correct behavior; the adapter should conform to expected patterns.
202
+
203
+ ## Implementation Phases
204
+
205
+ ### Phase 1: Fix LIKE and Complex Expression Generation
206
+ - Override `complex_expression_sql_append` to fix LIKE, ILIKE, and regex generation
207
+ - Add proper parentheses to all complex expressions
208
+ - Remove unwanted ESCAPE clauses from LIKE statements
209
+
210
+ ### Phase 2: Fix Table Alias Generation
211
+ - Override table alias methods to use standard `AS` syntax
212
+ - Ensure aliases work correctly in JOIN operations
213
+
214
+ ### Phase 3: Fix Qualified Column References
215
+ - Override qualified identifier methods to use dot notation
216
+ - Ensure subqueries use proper column references
217
+
218
+ ### Phase 4: Fix SQL Test Infrastructure
219
+ - Fix dataset creation issues in SQL tests
220
+ - Ensure tests use proper mock datasets
221
+
222
+ ### Phase 5: Verification and Documentation
223
+ - Run all tests to ensure they pass with fixed adapter
224
+ - Document the SQL generation patterns used by the adapter
225
+ - Create comprehensive documentation of DuckDB-specific SQL patterns
226
+ - Update API documentation with SQL generation examples
227
+
228
+ ## Documentation Strategy
229
+
230
+ ### SQL Pattern Documentation
231
+ To address Requirement 7, the adapter will include comprehensive documentation of SQL generation patterns:
232
+
233
+ **Location**: `API_DOCUMENTATION.md` and inline code comments
234
+
235
+ 1. **LIKE Clause Patterns**: Document how LIKE clauses are generated without ESCAPE clauses
236
+ 2. **Complex Expression Formatting**: Document parentheses usage in ILIKE and regex expressions
237
+ 3. **Qualified Column References**: Document dot notation usage for table.column references
238
+ 4. **Subquery Column References**: Document proper column referencing in correlated subqueries
239
+ 5. **DuckDB-Specific Optimizations**: Document any DuckDB-specific SQL optimizations used
240
+
241
+ **Example Documentation Structure**:
242
+ ```ruby
243
+ # SQL Generation Patterns for DuckDB Adapter
244
+ #
245
+ # LIKE Clauses:
246
+ # Input: Sequel.like(:name, "%John%")
247
+ # Output: (name LIKE '%John%')
248
+ #
249
+ # ILIKE Clauses:
250
+ # Input: Sequel.ilike(:name, "%john%")
251
+ # Output: (UPPER(name) LIKE UPPER('%john%'))
252
+ #
253
+ # Qualified Columns:
254
+ # Input: Sequel.qualify(:users, :id)
255
+ # Output: users.id
256
+ ```
257
+
258
+ This design fixes the root causes of the SQL generation issues rather than working around them, resulting in a more robust and standards-compliant adapter.
@@ -0,0 +1,84 @@
1
+ # Requirements Document
2
+
3
+ ## Introduction
4
+
5
+ This specification addresses SQL generation issues in the sequel-duckdb adapter where the adapter is generating non-standard SQL syntax that doesn't match expected Sequel conventions. The adapter should generate clean, standard SQL that follows Sequel's established patterns while being compatible with DuckDB. The issues are in the adapter's SQL generation logic, not in DuckDB's SQL support.
6
+
7
+ ## Requirements
8
+
9
+ ### Requirement 1: LIKE Clause Clean Generation
10
+
11
+ **User Story:** As a developer using Sequel with DuckDB, I want LIKE clauses to generate clean SQL without unnecessary ESCAPE clauses, so that the SQL matches standard Sequel patterns.
12
+
13
+ #### Acceptance Criteria
14
+
15
+ 1. WHEN I use `Sequel.like(:name, "%John%")` THEN the generated SQL SHALL be `(name LIKE '%John%')` without ESCAPE clause
16
+ 2. WHEN I use LIKE patterns with special characters THEN they SHALL work without requiring explicit ESCAPE clauses
17
+ 3. WHEN I use ILIKE patterns THEN they SHALL be converted to `(UPPER(column) LIKE UPPER(pattern))` with proper parentheses
18
+ 4. WHEN tests check LIKE clause generation THEN they SHALL expect clean SQL without ESCAPE additions
19
+
20
+ ### Requirement 2: Complex Expression Parentheses
21
+
22
+ **User Story:** As a developer using Sequel with DuckDB, I want complex expressions to be properly parenthesized in generated SQL, so that the SQL follows standard Sequel formatting conventions.
23
+
24
+ #### Acceptance Criteria
25
+
26
+ 1. WHEN I use ILIKE expressions THEN they SHALL be wrapped in parentheses: `(UPPER(column) LIKE UPPER(pattern))`
27
+ 2. WHEN I use regex expressions THEN they SHALL be wrapped in parentheses: `(column ~ 'pattern')`
28
+ 3. WHEN I use boolean comparisons with `=~` THEN they SHALL generate proper `IS` syntax with parentheses
29
+ 4. WHEN tests check complex expressions THEN they SHALL expect properly parenthesized SQL
30
+
31
+ ### Requirement 3: Standard Qualified Column References
32
+
33
+ **User Story:** As a developer using Sequel with DuckDB, I want qualified column references to use standard SQL dot notation, so that the generated SQL follows established SQL conventions.
34
+
35
+ #### Acceptance Criteria
36
+
37
+ 1. WHEN I reference columns across tables THEN they SHALL use `table.column` syntax
38
+ 2. WHEN I use qualified column names in subqueries THEN they SHALL use proper dot notation
39
+ 3. WHEN I use schema-qualified names THEN they SHALL use standard SQL format
40
+ 4. WHEN tests check qualified identifiers THEN they SHALL expect standard dot notation
41
+
42
+ ### Requirement 4: Proper Regular Expression Formatting
43
+
44
+ **User Story:** As a developer using Sequel with DuckDB, I want regular expression matching to generate properly formatted SQL with parentheses, so that the SQL follows Sequel's formatting conventions.
45
+
46
+ #### Acceptance Criteria
47
+
48
+ 1. WHEN I use regex matching with `~` operator THEN it SHALL generate `(column ~ 'pattern')` with parentheses
49
+ 2. WHEN I use case-insensitive regex THEN it SHALL be properly formatted with parentheses
50
+ 3. WHEN I use complex regex patterns THEN they SHALL be properly formatted and parenthesized
51
+ 4. WHEN tests check regex syntax THEN they SHALL expect properly parenthesized expressions
52
+
53
+ ### Requirement 5: Standard Subquery Column References
54
+
55
+ **User Story:** As a developer using Sequel with DuckDB, I want subqueries to properly reference outer query columns using standard SQL dot notation, so that correlated subqueries follow SQL conventions.
56
+
57
+ #### Acceptance Criteria
58
+
59
+ 1. WHEN I use correlated subqueries THEN column references SHALL use `table.column` syntax
60
+ 2. WHEN I reference outer query columns THEN they SHALL be properly qualified with dot notation
61
+ 3. WHEN I use EXISTS subqueries THEN column references SHALL use standard SQL format
62
+ 4. WHEN tests check subquery references THEN they SHALL expect standard dot notation
63
+
64
+ ### Requirement 6: Consistent SQL Generation Testing
65
+
66
+ **User Story:** As a developer maintaining the sequel-duckdb adapter, I want tests to verify that the adapter generates consistent, standard SQL, so that the adapter follows Sequel's established patterns.
67
+
68
+ #### Acceptance Criteria
69
+
70
+ 1. WHEN tests check SQL generation THEN they SHALL expect standard SQL syntax
71
+ 2. WHEN the adapter generates SQL THEN it SHALL be consistent with Sequel's conventions
72
+ 3. WHEN SQL generation issues are found THEN they SHALL be fixed in the adapter, not worked around in tests
73
+ 4. WHEN SQL correctness is verified THEN both functional and syntactic correctness SHALL be maintained
74
+
75
+ ### Requirement 7: Documentation of SQL Generation Patterns
76
+
77
+ **User Story:** As a developer using the sequel-duckdb adapter, I want to understand how the adapter generates SQL for DuckDB, so that I can write queries that work optimally with the adapter.
78
+
79
+ #### Acceptance Criteria
80
+
81
+ 1. WHEN the adapter generates specific SQL patterns THEN they SHALL be documented
82
+ 2. WHEN SQL generation differs from other Sequel adapters THEN the differences SHALL be explained
83
+ 3. WHEN DuckDB-specific optimizations are used THEN they SHALL be documented with examples
84
+ 4. WHEN SQL generation patterns change THEN documentation SHALL be updated accordingly
@@ -0,0 +1,94 @@
1
+ # Implementation Plan
2
+
3
+ - [x] 1. Identify failing tests and analyze DuckDB SQL generation
4
+ - Run existing test suite to identify tests failing due to SQL syntax differences
5
+ - Analyze what SQL the adapter currently generates vs what tests expect
6
+ - Document the specific DuckDB syntax patterns that are being generated
7
+ - Determine if adapter changes are needed or just test expectation updates
8
+ - _Requirements: 7.1, 7.2_
9
+
10
+ - [x] 2. Fix LIKE clause ESCAPE handling in adapter
11
+ - Current issue: LIKE generates `(name LIKE '%John%' ESCAPE '\')` instead of `(name LIKE '%John%')`
12
+ - Override LIKE handling in `complex_expression_sql_append` to remove ESCAPE clause
13
+ - Ensure LIKE clauses generate clean `(name LIKE '%John%')` without ESCAPE clause
14
+ - Write tests to verify LIKE functionality works correctly without the ESCAPE clause
15
+ - _Requirements: 1.1, 1.2, 1.4_
16
+
17
+ - [x] 3. Fix parentheses in complex expression SQL generation
18
+ - Current issue: ILIKE generates `UPPER(name) LIKE UPPER('%john%')` without outer parentheses
19
+ - Current issue: Regex generates `name ~ 'pattern'` without outer parentheses
20
+ - Update `complex_expression_sql_append` to add proper parentheses around expressions
21
+ - Ensure ILIKE generates `(UPPER(name) LIKE UPPER('%john%'))` with parentheses
22
+ - Ensure regex generates `(name ~ '^John')` with parentheses around the expression
23
+ - Write tests to verify all complex expressions have consistent parentheses
24
+ - _Requirements: 1.3, 5.1, 5.2, 5.3, 5.4_
25
+
26
+ - [x] 4. ~~Fix table alias syntax to use AS instead of triple underscore~~ (REMOVED - Not a core Sequel feature)
27
+ - The `table___alias` syntax is not a core Sequel feature and requires an extension
28
+ - Removed implementation and tests as this is not standard Sequel functionality
29
+ - Standard Sequel table aliases use `.as()` method: `db[:users].as(:u)`
30
+
31
+ - [x] 5. Fix qualified column references to use dot notation
32
+ - Current issue: Qualified columns generate `users__id` instead of `users.id`
33
+ - Override qualified identifier handling to generate proper dot notation
34
+ - Implement `qualified_identifier_sql_append` method to use dot notation
35
+ - Ensure subqueries use correct `users.id` column reference format
36
+ - Write tests to verify qualified column references work correctly in complex queries
37
+ - _Requirements: 4.1, 4.2, 4.3, 4.4, 6.1, 6.2, 6.3, 6.4_
38
+
39
+ - [x] 6. Fix SQL test infrastructure issues
40
+ - Current issue: SQL tests failing with dataset type assertion errors (`Expected #<Sequel::Dataset::_Subclass>`)
41
+ - Fix `SqlTest` class to use proper dataset creation instead of mock subclasses
42
+ - Update SQL generation tests to work with actual DuckDB adapter behavior
43
+ - Ensure tests create proper datasets for SQL generation testing
44
+ - Fix dataset type assertion issues in SQL tests
45
+ - _Requirements: 6.1, 6.2_
46
+
47
+ - [x] 7. Fix recursive CTE SQL generation
48
+ - Current issue: `WITH RECURSIVE` generates incorrect SQL without RECURSIVE keyword
49
+ - Implement proper recursive CTE handling in dataset methods
50
+ - Ensure recursive CTEs generate `WITH RECURSIVE` syntax correctly
51
+ - Test recursive CTE functionality with proper SQL generation
52
+ - _Requirements: 5.1, 5.2, 5.3, 5.4_
53
+
54
+ - [x] 8. Fix JOIN USING clause generation
55
+ - Current issue: `JOIN USING` clause not generating USING syntax correctly
56
+ - Implement proper USING clause handling in JOIN operations
57
+ - Ensure JOIN USING generates correct `INNER JOIN table USING (column)` syntax
58
+ - Test JOIN USING functionality with various column combinations
59
+ - _Requirements: 4.1, 4.2, 4.3, 4.4_
60
+
61
+ - [x] 9. Fix regex functionality integration
62
+ - Current issue: Regex matching returning 0 results instead of expected matches
63
+ - Debug regex operator implementation in complex_expression_sql_append
64
+ - Ensure regex patterns work correctly with DuckDB's regex syntax
65
+ - Test regex functionality with actual data and pattern matching
66
+ - _Requirements: 1.3, 5.1, 5.2, 5.3, 5.4_
67
+
68
+ - [x] 10. Fix model integration issues
69
+ - Current issue: Model tests failing with type mapping and update detection
70
+ - Fix time type mapping (currently returning :datetime instead of :time)
71
+ - Fix model update detection to properly track changed fields
72
+ - Ensure model DELETE operations work correctly with proper ID handling
73
+ - _Requirements: 2.1, 2.2, 2.3, 2.4_
74
+
75
+ - [x] 11. Fix table creation and schema issues
76
+ - Current issue: Tests failing because tables don't exist during SQL execution
77
+ - Ensure proper table creation in test setup for integration tests
78
+ - Fix primary key handling for tables without explicit primary keys
79
+ - Handle NOT NULL constraint violations properly in INSERT operations
80
+ - _Requirements: 3.1, 3.2, 3.3, 3.4_
81
+
82
+ - [x] 12. Verify all tests pass with consistent DuckDB SQL generation
83
+ - Run complete test suite to ensure all SQL generation tests pass
84
+ - Verify that adapter generates consistent, predictable SQL for DuckDB
85
+ - Test integration scenarios to ensure functional correctness
86
+ - Fix any remaining test expectation mismatches
87
+ - _Requirements: 6.1, 6.2, 6.3, 6.4_
88
+
89
+ - [x] 13. Document DuckDB-specific SQL syntax patterns
90
+ - Document the specific SQL syntax that the adapter generates for DuckDB
91
+ - Provide examples of DuckDB SQL patterns used by the adapter
92
+ - Explain why certain DuckDB syntax choices were made
93
+ - Create reference documentation for developers using the adapter
94
+ - _Requirements: 7.1, 7.2, 7.3_
@@ -0,0 +1,32 @@
1
+ # Requirements Document
2
+
3
+ ## Introduction
4
+
5
+ After comprehensive review of the current sequel-duckdb adapter implementation and testing against the actual behavior, all previously identified edge cases and validation issues have been determined to be either:
6
+
7
+ 1. **Already implemented** - The adapter includes comprehensive error handling, data type conversion, SQL injection prevention through parameterized queries, connection management, and transaction support.
8
+
9
+ 2. **Handled by Sequel core** - Query parameter validation (LIMIT/OFFSET edge cases) is handled by the Sequel framework itself before reaching database adapters.
10
+
11
+ 3. **Working as designed** - DuckDB's native behavior for edge cases is appropriate and doesn't require additional adapter-level handling.
12
+
13
+ 4. **Out of scope** - Some edge cases (like system resource management) are better handled at the application or infrastructure level rather than in a database adapter.
14
+
15
+ ## Current Status
16
+
17
+ The sequel-duckdb adapter currently provides:
18
+
19
+ - ✅ Comprehensive error handling and mapping to appropriate Sequel exception types
20
+ - ✅ Robust connection management with proper error handling
21
+ - ✅ Complete data type conversion and validation
22
+ - ✅ SQL injection prevention through parameterized queries
23
+ - ✅ Transaction support with proper rollback handling
24
+ - ✅ Schema introspection and DDL operations
25
+ - ✅ Performance optimizations for large datasets
26
+ - ✅ Memory management for streaming operations
27
+
28
+ ## Conclusion
29
+
30
+ No additional edge case or validation requirements have been identified that would add meaningful value to the adapter. The current implementation provides robust handling of edge cases appropriate for a production database adapter.
31
+
32
+ This specification is considered complete with no outstanding requirements.