sequel-duckdb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.kiro/specs/advanced-sql-features-implementation/design.md +24 -0
- data/.kiro/specs/advanced-sql-features-implementation/requirements.md +43 -0
- data/.kiro/specs/advanced-sql-features-implementation/tasks.md +24 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +258 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +84 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +94 -0
- data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +32 -0
- data/.kiro/specs/integration-test-database-setup/design.md +0 -0
- data/.kiro/specs/integration-test-database-setup/requirements.md +117 -0
- data/.kiro/specs/sequel-duckdb-adapter/design.md +542 -0
- data/.kiro/specs/sequel-duckdb-adapter/requirements.md +202 -0
- data/.kiro/specs/sequel-duckdb-adapter/tasks.md +247 -0
- data/.kiro/specs/sql-expression-handling-fix/design.md +298 -0
- data/.kiro/specs/sql-expression-handling-fix/requirements.md +86 -0
- data/.kiro/specs/sql-expression-handling-fix/tasks.md +22 -0
- data/.kiro/specs/test-infrastructure-improvements/requirements.md +106 -0
- data/.kiro/steering/product.md +22 -0
- data/.kiro/steering/structure.md +88 -0
- data/.kiro/steering/tech.md +124 -0
- data/.kiro/steering/testing.md +192 -0
- data/.rubocop.yml +103 -0
- data/.yardopts +8 -0
- data/API_DOCUMENTATION.md +919 -0
- data/CHANGELOG.md +131 -0
- data/LICENSE +21 -0
- data/MIGRATION_EXAMPLES.md +740 -0
- data/PERFORMANCE_OPTIMIZATIONS.md +723 -0
- data/README.md +692 -0
- data/Rakefile +27 -0
- data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +164 -0
- data/docs/DUCKDB_SQL_PATTERNS.md +410 -0
- data/docs/TASK_12_VERIFICATION_SUMMARY.md +122 -0
- data/lib/sequel/adapters/duckdb.rb +256 -0
- data/lib/sequel/adapters/shared/duckdb.rb +2349 -0
- data/lib/sequel/duckdb/version.rb +16 -0
- data/lib/sequel/duckdb.rb +43 -0
- data/sig/sequel/duckdb.rbs +6 -0
- metadata +235 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: eb4c8565280c6828b7fd8122ca87de1867da66e4cc5c7f4b640a0b34ee1880ac
|
4
|
+
data.tar.gz: c415c11256156a1b21e6b5f02268e2f22a786f3ce240e17596d6ba0e61f5a460
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: d282d7015f97c66fa0a9581552be9ff189b07dbf7678dca94a8990070d42f66335f2359056ca9fbeed4e8b1a89da1afc03884a0ba8994ecda23fc9e1282d5048
|
7
|
+
data.tar.gz: 237ea21f9f3a6d2d148b85e9f4f0338864efe2055b426c5a169ff2d2e3775f54f91957a10e061869066c414921abfc1fd0ae15c375090573e9bd1aa38575b7fd
|
@@ -0,0 +1,24 @@
|
|
1
|
+
# Design Document: Advanced SQL Features Implementation
|
2
|
+
|
3
|
+
## Overview
|
4
|
+
|
5
|
+
Most advanced SQL features are already implemented. This focuses on verification, testing, and minor enhancements.
|
6
|
+
|
7
|
+
**Status:**
|
8
|
+
- JOIN USING, recursive CTEs, set operations, error handling: ✅ Implemented
|
9
|
+
- Window functions: ✅ Basic support exists
|
10
|
+
- DuckDB configuration: ✅ Basic PRAGMA support exists
|
11
|
+
|
12
|
+
**Remaining Work:**
|
13
|
+
- Test window functions with actual DuckDB
|
14
|
+
- Test advanced expressions (arrays, JSON)
|
15
|
+
- Add user-friendly configuration methods
|
16
|
+
- Integration testing
|
17
|
+
|
18
|
+
## Implementation Approach
|
19
|
+
|
20
|
+
1. **Test existing functionality** - Verify implemented features work correctly
|
21
|
+
2. **Add convenience methods** - User-friendly configuration interface
|
22
|
+
3. **Integration testing** - Test with actual DuckDB databases
|
23
|
+
|
24
|
+
This is primarily testing and verification, not new feature implementation.
|
@@ -0,0 +1,43 @@
|
|
1
|
+
# Requirements Document
|
2
|
+
|
3
|
+
## Introduction
|
4
|
+
|
5
|
+
Most advanced SQL features are already implemented. This focuses on remaining verification and testing needs.
|
6
|
+
|
7
|
+
## Requirements
|
8
|
+
|
9
|
+
### Requirement 1: Window Function Verification
|
10
|
+
|
11
|
+
**User Story:** As a developer, I want to verify window functions work correctly with DuckDB.
|
12
|
+
|
13
|
+
#### Acceptance Criteria
|
14
|
+
|
15
|
+
1. WHEN I use window functions THEN they SHALL generate correct SQL and execute properly
|
16
|
+
2. WHEN I use LAG/LEAD functions THEN they SHALL support offset and default parameters
|
17
|
+
|
18
|
+
### Requirement 2: Advanced Expression Testing
|
19
|
+
|
20
|
+
**User Story:** As a developer, I want to test advanced DuckDB expressions work correctly.
|
21
|
+
|
22
|
+
#### Acceptance Criteria
|
23
|
+
|
24
|
+
1. WHEN I use array syntax THEN it SHALL use DuckDB's `[1, 2, 3]` format
|
25
|
+
2. WHEN I use JSON functions THEN they SHALL work with DuckDB's JSON support
|
26
|
+
|
27
|
+
### Requirement 3: Configuration Interface Enhancement
|
28
|
+
|
29
|
+
**User Story:** As a developer, I want user-friendly configuration methods.
|
30
|
+
|
31
|
+
#### Acceptance Criteria
|
32
|
+
|
33
|
+
1. WHEN I use `db.set_pragma(key, value)` THEN it SHALL execute PRAGMA statements
|
34
|
+
2. WHEN I use `db.configure_duckdb(options)` THEN it SHALL apply multiple settings
|
35
|
+
|
36
|
+
### Requirement 4: Integration Testing
|
37
|
+
|
38
|
+
**User Story:** As a developer, I want comprehensive integration tests.
|
39
|
+
|
40
|
+
#### Acceptance Criteria
|
41
|
+
|
42
|
+
1. WHEN I run tests THEN all advanced features SHALL work with actual DuckDB databases
|
43
|
+
2. WHEN errors occur THEN they SHALL be properly mapped to Sequel exceptions
|
@@ -0,0 +1,24 @@
|
|
1
|
+
# Implementation Plan
|
2
|
+
|
3
|
+
Most advanced SQL features are already implemented. Remaining tasks based on current code analysis:
|
4
|
+
|
5
|
+
- [x] 1. Test window functions
|
6
|
+
- ~~Add tests to `test/dataset_test.rb` for ROW_NUMBER, RANK, DENSE_RANK~~ ✅ Already implemented
|
7
|
+
- ~~Test LAG/LEAD functions with offset and default parameters~~ ✅ Already implemented
|
8
|
+
- _Requirements: 1.1, 1.2_
|
9
|
+
|
10
|
+
- [x] 2. Test advanced expressions
|
11
|
+
- Add tests to `test/sql_test.rb` for DuckDB array syntax `[1, 2, 3]`
|
12
|
+
- Test JSON functions like `json_extract`
|
13
|
+
- _Requirements: 2.1, 2.2_
|
14
|
+
|
15
|
+
- [x] 3. Add configuration convenience methods
|
16
|
+
- Add `set_pragma(key, value)` method to `DatabaseMethods` (user-friendly wrapper)
|
17
|
+
- Add `configure_duckdb(options)` method for batch configuration
|
18
|
+
- Add tests to `test/database_test.rb`
|
19
|
+
- _Requirements: 3.1, 3.2_
|
20
|
+
|
21
|
+
- [x] 4. Integration testing
|
22
|
+
- ~~Add integration tests to existing test files using actual DuckDB databases~~ ✅ Already implemented
|
23
|
+
- ~~Test that all advanced features work together correctly~~ ✅ Already implemented
|
24
|
+
- _Requirements: 4.1, 4.2_
|
@@ -0,0 +1,258 @@
|
|
1
|
+
# Design Document: DuckDB SQL Syntax Compatibility
|
2
|
+
|
3
|
+
## Overview
|
4
|
+
|
5
|
+
This design addresses SQL generation issues in the sequel-duckdb adapter where the adapter is generating non-standard SQL syntax that doesn't match Sequel's established conventions. The issues include unwanted ESCAPE clauses in LIKE statements, missing parentheses in complex expressions, incorrect qualified column reference format, and improper subquery column references.
|
6
|
+
|
7
|
+
The solution is to fix the adapter's SQL generation methods to produce clean, standard SQL that follows Sequel's patterns while being fully compatible with DuckDB. This ensures consistent SQL generation that matches Sequel's conventions across all operations including LIKE clauses, complex expressions, qualified identifiers, regex operations, and subquery column references.
|
8
|
+
|
9
|
+
## Architecture
|
10
|
+
|
11
|
+
### Design Philosophy
|
12
|
+
|
13
|
+
1. **Fix Root Causes**: Address SQL generation issues in the adapter rather than working around them in tests
|
14
|
+
2. **Standard SQL Generation**: Generate clean, standard SQL that follows Sequel's established patterns
|
15
|
+
3. **Minimal Adapter Changes**: Make targeted fixes to specific SQL generation methods
|
16
|
+
|
17
|
+
### Current Structure (Targeted Fixes)
|
18
|
+
|
19
|
+
```
|
20
|
+
lib/sequel/adapters/
|
21
|
+
├── duckdb.rb # Main adapter (unchanged)
|
22
|
+
└── shared/
|
23
|
+
└── duckdb.rb # DatasetMethods (fix SQL generation methods)
|
24
|
+
|
25
|
+
test/
|
26
|
+
├── sql_test.rb # Fix dataset creation issues
|
27
|
+
├── dataset_test.rb # Tests should pass with fixed adapter
|
28
|
+
└── spec_helper.rb # No changes needed
|
29
|
+
```
|
30
|
+
|
31
|
+
## Components and Interfaces
|
32
|
+
|
33
|
+
### 1. LIKE Clause Generation Fix
|
34
|
+
|
35
|
+
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
36
|
+
|
37
|
+
Override LIKE handling to prevent unwanted ESCAPE clauses:
|
38
|
+
|
39
|
+
```ruby
|
40
|
+
def complex_expression_sql_append(sql, op, args)
|
41
|
+
case op
|
42
|
+
when :LIKE
|
43
|
+
# Generate clean LIKE without ESCAPE clause
|
44
|
+
sql << "("
|
45
|
+
literal_append(sql, args[0])
|
46
|
+
sql << " LIKE "
|
47
|
+
literal_append(sql, args[1])
|
48
|
+
sql << ")"
|
49
|
+
when :ILIKE
|
50
|
+
# Convert ILIKE to UPPER() LIKE UPPER() with proper parentheses
|
51
|
+
sql << "(UPPER("
|
52
|
+
literal_append(sql, args[0])
|
53
|
+
sql << ") LIKE UPPER("
|
54
|
+
literal_append(sql, args[1])
|
55
|
+
sql << "))"
|
56
|
+
when :~
|
57
|
+
# Regex with proper parentheses
|
58
|
+
sql << "("
|
59
|
+
literal_append(sql, args[0])
|
60
|
+
sql << " ~ "
|
61
|
+
literal_append(sql, args[1])
|
62
|
+
sql << ")"
|
63
|
+
else
|
64
|
+
super
|
65
|
+
end
|
66
|
+
end
|
67
|
+
```
|
68
|
+
|
69
|
+
### 2. Table Alias Generation Fix
|
70
|
+
|
71
|
+
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
72
|
+
|
73
|
+
Override alias handling to use standard AS syntax:
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
def table_alias_sql_append(sql, table, alias_name)
|
77
|
+
sql << table.to_s
|
78
|
+
sql << " AS "
|
79
|
+
sql << alias_name.to_s
|
80
|
+
end
|
81
|
+
```
|
82
|
+
|
83
|
+
### 3. Qualified Column Reference Fix
|
84
|
+
|
85
|
+
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
86
|
+
|
87
|
+
Override qualified identifier handling to use dot notation:
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
def qualified_identifier_sql_append(sql, table, column)
|
91
|
+
sql << table.to_s
|
92
|
+
sql << "."
|
93
|
+
sql << column.to_s
|
94
|
+
end
|
95
|
+
```
|
96
|
+
|
97
|
+
### 4. Subquery Column Reference Fix
|
98
|
+
|
99
|
+
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
100
|
+
|
101
|
+
Ensure subqueries properly reference outer query columns using dot notation:
|
102
|
+
|
103
|
+
```ruby
|
104
|
+
def subquery_sql_append(sql, ds)
|
105
|
+
# Ensure subqueries use proper column references
|
106
|
+
sql << "("
|
107
|
+
subquery_sql_append_sql(sql, ds.sql)
|
108
|
+
sql << ")"
|
109
|
+
end
|
110
|
+
|
111
|
+
def exists_sql_append(sql, ds)
|
112
|
+
# EXISTS subqueries with proper column references
|
113
|
+
sql << "EXISTS ("
|
114
|
+
subquery_sql_append_sql(sql, ds.sql)
|
115
|
+
sql << ")"
|
116
|
+
end
|
117
|
+
```
|
118
|
+
|
119
|
+
### 5. SQL Test Infrastructure Fix
|
120
|
+
|
121
|
+
**Location**: `test/sql_test.rb`
|
122
|
+
|
123
|
+
Fix dataset creation issues in SQL tests:
|
124
|
+
|
125
|
+
```ruby
|
126
|
+
class SqlTest < SequelDuckDBTest::TestCase
|
127
|
+
def setup
|
128
|
+
super
|
129
|
+
# Use proper mock dataset creation instead of subclasses
|
130
|
+
@dataset = mock_dataset(:users)
|
131
|
+
end
|
132
|
+
|
133
|
+
def test_like_clause_generation
|
134
|
+
dataset = @dataset.where(Sequel.like(:name, "%John%"))
|
135
|
+
expected_sql = "SELECT * FROM users WHERE (name LIKE '%John%')"
|
136
|
+
assert_sql expected_sql, dataset
|
137
|
+
end
|
138
|
+
|
139
|
+
def test_ilike_clause_generation
|
140
|
+
dataset = @dataset.where(Sequel.ilike(:name, "%john%"))
|
141
|
+
expected_sql = "SELECT * FROM users WHERE (UPPER(name) LIKE UPPER('%john%'))"
|
142
|
+
assert_sql expected_sql, dataset
|
143
|
+
end
|
144
|
+
|
145
|
+
def test_subquery_column_references
|
146
|
+
subquery = @dataset.select(:id).where(Sequel.qualify(:users, :active) => true)
|
147
|
+
dataset = @dataset.where(id: subquery)
|
148
|
+
expected_sql = "SELECT * FROM users WHERE (id IN (SELECT id FROM users WHERE (users.active = true)))"
|
149
|
+
assert_sql expected_sql, dataset
|
150
|
+
end
|
151
|
+
end
|
152
|
+
```
|
153
|
+
|
154
|
+
## Data Models
|
155
|
+
|
156
|
+
No new data models needed. The existing adapter structure handles SQL generation correctly.
|
157
|
+
|
158
|
+
## Error Handling
|
159
|
+
|
160
|
+
Use existing Sequel error handling patterns. No special error handling needed for syntax variations since they are all valid SQL.
|
161
|
+
|
162
|
+
## Testing Strategy
|
163
|
+
|
164
|
+
### 1. SQL Generation Unit Tests
|
165
|
+
- Test all SQL generation methods using Sequel's mock database functionality
|
166
|
+
- Verify exact SQL syntax matches expected Sequel patterns
|
167
|
+
- Test LIKE clauses generate clean SQL without ESCAPE clauses
|
168
|
+
- Test complex expressions have proper parentheses
|
169
|
+
- Test qualified column references use dot notation
|
170
|
+
- Test regex expressions are properly formatted and parenthesized
|
171
|
+
- Test subquery column references use standard SQL format
|
172
|
+
|
173
|
+
### 2. Integration Tests
|
174
|
+
- Ensure actual database operations work correctly with generated SQL
|
175
|
+
- Verify functional correctness alongside syntactic correctness
|
176
|
+
- Test correlated subqueries with proper column references
|
177
|
+
- Test complex queries with multiple SQL generation features
|
178
|
+
|
179
|
+
### 3. Test Infrastructure Consistency
|
180
|
+
- Fix SQL test infrastructure to use proper dataset creation
|
181
|
+
- Ensure tests expect standard SQL syntax consistently
|
182
|
+
- Maintain comprehensive test coverage for all SQL generation patterns
|
183
|
+
- Use Test-Driven Development approach for all fixes
|
184
|
+
|
185
|
+
## Design Decisions and Rationales
|
186
|
+
|
187
|
+
### 1. Fix Adapter SQL Generation
|
188
|
+
**Decision**: Fix the root cause SQL generation issues in the adapter
|
189
|
+
**Rationale**: The adapter is generating non-standard SQL that doesn't follow Sequel conventions. Tests are correct to expect standard SQL.
|
190
|
+
|
191
|
+
### 2. Targeted Method Overrides
|
192
|
+
**Decision**: Override specific SQL generation methods in DatasetMethods
|
193
|
+
**Rationale**: Surgical fixes to specific issues without disrupting the overall adapter architecture.
|
194
|
+
|
195
|
+
### 3. Standard SQL Compliance
|
196
|
+
**Decision**: Generate SQL that follows standard SQL and Sequel conventions
|
197
|
+
**Rationale**: Ensures compatibility with existing Sequel patterns and makes the adapter more predictable.
|
198
|
+
|
199
|
+
### 4. Maintain Test Coverage
|
200
|
+
**Decision**: Keep all existing tests, fix the adapter to make them pass
|
201
|
+
**Rationale**: Tests are validating correct behavior; the adapter should conform to expected patterns.
|
202
|
+
|
203
|
+
## Implementation Phases
|
204
|
+
|
205
|
+
### Phase 1: Fix LIKE and Complex Expression Generation
|
206
|
+
- Override `complex_expression_sql_append` to fix LIKE, ILIKE, and regex generation
|
207
|
+
- Add proper parentheses to all complex expressions
|
208
|
+
- Remove unwanted ESCAPE clauses from LIKE statements
|
209
|
+
|
210
|
+
### Phase 2: Fix Table Alias Generation
|
211
|
+
- Override table alias methods to use standard `AS` syntax
|
212
|
+
- Ensure aliases work correctly in JOIN operations
|
213
|
+
|
214
|
+
### Phase 3: Fix Qualified Column References
|
215
|
+
- Override qualified identifier methods to use dot notation
|
216
|
+
- Ensure subqueries use proper column references
|
217
|
+
|
218
|
+
### Phase 4: Fix SQL Test Infrastructure
|
219
|
+
- Fix dataset creation issues in SQL tests
|
220
|
+
- Ensure tests use proper mock datasets
|
221
|
+
|
222
|
+
### Phase 5: Verification and Documentation
|
223
|
+
- Run all tests to ensure they pass with fixed adapter
|
224
|
+
- Document the SQL generation patterns used by the adapter
|
225
|
+
- Create comprehensive documentation of DuckDB-specific SQL patterns
|
226
|
+
- Update API documentation with SQL generation examples
|
227
|
+
|
228
|
+
## Documentation Strategy
|
229
|
+
|
230
|
+
### SQL Pattern Documentation
|
231
|
+
To address Requirement 7, the adapter will include comprehensive documentation of SQL generation patterns:
|
232
|
+
|
233
|
+
**Location**: `API_DOCUMENTATION.md` and inline code comments
|
234
|
+
|
235
|
+
1. **LIKE Clause Patterns**: Document how LIKE clauses are generated without ESCAPE clauses
|
236
|
+
2. **Complex Expression Formatting**: Document parentheses usage in ILIKE and regex expressions
|
237
|
+
3. **Qualified Column References**: Document dot notation usage for table.column references
|
238
|
+
4. **Subquery Column References**: Document proper column referencing in correlated subqueries
|
239
|
+
5. **DuckDB-Specific Optimizations**: Document any DuckDB-specific SQL optimizations used
|
240
|
+
|
241
|
+
**Example Documentation Structure**:
|
242
|
+
```ruby
|
243
|
+
# SQL Generation Patterns for DuckDB Adapter
|
244
|
+
#
|
245
|
+
# LIKE Clauses:
|
246
|
+
# Input: Sequel.like(:name, "%John%")
|
247
|
+
# Output: (name LIKE '%John%')
|
248
|
+
#
|
249
|
+
# ILIKE Clauses:
|
250
|
+
# Input: Sequel.ilike(:name, "%john%")
|
251
|
+
# Output: (UPPER(name) LIKE UPPER('%john%'))
|
252
|
+
#
|
253
|
+
# Qualified Columns:
|
254
|
+
# Input: Sequel.qualify(:users, :id)
|
255
|
+
# Output: users.id
|
256
|
+
```
|
257
|
+
|
258
|
+
This design fixes the root causes of the SQL generation issues rather than working around them, resulting in a more robust and standards-compliant adapter.
|
@@ -0,0 +1,84 @@
|
|
1
|
+
# Requirements Document
|
2
|
+
|
3
|
+
## Introduction
|
4
|
+
|
5
|
+
This specification addresses SQL generation issues in the sequel-duckdb adapter where the adapter is generating non-standard SQL syntax that doesn't match expected Sequel conventions. The adapter should generate clean, standard SQL that follows Sequel's established patterns while being compatible with DuckDB. The issues are in the adapter's SQL generation logic, not in DuckDB's SQL support.
|
6
|
+
|
7
|
+
## Requirements
|
8
|
+
|
9
|
+
### Requirement 1: LIKE Clause Clean Generation
|
10
|
+
|
11
|
+
**User Story:** As a developer using Sequel with DuckDB, I want LIKE clauses to generate clean SQL without unnecessary ESCAPE clauses, so that the SQL matches standard Sequel patterns.
|
12
|
+
|
13
|
+
#### Acceptance Criteria
|
14
|
+
|
15
|
+
1. WHEN I use `Sequel.like(:name, "%John%")` THEN the generated SQL SHALL be `(name LIKE '%John%')` without ESCAPE clause
|
16
|
+
2. WHEN I use LIKE patterns with special characters THEN they SHALL work without requiring explicit ESCAPE clauses
|
17
|
+
3. WHEN I use ILIKE patterns THEN they SHALL be converted to `(UPPER(column) LIKE UPPER(pattern))` with proper parentheses
|
18
|
+
4. WHEN tests check LIKE clause generation THEN they SHALL expect clean SQL without ESCAPE additions
|
19
|
+
|
20
|
+
### Requirement 2: Complex Expression Parentheses
|
21
|
+
|
22
|
+
**User Story:** As a developer using Sequel with DuckDB, I want complex expressions to be properly parenthesized in generated SQL, so that the SQL follows standard Sequel formatting conventions.
|
23
|
+
|
24
|
+
#### Acceptance Criteria
|
25
|
+
|
26
|
+
1. WHEN I use ILIKE expressions THEN they SHALL be wrapped in parentheses: `(UPPER(column) LIKE UPPER(pattern))`
|
27
|
+
2. WHEN I use regex expressions THEN they SHALL be wrapped in parentheses: `(column ~ 'pattern')`
|
28
|
+
3. WHEN I use boolean comparisons with `=~` THEN they SHALL generate proper `IS` syntax with parentheses
|
29
|
+
4. WHEN tests check complex expressions THEN they SHALL expect properly parenthesized SQL
|
30
|
+
|
31
|
+
### Requirement 3: Standard Qualified Column References
|
32
|
+
|
33
|
+
**User Story:** As a developer using Sequel with DuckDB, I want qualified column references to use standard SQL dot notation, so that the generated SQL follows established SQL conventions.
|
34
|
+
|
35
|
+
#### Acceptance Criteria
|
36
|
+
|
37
|
+
1. WHEN I reference columns across tables THEN they SHALL use `table.column` syntax
|
38
|
+
2. WHEN I use qualified column names in subqueries THEN they SHALL use proper dot notation
|
39
|
+
3. WHEN I use schema-qualified names THEN they SHALL use standard SQL format
|
40
|
+
4. WHEN tests check qualified identifiers THEN they SHALL expect standard dot notation
|
41
|
+
|
42
|
+
### Requirement 4: Proper Regular Expression Formatting
|
43
|
+
|
44
|
+
**User Story:** As a developer using Sequel with DuckDB, I want regular expression matching to generate properly formatted SQL with parentheses, so that the SQL follows Sequel's formatting conventions.
|
45
|
+
|
46
|
+
#### Acceptance Criteria
|
47
|
+
|
48
|
+
1. WHEN I use regex matching with `~` operator THEN it SHALL generate `(column ~ 'pattern')` with parentheses
|
49
|
+
2. WHEN I use case-insensitive regex THEN it SHALL be properly formatted with parentheses
|
50
|
+
3. WHEN I use complex regex patterns THEN they SHALL be properly formatted and parenthesized
|
51
|
+
4. WHEN tests check regex syntax THEN they SHALL expect properly parenthesized expressions
|
52
|
+
|
53
|
+
### Requirement 5: Standard Subquery Column References
|
54
|
+
|
55
|
+
**User Story:** As a developer using Sequel with DuckDB, I want subqueries to properly reference outer query columns using standard SQL dot notation, so that correlated subqueries follow SQL conventions.
|
56
|
+
|
57
|
+
#### Acceptance Criteria
|
58
|
+
|
59
|
+
1. WHEN I use correlated subqueries THEN column references SHALL use `table.column` syntax
|
60
|
+
2. WHEN I reference outer query columns THEN they SHALL be properly qualified with dot notation
|
61
|
+
3. WHEN I use EXISTS subqueries THEN column references SHALL use standard SQL format
|
62
|
+
4. WHEN tests check subquery references THEN they SHALL expect standard dot notation
|
63
|
+
|
64
|
+
### Requirement 6: Consistent SQL Generation Testing
|
65
|
+
|
66
|
+
**User Story:** As a developer maintaining the sequel-duckdb adapter, I want tests to verify that the adapter generates consistent, standard SQL, so that the adapter follows Sequel's established patterns.
|
67
|
+
|
68
|
+
#### Acceptance Criteria
|
69
|
+
|
70
|
+
1. WHEN tests check SQL generation THEN they SHALL expect standard SQL syntax
|
71
|
+
2. WHEN the adapter generates SQL THEN it SHALL be consistent with Sequel's conventions
|
72
|
+
3. WHEN SQL generation issues are found THEN they SHALL be fixed in the adapter, not worked around in tests
|
73
|
+
4. WHEN SQL correctness is verified THEN both functional and syntactic correctness SHALL be maintained
|
74
|
+
|
75
|
+
### Requirement 7: Documentation of SQL Generation Patterns
|
76
|
+
|
77
|
+
**User Story:** As a developer using the sequel-duckdb adapter, I want to understand how the adapter generates SQL for DuckDB, so that I can write queries that work optimally with the adapter.
|
78
|
+
|
79
|
+
#### Acceptance Criteria
|
80
|
+
|
81
|
+
1. WHEN the adapter generates specific SQL patterns THEN they SHALL be documented
|
82
|
+
2. WHEN SQL generation differs from other Sequel adapters THEN the differences SHALL be explained
|
83
|
+
3. WHEN DuckDB-specific optimizations are used THEN they SHALL be documented with examples
|
84
|
+
4. WHEN SQL generation patterns change THEN documentation SHALL be updated accordingly
|
@@ -0,0 +1,94 @@
|
|
1
|
+
# Implementation Plan
|
2
|
+
|
3
|
+
- [x] 1. Identify failing tests and analyze DuckDB SQL generation
|
4
|
+
- Run existing test suite to identify tests failing due to SQL syntax differences
|
5
|
+
- Analyze what SQL the adapter currently generates vs what tests expect
|
6
|
+
- Document the specific DuckDB syntax patterns that are being generated
|
7
|
+
- Determine if adapter changes are needed or just test expectation updates
|
8
|
+
- _Requirements: 7.1, 7.2_
|
9
|
+
|
10
|
+
- [x] 2. Fix LIKE clause ESCAPE handling in adapter
|
11
|
+
- Current issue: LIKE generates `(name LIKE '%John%' ESCAPE '\')` instead of `(name LIKE '%John%')`
|
12
|
+
- Override LIKE handling in `complex_expression_sql_append` to remove ESCAPE clause
|
13
|
+
- Ensure LIKE clauses generate clean `(name LIKE '%John%')` without ESCAPE clause
|
14
|
+
- Write tests to verify LIKE functionality works correctly without the ESCAPE clause
|
15
|
+
- _Requirements: 1.1, 1.2, 1.4_
|
16
|
+
|
17
|
+
- [x] 3. Fix parentheses in complex expression SQL generation
|
18
|
+
- Current issue: ILIKE generates `UPPER(name) LIKE UPPER('%john%')` without outer parentheses
|
19
|
+
- Current issue: Regex generates `name ~ 'pattern'` without outer parentheses
|
20
|
+
- Update `complex_expression_sql_append` to add proper parentheses around expressions
|
21
|
+
- Ensure ILIKE generates `(UPPER(name) LIKE UPPER('%john%'))` with parentheses
|
22
|
+
- Ensure regex generates `(name ~ '^John')` with parentheses around the expression
|
23
|
+
- Write tests to verify all complex expressions have consistent parentheses
|
24
|
+
- _Requirements: 1.3, 5.1, 5.2, 5.3, 5.4_
|
25
|
+
|
26
|
+
- [x] 4. ~~Fix table alias syntax to use AS instead of triple underscore~~ (REMOVED - Not a core Sequel feature)
|
27
|
+
- The `table___alias` syntax is not a core Sequel feature and requires an extension
|
28
|
+
- Removed implementation and tests as this is not standard Sequel functionality
|
29
|
+
- Standard Sequel table aliases use `.as()` method: `db[:users].as(:u)`
|
30
|
+
|
31
|
+
- [x] 5. Fix qualified column references to use dot notation
|
32
|
+
- Current issue: Qualified columns generate `users__id` instead of `users.id`
|
33
|
+
- Override qualified identifier handling to generate proper dot notation
|
34
|
+
- Implement `qualified_identifier_sql_append` method to use dot notation
|
35
|
+
- Ensure subqueries use correct `users.id` column reference format
|
36
|
+
- Write tests to verify qualified column references work correctly in complex queries
|
37
|
+
- _Requirements: 4.1, 4.2, 4.3, 4.4, 6.1, 6.2, 6.3, 6.4_
|
38
|
+
|
39
|
+
- [x] 6. Fix SQL test infrastructure issues
|
40
|
+
- Current issue: SQL tests failing with dataset type assertion errors (`Expected #<Sequel::Dataset::_Subclass>`)
|
41
|
+
- Fix `SqlTest` class to use proper dataset creation instead of mock subclasses
|
42
|
+
- Update SQL generation tests to work with actual DuckDB adapter behavior
|
43
|
+
- Ensure tests create proper datasets for SQL generation testing
|
44
|
+
- Fix dataset type assertion issues in SQL tests
|
45
|
+
- _Requirements: 6.1, 6.2_
|
46
|
+
|
47
|
+
- [x] 7. Fix recursive CTE SQL generation
|
48
|
+
- Current issue: `WITH RECURSIVE` generates incorrect SQL without RECURSIVE keyword
|
49
|
+
- Implement proper recursive CTE handling in dataset methods
|
50
|
+
- Ensure recursive CTEs generate `WITH RECURSIVE` syntax correctly
|
51
|
+
- Test recursive CTE functionality with proper SQL generation
|
52
|
+
- _Requirements: 5.1, 5.2, 5.3, 5.4_
|
53
|
+
|
54
|
+
- [x] 8. Fix JOIN USING clause generation
|
55
|
+
- Current issue: `JOIN USING` clause not generating USING syntax correctly
|
56
|
+
- Implement proper USING clause handling in JOIN operations
|
57
|
+
- Ensure JOIN USING generates correct `INNER JOIN table USING (column)` syntax
|
58
|
+
- Test JOIN USING functionality with various column combinations
|
59
|
+
- _Requirements: 4.1, 4.2, 4.3, 4.4_
|
60
|
+
|
61
|
+
- [x] 9. Fix regex functionality integration
|
62
|
+
- Current issue: Regex matching returning 0 results instead of expected matches
|
63
|
+
- Debug regex operator implementation in complex_expression_sql_append
|
64
|
+
- Ensure regex patterns work correctly with DuckDB's regex syntax
|
65
|
+
- Test regex functionality with actual data and pattern matching
|
66
|
+
- _Requirements: 1.3, 5.1, 5.2, 5.3, 5.4_
|
67
|
+
|
68
|
+
- [x] 10. Fix model integration issues
|
69
|
+
- Current issue: Model tests failing with type mapping and update detection
|
70
|
+
- Fix time type mapping (currently returning :datetime instead of :time)
|
71
|
+
- Fix model update detection to properly track changed fields
|
72
|
+
- Ensure model DELETE operations work correctly with proper ID handling
|
73
|
+
- _Requirements: 2.1, 2.2, 2.3, 2.4_
|
74
|
+
|
75
|
+
- [x] 11. Fix table creation and schema issues
|
76
|
+
- Current issue: Tests failing because tables don't exist during SQL execution
|
77
|
+
- Ensure proper table creation in test setup for integration tests
|
78
|
+
- Fix primary key handling for tables without explicit primary keys
|
79
|
+
- Handle NOT NULL constraint violations properly in INSERT operations
|
80
|
+
- _Requirements: 3.1, 3.2, 3.3, 3.4_
|
81
|
+
|
82
|
+
- [x] 12. Verify all tests pass with consistent DuckDB SQL generation
|
83
|
+
- Run complete test suite to ensure all SQL generation tests pass
|
84
|
+
- Verify that adapter generates consistent, predictable SQL for DuckDB
|
85
|
+
- Test integration scenarios to ensure functional correctness
|
86
|
+
- Fix any remaining test expectation mismatches
|
87
|
+
- _Requirements: 6.1, 6.2, 6.3, 6.4_
|
88
|
+
|
89
|
+
- [x] 13. Document DuckDB-specific SQL syntax patterns
|
90
|
+
- Document the specific SQL syntax that the adapter generates for DuckDB
|
91
|
+
- Provide examples of DuckDB SQL patterns used by the adapter
|
92
|
+
- Explain why certain DuckDB syntax choices were made
|
93
|
+
- Create reference documentation for developers using the adapter
|
94
|
+
- _Requirements: 7.1, 7.2, 7.3_
|
@@ -0,0 +1,32 @@
|
|
1
|
+
# Requirements Document
|
2
|
+
|
3
|
+
## Introduction
|
4
|
+
|
5
|
+
After comprehensive review of the current sequel-duckdb adapter implementation and testing against the actual behavior, all previously identified edge cases and validation issues have been determined to be either:
|
6
|
+
|
7
|
+
1. **Already implemented** - The adapter includes comprehensive error handling, data type conversion, SQL injection prevention through parameterized queries, connection management, and transaction support.
|
8
|
+
|
9
|
+
2. **Handled by Sequel core** - Query parameter validation (LIMIT/OFFSET edge cases) is handled by the Sequel framework itself before reaching database adapters.
|
10
|
+
|
11
|
+
3. **Working as designed** - DuckDB's native behavior for edge cases is appropriate and doesn't require additional adapter-level handling.
|
12
|
+
|
13
|
+
4. **Out of scope** - Some edge cases (like system resource management) are better handled at the application or infrastructure level rather than in a database adapter.
|
14
|
+
|
15
|
+
## Current Status
|
16
|
+
|
17
|
+
The sequel-duckdb adapter currently provides:
|
18
|
+
|
19
|
+
- ✅ Comprehensive error handling and mapping to appropriate Sequel exception types
|
20
|
+
- ✅ Robust connection management with proper error handling
|
21
|
+
- ✅ Complete data type conversion and validation
|
22
|
+
- ✅ SQL injection prevention through parameterized queries
|
23
|
+
- ✅ Transaction support with proper rollback handling
|
24
|
+
- ✅ Schema introspection and DDL operations
|
25
|
+
- ✅ Performance optimizations for large datasets
|
26
|
+
- ✅ Memory management for streaming operations
|
27
|
+
|
28
|
+
## Conclusion
|
29
|
+
|
30
|
+
No additional edge case or validation requirements have been identified that would add meaningful value to the adapter. The current implementation provides robust handling of edge cases appropriate for a production database adapter.
|
31
|
+
|
32
|
+
This specification is considered complete with no outstanding requirements.
|
File without changes
|