sequel-duckdb 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +31 -20
- data/lib/sequel/duckdb/version.rb +1 -4
- metadata +3 -59
- data/.beads/.beads-credential-key +0 -1
- data/.beads/.gitignore +0 -66
- data/.beads/README.md +0 -85
- data/.beads/config.yaml +0 -56
- data/.beads/hooks/post-checkout +0 -24
- data/.beads/hooks/post-merge +0 -24
- data/.beads/hooks/pre-commit +0 -24
- data/.beads/hooks/pre-push +0 -24
- data/.beads/hooks/prepare-commit-msg +0 -24
- data/.beads/metadata.json +0 -7
- data/.kiro/specs/advanced-sql-features-implementation/design.md +0 -26
- data/.kiro/specs/advanced-sql-features-implementation/requirements.md +0 -43
- data/.kiro/specs/advanced-sql-features-implementation/tasks.md +0 -28
- data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +0 -272
- data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +0 -84
- data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +0 -107
- data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +0 -32
- data/.kiro/specs/integration-test-database-setup/design.md +0 -0
- data/.kiro/specs/integration-test-database-setup/requirements.md +0 -117
- data/.kiro/specs/sequel-duckdb-adapter/design.md +0 -549
- data/.kiro/specs/sequel-duckdb-adapter/requirements.md +0 -202
- data/.kiro/specs/sequel-duckdb-adapter/tasks.md +0 -292
- data/.kiro/specs/sql-expression-handling-fix/design.md +0 -331
- data/.kiro/specs/sql-expression-handling-fix/requirements.md +0 -86
- data/.kiro/specs/sql-expression-handling-fix/tasks.md +0 -25
- data/.kiro/specs/test-infrastructure-improvements/requirements.md +0 -106
- data/.kiro/steering/product.md +0 -26
- data/.kiro/steering/structure.md +0 -88
- data/.kiro/steering/tech.md +0 -137
- data/.kiro/steering/testing.md +0 -213
- data/.mdformat.toml +0 -2
- data/.release-please-manifest.json +0 -3
- data/.rubocop.yml +0 -161
- data/.rubocop_todo.yml +0 -323
- data/.yardopts +0 -8
- data/AGENTS.md +0 -154
- data/API_DOCUMENTATION.md +0 -943
- data/FINAL_STATUS.md +0 -99
- data/MIGRATION_EXAMPLES.md +0 -740
- data/PERFORMANCE_OPTIMIZATIONS.md +0 -726
- data/REFACTORING_SUMMARY.md +0 -264
- data/Rakefile +0 -43
- data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +0 -182
- data/docs/DUCKDB_SQL_PATTERNS.md +0 -448
- data/docs/TASK_12_VERIFICATION_SUMMARY.md +0 -135
- data/justfile +0 -52
- data/plans/date_arithmetic.md +0 -420
- data/plans/engineering/Sequel.md +0 -471
- data/plans/engineering/duckdb.md +0 -712
- data/plans/engineering/sqlite.md +0 -453
- data/plans/mock_connection_bug.md +0 -333
- data/plans/mock_without_driver_gem.md +0 -371
- data/plans/over_engineering_analysis.md +0 -122
- data/plans/schema_management.md +0 -383
- data/release-please-config.json +0 -14
- data/sig/sequel/duckdb.rbs +0 -6
|
@@ -1,272 +0,0 @@
|
|
|
1
|
-
# Design Document: DuckDB SQL Syntax Compatibility
|
|
2
|
-
|
|
3
|
-
## Overview
|
|
4
|
-
|
|
5
|
-
This design addresses SQL generation issues in the sequel-duckdb adapter where the adapter is generating non-standard SQL syntax that doesn't match Sequel's established conventions. The issues include unwanted ESCAPE clauses in LIKE statements, missing parentheses in complex expressions, incorrect qualified column reference format, and improper subquery column references.
|
|
6
|
-
|
|
7
|
-
The solution is to fix the adapter's SQL generation methods to produce clean, standard SQL that follows Sequel's patterns while being fully compatible with DuckDB. This ensures consistent SQL generation that matches Sequel's conventions across all operations including LIKE clauses, complex expressions, qualified identifiers, regex operations, and subquery column references.
|
|
8
|
-
|
|
9
|
-
## Architecture
|
|
10
|
-
|
|
11
|
-
### Design Philosophy
|
|
12
|
-
|
|
13
|
-
1. **Fix Root Causes**: Address SQL generation issues in the adapter rather than working around them in tests
|
|
14
|
-
2. **Standard SQL Generation**: Generate clean, standard SQL that follows Sequel's established patterns
|
|
15
|
-
3. **Minimal Adapter Changes**: Make targeted fixes to specific SQL generation methods
|
|
16
|
-
|
|
17
|
-
### Current Structure (Targeted Fixes)
|
|
18
|
-
|
|
19
|
-
```
|
|
20
|
-
lib/sequel/adapters/
|
|
21
|
-
├── duckdb.rb # Main adapter (unchanged)
|
|
22
|
-
└── shared/
|
|
23
|
-
└── duckdb.rb # DatasetMethods (fix SQL generation methods)
|
|
24
|
-
|
|
25
|
-
test/
|
|
26
|
-
├── sql_test.rb # Fix dataset creation issues
|
|
27
|
-
├── dataset_test.rb # Tests should pass with fixed adapter
|
|
28
|
-
└── spec_helper.rb # No changes needed
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
## Components and Interfaces
|
|
32
|
-
|
|
33
|
-
### 1. LIKE Clause Generation Fix
|
|
34
|
-
|
|
35
|
-
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
|
36
|
-
|
|
37
|
-
Override LIKE handling to prevent unwanted ESCAPE clauses:
|
|
38
|
-
|
|
39
|
-
```ruby
|
|
40
|
-
def complex_expression_sql_append(sql, op, args)
|
|
41
|
-
case op
|
|
42
|
-
when :LIKE
|
|
43
|
-
# Generate clean LIKE without ESCAPE clause
|
|
44
|
-
sql << "("
|
|
45
|
-
literal_append(sql, args[0])
|
|
46
|
-
sql << " LIKE "
|
|
47
|
-
literal_append(sql, args[1])
|
|
48
|
-
sql << ")"
|
|
49
|
-
when :ILIKE
|
|
50
|
-
# Convert ILIKE to UPPER() LIKE UPPER() with proper parentheses
|
|
51
|
-
sql << "(UPPER("
|
|
52
|
-
literal_append(sql, args[0])
|
|
53
|
-
sql << ") LIKE UPPER("
|
|
54
|
-
literal_append(sql, args[1])
|
|
55
|
-
sql << "))"
|
|
56
|
-
when :~
|
|
57
|
-
# Regex with proper parentheses
|
|
58
|
-
sql << "("
|
|
59
|
-
literal_append(sql, args[0])
|
|
60
|
-
sql << " ~ "
|
|
61
|
-
literal_append(sql, args[1])
|
|
62
|
-
sql << ")"
|
|
63
|
-
else
|
|
64
|
-
super
|
|
65
|
-
end
|
|
66
|
-
end
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
### 2. Table Alias Generation Fix
|
|
70
|
-
|
|
71
|
-
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
|
72
|
-
|
|
73
|
-
Override alias handling to use standard AS syntax:
|
|
74
|
-
|
|
75
|
-
```ruby
|
|
76
|
-
def table_alias_sql_append(sql, table, alias_name)
|
|
77
|
-
sql << table.to_s
|
|
78
|
-
sql << " AS "
|
|
79
|
-
sql << alias_name.to_s
|
|
80
|
-
end
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
### 3. Qualified Column Reference Fix
|
|
84
|
-
|
|
85
|
-
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
|
86
|
-
|
|
87
|
-
Override qualified identifier handling to use dot notation:
|
|
88
|
-
|
|
89
|
-
```ruby
|
|
90
|
-
def qualified_identifier_sql_append(sql, table, column)
|
|
91
|
-
sql << table.to_s
|
|
92
|
-
sql << "."
|
|
93
|
-
sql << column.to_s
|
|
94
|
-
end
|
|
95
|
-
```
|
|
96
|
-
|
|
97
|
-
### 4. Subquery Column Reference Fix
|
|
98
|
-
|
|
99
|
-
**Location**: `lib/sequel/adapters/shared/duckdb.rb` - DatasetMethods
|
|
100
|
-
|
|
101
|
-
Ensure subqueries properly reference outer query columns using dot notation:
|
|
102
|
-
|
|
103
|
-
```ruby
|
|
104
|
-
def subquery_sql_append(sql, ds)
|
|
105
|
-
# Ensure subqueries use proper column references
|
|
106
|
-
sql << "("
|
|
107
|
-
subquery_sql_append_sql(sql, ds.sql)
|
|
108
|
-
sql << ")"
|
|
109
|
-
end
|
|
110
|
-
|
|
111
|
-
def exists_sql_append(sql, ds)
|
|
112
|
-
# EXISTS subqueries with proper column references
|
|
113
|
-
sql << "EXISTS ("
|
|
114
|
-
subquery_sql_append_sql(sql, ds.sql)
|
|
115
|
-
sql << ")"
|
|
116
|
-
end
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
### 5. SQL Test Infrastructure Fix
|
|
120
|
-
|
|
121
|
-
**Location**: `test/sql_test.rb`
|
|
122
|
-
|
|
123
|
-
Fix dataset creation issues in SQL tests:
|
|
124
|
-
|
|
125
|
-
```ruby
|
|
126
|
-
class SqlTest < SequelDuckDBTest::TestCase
|
|
127
|
-
def setup
|
|
128
|
-
super
|
|
129
|
-
# Use proper mock dataset creation instead of subclasses
|
|
130
|
-
@dataset = mock_dataset(:users)
|
|
131
|
-
end
|
|
132
|
-
|
|
133
|
-
def test_like_clause_generation
|
|
134
|
-
dataset = @dataset.where(Sequel.like(:name, "%John%"))
|
|
135
|
-
expected_sql = "SELECT * FROM users WHERE (name LIKE '%John%')"
|
|
136
|
-
assert_sql expected_sql, dataset
|
|
137
|
-
end
|
|
138
|
-
|
|
139
|
-
def test_ilike_clause_generation
|
|
140
|
-
dataset = @dataset.where(Sequel.ilike(:name, "%john%"))
|
|
141
|
-
expected_sql = "SELECT * FROM users WHERE (UPPER(name) LIKE UPPER('%john%'))"
|
|
142
|
-
assert_sql expected_sql, dataset
|
|
143
|
-
end
|
|
144
|
-
|
|
145
|
-
def test_subquery_column_references
|
|
146
|
-
subquery = @dataset.select(:id).where(Sequel.qualify(:users, :active) => true)
|
|
147
|
-
dataset = @dataset.where(id: subquery)
|
|
148
|
-
expected_sql = "SELECT * FROM users WHERE (id IN (SELECT id FROM users WHERE (users.active = true)))"
|
|
149
|
-
assert_sql expected_sql, dataset
|
|
150
|
-
end
|
|
151
|
-
end
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
## Data Models
|
|
155
|
-
|
|
156
|
-
No new data models needed. The existing adapter structure handles SQL generation correctly.
|
|
157
|
-
|
|
158
|
-
## Error Handling
|
|
159
|
-
|
|
160
|
-
Use existing Sequel error handling patterns. No special error handling needed for syntax variations since they are all valid SQL.
|
|
161
|
-
|
|
162
|
-
## Testing Strategy
|
|
163
|
-
|
|
164
|
-
### 1. SQL Generation Unit Tests
|
|
165
|
-
|
|
166
|
-
- Test all SQL generation methods using Sequel's mock database functionality
|
|
167
|
-
- Verify exact SQL syntax matches expected Sequel patterns
|
|
168
|
-
- Test LIKE clauses generate clean SQL without ESCAPE clauses
|
|
169
|
-
- Test complex expressions have proper parentheses
|
|
170
|
-
- Test qualified column references use dot notation
|
|
171
|
-
- Test regex expressions are properly formatted and parenthesized
|
|
172
|
-
- Test subquery column references use standard SQL format
|
|
173
|
-
|
|
174
|
-
### 2. Integration Tests
|
|
175
|
-
|
|
176
|
-
- Ensure actual database operations work correctly with generated SQL
|
|
177
|
-
- Verify functional correctness alongside syntactic correctness
|
|
178
|
-
- Test correlated subqueries with proper column references
|
|
179
|
-
- Test complex queries with multiple SQL generation features
|
|
180
|
-
|
|
181
|
-
### 3. Test Infrastructure Consistency
|
|
182
|
-
|
|
183
|
-
- Fix SQL test infrastructure to use proper dataset creation
|
|
184
|
-
- Ensure tests expect standard SQL syntax consistently
|
|
185
|
-
- Maintain comprehensive test coverage for all SQL generation patterns
|
|
186
|
-
- Use Test-Driven Development approach for all fixes
|
|
187
|
-
|
|
188
|
-
## Design Decisions and Rationales
|
|
189
|
-
|
|
190
|
-
### 1. Fix Adapter SQL Generation
|
|
191
|
-
|
|
192
|
-
**Decision**: Fix the root cause SQL generation issues in the adapter
|
|
193
|
-
**Rationale**: The adapter is generating non-standard SQL that doesn't follow Sequel conventions. Tests are correct to expect standard SQL.
|
|
194
|
-
|
|
195
|
-
### 2. Targeted Method Overrides
|
|
196
|
-
|
|
197
|
-
**Decision**: Override specific SQL generation methods in DatasetMethods
|
|
198
|
-
**Rationale**: Surgical fixes to specific issues without disrupting the overall adapter architecture.
|
|
199
|
-
|
|
200
|
-
### 3. Standard SQL Compliance
|
|
201
|
-
|
|
202
|
-
**Decision**: Generate SQL that follows standard SQL and Sequel conventions
|
|
203
|
-
**Rationale**: Ensures compatibility with existing Sequel patterns and makes the adapter more predictable.
|
|
204
|
-
|
|
205
|
-
### 4. Maintain Test Coverage
|
|
206
|
-
|
|
207
|
-
**Decision**: Keep all existing tests, fix the adapter to make them pass
|
|
208
|
-
**Rationale**: Tests are validating correct behavior; the adapter should conform to expected patterns.
|
|
209
|
-
|
|
210
|
-
## Implementation Phases
|
|
211
|
-
|
|
212
|
-
### Phase 1: Fix LIKE and Complex Expression Generation
|
|
213
|
-
|
|
214
|
-
- Override `complex_expression_sql_append` to fix LIKE, ILIKE, and regex generation
|
|
215
|
-
- Add proper parentheses to all complex expressions
|
|
216
|
-
- Remove unwanted ESCAPE clauses from LIKE statements
|
|
217
|
-
|
|
218
|
-
### Phase 2: Fix Table Alias Generation
|
|
219
|
-
|
|
220
|
-
- Override table alias methods to use standard `AS` syntax
|
|
221
|
-
- Ensure aliases work correctly in JOIN operations
|
|
222
|
-
|
|
223
|
-
### Phase 3: Fix Qualified Column References
|
|
224
|
-
|
|
225
|
-
- Override qualified identifier methods to use dot notation
|
|
226
|
-
- Ensure subqueries use proper column references
|
|
227
|
-
|
|
228
|
-
### Phase 4: Fix SQL Test Infrastructure
|
|
229
|
-
|
|
230
|
-
- Fix dataset creation issues in SQL tests
|
|
231
|
-
- Ensure tests use proper mock datasets
|
|
232
|
-
|
|
233
|
-
### Phase 5: Verification and Documentation
|
|
234
|
-
|
|
235
|
-
- Run all tests to ensure they pass with fixed adapter
|
|
236
|
-
- Document the SQL generation patterns used by the adapter
|
|
237
|
-
- Create comprehensive documentation of DuckDB-specific SQL patterns
|
|
238
|
-
- Update API documentation with SQL generation examples
|
|
239
|
-
|
|
240
|
-
## Documentation Strategy
|
|
241
|
-
|
|
242
|
-
### SQL Pattern Documentation
|
|
243
|
-
|
|
244
|
-
To address Requirement 7, the adapter will include comprehensive documentation of SQL generation patterns:
|
|
245
|
-
|
|
246
|
-
**Location**: `API_DOCUMENTATION.md` and inline code comments
|
|
247
|
-
|
|
248
|
-
1. **LIKE Clause Patterns**: Document how LIKE clauses are generated without ESCAPE clauses
|
|
249
|
-
2. **Complex Expression Formatting**: Document parentheses usage in ILIKE and regex expressions
|
|
250
|
-
3. **Qualified Column References**: Document dot notation usage for table.column references
|
|
251
|
-
4. **Subquery Column References**: Document proper column referencing in correlated subqueries
|
|
252
|
-
5. **DuckDB-Specific Optimizations**: Document any DuckDB-specific SQL optimizations used
|
|
253
|
-
|
|
254
|
-
**Example Documentation Structure**:
|
|
255
|
-
|
|
256
|
-
```ruby
|
|
257
|
-
# SQL Generation Patterns for DuckDB Adapter
|
|
258
|
-
#
|
|
259
|
-
# LIKE Clauses:
|
|
260
|
-
# Input: Sequel.like(:name, "%John%")
|
|
261
|
-
# Output: (name LIKE '%John%')
|
|
262
|
-
#
|
|
263
|
-
# ILIKE Clauses:
|
|
264
|
-
# Input: Sequel.ilike(:name, "%john%")
|
|
265
|
-
# Output: (UPPER(name) LIKE UPPER('%john%'))
|
|
266
|
-
#
|
|
267
|
-
# Qualified Columns:
|
|
268
|
-
# Input: Sequel.qualify(:users, :id)
|
|
269
|
-
# Output: users.id
|
|
270
|
-
```
|
|
271
|
-
|
|
272
|
-
This design fixes the root causes of the SQL generation issues rather than working around them, resulting in a more robust and standards-compliant adapter.
|
|
@@ -1,84 +0,0 @@
|
|
|
1
|
-
# Requirements Document
|
|
2
|
-
|
|
3
|
-
## Introduction
|
|
4
|
-
|
|
5
|
-
This specification addresses SQL generation issues in the sequel-duckdb adapter where the adapter is generating non-standard SQL syntax that doesn't match expected Sequel conventions. The adapter should generate clean, standard SQL that follows Sequel's established patterns while being compatible with DuckDB. The issues are in the adapter's SQL generation logic, not in DuckDB's SQL support.
|
|
6
|
-
|
|
7
|
-
## Requirements
|
|
8
|
-
|
|
9
|
-
### Requirement 1: LIKE Clause Clean Generation
|
|
10
|
-
|
|
11
|
-
**User Story:** As a developer using Sequel with DuckDB, I want LIKE clauses to generate clean SQL without unnecessary ESCAPE clauses, so that the SQL matches standard Sequel patterns.
|
|
12
|
-
|
|
13
|
-
#### Acceptance Criteria
|
|
14
|
-
|
|
15
|
-
1. WHEN I use `Sequel.like(:name, "%John%")` THEN the generated SQL SHALL be `(name LIKE '%John%')` without ESCAPE clause
|
|
16
|
-
2. WHEN I use LIKE patterns with special characters THEN they SHALL work without requiring explicit ESCAPE clauses
|
|
17
|
-
3. WHEN I use ILIKE patterns THEN they SHALL be converted to `(UPPER(column) LIKE UPPER(pattern))` with proper parentheses
|
|
18
|
-
4. WHEN tests check LIKE clause generation THEN they SHALL expect clean SQL without ESCAPE additions
|
|
19
|
-
|
|
20
|
-
### Requirement 2: Complex Expression Parentheses
|
|
21
|
-
|
|
22
|
-
**User Story:** As a developer using Sequel with DuckDB, I want complex expressions to be properly parenthesized in generated SQL, so that the SQL follows standard Sequel formatting conventions.
|
|
23
|
-
|
|
24
|
-
#### Acceptance Criteria
|
|
25
|
-
|
|
26
|
-
1. WHEN I use ILIKE expressions THEN they SHALL be wrapped in parentheses: `(UPPER(column) LIKE UPPER(pattern))`
|
|
27
|
-
2. WHEN I use regex expressions THEN they SHALL be wrapped in parentheses: `(column ~ 'pattern')`
|
|
28
|
-
3. WHEN I use boolean comparisons with `=~` THEN they SHALL generate proper `IS` syntax with parentheses
|
|
29
|
-
4. WHEN tests check complex expressions THEN they SHALL expect properly parenthesized SQL
|
|
30
|
-
|
|
31
|
-
### Requirement 3: Standard Qualified Column References
|
|
32
|
-
|
|
33
|
-
**User Story:** As a developer using Sequel with DuckDB, I want qualified column references to use standard SQL dot notation, so that the generated SQL follows established SQL conventions.
|
|
34
|
-
|
|
35
|
-
#### Acceptance Criteria
|
|
36
|
-
|
|
37
|
-
1. WHEN I reference columns across tables THEN they SHALL use `table.column` syntax
|
|
38
|
-
2. WHEN I use qualified column names in subqueries THEN they SHALL use proper dot notation
|
|
39
|
-
3. WHEN I use schema-qualified names THEN they SHALL use standard SQL format
|
|
40
|
-
4. WHEN tests check qualified identifiers THEN they SHALL expect standard dot notation
|
|
41
|
-
|
|
42
|
-
### Requirement 4: Proper Regular Expression Formatting
|
|
43
|
-
|
|
44
|
-
**User Story:** As a developer using Sequel with DuckDB, I want regular expression matching to generate properly formatted SQL with parentheses, so that the SQL follows Sequel's formatting conventions.
|
|
45
|
-
|
|
46
|
-
#### Acceptance Criteria
|
|
47
|
-
|
|
48
|
-
1. WHEN I use regex matching with `~` operator THEN it SHALL generate `(column ~ 'pattern')` with parentheses
|
|
49
|
-
2. WHEN I use case-insensitive regex THEN it SHALL be properly formatted with parentheses
|
|
50
|
-
3. WHEN I use complex regex patterns THEN they SHALL be properly formatted and parenthesized
|
|
51
|
-
4. WHEN tests check regex syntax THEN they SHALL expect properly parenthesized expressions
|
|
52
|
-
|
|
53
|
-
### Requirement 5: Standard Subquery Column References
|
|
54
|
-
|
|
55
|
-
**User Story:** As a developer using Sequel with DuckDB, I want subqueries to properly reference outer query columns using standard SQL dot notation, so that correlated subqueries follow SQL conventions.
|
|
56
|
-
|
|
57
|
-
#### Acceptance Criteria
|
|
58
|
-
|
|
59
|
-
1. WHEN I use correlated subqueries THEN column references SHALL use `table.column` syntax
|
|
60
|
-
2. WHEN I reference outer query columns THEN they SHALL be properly qualified with dot notation
|
|
61
|
-
3. WHEN I use EXISTS subqueries THEN column references SHALL use standard SQL format
|
|
62
|
-
4. WHEN tests check subquery references THEN they SHALL expect standard dot notation
|
|
63
|
-
|
|
64
|
-
### Requirement 6: Consistent SQL Generation Testing
|
|
65
|
-
|
|
66
|
-
**User Story:** As a developer maintaining the sequel-duckdb adapter, I want tests to verify that the adapter generates consistent, standard SQL, so that the adapter follows Sequel's established patterns.
|
|
67
|
-
|
|
68
|
-
#### Acceptance Criteria
|
|
69
|
-
|
|
70
|
-
1. WHEN tests check SQL generation THEN they SHALL expect standard SQL syntax
|
|
71
|
-
2. WHEN the adapter generates SQL THEN it SHALL be consistent with Sequel's conventions
|
|
72
|
-
3. WHEN SQL generation issues are found THEN they SHALL be fixed in the adapter, not worked around in tests
|
|
73
|
-
4. WHEN SQL correctness is verified THEN both functional and syntactic correctness SHALL be maintained
|
|
74
|
-
|
|
75
|
-
### Requirement 7: Documentation of SQL Generation Patterns
|
|
76
|
-
|
|
77
|
-
**User Story:** As a developer using the sequel-duckdb adapter, I want to understand how the adapter generates SQL for DuckDB, so that I can write queries that work optimally with the adapter.
|
|
78
|
-
|
|
79
|
-
#### Acceptance Criteria
|
|
80
|
-
|
|
81
|
-
1. WHEN the adapter generates specific SQL patterns THEN they SHALL be documented
|
|
82
|
-
2. WHEN SQL generation differs from other Sequel adapters THEN the differences SHALL be explained
|
|
83
|
-
3. WHEN DuckDB-specific optimizations are used THEN they SHALL be documented with examples
|
|
84
|
-
4. WHEN SQL generation patterns change THEN documentation SHALL be updated accordingly
|
|
@@ -1,107 +0,0 @@
|
|
|
1
|
-
# Implementation Plan
|
|
2
|
-
|
|
3
|
-
- [x] 1. Identify failing tests and analyze DuckDB SQL generation
|
|
4
|
-
|
|
5
|
-
- Run existing test suite to identify tests failing due to SQL syntax differences
|
|
6
|
-
- Analyze what SQL the adapter currently generates vs what tests expect
|
|
7
|
-
- Document the specific DuckDB syntax patterns that are being generated
|
|
8
|
-
- Determine if adapter changes are needed or just test expectation updates
|
|
9
|
-
- _Requirements: 7.1, 7.2_
|
|
10
|
-
|
|
11
|
-
- [x] 2. Fix LIKE clause ESCAPE handling in adapter
|
|
12
|
-
|
|
13
|
-
- Current issue: LIKE generates `(name LIKE '%John%' ESCAPE '\')` instead of `(name LIKE '%John%')`
|
|
14
|
-
- Override LIKE handling in `complex_expression_sql_append` to remove ESCAPE clause
|
|
15
|
-
- Ensure LIKE clauses generate clean `(name LIKE '%John%')` without ESCAPE clause
|
|
16
|
-
- Write tests to verify LIKE functionality works correctly without the ESCAPE clause
|
|
17
|
-
- _Requirements: 1.1, 1.2, 1.4_
|
|
18
|
-
|
|
19
|
-
- [x] 3. Fix parentheses in complex expression SQL generation
|
|
20
|
-
|
|
21
|
-
- Current issue: ILIKE generates `UPPER(name) LIKE UPPER('%john%')` without outer parentheses
|
|
22
|
-
- Current issue: Regex generates `name ~ 'pattern'` without outer parentheses
|
|
23
|
-
- Update `complex_expression_sql_append` to add proper parentheses around expressions
|
|
24
|
-
- Ensure ILIKE generates `(UPPER(name) LIKE UPPER('%john%'))` with parentheses
|
|
25
|
-
- Ensure regex generates `(name ~ '^John')` with parentheses around the expression
|
|
26
|
-
- Write tests to verify all complex expressions have consistent parentheses
|
|
27
|
-
- _Requirements: 1.3, 5.1, 5.2, 5.3, 5.4_
|
|
28
|
-
|
|
29
|
-
- [x] 4. ~~Fix table alias syntax to use AS instead of triple underscore~~ (REMOVED - Not a core Sequel feature)
|
|
30
|
-
|
|
31
|
-
- The `table___alias` syntax is not a core Sequel feature and requires an extension
|
|
32
|
-
- Removed implementation and tests as this is not standard Sequel functionality
|
|
33
|
-
- Standard Sequel table aliases use `.as()` method: `db[:users].as(:u)`
|
|
34
|
-
|
|
35
|
-
- [x] 5. Fix qualified column references to use dot notation
|
|
36
|
-
|
|
37
|
-
- Current issue: Qualified columns generate `users__id` instead of `users.id`
|
|
38
|
-
- Override qualified identifier handling to generate proper dot notation
|
|
39
|
-
- Implement `qualified_identifier_sql_append` method to use dot notation
|
|
40
|
-
- Ensure subqueries use correct `users.id` column reference format
|
|
41
|
-
- Write tests to verify qualified column references work correctly in complex queries
|
|
42
|
-
- _Requirements: 4.1, 4.2, 4.3, 4.4, 6.1, 6.2, 6.3, 6.4_
|
|
43
|
-
|
|
44
|
-
- [x] 6. Fix SQL test infrastructure issues
|
|
45
|
-
|
|
46
|
-
- Current issue: SQL tests failing with dataset type assertion errors (`Expected #<Sequel::Dataset::_Subclass>`)
|
|
47
|
-
- Fix `SqlTest` class to use proper dataset creation instead of mock subclasses
|
|
48
|
-
- Update SQL generation tests to work with actual DuckDB adapter behavior
|
|
49
|
-
- Ensure tests create proper datasets for SQL generation testing
|
|
50
|
-
- Fix dataset type assertion issues in SQL tests
|
|
51
|
-
- _Requirements: 6.1, 6.2_
|
|
52
|
-
|
|
53
|
-
- [x] 7. Fix recursive CTE SQL generation
|
|
54
|
-
|
|
55
|
-
- Current issue: `WITH RECURSIVE` generates incorrect SQL without RECURSIVE keyword
|
|
56
|
-
- Implement proper recursive CTE handling in dataset methods
|
|
57
|
-
- Ensure recursive CTEs generate `WITH RECURSIVE` syntax correctly
|
|
58
|
-
- Test recursive CTE functionality with proper SQL generation
|
|
59
|
-
- _Requirements: 5.1, 5.2, 5.3, 5.4_
|
|
60
|
-
|
|
61
|
-
- [x] 8. Fix JOIN USING clause generation
|
|
62
|
-
|
|
63
|
-
- Current issue: `JOIN USING` clause not generating USING syntax correctly
|
|
64
|
-
- Implement proper USING clause handling in JOIN operations
|
|
65
|
-
- Ensure JOIN USING generates correct `INNER JOIN table USING (column)` syntax
|
|
66
|
-
- Test JOIN USING functionality with various column combinations
|
|
67
|
-
- _Requirements: 4.1, 4.2, 4.3, 4.4_
|
|
68
|
-
|
|
69
|
-
- [x] 9. Fix regex functionality integration
|
|
70
|
-
|
|
71
|
-
- Current issue: Regex matching returning 0 results instead of expected matches
|
|
72
|
-
- Debug regex operator implementation in complex_expression_sql_append
|
|
73
|
-
- Ensure regex patterns work correctly with DuckDB's regex syntax
|
|
74
|
-
- Test regex functionality with actual data and pattern matching
|
|
75
|
-
- _Requirements: 1.3, 5.1, 5.2, 5.3, 5.4_
|
|
76
|
-
|
|
77
|
-
- [x] 10. Fix model integration issues
|
|
78
|
-
|
|
79
|
-
- Current issue: Model tests failing with type mapping and update detection
|
|
80
|
-
- Fix time type mapping (currently returning :datetime instead of :time)
|
|
81
|
-
- Fix model update detection to properly track changed fields
|
|
82
|
-
- Ensure model DELETE operations work correctly with proper ID handling
|
|
83
|
-
- _Requirements: 2.1, 2.2, 2.3, 2.4_
|
|
84
|
-
|
|
85
|
-
- [x] 11. Fix table creation and schema issues
|
|
86
|
-
|
|
87
|
-
- Current issue: Tests failing because tables don't exist during SQL execution
|
|
88
|
-
- Ensure proper table creation in test setup for integration tests
|
|
89
|
-
- Fix primary key handling for tables without explicit primary keys
|
|
90
|
-
- Handle NOT NULL constraint violations properly in INSERT operations
|
|
91
|
-
- _Requirements: 3.1, 3.2, 3.3, 3.4_
|
|
92
|
-
|
|
93
|
-
- [x] 12. Verify all tests pass with consistent DuckDB SQL generation
|
|
94
|
-
|
|
95
|
-
- Run complete test suite to ensure all SQL generation tests pass
|
|
96
|
-
- Verify that adapter generates consistent, predictable SQL for DuckDB
|
|
97
|
-
- Test integration scenarios to ensure functional correctness
|
|
98
|
-
- Fix any remaining test expectation mismatches
|
|
99
|
-
- _Requirements: 6.1, 6.2, 6.3, 6.4_
|
|
100
|
-
|
|
101
|
-
- [x] 13. Document DuckDB-specific SQL syntax patterns
|
|
102
|
-
|
|
103
|
-
- Document the specific SQL syntax that the adapter generates for DuckDB
|
|
104
|
-
- Provide examples of DuckDB SQL patterns used by the adapter
|
|
105
|
-
- Explain why certain DuckDB syntax choices were made
|
|
106
|
-
- Create reference documentation for developers using the adapter
|
|
107
|
-
- _Requirements: 7.1, 7.2, 7.3_
|
|
@@ -1,32 +0,0 @@
|
|
|
1
|
-
# Requirements Document
|
|
2
|
-
|
|
3
|
-
## Introduction
|
|
4
|
-
|
|
5
|
-
After comprehensive review of the current sequel-duckdb adapter implementation and testing against the actual behavior, all previously identified edge cases and validation issues have been determined to be either:
|
|
6
|
-
|
|
7
|
-
1. **Already implemented** - The adapter includes comprehensive error handling, data type conversion, SQL injection prevention through parameterized queries, connection management, and transaction support.
|
|
8
|
-
|
|
9
|
-
2. **Handled by Sequel core** - Query parameter validation (LIMIT/OFFSET edge cases) is handled by the Sequel framework itself before reaching database adapters.
|
|
10
|
-
|
|
11
|
-
3. **Working as designed** - DuckDB's native behavior for edge cases is appropriate and doesn't require additional adapter-level handling.
|
|
12
|
-
|
|
13
|
-
4. **Out of scope** - Some edge cases (like system resource management) are better handled at the application or infrastructure level rather than in a database adapter.
|
|
14
|
-
|
|
15
|
-
## Current Status
|
|
16
|
-
|
|
17
|
-
The sequel-duckdb adapter currently provides:
|
|
18
|
-
|
|
19
|
-
- ✅ Comprehensive error handling and mapping to appropriate Sequel exception types
|
|
20
|
-
- ✅ Robust connection management with proper error handling
|
|
21
|
-
- ✅ Complete data type conversion and validation
|
|
22
|
-
- ✅ SQL injection prevention through parameterized queries
|
|
23
|
-
- ✅ Transaction support with proper rollback handling
|
|
24
|
-
- ✅ Schema introspection and DDL operations
|
|
25
|
-
- ✅ Performance optimizations for large datasets
|
|
26
|
-
- ✅ Memory management for streaming operations
|
|
27
|
-
|
|
28
|
-
## Conclusion
|
|
29
|
-
|
|
30
|
-
No additional edge case or validation requirements have been identified that would add meaningful value to the adapter. The current implementation provides robust handling of edge cases appropriate for a production database adapter.
|
|
31
|
-
|
|
32
|
-
This specification is considered complete with no outstanding requirements.
|
|
File without changes
|
|
@@ -1,117 +0,0 @@
|
|
|
1
|
-
# Requirements Document
|
|
2
|
-
|
|
3
|
-
## Introduction
|
|
4
|
-
|
|
5
|
-
This specification addresses database schema and setup issues in integration tests for the sequel-duckdb adapter. Many integration tests are failing because they attempt to perform database operations on tables that don't exist or haven't been properly set up. The test infrastructure needs proper database setup, schema management, and teardown procedures to ensure integration tests run reliably.
|
|
6
|
-
|
|
7
|
-
## Requirements
|
|
8
|
-
|
|
9
|
-
### Requirement 1: Automatic Test Database Setup
|
|
10
|
-
|
|
11
|
-
**User Story:** As a developer running integration tests for the sequel-duckdb adapter, I want test databases to be automatically set up with required schemas, so that tests can focus on functionality rather than database preparation.
|
|
12
|
-
|
|
13
|
-
#### Acceptance Criteria
|
|
14
|
-
|
|
15
|
-
1. WHEN integration tests start THEN required test tables SHALL be automatically created
|
|
16
|
-
2. WHEN tests need specific schemas THEN they SHALL be set up before test execution
|
|
17
|
-
3. WHEN tests require sample data THEN it SHALL be inserted during setup
|
|
18
|
-
4. WHEN test setup fails THEN clear error messages SHALL indicate the specific failure
|
|
19
|
-
|
|
20
|
-
### Requirement 2: Test Table Schema Management
|
|
21
|
-
|
|
22
|
-
**User Story:** As a developer writing integration tests for the sequel-duckdb adapter, I want standardized test table schemas that cover all data types and scenarios, so that I can test comprehensive functionality without custom setup.
|
|
23
|
-
|
|
24
|
-
#### Acceptance Criteria
|
|
25
|
-
|
|
26
|
-
1. WHEN tests need a users table THEN it SHALL include common columns (id, name, email, age, active, created_at)
|
|
27
|
-
2. WHEN tests need type testing THEN tables SHALL include all supported DuckDB data types
|
|
28
|
-
3. WHEN tests need relationship testing THEN foreign key relationships SHALL be properly set up
|
|
29
|
-
4. WHEN tests need constraint testing THEN appropriate constraints SHALL be defined
|
|
30
|
-
|
|
31
|
-
### Requirement 3: Test Data Fixtures
|
|
32
|
-
|
|
33
|
-
**User Story:** As a developer running integration tests for the sequel-duckdb adapter, I want consistent test data fixtures, so that tests produce predictable results and can be easily debugged.
|
|
34
|
-
|
|
35
|
-
#### Acceptance Criteria
|
|
36
|
-
|
|
37
|
-
1. WHEN tests need sample users THEN standardized user records SHALL be available
|
|
38
|
-
2. WHEN tests need relational data THEN properly linked records SHALL be provided
|
|
39
|
-
3. WHEN tests need edge case data THEN fixtures SHALL include boundary values and special cases
|
|
40
|
-
4. WHEN tests need large datasets THEN performance test fixtures SHALL be available
|
|
41
|
-
|
|
42
|
-
### Requirement 4: Database Cleanup and Isolation
|
|
43
|
-
|
|
44
|
-
**User Story:** As a developer running integration tests for the sequel-duckdb adapter, I want each test to run in isolation with a clean database state, so that tests don't interfere with each other and produce consistent results.
|
|
45
|
-
|
|
46
|
-
#### Acceptance Criteria
|
|
47
|
-
|
|
48
|
-
1. WHEN each test starts THEN it SHALL have a clean database state
|
|
49
|
-
2. WHEN tests modify data THEN changes SHALL not affect subsequent tests
|
|
50
|
-
3. WHEN tests create temporary tables THEN they SHALL be cleaned up after the test
|
|
51
|
-
4. WHEN tests fail THEN database state SHALL be reset for the next test
|
|
52
|
-
|
|
53
|
-
### Requirement 5: Schema Introspection Test Support
|
|
54
|
-
|
|
55
|
-
**User Story:** As a developer testing schema introspection features of the sequel-duckdb adapter, I want test databases with comprehensive schema elements, so that I can verify all introspection functionality works correctly.
|
|
56
|
-
|
|
57
|
-
#### Acceptance Criteria
|
|
58
|
-
|
|
59
|
-
1. WHEN testing table listing THEN multiple test tables SHALL exist
|
|
60
|
-
2. WHEN testing column introspection THEN tables SHALL have diverse column types and properties
|
|
61
|
-
3. WHEN testing index introspection THEN various index types SHALL be present
|
|
62
|
-
4. WHEN testing constraint introspection THEN different constraint types SHALL be available
|
|
63
|
-
|
|
64
|
-
### Requirement 6: Transaction Testing Support
|
|
65
|
-
|
|
66
|
-
**User Story:** As a developer testing transaction functionality of the sequel-duckdb adapter, I want test scenarios that properly exercise transaction behavior, so that I can verify commit, rollback, and isolation work correctly.
|
|
67
|
-
|
|
68
|
-
#### Acceptance Criteria
|
|
69
|
-
|
|
70
|
-
1. WHEN testing transactions THEN test data SHALL support rollback verification
|
|
71
|
-
2. WHEN testing nested transactions THEN appropriate test scenarios SHALL be available
|
|
72
|
-
3. WHEN testing transaction isolation THEN concurrent test scenarios SHALL be supported
|
|
73
|
-
4. WHEN testing transaction errors THEN error conditions SHALL be reproducible
|
|
74
|
-
|
|
75
|
-
### Requirement 7: Performance Testing Database Setup
|
|
76
|
-
|
|
77
|
-
**User Story:** As a developer testing performance aspects of the sequel-duckdb adapter, I want test databases with appropriate data volumes, so that I can verify performance optimizations work correctly.
|
|
78
|
-
|
|
79
|
-
#### Acceptance Criteria
|
|
80
|
-
|
|
81
|
-
1. WHEN testing bulk operations THEN large datasets SHALL be available
|
|
82
|
-
2. WHEN testing query performance THEN indexed and non-indexed scenarios SHALL be set up
|
|
83
|
-
3. WHEN testing memory usage THEN datasets of various sizes SHALL be available
|
|
84
|
-
4. WHEN testing streaming THEN large result sets SHALL be available for testing
|
|
85
|
-
|
|
86
|
-
### Requirement 8: Error Condition Testing Setup
|
|
87
|
-
|
|
88
|
-
**User Story:** As a developer testing error handling in the sequel-duckdb adapter, I want test scenarios that reliably reproduce error conditions, so that I can verify proper error handling and exception mapping.
|
|
89
|
-
|
|
90
|
-
#### Acceptance Criteria
|
|
91
|
-
|
|
92
|
-
1. WHEN testing constraint violations THEN tables with constraints SHALL be available
|
|
93
|
-
2. WHEN testing connection errors THEN invalid database scenarios SHALL be reproducible
|
|
94
|
-
3. WHEN testing SQL errors THEN scenarios that trigger DuckDB errors SHALL be available
|
|
95
|
-
4. WHEN testing type errors THEN incompatible data scenarios SHALL be set up
|
|
96
|
-
|
|
97
|
-
### Requirement 9: Test Database Configuration
|
|
98
|
-
|
|
99
|
-
**User Story:** As a developer running integration tests for the sequel-duckdb adapter, I want flexible test database configuration, so that tests can run in different environments and scenarios.
|
|
100
|
-
|
|
101
|
-
#### Acceptance Criteria
|
|
102
|
-
|
|
103
|
-
1. WHEN tests run in CI THEN database configuration SHALL be automatically appropriate
|
|
104
|
-
2. WHEN tests run locally THEN database configuration SHALL support development workflows
|
|
105
|
-
3. WHEN tests need specific DuckDB settings THEN configuration SHALL be easily adjustable
|
|
106
|
-
4. WHEN tests need different database sizes THEN memory limits SHALL be configurable
|
|
107
|
-
|
|
108
|
-
### Requirement 10: Test Helper Integration
|
|
109
|
-
|
|
110
|
-
**User Story:** As a developer writing integration tests for the sequel-duckdb adapter, I want test helpers that work seamlessly with the database setup, so that I can write tests efficiently without boilerplate code.
|
|
111
|
-
|
|
112
|
-
#### Acceptance Criteria
|
|
113
|
-
|
|
114
|
-
1. WHEN using test helpers THEN they SHALL work with the established database schema
|
|
115
|
-
2. WHEN creating test data THEN helpers SHALL use the standardized fixtures
|
|
116
|
-
3. WHEN verifying results THEN helpers SHALL understand the test database structure
|
|
117
|
-
4. WHEN cleaning up THEN helpers SHALL properly reset database state
|