sequel-duckdb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.kiro/specs/advanced-sql-features-implementation/design.md +24 -0
- data/.kiro/specs/advanced-sql-features-implementation/requirements.md +43 -0
- data/.kiro/specs/advanced-sql-features-implementation/tasks.md +24 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +258 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +84 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +94 -0
- data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +32 -0
- data/.kiro/specs/integration-test-database-setup/design.md +0 -0
- data/.kiro/specs/integration-test-database-setup/requirements.md +117 -0
- data/.kiro/specs/sequel-duckdb-adapter/design.md +542 -0
- data/.kiro/specs/sequel-duckdb-adapter/requirements.md +202 -0
- data/.kiro/specs/sequel-duckdb-adapter/tasks.md +247 -0
- data/.kiro/specs/sql-expression-handling-fix/design.md +298 -0
- data/.kiro/specs/sql-expression-handling-fix/requirements.md +86 -0
- data/.kiro/specs/sql-expression-handling-fix/tasks.md +22 -0
- data/.kiro/specs/test-infrastructure-improvements/requirements.md +106 -0
- data/.kiro/steering/product.md +22 -0
- data/.kiro/steering/structure.md +88 -0
- data/.kiro/steering/tech.md +124 -0
- data/.kiro/steering/testing.md +192 -0
- data/.rubocop.yml +103 -0
- data/.yardopts +8 -0
- data/API_DOCUMENTATION.md +919 -0
- data/CHANGELOG.md +131 -0
- data/LICENSE +21 -0
- data/MIGRATION_EXAMPLES.md +740 -0
- data/PERFORMANCE_OPTIMIZATIONS.md +723 -0
- data/README.md +692 -0
- data/Rakefile +27 -0
- data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +164 -0
- data/docs/DUCKDB_SQL_PATTERNS.md +410 -0
- data/docs/TASK_12_VERIFICATION_SUMMARY.md +122 -0
- data/lib/sequel/adapters/duckdb.rb +256 -0
- data/lib/sequel/adapters/shared/duckdb.rb +2349 -0
- data/lib/sequel/duckdb/version.rb +16 -0
- data/lib/sequel/duckdb.rb +43 -0
- data/sig/sequel/duckdb.rbs +6 -0
- metadata +235 -0
@@ -0,0 +1,298 @@
|
|
1
|
+
# Design Document: SQL Expression Handling Fix
|
2
|
+
|
3
|
+
## Overview
|
4
|
+
|
5
|
+
This design addresses critical SQL expression handling issues in the sequel-duckdb adapter where SQL expressions, functions, and literal strings are incorrectly treated as regular string literals and quoted when they should be rendered as raw SQL. The fix involves implementing proper type detection and handling in the `literal_append` method to distinguish between different SQL object types.
|
6
|
+
|
7
|
+
The core issue is that the current adapter implementation doesn't properly handle Sequel's expression objects (`Sequel::LiteralString`, `Sequel::SQL::Function`, etc.) and treats them as regular Ruby strings, causing them to be quoted inappropriately in the generated SQL.
|
8
|
+
|
9
|
+
## Architecture
|
10
|
+
|
11
|
+
### Current Problem
|
12
|
+
|
13
|
+
The existing `literal_append` method in the DuckDB adapter handles `String` objects without first checking if they are `Sequel::LiteralString` objects. This causes `LiteralString` objects (created by `Sequel.lit()`) to be treated as regular strings and quoted inappropriately. The method correctly handles `Time` and `DateTime` objects but falls through to `literal_string_append` for all `String` objects, including `LiteralString`.
|
14
|
+
|
15
|
+
### Solution Architecture
|
16
|
+
|
17
|
+
The solution follows Sequel core's established pattern by checking for `LiteralString` as a special case of `String` before applying string quoting:
|
18
|
+
|
19
|
+
1. **Sequel Core Pattern Compliance**: Follows the exact pattern used in Sequel core's `literal_append` method
|
20
|
+
2. **LiteralString Special Handling**: Checks for `LiteralString` before regular `String` processing
|
21
|
+
3. **Minimal Change**: Only adds the missing `LiteralString` check to existing logic
|
22
|
+
4. **Parent Delegation**: Continues to delegate `SQL::Function` and other expressions to parent class
|
23
|
+
|
24
|
+
### Design Pattern
|
25
|
+
|
26
|
+
Following Sequel's adapter pattern, the fix will be implemented in the `DatasetMethods` module within `lib/sequel/adapters/shared/duckdb.rb`, allowing the main `Dataset` class to inherit the corrected behavior through the include mechanism.
|
27
|
+
|
28
|
+
## Components and Interfaces
|
29
|
+
|
30
|
+
### Core Component: Enhanced literal_append Method
|
31
|
+
|
32
|
+
**Location**: `Sequel::DuckDB::DatasetMethods` module
|
33
|
+
|
34
|
+
**Interface**:
|
35
|
+
```ruby
|
36
|
+
def literal_append(sql, v)
|
37
|
+
case v
|
38
|
+
when Time
|
39
|
+
literal_datetime_append(sql, v)
|
40
|
+
when DateTime
|
41
|
+
literal_datetime_append(sql, v)
|
42
|
+
when String
|
43
|
+
case v
|
44
|
+
when LiteralString
|
45
|
+
sql << v # Append directly without quoting
|
46
|
+
else
|
47
|
+
if v.encoding == Encoding::ASCII_8BIT
|
48
|
+
literal_blob_append(sql, v)
|
49
|
+
else
|
50
|
+
literal_string_append(sql, v)
|
51
|
+
end
|
52
|
+
end
|
53
|
+
else
|
54
|
+
super
|
55
|
+
end
|
56
|
+
end
|
57
|
+
```
|
58
|
+
|
59
|
+
### Type Handling Components
|
60
|
+
|
61
|
+
#### 1. LiteralString Handler
|
62
|
+
- **Purpose**: Process `Sequel.lit()` expressions as raw SQL following Sequel core pattern
|
63
|
+
- **Behavior**: Append string content directly without quoting (`sql << v`)
|
64
|
+
- **Input**: `Sequel::LiteralString` objects (subclass of `String`)
|
65
|
+
- **Output**: Raw SQL string appended to query
|
66
|
+
|
67
|
+
#### 2. Regular String Handler
|
68
|
+
- **Purpose**: Process regular Ruby strings with appropriate encoding handling
|
69
|
+
- **Behavior**: Apply existing binary/text string logic
|
70
|
+
- **Input**: Regular `String` objects (excluding `LiteralString`)
|
71
|
+
- **Output**: Properly quoted and escaped string literals
|
72
|
+
|
73
|
+
#### 3. Parent Class Delegation
|
74
|
+
- **Purpose**: Handle all other data types including `SQL::Function`
|
75
|
+
- **Behavior**: Delegate to parent class implementation (which already works correctly)
|
76
|
+
- **Input**: All other Ruby/Sequel objects including `SQL::Expression` subclasses
|
77
|
+
- **Output**: Appropriately formatted literals using Sequel core logic
|
78
|
+
|
79
|
+
### Integration Points
|
80
|
+
|
81
|
+
#### Dataset Class Integration
|
82
|
+
The enhanced `literal_append` method integrates with:
|
83
|
+
- **SQL Generation Pipeline**: Core Sequel SQL building process
|
84
|
+
- **Query Context Handling**: SELECT, WHERE, ORDER BY, GROUP BY, HAVING clauses
|
85
|
+
- **Expression Composition**: Nested and complex expressions
|
86
|
+
|
87
|
+
#### Sequel Framework Integration
|
88
|
+
- **Parent Class Delegation**: Leverages existing Sequel functionality
|
89
|
+
- **Type System Compatibility**: Works with Sequel's expression type hierarchy
|
90
|
+
- **Error Handling**: Integrates with Sequel's error reporting system
|
91
|
+
|
92
|
+
## Data Models
|
93
|
+
|
94
|
+
### Input Data Types
|
95
|
+
|
96
|
+
#### Sequel::LiteralString
|
97
|
+
```ruby
|
98
|
+
# Created by: Sequel.lit("YEAR(created_at)")
|
99
|
+
# Properties:
|
100
|
+
# - Contains raw SQL string
|
101
|
+
# - Should not be quoted
|
102
|
+
# - Used for expressions, functions, literals
|
103
|
+
```
|
104
|
+
|
105
|
+
#### Sequel::SQL::Function
|
106
|
+
```ruby
|
107
|
+
# Created by: Sequel.function(:count, :*)
|
108
|
+
# Properties:
|
109
|
+
# - Function name and arguments
|
110
|
+
# - Requires special rendering logic
|
111
|
+
# - May contain nested expressions
|
112
|
+
```
|
113
|
+
|
114
|
+
#### Regular Data Types
|
115
|
+
```ruby
|
116
|
+
# String, Integer, Float, Date, Time, etc.
|
117
|
+
# Properties:
|
118
|
+
# - Standard Ruby types
|
119
|
+
# - Require appropriate SQL formatting
|
120
|
+
# - Should be quoted/escaped as needed
|
121
|
+
```
|
122
|
+
|
123
|
+
### Output Data Model
|
124
|
+
|
125
|
+
#### SQL String Buffer
|
126
|
+
- **Type**: String (mutable)
|
127
|
+
- **Content**: Accumulated SQL query text
|
128
|
+
- **Modification**: Appended to by `literal_append`
|
129
|
+
|
130
|
+
## Error Handling
|
131
|
+
|
132
|
+
### Error Categories
|
133
|
+
|
134
|
+
#### 1. Unsupported Expression Types
|
135
|
+
- **Scenario**: Unknown SQL expression object encountered
|
136
|
+
- **Response**: Clear error message with object type information
|
137
|
+
- **Error Type**: `Sequel::Error` or subclass
|
138
|
+
|
139
|
+
#### 2. Expression Rendering Failures
|
140
|
+
- **Scenario**: Function or expression rendering fails
|
141
|
+
- **Response**: Context-aware error with failing expression details
|
142
|
+
- **Error Type**: `Sequel::DatabaseError`
|
143
|
+
|
144
|
+
#### 3. SQL Generation Failures
|
145
|
+
- **Scenario**: Overall SQL generation fails due to expression issues
|
146
|
+
- **Response**: Categorized error with query context
|
147
|
+
- **Error Type**: `Sequel::DatabaseError`
|
148
|
+
|
149
|
+
### Error Handling Strategy
|
150
|
+
|
151
|
+
```ruby
|
152
|
+
def literal_append(sql, v)
|
153
|
+
case v
|
154
|
+
when Time
|
155
|
+
literal_datetime_append(sql, v)
|
156
|
+
when DateTime
|
157
|
+
literal_datetime_append(sql, v)
|
158
|
+
when String
|
159
|
+
case v
|
160
|
+
when LiteralString
|
161
|
+
sql << v
|
162
|
+
else
|
163
|
+
if v.encoding == Encoding::ASCII_8BIT
|
164
|
+
literal_blob_append(sql, v)
|
165
|
+
else
|
166
|
+
literal_string_append(sql, v)
|
167
|
+
end
|
168
|
+
end
|
169
|
+
else
|
170
|
+
super
|
171
|
+
end
|
172
|
+
rescue => e
|
173
|
+
raise Sequel::DatabaseError, "Failed to render SQL literal: #{e.message}"
|
174
|
+
end
|
175
|
+
```
|
176
|
+
|
177
|
+
## Testing Strategy
|
178
|
+
|
179
|
+
### Test Categories
|
180
|
+
|
181
|
+
#### 1. Unit Tests - SQL Generation
|
182
|
+
- **Framework**: Sequel mock database
|
183
|
+
- **Purpose**: Test SQL generation without database connections
|
184
|
+
- **Coverage**: All expression types and combinations
|
185
|
+
- **Location**: `test/sql_test.rb`
|
186
|
+
|
187
|
+
#### 2. Integration Tests - Database Operations
|
188
|
+
- **Framework**: Real DuckDB in-memory databases
|
189
|
+
- **Purpose**: End-to-end expression functionality
|
190
|
+
- **Coverage**: Query execution with expressions
|
191
|
+
- **Location**: `test/dataset_test.rb`
|
192
|
+
|
193
|
+
#### 3. Regression Tests
|
194
|
+
- **Purpose**: Ensure backward compatibility
|
195
|
+
- **Coverage**: Existing functionality continues working
|
196
|
+
- **Location**: Multiple test files
|
197
|
+
|
198
|
+
### Test Implementation Approach
|
199
|
+
|
200
|
+
#### Test-Driven Development
|
201
|
+
1. **Write failing tests** for each expression type
|
202
|
+
2. **Implement minimal fix** to make tests pass
|
203
|
+
3. **Refactor** while maintaining green tests
|
204
|
+
4. **Add edge case tests** and handle them
|
205
|
+
|
206
|
+
#### Test Structure
|
207
|
+
```ruby
|
208
|
+
def test_literal_string_handling
|
209
|
+
# Test Sequel.lit() expressions
|
210
|
+
dataset = @db[:users].select(Sequel.lit("YEAR(created_at)"))
|
211
|
+
assert_equal "SELECT YEAR(created_at) FROM users", dataset.sql
|
212
|
+
end
|
213
|
+
|
214
|
+
def test_function_handling_still_works
|
215
|
+
# Test that Sequel.function() calls continue to work (already working)
|
216
|
+
dataset = @db[:users].select(Sequel.function(:count, :*))
|
217
|
+
assert_equal "SELECT count(*) FROM users", dataset.sql
|
218
|
+
end
|
219
|
+
|
220
|
+
def test_regular_string_still_quoted
|
221
|
+
# Test that regular strings are still properly quoted
|
222
|
+
dataset = @db[:users].where(name: "John's")
|
223
|
+
assert_equal "SELECT * FROM users WHERE (name = 'John''s')", dataset.sql
|
224
|
+
end
|
225
|
+
```
|
226
|
+
|
227
|
+
### Coverage Requirements
|
228
|
+
- **100% line coverage** for new `literal_append` method
|
229
|
+
- **Branch coverage** for all case statements
|
230
|
+
- **Edge case coverage** for error conditions
|
231
|
+
- **Integration coverage** for all SQL clause types
|
232
|
+
|
233
|
+
## Design Decisions and Rationales
|
234
|
+
|
235
|
+
### Decision 1: Follow Sequel Core Pattern Exactly
|
236
|
+
**Rationale**: Sequel core already has the correct pattern for handling `LiteralString` as a special case of `String`. Following this established pattern ensures compatibility and maintainability.
|
237
|
+
|
238
|
+
**Evidence**: Sequel core's `literal_append` method in `/sequel/dataset/sql.rb` shows the exact pattern:
|
239
|
+
```ruby
|
240
|
+
when String
|
241
|
+
case v
|
242
|
+
when LiteralString
|
243
|
+
sql << v
|
244
|
+
when SQL::Blob
|
245
|
+
literal_blob_append(sql, v)
|
246
|
+
else
|
247
|
+
literal_string_append(sql, v)
|
248
|
+
end
|
249
|
+
```
|
250
|
+
|
251
|
+
### Decision 2: Minimal Modification Approach
|
252
|
+
**Rationale**: The current implementation already works correctly for most types. Only the `LiteralString` handling is missing, so we add the minimal necessary check.
|
253
|
+
|
254
|
+
**Alternatives Considered**:
|
255
|
+
- Complete rewrite of literal_append (rejected - unnecessary and risky)
|
256
|
+
- Separate method for LiteralString (rejected - doesn't follow Sequel patterns)
|
257
|
+
|
258
|
+
### Decision 3: No Changes to Function Handling
|
259
|
+
**Rationale**: Testing revealed that `SQL::Function` objects already work correctly through parent class delegation. The issue is specifically with `LiteralString` objects being treated as regular strings.
|
260
|
+
|
261
|
+
**Evidence**:
|
262
|
+
- `db[:test].select(Sequel.function(:count, :*)).sql` produces correct `"SELECT count(*) FROM test"`
|
263
|
+
- `db[:test].select(Sequel.lit('YEAR(created_at)')).sql` incorrectly produces `"SELECT 'YEAR(created_at)' FROM test"`
|
264
|
+
|
265
|
+
### Decision 4: Preserve Existing Time/DateTime/Binary Handling
|
266
|
+
**Rationale**: The current implementation correctly handles `Time`, `DateTime`, and binary string encoding. These should remain unchanged.
|
267
|
+
|
268
|
+
**Benefits**:
|
269
|
+
- Maintains backward compatibility for existing functionality
|
270
|
+
- Focuses fix on the specific problem area
|
271
|
+
- Reduces risk of regression in working features
|
272
|
+
|
273
|
+
### Decision 5: Integration in DatasetMethods Module
|
274
|
+
**Rationale**: Following the established sequel-duckdb pattern where shared functionality is implemented in modules and included in main classes.
|
275
|
+
|
276
|
+
**Benefits**:
|
277
|
+
- Consistent with existing codebase architecture
|
278
|
+
- Allows for easy testing and maintenance
|
279
|
+
- Follows Sequel adapter conventions
|
280
|
+
|
281
|
+
## Implementation Phases
|
282
|
+
|
283
|
+
### Phase 1: Fix LiteralString Handling
|
284
|
+
- Modify existing `literal_append` method to check for `LiteralString` before regular `String`
|
285
|
+
- Add comprehensive unit tests for `LiteralString` handling
|
286
|
+
- Verify existing functionality remains intact
|
287
|
+
|
288
|
+
### Phase 2: Comprehensive Testing
|
289
|
+
- Add integration tests with real database operations
|
290
|
+
- Test all SQL clause contexts (SELECT, WHERE, ORDER BY, etc.)
|
291
|
+
- Add regression tests to ensure no existing functionality breaks
|
292
|
+
|
293
|
+
### Phase 3: Edge Cases and Error Handling
|
294
|
+
- Test complex nested expressions
|
295
|
+
- Verify error handling for malformed expressions
|
296
|
+
- Performance verification with existing benchmarks
|
297
|
+
|
298
|
+
This design provides a focused, low-risk solution that addresses the core SQL expression handling issues while maintaining full backward compatibility and following established Sequel adapter patterns.
|
@@ -0,0 +1,86 @@
|
|
1
|
+
# Requirements Document
|
2
|
+
|
3
|
+
## Introduction
|
4
|
+
|
5
|
+
This specification addresses critical SQL expression handling issues in the sequel-duckdb adapter. The adapter is currently incorrectly treating SQL expressions, functions, and literal strings as regular string literals, causing them to be quoted when they should be rendered as raw SQL. This breaks fundamental SQL generation functionality and prevents proper use of database functions, expressions, and literal SQL.
|
6
|
+
|
7
|
+
## Requirements
|
8
|
+
|
9
|
+
### Requirement 1: Sequel::LiteralString Handling
|
10
|
+
|
11
|
+
**User Story:** As a developer using Sequel with DuckDB, I want to use `Sequel.lit()` to include raw SQL expressions in my queries, so that I can use database functions and complex expressions without them being quoted as strings.
|
12
|
+
|
13
|
+
#### Acceptance Criteria
|
14
|
+
|
15
|
+
1. WHEN I use `Sequel.lit("YEAR(created_at)")` in a SELECT clause THEN the generated SQL SHALL contain `YEAR(created_at)` without quotes
|
16
|
+
2. WHEN I use `Sequel.lit("LENGTH(name) > 5")` in a WHERE clause THEN the generated SQL SHALL contain `LENGTH(name) > 5` without quotes
|
17
|
+
3. WHEN I use `Sequel.lit("CURRENT_TIMESTAMP")` in an UPDATE clause THEN the generated SQL SHALL contain `CURRENT_TIMESTAMP` without quotes
|
18
|
+
4. WHEN I use `Sequel.lit("name || ' ' || email AS full_info")` in a SELECT clause THEN the generated SQL SHALL contain the expression without quotes
|
19
|
+
|
20
|
+
### Requirement 2: Sequel::SQL::Function Handling
|
21
|
+
|
22
|
+
**User Story:** As a developer using Sequel with DuckDB, I want to use `Sequel.function()` to call database functions, so that function calls are properly generated in SQL without being quoted as strings.
|
23
|
+
|
24
|
+
#### Acceptance Criteria
|
25
|
+
|
26
|
+
1. WHEN I use `Sequel.function(:count, :*)` THEN the generated SQL SHALL contain `count(*)`
|
27
|
+
2. WHEN I use `Sequel.function(:sum, :amount)` THEN the generated SQL SHALL contain `sum(amount)`
|
28
|
+
3. WHEN I use `Sequel.function(:year, :created_at)` THEN the generated SQL SHALL contain `year(created_at)`
|
29
|
+
4. WHEN I use nested functions like `Sequel.function(:count, Sequel.function(:distinct, :name))` THEN the generated SQL SHALL contain `count(distinct(name))`
|
30
|
+
|
31
|
+
### Requirement 3: Complex Expression Handling
|
32
|
+
|
33
|
+
**User Story:** As a developer using Sequel with DuckDB, I want to use complex SQL expressions with operators and functions, so that they are properly rendered as SQL without being quoted.
|
34
|
+
|
35
|
+
#### Acceptance Criteria
|
36
|
+
|
37
|
+
1. WHEN I use expressions with mathematical operators THEN they SHALL be rendered as raw SQL
|
38
|
+
2. WHEN I use expressions with string concatenation operators THEN they SHALL be rendered as raw SQL
|
39
|
+
3. WHEN I use expressions with comparison operators THEN they SHALL be rendered as raw SQL
|
40
|
+
4. WHEN I use expressions with logical operators THEN they SHALL be rendered as raw SQL
|
41
|
+
|
42
|
+
### Requirement 4: Literal Append Method Override
|
43
|
+
|
44
|
+
**User Story:** As a developer maintaining the sequel-duckdb adapter, I want the `literal_append` method to properly distinguish between different types of SQL objects, so that each type is handled appropriately.
|
45
|
+
|
46
|
+
#### Acceptance Criteria
|
47
|
+
|
48
|
+
1. WHEN `literal_append` receives a `Sequel::LiteralString` object THEN it SHALL append the string content without quotes
|
49
|
+
2. WHEN `literal_append` receives a `Sequel::SQL::Function` object THEN it SHALL delegate to the appropriate function rendering method
|
50
|
+
3. WHEN `literal_append` receives a regular String object THEN it SHALL quote it as a string literal
|
51
|
+
4. WHEN `literal_append` receives other SQL expression objects THEN it SHALL delegate to the parent class method
|
52
|
+
|
53
|
+
### Requirement 5: Expression Context Preservation
|
54
|
+
|
55
|
+
**User Story:** As a developer using Sequel with DuckDB, I want SQL expressions to maintain their context when used in different parts of queries, so that they work correctly in SELECT, WHERE, ORDER BY, GROUP BY, and HAVING clauses.
|
56
|
+
|
57
|
+
#### Acceptance Criteria
|
58
|
+
|
59
|
+
1. WHEN I use expressions in SELECT clauses THEN they SHALL be rendered as raw SQL
|
60
|
+
2. WHEN I use expressions in WHERE clauses THEN they SHALL be rendered as raw SQL
|
61
|
+
3. WHEN I use expressions in ORDER BY clauses THEN they SHALL be rendered as raw SQL
|
62
|
+
4. WHEN I use expressions in GROUP BY clauses THEN they SHALL be rendered as raw SQL
|
63
|
+
5. WHEN I use expressions in HAVING clauses THEN they SHALL be rendered as raw SQL
|
64
|
+
6. WHEN I use expressions in UPDATE SET clauses THEN they SHALL be rendered as raw SQL
|
65
|
+
|
66
|
+
### Requirement 6: Backward Compatibility
|
67
|
+
|
68
|
+
**User Story:** As a developer using the sequel-duckdb adapter, I want existing functionality to continue working after expression handling is fixed, so that my current code doesn't break.
|
69
|
+
|
70
|
+
#### Acceptance Criteria
|
71
|
+
|
72
|
+
1. WHEN I use regular string values in queries THEN they SHALL still be properly quoted as string literals
|
73
|
+
2. WHEN I use numeric values in queries THEN they SHALL still be rendered as numeric literals
|
74
|
+
3. WHEN I use boolean values in queries THEN they SHALL still be rendered as boolean literals
|
75
|
+
4. WHEN I use date/time values in queries THEN they SHALL still be rendered with proper formatting
|
76
|
+
5. WHEN I use binary data in queries THEN it SHALL still be rendered as hex literals
|
77
|
+
|
78
|
+
### Requirement 7: Error Handling
|
79
|
+
|
80
|
+
**User Story:** As a developer using the sequel-duckdb adapter, I want clear error messages when expression handling fails, so that I can debug issues effectively.
|
81
|
+
|
82
|
+
#### Acceptance Criteria
|
83
|
+
|
84
|
+
1. WHEN an unsupported expression type is encountered THEN a clear error message SHALL be provided
|
85
|
+
2. WHEN expression rendering fails THEN the error SHALL include context about the failing expression
|
86
|
+
3. WHEN SQL generation fails due to expression issues THEN the error SHALL be properly categorized as a DatabaseError
|
@@ -0,0 +1,22 @@
|
|
1
|
+
# Implementation Plan
|
2
|
+
|
3
|
+
- [x] 1. Write tests for LiteralString handling
|
4
|
+
- Add test in `test/sql_test.rb` to verify `Sequel.lit()` expressions are not quoted
|
5
|
+
- Test LiteralString in SELECT clause: `db[:test].select(Sequel.lit('YEAR(created_at)')).sql`
|
6
|
+
- Test LiteralString in WHERE clause: `db[:test].where(Sequel.lit('age > 18')).sql`
|
7
|
+
- Add regression test to ensure regular strings are still quoted properly
|
8
|
+
- Add test to ensure SQL::Function continues working (should be unchanged)
|
9
|
+
- _Requirements: 1.1, 1.2, 4.1, 6.1_
|
10
|
+
|
11
|
+
- [x] 2. Fix literal_append method to handle LiteralString
|
12
|
+
- Modify existing `literal_append` method in `lib/sequel/adapters/shared/duckdb.rb`
|
13
|
+
- Add `LiteralString` check as special case of `String` following Sequel core pattern
|
14
|
+
- Change `when String` to nested case with `when LiteralString` that does `sql << v`
|
15
|
+
- Preserve all existing logic for Time, DateTime, and binary strings
|
16
|
+
- _Requirements: 1.1, 4.1, 4.2, 4.4, 6.1_
|
17
|
+
|
18
|
+
- [x] 3. Add integration test with real database
|
19
|
+
- Add test in `test/dataset_test.rb` using real DuckDB database
|
20
|
+
- Test that LiteralString expressions actually execute correctly
|
21
|
+
- Verify no regression in existing database operations
|
22
|
+
- _Requirements: 1.2, 5.1, 6.1_
|
@@ -0,0 +1,106 @@
|
|
1
|
+
# Requirements Document
|
2
|
+
|
3
|
+
## Introduction
|
4
|
+
|
5
|
+
This specification addresses test infrastructure issues in the sequel-duckdb adapter test suite. Many tests are failing due to incorrect assertions, wrong expectations about dataset types, and improper test setup rather than actual functionality problems. The test infrastructure needs to be improved to properly test the adapter's functionality while being flexible about implementation details.
|
6
|
+
|
7
|
+
## Requirements
|
8
|
+
|
9
|
+
### Requirement 1: Dataset Type Assertion Fixes
|
10
|
+
|
11
|
+
**User Story:** As a developer maintaining the sequel-duckdb adapter, I want tests to check functionality rather than exact class types, so that tests pass when the adapter works correctly regardless of internal Sequel implementation details.
|
12
|
+
|
13
|
+
#### Acceptance Criteria
|
14
|
+
|
15
|
+
1. WHEN tests check dataset types THEN they SHALL use `respond_to?` or duck typing instead of exact class matching
|
16
|
+
2. WHEN Sequel creates dataset subclasses THEN tests SHALL accept them as valid datasets
|
17
|
+
3. WHEN tests verify dataset functionality THEN they SHALL test behavior rather than class hierarchy
|
18
|
+
4. WHEN new Sequel versions change internal class structures THEN tests SHALL continue to pass
|
19
|
+
|
20
|
+
### Requirement 2: Mock Database Configuration
|
21
|
+
|
22
|
+
**User Story:** As a developer running tests for the sequel-duckdb adapter, I want the mock database to properly simulate DuckDB behavior, so that SQL generation tests accurately reflect real adapter behavior.
|
23
|
+
|
24
|
+
#### Acceptance Criteria
|
25
|
+
|
26
|
+
1. WHEN mock database is created THEN it SHALL include DuckDB-specific dataset methods
|
27
|
+
2. WHEN mock database generates SQL THEN it SHALL use DuckDB syntax rules
|
28
|
+
3. WHEN mock database handles identifiers THEN it SHALL use DuckDB quoting rules
|
29
|
+
4. WHEN mock database processes expressions THEN it SHALL use DuckDB literal handling
|
30
|
+
|
31
|
+
### Requirement 3: Test Helper Method Improvements
|
32
|
+
|
33
|
+
**User Story:** As a developer writing tests for the sequel-duckdb adapter, I want test helper methods that work correctly with DuckDB's SQL syntax variations, so that I can write reliable tests without worrying about syntax differences.
|
34
|
+
|
35
|
+
#### Acceptance Criteria
|
36
|
+
|
37
|
+
1. WHEN `assert_sql` helper is used THEN it SHALL handle DuckDB syntax variations gracefully
|
38
|
+
2. WHEN SQL comparison is needed THEN helper methods SHALL normalize minor syntax differences
|
39
|
+
3. WHEN testing SQL patterns THEN helper methods SHALL support flexible matching
|
40
|
+
4. WHEN debugging test failures THEN helper methods SHALL provide clear error messages
|
41
|
+
|
42
|
+
### Requirement 4: Integration Test Setup
|
43
|
+
|
44
|
+
**User Story:** As a developer running integration tests for the sequel-duckdb adapter, I want proper test database setup and teardown, so that integration tests run reliably and don't interfere with each other.
|
45
|
+
|
46
|
+
#### Acceptance Criteria
|
47
|
+
|
48
|
+
1. WHEN integration tests run THEN each test SHALL have a clean database state
|
49
|
+
2. WHEN tests create tables THEN they SHALL be properly cleaned up after the test
|
50
|
+
3. WHEN tests insert data THEN it SHALL not affect other tests
|
51
|
+
4. WHEN tests fail THEN database state SHALL be properly reset for subsequent tests
|
52
|
+
|
53
|
+
### Requirement 5: Test Data Management
|
54
|
+
|
55
|
+
**User Story:** As a developer writing tests for the sequel-duckdb adapter, I want consistent test data setup utilities, so that I can focus on testing functionality rather than data preparation.
|
56
|
+
|
57
|
+
#### Acceptance Criteria
|
58
|
+
|
59
|
+
1. WHEN tests need sample data THEN standardized helper methods SHALL be available
|
60
|
+
2. WHEN tests need specific table schemas THEN reusable setup methods SHALL be provided
|
61
|
+
3. WHEN tests need to verify data integrity THEN helper methods SHALL validate expected state
|
62
|
+
4. WHEN tests need to clean up data THEN helper methods SHALL ensure complete cleanup
|
63
|
+
|
64
|
+
### Requirement 6: Error Message Improvements
|
65
|
+
|
66
|
+
**User Story:** As a developer debugging failing tests in the sequel-duckdb adapter, I want clear and informative error messages, so that I can quickly identify and fix issues.
|
67
|
+
|
68
|
+
#### Acceptance Criteria
|
69
|
+
|
70
|
+
1. WHEN SQL assertion fails THEN the error message SHALL show both expected and actual SQL clearly
|
71
|
+
2. WHEN dataset type assertion fails THEN the error message SHALL explain what was expected and why
|
72
|
+
3. WHEN database operation fails THEN the error message SHALL include relevant context
|
73
|
+
4. WHEN test setup fails THEN the error message SHALL indicate the specific setup step that failed
|
74
|
+
|
75
|
+
### Requirement 7: Test Organization and Naming
|
76
|
+
|
77
|
+
**User Story:** As a developer maintaining the sequel-duckdb adapter test suite, I want tests to be well-organized and clearly named, so that I can easily understand what each test covers and find relevant tests when debugging.
|
78
|
+
|
79
|
+
#### Acceptance Criteria
|
80
|
+
|
81
|
+
1. WHEN tests are grouped by functionality THEN the grouping SHALL be logical and consistent
|
82
|
+
2. WHEN test methods are named THEN the names SHALL clearly describe what is being tested
|
83
|
+
3. WHEN tests cover edge cases THEN they SHALL be clearly identified as such
|
84
|
+
4. WHEN tests are added THEN they SHALL follow established naming and organization patterns
|
85
|
+
|
86
|
+
### Requirement 8: Test Performance Optimization
|
87
|
+
|
88
|
+
**User Story:** As a developer running the sequel-duckdb adapter test suite, I want tests to run quickly and efficiently, so that I can get fast feedback during development.
|
89
|
+
|
90
|
+
#### Acceptance Criteria
|
91
|
+
|
92
|
+
1. WHEN unit tests run THEN they SHALL complete quickly without database connections
|
93
|
+
2. WHEN integration tests run THEN they SHALL use efficient database operations
|
94
|
+
3. WHEN test suite runs THEN it SHALL minimize redundant setup and teardown operations
|
95
|
+
4. WHEN tests are parallelized THEN they SHALL not interfere with each other
|
96
|
+
|
97
|
+
### Requirement 9: Test Coverage Validation
|
98
|
+
|
99
|
+
**User Story:** As a developer maintaining the sequel-duckdb adapter, I want to ensure comprehensive test coverage, so that all functionality is properly tested and regressions are caught early.
|
100
|
+
|
101
|
+
#### Acceptance Criteria
|
102
|
+
|
103
|
+
1. WHEN new functionality is added THEN corresponding tests SHALL be required
|
104
|
+
2. WHEN tests are removed THEN coverage impact SHALL be evaluated
|
105
|
+
3. WHEN test coverage is measured THEN it SHALL include both unit and integration tests
|
106
|
+
4. WHEN coverage gaps are identified THEN they SHALL be prioritized for additional testing
|
@@ -0,0 +1,22 @@
|
|
1
|
+
# Product Overview
|
2
|
+
|
3
|
+
sequel-duckdb is a Ruby gem that provides a complete database adapter for the Sequel toolkit to work with DuckDB. This gem enables Ruby applications to connect to and interact with DuckDB databases through Sequel's comprehensive ORM and database abstraction interface.
|
4
|
+
|
5
|
+
## Key Features
|
6
|
+
- Full DuckDB database adapter for Sequel
|
7
|
+
- Ruby 3.1+ compatibility
|
8
|
+
- Complete SQL generation and query support
|
9
|
+
- Connection management via ruby-duckdb gem
|
10
|
+
- Comprehensive test coverage for all SQL operations
|
11
|
+
|
12
|
+
## Reference Projects
|
13
|
+
- **jeremyevans/sequel**: Official Sequel repository for coding conventions
|
14
|
+
- **sequel-hexspace**: Secondary reference for adapter structure
|
15
|
+
- **sequel_impala**: Tertiary reference for implementation patterns
|
16
|
+
|
17
|
+
## Target Users
|
18
|
+
- Ruby developers using Sequel ORM
|
19
|
+
- Applications requiring DuckDB integration
|
20
|
+
|
21
|
+
## Implementation Approach
|
22
|
+
This project follows the proven patterns from existing Sequel adapters, with implementation order guided by git history analysis of reference projects. All development is designed to be AI-agent friendly with thorough documentation and incremental implementation.
|
@@ -0,0 +1,88 @@
|
|
1
|
+
# Project Structure
|
2
|
+
|
3
|
+
## Root Directory
|
4
|
+
|
5
|
+
- **Gemfile**: Dependency specification
|
6
|
+
- **Rakefile**: Build tasks and automation
|
7
|
+
- **sequel-duckdb.gemspec**: Gem specification
|
8
|
+
- **.rubocop.yml**: Code style configuration
|
9
|
+
- **README.md**: Project documentation
|
10
|
+
- **CHANGELOG.md**: Version history
|
11
|
+
|
12
|
+
## Core Library Structure (Following sequel-hexspace Pattern)
|
13
|
+
|
14
|
+
```text
|
15
|
+
lib/
|
16
|
+
└── sequel/
|
17
|
+
└── adapters/
|
18
|
+
├── duckdb.rb # Main adapter file (Database & Dataset classes)
|
19
|
+
└── shared/
|
20
|
+
└── duckdb.rb # Shared DuckDB-specific functionality
|
21
|
+
```
|
22
|
+
|
23
|
+
## Organization Patterns
|
24
|
+
|
25
|
+
- **Namespace**: `Sequel::DuckDB` - follows Sequel's adapter pattern exactly like sequel-hexspace
|
26
|
+
- **Adapter Structure**: Main adapter in `lib/sequel/adapters/duckdb.rb` with Database and Dataset classes
|
27
|
+
- **Shared Functionality**: Common methods in `lib/sequel/adapters/shared/duckdb.rb` with DatabaseMethods and DatasetMethods modules
|
28
|
+
- **Database Class**: `Sequel::DuckDB::Database` includes `Sequel::DuckDB::DatabaseMethods` for connection management
|
29
|
+
- **Dataset Class**: `Sequel::DuckDB::Dataset` includes `Sequel::DuckDB::DatasetMethods` for SQL generation
|
30
|
+
- **Module Pattern**: Use include pattern exactly like sequel-hexspace to separate concerns between main classes and shared functionality
|
31
|
+
|
32
|
+
## Development Structure
|
33
|
+
|
34
|
+
- **bin/**: Development executables (console, setup)
|
35
|
+
- **test/**: Comprehensive test suite following sequel-hexspace pattern
|
36
|
+
- **sig/**: RBS type signatures for Ruby 3+ type checking
|
37
|
+
- **.kiro/**: AI assistant configuration and steering rules
|
38
|
+
|
39
|
+
## File Naming Conventions
|
40
|
+
|
41
|
+
- Use snake_case for file names
|
42
|
+
- Match file names to adapter names (duckdb.rb for DuckDB adapter)
|
43
|
+
- Follow sequel-hexspace directory structure exactly
|
44
|
+
- Standard Ruby gem layout with adapter-specific organization
|
45
|
+
|
46
|
+
## Key Architectural Decisions
|
47
|
+
|
48
|
+
- **Database Adapter Pattern**: Full Sequel database adapter implementation following sequel-hexspace
|
49
|
+
- **Reference-Driven Design**: Mirror sequel-hexspace structure and patterns exactly
|
50
|
+
- **Test-First Development**: Comprehensive test coverage before implementation
|
51
|
+
- **Incremental Implementation**: Small, clearly defined implementation pieces
|
52
|
+
- **AI-Agent Friendly**: Thorough documentation for autonomous development
|
53
|
+
|
54
|
+
## Implementation Order (based on sequel-hexspace git history)
|
55
|
+
|
56
|
+
1. **Connection Management**: Database connection and basic connectivity
|
57
|
+
2. **Schema Operations**: Table creation, modification, introspection
|
58
|
+
3. **Basic SQL Generation**: SELECT, INSERT, UPDATE, DELETE operations
|
59
|
+
4. **Advanced SQL Features**: JOINs, subqueries, window functions
|
60
|
+
5. **DuckDB-Specific Features**: Specialized functions and optimizations
|
61
|
+
6. **Performance Optimizations**: Bulk operations, prepared statements
|
62
|
+
|
63
|
+
## Adding New Features
|
64
|
+
|
65
|
+
- **Research First**: Study sequel-hexspace implementation patterns directly
|
66
|
+
- **Document Thoroughly**: Create detailed specifications before coding
|
67
|
+
- **Test-Driven**: Write comprehensive tests first
|
68
|
+
- **Incremental**: Implement in small, testable pieces
|
69
|
+
- **Follow Conventions**: Adhere to sequel-hexspace coding standards exactly
|
70
|
+
- Place main Database and Dataset classes in `lib/sequel/adapters/duckdb.rb`
|
71
|
+
- Place DatabaseMethods and DatasetMethods modules in `lib/sequel/adapters/shared/duckdb.rb`
|
72
|
+
- Use include pattern to mix shared functionality into main classes
|
73
|
+
- Mirror sequel-hexspace file organization and module structure exactly
|
74
|
+
|
75
|
+
## Testing Structure (Following sequel-hexspace Pattern)
|
76
|
+
|
77
|
+
- **test/all.rb**: Test runner
|
78
|
+
- **test/spec_helper.rb**: Test configuration and setup
|
79
|
+
- **test/database_test.rb**: Database connection and basic functionality tests
|
80
|
+
- **test/dataset_test.rb**: Comprehensive SQL generation tests
|
81
|
+
- **test/schema_test.rb**: Schema operations and introspection tests
|
82
|
+
- **test/prepared_statement_test.rb**: Prepared statement functionality
|
83
|
+
- **test/sql_test.rb**: SQL generation and syntax tests
|
84
|
+
- **test/type_test.rb**: Data type handling tests
|
85
|
+
- Mirror sequel-hexspace test organization exactly
|
86
|
+
- Test all SQL generation comprehensively
|
87
|
+
- Include integration tests with actual DuckDB instances
|
88
|
+
- Follow TDD methodology throughout development
|