clickhouse-ruby 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +80 -0
- data/LICENSE +21 -0
- data/README.md +251 -0
- data/lib/clickhouse_ruby/active_record/arel_visitor.rb +468 -0
- data/lib/clickhouse_ruby/active_record/connection_adapter.rb +723 -0
- data/lib/clickhouse_ruby/active_record/railtie.rb +192 -0
- data/lib/clickhouse_ruby/active_record/schema_statements.rb +693 -0
- data/lib/clickhouse_ruby/active_record.rb +121 -0
- data/lib/clickhouse_ruby/client.rb +471 -0
- data/lib/clickhouse_ruby/configuration.rb +145 -0
- data/lib/clickhouse_ruby/connection.rb +328 -0
- data/lib/clickhouse_ruby/connection_pool.rb +301 -0
- data/lib/clickhouse_ruby/errors.rb +144 -0
- data/lib/clickhouse_ruby/result.rb +189 -0
- data/lib/clickhouse_ruby/types/array.rb +183 -0
- data/lib/clickhouse_ruby/types/base.rb +77 -0
- data/lib/clickhouse_ruby/types/boolean.rb +68 -0
- data/lib/clickhouse_ruby/types/date_time.rb +163 -0
- data/lib/clickhouse_ruby/types/float.rb +115 -0
- data/lib/clickhouse_ruby/types/integer.rb +157 -0
- data/lib/clickhouse_ruby/types/low_cardinality.rb +58 -0
- data/lib/clickhouse_ruby/types/map.rb +249 -0
- data/lib/clickhouse_ruby/types/nullable.rb +73 -0
- data/lib/clickhouse_ruby/types/parser.rb +244 -0
- data/lib/clickhouse_ruby/types/registry.rb +148 -0
- data/lib/clickhouse_ruby/types/string.rb +83 -0
- data/lib/clickhouse_ruby/types/tuple.rb +206 -0
- data/lib/clickhouse_ruby/types/uuid.rb +84 -0
- data/lib/clickhouse_ruby/types.rb +69 -0
- data/lib/clickhouse_ruby/version.rb +5 -0
- data/lib/clickhouse_ruby.rb +101 -0
- metadata +150 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: d7855eb261fe694066b665b2d1408b837a751d257c26d88f74d29847869fe652
|
|
4
|
+
data.tar.gz: 524b169428f84ab3f24c26433b4b57ead1f41e6147551c412e0f0a1840a8f7c9
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 76da6aa320dabde964561304404eecfba2f500437ffc763f1bb15f6c5917c1cf703b1c14f2ec76024f7a585749993cd047dd7e6576dc2e33c2754e9677865556
|
|
7
|
+
data.tar.gz: 7bba509232194cef0a8dd560a08434d16a6ba3fd04a96d7868e1b00080d0421834e1a88c07f76c592589aadf2b473703ae6f3b5dd3cc692fcae6f3798843a446
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.1.0] - 2026-01-31
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
#### Core Features
|
|
15
|
+
- HTTP client with connection pooling for ClickHouse communication
|
|
16
|
+
- SSL/TLS support with certificate verification **enabled by default** (security fix vs existing gems)
|
|
17
|
+
- Configurable timeouts (connect, read, write)
|
|
18
|
+
- Basic authentication (username/password via headers)
|
|
19
|
+
- Connection health checks (ping)
|
|
20
|
+
|
|
21
|
+
#### Error Handling (Critical Fix)
|
|
22
|
+
- Proper HTTP status code checking - **never silently ignores errors**
|
|
23
|
+
- Clear exception hierarchy (`ClickhouseRuby::Error` → `QueryError`, `ConnectionError`, etc.)
|
|
24
|
+
- ClickHouse error code extraction and mapping to specific exception classes
|
|
25
|
+
- Actionable error messages with full context (SQL, error code, HTTP status)
|
|
26
|
+
|
|
27
|
+
#### Type System (Critical Fix)
|
|
28
|
+
- **AST-based type parser** - properly handles nested types (fixes regex-based parsing issues)
|
|
29
|
+
- Support for all basic types: String, Int8-256, UInt8-256, Float32/64, Bool
|
|
30
|
+
- Date/DateTime/DateTime64 with timezone awareness
|
|
31
|
+
- UUID with format validation
|
|
32
|
+
- Complex types: Array(T), Map(K,V), Tuple(T1,T2,...), Nullable(T), LowCardinality(T)
|
|
33
|
+
- Bidirectional type conversion (Ruby ↔ ClickHouse)
|
|
34
|
+
|
|
35
|
+
#### Query Execution
|
|
36
|
+
- SELECT queries with JSONCompact format
|
|
37
|
+
- Result object with Enumerable interface
|
|
38
|
+
- Column type information in results
|
|
39
|
+
- Query-level SETTINGS support
|
|
40
|
+
|
|
41
|
+
#### Bulk Insert
|
|
42
|
+
- JSONEachRow format support (5x faster than VALUES syntax)
|
|
43
|
+
- INSERT with SETTINGS support (async_insert, etc.)
|
|
44
|
+
- Batch insert API with proper error handling
|
|
45
|
+
|
|
46
|
+
#### ActiveRecord Integration
|
|
47
|
+
- Connection adapter registration (`adapter: clickhouse`)
|
|
48
|
+
- Basic CRUD operations with proper error propagation
|
|
49
|
+
- Schema introspection (tables, columns via system tables)
|
|
50
|
+
- Arel visitor for ClickHouse SQL dialect
|
|
51
|
+
- ALTER TABLE DELETE/UPDATE syntax for mutations
|
|
52
|
+
- Rails integration via Railtie
|
|
53
|
+
|
|
54
|
+
#### Project Infrastructure
|
|
55
|
+
- RSpec test suite with unit and integration tests
|
|
56
|
+
- VCR for HTTP interaction recording
|
|
57
|
+
- Docker Compose setup for ClickHouse testing
|
|
58
|
+
- GitHub Actions CI workflow
|
|
59
|
+
- RuboCop configuration
|
|
60
|
+
|
|
61
|
+
### Security
|
|
62
|
+
- SSL certificate verification is ON by default (unlike existing gems)
|
|
63
|
+
- Passwords not logged
|
|
64
|
+
|
|
65
|
+
### Known Limitations
|
|
66
|
+
- HTTP protocol only (native TCP protocol planned for future)
|
|
67
|
+
- No PREWHERE support yet (planned for v0.2.0)
|
|
68
|
+
- No streaming results yet (planned for v0.2.0)
|
|
69
|
+
- No migration DSL for table engines/partitions yet (planned for v0.2.0)
|
|
70
|
+
|
|
71
|
+
## Comparison with Existing Solutions
|
|
72
|
+
|
|
73
|
+
This gem addresses critical issues found in existing ClickHouse Ruby gems:
|
|
74
|
+
|
|
75
|
+
| Issue | clickhouse-activerecord | ClickhouseRuby |
|
|
76
|
+
|-------|------------------------|--------|
|
|
77
|
+
| Silent DELETE failures (#230) | ❌ Fails silently | ✅ Always raises errors |
|
|
78
|
+
| Type parsing (#210) | ❌ Regex breaks nested | ✅ AST-based parser |
|
|
79
|
+
| SSL verification | ❌ Disabled by default | ✅ Enabled by default |
|
|
80
|
+
| PREWHERE support (#228) | ❌ Missing | 🔜 Planned v0.2.0 |
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2024 Your Name
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,251 @@
|
|
|
1
|
+
# ClickhouseRuby
|
|
2
|
+
|
|
3
|
+
A lightweight Ruby client for ClickHouse with optional ActiveRecord integration.
|
|
4
|
+
|
|
5
|
+
## Installation
|
|
6
|
+
|
|
7
|
+
Add this line to your application's Gemfile:
|
|
8
|
+
|
|
9
|
+
```ruby
|
|
10
|
+
gem 'clickhouse-ruby'
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
And then execute:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
bundle install
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Or install it yourself as:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
gem install clickhouse-ruby
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Quick Start
|
|
26
|
+
|
|
27
|
+
```ruby
|
|
28
|
+
require 'clickhouse-ruby'
|
|
29
|
+
|
|
30
|
+
# Create a client
|
|
31
|
+
client = ClickhouseRuby::Client.new(
|
|
32
|
+
host: 'localhost',
|
|
33
|
+
port: 8123,
|
|
34
|
+
database: 'default'
|
|
35
|
+
)
|
|
36
|
+
|
|
37
|
+
# Execute a query
|
|
38
|
+
result = client.query('SELECT 1 + 1 AS result')
|
|
39
|
+
puts result.first['result'] # => 2
|
|
40
|
+
|
|
41
|
+
# Insert data
|
|
42
|
+
client.insert('events', [
|
|
43
|
+
{ date: '2024-01-01', event_type: 'click', count: 100 },
|
|
44
|
+
{ date: '2024-01-02', event_type: 'view', count: 250 }
|
|
45
|
+
])
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Configuration
|
|
49
|
+
|
|
50
|
+
### Basic Configuration
|
|
51
|
+
|
|
52
|
+
```ruby
|
|
53
|
+
ClickhouseRuby.configure do |config|
|
|
54
|
+
config.host = 'localhost'
|
|
55
|
+
config.port = 8123
|
|
56
|
+
config.database = 'default'
|
|
57
|
+
config.username = 'default'
|
|
58
|
+
config.password = ''
|
|
59
|
+
config.ssl = false
|
|
60
|
+
config.timeout = 60
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
# Use the default client
|
|
64
|
+
client = ClickhouseRuby.client
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Configuration Options
|
|
68
|
+
|
|
69
|
+
| Option | Description | Default |
|
|
70
|
+
|--------|-------------|---------|
|
|
71
|
+
| `host` | ClickHouse server hostname | `localhost` |
|
|
72
|
+
| `port` | HTTP interface port | `8123` |
|
|
73
|
+
| `database` | Default database | `default` |
|
|
74
|
+
| `username` | Authentication username | `default` |
|
|
75
|
+
| `password` | Authentication password | `''` |
|
|
76
|
+
| `ssl` | Enable HTTPS | `false` |
|
|
77
|
+
| `timeout` | Request timeout in seconds | `60` |
|
|
78
|
+
| `max_retries` | Number of retry attempts | `3` |
|
|
79
|
+
|
|
80
|
+
### Environment Variables
|
|
81
|
+
|
|
82
|
+
ClickhouseRuby can be configured via environment variables:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
CLICKHOUSE_HOST=localhost
|
|
86
|
+
CLICKHOUSE_PORT=8123
|
|
87
|
+
CLICKHOUSE_DATABASE=default
|
|
88
|
+
CLICKHOUSE_USERNAME=default
|
|
89
|
+
CLICKHOUSE_PASSWORD=secret
|
|
90
|
+
CLICKHOUSE_SSL=false
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## Usage
|
|
94
|
+
|
|
95
|
+
### Querying Data
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
# Simple query
|
|
99
|
+
result = client.query('SELECT * FROM events LIMIT 10')
|
|
100
|
+
|
|
101
|
+
# Query with parameters (prevents SQL injection)
|
|
102
|
+
result = client.query(
|
|
103
|
+
'SELECT * FROM events WHERE date = {date:Date}',
|
|
104
|
+
params: { date: '2024-01-01' }
|
|
105
|
+
)
|
|
106
|
+
|
|
107
|
+
# Query with specific format
|
|
108
|
+
result = client.query('SELECT * FROM events', format: 'JSONEachRow')
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
### Inserting Data
|
|
112
|
+
|
|
113
|
+
```ruby
|
|
114
|
+
# Insert hash array
|
|
115
|
+
client.insert('events', [
|
|
116
|
+
{ date: '2024-01-01', event_type: 'click', count: 100 },
|
|
117
|
+
{ date: '2024-01-02', event_type: 'view', count: 250 }
|
|
118
|
+
])
|
|
119
|
+
|
|
120
|
+
# Insert with explicit columns
|
|
121
|
+
client.insert('events', data, columns: [:date, :event_type, :count])
|
|
122
|
+
|
|
123
|
+
# Bulk insert from CSV
|
|
124
|
+
client.insert_from_file('events', '/path/to/data.csv', format: 'CSV')
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
### DDL Operations
|
|
128
|
+
|
|
129
|
+
```ruby
|
|
130
|
+
# Create table
|
|
131
|
+
client.execute(<<~SQL)
|
|
132
|
+
CREATE TABLE events (
|
|
133
|
+
date Date,
|
|
134
|
+
event_type String,
|
|
135
|
+
count UInt32
|
|
136
|
+
) ENGINE = MergeTree()
|
|
137
|
+
ORDER BY date
|
|
138
|
+
SQL
|
|
139
|
+
|
|
140
|
+
# Check if table exists
|
|
141
|
+
client.table_exists?('events') # => true
|
|
142
|
+
|
|
143
|
+
# Get table schema
|
|
144
|
+
schema = client.describe_table('events')
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
### Connection Management
|
|
148
|
+
|
|
149
|
+
```ruby
|
|
150
|
+
# Check connection
|
|
151
|
+
client.ping # => true
|
|
152
|
+
|
|
153
|
+
# Get server version
|
|
154
|
+
client.server_version # => "24.1.1.123"
|
|
155
|
+
|
|
156
|
+
# Execute multiple queries in a session
|
|
157
|
+
client.with_session do |session|
|
|
158
|
+
session.execute('SET max_memory_usage = 1000000000')
|
|
159
|
+
session.query('SELECT * FROM large_table')
|
|
160
|
+
end
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
## ActiveRecord Integration
|
|
164
|
+
|
|
165
|
+
ClickhouseRuby provides optional ActiveRecord integration for familiar model-based access.
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
# config/initializers/clickhouse-ruby.rb
|
|
169
|
+
require 'clickhouse-ruby/active_record'
|
|
170
|
+
|
|
171
|
+
ClickhouseRuby::ActiveRecord.establish_connection(
|
|
172
|
+
host: 'localhost',
|
|
173
|
+
database: 'analytics'
|
|
174
|
+
)
|
|
175
|
+
|
|
176
|
+
# app/models/event.rb
|
|
177
|
+
class Event < ClickhouseRuby::ActiveRecord::Base
|
|
178
|
+
self.table_name = 'events'
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
# Usage
|
|
182
|
+
Event.where(date: '2024-01-01').limit(10).each do |event|
|
|
183
|
+
puts event.event_type
|
|
184
|
+
end
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
## Error Handling
|
|
188
|
+
|
|
189
|
+
```ruby
|
|
190
|
+
begin
|
|
191
|
+
client.query('SELECT * FROM nonexistent_table')
|
|
192
|
+
rescue ClickhouseRuby::ConnectionError => e
|
|
193
|
+
# Handle connection issues
|
|
194
|
+
puts "Connection failed: #{e.message}"
|
|
195
|
+
rescue ClickhouseRuby::QueryError => e
|
|
196
|
+
# Handle query errors
|
|
197
|
+
puts "Query failed: #{e.message}"
|
|
198
|
+
puts "Error code: #{e.code}"
|
|
199
|
+
rescue ClickhouseRuby::TimeoutError => e
|
|
200
|
+
# Handle timeouts
|
|
201
|
+
puts "Request timed out: #{e.message}"
|
|
202
|
+
end
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
## Development
|
|
206
|
+
|
|
207
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests.
|
|
208
|
+
|
|
209
|
+
### Running Tests
|
|
210
|
+
|
|
211
|
+
```bash
|
|
212
|
+
# Start ClickHouse
|
|
213
|
+
docker-compose up -d
|
|
214
|
+
|
|
215
|
+
# Run all tests
|
|
216
|
+
bundle exec rake spec
|
|
217
|
+
|
|
218
|
+
# Run only unit tests
|
|
219
|
+
bundle exec rake spec_unit
|
|
220
|
+
|
|
221
|
+
# Run integration tests
|
|
222
|
+
bundle exec rake spec_integration
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
### Code Quality
|
|
226
|
+
|
|
227
|
+
```bash
|
|
228
|
+
# Run RuboCop
|
|
229
|
+
bundle exec rake rubocop
|
|
230
|
+
|
|
231
|
+
# Auto-fix issues
|
|
232
|
+
bundle exec rake rubocop_fix
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
## Contributing
|
|
236
|
+
|
|
237
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/yourusername/clickhouse-ruby.
|
|
238
|
+
|
|
239
|
+
1. Fork it
|
|
240
|
+
2. Create your feature branch (`git checkout -b feature/my-new-feature`)
|
|
241
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
|
242
|
+
4. Push to the branch (`git push origin feature/my-new-feature`)
|
|
243
|
+
5. Create a new Pull Request
|
|
244
|
+
|
|
245
|
+
## License
|
|
246
|
+
|
|
247
|
+
The gem is available as open source under the terms of the [MIT License](LICENSE).
|
|
248
|
+
|
|
249
|
+
## Documentation
|
|
250
|
+
|
|
251
|
+
Full documentation is available at [RubyDoc](https://rubydoc.info/gems/clickhouse-ruby).
|