timescaledb 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +192 -38
- data/bin/tsdb +28 -4
- data/examples/Gemfile +1 -1
- data/examples/Gemfile.lock +2 -2
- data/examples/all_in_one.rb +3 -6
- data/lib/timescale/chunk.rb +8 -0
- data/lib/timescale/continuous_aggregates.rb +9 -0
- data/lib/timescale/dimensions.rb +7 -0
- data/lib/timescale/hypertable.rb +7 -3
- data/lib/timescale/migration_helpers.rb +2 -2
- data/lib/timescale/schema_dumper.rb +24 -0
- data/lib/timescale/stats_report.rb +24 -17
- data/lib/timescale/version.rb +1 -1
- data/lib/timescale.rb +3 -2
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 91dfb6aac43ec8ab347ac9cf1bc553631310259cd4f6726c42f466c08f74c761
|
4
|
+
data.tar.gz: 4436ff5f2067264fc814592efebaa9ce32d69ece7607e7d728a95356db04f739
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 71492adbe4cd1771db48b2b796affa50d3f3157b420f59482e61d4a76dfc8fcec52f5face66047fc8069f5bcdd35bace4c8ff4e0fec88ffe1331872b3d625862
|
7
|
+
data.tar.gz: 4ecf26d66361af9d79cf33f99b9a25b8233df229a0ffa6ac33afe54d4f4c4d03e6c13609c60cceabc95dc30d8e49f7343588174fb0715bda044942360ebed53b
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -1,40 +1,30 @@
|
|
1
1
|
# Timescale
|
2
2
|
|
3
|
-
Welcome to the Timescale gem! To experiment with the code, start
|
4
|
-
|
3
|
+
Welcome to the Timescale gem! To experiment with the code, start installing the
|
4
|
+
gem:
|
5
5
|
|
6
6
|
```bash
|
7
|
-
|
8
|
-
cd timescale
|
9
|
-
bundle install
|
10
|
-
rake install
|
7
|
+
gem install timescaledb
|
11
8
|
```
|
12
9
|
|
13
|
-
|
10
|
+
## The `tsdb` CLI
|
14
11
|
|
15
|
-
|
16
|
-
|
17
|
-
```
|
12
|
+
When you install the gem locally, a new command line application named `tsdb`
|
13
|
+
will be linked in your command line.
|
18
14
|
|
19
|
-
|
20
|
-
|
15
|
+
It accepts a Postgresql URI and some extra flags that can help you to get more
|
16
|
+
info from your TimescaleDB server:
|
21
17
|
|
22
18
|
```bash
|
23
|
-
|
19
|
+
tsdb <uri> --stats
|
24
20
|
```
|
25
21
|
|
26
|
-
|
27
|
-
`tsdb`. Here is an example from the console:
|
22
|
+
Where the `<uri>` is replaced with params from your connection like:
|
28
23
|
|
29
24
|
```bash
|
30
|
-
tsdb
|
25
|
+
tsdb postgres://<user>@localhost:5432/<dbname> --stats
|
31
26
|
```
|
32
27
|
|
33
|
-
To join the console use `--console`:
|
34
|
-
|
35
|
-
```bash
|
36
|
-
tsdb "postgres://jonatasdp@localhost:5432/timescale_test" --console
|
37
|
-
```
|
38
28
|
|
39
29
|
Or just check the stats:
|
40
30
|
|
@@ -42,7 +32,7 @@ Or just check the stats:
|
|
42
32
|
tsdb "postgres://jonatasdp@localhost:5432/timescale_test" --stats
|
43
33
|
```
|
44
34
|
|
45
|
-
These is a sample output from
|
35
|
+
These is a sample output from database example with almost no data:
|
46
36
|
|
47
37
|
```ruby
|
48
38
|
{:hypertables=>
|
@@ -54,19 +44,153 @@ These is a sample output from an almost empty database:
|
|
54
44
|
:jobs_stats=>[{:success=>nil, :runs=>nil, :failures=>nil}]}
|
55
45
|
```
|
56
46
|
|
47
|
+
To start a interactive ruby/[pry](https://github.com/pry/pry) console use `--console`:
|
57
48
|
The console will dynamically create models for all hypertables that it finds
|
58
49
|
in the database.
|
59
50
|
|
60
|
-
|
61
|
-
|
51
|
+
Let's consider the [caggs.sql](https://gist.github.com/jonatas/95573ad8744994094ec9f284150004f9#file-caggs-sql)
|
52
|
+
as the example of database.
|
62
53
|
|
63
|
-
This library was started on [twitch.tv/timescaledb](https://twitch.tv/timescaledb).
|
64
|
-
You can watch all episodes here:
|
65
54
|
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
55
|
+
```bash
|
56
|
+
psql postgres://jonatasdp@localhost:5432/playground -f caggs.sql
|
57
|
+
```
|
58
|
+
|
59
|
+
Then use `tsdb` in the command line with the same URI and `--stats`:
|
60
|
+
|
61
|
+
```bash
|
62
|
+
tsdb postgres://jonatasdp@localhost:5432/playground --stats
|
63
|
+
{:hypertables=>
|
64
|
+
{:count=>1,
|
65
|
+
:uncompressed=>1,
|
66
|
+
:approximate_row_count=>{"ticks"=>352},
|
67
|
+
:chunks=>{:total=>1, :compressed=>0, :uncompressed=>1},
|
68
|
+
:size=>{:uncompressed=>"88 KB", :compressed=>"0 Bytes"}},
|
69
|
+
:continuous_aggregates=>{:total=>1},
|
70
|
+
:jobs_stats=>[{:success=>nil, :runs=>nil, :failures=>nil}]}
|
71
|
+
```
|
72
|
+
|
73
|
+
To have some interactive playground with the actual database using ruby, just
|
74
|
+
try the same command before changing from `--stats` to `--console`:
|
75
|
+
|
76
|
+
### tsdb --console
|
77
|
+
|
78
|
+
The same database from previous example, is used so
|
79
|
+
the context has a hypertable named `ticks` and a view named `ohlc_1m`.
|
80
|
+
|
81
|
+
|
82
|
+
```ruby
|
83
|
+
tsdb postgres://jonatasdp@localhost:5432/playground --console
|
84
|
+
pry(Timescale)>
|
85
|
+
```
|
86
|
+
|
87
|
+
The `tsdb` CLI will automatically create ActiveRecord models for hypertables and
|
88
|
+
continuous aggregates views.
|
89
|
+
|
90
|
+
```ruby
|
91
|
+
Tick
|
92
|
+
=> Timescale::Tick(time: datetime, symbol: string, price: decimal, volume: integer)
|
93
|
+
```
|
94
|
+
|
95
|
+
Note that it's only created for this session and will never be cached in the
|
96
|
+
library or any other place.
|
97
|
+
|
98
|
+
In this case, `Tick` model comes from `ticks` hypertable that was found in the database.
|
99
|
+
It contains several extra methods inherited from `acts_as_hypertable` macro.
|
100
|
+
|
101
|
+
Let's start with the `.hypertable` method.
|
102
|
+
|
103
|
+
```ruby
|
104
|
+
Tick.hypertable
|
105
|
+
=> #<Timescale::Hypertable:0x00007fe99c258900
|
106
|
+
hypertable_schema: "public",
|
107
|
+
hypertable_name: "ticks",
|
108
|
+
owner: "jonatasdp",
|
109
|
+
num_dimensions: 1,
|
110
|
+
num_chunks: 1,
|
111
|
+
compression_enabled: false,
|
112
|
+
is_distributed: false,
|
113
|
+
replication_factor: nil,
|
114
|
+
data_nodes: nil,
|
115
|
+
tablespaces: nil>
|
116
|
+
```
|
117
|
+
|
118
|
+
The core of the hypertables are the fragmentation of the data into chunks that
|
119
|
+
are the child tables that distribute the data. You can check all chunks directly
|
120
|
+
from the hypertable relation.
|
121
|
+
|
122
|
+
```ruby
|
123
|
+
Tick.hypertable.chunks
|
124
|
+
unknown OID 2206: failed to recognize type of 'primary_dimension_type'. It will be treated as String.
|
125
|
+
=> [#<Timescale::Chunk:0x00007fe99c31b068
|
126
|
+
hypertable_schema: "public",
|
127
|
+
hypertable_name: "ticks",
|
128
|
+
chunk_schema: "_timescaledb_internal",
|
129
|
+
chunk_name: "_hyper_33_17_chunk",
|
130
|
+
primary_dimension: "time",
|
131
|
+
primary_dimension_type: "timestamp without time zone",
|
132
|
+
range_start: 1999-12-30 00:00:00 +0000,
|
133
|
+
range_end: 2000-01-06 00:00:00 +0000,
|
134
|
+
range_start_integer: nil,
|
135
|
+
range_end_integer: nil,
|
136
|
+
is_compressed: false,
|
137
|
+
chunk_tablespace: nil,
|
138
|
+
data_nodes: nil>]
|
139
|
+
```
|
140
|
+
|
141
|
+
> Chunks are created by partitioning a hypertable's data into one
|
142
|
+
> (or potentially multiple) dimensions. All hypertables are partitioned by the
|
143
|
+
> values belonging to a time column, which may be in timestamp, date, or
|
144
|
+
> various integer forms. If the time partitioning interval is one day,
|
145
|
+
> for example, then rows with timestamps that belong to the same day are co-located
|
146
|
+
> within the same chunk, while rows belonging to different days belong to different chunks.
|
147
|
+
> Learn more [here](https://docs.timescale.com/timescaledb/latest/overview/core-concepts/hypertables-and-chunks/).
|
148
|
+
|
149
|
+
Another core concept of TimescaleDB is compression. With data partitioned, it
|
150
|
+
becomes very convenient to compress and decompress chunks independently.
|
151
|
+
|
152
|
+
```ruby
|
153
|
+
Tick.hypertable.chunks.first.compress!
|
154
|
+
ActiveRecord::StatementInvalid: PG::FeatureNotSupported: ERROR: compression not enabled on "ticks"
|
155
|
+
DETAIL: It is not possible to compress chunks on a hypertable that does not have compression enabled.
|
156
|
+
HINT: Enable compression using ALTER TABLE with the timescaledb.compress option.
|
157
|
+
```
|
158
|
+
|
159
|
+
As compression is not enabled, let's do it executing a plain SQL directly from
|
160
|
+
the actual context. To borrow a connection, let's use the Tick object.
|
161
|
+
|
162
|
+
```ruby
|
163
|
+
Tick.connection.execute("ALTER TABLE ticks SET (timescaledb.compress)") # => PG_OK
|
164
|
+
```
|
165
|
+
|
166
|
+
And now, it's possible to compress and decompress:
|
167
|
+
|
168
|
+
```ruby
|
169
|
+
Tick.hypertable.chunks.first.compress!
|
170
|
+
Tick.hypertable.chunks.first.decompress!
|
171
|
+
```
|
172
|
+
Learn more about TimescaleDB compression [here](https://docs.timescale.com/timescaledb/latest/overview/core-concepts/compression/).
|
173
|
+
|
174
|
+
The `ohlc_1m` view is also available as an ActiveRecord:
|
175
|
+
|
176
|
+
```ruby
|
177
|
+
Ohlc1m
|
178
|
+
=> Timescale::Ohlc1m(bucket: datetime, symbol: string, open: decimal, high: decimal, low: decimal, close: decimal, volume: integer)
|
179
|
+
```
|
180
|
+
|
181
|
+
And you can run any query as you do with regular active record queries.
|
182
|
+
|
183
|
+
```ruby
|
184
|
+
Ohlc1m.order(bucket: :desc).last
|
185
|
+
=> #<Timescale::Ohlc1m:0x00007fe99c2c38e0
|
186
|
+
bucket: 2000-01-01 00:00:00 UTC,
|
187
|
+
symbol: "SYMBOL",
|
188
|
+
open: 0.13e2,
|
189
|
+
high: 0.3e2,
|
190
|
+
low: 0.1e1,
|
191
|
+
close: 0.1e2,
|
192
|
+
volume: 27600>
|
193
|
+
```
|
70
194
|
|
71
195
|
## Installation
|
72
196
|
|
@@ -84,13 +208,14 @@ Or install it yourself as:
|
|
84
208
|
|
85
209
|
$ gem install timescaledb
|
86
210
|
|
211
|
+
|
87
212
|
## Usage
|
88
213
|
|
89
214
|
You can check the [all_in_one.rb](examples/all_in_one.rb) that will:
|
90
215
|
|
91
216
|
1. Create hypertable with compression settings
|
92
217
|
2. Insert data
|
93
|
-
3. Run some queries
|
218
|
+
3. Run some queries
|
94
219
|
4. Check chunk size per model
|
95
220
|
5. Compress a chunk
|
96
221
|
6. Check chunk status
|
@@ -140,7 +265,7 @@ end
|
|
140
265
|
Tick = Class.new(ActiveRecord::Base) do
|
141
266
|
self.table_name = 'ticks'
|
142
267
|
self.primary_key = 'symbol'
|
143
|
-
|
268
|
+
acts_as_hypertable
|
144
269
|
end
|
145
270
|
|
146
271
|
query = Tick.select(<<~QUERY)
|
@@ -167,14 +292,14 @@ create_continuous_aggregates('ohlc_1m', query, **options)
|
|
167
292
|
|
168
293
|
### Hypertable Helpers
|
169
294
|
|
170
|
-
You can
|
295
|
+
You can say `acts_as_hypertable` to get access to some basic scopes for your
|
171
296
|
model:
|
172
297
|
|
173
298
|
```ruby
|
174
299
|
class Event < ActiveRecord::Base
|
175
300
|
self.primary_key = "identifier"
|
176
301
|
|
177
|
-
|
302
|
+
acts_as_hypertable
|
178
303
|
end
|
179
304
|
```
|
180
305
|
|
@@ -265,14 +390,17 @@ In case you want to use TimescaleDB on a Rails environment, you may have some
|
|
265
390
|
issues as the schema dump used for tests is not considering hypertables
|
266
391
|
metadata.
|
267
392
|
|
268
|
-
If you add the `
|
269
|
-
|
393
|
+
If you add the `acts_as_hypertable` to your model, you can dynamically
|
394
|
+
verify if the `Timescale::ActsAsHypertable` module is included to
|
395
|
+
create the hypertable for testing environment.
|
396
|
+
|
397
|
+
Consider adding this hook to your `spec/rspec_helper.rb` file:
|
270
398
|
|
271
399
|
```ruby
|
272
400
|
config.before(:suite) do
|
273
401
|
hypertable_models = ApplicationRecord
|
274
402
|
.descendants
|
275
|
-
.select{|clazz| clazz.
|
403
|
+
.select{|clazz| clazz.included_modules.include?(Timescale::ActsAsHypertable)
|
276
404
|
hypertable_models.each do |clazz|
|
277
405
|
if clazz.hypertable.exists?
|
278
406
|
ApplicationRecord.logger.info "skip recreating hypertable for '#{clazz.table_name}'."
|
@@ -291,13 +419,39 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
|
|
291
419
|
|
292
420
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
293
421
|
|
422
|
+
You can create a `.env` file locally to run tests locally. Make sure to put your
|
423
|
+
own credentials there!
|
424
|
+
|
425
|
+
```bash
|
426
|
+
PG_URI_TEST="postgres://<user>@localhost:5432/<dbname>"
|
427
|
+
```
|
428
|
+
|
429
|
+
You can put some postgres URI directly as a parameter of
|
430
|
+
`tsdb`. Here is an example from the console:
|
431
|
+
|
432
|
+
```bash
|
433
|
+
tsdb "postgres://jonatasdp@localhost:5432/timescale_test"
|
434
|
+
```
|
435
|
+
|
436
|
+
## More resources
|
437
|
+
|
438
|
+
This library was started on [twitch.tv/timescaledb](https://twitch.tv/timescaledb).
|
439
|
+
You can watch all episodes here:
|
440
|
+
|
441
|
+
1. [Wrapping Functions to Ruby Helpers](https://www.youtube.com/watch?v=hGPsUxLFAYk).
|
442
|
+
2. [Extending ActiveRecord with Timescale Helpers](https://www.youtube.com/watch?v=IEyJIHk1Clk).
|
443
|
+
3. [Setup Hypertables for Rails testing environment](https://www.youtube.com/watch?v=wM6hVrZe7xA).
|
444
|
+
4. [Packing the code to this repository](https://www.youtube.com/watch?v=CMdGAl_XlL4).
|
445
|
+
4. [the code to this repository](https://www.youtube.com/watch?v=CMdGAl_XlL4).
|
446
|
+
5. [Working with Timescale continuous aggregates](https://youtu.be/co4HnBkHzVw).
|
447
|
+
6. [Creating the command-line application in Ruby to explore the Timescale API](https://www.youtube.com/watch?v=I3vM_q2m7T0).
|
448
|
+
|
294
449
|
### TODO
|
295
450
|
|
296
451
|
Here is a list of functions that would be great to have:
|
297
452
|
|
298
453
|
- [ ] Dump and Restore Timescale metadata - Like db/schema.rb but for Timescale configuration.
|
299
454
|
- [ ] Add data nodes support
|
300
|
-
- [ ] Implement the `timescale` CLI to explore the full API.
|
301
455
|
|
302
456
|
## Contributing
|
303
457
|
|
data/bin/tsdb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
require "bundler/setup"
|
3
3
|
require "timescale"
|
4
|
+
require "pry"
|
4
5
|
|
5
6
|
ActiveRecord::Base.establish_connection(ARGV[0])
|
6
7
|
|
@@ -8,17 +9,40 @@ Timescale::Hypertable.find_each do |hypertable|
|
|
8
9
|
class_name = hypertable.hypertable_name.singularize.camelize
|
9
10
|
model = Class.new(ActiveRecord::Base) do
|
10
11
|
self.table_name = hypertable.hypertable_name
|
11
|
-
|
12
|
-
include Timescale::HypertableHelpers
|
12
|
+
acts_as_hypertable
|
13
13
|
end
|
14
14
|
Timescale.const_set(class_name, model)
|
15
15
|
end
|
16
16
|
|
17
|
+
Timescale::ContinuousAggregates.find_each do |cagg|
|
18
|
+
class_name = cagg.view_name.singularize.camelize
|
19
|
+
model = Class.new(ActiveRecord::Base) do
|
20
|
+
self.table_name = cagg.view_name
|
21
|
+
acts_as_hypertable
|
22
|
+
end
|
23
|
+
Timescale.const_set(class_name, model)
|
24
|
+
end
|
25
|
+
|
26
|
+
def show(obj)
|
27
|
+
Pry::ColorPrinter.pp(obj)
|
28
|
+
end
|
29
|
+
|
17
30
|
if ARGV.index("--stats")
|
18
|
-
|
31
|
+
scope = Timescale::Hypertable.all
|
32
|
+
|
33
|
+
if (only = ARGV.index("--only"))
|
34
|
+
only_hypertables = ARGV[only+1].split(",")
|
35
|
+
scope = scope.where({hypertable_name: only_hypertables})
|
36
|
+
end
|
37
|
+
|
38
|
+
if (except = ARGV.index("--except"))
|
39
|
+
except_hypertables = ARGV[except+1].split(",")
|
40
|
+
scope = scope.where.not(hypertable_name: except_hypertables)
|
41
|
+
end
|
42
|
+
|
43
|
+
show(Timescale.stats(scope))
|
19
44
|
end
|
20
45
|
|
21
46
|
if ARGV.index("--console")
|
22
|
-
require "pry"
|
23
47
|
Pry.start(Timescale)
|
24
48
|
end
|
data/examples/Gemfile
CHANGED
data/examples/Gemfile.lock
CHANGED
data/examples/all_in_one.rb
CHANGED
@@ -2,16 +2,13 @@ require 'bundler/setup'
|
|
2
2
|
require 'timescale'
|
3
3
|
require 'pp'
|
4
4
|
require 'pry'
|
5
|
-
|
6
|
-
|
7
|
-
# set PG_URI=postgres://user:pass@host:port/db_name
|
8
|
-
ActiveRecord::Base.establish_connection(ENV['PG_URI_TEST'])
|
5
|
+
# ruby all_in_one.rb postgres://user:pass@host:port/db_name
|
6
|
+
ActiveRecord::Base.establish_connection( ARGV.last)
|
9
7
|
|
10
8
|
# Simple example
|
11
9
|
class Event < ActiveRecord::Base
|
12
10
|
self.primary_key = "identifier"
|
13
|
-
|
14
|
-
include Timescale::HypertableHelpers
|
11
|
+
acts_as_hypertable
|
15
12
|
end
|
16
13
|
|
17
14
|
# Setup Hypertable as in a migration
|
data/lib/timescale/chunk.rb
CHANGED
@@ -8,6 +8,14 @@ module Timescale
|
|
8
8
|
scope :compressed, -> { where(is_compressed: true) }
|
9
9
|
scope :uncompressed, -> { where(is_compressed: false) }
|
10
10
|
|
11
|
+
scope :resume, -> do
|
12
|
+
{
|
13
|
+
total: count,
|
14
|
+
compressed: compressed.count,
|
15
|
+
uncompressed: uncompressed.count
|
16
|
+
}
|
17
|
+
end
|
18
|
+
|
11
19
|
def compress!
|
12
20
|
execute("SELECT compress_chunk(#{chunk_relation})")
|
13
21
|
end
|
@@ -5,5 +5,14 @@ module Timescale
|
|
5
5
|
|
6
6
|
has_many :jobs, foreign_key: "hypertable_name",
|
7
7
|
class_name: "Timescale::Job"
|
8
|
+
|
9
|
+
has_many :chunks, foreign_key: "hypertable_name",
|
10
|
+
class_name: "Timescale::Chunk"
|
11
|
+
|
12
|
+
scope :resume, -> do
|
13
|
+
{
|
14
|
+
total: count
|
15
|
+
}
|
16
|
+
end
|
8
17
|
end
|
9
18
|
end
|
data/lib/timescale/hypertable.rb
CHANGED
@@ -1,16 +1,19 @@
|
|
1
1
|
module Timescale
|
2
2
|
class Hypertable < ActiveRecord::Base
|
3
3
|
self.table_name = "timescaledb_information.hypertables"
|
4
|
-
|
5
4
|
self.primary_key = "hypertable_name"
|
6
5
|
|
7
6
|
has_many :jobs, foreign_key: "hypertable_name"
|
8
7
|
has_many :chunks, foreign_key: "hypertable_name"
|
9
8
|
|
10
|
-
|
9
|
+
has_one :compression_settings,
|
11
10
|
foreign_key: "hypertable_name",
|
12
11
|
class_name: "Timescale::CompressionSettings"
|
13
12
|
|
13
|
+
has_one :dimensions,
|
14
|
+
foreign_key: "hypertable_name",
|
15
|
+
class_name: "Timescale::Dimensions"
|
16
|
+
|
14
17
|
has_many :continuous_aggregates,
|
15
18
|
foreign_key: "hypertable_name",
|
16
19
|
class_name: "Timescale::ContinuousAggregates"
|
@@ -24,7 +27,8 @@ module Timescale
|
|
24
27
|
end
|
25
28
|
|
26
29
|
def compression_stats
|
27
|
-
|
30
|
+
@compression_stats ||=
|
31
|
+
struct_from("SELECT * from hypertable_compression_stats('#{self.hypertable_name}')").first || {}
|
28
32
|
end
|
29
33
|
|
30
34
|
def detailed_size
|
@@ -21,12 +21,12 @@ module Timescale
|
|
21
21
|
# end
|
22
22
|
def create_table(table_name, id: :primary_key, primary_key: nil, force: nil, **options)
|
23
23
|
super
|
24
|
-
|
24
|
+
create_hypertable(table_name, **options[:hypertable]) if options.key?(:hypertable)
|
25
25
|
end
|
26
26
|
|
27
27
|
# Setup hypertable from options
|
28
28
|
# @see create_table with the hypertable options.
|
29
|
-
def
|
29
|
+
def create_hypertable(table_name,
|
30
30
|
time_column: 'created_at',
|
31
31
|
chunk_time_interval: '1 week',
|
32
32
|
compress_segmentby: nil,
|
@@ -0,0 +1,24 @@
|
|
1
|
+
ActiveRecord::ConnectionAdapters::PostgreSQL::SchemaDumper.class_eval do
|
2
|
+
def table(table_name, stream)
|
3
|
+
super(table_name, stream)
|
4
|
+
if hypertable=Timescale::Hypertable.find_by(hypertable_name: table_name)
|
5
|
+
dim = hypertable.dimensions
|
6
|
+
# TODO Build compression settings for the template:
|
7
|
+
# #{build_compression_settings_for(hypertable)})
|
8
|
+
stream.puts <<TEMPLATE
|
9
|
+
create_hypertable('#{table_name}',
|
10
|
+
time_column: '#{dim.column_name}',
|
11
|
+
chunk_time_interval: '#{dim.time_interval.inspect}')
|
12
|
+
TEMPLATE
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
=begin
|
18
|
+
def build_compression_settings_for(hypertable)
|
19
|
+
return if hypertable.compression_settings.nil?
|
20
|
+
hypertable.compression_settings.map do |settings|
|
21
|
+
", compress_segmentby: #{settings.segmentby_column_index},
|
22
|
+
compress_orderby: 'created_at',
|
23
|
+
compression_interval: nil)
|
24
|
+
=end
|
@@ -1,28 +1,35 @@
|
|
1
1
|
require "active_support/core_ext/numeric/conversions"
|
2
|
+
|
2
3
|
module Timescale
|
3
4
|
module StatsReport
|
4
5
|
module_function
|
5
|
-
def resume
|
6
|
+
def resume(scope=Hypertable.all)
|
7
|
+
base_filter = {hypertable_name: scope.pluck(:hypertable_name)}
|
6
8
|
{
|
7
9
|
hypertables: {
|
8
|
-
count:
|
9
|
-
uncompressed:
|
10
|
-
approximate_row_count:
|
11
|
-
|
12
|
-
|
13
|
-
chunks: {
|
14
|
-
total: Chunk.count,
|
15
|
-
compressed: Chunk.compressed.count,
|
16
|
-
uncompressed: Chunk.uncompressed.count
|
17
|
-
},
|
18
|
-
size: {
|
19
|
-
before_compressing: Hypertable.all.map{|h|h.before_total_bytes}.inject(:+).to_s(:human_size),
|
20
|
-
after_compressing: Hypertable.all.map{|h|h.after_total_bytes}.inject(:+).to_s(:human_size)
|
21
|
-
}
|
10
|
+
count: scope.count,
|
11
|
+
uncompressed: scope.to_a.count { |h| h.compression_stats.empty? },
|
12
|
+
approximate_row_count: approximate_row_count(scope),
|
13
|
+
chunks: Chunk.where(base_filter).resume,
|
14
|
+
size: compression_resume(scope)
|
22
15
|
},
|
23
|
-
continuous_aggregates:
|
24
|
-
jobs_stats: JobStats.resume
|
16
|
+
continuous_aggregates: ContinuousAggregates.where(base_filter).resume,
|
17
|
+
jobs_stats: JobStats.where(base_filter).resume
|
25
18
|
}
|
26
19
|
end
|
20
|
+
|
21
|
+
def compression_resume(scope)
|
22
|
+
sum = -> (method) { (scope.map(&method).inject(:+) || 0).to_s(:human_size)}
|
23
|
+
{
|
24
|
+
uncompressed: sum[:before_total_bytes],
|
25
|
+
compressed: sum[:after_total_bytes]
|
26
|
+
}
|
27
|
+
end
|
28
|
+
|
29
|
+
def approximate_row_count(scope)
|
30
|
+
scope.to_a.map do |hypertable|
|
31
|
+
{ hypertable.hypertable_name => hypertable.approximate_row_count }
|
32
|
+
end.inject(&:merge!)
|
33
|
+
end
|
27
34
|
end
|
28
35
|
end
|
data/lib/timescale/version.rb
CHANGED
data/lib/timescale.rb
CHANGED
@@ -4,6 +4,7 @@ require_relative 'timescale/acts_as_hypertable'
|
|
4
4
|
require_relative 'timescale/chunk'
|
5
5
|
require_relative 'timescale/compression_settings'
|
6
6
|
require_relative 'timescale/continuous_aggregates'
|
7
|
+
require_relative 'timescale/dimensions'
|
7
8
|
require_relative 'timescale/hypertable'
|
8
9
|
require_relative 'timescale/job'
|
9
10
|
require_relative 'timescale/job_stats'
|
@@ -38,8 +39,8 @@ module Timescale
|
|
38
39
|
JobStats.all
|
39
40
|
end
|
40
41
|
|
41
|
-
def
|
42
|
-
StatsReport.resume
|
42
|
+
def stats(scope=Hypertable.all)
|
43
|
+
StatsReport.resume(scope)
|
43
44
|
end
|
44
45
|
|
45
46
|
def default_hypertable_options
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: timescaledb
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jônatas Davi Paganini
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-10-
|
11
|
+
date: 2021-10-07 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: pg
|
@@ -136,10 +136,12 @@ files:
|
|
136
136
|
- lib/timescale/chunk.rb
|
137
137
|
- lib/timescale/compression_settings.rb
|
138
138
|
- lib/timescale/continuous_aggregates.rb
|
139
|
+
- lib/timescale/dimensions.rb
|
139
140
|
- lib/timescale/hypertable.rb
|
140
141
|
- lib/timescale/job.rb
|
141
142
|
- lib/timescale/job_stats.rb
|
142
143
|
- lib/timescale/migration_helpers.rb
|
144
|
+
- lib/timescale/schema_dumper.rb
|
143
145
|
- lib/timescale/stats_report.rb
|
144
146
|
- lib/timescale/version.rb
|
145
147
|
- timescale.gemspec
|