timescaledb 0.1.2 → 0.1.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +192 -38
- data/bin/tsdb +28 -4
- data/examples/Gemfile +1 -1
- data/examples/Gemfile.lock +2 -2
- data/examples/all_in_one.rb +3 -6
- data/lib/timescale/chunk.rb +8 -0
- data/lib/timescale/continuous_aggregates.rb +9 -0
- data/lib/timescale/dimensions.rb +7 -0
- data/lib/timescale/hypertable.rb +7 -3
- data/lib/timescale/migration_helpers.rb +2 -2
- data/lib/timescale/schema_dumper.rb +24 -0
- data/lib/timescale/stats_report.rb +24 -17
- data/lib/timescale/version.rb +1 -1
- data/lib/timescale.rb +3 -2
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 91dfb6aac43ec8ab347ac9cf1bc553631310259cd4f6726c42f466c08f74c761
|
4
|
+
data.tar.gz: 4436ff5f2067264fc814592efebaa9ce32d69ece7607e7d728a95356db04f739
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 71492adbe4cd1771db48b2b796affa50d3f3157b420f59482e61d4a76dfc8fcec52f5face66047fc8069f5bcdd35bace4c8ff4e0fec88ffe1331872b3d625862
|
7
|
+
data.tar.gz: 4ecf26d66361af9d79cf33f99b9a25b8233df229a0ffa6ac33afe54d4f4c4d03e6c13609c60cceabc95dc30d8e49f7343588174fb0715bda044942360ebed53b
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -1,40 +1,30 @@
|
|
1
1
|
# Timescale
|
2
2
|
|
3
|
-
Welcome to the Timescale gem! To experiment with the code, start
|
4
|
-
|
3
|
+
Welcome to the Timescale gem! To experiment with the code, start installing the
|
4
|
+
gem:
|
5
5
|
|
6
6
|
```bash
|
7
|
-
|
8
|
-
cd timescale
|
9
|
-
bundle install
|
10
|
-
rake install
|
7
|
+
gem install timescaledb
|
11
8
|
```
|
12
9
|
|
13
|
-
|
10
|
+
## The `tsdb` CLI
|
14
11
|
|
15
|
-
|
16
|
-
|
17
|
-
```
|
12
|
+
When you install the gem locally, a new command line application named `tsdb`
|
13
|
+
will be linked in your command line.
|
18
14
|
|
19
|
-
|
20
|
-
|
15
|
+
It accepts a Postgresql URI and some extra flags that can help you to get more
|
16
|
+
info from your TimescaleDB server:
|
21
17
|
|
22
18
|
```bash
|
23
|
-
|
19
|
+
tsdb <uri> --stats
|
24
20
|
```
|
25
21
|
|
26
|
-
|
27
|
-
`tsdb`. Here is an example from the console:
|
22
|
+
Where the `<uri>` is replaced with params from your connection like:
|
28
23
|
|
29
24
|
```bash
|
30
|
-
tsdb
|
25
|
+
tsdb postgres://<user>@localhost:5432/<dbname> --stats
|
31
26
|
```
|
32
27
|
|
33
|
-
To join the console use `--console`:
|
34
|
-
|
35
|
-
```bash
|
36
|
-
tsdb "postgres://jonatasdp@localhost:5432/timescale_test" --console
|
37
|
-
```
|
38
28
|
|
39
29
|
Or just check the stats:
|
40
30
|
|
@@ -42,7 +32,7 @@ Or just check the stats:
|
|
42
32
|
tsdb "postgres://jonatasdp@localhost:5432/timescale_test" --stats
|
43
33
|
```
|
44
34
|
|
45
|
-
These is a sample output from
|
35
|
+
These is a sample output from database example with almost no data:
|
46
36
|
|
47
37
|
```ruby
|
48
38
|
{:hypertables=>
|
@@ -54,19 +44,153 @@ These is a sample output from an almost empty database:
|
|
54
44
|
:jobs_stats=>[{:success=>nil, :runs=>nil, :failures=>nil}]}
|
55
45
|
```
|
56
46
|
|
47
|
+
To start a interactive ruby/[pry](https://github.com/pry/pry) console use `--console`:
|
57
48
|
The console will dynamically create models for all hypertables that it finds
|
58
49
|
in the database.
|
59
50
|
|
60
|
-
|
61
|
-
|
51
|
+
Let's consider the [caggs.sql](https://gist.github.com/jonatas/95573ad8744994094ec9f284150004f9#file-caggs-sql)
|
52
|
+
as the example of database.
|
62
53
|
|
63
|
-
This library was started on [twitch.tv/timescaledb](https://twitch.tv/timescaledb).
|
64
|
-
You can watch all episodes here:
|
65
54
|
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
55
|
+
```bash
|
56
|
+
psql postgres://jonatasdp@localhost:5432/playground -f caggs.sql
|
57
|
+
```
|
58
|
+
|
59
|
+
Then use `tsdb` in the command line with the same URI and `--stats`:
|
60
|
+
|
61
|
+
```bash
|
62
|
+
tsdb postgres://jonatasdp@localhost:5432/playground --stats
|
63
|
+
{:hypertables=>
|
64
|
+
{:count=>1,
|
65
|
+
:uncompressed=>1,
|
66
|
+
:approximate_row_count=>{"ticks"=>352},
|
67
|
+
:chunks=>{:total=>1, :compressed=>0, :uncompressed=>1},
|
68
|
+
:size=>{:uncompressed=>"88 KB", :compressed=>"0 Bytes"}},
|
69
|
+
:continuous_aggregates=>{:total=>1},
|
70
|
+
:jobs_stats=>[{:success=>nil, :runs=>nil, :failures=>nil}]}
|
71
|
+
```
|
72
|
+
|
73
|
+
To have some interactive playground with the actual database using ruby, just
|
74
|
+
try the same command before changing from `--stats` to `--console`:
|
75
|
+
|
76
|
+
### tsdb --console
|
77
|
+
|
78
|
+
The same database from previous example, is used so
|
79
|
+
the context has a hypertable named `ticks` and a view named `ohlc_1m`.
|
80
|
+
|
81
|
+
|
82
|
+
```ruby
|
83
|
+
tsdb postgres://jonatasdp@localhost:5432/playground --console
|
84
|
+
pry(Timescale)>
|
85
|
+
```
|
86
|
+
|
87
|
+
The `tsdb` CLI will automatically create ActiveRecord models for hypertables and
|
88
|
+
continuous aggregates views.
|
89
|
+
|
90
|
+
```ruby
|
91
|
+
Tick
|
92
|
+
=> Timescale::Tick(time: datetime, symbol: string, price: decimal, volume: integer)
|
93
|
+
```
|
94
|
+
|
95
|
+
Note that it's only created for this session and will never be cached in the
|
96
|
+
library or any other place.
|
97
|
+
|
98
|
+
In this case, `Tick` model comes from `ticks` hypertable that was found in the database.
|
99
|
+
It contains several extra methods inherited from `acts_as_hypertable` macro.
|
100
|
+
|
101
|
+
Let's start with the `.hypertable` method.
|
102
|
+
|
103
|
+
```ruby
|
104
|
+
Tick.hypertable
|
105
|
+
=> #<Timescale::Hypertable:0x00007fe99c258900
|
106
|
+
hypertable_schema: "public",
|
107
|
+
hypertable_name: "ticks",
|
108
|
+
owner: "jonatasdp",
|
109
|
+
num_dimensions: 1,
|
110
|
+
num_chunks: 1,
|
111
|
+
compression_enabled: false,
|
112
|
+
is_distributed: false,
|
113
|
+
replication_factor: nil,
|
114
|
+
data_nodes: nil,
|
115
|
+
tablespaces: nil>
|
116
|
+
```
|
117
|
+
|
118
|
+
The core of the hypertables are the fragmentation of the data into chunks that
|
119
|
+
are the child tables that distribute the data. You can check all chunks directly
|
120
|
+
from the hypertable relation.
|
121
|
+
|
122
|
+
```ruby
|
123
|
+
Tick.hypertable.chunks
|
124
|
+
unknown OID 2206: failed to recognize type of 'primary_dimension_type'. It will be treated as String.
|
125
|
+
=> [#<Timescale::Chunk:0x00007fe99c31b068
|
126
|
+
hypertable_schema: "public",
|
127
|
+
hypertable_name: "ticks",
|
128
|
+
chunk_schema: "_timescaledb_internal",
|
129
|
+
chunk_name: "_hyper_33_17_chunk",
|
130
|
+
primary_dimension: "time",
|
131
|
+
primary_dimension_type: "timestamp without time zone",
|
132
|
+
range_start: 1999-12-30 00:00:00 +0000,
|
133
|
+
range_end: 2000-01-06 00:00:00 +0000,
|
134
|
+
range_start_integer: nil,
|
135
|
+
range_end_integer: nil,
|
136
|
+
is_compressed: false,
|
137
|
+
chunk_tablespace: nil,
|
138
|
+
data_nodes: nil>]
|
139
|
+
```
|
140
|
+
|
141
|
+
> Chunks are created by partitioning a hypertable's data into one
|
142
|
+
> (or potentially multiple) dimensions. All hypertables are partitioned by the
|
143
|
+
> values belonging to a time column, which may be in timestamp, date, or
|
144
|
+
> various integer forms. If the time partitioning interval is one day,
|
145
|
+
> for example, then rows with timestamps that belong to the same day are co-located
|
146
|
+
> within the same chunk, while rows belonging to different days belong to different chunks.
|
147
|
+
> Learn more [here](https://docs.timescale.com/timescaledb/latest/overview/core-concepts/hypertables-and-chunks/).
|
148
|
+
|
149
|
+
Another core concept of TimescaleDB is compression. With data partitioned, it
|
150
|
+
becomes very convenient to compress and decompress chunks independently.
|
151
|
+
|
152
|
+
```ruby
|
153
|
+
Tick.hypertable.chunks.first.compress!
|
154
|
+
ActiveRecord::StatementInvalid: PG::FeatureNotSupported: ERROR: compression not enabled on "ticks"
|
155
|
+
DETAIL: It is not possible to compress chunks on a hypertable that does not have compression enabled.
|
156
|
+
HINT: Enable compression using ALTER TABLE with the timescaledb.compress option.
|
157
|
+
```
|
158
|
+
|
159
|
+
As compression is not enabled, let's do it executing a plain SQL directly from
|
160
|
+
the actual context. To borrow a connection, let's use the Tick object.
|
161
|
+
|
162
|
+
```ruby
|
163
|
+
Tick.connection.execute("ALTER TABLE ticks SET (timescaledb.compress)") # => PG_OK
|
164
|
+
```
|
165
|
+
|
166
|
+
And now, it's possible to compress and decompress:
|
167
|
+
|
168
|
+
```ruby
|
169
|
+
Tick.hypertable.chunks.first.compress!
|
170
|
+
Tick.hypertable.chunks.first.decompress!
|
171
|
+
```
|
172
|
+
Learn more about TimescaleDB compression [here](https://docs.timescale.com/timescaledb/latest/overview/core-concepts/compression/).
|
173
|
+
|
174
|
+
The `ohlc_1m` view is also available as an ActiveRecord:
|
175
|
+
|
176
|
+
```ruby
|
177
|
+
Ohlc1m
|
178
|
+
=> Timescale::Ohlc1m(bucket: datetime, symbol: string, open: decimal, high: decimal, low: decimal, close: decimal, volume: integer)
|
179
|
+
```
|
180
|
+
|
181
|
+
And you can run any query as you do with regular active record queries.
|
182
|
+
|
183
|
+
```ruby
|
184
|
+
Ohlc1m.order(bucket: :desc).last
|
185
|
+
=> #<Timescale::Ohlc1m:0x00007fe99c2c38e0
|
186
|
+
bucket: 2000-01-01 00:00:00 UTC,
|
187
|
+
symbol: "SYMBOL",
|
188
|
+
open: 0.13e2,
|
189
|
+
high: 0.3e2,
|
190
|
+
low: 0.1e1,
|
191
|
+
close: 0.1e2,
|
192
|
+
volume: 27600>
|
193
|
+
```
|
70
194
|
|
71
195
|
## Installation
|
72
196
|
|
@@ -84,13 +208,14 @@ Or install it yourself as:
|
|
84
208
|
|
85
209
|
$ gem install timescaledb
|
86
210
|
|
211
|
+
|
87
212
|
## Usage
|
88
213
|
|
89
214
|
You can check the [all_in_one.rb](examples/all_in_one.rb) that will:
|
90
215
|
|
91
216
|
1. Create hypertable with compression settings
|
92
217
|
2. Insert data
|
93
|
-
3. Run some queries
|
218
|
+
3. Run some queries
|
94
219
|
4. Check chunk size per model
|
95
220
|
5. Compress a chunk
|
96
221
|
6. Check chunk status
|
@@ -140,7 +265,7 @@ end
|
|
140
265
|
Tick = Class.new(ActiveRecord::Base) do
|
141
266
|
self.table_name = 'ticks'
|
142
267
|
self.primary_key = 'symbol'
|
143
|
-
|
268
|
+
acts_as_hypertable
|
144
269
|
end
|
145
270
|
|
146
271
|
query = Tick.select(<<~QUERY)
|
@@ -167,14 +292,14 @@ create_continuous_aggregates('ohlc_1m', query, **options)
|
|
167
292
|
|
168
293
|
### Hypertable Helpers
|
169
294
|
|
170
|
-
You can
|
295
|
+
You can say `acts_as_hypertable` to get access to some basic scopes for your
|
171
296
|
model:
|
172
297
|
|
173
298
|
```ruby
|
174
299
|
class Event < ActiveRecord::Base
|
175
300
|
self.primary_key = "identifier"
|
176
301
|
|
177
|
-
|
302
|
+
acts_as_hypertable
|
178
303
|
end
|
179
304
|
```
|
180
305
|
|
@@ -265,14 +390,17 @@ In case you want to use TimescaleDB on a Rails environment, you may have some
|
|
265
390
|
issues as the schema dump used for tests is not considering hypertables
|
266
391
|
metadata.
|
267
392
|
|
268
|
-
If you add the `
|
269
|
-
|
393
|
+
If you add the `acts_as_hypertable` to your model, you can dynamically
|
394
|
+
verify if the `Timescale::ActsAsHypertable` module is included to
|
395
|
+
create the hypertable for testing environment.
|
396
|
+
|
397
|
+
Consider adding this hook to your `spec/rspec_helper.rb` file:
|
270
398
|
|
271
399
|
```ruby
|
272
400
|
config.before(:suite) do
|
273
401
|
hypertable_models = ApplicationRecord
|
274
402
|
.descendants
|
275
|
-
.select{|clazz| clazz.
|
403
|
+
.select{|clazz| clazz.included_modules.include?(Timescale::ActsAsHypertable)
|
276
404
|
hypertable_models.each do |clazz|
|
277
405
|
if clazz.hypertable.exists?
|
278
406
|
ApplicationRecord.logger.info "skip recreating hypertable for '#{clazz.table_name}'."
|
@@ -291,13 +419,39 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
|
|
291
419
|
|
292
420
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
293
421
|
|
422
|
+
You can create a `.env` file locally to run tests locally. Make sure to put your
|
423
|
+
own credentials there!
|
424
|
+
|
425
|
+
```bash
|
426
|
+
PG_URI_TEST="postgres://<user>@localhost:5432/<dbname>"
|
427
|
+
```
|
428
|
+
|
429
|
+
You can put some postgres URI directly as a parameter of
|
430
|
+
`tsdb`. Here is an example from the console:
|
431
|
+
|
432
|
+
```bash
|
433
|
+
tsdb "postgres://jonatasdp@localhost:5432/timescale_test"
|
434
|
+
```
|
435
|
+
|
436
|
+
## More resources
|
437
|
+
|
438
|
+
This library was started on [twitch.tv/timescaledb](https://twitch.tv/timescaledb).
|
439
|
+
You can watch all episodes here:
|
440
|
+
|
441
|
+
1. [Wrapping Functions to Ruby Helpers](https://www.youtube.com/watch?v=hGPsUxLFAYk).
|
442
|
+
2. [Extending ActiveRecord with Timescale Helpers](https://www.youtube.com/watch?v=IEyJIHk1Clk).
|
443
|
+
3. [Setup Hypertables for Rails testing environment](https://www.youtube.com/watch?v=wM6hVrZe7xA).
|
444
|
+
4. [Packing the code to this repository](https://www.youtube.com/watch?v=CMdGAl_XlL4).
|
445
|
+
4. [the code to this repository](https://www.youtube.com/watch?v=CMdGAl_XlL4).
|
446
|
+
5. [Working with Timescale continuous aggregates](https://youtu.be/co4HnBkHzVw).
|
447
|
+
6. [Creating the command-line application in Ruby to explore the Timescale API](https://www.youtube.com/watch?v=I3vM_q2m7T0).
|
448
|
+
|
294
449
|
### TODO
|
295
450
|
|
296
451
|
Here is a list of functions that would be great to have:
|
297
452
|
|
298
453
|
- [ ] Dump and Restore Timescale metadata - Like db/schema.rb but for Timescale configuration.
|
299
454
|
- [ ] Add data nodes support
|
300
|
-
- [ ] Implement the `timescale` CLI to explore the full API.
|
301
455
|
|
302
456
|
## Contributing
|
303
457
|
|
data/bin/tsdb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
require "bundler/setup"
|
3
3
|
require "timescale"
|
4
|
+
require "pry"
|
4
5
|
|
5
6
|
ActiveRecord::Base.establish_connection(ARGV[0])
|
6
7
|
|
@@ -8,17 +9,40 @@ Timescale::Hypertable.find_each do |hypertable|
|
|
8
9
|
class_name = hypertable.hypertable_name.singularize.camelize
|
9
10
|
model = Class.new(ActiveRecord::Base) do
|
10
11
|
self.table_name = hypertable.hypertable_name
|
11
|
-
|
12
|
-
include Timescale::HypertableHelpers
|
12
|
+
acts_as_hypertable
|
13
13
|
end
|
14
14
|
Timescale.const_set(class_name, model)
|
15
15
|
end
|
16
16
|
|
17
|
+
Timescale::ContinuousAggregates.find_each do |cagg|
|
18
|
+
class_name = cagg.view_name.singularize.camelize
|
19
|
+
model = Class.new(ActiveRecord::Base) do
|
20
|
+
self.table_name = cagg.view_name
|
21
|
+
acts_as_hypertable
|
22
|
+
end
|
23
|
+
Timescale.const_set(class_name, model)
|
24
|
+
end
|
25
|
+
|
26
|
+
def show(obj)
|
27
|
+
Pry::ColorPrinter.pp(obj)
|
28
|
+
end
|
29
|
+
|
17
30
|
if ARGV.index("--stats")
|
18
|
-
|
31
|
+
scope = Timescale::Hypertable.all
|
32
|
+
|
33
|
+
if (only = ARGV.index("--only"))
|
34
|
+
only_hypertables = ARGV[only+1].split(",")
|
35
|
+
scope = scope.where({hypertable_name: only_hypertables})
|
36
|
+
end
|
37
|
+
|
38
|
+
if (except = ARGV.index("--except"))
|
39
|
+
except_hypertables = ARGV[except+1].split(",")
|
40
|
+
scope = scope.where.not(hypertable_name: except_hypertables)
|
41
|
+
end
|
42
|
+
|
43
|
+
show(Timescale.stats(scope))
|
19
44
|
end
|
20
45
|
|
21
46
|
if ARGV.index("--console")
|
22
|
-
require "pry"
|
23
47
|
Pry.start(Timescale)
|
24
48
|
end
|
data/examples/Gemfile
CHANGED
data/examples/Gemfile.lock
CHANGED
data/examples/all_in_one.rb
CHANGED
@@ -2,16 +2,13 @@ require 'bundler/setup'
|
|
2
2
|
require 'timescale'
|
3
3
|
require 'pp'
|
4
4
|
require 'pry'
|
5
|
-
|
6
|
-
|
7
|
-
# set PG_URI=postgres://user:pass@host:port/db_name
|
8
|
-
ActiveRecord::Base.establish_connection(ENV['PG_URI_TEST'])
|
5
|
+
# ruby all_in_one.rb postgres://user:pass@host:port/db_name
|
6
|
+
ActiveRecord::Base.establish_connection( ARGV.last)
|
9
7
|
|
10
8
|
# Simple example
|
11
9
|
class Event < ActiveRecord::Base
|
12
10
|
self.primary_key = "identifier"
|
13
|
-
|
14
|
-
include Timescale::HypertableHelpers
|
11
|
+
acts_as_hypertable
|
15
12
|
end
|
16
13
|
|
17
14
|
# Setup Hypertable as in a migration
|
data/lib/timescale/chunk.rb
CHANGED
@@ -8,6 +8,14 @@ module Timescale
|
|
8
8
|
scope :compressed, -> { where(is_compressed: true) }
|
9
9
|
scope :uncompressed, -> { where(is_compressed: false) }
|
10
10
|
|
11
|
+
scope :resume, -> do
|
12
|
+
{
|
13
|
+
total: count,
|
14
|
+
compressed: compressed.count,
|
15
|
+
uncompressed: uncompressed.count
|
16
|
+
}
|
17
|
+
end
|
18
|
+
|
11
19
|
def compress!
|
12
20
|
execute("SELECT compress_chunk(#{chunk_relation})")
|
13
21
|
end
|
@@ -5,5 +5,14 @@ module Timescale
|
|
5
5
|
|
6
6
|
has_many :jobs, foreign_key: "hypertable_name",
|
7
7
|
class_name: "Timescale::Job"
|
8
|
+
|
9
|
+
has_many :chunks, foreign_key: "hypertable_name",
|
10
|
+
class_name: "Timescale::Chunk"
|
11
|
+
|
12
|
+
scope :resume, -> do
|
13
|
+
{
|
14
|
+
total: count
|
15
|
+
}
|
16
|
+
end
|
8
17
|
end
|
9
18
|
end
|
data/lib/timescale/hypertable.rb
CHANGED
@@ -1,16 +1,19 @@
|
|
1
1
|
module Timescale
|
2
2
|
class Hypertable < ActiveRecord::Base
|
3
3
|
self.table_name = "timescaledb_information.hypertables"
|
4
|
-
|
5
4
|
self.primary_key = "hypertable_name"
|
6
5
|
|
7
6
|
has_many :jobs, foreign_key: "hypertable_name"
|
8
7
|
has_many :chunks, foreign_key: "hypertable_name"
|
9
8
|
|
10
|
-
|
9
|
+
has_one :compression_settings,
|
11
10
|
foreign_key: "hypertable_name",
|
12
11
|
class_name: "Timescale::CompressionSettings"
|
13
12
|
|
13
|
+
has_one :dimensions,
|
14
|
+
foreign_key: "hypertable_name",
|
15
|
+
class_name: "Timescale::Dimensions"
|
16
|
+
|
14
17
|
has_many :continuous_aggregates,
|
15
18
|
foreign_key: "hypertable_name",
|
16
19
|
class_name: "Timescale::ContinuousAggregates"
|
@@ -24,7 +27,8 @@ module Timescale
|
|
24
27
|
end
|
25
28
|
|
26
29
|
def compression_stats
|
27
|
-
|
30
|
+
@compression_stats ||=
|
31
|
+
struct_from("SELECT * from hypertable_compression_stats('#{self.hypertable_name}')").first || {}
|
28
32
|
end
|
29
33
|
|
30
34
|
def detailed_size
|
@@ -21,12 +21,12 @@ module Timescale
|
|
21
21
|
# end
|
22
22
|
def create_table(table_name, id: :primary_key, primary_key: nil, force: nil, **options)
|
23
23
|
super
|
24
|
-
|
24
|
+
create_hypertable(table_name, **options[:hypertable]) if options.key?(:hypertable)
|
25
25
|
end
|
26
26
|
|
27
27
|
# Setup hypertable from options
|
28
28
|
# @see create_table with the hypertable options.
|
29
|
-
def
|
29
|
+
def create_hypertable(table_name,
|
30
30
|
time_column: 'created_at',
|
31
31
|
chunk_time_interval: '1 week',
|
32
32
|
compress_segmentby: nil,
|
@@ -0,0 +1,24 @@
|
|
1
|
+
ActiveRecord::ConnectionAdapters::PostgreSQL::SchemaDumper.class_eval do
|
2
|
+
def table(table_name, stream)
|
3
|
+
super(table_name, stream)
|
4
|
+
if hypertable=Timescale::Hypertable.find_by(hypertable_name: table_name)
|
5
|
+
dim = hypertable.dimensions
|
6
|
+
# TODO Build compression settings for the template:
|
7
|
+
# #{build_compression_settings_for(hypertable)})
|
8
|
+
stream.puts <<TEMPLATE
|
9
|
+
create_hypertable('#{table_name}',
|
10
|
+
time_column: '#{dim.column_name}',
|
11
|
+
chunk_time_interval: '#{dim.time_interval.inspect}')
|
12
|
+
TEMPLATE
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
=begin
|
18
|
+
def build_compression_settings_for(hypertable)
|
19
|
+
return if hypertable.compression_settings.nil?
|
20
|
+
hypertable.compression_settings.map do |settings|
|
21
|
+
", compress_segmentby: #{settings.segmentby_column_index},
|
22
|
+
compress_orderby: 'created_at',
|
23
|
+
compression_interval: nil)
|
24
|
+
=end
|
@@ -1,28 +1,35 @@
|
|
1
1
|
require "active_support/core_ext/numeric/conversions"
|
2
|
+
|
2
3
|
module Timescale
|
3
4
|
module StatsReport
|
4
5
|
module_function
|
5
|
-
def resume
|
6
|
+
def resume(scope=Hypertable.all)
|
7
|
+
base_filter = {hypertable_name: scope.pluck(:hypertable_name)}
|
6
8
|
{
|
7
9
|
hypertables: {
|
8
|
-
count:
|
9
|
-
uncompressed:
|
10
|
-
approximate_row_count:
|
11
|
-
|
12
|
-
|
13
|
-
chunks: {
|
14
|
-
total: Chunk.count,
|
15
|
-
compressed: Chunk.compressed.count,
|
16
|
-
uncompressed: Chunk.uncompressed.count
|
17
|
-
},
|
18
|
-
size: {
|
19
|
-
before_compressing: Hypertable.all.map{|h|h.before_total_bytes}.inject(:+).to_s(:human_size),
|
20
|
-
after_compressing: Hypertable.all.map{|h|h.after_total_bytes}.inject(:+).to_s(:human_size)
|
21
|
-
}
|
10
|
+
count: scope.count,
|
11
|
+
uncompressed: scope.to_a.count { |h| h.compression_stats.empty? },
|
12
|
+
approximate_row_count: approximate_row_count(scope),
|
13
|
+
chunks: Chunk.where(base_filter).resume,
|
14
|
+
size: compression_resume(scope)
|
22
15
|
},
|
23
|
-
continuous_aggregates:
|
24
|
-
jobs_stats: JobStats.resume
|
16
|
+
continuous_aggregates: ContinuousAggregates.where(base_filter).resume,
|
17
|
+
jobs_stats: JobStats.where(base_filter).resume
|
25
18
|
}
|
26
19
|
end
|
20
|
+
|
21
|
+
def compression_resume(scope)
|
22
|
+
sum = -> (method) { (scope.map(&method).inject(:+) || 0).to_s(:human_size)}
|
23
|
+
{
|
24
|
+
uncompressed: sum[:before_total_bytes],
|
25
|
+
compressed: sum[:after_total_bytes]
|
26
|
+
}
|
27
|
+
end
|
28
|
+
|
29
|
+
def approximate_row_count(scope)
|
30
|
+
scope.to_a.map do |hypertable|
|
31
|
+
{ hypertable.hypertable_name => hypertable.approximate_row_count }
|
32
|
+
end.inject(&:merge!)
|
33
|
+
end
|
27
34
|
end
|
28
35
|
end
|
data/lib/timescale/version.rb
CHANGED
data/lib/timescale.rb
CHANGED
@@ -4,6 +4,7 @@ require_relative 'timescale/acts_as_hypertable'
|
|
4
4
|
require_relative 'timescale/chunk'
|
5
5
|
require_relative 'timescale/compression_settings'
|
6
6
|
require_relative 'timescale/continuous_aggregates'
|
7
|
+
require_relative 'timescale/dimensions'
|
7
8
|
require_relative 'timescale/hypertable'
|
8
9
|
require_relative 'timescale/job'
|
9
10
|
require_relative 'timescale/job_stats'
|
@@ -38,8 +39,8 @@ module Timescale
|
|
38
39
|
JobStats.all
|
39
40
|
end
|
40
41
|
|
41
|
-
def
|
42
|
-
StatsReport.resume
|
42
|
+
def stats(scope=Hypertable.all)
|
43
|
+
StatsReport.resume(scope)
|
43
44
|
end
|
44
45
|
|
45
46
|
def default_hypertable_options
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: timescaledb
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jônatas Davi Paganini
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-10-
|
11
|
+
date: 2021-10-07 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: pg
|
@@ -136,10 +136,12 @@ files:
|
|
136
136
|
- lib/timescale/chunk.rb
|
137
137
|
- lib/timescale/compression_settings.rb
|
138
138
|
- lib/timescale/continuous_aggregates.rb
|
139
|
+
- lib/timescale/dimensions.rb
|
139
140
|
- lib/timescale/hypertable.rb
|
140
141
|
- lib/timescale/job.rb
|
141
142
|
- lib/timescale/job_stats.rb
|
142
143
|
- lib/timescale/migration_helpers.rb
|
144
|
+
- lib/timescale/schema_dumper.rb
|
143
145
|
- lib/timescale/stats_report.rb
|
144
146
|
- lib/timescale/version.rb
|
145
147
|
- timescale.gemspec
|