elasticshelf 0.0.2 → 0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/Gemfile +0 -3
- data/README.md +148 -20
- data/elasticshelf.gemspec +1 -2
- data/examples/wither_older_than_days.rb +17 -0
- data/lib/elasticshelf/version.rb +1 -1
- data/lib/elasticshelf.rb +325 -43
- metadata +5 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 600e76e898a7092c714c0eb4eb8065af93c6690d
|
4
|
+
data.tar.gz: 1aae40b15a8349ea1a4e7616f40e444e58e48229
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 600f1e3260425d1b89b5087c6cf8b7729dd0dfb377a1486bdde25550ed34b4e2b7222211980eaf11908aeebe264a75f150b5ca744b6c9b1287fbdcc1f76771cc
|
7
|
+
data.tar.gz: 04c382b1ca40101f3c7ba9a8049512919b32d78f9fb382f90a024b7283988167f6859268529f29fe044ef925431e73e3234d646cf5530530d4ba2df19cf40b0d
|
data/.gitignore
CHANGED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,26 +1,21 @@
|
|
1
1
|
# Elasticshelf
|
2
2
|
|
3
|
-
Ruby integrations for Elasticsearch indices management using close,
|
3
|
+
Ruby integrations for Elasticsearch indices management using close, delete, snapshot and restore.
|
4
4
|
|
5
|
-
Inspired by Curator at https://github.com/elasticsearch/curator.git, but if
|
5
|
+
Inspired by Curator at https://github.com/elasticsearch/curator.git, but if
|
6
6
|
you are an ophidiophobe, no worries, this is written in Ruby :-)
|
7
7
|
|
8
8
|
Honestly, there is nothing special here that can not already be done with the elasticsearch-ruby gem,
|
9
9
|
but it's just packaged up as a convenience gem.
|
10
10
|
|
11
11
|
The goal is to perform the following on Elasticsearch indices:
|
12
|
-
* **wither** (
|
12
|
+
* **wither** (i.e. disable bloom filter)
|
13
13
|
* **close**
|
14
14
|
* **open**
|
15
15
|
* **delete**
|
16
16
|
* **snapshot**
|
17
17
|
* **restore**
|
18
18
|
|
19
|
-
## TODO's:
|
20
|
-
|
21
|
-
Loop thru expired indices hash and perform an action:
|
22
|
-
close, open, delete, wither, snapshot, or restore
|
23
|
-
|
24
19
|
## Installation
|
25
20
|
|
26
21
|
Add this line to your application's Gemfile:
|
@@ -37,33 +32,166 @@ Or install it yourself as:
|
|
37
32
|
|
38
33
|
## Usage
|
39
34
|
|
35
|
+
> Note: see the examples folder for more command line ruby scripts.
|
36
|
+
|
37
|
+
initial setup:
|
38
|
+
```
|
39
|
+
require 'elasticshelf'
|
40
|
+
es = Elasticshelf::Client.new(:host => '127.0.0.1:9200')
|
41
|
+
```
|
42
|
+
if required be sure to override these defaults:
|
43
|
+
```
|
44
|
+
es.indices_prefix = 'logstash-'
|
45
|
+
es.date_separator = '.'
|
46
|
+
```
|
47
|
+
> Note: an asterisk '*' is appended to indices_prefix.
|
48
|
+
|
49
|
+
ensure the cutoff date is as desired:
|
50
|
+
```
|
51
|
+
es.cutoff_days = 30
|
52
|
+
es.cutoff_days_as_date_to_s
|
53
|
+
```
|
54
|
+
do a "dry run" to see what indices will be affected:
|
55
|
+
```
|
56
|
+
es.find_expired_indices
|
57
|
+
```
|
58
|
+
|
59
|
+
### Wither expired indices (i.e. remove bloom filter)
|
60
|
+
|
61
|
+
remove the bloom filter on any indices older than 30 days, but only if they are not closed already:
|
62
|
+
```
|
63
|
+
es.cutoff_days = 30
|
64
|
+
es.wither_indices
|
65
|
+
```
|
66
|
+
or wither a single index:
|
67
|
+
```
|
68
|
+
es.wither_index("index_name")
|
69
|
+
```
|
70
|
+
|
71
|
+
### Close expired indices
|
72
|
+
|
73
|
+
close any indices older than 60 days:
|
74
|
+
```
|
75
|
+
es.cutoff_days = 60
|
76
|
+
es.close_expired_indices
|
77
|
+
```
|
78
|
+
or close a single index:
|
79
|
+
```
|
80
|
+
es.close_index("index_name")
|
81
|
+
```
|
82
|
+
|
83
|
+
### Open index
|
84
|
+
|
85
|
+
> just in case one was closed by mistake
|
86
|
+
```
|
87
|
+
es.open_index("index_name")
|
88
|
+
```
|
89
|
+
|
90
|
+
### Delete expired indices
|
91
|
+
|
92
|
+
delete any indices older than 60 days:
|
93
|
+
```
|
94
|
+
es.cutoff_days = 60
|
95
|
+
es.delete_expired_indices
|
96
|
+
```
|
97
|
+
or delete a single index:
|
98
|
+
```
|
99
|
+
es.delete_index("index_name")
|
40
100
|
```
|
41
|
-
$ es = Elasticshelf::Client.new(:host => '127.0.0.1:9200')
|
42
101
|
|
43
|
-
|
102
|
+
### Snapshot create repository
|
44
103
|
|
45
|
-
|
104
|
+
> Notes:
|
105
|
+
* some sysadmin prep is needed to ensure the **location** is accessible by Elasticsearch
|
106
|
+
* **repo_type** defaults to 'fs'
|
107
|
+
* **repo_compressed** defaults to true
|
46
108
|
|
47
|
-
$ es.find_expired_indices() ... default is 'logstash-*' is not specified
|
48
109
|
```
|
110
|
+
es.repo = 'name_of_the_snapshot_repository'
|
111
|
+
es.repo_location = '/var/elasticsearch_snapshots' ... or what was prepped by the sysadmin
|
112
|
+
es.snapshot_create_repository
|
113
|
+
```
|
114
|
+
|
115
|
+
### Snapshot get repository
|
49
116
|
|
50
|
-
|
117
|
+
> Note: if **es.repo** is not set this gets information about all snapshot repositories.
|
51
118
|
|
52
119
|
```
|
53
|
-
|
120
|
+
es.snapshot_get_repository
|
121
|
+
```
|
54
122
|
|
55
|
-
|
123
|
+
### Snapshot delete repository
|
56
124
|
|
57
|
-
|
125
|
+
```
|
126
|
+
es.repo = 'name_of_the_snapshot_repository'
|
127
|
+
es.snapshot_delete_repository
|
128
|
+
```
|
58
129
|
|
59
|
-
|
130
|
+
### Snapshot expired indices
|
60
131
|
|
61
|
-
|
132
|
+
take a snapshot of all indices older than 22 days:
|
133
|
+
```
|
134
|
+
es.cutoff_days = 22
|
135
|
+
es.repo = 'name_of_the_snapshot_repository'
|
136
|
+
es.snapshot_expired_indices
|
137
|
+
```
|
138
|
+
> Note: this snapshot is auto-named like "2014-04-02_15:07:55_utc_logstash-_cutoff_22_days",
|
139
|
+
> as there may be many indices within this snapshot. Override this as described below.
|
62
140
|
|
63
|
-
|
141
|
+
override snapshot auto-naming behavior:
|
142
|
+
```
|
143
|
+
es.snapshot_expired_indices_name = "more_desirable_name"
|
144
|
+
es.cutoff_days = 22
|
145
|
+
es.repo = 'name_of_the_snapshot_repository'
|
146
|
+
es.snapshot_expired_indices
|
147
|
+
```
|
148
|
+
|
149
|
+
### Snapshot one or more indices
|
64
150
|
|
65
|
-
$ es.get_index_state("index_name")
|
66
151
|
```
|
152
|
+
es.indices = "index1" ... single
|
153
|
+
es.indices = "index1,index2" ... multiples
|
154
|
+
es.repo = 'name_of_the_snapshot_repository'
|
155
|
+
es.snapshot = 'name_of_this_snapshot'
|
156
|
+
es.snapshot_expired_indices
|
157
|
+
```
|
158
|
+
|
159
|
+
### Snapshot get
|
160
|
+
|
161
|
+
> Note: if **es.snapshot** is not set this gets information about all snapshots,
|
162
|
+
> i.e. to see all snapshots do es.snapshot=nil.
|
163
|
+
|
164
|
+
```
|
165
|
+
es.snapshot_get
|
166
|
+
```
|
167
|
+
|
168
|
+
### Snapshot delete
|
169
|
+
|
170
|
+
```
|
171
|
+
es.snapshot = 'name_of_this_snapshot'
|
172
|
+
es.snapshot_delete
|
173
|
+
```
|
174
|
+
|
175
|
+
### Other usage
|
176
|
+
|
177
|
+
```
|
178
|
+
es.index_closed?("index_name")
|
179
|
+
es.get_elasticsearch_version
|
180
|
+
es.get_lucene_version
|
181
|
+
es.get_info
|
182
|
+
es.get_cluster_state("index_name")
|
183
|
+
es.get_index_state("index_name")
|
184
|
+
es.version ... of this gem
|
185
|
+
```
|
186
|
+
|
187
|
+
### Notes
|
188
|
+
* both open and closed indices are included when finding expired indices
|
189
|
+
* can not wither a closed index
|
190
|
+
* can not snapshot a closed index
|
191
|
+
* can not restore an open index from a snapshot
|
192
|
+
* deleting all (i.e. '*' or '_all') indices is not allowed
|
193
|
+
* both es.errors and es.results should be checked after most usage
|
194
|
+
* exceptions are not re-raised from the elasticsearch-ruby gem, but the class/message are echoed in es.errors
|
67
195
|
|
68
196
|
## Contributing
|
69
197
|
|
data/elasticshelf.gemspec
CHANGED
@@ -9,7 +9,7 @@ Gem::Specification.new do |spec|
|
|
9
9
|
spec.authors = ["cLee Smith"]
|
10
10
|
spec.email = ["cleesmith2006@gmail.com"]
|
11
11
|
spec.summary = "Ruby integrations for Elasticsearch indices management"
|
12
|
-
spec.description = "Ruby integrations for Elasticsearch indices management using close,
|
12
|
+
spec.description = "Ruby integrations for Elasticsearch indices management using close, delete, snapshot, and restore"
|
13
13
|
spec.homepage = "https://github.com/cleesmith/elasticshelf.git"
|
14
14
|
spec.license = "MIT"
|
15
15
|
|
@@ -19,7 +19,6 @@ Gem::Specification.new do |spec|
|
|
19
19
|
spec.require_paths = ["lib"]
|
20
20
|
|
21
21
|
spec.add_dependency "elasticsearch", '1.0.1'
|
22
|
-
spec.add_dependency "time_diff", '0.3.0'
|
23
22
|
|
24
23
|
spec.add_development_dependency "bundler", "~> 1.5"
|
25
24
|
spec.add_development_dependency "rake"
|
@@ -0,0 +1,17 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require 'elasticshelf'
|
3
|
+
|
4
|
+
if ARGV.empty?
|
5
|
+
puts "please enter the cutoff days number"
|
6
|
+
exit(1)
|
7
|
+
end
|
8
|
+
days = ARGV[0].to_i
|
9
|
+
|
10
|
+
es = Elasticshelf::Client.new(:host => '127.0.0.1:9200')
|
11
|
+
|
12
|
+
# optional, as 'logstash-' and '.' are the defaults:
|
13
|
+
# es.indices_prefix = '???-'
|
14
|
+
# es.date_separator = '.'
|
15
|
+
|
16
|
+
es.cutoff_days = days
|
17
|
+
es.wither_indices
|
data/lib/elasticshelf/version.rb
CHANGED
data/lib/elasticshelf.rb
CHANGED
@@ -5,37 +5,73 @@ module Elasticshelf
|
|
5
5
|
|
6
6
|
module Client
|
7
7
|
|
8
|
-
attr_reader :client, :
|
8
|
+
attr_reader :client, :results, :errors
|
9
|
+
|
10
|
+
attr_reader :get_info, :version
|
11
|
+
|
12
|
+
attr_reader :expired_indices, :indices_affected, :indices_rejected, :indices_closed, :indices_deleted, :indices_withered
|
13
|
+
|
14
|
+
attr_accessor :indices_prefix, :date_separator, :cutoff_days
|
15
|
+
|
16
|
+
attr_accessor :repo, :repo_type, :repo_location, :repo_compressed, :snapshot, :indices
|
17
|
+
|
18
|
+
attr_accessor :wait_for_completion, :ignore_unavailable, :include_global_state
|
19
|
+
|
20
|
+
attr_accessor :snapshot_expired_indices_name
|
9
21
|
|
10
22
|
def new(arguments={})
|
11
23
|
@client = Elasticsearch::Transport::Client.new(arguments)
|
24
|
+
@wait_for_completion = true
|
25
|
+
@ignore_unavailable = true
|
26
|
+
@include_global_state = false
|
27
|
+
@snapshot_expired_indices_name = nil
|
28
|
+
@date_separator = '.'
|
29
|
+
@indices_prefix = 'logstash-'
|
30
|
+
@cutoff_days = -1
|
31
|
+
@repo_type = 'fs'
|
32
|
+
@repo_compressed = true
|
33
|
+
@indices_affected = {}
|
34
|
+
@indices_rejected = {}
|
35
|
+
@expired_indices = {}
|
36
|
+
@indices_closed = {}
|
37
|
+
@indices_deleted = {}
|
38
|
+
@indices_withered = {}
|
39
|
+
@results = {}
|
40
|
+
@errors = {}
|
12
41
|
self
|
13
42
|
end
|
14
43
|
extend self
|
15
44
|
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
45
|
+
def indices_prefix=(str)
|
46
|
+
@indices_prefix = str.chomp('*') + '*'
|
47
|
+
end
|
48
|
+
|
49
|
+
def find_expired_indices
|
50
|
+
try do
|
51
|
+
return {} if blank? @indices_prefix
|
52
|
+
return {} if blank? @date_separator
|
53
|
+
return {} if @cutoff_days.nil?
|
54
|
+
return {} if @cutoff_days.to_i < 0
|
55
|
+
@expired_indices = {}
|
56
|
+
# cutoff_date = cutoff_days_as_date
|
57
|
+
# cutoff_date_beginning_of_day = set_time_to_beginning_of_day(cutoff_date.strftime("%Y.%m.%d"))
|
58
|
+
cutoff_date_beginning_of_day = cutoff_days_as_date
|
59
|
+
required_date_parts = 3 # for now, we only do days not hours
|
60
|
+
# the following should only return 'closed' indices but it returns all indices ???
|
61
|
+
# sorted_indices = @client.indices.get_settings(:index => @indices_prefix, :expand_wildcards => 'closed').keys
|
62
|
+
# sorted_indices = @client.indices.get_settings(:index => @indices_prefix, :expand_wildcards => 'open').keys
|
63
|
+
sorted_indices = @client.indices.get_settings(:index => @indices_prefix, :expand_wildcards => 'open,closed').keys
|
64
|
+
sorted_indices.each do |index_name|
|
65
|
+
unprefixed_index_name = index_name.slice(@indices_prefix.size-1..-1)
|
66
|
+
date_parts = unprefixed_index_name.split(@date_separator)
|
67
|
+
next unless date_parts.size == required_date_parts
|
68
|
+
index_time = set_time_to_beginning_of_day(unprefixed_index_name, separator=@date_separator)
|
69
|
+
expiry = (cutoff_date_beginning_of_day - index_time).to_i
|
70
|
+
if expiry > 0
|
71
|
+
@expired_indices[index_name] = expiry
|
72
|
+
end
|
36
73
|
end
|
37
74
|
end
|
38
|
-
expired_indices
|
39
75
|
end
|
40
76
|
|
41
77
|
def index_closed?(index_name)
|
@@ -43,16 +79,69 @@ module Elasticshelf
|
|
43
79
|
index_metadata['metadata']['indices'][index_name]['state'] == 'close'
|
44
80
|
end
|
45
81
|
|
82
|
+
def open_index(index_name)
|
83
|
+
try do
|
84
|
+
@results[index_name] = @client.indices.open(index: index_name)
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
88
|
+
def close_index(index_name)
|
89
|
+
try do
|
90
|
+
@indices_closed[index_name] = @client.indices.close(index: index_name)
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
def close_expired_indices
|
95
|
+
find_expired_indices if blank? @expired_indices
|
96
|
+
@expired_indices.each do |k,v|
|
97
|
+
close_index(k)
|
98
|
+
puts "closed index '#{k}' expired #{v} days ago"
|
99
|
+
end
|
100
|
+
end
|
101
|
+
|
102
|
+
def delete_index(index_name)
|
103
|
+
if ['_all','*'].include? index_name.downcase
|
104
|
+
@errors = {"error" => "deleting all indices is not allowed"}
|
105
|
+
return
|
106
|
+
end
|
107
|
+
try do
|
108
|
+
@indices_deleted[index_name] = @client.indices.delete(index: index_name)
|
109
|
+
end
|
110
|
+
end
|
111
|
+
|
112
|
+
def delete_expired_indices
|
113
|
+
reset_errors_results
|
114
|
+
find_expired_indices if blank? @expired_indices
|
115
|
+
@expired_indices.each do |k,v|
|
116
|
+
delete_index(k)
|
117
|
+
puts "deleted '#{k}' expired #{v} days ago"
|
118
|
+
end
|
119
|
+
end
|
120
|
+
|
121
|
+
def get_settings_index(index_name)
|
122
|
+
reset_errors_results
|
123
|
+
@results[index_name] = @client.indices.get_settings(:index => index_name)
|
124
|
+
end
|
125
|
+
|
46
126
|
def wither_index(index_name)
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
127
|
+
try do
|
128
|
+
# don't try to disable bloom filter on a closed index, it will re-open it:
|
129
|
+
if index_closed?(index_name)
|
130
|
+
@errors = {"failed" => "Unable to wither (disable bloom filter) index #{index_name}: It's already closed."}
|
131
|
+
return
|
132
|
+
else
|
133
|
+
@indices_withered[index_name] = @client.indices.put_settings(:index => index_name, :body => 'index.codec.bloom.load=false')
|
134
|
+
end
|
135
|
+
end
|
136
|
+
end
|
137
|
+
|
138
|
+
def wither_expired_indices
|
139
|
+
reset_errors_results
|
140
|
+
find_expired_indices if blank? @expired_indices
|
141
|
+
@expired_indices.each do |k,v|
|
142
|
+
wither_index(k)
|
143
|
+
puts "withered '#{k}' expired #{v} days ago"
|
54
144
|
end
|
55
|
-
result
|
56
145
|
end
|
57
146
|
|
58
147
|
def get_index_state(index_name, metric='metadata')
|
@@ -65,38 +154,231 @@ module Elasticshelf
|
|
65
154
|
index_metadata
|
66
155
|
end
|
67
156
|
|
157
|
+
def snapshot_create_repository
|
158
|
+
@results = @errors = {}
|
159
|
+
# ensure the location is accessible by Elasticsearch
|
160
|
+
@repo_type = 'fs' if blank? @repo_type
|
161
|
+
@repo_location = '/var/lib/elasticsearch_snapshots' if blank? @repo_location
|
162
|
+
@repo_compressed = true if blank? @repo_compressed
|
163
|
+
try do
|
164
|
+
@results = @client.snapshot.create_repository(
|
165
|
+
repository: @repo,
|
166
|
+
body: {
|
167
|
+
type: @repo_type,
|
168
|
+
settings: {
|
169
|
+
location: @repo_location,
|
170
|
+
compress: @repo_compressed
|
171
|
+
}
|
172
|
+
}
|
173
|
+
)
|
174
|
+
end
|
175
|
+
end
|
176
|
+
|
177
|
+
def snapshot_get_repository
|
178
|
+
@results = @errors = {}
|
179
|
+
try do
|
180
|
+
if @repo
|
181
|
+
@results = @client.snapshot.get_repository(repository: @repo)
|
182
|
+
else
|
183
|
+
@results = @client.snapshot.get_repository
|
184
|
+
end
|
185
|
+
end
|
186
|
+
end
|
187
|
+
|
188
|
+
def snapshot_delete_repository
|
189
|
+
@results = @errors = {}
|
190
|
+
try do
|
191
|
+
@results = @client.snapshot.delete_repository(repository: @repo)
|
192
|
+
end
|
193
|
+
end
|
194
|
+
|
195
|
+
def snapshot_expired_indices
|
196
|
+
@results = @errors = {}
|
197
|
+
find_expired_indices
|
198
|
+
if blank? @expired_indices
|
199
|
+
@results = {"result" => "no expired indices found"}
|
200
|
+
return
|
201
|
+
end
|
202
|
+
open_indices = []
|
203
|
+
@expired_indices.keys.each do |index_name|
|
204
|
+
# only open indices can be used in a snapshot:
|
205
|
+
if index_closed?(index_name)
|
206
|
+
@indices_rejected[index_name] = "closed index can't snapshot"
|
207
|
+
else
|
208
|
+
@indices_affected[index_name] = "open can snapshot"
|
209
|
+
open_indices << index_name
|
210
|
+
end
|
211
|
+
end
|
212
|
+
if blank? open_indices
|
213
|
+
@results = {"result" => "none of the expired indices are open so can't snapshot"}
|
214
|
+
return
|
215
|
+
end
|
216
|
+
@indices = open_indices.join(',')
|
217
|
+
# allow user to override this name:
|
218
|
+
if blank? @snapshot_expired_indices_name
|
219
|
+
@snapshot = (Time.now.utc.to_s.gsub(' ', '_') +
|
220
|
+
"_#{@indices_prefix.chomp('*')}" +
|
221
|
+
"_cutoff_#{@cutoff_days}_days").downcase
|
222
|
+
else
|
223
|
+
@snapshot = @snapshot_expired_indices_name
|
224
|
+
end
|
225
|
+
@results = snapshot_create
|
226
|
+
# {"snapshot"=>{
|
227
|
+
# "snapshot"=>"2014-04-02_19:23:35_utc_logstash-_cutoff_22_days",
|
228
|
+
# "indices"=>[],
|
229
|
+
# "state"=>"SUCCESS",
|
230
|
+
# "start_time"=>"2014-04-02T19:23:35.715Z",
|
231
|
+
# "start_time_in_millis"=>1396466615715,
|
232
|
+
# "end_time"=>"2014-04-02T19:23:35.721Z",
|
233
|
+
# "end_time_in_millis"=>1396466615721,
|
234
|
+
# "duration_in_millis"=>6,
|
235
|
+
# "failures"=>[],
|
236
|
+
# "shards"=>{"total"=>0, "failed"=>0, "successful"=>0}
|
237
|
+
# }}
|
238
|
+
# @results['snapshot']['state'].downcase == 'success'
|
239
|
+
# @results['snapshot']['failures']
|
240
|
+
end
|
241
|
+
|
242
|
+
def snapshot_create
|
243
|
+
# naming errors as seen in elasticsearch log:
|
244
|
+
# Invalid snapshot name [2014-04-01 20:12:07 UTC], must not contain whitespace
|
245
|
+
# Invalid snapshot name [2014-04-01_20:12:38_UTC], must be lowercase
|
246
|
+
try do
|
247
|
+
if blank? @indices
|
248
|
+
# the whole cluster:
|
249
|
+
@results = @client.snapshot.create(repository: @repo,
|
250
|
+
snapshot: @snapshot,
|
251
|
+
wait_for_completion: @wait_for_completion,
|
252
|
+
body: {
|
253
|
+
ignore_unavailable: @ignore_unavailable,
|
254
|
+
include_global_state: @include_global_state
|
255
|
+
}
|
256
|
+
)
|
257
|
+
else
|
258
|
+
@results = @client.snapshot.create(repository: @repo,
|
259
|
+
snapshot: @snapshot,
|
260
|
+
wait_for_completion: @wait_for_completion,
|
261
|
+
body: {
|
262
|
+
indices: @indices,
|
263
|
+
ignore_unavailable: @ignore_unavailable,
|
264
|
+
include_global_state: @include_global_state
|
265
|
+
}
|
266
|
+
)
|
267
|
+
end
|
268
|
+
end
|
269
|
+
end
|
270
|
+
|
271
|
+
def snapshot_get
|
272
|
+
reset_errors_results
|
273
|
+
try do
|
274
|
+
if blank? @snapshot
|
275
|
+
@results = @client.snapshot.get(repository: @repo, snapshot: '_all')
|
276
|
+
else
|
277
|
+
@results = @client.snapshot.get(repository: @repo, snapshot: @snapshot)
|
278
|
+
end
|
279
|
+
end
|
280
|
+
end
|
281
|
+
|
282
|
+
def snapshot_delete
|
283
|
+
try do
|
284
|
+
@results = @client.snapshot.delete(repository: @repo, snapshot: @snapshot)
|
285
|
+
end
|
286
|
+
end
|
287
|
+
|
288
|
+
def snapshot_restore
|
289
|
+
if blank? @indices
|
290
|
+
@results = {"error" => "indices must be specified to restore from a snapshot"}
|
291
|
+
return
|
292
|
+
end
|
293
|
+
try do
|
294
|
+
# @example Restore from the `snapshot-1` snapshot
|
295
|
+
# client.snapshot.restore repository: 'my-backups', snapshot: 'snapshot-1'
|
296
|
+
# @example Restore a specific index under a different name
|
297
|
+
# client.snapshot.restore repository: 'my-backups',
|
298
|
+
# snapshot: 'snapshot-5',
|
299
|
+
# body: {
|
300
|
+
# rename_pattern: "^(.*)$",
|
301
|
+
# rename_replacement: "restored_$1"
|
302
|
+
# }
|
303
|
+
@results = @client.snapshot.restore(
|
304
|
+
repository: @repo,
|
305
|
+
snapshot: @snapshot,
|
306
|
+
wait_for_completion: @wait_for_completion,
|
307
|
+
body: {
|
308
|
+
indices: @indices,
|
309
|
+
ignore_unavailable: true,
|
310
|
+
include_global_state: false
|
311
|
+
}
|
312
|
+
)
|
313
|
+
end
|
314
|
+
end
|
315
|
+
|
68
316
|
def get_info
|
69
|
-
|
317
|
+
try do
|
318
|
+
@client.info
|
319
|
+
end
|
70
320
|
end
|
71
321
|
|
72
322
|
def get_elasticsearch_version
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
323
|
+
try do
|
324
|
+
info_hash = @client.info
|
325
|
+
return nil if blank? info_hash
|
326
|
+
return nil unless info_hash['status'] == 200
|
327
|
+
return nil if blank? info_hash['version']
|
328
|
+
info_hash['version']['number']
|
329
|
+
end
|
79
330
|
end
|
80
331
|
|
81
332
|
def get_lucene_version
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
|
333
|
+
try do
|
334
|
+
info_hash = @client.info
|
335
|
+
return nil if blank? info_hash
|
336
|
+
return nil unless info_hash['status'] == 200
|
337
|
+
return nil if blank? info_hash['version']
|
338
|
+
info_hash['version']['lucene_version']
|
339
|
+
end
|
88
340
|
end
|
89
341
|
|
90
342
|
def version
|
91
343
|
Elasticshelf::VERSION
|
92
344
|
end
|
93
345
|
|
346
|
+
def cutoff_days_as_date
|
347
|
+
return {"error" => "cutoff_days must be zero or greater"} if @cutoff_days.to_i < 0
|
348
|
+
cutoff_date = DateTime.now.new_offset(0) - @cutoff_days
|
349
|
+
set_time_to_beginning_of_day(cutoff_date.strftime("%Y.%m.%d"))
|
350
|
+
end
|
351
|
+
|
352
|
+
def cutoff_days_as_date_to_s
|
353
|
+
cdad = cutoff_days_as_date
|
354
|
+
cdad = cdad.strftime("%Y.%m.%d") if cdad.is_a? DateTime
|
355
|
+
cdad
|
356
|
+
end
|
357
|
+
|
94
358
|
private
|
95
359
|
|
360
|
+
def try(&block)
|
361
|
+
yield
|
362
|
+
rescue Exception => ex
|
363
|
+
# a consistent response for all exceptions similar to Elasticsearch:
|
364
|
+
@errors = {"error" => "#{ex.class}: #{ex.message}"}
|
365
|
+
# raise ex
|
366
|
+
end
|
367
|
+
|
96
368
|
def set_time_to_beginning_of_day(index_timestamp, separator='.')
|
97
369
|
return DateTime.strptime(index_timestamp, ['%Y', '%m', '%d'].join(separator))
|
98
370
|
end
|
99
371
|
|
372
|
+
def blank?(some_object)
|
373
|
+
some_object = some_object.strip if some_object.is_a? String
|
374
|
+
some_object.respond_to?(:empty?) ? some_object.empty? : !some_object
|
375
|
+
end
|
376
|
+
|
377
|
+
def reset_errors_results
|
378
|
+
@results = {}
|
379
|
+
@errors = {}
|
380
|
+
end
|
381
|
+
|
100
382
|
end
|
101
383
|
|
102
384
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: elasticshelf
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- cLee Smith
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-03
|
11
|
+
date: 2014-04-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: elasticsearch
|
@@ -24,20 +24,6 @@ dependencies:
|
|
24
24
|
- - '='
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: 1.0.1
|
27
|
-
- !ruby/object:Gem::Dependency
|
28
|
-
name: time_diff
|
29
|
-
requirement: !ruby/object:Gem::Requirement
|
30
|
-
requirements:
|
31
|
-
- - '='
|
32
|
-
- !ruby/object:Gem::Version
|
33
|
-
version: 0.3.0
|
34
|
-
type: :runtime
|
35
|
-
prerelease: false
|
36
|
-
version_requirements: !ruby/object:Gem::Requirement
|
37
|
-
requirements:
|
38
|
-
- - '='
|
39
|
-
- !ruby/object:Gem::Version
|
40
|
-
version: 0.3.0
|
41
27
|
- !ruby/object:Gem::Dependency
|
42
28
|
name: bundler
|
43
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -66,8 +52,8 @@ dependencies:
|
|
66
52
|
- - ">="
|
67
53
|
- !ruby/object:Gem::Version
|
68
54
|
version: '0'
|
69
|
-
description: Ruby integrations for Elasticsearch indices management using close,
|
70
|
-
|
55
|
+
description: Ruby integrations for Elasticsearch indices management using close, delete,
|
56
|
+
snapshot, and restore
|
71
57
|
email:
|
72
58
|
- cleesmith2006@gmail.com
|
73
59
|
executables: []
|
@@ -80,6 +66,7 @@ files:
|
|
80
66
|
- README.md
|
81
67
|
- Rakefile
|
82
68
|
- elasticshelf.gemspec
|
69
|
+
- examples/wither_older_than_days.rb
|
83
70
|
- lib/elasticshelf.rb
|
84
71
|
- lib/elasticshelf/version.rb
|
85
72
|
homepage: https://github.com/cleesmith/elasticshelf.git
|