logstash-filter-rediss 0.3.5 → 1.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2e73636b9648b878f1d962d5fe80287c306c9ca68cfb4611ffc18789712a36c9
4
- data.tar.gz: cc85150046052b65002092f9000997065056290d68159030cdd0be58ce48c93f
3
+ metadata.gz: 300e9f93677fcf02905c91573ab605abcee5760a600d30d6cb7dddf83ca655be
4
+ data.tar.gz: a1034196a4ddeb9b22c3d2ee2def4cb5145a7ef1864917964a74a8393098ffbe
5
5
  SHA512:
6
- metadata.gz: 85774259a88a2df9e31439a077d22c590b2a3d87533056637368d626a198dfefeacce1dd2dda1cc23f31ed85fb8b2b427f51ccec8ba38d3bb057074d25aac018
7
- data.tar.gz: c0ad14407e0736b3b768a01ae7cdbbcd293dddb3f4048023802c6dea3be459d446a81247c72cbb5e030fa79450d1586518c294fc0917ecc6afa12027a7becb83
6
+ metadata.gz: '0313399da64d527725b273f983c369a438815044776a54617476ec678d7058b18ded2c459a9e0372cccb0ceac353a2eec906bab7c1406bd0a7a46ede443d36aa'
7
+ data.tar.gz: ea8144cdf41d8fcfd8da4f87d849043d007ac7d783e8a2dd5877846e34849f56126d8a98bb6f335a454932a9a1d06d3181eaba456e7f52576755a4e57385e20d
@@ -1,19 +1,9 @@
1
- ## 0.3.2
2
- - Introduce redis command `sismember`
3
-
4
- ## 0.3.1
5
- - Prevent infinite loops for retries after errors or failed lock acquisitions
1
+ ## 1.0.2
2
+ - Support for GET, HGET, SET, HSET, SETEX, EXISTS, DEL, SADD, SMEMBERS, SISMEMBER and SCARD;
3
+ - Support SSL (redis or rediss protocol);
4
+ - Support TTL (when you uses SETEX);
5
+ - Introduce new control field `cmd_key_is_formatted` for declaring commands
6
+ to be resolved through - see %{foo} handling;
6
7
  - Introduce optional configuration options for controlling the count and
7
8
  the interval of retries - see :max_retries, :lock_retry_interval and
8
- :max_lock_retries
9
-
10
- ## 0.3.0
11
- - Support for GET, SET, EXISTS, DEL, SADD, SMEMBERS and SCARD
12
- - Introduce new control field `cmd_key_is_formatted` for declaring commands
13
- to be resolved through - see %{foo} handling
14
-
15
- ## 0.2.0
16
- - Support for HSET and HGET
17
-
18
- ## 0.1.0
19
- - Initial version; simple caching with redis
9
+ :max_lock_retries;
@@ -1,10 +1,4 @@
1
- The following is a list of people who have contributed ideas, code, bug
2
- reports, or in general have helped logstash-filter-cache-redis along its way.
3
-
4
1
  Contributors:
2
+ * Thiago Barradas (ThiagoBarradas)
3
+ * Bruno Sales (bruno-sales)
5
4
  * David Robakowski (drobakowski)
6
-
7
- Note: If you've sent us patches, bug reports, or otherwise contributed to
8
- logstash-filter-cache-redis, and you aren't on the list above and want to be,
9
- please let us know and we'll make sure you're here. Contributions from folks
10
- like you are what make open source awesome.
data/LICENSE CHANGED
@@ -1,6 +1,6 @@
1
1
  MIT License
2
2
 
3
- Copyright (c) 2016, Synlay Technologies UG & Co. KG.
3
+ Copyright (c) 2019, Thiago Barradas
4
4
 
5
5
  Permission is hereby granted, free of charge, to any person obtaining a copy
6
6
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -1,23 +1,61 @@
1
- # Logstash Filter Redis Cache Plugin
1
+ # Logstash Filter Rediss Plugin
2
2
 
3
- [![Build Status](https://travis-ci.org/synlay/logstash-filter-cache-redis.svg)](https://travis-ci.org/synlay/logstash-filter-cache-redis)
4
- [![Coverage Status](https://coveralls.io/repos/github/synlay/logstash-filter-cache-redis/badge.svg?branch=master)](https://coveralls.io/github/synlay/logstash-filter-cache-redis?branch=master)
5
- [![Gem Version](https://badge.fury.io/rb/logstash-filter-cache-redis.svg)](https://badge.fury.io/rb/logstash-filter-cache-redis)
6
- [![GitHub license](https://img.shields.io/github/license/synlay/logstash-filter-cache-redis.svg)](https://github.com/synlay/logstash-filter-cache-redis)
3
+ [![Build Status](https://travis-ci.org/thiagobarradas/logstash-filter-rediss.svg)](https://travis-ci.org/thiagobarradas/logstash-filter-rediss)
4
+ [![Gem Version](https://badge.fury.io/rb/logstash-filter-rediss.svg)](https://badge.fury.io/rb/logstash-filter-rediss)
5
+ [![GitHub license](https://img.shields.io/github/license/thiagobarradas/logstash-filter-rediss.svg)](https://github.com/thiagobarradas/logstash-filter-rediss)
7
6
 
8
7
  This is a plugin for [Logstash](https://github.com/elastic/logstash).
9
8
 
10
- It is fully free and fully open source. The license is MIT, see [LICENSE](http://github.com/synlay/logstash-filter-cache-redis/LICENSE) for further infos.
9
+ It is fully free and fully open source. The license is MIT, see [LICENSE](http://github.com/thiagobarradas/logstash-filter-rediss/LICENSE) for further infos.
11
10
 
12
11
  ## Documentation
13
12
 
14
- This filter will store and retrieve data from Redis data cache. The fields `source` and `target` are used alternatively as data in- or output fields, while the value of a defined command like `rpush` will look for the corresponding event and use that value as the key. The following example for instance will store data from the event `ProductEntity` under the key based upon the data from the event `ProductId`:
13
+ Actions allowed:
14
+ - `get` - Get cache with key in :get and set value in :target
15
+ - `set` - Set cache with key in :set and value from :source
16
+ - `setex` - Set cache with key in :setex, considering TTL from :ttl (in seconds) and value from :source
17
+ - `exists` - Checks if key :exists exists and save result in :target
18
+ - `del` - Deletes cache with key :del
19
+ - `llen` - Get length of a list with key :llen and save result in :target
20
+ - `rpush` - Appends a value :source in a list with key :rpush
21
+ - `rpushnx` - Appends a value :source in a list with key :rpush only with key not exists. This operation uses red lock;
22
+ - `hset` - Set hash with key in :hset, field from : field and value from :source
23
+ - `hget` - Get a value into :target from a hash field with :hget key
24
+ - `sadd` - Adds one or more members from :sadd to a set :source
25
+ - `sismember` - Determine if a :source value is a member of a set :sismember and save in :target
26
+ - `smembers` - Get all members from :smembers key and put in :target
27
+ - `scard` - Get number of members of set :scard and put in :target
28
+ - `rpop` - Removes and get last element from list :rpop and save in :target
29
+ - `lpop` - Removes and get first element from list :lpop and save in :target
30
+ - `lget` Get all elements from a list :lget and save in :target
31
+
32
+ Get Sample:
33
+
34
+ ```ruby
35
+ filter {
36
+ rediss {
37
+ host => "redis.company.com"
38
+ port => 6379
39
+ db => 0
40
+ password => "authtoken"
41
+ get => "[data][id]"
42
+ target => "[data][result]"
43
+ }
44
+ }
45
+ ```
46
+
47
+ Set with TTL Sample:
15
48
 
16
49
  ```ruby
17
50
  filter {
18
- cache_redis {
19
- rpush => "ProductId"
20
- source => "ProductEntity"
51
+ rediss {
52
+ host => "redis.company.com"
53
+ port => 6379
54
+ db => 0
55
+ password => "authtoken"
56
+ ttl => 300
57
+ setex => "[data][id]"
58
+ source => "[data][content]"
21
59
  }
22
60
  }
23
61
  ```
@@ -8,20 +8,19 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
8
8
  config_name "rediss"
9
9
 
10
10
  # The field to perform filter
11
- #
12
11
  config :source, :validate => :string, :default => "message"
13
12
 
14
13
  # The name of the container to put the result
15
- #
16
14
  config :target, :validate => :string, :default => "message"
17
15
 
16
+ # The field to use in hset
18
17
  config :field, :validate => :string
19
18
 
20
- # Expire time in seconds
19
+ # Expire time in seconds for setex
21
20
  config :ttl => :number
22
21
 
23
22
  # Informs if the connection is to be made with SSL or not
24
- config :ssl => :boolean
23
+ config :ssl, :validate => :boolean, :default => false
25
24
 
26
25
  # For now only working for rpushnx and llen!
27
26
  config :cmd_key_is_formatted, :validate => :boolean, :default => false
@@ -62,62 +61,83 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
62
61
  # Interval for retrying to acquire a lock
63
62
  config :lock_retry_interval, :validate => :number, :default => 1
64
63
 
64
+ # config :get, :validate => :boolean, :default => false
65
+ config :lock_timeout, :validate => :number, :default => 5000
66
+
65
67
  # Maximal count of retries to acquire a lock
66
68
  config :max_lock_retries, :validate => :number, :default => 3
67
69
 
70
+ ################# ACTIONS
71
+
72
+ # Sets the action. If has value, this action will be executed
73
+ # GET: get cache with key in :get and set value in :target
68
74
  config :get, :validate => :string
69
75
 
76
+ # Sets the action. If has value, this action will be executed
77
+ # SET: set cache with key in :set and value from :source
70
78
  config :set, :validate => :string
71
79
 
80
+ # Sets the action. If has value, this action will be executed
81
+ # SETEX: set cache with key in :setex, considering TTL from :ttl (in seconds) and value from :source
72
82
  config :setex, :validate => :string
73
83
 
84
+ # Sets the action. If has value, this action will be executed
85
+ # EXISTS: perform exists operation in :exists key and save result in :target
74
86
  config :exists, :validate => :string
75
87
 
88
+ # Sets the action. If has value, this action will be executed
89
+ # DEL: deletes cache with key :del
76
90
  config :del, :validate => :string
77
91
 
78
- # # Sets the action. If set to true, it will get the data from redis cache
79
- # config :get, :validate => :boolean, :default => false
92
+ # Sets the action. If has value, this action will be executed
93
+ # LLEN: get length of a list with key :llen and save result in :target
80
94
  config :llen, :validate => :string
81
95
 
82
- # # Sets the action. If set to true, it will get the data from redis cache
83
- # config :get, :validate => :boolean, :default => false
96
+ # Sets the action. If has value, this action will be executed
97
+ # RPUSH: aapend value :source in a list with key :rpush
84
98
  config :rpush, :validate => :string
85
99
 
86
- # # Sets the action. If set to true, it will get the data from redis cache
87
- # config :get, :validate => :boolean, :default => false
100
+ # Sets the action. If has value, this action will be executed
101
+ # RPUSHNX: aapend value :source in a list with key :rpush only with key not exists
102
+ # This operation uses red lock;
88
103
  config :rpushnx, :validate => :string
89
104
 
90
- # Sets the action. If set to true, it will get the data from redis cache
105
+ # Sets the action. If has value, this action will be executed
106
+ # HSET: set hash with key in :hset, field from : field and value from :source
91
107
  config :hset, :validate => :string
92
108
 
93
- # Sets the action. If set to true, it will get the data from redis cache
109
+ # Sets the action. If has value, this action will be executed
110
+ # HGET: get a value into :target from a hash field with :hget key
94
111
  config :hget, :validate => :string
95
112
 
113
+ # Sets the action. If has value, this action will be executed
114
+ # SADD: add one or more members from :sadd to a set :source
96
115
  config :sadd, :validate => :string
97
116
 
117
+ # Sets the action. If has value, this action will be executed
118
+ # SISMEMBER: Determine if a :source value is a member of a set :sismember and save in :target
98
119
  config :sismember, :validate => :string
99
120
 
121
+ # Sets the action. If has value, this action will be executed
122
+ # SMEMBERS: Get all members from :smembers key and put in :target
100
123
  config :smembers, :validate => :string
101
124
 
125
+ # Sets the action. If has value, this action will be executed
126
+ # SCARD: Get number of members of set :scard and put in :target
102
127
  config :scard, :validate => :string
103
128
 
104
- # config :get, :validate => :boolean, :default => false
105
- config :lock_timeout, :validate => :number, :default => 5000
106
-
107
- # # Sets the action. If set to true, it will get the data from redis cache
108
- # config :get, :validate => :boolean, :default => false
129
+ # Sets the action. If has value, this action will be executed
130
+ # RPOP: Remove and get last element from list :rpop and save in :target
109
131
  config :rpop, :validate => :string
110
132
 
111
- # # Sets the action. If set to true, it will get the data from redis cache
112
- # config :get, :validate => :boolean, :default => false
133
+ # Sets the action. If has value, this action will be executed
134
+ # LPOP: Remove and get first element from list :lpop and save in :target
113
135
  config :lpop, :validate => :string
114
136
 
115
- # # Sets the action. If set to true, it will get the data from redis cache
116
- # config :get, :validate => :boolean, :default => false
117
- # O(N)
137
+ # Sets the action. If has value, this action will be executed
138
+ # LGET: Get all elements from a list :lget and save in :target
118
139
  config :lget, :validate => :string
119
140
 
120
-
121
141
  public
122
142
  def register
123
143
  @redis = nil
@@ -128,7 +148,6 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
128
148
  @host_idx = 0
129
149
  end # def register
130
150
 
131
-
132
151
  def filter(event)
133
152
 
134
153
  # TODO: Maybe refactor the interface into a more flexible one with two
@@ -153,7 +172,7 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
153
172
  end
154
173
 
155
174
  if @exists
156
- @redis.exists(event.get(@exists))
175
+ event.set(@target, @redis.exists(event.get(@exists)))
157
176
  end
158
177
 
159
178
  if @del
@@ -177,7 +196,7 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
177
196
  end
178
197
 
179
198
  if @smembers
180
- @redis.smembers(event.get(@smembers))
199
+ event.set(@target, @redis.smembers(event.get(@smembers)))
181
200
  end
182
201
 
183
202
  if @scard
@@ -242,7 +261,6 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
242
261
  filter_matched(event)
243
262
  end # def filter
244
263
 
245
-
246
264
  private
247
265
  def connect
248
266
  @current_host, @current_port = @host[@host_idx].split(':')
@@ -260,7 +278,7 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
260
278
  :ssl => @ssl
261
279
  }
262
280
 
263
- @logger.debug("connection params", params)
281
+ @logger.debug("Connection params", params)
264
282
 
265
283
  if @password
266
284
  params[:password] = @password.value
@@ -279,6 +297,6 @@ class LogStash::Filters::Rediss < LogStash::Filters::Base
279
297
  @logger.debug("lock_manager hosts", hosts)
280
298
 
281
299
  Redlock::Client.new(hosts)
282
- end # def connect
300
+ end # def connect_lockmanager
283
301
 
284
- end # class LogStash::Filters::Example
302
+ end # class LogStash::Filters::Rediss
@@ -1,18 +1,19 @@
1
1
  Gem::Specification.new do |s|
2
- s.name = 'logstash-filter-rediss'
3
- s.version = '0.3.5'
2
+ s.name = 'logstash-filter-rediss'
3
+ s.version = '1.0.2'
4
4
  s.licenses = ['MIT']
5
- s.summary = "Redis Cache Filter for Logstash"
5
+ s.summary = "Redis (with SSL) Filter for Logstash"
6
6
  s.description = "A Logstash filter plugin for storing and retrieving data from redis cache. This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gem_name. This gem is not a stand-alone program."
7
- s.authors = ["David Robakowski"]
8
- s.email = 'david.robakowski@synlay.com'
9
- s.homepage = "https://github.com/bruno-sales/logstash-filter-cache-redis"
10
- s.require_paths = ["lib"]
11
- s.platform = Gem::Platform::JAVA if defined?(RUBY_ENGINE) && RUBY_ENGINE == 'jruby'
7
+ s.authors = ["Thiago Barradas", "Bruno Sales", "David Robakowski"]
8
+ s.email = 'th.barradas@gmail.com'
9
+ s.homepage = "https://github.com/thiagobarradas/logstash-filter-rediss"
10
+ s.require_paths = ["lib"]
11
+ s.platform = Gem::Platform::JAVA if defined?(RUBY_ENGINE) && RUBY_ENGINE == 'jruby'
12
12
 
13
13
  # Files
14
14
  s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE']
15
- # Tests
15
+
16
+ # Tests
16
17
  s.test_files = s.files.grep(%r{^(test|spec|features)/})
17
18
 
18
19
  # Special flag to let us know this is actually a logstash plugin
metadata CHANGED
@@ -1,9 +1,11 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-filter-rediss
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.5
4
+ version: 1.0.2
5
5
  platform: ruby
6
6
  authors:
7
+ - Thiago Barradas
8
+ - Bruno Sales
7
9
  - David Robakowski
8
10
  autorequire:
9
11
  bindir: bin
@@ -82,7 +84,7 @@ description: A Logstash filter plugin for storing and retrieving data from redis
82
84
  This gem is a Logstash plugin required to be installed on top of the Logstash core
83
85
  pipeline using $LS_HOME/bin/logstash-plugin install gem_name. This gem is not a
84
86
  stand-alone program.
85
- email: david.robakowski@synlay.com
87
+ email: th.barradas@gmail.com
86
88
  executables: []
87
89
  extensions: []
88
90
  extra_rdoc_files: []
@@ -94,7 +96,7 @@ files:
94
96
  - README.md
95
97
  - lib/logstash/filters/rediss.rb
96
98
  - logstash-filter-rediss.gemspec
97
- homepage: https://github.com/bruno-sales/logstash-filter-cache-redis
99
+ homepage: https://github.com/thiagobarradas/logstash-filter-rediss
98
100
  licenses:
99
101
  - MIT
100
102
  metadata:
@@ -119,5 +121,5 @@ rubyforge_project:
119
121
  rubygems_version: 2.7.7
120
122
  signing_key:
121
123
  specification_version: 4
122
- summary: Redis Cache Filter for Logstash
124
+ summary: Redis (with SSL) Filter for Logstash
123
125
  test_files: []