logstash-filter-jsonvalidate 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: d09954895846250eb1688c31f45bc1d651616421
4
+ data.tar.gz: cb863677b74a16ef1c69bcf344e45df0d82dfa39
5
+ SHA512:
6
+ metadata.gz: 0d27605fca5df239bdfcad5f10a652145e66b12272b92c42ddd3a0f1dcdd76fbe02001659130bee9f822c6c3b032b31cb0e3496151f9f409c90c50564a443e8b
7
+ data.tar.gz: 26e24e2a7818bb06ccbf8eb3d8e3abfba58de9f316c6019c322d785431276ce07f8acad86cf4d4ffdc9b523996f977acc458541e3447e7722a54e49a8b6f7b15
data/CHANGELOG.md ADDED
@@ -0,0 +1,3 @@
1
+ ## 0.0.1
2
+ - initial create
3
+
data/CONTRIBUTORS ADDED
@@ -0,0 +1,11 @@
1
+ The following is a list of people who have contributed ideas, code, bug
2
+ reports, or in general have helped this logstash plugin along its way.
3
+
4
+ Contributors:
5
+ * Patrick Christopher (coffeepac)
6
+
7
+ Note: If you've sent us patches, bug reports, or otherwise contributed to
8
+ this Logstash plugin, and you aren't on the list above and want to be, please let
9
+ us know and we'll make sure you're here. Contributions from folks like you are
10
+ what make
11
+ open source awesome.
data/DEVELOPER.md ADDED
@@ -0,0 +1,3 @@
1
+ # logstash-filter-jsonvalidate
2
+ test if a specified field is json.
3
+ set a tag if it is not
data/Gemfile ADDED
@@ -0,0 +1,2 @@
1
+ source 'https://rubygems.org'
2
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,13 @@
1
+ Copyright (c) 2012–2015 Elasticsearch <http://www.elastic.co>
2
+
3
+ Licensed under the Apache License, Version 2.0 (the "License");
4
+ you may not use this file except in compliance with the License.
5
+ You may obtain a copy of the License at
6
+
7
+ http://www.apache.org/licenses/LICENSE-2.0
8
+
9
+ Unless required by applicable law or agreed to in writing, software
10
+ distributed under the License is distributed on an "AS IS" BASIS,
11
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ See the License for the specific language governing permissions and
13
+ limitations under the License.
data/NOTICE.TXT ADDED
@@ -0,0 +1,5 @@
1
+ Elasticsearch
2
+ Copyright 2012-2015 Elasticsearch
3
+
4
+ This product includes software developed by The Apache Software
5
+ Foundation (http://www.apache.org/).
data/README.md ADDED
@@ -0,0 +1,83 @@
1
+ # Logstash Plugin
2
+
3
+ This is a plugin for [Logstash](https://github.com/elastic/logstash).
4
+
5
+ It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
6
+
7
+ ## Documentation
8
+
9
+ This plugin will check if the specified set of fields are valid json. If the fields are valid json, no action is taken. If any of the fields
10
+ is not valid json the tag 'not_json' is added to the tag list. This value can by overriden in the plugin configuration.
11
+ ## Need Help?
12
+
13
+ Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
14
+
15
+ ## Developing
16
+
17
+ ### 1. Plugin Developement and Testing
18
+
19
+ #### Code
20
+ - To get started, you'll need JRuby with the Bundler gem installed.
21
+
22
+ - Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
23
+
24
+ - Install dependencies
25
+ ```sh
26
+ bundle install
27
+ ```
28
+
29
+ #### Test
30
+
31
+ - Update your dependencies
32
+
33
+ ```sh
34
+ bundle install
35
+ ```
36
+
37
+ - Run tests
38
+
39
+ ```sh
40
+ bundle exec rspec
41
+ ```
42
+
43
+ ### 2. Running your unpublished Plugin in Logstash
44
+
45
+ #### 2.1 Run in a local Logstash clone
46
+
47
+ - Edit Logstash `Gemfile` and add the local plugin path, for example:
48
+ ```ruby
49
+ gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
50
+ ```
51
+ - Install plugin
52
+ ```sh
53
+ bin/plugin install --no-verify
54
+ ```
55
+ - Run Logstash with your plugin
56
+ ```sh
57
+ bin/logstash -e 'filter {awesome {}}'
58
+ ```
59
+ At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
60
+
61
+ #### 2.2 Run in an installed Logstash
62
+
63
+ You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
64
+
65
+ - Build your plugin gem
66
+ ```sh
67
+ gem build logstash-filter-awesome.gemspec
68
+ ```
69
+ - Install the plugin from the Logstash home
70
+ ```sh
71
+ bin/plugin install /your/local/plugin/logstash-filter-awesome.gem
72
+ ```
73
+ - Start Logstash and proceed to test the plugin
74
+
75
+ ## Contributing
76
+
77
+ All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
78
+
79
+ Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
80
+
81
+ It is more important to the community that you are able to contribute.
82
+
83
+ For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
@@ -0,0 +1,60 @@
1
+ # encoding: utf-8
2
+ require "logstash/filters/base"
3
+ require "logstash/namespace"
4
+ require "json"
5
+
6
+ # This filter requires a set of intput fields that will
7
+ # be joined in csv format and saved into the specified
8
+ # csv output field
9
+ #
10
+ class LogStash::Filters::Jsonvalidate < LogStash::Filters::Base
11
+
12
+ # Setting the config_name here is required. This is how you
13
+ # configure this filter from your Logstash config.
14
+ #
15
+ # filter {
16
+ # jsonvalidate {
17
+ # test_fields => ['call', 'response']
18
+ # failed_test_tag => 'not_json'
19
+ # }
20
+ # }
21
+ #
22
+ config_name "jsonvalidate"
23
+
24
+ # Fields to use as source for map values
25
+ config :test_fields, :validate => :array
26
+
27
+ # tag to add to tag list if not all specified fields are json
28
+ config :failed_test_tag, :validate => :string, :default => "not_json"
29
+
30
+ public
31
+ def register
32
+ # Add instance variables
33
+ end # def register
34
+
35
+ public
36
+ def filter(event)
37
+ @logger.debug? and @logger.debug("Running jsonvalidate filter", :event => event)
38
+
39
+ @test_fields.each do |field|
40
+ if !valid_json?(event[field])
41
+ event["tags"] << @failed_test_tag unless event["tags"].include?(@failed_test_tag)
42
+ end
43
+ end
44
+
45
+ filter_matched(event)
46
+ @logger.debug? and @logger.debug("Event now: ", :event => event)
47
+
48
+ end # def filter
49
+
50
+ private
51
+ def valid_json?(json)
52
+ begin
53
+ JSON.parse(json)
54
+ return true
55
+ rescue JSON::ParserError => e
56
+ @logger.debug? and @logger.debug("Exeception: ", :exception => e)
57
+ return false
58
+ end
59
+ end
60
+ end # class LogStash::Filters::Example
@@ -0,0 +1,23 @@
1
+ Gem::Specification.new do |s|
2
+ s.name = 'logstash-filter-jsonvalidate'
3
+ s.version = '0.0.1'
4
+ s.licenses = ['Apache License (2.0)']
5
+ s.summary = "This filter will test if a specified field is json. If not a tag will be set"
6
+ s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
7
+ s.authors = ["Patrick Christopher"]
8
+ s.email = 'coffeepac@gmail.com'
9
+ s.homepage = "http://www.elastic.co/guide/en/logstash/current/index.html"
10
+ s.require_paths = ["lib"]
11
+
12
+ # Files
13
+ s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
14
+ # Tests
15
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
16
+
17
+ # Special flag to let us know this is actually a logstash plugin
18
+ s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
19
+
20
+ # Gem dependencies
21
+ s.add_runtime_dependency "logstash-core", ">= 2.0.0", "< 3.0.0"
22
+ s.add_development_dependency 'logstash-devutils'
23
+ end
@@ -0,0 +1,37 @@
1
+ # encoding: utf-8
2
+ require 'spec_helper'
3
+ require "logstash/filters/jsonvalidate"
4
+
5
+ describe LogStash::Filters::Jsonvalidate do
6
+ describe "Valid Json" do
7
+ let(:config) do <<-CONFIG
8
+ filter {
9
+ jsonvalidate {
10
+ test_fields => ['call', 'response']
11
+ }
12
+ }
13
+ CONFIG
14
+ end
15
+
16
+ sample("call" => "{\"Marco\":\"polo\"}", "response" => "{\"herber\":\"hoover\"}", "tags" =>[]) do
17
+ expect(subject).to include("tags")
18
+ expect(subject['tags']).to eq([])
19
+ end
20
+ end
21
+
22
+ describe "Invalid Json" do
23
+ let(:config) do <<-CONFIG
24
+ filter {
25
+ jsonvalidate {
26
+ test_fields => ['call', 'response']
27
+ }
28
+ }
29
+ CONFIG
30
+ end
31
+
32
+ sample("call" => "{\"marco\":\"polo\"}", "response" => "totes not json", "tags" => []) do
33
+ expect(subject).to include("tags")
34
+ expect(subject['tags']).to include("not_json")
35
+ end
36
+ end
37
+ end
@@ -0,0 +1,2 @@
1
+ # encoding: utf-8
2
+ require "logstash/devutils/rspec/spec_helper"
metadata ADDED
@@ -0,0 +1,92 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: logstash-filter-jsonvalidate
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ platform: ruby
6
+ authors:
7
+ - Patrick Christopher
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2016-06-30 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - '>='
17
+ - !ruby/object:Gem::Version
18
+ version: 2.0.0
19
+ - - <
20
+ - !ruby/object:Gem::Version
21
+ version: 3.0.0
22
+ name: logstash-core
23
+ prerelease: false
24
+ type: :runtime
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - '>='
28
+ - !ruby/object:Gem::Version
29
+ version: 2.0.0
30
+ - - <
31
+ - !ruby/object:Gem::Version
32
+ version: 3.0.0
33
+ - !ruby/object:Gem::Dependency
34
+ requirement: !ruby/object:Gem::Requirement
35
+ requirements:
36
+ - - '>='
37
+ - !ruby/object:Gem::Version
38
+ version: '0'
39
+ name: logstash-devutils
40
+ prerelease: false
41
+ type: :development
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - '>='
45
+ - !ruby/object:Gem::Version
46
+ version: '0'
47
+ description: This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program
48
+ email: coffeepac@gmail.com
49
+ executables: []
50
+ extensions: []
51
+ extra_rdoc_files: []
52
+ files:
53
+ - CHANGELOG.md
54
+ - CONTRIBUTORS
55
+ - DEVELOPER.md
56
+ - Gemfile
57
+ - LICENSE
58
+ - NOTICE.TXT
59
+ - README.md
60
+ - lib/logstash/filters/jsonvalidate.rb
61
+ - logstash-filter-jsonvalidate.gemspec
62
+ - spec/filters/jsonvalidate_spec.rb
63
+ - spec/spec_helper.rb
64
+ homepage: http://www.elastic.co/guide/en/logstash/current/index.html
65
+ licenses:
66
+ - Apache License (2.0)
67
+ metadata:
68
+ logstash_plugin: 'true'
69
+ logstash_group: filter
70
+ post_install_message:
71
+ rdoc_options: []
72
+ require_paths:
73
+ - lib
74
+ required_ruby_version: !ruby/object:Gem::Requirement
75
+ requirements:
76
+ - - '>='
77
+ - !ruby/object:Gem::Version
78
+ version: '0'
79
+ required_rubygems_version: !ruby/object:Gem::Requirement
80
+ requirements:
81
+ - - '>='
82
+ - !ruby/object:Gem::Version
83
+ version: '0'
84
+ requirements: []
85
+ rubyforge_project:
86
+ rubygems_version: 2.4.8
87
+ signing_key:
88
+ specification_version: 4
89
+ summary: This filter will test if a specified field is json. If not a tag will be set
90
+ test_files:
91
+ - spec/filters/jsonvalidate_spec.rb
92
+ - spec/spec_helper.rb