logstash-filter-jsonvalidate 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d09954895846250eb1688c31f45bc1d651616421
+  data.tar.gz: cb863677b74a16ef1c69bcf344e45df0d82dfa39
+SHA512:
+  metadata.gz: 0d27605fca5df239bdfcad5f10a652145e66b12272b92c42ddd3a0f1dcdd76fbe02001659130bee9f822c6c3b032b31cb0e3496151f9f409c90c50564a443e8b
+  data.tar.gz: 26e24e2a7818bb06ccbf8eb3d8e3abfba58de9f316c6019c322d785431276ce07f8acad86cf4d4ffdc9b523996f977acc458541e3447e7722a54e49a8b6f7b15

data/CHANGELOG.md

@@ -0,0 +1,3 @@
+## 0.0.1
+ - initial create
+

data/CONTRIBUTORS

@@ -0,0 +1,11 @@
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped this logstash plugin along its way.
+
+Contributors:
+* Patrick Christopher (coffeepac)
+
+Note: If you've sent us patches, bug reports, or otherwise contributed to
+this Logstash plugin, and you aren't on the list above and want to be, please let 
+us know and we'll make sure you're here. Contributions from folks like you are 
+what make
+open source awesome.

data/DEVELOPER.md

@@ -0,0 +1,3 @@
+# logstash-filter-jsonvalidate
+test if a specified field is json.
+set a tag if it is not

data/Gemfile

@@ -0,0 +1,2 @@
+source 'https://rubygems.org'
+gemspec

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2012–2015 Elasticsearch <http://www.elastic.co>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/NOTICE.TXT

@@ -0,0 +1,5 @@
+Elasticsearch
+Copyright 2012-2015 Elasticsearch
+
+This product includes software developed by The Apache Software
+Foundation (http://www.apache.org/).

data/README.md

@@ -0,0 +1,83 @@
+# Logstash Plugin
+
+This is a plugin for [Logstash](https://github.com/elastic/logstash).
+
+It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
+
+## Documentation
+
+This plugin will check if the specified set of fields are valid json.  If the fields are valid json, no action is taken.  If any of the fields
+is not valid json the tag 'not_json' is added to the tag list.  This value can by overriden in the plugin configuration.
+## Need Help?
+
+Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
+
+## Developing
+
+### 1. Plugin Developement and Testing
+
+#### Code
+- To get started, you'll need JRuby with the Bundler gem installed.
+
+- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
+
+- Install dependencies
+```sh
+bundle install
+```
+
+#### Test
+
+- Update your dependencies
+
+```sh
+bundle install
+```
+
+- Run tests
+
+```sh
+bundle exec rspec
+```
+
+### 2. Running your unpublished Plugin in Logstash
+
+#### 2.1 Run in a local Logstash clone
+
+- Edit Logstash `Gemfile` and add the local plugin path, for example:
+```ruby
+gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
+```
+- Install plugin
+```sh
+bin/plugin install --no-verify
+```
+- Run Logstash with your plugin
+```sh
+bin/logstash -e 'filter {awesome {}}'
+```
+At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
+
+#### 2.2 Run in an installed Logstash
+
+You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
+
+- Build your plugin gem
+```sh
+gem build logstash-filter-awesome.gemspec
+```
+- Install the plugin from the Logstash home
+```sh
+bin/plugin install /your/local/plugin/logstash-filter-awesome.gem
+```
+- Start Logstash and proceed to test the plugin
+
+## Contributing
+
+All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
+
+Programming is not a required skill. Whatever you've seen about open source and maintainers or community members  saying "send patches or die" - you will not see that here.
+
+It is more important to the community that you are able to contribute.
+
+For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.

data/lib/logstash/filters/jsonvalidate.rb

@@ -0,0 +1,60 @@
+# encoding: utf-8
+require "logstash/filters/base"
+require "logstash/namespace"
+require "json"
+
+# This filter requires a set of intput fields that will
+# be joined in csv format and saved into the specified
+# csv output field
+#
+class LogStash::Filters::Jsonvalidate < LogStash::Filters::Base
+
+  # Setting the config_name here is required. This is how you
+  # configure this filter from your Logstash config.
+  #
+  # filter {
+  #   jsonvalidate {
+  #     test_fields => ['call', 'response']
+  #     failed_test_tag => 'not_json'
+  #   }
+  # }
+  #
+  config_name "jsonvalidate"
+  
+  # Fields to use as source for map values
+  config :test_fields, :validate => :array
+
+  # tag to add to tag list if not all specified fields are json
+  config :failed_test_tag, :validate => :string, :default => "not_json"
+
+  public
+  def register
+    # Add instance variables 
+  end # def register
+
+  public
+  def filter(event)
+    @logger.debug? and @logger.debug("Running jsonvalidate filter", :event => event)
+
+    @test_fields.each do |field| 
+      if !valid_json?(event[field])
+        event["tags"] << @failed_test_tag unless event["tags"].include?(@failed_test_tag)
+      end
+    end
+
+    filter_matched(event)
+    @logger.debug? and @logger.debug("Event now: ", :event => event)
+
+  end # def filter
+
+  private
+  def valid_json?(json)
+    begin 
+      JSON.parse(json)
+      return true
+    rescue JSON::ParserError => e
+      @logger.debug? and @logger.debug("Exeception: ", :exception => e)
+      return false
+    end
+  end
+end # class LogStash::Filters::Example

data/logstash-filter-jsonvalidate.gemspec

@@ -0,0 +1,23 @@
+Gem::Specification.new do |s|
+  s.name = 'logstash-filter-jsonvalidate'
+  s.version         = '0.0.1'
+  s.licenses = ['Apache License (2.0)']
+  s.summary = "This filter will test if a specified field is json.  If not a tag will be set"
+  s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
+  s.authors = ["Patrick Christopher"]
+  s.email = 'coffeepac@gmail.com'
+  s.homepage = "http://www.elastic.co/guide/en/logstash/current/index.html"
+  s.require_paths = ["lib"]
+
+  # Files
+  s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
+   # Tests
+  s.test_files = s.files.grep(%r{^(test|spec|features)/})
+
+  # Special flag to let us know this is actually a logstash plugin
+  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
+
+  # Gem dependencies
+  s.add_runtime_dependency "logstash-core", ">= 2.0.0", "< 3.0.0"
+  s.add_development_dependency 'logstash-devutils'
+end

data/spec/filters/jsonvalidate_spec.rb

@@ -0,0 +1,37 @@
+# encoding: utf-8
+require 'spec_helper'
+require "logstash/filters/jsonvalidate"
+
+describe LogStash::Filters::Jsonvalidate do
+  describe "Valid Json" do
+    let(:config) do <<-CONFIG
+      filter {
+        jsonvalidate {
+          test_fields => ['call', 'response']
+        }
+      }
+    CONFIG
+    end
+
+    sample("call" => "{\"Marco\":\"polo\"}", "response" => "{\"herber\":\"hoover\"}", "tags" =>[]) do
+      expect(subject).to include("tags")
+      expect(subject['tags']).to eq([])
+    end
+  end
+
+  describe "Invalid Json" do
+    let(:config) do <<-CONFIG
+      filter {
+        jsonvalidate {
+          test_fields => ['call', 'response']
+        }
+      }
+    CONFIG
+    end
+
+    sample("call" => "{\"marco\":\"polo\"}", "response" => "totes not json", "tags" => []) do
+      expect(subject).to include("tags")
+      expect(subject['tags']).to include("not_json")
+    end
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+# encoding: utf-8
+require "logstash/devutils/rspec/spec_helper"

metadata

@@ -0,0 +1,92 @@
+--- !ruby/object:Gem::Specification
+name: logstash-filter-jsonvalidate
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Patrick Christopher
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2016-06-30 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.0
+    - - <
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+  name: logstash-core
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.0
+    - - <
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: logstash-devutils
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program
+email: coffeepac@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- CONTRIBUTORS
+- DEVELOPER.md
+- Gemfile
+- LICENSE
+- NOTICE.TXT
+- README.md
+- lib/logstash/filters/jsonvalidate.rb
+- logstash-filter-jsonvalidate.gemspec
+- spec/filters/jsonvalidate_spec.rb
+- spec/spec_helper.rb
+homepage: http://www.elastic.co/guide/en/logstash/current/index.html
+licenses:
+- Apache License (2.0)
+metadata:
+  logstash_plugin: 'true'
+  logstash_group: filter
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project:
+rubygems_version: 2.4.8
+signing_key:
+specification_version: 4
+summary: This filter will test if a specified field is json.  If not a tag will be set
+test_files:
+- spec/filters/jsonvalidate_spec.rb
+- spec/spec_helper.rb