logstash-filter-jwt-decoder 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 86e9b95e13fe378117fbcfabf261b6dc40645f67fae4fafa3488f3ae6438d641
+  data.tar.gz: 78d5473c390c19d062c46c4331eb3aeb56344a49c6af7ffa895076804eda20a7
+SHA512:
+  metadata.gz: f8d11bd6c26da92df1c03f7e3214f75b8f29b37c04dda85711e6a29bd3445a57f16efd8323c0707c90c01545641dde3cb35a7b944f1b9d1e322a954923de9f36
+  data.tar.gz: 3151853dfb5ccfb669d96a6dc14d03c119ddb1c872decb51c5066303503b75964596e4312b886172cd3953bd44c6dd90d5cd2d037d8bd86a24976daab532e5c7

data/CHANGELOG.md

@@ -0,0 +1,7 @@
+## 3.0.2
+ - Docs: Add documentation template
+## 2.0.0
+ - Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully, 
+   instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
+ - Dependency on logstash-core update to 2.0
+

data/CONTRIBUTORS

@@ -0,0 +1,11 @@
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped logstash along its way.
+
+Contributors:
+* Aaron Mildenstein (untergeek)
+* Pier-Hugues Pellerin (ph)
+
+Note: If you've sent us patches, bug reports, or otherwise contributed to
+Logstash, and you aren't on the list above and want to be, please let us know
+and we'll make sure you're here. Contributions from folks like you are what make
+open source awesome.

data/DEVELOPER.md

@@ -0,0 +1,2 @@
+# logstash-filter-example
+Example filter plugin. This should help bootstrap your effort to write your own filter plugin!

data/Gemfile

@@ -0,0 +1,11 @@
+source 'https://rubygems.org'
+
+gemspec
+
+logstash_path = ENV["LOGSTASH_PATH"] || "../../logstash"
+use_logstash_source = ENV["LOGSTASH_SOURCE"] && ENV["LOGSTASH_SOURCE"].to_s == "1"
+
+if Dir.exist?(logstash_path) && use_logstash_source
+  gem 'logstash-core', :path => "#{logstash_path}/logstash-core"
+  gem 'logstash-core-plugin-api', :path => "#{logstash_path}/logstash-core-plugin-api"
+end

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2012-2018 Elasticsearch <http://www.elastic.co>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/NOTICE.TXT

@@ -0,0 +1,5 @@
+Elasticsearch
+Copyright 2012-2015 Elasticsearch
+
+This product includes software developed by The Apache Software
+Foundation (http://www.apache.org/).

data/README.md

@@ -0,0 +1,98 @@
+# Logstash Plugin
+
+[![Travis Build Status](https://travis-ci.org/iqueiroz/logstash-filter-jwt-decoder.svg)](https://travis-ci.org/iqueiroz/logstash-filter-jwt-decoder)
+
+This is a plugin for [Logstash](https://github.com/elastic/logstash).
+
+It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
+
+## Documentation
+
+Logstash provides infrastructure to automatically build documentation for this plugin. We provide a template file, index.asciidoc, where you can add documentation. The contents of this file will be converted into html and then placed with other plugin documentation in a [central location](http://www.elastic.co/guide/en/logstash/current/).
+
+- For formatting config examples, you can use the asciidoc `[source,json]` directive
+- For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide
+
+## Need Help?
+
+Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
+
+## Developing
+
+### 1. Plugin Developement and Testing
+
+#### Code
+- To get started, you'll need JRuby with the Bundler gem installed.
+
+- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
+
+- Install dependencies
+```sh
+bundle install
+```
+
+#### Test
+
+- Update your dependencies
+
+```sh
+bundle install
+```
+
+- Run tests
+
+```sh
+bundle exec rspec
+```
+
+### 2. Running your unpublished Plugin in Logstash
+
+#### 2.1 Run in a local Logstash clone
+
+- Edit Logstash `Gemfile` and add the local plugin path, for example:
+```ruby
+gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
+```
+- Install plugin
+```sh
+# Logstash 2.3 and higher
+bin/logstash-plugin install --no-verify
+
+# Prior to Logstash 2.3
+bin/plugin install --no-verify
+
+```
+- Run Logstash with your plugin
+```sh
+bin/logstash -e 'filter {awesome {}}'
+```
+At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
+
+#### 2.2 Run in an installed Logstash
+
+You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
+
+- Build your plugin gem
+```sh
+gem build logstash-filter-awesome.gemspec
+```
+- Install the plugin from the Logstash home
+```sh
+# Logstash 2.3 and higher
+bin/logstash-plugin install --no-verify
+
+# Prior to Logstash 2.3
+bin/plugin install --no-verify
+
+```
+- Start Logstash and proceed to test the plugin
+
+## Contributing
+
+All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
+
+Programming is not a required skill. Whatever you've seen about open source and maintainers or community members  saying "send patches or die" - you will not see that here.
+
+It is more important to the community that you are able to contribute.
+
+For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.

data/lib/logstash/filters/jwt_decoder.rb

@@ -0,0 +1,55 @@
+# encoding: utf-8
+require "logstash/filters/base"
+require "logstash/namespace"
+require "jwt"
+require "jsonpath"
+
+class LogStash::Filters::JwtDecoder < LogStash::Filters::Base
+
+  config_name "jwt_decoder"
+
+  # The pattern to match the token
+  config :token_pattern, :validate => :string, :default => /^[Bb]earer\s+(.+)$/
+
+  # The capture group index of the matched token
+  config :match_group_index, :validate => :number, :default => 0
+
+  # The path to the access token field
+  config :access_token_field, :validate => :string, :default => "message"
+
+  # The output field
+  config :output_field, :validate => :string, :default => "jwt_decoded"
+
+  # fields to be extracted
+  config :extract, :validate => :hash, :default => { "userId" => "$..[0].sub"}
+
+  public
+  def register
+    # Add instance variables
+  end # def register
+
+  public
+  def filter(event)
+      # Replace the event message with our message as configured in the
+      # config file.
+      raw_message = event.get(@access_token_field)
+      if(raw_message)
+        if match = raw_message.match(@token_pattern)
+          token = match.captures[@match_group_index]
+          decoded_token = JWT.decode token, nil, false
+
+          result = Hash.new
+
+          @extract.each do |key, value|
+            jsonPath = JsonPath.new(value)
+            result[key] = jsonPath.first(decoded_token)
+          end
+
+          event.set(@output_field, result)
+
+          filter_matched(event)
+        end
+      end
+  end # def filter
+end # class LogStash::Filters::JwtDecoder
+

data/logstash-filter-jwt-decoder.gemspec

@@ -0,0 +1,26 @@
+Gem::Specification.new do |s|
+  s.name = 'logstash-filter-jwt-decoder'
+  s.version         = '1.0.0'
+  s.licenses = ['Apache License (2.0)']
+  s.summary = "This plugin allows you to extract any data encoded in a JWT Token that you received in your log."
+  s.description     = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"
+  s.authors = ["Ivan de Queiroz"]
+  s.email = 'iqueiroz.go@gmail.com'
+  s.homepage = "https://github.com/iqueiroz/logstash-filter-jwt-decoder"
+  s.require_paths = ["lib"]
+
+  # Files
+  s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
+   # Tests
+  s.test_files = s.files.grep(%r{^(test|spec|features)/})
+
+  # Special flag to let us know this is actually a logstash plugin
+  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
+
+  # Gem dependencies
+  s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
+  s.add_runtime_dependency 'jwt', "~> 2.1"
+  s.add_runtime_dependency 'jsonpath', "~> 0.5"
+  s.add_development_dependency 'logstash-devutils'
+  
+end

data/spec/filters/jwt_decoder_spec.rb

@@ -0,0 +1,127 @@
+# encoding: utf-8
+require 'spec_helper'
+require "logstash/filters/jwt_decoder"
+
+describe LogStash::Filters::JwtDecoder do
+
+  let(:config) { { } }
+  let(:attrs) { { } }
+  let(:event) { LogStash::Event.new(attrs) }
+
+  # Sample JWT from https://jwt.io/
+  sampleJwt = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c"
+
+  describe "Empty configuration" do
+    let(:config) do <<-CONFIG
+      filter {
+        jwt_decoder {
+        }
+      }
+    CONFIG
+    end
+    
+    context "when jwt field exists" do
+      sample("message" => "Bearer #{sampleJwt}") do
+        expect(subject.get("jwt_decoded")).to eq({ "userId" => "1234567890" })
+      end
+    end
+
+    context "when jwt field does not exists" do
+      sample("somefield" => "somevalue") do
+        expect(subject.get("jwt_decoded")).to be_nil
+      end
+    end
+  end
+
+
+  describe "With user defined output field" do
+    let(:config) do <<-CONFIG
+      filter {
+        jwt_decoder {
+          output_field => "customfield"
+        }
+      }
+    CONFIG
+    end
+    
+    context "when jwt field exists" do
+      sample("message" => "Bearer #{sampleJwt}") do
+        expect(subject.get("customfield")).to eq({ "userId" => "1234567890" })
+      end
+    end
+  end
+
+  describe "With user defined access token field" do
+    let(:config) do <<-CONFIG
+      filter {
+        jwt_decoder {
+          access_token_field => "accesstoken"
+        }
+      }
+    CONFIG
+    end
+    
+    context "should be able to decode the token" do
+      sample("accesstoken" => "Bearer #{sampleJwt}") do
+        expect(subject.get("jwt_decoded")).to eq({ "userId" => "1234567890" })
+      end
+    end
+  end
+
+  describe "With user defined access token pattern" do
+    let(:config) do <<-CONFIG
+      filter {
+        jwt_decoder {
+          token_pattern => "^CustomPrefix\s+(.+)$"
+        }
+      }
+    CONFIG
+    end
+    
+    context "should be able to decode the token" do
+      sample("message" => "CustomPrefix #{sampleJwt}") do
+        expect(subject.get("jwt_decoded")).to eq({ "userId" => "1234567890" })
+      end
+    end
+  end
+
+  describe "With user defined access token pattern and capture group" do
+    let(:config) do <<-CONFIG
+      filter {
+        jwt_decoder {
+          token_pattern => "^(CustomPrefix)(\s+)(.+)$"
+          match_group_index => 2
+        }
+      }
+    CONFIG
+    end
+    
+    context "should be able to decode the token" do
+      sample("message" => "CustomPrefix #{sampleJwt}") do
+        expect(subject.get("jwt_decoded")).to eq({ "userId" => "1234567890" })
+      end
+    end
+  end
+
+
+  describe "With multiple fields to be extract" do
+    let(:config) do <<-CONFIG
+      filter {
+        jwt_decoder {
+          extract => {
+            "userId" => "$..[0].sub"
+            "name" => "$..[0].name"
+          }
+        }
+      }
+    CONFIG
+    end
+    
+    context "should be able to decode the token" do
+      sample("message" => "Bearer #{sampleJwt}") do
+        expect(subject.get("jwt_decoded")).to eq({ "userId" => "1234567890", "name"=>"John Doe" })
+      end
+    end
+  end
+
+end

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+# encoding: utf-8
+require "logstash/devutils/rspec/spec_helper"

metadata

@@ -0,0 +1,117 @@
+--- !ruby/object:Gem::Specification
+name: logstash-filter-jwt-decoder
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Ivan de Queiroz
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2018-09-24 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  name: logstash-core-plugin-api
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.1'
+  name: jwt
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.1'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+  name: jsonpath
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: logstash-devutils
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: This gem is a Logstash plugin required to be installed on top of the
+  Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This
+  gem is not a stand-alone program
+email: iqueiroz.go@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- CONTRIBUTORS
+- DEVELOPER.md
+- Gemfile
+- LICENSE
+- NOTICE.TXT
+- README.md
+- lib/logstash/filters/jwt_decoder.rb
+- logstash-filter-jwt-decoder.gemspec
+- spec/filters/jwt_decoder_spec.rb
+- spec/spec_helper.rb
+homepage: https://github.com/iqueiroz/logstash-filter-jwt-decoder
+licenses:
+- Apache License (2.0)
+metadata:
+  logstash_plugin: 'true'
+  logstash_group: filter
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project:
+rubygems_version: 2.7.6
+signing_key:
+specification_version: 4
+summary: This plugin allows you to extract any data encoded in a JWT Token that you
+  received in your log.
+test_files:
+- spec/filters/jwt_decoder_spec.rb
+- spec/spec_helper.rb