fluent-plugin-application-insights 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: f77dfcd3f36125b8dc0a3fd720db0c4303274c54
-  data.tar.gz: 3129c6bc54acac8cbfb71fe821e5e3675f35728a
+  metadata.gz: bfa9d92dcee9f67b521133f83b7381e1a73d6b35
+  data.tar.gz: c8a69e7f304425e4f915dd17e8ad979399333af9
 SHA512:
-  metadata.gz: da8c5a97877de1e48c507c586fbb352dfd7aa9a0e092edb4a9d883a1f07d861fa64e953c6761030bbb0a5b6ffa58230755194ade69443eb6b6798700dc3d4318
-  data.tar.gz: 49d85bf6fc2822ebf417ca85536d501be120f887e517dd3aff44fc1d454a6abe70042059db5010bfda1ae7a7711f3d6aeff3f2cd5a66d2fb30bd0174fd777bb7
+  metadata.gz: 6794875d0ae52ea3f9d97da680958902c20dd187471312d6332b889148c7c00f0c20102b5f6e75ba4532a0ab282e12c95bfb188410abc4de70fcb8816c3566a8
+  data.tar.gz: ed67f84b150d180cbb43e6301c93b9031a004f4515184203575f3678190f98967be764ba44ee0bef15e7e24a97ce1591b46ba5fcdac27035a9c20a80e4a02df4

data/CONTRIBUTING.md

@@ -0,0 +1,33 @@
+# Development
+## Build Gem
+Run ```gem build fluent-plugin-application-insights.gemspec```.
+
+## Run Test
+Make sure you have bundler installed, you can install it by ```sudo gem install bundler```. And run ```bundler install``` once to install all dependencies.
+
+Run ```rake test```.
+
+## Release
+If you are the current maintainer of this plugin:
+1. Ensure all tests passed
+2. Bump up version in ```fluent-plugin-application-insights.gemspec```
+3. Build the gem, install it locally and test it
+4. Create a PR with whatever changes needed before releasing, e.g., version bump up, documentation
+5. Tag and push: ```git tag vx.xx.xx; git push --tags```
+6. Create a github release with the pushed tag
+7. Push to rubygems.org: ```gem push fluent-plugin-application-insights-<version>.gem```
+
+# Contributing
+
+This project welcomes contributions and suggestions. Most contributions require you to
+agree to a Contributor License Agreement (CLA) declaring that you have the right to,
+and actually do, grant us the rights to use your contribution. For details, visit
+https://cla.microsoft.com.
+
+When you submit a pull request, a CLA-bot will automatically determine whether you need
+to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the
+instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
+
+This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
+For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
+or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

data/README.md

@@ -1,4 +1,8 @@
 # fluent-plugin-application-insights
+
+[![Gem Version](https://badge.fury.io/rb/fluent-plugin-application-insights.svg)](https://badge.fury.io/rb/fluent-plugin-application-insights)
+[![Build Status](https://travis-ci.org/Microsoft/fluent-plugin-application-insights.svg?branch=master)](https://travis-ci.org/Microsoft/fluent-plugin-application-insights)
+
 This is the [Fluentd](https://fluentd.org/) output plugin for [Azure Application Insights](https://docs.microsoft.com/azure/application-insights/)
 
 Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms.
@@ -30,18 +34,20 @@ Here is the configuration options for this plugin:
 * `standard_schema` - The parameter indicating whether the record is in standard schema. i.e., the format that is recognized by Application Insights backend (default `false`).
 If the record is not in standard schema, it will be tracked as Application Insights trace telemetry. Otherwise, the record is just forwarded to the backend. See [Standard Schema](#standard-schema) for more info.
 * `message_property` - The property name for the trace message (default `message`).
-* `time_property` - The property name for the timestamp (default `nil`). Fluentd input plugin will assign a timestamp for each emitted record, and this timestamp is used as the telemetry creation time by default. Set the `time_property` if you want to use the value of this property instead of the one assigned by the input plugin.
+* `time_property` - The property name for the timestamp (default `nil`).  
+    Fluentd input plugin will assign a timestamp for each emitted record, and this timestamp is used as the telemetry creation time by default. Set the `time_property` if you want to use the value of this property instead of the one assigned by the input plugin.
 * `severity_property` - The property name for severity level (default `severity`). If the severity property doesn't exist, the record will be treated as information level. See [Severity Level](https://docs.microsoft.com/azure/application-insights/application-insights-data-model-trace-telemetry#severity-level) for more info.
 * `severity_level_verbose` - The value of severity property that maps to Application Insights' verbose severity level (default `verbose`).
 * `severity_level_information` - The value of severity property that maps to Application Insights' information severity level (default `information`).
 * `severity_level_warning` - The value of severity property that maps to Application Insights' warning severity level (default `warning`).
 * `severity_level_error` - The value of severity property that maps to Application Insights' error severity level (default `error`).
 * `severity_level_critical` - The value of severity property that maps to Application Insights' critical severity level (default `critical`).
-* `context_tag_sources` - The dictionary that instructs the Application Insights plugin to set Application Insights context tags using record properties. In this dictionary keys are Application Insights context tags to set, and values are names of properties to use as source of data. For example:
+* `context_tag_sources` - The dictionary that instructs the Application Insights plugin to set Application Insights context tags using record properties.  
+    In this dictionary keys are Application Insights context tags to set, and values are the source properties that are used to set the context tags value. For the source property, you can specify the property name or jsonpath like syntax for nested property, see [record_accessor syntax](https://docs.fluentd.org/v1.0/articles/api-plugin-helper-record_accessor#syntax) for more info. For example:
     ```
     context_tag_sources {
       "ai.cloud.role": "kubernetes_container_name",
-      "ai.cloud.roleInstance": "kubernetes_pod_name"
+      "ai.cloud.roleInstance": "$.docker.container_id"
     }
     ```
     Here is the list of all [context tag keys](https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/Schema/PublicSchema/ContextTagKeys.bond)
@@ -74,36 +80,9 @@ Below is an example of a Request telemetry in standard schema format. `name`, `t
 }
 ```
 
-## Development
-### Build Gem
-Run ```gem build fluent-plugin-application-insights.gemspec```.
-
-### Run Test
-Make sure you have bundler installed, you can install it by ```sudo gem install bundler```. And run ```bundler install``` once to install all dependencies.
-
-Run ```rake test```.
-
-### Release
-If you are the current maintainer of this plugin:
-1. Ensure all tests passed
-2. Bump up version in ```fluent-plugin-application-insights.gemspec```
-3. Build the gem, install it locally and test it
-4. Create a PR with whatever changes needed before releasing, e.g., version bump up, documentation
-5. Tag and push: ```git tag vx.xx.xx; git push --tags```
-6. Create a github release with the pushed tag
-7. Push to rubygems.org: ```gem push fluent-plugin-application-insights-<version>.gem```
-
 ## Contributing
-
-This project welcomes contributions and suggestions. Most contributions require you to
-agree to a Contributor License Agreement (CLA) declaring that you have the right to,
-and actually do, grant us the rights to use your contribution. For details, visit
-https://cla.microsoft.com.
-
-When you submit a pull request, a CLA-bot will automatically determine whether you need
-to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the
-instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
+Refer to [Contributing Guide](CONTRIBUTING.md).
 
 This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
-For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
-or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
+For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
+contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

data/fluent-plugin-application-insights.gemspec

@@ -3,18 +3,13 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name    = "fluent-plugin-application-insights"
-  spec.version = "0.1.0"
+  spec.version = "0.2.0"
   spec.authors = ["Microsoft Corporation"]
   spec.email   = ["ctdiagcore@microsoft.com"]
 
   spec.summary       = "This is the fluentd output plugin for Azure Application Insights."
-  spec.description   = <<-eos
-  Fluentd output plugin for Azure Application Insights.
-  Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms.
-  Use it to monitor your live web application. It will automatically detect performance anomalies. It includes powerful analytics
-  tools to help you diagnose issues and to understand what users actually do with your app.
-  It's designed to help you continuously improve performance and usability.
-eos
+  spec.description   = "Fluentd output plugin for Azure Application Insights."
+
   spec.homepage      = "https://github.com/Microsoft/fluent-plugin-application-insights"
   spec.license       = "MIT"
 
@@ -29,6 +24,6 @@ eos
   spec.add_development_dependency "bundler", "~> 1.14"
   spec.add_development_dependency "rake", "~> 12.0"
   spec.add_development_dependency "test-unit", "~> 3.0"
-  spec.add_runtime_dependency "fluentd", [">= 0.14.10", "< 2"]
+  spec.add_runtime_dependency "fluentd", [">= 1.0", "< 2"]
   spec.add_runtime_dependency "application_insights", "~> 0.5.5"
 end

data/lib/fluent/plugin/out_application_insights.rb

@@ -11,6 +11,8 @@ module Fluent::Plugin
 
     attr_accessor :tc
 
+    helpers :record_accessor
+
     # The Application Insights instrumentation key
     config_param :instrumentation_key, :string
     # The batch size to send data to Application Insights service.
@@ -59,8 +61,11 @@ module Fluent::Plugin
       context_tag_keys.concat Channel::Contracts::Session.json_mappings.values
       context_tag_keys.concat Channel::Contracts::User.json_mappings.values
 
-      context_tag_sources.keys.each do |tag|
+      @context_tag_accessors = {}
+      context_tag_sources.each do |tag, property_path|
         raise ArgumentError.new("Context tag '#{tag}' is invalid!") unless context_tag_keys.include?(tag)
+
+        @context_tag_accessors[tag] = record_accessor_create(property_path)
       end
     end
 
@@ -136,9 +141,9 @@ module Fluent::Plugin
       return if @context_tag_sources.length == 0
 
       record["tags"] = record["tags"] || {}
-      @context_tag_sources.each do |context_tag, source_property|
-        context_value = record.delete source_property
-        record["tags"][context_tag] = context_value if context_value
+      @context_tag_accessors.each do |tag, accessor|
+        tag_value = accessor.call(record)
+        record["tags"][tag] = tag_value if !tag_value.nil?
       end
     end
 
@@ -164,16 +169,16 @@ module Fluent::Plugin
       context.instrumentation_key = @instrumentation_key
       return context if @context_tag_sources.length == 0
 
-      @context_tag_sources.each do |context_tag, source_property|
-        if record[source_property]
-          set_context_tag context, context_tag, record[source_property]
-        end
+      @context_tag_accessors.each do |tag, accessor|
+        set_context_tag context, tag, accessor.call(record)
       end
 
       return context
     end
 
     def set_context_tag(context, tag_name, tag_value)
+      return if tag_value.nil?
+
       context_set = [context.application, context.cloud, context.device, context.location, context.operation, context.session, context.user]
       context_set.each do |c|
         c.class.json_mappings.each do |attr, name|

data/test/plugin/test_out_application_insights.rb

@@ -153,13 +153,42 @@ class ApplicationInsightsOutputTest < Test::Unit::TestCase
       assert_nil envelope.data["baseData"]["properties"]
     end
 
+    test 'context tag source is nested property path' do
+      config = %[
+        instrumentation_key ikey
+        standard_schema true
+        context_tag_sources {
+          "ai.cloud.role": "$.kubernetes.container_name",
+          "ai.cloud.roleInstance": "$.kubernetes.not_exist"
+        }
+      ]
+      d = create_driver config
+
+      time = event_time("2011-01-02 13:14:15 UTC")
+      d.run(default_tag: 'test', shutdown: false) do
+        d.feed(time, {
+          "name" => "telemetry name",
+          "data" => { "baseType" => "RequestData", "baseData" => {} },
+          "kubernetes" => {
+            "container_name" => "frontend",
+            "container_id" => "c42c557e1615511dd3a3cb1d6e8f14984464bb0f"
+          }
+        })
+      end
+
+      envelope = d.instance.tc.channel.queue[0]
+      assert_not_nil envelope.tags
+      assert_equal "frontend", envelope.tags["ai.cloud.role"]
+      assert_nil envelope.tags["ai.cloud.roleInstance"]
+    end
+
     test 'multiple context tag keys' do
       config = %[
         instrumentation_key ikey
         standard_schema true
         context_tag_sources {
           "ai.cloud.role": "kubernetes_container_name",
-          "ai.cloud.roleInstance": "kubernetes_container_id"
+          "ai.cloud.roleInstance": "$.docker.container_id"
         }
       ]
       d = create_driver config
@@ -169,8 +198,10 @@ class ApplicationInsightsOutputTest < Test::Unit::TestCase
         d.feed(time, {
           "name" => "telemetry name",
           "data" => { "baseType" => "RequestData", "baseData" => {} },
-          "kubernetes_container_name" => "frontend",
-          "kubernetes_container_id" => "c42c557e1615511dd3a3cb1d6e8f14984464bb0f"
+          "docker" => {
+            "container_id" => "c42c557e1615511dd3a3cb1d6e8f14984464bb0f"
+          },
+          "kubernetes_container_name" => "frontend"
         })
       end
 
@@ -353,12 +384,40 @@ class ApplicationInsightsOutputTest < Test::Unit::TestCase
       assert_not_nil envelope.data.base_data.properties["other_prop"]
     end
 
+    test 'context tag source is nested property path' do
+      config = %[
+        instrumentation_key ikey
+        context_tag_sources {
+          "ai.cloud.role": "$.kubernetes.container_name",
+          "ai.cloud.roleInstance": "$.kubernetes.not_exist"
+        }
+      ]
+      d = create_driver config
+
+      time = event_time("2011-01-02 13:14:15 UTC")
+      d.run(default_tag: 'test', shutdown: false) do
+        d.feed(time, {
+          "name" => "telemetry name",
+          "data" => { "baseType" => "RequestData", "baseData" => {} },
+          "kubernetes" => {
+            "container_name" => "frontend",
+            "container_id" => "c42c557e1615511dd3a3cb1d6e8f14984464bb0f"
+          }
+        })
+      end
+
+      envelope = d.instance.tc.channel.queue[0]
+      assert_not_nil envelope.tags
+      assert_equal "frontend", envelope.tags["ai.cloud.role"]
+      assert_nil envelope.tags["ai.cloud.roleInstance"]
+    end
+
     test 'multiple context tag keys' do
       config = %[
         instrumentation_key ikey
         context_tag_sources {
           "ai.cloud.role": "kubernetes_container_name",
-          "ai.cloud.roleInstance": "kubernetes_container_id"
+          "ai.cloud.roleInstance": "$.docker.container_id"
         }
       ]
       d = create_driver config
@@ -367,8 +426,10 @@ class ApplicationInsightsOutputTest < Test::Unit::TestCase
       d.run(default_tag: 'test', shutdown: false) do
         d.feed(time, {
           "message" => "my message",
-          "kubernetes_container_name" => "frontend",
-          "kubernetes_container_id" => "c42c557e1615511dd3a3cb1d6e8f14984464bb0f"
+          "docker" => {
+            "container_id" => "c42c557e1615511dd3a3cb1d6e8f14984464bb0f"
+          },
+          "kubernetes_container_name" => "frontend"
         })
       end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-application-insights
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Microsoft Corporation
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-05-03 00:00:00.000000000 Z
+date: 2018-05-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -58,7 +58,7 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.14.10
+        version: '1.0'
     - - "<"
       - !ruby/object:Gem::Version
         version: '2'
@@ -68,7 +68,7 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.14.10
+        version: '1.0'
     - - "<"
       - !ruby/object:Gem::Version
         version: '2'
@@ -86,12 +86,7 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.5.5
-description: |2
-    Fluentd output plugin for Azure Application Insights.
-    Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms.
-    Use it to monitor your live web application. It will automatically detect performance anomalies. It includes powerful analytics
-    tools to help you diagnose issues and to understand what users actually do with your app.
-    It's designed to help you continuously improve performance and usability.
+description: Fluentd output plugin for Azure Application Insights.
 email:
 - ctdiagcore@microsoft.com
 executables: []
@@ -100,6 +95,7 @@ extra_rdoc_files: []
 files:
 - ".gitignore"
 - ".travis.yml"
+- CONTRIBUTING.md
 - Gemfile
 - LICENSE
 - README.md