fluent-plugin-pi 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 27e54a4517d037bd4d94bfd82de9cad602b64762
+  data.tar.gz: b97da9892c3d3ff312832f8cb92b65a47f8eb147
+SHA512:
+  metadata.gz: 63b04f2f4715f67f5135f99bdd344e936634fa94464ab9ed9d0f9f647dea234b3b57af2e583227dc11d63b5f2308af708336b5e66ddfa681e7e19d446fbf06b4
+  data.tar.gz: 6930855b64fca1bc4287e4ebd8d051a732b6e7dcab934647503869a7a813cd92c9802c78361c6371cc982e966145b6cfebd7d4bf8bd7bdf2b7fdfd7af4be3ea8

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at dendres@osisoft.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in fluent-plugin-pi.gemspec
+gemspec

data/Gemfile.lock

@@ -0,0 +1,38 @@
+PATH
+  remote: .
+  specs:
+    fluent-plugin-pi (0.0.1)
+      adal (~> 1.0.0)
+      certified (~> 1.0.0)
+      rest-client (~> 1.6.9)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    adal (1.0.0)
+      jwt (~> 1.5)
+      nokogiri (~> 1.6)
+      uri_template (~> 0.7)
+    certified (1.0.0)
+    jwt (1.5.6)
+    mime-types (1.25.1)
+    mini_portile2 (2.1.0)
+    nokogiri (1.6.8-x86-mingw32)
+      mini_portile2 (~> 2.1.0)
+      pkg-config (~> 1.1.7)
+    pkg-config (1.1.7)
+    rake (10.4.2)
+    rest-client (1.6.9)
+      mime-types (~> 1.16)
+    uri_template (0.7.0)
+
+PLATFORMS
+  x86-mingw32
+
+DEPENDENCIES
+  bundler (~> 1.13)
+  fluent-plugin-pi!
+  rake (~> 10.0)
+
+BUNDLED WITH
+   1.13.2

data/README.md

@@ -0,0 +1,36 @@
+# Fluent::Plugin::Pi
+
+Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/fluent/plugin/pi`. To experiment with that code, run `bin/console` for an interactive prompt.
+
+TODO: Delete this and the text above, and describe your gem
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'fluent-plugin-pi'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install fluent-plugin-pi
+
+## Usage
+
+TODO: Write usage instructions here
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/fluent-plugin-pi. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+

data/Rakefile

@@ -0,0 +1,2 @@
+require "bundler/gem_tasks"
+task :default => :spec

data/fluent-plugin-pi.gemspec

@@ -0,0 +1,27 @@
+# coding: utf-8
+
+
+Gem::Specification.new do |spec|
+  spec.name          = "fluent-plugin-pi"
+  spec.version       = "0.0.1"
+  spec.authors       = ["Derek Endres"]
+  spec.email         = ["dendres@osisoft.com"]
+
+  spec.summary       = %q{Fluentd plugin to either get data from PI, send to PI or send to QI.}
+  spec.description   = %q{Fluentd plugin to either get data from PI, send to PI or send to QI}
+  spec.homepage      = "https://github.com/osisoftresearch"
+  spec.license       = "OSISoft LLC"
+
+  root_path      	 = File.dirname(__FILE__)
+  spec.files         =  Dir.chdir(root_path) { Dir.glob("**/{*,.*}") }
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "rest-client", "~> 1.6.9"
+  spec.add_dependency "adal", "~> 1.0.0"
+  spec.add_dependency "certified", "~> 1.0.0"
+  
+  spec.add_development_dependency "bundler", "~> 1.13"
+  spec.add_development_dependency "rake", "~> 10.0"
+end

data/lib/fluent/plugin/in_piin.rb

@@ -0,0 +1,101 @@
+#     Copyright (c) 2013-2016 OSIsoft, LLC. All rights reserved.
+# 
+#     THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS OF
+#     OSIsoft, LLC.  USE, DISCLOSURE, OR REPRODUCTION IS PROHIBITED WITHOUT
+#     THE PRIOR EXPRESS WRITTEN PERMISSION OF OSIsoft, LLC.
+# 
+#     RESTRICTED RIGHTS LEGEND
+#     Use, duplication, or disclosure by the Government is subject to restrictions
+#     as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and
+#     Computer Software clause at DFARS 252.227.7013
+# 
+#     OSIsoft, LLC
+#     777 Davis Street, Suite 250, San Leandro CA 94577
+
+require 'fluent/input'
+require 'rest-client'
+require 'json'
+require 'time'
+
+module Fluent
+  class Piin < Input
+    # First, register the plugin. NAME is the name of this plugin
+    # and identifies the plugin in the configuration file.
+    Fluent::Plugin.register_input('piin', self)
+
+    stop_flag = false
+    webid = nil
+    piwebapiurl = 'localhost', tagpath = nil 
+    tag = nil
+    regetrate = 10
+
+    # This method is called before starting.
+    # 'conf' is a Hash that includes configuration parameters.
+    # If the configuration is invalid, raise Fluent::ConfigError.
+    def configure(conf)
+      super
+      @piwebapiurl = conf['piwebapiurl']
+      (conf.key?('tagpath')) ? @tagpath = conf['tagpath'] : @tagpath = nil
+      (conf.key?('webid')) ? @webid = conf['webid'] : @webid = nil
+      (conf.key?('username')) ? @username = conf['username'] : @username = nil    
+      (conf.key?('pwd')) ? @pwd = conf['pwd'] : @pwd = nil
+      (conf.key?('anon')) ? @anon = conf['anon'] : @anon = false
+      (conf.key?('tag')) ? @tag = conf['tag'] : @tag = 'piin'
+      (conf.key?('regetrate')) ? @regetrate = conf['regetrate'] : @regetrate = 10
+    end
+
+    # This method is called when starting.
+    # Open sockets or files and create a thread here.
+    def start
+      super
+      
+      if(@webid == nil)
+        url = "https://#{@piwebapiurl}/piwebapi/points/?path=#{@tagpath}"
+        begin
+          client = RestClient::Resource.new(url, :verify_ssl => OpenSSL::SSL::VERIFY_NONE, :user => @username, :password =>@pwd)
+          response = client.get
+          @webid = (JSON.parse(response.body))['WebId']
+        rescue
+          log.warn "Errors. Web Id not found. Not able to bring in data from the specified address"
+          log.warn url
+          @webid = nil
+        end
+      end
+      
+      @stop_flag = false
+      @thread = Thread.new(&method(:thread_main))
+    end
+
+    # This method is called when shutting down.
+    # Shutdown the thread and close sockets or files here.
+    def shutdown    
+      super  
+      @stop_flag = true
+      $log.debug "Waiting for thread to finish"
+      @thread.join
+    end   
+
+    def thread_main
+      if(@webid == nil)
+        return
+      end
+      url = "https://#{@piwebapiurl}/piwebapi/streams/#{@webid}/value"
+      client = RestClient::Resource.new(url, :verify_ssl => OpenSSL::SSL::VERIFY_NONE,:user => @username, :password =>@pwd)
+      until @stop_flag
+        sleep @regetrate
+        begin
+            response = client.get :content_type => 'application/json'
+            responseObj = JSON.parse(response)
+            time = Time.parse(responseObj['Timestamp'])
+            if (time == nil)
+              time = Engine.now
+            end
+            router.emit(@tag, time, responseObj)
+        rescue => e
+            log.warn "Having problems connecting to PI WebAPI.  Will try again."
+        end
+      end      
+    end
+    
+  end
+end

data/lib/fluent/plugin/out_pi.rb

@@ -0,0 +1,150 @@
+#     Copyright (c) 2013-2016 OSIsoft, LLC. All rights reserved.
+# 
+#     THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS OF
+#     OSIsoft, LLC.  USE, DISCLOSURE, OR REPRODUCTION IS PROHIBITED WITHOUT
+#     THE PRIOR EXPRESS WRITTEN PERMISSION OF OSIsoft, LLC.
+# 
+#     RESTRICTED RIGHTS LEGEND
+#     Use, duplication, or disclosure by the Government is subject to restrictions
+#     as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and
+#     Computer Software clause at DFARS 252.227.7013
+# 
+#     OSIsoft, LLC
+#     777 Davis Street, Suite 250, San Leandro CA 94577
+
+require 'fluent/plugin/output'
+require 'json'
+require 'rest-client'
+
+module Fluent
+  class Pi < BufferedOutput
+    Fluent::Plugin.register_output('pi', self)
+
+    helpers :inject, :formatter, :compat_parameters
+
+    piwebapiserver = 'localhost', tagpath = nil 
+    valuetag = 'value'
+    username = nil, pwd = nil, anon = false
+    webid = nil
+
+    DEFAULT_FORMAT_TYPE = 'json'
+
+    config_section :buffer do
+      config_set_default :chunk_keys, ['tag']
+      config_set_default :flush_at_shutdown, true
+      config_set_default :chunk_limit_size, 10 * 1024
+    end
+
+    config_section :format do
+      config_set_default :@type, DEFAULT_FORMAT_TYPE
+    end
+
+    def prefer_buffered_processing
+      false
+    end
+
+    def prefer_delayed_commit
+      @delayed
+    end
+
+    attr_accessor :delayed
+
+    def initialize
+      super
+      @delayed = false
+    end
+
+    def configure(conf)
+      if conf['output_type'] && !conf['format']
+        conf['format'] = conf['output_type']
+      end
+      compat_parameters_convert(conf, :inject, :formatter)
+
+      super
+
+      @formatter = formatter_create(conf: conf.elements('format').first, default_type: DEFAULT_FORMAT_TYPE)
+
+      #set piwebapi URI information
+      @piwebapiserver = conf['piwebapiserver']
+      
+      (conf.key?('tagpath')) ? @tagpath = conf['tagpath'] : @tagpath = nil
+      (conf.key?('attpath')) ? @attpath = conf['attpath'] : @attpath = nil
+       
+      if(@tagpath == nil && @attpath == nil)
+        $log.write("Neither a tag nor attribute is specified.  Unable to log to PI.\n")
+        return
+      end
+
+      if(@tagpath != nil && @attpath != nil)
+        $log.write("Both a tag and attribute is specified.  Using the attribute.\n")
+      end
+      
+      (conf.key?('valuetag')) ? @valuetag = conf['valuetag'] : @valuetag = 'Value'
+      (conf.key?('username')) ? @username = conf['username'] : @username = nil    
+      (conf.key?('pwd')) ? @pwd = conf['pwd'] : @pwd = nil
+      (conf.key?('anon')) ? @anon = conf['anon'] : @anon = false
+
+      (conf.key?('webid')) ? @webid = conf['webid'] : @webid = nil
+      
+      if((@username == nil) && !@anon)
+        $log.write("Please specify a username, or specify that anonymous will be used.\n")
+        return
+      end
+    end
+
+    def start
+      super
+      # need to populate webid if not provided
+
+      if(@webid != nil)
+        return
+      end
+
+      client = nil  
+     #PI Web API search doesn't work from basic or anonymous.  Kerberos from Ruby isn't working.  I think we have to use the absolute path.  With Kerberos we could looking based on search and use top result
+      if(@attpath == nil)
+        url = "https://#{@piwebapiserver}/piwebapi/points/?path=#{@tagpath}"
+        if(@anon)
+          client = RestClient::Resource.new(url, :verify_ssl => OpenSSL::SSL::VERIFY_NONE )
+        else
+          client = RestClient::Resource.new(url, :verify_ssl => OpenSSL::SSL::VERIFY_NONE, :user => @username, :password =>@pwd)
+        end
+      else 
+        url = "https://#{@piwebapiserver}/piwebapi/attributes/?path=#{@attpath}"
+        if(@anon)
+          client = RestClient::Resource.new(url, :verify_ssl => OpenSSL::SSL::VERIFY_NONE )
+        else
+          client = RestClient::Resource.new(url, :verify_ssl => OpenSSL::SSL::VERIFY_NONE, :user => @username, :password =>@pwd)
+        end
+      end
+                                  
+      response = client.get
+      @webid = (JSON.parse(response.body))['WebId']
+      if(@webid == nil)
+        $log.write("WedID is lost.  Unable to log to PI\n")
+        return
+      end
+
+      @writeURL = "https://#{@piwebapiserver}/piwebapi/streams/#{@webid}/recorded"
+      if(@anon)
+        @writeclient = RestClient::Resource.new(@writeURL, :verify_ssl => OpenSSL::SSL::VERIFY_NONE)
+      else
+        @writeclient = RestClient::Resource.new(@writeURL, :verify_ssl => OpenSSL::SSL::VERIFY_NONE,:user => @username, :password =>@pwd)
+      end
+
+    end
+
+    def write(chunk)
+        data= chunk.read
+        jsonObj = "[" + data.chop + "]" # removes trailing ',' and makes a valid json object            
+
+        response = @writeclient.post jsonObj, :content_type => 'application/json'                   
+        $log.write("PI successful writes\n")
+    end
+
+    def format(tag, time, record)
+        value = record[@valuetag].to_s
+        "{\"Timestamp\":\"#{Time.at(time).localtime}\",\"Value\":\"#{value}\"},"
+    end    
+  end
+end

data/lib/fluent/plugin/out_qi.rb

@@ -0,0 +1,198 @@
+#     Copyright (c) 2013-2016 OSIsoft, LLC. All rights reserved.
+# 
+#     THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS OF
+#     OSIsoft, LLC.  USE, DISCLOSURE, OR REPRODUCTION IS PROHIBITED WITHOUT
+#     THE PRIOR EXPRESS WRITTEN PERMISSION OF OSIsoft, LLC.
+# 
+#     RESTRICTED RIGHTS LEGEND
+#     Use, duplication, or disclosure by the Government is subject to restrictions
+#     as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and
+#     Computer Software clause at DFARS 252.227.7013
+# 
+#     OSIsoft, LLC
+#     777 Davis Street, Suite 250, San Leandro CA 94577
+
+require 'fluent/output'
+require 'json'
+require 'rest-client'
+require 'adal'
+require 'certified'
+
+module Fluent
+  class Qi < BufferedOutput
+    Fluent::Plugin.register_output('qi', self)
+
+    helpers :inject, :formatter, :compat_parameters
+    
+    typeID = nil, streamID = nil, tenantId = nil, namespaceId = nil    
+    appId = nil, appKey = nil
+    valuetag = 'value'
+    access_token = nil, full_token = nil, refresh_token = nil
+    authHeader = nil
+
+    secondWriteAttempt = false
+
+    DEFAULT_FORMAT_TYPE = 'json'
+
+    config_section :buffer do
+      config_set_default :chunk_keys, ['tag']
+      config_set_default :flush_at_shutdown, true
+      config_set_default :chunk_limit_size, 10 * 1024
+    end
+
+    config_section :format do
+      config_set_default :@type, DEFAULT_FORMAT_TYPE
+    end
+
+    def prefer_buffered_processing
+      false
+    end
+
+    def prefer_delayed_commit
+      @delayed
+    end
+
+    attr_accessor :delayed
+
+    def initialize
+      super
+      @delayed = false
+    end
+
+    def configure(conf)
+      if conf['output_type'] && !conf['format']
+        conf['format'] = conf['output_type']
+      end
+      compat_parameters_convert(conf, :inject, :formatter)
+
+      super
+
+      @formatter = formatter_create(conf: conf.elements('format').first, default_type: DEFAULT_FORMAT_TYPE)
+      
+      (conf.key?('tenantId')) ? @tenantId = conf['tenantId'] : @tenantId = nil
+      (conf.key?('namespaceId')) ? @namespaceId = conf['namespaceId'] : @attpath = nil
+      
+      (conf.key?('appId')) ? @appId = conf['appId'] : @appId = nil
+      (conf.key?('appKey')) ? @appKey = conf['appKey'] : @appKey = nil
+
+      (conf.key?('typeID')) ? @typeID = conf['typeID'] : @typeID = nil 
+      (conf.key?('streamID')) ? @streamID = conf['streamID'] : @streamID = 'nil'
+      (conf.key?('valuetag')) ? @valuetag = conf['valuetag'] : @valuetag = 'Value'    
+       
+      if(@tenantId == nil  || @namespaceId == nil)
+        #could change requirement and create a namespace as it is needed, but currently it must be already created
+        $log.write("Please specify a tenantID and namespaceID.  Unable to send to QI.\n")
+        return
+      end
+      
+      if(@appId == nil  || @appKey == nil)
+        $log.write("Please specify a appId and appKey.  Unable to send to QI.\n")
+        return
+      end        
+    end
+    
+    def start
+      super
+      gettoken(true) 
+      @secondWriteAttempt = false
+    end   
+
+    def write(chunk)
+      data = chunk.read
+      jsonObj = "[" + data.chop + "]" # removes trailing ',' and makes a valid json object        
+
+      if(Time.at(@full_token.expires_on).utc  < (Time.now.utc + 30))
+        gettoken(false) #the refresh token isn't working so when we are withing 30 seconds get a new token
+      end
+
+      sendDataURL = "https://qi-data.osisoft.com/Qi/#{@tenantId}/#{@namespaceId}/Streams/#{@streamID}/Data/UpdateValues"
+
+      clientGetStream = RestClient::Resource.new(sendDataURL, :verify_ssl => OpenSSL::SSL::VERIFY_NONE)
+      clientGetStream.put(jsonObj, :authorization => @authHeader, :content_type => "application/json", :accept => "text/plain") { |response, request, result, &block|
+          case response.code
+          when 200,201,302   
+            #201 means it is created, 302 means it was created and it was just retrieved.
+            secondWriteAttempt=false
+          else
+            $log.write("Failed to write the value.  Regetting stream and type to see if this helps.")    
+            if(!secondWriteAttempt) 
+              secondWriteAttempt=true
+              createTypeAndStream()   
+              write(chunk)
+            else
+              $log.write("Failed to write the value.  Regetting stream and type did not help.")    
+              raise "Writing Values failed"  
+            end
+          end
+          }           
+        $log.write("QI successful writes\n")
+    end
+
+    def gettoken(getstream)
+        auth_ctx = ADAL::AuthenticationContext.new("login.windows.net","365cc376-d621-41b0-b32f-7a982b22a63c")
+        client_cred = ADAL::ClientCredential.new(@appId, @appKey)
+
+        token_response = auth_ctx.acquire_token_for_client("https://qihomeprod.onmicrosoft.com/historian",client_cred)
+        case token_response
+        when ADAL::SuccessResponse
+            # ADAL successfully exchanged the authorization code for an access_token.
+            # The token_response includes other information but we only care about the
+            # access token and the refresh token.
+            @full_token = token_response
+            @access_token = token_response.access_token
+            @refresh_token = token_response.refresh_token    
+            @authHeader = ("bearer " + @access_token)
+            
+            if(getstream)
+              createTypeAndStream()
+            end
+
+        when ADAL::ErrorResponse
+            # ADAL failed to exchange the authorization code for an access_token.
+            $log.write("Failed to get an access_token")
+            raise "ADAL token failed"  
+        end
+    end
+
+    def createTypeAndStream
+        typeURL = ("https://qi-data.osisoft.com/Qi/#{@tenantId}/#{@namespaceId}/Types")
+
+        # could open this up to let configuration dictate how the object is created.
+        # need to figure out if this should be guided or open (either ask for information and it creates JSON, or just import the JSON and use directly).     
+        jsonObj =  "{\"Id\":\"#{@typeID}\",\"Name\":\"FluentD\",\"Description\":\"This is a type for fluentd events\",\"QiTypeCode\":0,\"Properties\":[{\"Id\":\"Timestamp\",\"Name\":null,\"Description\":null,\"QiType\":{\"Id\":\"string\",\"Name\":null,\"Description\":null,\"QiTypeCode\":18,\"Properties\":null},\"IsKey\":true},{\"Id\":\"Value\",\"Name\":null,\"Description\":null,\"QiType\":{\"Id\":\"string\",\"Name\":null,\"Description\":null,\"QiTypeCode\":18,\"Properties\":null},\"IsKey\":false}]}"
+        
+        client = RestClient::Resource.new(typeURL, :verify_ssl => OpenSSL::SSL::VERIFY_NONE)
+    
+        client.post(jsonObj, :authorization => @authHeader, :content_type => "application/json", :accept => "text/plain") { |response, request, result, &block|
+            case response.code
+            when 200,201,302                  
+                #this returns a 201 when the create works and a 302 when the thing already existed
+
+                getStreamURL = "https://qi-data.osisoft.com/Qi/#{@tenantId}/#{@namespaceId}/Streams"
+                jsonObjGetStream = "{\"Id\":\"#{@streamID}\",\"Name\":\"WaveData_SampleStream\",\"Description\":null,\"TypeId\":\"#{@typeID}\",\"BehaviorId\":null}"
+                
+                clientGetStream = RestClient::Resource.new(getStreamURL, :verify_ssl => OpenSSL::SSL::VERIFY_NONE)
+                clientGetStream.post(jsonObjGetStream, :authorization => @authHeader, :content_type => "application/json", :accept => "text/plain") { |response, request, result, &block|
+                    case response.code
+                    when 201,302
+                    #this returns a 201 when the create works and a 302 when the thing already existed
+
+                    else          
+                        $log.write("Failed to get the stream created or found ")           
+                        raise "Stream creation failed"  
+                    end                    
+                    }
+            else
+            $log.write("Failed to get the type created or found ")      
+            raise "Type creation failed"  
+            end
+        }
+    end
+
+    def format(tag, time, record)
+      #assumes time is a bignum in FluentD pattern not a PI Time
+        value = record[@valuetag].to_s
+        return "{\"Timestamp\":\"#{Time.at(time).localtime}\",\"Value\":\"#{value}\"},"
+    end
+  end
+end

metadata

@@ -0,0 +1,124 @@
+--- !ruby/object:Gem::Specification
+name: fluent-plugin-pi
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Derek Endres
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2016-10-04 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rest-client
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.6.9
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.6.9
+- !ruby/object:Gem::Dependency
+  name: adal
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+- !ruby/object:Gem::Dependency
+  name: certified
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.13'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.13'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+description: Fluentd plugin to either get data from PI, send to PI or send to QI
+email:
+- dendres@osisoft.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CODE_OF_CONDUCT.md
+- Gemfile
+- Gemfile.lock
+- README.md
+- Rakefile
+- fluent-plugin-pi.gemspec
+- lib/fluent/plugin/in_piin.rb
+- lib/fluent/plugin/out_pi.rb
+- lib/fluent/plugin/out_qi.rb
+- pkg/fluent-plugin-pi-0.0.1.gem
+homepage: https://github.com/osisoftresearch
+licenses:
+- OSISoft LLC
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.7
+signing_key: 
+specification_version: 4
+summary: Fluentd plugin to either get data from PI, send to PI or send to QI.
+test_files: []