logstash-output-documentdb 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 5f78be4c25e8e6aa60117d8e38fce205e9d8a140
+  data.tar.gz: c7d4ae94f9fb1dd05bef5ae097597e5c6c72281c
+SHA512:
+  metadata.gz: e2765023f4e8f62f069273e52a7ed3136fb398bd0eeb662c3b7283357b0681ec2654ae2a6028e6e9e272538d9421dc23b05e159dea4682aa7ebf6b9374daf03d
+  data.tar.gz: dad8180f0c381e813962a6438c902978c37ab862c4f2c72f6c16774c1308a39128eebdd22960bab57b2d638441cd8277c8f469f48ad7777dea996e25c2eb8f83

data/CHANGELOG.md

@@ -0,0 +1,8 @@
+## 0.1.1
+
+* changed required package version in gemspec
+* fixup bug on error log messages output
+
+## 0.1.0
+
+* Initial Release

data/CONTRIBUTORS

@@ -0,0 +1,10 @@
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped logstash along its way.
+
+Contributors:
+* Yoichi Kawasaki (yokawasa)
+
+Note: If you've sent us patches, bug reports, or otherwise contributed to
+Logstash, and you aren't on the list above and want to be, please let us know
+and we'll make sure you're here. Contributions from folks like you are what make
+open source awesome.

data/Gemfile

@@ -0,0 +1,2 @@
+source 'https://rubygems.org'
+gemspec

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2012–2015 Elasticsearch <http://www.elastic.co>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -0,0 +1,127 @@
+# Azure DocumentDB output plugin for Logstash
+
+logstash-output-documentdb is a logstash plugin to output to Azure DocumentDB. [Logstash](https://www.elastic.co/products/logstash) is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite [destinations](https://www.elastic.co/products/logstash). [Azure DocumentDB](https://azure.microsoft.com/en-us/services/documentdb/) is a managed NoSQL database service provided by Microsoft Azure. It’s schemaless, natively support JSON, very easy-to-use, very fast, highly reliable, and enables rapid deployment, you name it.
+
+## Installation
+
+You can install this plugin using the Logstash "plugin" or "logstash-plugin" (for newer versions of Logstash) command:
+```
+bin/plugin install logstash-output-documentdb
+# or 
+bin/logstash-plugin install logstash-output-documentdb  (Newer versions of Logstash)
+```
+Please see [Logstash reference](https://www.elastic.co/guide/en/logstash/current/offline-plugins.html) for more information.
+
+## Configuration
+
+```
+output {
+    documentdb {
+        docdb_endpoint => "https://<YOUR ACCOUNT>.documents.azure.com:443/"
+        docdb_account_key => "<ACCOUNT KEY>"
+        docdb_database => "<DATABASE NAME>"
+        docdb_collection => "<COLLECTION NAME>"
+        auto_create_database => true|false
+        auto_create_collection => true|false
+        partitioned_collection => true|false
+        partition_key =>  "<PARTITIONED KEY NAME>"
+        offer_throughput => <THROUGHPUT NUM>
+    }
+}
+```
+
+ * **docdb\_endpoint (required)** - Azure DocumentDB Account endpoint URI
+ * **docdb\_account\_key (required)** - Azure DocumentDB Account key (master key). You must NOT set a read-only key
+ * **docdb\_database (required)** - DocumentDB database nameb
+ * **docdb\_collection (required)** - DocumentDB collection name
+ * **auto\_create\_database (optional)** - Default:true. By default, DocumentDB database named **docdb\_database** will be automatically created if it does not exist
+ * **auto\_create\_collection (optional)** - Default:true. By default, DocumentDB collection named **docdb\_collection** will be automatically created if it does not exist
+ * **partitioned\_collection (optional)** - Default:false. Set true if you want to create and/or store records to partitioned collection. Set false for single-partition collection
+ * **partition\_key (optional)** - Default:nil. Partition key must be specified for paritioned collection (partitioned\_collection set to be true)
+ * **offer\_throughput (optional)** - Default:10100. Throughput for the collection expressed in units of 100 request units per second. This is only effective when you newly create a partitioned collection (ie. Both auto\_create\_collection and partitioned\_collection are set to be true )
+
+
+## Tests
+
+logstash-output-documentdb adds id attribute (UUID format) to in-coming events automatically and send them to DocumentDB. Here is an example configuration where Logstash's event source and destination are configured as Apache2 access log and DocumentDB respectively.
+
+### Example Configuration
+```
+input {
+    file {
+        path => "/var/log/apache2/access.log"
+        start_position => "beginning"
+    }
+}
+
+filter {
+    if [path] =~ "access" {
+        mutate { replace => { "type" => "apache_access" } }
+        grok {
+            match => { "message" => "%{COMBINEDAPACHELOG}" }
+        }
+    }
+    date {
+        match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
+    }
+}
+
+output {
+    documentdb {
+        docdb_endpoint => "https://yoichikademo.documents.azure.com:443/"
+        docdb_account_key => "EMwUa3EzsAtJ1qYfzwo9nQxxydofsXNm3xLh1SLffKkUHMFl80OZRZIVu4lxdKRKxkgVAj0c2mv9BZSyMN7tdg==(dummy)"
+        docdb_database => "testdb"
+        docdb_collection => "apache_access"
+        auto_create_database => true
+        auto_create_collection => true
+    }
+    # for debug
+    stdout { codec => rubydebug }
+}
+```
+You can find example configuration files in logstash-output-documentdb/examples.
+
+### Run the plugin with the example configuration
+
+Now you run logstash with the the example configuration like this:
+```
+# Test your logstash configuration before actually running the logstash
+bin/logstash -f logstash-apache2-to-documentdb.conf  --configtest
+# run
+bin/logstash -f logstash-apache2-to-documentdb.conf
+```
+
+Here is an expected output for sample input (Apache2 access log):
+
+<u>Apache2 access log</u>
+```
+124.211.152.166 - - [27/Dec/2016:02:12:28 +0000] "GET /test.html HTTP/1.1" 200 316 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
+```
+
+<u>Output (rubydebug)</u>
+```
+{
+        "message" => "124.211.152.166 - - [27/Dec/2016:02:12:28 +0000] \"GET /test.html HTTP/1.1\" 200 316 \"-\" \"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36\"",
+       "@version" => "1",
+     "@timestamp" => "2016-12-27T02:12:28.000Z",
+           "path" => "/var/log/apache2/access.log",
+           "host" => "yoichitest01",
+           "type" => "apache_access",
+       "clientip" => "124.211.152.166",
+          "ident" => "-",
+           "auth" => "-",
+      "timestamp" => "27/Dec/2016:02:12:28 +0000",
+           "verb" => "GET",
+        "request" => "/test.html",
+    "httpversion" => "1.1",
+       "response" => "200",
+          "bytes" => "316",
+       "referrer" => "\"-\"",
+          "agent" => "\"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36\"",
+             "id" => "0cae1966-b7ab-4f32-8893-b4fabc7800ae"
+}
+```
+
+## Contributing
+Bug reports and pull requests are welcome on GitHub at https://github.com/yokawasa/logstash-output-documentdb.
+

data/VERSION

@@ -0,0 +1 @@
+0.1.1

data/lib/logstash/outputs/documentdb.rb

@@ -0,0 +1,101 @@
+# encoding: utf-8
+require "logstash/outputs/base"
+require "logstash/namespace"
+require 'time'
+require 'securerandom'
+require_relative 'documentdb/client'
+require_relative 'documentdb/partitioned_coll_client'
+require_relative 'documentdb/header'
+require_relative 'documentdb/resource'
+
+
+class LogStash::Outputs::Documentdb < LogStash::Outputs::Base
+  config_name "documentdb"
+
+  config :docdb_endpoint, :validate => :string, :required => true
+  config :docdb_account_key, :validate => :string, :required => true
+  config :docdb_database, :validate => :string, :required => true
+  config :docdb_collection, :validate => :string, :required => true
+  config :auto_create_database, :validate => :boolean, :default => true
+  config :auto_create_collection, :validate => :boolean, :default => true
+  config :partitioned_collection, :validate => :boolean, :default => false
+  config :partition_key, :validate => :string, :default => nil
+  config :offer_throughput, :validate =>:number, :default => AzureDocumentDB::PARTITIONED_COLL_MIN_THROUGHPUT
+
+  public
+  def register
+    ## Configure
+    if @partitioned_collection
+      raise ArgumentError, 'partition_key must be set in partitioned collection mode' if @partition_key.empty?
+      if (@auto_create_collection &&
+             @offer_throughput < AzureDocumentDB::PARTITIONED_COLL_MIN_THROUGHPUT)
+          raise ArgumentError, sprintf("offer_throughput must be more than and equals to %s",
+                                 AzureDocumentDB::PARTITIONED_COLL_MIN_THROUGHPUT)
+      end
+    end
+
+    ## Start 
+    begin
+      @client = nil
+      if @partitioned_collection
+        @client = AzureDocumentDB::PartitionedCollectionClient.new(@docdb_account_key,@docdb_endpoint)
+      else
+        @client = AzureDocumentDB::Client.new(@docdb_account_key,@docdb_endpoint)
+      end
+
+      # initial operations for database
+      res = @client.find_databases_by_name(@docdb_database)
+      if( res[:body]["_count"].to_i == 0 )
+        raise RuntimeError, "No database (#{docdb_database}) exists! Enable auto_create_database or create it by useself" if !@auto_create_database
+        # create new database as it doesn't exists
+        @client.create_database(@docdb_database)
+      end
+
+      # initial operations for collection
+      database_resource = @client.get_database_resource(@docdb_database)
+      res = @client.find_collections_by_name(database_resource, @docdb_collection)
+      if( res[:body]["_count"].to_i == 0 )
+        raise "No collection (#{docdb_collection}) exists! Enable auto_create_collection or create it by useself" if !@auto_create_collection
+        # create new collection as it doesn't exists
+        if @partitioned_collection
+          partition_key_paths = ["/#{@partition_key}"]
+          @client.create_collection(database_resource,
+                      @docdb_collection, partition_key_paths, @offer_throughput)
+        else
+          @client.create_collection(database_resource, @docdb_collection)
+        end
+      end
+      @coll_resource = @client.get_collection_resource(database_resource, @docdb_collection)
+
+    rescue Exception =>ex
+      @logger.error("Documentdb output plugin's register Error: '#{ex}'")
+      exit!
+    end
+  end # def register
+
+  public
+  def receive(event)
+    document = event.to_hash()
+    document['id'] =  SecureRandom.uuid
+
+    ## Writing document to DocumentDB
+    unique_doc_identifier = document['id']
+    begin
+      if @partitioned_collection
+        @client.create_document(@coll_resource, unique_doc_identifier, document, @partition_key)
+      else
+        @client.create_document(@coll_resource, unique_doc_identifier, document)
+      end
+    rescue RestClient::ExceptionWithResponse => rcex
+      exdict = JSON.parse(rcex.response)
+      if exdict['code'] == 'Conflict'
+        $logger.error("Duplicate Error: document #{unique_doc_identifier} already exists, data=>" + (document.to_json).to_s)
+      else
+        $logger.error("RestClient Error: '#{rcex.response}', data=>" + (document.to_json).to_s)
+      end
+    rescue => ex
+      $logger.error("UnknownError: '#{ex}', uniqueid=>#{unique_doc_identifier}, data=>" + (document.to_json).to_s )
+    end
+  end # def event
+
+end # class LogStash::Outputs::Documentdb

data/lib/logstash/outputs/documentdb/client.rb

@@ -0,0 +1,167 @@
+require 'rest-client'
+require 'json'
+require_relative 'constants'
+require_relative 'header'
+require_relative 'resource'
+
+module AzureDocumentDB
+
+  class Client
+
+    def initialize (master_key, url_endpoint)
+      @master_key = master_key
+      @url_endpoint = url_endpoint
+      @header = AzureDocumentDB::Header.new(@master_key)
+    end
+
+    def create_database (database_name)
+      url = "#{@url_endpoint}/dbs"
+      custom_headers = {'Content-Type' => 'application/json'}
+      headers = @header.generate('post', AzureDocumentDB::RESOURCE_TYPE_DATABASE, '', custom_headers )
+      body_json = { 'id' => database_name }.to_json
+      res = RestClient.post( url, body_json, headers)
+      JSON.parse(res)
+    end
+
+    def find_databases_by_name (database_name)
+      query_params = []
+      query_text = "SELECT * FROM root r WHERE r.id=@id"
+      query_params.push( {:name=>"@id", :value=> database_name } )
+      url = sprintf("%s/dbs", @url_endpoint )
+      res = _query(AzureDocumentDB::RESOURCE_TYPE_DATABASE, '', url, query_text, query_params)
+      res 
+    end
+
+    def get_database_resource (database_name) 
+      resource  = nil
+      res = find_databases_by_name (database_name)
+      if( res[:body]["_count"].to_i == 0 )
+        p "no #{database_name} database exists"
+        return resource
+      end
+      res[:body]['Databases'].select do |db| 
+        if (db['id'] == database_name )
+          resource = AzureDocumentDB::DatabaseResource.new(db['_rid'])
+        end
+      end
+      resource
+    end
+
+    def create_collection(database_resource, collection_name, colls_options={}, custom_headers={} )
+      if !database_resource 
+        raise ArgumentError.new 'No database_resource!'
+      end
+      url = sprintf("%s/dbs/%s/colls", @url_endpoint, database_resource.database_rid )
+      custom_headers['Content-Type'] = 'application/json'
+      headers = @header.generate('post',
+            AzureDocumentDB::RESOURCE_TYPE_COLLECTION,
+            database_resource.database_rid, custom_headers )
+      body = {'id' => collection_name }
+      colls_options.each{|k, v|
+        if k == 'indexingPolicy' || k == 'partitionKey'
+          body[k] = v
+        end
+      }
+      res = RestClient.post( url, body.to_json, headers)
+      JSON.parse(res)
+    end
+
+    def find_collections_by_name(database_resource, collection_name)
+      if !database_resource 
+        raise ArgumentError.new 'No database_resource!'
+      end
+      ret = {}
+      query_params = []
+      query_text = "SELECT * FROM root r WHERE r.id=@id"
+      query_params.push( {:name=>"@id", :value=> collection_name } )
+      url = sprintf("%s/dbs/%s/colls", @url_endpoint, database_resource.database_rid)
+      ret = _query(AzureDocumentDB::RESOURCE_TYPE_COLLECTION,
+                database_resource.database_rid, url, query_text, query_params)
+      ret
+    end
+
+    def get_collection_resource (database_resource, collection_name)
+      _collection_rid = ''
+      if !database_resource 
+        raise ArgumentError.new 'No database_resource!'
+      end
+      res = find_collections_by_name(database_resource, collection_name)
+      res[:body]['DocumentCollections'].select do |col| 
+        if (col['id'] == collection_name )
+          _collection_rid = col['_rid']
+        end
+      end
+      if _collection_rid.empty?
+        p "no #{collection_name} collection exists"
+        return nil
+      end
+      AzureDocumentDB::CollectionResource.new(database_resource.database_rid, _collection_rid)      
+    end
+    
+    def create_document(collection_resource, document_id, document, custom_headers={} )
+      if !collection_resource 
+        raise ArgumentError.new 'No collection_resource!'
+      end
+      if document['id'] && document_id != document['id'] 
+        raise ArgumentError.new "Document id mismatch error (#{document_id})!"
+      end
+      body = { 'id' => document_id }.merge document
+      url = sprintf("%s/dbs/%s/colls/%s/docs",
+                  @url_endpoint, collection_resource.database_rid, collection_resource.collection_rid)
+      custom_headers['Content-Type'] = 'application/json'
+      headers = @header.generate('post', AzureDocumentDB::RESOURCE_TYPE_DOCUMENT,
+                                  collection_resource.collection_rid, custom_headers )
+      res = RestClient.post( url, body.to_json, headers)
+      JSON.parse(res)
+    end
+
+    def find_documents(collection_resource, document_id, custom_headers={})
+      if !collection_resource 
+        raise ArgumentError.new 'No collection_resource!'
+      end
+      ret = {}
+      query_params = []
+      query_text = "SELECT * FROM c WHERE c.id=@id"
+      query_params.push( {:name=>"@id", :value=> document_id } )
+      url = sprintf("%s/dbs/%s/colls/%s/docs",
+              @url_endpoint, collection_resource.database_rid, collection_resource.collection_rid)
+      ret = _query(AzureDocumentDB::RESOURCE_TYPE_DOCUMENT,
+              collection_resource.collection_rid, url, query_text, query_params, custom_headers)
+      ret
+    end
+
+    def query_documents( collection_resource, query_text, query_params, custom_headers={} )
+      if !collection_resource 
+        raise ArgumentError.new 'No collection_resource!'
+      end
+      ret = {}
+      url = sprintf("%s/dbs/%s/colls/%s/docs",
+              @url_endpoint, collection_resource.database_rid, collection_resource.collection_rid)
+      ret = _query(AzureDocumentDB::RESOURCE_TYPE_DOCUMENT,
+              collection_resource.collection_rid, url, query_text, query_params, custom_headers)
+      ret
+    end
+
+    protected
+
+    def _query( resource_type, parent_resource_id, url, query_text, query_params, custom_headers={} )
+      query_specific_header = {
+              'x-ms-documentdb-isquery' => 'True',
+              'Content-Type' => 'application/query+json',
+              'Accept' => 'application/json'
+             }
+      query_specific_header.merge! custom_headers
+      headers = @header.generate('post', resource_type, parent_resource_id, query_specific_header)
+      body_json = {
+              :query => query_text,
+              :parameters => query_params
+             }.to_json
+
+      res = RestClient.post( url, body_json, headers) 
+      result = {
+          :header => res.headers,
+          :body => JSON.parse(res.body) }
+      return result
+    end
+  end
+end

data/lib/logstash/outputs/documentdb/constants.rb

@@ -0,0 +1,10 @@
+module AzureDocumentDB
+  API_VERSION = '2015-12-16'.freeze
+  RESOURCE_TYPE_DATABASE='dbs'.freeze
+  RESOURCE_TYPE_COLLECTION='colls'.freeze
+  RESOURCE_TYPE_DOCUMENT='docs'.freeze
+  AUTH_TOKEN_VERSION = '1.0'.freeze
+  AUTH_TOKEN_TYPE_MASTER = 'master'.freeze
+  AUTH_TOKEN_TYPE_RESOURCE = 'resource'.freeze
+  PARTITIONED_COLL_MIN_THROUGHPUT = 10100.freeze
+end

data/lib/logstash/outputs/documentdb/header.rb

@@ -0,0 +1,55 @@
+require 'time'
+require 'openssl'
+require 'base64'
+require 'erb'
+
+module AzureDocumentDB
+
+  class Header
+
+    def initialize (master_key)
+      @master_key = master_key
+    end
+
+    def generate (verb, resource_type, parent_resource_id, api_specific_headers = {} )
+      headers = {}
+      utc_date = get_httpdate()
+      auth_token = generate_auth_token(verb, resource_type, parent_resource_id, utc_date )
+      default_headers = { 
+            'x-ms-version' => AzureDocumentDB::API_VERSION,
+            'x-ms-date' => utc_date,
+            'authorization' => auth_token 
+            }.freeze
+      headers.merge!(default_headers)
+      headers.merge(api_specific_headers)
+    end
+
+    private
+    
+    def generate_auth_token ( verb, resource_type, resource_id, utc_date)
+      payload = sprintf("%s\n%s\n%s\n%s\n%s\n",
+                verb,
+                resource_type,
+                resource_id,
+                utc_date,
+                "" )
+      sig = hmac_base64encode(payload)
+
+      ERB::Util.url_encode sprintf("type=%s&ver=%s&sig=%s",
+                AzureDocumentDB::AUTH_TOKEN_TYPE_MASTER,
+                AzureDocumentDB::AUTH_TOKEN_VERSION,
+                sig )
+    end
+
+    def get_httpdate
+      Time.now.httpdate
+    end
+
+    def hmac_base64encode( text )
+      key = Base64.urlsafe_decode64 @master_key
+      hmac = OpenSSL::HMAC.digest('sha256', key, text.downcase)
+      Base64.encode64(hmac).strip
+    end
+
+  end
+end

data/lib/logstash/outputs/documentdb/partitioned_coll_client.rb

@@ -0,0 +1,62 @@
+require 'rest-client'
+require 'json'
+require_relative 'constants'
+require_relative 'header'
+require_relative 'resource'
+
+module AzureDocumentDB
+
+  class PartitionedCollectionClient < Client
+
+    def create_collection(database_resource, collection_name,
+          partition_key_paths, offer_throughput = AzureDocumentDB::PARTITIONED_COLL_MIN_THROUGHPUT )
+
+      if (offer_throughput <  AzureDocumentDB::PARTITIONED_COLL_MIN_THROUGHPUT) 
+        raise ArgumentError.new sprintf("Offeer thoughput need to be more than %d !",
+                          AzureDocumentDB::PARTITIONED_COLL_MIN_THROUGHPUT)
+      end 
+      if (partition_key_paths.length < 1 )
+        raise ArgumentError.new "No PartitionKey paths!"
+      end
+      colls_options = {
+            'indexingPolicy' => { 'indexingMode' => "consistent", 'automatic'=>true },
+            'partitionKey' => { "paths" => partition_key_paths, "kind" => "Hash" }
+      }
+      custom_headers= {'x-ms-offer-throughput' => offer_throughput }
+      super(database_resource, collection_name, colls_options, custom_headers)
+    end
+
+
+    def create_document(collection_resource, document_id, document, partitioned_key )
+      if partitioned_key.empty?
+        raise ArgumentError.new "No partitioned key!"
+      end
+      if !document.key?(partitioned_key)
+        raise ArgumentError.new "No partitioned key in your document!"
+      end
+      partitioned_key_value = document[partitioned_key]
+      custom_headers = {
+          'x-ms-documentdb-partitionkey' => "[\"#{partitioned_key_value}\"]"
+        }
+      super(collection_resource, document_id, document, custom_headers) 
+    end
+
+    def find_documents(collection_resource, document_id,
+                partitioned_key, partitioned_key_value, custom_headers={})
+      if !collection_resource 
+        raise ArgumentError.new "No collection_resource!"
+      end
+      ret = {}
+      query_params = []
+      query_text = sprintf("SELECT * FROM c WHERE c.id=@id AND c.%s=@value", partitioned_key)
+      query_params.push( {:name=>"@id", :value=> document_id } )
+      query_params.push( {:name=>"@value", :value=> partitioned_key_value } )
+      url = sprintf("%s/dbs/%s/colls/%s/docs",
+              @url_endpoint, collection_resource.database_rid, collection_resource.collection_rid)
+      ret = query(AzureDocumentDB::RESOURCE_TYPE_DOCUMENT,
+              collection_resource.collection_rid, url, query_text, query_params, custom_headers)
+      ret
+    end
+
+  end
+end

data/lib/logstash/outputs/documentdb/resource.rb

@@ -0,0 +1,40 @@
+module AzureDocumentDB
+
+  class Resource
+    def initialize 
+      @r = {}
+    end
+    protected
+      attr_accessor :r
+  end
+
+  class DatabaseResource < Resource
+  
+    def initialize (database_rid) 
+      super()
+      @r['database_rid'] = database_rid
+    end
+
+    def database_rid
+      @r['database_rid']
+    end
+  end
+
+  class CollectionResource < Resource
+  
+    def initialize (database_rid, collection_rid) 
+      super()
+      @r['database_rid'] = database_rid
+      @r['collection_rid'] = collection_rid
+    end
+
+    def database_rid
+      @r['database_rid']
+    end
+
+    def collection_rid
+      @r['collection_rid']
+    end
+  end
+
+end

data/logstash-output-documentdb.gemspec

@@ -0,0 +1,25 @@
+Gem::Specification.new do |s|
+  s.name = 'logstash-output-documentdb'
+  s.version    =  File.read("VERSION").strip
+  s.authors = ["Yoichi Kawasaki"]
+  s.email = "yoichi.kawasaki@outlook.com"
+  s.summary = %q{logstash output plugin to store events into Azure DocumentDB}
+  s.description = s.summary
+  s.homepage = "http://github.com/yokawasa/logstash-output-documentdb"
+  s.licenses = ["Apache License (2.0)"]
+  s.require_paths = ["lib"]
+
+  # Files
+  s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT', 'VERSION']
+   # Tests
+  s.test_files = s.files.grep(%r{^(test|spec|features)/})
+
+  # Special flag to let us know this is actually a logstash plugin
+  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "output" }
+
+  # Gem dependencies
+  s.add_runtime_dependency "rest-client"
+  s.add_runtime_dependency "logstash-core", ">= 2.0.0", "< 3.0.0"
+  s.add_runtime_dependency "logstash-codec-plain"
+  s.add_development_dependency "logstash-devutils"
+end

data/spec/outputs/documentdb_spec.rb

@@ -0,0 +1,41 @@
+# encoding: utf-8
+require "logstash/devutils/rspec/spec_helper"
+require "logstash/outputs/documentdb"
+require "logstash/codecs/plain"
+require "logstash/event"
+
+describe LogStash::Outputs::Documentdb do
+
+  let(:docdb_endpoint) { 'https://<YOUR ACCOUNT>.documents.azure.com:443/' }
+  let(:docdb_account_key) { '<ACCOUNT KEY>' }
+  let(:docdb_database) { '<DATABASE NAME>' }
+  let(:docdb_collection) { '<COLLECTION NAME>' }
+  let(:auto_create_database) { true }
+  let(:auto_create_collection) { true }
+
+  let(:docdb_config) {
+    { 
+      "docdb_endpoint" => docdb_endpoint, 
+      "docdb_account_key" => docdb_account_key,
+      "docdb_database" => docdb_database,
+      "docdb_collection" => docdb_collection,
+      "auto_create_database" => auto_create_database,
+      "auto_create_collection" => auto_create_collection,
+    }
+  }
+
+  let(:docdb_output) { LogStash::Outputs::Documentdb.new(docdb_config) }
+
+  before do
+      docdb_output.register
+  end 
+
+  describe "#recieve" do
+    it "Should successfully send the event to documentdb" do
+      properties = { "a" => 1, "b" => 2, "c" => 3 }
+      event =  LogStash::Event.new(properties) 
+      expect {docdb_output.receive(event)}.to_not raise_error
+    end
+  end
+
+end

metadata

@@ -0,0 +1,122 @@
+--- !ruby/object:Gem::Specification
+name: logstash-output-documentdb
+version: !ruby/object:Gem::Version
+  version: 0.1.1
+platform: ruby
+authors:
+- Yoichi Kawasaki
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2016-12-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: rest-client
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+  name: logstash-core
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: logstash-codec-plain
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: logstash-devutils
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: logstash output plugin to store events into Azure DocumentDB
+email: yoichi.kawasaki@outlook.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- CONTRIBUTORS
+- Gemfile
+- LICENSE
+- README.md
+- VERSION
+- lib/logstash/outputs/documentdb.rb
+- lib/logstash/outputs/documentdb/client.rb
+- lib/logstash/outputs/documentdb/constants.rb
+- lib/logstash/outputs/documentdb/header.rb
+- lib/logstash/outputs/documentdb/partitioned_coll_client.rb
+- lib/logstash/outputs/documentdb/resource.rb
+- logstash-output-documentdb.gemspec
+- spec/outputs/documentdb_spec.rb
+homepage: http://github.com/yokawasa/logstash-output-documentdb
+licenses:
+- Apache License (2.0)
+metadata:
+  logstash_plugin: 'true'
+  logstash_group: output
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project:
+rubygems_version: 2.4.8
+signing_key:
+specification_version: 4
+summary: logstash output plugin to store events into Azure DocumentDB
+test_files:
+- spec/outputs/documentdb_spec.rb