logstash-output-jdbc 0.2.2.rc1 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 136a9e59dadead66c0b3aa3b4ae87b194a66778d
-  data.tar.gz: a53165aa0c81c5f945d2c7776e9c6ad01ed528e0
+  metadata.gz: 9d9c04c7056a8af5071a1da2710a14339093abf3
+  data.tar.gz: 4dd0af7eca9417f9c51944b0fb325c42c4cc4ea7
 SHA512:
-  metadata.gz: 3034d07b2ce2bcc743eca63d5311903b52c57b103aab548c6840b67608413fdb87efed6abf86228b81c37a0c48ba61002b4519137b60a463bc3c9d20acaba762
-  data.tar.gz: 9e54e34966849f7bf8bd39f6ae8a603ea391da685ba840c09a045c7d3ab70be7374167a931826bbc4f822d9b512f54abb85c1c4ba1efe72755103f28ac96387c
+  metadata.gz: abb87d28ba5e8cb5031d5a9a3b23ab44d925608cd8ce81646406366a4f45bac690c356618c74e1b28b57fe63f3c48b6c0cdea78e095882126da36ef73d52634c
+  data.tar.gz: 7ea9fc64a0a90155228dc276af38d394ac135e5cbb91af37ee05cad8b64d8355c339da076eaabdc5e2e3fa404ad86af2fc16d3744ca079a38fb830516bb4384e

data/README.md

@@ -17,6 +17,9 @@ particular location. Please ensure you read the 4 installation lines below.
   - Altered exception handling to now count sequential flushes with exceptions thrown in 0.2.0 
 
 ## Versions
+Released versions are are tagged as of v0.2.1, and available via rubygems.
+
+For development:
   - See master branch for logstash v2+
   - See v1.5 branch for logstash v1.5 
   - See v1.4 branch for logstash 1.4
@@ -24,7 +27,7 @@ particular location. Please ensure you read the 4 installation lines below.
 ## Installation
   - Run `bin/plugin install logstash-output-jdbc` in your logstash installation directory
   - Now either:
-    - Use driver_class in your configuraton to specify a path to your jar file
+    - Use driver_path in your configuraton to specify a path to your jar file
   - Or:
     - Create the directory vendor/jar/jdbc in your logstash installation (`mkdir -p vendor/jar/jdbc/`)
     - Add JDBC jar files to vendor/jar/jdbc in your logstash installation
@@ -56,120 +59,6 @@ Tests are not yet 100% complete.
 | max_flush_exceptions | Number | Number of sequential flushes which cause an exception, before we stop logstash. Set to a value less than 1 if you never want it to stop. This should be carefully configured with relation to idle_flush_time if your SQL instance is not highly available. | No | 0 |
 
 ## Example configurations
-If you have a working sample configuration, for a DB thats not listed, pull requests are welcome.
-
-### SQLite3
-  * Tested using https://bitbucket.org/xerial/sqlite-jdbc
-  * SQLite setup - `echo "CREATE table log (host text, timestamp datetime, message text);" | sqlite3 test.db`
-```
-input
-{
-	stdin { }
-}
-output {
-	stdout { }
-
-	jdbc {
-		connection_string => 'jdbc:sqlite:test.db'
-		statement => [ "INSERT INTO log (host, timestamp, message) VALUES(?, ?, ?)", "host", "@timestamp", "message" ]
-	}
-}
-```
-
-### SQL Server
-  * Tested using http://msdn.microsoft.com/en-gb/sqlserver/aa937724.aspx
-```
-input
-{
-	stdin { }
-}
-output {
-	jdbc {
-		connection_string => "jdbc:sqlserver://server:1433;databaseName=databasename;user=username;password=password;autoReconnect=true;"
-		statement => [ "INSERT INTO log (host, timestamp, message) VALUES(?, ?, ?)", "host", "@timestamp", "message" ]
-	}
-}
-```
-
-### Postgres
-With thanks to [@roflmao](https://github.com/roflmao)
-```
-input
-{
-	stdin { }
-}
-output {
-	jdbc {
-		connection_string => 'jdbc:postgresql://hostname:5432/database?user=username&password=password'
-		statement => [ "INSERT INTO log (host, timestamp, message) VALUES(?, CAST (? AS timestamp), ?)", "host", "@timestamp", "message" ]
-	}
-}
-```
+Example logstash configurations, can now be found in the examples directory. Where possible we try to link every configuration with a tested jar.
 
-### Oracle
-With thanks to [@josemazo](https://github.com/josemazo)
-  * Tested with Express Edition 11g Release 2
-  * Tested using http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html (ojdbc6.jar)
-```
-input
-{
-	stdin { }
-}
-output {
-	jdbc {
-		connection_string => "jdbc:oracle:thin:USER/PASS@HOST:PORT:SID"
-		statement => [ "INSERT INTO log (host, timestamp, message) VALUES(?, CAST (? AS timestamp), ?)", "host", "@timestamp", "message" ]
-	}
-}
-```
-
-### Mysql
-With thanks to [@jMonsinjon](https://github.com/jMonsinjon) 
-  * Tested with Version 14.14 Distrib 5.5.43, for debian-linux-gnu (x86_64)
-  * Tested using http://dev.mysql.com/downloads/file.php?id=457911 (mysql-connector-java-5.1.36-bin.jar)
-```
-input
-{
-	stdin { }
-}
-output {
-	jdbc {
-		connection_string => "jdbc:mysql://HOSTNAME/DATABASE?user=USER&password=PASSWORD"
-		statement => [ "INSERT INTO log (host, timestamp, message) VALUES(?, CAST (? AS timestamp), ?)", "host", "@timestamp", "message" ]
-	}
-}
-```
-
-### MariaDB
-  * Tested with Ubuntu 14.04.3 LTS, Server version: 10.1.9-MariaDB-1~trusty-log mariadb.org binary distribution
-  * Tested using https://downloads.mariadb.com/enterprise/tqge-whfa/connectors/java/connector-java-1.3.2/mariadb-java-client-1.3.2.jar (mariadb-java-client-1.3.2.jar)
-```
-input
-{
-    stdin { }
-}
-output {
-    jdbc {
-        connection_string => "jdbc:mariadb://HOSTNAME/DATABASE?user=USER&password=PASSWORD"
-        statement => [ "INSERT INTO log (host, timestamp, message) VALUES(?, ?, ?)", "host", "@timestamp", "message" ]
-    }
-
-}
-```
-
-### Apache Phoenix (HBase SQL)
-  * Tested with Ubuntu 14.04.03 / Logstash 2.1 / Apache Phoenix 4.6
-  * <!> HBase and Zookeeper must be both accessible from logstash machine <!>
-```
-input
-{
-    stdin { }
-}
-output {
-    jdbc {
-        connection_string => "jdbc:phoenix:ZOOKEEPER_HOSTNAME"
-        statement => [ "UPSERT INTO EVENTS log (host, timestamp, message) VALUES(?, ?, ?)", "host", "@timestamp", "message" ]
-    }
-
-}
-```
+If you have a working sample configuration, for a DB thats not listed, pull requests are welcome.

data/lib/logstash/outputs/jdbc.rb

@@ -83,25 +83,22 @@ class LogStash::Outputs::Jdbc < LogStash::Outputs::Base
 
     load_jar_files!
 
-    @pool = Java::ComZaxxerHikari::HikariDataSource.new
-
-    @pool.setAutoCommit(@driver_auto_commit)
-    @pool.setDriverClassName(@driver_class) if @driver_class
-
-    @pool.setJdbcUrl(@connection_string)
-
-    @pool.setUsername(@username) if @username
-    @pool.setPassword(@password) if @password
-
-    @pool.setMaximumPoolSize(@max_pool_size)
-    @pool.setConnectionTimeout(@connection_timeout)
-
     @exceptions_tracker = RingBuffer.new(@max_flush_exceptions)
 
     if (@flush_size > 1000)
       @logger.warn("JDBC - Flush size is set to > 1000")
     end
 
+    if @statement.length < 1
+      @logger.error("JDBC - No statement provided. Configuration error.")
+    end
+
+    if (!@unsafe_statement and @statement.length < 2) 
+      @logger.error("JDBC - Statement has no parameters. No events will be inserted into SQL as you're not passing any event data. Likely configuration error.")
+    end
+
+    setup_and_test_pool!
+    
     buffer_initialize(
       :max_items => @flush_size,
       :max_interval => @idle_flush_time,
@@ -144,6 +141,29 @@ class LogStash::Outputs::Jdbc < LogStash::Outputs::Base
 
   private
 
+  def setup_and_test_pool!  
+    # Setup pool
+    @pool = Java::ComZaxxerHikari::HikariDataSource.new
+
+    @pool.setAutoCommit(@driver_auto_commit)
+    @pool.setDriverClassName(@driver_class) if @driver_class
+
+    @pool.setJdbcUrl(@connection_string)
+
+    @pool.setUsername(@username) if @username
+    @pool.setPassword(@password) if @password
+
+    @pool.setMaximumPoolSize(@max_pool_size)
+    @pool.setConnectionTimeout(@connection_timeout)
+
+    # Test connection
+    test_connection = @pool.getConnection()
+    unless test_connection.isValid(10)
+      @logger.error("JDBC - Connection is not valid. Please check connection string or that your JDBC endpoint is available.")
+    end
+    test_connection.close()
+  end
+
   def load_jar_files!
     # Load jar from driver path
     unless @driver_jar_path.nil?

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: logstash-output-jdbc
 version: !ruby/object:Gem::Version
-  version: 0.2.2.rc1
+  version: 0.2.2
 platform: ruby
 authors:
 - the_angry_angel
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-12-23 00:00:00.000000000 Z
+date: 2015-12-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: logstash-core
@@ -107,9 +107,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>'
+  - - '>='
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubyforge_project: 
 rubygems_version: 2.0.14