logstash-output-charrington 0.3.20 → 0.3.21

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2ae644e2b642cfbf37d315289ff166d7b03e19865dd50320126e71f50954965c
-  data.tar.gz: 5af164f7c8a73ed16001ef89922095bb0d37699fbc7c144c06eff4ab23e0481a
+  metadata.gz: b495c8c2b48eb4a9f0a61ebd67cb215003b4971d58d63241f4ca4678da1ccb26
+  data.tar.gz: 424fd9a3e3eac5e0c7f893cd3613e9f38a38d15ef63c19a843605c8651aef06a
 SHA512:
-  metadata.gz: 5f6583538a0632625e50bb48e6b48864455c2edb0987d434bda4f31edd0479b4805e72974feb0213496f6db77bb6e133661e09d448ac4423cd69bce5ddf07c9e
-  data.tar.gz: 7f3cf086ba6be74f724beed922eaf3fc8d8dc8dcabf24a599cce0ac5acbc1cc8ae927d0d10f0da8e287c00145cdbaa770141d7608e79966dcfb521bdde91c76e
+  metadata.gz: 5eb47165c378727eee8e7371defca71ffbb840a2e555b91a0e802bb75722888ec770938497a2b956ed8a7ecd874d0bd632ad8e76673afb93fc6a85fa1d784370
+  data.tar.gz: ed5876a86067c11dcbc1776ab8b11388783c58d758a815ba61bcaa991de0c606aff1c9d47c7fd4fc164fbb50f63878211385431121ce8b34e1c56cc4f56542b3

data/logstash-output-charrington.gemspec

@@ -1,6 +1,6 @@
 Gem::Specification.new do |s|
   s.name = 'logstash-output-charrington'
-  s.version = '0.3.20'
+  s.version = '0.3.21'
 
   s.licenses = ['Apache-2.0']
   s.homepage = 'https://gitlab.podium.com/engineering/analytics/logstash-output-charrington'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: logstash-output-charrington
 version: !ruby/object:Gem::Version
-  version: 0.3.20
+  version: 0.3.21
 platform: ruby
 authors:
 - dconger
@@ -176,14 +176,8 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- CHANGELOG.md
 - Gemfile
-- LICENSE.txt
-- README.md
-- THANKS.md
 - lib/com/zaxxer/HikariCP/2.7.2/HikariCP-2.7.2.jar
-- lib/commons-io/commons-io/2.4/commons-io-2.4.jar
-- lib/de/flapdoodle/embed/de.flapdoodle.embed.process/2.0.2/de.flapdoodle.embed.process-2.0.2.jar
 - lib/logstash-output-charrington_jars.rb
 - lib/logstash/outputs/charrington.rb
 - lib/logstash/outputs/charrington/alter_postgres_table.rb
@@ -195,16 +189,9 @@ files:
 - lib/logstash/outputs/charrington/service.rb
 - lib/logstash/outputs/charrington/transform_postgres.rb
 - lib/logstash/outputs/charrington/transform_redshift.rb
-- lib/net/java/dev/jna/jna-platform/4.0.0/jna-platform-4.0.0.jar
-- lib/net/java/dev/jna/jna/4.0.0/jna-4.0.0.jar
-- lib/org/apache/commons/commons-compress/1.10/commons-compress-1.10.jar
-- lib/org/apache/commons/commons-lang3/3.1/commons-lang3-3.1.jar
 - lib/org/apache/logging/log4j/log4j-api/2.6.2/log4j-api-2.6.2.jar
 - lib/org/apache/logging/log4j/log4j-slf4j-impl/2.6.2/log4j-slf4j-impl-2.6.2.jar
-- lib/org/postgresql/postgresql/42.2.5/postgresql-42.2.5.jar
 - lib/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar
-- lib/ru/yandex/qatools/embed/postgresql-embedded/2.10/postgresql-embedded-2.10.jar
-- lib/ru/yandex/qatools/embed/postgresql-embedded/2.8/postgresql-embedded-2.8.jar
 - logstash-output-charrington.gemspec
 - spec/charrington_spec_helper.rb
 - spec/logstash-output-charrington_test_jars.rb
@@ -239,7 +226,7 @@ requirements:
 - jar 'com.zaxxer:HikariCP', '2.7.2'
 - jar 'org.apache.logging.log4j:log4j-slf4j-impl', '2.6.2'
 rubyforge_project:
-rubygems_version: 2.7.6
+rubygems_version: 2.7.9
 signing_key:
 specification_version: 4
 summary: This plugin allows you to output to SQL, via JDBC

data/CHANGELOG.md

@@ -1,64 +0,0 @@
-# Change Log
-All notable changes to this project will be documented in this file, from 0.2.0.
-
-## [5.3.0] - 2017-11-08
-  - Adds configuration options `enable_event_as_json_keyword` and `event_as_json_keyword`
-  - Adds BigDecimal support
-  - Adds additional logging for debugging purposes (with thanks to @mlkmhd's work)
-
-## [5.2.1] - 2017-04-09
-  - Adds Array and Hash to_json support for non-sprintf syntax
-
-## [5.2.0] - 2017-04-01
-  - Upgrades HikariCP to latest
-  - Fixes HikariCP logging integration issues
-
-## [5.1.0] - 2016-12-17
-  - phoenix-thin fixes for issue #60
-
-## [5.0.0] - 2016-11-03
-  - logstash v5 support
-
-## [0.3.1] - 2016-08-28
-  - Adds connection_test configuration option, to prevent the connection test from occuring, allowing the error to be suppressed.
-    Useful for cockroachdb deployments. https://github.com/theangryangel/logstash-output-jdbc/issues/53 
-
-## [0.3.0] - 2016-07-24
-  - Brings tests from v5 branch, providing greater coverage
-  - Removes bulk update support, due to inconsistent behaviour
-  - Plugin now marked as threadsafe, meaning only 1 instance per-Logstash
-    - Raises default max_pool_size to match the default number of workers (1 connection per worker)
-
-## [0.2.10] - 2016-07-07
-  - Support non-string entries in statement array
-  - Adds backtrace to exception logging
-
-## [0.2.9] - 2016-06-29
-  - Fix NameError exception. 
-  - Moved log_jdbc_exception calls
-
-## [0.2.7] - 2016-05-29
-  - Backport retry exception logic from v5 branch
-  - Backport improved timestamp compatibility from v5 branch
-
-## [0.2.6] - 2016-05-02
-  - Fix for exception infinite loop
-
-## [0.2.5] - 2016-04-11
-### Added
-  - Basic tests running against DerbyDB
-  - Fix for converting Logstash::Timestamp to iso8601 from @hordijk
-
-## [0.2.4] - 2016-04-07
-  - Documentation fixes from @hordijk
-
-## [0.2.3] - 2016-02-16
-  - Bug fixes
-
-## [0.2.2] - 2015-12-30
-  - Bug fixes
-
-## [0.2.1] -  2015-12-22
-  - Support for connection pooling support added through HikariCP
-  - Support for unsafe statement handling (allowing dynamic queries)
-  - Altered exception handling to now count sequential flushes with exceptions thrown

data/LICENSE.txt

@@ -1,21 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2014
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

data/README.md

@@ -1,130 +0,0 @@
-# Logstash Output Charrington
-
-Logstash Output Plugin to handle batching messages and dynamically create and alter tables.
-
-Forked From [logstash-output-jdbc](https://github.com/theangryangel/logstash-output-jdbc)
-## Configure
-Example Logstash Config
-```bash
-input {
-  kafka {
-    auto_offset_reset => earliest
-    bootstrap_servers => "${LS_KAFKA_BOOTSTRAP_SERVERS:kafka:9092}"
-    codec => json
-    group_id => "winston_ls_pg_sinks_cg"
-    topics => [
-      "orwell.analytics.v1.json"
-    ]
-  }
-}
-
-filter {
-  if "_jsonparsefailure" in [tags] {
-    drop { }
-  }
-
-  ruby {
-    code => "
-      hash = event.to_hash
-      hash.each do |k,v|
-        if v == nil
-          event.remove(k)
-        end
-      end
-    "
-  }
-}
-
-output {
-  charrington {
-    driver_jar_path => "${DRIVER_PATH:/usr/share/logstash/vendor/jar/jdbc/postgresql-42.2.5.jar}"
-    connection_string => "jdbc:postgresql://${WINSTON_DATABASE_HOST}:5432/${WINSTON_DATABASE}?user=${WINSTON_DATABASE_USERNAME}&password=${WINSTON_DATABASE_PASSWORD}"
-    schema => "dea"
-  }
-
-  if "${LS_SHOW_DEBUG_OUTPUT}" == "true" {
-    stdout { codec => rubydebug }
-  }
-}
-```
-
-## Build & Publish
-
-**TLDR**
-```bash
-bundle exec rake build                          # build gem
-gem push logstash-output-charrington-x.x.x.gem  # publish
-```
-
-Everything in the [vendor](./vendor) directory (including `test-jars` if it exists) will be packaged with the generated gem.
-Avoid including unnecessary dependencies, by removing the vendor directory and regenerating it with the rake task. 
-See the first command below: 
- 
-```bash
-rm -r vendor && bundle exec rake install_jars       # clean vendor directory and re-install runtime jar dependencies
-gem build logstash-output-charrington.gemspec       # build
-gem push logstash-output-charrington-x.x.x.gem      # publish
-gem owner --add <email> logstash-output-charrington # add another authorized publisher
-```
-
-## Testing
-**NOTE:** Downloading test_jars requires maven and Java. You can set them up with asdf: [asdf-java](https://github.com/skotchpine/asdf-java), [asdf-maven](https://github.com/skotchpine/asdf-maven)
-* There is currently an [open bug](https://github.com/skotchpine/asdf-maven/pull/17) with how `asdf-maven` sets `$JAVA_HOME` on a mac. Edit your `~/.asdf/plugins/maven/bin/exec-env` file to fix the bug 
-
-```bash
-bundle exec rake install_test_jars      # install test runtime jar dependencies
-bundle exec rspec
-```
-
-#### Dangling Embedded Postgres
-**TLDR**
-```bash
-lsof -PiTCP -sTCP:LISTEN | grep postgres | awk '{print $2}' | xargs kill
-```
-Sometimes the embedded postgres JAR can leave dangling open processes. This is obvious when you have `export TEST_DEBUG=true`
-and the test output contains a stacktrace that starts with `[2019-06-21T15:18:06,734][ERROR][ru.yandex.qatools.embed.postgresql.PostgresProcess] Failed to read PID file (File '/var/folders/...`.
-To resolve this issue, check for dangling processes using `lsof -PiTCP -sTCP:LISTEN | grep postgres`, then you can kill the process ids
-
-#### Tracks Table
-**Create Statment for Driver is Postgres and Transform is Redshift**
- * See `@@redshift_tracks_columns` in [insert.rb](./lib/logstash/outputs/charrington/insert.rb) for column list
-```sql
-create table <schema>.tracks
-(
-	id varchar(512) not null constraint tracks_pkey primary key,
-	anonymous_id varchar(512),
-	app_name varchar(512),
-	context_campaign_content varchar(512)
-	context_campaign_medium varchar(512),
-	context_campaign_name varchar(512),
-	context_campaign_source varchar(512),
-	context_ip varchar(512),
-	context_library_name varchar(512),
-	context_library_version varchar(512),
-	context_page_path varchar(512),
-	context_page_referrer varchar(512),
-	context_page_search varchar(512),
-	context_page_title varchar(512),
-	context_page_url varchar(512),
-	context_user_agent varchar(512),
-	event varchar(512),
-	event_text varchar(512),
-	original_timestamp timestamp,
-	received_at timestamp,
-	segment_dedupe_id varchar(512),
-	sent_at timestamp,
-	timestamp timestamp,
-	user_id varchar(512),
-	user_uid varchar(512),
-	uuid bigint,
-	uuid_ts timestamp default ('now'::text)::timestamp without time zone,
-)
-
-```
-
-#### Resources
-* [logstash-devutils](https://rubygems.org/gems/logstash-devutils) gem with logstash helper methods and tools
-    * [spec_helper.rb source](https://github.com/elastic/logstash-devutils/blob/master/lib/logstash/devutils/rspec/spec_helper.rb)  
-    * [logstash_helpers.rb source](https://github.com/elastic/logstash-devutils/blob/master/lib/logstash/devutils/rspec/logstash_helpers.rb)
-* [logstash-output-file spec test](https://github.com/logstash-plugins/logstash-output-file/blob/master/spec/outputs/file_spec.rb)
-

data/THANKS.md

@@ -1,18 +0,0 @@
-logstash-output-jdbc is a project originally created by Karl Southern 
-(the_angry_angel), but there are a number of people that have contributed 
-or implemented key features over time. We do our best to keep this list 
-up-to-date, but you can also have a look at the nice contributor graphs
-produced by GitHub: https://github.com/theangryangel/logstash-output-jdbc/graphs/contributors
-
- * [hordijk](https://github.com/hordijk)
- * [dmitryakadiamond](https://github.com/dmitryakadiamond)
- * [MassimoSporchia](https://github.com/MassimoSporchia)
- * [ebuildy](https://github.com/ebuildy)
- * [kushtrimjunuzi](https://github.com/kushtrimjunuzi)
- * [josemazo](https://github.com/josemazo)
- * [aceoliver](https://github.com/aceoliver)
- * [roflmao](https://github.com/roflmao)
- * [onesuper](https://github.com/onesuper)
- * [phr0gz](https://github.com/phr0gz)
- * [jMonsinjon](https://github.com/jMonsinjon)
- * [mlkmhd](https://github.com/mlkmhd)