fluent-plugin-sql 1.0.0.rc1 → 2.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 3e683b76d37276e4bbe276dc6148216c113e5267
4
- data.tar.gz: 374e957bfaf1f9eea818a6d3bc9c9b370c58604f
2
+ SHA256:
3
+ metadata.gz: 02d185dee05a4611437c046d0d279fb7299a874dd6c46a76d174a03c983b2f16
4
+ data.tar.gz: 71708e3128e2ec604ccccaf4ca7aeeb13c20853f2cdf0f72bbf4264125706ea2
5
5
  SHA512:
6
- metadata.gz: 1a71af14e2817621d22c85ac31a49d54d6b7f224b605ce2febaf09fbe369b435b33df779b23196e37ac6fb5d3c5a26d2039cb5aadf24dc90ec5d105ba4b283c6
7
- data.tar.gz: 593be077cfbc429c9502ad7b56c404cac24afbae65268d095c723339bb77a4645a5c4297d03a6cbf73dd269cdda69824d3e2e5d4d4b498406e8871c5e0fb775e
6
+ metadata.gz: 94aff71abbd5fea54eca7b2d9534fe2615992586add9f2ced691770a21aae95e45c4276ed6b076c5505eca634d3ff82b1eb321a13959021791704fde88691533
7
+ data.tar.gz: ad2d4fe75de3032775ef25c6d6c5062a43ec01fc715a9a4e59371022975ae421c9e957282da0c8777a159f873b04280f09bd9aa34e2dc3fd512ac54136599ed6
@@ -0,0 +1,69 @@
1
+ name: linux
2
+ on:
3
+ - push
4
+ - pull_request
5
+
6
+ jobs:
7
+ build:
8
+ runs-on: ${{ matrix.os }}
9
+ continue-on-error: ${{ matrix.experimental }}
10
+ strategy:
11
+ fail-fast: false
12
+ matrix:
13
+ ruby: [ '2.5', '2.6', '2.7', '3.0' ]
14
+ postgres: [ '9', '10', '11', '12', '13' ]
15
+ os:
16
+ - ubuntu-latest
17
+ experimental: [false]
18
+ include:
19
+ - ruby: head
20
+ postgres: '9'
21
+ os: ubuntu-latest
22
+ experimental: true
23
+ - ruby: head
24
+ postgres: '10'
25
+ os: ubuntu-latest
26
+ experimental: true
27
+ - ruby: head
28
+ postgres: '11'
29
+ os: ubuntu-latest
30
+ experimental: true
31
+ - ruby: head
32
+ postgres: '12'
33
+ os: ubuntu-latest
34
+ experimental: true
35
+ - ruby: head
36
+ postgres: '13'
37
+ os: ubuntu-latest
38
+ experimental: true
39
+ services:
40
+ postgres:
41
+ image: postgres:${{ matrix.postgres }}
42
+ ports:
43
+ - 5432:5432
44
+ env:
45
+ POSTGRES_USER: postgres
46
+ POSTGRES_PASSWORD: password
47
+ POSTGRES_DB: test
48
+ options: >-
49
+ --health-cmd pg_isready
50
+ --health-interval 10s
51
+ --health-timeout 5s
52
+ --health-retries 5
53
+ name: Ruby ${{ matrix.ruby }} with PostgreSQL ${{ matrix.postgres }} unit testing on ${{ matrix.os }}
54
+ steps:
55
+ - uses: actions/checkout@v2
56
+ - uses: ruby/setup-ruby@v1
57
+ with:
58
+ ruby-version: ${{ matrix.ruby }}
59
+ - name: prepare database
60
+ env:
61
+ PGPASSWORD: password
62
+ run: |
63
+ psql -h localhost -p 5432 -U postgres -c "CREATE ROLE fluentd WITH LOGIN ENCRYPTED PASSWORD 'fluentd';"
64
+ psql -h localhost -p 5432 -U postgres -c "CREATE DATABASE fluentd_test OWNER fluentd;"
65
+ - name: unit testing
66
+ run: |
67
+ gem install bundler rake
68
+ bundle install --jobs 4 --retry 3
69
+ bundle exec rake test
data/ChangeLog ADDED
@@ -0,0 +1,24 @@
1
+ Release 2.2.0 - 2021/04/20
2
+
3
+  * Upgrade ActiveRecord to 6.1
4
+
5
+ Release 2.1.0 - 2020/09/09
6
+
7
+  * in_sql: Use Fluent::EventTime instead of Integer to preserve millisecond precision
8
+
9
+ Release 2.0.0 - 2020/04/22
10
+
11
+ * out_sql: Remove v0.12 API code and use full v1 API. Change buffer format.
12
+
13
+ Release 1.1.1 - 2019/05/10
14
+
15
+ * out_sql: Support schema_search_path option of PostgreSQL
16
+
17
+ Release 1.1.0 - 2018/10/04
18
+
19
+ * Upgrade ActiveRecord to 5.1
20
+
21
+ Release 1.0.0 - 2018/04/06
22
+
23
+ * Support v1 API
24
+
data/README.md CHANGED
@@ -5,7 +5,7 @@
5
5
  This SQL plugin has two parts:
6
6
 
7
7
  1. SQL **input** plugin reads records from RDBMSes periodically. An example use case would be getting "diffs" of a table (based on the "updated_at" field).
8
- 2. SQL **output** plugin that writes records into RDBMes. An example use case would be aggregating server/app/sensor logs into RDBMS systems.
8
+ 2. SQL **output** plugin that writes records into RDBMSes. An example use case would be aggregating server/app/sensor logs into RDBMS systems.
9
9
 
10
10
  ## Requirements
11
11
 
@@ -14,7 +14,7 @@ This SQL plugin has two parts:
14
14
  | >= 1.0.0 | >= v0.14.4 | >= 2.1 |
15
15
  | < 1.0.0 | < v0.14.0 | >= 1.9 |
16
16
 
17
- NOTE: fluent-plugin-sql v1.0.0 is now RC. We will release stable v1.0.0 soon.
17
+ NOTE: fluent-plugin-sql v2's buffer format is different from v1. If you update the plugin to v2, don't reuse v1's buffer.
18
18
 
19
19
  ## Installation
20
20
 
@@ -25,6 +25,8 @@ You should install actual RDBMS driver gem together. `pg` gem for postgresql ada
25
25
 
26
26
  We recommend that mysql2 gem is higher than `0.3.12` and pg gem is higher than `0.16.0`.
27
27
 
28
+ If you use ruby 2.1, use pg gem 0.21.0 (< 1.0.0) because ActiveRecord 5.1.4 or earlier doesn't support Ruby 2.1.
29
+
28
30
  ## Input: How It Works
29
31
 
30
32
  This plugin runs following SQL periodically:
@@ -148,10 +150,12 @@ This plugin takes advantage of ActiveRecord underneath. For `host`, `port`, `dat
148
150
  * **username** RDBMS login user name
149
151
  * **password** RDBMS login password
150
152
  * **socket** RDBMS socket path
153
+ * **pool** A connection pool synchronizes thread access to a limited number of database connections
154
+ * **timeout** RDBMS connection timeout
151
155
  * **remove_tag_prefix** remove the given prefix from the events. See "tag_prefix" in "Input: Configuration". (optional)
152
156
 
153
157
  \<table\> sections:
154
158
 
155
159
  * **table** RDBM table name
156
160
  * **column_mapping**: [Required] Record to table schema mapping. The format is consists of `from:to` or `key` values are separated by `,`. For example, if set 'item_id:id,item_text:data,updated_at' to **column_mapping**, `item_id` field of record is stored into `id` column and `updated_at` field of record is stored into `updated_at` column.
157
- * **<table pattern>**: the pattern to which the incoming event's tag (after it goes through `remove_tag_prefix`, if given). The patterns should follow the same syntax as [that of <match>](http://docs.fluentd.org/articles/config-file#match-pattern-how-you-control-the-event-flow-inside-fluentd). **Exactly one <table> element must NOT have this parameter so that it becomes the default table to store data**.
161
+ * **\<table pattern\>**: the pattern to which the incoming event's tag (after it goes through `remove_tag_prefix`, if given). The patterns should follow the same syntax as [that of \<match\>](https://docs.fluentd.org/configuration/config-file#how-match-patterns-work). **Exactly one \<table\> element must NOT have this parameter so that it becomes the default table to store data**.
data/VERSION CHANGED
@@ -1 +1 @@
1
- 1.0.0.rc1
1
+ 2.2.0
@@ -4,12 +4,11 @@ $:.push File.expand_path('../lib', __FILE__)
4
4
  Gem::Specification.new do |gem|
5
5
  gem.name = "fluent-plugin-sql"
6
6
  gem.description = "SQL input/output plugin for Fluentd event collector"
7
- gem.homepage = "https://github.com/frsyuki/fluent-plugin-sql"
7
+ gem.homepage = "https://github.com/fluent/fluent-plugin-sql"
8
8
  gem.summary = gem.description
9
9
  gem.version = File.read("VERSION").strip
10
10
  gem.authors = ["Sadayuki Furuhashi"]
11
11
  gem.email = "frsyuki@gmail.com"
12
- gem.has_rdoc = false
13
12
  #gem.platform = Gem::Platform::RUBY
14
13
  gem.files = `git ls-files`.split("\n")
15
14
  gem.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
@@ -17,12 +16,12 @@ Gem::Specification.new do |gem|
17
16
  gem.require_paths = ['lib']
18
17
  gem.license = "Apache-2.0"
19
18
 
20
- gem.add_dependency "fluentd", [">= 0.12.17", "< 2"]
21
- gem.add_dependency 'activerecord', "~> 4.2"
22
- gem.add_dependency 'activerecord-import', "~> 0.7"
19
+ gem.add_dependency "fluentd", [">= 1.7.0", "< 2"]
20
+ gem.add_dependency 'activerecord', "~> 6.1"
21
+ gem.add_dependency 'activerecord-import', "~> 1.0"
23
22
  gem.add_development_dependency "rake", ">= 0.9.2"
24
- gem.add_development_dependency "test-unit", "~> 3.1.0"
23
+ gem.add_development_dependency "test-unit", "> 3.1.0"
25
24
  gem.add_development_dependency "test-unit-rr"
26
25
  gem.add_development_dependency "test-unit-notify"
27
- gem.add_development_dependency "pg"
26
+ gem.add_development_dependency "pg", '~> 1.0'
28
27
  end
@@ -39,6 +39,8 @@ module Fluent::Plugin
39
39
  config_param :password, :string, default: nil, secret: true
40
40
  desc 'RDBMS socket path'
41
41
  config_param :socket, :string, default: nil
42
+ desc 'PostgreSQL schema search path'
43
+ config_param :schema_search_path, :string, default: nil
42
44
 
43
45
  desc 'path to a file to store last rows'
44
46
  config_param :state_file, :string, default: nil
@@ -58,13 +60,16 @@ module Fluent::Plugin
58
60
  config_param :time_column, :string, default: nil
59
61
  config_param :primary_key, :string, default: nil
60
62
 
63
+ attr_reader :log
64
+
61
65
  def configure(conf)
62
66
  super
63
67
  end
64
68
 
65
- def init(tag_prefix, base_model, router)
69
+ def init(tag_prefix, base_model, router, log)
66
70
  @router = router
67
71
  @tag = "#{tag_prefix}.#{@tag}" if tag_prefix
72
+ @log = log
68
73
 
69
74
  # creates a model for this table
70
75
  table_name = @table
@@ -106,6 +111,17 @@ module Fluent::Plugin
106
111
  end
107
112
  end
108
113
 
114
+ # Make sure we always have a Fluent::EventTime object regardless of what comes in
115
+ def normalized_time(tv, now)
116
+ return Fluent::EventTime.from_time(tv) if tv.is_a?(Time)
117
+ begin
118
+ Fluent::EventTime.parse(tv.to_s)
119
+ rescue
120
+ log.warn "Message contains invalid timestamp, using current time instead (#{now.inspect})"
121
+ now
122
+ end
123
+ end
124
+
109
125
  # emits next records and returns the last record of emitted records
110
126
  def emit_next_records(last_record, limit)
111
127
  relation = @model
@@ -121,15 +137,13 @@ module Fluent::Plugin
121
137
  relation.each do |obj|
122
138
  record = obj.serializable_hash rescue nil
123
139
  if record
124
- if @time_column && tv = obj.read_attribute(@time_column)
125
- if tv.is_a?(Time)
126
- time = tv.to_i
140
+ time =
141
+ if @time_column && (tv = obj.read_attribute(@time_column))
142
+ normalized_time(tv, now)
127
143
  else
128
- time = Time.parse(tv.to_s).to_i rescue now
144
+ now
129
145
  end
130
- else
131
- time = now
132
- end
146
+
133
147
  me.add(time, record)
134
148
  last_record = record
135
149
  end
@@ -176,6 +190,7 @@ module Fluent::Plugin
176
190
  username: @username,
177
191
  password: @password,
178
192
  socket: @socket,
193
+ schema_search_path: @schema_search_path,
179
194
  }
180
195
 
181
196
  # creates subclass of ActiveRecord::Base so that it can have different
@@ -214,7 +229,7 @@ module Fluent::Plugin
214
229
  # ignore tables if TableElement#init failed
215
230
  @tables.reject! do |te|
216
231
  begin
217
- te.init(@tag_prefix, @base_model, router)
232
+ te.init(@tag_prefix, @base_model, router, log)
218
233
  log.info "Selecting '#{te.table}' table"
219
234
  false
220
235
  rescue => e
@@ -1,11 +1,12 @@
1
1
  require "fluent/plugin/output"
2
2
 
3
+ require 'active_record'
4
+ require 'activerecord-import'
5
+
3
6
  module Fluent::Plugin
4
7
  class SQLOutput < Output
5
8
  Fluent::Plugin.register_output('sql', self)
6
9
 
7
- DEFAULT_BUFFER_TYPE = "memory"
8
-
9
10
  helpers :inject, :compat_parameters, :event_emitter
10
11
 
11
12
  desc 'RDBMS host'
@@ -22,13 +23,19 @@ module Fluent::Plugin
22
23
  config_param :database, :string
23
24
  desc 'RDBMS socket path'
24
25
  config_param :socket, :string, default: nil
26
+ desc 'PostgreSQL schema search path'
27
+ config_param :schema_search_path, :string, default: nil
25
28
  desc 'remove the given prefix from the events'
26
29
  config_param :remove_tag_prefix, :string, default: nil
27
30
  desc 'enable fallback'
28
31
  config_param :enable_fallback, :bool, default: true
32
+ desc "size of ActiveRecord's connection pool"
33
+ config_param :pool, :integer, default: 5
34
+ desc "specifies the timeout to establish a new connection to the database before failing"
35
+ config_param :timeout, :integer, default: 5000
29
36
 
30
37
  config_section :buffer do
31
- config_set_default :@type, DEFAULT_BUFFER_TYPE
38
+ config_set_default :chunk_keys, ["tag"]
32
39
  end
33
40
 
34
41
  attr_accessor :tables
@@ -81,11 +88,12 @@ module Fluent::Plugin
81
88
  # @model.column_names
82
89
  end
83
90
 
84
- def import(chunk)
91
+ def import(chunk, output)
92
+ tag = chunk.metadata.tag
85
93
  records = []
86
- chunk.msgpack_each { |tag, time, data|
94
+ chunk.msgpack_each { |time, data|
87
95
  begin
88
- # format process should be moved to emit / format after supports error stream.
96
+ data = output.inject_values_to_record(tag, time, data)
89
97
  records << @model.new(@format_proc.call(data))
90
98
  rescue => e
91
99
  args = {error: e, table: @table, record: Yajl.dump(data)}
@@ -100,7 +108,7 @@ module Fluent::Plugin
100
108
  @log.warn "Got deterministic error. Fallback to one-by-one import", error: e
101
109
  one_by_one_import(records)
102
110
  else
103
- $log.warn "Got deterministic error. Fallback is disabled", error: e
111
+ @log.warn "Got deterministic error. Fallback is disabled", error: e
104
112
  raise e
105
113
  end
106
114
  end
@@ -142,8 +150,6 @@ module Fluent::Plugin
142
150
 
143
151
  def initialize
144
152
  super
145
- require 'active_record'
146
- require 'activerecord-import'
147
153
  end
148
154
 
149
155
  def configure(conf)
@@ -169,7 +175,10 @@ module Fluent::Plugin
169
175
  @tables << te
170
176
  end
171
177
  }
172
- @only_default = @tables.empty?
178
+
179
+ if @pool < @buffer_config.flush_thread_count
180
+ log.warn "connection pool size is smaller than buffer's flush_thread_count. Recommend to increase pool value", :pool => @pool, :flush_thread_count => @buffer_config.flush_thread_count
181
+ end
173
182
 
174
183
  if @default_table.nil?
175
184
  raise Fluent::ConfigError, "There is no default table. <table> is required in sql output"
@@ -187,6 +196,9 @@ module Fluent::Plugin
187
196
  username: @username,
188
197
  password: @password,
189
198
  socket: @socket,
199
+ schema_search_path: @schema_search_path,
200
+ pool: @pool,
201
+ timeout: @timeout,
190
202
  }
191
203
 
192
204
  @base_model = Class.new(ActiveRecord::Base) do
@@ -207,19 +219,6 @@ module Fluent::Plugin
207
219
  super
208
220
  end
209
221
 
210
- def emit(tag, es, chain)
211
- if @only_default
212
- super(tag, es, chain)
213
- else
214
- super(tag, es, chain, format_tag(tag))
215
- end
216
- end
217
-
218
- def format(tag, time, record)
219
- record = inject_values_to_record(tag, time, record)
220
- [tag, time, record].to_msgpack
221
- end
222
-
223
222
  def formatted_to_msgpack_binary
224
223
  true
225
224
  end
@@ -228,11 +227,12 @@ module Fluent::Plugin
228
227
  ActiveRecord::Base.connection_pool.with_connection do
229
228
 
230
229
  @tables.each { |table|
231
- if table.pattern.match(chunk.key)
232
- return table.import(chunk)
230
+ tag = format_tag(chunk.metadata.tag)
231
+ if table.pattern.match(tag)
232
+ return table.import(chunk, self)
233
233
  end
234
234
  }
235
- @default_table.import(chunk)
235
+ @default_table.import(chunk, self)
236
236
  end
237
237
  end
238
238
 
@@ -251,7 +251,7 @@ module Fluent::Plugin
251
251
  end
252
252
 
253
253
  def format_tag(tag)
254
- if @remove_tag_prefix
254
+ if tag && @remove_tag_prefix
255
255
  tag.gsub(@remove_tag_prefix, '')
256
256
  else
257
257
  tag
@@ -20,5 +20,12 @@ ActiveRecord::Schema.define(version: 20160225030107) do
20
20
  t.datetime "created_at", null: false
21
21
  t.datetime "updated_at", null: false
22
22
  end
23
+
24
+ create_table "messages_custom_time", force: :cascade do |t|
25
+ t.string "message"
26
+ t.datetime "created_at", null: false
27
+ t.datetime "updated_at", null: false
28
+ t.string "custom_time"
29
+ end
23
30
  end
24
31
 
@@ -18,6 +18,8 @@ class SqlInputTest < Test::Unit::TestCase
18
18
  username fluentd
19
19
  password fluentd
20
20
 
21
+ schema_search_path public
22
+
21
23
  tag_prefix db
22
24
 
23
25
  <table>
@@ -41,6 +43,7 @@ class SqlInputTest < Test::Unit::TestCase
41
43
  database: "fluentd_test",
42
44
  username: "fluentd",
43
45
  password: "fluentd",
46
+ schema_search_path: "public",
44
47
  tag_prefix: "db"
45
48
  }
46
49
  actual = {
@@ -50,6 +53,7 @@ class SqlInputTest < Test::Unit::TestCase
50
53
  database: d.instance.database,
51
54
  username: d.instance.username,
52
55
  password: d.instance.password,
56
+ schema_search_path: d.instance.schema_search_path,
53
57
  tag_prefix: d.instance.tag_prefix
54
58
  }
55
59
  assert_equal(expected, actual)
@@ -78,9 +82,9 @@ class SqlInputTest < Test::Unit::TestCase
78
82
  [d.events[2][1], "message 3"],
79
83
  ]
80
84
  actual = [
81
- [Time.parse(d.events[0][2]["updated_at"]).to_i, d.events[0][2]["message"]],
82
- [Time.parse(d.events[1][2]["updated_at"]).to_i, d.events[1][2]["message"]],
83
- [Time.parse(d.events[2][2]["updated_at"]).to_i, d.events[2][2]["message"]],
85
+ [Fluent::EventTime.parse(d.events[0][2]["updated_at"]), d.events[0][2]["message"]],
86
+ [Fluent::EventTime.parse(d.events[1][2]["updated_at"]), d.events[1][2]["message"]],
87
+ [Fluent::EventTime.parse(d.events[2][2]["updated_at"]), d.events[2][2]["message"]],
84
88
  ]
85
89
  assert_equal(expected, actual)
86
90
  end
@@ -0,0 +1,114 @@
1
+ require "helper"
2
+ require "fluent/test/driver/input"
3
+
4
+ class SqlInputCustomTimeTest < Test::Unit::TestCase
5
+ def setup
6
+ Fluent::Test.setup
7
+ end
8
+
9
+ def teardown
10
+ end
11
+
12
+ CONFIG = %[
13
+ adapter postgresql
14
+ host localhost
15
+ port 5432
16
+ database fluentd_test
17
+
18
+ username fluentd
19
+ password fluentd
20
+
21
+ schema_search_path public
22
+
23
+ tag_prefix db
24
+
25
+ <table>
26
+ table messages_custom_time
27
+ tag logs
28
+ update_column updated_at
29
+ time_column custom_time
30
+ </table>
31
+ ]
32
+
33
+ def create_driver(conf = CONFIG)
34
+ Fluent::Test::Driver::Input.new(Fluent::Plugin::SQLInput).configure(conf)
35
+ end
36
+
37
+ def test_configure
38
+ d = create_driver
39
+ expected = {
40
+ host: "localhost",
41
+ port: 5432,
42
+ adapter: "postgresql",
43
+ database: "fluentd_test",
44
+ username: "fluentd",
45
+ password: "fluentd",
46
+ schema_search_path: "public",
47
+ tag_prefix: "db"
48
+ }
49
+ actual = {
50
+ host: d.instance.host,
51
+ port: d.instance.port,
52
+ adapter: d.instance.adapter,
53
+ database: d.instance.database,
54
+ username: d.instance.username,
55
+ password: d.instance.password,
56
+ schema_search_path: d.instance.schema_search_path,
57
+ tag_prefix: d.instance.tag_prefix
58
+ }
59
+ assert_equal(expected, actual)
60
+ tables = d.instance.instance_variable_get(:@tables)
61
+ assert_equal(1, tables.size)
62
+ messages_custom_time = tables.first
63
+ assert_equal("messages_custom_time", messages_custom_time.table)
64
+ assert_equal("logs", messages_custom_time.tag)
65
+ end
66
+
67
+ def test_message
68
+ d = create_driver(CONFIG + "select_interval 1")
69
+
70
+ start_time = Fluent::EventTime.now
71
+
72
+ # Create one message with a valid timestamp containing milliseconds and a time zone
73
+ Message.create!(message: "message 1", custom_time: '2020-08-27 15:00:16.100758000 -0400')
74
+
75
+ # Create one message without a timestamp so that we can test auto-creation
76
+ Message.create!(message: "message 2 (no timestamp)", custom_time: nil)
77
+
78
+ # Create one message with an unparseable timestamp so that we can check that a valid
79
+ # one is auto-generated.
80
+ Message.create!(message: "message 3 (bad timestamp)", custom_time: 'foo')
81
+
82
+ d.end_if do
83
+ d.record_count >= 3
84
+ end
85
+ d.run(timeout: 5)
86
+
87
+ assert_equal("db.logs", d.events[0][0])
88
+ expected = [
89
+ [d.events[0][1], "message 1"],
90
+ [d.events[1][1], "message 2 (no timestamp)"],
91
+ [d.events[2][1], "message 3 (bad timestamp)"],
92
+ ]
93
+
94
+ actual = [
95
+ [Fluent::EventTime.parse(d.events[0][2]["custom_time"]), d.events[0][2]["message"]],
96
+ d.events[1][2]["message"],
97
+ d.events[2][2]["message"],
98
+ ]
99
+
100
+ assert_equal(expected[0], actual[0])
101
+
102
+ # Messages 2 and 3 should have the same messages but (usually) a slightly later
103
+ # timestamps because they are generated by the input plugin instead of the test
104
+ # code
105
+ [1,2].each do |i|
106
+ assert_equal(expected[i][1], actual[i])
107
+ assert_operator(expected[i][0], :>=, start_time)
108
+ end
109
+ end
110
+
111
+ class Message < ActiveRecord::Base
112
+ self.table_name = "messages_custom_time"
113
+ end
114
+ end
@@ -18,6 +18,8 @@ class SqlOutputTest < Test::Unit::TestCase
18
18
  username fluentd
19
19
  password fluentd
20
20
 
21
+ schema_search_path public
22
+
21
23
  remove_tag_prefix db
22
24
 
23
25
  <table>
@@ -39,8 +41,10 @@ class SqlOutputTest < Test::Unit::TestCase
39
41
  database: "fluentd_test",
40
42
  username: "fluentd",
41
43
  password: "fluentd",
44
+ schema_search_path: 'public',
42
45
  remove_tag_suffix: /^db/,
43
- enable_fallback: true
46
+ enable_fallback: true,
47
+ pool: 5
44
48
  }
45
49
  actual = {
46
50
  host: d.instance.host,
@@ -49,8 +53,10 @@ class SqlOutputTest < Test::Unit::TestCase
49
53
  database: d.instance.database,
50
54
  username: d.instance.username,
51
55
  password: d.instance.password,
56
+ schema_search_path: d.instance.schema_search_path,
52
57
  remove_tag_suffix: d.instance.remove_tag_prefix,
53
- enable_fallback: d.instance.enable_fallback
58
+ enable_fallback: d.instance.enable_fallback,
59
+ pool: d.instance.pool
54
60
  }
55
61
  assert_equal(expected, actual)
56
62
  assert_empty(d.instance.tables)
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-sql
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0.rc1
4
+ version: 2.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sadayuki Furuhashi
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-10-30 00:00:00.000000000 Z
11
+ date: 2021-04-20 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fluentd
@@ -16,7 +16,7 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 0.12.17
19
+ version: 1.7.0
20
20
  - - "<"
21
21
  - !ruby/object:Gem::Version
22
22
  version: '2'
@@ -26,7 +26,7 @@ dependencies:
26
26
  requirements:
27
27
  - - ">="
28
28
  - !ruby/object:Gem::Version
29
- version: 0.12.17
29
+ version: 1.7.0
30
30
  - - "<"
31
31
  - !ruby/object:Gem::Version
32
32
  version: '2'
@@ -36,28 +36,28 @@ dependencies:
36
36
  requirements:
37
37
  - - "~>"
38
38
  - !ruby/object:Gem::Version
39
- version: '4.2'
39
+ version: '6.1'
40
40
  type: :runtime
41
41
  prerelease: false
42
42
  version_requirements: !ruby/object:Gem::Requirement
43
43
  requirements:
44
44
  - - "~>"
45
45
  - !ruby/object:Gem::Version
46
- version: '4.2'
46
+ version: '6.1'
47
47
  - !ruby/object:Gem::Dependency
48
48
  name: activerecord-import
49
49
  requirement: !ruby/object:Gem::Requirement
50
50
  requirements:
51
51
  - - "~>"
52
52
  - !ruby/object:Gem::Version
53
- version: '0.7'
53
+ version: '1.0'
54
54
  type: :runtime
55
55
  prerelease: false
56
56
  version_requirements: !ruby/object:Gem::Requirement
57
57
  requirements:
58
58
  - - "~>"
59
59
  - !ruby/object:Gem::Version
60
- version: '0.7'
60
+ version: '1.0'
61
61
  - !ruby/object:Gem::Dependency
62
62
  name: rake
63
63
  requirement: !ruby/object:Gem::Requirement
@@ -76,14 +76,14 @@ dependencies:
76
76
  name: test-unit
77
77
  requirement: !ruby/object:Gem::Requirement
78
78
  requirements:
79
- - - "~>"
79
+ - - ">"
80
80
  - !ruby/object:Gem::Version
81
81
  version: 3.1.0
82
82
  type: :development
83
83
  prerelease: false
84
84
  version_requirements: !ruby/object:Gem::Requirement
85
85
  requirements:
86
- - - "~>"
86
+ - - ">"
87
87
  - !ruby/object:Gem::Version
88
88
  version: 3.1.0
89
89
  - !ruby/object:Gem::Dependency
@@ -118,23 +118,24 @@ dependencies:
118
118
  name: pg
119
119
  requirement: !ruby/object:Gem::Requirement
120
120
  requirements:
121
- - - ">="
121
+ - - "~>"
122
122
  - !ruby/object:Gem::Version
123
- version: '0'
123
+ version: '1.0'
124
124
  type: :development
125
125
  prerelease: false
126
126
  version_requirements: !ruby/object:Gem::Requirement
127
127
  requirements:
128
- - - ">="
128
+ - - "~>"
129
129
  - !ruby/object:Gem::Version
130
- version: '0'
130
+ version: '1.0'
131
131
  description: SQL input/output plugin for Fluentd event collector
132
132
  email: frsyuki@gmail.com
133
133
  executables: []
134
134
  extensions: []
135
135
  extra_rdoc_files: []
136
136
  files:
137
- - ".travis.yml"
137
+ - ".github/workflows/linux.yml"
138
+ - ChangeLog
138
139
  - Gemfile
139
140
  - Gemfile.v0.12
140
141
  - README.md
@@ -146,8 +147,9 @@ files:
146
147
  - test/fixtures/schema.rb
147
148
  - test/helper.rb
148
149
  - test/plugin/test_in_sql.rb
150
+ - test/plugin/test_in_sql_with_custom_time.rb
149
151
  - test/plugin/test_out_sql.rb
150
- homepage: https://github.com/frsyuki/fluent-plugin-sql
152
+ homepage: https://github.com/fluent/fluent-plugin-sql
151
153
  licenses:
152
154
  - Apache-2.0
153
155
  metadata: {}
@@ -162,17 +164,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
162
164
  version: '0'
163
165
  required_rubygems_version: !ruby/object:Gem::Requirement
164
166
  requirements:
165
- - - ">"
167
+ - - ">="
166
168
  - !ruby/object:Gem::Version
167
- version: 1.3.1
169
+ version: '0'
168
170
  requirements: []
169
- rubyforge_project:
170
- rubygems_version: 2.6.13
171
+ rubygems_version: 3.2.3
171
172
  signing_key:
172
173
  specification_version: 4
173
174
  summary: SQL input/output plugin for Fluentd event collector
174
- test_files:
175
- - test/fixtures/schema.rb
176
- - test/helper.rb
177
- - test/plugin/test_in_sql.rb
178
- - test/plugin/test_out_sql.rb
175
+ test_files: []
data/.travis.yml DELETED
@@ -1,32 +0,0 @@
1
- sudo: false
2
- language: ruby
3
-
4
- addons:
5
- postgresql: "9.4"
6
-
7
- rvm:
8
- - 2.0
9
- - 2.1
10
- - 2.2.4
11
- - 2.3.1
12
- - 2.4.1
13
- - ruby-head
14
-
15
- gemfile:
16
- - Gemfile
17
-
18
- before_install:
19
- - gem install bundler
20
-
21
- before_script:
22
- - psql -U postgres -c "CREATE ROLE fluentd WITH LOGIN ENCRYPTED PASSWORD 'fluentd';"
23
- - psql -U postgres -c "CREATE DATABASE fluentd_test OWNER fluentd;"
24
-
25
- script: bundle exec rake test
26
-
27
- matrix:
28
- allow_failures:
29
- - rvm: ruby-head
30
- exclude:
31
- - rvm: 2.0
32
- gemfile: Gemfile