embulk-filter-mysql_lookup 0.1.6 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (14) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +79 -65
  3. data/build.gradle +1 -1
  4. data/classpath/embulk-filter-mysql_lookup-0.1.7.jar +0 -0
  5. data/example/mysql-filter-lookup_testing.yml.liquid +37 -0
  6. data/example/mysql-filter-lookup_testing_csv.yml.liquid +54 -0
  7. data/example/mysql-filter-lookup_testing_mssql.yml.liquid +39 -0
  8. data/example/mysql-filter-lookup_testing_oracle.yml.liquid +42 -0
  9. data/example/mysql-filter-lookup_testing_postgres.yml.liquid +42 -0
  10. data/src/main/java/org/embulk/filter/mysql_lookup/DatabaseConnection.java +32 -28
  11. data/src/main/java/org/embulk/filter/mysql_lookup/MysqlLookupFilterPlugin.java +21 -4
  12. metadata +8 -4
  13. data/classpath/embulk-filter-mysql_lookup-0.1.6.jar +0 -0
  14. data/example/config1.yml +0 -47
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b21673bee3d44790b92f529c7e231d0d59d5a840b9333969036cb54692273294
4
- data.tar.gz: 785a6ed85c5ed162116f643a32fa7912fbbfd5df28e47578e8723bf35f77356d
3
+ metadata.gz: 7486a3575cd34e562c8298dee178b17fc06cd2b1bb0446c84465874200a2fd55
4
+ data.tar.gz: 2f32f73302b28d4a6bcc57e3d69aaeb9dab5825d1d66428e00087e73c8db04f2
5
5
  SHA512:
6
- metadata.gz: 4d414ef92f46c1ff72dd0dd38a385d06e0854f722fdfec3442f906e950f5b78626c186ed79d030305bfb208062f11a8f1e81b0138abce7ba8c7e5b2f505c5b8d
7
- data.tar.gz: 87bec41710718ee63d33659e204d7925387277ab83c9405972a33b614649bcb2d2125143112bc1a1a293721ee69b73a345ace1710ff89b5a75bcb6758f6a6b93
6
+ metadata.gz: 15732f29be816f047cac06d2b9eff18466692e6900c0dd9854735314421fa7bb62fdb92db01cf963700575373c30edf4ba0a75e2134815524637fabe578f2af2
7
+ data.tar.gz: 98a5ac748904ee32de5e51afe3f610fde51082e9bb8357c1e070605cdafd85fcbf6493ec72ccdaa28512df02de375964bd008263952d7cff27a7a3e2a5e38f14
data/README.md CHANGED
@@ -30,93 +30,107 @@ An Embulk filter plugin for Lookup Transformation with MySQL database
30
30
  - **Name of column-1**: column name-1 from input file
31
31
  - **Name of column-2**: column name-2 from input file
32
32
  - **new_columns**: (New generated column names) (required)
33
- - **Name-1,Type-1**: Any Name, Type of the name (name: country_name, type: string)
34
- - **Name-2,Type-2**: Any Name, Type of the name (name: country_address, type: string) etc ...
33
+ - **Name-1,Type-1**: Any Name, Type of the name (name: pin, type: string)
34
+ - **Name-2,Type-2**: Any Name, Type of the name (name: gender, type: string)
35
+ - **Name-3,Type-3**: Any Name, Type of the name (name: phone_number, type: string) etc ...
35
36
  ## Example - columns
36
37
  ​
37
- Input1 for table 1 is as follows :-
38
- ​
38
+ Customer.csv for table 1 is as follows :-
39
+
39
40
  ```
40
- year country_code country_name literacy_rate
41
-
42
- 1990 1 India 80%
43
- 1993 2 USA 83%
44
- 1997 3 JAPAN
45
- 1999 4 China 72%
46
- 2000 5 Ukraine 68%
47
- 2002 6 Italy 79%
48
- 2004 7 UK 75%
49
- 2011 8 NULL 42%
41
+ id customer_name address email car_name company
42
+ 1 John Doe 123 Main St, Anytown USA john.doe@example.com Civic Honda
43
+ 2 Jane Smith 456 Elm St, Anytown USA jane.smith@example.com E-Class Mercedes-Benz
44
+ 3 Bob Johnson 789 Oak St, Anytown USA bob.johnson@example.com GLE-Class Mercedes-Benz
45
+ 4 Amanda Hernandez 999 Cedar St, Anytown USA amanda.hernandez@example.com 911 119
46
+ 5 Tom Brown 567 Pine St, Anytown USA tom.brown@example.com C-Class Mercedes-Benz
47
+ 6 Samantha Davis 890 Cedar St, Anytown USA samantha.davis@example.com Civic Honda
48
+ 7 Mike Wilson 1234 Spruce St, Anytown USA mike.wilson@example.com GLE-Class Mercedes-Benz
49
+ 8 Jason Brown 888 Pine St, Anytown USA jason.brown@example.com 911 Porsche
50
+ 9 David Rodriguez 9010 Oak St, Anytown USA david.rodriguez@example.com GLC-Class Mercedes-Benz
51
+ 10 Mark Davis 666 Spruce St, Anytown USA mark.davis@example.com C-Class Mercedes-Benz
52
+ 11 Chris Thompson 222 Cedar St, Anytown USA chris.thompson@example.com Cayenne Porsche
53
+ 12 Linda Young 555 Birch St, Anytown USA linda.young@example.com RAV4
54
+ 13 Kevin Hernandez 444 Maple St, Anytown USA kevin.hernandez@example.com 911 119
50
55
  ```
51
- ​
52
- Input2 for table 2 is as follows :-
53
- ​
56
+
57
+ Car.csv for table 2 is as follows :-
58
+
54
59
  ```
55
- id country_population country_address country_GDP
56
-
57
- 1 11.3 India 1.67
58
- 2 18.2 USA 16.72
59
- 3 30 JAPAN 5.00
60
- 4 4 China 9.33
61
- 5 57 Ukraine 1.08
62
- 6 63 Italy 2.068
63
- 7 17 UK 2.49
64
- 8 28 UAE 1.18
65
-
66
-
67
- Note: country_population is calculated in Billion and country_GDP is calculated in $USD Trillion
60
+ car_id model brand category fuel_capacity
61
+ 87 GLE-Class Mercedes-Benz SUV 80
62
+ 101 Cayenne Porsche SUV 75
63
+ 119 911 Porsche Sports Car 64
64
+ 205 Accord Honda Sedan 56
65
+ 334 Pilot Honda SUV 70
66
+ 434 CR-v Honda SUV 64
67
+ 559 C-Class Mercedes-Benz Sedan 66
68
+ 603 Civic Honda Sedan 42
69
+ 697 E-Class Mercedes-Benz Sedan 72
70
+ 812 GLC-Class Mercedes-Benz Sedan 68
71
+
72
+
68
73
  ```
69
- ​
74
+
70
75
  As shown in yaml below, columns mentioned in mapping_from will be mapped with columns mentioned in mapping_to
71
76
  ie:
72
77
 
73
- ​
74
- country_code : id
75
- country_name : country_address
78
+ car_name : model
79
+ company : brand
76
80
 
77
- After successful mapping an Output.csv file containing the columns mentioned in new_columns will be generated
78
- ​
79
- ​
81
+ After successful mapping an Output.csv file containing the columns mentioned in new_columns will be generated
80
82
 
81
83
  Output File generated :-
82
- ​
84
+
83
85
  ```
84
- year country_code country_name literacy_rate country_GDP country_population
85
-
86
- 1990 1 India 80% 1.67 11.3
87
- 1993 2 USA 83% 16.72 18.2
88
- 1997 3 JAPAN 5.00 30
89
- 1999 4 China 72% 9.33 4
90
- 2000 5 Ukraine 68% 1.08 57
91
- 2002 6 Italy 79% 2.068 63
92
- 2004 7 UK 75% 2.49 17
93
- 2011 8 NULL 42%
86
+ id customer_name address email car_name company car_id category fuel_capacity
87
+ 1 John Doe 123 Main St, Anytown USA john.doe@example.com Civic Honda 603 Sedan 42
88
+ 2 Jane Smith 456 Elm St, Anytown USA jane.smith@example.com E-Class Mercedes-Benz 697 Sedan 72
89
+ 3 Bob Johnson 789 Oak St, Anytown USA bob.johnson@example.com GLE-Class Mercedes-Benz 87 SUV 80
90
+ 4 Amanda Hernandez 999 Cedar St, Anytown USA amanda.hernandez@example.com 911 119 0
91
+ 5 Tom Brown 567 Pine St, Anytown USA tom.brown@example.com C-Class Mercedes-Benz 559 Sedan 66
92
+ 6 Samantha Davis 890 Cedar St, Anytown USA samantha.davis@example.com Civic Honda 603 Sedan 42
93
+ 7 Mike Wilson 1234 Spruce St, Anytown USA mike.wilson@example.com GLE-Class Mercedes-Benz 87 SUV 80
94
+ 8 Jason Brown 888 Pine St, Anytown USA jason.brown@example.com 911 Porsche 119 Sport Car 64
95
+ 9 David Rodriguez 9010 Oak St, Anytown USA david.rodriguez@example.com GLC-Class Mercedes-Benz 812 SUV 68
96
+ 10 Mark Davis 666 Spruce St, Anytown USA mark.davis@example.com C-Class Mercedes-Benz 559 Sedan 66
97
+ 11 Chris Thompson 222 Cedar St, Anytown USA chris.thompson@example.com Cayenne Porsche 101 SUV 75
98
+ 12 Linda Young 555 Birch St, Anytown USA linda.young@example.com RAV4 \N 0
99
+ 13 Kevin Hernandez 444 Maple St, Anytown USA kevin.hernandez@example.com 911 119 0
94
100
  ```
95
101
  ​
96
102
  ​
97
103
  ​
98
104
  ```yaml
99
- - type: mysql_lookup
100
- host: localhost
101
- port: 1433
102
- database: test
103
- tablename: country_details
104
- username: root
105
- password: root
106
- mapping_from:
107
- - country_code
108
- - country_name
109
- mapping_to:
110
- - id
111
- - country_address
112
- new_columns:
113
- - { name: country_GDP, type: string }
114
- - { name: country_population, type: string }
105
+ filters:
106
+ - type: mysql_lookup
107
+ host: localhost
108
+ port: 3306
109
+ database: test
110
+ table_name: car
111
+ username: root
112
+ password: 'passsword'
113
+ mapping_from:
114
+ - car_name
115
+ - company
116
+ mapping_to:
117
+ - model
118
+ - brand
119
+ new_columns:
120
+ - { name: car_id, type: string }
121
+ - { name: category, type: string }
122
+ - { name: fuel_capacity, type: string }
115
123
  ```
116
124
  ​
117
125
  Notes:
118
- 1. mapping_from attribute should be in same order as mentioned in input file.
126
+ 1. mapping_from attribute should be in the same order as mentioned in the input file.
119
127
  ​
128
+ 2. In case with JDBC plugin if any integer column returned as float/decimal then use to cast that column as long as below
129
+ ```
130
+ column_options:
131
+ id: {value_type: long}
132
+ ```
133
+ 3. Matching columns data types must be int,long and String
120
134
  ## Development
121
135
  ​
122
136
  Run example:
data/build.gradle CHANGED
@@ -13,7 +13,7 @@ configurations {
13
13
  provided
14
14
  }
15
15
 
16
- version = "0.1.6"
16
+ version = "0.1.7"
17
17
 
18
18
  sourceCompatibility = 1.8
19
19
  targetCompatibility = 1.8
data/example/mysql-filter-lookup_testing.yml.liquid ADDED
@@ -0,0 +1,37 @@
1
+ exec:
2
+ max_threads: 1
3
+ min_output_tasks: 1
4
+ in:
5
+ type: mysql
6
+ host: localhost
7
+ user: root
8
+ password: 'passsword'
9
+ database: test
10
+ table: customers
11
+ filters:
12
+ - type: mysql_lookup
13
+ host: localhost
14
+ port: 3306
15
+ database: test
16
+ table_name: car
17
+ username: root
18
+ password: 'passsword'
19
+ mapping_from:
20
+ - car_name
21
+ - company
22
+ mapping_to:
23
+ - model
24
+ - brand
25
+ new_columns:
26
+ - { name: car_id, type: string }
27
+ - { name: category, type: string }
28
+ - { name: fuel_capacity, type: string }
29
+
30
+ out:
31
+ type: mysql
32
+ host: localhost
33
+ user: root
34
+ password: 'passsword'
35
+ database: test
36
+ table: output
37
+ mode: truncate_insert
data/example/mysql-filter-lookup_testing_csv.yml.liquid ADDED
@@ -0,0 +1,54 @@
1
+ exec:
2
+ max_threads: 1
3
+ min_output_tasks: 1
4
+ in:
5
+ type: file
6
+ path_prefix: "..path../customer.csv"
7
+ parser:
8
+ charset: UTF-8
9
+ type: csv
10
+ delimiter: ','
11
+ quote: '"'
12
+ header_line: true
13
+ columns:
14
+ - { name: id, type: long }
15
+ - { name: customer_name, type: string }
16
+ - { name: address, type: string }
17
+ - { name: email, type: string }
18
+ - { name: car_name, type: string }
19
+ - { name: company, type: string }
20
+ filters:
21
+ - type: mysql_lookup
22
+ host: localhost
23
+ port: 3306
24
+ database: test
25
+ table_name: car
26
+ username: root
27
+ password: 'passsword'
28
+ mapping_from:
29
+ - car_name
30
+ - company
31
+ mapping_to:
32
+ - model
33
+ - brand
34
+ new_columns:
35
+ - { name: car_id, type: string }
36
+ - { name: category, type: string }
37
+ - { name: fuel_capacity, type: string }
38
+
39
+
40
+ out:
41
+ type: file
42
+ path_prefix: "..path../output.csv"
43
+ file_ext: csv
44
+ formatter:
45
+ type: csv
46
+ delimiter: ","
47
+ newline: CRLF
48
+ newline_in_field: LF
49
+ charset: UTF-8
50
+ quote_policy: MINIMAL
51
+ quote: '"'
52
+ escape: "\\"
53
+ null_string: "\\N"
54
+ default_timezone: 'UTC'
data/example/mysql-filter-lookup_testing_mssql.yml.liquid ADDED
@@ -0,0 +1,39 @@
1
+ exec:
2
+ max_threads: 1
3
+ min_output_tasks: 1
4
+ in:
5
+ type: sqlserver
6
+ host: Localhost
7
+ driver_path: "..path../mssql-jdbc-10.2.0.jre17.jar"
8
+ user: "user"
9
+ password: "password"
10
+ database: "test"
11
+ table: customers
12
+ filters:
13
+ - type: mysql_lookup
14
+ host: localhost
15
+ port: 3306
16
+ database: test
17
+ table_name: car
18
+ username: root
19
+ password: 'passsword'
20
+ mapping_from:
21
+ - car_name
22
+ - company
23
+ mapping_to:
24
+ - model
25
+ - brand
26
+ new_columns:
27
+ - { name: car_id, type: string }
28
+ - { name: category, type: string }
29
+ - { name: fuel_capacity, type: string }
30
+
31
+ out:
32
+ type: sqlserver
33
+ host: Localhost
34
+ driver_path: "..path../mssql-jdbc-10.2.0.jre17.jar"
35
+ user: "usert"
36
+ password: "password"
37
+ database: "test"
38
+ table: output_table
39
+ mode: truncate_insert
data/example/mysql-filter-lookup_testing_oracle.yml.liquid ADDED
@@ -0,0 +1,42 @@
1
+ exec:
2
+ max_threads: 1
3
+ min_output_tasks: 1
4
+ in:
5
+ type: jdbc
6
+ host: localhost
7
+ driver_path: "..path../ojdbc8.jar"
8
+ driver_class: 'oracle.jdbc.driver.OracleDriver'
9
+ url: jdbc:oracle:thin:@localhost:1521:orcl
10
+ user: MYUSER
11
+ password: ABCD
12
+ database: DEMO
13
+ table: customers
14
+ filters:
15
+ - type: mysql_lookup
16
+ host: localhost
17
+ port: 3306
18
+ database: test
19
+ table_name: car
20
+ username: root
21
+ password: 'passsword'
22
+ mapping_from:
23
+ - car_name
24
+ - company
25
+ mapping_to:
26
+ - model
27
+ - brand
28
+ new_columns:
29
+ - { name: car_id, type: string }
30
+ - { name: category, type: string }
31
+ - { name: fuel_capacity, type: string }
32
+ out:
33
+ type: jdbc
34
+ host: localhost
35
+ driver_path: "..path../ojdbc8.jar"
36
+ driver_class: 'oracle.jdbc.driver.OracleDriver'
37
+ url: jdbc:oracle:thin:@localhost:1521:orcl
38
+ user: MYUSER
39
+ password: ABCD
40
+ database: DEMO
41
+ table: output_table
42
+ mode: truncate_insert
data/example/mysql-filter-lookup_testing_postgres.yml.liquid ADDED
@@ -0,0 +1,42 @@
1
+ exec:
2
+ max_threads: 1
3
+ min_output_tasks: 1
4
+ in:
5
+ type: postgresql
6
+ host: localhost
7
+ port: 5432
8
+ user: postgres
9
+ password: 1234
10
+ schema: public
11
+ database: test
12
+ table: customer
13
+ filters:
14
+ - type: mysql_lookup
15
+ host: localhost
16
+ port: 3306
17
+ database: test
18
+ table_name: car
19
+ username: root
20
+ password: 'passsword'
21
+ mapping_from:
22
+ - car_name
23
+ - company
24
+ mapping_to:
25
+ - model
26
+ - brand
27
+ new_columns:
28
+ - { name: car_id, type: string }
29
+ - { name: category, type: string }
30
+ - { name: fuel_capacity, type: string }
31
+
32
+ out:
33
+ type: postgresql
34
+ host: localhost
35
+ port: 5432
36
+ database: test
37
+ user: postgres
38
+ password: 1234
39
+ schema: public
40
+ table: output_table
41
+ mode: truncate_insert
42
+
data/src/main/java/org/embulk/filter/mysql_lookup/DatabaseConnection.java CHANGED
@@ -66,6 +66,11 @@ public class DatabaseConnection {
66
66
  return mysqlJdbcDriver.get();
67
67
  }
68
68
 
69
+ if (driverPath.isPresent()) {
70
+ logger.info(
71
+ "\"driver_path\" is set to load the MySQL JDBC driver class \"{}\". Adding it to classpath.", className);
72
+ this.addDriverJarToClasspath(driverPath.get());
73
+ }
69
74
  try {
70
75
  // If the class is found from the ClassLoader of the plugin, that is prioritized the highest.
71
76
  final Class<? extends java.sql.Driver> found = loadJdbcDriverClassForName(className);
@@ -79,46 +84,45 @@ public class DatabaseConnection {
79
84
  return found;
80
85
  }
81
86
  catch (final ClassNotFoundException ex) {
82
- // Pass-through once.
83
- }
84
-
85
- if (driverPath.isPresent()) {
86
- logger.info(
87
- "\"driver_path\" is set to load the MySQL JDBC driver class \"{}\". Adding it to classpath.", className);
88
- this.addDriverJarToClasspath(driverPath.get());
87
+ //throw new ConfigException("The MySQL JDBC driver for the class \"" + className + "\" is not found.", ex);
89
88
  }
90
- else {
91
- final File root = this.findPluginRoot();
92
- final File driverLib = new File(root, "default_jdbc_driver");
93
- final File[] files = driverLib.listFiles(new FileFilter() {
94
- @Override
95
- public boolean accept(final File file)
96
- {
97
- return file.isFile() && file.getName().endsWith(".jar");
98
- }
99
- });
100
- if (files == null || files.length == 0) {
101
- throw new ConfigException(new ClassNotFoundException(
102
- "The MySQL JDBC driver for the class \"" + className + "\" is not found"
103
- + " in \"default_jdbc_driver\" (" + root.getAbsolutePath() + ")."));
104
- }
105
- for (final File file : files) {
106
- logger.info(
107
- "The MySQL JDBC driver for the class \"{}\" is expected to be found"
108
- + " in \"default_jdbc_driver\" at {}.", className, file.getAbsolutePath());
109
- this.addDriverJarToClasspath(file.getAbsolutePath());
89
+ final File root = this.findPluginRoot();
90
+ final File driverLib = new File(root, "default_jdbc_driver");
91
+ final File[] files = driverLib.listFiles(new FileFilter() {
92
+ @Override
93
+ public boolean accept(final File file)
94
+ {
95
+ return file.isFile() && file.getName().endsWith(".jar");
110
96
  }
97
+ });
98
+ if (files == null || files.length == 0) {
99
+ throw new ConfigException(new ClassNotFoundException(
100
+ "The MySQL JDBC driver for the class \"" + className + "\" is not found"
101
+ + " in \"default_jdbc_driver\" (" + root.getAbsolutePath() + ")."));
102
+ }
103
+ for (final File file : files) {
104
+ logger.info(
105
+ "The MySQL JDBC driver for the class \"{}\" is expected to be found"
106
+ + " in \"default_jdbc_driver\" at {}.", className, file.getAbsolutePath());
107
+ this.addDriverJarToClasspath(file.getAbsolutePath());
111
108
  }
112
109
 
113
110
  try {
114
- // Retrying to find the class from the ClassLoader of the plugin.
111
+ // If the class is found from the ClassLoader of the plugin, that is prioritized the highest.
115
112
  final Class<? extends java.sql.Driver> found = loadJdbcDriverClassForName(className);
116
113
  mysqlJdbcDriver.compareAndSet(null, found);
114
+
115
+ if (driverPath.isPresent()) {
116
+ logger.warn(
117
+ "\"driver_path\" is set while the MySQL JDBC driver class \"{}\" is found from the PluginClassLoader."
118
+ + " \"driver_path\" is ignored.", className);
119
+ }
117
120
  return found;
118
121
  }
119
122
  catch (final ClassNotFoundException ex) {
120
123
  throw new ConfigException("The MySQL JDBC driver for the class \"" + className + "\" is not found.", ex);
121
124
  }
125
+
122
126
  }
123
127
  }
124
128
 
data/src/main/java/org/embulk/filter/mysql_lookup/MysqlLookupFilterPlugin.java CHANGED
@@ -4,15 +4,18 @@ import org.embulk.config.*;
4
4
  import org.embulk.spi.*;
5
5
  import org.embulk.spi.time.Timestamp;
6
6
  import org.embulk.spi.type.Types;
7
-
8
7
  import java.sql.*;
9
8
  import java.time.Instant;
10
9
  import java.util.*;
11
-
10
+ import org.slf4j.Logger;
11
+ import org.slf4j.LoggerFactory;
12
12
  public class MysqlLookupFilterPlugin
13
13
  implements FilterPlugin {
14
+
15
+ private static final Logger logger = LoggerFactory.getLogger(MysqlLookupFilterPlugin.class);
14
16
  public interface PluginTask
15
17
  extends Task {
18
+
16
19
  @Config("host")
17
20
  public String getHost();
18
21
 
@@ -185,7 +188,8 @@ public class MysqlLookupFilterPlugin
185
188
  for (ColumnConfig columnConfig : task.getNewColumns().getColumns()) {
186
189
  columnConfigList.add(columnConfig);
187
190
  }
188
-
191
+ Set<String> unmatchedData = new LinkedHashSet<>();
192
+ List<String> keyColumns = task.getMappingFrom();
189
193
  while (reader.nextRecord()) {
190
194
 
191
195
  int colNum = 0;
@@ -218,6 +222,8 @@ public class MysqlLookupFilterPlugin
218
222
  List<String> matchedData = new ArrayList<>();
219
223
  if (keyValuePair.containsKey(key)) {
220
224
  matchedData = keyValuePair.get(key);
225
+ }else{
226
+ unmatchedData.add(key);
221
227
  }
222
228
 
223
229
  if (matchedData.size() == 0) {
@@ -233,7 +239,18 @@ public class MysqlLookupFilterPlugin
233
239
  }
234
240
  builder.addRecord();
235
241
  }
236
-
242
+ String info="\n--------------------Unmatched rows.....................\nMapping Key Columns: ";
243
+ for(int i=0;i<keyColumns.size();i++){
244
+ info+= keyColumns.get(i);
245
+ if(i!=keyColumns.size()-1){
246
+ info+=",";
247
+ }
248
+ }
249
+ info+="\n";
250
+ for(String key: unmatchedData){
251
+ info+= key+"\n";
252
+ }
253
+ logger.info(info);
237
254
  }
238
255
 
239
256
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: embulk-filter-mysql_lookup
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.6
4
+ version: 0.1.7
5
5
  platform: ruby
6
6
  authors:
7
7
  - InfoObjects Inc.
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-02-10 00:00:00.000000000 Z
11
+ date: 2023-03-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -49,12 +49,16 @@ files:
49
49
  - LICENSE
50
50
  - README.md
51
51
  - build.gradle
52
- - classpath/embulk-filter-mysql_lookup-0.1.6.jar
52
+ - classpath/embulk-filter-mysql_lookup-0.1.7.jar
53
53
  - classpath/mysql-connector-java-8.0.19.jar
54
54
  - classpath/protobuf-java-3.6.1.jar
55
55
  - config/checkstyle/checkstyle.xml
56
56
  - config/checkstyle/default.xml
57
- - example/config1.yml
57
+ - example/mysql-filter-lookup_testing.yml.liquid
58
+ - example/mysql-filter-lookup_testing_csv.yml.liquid
59
+ - example/mysql-filter-lookup_testing_mssql.yml.liquid
60
+ - example/mysql-filter-lookup_testing_oracle.yml.liquid
61
+ - example/mysql-filter-lookup_testing_postgres.yml.liquid
58
62
  - gradle/wrapper/gradle-wrapper.jar
59
63
  - gradle/wrapper/gradle-wrapper.properties
60
64
  - gradlew
data/example/config1.yml DELETED
@@ -1,47 +0,0 @@
1
- exec:
2
- max_threads: 2
3
- min_output_tasks: 1
4
- in:
5
- type: file
6
- path_prefix: C:\Users\AnkitKumar\Desktop\embulk-filter-mysqllookup\calendar.csv
7
- parser:
8
- type: csv
9
- columns:
10
- - { name: dim_calendar_key, type: long }
11
- - { name: year_number, type: long }
12
- - { name: quarter_number, type: long }
13
- - { name: attr_1, type: string }
14
- filters:
15
- - type: mysql_lookup
16
- host: localhost
17
- port: 3306
18
- driver_path: C:/Users/AnkitKumar/Desktop/embulk-plugins/mysql-connector-java-8.0.19.jar
19
- driver_class: com.mysql.cj.jdbc.Driver
20
- database: information
21
- table_name: country
22
- username: root
23
- password: root
24
- mapping_from:
25
- - quarter_number
26
- - attr_1
27
- mapping_to:
28
- - id
29
- - country_code
30
- new_columns:
31
- - { name: country_address, type: string }
32
- - { name: country_code,type: double}
33
- out:
34
- type: file
35
- path_prefix: C:/Users/AnkitKumar/Desktop/embulk-files/sample/output.csv
36
- file_ext: csv
37
- formatter:
38
- type: csv
39
- delimiter: "\t"
40
- newline: CRLF
41
- newline_in_field: LF
42
- charset: UTF-8
43
- quote_policy: MINIMAL
44
- quote: '"'
45
- escape: "\\"
46
- null_string: "\\N"
47
- default_timezone: 'UTC'