embulk-output-postgresql 0.2.4 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 7401277e85bce3f1f2370d0aaa62d9811d8f4173
-  data.tar.gz: 2b308d4f179ad2191d1f2cd9ca4c0e506bdc7c1c
+  metadata.gz: 20e8259bf0d8db2e0a6486cf6d6e8122a35eac4e
+  data.tar.gz: c51d55c5847e97b55862f85bc7d7c36fbfd825be
 SHA512:
-  metadata.gz: 6fb3a523e6717894d079460c8a2c850ada1ed48f0fa484e38d7a7d6164a9a6600991a9b904273743ce084eee9fa219d8014954f81ed2bf9b0fd20df95cb3a06d
-  data.tar.gz: ad686f50fa358911d4a86fb0de4433cb75723e656ae71cbb598e43e4aefd074fa8e64319f24adc3bec17a2d90accf7d561099ae5309af6243803341492d8fa74
+  metadata.gz: 77bddc402cbe94140137f501359842fd101206b26602908aba64f632c6b56a0e02c73e4bbc83c6a20b1a4047034fd95d5667a1ee0487c4792e1805724339ab1a
+  data.tar.gz: b8cc2770741d36bb12fb02868b164833f35d57f9d6015115af25f972b2a791e84cc86c4701521547d0004d377e39e4c171a89bcd5b30a6c1885231f85e7c2f77

data/README.md

@@ -1,43 +1,89 @@
-# PostgreSQL output plugins for Embulk
-
-PostgreSQL output plugins for Embulk loads records to PostgreSQL.
-
-## Overview
-
-* **Plugin type**: output
-* **Load all or nothing**: depnds on the mode:
-  * **insert**: no
-  * **replace**: yes
-* **Resume supported**: no
-
-## Configuration
-
-- **host**: database host name (string, required)
-- **port**: database port number (integer, default: 5432)
-- **user**: database login user name (string, required)
-- **password**: database login password (string, default: "")
-- **database**: destination database name (string, required)
-- **schema**: destination schema name (string, default: "public")
-- **table**: destination table name (string, required)
-- **mode**: "replace", "merge" or "insert" (string, required)
-- **batch_size**: size of a single batch insert (integer, default: 16777216)
-- **options**: extra connection properties (hash, default: {})
-
-### Example
-
-```yaml
-out:
-  type: postgresql
-  host: localhost
-  user: pg
-  password: ""
-  database: my_database
-  table: my_table
-  mode: insert
-```
-
-### Build
-
-```
-$ ./gradlew gem
-```
+# PostgreSQL output plugins for Embulk
+
+PostgreSQL output plugins for Embulk loads records to PostgreSQL.
+
+## Overview
+
+* **Plugin type**: output
+* **Load all or nothing**: depnds on the mode. see bellow.
+* **Resume supported**: depnds on the mode. see bellow.
+
+## Configuration
+
+- **host**: database host name (string, required)
+- **port**: database port number (integer, default: 5432)
+- **user**: database login user name (string, required)
+- **password**: database login password (string, default: "")
+- **database**: destination database name (string, required)
+- **schema**: destination schema name (string, default: "public")
+- **table**: destination table name (string, required)
+- **options**: extra connection properties (hash, default: {})
+- **mode**: "replace", "merge" or "insert" (string, required)
+- **batch_size**: size of a single batch insert (integer, default: 16777216)
+- **default_timezone**: If input column type (embulk type) is timestamp and destination column type is `string` or `nstring`, this plugin needs to format the timestamp into a string. This default_timezone option is used to control the timezone. You can overwrite timezone for each columns using column_options option. (string, default: `UTC`)
+- **column_options**: advanced: a key-value pairs where key is a column name and value is options for the column.
+  - **type**: type of a column when this plugin creates new tables (e.g. `VARCHAR(255)`, `INTEGER NOT NULL UNIQUE`). This used when this plugin creates intermediate tables (insert, truncate_insert and merge modes), when it creates the target table (insert_direct and replace modes), and when it creates nonexistent target table automatically. (string, default: depends on input column type. `BIGINT` if input column type is long, `BOOLEAN` if boolean, `DOUBLE PRECISION` if double, `CLOB` if string, `TIMESTAMP` if timestamp)
+  - **value_type**: This plugin converts input column type (embulk type) into a database type to build a INSERT statement. This value_type option controls the type of the value in a INSERT statement. (string, default: depends on input column type. Available values options are: `byte`, `short`, `int`, `long`, `double`, `float`, `boolean`, `string`, `nstring`, `date`, `time`, `timestamp`, `decimal`, `null`, `pass`)
+  - **timestamp_format**: If input column type (embulk type) is timestamp and value_type is `string` or `nstring`, this plugin needs to format the timestamp value into a string. This timestamp_format option is used to control the format of the timestamp. (string, default: `%Y-%m-%d %H:%M:%S.%6N`)
+  - **timezone**: If input column type (embulk type) is timestamp and value_type is `string` or `nstring`, this plugin needs to format the timestamp value into a string. And if the input column type is timestamp and value_type is `date`, this plugin needs to consider timezone. In those cases, this timezone option is used to control the timezone. (string, value of default_timezone option is used by default)
+
+### Modes
+
+* **insert**:
+  * Behavior: This mode writes rows to some intermediate tables first. If all those tasks run correctly, runs `INSERT INTO <target_table> SELECT * FROM <intermediate_table_1> UNION ALL SELECT * FROM <intermediate_table_2> UNION ALL ...` query.
+  * Transactional: Yes. This mode successfully writes all rows, or fails with writing zero rows.
+  * Resumable: Yes.
+* **insert_direct**:
+  * Behavior: This mode inserts rows to the target table directly.
+  * Transactional: No. If fails, the target table could have some rows inserted.
+  * Resumable: No.
+* **truncate_insert**:
+  * Behavior: Same with `insert` mode excepting that it truncates the target table right before the last `INSERT ...` query.
+  * Transactional: Yes.
+  * Resumable: Yes.
+* **merge**:
+  * Behavior: This mode writes rows to some intermediate tables first. If all those tasks run correctly, runs `INSERT INTO <target_table> SELECT * FROM <intermediate_table_1> UNION ALL SELECT * FROM <intermediate_table_2> UNION ALL ... ON DUPLICATE KEY UPDATE ...` query.
+  * Transactional: Yes.
+  * Resumable: Yes.
+* **replace**:
+  * Behavior: Same with `insert` mode excepting that it truncates the target table right before the last `INSERT ...` query.
+  * Transactional: Yes.
+  * Resumable: No.
+
+### Example
+
+```yaml
+out:
+  type: postgresql
+  host: localhost
+  user: pg
+  password: ""
+  database: my_database
+  table: my_table
+  mode: insert
+```
+
+Advanced configuration:
+
+```yaml
+out:
+  type: postgresql
+  host: localhost
+  user: pg
+  password: ""
+  database: my_database
+  table: my_table
+  options: {loglevel: 2}
+  mode: insert_direct
+  column_options:
+    my_col_1: {type: 'BIGSERIAL'}
+    my_col_3: {type: 'INT NOT NULL'}
+    my_col_4: {value_type: string, timestamp_format: `%Y-%m-%d %H:%M:%S %z`, timezone: '-0700'}
+    my_col_5: {type: 'DECIMAL(18,9)', value_type: pass}
+```
+
+### Build
+
+```
+$ ./gradlew gem
+```

data/build.gradle

@@ -1,7 +1,7 @@
-dependencies {
-    compile project(':embulk-output-jdbc')
-
-    compile 'org.postgresql:postgresql:9.4-1200-jdbc41'
-
-    testCompile project(':embulk-output-jdbc').sourceSets.test.output
-}
+dependencies {
+    compile project(':embulk-output-jdbc')
+
+    compile 'org.postgresql:postgresql:9.4-1200-jdbc41'
+
+    testCompile project(':embulk-output-jdbc').sourceSets.test.output
+}

data/lib/embulk/output/postgresql.rb

@@ -1,3 +1,3 @@
-Embulk::JavaPlugin.register_output(
-  :postgresql, "org.embulk.output.PostgreSQLOutputPlugin",
-  File.expand_path('../../../../classpath', __FILE__))
+Embulk::JavaPlugin.register_output(
+  :postgresql, "org.embulk.output.PostgreSQLOutputPlugin",
+  File.expand_path('../../../../classpath', __FILE__))

data/src/main/java/org/embulk/output/PostgreSQLOutputPlugin.java

@@ -1,142 +1,109 @@
-package org.embulk.output;
-
-import java.util.List;
-import java.util.Properties;
-import java.io.IOException;
-import java.sql.SQLException;
-
-import org.embulk.output.jdbc.setter.ColumnSetter;
-import org.embulk.output.postgresql.PostgresqlBatchUpsert;
-import org.embulk.spi.Exec;
-import org.embulk.config.Config;
-import org.embulk.config.ConfigDefault;
-import org.embulk.output.jdbc.AbstractJdbcOutputPlugin;
-import org.embulk.output.jdbc.BatchInsert;
-import org.embulk.output.postgresql.PostgreSQLOutputConnector;
-import org.embulk.output.postgresql.PostgreSQLCopyBatchInsert;
-import org.embulk.spi.PageReader;
-
-public class PostgreSQLOutputPlugin
-        extends AbstractJdbcOutputPlugin
-{
-    public interface PostgreSQLPluginTask
-            extends PluginTask
-    {
-        @Config("host")
-        public String getHost();
-
-        @Config("port")
-        @ConfigDefault("5432")
-        public int getPort();
-
-        @Config("user")
-        public String getUser();
-
-        @Config("password")
-        @ConfigDefault("\"\"")
-        public String getPassword();
-
-        @Config("database")
-        public String getDatabase();
-
-        @Config("schema")
-        @ConfigDefault("\"public\"")
-        public String getSchema();
-    }
-
-    @Override
-    protected Class<? extends PluginTask> getTaskClass()
-    {
-        return PostgreSQLPluginTask.class;
-    }
-
-    @Override
-    protected PostgreSQLOutputConnector getConnector(PluginTask task, boolean retryableMetadataOperation)
-    {
-        PostgreSQLPluginTask t = (PostgreSQLPluginTask) task;
-
-        String url = String.format("jdbc:postgresql://%s:%d/%s",
-                t.getHost(), t.getPort(), t.getDatabase());
-
-        Properties props = new Properties();
-        props.setProperty("user", t.getUser());
-        props.setProperty("password", t.getPassword());
-        props.setProperty("loginTimeout",   "300"); // seconds
-        props.setProperty("socketTimeout", "1800"); // seconds
-
-        // Enable keepalive based on tcp_keepalive_time, tcp_keepalive_intvl and tcp_keepalive_probes kernel parameters.
-        // Socket options TCP_KEEPCNT, TCP_KEEPIDLE, and TCP_KEEPINTVL are not configurable.
-        props.setProperty("tcpKeepAlive", "true");
-
-        // TODO
-        //switch t.getSssl() {
-        //when "disable":
-        //    break;
-        //when "enable":
-        //    props.setProperty("sslfactory", "org.postgresql.ssl.NonValidatingFactory");  // disable server-side validation
-        //when "verify":
-        //    props.setProperty("ssl", "true");
-        //    break;
-        //}
-
-        if (!retryableMetadataOperation) {
-            // non-retryable batch operation uses longer timeout
-            props.setProperty("loginTimeout",    "300");  // seconds
-            props.setProperty("socketTimeout", "28800");  // seconds
-        }
-
-        props.putAll(t.getOptions());
-
-        return new PostgreSQLOutputConnector(url, props, t.getSchema());
-    }
-
-    @Override
-    protected PluginPageOutput newPluginPageOutput(PageReader reader,
-                                                   BatchInsert batch, List<ColumnSetter> columnSetters,
-                                                   PluginTask task)
-    {
-        if (task.getMode().isMerge()) {
-            return new PostgresPluginPageOutput(reader, batch, columnSetters, task.getBatchSize());
-        }
-        return super.newPluginPageOutput(reader, batch, columnSetters, task);
-    }
-
-    public static class PostgresPluginPageOutput extends PluginPageOutput
-    {
-
-        public PostgresPluginPageOutput(PageReader pageReader, BatchInsert batch, List<ColumnSetter> columnSetters, int batchSize)
-        {
-            super(pageReader, batch, columnSetters, batchSize);
-        }
-
-        @Override
-        protected void handleColumnsSetters()
-        {
-            int size = columnSetters.size();
-            for (int i=0; i < size; i++) {
-                ColumnSetter columnSetter = columnSetters.get(i);
-                if (!columnSetter.getColumn().isPrimaryKey()) {
-                    columns.get(i).visit(columnSetter);
-                }
-            }
-            for (int i=0; i < size; i++) {
-                ColumnSetter columnSetter = columnSetters.get(i);
-                if (columnSetter.getColumn().isPrimaryKey()) {
-                    columns.get(i).visit(columnSetter);
-                }
-            }
-            for (int i=0; i < size; i++) {
-                columns.get(i).visit(columnSetters.get(i));
-            }
-        }
-
-    }
-
-    @Override
-    protected BatchInsert newBatchInsert(PluginTask task) throws IOException, SQLException
-    {
-        PostgreSQLOutputConnector connector = getConnector(task, true);
-        return task.getMode().isMerge() ? new PostgresqlBatchUpsert(connector) :
-                new PostgreSQLCopyBatchInsert(getConnector(task, true));
-    }
-}
+package org.embulk.output;
+
+import java.util.List;
+import java.util.Properties;
+import java.io.IOException;
+import java.sql.SQLException;
+import com.google.common.base.Optional;
+import com.google.common.collect.ImmutableSet;
+import org.embulk.config.Config;
+import org.embulk.config.ConfigDefault;
+import org.embulk.output.jdbc.AbstractJdbcOutputPlugin;
+import org.embulk.output.jdbc.BatchInsert;
+import org.embulk.output.postgresql.PostgreSQLOutputConnector;
+import org.embulk.output.postgresql.PostgreSQLCopyBatchInsert;
+
+public class PostgreSQLOutputPlugin
+        extends AbstractJdbcOutputPlugin
+{
+    public interface PostgreSQLPluginTask
+            extends PluginTask
+    {
+        @Config("host")
+        public String getHost();
+
+        @Config("port")
+        @ConfigDefault("5432")
+        public int getPort();
+
+        @Config("user")
+        public String getUser();
+
+        @Config("password")
+        @ConfigDefault("\"\"")
+        public String getPassword();
+
+        @Config("database")
+        public String getDatabase();
+
+        @Config("schema")
+        @ConfigDefault("\"public\"")
+        public String getSchema();
+    }
+
+    @Override
+    protected Class<? extends PluginTask> getTaskClass()
+    {
+        return PostgreSQLPluginTask.class;
+    }
+
+    @Override
+    protected Features getFeatures(PluginTask task)
+    {
+        return new Features()
+            .setMaxTableNameLength(30)
+            .setSupportedModes(ImmutableSet.of(Mode.INSERT, Mode.INSERT_DIRECT, Mode.MERGE, Mode.TRUNCATE_INSERT, Mode.REPLACE))
+            .setIgnoreMergeKeys(false);
+    }
+
+    @Override
+    protected PostgreSQLOutputConnector getConnector(PluginTask task, boolean retryableMetadataOperation)
+    {
+        PostgreSQLPluginTask t = (PostgreSQLPluginTask) task;
+
+        String url = String.format("jdbc:postgresql://%s:%d/%s",
+                t.getHost(), t.getPort(), t.getDatabase());
+
+        Properties props = new Properties();
+        props.setProperty("loginTimeout",   "300"); // seconds
+        props.setProperty("socketTimeout", "1800"); // seconds
+
+        // Enable keepalive based on tcp_keepalive_time, tcp_keepalive_intvl and tcp_keepalive_probes kernel parameters.
+        // Socket options TCP_KEEPCNT, TCP_KEEPIDLE, and TCP_KEEPINTVL are not configurable.
+        props.setProperty("tcpKeepAlive", "true");
+
+        // TODO
+        //switch t.getSssl() {
+        //when "disable":
+        //    break;
+        //when "enable":
+        //    props.setProperty("sslfactory", "org.postgresql.ssl.NonValidatingFactory");  // disable server-side validation
+        //when "verify":
+        //    props.setProperty("ssl", "true");
+        //    break;
+        //}
+
+        if (!retryableMetadataOperation) {
+            // non-retryable batch operation uses longer timeout
+            props.setProperty("loginTimeout",    "300");  // seconds
+            props.setProperty("socketTimeout", "28800");  // seconds
+        }
+
+        props.putAll(t.getOptions());
+
+        props.setProperty("user", t.getUser());
+        logger.info("Connecting to {} options {}", url, props);
+        props.setProperty("password", t.getPassword());
+
+        return new PostgreSQLOutputConnector(url, props, t.getSchema());
+    }
+
+    @Override
+    protected BatchInsert newBatchInsert(PluginTask task, Optional<List<String>> mergeKeys) throws IOException, SQLException
+    {
+        if (mergeKeys.isPresent()) {
+            throw new UnsupportedOperationException("PostgreSQL output plugin doesn't support 'merge_direct' mode. Use 'merge' mode instead.");
+        }
+        return new PostgreSQLCopyBatchInsert(getConnector(task, true));
+    }
+}

data/src/main/java/org/embulk/output/postgresql/AbstractPostgreSQLCopyBatchInsert.java

@@ -1,217 +1,215 @@
-package org.embulk.output.postgresql;
-
-import java.io.File;
-import java.io.FileOutputStream;
-import java.io.Writer;
-import java.io.BufferedWriter;
-import java.io.OutputStreamWriter;
-import java.io.IOException;
-import java.nio.charset.Charset;
-import java.math.BigDecimal;
-import java.sql.Date;
-import java.sql.Time;
-import java.sql.Timestamp;
-import java.sql.SQLException;
-import org.embulk.spi.Exec;
-import org.embulk.output.jdbc.JdbcSchema;
-import org.embulk.output.jdbc.BatchInsert;
-
-public abstract class AbstractPostgreSQLCopyBatchInsert
-        implements BatchInsert
-{
-    protected static final Charset FILE_CHARSET = Charset.forName("UTF-8");
-
-    protected static final String nullString = "\\N";
-    protected static final String newLineString = "\n";
-    protected static final String delimiterString = "\t";
-
-    protected File currentFile;
-    protected BufferedWriter writer;
-    protected int index;
-    protected int batchRows;
-
-    protected AbstractPostgreSQLCopyBatchInsert() throws IOException
-    {
-        this.index = 0;
-        openNewFile();
-    }
-
-    private File createTempFile() throws IOException
-    {
-        return File.createTempFile("embulk-output-postgres-copy-", ".tsv.tmp");  // TODO configurable temporary file path
-    }
-
-    protected File openNewFile() throws IOException
-    {
-        File newFile = createTempFile();
-        File oldFile = closeCurrentFile();
-        this.writer = openWriter(newFile);
-        currentFile = newFile;
-        return oldFile;
-    }
-
-    protected File closeCurrentFile() throws IOException
-    {
-        if(writer != null) {
-            writer.close();
-            writer = null;
-        }
-        return currentFile;
-    }
-
-    protected BufferedWriter openWriter(File newFile) throws IOException
-    {
-        return new BufferedWriter(new OutputStreamWriter(new FileOutputStream(newFile), FILE_CHARSET));
-    }
-
-    public int getBatchWeight()
-    {
-        long fsize = currentFile.length();
-        if (fsize > Integer.MAX_VALUE) {
-            return Integer.MAX_VALUE;
-        } else {
-            return (int) fsize;
-        }
-    }
-
-    public void finish() throws IOException, SQLException
-    {
-        closeCurrentFile();  // this is necessary to make getBatchWeight() work
-        if (getBatchWeight() != 0) {
-            flush();
-        }
-    }
-
-    public void add() throws IOException
-    {
-        writer.write(newLineString);
-        batchRows++;
-        index = 0;
-    }
-
-    private void appendDelimiter() throws IOException
-    {
-        if(index != 0) {
-            writer.write(delimiterString);
-        }
-        index++;
-    }
-
-    public void setNull(int sqlType) throws IOException
-    {
-        appendDelimiter();
-        writer.write(nullString);
-    }
-
-    public void setBoolean(boolean v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setByte(byte v) throws IOException
-    {
-        appendDelimiter();
-        setEscapedString(String.valueOf(v));
-    }
-
-    public void setShort(short v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setInt(int v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setLong(long v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setFloat(float v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setDouble(double v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setBigDecimal(BigDecimal v) throws IOException
-    {
-        appendDelimiter();
-        writer.write(String.valueOf(v));
-    }
-
-    public void setString(String v) throws IOException
-    {
-        appendDelimiter();
-        setEscapedString(v);
-    }
-
-    public void setNString(String v) throws IOException
-    {
-        appendDelimiter();
-        setEscapedString(v);
-    }
-
-    public void setBytes(byte[] v) throws IOException
-    {
-        appendDelimiter();
-        setEscapedString(String.valueOf(v));
-    }
-
-    public void setSqlDate(Date v, int sqlType) throws IOException
-    {
-        appendDelimiter();
-        writer.write(v.toString());
-    }
-
-    public void setSqlTime(Time v, int sqlType) throws IOException
-    {
-        appendDelimiter();
-        writer.write(v.toString());
-    }
-
-    public void setSqlTimestamp(Timestamp v, int sqlType) throws IOException
-    {
-        appendDelimiter();
-        writer.write(v.toString());
-    }
-
-    // Escape \, \n, \t, \r
-    // Remove \0
-    private void setEscapedString(String v) throws IOException{
-        for (char c : v.toCharArray()) {
-            String s;
-            switch (c) {
-            case '\\':
-                s = "\\\\";
-                break;
-            case '\n':
-                s = "\\n";
-                break;
-            case '\t':
-                s = "\\t";
-                break;
-            case '\r':
-                s = "\\r";
-                break;
-            case 0:
-                s = "";
-                break;
-            default:
-                s = String.valueOf(c);
-            }
-            writer.write(s);
-        }
-    }
-}
+package org.embulk.output.postgresql;
+
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.Writer;
+import java.io.BufferedWriter;
+import java.io.OutputStreamWriter;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.math.BigDecimal;
+import java.sql.Date;
+import java.sql.Time;
+import java.sql.Timestamp;
+import java.sql.SQLException;
+import org.embulk.output.jdbc.BatchInsert;
+
+public abstract class AbstractPostgreSQLCopyBatchInsert
+        implements BatchInsert
+{
+    protected static final Charset FILE_CHARSET = Charset.forName("UTF-8");
+
+    protected static final String nullString = "\\N";
+    protected static final String newLineString = "\n";
+    protected static final String delimiterString = "\t";
+
+    protected File currentFile;
+    protected BufferedWriter writer;
+    protected int index;
+    protected int batchRows;
+
+    protected AbstractPostgreSQLCopyBatchInsert() throws IOException
+    {
+        this.index = 0;
+        openNewFile();
+    }
+
+    private File createTempFile() throws IOException
+    {
+        return File.createTempFile("embulk-output-postgres-copy-", ".tsv.tmp");  // TODO configurable temporary file path
+    }
+
+    protected File openNewFile() throws IOException
+    {
+        File newFile = createTempFile();
+        File oldFile = closeCurrentFile();
+        this.writer = openWriter(newFile);
+        currentFile = newFile;
+        return oldFile;
+    }
+
+    protected File closeCurrentFile() throws IOException
+    {
+        if(writer != null) {
+            writer.close();
+            writer = null;
+        }
+        return currentFile;
+    }
+
+    protected BufferedWriter openWriter(File newFile) throws IOException
+    {
+        return new BufferedWriter(new OutputStreamWriter(new FileOutputStream(newFile), FILE_CHARSET));
+    }
+
+    public int getBatchWeight()
+    {
+        long fsize = currentFile.length();
+        if (fsize > Integer.MAX_VALUE) {
+            return Integer.MAX_VALUE;
+        } else {
+            return (int) fsize;
+        }
+    }
+
+    public void finish() throws IOException, SQLException
+    {
+        closeCurrentFile();  // this is necessary to make getBatchWeight() work
+        if (getBatchWeight() != 0) {
+            flush();
+        }
+    }
+
+    public void add() throws IOException
+    {
+        writer.write(newLineString);
+        batchRows++;
+        index = 0;
+    }
+
+    private void appendDelimiter() throws IOException
+    {
+        if(index != 0) {
+            writer.write(delimiterString);
+        }
+        index++;
+    }
+
+    public void setNull(int sqlType) throws IOException
+    {
+        appendDelimiter();
+        writer.write(nullString);
+    }
+
+    public void setBoolean(boolean v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setByte(byte v) throws IOException
+    {
+        appendDelimiter();
+        setEscapedString(String.valueOf(v));
+    }
+
+    public void setShort(short v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setInt(int v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setLong(long v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setFloat(float v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setDouble(double v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setBigDecimal(BigDecimal v) throws IOException
+    {
+        appendDelimiter();
+        writer.write(String.valueOf(v));
+    }
+
+    public void setString(String v) throws IOException
+    {
+        appendDelimiter();
+        setEscapedString(v);
+    }
+
+    public void setNString(String v) throws IOException
+    {
+        appendDelimiter();
+        setEscapedString(v);
+    }
+
+    public void setBytes(byte[] v) throws IOException
+    {
+        appendDelimiter();
+        setEscapedString(String.valueOf(v));
+    }
+
+    public void setSqlDate(Date v, int sqlType) throws IOException
+    {
+        appendDelimiter();
+        writer.write(v.toString());
+    }
+
+    public void setSqlTime(Time v, int sqlType) throws IOException
+    {
+        appendDelimiter();
+        writer.write(v.toString());
+    }
+
+    public void setSqlTimestamp(Timestamp v, int sqlType) throws IOException
+    {
+        appendDelimiter();
+        writer.write(v.toString());
+    }
+
+    // Escape \, \n, \t, \r
+    // Remove \0
+    private void setEscapedString(String v) throws IOException{
+        for (char c : v.toCharArray()) {
+            String s;
+            switch (c) {
+            case '\\':
+                s = "\\\\";
+                break;
+            case '\n':
+                s = "\\n";
+                break;
+            case '\t':
+                s = "\\t";
+                break;
+            case '\r':
+                s = "\\r";
+                break;
+            case 0:
+                s = "";
+                break;
+            default:
+                s = String.valueOf(c);
+            }
+            writer.write(s);
+        }
+    }
+}