RubyGems - embulk - Versions diffs - 0.8.39-java → 0.10.24-java - Mend

embulk 0.8.39-java → 0.10.24-java

Files changed (597) hide show

data/.gitignore DELETED

@@ -1,15 +0,0 @@
-target/
-build/
-pkg/
-*.iml
-*~
-._*
-.idea
-tmp/
-vendor/
-/classpath/
-/.bundle
-.yardoc
-/embulk-*.jar
-/.gradle
-/coverage

data/.ruby-version DELETED

	@@ -1 +0,0 @@
1	- 2.2.2

data/.travis.yml DELETED

@@ -1,28 +0,0 @@
-language: java
-jdk:
-  - oraclejdk8
-cache:
-  directories:  # run "travis cache --delete" to delete caches
-    - $HOME/.gradle
-sudo: false
-install:
-  - pip install --user sphinx
-script:
-  - ./gradlew --info $GRADLE_TASK # each Travis process configures GRADLE_TASK by env.matrix
-after_success:
-  - PATH="$HOME/.local/bin:$PATH" ./embulk-docs/push-gh-pages.sh
-env:
-  global:
-    - GIT_USER_NAME=travis
-    - GIT_USER_EMAIL=travis@embulk.org
-    - secure: K5qT2PcCP/40dNW+1H4NZ6y1+GAZbyP/lMQ1tSMsAICGkMQ/A+Mp5wtnIGIsAf6JGcJ1PvpCoLE1V6wKFL5fEwxi4SRcTnZTh9PkeAk8dgezOMoX4EqeZiQAYv4MM2zKL+Gr6QivjmRA7I5jrZCo8JyaA5XfQ7ygjICKNJy8NaE=
-  # To speed up the Travis build, the build can be broken up into several parts by env.matrix.
-  # They'll run in parallel. Each Travis process set environment variables declared in each
-  # env.matrix like GRADLE_TASK. See: https://docs.travis-ci.com/user/speeding-up-the-build/
-  matrix:
-    - GRADLE_TASK=':embulk-core:check'
-    - GRADLE_TASK=':embulk-standards:check'
-    - GRADLE_TASK=':embulk-jruby-strptime:check'
-    - GRADLE_TASK=':embulk-test:check'
-    - GRADLE_TASK=':embulk-cli:check'
-    - GRADLE_TASK='rubyTest'

data/COPYING DELETED

@@ -1,14 +0,0 @@
-Copyright (C) 2015 Sadayuki Furuhashi
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-       http://www.apache.org/licenses/LICENSE-2.0
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.

data/Gemfile.lock DELETED

@@ -1,30 +0,0 @@
-PATH
-  remote: .
-  specs:
-    embulk (0.8.13)
-      jruby-jars (= 9.1.13.0)
-GEM
-  remote: https://rubygems.org/
-  specs:
-    jruby-jars (9.1.13.0)
-    kramdown (1.5.0)
-    power_assert (0.2.2)
-    rake (10.4.2)
-    test-unit (3.0.9)
-      power_assert
-    yard (0.8.7.6)
-PLATFORMS
-  java
-  ruby
-DEPENDENCIES
-  embulk!
-  kramdown (~> 1.5.0)
-  rake (>= 0.10.0)
-  test-unit (~> 3.0.9)
-  yard (~> 0.8.7)
-BUNDLED WITH
-   1.13.2

data/README.md DELETED

@@ -1,229 +0,0 @@
-# What's Embulk?
-Embulk is a parallel bulk data loader that **helps data transfer between various storages, databases, NoSQL and cloud services**.
-**Embulk supports plugins** to add functions. You can [share the plugins](http://www.embulk.org/plugins/) to keep your custom scripts readable, maintainable, and reusable.
-[![Embulk](https://gist.githubusercontent.com/frsyuki/f322a77ee2766a508ba9/raw/e8539b6b4fda1b3357e8c79d3966aa8148dbdbd3/embulk-overview.png)](http://www.slideshare.net/frsyuki/embuk-making-data-integration-works-relaxed/12)
-[Embulk, an open-source plugin-based parallel bulk data loader](http://www.slideshare.net/frsyuki/embuk-making-data-integration-works-relaxed) at Slideshare
-# Document
-Embulk documents: http://www.embulk.org/docs/
-# Mailing list
-* [Embulk-announce](https://groups.google.com/forum/#!forum/embulk-announce): Embulk core members post important updates such as **key releases**, **compatibility information**, and **feedback requests to users**.
-## Quick Start
-### Linux & Mac & BSD
-Embulk is a Java application. Please make sure that [Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html) is installed.
-Following 4 commands install embulk to your home directory:
-```sh
-curl --create-dirs -o ~/.embulk/bin/embulk -L "https://dl.embulk.org/embulk-latest.jar"
-chmod +x ~/.embulk/bin/embulk
-echo 'export PATH="$HOME/.embulk/bin:$PATH"' >> ~/.bashrc
-source ~/.bashrc
-```
-Next step: [Running example in 4 commands](#running-example)
-### Windows
-Embulk is a Java application. Please make sure that [Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html) is installed.
-You can download `embulk.bat` using this command on cmd.exe or PowerShell.exe:
-```
-PowerShell -Command "& {Invoke-WebRequest http://dl.embulk.org/embulk-latest.jar -OutFile embulk.bat}"
-```
-Next step: [Running example in 4 commands](#running-example)
-### Running example
-`embulk example` command generates a sample CSV file so that you can try embulk quickly:
-```
-embulk example ./try1
-embulk guess   ./try1/seed.yml -o config.yml
-embulk preview config.yml
-embulk run     config.yml
-```
-Next step: [Using plugins](#using-plugins)
-### Using plugins
-You can use plugins to load data from/to various systems and file formats. Here is the list of publicly released plugins: [list of plugins by category](http://www.embulk.org/plugins/).
-An example is [embulk-output-command](https://github.com/embulk/embulk-output-command) plugin. It executes an external command to output the records.
-To install plugins, you can use `embulk gem install <name>` command:
-```
-embulk gem install embulk-output-command
-embulk gem list
-```
-Embulk bundles some built-in plugins such as `embulk-encoder-gzip` or `embulk-formatter-csv`. You can use those plugins with following configuration file:
-```yaml
-in:
-  type: file
-  path_prefix: "./try1/csv/sample_"
-  ...
-out:
-  type: command
-  command: "cat - > task.$INDEX.$SEQID.csv.gz"
-  encoders:
-    - {type: gzip}
-  formatter:
-    type: csv
-```
-### Resuming a failed transaction
-Embulk supports resuming failed transactions.
-To enable resuming, you need to start transaction with `-r PATH` option:
-```
-embulk run config.yml -r resume-state.yml
-```
-If the transaction fails, embulk stores state some states to the yaml file. You can retry the transaction using exactly same command:
-```
-embulk run config.yml -r resume-state.yml
-```
-If you give up on resuming the transaction, you can use `embulk cleanup` subcommand to delete intermediate data:
-```
-embulk cleanup config.yml -r resume-state.yml
-```
-### Using plugin bundle
-`embulk mkbundle` subcommand creates a isolated bundle of plugins. You can install plugins (gems) to the bundle directory instead of ~/.embulk directory. This makes it easy to manage versions of plugins.
-To use the bundle, add `-b <bundle_dir>` option to `guess`, `preview`, or `run` subcommand. `embulk mkbundle` also generates some example plugins to \<bundle_dir>/embulk/\*.rb directory.
-See the generated \<bundle_dir>/Gemfile file how to plugin bundles work.
-```
-embulk mkbundle ./embulk_bundle  # please edit ./embulk_bundle/Gemfile to add plugins. Detailed usage is written in the Gemfile
-embulk guess -b ./embulk_bundle ...
-embulk run   -b ./embulk_bundle ...
-```
-## Use cases
-* [Scheduled bulk data loading to Elasticsearch + Kibana 5 from CSV files](http://www.embulk.org/docs/recipe/scheduled-csv-load-to-elasticsearch-kibana5.html)
-For further details, visit [Embulk documentation](http://www.embulk.org/docs/).
-## Upgrading to the latest version
-Following command updates embulk itself to the latest released version.
-```sh
-embulk selfupdate
-```
-Following command updates embulk itself to the specific released version.
-```sh
-embulk selfupdate x.y.z
-```
-Older versions are available at [dl.embulk.org](http://dl.embulk.org).
-## Embulk Development
-### Build
-```
-./gradlew cli  # creates pkg/embulk-VERSION.jar
-./gradlew gem  # creates pkg/embulk-VERSION.gem
-```
-You can see JaCoCo's test coverage report at `${project}/build/reports/tests/index.html`
-You can see Findbug's report at `${project}/build/reports/findbug/main.html`  # FIXME coverage information is not included somehow
-You can use `classpath` task to use `bundle exec ./bin/embulk` for development:
-```
-./gradlew -t classpath  # -x test: skip test
-./bin/embulk
-```
-To deploy artifacts to your local maven repository at ~/.m2/repository/:
-```
-./gradlew install
-```
-To compile the source code of embulk-core project only:
-```
-./gradlew :embulk-core:compileJava
-```
-Task `dependencies` shows dependency tree of embulk-core project:
-```
-./gradlew :embulk-core:dependencies
-```
-### Update JRuby
-Task `updateJRuby` updates JRuby version of embulk-core project.
-This is an example to update JRuby to `9.1.13.0`.
-```
-./gradlew updateJRuby -Pto=9.1.13.0
-```
-### Documents
-Embulk uses Sphinx, YARD (Ruby API) and JavaDoc (Java API) for document generation.
-```
-brew install python
-pip install sphinx
-gem install yard
-./gradlew site
-# documents are: embulk-docs/build/html
-```
-### Release
-You need to add your bintray account information to ~/.gradle/gradle.properties
-```
-bintray_user=(bintray user name)
-bintray_api_key=(bintray api key)
-```
-Run following commands and follow its instruction:
-```
-./gradlew setVersion -Pto=$VERSION
-```
-```
-./gradlew releaseCheck
-./gradlew clean cli gem && ./gradlew release
-git commit -am v$VERSION
-git tag v$VERSION
-```
-See also:
-* [Bintray](https://bintray.com)
-* [How to acquire bintray API Keys](https://bintray.com/docs/usermanual/interacting/interacting_editingyouruserprofile.html#anchorAPIKEY)

data/Rakefile DELETED

@@ -1,26 +0,0 @@
-#!/usr/bin/env rake
-require 'bundler/gem_tasks'
-require 'rake/testtask'
-require 'rake/clean'
-require 'bundler'
-require 'yard'
-desc 'Generate YARD document'
-YARD::Rake::YardocTask.new(:doc) do |t|
-  t.files   = ['lib/**/*.rb']
-  t.options = %w[-M kramdown]
-  t.options << '--debug' << '--verbose' if $trace
-end
-desc "Clean java code and copy files"
-task :clean do
-  sh "./gradlew clean"
-end
-desc "Build and copy jar files to ./classpath"
-task :classpath do
-  sh "./gradlew classpath"
-end
-task :default => :classpath

data/appveyor.yml DELETED

@@ -1,28 +0,0 @@
-version: "{branch} {build}"
-build:
-  verbosity: detailed
-install:
- - if not exist C:\Users\appveyor\handle mkdir C:\Users\appveyor\handle
- - if not exist C:\Users\appveyor\handle\handle.exe appveyor DownloadFile "https://live.sysinternals.com/handle.exe" -FileName "C:\Users\appveyor\handle\handle.exe"
-# To avoid call-selfrun.bat FileNotFound exception, it installs and uses handle.exe as workaround provided by AppVeyor support.
-# see http://help.appveyor.com/discussions/problems/5975-the-process-cannot-access-the-file-because-it-is-being-used-by-another-process
-build_script:
-  - C:\Users\appveyor\handle\handle.exe -a -u C:\projects\embulk\embulk-cli\build\classes\test\org\embulk\cli\call-selfrun.bat -nobanner
-  - gradlew.bat --info --no-daemon check rubyTest
-cache:
-  - C:\Users\appveyor\.gradle
-  - C:\Users\appveyor\handle
-environment:
-  matrix:
-  - JAVA_HOME: C:\Program Files\Java\jdk1.8.0
-  # - JAVA_HOME: C:\Program Files (x86)\Java\jdk1.8.0
-test: off
-matrix:
-  fast_finish: true

data/bin/embulk DELETED

@@ -1,139 +0,0 @@
-#!/usr/bin/env ruby
-if RUBY_PLATFORM =~ /java/i
-  STDERR.puts "################################################################################"
-  STDERR.puts "[WARN] embulk.gem is deprecated, and will be removed from v0.9."
-  STDERR.puts "[WARN] embulk.gem's bin/embulk will be also removed from v0.9."
-  STDERR.puts "[WARN] Use the jar version installed from http://dl.embulk.org/ instead."
-  STDERR.puts "[WARN] See the issue and comment at: https://github.com/embulk/embulk/issues/628"
-  STDERR.puts "################################################################################"
-  STDERR.puts ""
-  require 'java'
-  if ENV['EMBULK_BIN_ENABLE_BUNDLE'] == File.expand_path(__FILE__)
-    # bin/embulk is started by CRuby (embulk gem for CRuby is installed), and re-run by JRuby at the end of the script.
-    # Bundler may be enabled in this path.
-    ENV.delete('EMBULK_BIN_ENABLE_BUNDLE')
-  else
-    # bin/embulk is started by JRuby (embulk gem for JRuby is installed).
-    # Bundler must be disabled in this path not to bother the JRuby's bundler.
-    # org.embulk.EmbulkRunner processes the -b or --bundle option. The option should be removed to disable Bundler.
-    # Search for -b or --bundle, and just remove it.
-    bundle_option_index = ARGV.find_index {|arg| arg == '-b' || arg == '--bundle' }
-    if bundle_option_index
-      STDERR.puts "-b or --bundle options do not work when Embulk is triggered from command line by JRuby."
-      ARGV.slice!(bundle_option_index, 2)[1]
-    end
-    $LOAD_PATH << File.expand_path('../lib', File.dirname(__FILE__))
-    Dir.glob("#{File.expand_path('../classpath', File.dirname(__FILE__))}/*").each do |f_jar|
-      $CLASSPATH << f_jar
-    end
-  end
-  require 'embulk/version'
-  embulk_run = Java::org.embulk.cli.EmbulkRun.new(Embulk::VERSION_INTERNAL)
-  embulk_run.run(Java::JavaUtil::ArrayList.new(ARGV), Java::JavaUtil::ArrayList.new)
-  exit 0
-end
-java_args = []
-jruby_args = []
-default_optimize = false
-overwrite_optimize = nil
-until ARGV.empty?
-  v = ARGV[0]
-  case v
-  when /-E(.*)/
-    external_script = v[2..-1]
-    STDERR.puts "Running an external Ruby script: #{external_script}"
-    load external_script
-    ARGV.shift
-  when "-J+O"
-    overwrite_optimize = true
-    ARGV.shift
-  when "-J-O"
-    overwrite_optimize = false
-    ARGV.shift
-  when "-J"
-    prop_path = ARGV[1]
-    unless prop_path
-      STDERR.puts "-J option requires an option"
-      exit 1
-    end
-    props = File.read(prop_path)
-    java_props = props.split("\n").reject {|prop| prop.strip.empty? }
-    java_args = java_props + java_args
-    ARGV.shift
-    ARGV.shift
-  when /-J(.*)/
-    java_args << v[2..-1]
-    ARGV.shift
-  when /-R(.*)/
-    jruby_args << v[2..-1]
-    ARGV.shift
-  when "run"
-    default_optimize = true
-    break
-  else
-    break
-  end
-end
-# TODO cygwin check
-cygwin = false
-java_cmd = ENV['JAVACMD']
-unless java_cmd
-  if java_home = ENV['JAVA_HOME']
-    if cygwin
-      java_cmd = "#{`cygpath -u "#{java_home}"`.strip}/bin/java"
-    else
-      java_cmd = "#{java_home}/bin/java"
-    end
-  else
-    java_cmd = "java"
-  end
-end
-if overwrite_optimize == true || (default_optimize == true && overwrite_optimize != false)
-  java_args = %w[-XX:+AggressiveOpts -XX:+UseConcMarkSweepGC] + java_args
-else
-  java_args = %w[-XX:+AggressiveOpts -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Xverify:none] + java_args
-end
-# Disable bundler options of CRuby before starting JRuby
-ENV.delete('GEM_HOME')
-ENV.delete('GEM_PATH')
-ENV.delete('BUNDLE_GEMFILE')
-ENV.delete('BUNDLE_BIN_PATH')
-ENV.delete('RUBYLIB')
-ENV.delete('RUBYOPT')
-env = {
-  'EMBULK_BIN_ENABLE_BUNDLE' => File.expand_path(__FILE__)
-}
-begin
-  require 'jruby-jars'
-rescue LoadError => e
-  STDERR.puts  "Could not load jruby-jar.gem. Did you run with \"bundle exec\"?"
-  raise e
-end
-jruby_cp = "#{File.dirname(JRubyJars.core_jar_path)}/*"
-embulk_cp = "#{File.expand_path('../../classpath', __FILE__)}/*"  # bundler is included in embulk-core.jar
-# java ... -jar ruby-complete.jar bin/embulk "$@"
-cmdline = [java_cmd]
-cmdline.concat java_args
-cmdline << '-cp' << [jruby_cp, embulk_cp].join(File::PATH_SEPARATOR)
-cmdline << 'org.jruby.Main'
-cmdline.concat jruby_args
-cmdline << __FILE__
-cmdline.concat ARGV
-exec env, *cmdline
-exit 127