s3_website 1.8.2 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 42a545379161c7267e94d88d44653f3f80d07b50
-  data.tar.gz: 5e426809c3fca74b91ebac4ff31bb2ea32cbb516
+  metadata.gz: 5e76d45fa334b0d884f2a0db53a4d87e1fee84dd
+  data.tar.gz: 04992267d1790b3bdc5a0e26b1f234ad5975c8ca
 SHA512:
-  metadata.gz: 877b0650078a1b3b50a54f51e7a7c83accc5639f362780b4942187f491221e623a5954783429cb4f691c365ca51167f121a9a00416719332a8c401bf5e919608
-  data.tar.gz: 6a4a3508b51befc22468b1cb05f4eef7f58ec4f7d31137ffe953c0d41fa6edb15d03bd102963a7556cc024d18807fc7a13c064443a756f333d66396f0697acea
+  metadata.gz: 63f49935c01027dcbdabe02d0ee8894652e1ee1499ca3ad779f1f39266a1acbc69c04ea75e593aaf5fa7a4049da9a25a89ffdcd9d99236212dbd6abf12834e97
+  data.tar.gz: ab7f8fecd30b1377a0202a3065543c840095636793d7580b66e368ce0b805ba38c96ee2f665f771be687f59f5ebeaea3a7507bb10e53baba3f0f34f1915c08f8

data/.gitignore

@@ -4,3 +4,8 @@
 pkg/*
 tmp
 Gemfile.lock
+
+.idea
+target
+.idea_modules
+.vagrant

data/.travis.yml

@@ -1,5 +1,3 @@
-language: ruby
-rvm:
-- 1.9.3
-- 2.0.0
-- 2.1.0
+language: scala
+scala:
+   2.11.1

data/README.md

@@ -15,10 +15,10 @@
 
 ## Install
 
-    gem install s3_website -v '~> 1'
+    gem install s3_website
 
-`s3_website` requires Ruby. Here is documentation on installing Ruby:
-<http://www.ruby-lang.org/en/downloads/>.
+s3_website needs both [Ruby](https://www.ruby-lang.org/en/downloads/)
+and [Java](http://java.com) to run.
 
 ## Usage
 
@@ -33,11 +33,6 @@ Here's how you can get started:
   S3 website. If the bucket does not exist, the command will create it for you.
 * Run `s3_website push` to push your website to S3. Congratulations! You are live.
 
-**Important security note:** if the source code of your website is publicly
-available, ensure that the `s3_website.yml` file is in the list of ignored files.
-For git users this means that the file `.gitignore` should mention the
-`s3_website.yml` file.
-
 ### For Jekyll users
 
 S3_website will automatically discover your website in the *_site* directory.
@@ -146,25 +141,6 @@ gzip:
 Remember that the extensions here are referring to the *compiled* extensions,
 not the pre-processed extensions.
 
-#### Gzip zopfli
-
-By adding the line `gzip_zopfli: true` into the config file, you can benefit
-from the zopfli algorithm, which is 100% compatible with the traditional gzip
-algorithm. A zopfli compression takes longer but results in about 5% smaller
-files.
-
-### Specifying a MIME type for files without extensions
-
-`s3_website` will look up the MIME type of each file it uploads, and infer the Content-Type from it automatically. By default, files without an extension will have a blank Content-Type.
-
-You can specify a default MIME type for files without an extension using a line like this in `s3_website.yml`:
-
-```yaml
-extensionless_mime_type: text/html
-```
-
-This is useful when you are uploading HTML files for which you want 'clean' URLs, e.g. `www.domain.com/info`.
-
 ### Using non-standard AWS regions
 
 By default, `s3_website` uses the US Standard Region. You can upload your
@@ -285,15 +261,6 @@ index file, your source bucket in Cloudfront likely is pointing to the S3 Origin
 *example.com.s3.amazonaws.com*. Update the source to the S3 Website Endpoint,
 *e.g. example.com.s3-website-us-east-1.amazonaws.com*, to fix this.
 
-### The headless mode
-
-s3_website has a headless mode, where human interactions are disabled.
-
-In the headless mode, `s3_website` will automatically delete the files on the S3
-bucket that are not on your local computer.
-
-Enable the headless mode by adding the `--headless` argument after `s3_website`.
-
 ### Configuring redirects on your S3 website
 
 You can set HTTP redirects on your S3 website in two ways. If you only need
@@ -340,32 +307,6 @@ For more information on configuring redirects, see the documentation of the
 gem, which comes as a transitive dependency of the `s3_website` gem. (The
 command `s3_website cfg apply` internally calls the `configure-s3-website` gem.)
 
-### Using `s3_website` as a library
-
-By nature, `s3_website` is a command-line interface tool. You can, however, use
-it programmatically by calling the same API as the executable `s3_website` does:
-
-````ruby
-require 's3_website'
-is_headless = true
-S3Website::Tasks.push('/website/root', '/path/to/your/website/_site/', is_headless)
-````
-
-You can also use a basic `Hash` instead of a `s3_website.yml` file:
-
-```ruby
-config = {
-  "s3_id"     => YOUR_AWS_S3_ACCESS_KEY_ID,
-  "s3_secret" => YOUR_AWS_S3_SECRET_ACCESS_KEY,
-  "s3_bucket" => "your.blog.bucket.com"
-}
-in_headless = true
-S3Website::Uploader.run('/path/to/your/website/_site/', config, in_headless)
-```
-
-The code above will assume that you have the `s3_website.yml` in the directory
-`/path/to/your/website`.
-
 ### Specifying custom concurrency level
 
 By default, `s3_website` does 3 operations in parallel. An operation can be an
@@ -387,18 +328,41 @@ If you experience the "too many open files" error, either increase the amount of
 maximum open files (on Unix-like systems, see `man ulimit`) or decrease the
 `concurrency_level` setting.
 
+### Simulating deployments
+
+You can simulate the `s3_website push` operation by adding the
+`--dry-run` switch. The dry run mode will not apply any modifications on your S3
+bucket or CloudFront distribution. It will merely print out what the `push`
+operation would actually do if run without the dry switch.
+
+You can use the dry run mode if you are unsure what kind of effects the `push`
+operation would cause to your live website.
+
+## Migrating from v1 to v2
+
+Please read the [release note](/changelog.md#200) on version 2. It contains
+information on backward incompatible changes.
+
 ## Example configurations
 
 See
 <https://github.com/laurilehmijoki/s3_website/blob/master/additional-docs/example-configurations.md>.
 
+## On security
+
+If the source code of your website is publicly
+available, ensure that the `s3_website.yml` file is in the list of ignored files.
+For git users this means that the file `.gitignore` should mention the
+`s3_website.yml` file.
+
+If you use the .dotenv gem, ensure that you do not push the `.env` file to a
+public git repository.
+
 ## Known issues
 
 Please create an issue and send a pull request if you spot any.
 
-## Development
-
-### Versioning
+## Versioning
 
 s3_website uses [Semantic Versioning](http://semver.org).
 
@@ -406,10 +370,9 @@ In the spirit of semantic versioning, here is the definition of public API for
 s3_website: Within a major version, s3_website will not break
 backwards-compatibility of anything that is mentioned in this README file.
 
-### Tests
+## Development
 
-  * Install bundler and run `bundle install`
-  * Run all tests by invoking `rake test`
+See [development](additional-docs/development.md).
 
 ### Contributing
 
@@ -425,12 +388,6 @@ If you are not sure how to test your pull request, you can ask the [gem owners
 However, by including proper tests, you increase the chances of your pull
 request being incorporated into future releases.
 
-#### Checklist for new features
-
-* Is it tested?
-* Is it documented in README?
-* Is it mentioned in `resources/configuration_file_template.yml`?
-
 ## License
 
 MIT. See the LICENSE file for more information.

data/Rakefile

@@ -1,24 +1,2 @@
 require 'bundler'
 Bundler::GemHelper.install_tasks
-
-desc "Build the project"
-task :default => 'test'
-
-desc "Run tests"
-task :test do
-  ENV['disable_parallel_processing'] = 'true'
-  sh "bundle exec rspec"
-  sh "bundle exec cucumber --tags ~@skip-on-travis"
-end
-
-desc 'Run work-in-progress features'
-task "cucumber:wip" do
-  ENV['disable_parallel_processing'] = 'true'
-  sh "bundle exec cucumber --tags @wip"
-end
-
-desc 'Run fast tests'
-task "cucumber:fast" do
-  ENV['disable_parallel_processing'] = 'true'
-  sh "bundle exec cucumber --tags ~@network-io --tags ~@starts-new-os-process"
-end

data/additional-docs/development.md

@@ -0,0 +1,29 @@
+## Coding
+
+Install a Scala editor. Intellij IDEA has great Scala support.
+
+If you use IDEA, install the [Grep
+Console](http://plugins.jetbrains.com/plugin/?idea&pluginId=7125) plugin. It
+shows the ANSI colors in your IDEA console.
+
+### Test runs with IDEA
+
+1. Create a run profile: *Run* –> *Edit Configurations...*.
+2. Add *Application*
+3. Set *Main class* to `s3.website.Push`
+4. Set *Program arguments* to `--site=/Users/you/yourtestsite/_site --config-dir=/Users/you/yourtestsite --verbose`
+
+## Automated tests
+
+    ./sbt test
+
+## Test Linux distributions
+
+Use Vagrant for testing the installation procedure on Linux.
+
+Here's howto:
+
+1. Install <https://www.vagrantup.com/downloads.html>
+2. `cd vagrant && vagrant status`
+3. launch with `vagrant up <name>` and ssh into with `vagrant ssh <name>`
+4. test the latest release with `gem install s3_website && s3_website push`

data/assembly.sbt

@@ -0,0 +1,3 @@
+import AssemblyKeys._ // put this at the top of the file
+
+assemblySettings

data/bin/s3_website_monadic

@@ -0,0 +1,222 @@
+#!/usr/bin/env ruby
+
+require File.expand_path(File.dirname(__FILE__)+ '/../lib/s3_website')
+require 'colored'
+
+class Cfg < Thor
+  desc 'create', 'Create a config file with placeholder values'
+  def create
+    config_file_src = File.dirname(__FILE__) + '/../resources/configuration_file_template.yml'
+    config_file = Dir.pwd + '/s3_website.yml'
+    unless File.exists? config_file
+      require 'fileutils'
+      FileUtils.cp config_file_src, config_file
+      puts "Created the config file s3_website.yml. Go fill it with your settings."
+    end
+  rescue Exception => error
+    puts "#{error.message} (#{error.class})"
+    exit 1
+  end
+
+  desc 'apply', 'Apply the configuration on the AWS services'
+  long_desc <<-LONGDESC
+    `s3_website cfg apply` will apply the configuration the S3 bucket.
+
+    In addition, if you CloudFront related settings, this command will apply them on the CloudFront distribution.
+
+    If the S3 bucket does not exist, this command will create it
+    and configure it to function as a website.
+  LONGDESC
+  def apply
+    puts 'Applying the configurations in s3_website.yml on the AWS services ...'
+    require 'configure-s3-website'
+    config_source = ConfigureS3Website::FileConfigSource.new 's3_website.yml'
+    ConfigureS3Website::Runner.run({
+      :config_source => config_source
+    })
+  rescue Exception => error
+    puts "#{error.message} (#{error.class})"
+    exit 1
+  end
+end
+
+class Cli < Thor
+  option(
+    :site,
+    :type => :string,
+    :desc => "The directory where your website files are. When not defined, s3_website will look for the site in either _site or public/output."
+  )
+  option(
+    :config_dir,
+    :type => :string,
+    :desc => "The directory where your config file is. When not defined, s3_website will look in the current working directory."
+  )
+  option(
+    :verbose,
+    :type => :boolean,
+    :default => false,
+    :desc => "Print verbose output"
+  )
+  option(
+    :dry_run,
+    :type => :boolean,
+    :default => false,
+    :desc => "Run the operation without actually making the modifications. When this switch is on, changes will not be applied on the S3 website. They will be only printed to the console."
+  )
+  desc 'push', 'Push local files with the S3 website'
+  long_desc <<-LONGDESC
+    `s3_website push` will upload new and changes files to S3. It will
+    also delete from S3 the files that you no longer have locally.
+  LONGDESC
+  def push
+    call_dir = Dir.pwd
+    project_root = File.expand_path(File.dirname(__FILE__)+ '/..')
+    logger = Logger.new(options[:verbose])
+    # Find the jar
+    jar_file = resolve_jar project_root, logger
+    # Then run it
+    run_s3_website_jar(jar_file, call_dir, logger)
+  end
+
+  desc 'cfg SUBCOMMAND ...ARGS', 'Operate on the config file'
+  subcommand 'cfg', Cfg
+end
+
+def run_s3_website_jar(jar_file, call_dir, logger)
+  java_installed = resolve_exit_status('which java') or resolve_exit_status('java -version')
+  unless java_installed
+    logger.info_msg "Cannot find Java. s3_website push is implemented in Scala, and it needs Java to run."
+    autoinstall_java_or_print_help_and_exit(logger)
+  end
+  args = ARGV.join(' ').sub('push', '')
+  logger.debug_msg "Using #{jar_file}"
+  if system("java -cp #{jar_file} s3.website.Push #{args}")
+    exit 0
+  else
+    exit 1
+  end
+end
+
+def resolve_exit_status(cmd)
+  `#{cmd}`
+  cmd_succeeded = $? == 0
+rescue
+  cmd_succeeded = false
+end
+
+def autoinstall_java_or_print_help_and_exit(logger)
+  @logger = logger
+  automatic_methods = [
+    {
+      :package_manager_lookup => 'which apt-get',
+      :install_command => 'sudo apt-get install --assume-yes openjdk-7-jre'
+    },
+    {
+      :package_manager_lookup => 'which yum',
+      :install_command => 'sudo yum install --assumeyes java-1.7.0-openjdk'
+    }
+  ]
+
+  def print_manual_method_and_exit
+    @logger.info_msg 'Go to http://java.com, install Java and then try again.'
+    @logger.info_msg "(If you cannot or do not want to install Java, you can use latest 1.x version of this gem, which requires only Ruby. For more info, see https://github.com/laurilehmijoki/s3_website/tree/1.x)"
+    exit 1
+  end
+
+  manual_method_help =
+
+  automatic_method = automatic_methods.find { |automatic_method|
+    resolve_exit_status automatic_method.fetch(:package_manager_lookup)
+  }
+  if automatic_method
+    @logger.info_msg "Do you want me to install Java with the command `#{automatic_method.fetch(:install_command)}`? [Y/n]"
+    user_answer = $stdin.gets
+    if user_answer.chomp.downcase == 'y' or user_answer == "\n"
+      automatic_method_succeeded = system automatic_method.fetch(:install_command)
+      unless automatic_method_succeeded
+        @logger.fail_msg "Could not automatically install Java. Try setting it up manually:"
+        print_manual_method_and_exit
+      end
+    else
+      print_manual_method_and_exit
+    end
+  else
+    print_manual_method_and_exit
+  end
+end
+
+def resolve_jar(project_root, logger)
+  development_jar_path =
+    project_root + '/target/scala-2.11/s3_website.jar'
+  released_jar_lookup_paths = [
+    project_root + "/s3_website-#{S3Website::VERSION}.jar",
+    (ENV['TMPDIR'] || '/tmp') + "/s3_website-#{S3Website::VERSION}.jar"
+  ]
+  found_jar = ([development_jar_path] + released_jar_lookup_paths).
+    select { |jar_path|
+      File.exists? jar_path
+    }.
+    first
+  jar_file =
+    if found_jar
+      found_jar
+    else
+      is_development = File.exists?(project_root + '/.git')
+      if is_development
+        Dir.chdir(project_root) {
+          system "./sbt assembly"
+        }
+        development_jar_path
+      else
+        download_jar(released_jar_lookup_paths, logger)
+      end
+    end
+end
+
+def download_jar(released_jar_lookup_paths, logger)
+  tag_name = "v#{S3Website::VERSION}"
+  downloaded_jar = released_jar_lookup_paths.select { |jar_path|
+    File.writable? File.dirname(jar_path)
+  }.first
+  unless downloaded_jar
+    logger.fail_msg "Neither #{released_jar_lookup_paths.join ' or '} is writable. Cannot download s3_website.jar."
+    logger.fail_msg "Set either directory as writable to the current user and try again."
+    exit 1
+  end
+  download_url = "https://github.com/laurilehmijoki/s3_website/releases/download/#{tag_name}/s3_website.jar"
+  logger.info_msg "Downloading #{download_url} into #{downloaded_jar}"
+  require 'open-uri'
+  open(downloaded_jar, 'wb') do |file|
+    file << open(download_url).read
+  end
+  downloaded_jar
+end
+
+class Logger
+  attr_reader :verbose
+  def initialize(verbose)
+    @verbose = verbose
+  end
+
+  def debug_msg(msg)
+    if verbose
+      print_msg 'debg'.cyan, msg
+    end
+  end
+
+  def info_msg(msg)
+    print_msg 'info'.blue, msg
+  end
+
+  def fail_msg(msg)
+    print_msg 'fail'.red, msg
+  end
+
+  private
+
+  def print_msg(prefix, msg)
+    puts "[#{prefix}] #{msg}"
+  end
+end
+
+Cli.start(ARGV)