appengine 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 4841b00bed08c35d62dfd337ccd15cf5df775de9
-  data.tar.gz: a3158e01aaccb61471b504d7a66983c24a58a560
+  metadata.gz: 06eae82181654cc6c73ea33d6fda3c5cc5f2469a
+  data.tar.gz: 33c1eae652eb721b503eb848e7e6034f0843c279
 SHA512:
-  metadata.gz: 27445f16fa0d99b081992a8ea08427730dc4375b05f7a083763c5e6f9ee338652a6e8798eb47bc5293e51d9a4a22fe3c79322ca934284973826c3f7f6faf5741
-  data.tar.gz: 734a601f646ee785f5af7f9cbdf77af271a6e2bc25ae80abeaad0a8c92a26562462d2be953fa981ea4a04b48b1646c316dc08022658ad4d954e755dc5fb48406
+  metadata.gz: e8fe8e5431795a42492b08e7b86d61a5c9e3d21d2aa28625a1928da674fad328ba3e080ff4c7f2df87e54b0621c51ab797d44d4f0f10c9e327e7122a443a3ea3
+  data.tar.gz: 67586c5fae41dbbe9ca43dca1dfc9c59f8dc5f5b6e447902826b47c0b5313f339852e07c26f234f3dca19213c595e6bf663048c72ad7896d2e3921c4c8e14b39

data/CHANGELOG.md

@@ -2,6 +2,19 @@
 
 This is the change history for the appengine gem.
 
+## v0.4.0 (2017-07-17)
+
+*   Provided rake tasks to access App Engine remote execution.
+*   Reimplemented AppEngine::Env as an alias to the google-cloud-env gem's
+    functionality.
+
+## v0.3.0 (2017-02-28)
+
+*   Replaced logger integration with a dependency on the stackdriver gem,
+    which supersedes it.
+*   Stubbed out the env class in preparation to change it to a dependency
+    on the upcoming google-cloud-env gem.
+
 ## v0.2.0 (2016-05-04)
 
 *   Tools for integration with the Google Cloud Console logger, including

data/README.md

@@ -4,18 +4,23 @@ Google App Engine Integration Tools
 This repository contains the "appengine" gem, a collection of libraries and
 plugins for integrating Ruby apps with Google App Engine. It is not required
 for deploying a ruby application to Google App Engine, but it provides a
-number of convenience hooks for better integrating into the App Engine
+number of convenience hooks and tools for integrating into the App Engine
 environment.
 
 Currently, it includes:
 
 * Automatic Stackdriver instrumentation for Rails apps. This means logs,
-  error reports, and latency traces are reported to the cloud console.
-* A placeholder for a class that provides app engine environment information
-  such as project ID and VM info.
+  error reports, and latency traces are reported to the cloud console,
+  and debugger integration is available.
+* A client and rake tasks for executing application commands in the App
+  Engine environment against production resources, useful for tasks such as
+  running production database migrations.
+* Convenient access to environment information such as project ID and VM
+  properties.
 
 Planned for the near future:
 
+* Tools for generating "app.yaml" configuration files for Ruby applications.
 * Streamlined implementation of health checks and other lifecycle hooks.
 
 For more information on using Google Cloud Platform to deploy Ruby apps,
@@ -36,6 +41,9 @@ may need to include the line:
 in your `config/application.rb` file if you aren't already requiring all
 bundled gems.
 
+If you are running a different web framework, you may need to add some
+initialization code to activate the features listed below.
+
 ## Logging and monitoring
 
 This library automatically installs the "stackdriver" gem, which instruments
@@ -47,6 +55,37 @@ monitoring features of Google App Engine, see:
 * [google-cloud-error_reporting instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-error_reporting/latest/guides/instrumentation)
 * [google-cloud-trace instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-trace/latest/guides/instrumentation)
 
+Rails applications automatically activate this instrumentation when the gem
+is present. You may opt out of individual services by providing appropriate
+Rails configuration. See {AppEngine::Railtie} for more information.
+
+Non-Rails applications must provide initialization code to activate this
+instrumentation, typically by installing a Rack middleware. See the individual
+service documentation links above for more information.
+
+## App Engine remote execution
+
+This library provides rake tasks for App Engine remote execution, allowing
+App Engine applications to perform on-demand tasks in the App Engine
+environment. This may be used for safe running of ops and maintenance tasks,
+such as database migrations, that access production cloud resources. For
+example, you could run a production database migration in a Rails app using:
+
+    bundle exec rake appengine:exec -- bundle exec rake db:migrate
+
+The migration would be run in VMs provided by Google Cloud. It uses a
+privileged service account that will have access to the production cloud
+resources, such as Cloud SQL instances, used by the application. This mechanism
+is often much easier and safer than running the task on a local workstation and
+granting that workstation direct access to those Cloud SQL instances.
+
+See {AppEngine::Exec} for more information on App Engine remote execution.
+
+See {AppEngine::Tasks} for more information on running the rake tasks. The
+tasks are available automatically in Rails applications when the gem is
+present. Non-Rails applications may install the tasks by adding the line
+`require "appengine/tasks"` to the `Rakefile`.
+
 ## Development and support
 
 The source code for this gem is available on Github at

data/Rakefile

@@ -33,4 +33,10 @@ end
   rd.options << '--all'
 end
 
+require "yard"
+require "yard/rake/yardoc_task"
+YARD::Rake::YardocTask.new
+
+load "lib/appengine/tasks.rb"
+
 task :default => [:test]

data/lib/appengine.rb

@@ -13,7 +13,7 @@
 # limitations under the License.
 ;
 
-# == Google AppEngine integration
+# # Google AppEngine integration
 #
 # The AppEngine module includes optional tools helping Ruby applications to
 # integrate more closely with the Google App Engine environment.
@@ -22,8 +22,9 @@ module AppEngine
 end
 
 
-require 'appengine/version'
-require 'appengine/env'
-require 'appengine/railtie' if defined?(::Rails)
+require "appengine/version"
+require "appengine/env"
+require "appengine/exec"
+require "appengine/railtie" if defined? ::Rails
 
 require "stackdriver"

data/lib/appengine/env.rb

@@ -13,18 +13,26 @@
 # limitations under the License.
 ;
 
+require "google/cloud/env"
+
 
 module AppEngine
 
   ##
-  # A collection of functions for extracting App Engine environment information.
+  # A convenience object that provides information on the Google Cloud
+  # hosting environment. For example, you can call
   #
-  module Env
-
-    def self.app_engine?
-      ::ENV["GAE_INSTANCE"] ? true : false
-    end
-
-  end
+  #     if AppEngine::Env.app_engine?
+  #       # Do something
+  #     end
+  #
+  # Technically, `Env` is not actually a module, but is simply set to the
+  # `Google::Cloud.env` object.
+  #
+  # This is provided mostly for backward compatibility with previous usage, and
+  # is mildly deprecated. Generally, you should call `Google::Cloud.env`
+  # directly instead. See the documentation for the `google-cloud-env` gem.
+  #
+  Env = ::Google::Cloud.env
 
 end

data/lib/appengine/exec.rb

@@ -0,0 +1,421 @@
+# Copyright 2017 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+;
+
+require "yaml"
+require "json"
+
+require "appengine/util/gcloud"
+
+
+module AppEngine
+
+  ##
+  # # App Engine remote execution
+  #
+  # This class provides a client for App Engine remote execution, allowing
+  # App Engine applications to perform on-demand tasks in the App Engine
+  # environment. This may be used for safe running of ops and maintenance
+  # tasks, such as database migrations, that access production cloud resources.
+  #
+  # ## About App Engine execution
+  #
+  # App Engine execution spins up an image of a deployed App Engine app, and
+  # runs a command in that image. For example, if your app runs on Ruby on
+  # Rails, then your app provides a `bin/rails` tool, and you may invoke it
+  # using App Engine execution---for example to run a command such as
+  # `bundle exec bin/rails db:migrate` in the image.
+  #
+  # When App Engine execution runs your command, it provides access to key
+  # elements of the App Engine environment, including:
+  # * The same runtime that runs your application in App Engine itself.
+  # * Any Cloud SQL connections requested by your application.
+  # * Any environment variables set by your application.
+  #
+  # The command runs on virtual machines provided by Google Cloud Container
+  # Builder, and has access to the credentials of the Cloud Container Builder
+  # service account.
+  #
+  # ## Prerequisites
+  #
+  # To use App Engine remote execution, you will need:
+  #
+  # * An app deployed to Google App Engine, of course!
+  # * The gcloud SDK installed and configured. See https://cloud.google.com/sdk/
+  # * The `appengine` gem.
+  #
+  # You may also need to grant the Cloud Container Builder service account
+  # any permissions needed by your command. Often, Project Editor permissions
+  # will be sufficient for most tasks. You can find the service account
+  # configuration in the IAM tab in the Cloud Console under the name
+  # `[your-project-number]@cloudbuild.gserviceaccount.com`.
+  #
+  # You may use the `AppEngine::Exec` class to run commands directly. However,
+  # in most cases, it will be easier to run commands via the provided rake
+  # tasks. See {AppEngine::Tasks} for more info.
+  #
+  # ## Configuring
+  #
+  # This class uses three parameters to specify which application image to use
+  # to run your command: `service`, `config_path`, and `version`.
+  #
+  # In most cases, you can use the defaults. The Exec class will look in your
+  # current directory for a file called `./app.yaml` which describes your App
+  # Engine service. It gets the service name from this file (or uses the
+  # "default" name if none is specified), then looks up the most recently
+  # created deployment version for that service. That deployment version then
+  # provides the application image that runs your command.
+  #
+  # If your app has multiple services, you may specify which config file
+  # (other than `./app.yaml`) describes the desired service, by providing the
+  # `config_path` parameter. Alternately, you may specify a service name
+  # directly by providing the `service` parameter. If you provide both
+  # parameters, `service` takes precedence.
+  #
+  # Usually, App Engine execution uses the image for the most recently created
+  # version of the service. (Note: the most recently created version is used,
+  # regardless of whether that version is currently receiving traffic.) If you
+  # want to use the image for a different version, you may specify a version
+  # by providing the `version` parameter.
+  #
+  # You may also provide a timeout, which is the length of time that App
+  # Engine execution will allow your command to run before it is considered to
+  # have stalled and is terminated. The timeout should be a string of the form
+  # `2h15m10s`. The default is `10m`.
+  #
+  # ## Resource usage and billing
+  #
+  # App Engine remote execution uses virtual machine resources provided by
+  # Google Cloud Container Builder. Generally, a certain number of usage
+  # minutes per day is covered under a free tier, but additional compute usage
+  # beyond that time is billed to your Google Cloud account. For more details,
+  # see https://cloud.google.com/container-builder/pricing
+  #
+  # If your command makes API calls or utilizes other cloud resources, you may
+  # also be billed for that usage. However, remote execution does not use
+  # actual App Engine instances, and you will not be billed for additional App
+  # Engine instance usage.
+  #
+  class Exec
+
+    @default_timeout = "10m".freeze
+    @default_service = "default".freeze
+    @default_config_path = "./app.yaml".freeze
+    @default_wrapper_image = "gcr.io/google-appengine/exec-wrapper".freeze
+
+
+    ##
+    # Base class for exec-related usage errors.
+    #
+    class UsageError < ::StandardError
+    end
+
+
+    ##
+    # Exception raised when the App Engine config file could not be found.
+    #
+    class ConfigFileNotFound < UsageError
+      def initialize config_path
+        @config_path = config_path
+        super "Config file #{config_path} not found."
+      end
+      attr_reader :config_path
+    end
+
+    ##
+    # Exception raised when the App Engine config file could not be parsed.
+    #
+    class BadConfigFileFormat < UsageError
+      def initialize config_path
+        @config_path = config_path
+        super "Config file #{config_path} malformed."
+      end
+      attr_reader :config_path
+    end
+
+    ##
+    # Exception raised when the given version could not be found, or no
+    # versions at all could be found for the given service.
+    #
+    class NoSuchVersion < UsageError
+      def initialize service, version=nil
+        @service = service
+        @version = version
+        if version
+          super "No such version \"#{version}\" for service \"#{service}\""
+        else
+          super "No versions found for service \"#{service}\""
+        end
+      end
+      attr_reader :service
+      attr_reader :version
+    end
+
+    ##
+    # Exception raised when an explicitly-specified service name conflicts with
+    # a config-specified service name.
+    #
+    class ServiceNameConflict < UsageError
+      def initialize service_name, config_name, config_path
+        @service_name = service_name
+        @config_name = config_name
+        @config_path = config_path
+        super "Service name conflicts with config file"
+      end
+      attr_reader :service_name
+      attr_reader :config_name
+      attr_reader :config_path
+    end
+
+
+    class << self
+
+      ## @return [String] Default command timeout.
+      attr_accessor :default_timeout
+
+      ## @return [String] Default service name if the config doesn't specify.
+      attr_accessor :default_service
+
+      ## @return [String] Path to default config file.
+      attr_accessor :default_config_path
+
+      ## @return [String] Docker image that implements the app engine wrapper.
+      attr_accessor :default_wrapper_image
+
+      ##
+      # Create an execution for a rake task.
+      #
+      # @param name [String] Name of the task
+      # @param args [Array<String>] Args to pass to the task
+      # @param env_args [Array<String>] Environment variable settings, each
+      #     of the form `NAME=value`.
+      # @param service [String,nil] Name of the service. If omitted, obtains
+      #     the service name from the config file.
+      # @param config_path [String,nil] App Engine config file to get the
+      #     service name from if the service name is not provided directly.
+      #     Defaults to the value of `AppEngine::Exec.default_config_path`.
+      # @param version [String,nil] Version string. Defaults to the most
+      #     recently created version of the given service (which may not be the
+      #     one currently receiving traffic).
+      # @param timeout [String,nil] Timeout string. Defaults to the value of
+      #     `AppEngine::Exec.default_timeout`.
+      #
+      def new_rake_task name, args: [], env_args: [],
+                        service: nil, config_path: nil, version: nil,
+                        timeout: nil
+        escaped_args = args.map{ |arg|
+          arg.gsub(/[,\[\]]/){ |m| "\\#{m}" }
+        }
+        if escaped_args.empty?
+          name_with_args = name
+        else
+          name_with_args = "#{name}[#{escaped_args.join ','}]"
+        end
+        new ["bundle", "exec", "rake", name_with_args] + env_args,
+            service: service, config_path: config_path, version: version,
+            timeout: timeout
+      end
+
+    end
+
+
+    ##
+    # Create an execution for the given command.
+    #
+    # @param command [Array<String>] The command in array form.
+    # @param service [String,nil] Name of the service. If omitted, obtains
+    #     the service name from the config file.
+    # @param config_path [String,nil] App Engine config file to get the
+    #     service name from if the service name is not provided directly.
+    #     Defaults to the value of `AppEngine::Exec.default_config_path`.
+    # @param version [String,nil] Version string. Defaults to the most
+    #     recently created version of the given service (which may not be the
+    #     one currently receiving traffic).
+    # @param timeout [String,nil] Timeout string. Defaults to the value of
+    #     `AppEngine::Exec.default_timeout`.
+    #
+    def initialize command,
+                   service: nil, config_path: nil, version: nil, timeout: nil
+      @command = command
+      @service = service
+      @config_path = config_path
+      @version = version
+      @timeout = timeout
+      @wrapper_image = nil
+
+      yield self if block_given?
+    end
+
+
+    ## @return [String,nil] The service name, or nil to read from the config.
+    attr_accessor :service
+
+    ## @return [String,nil] Path to the config file, or nil to use the default.
+    attr_accessor :config_path
+
+    ## @return [String,nil] Service version, or nil to use the most recent.
+    attr_accessor :version
+
+    ## @return [String,nil] Command timeout, or nil to use the default.
+    attr_accessor :timeout
+
+    ## @return [String,Array<String>] Command to run.
+    attr_accessor :command
+
+    ## @return [String] Custom wrapper image to use, or nil to use the default.
+    attr_accessor :wrapper_image
+
+
+    ##
+    # Executes the command synchronously. Streams the logs back to standard out
+    # and does not return until the command has completed or timed out.
+    #
+    def start
+      resolve_parameters
+
+      version_info = version_info @service, @version
+      env_variables = version_info["envVariables"] || {}
+      beta_settings = version_info["betaSettings"] || {}
+      cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
+      image = version_info["deployment"]["container"]["image"]
+
+      config = build_config command, image, env_variables, cloud_sql_instances
+      file = ::Tempfile.new ["cloudbuild_", ".json"]
+      begin
+        ::JSON.dump config, file
+        file.flush
+        Util::Gcloud.execute [
+            "container", "builds", "submit",
+            "--no-source",
+            "--config=#{file.path}",
+            "--timeout=#{@timeout}"]
+      ensure
+        file.close!
+      end
+    end
+
+
+    private
+
+    ##
+    # @private
+    # Resolves and canonicalizes all the parameters.
+    #
+    def resolve_parameters
+      unless @command.is_a? Array
+        @command = ::Shellwords.parse @command.to_s
+      end
+
+      config_service = config_path = nil
+      if @config_path || !@service
+        config_service = begin
+          config_path = @config_path || Exec.default_config_path
+          ::YAML.load_file(config_path)["service"] || Exec.default_service
+        rescue ::Errno::ENOENT
+          raise ConfigFileNotFound.new config_path
+        rescue
+          raise BadConfigFileFormat.new config_path
+        end
+      end
+      if @service && config_service && @service != config_service
+        raise ServiceNameConflict.new @service, config_service, config_path
+      end
+
+      @service ||= config_service
+      @version ||= latest_version @service
+      @timeout ||= Exec.default_timeout
+      @wrapper_image ||= Exec.default_wrapper_image
+    end
+
+    ##
+    # @private
+    # Builds a cloudbuild config as a data structure.
+    #
+    # @param command [Array<String>] The command in array form.
+    # @param image [String] The fully qualified image path.
+    # @param env_variables[Hash<String,String>] Environment variables.
+    # @param cloud_sql_instances[String,Array<String>] Names of cloud sql
+    #     instances to connect to.
+    #
+    def build_config command, image, env_variables, cloud_sql_instances
+      args = ["-i", image]
+      env_variables.each do |k, v|
+        args << "-e" << "#{k}=#{v}"
+      end
+      unless cloud_sql_instances.empty?
+        cloud_sql_instances = Array(cloud_sql_instances)
+        cloud_sql_instances.each do |sql|
+          args << "-s" << sql
+        end
+      end
+      args << "--"
+      args += command
+
+      {
+        "steps" => [
+          "name" => @wrapper_image,
+          "args" => args
+        ]
+      }
+    end
+
+    ##
+    # @private
+    # Returns the name of the most recently created version of the given
+    # service.
+    #
+    # @param service [String] Name of the service.
+    # @return [String] Name of the most recent version.
+    #
+    def latest_version service
+      result = Util::Gcloud.execute [
+          "app", "versions", "list",
+          "--service=#{service}",
+          "--format=get(version.id)",
+          "--sort-by=~version.createTime",
+          "--limit=1"],
+          capture: true, assert: false
+      result = result.split.first
+      raise NoSuchVersion.new(service) unless result
+      result
+    end
+
+    ##
+    # @private
+    # Returns full information on the given version of the given service.
+    #
+    # @param service [String] Name of the service. If omitted, the service
+    #     "default" is used.
+    # @param version [String] Name of the version. If omitted, the most
+    #     recently deployed is used.
+    # @return [Hash,nil] A collection of fields parsed from the JSON
+    #     representation of the version, or nil if the requested version
+    #     doesn't exist.
+    #
+    def version_info service, version
+      service ||= "default"
+      version ||= latest_version service
+      result = Util::Gcloud.execute [
+          "app", "versions", "describe", version,
+          "--service=#{service}",
+          "--format=json"],
+          capture: true, assert: false
+      result.strip!
+      raise NoSuchVersion.new(service, version) if result.empty?
+      ::JSON.parse result
+    end
+
+  end
+
+end

data/lib/appengine/railtie.rb

@@ -13,35 +13,77 @@
 # limitations under the License.
 ;
 
-require 'fileutils'
-
 
 module AppEngine
 
-
-  # == AppEngine Rails integration
+  ##
+  # # AppEngine Rails integration
   #
   # A Railtie providing Rails integration with the Google App Engine runtime
-  # environment. Sets up the Rails logger to log to the Google Cloud Console
-  # in production.
+  # environment.
+  #
+  # Specifically:
+  #
+  # * It installs the Stackdriver instrumentation, providing application
+  #   diagnostics to the project's Stackdriver account.
+  # * It installs the rake tasks, providing the ability to execute commands
+  #   on demand in the production App Engine environment.
   #
   # To use, just include the "appengine" gem in your gemfile, and make sure
   # it is required in your config/application.rb (if you are not already
   # using Bundler.require).
   #
-  # === Configuration
+  # ## Configuration
+  #
+  # You may selectively deactivate features of this Railtie using Rails
+  # configuration keys. For example, to disable rake tasks, include the
+  # following line in one of your Rails configuration files:
+  #
+  #     config.appengine.define_tasks = false
+  #
+  # The following configuration keys are supported. Additional keys specific
+  # to the various Stackdriver services may be defined in the individual
+  # libraries.
+  #
+  # ### appengine.define_tasks
+  #
+  # Causes rake tasks to be added to the application. Default is true. Set it
+  # to false to disable App Engine rake tasks.
+  #
+  # ### google_cloud.use_logging
+  #
+  # Activates Stackdriver Logging, collecting Rails logs so they appear on
+  # the Google Cloud console. Default is true. Set it to false to disable
+  # logging instrumentation.
+  #
+  # ### google_cloud.use_error_reporting
+  #
+  # Activates Stackdriver Error Reporting, collecting exceptions so they appear
+  # on the Google Cloud console. Default is true. Set it to false to disable
+  # error instrumentation.
+  #
+  # ### google_cloud.use_trace
+  #
+  # Activates Stackdriver Trace instrumentation, collecting application latency
+  # trace data so it appears on the Google Cloud conosle. Default is true. Set
+  # it to false to disable trace instrumentation.
+  #
+  # ### google_cloud.use_debugger
+  #
+  # Enables the Stackdriver Debugger. Default is true. Set it to false to
+  # disable debugging.
   #
-  # This is a placeholder for now.
-
   class Railtie < ::Rails::Railtie
 
-    # :stopdoc:
-
     config.appengine = ::ActiveSupport::OrderedOptions.new
+    config.appengine.define_tasks = true
 
-    # :startdoc:
+    rake_tasks do |app|
+      if app.config.appengine.define_tasks
+        require "appengine/tasks"
+      end
+    end
 
   end
 
-
 end

data/lib/appengine/tasks.rb

@@ -0,0 +1,308 @@
+# Copyright 2017 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+;
+
+# This file should be loaded or required from a Rakefile to define AppEngine
+# related tasks.
+
+require "shellwords"
+
+require "appengine/util/gcloud"
+require "appengine/exec"
+
+
+module AppEngine
+
+  ##
+  # # App Engine Rake Tasks.
+  #
+  # To make these tasks available, add the line `require "appengine/tasks"`
+  # to your Rakefile. If your app uses Ruby on Rails, then the appengine gem
+  # provides a railtie that adds its tasks automatically, so you don't have
+  # to do anything beyond adding the gem to your Gemfile.
+  #
+  # The following tasks are defined:
+  #
+  # ## Rake appengine:exec
+  #
+  # Executes a given command in the context of an App Engine application, using
+  # App Engine remote execution. See {AppEngine::Exec} for more information on
+  # this capability.
+  #
+  # The command to be run may either be provided as a rake argument, or as
+  # command line arguments, delimited by two dashes `--`. (The dashes are
+  # needed to separate your command from rake arguments and flags.)
+  # For example, to run a production database migration, you can run either of
+  # the following equivalent commands:
+  #
+  #     bundle exec rake "appengine:exec[bundle exec bin/rails db:migrate]"
+  #     bundle exec rake appengine:exec -- bundle exec bin/rails db:migrate
+  #
+  # To display usage instructions, provide two dashes but no command:
+  #
+  #     bundle exec rake appengine:exec --
+  #
+  # ### Parameters
+  #
+  # You may customize the behavior of App Engine execution through a few
+  # enviroment variable parameters. These are set via the normal mechanism at
+  # the end of a rake command line. For example, to set GAE_CONFIG:
+  #
+  #     bundle exec rake appengine:exec GAE_CONFIG=myservice.yaml -- bundle exec bin/rails db:migrate
+  #
+  # Be sure to set these parameters before the double dash. Any arguments
+  # following the double dash are interpreted as part of the command itself.
+  #
+  # The following environment variable parameters are supported:
+  #
+  # #### GAE_CONFIG
+  #
+  # Path to the App Engine config file, used when your app has multiple
+  # services, or the config file is otherwise not called `./app.yaml`. The
+  # config file is used to determine the name of the App Engine service.
+  #
+  # #### GAE_SERVICE
+  #
+  # Name of the service to be used. If both `GAE_CONFIG` and `GAE_SERVICE` are
+  # provided and imply different service names, an error will be raised.
+  #
+  # #### GAE_VERSION
+  #
+  # The version of the service, used to identify which application image to
+  # use to run your command. If not specified, uses the most recently created
+  # version, regardless of whether that version is actually serving traffic.
+  #
+  # #### GAE_TIMEOUT
+  #
+  # Amount of time to wait before appengine:exec terminates the command.
+  # Expressed as a string formatted like: "2h15m10s". Default is "10m".
+  #
+  module Tasks
+
+    ## @private
+    CONFIG_ENV = "GAE_CONFIG"
+    ## @private
+    SERVICE_ENV = "GAE_SERVICE"
+    ## @private
+    VERSION_ENV = "GAE_VERSION"
+    ## @private
+    TIMEOUT_ENV = "GAE_TIMEOUT"
+
+    @defined = false
+
+    class << self
+
+      ##
+      # @private
+      # Define rake tasks.
+      #
+      def define
+        if @defined
+          puts "AppEngine rake tasks already defined."
+          return
+        end
+        @defined = true
+
+        setup_exec_task
+      end
+
+      private
+
+      def setup_exec_task
+        ::Rake.application.last_description =
+            "Execute the given command in Google App Engine."
+        ::Rake::Task.define_task "appengine:exec", [:cmd] do |t, args|
+          verify_gcloud_and_report_errors
+          if args[:cmd]
+            command = ::Shellwords.split args[:cmd]
+          else
+            i = (::ARGV.index{ |a| a.to_s == "--" } || -1) + 1
+            if i == 0
+              report_error <<~MESSAGE
+No command provided for appengine:exec.
+Did you remember to delimit it with two dashes? e.g.
+  bundle exec rake appengine:exec -- bundle exec ruby myscript.rb
+For detailed usage instructions, provide two dashes but no command:
+  bundle exec rake appengine:exec --
+              MESSAGE
+            end
+            command = ::ARGV[i..-1]
+            if command.empty?
+              show_usage
+              exit
+            end
+          end
+          app_exec = Exec.new \
+              command,
+              service: ::ENV[SERVICE_ENV],
+              config_path: ::ENV[CONFIG_ENV],
+              version: ::ENV[VERSION_ENV],
+              timeout: ::ENV[TIMEOUT_ENV]
+          start_and_report_errors app_exec
+          exit
+        end
+      end
+
+      def show_usage
+        puts <<~USAGE
+rake appengine:exec
+
+This Rake task executes a given command in the context of an App Engine
+application, using App Engine remote execution. For more information,
+on this capability, see the AppEngine::Exec documentation at
+http://www.rubydoc.info/gems/appengine/AppEngine/Exec
+
+The command to be run may either be provided as a rake argument, or as
+command line arguments delimited by two dashes `--`. (The dashes are
+needed to separate your command from rake arguments and flags.)
+For example, to run a production database migration, you can run either
+of the following equivalent commands:
+
+    bundle exec rake "appengine:exec[bundle exec bin/rails db:migrate]"
+    bundle exec rake appengine:exec -- bundle exec bin/rails db:migrate
+
+To display these usage instructions, provide two dashes but no command:
+
+    bundle exec rake appengine:exec --
+
+You may customize the behavior of App Engine execution through a few
+enviroment variable parameters. These are set via the normal mechanism at
+the end of a rake command line. For example, to set GAE_CONFIG:
+
+    bundle exec rake appengine:exec GAE_CONFIG=myservice.yaml -- bundle exec bin/rails db:migrate
+
+Be sure to set these parameters before the double dash. Any arguments
+following the double dash are interpreted as part of the command itself.
+
+The following environment variable parameters are supported:
+
+GAE_CONFIG
+
+  Path to the App Engine config file, used when your app has multiple
+  services, or the config file is otherwise not called `./app.yaml`. The
+  config file is used to determine the name of the App Engine service.
+
+GAE_SERVICE
+
+  Name of the service to be used. If both `GAE_CONFIG` and `GAE_SERVICE`
+  are provided and imply different service names, an error will be raised.
+
+GAE_VERSION
+
+  The version of the service, used to identify which application image to
+  use to run your command. If not specified, uses the most recently created
+  version, regardless of whether that version is actually serving traffic.
+
+GAE_TIMEOUT
+
+  Amount of time to wait before appengine:exec terminates the command.
+  Expressed as a string formatted like: "2h15m10s". Default is "10m".
+
+This rake task is provided by the "appengine" gem. To make these tasks
+available, add the following line to your Rakefile:
+
+    require "appengine/tasks"
+
+If your app uses Ruby on Rails, the gem provides a railtie that adds its
+tasks automatically, so you don't have to do anything beyond adding the
+gem to your Gemfile.
+
+For more information or to report issues, visit the Github page:
+https://github.com/GoogleCloudPlatform/appengine-ruby
+        USAGE
+      end
+
+      def verify_gcloud_and_report_errors
+        Util::Gcloud.verify!
+      rescue Gcloud::BinaryNotFound
+        report_error <<~MESSAGE
+Could not find the `gcloud` binary in your system path.
+This tool requires the Google Cloud SDK. To download and install it,
+visit https://cloud.google.com/sdk/downloads
+        MESSAGE
+      rescue Gcloud::GcloudNotAuthenticated
+        report_error <<~MESSAGE
+The gcloud authorization has not been completed. If you have not yet
+initialized the Google Cloud SDK, we recommend running the `gcloud init`
+command as described at https://cloud.google.com/sdk/docs/initializing
+Alternately, you may log in directly by running `gcloud auth login`.
+        MESSAGE
+      rescue Gcloud::ProjectNotSet
+        report_error <<~MESSAGE
+The gcloud project configuration has not been set. If you have not yet
+initialized the Google Cloud SDK, we recommend running the `gcloud init`
+command as described at https://cloud.google.com/sdk/docs/initializing
+Alternately, you may set the default project configuration directly by
+running `gcloud config set project <project-name>`.
+        MESSAGE
+      end
+
+      def start_and_report_errors app_exec
+        app_exec.start
+      rescue Exec::ConfigFileNotFound => ex
+        report_error <<~MESSAGE
+Could not determine which service should run this command because the App
+Engine config file "#{ex.config_path}" was not found.
+Specify the config file using the GAE_CONFIG argument. e.g.
+  bundle exec rake appengine:exec GAE_CONFIG=myapp.yaml -- myscript.sh
+Alternately, you may specify a service name directly with GAE_SERVICE. e.g.
+  bundle exec rake appengine:exec GAE_SERVICE=myservice -- myscript.sh
+MESSAGE
+      rescue Exec::BadConfigFileFormat => ex
+        report_error <<~MESSAGE
+Could not determine which service should run this command because the App
+Engine config file "#{ex.config_path}" was malformed.
+It must be a valid YAML file.
+Specify the config file using the GAE_CONFIG argument. e.g.
+  bundle exec rake appengine:exec GAE_CONFIG=myapp.yaml -- myscript.sh
+Alternately, you may specify a service name directly with GAE_SERVICE. e.g.
+  bundle exec rake appengine:exec GAE_SERVICE=myservice -- myscript.sh
+MESSAGE
+      rescue Exec::NoSuchVersion => ex
+        if ex.version
+          report_error <<~MESSAGE
+Could not find version "#{ex.version}" of service "#{ex.service}".
+Please double-check the version exists. To use the most recent version by
+default, omit the GAE_VERSION argument.
+MESSAGE
+        else
+          report_error <<~MESSAGE
+Could not find any versions of service "#{ex.service}".
+Please double-check that you have deployed this service. If you want to run
+a command against a different service, you may provide a GAE_CONFIG argument
+pointing to your App Engine config file, or a GAE_SERVICE argument to specify
+a service directly.
+MESSAGE
+        end
+      rescue Exec::ServiceNameConflict => ex
+        report_error <<~MESSAGE
+The explicit service name "#{ex.service_name}" was requested
+but conflicts with the service "#{ex.config_name}" from the
+config file "#{ex.config_path}"
+You should specify either GAE_SERVICE or GAE_CONFIG but not both.
+MESSAGE
+      end
+
+      def report_error str
+        ::STDERR.puts str
+        exit 1
+      end
+
+    end
+
+  end
+end
+
+
+::AppEngine::Tasks.define

data/lib/appengine/util/gcloud.rb

@@ -0,0 +1,190 @@
+# Copyright 2017 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+;
+
+require "shellwords"
+
+
+module AppEngine
+  module Util
+
+    ##
+    # A collection of utility functions and classes for interacting with an
+    # installation of the gcloud SDK.
+    #
+    module Gcloud
+
+      ##
+      # Base class for gcloud related errors.
+      #
+      class Error < ::StandardError
+      end
+
+      ##
+      # Exception raised when the gcloud binary could not be found.
+      #
+      class BinaryNotFound < Gcloud::Error
+        def initialize
+          super "GCloud binary not found in path"
+        end
+      end
+
+      ##
+      # Exception raised when the project gcloud config is not set.
+      #
+      class ProjectNotSet < Gcloud::Error
+        def initialize
+          super "GCloud project configuration not set"
+        end
+      end
+
+      ##
+      # Exception raised when gcloud auth has not been completed.
+      #
+      class GcloudNotAuthenticated < Gcloud::Error
+        def initialize
+          super "GCloud not authenticated"
+        end
+      end
+
+      ##
+      # Exception raised when gcloud fails and returns an error.
+      #
+      class GcloudFailed < Gcloud::Error
+        def initialize code
+          super "GCloud failed with result code #{code}"
+        end
+      end
+
+      class << self
+
+        ##
+        # @private
+        # Returns the path to the gcloud binary, or nil if the binary could
+        # not be found.
+        #
+        # @return [String,nil] Path to the gcloud binary.
+        #
+        def binary_path
+          unless defined? @binary_path
+            @binary_path = `which gcloud`.strip
+            @binary_path = nil if @binary_path.empty?
+          end
+          @binary_path
+        end
+
+        ##
+        # @private
+        # Returns the path to the gcloud binary. Raises BinaryNotFound if the
+        # binary could not be found.
+        #
+        # @return [String] Path to the gcloud binary.
+        # @raise [BinaryNotFound] The gcloud binary is not present.
+        #
+        def binary_path!
+          value = binary_path
+          raise BinaryNotFound unless value
+          value
+        end
+
+        ##
+        # @private
+        # Returns the ID of the current project, or nil if no project has
+        # been set.
+        #
+        # @return [String,nil] ID of the current project.
+        #
+        def current_project
+          unless defined? @current_project
+            @current_project = execute [
+              "config", "list", "core/project", "--format=value(core.project)"
+            ], capture: true
+            @current_project = nil if @current_project.empty?
+          end
+          @current_project
+        end
+
+        ##
+        # @private
+        # Returns the ID of the current project. Raises ProjectNotSet if no
+        # project has been set in the gcloud configuration.
+        #
+        # @return [String] ID of the current project.
+        # @raise [ProjectNotSet] The project config has not been set.
+        #
+        def current_project!
+          value = current_project
+          raise ProjectNotSet if value.empty?
+          value
+        end
+
+        ##
+        # @private
+        # Verifies that all gcloud related dependencies are satisfied.
+        # Specifically, verifies that the gcloud binary is installed and
+        # authenticated, and a project has been set.
+        #
+        # @raise [BinaryNotFound] The gcloud binary is not present.
+        # @raise [ProjectNotSet] The project config has not been set.
+        # @raise [GcloudNotAuthenticated] Gcloud has not been authenticated.
+        #
+        def verify!
+          binary_path!
+          current_project!
+          auths = execute ["auth", "list", "--format=value(account)"],
+              capture: true
+          raise GcloudNotAuthenticated if auths.empty?
+        end
+
+        ##
+        # @private
+        # Execute a given gcloud command in a subshell.
+        #
+        # @param args [Array<String>] The gcloud args.
+        # @param echo [boolean] Whether to echo the command to the terminal.
+        #     Defaults to false.
+        # @param capture [boolean] If true, return the output. If false, return
+        #     a boolean indicating success or failure. Defaults to false.
+        # @param assert [boolean] If true, raise GcloudFailed on failure.
+        #     Defaults to true.
+        # @return [String,Integer] Either the output or the success status,
+        #     depending on the value of the `capture` parameter.
+        #
+        def execute args, echo: false, capture: false, assert: true
+          joined_args = ::Shellwords.join args
+          cmd = "#{binary_path!} #{joined_args}"
+          puts cmd if echo
+          result = capture ? `#{cmd}` : system(cmd)
+          code = $?.exitstatus
+          raise GcloudFailed.new($?.exitstatus) if assert && code != 0
+          result
+        end
+
+        ##
+        # @private
+        # Execute a given gcloud command in a subshell, and return the output
+        # as a string.
+        #
+        # @param args [Array<String>] The gcloud args.
+        # @return [String] The command output.
+        #
+        def capture args
+          execute args, capture: true
+        end
+
+      end
+    end
+
+  end
+end

data/lib/appengine/version.rb

@@ -16,6 +16,6 @@
 module AppEngine
 
   # The current version of this gem, as a string.
-  VERSION = '0.3.0'.freeze
+  VERSION = '0.4.0'.freeze
 
 end

data/test/test_env.rb

@@ -22,7 +22,6 @@ module AppEngine
 
     class TestEnv < ::Minitest::Test  # :nodoc:
 
-
       def test_app_engine
         ::ENV["GAE_INSTANCE"] = "instance-123"
         assert Env.app_engine?
@@ -30,7 +29,6 @@ module AppEngine
         refute Env.app_engine?
       end
 
-
     end
 
   end

metadata

@@ -1,50 +1,57 @@
 --- !ruby/object:Gem::Specification
 name: appengine
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
-- Adam Tanner
 - Daniel Azuma
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-02-28 00:00:00.000000000 Z
+date: 2017-07-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: stackdriver
+  name: google-cloud-env
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.4'
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.4.1
+        version: '1.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.4'
-    - - ">="
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: stackdriver
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.7'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.4.1
+        version: '0.7'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.14'
+        version: '1.15'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.14'
+        version: '1.15'
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
@@ -87,12 +94,27 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '4.2'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.9'
 description: The appengine gem is a set of classes, plugins, and tools for integration
   with Google App Engine. It provides access to the App Engine runtime environment,
   including logging to the Google Cloud Console and interrogation of hosting properties.
-  However, it is not required for deploying your Ruby application to App Engine.
+  It also provides rake tasks for managing your App Engine application, for example
+  to run production maintenance commands. This gem is not required to deploy your
+  Ruby application to App Engine.
 email:
-- adamtanner@google.com
 - dazuma@gmail.com
 executables: []
 extensions: []
@@ -105,7 +127,10 @@ files:
 - Rakefile
 - lib/appengine.rb
 - lib/appengine/env.rb
+- lib/appengine/exec.rb
 - lib/appengine/railtie.rb
+- lib/appengine/tasks.rb
+- lib/appengine/util/gcloud.rb
 - lib/appengine/version.rb
 - test/test_env.rb
 homepage: https://github.com/GoogleCloudPlatform/appengine-ruby
@@ -128,7 +153,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.8
+rubygems_version: 2.6.11
 signing_key: 
 specification_version: 4
 summary: Google App Engine integration tools