appengine 0.4.3 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 2f0b0ccf502def04a72e834f3086b4d49067616e
4
- data.tar.gz: 6911772b6e335e325cb7569d4bc83e2662888974
2
+ SHA256:
3
+ metadata.gz: 8985bf9ca824b3567577328e9c844eabcab5691907b0eb8a3385c181c91d2f13
4
+ data.tar.gz: 04d8e4d73ee85d186798b69748191ec14423aecabfa1cbc5446d6e79e29b438c
5
5
  SHA512:
6
- metadata.gz: 1e2144f96a6afb55b57933f1c5f8603df938a23bdaa778ba7b2bf5c7e1040dc410abf1941d1ee8b2724dce8e7b188372e50f3fef5d18ef2e4c3f50b9bb6d1b96
7
- data.tar.gz: 747253cd3457c8ec7c9aa909c1927e5e622d49df4baab4649a9a4930b4b8d16666d056cea9056fda78fab43379e093111ea4f180b8b70f4434f44d3c92fcc79e
6
+ metadata.gz: fa651f55f75abf3d21813891a34c26d9c27ffe0656f7b77e54f440a2bb8d747f751317327cdc250a72fc3e286e4a76b79ac4f44ace1f70f01d8d46d3a27866ec
7
+ data.tar.gz: a03743e4a99da424fb1c588f83767389c36b82ddcb5271cba0462a5c7433bd113d3db1b668159dad5520eaa7fcb11ffcb88694133a0f46feb557ac69ada0bab2
data/.yardopts CHANGED
@@ -1,7 +1,8 @@
1
1
  --no-private
2
2
  --title=AppEngine
3
3
  --markup markdown
4
-
4
+ --markup-provider redcarpet
5
+ --main=README.md
5
6
  ./lib/**/*.rb
6
7
  -
7
8
  README.md
@@ -2,6 +2,34 @@
2
2
 
3
3
  This is the change history for the appengine gem.
4
4
 
5
+ ## v0.6.0 (2020-12-02)
6
+
7
+ * Fix failure in the appengine:exec cloud_build strategy when App Engine doesn't provide the image
8
+ * Fix exception when appengine:exec is provided a shell command rather than a command array (tpbowden)
9
+ * Update stackdriver dependency to 0.20 and google-cloud-env dependency to 1.4
10
+
11
+ ## v0.5.0 (2019-07-15)
12
+
13
+ * appengine:exec supports the App Engine standard environment.
14
+ * appengine:exec supports setting the project via `GAE_PROJECT`.
15
+ * Support for an alternate appengine:exec strategy for flexible environment apps that talk to a database via a private IP.
16
+ * Fix crash when the gcloud path includes directories with spaces.
17
+ * Escape `$` symbols in environment configs. (tpbowden)
18
+
19
+ ## v0.4.6 (2018-09-17)
20
+
21
+ * Use gcloud builds submit instead of gcloud container builds submit. (tbpg)
22
+ * Update stackdriver dependency to 0.15.
23
+
24
+ ## v0.4.5 (2017-12-04)
25
+
26
+ * Ensure tempfile is required when needed.
27
+ * Update stackdriver dependency to 0.11.
28
+
29
+ ## v0.4.4 (2017-10-03)
30
+
31
+ * Windows compatibility for appengine:exec task. (gkaykck)
32
+
5
33
  ## v0.4.3 (2017-09-20)
6
34
 
7
35
  * Fixed incorrect namespace in the handler for gcloud errors.
data/README.md CHANGED
@@ -1,6 +1,9 @@
1
1
  Google App Engine Integration Tools
2
2
  ===================================
3
3
 
4
+ [![CircleCI](https://circleci.com/gh/GoogleCloudPlatform/appengine-ruby.svg?style=svg)](https://circleci.com/gh/GoogleCloudPlatform/appengine-ruby)
5
+ [![Gem Version](https://badge.fury.io/rb/appengine.svg)](https://badge.fury.io/rb/appengine)
6
+
4
7
  This repository contains the "appengine" gem, a collection of libraries and
5
8
  plugins for integrating Ruby apps with Google App Engine. It is not required
6
9
  for deploying a ruby application to Google App Engine, but it provides a
@@ -18,7 +21,7 @@ Currently, it includes:
18
21
  * Convenient access to environment information such as project ID and VM
19
22
  properties.
20
23
 
21
- Planned for the near future:
24
+ Potential future directions:
22
25
 
23
26
  * Tools for generating "app.yaml" configuration files for Ruby applications.
24
27
  * Streamlined implementation of health checks and other lifecycle hooks.
@@ -32,6 +35,10 @@ To install, include the "appengine" gem in your Gemfile. e.g.
32
35
 
33
36
  gem "appengine"
34
37
 
38
+ ## Quick Start
39
+
40
+ ### Rails Quick Start
41
+
35
42
  If you are running [Ruby On Rails](http://rubyonrails.org/) 4.0 or later, this
36
43
  gem will automatically install a Railtie that provides its capabilities. You
37
44
  may need to include the line:
@@ -41,10 +48,48 @@ may need to include the line:
41
48
  in your `config/application.rb` file if you aren't already requiring all
42
49
  bundled gems.
43
50
 
44
- If you are running a different web framework, you may need to add some
45
- initialization code to activate the features listed below.
51
+ ### Rack Quick Start
52
+
53
+ If you are running a different Rack-based web framework, include the following
54
+ line in your main Ruby file or `config.ru`:
55
+
56
+ require "appengine"
57
+
58
+ Then, to activate Stackdriver instrumentation, add the following middleware:
59
+
60
+ use Google::Cloud::Logging::Middleware
61
+ use Google::Cloud::ErrorReporting::Middleware
62
+ use Google::Cloud::Trace::Middleware
63
+ use Google::Cloud::Debugger::Middleware
64
+
65
+ You can add the Rake tasks to your application by adding the following to your
66
+ Rakefile:
67
+
68
+ require "appengine/tasks"
69
+
70
+ ### Setting up appengine:exec remote execution
71
+
72
+ This gem is commonly used for its `appengine:exec` Rake task that provides a
73
+ way to run production tasks such as database migrations in the cloud. If you
74
+ are getting started with this feature, you should read the documentation
75
+ (available on the
76
+ [AppEngine::Exec module](http://www.rubydoc.info/gems/appengine/AppEngine/Exec))
77
+ carefully, for important tips. In particular:
78
+
79
+ * The strategy used by the gem is different depending on whether your app is
80
+ deployed to the App Engine standard environment or flexible environment.
81
+ It is important to understand which strategy is in use, because it affects
82
+ which version of your application code is used to run the task, and various
83
+ other factors.
84
+ * You may need to grant additional permissions to the service account that
85
+ runs the task. Again, the documentation will describe this in detail.
86
+ * If your app is running on the flexible environment and uses a VPC (and
87
+ connects to your database via a private IP address), then you will need to
88
+ use a special configuration for the task.
89
+
90
+ ## Using this library
46
91
 
47
- ## Logging and monitoring
92
+ ### Logging and monitoring
48
93
 
49
94
  This library automatically installs the "stackdriver" gem, which instruments
50
95
  your application to report logs, unhandled exceptions, and latency traces to
@@ -54,16 +99,21 @@ monitoring features of Google App Engine, see:
54
99
  * [google-cloud-logging instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-logging/latest/guides/instrumentation)
55
100
  * [google-cloud-error_reporting instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-error_reporting/latest/guides/instrumentation)
56
101
  * [google-cloud-trace instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-trace/latest/guides/instrumentation)
102
+ * [google-cloud-debugger instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-debugger/latest/guides/instrumentation)
57
103
 
58
104
  Rails applications automatically activate this instrumentation when the gem
59
105
  is present. You may opt out of individual services by providing appropriate
60
- Rails configuration. See {AppEngine::Railtie} for more information.
106
+ Rails configuration. See
107
+ [AppEngine::Railtie](http://www.rubydoc.info/gems/appengine/AppEngine/Railtie)
108
+ for more information.
61
109
 
62
110
  Non-Rails applications must provide initialization code to activate this
63
- instrumentation, typically by installing a Rack middleware. See the individual
64
- service documentation links above for more information.
111
+ instrumentation, typically by installing a Rack middleware. You can find the
112
+ basic code for installing these middlewares in the Rack Quick Start section
113
+ above. See the individual service documentation links above for more
114
+ information and configuration options.
65
115
 
66
- ## App Engine remote execution
116
+ ### App Engine remote execution
67
117
 
68
118
  This library provides rake tasks for App Engine remote execution, allowing
69
119
  App Engine applications to perform on-demand tasks in the App Engine
@@ -73,17 +123,17 @@ example, you could run a production database migration in a Rails app using:
73
123
 
74
124
  bundle exec rake appengine:exec -- bundle exec rake db:migrate
75
125
 
76
- The migration would be run in VMs provided by Google Cloud. It uses a
77
- privileged service account that will have access to the production cloud
78
- resources, such as Cloud SQL instances, used by the application. This mechanism
79
- is often much easier and safer than running the task on a local workstation and
80
- granting that workstation direct access to those Cloud SQL instances.
126
+ The migration would be run in containers on Google Cloud infrastructure, which
127
+ is much easier and safer than running the task on a local workstation and
128
+ granting that workstation direct access to your production database.
81
129
 
82
- See {AppEngine::Exec} for more information on App Engine remote execution.
130
+ See [AppEngine::Exec](http://www.rubydoc.info/gems/appengine/AppEngine/Exec)
131
+ for more information on App Engine remote execution.
83
132
 
84
- See {AppEngine::Tasks} for more information on running the rake tasks. The
85
- tasks are available automatically in Rails applications when the gem is
86
- present. Non-Rails applications may install the tasks by adding the line
133
+ See [AppEngine::Tasks](http://www.rubydoc.info/gems/appengine/AppEngine/Tasks)
134
+ for more information on running the Rake tasks. The tasks are available
135
+ automatically in Rails applications when the gem is present. Non-Rails
136
+ applications may install the tasks by adding the line
87
137
  `require "appengine/tasks"` to the `Rakefile`.
88
138
 
89
139
  ## Development and support
@@ -0,0 +1,116 @@
1
+ # Copyright 2019 Google LLC
2
+ #
3
+ # Licensed under the Apache License, Version 2.0 (the "License");
4
+ # you may not use this file except in compliance with the License.
5
+ # You may obtain a copy of the License at
6
+ #
7
+ # http://www.apache.org/licenses/LICENSE-2.0
8
+ #
9
+ # Unless required by applicable law or agreed to in writing, software
10
+ # distributed under the License is distributed on an "AS IS" BASIS,
11
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ # See the License for the specific language governing permissions and
13
+ # limitations under the License.
14
+
15
+ require "webrick"
16
+ require "monitor"
17
+ require "json"
18
+
19
+ SECRET = <%= secret %>
20
+ COMMAND = Array(<%= command %>)
21
+
22
+ port = Integer ENV["PORT"]
23
+ server = ::WEBrick::HTTPServer.new Port: port
24
+
25
+ Status = ::Struct.new :out_lines, :err_lines, :exit_status, :pid, :start_time
26
+ $status = Status.new [], [], nil, nil, nil
27
+ $status.extend ::MonitorMixin
28
+
29
+ def do_start
30
+ $status.synchronize do
31
+ return unless $status.pid.nil?
32
+ end
33
+ $stdout.puts "Executing: #{COMMAND.inspect}"
34
+ $stdout.flush
35
+ rout, wout = ::IO.pipe
36
+ rerr, werr = ::IO.pipe
37
+ ::Thread.new do
38
+ rout.each_line do |line|
39
+ $status.synchronize { $status.out_lines << line }
40
+ $stdout.puts line
41
+ $stdout.flush
42
+ end
43
+ end
44
+ ::Thread.new do
45
+ rerr.each_line do |line|
46
+ $status.synchronize { $status.err_lines << line }
47
+ $stderr.puts line
48
+ $stderr.flush
49
+ end
50
+ end
51
+ start_time = Time.now.to_i
52
+ pid = ::Process.spawn *COMMAND, err: werr, out: wout
53
+ werr.close
54
+ wout.close
55
+ $status.synchronize do
56
+ $status.pid = pid
57
+ $status.start_time = start_time
58
+ end
59
+ ::Thread.new do
60
+ _pid, status = ::Process.wait2 pid
61
+ $status.synchronize do
62
+ $status.exit_status = status.exitstatus
63
+ end
64
+ end
65
+ end
66
+
67
+ def get_status outpos: 0, errpos: 0
68
+ outlines, errlines, status, start_time =
69
+ $status.synchronize do
70
+ [
71
+ $status.out_lines[outpos..-1],
72
+ $status.err_lines[errpos..-1],
73
+ $status.exit_status,
74
+ $status.start_time
75
+ ]
76
+ end
77
+ {
78
+ "outpos" => outpos + outlines.size,
79
+ "errpos" => errpos + errlines.size,
80
+ "outlines" => outlines,
81
+ "errlines" => errlines,
82
+ "status" => status,
83
+ "time" => Time.now.to_i - start_time
84
+ }
85
+ end
86
+
87
+ server.mount_proc "/_ah/start" do |req, res|
88
+ do_start
89
+ end
90
+
91
+ server.mount_proc "/#{SECRET}" do |req, res|
92
+ do_start
93
+ status = get_status outpos: req.query["outpos"].to_i,
94
+ errpos: req.query["errpos"].to_i
95
+ res.body = JSON.dump status
96
+ end
97
+
98
+ server.mount_proc "/#{SECRET}/kill" do |req, res|
99
+ unless req.request_method == "POST"
100
+ res.status = 404
101
+ return
102
+ end
103
+ $status.synchronize do
104
+ if $status.pid.nil?
105
+ res.status = 404
106
+ return
107
+ end
108
+ ::Process.kill "SIGTERM", $status.pid
109
+ end
110
+ end
111
+
112
+ begin
113
+ server.start
114
+ ensure
115
+ server.shutdown
116
+ end
@@ -1,4 +1,6 @@
1
- # Copyright 2016 Google Inc. All rights reserved.
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2019 Google LLC
2
4
  #
3
5
  # Licensed under the Apache License, Version 2.0 (the "License");
4
6
  # you may not use this file except in compliance with the License.
@@ -11,17 +13,22 @@
11
13
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
14
  # See the License for the specific language governing permissions and
13
15
  # limitations under the License.
14
- ;
15
16
 
16
- # # Google AppEngine integration
17
+
18
+ ##
19
+ # ## Google AppEngine integration
17
20
  #
18
21
  # The AppEngine module includes optional tools helping Ruby applications to
19
22
  # integrate more closely with the Google App Engine environment.
20
-
23
+ #
21
24
  module AppEngine
25
+ ##
26
+ # Internal utilities
27
+ #
28
+ module Util
29
+ end
22
30
  end
23
31
 
24
-
25
32
  require "appengine/version"
26
33
  require "appengine/env"
27
34
  require "appengine/exec"
@@ -1,4 +1,6 @@
1
- # Copyright 2016 Google Inc. All rights reserved.
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2019 Google LLC
2
4
  #
3
5
  # Licensed under the Apache License, Version 2.0 (the "License");
4
6
  # you may not use this file except in compliance with the License.
@@ -11,13 +13,11 @@
11
13
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
14
  # See the License for the specific language governing permissions and
13
15
  # limitations under the License.
14
- ;
15
16
 
16
- require "google/cloud/env"
17
17
 
18
+ require "google/cloud/env"
18
19
 
19
20
  module AppEngine
20
-
21
21
  ##
22
22
  # A convenience object that provides information on the Google Cloud
23
23
  # hosting environment. For example, you can call
@@ -34,5 +34,4 @@ module AppEngine
34
34
  # directly instead. See the documentation for the `google-cloud-env` gem.
35
35
  #
36
36
  Env = ::Google::Cloud.env
37
-
38
37
  end
@@ -1,4 +1,6 @@
1
- # Copyright 2017 Google Inc. All rights reserved.
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2019 Google LLC
2
4
  #
3
5
  # Licensed under the Apache License, Version 2.0 (the "License");
4
6
  # you may not use this file except in compliance with the License.
@@ -11,16 +13,20 @@
11
13
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
14
  # See the License for the specific language governing permissions and
13
15
  # limitations under the License.
14
- ;
15
16
 
16
- require "yaml"
17
+
18
+ require "date"
19
+ require "erb"
17
20
  require "json"
21
+ require "net/http"
22
+ require "securerandom"
23
+ require "shellwords"
24
+ require "tempfile"
25
+ require "yaml"
18
26
 
19
27
  require "appengine/util/gcloud"
20
28
 
21
-
22
29
  module AppEngine
23
-
24
30
  ##
25
31
  # # App Engine remote execution
26
32
  #
@@ -31,90 +37,219 @@ module AppEngine
31
37
  #
32
38
  # ## About App Engine execution
33
39
  #
34
- # App Engine execution spins up an image of a deployed App Engine app, and
35
- # runs a command in that image. For example, if your app runs on Ruby on
36
- # Rails, then your app provides a `bin/rails` tool, and you may invoke it
37
- # using App Engine execution---for example to run a command such as
38
- # `bundle exec bin/rails db:migrate` in the image.
40
+ # App Engine execution spins up a one-off copy of an App Engine app, and runs
41
+ # a command against it. For example, if your app runs on Ruby on Rails, then
42
+ # you might use App Engine execution to run a command such as
43
+ # `bundle exec bin/rails db:migrate` in production infrastructure (to avoid
44
+ # having to connect directly to your production database from a local
45
+ # workstation).
46
+ #
47
+ # App Engine execution provides two strategies for generating that "one-off
48
+ # copy":
49
+ #
50
+ # * A `deployment` strategy, which deploys a temporary version of your app
51
+ # to a single backend instance and runs the command there.
52
+ # * A `cloud_build` strategy, which deploys your application image to
53
+ # Google Cloud Build and runs the command there.
39
54
  #
40
- # When App Engine execution runs your command, it provides access to key
41
- # elements of the App Engine environment, including:
55
+ # Both strategies are generally designed to emulate the App Engine runtime
56
+ # environment on cloud VMs similar to those used by actual deployments of
57
+ # your app. Both provide your application code and environment variables, and
58
+ # both provide access to Cloud SQL connections used by your app. However,
59
+ # they differ in what *version* of your app code they run against, and in
60
+ # certain other constraints and performance characteristics. More detailed
61
+ # information on using the two strategies is provided in the sections below.
42
62
  #
43
- # * The same runtime that runs your application in App Engine itself.
44
- # * Any Cloud SQL connections requested by your application.
45
- # * Any environment variables set by your application.
63
+ # Apps deployed to the App Engine *flexible environment* will use the
64
+ # `cloud_build` strategy by default. However, you can force an app to use the
65
+ # `deployment` strategy instead. (You might do so if you need to connect to a
66
+ # Cloud SQL database on a VPC using a private IP, because the `cloud_build`
67
+ # strategy does not support private IPs.) To force use of `deployment`, set
68
+ # the `strategy` parameter in the {AppEngine::Exec} constructor (or the
69
+ # corresponding `GAE_EXEC_STRATEGY` parameter in the Rake task). Note that
70
+ # the `deployment` strategy is usually significantly slower than
71
+ # `cloud_build` for apps in the flexible environment.
46
72
  #
47
- # The command runs on virtual machines provided by Google Cloud Container
48
- # Builder, and has access to the credentials of the Cloud Container Builder
49
- # service account.
73
+ # Apps deployed to the App Engine *standard environment* will *always* use
74
+ # the `deployment` strategy. You cannot force use of the `cloud_build`
75
+ # strategy.
50
76
  #
51
77
  # ## Prerequisites
52
78
  #
53
79
  # To use App Engine remote execution, you will need:
54
80
  #
55
81
  # * An app deployed to Google App Engine, of course!
56
- # * The gcloud SDK installed and configured. See https://cloud.google.com/sdk/
82
+ # * The [gcloud SDK](https://cloud.google.com/sdk/) installed and configured.
57
83
  # * The `appengine` gem.
58
84
  #
59
- # You may also need to grant the Cloud Container Builder service account
60
- # any permissions needed by your command. Often, Project Editor permissions
61
- # will be sufficient for most tasks. You can find the service account
62
- # configuration in the IAM tab in the Cloud Console under the name
63
- # `[your-project-number]@cloudbuild.gserviceaccount.com`.
64
- #
65
85
  # You may use the `AppEngine::Exec` class to run commands directly. However,
66
86
  # in most cases, it will be easier to run commands via the provided rake
67
- # tasks. See {AppEngine::Tasks} for more info.
87
+ # tasks (see {AppEngine::Tasks}).
88
+ #
89
+ # ## Using the "deployment" strategy
90
+ #
91
+ # The `deployment` strategy deploys a temporary version of your app to a
92
+ # single backend App Engine instance, runs the command there, and then
93
+ # deletes the temporary version when it is finished.
94
+ #
95
+ # This is the default strategy (and indeed the only option) for apps running
96
+ # on the App Engine standard environment. It can also be used for flexible
97
+ # environment apps, but this is not commonly done because deployment of
98
+ # flexible environment apps can take a long time.
99
+ #
100
+ # Because the `deployment` strategy deploys a temporary version of your app,
101
+ # it runs against the *current application code* present where the command
102
+ # was initiated (i.e. the code currently on your workstation if you run the
103
+ # rake task from your workstation, or the current code on the branch if you
104
+ # are running from a CI/CD system.) This may be different from the code
105
+ # actually running in production, so it is important that you run from a
106
+ # compatible code branch.
68
107
  #
69
- # ## Configuring
108
+ # ### Specifying the host application
70
109
  #
71
- # This class uses three parameters to specify which application image to use
72
- # to run your command: `service`, `config_path`, and `version`.
110
+ # The `deployment` strategy works by deploying a temporary version of your
111
+ # app, so that it has access to your app's project and settings in App
112
+ # Engine. In most cases, it can determine this automatically, but depending
113
+ # on how your app or environment is structured, you may need to give it some
114
+ # help.
73
115
  #
74
- # In most cases, you can use the defaults. The Exec class will look in your
75
- # current directory for a file called `./app.yaml` which describes your App
76
- # Engine service. It gets the service name from this file (or uses the
77
- # "default" name if none is specified), then looks up the most recently
78
- # created deployment version for that service. That deployment version then
79
- # provides the application image that runs your command.
116
+ # By default, your Google Cloud project is taken from the current gcloud
117
+ # project. If you need to override this, set the `:project` parameter in the
118
+ # {AppEngine::Exec} constructor (or the corresponding `GAE_PROJECT`
119
+ # parameter in the Rake task).
80
120
  #
81
- # If your app has multiple services, you may specify which config file
82
- # (other than `./app.yaml`) describes the desired service, by providing the
83
- # `config_path` parameter. Alternately, you may specify a service name
84
- # directly by providing the `service` parameter. If you provide both
85
- # parameters, `service` takes precedence.
121
+ # By default, the service name is taken from the App Engine config file.
122
+ # App Engine execution will assume this file is called `app.yaml` in the
123
+ # current directory. To use a different config file, set the `config_path`
124
+ # parameter in the {AppEngine::Exec} constructor (or the corresponding
125
+ # `GAE_CONFIG` parameter in the Rake task). You may also set the service name
126
+ # directly, using the `service` parameter (or `GAE_SERVICE` in Rake).
86
127
  #
87
- # Usually, App Engine execution uses the image for the most recently created
88
- # version of the service. (Note: the most recently created version is used,
89
- # regardless of whether that version is currently receiving traffic.) If you
90
- # want to use the image for a different version, you may specify a version
91
- # by providing the `version` parameter.
128
+ # ### Providing credentials
129
+ #
130
+ # Your command will effectively be a deployment of your App Engine app
131
+ # itself, and will have access to the same credentials. For example, App
132
+ # Engine provides a service account by default for your app, or your app may
133
+ # be making use of its own service account key. In either case, make sure the
134
+ # service account has sufficient access for the command you want to run
135
+ # (such as database admin credentials).
136
+ #
137
+ # ### Other options
92
138
  #
93
139
  # You may also provide a timeout, which is the length of time that App
94
140
  # Engine execution will allow your command to run before it is considered to
95
141
  # have stalled and is terminated. The timeout should be a string of the form
96
142
  # `2h15m10s`. The default is `10m`.
97
143
  #
98
- # ## Resource usage and billing
144
+ # The timeout is set via the `timeout` parameter to the {AppEngine::Exec}
145
+ # constructor, or by setting the `GAE_TIMEOUT` environment variable when
146
+ # invoking using Rake.
147
+ #
148
+ # ### Resource usage and billing
149
+ #
150
+ # The `deployment` strategy deploys to a temporary instance of your app in
151
+ # order to run the command. You may be billed for that usage. However, the
152
+ # cost should be minimal, because it will then immediately delete that
153
+ # instance in order to minimize usage.
154
+ #
155
+ # If you interrupt the execution (or it crashes), it is possible that the
156
+ # temporary instance may not get deleted properly. If you suspect this may
157
+ # have happened, go to the App Engine tab in the cloud console, under
158
+ # "versions" of your service, and delete the temporary version manually. It
159
+ # will have a name matching the pattern `appengine-exec-<timestamp>`.
160
+ #
161
+ # ## Using the "cloud_build" strategy
162
+ #
163
+ # The `cloud_build` strategy takes the application image that App Engine is
164
+ # actually using to run your app, and uses it to spin up a copy of your app
165
+ # in [Google Cloud Build](https://cloud.google.com/cloud-build) (along with
166
+ # an emulation layer that emulates certain App Engine services such as Cloud
167
+ # SQL connection sockets). The command then gets run in the Cloud Build
168
+ # environment.
169
+ #
170
+ # This is the default strategy for apps running on the App Engine flexible
171
+ # environment. (It is not available for standard environment apps.) Note that
172
+ # the `cloud_build` strategy cannot be used if your command needs to connect
173
+ # to a database over a [VPC](https://cloud.google.com/vpc/) private IP
174
+ # address. This is because it runs on virtual machines provided by the Cloud
175
+ # Build service, which are not part of your VPC. If your database can be
176
+ # accessed only over a private IP, you should use the `deployment` strategy
177
+ # instead.
178
+ #
179
+ # The Cloud Build log is output to the directory specified by
180
+ # "CLOUD_BUILD_GCS_LOG_DIR". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
181
+ # By default, log directory name is
182
+ # "gs://[PROJECT_NUMBER].cloudbuild-logs.googleusercontent.com/".
183
+ #
184
+ # ### Specifying the host application
185
+ #
186
+ # The `cloud_build` strategy needs to know exactly which app, service, and
187
+ # version of your app, to identify the application image to use.
188
+ #
189
+ # By default, your Google Cloud project is taken from the current gcloud
190
+ # project. If you need to override this, set the `:project` parameter in the
191
+ # {AppEngine::Exec} constructor (or the corresponding `GAE_PROJECT`
192
+ # parameter in the Rake task).
193
+ #
194
+ # By default, the service name is taken from the App Engine config file.
195
+ # App Engine execution will assume this file is called `app.yaml` in the
196
+ # current directory. To use a different config file, set the `config_path`
197
+ # parameter in the {AppEngine::Exec} constructor (or the corresponding
198
+ # `GAE_CONFIG` parameter in the Rake task). You may also set the service name
199
+ # directly, using the `service` parameter (or `GAE_SERVICE` in Rake).
200
+ #
201
+ # By default, the image of the most recently deployed version of your app is
202
+ # used. (Note that this most recently deployed version may not be the same
203
+ # version that is currently receiving traffic: for example, if you deployed
204
+ # with `--no-promote`.) To use a different version, set the `version`
205
+ # parameter in the {AppEngine::Exec} constructor (or the corresponding
206
+ # `GAE_VERSION` parameter in the Rake task).
99
207
  #
100
- # App Engine remote execution uses virtual machine resources provided by
101
- # Google Cloud Container Builder. Generally, a certain number of usage
102
- # minutes per day is covered under a free tier, but additional compute usage
103
- # beyond that time is billed to your Google Cloud account. For more details,
104
- # see https://cloud.google.com/container-builder/pricing
208
+ # ### Providing credentials
209
+ #
210
+ # By default, the `cloud_build` strategy uses your project's Cloud Build
211
+ # service account for its credentials. Unless your command provides its own
212
+ # service account key, you may need to grant the Cloud Build service account
213
+ # any permissions needed to execute your command (such as access to your
214
+ # database). For most tasks, it is sufficient to grant Project Editor
215
+ # permissions to the service account. You can find the service account
216
+ # configuration in the IAM tab in the Cloud Console under the name
217
+ # `[your-project-number]@cloudbuild.gserviceaccount.com`.
218
+ #
219
+ # ### Other options
220
+ #
221
+ # You may also provide a timeout, which is the length of time that App
222
+ # Engine execution will allow your command to run before it is considered to
223
+ # have stalled and is terminated. The timeout should be a string of the form
224
+ # `2h15m10s`. The default is `10m`.
225
+ #
226
+ # The timeout is set via the `timeout` parameter to the {AppEngine::Exec}
227
+ # constructor, or by setting the `GAE_TIMEOUT` environment variable when
228
+ # invoking using Rake.
229
+ #
230
+ # You can also set the wrapper image used to emulate the App Engine runtime
231
+ # environment, by setting the `wrapper_image` parameter to the constructor,
232
+ # or by setting the `GAE_EXEC_WRAPPER_IMAGE` environment variable. Generally,
233
+ # you will not need to do this unless you are testing a new wrapper image.
234
+ #
235
+ # ### Resource usage and billing
236
+ #
237
+ # The `cloud_build` strategy uses virtual machine resources provided by
238
+ # Google Cloud Build. Generally, a certain number of usage minutes per day is
239
+ # covered under a free tier, but additional compute usage beyond that time is
240
+ # billed to your Google Cloud account. For more details,
241
+ # see https://cloud.google.com/cloud-build/pricing
105
242
  #
106
243
  # If your command makes API calls or utilizes other cloud resources, you may
107
- # also be billed for that usage. However, remote execution does not use
108
- # actual App Engine instances, and you will not be billed for additional App
109
- # Engine instance usage.
244
+ # also be billed for that usage. However, the `cloud_build` strategy (unlike
245
+ # the `deployment` strategy) does not use actual App Engine instances, and
246
+ # you will not be billed for additional App Engine instance usage.
110
247
  #
111
248
  class Exec
112
-
113
- @default_timeout = "10m".freeze
114
- @default_service = "default".freeze
115
- @default_config_path = "./app.yaml".freeze
116
- @default_wrapper_image = "gcr.io/google-appengine/exec-wrapper".freeze
117
-
249
+ @default_timeout = "10m"
250
+ @default_service = "default"
251
+ @default_config_path = "./app.yaml"
252
+ @default_wrapper_image = "gcr.io/google-appengine/exec-wrapper:latest"
118
253
 
119
254
  ##
120
255
  # Base class for exec-related usage errors.
@@ -122,6 +257,41 @@ module AppEngine
122
257
  class UsageError < ::StandardError
123
258
  end
124
259
 
260
+ ##
261
+ # Unsupported strategy
262
+ #
263
+ class UnsupportedStrategy < UsageError
264
+ def initialize strategy, app_env
265
+ @strategy = strategy
266
+ @app_env = app_env
267
+ super "Strategy \"#{strategy}\" not supported for the #{app_env}" \
268
+ " environment"
269
+ end
270
+ attr_reader :strategy
271
+ attr_reader :app_env
272
+ end
273
+
274
+ ##
275
+ # Exception raised when a parameter is malformed.
276
+ #
277
+ class BadParameter < UsageError
278
+ def initialize param, value
279
+ @param_name = param
280
+ @value = value
281
+ super "Bad value for #{param}: #{value}"
282
+ end
283
+ attr_reader :param_name
284
+ attr_reader :value
285
+ end
286
+
287
+ ##
288
+ # Exception raised when gcloud has no default project.
289
+ #
290
+ class NoDefaultProject < UsageError
291
+ def initialize
292
+ super "No default project set."
293
+ end
294
+ end
125
295
 
126
296
  ##
127
297
  # Exception raised when the App Engine config file could not be found.
@@ -150,7 +320,7 @@ module AppEngine
150
320
  # versions at all could be found for the given service.
151
321
  #
152
322
  class NoSuchVersion < UsageError
153
- def initialize service, version=nil
323
+ def initialize service, version = nil
154
324
  @service = service
155
325
  @version = version
156
326
  if version
@@ -163,25 +333,7 @@ module AppEngine
163
333
  attr_reader :version
164
334
  end
165
335
 
166
- ##
167
- # Exception raised when an explicitly-specified service name conflicts with
168
- # a config-specified service name.
169
- #
170
- class ServiceNameConflict < UsageError
171
- def initialize service_name, config_name, config_path
172
- @service_name = service_name
173
- @config_name = config_name
174
- @config_path = config_path
175
- super "Service name conflicts with config file"
176
- end
177
- attr_reader :service_name
178
- attr_reader :config_name
179
- attr_reader :config_path
180
- end
181
-
182
-
183
336
  class << self
184
-
185
337
  ## @return [String] Default command timeout.
186
338
  attr_accessor :default_timeout
187
339
 
@@ -205,78 +357,138 @@ module AppEngine
205
357
  # the service name from the config file.
206
358
  # @param config_path [String,nil] App Engine config file to get the
207
359
  # service name from if the service name is not provided directly.
208
- # Defaults to the value of `AppEngine::Exec.default_config_path`.
209
- # @param version [String,nil] Version string. Defaults to the most
210
- # recently created version of the given service (which may not be the
211
- # one currently receiving traffic).
212
- # @param timeout [String,nil] Timeout string. Defaults to the value of
213
- # `AppEngine::Exec.default_timeout`.
360
+ # If omitted, defaults to the value returned by
361
+ # {AppEngine::Exec.default_config_path}.
362
+ # @param version [String,nil] Version string. If omitted, defaults to the
363
+ # most recently created version of the given service (which may not
364
+ # be the one currently receiving traffic).
365
+ # @param timeout [String,nil] Timeout string. If omitted, defaults to the
366
+ # value returned by {AppEngine::Exec.default_timeout}.
367
+ # @param wrapper_image [String,nil] The fully qualified name of the
368
+ # wrapper image to use. (Applies only to the "cloud_build" strategy.)
369
+ # @param strategy [String,nil] The execution strategy to use, or `nil` to
370
+ # choose a default based on the App Engine environment (flexible or
371
+ # standard). Allowed values are `nil`, `"deployment"` (which is the
372
+ # default for Standard), and `"cloud_build"` (which is the default
373
+ # for Flexible).
374
+ # @param gcs_log_dir [String,nil] GCS bucket name of the cloud build log
375
+ # when strategy is "cloud_build". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
214
376
  #
215
377
  def new_rake_task name, args: [], env_args: [],
216
378
  service: nil, config_path: nil, version: nil,
217
- timeout: nil
218
- escaped_args = args.map{ |arg|
219
- arg.gsub(/[,\[\]]/){ |m| "\\#{m}" }
220
- }
221
- if escaped_args.empty?
222
- name_with_args = name
223
- else
224
- name_with_args = "#{name}[#{escaped_args.join ','}]"
379
+ timeout: nil, project: nil, wrapper_image: nil,
380
+ strategy: nil, gcs_log_dir: nil
381
+ escaped_args = args.map do |arg|
382
+ arg.gsub(/[,\[\]]/) { |m| "\\#{m}" }
225
383
  end
384
+ name_with_args =
385
+ if escaped_args.empty?
386
+ name
387
+ else
388
+ "#{name}[#{escaped_args.join ','}]"
389
+ end
226
390
  new ["bundle", "exec", "rake", name_with_args] + env_args,
227
391
  service: service, config_path: config_path, version: version,
228
- timeout: timeout
392
+ timeout: timeout, project: project, wrapper_image: wrapper_image,
393
+ strategy: strategy, gcs_log_dir: gcs_log_dir
229
394
  end
230
-
231
395
  end
232
396
 
233
-
234
397
  ##
235
398
  # Create an execution for the given command.
236
399
  #
237
400
  # @param command [Array<String>] The command in array form.
401
+ # @param project [String,nil] ID of the project. If omitted, obtains
402
+ # the project from gcloud.
238
403
  # @param service [String,nil] Name of the service. If omitted, obtains
239
404
  # the service name from the config file.
240
405
  # @param config_path [String,nil] App Engine config file to get the
241
406
  # service name from if the service name is not provided directly.
242
- # Defaults to the value of `AppEngine::Exec.default_config_path`.
243
- # @param version [String,nil] Version string. Defaults to the most
244
- # recently created version of the given service (which may not be the
245
- # one currently receiving traffic).
246
- # @param timeout [String,nil] Timeout string. Defaults to the value of
247
- # `AppEngine::Exec.default_timeout`.
407
+ # If omitted, defaults to the value returned by
408
+ # {AppEngine::Exec.default_config_path}.
409
+ # @param version [String,nil] Version string. If omitted, defaults to the
410
+ # most recently created version of the given service (which may not be
411
+ # the one currently receiving traffic).
412
+ # @param timeout [String,nil] Timeout string. If omitted, defaults to the
413
+ # value returned by {AppEngine::Exec.default_timeout}.
414
+ # @param wrapper_image [String,nil] The fully qualified name of the wrapper
415
+ # image to use. (Applies only to the "cloud_build" strategy.)
416
+ # @param strategy [String,nil] The execution strategy to use, or `nil` to
417
+ # choose a default based on the App Engine environment (flexible or
418
+ # standard). Allowed values are `nil`, `"deployment"` (which is the
419
+ # default for Standard), and `"cloud_build"` (which is the default for
420
+ # Flexible).
421
+ # @param gcs_log_dir [String,nil] GCS bucket name of the cloud build log
422
+ # when strategy is "cloud_build". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
248
423
  #
249
424
  def initialize command,
250
- service: nil, config_path: nil, version: nil, timeout: nil
425
+ project: nil, service: nil, config_path: nil, version: nil,
426
+ timeout: nil, wrapper_image: nil, strategy: nil, gcs_log_dir: nil
251
427
  @command = command
252
428
  @service = service
253
429
  @config_path = config_path
254
430
  @version = version
255
431
  @timeout = timeout
256
- @wrapper_image = nil
432
+ @project = project
433
+ @wrapper_image = wrapper_image
434
+ @strategy = strategy
435
+ @gcs_log_dir = gcs_log_dir
257
436
 
258
437
  yield self if block_given?
259
438
  end
260
439
 
440
+ ##
441
+ # @return [String] The project ID.
442
+ # @return [nil] if the default gcloud project should be used.
443
+ #
444
+ attr_accessor :project
261
445
 
262
- ## @return [String,nil] The service name, or nil to read from the config.
446
+ ##
447
+ # @return [String] The service name.
448
+ # @return [nil] if the service should be obtained from the app config.
449
+ #
263
450
  attr_accessor :service
264
451
 
265
- ## @return [String,nil] Path to the config file, or nil to use the default.
452
+ ##
453
+ # @return [String] Path to the config file.
454
+ # @return [nil] if the default of `./app.yaml` should be used.
455
+ #
266
456
  attr_accessor :config_path
267
457
 
268
- ## @return [String,nil] Service version, or nil to use the most recent.
458
+ ##
459
+ # @return [String] Service version of the image to use.
460
+ # @return [nil] if the most recent should be used.
461
+ #
269
462
  attr_accessor :version
270
463
 
271
- ## @return [String,nil] Command timeout, or nil to use the default.
464
+ ##
465
+ # @return [String] The command timeout, in `1h23m45s` format.
466
+ # @return [nil] if the default of `10m` should be used.
467
+ #
272
468
  attr_accessor :timeout
273
469
 
274
- ## @return [String,Array<String>] Command to run.
470
+ ##
471
+ # The command to run.
472
+ #
473
+ # @return [String] if the command is a script to be run in a shell.
474
+ # @return [Array<String>] if the command is a posix command to be run
475
+ # directly without a shell.
476
+ #
275
477
  attr_accessor :command
276
478
 
277
- ## @return [String] Custom wrapper image to use, or nil to use the default.
479
+ ##
480
+ # @return [String] Custom wrapper image to use.
481
+ # @return [nil] if the default should be used.
482
+ #
278
483
  attr_accessor :wrapper_image
279
484
 
485
+ ##
486
+ # @return [String] The execution strategy to use. Allowed values are
487
+ # `"deployment"` and `"cloud_build"`.
488
+ # @return [nil] to choose a default based on the App Engine environment
489
+ # (flexible or standard).
490
+ #
491
+ attr_accessor :strategy
280
492
 
281
493
  ##
282
494
  # Executes the command synchronously. Streams the logs back to standard out
@@ -284,29 +496,15 @@ module AppEngine
284
496
  #
285
497
  def start
286
498
  resolve_parameters
287
-
288
- version_info = version_info @service, @version
289
- env_variables = version_info["envVariables"] || {}
290
- beta_settings = version_info["betaSettings"] || {}
291
- cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
292
- image = version_info["deployment"]["container"]["image"]
293
-
294
- config = build_config command, image, env_variables, cloud_sql_instances
295
- file = ::Tempfile.new ["cloudbuild_", ".json"]
296
- begin
297
- ::JSON.dump config, file
298
- file.flush
299
- Util::Gcloud.execute [
300
- "container", "builds", "submit",
301
- "--no-source",
302
- "--config=#{file.path}",
303
- "--timeout=#{@timeout}"]
304
- ensure
305
- file.close!
499
+ app_info = version_info @service, @version
500
+ resolve_strategy app_info["env"]
501
+ if @strategy == "cloud_build"
502
+ start_build_strategy app_info
503
+ else
504
+ start_deployment_strategy app_info
306
505
  end
307
506
  end
308
507
 
309
-
310
508
  private
311
509
 
312
510
  ##
@@ -314,61 +512,55 @@ module AppEngine
314
512
  # Resolves and canonicalizes all the parameters.
315
513
  #
316
514
  def resolve_parameters
317
- unless @command.is_a? Array
318
- @command = ::Shellwords.parse @command.to_s
319
- end
320
-
321
- config_service = config_path = nil
322
- if @config_path || !@service
323
- config_service = begin
324
- config_path = @config_path || Exec.default_config_path
325
- ::YAML.load_file(config_path)["service"] || Exec.default_service
326
- rescue ::Errno::ENOENT
327
- raise ConfigFileNotFound.new config_path
328
- rescue
329
- raise BadConfigFileFormat.new config_path
330
- end
331
- end
332
- if @service && config_service && @service != config_service
333
- raise ServiceNameConflict.new @service, config_service, config_path
334
- end
335
-
336
- @service ||= config_service
515
+ @timestamp_suffix = ::Time.now.strftime "%Y%m%d%H%M%S"
516
+ @command = ::Shellwords.split @command.to_s unless @command.is_a? Array
517
+ @project ||= default_project
518
+ @service ||= service_from_config || Exec.default_service
337
519
  @version ||= latest_version @service
338
520
  @timeout ||= Exec.default_timeout
521
+ @timeout_seconds = parse_timeout @timeout
339
522
  @wrapper_image ||= Exec.default_wrapper_image
523
+ self
340
524
  end
341
525
 
342
- ##
343
- # @private
344
- # Builds a cloudbuild config as a data structure.
345
- #
346
- # @param command [Array<String>] The command in array form.
347
- # @param image [String] The fully qualified image path.
348
- # @param env_variables[Hash<String,String>] Environment variables.
349
- # @param cloud_sql_instances[String,Array<String>] Names of cloud sql
350
- # instances to connect to.
351
- #
352
- def build_config command, image, env_variables, cloud_sql_instances
353
- args = ["-i", image]
354
- env_variables.each do |k, v|
355
- args << "-e" << "#{k}=#{v}"
526
+ def resolve_strategy app_env
527
+ @strategy = @strategy.to_s.downcase
528
+ if @strategy.empty?
529
+ @strategy = app_env == "flexible" ? "cloud_build" : "deployment"
356
530
  end
357
- unless cloud_sql_instances.empty?
358
- cloud_sql_instances = Array(cloud_sql_instances)
359
- cloud_sql_instances.each do |sql|
360
- args << "-s" << sql
361
- end
531
+ if app_env == "standard" && @strategy == "cloud_build" ||
532
+ @strategy != "cloud_build" && @strategy != "deployment"
533
+ raise UnsupportedStrategy.new @strategy, app_env
362
534
  end
363
- args << "--"
364
- args += command
535
+ @strategy
536
+ end
365
537
 
366
- {
367
- "steps" => [
368
- "name" => @wrapper_image,
369
- "args" => args
370
- ]
371
- }
538
+ def service_from_config
539
+ return nil if !@config_path && @service
540
+ @config_path ||= Exec.default_config_path
541
+ ::YAML.load_file(config_path)["service"]
542
+ rescue ::Errno::ENOENT
543
+ raise ConfigFileNotFound, @config_path
544
+ rescue ::StandardError
545
+ raise BadConfigFileFormat, @config_path
546
+ end
547
+
548
+ def default_project
549
+ result = Util::Gcloud.execute \
550
+ ["config", "get-value", "project"],
551
+ capture: true, assert: false
552
+ result.strip!
553
+ raise NoDefaultProject if result.empty?
554
+ result
555
+ end
556
+
557
+ def parse_timeout timeout_str
558
+ matched = timeout_str =~ /^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)s?)?$/
559
+ raise BadParameter.new "timeout", timeout_str unless matched
560
+ hours = ::Regexp.last_match(1).to_i
561
+ minutes = ::Regexp.last_match(2).to_i
562
+ seconds = ::Regexp.last_match(3).to_i
563
+ hours * 3600 + minutes * 60 + seconds
372
564
  end
373
565
 
374
566
  ##
@@ -380,15 +572,18 @@ module AppEngine
380
572
  # @return [String] Name of the most recent version.
381
573
  #
382
574
  def latest_version service
383
- result = Util::Gcloud.execute [
575
+ result = Util::Gcloud.execute \
576
+ [
384
577
  "app", "versions", "list",
385
- "--service=#{service}",
386
- "--format=get(version.id)",
387
- "--sort-by=~version.createTime",
388
- "--limit=1"],
389
- capture: true, assert: false
578
+ "--project", @project,
579
+ "--service", service,
580
+ "--format", "get(version.id)",
581
+ "--sort-by", "~version.createTime",
582
+ "--limit", "1"
583
+ ],
584
+ capture: true, assert: false
390
585
  result = result.split.first
391
- raise NoSuchVersion.new(service) unless result
586
+ raise NoSuchVersion, service unless result
392
587
  result
393
588
  end
394
589
 
@@ -400,23 +595,263 @@ module AppEngine
400
595
  # "default" is used.
401
596
  # @param version [String] Name of the version. If omitted, the most
402
597
  # recently deployed is used.
403
- # @return [Hash,nil] A collection of fields parsed from the JSON
404
- # representation of the version, or nil if the requested version
405
- # doesn't exist.
598
+ # @return [Hash] A collection of fields parsed from the JSON representation
599
+ # of the version
600
+ # @return [nil] if the requested version doesn't exist.
406
601
  #
407
602
  def version_info service, version
408
603
  service ||= "default"
409
604
  version ||= latest_version service
410
- result = Util::Gcloud.execute [
605
+ result = Util::Gcloud.execute \
606
+ [
411
607
  "app", "versions", "describe", version,
412
- "--service=#{service}",
413
- "--format=json"],
414
- capture: true, assert: false
608
+ "--project", @project,
609
+ "--service", service,
610
+ "--format", "json"
611
+ ],
612
+ capture: true, assert: false
415
613
  result.strip!
416
614
  raise NoSuchVersion.new(service, version) if result.empty?
417
615
  ::JSON.parse result
418
616
  end
419
617
 
420
- end
618
+ ##
619
+ # @private
620
+ # Performs exec on a GAE standard app.
621
+ #
622
+ def start_deployment_strategy app_info
623
+ describe_deployment_strategy
624
+ entrypoint_file = app_yaml_file = temp_version = nil
625
+ begin
626
+ puts "\n---------- DEPLOY COMMAND ----------"
627
+ secret = create_secret
628
+ entrypoint_file = copy_entrypoint secret
629
+ app_yaml_file = copy_app_yaml app_info, entrypoint_file
630
+ temp_version = deploy_temp_app app_yaml_file
631
+ puts "\n---------- EXECUTE COMMAND ----------"
632
+ puts "COMMAND: #{@command.inspect}\n\n"
633
+ exit_status = track_status temp_version, secret
634
+ puts "\nEXIT STATUS: #{exit_status}"
635
+ ensure
636
+ puts "\n---------- CLEANUP ----------"
637
+ ::File.unlink entrypoint_file if entrypoint_file
638
+ ::File.unlink app_yaml_file if app_yaml_file
639
+ delete_temp_version temp_version
640
+ end
641
+ end
642
+
643
+ def describe_deployment_strategy
644
+ puts "\nUsing the `deployment` strategy for appengine:exec"
645
+ puts "(i.e. deploying a temporary version of your app)"
646
+ puts "PROJECT: #{@project}"
647
+ puts "SERVICE: #{@service}"
648
+ puts "TIMEOUT: #{@timeout}"
649
+ end
650
+
651
+ def create_secret
652
+ ::SecureRandom.alphanumeric 20
653
+ end
654
+
655
+ def copy_entrypoint secret
656
+ entrypoint_template =
657
+ ::File.join(::File.dirname(::File.dirname(__dir__)),
658
+ "data", "exec_standard_entrypoint.rb.erb")
659
+ entrypoint_file = "appengine_exec_entrypoint_#{@timestamp_suffix}.rb"
660
+ erb = ::ERB.new ::File.read entrypoint_template
661
+ data = {
662
+ secret: secret.inspect, command: command.inspect
663
+ }
664
+ result = erb.result_with_hash data
665
+ ::File.open entrypoint_file, "w" do |file|
666
+ file.write result
667
+ end
668
+ entrypoint_file
669
+ end
670
+
671
+ def copy_app_yaml app_info, entrypoint_file
672
+ yaml_data = {
673
+ "runtime" => app_info["runtime"],
674
+ "service" => @service,
675
+ "entrypoint" => "ruby #{entrypoint_file}",
676
+ "env_variables" => app_info["envVariables"],
677
+ "manual_scaling" => { "instances" => 1 }
678
+ }
679
+ if app_info["env"] == "flexible"
680
+ complete_flex_app_yaml yaml_data, app_info
681
+ else
682
+ complete_standard_app_yaml yaml_data, app_info
683
+ end
684
+ app_yaml_file = "appengine_exec_config_#{@timestamp_suffix}.yaml"
685
+ ::File.open app_yaml_file, "w" do |file|
686
+ ::Psych.dump yaml_data, file
687
+ end
688
+ app_yaml_file
689
+ end
690
+
691
+ def complete_flex_app_yaml yaml_data, app_info
692
+ yaml_data["env"] = "flex"
693
+ orig_path = (app_info["betaSettings"] || {})["module_yaml_path"]
694
+ return unless orig_path
695
+ orig_yaml = ::YAML.load_file orig_path
696
+ copy_keys = ["skip_files", "resources", "network", "runtime_config",
697
+ "beta_settings"]
698
+ copy_keys.each do |key|
699
+ yaml_data[key] = orig_yaml[key] if orig_yaml[key]
700
+ end
701
+ end
702
+
703
+ def complete_standard_app_yaml yaml_data, app_info
704
+ yaml_data["instance_class"] = app_info["instanceClass"].sub(/^F/, "B")
705
+ end
706
+
707
+ def deploy_temp_app app_yaml_file
708
+ temp_version = "appengine-exec-#{@timestamp_suffix}"
709
+ Util::Gcloud.execute [
710
+ "app", "deploy", app_yaml_file,
711
+ "--project", @project,
712
+ "--version", temp_version,
713
+ "--no-promote", "--quiet"
714
+ ]
715
+ temp_version
716
+ end
717
+
718
+ def track_status temp_version, secret
719
+ host = "#{temp_version}.#{@service}.#{@project}.appspot.com"
720
+ ::Net::HTTP.start host do |http|
721
+ outpos = errpos = 0
722
+ delay = 0.0
723
+ loop do
724
+ sleep delay
725
+ uri = URI("http://#{host}/#{secret}")
726
+ uri.query = ::URI.encode_www_form outpos: outpos, errpos: errpos
727
+ response = http.request_get uri
728
+ data = ::JSON.parse response.body
729
+ data["outlines"].each { |line| puts "[STDOUT] #{line}" }
730
+ data["errlines"].each { |line| puts "[STDERR] #{line}" }
731
+ outpos = data["outpos"]
732
+ errpos = data["errpos"]
733
+ return data["status"] if data["status"]
734
+ if data["time"] > @timeout_seconds
735
+ http.request_post "/#{secret}/kill", ""
736
+ return "timeout"
737
+ end
738
+ if data["outlines"].empty? && data["errlines"].empty?
739
+ delay += 0.1
740
+ delay = 1.0 if delay > 1.0
741
+ else
742
+ delay = 0.0
743
+ end
744
+ end
745
+ end
746
+ end
747
+
748
+ def delete_temp_version temp_version
749
+ Util::Gcloud.execute [
750
+ "app", "versions", "delete", temp_version,
751
+ "--project", @project,
752
+ "--service", @service,
753
+ "--quiet"
754
+ ]
755
+ end
756
+
757
+ ##
758
+ # @private
759
+ # Performs exec on a GAE flexible app.
760
+ #
761
+ def start_build_strategy app_info
762
+ env_variables = app_info["envVariables"] || {}
763
+ beta_settings = app_info["betaSettings"] || {}
764
+ cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
765
+ container = app_info["deployment"]["container"]
766
+ image = container ? container["image"] : image_from_build(app_info)
767
+
768
+ describe_build_strategy
769
+
770
+ config = build_config command, image, env_variables, cloud_sql_instances
771
+ file = ::Tempfile.new ["cloudbuild_", ".json"]
772
+ begin
773
+ ::JSON.dump config, file
774
+ file.flush
775
+ execute_command = [
776
+ "builds", "submit",
777
+ "--project", @project,
778
+ "--no-source",
779
+ "--config", file.path,
780
+ "--timeout", @timeout
781
+ ]
782
+ execute_command.concat ["--gcs-log-dir", @gcs_log_dir] unless @gcs_log_dir.nil?
783
+ Util::Gcloud.execute execute_command
784
+ ensure
785
+ file.close!
786
+ end
787
+ end
788
+
789
+ ##
790
+ # @private
791
+ # Workaround for https://github.com/GoogleCloudPlatform/appengine-ruby/issues/33
792
+ # Determines the image by looking it up in Cloud Build
793
+ #
794
+ def image_from_build app_info
795
+ create_time = ::DateTime.parse(app_info["createTime"]).to_time.utc
796
+ after_time = (create_time - 3600).strftime "%Y-%m-%dT%H:%M:%SZ"
797
+ before_time = (create_time + 3600).strftime "%Y-%m-%dT%H:%M:%SZ"
798
+ partial_uri = "gcr.io/#{@project}/appengine/#{@service}.#{@version}"
799
+ filter = "createTime>#{after_time} createTime<#{before_time} images[]:#{partial_uri}"
800
+ result = Util::Gcloud.execute \
801
+ [
802
+ "builds", "list",
803
+ "--project", @project,
804
+ "--filter", filter,
805
+ "--format", "json"
806
+ ],
807
+ capture: true, assert: false
808
+ result.strip!
809
+ raise NoSuchVersion.new(@service, @version) if result.empty?
810
+ build_info = ::JSON.parse(result).first
811
+ build_info["images"].first
812
+ end
813
+
814
+ def describe_build_strategy
815
+ puts "\nUsing the `cloud_build` strategy for appengine:exec"
816
+ puts "(i.e. running your app image in Cloud Build)"
817
+ puts "PROJECT: #{@project}"
818
+ puts "SERVICE: #{@service}"
819
+ puts "VERSION: #{@version}"
820
+ puts "TIMEOUT: #{@timeout}"
821
+ puts ""
822
+ end
823
+
824
+ ##
825
+ # @private
826
+ # Builds a cloudbuild config as a data structure.
827
+ #
828
+ # @param command [Array<String>] The command in array form.
829
+ # @param image [String] The fully qualified image path.
830
+ # @param env_variables [Hash<String,String>] Environment variables.
831
+ # @param cloud_sql_instances [String,Array<String>] Names of cloud sql
832
+ # instances to connect to.
833
+ #
834
+ def build_config command, image, env_variables, cloud_sql_instances
835
+ args = ["-i", image]
836
+ env_variables.each do |k, v|
837
+ v = v.gsub "$", "$$"
838
+ args << "-e" << "#{k}=#{v}"
839
+ end
840
+ unless cloud_sql_instances.empty?
841
+ cloud_sql_instances = Array(cloud_sql_instances)
842
+ cloud_sql_instances.each do |sql|
843
+ args << "-s" << sql
844
+ end
845
+ end
846
+ args << "--"
847
+ args += command
421
848
 
849
+ {
850
+ "steps" => [
851
+ "name" => @wrapper_image,
852
+ "args" => args
853
+ ]
854
+ }
855
+ end
856
+ end
422
857
  end