appengine 0.4.3 → 0.6.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 2f0b0ccf502def04a72e834f3086b4d49067616e
4
- data.tar.gz: 6911772b6e335e325cb7569d4bc83e2662888974
2
+ SHA256:
3
+ metadata.gz: 8985bf9ca824b3567577328e9c844eabcab5691907b0eb8a3385c181c91d2f13
4
+ data.tar.gz: 04d8e4d73ee85d186798b69748191ec14423aecabfa1cbc5446d6e79e29b438c
5
5
  SHA512:
6
- metadata.gz: 1e2144f96a6afb55b57933f1c5f8603df938a23bdaa778ba7b2bf5c7e1040dc410abf1941d1ee8b2724dce8e7b188372e50f3fef5d18ef2e4c3f50b9bb6d1b96
7
- data.tar.gz: 747253cd3457c8ec7c9aa909c1927e5e622d49df4baab4649a9a4930b4b8d16666d056cea9056fda78fab43379e093111ea4f180b8b70f4434f44d3c92fcc79e
6
+ metadata.gz: fa651f55f75abf3d21813891a34c26d9c27ffe0656f7b77e54f440a2bb8d747f751317327cdc250a72fc3e286e4a76b79ac4f44ace1f70f01d8d46d3a27866ec
7
+ data.tar.gz: a03743e4a99da424fb1c588f83767389c36b82ddcb5271cba0462a5c7433bd113d3db1b668159dad5520eaa7fcb11ffcb88694133a0f46feb557ac69ada0bab2
data/.yardopts CHANGED
@@ -1,7 +1,8 @@
1
1
  --no-private
2
2
  --title=AppEngine
3
3
  --markup markdown
4
-
4
+ --markup-provider redcarpet
5
+ --main=README.md
5
6
  ./lib/**/*.rb
6
7
  -
7
8
  README.md
@@ -2,6 +2,34 @@
2
2
 
3
3
  This is the change history for the appengine gem.
4
4
 
5
+ ## v0.6.0 (2020-12-02)
6
+
7
+ * Fix failure in the appengine:exec cloud_build strategy when App Engine doesn't provide the image
8
+ * Fix exception when appengine:exec is provided a shell command rather than a command array (tpbowden)
9
+ * Update stackdriver dependency to 0.20 and google-cloud-env dependency to 1.4
10
+
11
+ ## v0.5.0 (2019-07-15)
12
+
13
+ * appengine:exec supports the App Engine standard environment.
14
+ * appengine:exec supports setting the project via `GAE_PROJECT`.
15
+ * Support for an alternate appengine:exec strategy for flexible environment apps that talk to a database via a private IP.
16
+ * Fix crash when the gcloud path includes directories with spaces.
17
+ * Escape `$` symbols in environment configs. (tpbowden)
18
+
19
+ ## v0.4.6 (2018-09-17)
20
+
21
+ * Use gcloud builds submit instead of gcloud container builds submit. (tbpg)
22
+ * Update stackdriver dependency to 0.15.
23
+
24
+ ## v0.4.5 (2017-12-04)
25
+
26
+ * Ensure tempfile is required when needed.
27
+ * Update stackdriver dependency to 0.11.
28
+
29
+ ## v0.4.4 (2017-10-03)
30
+
31
+ * Windows compatibility for appengine:exec task. (gkaykck)
32
+
5
33
  ## v0.4.3 (2017-09-20)
6
34
 
7
35
  * Fixed incorrect namespace in the handler for gcloud errors.
data/README.md CHANGED
@@ -1,6 +1,9 @@
1
1
  Google App Engine Integration Tools
2
2
  ===================================
3
3
 
4
+ [![CircleCI](https://circleci.com/gh/GoogleCloudPlatform/appengine-ruby.svg?style=svg)](https://circleci.com/gh/GoogleCloudPlatform/appengine-ruby)
5
+ [![Gem Version](https://badge.fury.io/rb/appengine.svg)](https://badge.fury.io/rb/appengine)
6
+
4
7
  This repository contains the "appengine" gem, a collection of libraries and
5
8
  plugins for integrating Ruby apps with Google App Engine. It is not required
6
9
  for deploying a ruby application to Google App Engine, but it provides a
@@ -18,7 +21,7 @@ Currently, it includes:
18
21
  * Convenient access to environment information such as project ID and VM
19
22
  properties.
20
23
 
21
- Planned for the near future:
24
+ Potential future directions:
22
25
 
23
26
  * Tools for generating "app.yaml" configuration files for Ruby applications.
24
27
  * Streamlined implementation of health checks and other lifecycle hooks.
@@ -32,6 +35,10 @@ To install, include the "appengine" gem in your Gemfile. e.g.
32
35
 
33
36
  gem "appengine"
34
37
 
38
+ ## Quick Start
39
+
40
+ ### Rails Quick Start
41
+
35
42
  If you are running [Ruby On Rails](http://rubyonrails.org/) 4.0 or later, this
36
43
  gem will automatically install a Railtie that provides its capabilities. You
37
44
  may need to include the line:
@@ -41,10 +48,48 @@ may need to include the line:
41
48
  in your `config/application.rb` file if you aren't already requiring all
42
49
  bundled gems.
43
50
 
44
- If you are running a different web framework, you may need to add some
45
- initialization code to activate the features listed below.
51
+ ### Rack Quick Start
52
+
53
+ If you are running a different Rack-based web framework, include the following
54
+ line in your main Ruby file or `config.ru`:
55
+
56
+ require "appengine"
57
+
58
+ Then, to activate Stackdriver instrumentation, add the following middleware:
59
+
60
+ use Google::Cloud::Logging::Middleware
61
+ use Google::Cloud::ErrorReporting::Middleware
62
+ use Google::Cloud::Trace::Middleware
63
+ use Google::Cloud::Debugger::Middleware
64
+
65
+ You can add the Rake tasks to your application by adding the following to your
66
+ Rakefile:
67
+
68
+ require "appengine/tasks"
69
+
70
+ ### Setting up appengine:exec remote execution
71
+
72
+ This gem is commonly used for its `appengine:exec` Rake task that provides a
73
+ way to run production tasks such as database migrations in the cloud. If you
74
+ are getting started with this feature, you should read the documentation
75
+ (available on the
76
+ [AppEngine::Exec module](http://www.rubydoc.info/gems/appengine/AppEngine/Exec))
77
+ carefully, for important tips. In particular:
78
+
79
+ * The strategy used by the gem is different depending on whether your app is
80
+ deployed to the App Engine standard environment or flexible environment.
81
+ It is important to understand which strategy is in use, because it affects
82
+ which version of your application code is used to run the task, and various
83
+ other factors.
84
+ * You may need to grant additional permissions to the service account that
85
+ runs the task. Again, the documentation will describe this in detail.
86
+ * If your app is running on the flexible environment and uses a VPC (and
87
+ connects to your database via a private IP address), then you will need to
88
+ use a special configuration for the task.
89
+
90
+ ## Using this library
46
91
 
47
- ## Logging and monitoring
92
+ ### Logging and monitoring
48
93
 
49
94
  This library automatically installs the "stackdriver" gem, which instruments
50
95
  your application to report logs, unhandled exceptions, and latency traces to
@@ -54,16 +99,21 @@ monitoring features of Google App Engine, see:
54
99
  * [google-cloud-logging instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-logging/latest/guides/instrumentation)
55
100
  * [google-cloud-error_reporting instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-error_reporting/latest/guides/instrumentation)
56
101
  * [google-cloud-trace instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-trace/latest/guides/instrumentation)
102
+ * [google-cloud-debugger instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-debugger/latest/guides/instrumentation)
57
103
 
58
104
  Rails applications automatically activate this instrumentation when the gem
59
105
  is present. You may opt out of individual services by providing appropriate
60
- Rails configuration. See {AppEngine::Railtie} for more information.
106
+ Rails configuration. See
107
+ [AppEngine::Railtie](http://www.rubydoc.info/gems/appengine/AppEngine/Railtie)
108
+ for more information.
61
109
 
62
110
  Non-Rails applications must provide initialization code to activate this
63
- instrumentation, typically by installing a Rack middleware. See the individual
64
- service documentation links above for more information.
111
+ instrumentation, typically by installing a Rack middleware. You can find the
112
+ basic code for installing these middlewares in the Rack Quick Start section
113
+ above. See the individual service documentation links above for more
114
+ information and configuration options.
65
115
 
66
- ## App Engine remote execution
116
+ ### App Engine remote execution
67
117
 
68
118
  This library provides rake tasks for App Engine remote execution, allowing
69
119
  App Engine applications to perform on-demand tasks in the App Engine
@@ -73,17 +123,17 @@ example, you could run a production database migration in a Rails app using:
73
123
 
74
124
  bundle exec rake appengine:exec -- bundle exec rake db:migrate
75
125
 
76
- The migration would be run in VMs provided by Google Cloud. It uses a
77
- privileged service account that will have access to the production cloud
78
- resources, such as Cloud SQL instances, used by the application. This mechanism
79
- is often much easier and safer than running the task on a local workstation and
80
- granting that workstation direct access to those Cloud SQL instances.
126
+ The migration would be run in containers on Google Cloud infrastructure, which
127
+ is much easier and safer than running the task on a local workstation and
128
+ granting that workstation direct access to your production database.
81
129
 
82
- See {AppEngine::Exec} for more information on App Engine remote execution.
130
+ See [AppEngine::Exec](http://www.rubydoc.info/gems/appengine/AppEngine/Exec)
131
+ for more information on App Engine remote execution.
83
132
 
84
- See {AppEngine::Tasks} for more information on running the rake tasks. The
85
- tasks are available automatically in Rails applications when the gem is
86
- present. Non-Rails applications may install the tasks by adding the line
133
+ See [AppEngine::Tasks](http://www.rubydoc.info/gems/appengine/AppEngine/Tasks)
134
+ for more information on running the Rake tasks. The tasks are available
135
+ automatically in Rails applications when the gem is present. Non-Rails
136
+ applications may install the tasks by adding the line
87
137
  `require "appengine/tasks"` to the `Rakefile`.
88
138
 
89
139
  ## Development and support
@@ -0,0 +1,116 @@
1
+ # Copyright 2019 Google LLC
2
+ #
3
+ # Licensed under the Apache License, Version 2.0 (the "License");
4
+ # you may not use this file except in compliance with the License.
5
+ # You may obtain a copy of the License at
6
+ #
7
+ # http://www.apache.org/licenses/LICENSE-2.0
8
+ #
9
+ # Unless required by applicable law or agreed to in writing, software
10
+ # distributed under the License is distributed on an "AS IS" BASIS,
11
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ # See the License for the specific language governing permissions and
13
+ # limitations under the License.
14
+
15
+ require "webrick"
16
+ require "monitor"
17
+ require "json"
18
+
19
+ SECRET = <%= secret %>
20
+ COMMAND = Array(<%= command %>)
21
+
22
+ port = Integer ENV["PORT"]
23
+ server = ::WEBrick::HTTPServer.new Port: port
24
+
25
+ Status = ::Struct.new :out_lines, :err_lines, :exit_status, :pid, :start_time
26
+ $status = Status.new [], [], nil, nil, nil
27
+ $status.extend ::MonitorMixin
28
+
29
+ def do_start
30
+ $status.synchronize do
31
+ return unless $status.pid.nil?
32
+ end
33
+ $stdout.puts "Executing: #{COMMAND.inspect}"
34
+ $stdout.flush
35
+ rout, wout = ::IO.pipe
36
+ rerr, werr = ::IO.pipe
37
+ ::Thread.new do
38
+ rout.each_line do |line|
39
+ $status.synchronize { $status.out_lines << line }
40
+ $stdout.puts line
41
+ $stdout.flush
42
+ end
43
+ end
44
+ ::Thread.new do
45
+ rerr.each_line do |line|
46
+ $status.synchronize { $status.err_lines << line }
47
+ $stderr.puts line
48
+ $stderr.flush
49
+ end
50
+ end
51
+ start_time = Time.now.to_i
52
+ pid = ::Process.spawn *COMMAND, err: werr, out: wout
53
+ werr.close
54
+ wout.close
55
+ $status.synchronize do
56
+ $status.pid = pid
57
+ $status.start_time = start_time
58
+ end
59
+ ::Thread.new do
60
+ _pid, status = ::Process.wait2 pid
61
+ $status.synchronize do
62
+ $status.exit_status = status.exitstatus
63
+ end
64
+ end
65
+ end
66
+
67
+ def get_status outpos: 0, errpos: 0
68
+ outlines, errlines, status, start_time =
69
+ $status.synchronize do
70
+ [
71
+ $status.out_lines[outpos..-1],
72
+ $status.err_lines[errpos..-1],
73
+ $status.exit_status,
74
+ $status.start_time
75
+ ]
76
+ end
77
+ {
78
+ "outpos" => outpos + outlines.size,
79
+ "errpos" => errpos + errlines.size,
80
+ "outlines" => outlines,
81
+ "errlines" => errlines,
82
+ "status" => status,
83
+ "time" => Time.now.to_i - start_time
84
+ }
85
+ end
86
+
87
+ server.mount_proc "/_ah/start" do |req, res|
88
+ do_start
89
+ end
90
+
91
+ server.mount_proc "/#{SECRET}" do |req, res|
92
+ do_start
93
+ status = get_status outpos: req.query["outpos"].to_i,
94
+ errpos: req.query["errpos"].to_i
95
+ res.body = JSON.dump status
96
+ end
97
+
98
+ server.mount_proc "/#{SECRET}/kill" do |req, res|
99
+ unless req.request_method == "POST"
100
+ res.status = 404
101
+ return
102
+ end
103
+ $status.synchronize do
104
+ if $status.pid.nil?
105
+ res.status = 404
106
+ return
107
+ end
108
+ ::Process.kill "SIGTERM", $status.pid
109
+ end
110
+ end
111
+
112
+ begin
113
+ server.start
114
+ ensure
115
+ server.shutdown
116
+ end
@@ -1,4 +1,6 @@
1
- # Copyright 2016 Google Inc. All rights reserved.
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2019 Google LLC
2
4
  #
3
5
  # Licensed under the Apache License, Version 2.0 (the "License");
4
6
  # you may not use this file except in compliance with the License.
@@ -11,17 +13,22 @@
11
13
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
14
  # See the License for the specific language governing permissions and
13
15
  # limitations under the License.
14
- ;
15
16
 
16
- # # Google AppEngine integration
17
+
18
+ ##
19
+ # ## Google AppEngine integration
17
20
  #
18
21
  # The AppEngine module includes optional tools helping Ruby applications to
19
22
  # integrate more closely with the Google App Engine environment.
20
-
23
+ #
21
24
  module AppEngine
25
+ ##
26
+ # Internal utilities
27
+ #
28
+ module Util
29
+ end
22
30
  end
23
31
 
24
-
25
32
  require "appengine/version"
26
33
  require "appengine/env"
27
34
  require "appengine/exec"
@@ -1,4 +1,6 @@
1
- # Copyright 2016 Google Inc. All rights reserved.
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2019 Google LLC
2
4
  #
3
5
  # Licensed under the Apache License, Version 2.0 (the "License");
4
6
  # you may not use this file except in compliance with the License.
@@ -11,13 +13,11 @@
11
13
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
14
  # See the License for the specific language governing permissions and
13
15
  # limitations under the License.
14
- ;
15
16
 
16
- require "google/cloud/env"
17
17
 
18
+ require "google/cloud/env"
18
19
 
19
20
  module AppEngine
20
-
21
21
  ##
22
22
  # A convenience object that provides information on the Google Cloud
23
23
  # hosting environment. For example, you can call
@@ -34,5 +34,4 @@ module AppEngine
34
34
  # directly instead. See the documentation for the `google-cloud-env` gem.
35
35
  #
36
36
  Env = ::Google::Cloud.env
37
-
38
37
  end
@@ -1,4 +1,6 @@
1
- # Copyright 2017 Google Inc. All rights reserved.
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2019 Google LLC
2
4
  #
3
5
  # Licensed under the Apache License, Version 2.0 (the "License");
4
6
  # you may not use this file except in compliance with the License.
@@ -11,16 +13,20 @@
11
13
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
14
  # See the License for the specific language governing permissions and
13
15
  # limitations under the License.
14
- ;
15
16
 
16
- require "yaml"
17
+
18
+ require "date"
19
+ require "erb"
17
20
  require "json"
21
+ require "net/http"
22
+ require "securerandom"
23
+ require "shellwords"
24
+ require "tempfile"
25
+ require "yaml"
18
26
 
19
27
  require "appengine/util/gcloud"
20
28
 
21
-
22
29
  module AppEngine
23
-
24
30
  ##
25
31
  # # App Engine remote execution
26
32
  #
@@ -31,90 +37,219 @@ module AppEngine
31
37
  #
32
38
  # ## About App Engine execution
33
39
  #
34
- # App Engine execution spins up an image of a deployed App Engine app, and
35
- # runs a command in that image. For example, if your app runs on Ruby on
36
- # Rails, then your app provides a `bin/rails` tool, and you may invoke it
37
- # using App Engine execution---for example to run a command such as
38
- # `bundle exec bin/rails db:migrate` in the image.
40
+ # App Engine execution spins up a one-off copy of an App Engine app, and runs
41
+ # a command against it. For example, if your app runs on Ruby on Rails, then
42
+ # you might use App Engine execution to run a command such as
43
+ # `bundle exec bin/rails db:migrate` in production infrastructure (to avoid
44
+ # having to connect directly to your production database from a local
45
+ # workstation).
46
+ #
47
+ # App Engine execution provides two strategies for generating that "one-off
48
+ # copy":
49
+ #
50
+ # * A `deployment` strategy, which deploys a temporary version of your app
51
+ # to a single backend instance and runs the command there.
52
+ # * A `cloud_build` strategy, which deploys your application image to
53
+ # Google Cloud Build and runs the command there.
39
54
  #
40
- # When App Engine execution runs your command, it provides access to key
41
- # elements of the App Engine environment, including:
55
+ # Both strategies are generally designed to emulate the App Engine runtime
56
+ # environment on cloud VMs similar to those used by actual deployments of
57
+ # your app. Both provide your application code and environment variables, and
58
+ # both provide access to Cloud SQL connections used by your app. However,
59
+ # they differ in what *version* of your app code they run against, and in
60
+ # certain other constraints and performance characteristics. More detailed
61
+ # information on using the two strategies is provided in the sections below.
42
62
  #
43
- # * The same runtime that runs your application in App Engine itself.
44
- # * Any Cloud SQL connections requested by your application.
45
- # * Any environment variables set by your application.
63
+ # Apps deployed to the App Engine *flexible environment* will use the
64
+ # `cloud_build` strategy by default. However, you can force an app to use the
65
+ # `deployment` strategy instead. (You might do so if you need to connect to a
66
+ # Cloud SQL database on a VPC using a private IP, because the `cloud_build`
67
+ # strategy does not support private IPs.) To force use of `deployment`, set
68
+ # the `strategy` parameter in the {AppEngine::Exec} constructor (or the
69
+ # corresponding `GAE_EXEC_STRATEGY` parameter in the Rake task). Note that
70
+ # the `deployment` strategy is usually significantly slower than
71
+ # `cloud_build` for apps in the flexible environment.
46
72
  #
47
- # The command runs on virtual machines provided by Google Cloud Container
48
- # Builder, and has access to the credentials of the Cloud Container Builder
49
- # service account.
73
+ # Apps deployed to the App Engine *standard environment* will *always* use
74
+ # the `deployment` strategy. You cannot force use of the `cloud_build`
75
+ # strategy.
50
76
  #
51
77
  # ## Prerequisites
52
78
  #
53
79
  # To use App Engine remote execution, you will need:
54
80
  #
55
81
  # * An app deployed to Google App Engine, of course!
56
- # * The gcloud SDK installed and configured. See https://cloud.google.com/sdk/
82
+ # * The [gcloud SDK](https://cloud.google.com/sdk/) installed and configured.
57
83
  # * The `appengine` gem.
58
84
  #
59
- # You may also need to grant the Cloud Container Builder service account
60
- # any permissions needed by your command. Often, Project Editor permissions
61
- # will be sufficient for most tasks. You can find the service account
62
- # configuration in the IAM tab in the Cloud Console under the name
63
- # `[your-project-number]@cloudbuild.gserviceaccount.com`.
64
- #
65
85
  # You may use the `AppEngine::Exec` class to run commands directly. However,
66
86
  # in most cases, it will be easier to run commands via the provided rake
67
- # tasks. See {AppEngine::Tasks} for more info.
87
+ # tasks (see {AppEngine::Tasks}).
88
+ #
89
+ # ## Using the "deployment" strategy
90
+ #
91
+ # The `deployment` strategy deploys a temporary version of your app to a
92
+ # single backend App Engine instance, runs the command there, and then
93
+ # deletes the temporary version when it is finished.
94
+ #
95
+ # This is the default strategy (and indeed the only option) for apps running
96
+ # on the App Engine standard environment. It can also be used for flexible
97
+ # environment apps, but this is not commonly done because deployment of
98
+ # flexible environment apps can take a long time.
99
+ #
100
+ # Because the `deployment` strategy deploys a temporary version of your app,
101
+ # it runs against the *current application code* present where the command
102
+ # was initiated (i.e. the code currently on your workstation if you run the
103
+ # rake task from your workstation, or the current code on the branch if you
104
+ # are running from a CI/CD system.) This may be different from the code
105
+ # actually running in production, so it is important that you run from a
106
+ # compatible code branch.
68
107
  #
69
- # ## Configuring
108
+ # ### Specifying the host application
70
109
  #
71
- # This class uses three parameters to specify which application image to use
72
- # to run your command: `service`, `config_path`, and `version`.
110
+ # The `deployment` strategy works by deploying a temporary version of your
111
+ # app, so that it has access to your app's project and settings in App
112
+ # Engine. In most cases, it can determine this automatically, but depending
113
+ # on how your app or environment is structured, you may need to give it some
114
+ # help.
73
115
  #
74
- # In most cases, you can use the defaults. The Exec class will look in your
75
- # current directory for a file called `./app.yaml` which describes your App
76
- # Engine service. It gets the service name from this file (or uses the
77
- # "default" name if none is specified), then looks up the most recently
78
- # created deployment version for that service. That deployment version then
79
- # provides the application image that runs your command.
116
+ # By default, your Google Cloud project is taken from the current gcloud
117
+ # project. If you need to override this, set the `:project` parameter in the
118
+ # {AppEngine::Exec} constructor (or the corresponding `GAE_PROJECT`
119
+ # parameter in the Rake task).
80
120
  #
81
- # If your app has multiple services, you may specify which config file
82
- # (other than `./app.yaml`) describes the desired service, by providing the
83
- # `config_path` parameter. Alternately, you may specify a service name
84
- # directly by providing the `service` parameter. If you provide both
85
- # parameters, `service` takes precedence.
121
+ # By default, the service name is taken from the App Engine config file.
122
+ # App Engine execution will assume this file is called `app.yaml` in the
123
+ # current directory. To use a different config file, set the `config_path`
124
+ # parameter in the {AppEngine::Exec} constructor (or the corresponding
125
+ # `GAE_CONFIG` parameter in the Rake task). You may also set the service name
126
+ # directly, using the `service` parameter (or `GAE_SERVICE` in Rake).
86
127
  #
87
- # Usually, App Engine execution uses the image for the most recently created
88
- # version of the service. (Note: the most recently created version is used,
89
- # regardless of whether that version is currently receiving traffic.) If you
90
- # want to use the image for a different version, you may specify a version
91
- # by providing the `version` parameter.
128
+ # ### Providing credentials
129
+ #
130
+ # Your command will effectively be a deployment of your App Engine app
131
+ # itself, and will have access to the same credentials. For example, App
132
+ # Engine provides a service account by default for your app, or your app may
133
+ # be making use of its own service account key. In either case, make sure the
134
+ # service account has sufficient access for the command you want to run
135
+ # (such as database admin credentials).
136
+ #
137
+ # ### Other options
92
138
  #
93
139
  # You may also provide a timeout, which is the length of time that App
94
140
  # Engine execution will allow your command to run before it is considered to
95
141
  # have stalled and is terminated. The timeout should be a string of the form
96
142
  # `2h15m10s`. The default is `10m`.
97
143
  #
98
- # ## Resource usage and billing
144
+ # The timeout is set via the `timeout` parameter to the {AppEngine::Exec}
145
+ # constructor, or by setting the `GAE_TIMEOUT` environment variable when
146
+ # invoking using Rake.
147
+ #
148
+ # ### Resource usage and billing
149
+ #
150
+ # The `deployment` strategy deploys to a temporary instance of your app in
151
+ # order to run the command. You may be billed for that usage. However, the
152
+ # cost should be minimal, because it will then immediately delete that
153
+ # instance in order to minimize usage.
154
+ #
155
+ # If you interrupt the execution (or it crashes), it is possible that the
156
+ # temporary instance may not get deleted properly. If you suspect this may
157
+ # have happened, go to the App Engine tab in the cloud console, under
158
+ # "versions" of your service, and delete the temporary version manually. It
159
+ # will have a name matching the pattern `appengine-exec-<timestamp>`.
160
+ #
161
+ # ## Using the "cloud_build" strategy
162
+ #
163
+ # The `cloud_build` strategy takes the application image that App Engine is
164
+ # actually using to run your app, and uses it to spin up a copy of your app
165
+ # in [Google Cloud Build](https://cloud.google.com/cloud-build) (along with
166
+ # an emulation layer that emulates certain App Engine services such as Cloud
167
+ # SQL connection sockets). The command then gets run in the Cloud Build
168
+ # environment.
169
+ #
170
+ # This is the default strategy for apps running on the App Engine flexible
171
+ # environment. (It is not available for standard environment apps.) Note that
172
+ # the `cloud_build` strategy cannot be used if your command needs to connect
173
+ # to a database over a [VPC](https://cloud.google.com/vpc/) private IP
174
+ # address. This is because it runs on virtual machines provided by the Cloud
175
+ # Build service, which are not part of your VPC. If your database can be
176
+ # accessed only over a private IP, you should use the `deployment` strategy
177
+ # instead.
178
+ #
179
+ # The Cloud Build log is output to the directory specified by
180
+ # "CLOUD_BUILD_GCS_LOG_DIR". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
181
+ # By default, log directory name is
182
+ # "gs://[PROJECT_NUMBER].cloudbuild-logs.googleusercontent.com/".
183
+ #
184
+ # ### Specifying the host application
185
+ #
186
+ # The `cloud_build` strategy needs to know exactly which app, service, and
187
+ # version of your app, to identify the application image to use.
188
+ #
189
+ # By default, your Google Cloud project is taken from the current gcloud
190
+ # project. If you need to override this, set the `:project` parameter in the
191
+ # {AppEngine::Exec} constructor (or the corresponding `GAE_PROJECT`
192
+ # parameter in the Rake task).
193
+ #
194
+ # By default, the service name is taken from the App Engine config file.
195
+ # App Engine execution will assume this file is called `app.yaml` in the
196
+ # current directory. To use a different config file, set the `config_path`
197
+ # parameter in the {AppEngine::Exec} constructor (or the corresponding
198
+ # `GAE_CONFIG` parameter in the Rake task). You may also set the service name
199
+ # directly, using the `service` parameter (or `GAE_SERVICE` in Rake).
200
+ #
201
+ # By default, the image of the most recently deployed version of your app is
202
+ # used. (Note that this most recently deployed version may not be the same
203
+ # version that is currently receiving traffic: for example, if you deployed
204
+ # with `--no-promote`.) To use a different version, set the `version`
205
+ # parameter in the {AppEngine::Exec} constructor (or the corresponding
206
+ # `GAE_VERSION` parameter in the Rake task).
99
207
  #
100
- # App Engine remote execution uses virtual machine resources provided by
101
- # Google Cloud Container Builder. Generally, a certain number of usage
102
- # minutes per day is covered under a free tier, but additional compute usage
103
- # beyond that time is billed to your Google Cloud account. For more details,
104
- # see https://cloud.google.com/container-builder/pricing
208
+ # ### Providing credentials
209
+ #
210
+ # By default, the `cloud_build` strategy uses your project's Cloud Build
211
+ # service account for its credentials. Unless your command provides its own
212
+ # service account key, you may need to grant the Cloud Build service account
213
+ # any permissions needed to execute your command (such as access to your
214
+ # database). For most tasks, it is sufficient to grant Project Editor
215
+ # permissions to the service account. You can find the service account
216
+ # configuration in the IAM tab in the Cloud Console under the name
217
+ # `[your-project-number]@cloudbuild.gserviceaccount.com`.
218
+ #
219
+ # ### Other options
220
+ #
221
+ # You may also provide a timeout, which is the length of time that App
222
+ # Engine execution will allow your command to run before it is considered to
223
+ # have stalled and is terminated. The timeout should be a string of the form
224
+ # `2h15m10s`. The default is `10m`.
225
+ #
226
+ # The timeout is set via the `timeout` parameter to the {AppEngine::Exec}
227
+ # constructor, or by setting the `GAE_TIMEOUT` environment variable when
228
+ # invoking using Rake.
229
+ #
230
+ # You can also set the wrapper image used to emulate the App Engine runtime
231
+ # environment, by setting the `wrapper_image` parameter to the constructor,
232
+ # or by setting the `GAE_EXEC_WRAPPER_IMAGE` environment variable. Generally,
233
+ # you will not need to do this unless you are testing a new wrapper image.
234
+ #
235
+ # ### Resource usage and billing
236
+ #
237
+ # The `cloud_build` strategy uses virtual machine resources provided by
238
+ # Google Cloud Build. Generally, a certain number of usage minutes per day is
239
+ # covered under a free tier, but additional compute usage beyond that time is
240
+ # billed to your Google Cloud account. For more details,
241
+ # see https://cloud.google.com/cloud-build/pricing
105
242
  #
106
243
  # If your command makes API calls or utilizes other cloud resources, you may
107
- # also be billed for that usage. However, remote execution does not use
108
- # actual App Engine instances, and you will not be billed for additional App
109
- # Engine instance usage.
244
+ # also be billed for that usage. However, the `cloud_build` strategy (unlike
245
+ # the `deployment` strategy) does not use actual App Engine instances, and
246
+ # you will not be billed for additional App Engine instance usage.
110
247
  #
111
248
  class Exec
112
-
113
- @default_timeout = "10m".freeze
114
- @default_service = "default".freeze
115
- @default_config_path = "./app.yaml".freeze
116
- @default_wrapper_image = "gcr.io/google-appengine/exec-wrapper".freeze
117
-
249
+ @default_timeout = "10m"
250
+ @default_service = "default"
251
+ @default_config_path = "./app.yaml"
252
+ @default_wrapper_image = "gcr.io/google-appengine/exec-wrapper:latest"
118
253
 
119
254
  ##
120
255
  # Base class for exec-related usage errors.
@@ -122,6 +257,41 @@ module AppEngine
122
257
  class UsageError < ::StandardError
123
258
  end
124
259
 
260
+ ##
261
+ # Unsupported strategy
262
+ #
263
+ class UnsupportedStrategy < UsageError
264
+ def initialize strategy, app_env
265
+ @strategy = strategy
266
+ @app_env = app_env
267
+ super "Strategy \"#{strategy}\" not supported for the #{app_env}" \
268
+ " environment"
269
+ end
270
+ attr_reader :strategy
271
+ attr_reader :app_env
272
+ end
273
+
274
+ ##
275
+ # Exception raised when a parameter is malformed.
276
+ #
277
+ class BadParameter < UsageError
278
+ def initialize param, value
279
+ @param_name = param
280
+ @value = value
281
+ super "Bad value for #{param}: #{value}"
282
+ end
283
+ attr_reader :param_name
284
+ attr_reader :value
285
+ end
286
+
287
+ ##
288
+ # Exception raised when gcloud has no default project.
289
+ #
290
+ class NoDefaultProject < UsageError
291
+ def initialize
292
+ super "No default project set."
293
+ end
294
+ end
125
295
 
126
296
  ##
127
297
  # Exception raised when the App Engine config file could not be found.
@@ -150,7 +320,7 @@ module AppEngine
150
320
  # versions at all could be found for the given service.
151
321
  #
152
322
  class NoSuchVersion < UsageError
153
- def initialize service, version=nil
323
+ def initialize service, version = nil
154
324
  @service = service
155
325
  @version = version
156
326
  if version
@@ -163,25 +333,7 @@ module AppEngine
163
333
  attr_reader :version
164
334
  end
165
335
 
166
- ##
167
- # Exception raised when an explicitly-specified service name conflicts with
168
- # a config-specified service name.
169
- #
170
- class ServiceNameConflict < UsageError
171
- def initialize service_name, config_name, config_path
172
- @service_name = service_name
173
- @config_name = config_name
174
- @config_path = config_path
175
- super "Service name conflicts with config file"
176
- end
177
- attr_reader :service_name
178
- attr_reader :config_name
179
- attr_reader :config_path
180
- end
181
-
182
-
183
336
  class << self
184
-
185
337
  ## @return [String] Default command timeout.
186
338
  attr_accessor :default_timeout
187
339
 
@@ -205,78 +357,138 @@ module AppEngine
205
357
  # the service name from the config file.
206
358
  # @param config_path [String,nil] App Engine config file to get the
207
359
  # service name from if the service name is not provided directly.
208
- # Defaults to the value of `AppEngine::Exec.default_config_path`.
209
- # @param version [String,nil] Version string. Defaults to the most
210
- # recently created version of the given service (which may not be the
211
- # one currently receiving traffic).
212
- # @param timeout [String,nil] Timeout string. Defaults to the value of
213
- # `AppEngine::Exec.default_timeout`.
360
+ # If omitted, defaults to the value returned by
361
+ # {AppEngine::Exec.default_config_path}.
362
+ # @param version [String,nil] Version string. If omitted, defaults to the
363
+ # most recently created version of the given service (which may not
364
+ # be the one currently receiving traffic).
365
+ # @param timeout [String,nil] Timeout string. If omitted, defaults to the
366
+ # value returned by {AppEngine::Exec.default_timeout}.
367
+ # @param wrapper_image [String,nil] The fully qualified name of the
368
+ # wrapper image to use. (Applies only to the "cloud_build" strategy.)
369
+ # @param strategy [String,nil] The execution strategy to use, or `nil` to
370
+ # choose a default based on the App Engine environment (flexible or
371
+ # standard). Allowed values are `nil`, `"deployment"` (which is the
372
+ # default for Standard), and `"cloud_build"` (which is the default
373
+ # for Flexible).
374
+ # @param gcs_log_dir [String,nil] GCS bucket name of the cloud build log
375
+ # when strategy is "cloud_build". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
214
376
  #
215
377
  def new_rake_task name, args: [], env_args: [],
216
378
  service: nil, config_path: nil, version: nil,
217
- timeout: nil
218
- escaped_args = args.map{ |arg|
219
- arg.gsub(/[,\[\]]/){ |m| "\\#{m}" }
220
- }
221
- if escaped_args.empty?
222
- name_with_args = name
223
- else
224
- name_with_args = "#{name}[#{escaped_args.join ','}]"
379
+ timeout: nil, project: nil, wrapper_image: nil,
380
+ strategy: nil, gcs_log_dir: nil
381
+ escaped_args = args.map do |arg|
382
+ arg.gsub(/[,\[\]]/) { |m| "\\#{m}" }
225
383
  end
384
+ name_with_args =
385
+ if escaped_args.empty?
386
+ name
387
+ else
388
+ "#{name}[#{escaped_args.join ','}]"
389
+ end
226
390
  new ["bundle", "exec", "rake", name_with_args] + env_args,
227
391
  service: service, config_path: config_path, version: version,
228
- timeout: timeout
392
+ timeout: timeout, project: project, wrapper_image: wrapper_image,
393
+ strategy: strategy, gcs_log_dir: gcs_log_dir
229
394
  end
230
-
231
395
  end
232
396
 
233
-
234
397
  ##
235
398
  # Create an execution for the given command.
236
399
  #
237
400
  # @param command [Array<String>] The command in array form.
401
+ # @param project [String,nil] ID of the project. If omitted, obtains
402
+ # the project from gcloud.
238
403
  # @param service [String,nil] Name of the service. If omitted, obtains
239
404
  # the service name from the config file.
240
405
  # @param config_path [String,nil] App Engine config file to get the
241
406
  # service name from if the service name is not provided directly.
242
- # Defaults to the value of `AppEngine::Exec.default_config_path`.
243
- # @param version [String,nil] Version string. Defaults to the most
244
- # recently created version of the given service (which may not be the
245
- # one currently receiving traffic).
246
- # @param timeout [String,nil] Timeout string. Defaults to the value of
247
- # `AppEngine::Exec.default_timeout`.
407
+ # If omitted, defaults to the value returned by
408
+ # {AppEngine::Exec.default_config_path}.
409
+ # @param version [String,nil] Version string. If omitted, defaults to the
410
+ # most recently created version of the given service (which may not be
411
+ # the one currently receiving traffic).
412
+ # @param timeout [String,nil] Timeout string. If omitted, defaults to the
413
+ # value returned by {AppEngine::Exec.default_timeout}.
414
+ # @param wrapper_image [String,nil] The fully qualified name of the wrapper
415
+ # image to use. (Applies only to the "cloud_build" strategy.)
416
+ # @param strategy [String,nil] The execution strategy to use, or `nil` to
417
+ # choose a default based on the App Engine environment (flexible or
418
+ # standard). Allowed values are `nil`, `"deployment"` (which is the
419
+ # default for Standard), and `"cloud_build"` (which is the default for
420
+ # Flexible).
421
+ # @param gcs_log_dir [String,nil] GCS bucket name of the cloud build log
422
+ # when strategy is "cloud_build". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
248
423
  #
249
424
  def initialize command,
250
- service: nil, config_path: nil, version: nil, timeout: nil
425
+ project: nil, service: nil, config_path: nil, version: nil,
426
+ timeout: nil, wrapper_image: nil, strategy: nil, gcs_log_dir: nil
251
427
  @command = command
252
428
  @service = service
253
429
  @config_path = config_path
254
430
  @version = version
255
431
  @timeout = timeout
256
- @wrapper_image = nil
432
+ @project = project
433
+ @wrapper_image = wrapper_image
434
+ @strategy = strategy
435
+ @gcs_log_dir = gcs_log_dir
257
436
 
258
437
  yield self if block_given?
259
438
  end
260
439
 
440
+ ##
441
+ # @return [String] The project ID.
442
+ # @return [nil] if the default gcloud project should be used.
443
+ #
444
+ attr_accessor :project
261
445
 
262
- ## @return [String,nil] The service name, or nil to read from the config.
446
+ ##
447
+ # @return [String] The service name.
448
+ # @return [nil] if the service should be obtained from the app config.
449
+ #
263
450
  attr_accessor :service
264
451
 
265
- ## @return [String,nil] Path to the config file, or nil to use the default.
452
+ ##
453
+ # @return [String] Path to the config file.
454
+ # @return [nil] if the default of `./app.yaml` should be used.
455
+ #
266
456
  attr_accessor :config_path
267
457
 
268
- ## @return [String,nil] Service version, or nil to use the most recent.
458
+ ##
459
+ # @return [String] Service version of the image to use.
460
+ # @return [nil] if the most recent should be used.
461
+ #
269
462
  attr_accessor :version
270
463
 
271
- ## @return [String,nil] Command timeout, or nil to use the default.
464
+ ##
465
+ # @return [String] The command timeout, in `1h23m45s` format.
466
+ # @return [nil] if the default of `10m` should be used.
467
+ #
272
468
  attr_accessor :timeout
273
469
 
274
- ## @return [String,Array<String>] Command to run.
470
+ ##
471
+ # The command to run.
472
+ #
473
+ # @return [String] if the command is a script to be run in a shell.
474
+ # @return [Array<String>] if the command is a posix command to be run
475
+ # directly without a shell.
476
+ #
275
477
  attr_accessor :command
276
478
 
277
- ## @return [String] Custom wrapper image to use, or nil to use the default.
479
+ ##
480
+ # @return [String] Custom wrapper image to use.
481
+ # @return [nil] if the default should be used.
482
+ #
278
483
  attr_accessor :wrapper_image
279
484
 
485
+ ##
486
+ # @return [String] The execution strategy to use. Allowed values are
487
+ # `"deployment"` and `"cloud_build"`.
488
+ # @return [nil] to choose a default based on the App Engine environment
489
+ # (flexible or standard).
490
+ #
491
+ attr_accessor :strategy
280
492
 
281
493
  ##
282
494
  # Executes the command synchronously. Streams the logs back to standard out
@@ -284,29 +496,15 @@ module AppEngine
284
496
  #
285
497
  def start
286
498
  resolve_parameters
287
-
288
- version_info = version_info @service, @version
289
- env_variables = version_info["envVariables"] || {}
290
- beta_settings = version_info["betaSettings"] || {}
291
- cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
292
- image = version_info["deployment"]["container"]["image"]
293
-
294
- config = build_config command, image, env_variables, cloud_sql_instances
295
- file = ::Tempfile.new ["cloudbuild_", ".json"]
296
- begin
297
- ::JSON.dump config, file
298
- file.flush
299
- Util::Gcloud.execute [
300
- "container", "builds", "submit",
301
- "--no-source",
302
- "--config=#{file.path}",
303
- "--timeout=#{@timeout}"]
304
- ensure
305
- file.close!
499
+ app_info = version_info @service, @version
500
+ resolve_strategy app_info["env"]
501
+ if @strategy == "cloud_build"
502
+ start_build_strategy app_info
503
+ else
504
+ start_deployment_strategy app_info
306
505
  end
307
506
  end
308
507
 
309
-
310
508
  private
311
509
 
312
510
  ##
@@ -314,61 +512,55 @@ module AppEngine
314
512
  # Resolves and canonicalizes all the parameters.
315
513
  #
316
514
  def resolve_parameters
317
- unless @command.is_a? Array
318
- @command = ::Shellwords.parse @command.to_s
319
- end
320
-
321
- config_service = config_path = nil
322
- if @config_path || !@service
323
- config_service = begin
324
- config_path = @config_path || Exec.default_config_path
325
- ::YAML.load_file(config_path)["service"] || Exec.default_service
326
- rescue ::Errno::ENOENT
327
- raise ConfigFileNotFound.new config_path
328
- rescue
329
- raise BadConfigFileFormat.new config_path
330
- end
331
- end
332
- if @service && config_service && @service != config_service
333
- raise ServiceNameConflict.new @service, config_service, config_path
334
- end
335
-
336
- @service ||= config_service
515
+ @timestamp_suffix = ::Time.now.strftime "%Y%m%d%H%M%S"
516
+ @command = ::Shellwords.split @command.to_s unless @command.is_a? Array
517
+ @project ||= default_project
518
+ @service ||= service_from_config || Exec.default_service
337
519
  @version ||= latest_version @service
338
520
  @timeout ||= Exec.default_timeout
521
+ @timeout_seconds = parse_timeout @timeout
339
522
  @wrapper_image ||= Exec.default_wrapper_image
523
+ self
340
524
  end
341
525
 
342
- ##
343
- # @private
344
- # Builds a cloudbuild config as a data structure.
345
- #
346
- # @param command [Array<String>] The command in array form.
347
- # @param image [String] The fully qualified image path.
348
- # @param env_variables[Hash<String,String>] Environment variables.
349
- # @param cloud_sql_instances[String,Array<String>] Names of cloud sql
350
- # instances to connect to.
351
- #
352
- def build_config command, image, env_variables, cloud_sql_instances
353
- args = ["-i", image]
354
- env_variables.each do |k, v|
355
- args << "-e" << "#{k}=#{v}"
526
+ def resolve_strategy app_env
527
+ @strategy = @strategy.to_s.downcase
528
+ if @strategy.empty?
529
+ @strategy = app_env == "flexible" ? "cloud_build" : "deployment"
356
530
  end
357
- unless cloud_sql_instances.empty?
358
- cloud_sql_instances = Array(cloud_sql_instances)
359
- cloud_sql_instances.each do |sql|
360
- args << "-s" << sql
361
- end
531
+ if app_env == "standard" && @strategy == "cloud_build" ||
532
+ @strategy != "cloud_build" && @strategy != "deployment"
533
+ raise UnsupportedStrategy.new @strategy, app_env
362
534
  end
363
- args << "--"
364
- args += command
535
+ @strategy
536
+ end
365
537
 
366
- {
367
- "steps" => [
368
- "name" => @wrapper_image,
369
- "args" => args
370
- ]
371
- }
538
+ def service_from_config
539
+ return nil if !@config_path && @service
540
+ @config_path ||= Exec.default_config_path
541
+ ::YAML.load_file(config_path)["service"]
542
+ rescue ::Errno::ENOENT
543
+ raise ConfigFileNotFound, @config_path
544
+ rescue ::StandardError
545
+ raise BadConfigFileFormat, @config_path
546
+ end
547
+
548
+ def default_project
549
+ result = Util::Gcloud.execute \
550
+ ["config", "get-value", "project"],
551
+ capture: true, assert: false
552
+ result.strip!
553
+ raise NoDefaultProject if result.empty?
554
+ result
555
+ end
556
+
557
+ def parse_timeout timeout_str
558
+ matched = timeout_str =~ /^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)s?)?$/
559
+ raise BadParameter.new "timeout", timeout_str unless matched
560
+ hours = ::Regexp.last_match(1).to_i
561
+ minutes = ::Regexp.last_match(2).to_i
562
+ seconds = ::Regexp.last_match(3).to_i
563
+ hours * 3600 + minutes * 60 + seconds
372
564
  end
373
565
 
374
566
  ##
@@ -380,15 +572,18 @@ module AppEngine
380
572
  # @return [String] Name of the most recent version.
381
573
  #
382
574
  def latest_version service
383
- result = Util::Gcloud.execute [
575
+ result = Util::Gcloud.execute \
576
+ [
384
577
  "app", "versions", "list",
385
- "--service=#{service}",
386
- "--format=get(version.id)",
387
- "--sort-by=~version.createTime",
388
- "--limit=1"],
389
- capture: true, assert: false
578
+ "--project", @project,
579
+ "--service", service,
580
+ "--format", "get(version.id)",
581
+ "--sort-by", "~version.createTime",
582
+ "--limit", "1"
583
+ ],
584
+ capture: true, assert: false
390
585
  result = result.split.first
391
- raise NoSuchVersion.new(service) unless result
586
+ raise NoSuchVersion, service unless result
392
587
  result
393
588
  end
394
589
 
@@ -400,23 +595,263 @@ module AppEngine
400
595
  # "default" is used.
401
596
  # @param version [String] Name of the version. If omitted, the most
402
597
  # recently deployed is used.
403
- # @return [Hash,nil] A collection of fields parsed from the JSON
404
- # representation of the version, or nil if the requested version
405
- # doesn't exist.
598
+ # @return [Hash] A collection of fields parsed from the JSON representation
599
+ # of the version
600
+ # @return [nil] if the requested version doesn't exist.
406
601
  #
407
602
  def version_info service, version
408
603
  service ||= "default"
409
604
  version ||= latest_version service
410
- result = Util::Gcloud.execute [
605
+ result = Util::Gcloud.execute \
606
+ [
411
607
  "app", "versions", "describe", version,
412
- "--service=#{service}",
413
- "--format=json"],
414
- capture: true, assert: false
608
+ "--project", @project,
609
+ "--service", service,
610
+ "--format", "json"
611
+ ],
612
+ capture: true, assert: false
415
613
  result.strip!
416
614
  raise NoSuchVersion.new(service, version) if result.empty?
417
615
  ::JSON.parse result
418
616
  end
419
617
 
420
- end
618
+ ##
619
+ # @private
620
+ # Performs exec on a GAE standard app.
621
+ #
622
+ def start_deployment_strategy app_info
623
+ describe_deployment_strategy
624
+ entrypoint_file = app_yaml_file = temp_version = nil
625
+ begin
626
+ puts "\n---------- DEPLOY COMMAND ----------"
627
+ secret = create_secret
628
+ entrypoint_file = copy_entrypoint secret
629
+ app_yaml_file = copy_app_yaml app_info, entrypoint_file
630
+ temp_version = deploy_temp_app app_yaml_file
631
+ puts "\n---------- EXECUTE COMMAND ----------"
632
+ puts "COMMAND: #{@command.inspect}\n\n"
633
+ exit_status = track_status temp_version, secret
634
+ puts "\nEXIT STATUS: #{exit_status}"
635
+ ensure
636
+ puts "\n---------- CLEANUP ----------"
637
+ ::File.unlink entrypoint_file if entrypoint_file
638
+ ::File.unlink app_yaml_file if app_yaml_file
639
+ delete_temp_version temp_version
640
+ end
641
+ end
642
+
643
+ def describe_deployment_strategy
644
+ puts "\nUsing the `deployment` strategy for appengine:exec"
645
+ puts "(i.e. deploying a temporary version of your app)"
646
+ puts "PROJECT: #{@project}"
647
+ puts "SERVICE: #{@service}"
648
+ puts "TIMEOUT: #{@timeout}"
649
+ end
650
+
651
+ def create_secret
652
+ ::SecureRandom.alphanumeric 20
653
+ end
654
+
655
+ def copy_entrypoint secret
656
+ entrypoint_template =
657
+ ::File.join(::File.dirname(::File.dirname(__dir__)),
658
+ "data", "exec_standard_entrypoint.rb.erb")
659
+ entrypoint_file = "appengine_exec_entrypoint_#{@timestamp_suffix}.rb"
660
+ erb = ::ERB.new ::File.read entrypoint_template
661
+ data = {
662
+ secret: secret.inspect, command: command.inspect
663
+ }
664
+ result = erb.result_with_hash data
665
+ ::File.open entrypoint_file, "w" do |file|
666
+ file.write result
667
+ end
668
+ entrypoint_file
669
+ end
670
+
671
+ def copy_app_yaml app_info, entrypoint_file
672
+ yaml_data = {
673
+ "runtime" => app_info["runtime"],
674
+ "service" => @service,
675
+ "entrypoint" => "ruby #{entrypoint_file}",
676
+ "env_variables" => app_info["envVariables"],
677
+ "manual_scaling" => { "instances" => 1 }
678
+ }
679
+ if app_info["env"] == "flexible"
680
+ complete_flex_app_yaml yaml_data, app_info
681
+ else
682
+ complete_standard_app_yaml yaml_data, app_info
683
+ end
684
+ app_yaml_file = "appengine_exec_config_#{@timestamp_suffix}.yaml"
685
+ ::File.open app_yaml_file, "w" do |file|
686
+ ::Psych.dump yaml_data, file
687
+ end
688
+ app_yaml_file
689
+ end
690
+
691
+ def complete_flex_app_yaml yaml_data, app_info
692
+ yaml_data["env"] = "flex"
693
+ orig_path = (app_info["betaSettings"] || {})["module_yaml_path"]
694
+ return unless orig_path
695
+ orig_yaml = ::YAML.load_file orig_path
696
+ copy_keys = ["skip_files", "resources", "network", "runtime_config",
697
+ "beta_settings"]
698
+ copy_keys.each do |key|
699
+ yaml_data[key] = orig_yaml[key] if orig_yaml[key]
700
+ end
701
+ end
702
+
703
+ def complete_standard_app_yaml yaml_data, app_info
704
+ yaml_data["instance_class"] = app_info["instanceClass"].sub(/^F/, "B")
705
+ end
706
+
707
+ def deploy_temp_app app_yaml_file
708
+ temp_version = "appengine-exec-#{@timestamp_suffix}"
709
+ Util::Gcloud.execute [
710
+ "app", "deploy", app_yaml_file,
711
+ "--project", @project,
712
+ "--version", temp_version,
713
+ "--no-promote", "--quiet"
714
+ ]
715
+ temp_version
716
+ end
717
+
718
+ def track_status temp_version, secret
719
+ host = "#{temp_version}.#{@service}.#{@project}.appspot.com"
720
+ ::Net::HTTP.start host do |http|
721
+ outpos = errpos = 0
722
+ delay = 0.0
723
+ loop do
724
+ sleep delay
725
+ uri = URI("http://#{host}/#{secret}")
726
+ uri.query = ::URI.encode_www_form outpos: outpos, errpos: errpos
727
+ response = http.request_get uri
728
+ data = ::JSON.parse response.body
729
+ data["outlines"].each { |line| puts "[STDOUT] #{line}" }
730
+ data["errlines"].each { |line| puts "[STDERR] #{line}" }
731
+ outpos = data["outpos"]
732
+ errpos = data["errpos"]
733
+ return data["status"] if data["status"]
734
+ if data["time"] > @timeout_seconds
735
+ http.request_post "/#{secret}/kill", ""
736
+ return "timeout"
737
+ end
738
+ if data["outlines"].empty? && data["errlines"].empty?
739
+ delay += 0.1
740
+ delay = 1.0 if delay > 1.0
741
+ else
742
+ delay = 0.0
743
+ end
744
+ end
745
+ end
746
+ end
747
+
748
+ def delete_temp_version temp_version
749
+ Util::Gcloud.execute [
750
+ "app", "versions", "delete", temp_version,
751
+ "--project", @project,
752
+ "--service", @service,
753
+ "--quiet"
754
+ ]
755
+ end
756
+
757
+ ##
758
+ # @private
759
+ # Performs exec on a GAE flexible app.
760
+ #
761
+ def start_build_strategy app_info
762
+ env_variables = app_info["envVariables"] || {}
763
+ beta_settings = app_info["betaSettings"] || {}
764
+ cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
765
+ container = app_info["deployment"]["container"]
766
+ image = container ? container["image"] : image_from_build(app_info)
767
+
768
+ describe_build_strategy
769
+
770
+ config = build_config command, image, env_variables, cloud_sql_instances
771
+ file = ::Tempfile.new ["cloudbuild_", ".json"]
772
+ begin
773
+ ::JSON.dump config, file
774
+ file.flush
775
+ execute_command = [
776
+ "builds", "submit",
777
+ "--project", @project,
778
+ "--no-source",
779
+ "--config", file.path,
780
+ "--timeout", @timeout
781
+ ]
782
+ execute_command.concat ["--gcs-log-dir", @gcs_log_dir] unless @gcs_log_dir.nil?
783
+ Util::Gcloud.execute execute_command
784
+ ensure
785
+ file.close!
786
+ end
787
+ end
788
+
789
+ ##
790
+ # @private
791
+ # Workaround for https://github.com/GoogleCloudPlatform/appengine-ruby/issues/33
792
+ # Determines the image by looking it up in Cloud Build
793
+ #
794
+ def image_from_build app_info
795
+ create_time = ::DateTime.parse(app_info["createTime"]).to_time.utc
796
+ after_time = (create_time - 3600).strftime "%Y-%m-%dT%H:%M:%SZ"
797
+ before_time = (create_time + 3600).strftime "%Y-%m-%dT%H:%M:%SZ"
798
+ partial_uri = "gcr.io/#{@project}/appengine/#{@service}.#{@version}"
799
+ filter = "createTime>#{after_time} createTime<#{before_time} images[]:#{partial_uri}"
800
+ result = Util::Gcloud.execute \
801
+ [
802
+ "builds", "list",
803
+ "--project", @project,
804
+ "--filter", filter,
805
+ "--format", "json"
806
+ ],
807
+ capture: true, assert: false
808
+ result.strip!
809
+ raise NoSuchVersion.new(@service, @version) if result.empty?
810
+ build_info = ::JSON.parse(result).first
811
+ build_info["images"].first
812
+ end
813
+
814
+ def describe_build_strategy
815
+ puts "\nUsing the `cloud_build` strategy for appengine:exec"
816
+ puts "(i.e. running your app image in Cloud Build)"
817
+ puts "PROJECT: #{@project}"
818
+ puts "SERVICE: #{@service}"
819
+ puts "VERSION: #{@version}"
820
+ puts "TIMEOUT: #{@timeout}"
821
+ puts ""
822
+ end
823
+
824
+ ##
825
+ # @private
826
+ # Builds a cloudbuild config as a data structure.
827
+ #
828
+ # @param command [Array<String>] The command in array form.
829
+ # @param image [String] The fully qualified image path.
830
+ # @param env_variables [Hash<String,String>] Environment variables.
831
+ # @param cloud_sql_instances [String,Array<String>] Names of cloud sql
832
+ # instances to connect to.
833
+ #
834
+ def build_config command, image, env_variables, cloud_sql_instances
835
+ args = ["-i", image]
836
+ env_variables.each do |k, v|
837
+ v = v.gsub "$", "$$"
838
+ args << "-e" << "#{k}=#{v}"
839
+ end
840
+ unless cloud_sql_instances.empty?
841
+ cloud_sql_instances = Array(cloud_sql_instances)
842
+ cloud_sql_instances.each do |sql|
843
+ args << "-s" << sql
844
+ end
845
+ end
846
+ args << "--"
847
+ args += command
421
848
 
849
+ {
850
+ "steps" => [
851
+ "name" => @wrapper_image,
852
+ "args" => args
853
+ ]
854
+ }
855
+ end
856
+ end
422
857
  end