appengine 0.4.3 → 0.6.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.yardopts +2 -1
- data/CHANGELOG.md +28 -0
- data/README.md +67 -17
- data/data/exec_standard_entrypoint.rb.erb +116 -0
- data/lib/appengine.rb +12 -5
- data/lib/appengine/env.rb +4 -5
- data/lib/appengine/exec.rb +627 -192
- data/lib/appengine/railtie.rb +4 -9
- data/lib/appengine/tasks.rb +219 -142
- data/lib/appengine/util/gcloud.rb +25 -17
- data/lib/appengine/version.rb +5 -5
- metadata +97 -22
- data/Rakefile +0 -42
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 8985bf9ca824b3567577328e9c844eabcab5691907b0eb8a3385c181c91d2f13
|
4
|
+
data.tar.gz: 04d8e4d73ee85d186798b69748191ec14423aecabfa1cbc5446d6e79e29b438c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fa651f55f75abf3d21813891a34c26d9c27ffe0656f7b77e54f440a2bb8d747f751317327cdc250a72fc3e286e4a76b79ac4f44ace1f70f01d8d46d3a27866ec
|
7
|
+
data.tar.gz: a03743e4a99da424fb1c588f83767389c36b82ddcb5271cba0462a5c7433bd113d3db1b668159dad5520eaa7fcb11ffcb88694133a0f46feb557ac69ada0bab2
|
data/.yardopts
CHANGED
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,34 @@
|
|
2
2
|
|
3
3
|
This is the change history for the appengine gem.
|
4
4
|
|
5
|
+
## v0.6.0 (2020-12-02)
|
6
|
+
|
7
|
+
* Fix failure in the appengine:exec cloud_build strategy when App Engine doesn't provide the image
|
8
|
+
* Fix exception when appengine:exec is provided a shell command rather than a command array (tpbowden)
|
9
|
+
* Update stackdriver dependency to 0.20 and google-cloud-env dependency to 1.4
|
10
|
+
|
11
|
+
## v0.5.0 (2019-07-15)
|
12
|
+
|
13
|
+
* appengine:exec supports the App Engine standard environment.
|
14
|
+
* appengine:exec supports setting the project via `GAE_PROJECT`.
|
15
|
+
* Support for an alternate appengine:exec strategy for flexible environment apps that talk to a database via a private IP.
|
16
|
+
* Fix crash when the gcloud path includes directories with spaces.
|
17
|
+
* Escape `$` symbols in environment configs. (tpbowden)
|
18
|
+
|
19
|
+
## v0.4.6 (2018-09-17)
|
20
|
+
|
21
|
+
* Use gcloud builds submit instead of gcloud container builds submit. (tbpg)
|
22
|
+
* Update stackdriver dependency to 0.15.
|
23
|
+
|
24
|
+
## v0.4.5 (2017-12-04)
|
25
|
+
|
26
|
+
* Ensure tempfile is required when needed.
|
27
|
+
* Update stackdriver dependency to 0.11.
|
28
|
+
|
29
|
+
## v0.4.4 (2017-10-03)
|
30
|
+
|
31
|
+
* Windows compatibility for appengine:exec task. (gkaykck)
|
32
|
+
|
5
33
|
## v0.4.3 (2017-09-20)
|
6
34
|
|
7
35
|
* Fixed incorrect namespace in the handler for gcloud errors.
|
data/README.md
CHANGED
@@ -1,6 +1,9 @@
|
|
1
1
|
Google App Engine Integration Tools
|
2
2
|
===================================
|
3
3
|
|
4
|
+
[![CircleCI](https://circleci.com/gh/GoogleCloudPlatform/appengine-ruby.svg?style=svg)](https://circleci.com/gh/GoogleCloudPlatform/appengine-ruby)
|
5
|
+
[![Gem Version](https://badge.fury.io/rb/appengine.svg)](https://badge.fury.io/rb/appengine)
|
6
|
+
|
4
7
|
This repository contains the "appengine" gem, a collection of libraries and
|
5
8
|
plugins for integrating Ruby apps with Google App Engine. It is not required
|
6
9
|
for deploying a ruby application to Google App Engine, but it provides a
|
@@ -18,7 +21,7 @@ Currently, it includes:
|
|
18
21
|
* Convenient access to environment information such as project ID and VM
|
19
22
|
properties.
|
20
23
|
|
21
|
-
|
24
|
+
Potential future directions:
|
22
25
|
|
23
26
|
* Tools for generating "app.yaml" configuration files for Ruby applications.
|
24
27
|
* Streamlined implementation of health checks and other lifecycle hooks.
|
@@ -32,6 +35,10 @@ To install, include the "appengine" gem in your Gemfile. e.g.
|
|
32
35
|
|
33
36
|
gem "appengine"
|
34
37
|
|
38
|
+
## Quick Start
|
39
|
+
|
40
|
+
### Rails Quick Start
|
41
|
+
|
35
42
|
If you are running [Ruby On Rails](http://rubyonrails.org/) 4.0 or later, this
|
36
43
|
gem will automatically install a Railtie that provides its capabilities. You
|
37
44
|
may need to include the line:
|
@@ -41,10 +48,48 @@ may need to include the line:
|
|
41
48
|
in your `config/application.rb` file if you aren't already requiring all
|
42
49
|
bundled gems.
|
43
50
|
|
44
|
-
|
45
|
-
|
51
|
+
### Rack Quick Start
|
52
|
+
|
53
|
+
If you are running a different Rack-based web framework, include the following
|
54
|
+
line in your main Ruby file or `config.ru`:
|
55
|
+
|
56
|
+
require "appengine"
|
57
|
+
|
58
|
+
Then, to activate Stackdriver instrumentation, add the following middleware:
|
59
|
+
|
60
|
+
use Google::Cloud::Logging::Middleware
|
61
|
+
use Google::Cloud::ErrorReporting::Middleware
|
62
|
+
use Google::Cloud::Trace::Middleware
|
63
|
+
use Google::Cloud::Debugger::Middleware
|
64
|
+
|
65
|
+
You can add the Rake tasks to your application by adding the following to your
|
66
|
+
Rakefile:
|
67
|
+
|
68
|
+
require "appengine/tasks"
|
69
|
+
|
70
|
+
### Setting up appengine:exec remote execution
|
71
|
+
|
72
|
+
This gem is commonly used for its `appengine:exec` Rake task that provides a
|
73
|
+
way to run production tasks such as database migrations in the cloud. If you
|
74
|
+
are getting started with this feature, you should read the documentation
|
75
|
+
(available on the
|
76
|
+
[AppEngine::Exec module](http://www.rubydoc.info/gems/appengine/AppEngine/Exec))
|
77
|
+
carefully, for important tips. In particular:
|
78
|
+
|
79
|
+
* The strategy used by the gem is different depending on whether your app is
|
80
|
+
deployed to the App Engine standard environment or flexible environment.
|
81
|
+
It is important to understand which strategy is in use, because it affects
|
82
|
+
which version of your application code is used to run the task, and various
|
83
|
+
other factors.
|
84
|
+
* You may need to grant additional permissions to the service account that
|
85
|
+
runs the task. Again, the documentation will describe this in detail.
|
86
|
+
* If your app is running on the flexible environment and uses a VPC (and
|
87
|
+
connects to your database via a private IP address), then you will need to
|
88
|
+
use a special configuration for the task.
|
89
|
+
|
90
|
+
## Using this library
|
46
91
|
|
47
|
-
|
92
|
+
### Logging and monitoring
|
48
93
|
|
49
94
|
This library automatically installs the "stackdriver" gem, which instruments
|
50
95
|
your application to report logs, unhandled exceptions, and latency traces to
|
@@ -54,16 +99,21 @@ monitoring features of Google App Engine, see:
|
|
54
99
|
* [google-cloud-logging instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-logging/latest/guides/instrumentation)
|
55
100
|
* [google-cloud-error_reporting instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-error_reporting/latest/guides/instrumentation)
|
56
101
|
* [google-cloud-trace instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-trace/latest/guides/instrumentation)
|
102
|
+
* [google-cloud-debugger instrumentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-debugger/latest/guides/instrumentation)
|
57
103
|
|
58
104
|
Rails applications automatically activate this instrumentation when the gem
|
59
105
|
is present. You may opt out of individual services by providing appropriate
|
60
|
-
Rails configuration. See
|
106
|
+
Rails configuration. See
|
107
|
+
[AppEngine::Railtie](http://www.rubydoc.info/gems/appengine/AppEngine/Railtie)
|
108
|
+
for more information.
|
61
109
|
|
62
110
|
Non-Rails applications must provide initialization code to activate this
|
63
|
-
instrumentation, typically by installing a Rack middleware.
|
64
|
-
|
111
|
+
instrumentation, typically by installing a Rack middleware. You can find the
|
112
|
+
basic code for installing these middlewares in the Rack Quick Start section
|
113
|
+
above. See the individual service documentation links above for more
|
114
|
+
information and configuration options.
|
65
115
|
|
66
|
-
|
116
|
+
### App Engine remote execution
|
67
117
|
|
68
118
|
This library provides rake tasks for App Engine remote execution, allowing
|
69
119
|
App Engine applications to perform on-demand tasks in the App Engine
|
@@ -73,17 +123,17 @@ example, you could run a production database migration in a Rails app using:
|
|
73
123
|
|
74
124
|
bundle exec rake appengine:exec -- bundle exec rake db:migrate
|
75
125
|
|
76
|
-
The migration would be run in
|
77
|
-
|
78
|
-
|
79
|
-
is often much easier and safer than running the task on a local workstation and
|
80
|
-
granting that workstation direct access to those Cloud SQL instances.
|
126
|
+
The migration would be run in containers on Google Cloud infrastructure, which
|
127
|
+
is much easier and safer than running the task on a local workstation and
|
128
|
+
granting that workstation direct access to your production database.
|
81
129
|
|
82
|
-
See
|
130
|
+
See [AppEngine::Exec](http://www.rubydoc.info/gems/appengine/AppEngine/Exec)
|
131
|
+
for more information on App Engine remote execution.
|
83
132
|
|
84
|
-
See
|
85
|
-
|
86
|
-
|
133
|
+
See [AppEngine::Tasks](http://www.rubydoc.info/gems/appengine/AppEngine/Tasks)
|
134
|
+
for more information on running the Rake tasks. The tasks are available
|
135
|
+
automatically in Rails applications when the gem is present. Non-Rails
|
136
|
+
applications may install the tasks by adding the line
|
87
137
|
`require "appengine/tasks"` to the `Rakefile`.
|
88
138
|
|
89
139
|
## Development and support
|
@@ -0,0 +1,116 @@
|
|
1
|
+
# Copyright 2019 Google LLC
|
2
|
+
#
|
3
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
# you may not use this file except in compliance with the License.
|
5
|
+
# You may obtain a copy of the License at
|
6
|
+
#
|
7
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
#
|
9
|
+
# Unless required by applicable law or agreed to in writing, software
|
10
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
# See the License for the specific language governing permissions and
|
13
|
+
# limitations under the License.
|
14
|
+
|
15
|
+
require "webrick"
|
16
|
+
require "monitor"
|
17
|
+
require "json"
|
18
|
+
|
19
|
+
SECRET = <%= secret %>
|
20
|
+
COMMAND = Array(<%= command %>)
|
21
|
+
|
22
|
+
port = Integer ENV["PORT"]
|
23
|
+
server = ::WEBrick::HTTPServer.new Port: port
|
24
|
+
|
25
|
+
Status = ::Struct.new :out_lines, :err_lines, :exit_status, :pid, :start_time
|
26
|
+
$status = Status.new [], [], nil, nil, nil
|
27
|
+
$status.extend ::MonitorMixin
|
28
|
+
|
29
|
+
def do_start
|
30
|
+
$status.synchronize do
|
31
|
+
return unless $status.pid.nil?
|
32
|
+
end
|
33
|
+
$stdout.puts "Executing: #{COMMAND.inspect}"
|
34
|
+
$stdout.flush
|
35
|
+
rout, wout = ::IO.pipe
|
36
|
+
rerr, werr = ::IO.pipe
|
37
|
+
::Thread.new do
|
38
|
+
rout.each_line do |line|
|
39
|
+
$status.synchronize { $status.out_lines << line }
|
40
|
+
$stdout.puts line
|
41
|
+
$stdout.flush
|
42
|
+
end
|
43
|
+
end
|
44
|
+
::Thread.new do
|
45
|
+
rerr.each_line do |line|
|
46
|
+
$status.synchronize { $status.err_lines << line }
|
47
|
+
$stderr.puts line
|
48
|
+
$stderr.flush
|
49
|
+
end
|
50
|
+
end
|
51
|
+
start_time = Time.now.to_i
|
52
|
+
pid = ::Process.spawn *COMMAND, err: werr, out: wout
|
53
|
+
werr.close
|
54
|
+
wout.close
|
55
|
+
$status.synchronize do
|
56
|
+
$status.pid = pid
|
57
|
+
$status.start_time = start_time
|
58
|
+
end
|
59
|
+
::Thread.new do
|
60
|
+
_pid, status = ::Process.wait2 pid
|
61
|
+
$status.synchronize do
|
62
|
+
$status.exit_status = status.exitstatus
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
def get_status outpos: 0, errpos: 0
|
68
|
+
outlines, errlines, status, start_time =
|
69
|
+
$status.synchronize do
|
70
|
+
[
|
71
|
+
$status.out_lines[outpos..-1],
|
72
|
+
$status.err_lines[errpos..-1],
|
73
|
+
$status.exit_status,
|
74
|
+
$status.start_time
|
75
|
+
]
|
76
|
+
end
|
77
|
+
{
|
78
|
+
"outpos" => outpos + outlines.size,
|
79
|
+
"errpos" => errpos + errlines.size,
|
80
|
+
"outlines" => outlines,
|
81
|
+
"errlines" => errlines,
|
82
|
+
"status" => status,
|
83
|
+
"time" => Time.now.to_i - start_time
|
84
|
+
}
|
85
|
+
end
|
86
|
+
|
87
|
+
server.mount_proc "/_ah/start" do |req, res|
|
88
|
+
do_start
|
89
|
+
end
|
90
|
+
|
91
|
+
server.mount_proc "/#{SECRET}" do |req, res|
|
92
|
+
do_start
|
93
|
+
status = get_status outpos: req.query["outpos"].to_i,
|
94
|
+
errpos: req.query["errpos"].to_i
|
95
|
+
res.body = JSON.dump status
|
96
|
+
end
|
97
|
+
|
98
|
+
server.mount_proc "/#{SECRET}/kill" do |req, res|
|
99
|
+
unless req.request_method == "POST"
|
100
|
+
res.status = 404
|
101
|
+
return
|
102
|
+
end
|
103
|
+
$status.synchronize do
|
104
|
+
if $status.pid.nil?
|
105
|
+
res.status = 404
|
106
|
+
return
|
107
|
+
end
|
108
|
+
::Process.kill "SIGTERM", $status.pid
|
109
|
+
end
|
110
|
+
end
|
111
|
+
|
112
|
+
begin
|
113
|
+
server.start
|
114
|
+
ensure
|
115
|
+
server.shutdown
|
116
|
+
end
|
data/lib/appengine.rb
CHANGED
@@ -1,4 +1,6 @@
|
|
1
|
-
#
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Copyright 2019 Google LLC
|
2
4
|
#
|
3
5
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
4
6
|
# you may not use this file except in compliance with the License.
|
@@ -11,17 +13,22 @@
|
|
11
13
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
14
|
# See the License for the specific language governing permissions and
|
13
15
|
# limitations under the License.
|
14
|
-
;
|
15
16
|
|
16
|
-
|
17
|
+
|
18
|
+
##
|
19
|
+
# ## Google AppEngine integration
|
17
20
|
#
|
18
21
|
# The AppEngine module includes optional tools helping Ruby applications to
|
19
22
|
# integrate more closely with the Google App Engine environment.
|
20
|
-
|
23
|
+
#
|
21
24
|
module AppEngine
|
25
|
+
##
|
26
|
+
# Internal utilities
|
27
|
+
#
|
28
|
+
module Util
|
29
|
+
end
|
22
30
|
end
|
23
31
|
|
24
|
-
|
25
32
|
require "appengine/version"
|
26
33
|
require "appengine/env"
|
27
34
|
require "appengine/exec"
|
data/lib/appengine/env.rb
CHANGED
@@ -1,4 +1,6 @@
|
|
1
|
-
#
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Copyright 2019 Google LLC
|
2
4
|
#
|
3
5
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
4
6
|
# you may not use this file except in compliance with the License.
|
@@ -11,13 +13,11 @@
|
|
11
13
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
14
|
# See the License for the specific language governing permissions and
|
13
15
|
# limitations under the License.
|
14
|
-
;
|
15
16
|
|
16
|
-
require "google/cloud/env"
|
17
17
|
|
18
|
+
require "google/cloud/env"
|
18
19
|
|
19
20
|
module AppEngine
|
20
|
-
|
21
21
|
##
|
22
22
|
# A convenience object that provides information on the Google Cloud
|
23
23
|
# hosting environment. For example, you can call
|
@@ -34,5 +34,4 @@ module AppEngine
|
|
34
34
|
# directly instead. See the documentation for the `google-cloud-env` gem.
|
35
35
|
#
|
36
36
|
Env = ::Google::Cloud.env
|
37
|
-
|
38
37
|
end
|
data/lib/appengine/exec.rb
CHANGED
@@ -1,4 +1,6 @@
|
|
1
|
-
#
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Copyright 2019 Google LLC
|
2
4
|
#
|
3
5
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
4
6
|
# you may not use this file except in compliance with the License.
|
@@ -11,16 +13,20 @@
|
|
11
13
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
14
|
# See the License for the specific language governing permissions and
|
13
15
|
# limitations under the License.
|
14
|
-
;
|
15
16
|
|
16
|
-
|
17
|
+
|
18
|
+
require "date"
|
19
|
+
require "erb"
|
17
20
|
require "json"
|
21
|
+
require "net/http"
|
22
|
+
require "securerandom"
|
23
|
+
require "shellwords"
|
24
|
+
require "tempfile"
|
25
|
+
require "yaml"
|
18
26
|
|
19
27
|
require "appengine/util/gcloud"
|
20
28
|
|
21
|
-
|
22
29
|
module AppEngine
|
23
|
-
|
24
30
|
##
|
25
31
|
# # App Engine remote execution
|
26
32
|
#
|
@@ -31,90 +37,219 @@ module AppEngine
|
|
31
37
|
#
|
32
38
|
# ## About App Engine execution
|
33
39
|
#
|
34
|
-
# App Engine execution spins up
|
35
|
-
#
|
36
|
-
#
|
37
|
-
#
|
38
|
-
#
|
40
|
+
# App Engine execution spins up a one-off copy of an App Engine app, and runs
|
41
|
+
# a command against it. For example, if your app runs on Ruby on Rails, then
|
42
|
+
# you might use App Engine execution to run a command such as
|
43
|
+
# `bundle exec bin/rails db:migrate` in production infrastructure (to avoid
|
44
|
+
# having to connect directly to your production database from a local
|
45
|
+
# workstation).
|
46
|
+
#
|
47
|
+
# App Engine execution provides two strategies for generating that "one-off
|
48
|
+
# copy":
|
49
|
+
#
|
50
|
+
# * A `deployment` strategy, which deploys a temporary version of your app
|
51
|
+
# to a single backend instance and runs the command there.
|
52
|
+
# * A `cloud_build` strategy, which deploys your application image to
|
53
|
+
# Google Cloud Build and runs the command there.
|
39
54
|
#
|
40
|
-
#
|
41
|
-
#
|
55
|
+
# Both strategies are generally designed to emulate the App Engine runtime
|
56
|
+
# environment on cloud VMs similar to those used by actual deployments of
|
57
|
+
# your app. Both provide your application code and environment variables, and
|
58
|
+
# both provide access to Cloud SQL connections used by your app. However,
|
59
|
+
# they differ in what *version* of your app code they run against, and in
|
60
|
+
# certain other constraints and performance characteristics. More detailed
|
61
|
+
# information on using the two strategies is provided in the sections below.
|
42
62
|
#
|
43
|
-
#
|
44
|
-
#
|
45
|
-
#
|
63
|
+
# Apps deployed to the App Engine *flexible environment* will use the
|
64
|
+
# `cloud_build` strategy by default. However, you can force an app to use the
|
65
|
+
# `deployment` strategy instead. (You might do so if you need to connect to a
|
66
|
+
# Cloud SQL database on a VPC using a private IP, because the `cloud_build`
|
67
|
+
# strategy does not support private IPs.) To force use of `deployment`, set
|
68
|
+
# the `strategy` parameter in the {AppEngine::Exec} constructor (or the
|
69
|
+
# corresponding `GAE_EXEC_STRATEGY` parameter in the Rake task). Note that
|
70
|
+
# the `deployment` strategy is usually significantly slower than
|
71
|
+
# `cloud_build` for apps in the flexible environment.
|
46
72
|
#
|
47
|
-
#
|
48
|
-
#
|
49
|
-
#
|
73
|
+
# Apps deployed to the App Engine *standard environment* will *always* use
|
74
|
+
# the `deployment` strategy. You cannot force use of the `cloud_build`
|
75
|
+
# strategy.
|
50
76
|
#
|
51
77
|
# ## Prerequisites
|
52
78
|
#
|
53
79
|
# To use App Engine remote execution, you will need:
|
54
80
|
#
|
55
81
|
# * An app deployed to Google App Engine, of course!
|
56
|
-
# * The gcloud SDK
|
82
|
+
# * The [gcloud SDK](https://cloud.google.com/sdk/) installed and configured.
|
57
83
|
# * The `appengine` gem.
|
58
84
|
#
|
59
|
-
# You may also need to grant the Cloud Container Builder service account
|
60
|
-
# any permissions needed by your command. Often, Project Editor permissions
|
61
|
-
# will be sufficient for most tasks. You can find the service account
|
62
|
-
# configuration in the IAM tab in the Cloud Console under the name
|
63
|
-
# `[your-project-number]@cloudbuild.gserviceaccount.com`.
|
64
|
-
#
|
65
85
|
# You may use the `AppEngine::Exec` class to run commands directly. However,
|
66
86
|
# in most cases, it will be easier to run commands via the provided rake
|
67
|
-
# tasks
|
87
|
+
# tasks (see {AppEngine::Tasks}).
|
88
|
+
#
|
89
|
+
# ## Using the "deployment" strategy
|
90
|
+
#
|
91
|
+
# The `deployment` strategy deploys a temporary version of your app to a
|
92
|
+
# single backend App Engine instance, runs the command there, and then
|
93
|
+
# deletes the temporary version when it is finished.
|
94
|
+
#
|
95
|
+
# This is the default strategy (and indeed the only option) for apps running
|
96
|
+
# on the App Engine standard environment. It can also be used for flexible
|
97
|
+
# environment apps, but this is not commonly done because deployment of
|
98
|
+
# flexible environment apps can take a long time.
|
99
|
+
#
|
100
|
+
# Because the `deployment` strategy deploys a temporary version of your app,
|
101
|
+
# it runs against the *current application code* present where the command
|
102
|
+
# was initiated (i.e. the code currently on your workstation if you run the
|
103
|
+
# rake task from your workstation, or the current code on the branch if you
|
104
|
+
# are running from a CI/CD system.) This may be different from the code
|
105
|
+
# actually running in production, so it is important that you run from a
|
106
|
+
# compatible code branch.
|
68
107
|
#
|
69
|
-
#
|
108
|
+
# ### Specifying the host application
|
70
109
|
#
|
71
|
-
#
|
72
|
-
# to
|
110
|
+
# The `deployment` strategy works by deploying a temporary version of your
|
111
|
+
# app, so that it has access to your app's project and settings in App
|
112
|
+
# Engine. In most cases, it can determine this automatically, but depending
|
113
|
+
# on how your app or environment is structured, you may need to give it some
|
114
|
+
# help.
|
73
115
|
#
|
74
|
-
#
|
75
|
-
#
|
76
|
-
#
|
77
|
-
#
|
78
|
-
# created deployment version for that service. That deployment version then
|
79
|
-
# provides the application image that runs your command.
|
116
|
+
# By default, your Google Cloud project is taken from the current gcloud
|
117
|
+
# project. If you need to override this, set the `:project` parameter in the
|
118
|
+
# {AppEngine::Exec} constructor (or the corresponding `GAE_PROJECT`
|
119
|
+
# parameter in the Rake task).
|
80
120
|
#
|
81
|
-
#
|
82
|
-
#
|
83
|
-
#
|
84
|
-
#
|
85
|
-
#
|
121
|
+
# By default, the service name is taken from the App Engine config file.
|
122
|
+
# App Engine execution will assume this file is called `app.yaml` in the
|
123
|
+
# current directory. To use a different config file, set the `config_path`
|
124
|
+
# parameter in the {AppEngine::Exec} constructor (or the corresponding
|
125
|
+
# `GAE_CONFIG` parameter in the Rake task). You may also set the service name
|
126
|
+
# directly, using the `service` parameter (or `GAE_SERVICE` in Rake).
|
86
127
|
#
|
87
|
-
#
|
88
|
-
#
|
89
|
-
#
|
90
|
-
#
|
91
|
-
# by
|
128
|
+
# ### Providing credentials
|
129
|
+
#
|
130
|
+
# Your command will effectively be a deployment of your App Engine app
|
131
|
+
# itself, and will have access to the same credentials. For example, App
|
132
|
+
# Engine provides a service account by default for your app, or your app may
|
133
|
+
# be making use of its own service account key. In either case, make sure the
|
134
|
+
# service account has sufficient access for the command you want to run
|
135
|
+
# (such as database admin credentials).
|
136
|
+
#
|
137
|
+
# ### Other options
|
92
138
|
#
|
93
139
|
# You may also provide a timeout, which is the length of time that App
|
94
140
|
# Engine execution will allow your command to run before it is considered to
|
95
141
|
# have stalled and is terminated. The timeout should be a string of the form
|
96
142
|
# `2h15m10s`. The default is `10m`.
|
97
143
|
#
|
98
|
-
#
|
144
|
+
# The timeout is set via the `timeout` parameter to the {AppEngine::Exec}
|
145
|
+
# constructor, or by setting the `GAE_TIMEOUT` environment variable when
|
146
|
+
# invoking using Rake.
|
147
|
+
#
|
148
|
+
# ### Resource usage and billing
|
149
|
+
#
|
150
|
+
# The `deployment` strategy deploys to a temporary instance of your app in
|
151
|
+
# order to run the command. You may be billed for that usage. However, the
|
152
|
+
# cost should be minimal, because it will then immediately delete that
|
153
|
+
# instance in order to minimize usage.
|
154
|
+
#
|
155
|
+
# If you interrupt the execution (or it crashes), it is possible that the
|
156
|
+
# temporary instance may not get deleted properly. If you suspect this may
|
157
|
+
# have happened, go to the App Engine tab in the cloud console, under
|
158
|
+
# "versions" of your service, and delete the temporary version manually. It
|
159
|
+
# will have a name matching the pattern `appengine-exec-<timestamp>`.
|
160
|
+
#
|
161
|
+
# ## Using the "cloud_build" strategy
|
162
|
+
#
|
163
|
+
# The `cloud_build` strategy takes the application image that App Engine is
|
164
|
+
# actually using to run your app, and uses it to spin up a copy of your app
|
165
|
+
# in [Google Cloud Build](https://cloud.google.com/cloud-build) (along with
|
166
|
+
# an emulation layer that emulates certain App Engine services such as Cloud
|
167
|
+
# SQL connection sockets). The command then gets run in the Cloud Build
|
168
|
+
# environment.
|
169
|
+
#
|
170
|
+
# This is the default strategy for apps running on the App Engine flexible
|
171
|
+
# environment. (It is not available for standard environment apps.) Note that
|
172
|
+
# the `cloud_build` strategy cannot be used if your command needs to connect
|
173
|
+
# to a database over a [VPC](https://cloud.google.com/vpc/) private IP
|
174
|
+
# address. This is because it runs on virtual machines provided by the Cloud
|
175
|
+
# Build service, which are not part of your VPC. If your database can be
|
176
|
+
# accessed only over a private IP, you should use the `deployment` strategy
|
177
|
+
# instead.
|
178
|
+
#
|
179
|
+
# The Cloud Build log is output to the directory specified by
|
180
|
+
# "CLOUD_BUILD_GCS_LOG_DIR". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
|
181
|
+
# By default, log directory name is
|
182
|
+
# "gs://[PROJECT_NUMBER].cloudbuild-logs.googleusercontent.com/".
|
183
|
+
#
|
184
|
+
# ### Specifying the host application
|
185
|
+
#
|
186
|
+
# The `cloud_build` strategy needs to know exactly which app, service, and
|
187
|
+
# version of your app, to identify the application image to use.
|
188
|
+
#
|
189
|
+
# By default, your Google Cloud project is taken from the current gcloud
|
190
|
+
# project. If you need to override this, set the `:project` parameter in the
|
191
|
+
# {AppEngine::Exec} constructor (or the corresponding `GAE_PROJECT`
|
192
|
+
# parameter in the Rake task).
|
193
|
+
#
|
194
|
+
# By default, the service name is taken from the App Engine config file.
|
195
|
+
# App Engine execution will assume this file is called `app.yaml` in the
|
196
|
+
# current directory. To use a different config file, set the `config_path`
|
197
|
+
# parameter in the {AppEngine::Exec} constructor (or the corresponding
|
198
|
+
# `GAE_CONFIG` parameter in the Rake task). You may also set the service name
|
199
|
+
# directly, using the `service` parameter (or `GAE_SERVICE` in Rake).
|
200
|
+
#
|
201
|
+
# By default, the image of the most recently deployed version of your app is
|
202
|
+
# used. (Note that this most recently deployed version may not be the same
|
203
|
+
# version that is currently receiving traffic: for example, if you deployed
|
204
|
+
# with `--no-promote`.) To use a different version, set the `version`
|
205
|
+
# parameter in the {AppEngine::Exec} constructor (or the corresponding
|
206
|
+
# `GAE_VERSION` parameter in the Rake task).
|
99
207
|
#
|
100
|
-
#
|
101
|
-
#
|
102
|
-
#
|
103
|
-
#
|
104
|
-
#
|
208
|
+
# ### Providing credentials
|
209
|
+
#
|
210
|
+
# By default, the `cloud_build` strategy uses your project's Cloud Build
|
211
|
+
# service account for its credentials. Unless your command provides its own
|
212
|
+
# service account key, you may need to grant the Cloud Build service account
|
213
|
+
# any permissions needed to execute your command (such as access to your
|
214
|
+
# database). For most tasks, it is sufficient to grant Project Editor
|
215
|
+
# permissions to the service account. You can find the service account
|
216
|
+
# configuration in the IAM tab in the Cloud Console under the name
|
217
|
+
# `[your-project-number]@cloudbuild.gserviceaccount.com`.
|
218
|
+
#
|
219
|
+
# ### Other options
|
220
|
+
#
|
221
|
+
# You may also provide a timeout, which is the length of time that App
|
222
|
+
# Engine execution will allow your command to run before it is considered to
|
223
|
+
# have stalled and is terminated. The timeout should be a string of the form
|
224
|
+
# `2h15m10s`. The default is `10m`.
|
225
|
+
#
|
226
|
+
# The timeout is set via the `timeout` parameter to the {AppEngine::Exec}
|
227
|
+
# constructor, or by setting the `GAE_TIMEOUT` environment variable when
|
228
|
+
# invoking using Rake.
|
229
|
+
#
|
230
|
+
# You can also set the wrapper image used to emulate the App Engine runtime
|
231
|
+
# environment, by setting the `wrapper_image` parameter to the constructor,
|
232
|
+
# or by setting the `GAE_EXEC_WRAPPER_IMAGE` environment variable. Generally,
|
233
|
+
# you will not need to do this unless you are testing a new wrapper image.
|
234
|
+
#
|
235
|
+
# ### Resource usage and billing
|
236
|
+
#
|
237
|
+
# The `cloud_build` strategy uses virtual machine resources provided by
|
238
|
+
# Google Cloud Build. Generally, a certain number of usage minutes per day is
|
239
|
+
# covered under a free tier, but additional compute usage beyond that time is
|
240
|
+
# billed to your Google Cloud account. For more details,
|
241
|
+
# see https://cloud.google.com/cloud-build/pricing
|
105
242
|
#
|
106
243
|
# If your command makes API calls or utilizes other cloud resources, you may
|
107
|
-
# also be billed for that usage. However,
|
108
|
-
# actual App Engine instances, and
|
109
|
-
# Engine instance usage.
|
244
|
+
# also be billed for that usage. However, the `cloud_build` strategy (unlike
|
245
|
+
# the `deployment` strategy) does not use actual App Engine instances, and
|
246
|
+
# you will not be billed for additional App Engine instance usage.
|
110
247
|
#
|
111
248
|
class Exec
|
112
|
-
|
113
|
-
@
|
114
|
-
@
|
115
|
-
@
|
116
|
-
@default_wrapper_image = "gcr.io/google-appengine/exec-wrapper".freeze
|
117
|
-
|
249
|
+
@default_timeout = "10m"
|
250
|
+
@default_service = "default"
|
251
|
+
@default_config_path = "./app.yaml"
|
252
|
+
@default_wrapper_image = "gcr.io/google-appengine/exec-wrapper:latest"
|
118
253
|
|
119
254
|
##
|
120
255
|
# Base class for exec-related usage errors.
|
@@ -122,6 +257,41 @@ module AppEngine
|
|
122
257
|
class UsageError < ::StandardError
|
123
258
|
end
|
124
259
|
|
260
|
+
##
|
261
|
+
# Unsupported strategy
|
262
|
+
#
|
263
|
+
class UnsupportedStrategy < UsageError
|
264
|
+
def initialize strategy, app_env
|
265
|
+
@strategy = strategy
|
266
|
+
@app_env = app_env
|
267
|
+
super "Strategy \"#{strategy}\" not supported for the #{app_env}" \
|
268
|
+
" environment"
|
269
|
+
end
|
270
|
+
attr_reader :strategy
|
271
|
+
attr_reader :app_env
|
272
|
+
end
|
273
|
+
|
274
|
+
##
|
275
|
+
# Exception raised when a parameter is malformed.
|
276
|
+
#
|
277
|
+
class BadParameter < UsageError
|
278
|
+
def initialize param, value
|
279
|
+
@param_name = param
|
280
|
+
@value = value
|
281
|
+
super "Bad value for #{param}: #{value}"
|
282
|
+
end
|
283
|
+
attr_reader :param_name
|
284
|
+
attr_reader :value
|
285
|
+
end
|
286
|
+
|
287
|
+
##
|
288
|
+
# Exception raised when gcloud has no default project.
|
289
|
+
#
|
290
|
+
class NoDefaultProject < UsageError
|
291
|
+
def initialize
|
292
|
+
super "No default project set."
|
293
|
+
end
|
294
|
+
end
|
125
295
|
|
126
296
|
##
|
127
297
|
# Exception raised when the App Engine config file could not be found.
|
@@ -150,7 +320,7 @@ module AppEngine
|
|
150
320
|
# versions at all could be found for the given service.
|
151
321
|
#
|
152
322
|
class NoSuchVersion < UsageError
|
153
|
-
def initialize service, version=nil
|
323
|
+
def initialize service, version = nil
|
154
324
|
@service = service
|
155
325
|
@version = version
|
156
326
|
if version
|
@@ -163,25 +333,7 @@ module AppEngine
|
|
163
333
|
attr_reader :version
|
164
334
|
end
|
165
335
|
|
166
|
-
##
|
167
|
-
# Exception raised when an explicitly-specified service name conflicts with
|
168
|
-
# a config-specified service name.
|
169
|
-
#
|
170
|
-
class ServiceNameConflict < UsageError
|
171
|
-
def initialize service_name, config_name, config_path
|
172
|
-
@service_name = service_name
|
173
|
-
@config_name = config_name
|
174
|
-
@config_path = config_path
|
175
|
-
super "Service name conflicts with config file"
|
176
|
-
end
|
177
|
-
attr_reader :service_name
|
178
|
-
attr_reader :config_name
|
179
|
-
attr_reader :config_path
|
180
|
-
end
|
181
|
-
|
182
|
-
|
183
336
|
class << self
|
184
|
-
|
185
337
|
## @return [String] Default command timeout.
|
186
338
|
attr_accessor :default_timeout
|
187
339
|
|
@@ -205,78 +357,138 @@ module AppEngine
|
|
205
357
|
# the service name from the config file.
|
206
358
|
# @param config_path [String,nil] App Engine config file to get the
|
207
359
|
# service name from if the service name is not provided directly.
|
208
|
-
#
|
209
|
-
#
|
210
|
-
#
|
211
|
-
#
|
212
|
-
#
|
213
|
-
#
|
360
|
+
# If omitted, defaults to the value returned by
|
361
|
+
# {AppEngine::Exec.default_config_path}.
|
362
|
+
# @param version [String,nil] Version string. If omitted, defaults to the
|
363
|
+
# most recently created version of the given service (which may not
|
364
|
+
# be the one currently receiving traffic).
|
365
|
+
# @param timeout [String,nil] Timeout string. If omitted, defaults to the
|
366
|
+
# value returned by {AppEngine::Exec.default_timeout}.
|
367
|
+
# @param wrapper_image [String,nil] The fully qualified name of the
|
368
|
+
# wrapper image to use. (Applies only to the "cloud_build" strategy.)
|
369
|
+
# @param strategy [String,nil] The execution strategy to use, or `nil` to
|
370
|
+
# choose a default based on the App Engine environment (flexible or
|
371
|
+
# standard). Allowed values are `nil`, `"deployment"` (which is the
|
372
|
+
# default for Standard), and `"cloud_build"` (which is the default
|
373
|
+
# for Flexible).
|
374
|
+
# @param gcs_log_dir [String,nil] GCS bucket name of the cloud build log
|
375
|
+
# when strategy is "cloud_build". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
|
214
376
|
#
|
215
377
|
def new_rake_task name, args: [], env_args: [],
|
216
378
|
service: nil, config_path: nil, version: nil,
|
217
|
-
timeout: nil
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
if escaped_args.empty?
|
222
|
-
name_with_args = name
|
223
|
-
else
|
224
|
-
name_with_args = "#{name}[#{escaped_args.join ','}]"
|
379
|
+
timeout: nil, project: nil, wrapper_image: nil,
|
380
|
+
strategy: nil, gcs_log_dir: nil
|
381
|
+
escaped_args = args.map do |arg|
|
382
|
+
arg.gsub(/[,\[\]]/) { |m| "\\#{m}" }
|
225
383
|
end
|
384
|
+
name_with_args =
|
385
|
+
if escaped_args.empty?
|
386
|
+
name
|
387
|
+
else
|
388
|
+
"#{name}[#{escaped_args.join ','}]"
|
389
|
+
end
|
226
390
|
new ["bundle", "exec", "rake", name_with_args] + env_args,
|
227
391
|
service: service, config_path: config_path, version: version,
|
228
|
-
timeout: timeout
|
392
|
+
timeout: timeout, project: project, wrapper_image: wrapper_image,
|
393
|
+
strategy: strategy, gcs_log_dir: gcs_log_dir
|
229
394
|
end
|
230
|
-
|
231
395
|
end
|
232
396
|
|
233
|
-
|
234
397
|
##
|
235
398
|
# Create an execution for the given command.
|
236
399
|
#
|
237
400
|
# @param command [Array<String>] The command in array form.
|
401
|
+
# @param project [String,nil] ID of the project. If omitted, obtains
|
402
|
+
# the project from gcloud.
|
238
403
|
# @param service [String,nil] Name of the service. If omitted, obtains
|
239
404
|
# the service name from the config file.
|
240
405
|
# @param config_path [String,nil] App Engine config file to get the
|
241
406
|
# service name from if the service name is not provided directly.
|
242
|
-
#
|
243
|
-
#
|
244
|
-
#
|
245
|
-
#
|
246
|
-
#
|
247
|
-
#
|
407
|
+
# If omitted, defaults to the value returned by
|
408
|
+
# {AppEngine::Exec.default_config_path}.
|
409
|
+
# @param version [String,nil] Version string. If omitted, defaults to the
|
410
|
+
# most recently created version of the given service (which may not be
|
411
|
+
# the one currently receiving traffic).
|
412
|
+
# @param timeout [String,nil] Timeout string. If omitted, defaults to the
|
413
|
+
# value returned by {AppEngine::Exec.default_timeout}.
|
414
|
+
# @param wrapper_image [String,nil] The fully qualified name of the wrapper
|
415
|
+
# image to use. (Applies only to the "cloud_build" strategy.)
|
416
|
+
# @param strategy [String,nil] The execution strategy to use, or `nil` to
|
417
|
+
# choose a default based on the App Engine environment (flexible or
|
418
|
+
# standard). Allowed values are `nil`, `"deployment"` (which is the
|
419
|
+
# default for Standard), and `"cloud_build"` (which is the default for
|
420
|
+
# Flexible).
|
421
|
+
# @param gcs_log_dir [String,nil] GCS bucket name of the cloud build log
|
422
|
+
# when strategy is "cloud_build". (ex. "gs://BUCKET-NAME/FOLDER-NAME")
|
248
423
|
#
|
249
424
|
def initialize command,
|
250
|
-
|
425
|
+
project: nil, service: nil, config_path: nil, version: nil,
|
426
|
+
timeout: nil, wrapper_image: nil, strategy: nil, gcs_log_dir: nil
|
251
427
|
@command = command
|
252
428
|
@service = service
|
253
429
|
@config_path = config_path
|
254
430
|
@version = version
|
255
431
|
@timeout = timeout
|
256
|
-
@
|
432
|
+
@project = project
|
433
|
+
@wrapper_image = wrapper_image
|
434
|
+
@strategy = strategy
|
435
|
+
@gcs_log_dir = gcs_log_dir
|
257
436
|
|
258
437
|
yield self if block_given?
|
259
438
|
end
|
260
439
|
|
440
|
+
##
|
441
|
+
# @return [String] The project ID.
|
442
|
+
# @return [nil] if the default gcloud project should be used.
|
443
|
+
#
|
444
|
+
attr_accessor :project
|
261
445
|
|
262
|
-
##
|
446
|
+
##
|
447
|
+
# @return [String] The service name.
|
448
|
+
# @return [nil] if the service should be obtained from the app config.
|
449
|
+
#
|
263
450
|
attr_accessor :service
|
264
451
|
|
265
|
-
##
|
452
|
+
##
|
453
|
+
# @return [String] Path to the config file.
|
454
|
+
# @return [nil] if the default of `./app.yaml` should be used.
|
455
|
+
#
|
266
456
|
attr_accessor :config_path
|
267
457
|
|
268
|
-
##
|
458
|
+
##
|
459
|
+
# @return [String] Service version of the image to use.
|
460
|
+
# @return [nil] if the most recent should be used.
|
461
|
+
#
|
269
462
|
attr_accessor :version
|
270
463
|
|
271
|
-
##
|
464
|
+
##
|
465
|
+
# @return [String] The command timeout, in `1h23m45s` format.
|
466
|
+
# @return [nil] if the default of `10m` should be used.
|
467
|
+
#
|
272
468
|
attr_accessor :timeout
|
273
469
|
|
274
|
-
##
|
470
|
+
##
|
471
|
+
# The command to run.
|
472
|
+
#
|
473
|
+
# @return [String] if the command is a script to be run in a shell.
|
474
|
+
# @return [Array<String>] if the command is a posix command to be run
|
475
|
+
# directly without a shell.
|
476
|
+
#
|
275
477
|
attr_accessor :command
|
276
478
|
|
277
|
-
##
|
479
|
+
##
|
480
|
+
# @return [String] Custom wrapper image to use.
|
481
|
+
# @return [nil] if the default should be used.
|
482
|
+
#
|
278
483
|
attr_accessor :wrapper_image
|
279
484
|
|
485
|
+
##
|
486
|
+
# @return [String] The execution strategy to use. Allowed values are
|
487
|
+
# `"deployment"` and `"cloud_build"`.
|
488
|
+
# @return [nil] to choose a default based on the App Engine environment
|
489
|
+
# (flexible or standard).
|
490
|
+
#
|
491
|
+
attr_accessor :strategy
|
280
492
|
|
281
493
|
##
|
282
494
|
# Executes the command synchronously. Streams the logs back to standard out
|
@@ -284,29 +496,15 @@ module AppEngine
|
|
284
496
|
#
|
285
497
|
def start
|
286
498
|
resolve_parameters
|
287
|
-
|
288
|
-
|
289
|
-
|
290
|
-
|
291
|
-
|
292
|
-
|
293
|
-
|
294
|
-
config = build_config command, image, env_variables, cloud_sql_instances
|
295
|
-
file = ::Tempfile.new ["cloudbuild_", ".json"]
|
296
|
-
begin
|
297
|
-
::JSON.dump config, file
|
298
|
-
file.flush
|
299
|
-
Util::Gcloud.execute [
|
300
|
-
"container", "builds", "submit",
|
301
|
-
"--no-source",
|
302
|
-
"--config=#{file.path}",
|
303
|
-
"--timeout=#{@timeout}"]
|
304
|
-
ensure
|
305
|
-
file.close!
|
499
|
+
app_info = version_info @service, @version
|
500
|
+
resolve_strategy app_info["env"]
|
501
|
+
if @strategy == "cloud_build"
|
502
|
+
start_build_strategy app_info
|
503
|
+
else
|
504
|
+
start_deployment_strategy app_info
|
306
505
|
end
|
307
506
|
end
|
308
507
|
|
309
|
-
|
310
508
|
private
|
311
509
|
|
312
510
|
##
|
@@ -314,61 +512,55 @@ module AppEngine
|
|
314
512
|
# Resolves and canonicalizes all the parameters.
|
315
513
|
#
|
316
514
|
def resolve_parameters
|
317
|
-
|
318
|
-
|
319
|
-
|
320
|
-
|
321
|
-
config_service = config_path = nil
|
322
|
-
if @config_path || !@service
|
323
|
-
config_service = begin
|
324
|
-
config_path = @config_path || Exec.default_config_path
|
325
|
-
::YAML.load_file(config_path)["service"] || Exec.default_service
|
326
|
-
rescue ::Errno::ENOENT
|
327
|
-
raise ConfigFileNotFound.new config_path
|
328
|
-
rescue
|
329
|
-
raise BadConfigFileFormat.new config_path
|
330
|
-
end
|
331
|
-
end
|
332
|
-
if @service && config_service && @service != config_service
|
333
|
-
raise ServiceNameConflict.new @service, config_service, config_path
|
334
|
-
end
|
335
|
-
|
336
|
-
@service ||= config_service
|
515
|
+
@timestamp_suffix = ::Time.now.strftime "%Y%m%d%H%M%S"
|
516
|
+
@command = ::Shellwords.split @command.to_s unless @command.is_a? Array
|
517
|
+
@project ||= default_project
|
518
|
+
@service ||= service_from_config || Exec.default_service
|
337
519
|
@version ||= latest_version @service
|
338
520
|
@timeout ||= Exec.default_timeout
|
521
|
+
@timeout_seconds = parse_timeout @timeout
|
339
522
|
@wrapper_image ||= Exec.default_wrapper_image
|
523
|
+
self
|
340
524
|
end
|
341
525
|
|
342
|
-
|
343
|
-
|
344
|
-
|
345
|
-
|
346
|
-
# @param command [Array<String>] The command in array form.
|
347
|
-
# @param image [String] The fully qualified image path.
|
348
|
-
# @param env_variables[Hash<String,String>] Environment variables.
|
349
|
-
# @param cloud_sql_instances[String,Array<String>] Names of cloud sql
|
350
|
-
# instances to connect to.
|
351
|
-
#
|
352
|
-
def build_config command, image, env_variables, cloud_sql_instances
|
353
|
-
args = ["-i", image]
|
354
|
-
env_variables.each do |k, v|
|
355
|
-
args << "-e" << "#{k}=#{v}"
|
526
|
+
def resolve_strategy app_env
|
527
|
+
@strategy = @strategy.to_s.downcase
|
528
|
+
if @strategy.empty?
|
529
|
+
@strategy = app_env == "flexible" ? "cloud_build" : "deployment"
|
356
530
|
end
|
357
|
-
|
358
|
-
|
359
|
-
|
360
|
-
args << "-s" << sql
|
361
|
-
end
|
531
|
+
if app_env == "standard" && @strategy == "cloud_build" ||
|
532
|
+
@strategy != "cloud_build" && @strategy != "deployment"
|
533
|
+
raise UnsupportedStrategy.new @strategy, app_env
|
362
534
|
end
|
363
|
-
|
364
|
-
|
535
|
+
@strategy
|
536
|
+
end
|
365
537
|
|
366
|
-
|
367
|
-
|
368
|
-
|
369
|
-
|
370
|
-
|
371
|
-
|
538
|
+
def service_from_config
|
539
|
+
return nil if !@config_path && @service
|
540
|
+
@config_path ||= Exec.default_config_path
|
541
|
+
::YAML.load_file(config_path)["service"]
|
542
|
+
rescue ::Errno::ENOENT
|
543
|
+
raise ConfigFileNotFound, @config_path
|
544
|
+
rescue ::StandardError
|
545
|
+
raise BadConfigFileFormat, @config_path
|
546
|
+
end
|
547
|
+
|
548
|
+
def default_project
|
549
|
+
result = Util::Gcloud.execute \
|
550
|
+
["config", "get-value", "project"],
|
551
|
+
capture: true, assert: false
|
552
|
+
result.strip!
|
553
|
+
raise NoDefaultProject if result.empty?
|
554
|
+
result
|
555
|
+
end
|
556
|
+
|
557
|
+
def parse_timeout timeout_str
|
558
|
+
matched = timeout_str =~ /^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)s?)?$/
|
559
|
+
raise BadParameter.new "timeout", timeout_str unless matched
|
560
|
+
hours = ::Regexp.last_match(1).to_i
|
561
|
+
minutes = ::Regexp.last_match(2).to_i
|
562
|
+
seconds = ::Regexp.last_match(3).to_i
|
563
|
+
hours * 3600 + minutes * 60 + seconds
|
372
564
|
end
|
373
565
|
|
374
566
|
##
|
@@ -380,15 +572,18 @@ module AppEngine
|
|
380
572
|
# @return [String] Name of the most recent version.
|
381
573
|
#
|
382
574
|
def latest_version service
|
383
|
-
result = Util::Gcloud.execute
|
575
|
+
result = Util::Gcloud.execute \
|
576
|
+
[
|
384
577
|
"app", "versions", "list",
|
385
|
-
"--
|
386
|
-
"--
|
387
|
-
"--
|
388
|
-
"--
|
389
|
-
|
578
|
+
"--project", @project,
|
579
|
+
"--service", service,
|
580
|
+
"--format", "get(version.id)",
|
581
|
+
"--sort-by", "~version.createTime",
|
582
|
+
"--limit", "1"
|
583
|
+
],
|
584
|
+
capture: true, assert: false
|
390
585
|
result = result.split.first
|
391
|
-
raise NoSuchVersion
|
586
|
+
raise NoSuchVersion, service unless result
|
392
587
|
result
|
393
588
|
end
|
394
589
|
|
@@ -400,23 +595,263 @@ module AppEngine
|
|
400
595
|
# "default" is used.
|
401
596
|
# @param version [String] Name of the version. If omitted, the most
|
402
597
|
# recently deployed is used.
|
403
|
-
# @return [Hash
|
404
|
-
#
|
405
|
-
#
|
598
|
+
# @return [Hash] A collection of fields parsed from the JSON representation
|
599
|
+
# of the version
|
600
|
+
# @return [nil] if the requested version doesn't exist.
|
406
601
|
#
|
407
602
|
def version_info service, version
|
408
603
|
service ||= "default"
|
409
604
|
version ||= latest_version service
|
410
|
-
result = Util::Gcloud.execute
|
605
|
+
result = Util::Gcloud.execute \
|
606
|
+
[
|
411
607
|
"app", "versions", "describe", version,
|
412
|
-
"--
|
413
|
-
"--
|
414
|
-
|
608
|
+
"--project", @project,
|
609
|
+
"--service", service,
|
610
|
+
"--format", "json"
|
611
|
+
],
|
612
|
+
capture: true, assert: false
|
415
613
|
result.strip!
|
416
614
|
raise NoSuchVersion.new(service, version) if result.empty?
|
417
615
|
::JSON.parse result
|
418
616
|
end
|
419
617
|
|
420
|
-
|
618
|
+
##
|
619
|
+
# @private
|
620
|
+
# Performs exec on a GAE standard app.
|
621
|
+
#
|
622
|
+
def start_deployment_strategy app_info
|
623
|
+
describe_deployment_strategy
|
624
|
+
entrypoint_file = app_yaml_file = temp_version = nil
|
625
|
+
begin
|
626
|
+
puts "\n---------- DEPLOY COMMAND ----------"
|
627
|
+
secret = create_secret
|
628
|
+
entrypoint_file = copy_entrypoint secret
|
629
|
+
app_yaml_file = copy_app_yaml app_info, entrypoint_file
|
630
|
+
temp_version = deploy_temp_app app_yaml_file
|
631
|
+
puts "\n---------- EXECUTE COMMAND ----------"
|
632
|
+
puts "COMMAND: #{@command.inspect}\n\n"
|
633
|
+
exit_status = track_status temp_version, secret
|
634
|
+
puts "\nEXIT STATUS: #{exit_status}"
|
635
|
+
ensure
|
636
|
+
puts "\n---------- CLEANUP ----------"
|
637
|
+
::File.unlink entrypoint_file if entrypoint_file
|
638
|
+
::File.unlink app_yaml_file if app_yaml_file
|
639
|
+
delete_temp_version temp_version
|
640
|
+
end
|
641
|
+
end
|
642
|
+
|
643
|
+
def describe_deployment_strategy
|
644
|
+
puts "\nUsing the `deployment` strategy for appengine:exec"
|
645
|
+
puts "(i.e. deploying a temporary version of your app)"
|
646
|
+
puts "PROJECT: #{@project}"
|
647
|
+
puts "SERVICE: #{@service}"
|
648
|
+
puts "TIMEOUT: #{@timeout}"
|
649
|
+
end
|
650
|
+
|
651
|
+
def create_secret
|
652
|
+
::SecureRandom.alphanumeric 20
|
653
|
+
end
|
654
|
+
|
655
|
+
def copy_entrypoint secret
|
656
|
+
entrypoint_template =
|
657
|
+
::File.join(::File.dirname(::File.dirname(__dir__)),
|
658
|
+
"data", "exec_standard_entrypoint.rb.erb")
|
659
|
+
entrypoint_file = "appengine_exec_entrypoint_#{@timestamp_suffix}.rb"
|
660
|
+
erb = ::ERB.new ::File.read entrypoint_template
|
661
|
+
data = {
|
662
|
+
secret: secret.inspect, command: command.inspect
|
663
|
+
}
|
664
|
+
result = erb.result_with_hash data
|
665
|
+
::File.open entrypoint_file, "w" do |file|
|
666
|
+
file.write result
|
667
|
+
end
|
668
|
+
entrypoint_file
|
669
|
+
end
|
670
|
+
|
671
|
+
def copy_app_yaml app_info, entrypoint_file
|
672
|
+
yaml_data = {
|
673
|
+
"runtime" => app_info["runtime"],
|
674
|
+
"service" => @service,
|
675
|
+
"entrypoint" => "ruby #{entrypoint_file}",
|
676
|
+
"env_variables" => app_info["envVariables"],
|
677
|
+
"manual_scaling" => { "instances" => 1 }
|
678
|
+
}
|
679
|
+
if app_info["env"] == "flexible"
|
680
|
+
complete_flex_app_yaml yaml_data, app_info
|
681
|
+
else
|
682
|
+
complete_standard_app_yaml yaml_data, app_info
|
683
|
+
end
|
684
|
+
app_yaml_file = "appengine_exec_config_#{@timestamp_suffix}.yaml"
|
685
|
+
::File.open app_yaml_file, "w" do |file|
|
686
|
+
::Psych.dump yaml_data, file
|
687
|
+
end
|
688
|
+
app_yaml_file
|
689
|
+
end
|
690
|
+
|
691
|
+
def complete_flex_app_yaml yaml_data, app_info
|
692
|
+
yaml_data["env"] = "flex"
|
693
|
+
orig_path = (app_info["betaSettings"] || {})["module_yaml_path"]
|
694
|
+
return unless orig_path
|
695
|
+
orig_yaml = ::YAML.load_file orig_path
|
696
|
+
copy_keys = ["skip_files", "resources", "network", "runtime_config",
|
697
|
+
"beta_settings"]
|
698
|
+
copy_keys.each do |key|
|
699
|
+
yaml_data[key] = orig_yaml[key] if orig_yaml[key]
|
700
|
+
end
|
701
|
+
end
|
702
|
+
|
703
|
+
def complete_standard_app_yaml yaml_data, app_info
|
704
|
+
yaml_data["instance_class"] = app_info["instanceClass"].sub(/^F/, "B")
|
705
|
+
end
|
706
|
+
|
707
|
+
def deploy_temp_app app_yaml_file
|
708
|
+
temp_version = "appengine-exec-#{@timestamp_suffix}"
|
709
|
+
Util::Gcloud.execute [
|
710
|
+
"app", "deploy", app_yaml_file,
|
711
|
+
"--project", @project,
|
712
|
+
"--version", temp_version,
|
713
|
+
"--no-promote", "--quiet"
|
714
|
+
]
|
715
|
+
temp_version
|
716
|
+
end
|
717
|
+
|
718
|
+
def track_status temp_version, secret
|
719
|
+
host = "#{temp_version}.#{@service}.#{@project}.appspot.com"
|
720
|
+
::Net::HTTP.start host do |http|
|
721
|
+
outpos = errpos = 0
|
722
|
+
delay = 0.0
|
723
|
+
loop do
|
724
|
+
sleep delay
|
725
|
+
uri = URI("http://#{host}/#{secret}")
|
726
|
+
uri.query = ::URI.encode_www_form outpos: outpos, errpos: errpos
|
727
|
+
response = http.request_get uri
|
728
|
+
data = ::JSON.parse response.body
|
729
|
+
data["outlines"].each { |line| puts "[STDOUT] #{line}" }
|
730
|
+
data["errlines"].each { |line| puts "[STDERR] #{line}" }
|
731
|
+
outpos = data["outpos"]
|
732
|
+
errpos = data["errpos"]
|
733
|
+
return data["status"] if data["status"]
|
734
|
+
if data["time"] > @timeout_seconds
|
735
|
+
http.request_post "/#{secret}/kill", ""
|
736
|
+
return "timeout"
|
737
|
+
end
|
738
|
+
if data["outlines"].empty? && data["errlines"].empty?
|
739
|
+
delay += 0.1
|
740
|
+
delay = 1.0 if delay > 1.0
|
741
|
+
else
|
742
|
+
delay = 0.0
|
743
|
+
end
|
744
|
+
end
|
745
|
+
end
|
746
|
+
end
|
747
|
+
|
748
|
+
def delete_temp_version temp_version
|
749
|
+
Util::Gcloud.execute [
|
750
|
+
"app", "versions", "delete", temp_version,
|
751
|
+
"--project", @project,
|
752
|
+
"--service", @service,
|
753
|
+
"--quiet"
|
754
|
+
]
|
755
|
+
end
|
756
|
+
|
757
|
+
##
|
758
|
+
# @private
|
759
|
+
# Performs exec on a GAE flexible app.
|
760
|
+
#
|
761
|
+
def start_build_strategy app_info
|
762
|
+
env_variables = app_info["envVariables"] || {}
|
763
|
+
beta_settings = app_info["betaSettings"] || {}
|
764
|
+
cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
|
765
|
+
container = app_info["deployment"]["container"]
|
766
|
+
image = container ? container["image"] : image_from_build(app_info)
|
767
|
+
|
768
|
+
describe_build_strategy
|
769
|
+
|
770
|
+
config = build_config command, image, env_variables, cloud_sql_instances
|
771
|
+
file = ::Tempfile.new ["cloudbuild_", ".json"]
|
772
|
+
begin
|
773
|
+
::JSON.dump config, file
|
774
|
+
file.flush
|
775
|
+
execute_command = [
|
776
|
+
"builds", "submit",
|
777
|
+
"--project", @project,
|
778
|
+
"--no-source",
|
779
|
+
"--config", file.path,
|
780
|
+
"--timeout", @timeout
|
781
|
+
]
|
782
|
+
execute_command.concat ["--gcs-log-dir", @gcs_log_dir] unless @gcs_log_dir.nil?
|
783
|
+
Util::Gcloud.execute execute_command
|
784
|
+
ensure
|
785
|
+
file.close!
|
786
|
+
end
|
787
|
+
end
|
788
|
+
|
789
|
+
##
|
790
|
+
# @private
|
791
|
+
# Workaround for https://github.com/GoogleCloudPlatform/appengine-ruby/issues/33
|
792
|
+
# Determines the image by looking it up in Cloud Build
|
793
|
+
#
|
794
|
+
def image_from_build app_info
|
795
|
+
create_time = ::DateTime.parse(app_info["createTime"]).to_time.utc
|
796
|
+
after_time = (create_time - 3600).strftime "%Y-%m-%dT%H:%M:%SZ"
|
797
|
+
before_time = (create_time + 3600).strftime "%Y-%m-%dT%H:%M:%SZ"
|
798
|
+
partial_uri = "gcr.io/#{@project}/appengine/#{@service}.#{@version}"
|
799
|
+
filter = "createTime>#{after_time} createTime<#{before_time} images[]:#{partial_uri}"
|
800
|
+
result = Util::Gcloud.execute \
|
801
|
+
[
|
802
|
+
"builds", "list",
|
803
|
+
"--project", @project,
|
804
|
+
"--filter", filter,
|
805
|
+
"--format", "json"
|
806
|
+
],
|
807
|
+
capture: true, assert: false
|
808
|
+
result.strip!
|
809
|
+
raise NoSuchVersion.new(@service, @version) if result.empty?
|
810
|
+
build_info = ::JSON.parse(result).first
|
811
|
+
build_info["images"].first
|
812
|
+
end
|
813
|
+
|
814
|
+
def describe_build_strategy
|
815
|
+
puts "\nUsing the `cloud_build` strategy for appengine:exec"
|
816
|
+
puts "(i.e. running your app image in Cloud Build)"
|
817
|
+
puts "PROJECT: #{@project}"
|
818
|
+
puts "SERVICE: #{@service}"
|
819
|
+
puts "VERSION: #{@version}"
|
820
|
+
puts "TIMEOUT: #{@timeout}"
|
821
|
+
puts ""
|
822
|
+
end
|
823
|
+
|
824
|
+
##
|
825
|
+
# @private
|
826
|
+
# Builds a cloudbuild config as a data structure.
|
827
|
+
#
|
828
|
+
# @param command [Array<String>] The command in array form.
|
829
|
+
# @param image [String] The fully qualified image path.
|
830
|
+
# @param env_variables [Hash<String,String>] Environment variables.
|
831
|
+
# @param cloud_sql_instances [String,Array<String>] Names of cloud sql
|
832
|
+
# instances to connect to.
|
833
|
+
#
|
834
|
+
def build_config command, image, env_variables, cloud_sql_instances
|
835
|
+
args = ["-i", image]
|
836
|
+
env_variables.each do |k, v|
|
837
|
+
v = v.gsub "$", "$$"
|
838
|
+
args << "-e" << "#{k}=#{v}"
|
839
|
+
end
|
840
|
+
unless cloud_sql_instances.empty?
|
841
|
+
cloud_sql_instances = Array(cloud_sql_instances)
|
842
|
+
cloud_sql_instances.each do |sql|
|
843
|
+
args << "-s" << sql
|
844
|
+
end
|
845
|
+
end
|
846
|
+
args << "--"
|
847
|
+
args += command
|
421
848
|
|
849
|
+
{
|
850
|
+
"steps" => [
|
851
|
+
"name" => @wrapper_image,
|
852
|
+
"args" => args
|
853
|
+
]
|
854
|
+
}
|
855
|
+
end
|
856
|
+
end
|
422
857
|
end
|