concourse 0.21.0 → 0.22.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +15 -0
- data/README.md +52 -27
- data/concourse.gemspec +2 -0
- data/lib/concourse.rb +65 -31
- data/lib/concourse/version.rb +1 -1
- metadata +17 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b0107e8ffc49ba8bd8ad43183da66ecb29d89478440fd0cb70b9eb2e1305c427
|
4
|
+
data.tar.gz: 40fa32805af4c9d08d2b2b4f0ebcde942bd6e1135a98b491bba6f148cea1c1ee
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5e9e9633794c23f7306688fd979167def21e3f2efd5ab4ad3de73ba18f64ee87c3fe45accf0d19c4d3afb7cac8a54e198304e2b023b39bc3db40f1d65ffe26b9
|
7
|
+
data.tar.gz: af0504712cfbef8764296617f523eea2d1f97171857c58ea4b85d8e3da70a2e855f3baed3cffebff6af9f7e379711cba2d69d6c9fe2c63d7b68a2b31acddba81
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,20 @@
|
|
1
1
|
# concourse-gem changelog
|
2
2
|
|
3
|
+
## 0.22.0 / 2019-01-18
|
4
|
+
|
5
|
+
### Breaking changes
|
6
|
+
|
7
|
+
* The name of the generated, final pipeline file is now `<pipeline_erb_filename>.generated`.
|
8
|
+
|
9
|
+
|
10
|
+
### Features
|
11
|
+
|
12
|
+
* Introduce rake task `concourse:prune-stalled-workers`
|
13
|
+
* `Concourse.new` now takes an optional `:fly_target` named param to avoid passing this to each task. [#2] (Thanks, @ebmeierj!)
|
14
|
+
* `Concourse.new` now takes an optional `:pipeline_erb_filename` named param to set the name of the pipeline file used. [#1] (Thanks, @ebmeierj!)
|
15
|
+
* `Concourse.new` now takes an optional `:secrets_filename` named param to set the name of the private variable file used. [#1] (Thanks, @ebmeierj!)
|
16
|
+
|
17
|
+
|
3
18
|
## 0.21.0 / 2018-12-26
|
4
19
|
|
5
20
|
* Introduce ruby 2.6 final (and remove 2.6-rc)
|
data/README.md
CHANGED
@@ -12,7 +12,7 @@ In your Rakefile,
|
|
12
12
|
``` ruby
|
13
13
|
require 'concourse'
|
14
14
|
|
15
|
-
Concourse.new("myproject").create_tasks!
|
15
|
+
Concourse.new("myproject", fly_target: "myci").create_tasks!
|
16
16
|
```
|
17
17
|
|
18
18
|
This will create a set of rake tasks for you.
|
@@ -21,21 +21,54 @@ This will create a set of rake tasks for you.
|
|
21
21
|
rake concourse:init
|
22
22
|
```
|
23
23
|
|
24
|
-
The `concourse:init` task will create a subdirectory named `concourse`, and create a Concourse pipeline file named
|
24
|
+
The `concourse:init` task will create a subdirectory named `concourse`, and create a Concourse pipeline file named `<myproject>.yml`, which will be interpreted as an ERB template. It will also ensure that files with sensitive data (`concourse/private.yml` and `concourse/myproject.yml.generated`) are in `.gitignore`.
|
25
25
|
|
26
26
|
|
27
27
|
### Concourse subdirectory name
|
28
28
|
|
29
|
-
You can choose a directory name other than `concourse`:
|
29
|
+
You can choose a directory name other than the default `concourse`:
|
30
30
|
|
31
31
|
``` ruby
|
32
32
|
Concourse.new("myproject", directory: "ci").create_tasks!
|
33
33
|
```
|
34
34
|
|
35
35
|
|
36
|
+
### Concourse `fly` target name
|
37
|
+
|
38
|
+
If the initializer is given no additional parameters, your fly target is assumed to be named "default":
|
39
|
+
|
40
|
+
``` ruby
|
41
|
+
Concourse.new("myproject").create_tasks! # `fly -t default <command>`
|
42
|
+
```
|
43
|
+
|
44
|
+
But this is likely an inappropriate default, and so you can specify your target name:
|
45
|
+
|
46
|
+
``` ruby
|
47
|
+
Concourse.new("myproject", fly_target: "myci").create_tasks! # `fly -t myci <command>`
|
48
|
+
```
|
49
|
+
|
50
|
+
|
51
|
+
### Pipeline file
|
52
|
+
|
53
|
+
By default the pipeline file will be named `<myproject>.yml`, but can be set to something else:
|
54
|
+
|
55
|
+
``` ruby
|
56
|
+
Concourse.new("myproject", pipeline_erb_filename: "pipeline.yml").create_tasks!
|
57
|
+
```
|
58
|
+
|
59
|
+
Note that the generated, final pipeline file is always named `<pipeline_erb_filename>.generated`.
|
60
|
+
|
61
|
+
|
62
|
+
|
36
63
|
### Keeping credentials private
|
37
64
|
|
38
|
-
|
65
|
+
You can use a separate file to keep your pipeline variables secure. By default, `concourse/private.yml` will be used. You can specify a different filename:
|
66
|
+
|
67
|
+
``` ruby
|
68
|
+
Concourse.new("myproject", secrets_filename: "secrets.yml").create_tasks!
|
69
|
+
```
|
70
|
+
|
71
|
+
If the secrets file exists, it will be passed to the `fly` commandline with the `-l` option to fill in template values.
|
39
72
|
|
40
73
|
For example, I might have a concourse config that looks like this:
|
41
74
|
|
@@ -90,54 +123,50 @@ jobs:
|
|
90
123
|
Note that the `windows` rubies are not Docker images, since Concourse's Houdini backend doesn't use Docker. Instead, these are implicitly referring to the supported ruby versions installed by the BOSH release at https://github.com/flavorjones/windows-ruby-dev-tools-release
|
91
124
|
|
92
125
|
|
93
|
-
### `fly_target`
|
94
|
-
|
95
|
-
Any rake task that needs to interact with your Concourse ATC requires a `fly_target` argument. The value should be a fly target `name`, and it's assumed that you're already logged in to that target.
|
96
|
-
|
97
|
-
|
98
126
|
### Managing your Concourse pipeline
|
99
127
|
|
100
128
|
Tasks to manage a local pipeline file, generated from an ERB template:
|
101
129
|
|
102
130
|
```
|
103
|
-
rake concourse:clean
|
104
|
-
rake concourse:generate
|
131
|
+
rake concourse:clean # remove generate pipeline file
|
132
|
+
rake concourse:generate # generate and validate the pipeline file for myproject
|
105
133
|
```
|
106
134
|
|
107
135
|
A task to update your pipeline configuration:
|
108
136
|
|
109
137
|
```
|
110
|
-
rake concourse:set
|
138
|
+
rake concourse:set # upload the pipeline file for myproject
|
111
139
|
```
|
112
140
|
|
113
141
|
Tasks to publicly expose or hide your pipeline:
|
114
142
|
|
115
143
|
```
|
116
|
-
rake concourse:expose
|
117
|
-
rake concourse:hide
|
144
|
+
rake concourse:expose # expose the myproject pipeline
|
145
|
+
rake concourse:hide # hide the myproject pipeline
|
118
146
|
```
|
119
147
|
|
120
148
|
Tasks to pause and unpause your pipeline:
|
121
149
|
|
122
150
|
```
|
123
|
-
rake concourse:pause
|
124
|
-
rake concourse:unpause
|
151
|
+
rake concourse:pause # pause the myproject pipeline
|
152
|
+
rake concourse:unpause # unpause the myproject pipeline
|
125
153
|
```
|
126
154
|
|
127
155
|
And, should you ever need to [nuke the site from orbit][ripley], a task to destroy your pipeline:
|
128
156
|
|
129
157
|
```
|
130
|
-
rake concourse:destroy
|
158
|
+
rake concourse:destroy # destroy the myproject pipeline
|
131
159
|
```
|
132
160
|
|
133
161
|
|
134
162
|
[ripley]: https://www.youtube.com/watch?v=aCbfMkh940Q
|
135
163
|
|
164
|
+
|
136
165
|
### Running tasks with `fly execute`
|
137
166
|
|
138
167
|
```
|
139
|
-
rake concourse:tasks
|
140
|
-
rake concourse:task[
|
168
|
+
rake concourse:tasks # list all the available tasks from the nokogiri pipeline
|
169
|
+
rake concourse:task[job_task,fly_execute_args] # fly execute the specified task
|
141
170
|
```
|
142
171
|
|
143
172
|
where:
|
@@ -149,22 +178,18 @@ where:
|
|
149
178
|
### Aborting running builds
|
150
179
|
|
151
180
|
```
|
152
|
-
rake concourse:abort-builds
|
181
|
+
rake concourse:abort-builds # abort all running builds for this pipeline
|
153
182
|
```
|
154
183
|
|
155
184
|
|
156
|
-
###
|
185
|
+
### Pruning stalled concourse workers
|
157
186
|
|
158
|
-
|
187
|
+
Especially useful if you're deploying via BOSH, which often results in stalled workers;
|
159
188
|
|
160
189
|
```
|
161
|
-
rake concourse:
|
190
|
+
rake concourse:prune-stalled-workers # prune any stalled workers
|
162
191
|
```
|
163
192
|
|
164
|
-
where:
|
165
|
-
|
166
|
-
* _optional_: `team_name` is the name of the pipeline's Concourse Team. Defaults to `main`.
|
167
|
-
|
168
193
|
|
169
194
|
## Installation
|
170
195
|
|
data/concourse.gemspec
CHANGED
@@ -21,6 +21,8 @@ Gem::Specification.new do |spec|
|
|
21
21
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
22
22
|
spec.require_paths = ["lib"]
|
23
23
|
|
24
|
+
spec.add_dependency "term-ansicolor"
|
25
|
+
|
24
26
|
spec.add_development_dependency "bundler", "~> 1.14"
|
25
27
|
spec.add_development_dependency "rake", "~> 12.0"
|
26
28
|
spec.add_development_dependency "rspec", "~> 3.0"
|
data/lib/concourse.rb
CHANGED
@@ -1,9 +1,14 @@
|
|
1
1
|
require "concourse/version"
|
2
2
|
require "yaml"
|
3
3
|
require "tempfile"
|
4
|
+
require 'term/ansicolor'
|
4
5
|
|
5
6
|
class Concourse
|
6
7
|
include Rake::DSL
|
8
|
+
include Term::ANSIColor
|
9
|
+
|
10
|
+
GITIGNORE_FILE = ".gitignore"
|
11
|
+
GITATTRIBUTES_FILE = ".gitattributes"
|
7
12
|
|
8
13
|
# these numbers/names align with public docker image names
|
9
14
|
RUBIES = {
|
@@ -15,14 +20,11 @@ class Concourse
|
|
15
20
|
|
16
21
|
DEFAULT_DIRECTORY = "concourse"
|
17
22
|
|
18
|
-
attr_reader :project_name
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
end
|
24
|
-
return task_args[:fly_target]
|
25
|
-
end
|
23
|
+
attr_reader :project_name
|
24
|
+
attr_reader :directory
|
25
|
+
attr_reader :pipeline_filename, :pipeline_erb_filename
|
26
|
+
attr_reader :fly_target
|
27
|
+
attr_reader :secrets_filename
|
26
28
|
|
27
29
|
def self.url_for fly_target
|
28
30
|
matching_line = `fly targets`.split("\n").grep(/^#{fly_target}/).first
|
@@ -46,12 +48,17 @@ class Concourse
|
|
46
48
|
RUBIES[:mri].select { |r| r =~ /rc/ }
|
47
49
|
end
|
48
50
|
|
49
|
-
def initialize project_name,
|
51
|
+
def initialize project_name, options={}
|
50
52
|
@project_name = project_name
|
51
|
-
|
52
|
-
@
|
53
|
-
@
|
54
|
-
|
53
|
+
|
54
|
+
@directory = options[:directory] || DEFAULT_DIRECTORY
|
55
|
+
@fly_target = options[:fly_target] || "default"
|
56
|
+
base_pipeline_erb_filename = options[:pipeline_erb_filename] || "#{project_name}.yml"
|
57
|
+
base_secrets_filename = options[:secrets_filename] || "private.yml"
|
58
|
+
|
59
|
+
@pipeline_filename = File.join(@directory, "#{base_pipeline_erb_filename}.generated")
|
60
|
+
@pipeline_erb_filename = File.join(@directory, base_pipeline_erb_filename)
|
61
|
+
@secrets_filename = File.join(@directory, base_secrets_filename)
|
55
62
|
end
|
56
63
|
|
57
64
|
def erbify document_string, *args
|
@@ -61,10 +68,32 @@ class Concourse
|
|
61
68
|
def rake_init
|
62
69
|
FileUtils.mkdir_p File.join(directory, "tasks")
|
63
70
|
FileUtils.touch pipeline_erb_filename
|
64
|
-
|
65
|
-
|
66
|
-
|
71
|
+
ensure_in_gitignore secrets_filename
|
72
|
+
ensure_in_gitignore pipeline_filename
|
73
|
+
end
|
74
|
+
|
75
|
+
def ensure_in_gitignore file_glob
|
76
|
+
if File.exist?(GITIGNORE_FILE)
|
77
|
+
if File.read(GITIGNORE_FILE).split("\n").include?(file_glob)
|
78
|
+
note "found '#{file_glob}' already present in #{GITIGNORE_FILE}"
|
79
|
+
return
|
80
|
+
end
|
67
81
|
end
|
82
|
+
note "adding '#{file_glob}' to #{GITIGNORE_FILE}"
|
83
|
+
File.open(GITIGNORE_FILE, "a") { |f| f.puts file_glob }
|
84
|
+
end
|
85
|
+
|
86
|
+
def sh command
|
87
|
+
running "(in #{Dir.pwd}) #{command}"
|
88
|
+
super command, verbose: false
|
89
|
+
end
|
90
|
+
|
91
|
+
def running message
|
92
|
+
print bold, red, "RUNNING: ", reset, message, "\n"
|
93
|
+
end
|
94
|
+
|
95
|
+
def note message
|
96
|
+
print bold, green, "NOTE: ", reset, message, "\n"
|
68
97
|
end
|
69
98
|
|
70
99
|
def create_tasks!
|
@@ -99,23 +128,21 @@ class Concourse
|
|
99
128
|
end
|
100
129
|
|
101
130
|
desc "upload the pipeline file for #{project_name}"
|
102
|
-
task "set"
|
103
|
-
fly_target = Concourse.validate_fly_target t, args
|
131
|
+
task "set" => ["generate"] do |t, args|
|
104
132
|
options = [
|
105
133
|
"-p '#{project_name}'",
|
106
134
|
"-c '#{pipeline_filename}'",
|
107
135
|
]
|
108
|
-
if File.exist?
|
109
|
-
|
110
|
-
options << "-l '#{
|
136
|
+
if File.exist? secrets_filename
|
137
|
+
note "using #{secrets_filename} to resolve template vars"
|
138
|
+
options << "-l '#{secrets_filename}'"
|
111
139
|
end
|
112
140
|
sh "fly -t #{fly_target} set-pipeline #{options.join(" ")}"
|
113
141
|
end
|
114
142
|
|
115
143
|
%w[expose hide pause unpause destroy].each do |command|
|
116
144
|
desc "#{command} the #{project_name} pipeline"
|
117
|
-
task
|
118
|
-
fly_target = Concourse.validate_fly_target t, args
|
145
|
+
task command do |t, args|
|
119
146
|
sh "fly -t #{fly_target} #{command}-pipeline -p #{project_name}"
|
120
147
|
end
|
121
148
|
end
|
@@ -136,14 +163,12 @@ class Concourse
|
|
136
163
|
tasks << "#{job["name"]}/#{task["task"]}"
|
137
164
|
end
|
138
165
|
|
139
|
-
|
166
|
+
note "Available Concourse tasks for #{project_name} are:"
|
140
167
|
tasks.sort.each { |task| puts " * #{task}" }
|
141
168
|
end
|
142
169
|
|
143
170
|
desc "fly execute the specified task"
|
144
|
-
task "task", [:
|
145
|
-
fly_target = Concourse.validate_fly_target t, args
|
146
|
-
|
171
|
+
task "task", [:job_task, :fly_execute_args] => "generate" do |t, args|
|
147
172
|
job_task = args[:job_task]
|
148
173
|
unless job_task
|
149
174
|
raise "ERROR: must specify a task name, like `rake #{t.name}[target,taskname]`"
|
@@ -168,16 +193,25 @@ class Concourse
|
|
168
193
|
# builds commands
|
169
194
|
#
|
170
195
|
desc "abort all running builds for this pipeline"
|
171
|
-
task "abort-builds"
|
172
|
-
fly_target
|
173
|
-
|
174
|
-
`fly -t #{fly_target} builds`.split("\n").each do |line|
|
196
|
+
task "abort-builds" do |t, args|
|
197
|
+
`fly -t #{fly_target} builds`.each_line do |line|
|
175
198
|
pipeline_job, build_id, status = *line.split(/\s+/)[1,3]
|
176
199
|
next unless status == "started"
|
177
200
|
|
178
201
|
sh "fly -t #{fly_target} abort-build -j #{pipeline_job} -b #{build_id}"
|
179
202
|
end
|
180
203
|
end
|
204
|
+
|
205
|
+
#
|
206
|
+
# worker commands
|
207
|
+
#
|
208
|
+
desc "prune any stalled workers"
|
209
|
+
task "prune-stalled-workers" do |t, args|
|
210
|
+
`fly -t #{fly_target} workers | fgrep stalled`.each_line do |line|
|
211
|
+
worker_id = line.split.first
|
212
|
+
system("fly -t #{fly_target} prune-worker -w #{worker_id}")
|
213
|
+
end
|
214
|
+
end
|
181
215
|
end
|
182
216
|
end
|
183
217
|
|
data/lib/concourse/version.rb
CHANGED
metadata
CHANGED
@@ -1,15 +1,29 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: concourse
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.22.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Dalessio
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2019-01-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: term-ansicolor
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
13
27
|
- !ruby/object:Gem::Dependency
|
14
28
|
name: bundler
|
15
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -90,8 +104,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
90
104
|
- !ruby/object:Gem::Version
|
91
105
|
version: '0'
|
92
106
|
requirements: []
|
93
|
-
|
94
|
-
rubygems_version: 2.7.8
|
107
|
+
rubygems_version: 3.0.1
|
95
108
|
signing_key:
|
96
109
|
specification_version: 4
|
97
110
|
summary: Rake tasks for Concourse pipelines.
|