librarian-chef 0.0.1.beta.1
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +9 -0
- data/.rspec +1 -0
- data/.travis.yml +19 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +22 -0
- data/README.md +386 -0
- data/Rakefile +1 -0
- data/lib/librarian-chef.rb +1 -0
- data/lib/librarian/chef.rb +1 -0
- data/lib/librarian/chef/cli.rb +47 -0
- data/lib/librarian/chef/dsl.rb +16 -0
- data/lib/librarian/chef/environment.rb +32 -0
- data/lib/librarian/chef/extension.rb +9 -0
- data/lib/librarian/chef/integration/knife.rb +46 -0
- data/lib/librarian/chef/manifest_reader.rb +59 -0
- data/lib/librarian/chef/source.rb +4 -0
- data/lib/librarian/chef/source/git.rb +25 -0
- data/lib/librarian/chef/source/github.rb +27 -0
- data/lib/librarian/chef/source/local.rb +74 -0
- data/lib/librarian/chef/source/path.rb +12 -0
- data/lib/librarian/chef/source/site.rb +442 -0
- data/lib/librarian/chef/templates/Cheffile +15 -0
- data/lib/librarian/chef/version.rb +5 -0
- data/librarian-chef.gemspec +27 -0
- data/spec/functional/chef/cli_spec.rb +195 -0
- data/spec/functional/chef/source/site_spec.rb +266 -0
- data/spec/integration/chef/source/git_spec.rb +451 -0
- data/spec/integration/chef/source/site_spec.rb +217 -0
- metadata +176 -0
data/.gitignore
ADDED
data/.rspec
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
--color
|
data/.travis.yml
ADDED
@@ -0,0 +1,19 @@
|
|
1
|
+
language: ruby
|
2
|
+
script: rspec spec
|
3
|
+
rvm:
|
4
|
+
- 1.8.7
|
5
|
+
- 1.9.2
|
6
|
+
- 1.9.3
|
7
|
+
- 2.0.0
|
8
|
+
- rbx-18mode
|
9
|
+
- rbx-19mode
|
10
|
+
- jruby-18mode
|
11
|
+
- jruby-19mode
|
12
|
+
- ruby-head
|
13
|
+
matrix:
|
14
|
+
allow_failures:
|
15
|
+
- rvm: rbx-18mode
|
16
|
+
- rvm: rbx-19mode
|
17
|
+
- rvm: jruby-18mode
|
18
|
+
- rvm: jruby-19mode
|
19
|
+
- rvm: ruby-head
|
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2011 ApplicationsOnline, LLC.
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,386 @@
|
|
1
|
+
Librarian-Chef [![Build Status](https://secure.travis-ci.org/applicationsonline/librarian-chef.png)](http://travis-ci.org/applicationsonline/librarian-chef) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/applicationsonline/librarian-chef)
|
2
|
+
=========
|
3
|
+
|
4
|
+
Librarian-Chef is a bundler for your Chef-based infrastructure repositories.
|
5
|
+
|
6
|
+
It is a tool that helps you manage the cookbooks that your chef-repo depends on.
|
7
|
+
Here are some more details.
|
8
|
+
|
9
|
+
Librarian-Chef is a bundler for infrastructure repositories using Chef. You can
|
10
|
+
use Librarian-Chef to resolve your infrastructure's cookbook dependencies, fetch
|
11
|
+
them, and install them into your infrastructure repository.
|
12
|
+
|
13
|
+
Librarian-Chef can resolve and fetch third-party, publicly-released cookbooks,
|
14
|
+
and install them into your infrastructure repository. It can also source
|
15
|
+
cookbooks directly from their own source control repositories.
|
16
|
+
|
17
|
+
Librarian-Chef can also deal with cookbooks you may actively be working on
|
18
|
+
outside your infrastructure repository. For example, it can deal with cookbooks
|
19
|
+
directly from their own private source control repositories, whether they are
|
20
|
+
remote or local to your machine, and it can deal with cookbooks released to and
|
21
|
+
hosted on a private cookbooks server.
|
22
|
+
|
23
|
+
Librarian-Chef is not primarily intended for dealing with the cookbooks you are
|
24
|
+
actively working on *within* your infrastructure repository. In such a case, you
|
25
|
+
can still use Librarian-Chef, but it is likely unnecessary.
|
26
|
+
|
27
|
+
Librarian-Chef *takes over* your `cookbooks/` directory and manages it for you
|
28
|
+
based on your `Cheffile`. Your `Cheffile` becomes the authoritative source for
|
29
|
+
the cookbooks your infrastructure repository depends on. You should not modify
|
30
|
+
the contents of your `cookbooks/` directory when using Librarian-Chef. If you
|
31
|
+
have cookbooks which are, rather than being separate projects, inherently part
|
32
|
+
of your infrastructure repository, then they should go in a separate directory,
|
33
|
+
like your `site-cookbooks/` directory, and you do not need to use Librarian-Chef
|
34
|
+
to manage them.
|
35
|
+
|
36
|
+
### The Cheffile
|
37
|
+
|
38
|
+
Every infrastruture repository that uses Librarian-Chef will have a file named
|
39
|
+
`Cheffile` in the root directory of that repository. The full specification for
|
40
|
+
which third-party, publicly-released cookbooks your infrastructure repository
|
41
|
+
depends will go here.
|
42
|
+
|
43
|
+
Here's an example `Cheffile`:
|
44
|
+
|
45
|
+
site "http://community.opscode.com/api/v1"
|
46
|
+
|
47
|
+
cookbook "ntp"
|
48
|
+
cookbook "timezone", "0.0.1"
|
49
|
+
|
50
|
+
cookbook "rvm",
|
51
|
+
:git => "https://github.com/fnichol/chef-rvm",
|
52
|
+
:ref => "v0.7.1"
|
53
|
+
|
54
|
+
cookbook "cloudera",
|
55
|
+
:path => "vendor/cookbooks/cloudera-cookbook"
|
56
|
+
|
57
|
+
Here's how it works:
|
58
|
+
|
59
|
+
We start off by declaring the *default source* for this `Cheffile`.
|
60
|
+
|
61
|
+
site "http://community.opscode.com/api/v1"
|
62
|
+
|
63
|
+
This default source in this example is the Opscode Community Site API. This is
|
64
|
+
most likely what you will want for your default source. However, you can
|
65
|
+
certainly set up your own API-compatible HTTP endpoint if you want more control.
|
66
|
+
|
67
|
+
Any time we declare a cookbook dependency without also declaring a source for
|
68
|
+
that cookbook dependency, Librarian-Chef assumes we want it to look for that
|
69
|
+
cookbook in the default source.
|
70
|
+
|
71
|
+
Any time we declare a cookbook dependency that has subsidiary cookbook
|
72
|
+
dependencies of its own, Librarian-Chef assumes we want it to look for the
|
73
|
+
subsidiary cookbook dependencies in the default source.
|
74
|
+
|
75
|
+
cookbook "ntp"
|
76
|
+
|
77
|
+
Our infrastructure repository depends on the `ntp` cookbook from the default
|
78
|
+
source. Any version of the `ntp` cookbook will fulfill our requirements.
|
79
|
+
|
80
|
+
cookbook "timezone", "0.0.1"
|
81
|
+
|
82
|
+
Our infrastructure repository depends on the `timezone` cookbook from the
|
83
|
+
default source. But only version `0.0.1` of that cookbook will do.
|
84
|
+
|
85
|
+
cookbook "rvm",
|
86
|
+
:git => "https://github.com/fnichol/chef-rvm",
|
87
|
+
:ref => "v0.7.1"
|
88
|
+
|
89
|
+
Our infrastructure repository depends on the `rvm` cookbook, but not the one
|
90
|
+
from the default source. Instead, the cookbook is to be fetched from the
|
91
|
+
specified Git repository and from the specified Git tag only.
|
92
|
+
|
93
|
+
When using a Git source, we do not have to use a `:ref =>`. If we do not,
|
94
|
+
then Librarian-Chef will assume we meant the `master` branch. (In the future,
|
95
|
+
this will be changed to whatever branch is the default branch according to
|
96
|
+
the Git remote, which may not be `master`.)
|
97
|
+
|
98
|
+
If we use a `:ref =>`, we can use anything that Git will recognize as a ref.
|
99
|
+
This includes any branch name, tag name, SHA, or SHA unique prefix. If we use a
|
100
|
+
branch, we can later ask Librarian-Chef to update the cookbook by fetching the
|
101
|
+
most recent version of the cookbook from that same branch.
|
102
|
+
|
103
|
+
The Git source also supports a `:path =>` option. If we use the path option,
|
104
|
+
Librarian-Chef will navigate down into the Git repository and only use the
|
105
|
+
specified subdirectory. Many people have the habit of having a single repository
|
106
|
+
with many cookbooks in it. If we need a cookbook from such a repository, we can
|
107
|
+
use the `:path =>` option here to help Librarian-Chef drill down and find the
|
108
|
+
cookbook subdirectory.
|
109
|
+
|
110
|
+
cookbook "cloudera",
|
111
|
+
:path => "vendor/cookbooks/cloudera-cookbook"
|
112
|
+
|
113
|
+
Our infrastructure repository depends on the `cloudera` cookbook, which we have
|
114
|
+
downloaded and copied into our repository. In this example, `vendor/cookbooks/`
|
115
|
+
is only for use with Librarian-Chef. This directory should not appear in the
|
116
|
+
`.chef/knife.rb`. Librarian-Chef will, instead, copy this cookbook from where
|
117
|
+
we vendored it in our repository into the `cookbooks/` directory for us.
|
118
|
+
|
119
|
+
The `:path =>` source won't be confused with the `:git =>` source's `:path =>`
|
120
|
+
option.
|
121
|
+
|
122
|
+
Also, there is shortcut for cookbooks hosted on GitHub, so we may write:
|
123
|
+
|
124
|
+
cookbook "rvm",
|
125
|
+
:github => "fnichol/chef-rvm"
|
126
|
+
|
127
|
+
### How to Use
|
128
|
+
|
129
|
+
Install Librarian-Chef:
|
130
|
+
|
131
|
+
$ gem install librarian-chef
|
132
|
+
|
133
|
+
Prepare your infrastructure repository:
|
134
|
+
|
135
|
+
$ cd ~/path/to/chef-repo
|
136
|
+
$ git rm -r cookbooks
|
137
|
+
$ echo /cookbooks >> .gitignore
|
138
|
+
$ echo /tmp >> .gitignore
|
139
|
+
|
140
|
+
Librarian-Chef takes over your `cookbooks/` directory, and will always reinstall
|
141
|
+
the cookbooks listed the `Cheffile.lock` into your `cookbooks/` directory. Hence
|
142
|
+
you do not need your `cookbooks/` directory to be tracked in Git. If you
|
143
|
+
nevertheless want your `cookbooks/` directory to be tracked in Git, simple don't
|
144
|
+
`.gitignore` the directory.
|
145
|
+
|
146
|
+
If you are manually tracking/vendoring outside cookbooks within the repository,
|
147
|
+
put them in another directory such as `vendor/cookbooks/` and use the `:path =>`
|
148
|
+
source when declaring these cookbooks in your `Cheffile`. Most people will
|
149
|
+
typically not be manually tracking/vendoring outside cookbooks.
|
150
|
+
|
151
|
+
Librarian-Chef uses your `tmp/` directory for tempfiles and caches. You do not
|
152
|
+
need to track this directory in Git.
|
153
|
+
|
154
|
+
Make a Cheffile:
|
155
|
+
|
156
|
+
$ librarian-chef init
|
157
|
+
|
158
|
+
This creates an empty `Cheffile` with the Opscode Community Site API as the
|
159
|
+
default source.
|
160
|
+
|
161
|
+
Add dependencies and their sources to the `Cheffile`:
|
162
|
+
|
163
|
+
$ cat Cheffile
|
164
|
+
site 'http://community.opscode.com/api/v1'
|
165
|
+
cookbook 'ntp'
|
166
|
+
cookbook 'timezone', '0.0.1'
|
167
|
+
cookbook 'rvm',
|
168
|
+
:git => 'https://github.com/fnichol/chef-rvm',
|
169
|
+
:ref => 'v0.7.1'
|
170
|
+
cookbook 'cloudera',
|
171
|
+
:path => 'vendor/cookbooks/cloudera-cookbook'
|
172
|
+
|
173
|
+
This is the same `Cheffile` we saw above.
|
174
|
+
|
175
|
+
$ librarian-chef install [--clean] [--verbose]
|
176
|
+
|
177
|
+
This command looks at each `cookbook` declaration and fetches the cookbook from
|
178
|
+
the source specified, or from the default source if none is provided.
|
179
|
+
|
180
|
+
Each cookbook is inspected, its dependencies are determined, and each dependency
|
181
|
+
is also fetched. For example, if you declare `cookbook 'nagios'`, which
|
182
|
+
depends on other cookbooks such as `'php'`, then those other cookbooks
|
183
|
+
including `'php'` will be fetched. This goes all the way down the chain of
|
184
|
+
dependencies.
|
185
|
+
|
186
|
+
This command writes the complete resolution into `Cheffile.lock`.
|
187
|
+
|
188
|
+
This command then copies all of the fetched cookbooks into your `cookbooks/`
|
189
|
+
directory, overwriting whatever was there before. You can then use `knife
|
190
|
+
cookbook upload -all` to upload the cookbooks to your chef-server, if you are
|
191
|
+
using the client-server model.
|
192
|
+
|
193
|
+
Check your `Cheffile` and `Cheffile.lock` into version control:
|
194
|
+
|
195
|
+
$ git add Cheffile
|
196
|
+
$ git add Cheffile.lock
|
197
|
+
$ git commit -m "I want these particular versions of these particular cookbooks from these particular."
|
198
|
+
|
199
|
+
Make sure you check your `Cheffile.lock` into version control. This will ensure
|
200
|
+
dependencies do not need to be resolved every run, greatly reducing dependency
|
201
|
+
resolution time.
|
202
|
+
|
203
|
+
Get an overview of your `Cheffile.lock` with:
|
204
|
+
|
205
|
+
$ librarian-chef show
|
206
|
+
|
207
|
+
Inspect the details of specific resolved dependencies with:
|
208
|
+
|
209
|
+
$ librarian-chef show NAME1 [NAME2, ...]
|
210
|
+
|
211
|
+
Update your `Cheffile` with new/changed/removed constraints/sources/dependencies:
|
212
|
+
|
213
|
+
$ cat Cheffile
|
214
|
+
site 'http://community.opscode.com/api/v1'
|
215
|
+
cookbook 'ntp'
|
216
|
+
cookbook 'timezone', '0.0.1'
|
217
|
+
cookbook 'rvm',
|
218
|
+
:git => 'https://github.com/fnichol/chef-rvm',
|
219
|
+
:ref => 'v0.7.1'
|
220
|
+
cookbook 'monit' # new!
|
221
|
+
$ git diff Cheffile
|
222
|
+
$ librarian-chef install [--verbose]
|
223
|
+
$ git diff Cheffile.lock
|
224
|
+
$ git add Cheffile
|
225
|
+
$ git add Cheffile.lock
|
226
|
+
$ git commit -m "I also want these additional cookbooks."
|
227
|
+
|
228
|
+
Find out which dependencies are outdated and may be updated:
|
229
|
+
|
230
|
+
$ librarian-chef outdated [--verbose]
|
231
|
+
|
232
|
+
Update the version of a dependency:
|
233
|
+
|
234
|
+
$ librarian-chef update ntp timezone monit [--verbose]
|
235
|
+
$ git diff Cheffile.lock
|
236
|
+
$ git add Cheffile.lock
|
237
|
+
$ git commit -m "I want updated versions of these cookbooks."
|
238
|
+
|
239
|
+
Push your changes to the git repository:
|
240
|
+
|
241
|
+
$ git push origin master
|
242
|
+
|
243
|
+
Upload the cookbooks to your chef-server:
|
244
|
+
|
245
|
+
$ knife cookbook upload --all
|
246
|
+
|
247
|
+
### Configuration
|
248
|
+
|
249
|
+
Configuration comes from three sources with the following highest-to-lowest
|
250
|
+
precedence:
|
251
|
+
|
252
|
+
* The local config (`./.librarian/chef/config`)
|
253
|
+
* The environment
|
254
|
+
* The global config (`~/.librarian/chef/config`)
|
255
|
+
|
256
|
+
You can inspect the final configuration with:
|
257
|
+
|
258
|
+
$ librarian-chef config
|
259
|
+
|
260
|
+
You can find out where a particular key is set with:
|
261
|
+
|
262
|
+
$ librarian-chef config KEY
|
263
|
+
|
264
|
+
You can set a key at the global level with:
|
265
|
+
|
266
|
+
$ librarian-chef config KEY VALUE --global
|
267
|
+
|
268
|
+
And remove it with:
|
269
|
+
|
270
|
+
$ librarian-chef config KEY --global --delete
|
271
|
+
|
272
|
+
You can set a key at the local level with:
|
273
|
+
|
274
|
+
$ librarian-chef config KEY VALUE --local
|
275
|
+
|
276
|
+
And remove it with:
|
277
|
+
|
278
|
+
$ librarian-chef config KEY --local --delete
|
279
|
+
|
280
|
+
You cannot set or delete environment-level config keys with the CLI.
|
281
|
+
|
282
|
+
Configuration set at either the global or local level will affect subsequent
|
283
|
+
invocations of `librarian-chef`. Configurations set at the environment level are
|
284
|
+
not saved and will not affect subsequent invocations of `librarian-chef`.
|
285
|
+
|
286
|
+
You can pass a config at the environment level by taking the original config key
|
287
|
+
and transforming it: replace hyphens (`-`) with underscores (`_`) and periods
|
288
|
+
(`.`) with doubled underscores (`__`), uppercase, and finally prefix with
|
289
|
+
`LIBRARIAN_CHEF_`. For example, to pass a config in the environment for the key
|
290
|
+
`part-one.part-two`, set the environment variable
|
291
|
+
`LIBRARIAN_CHEF_PART_ONE__PART_TWO`.
|
292
|
+
|
293
|
+
Configuration affects how various commands operate.
|
294
|
+
|
295
|
+
* The `path` config sets the cookbooks directory to install to. If a relative
|
296
|
+
path, it is relative to the directory containing the `Cheffile`. The
|
297
|
+
equivalent environment variable is `LIBRARIAN_CHEF_PATH`.
|
298
|
+
|
299
|
+
* The `tmp` config sets the cache directory for librarian. If a relative
|
300
|
+
path, it is relative to the directory containing the `Cheffile`. The
|
301
|
+
equivalent environment variable is `LIBRARIAN_CHEF_TMP`.
|
302
|
+
|
303
|
+
* The `install.strip-dot-git` config causes the `.git/` directory to be stripped
|
304
|
+
out when installing cookbooks from a git source. This must be set to exactly
|
305
|
+
"1" to cause this behavior. The equivalent environment variable is
|
306
|
+
`LIBRARIAN_CHEF_INSTALL__STRIP_DOT_GIT`.
|
307
|
+
|
308
|
+
Configuration can be set by passing specific options to other commands.
|
309
|
+
|
310
|
+
* The `path` config can be set at the local level by passing the `--path` option
|
311
|
+
to the `install` command. It can be unset at the local level by passing the
|
312
|
+
`--no-path` option to the `install` command. Note that if this is set at the
|
313
|
+
environment or global level then, even if `--no-path` is given as an option,
|
314
|
+
the environment or global config will be used.
|
315
|
+
|
316
|
+
* The `install.strip-dot-git` config can be set at the local level by passing
|
317
|
+
the `--strip-dot-git` option to the `install` command. It can be unset at the
|
318
|
+
local level by passing the `--no-strip-dot-git` option.
|
319
|
+
|
320
|
+
### Knife Integration
|
321
|
+
|
322
|
+
You can integrate your `knife.rb` with Librarian-Chef.
|
323
|
+
|
324
|
+
Stick the following in your `knife.rb`:
|
325
|
+
|
326
|
+
require 'librarian/chef/integration/knife'
|
327
|
+
cookbook_path Librarian::Chef.install_path,
|
328
|
+
"/path/to/chef-repo/site-cookbooks"
|
329
|
+
|
330
|
+
In the above, do *not* to include the path to your `cookbooks/` directory. If
|
331
|
+
you have additional cookbooks directories in your chef-repo that you use for
|
332
|
+
vendored cookbooks (where you use the `:path =>` source in your `Cheffile`),
|
333
|
+
make sure *not* to include the paths to those additional cookbooks directories
|
334
|
+
either.
|
335
|
+
|
336
|
+
You still need to include your `site-cookbooks/` directory in the above list.
|
337
|
+
|
338
|
+
What this integration does is whenever you use any `knife` command, it will:
|
339
|
+
|
340
|
+
* Enforce that your `Cheffile` and `Cheffile.lock` are in sync
|
341
|
+
* Install the resolved cookbooks to a temporary directory
|
342
|
+
* Configure Knife to look in the temporary directory for the installed cookbooks
|
343
|
+
and not in the normal `cookbooks/` directory.
|
344
|
+
|
345
|
+
When you use this integration, any changes you make to anything in the
|
346
|
+
`cookbooks/` directory will be ignored by Knife, because Knife won't look in
|
347
|
+
that directory for your cookbooks.
|
348
|
+
|
349
|
+
How to Contribute
|
350
|
+
-----------------
|
351
|
+
|
352
|
+
### Running the tests
|
353
|
+
|
354
|
+
Ensure the gem dependencies are installed:
|
355
|
+
|
356
|
+
$ bundle
|
357
|
+
|
358
|
+
Run the tests
|
359
|
+
|
360
|
+
$ [bundle exec] rspec spec
|
361
|
+
|
362
|
+
### Installing locally
|
363
|
+
|
364
|
+
Ensure the gem dependencies are installed:
|
365
|
+
|
366
|
+
$ bundle
|
367
|
+
|
368
|
+
Install from the repository:
|
369
|
+
|
370
|
+
$ [bundle exec] rake install
|
371
|
+
|
372
|
+
### Reporting Issues
|
373
|
+
|
374
|
+
Please include relevant `Cheffile` and `Cheffile.lock` files. Please run the
|
375
|
+
`librarian-chef` commands in verbose mode by using the `--verbose` flag, and
|
376
|
+
include the verbose output in the bug report as well.
|
377
|
+
|
378
|
+
License
|
379
|
+
-------
|
380
|
+
|
381
|
+
Written by Jay Feldblum.
|
382
|
+
|
383
|
+
Copyright (c) 2013 ApplicationsOnline, LLC.
|
384
|
+
|
385
|
+
Released under the terms of the MIT License. For further information, please see
|
386
|
+
the file `LICENSE.txt`.
|
data/Rakefile
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "bundler/gem_tasks"
|