librarian 0.0.20 → 0.0.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.travis.yml +1 -0
- data/CHANGELOG.md +16 -0
- data/README.md +25 -18
- data/features/chef/cli/show.feature +65 -0
- data/features/support/env.rb +5 -1
- data/lib/librarian/action/clean.rb +0 -12
- data/lib/librarian/action/install.rb +1 -1
- data/lib/librarian/chef/manifest_reader.rb +47 -0
- data/lib/librarian/chef/source/local.rb +51 -3
- data/lib/librarian/chef/source/site.rb +318 -109
- data/lib/librarian/cli.rb +24 -9
- data/lib/librarian/cli/manifest_presenter.rb +79 -0
- data/lib/librarian/dependency.rb +2 -2
- data/lib/librarian/dsl/target.rb +1 -1
- data/lib/librarian/environment.rb +4 -3
- data/lib/librarian/manifest.rb +38 -24
- data/lib/librarian/manifest_set.rb +36 -20
- data/lib/librarian/mock/source/mock.rb +33 -21
- data/lib/librarian/resolution.rb +11 -0
- data/lib/librarian/source/git.rb +14 -2
- data/lib/librarian/source/git/repository.rb +79 -58
- data/lib/librarian/source/local.rb +19 -5
- data/lib/librarian/source/path.rb +13 -1
- data/lib/librarian/specfile.rb +1 -1
- data/lib/librarian/version.rb +1 -1
- data/librarian.gemspec +1 -1
- data/spec/unit/action/clean_spec.rb +0 -31
- data/spec/unit/action/install_spec.rb +7 -3
- data/spec/unit/dsl_spec.rb +14 -0
- data/spec/unit/manifest_set_spec.rb +202 -0
- data/spec/unit/resolver_spec.rb +36 -16
- data/spec/unit/source/git_spec.rb +29 -0
- metadata +28 -25
- data/lib/librarian/chef/manifest.rb +0 -43
- data/lib/librarian/chef/source/local/manifest.rb +0 -82
- data/lib/librarian/chef/source/site/manifest.rb +0 -94
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,21 @@
|
|
1
1
|
# Change Log
|
2
2
|
|
3
|
+
## 0.0.21
|
4
|
+
|
5
|
+
* \#64. Sources now raise when given unrecognized options in the specfile.
|
6
|
+
|
7
|
+
* A new `show` CLI command.
|
8
|
+
|
9
|
+
* Changed the `clean` CLI command no longer to delete the lockfile.
|
10
|
+
|
11
|
+
* Simplify the architecture of `Librarian::Manifest` vis-a-vis sources. It is no
|
12
|
+
longer a base class for adapters to inherit. Now, sources expose a small
|
13
|
+
interface, which `Librarian::Manifest` can call, for delay-loading attributes.
|
14
|
+
|
15
|
+
* The git source now resolves the `git` binary before `chdir`ing.
|
16
|
+
|
17
|
+
* Test on Rubinius.
|
18
|
+
|
3
19
|
## 0.0.20
|
4
20
|
|
5
21
|
* A command to print outdated dependencies.
|
data/README.md
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
Librarian
|
1
|
+
Librarian [](http://travis-ci.org/applicationsonline/librarian)
|
2
2
|
=========
|
3
3
|
|
4
4
|
Librarian is a framework for writing bundlers, which are tools that resolve,
|
@@ -19,7 +19,7 @@ Librarian-Chef
|
|
19
19
|
---------------
|
20
20
|
|
21
21
|
Librarian-Chef is a bundler for infrastructure repositories using Chef. You can
|
22
|
-
use Librarian-Chef to resolve your infrastructure's cookbook dependencies
|
22
|
+
use Librarian-Chef to resolve your infrastructure's cookbook dependencies,
|
23
23
|
fetch them and install them into your infrastructure.
|
24
24
|
|
25
25
|
Librarian-Chef is for resolving and fetching third-party, publicly-released
|
@@ -29,15 +29,15 @@ infrastructure repository.
|
|
29
29
|
|
30
30
|
Librarian-Chef *takes over* your `cookbooks/` directory and manages it for you
|
31
31
|
based on your `Cheffile`. Your `Cheffile` becomes the authoritative source for
|
32
|
-
|
32
|
+
the cookbooks your infrastructure repository depends on. You should not modify
|
33
33
|
the contents of your `cookbooks/` directory when using Librarian-Chef. If you
|
34
|
-
have custom cookbooks specific to your infrastructure repository
|
35
|
-
|
34
|
+
have custom cookbooks which are specific to your infrastructure repository,
|
35
|
+
they should go in your `site-cookbooks/` directory.
|
36
36
|
|
37
37
|
### The Cheffile
|
38
38
|
|
39
39
|
Every infrastruture repository that uses Librarian-Chef will have a file named
|
40
|
-
`Cheffile` in the root directory of
|
40
|
+
`Cheffile` in the root directory of that repository. The full specification for
|
41
41
|
which third-party, publicly-rleased cookbooks your infrastructure repository
|
42
42
|
depends will go here.
|
43
43
|
|
@@ -91,10 +91,10 @@ Our infrastructure repository depends on the `rvm` cookbook, but not the one
|
|
91
91
|
from the default source. Instead, the cookbook is to be fetched from the
|
92
92
|
specified Git repository and from the specified Git tag only.
|
93
93
|
|
94
|
-
|
95
|
-
we meant the
|
96
|
-
branch is the default branch according to
|
97
|
-
`master`.)
|
94
|
+
When using a Git source, we do not have to use a `:ref =>`. If we do not,
|
95
|
+
then Librarian-Chef will assume we meant the `master` branch. (In the future,
|
96
|
+
this will be changed to whatever branch is the default branch according to
|
97
|
+
the Git remote, which may not be `master`.)
|
98
98
|
|
99
99
|
If we use a `:ref =>`, we can use anything that Git will recognize as a ref.
|
100
100
|
This includes any branch name, tag name, SHA, or SHA unique prefix. If we use a
|
@@ -122,7 +122,7 @@ option.
|
|
122
122
|
|
123
123
|
### How to Use
|
124
124
|
|
125
|
-
Install
|
125
|
+
Install Librarian-Chef:
|
126
126
|
|
127
127
|
$ gem install librarian
|
128
128
|
|
@@ -171,12 +171,11 @@ This is the same `Cheffile` we saw above.
|
|
171
171
|
$ librarian-chef install [--clean] [--verbose]
|
172
172
|
|
173
173
|
This command looks at each `cookbook` declaration and fetches the cookbook from
|
174
|
-
the source specified
|
175
|
-
provided for that cookbook.
|
174
|
+
the source specified, or from the default source if none is provided.
|
176
175
|
|
177
|
-
Each cookbook is inspected
|
178
|
-
is also fetched. For example, if you declare `cookbook 'nagios'`,
|
179
|
-
|
176
|
+
Each cookbook is inspected, its dependencies are determined, and each dependency
|
177
|
+
is also fetched. For example, if you declare `cookbook 'nagios'`, which
|
178
|
+
depends on other cookbooks such as `'php'`, then those other cookbooks
|
180
179
|
including `'php'` will be fetched. This goes all the way down the chain of
|
181
180
|
dependencies.
|
182
181
|
|
@@ -193,11 +192,19 @@ Check your `Cheffile` and `Cheffile.lock` into version control:
|
|
193
192
|
$ git add Cheffile.lock
|
194
193
|
$ git commit -m "I want these particular versions of these particular cookbooks from these particular."
|
195
194
|
|
196
|
-
Make sure you check your Cheffile.lock into version control. This will ensure
|
195
|
+
Make sure you check your `Cheffile.lock` into version control. This will ensure
|
197
196
|
dependencies do not need to be resolved every run, greatly reducing dependency
|
198
197
|
resolution time.
|
199
198
|
|
200
|
-
|
199
|
+
Get an overview of your `Cheffile.lock` with:
|
200
|
+
|
201
|
+
$ librarian-chef show
|
202
|
+
|
203
|
+
Inspect the details of specific resolved dependencies with:
|
204
|
+
|
205
|
+
$ librarian-chef show NAME1 [NAME2, ...]
|
206
|
+
|
207
|
+
Update your `Cheffile` with new/changed/removed constraints/sources/dependencies:
|
201
208
|
|
202
209
|
$ cat Cheffile
|
203
210
|
site 'http://community.opscode.com/api/v1'
|
@@ -0,0 +1,65 @@
|
|
1
|
+
Feature: cli/show
|
2
|
+
|
3
|
+
|
4
|
+
|
5
|
+
Background: A simple Cheffile with one cookbook with one dependency
|
6
|
+
Given a file named "cookbook-sources/main/metadata.yaml" with:
|
7
|
+
"""
|
8
|
+
name: main
|
9
|
+
version: 1.0.0
|
10
|
+
dependencies:
|
11
|
+
sub: 1.0.0
|
12
|
+
"""
|
13
|
+
Given a file named "cookbook-sources/sub/metadata.yaml" with:
|
14
|
+
"""
|
15
|
+
name: sub
|
16
|
+
version: 1.0.0
|
17
|
+
dependencies: {}
|
18
|
+
"""
|
19
|
+
Given a file named "Cheffile" with:
|
20
|
+
"""
|
21
|
+
path 'cookbook-sources'
|
22
|
+
cookbook 'main'
|
23
|
+
"""
|
24
|
+
Given I run `librarian-chef install`
|
25
|
+
|
26
|
+
|
27
|
+
|
28
|
+
Scenario: Showing all
|
29
|
+
When I run `librarian-chef show`
|
30
|
+
Then the exit status should be 0
|
31
|
+
Then the output should contain exactly:
|
32
|
+
"""
|
33
|
+
main (1.0.0)
|
34
|
+
sub (1.0.0)
|
35
|
+
|
36
|
+
"""
|
37
|
+
|
38
|
+
|
39
|
+
|
40
|
+
Scenario: Showing one without dependencies
|
41
|
+
When I run `librarian-chef show sub`
|
42
|
+
Then the exit status should be 0
|
43
|
+
Then the output should contain exactly:
|
44
|
+
"""
|
45
|
+
sub (1.0.0)
|
46
|
+
source: cookbook-sources
|
47
|
+
|
48
|
+
"""
|
49
|
+
|
50
|
+
|
51
|
+
|
52
|
+
Scenario: Showing one with dependencies
|
53
|
+
When I run `librarian-chef show main`
|
54
|
+
Then the exit status should be 0
|
55
|
+
Then the output should contain exactly:
|
56
|
+
"""
|
57
|
+
main (1.0.0)
|
58
|
+
source: cookbook-sources
|
59
|
+
dependencies:
|
60
|
+
sub (= 1.0.0)
|
61
|
+
|
62
|
+
"""
|
63
|
+
|
64
|
+
|
65
|
+
|
data/features/support/env.rb
CHANGED
@@ -7,7 +7,6 @@ module Librarian
|
|
7
7
|
def run
|
8
8
|
clean_cache_path
|
9
9
|
clean_install_path
|
10
|
-
clean_lockfile_path
|
11
10
|
end
|
12
11
|
|
13
12
|
private
|
@@ -28,13 +27,6 @@ module Librarian
|
|
28
27
|
end
|
29
28
|
end
|
30
29
|
|
31
|
-
def clean_lockfile_path
|
32
|
-
if lockfile_path.exist?
|
33
|
-
debug { "Deleting #{project_relative_path_to(lockfile_path)}" }
|
34
|
-
lockfile_path.rmtree
|
35
|
-
end
|
36
|
-
end
|
37
|
-
|
38
30
|
def cache_path
|
39
31
|
environment.cache_path
|
40
32
|
end
|
@@ -43,10 +35,6 @@ module Librarian
|
|
43
35
|
environment.install_path
|
44
36
|
end
|
45
37
|
|
46
|
-
def lockfile_path
|
47
|
-
environment.lockfile_path
|
48
|
-
end
|
49
|
-
|
50
38
|
def project_relative_path_to(path)
|
51
39
|
environment.project_relative_path_to(path)
|
52
40
|
end
|
@@ -0,0 +1,47 @@
|
|
1
|
+
require 'json'
|
2
|
+
require 'yaml'
|
3
|
+
|
4
|
+
require 'librarian/manifest'
|
5
|
+
|
6
|
+
module Librarian
|
7
|
+
module Chef
|
8
|
+
module ManifestReader
|
9
|
+
extend self
|
10
|
+
|
11
|
+
MANIFESTS = %w(metadata.json metadata.yml metadata.yaml metadata.rb)
|
12
|
+
|
13
|
+
def manifest_path(path)
|
14
|
+
MANIFESTS.map{|s| path.join(s)}.find{|s| s.exist?}
|
15
|
+
end
|
16
|
+
|
17
|
+
def read_manifest(name, manifest_path)
|
18
|
+
case manifest_path.extname
|
19
|
+
when ".json" then JSON.parse(manifest_path.read)
|
20
|
+
when ".yml", ".yaml" then YAML.load(manifest_path.read)
|
21
|
+
when ".rb" then compile_manifest(name, manifest_path.dirname)
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
def compile_manifest(name, path)
|
26
|
+
# Inefficient, if there are many cookbooks with uncompiled metadata.
|
27
|
+
require 'chef/json_compat'
|
28
|
+
require 'chef/cookbook/metadata'
|
29
|
+
md = ::Chef::Cookbook::Metadata.new
|
30
|
+
md.name(name)
|
31
|
+
md.from_file(path.join('metadata.rb').to_s)
|
32
|
+
{"name" => md.name, "version" => md.version, "dependencies" => md.dependencies}
|
33
|
+
end
|
34
|
+
|
35
|
+
def manifest?(name, path)
|
36
|
+
path = Pathname.new(path)
|
37
|
+
!!manifest_path(path)
|
38
|
+
end
|
39
|
+
|
40
|
+
def check_manifest(name, manifest_path)
|
41
|
+
manifest = read_manifest(name, manifest_path)
|
42
|
+
manifest["name"] == name
|
43
|
+
end
|
44
|
+
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
@@ -1,12 +1,60 @@
|
|
1
|
-
require 'librarian/chef/
|
1
|
+
require 'librarian/chef/manifest_reader'
|
2
2
|
|
3
3
|
module Librarian
|
4
4
|
module Chef
|
5
5
|
module Source
|
6
6
|
module Local
|
7
7
|
|
8
|
-
def
|
9
|
-
|
8
|
+
def install!(manifest)
|
9
|
+
manifest.source == self or raise ArgumentError
|
10
|
+
|
11
|
+
debug { "Installing #{manifest}" }
|
12
|
+
|
13
|
+
name, version = manifest.name, manifest.version
|
14
|
+
found_path = found_path(name)
|
15
|
+
|
16
|
+
install_path = environment.install_path.join(name)
|
17
|
+
if install_path.exist?
|
18
|
+
debug { "Deleting #{relative_path_to(install_path)}" }
|
19
|
+
install_path.rmtree
|
20
|
+
end
|
21
|
+
|
22
|
+
debug { "Copying #{relative_path_to(found_path)} to #{relative_path_to(install_path)}" }
|
23
|
+
FileUtils.cp_r(found_path, install_path)
|
24
|
+
end
|
25
|
+
|
26
|
+
def fetch_version(name, extra)
|
27
|
+
manifest_data(name)["version"]
|
28
|
+
end
|
29
|
+
|
30
|
+
def fetch_dependencies(name, version, extra)
|
31
|
+
manifest_data(name)["dependencies"]
|
32
|
+
end
|
33
|
+
|
34
|
+
private
|
35
|
+
|
36
|
+
def manifest_data(name)
|
37
|
+
@manifest_data ||= { }
|
38
|
+
@manifest_data[name] ||= fetch_manifest_data(name)
|
39
|
+
end
|
40
|
+
|
41
|
+
def fetch_manifest_data(name)
|
42
|
+
expect_manifest!(name)
|
43
|
+
|
44
|
+
found_path = found_path(name)
|
45
|
+
manifest_path = ManifestReader.manifest_path(found_path)
|
46
|
+
ManifestReader.read_manifest(name, manifest_path)
|
47
|
+
end
|
48
|
+
|
49
|
+
def manifest?(name, path)
|
50
|
+
ManifestReader.manifest?(name, path)
|
51
|
+
end
|
52
|
+
|
53
|
+
def expect_manifest!(name)
|
54
|
+
found_path = found_path(name)
|
55
|
+
return if found_path && ManifestReader.manifest_path(found_path)
|
56
|
+
|
57
|
+
raise Error, "No metadata file found for #{name} from #{self}! If this should be a cookbook, you might consider contributing a metadata file upstream or forking the cookbook to add your own metadata file."
|
10
58
|
end
|
11
59
|
|
12
60
|
end
|
@@ -5,11 +5,12 @@ require 'net/http'
|
|
5
5
|
require 'json'
|
6
6
|
require 'digest'
|
7
7
|
require 'zlib'
|
8
|
+
require 'tmpdir'
|
8
9
|
require 'archive/tar/minitar'
|
9
10
|
|
10
11
|
require 'librarian/helpers/debug'
|
11
12
|
|
12
|
-
require 'librarian/chef/
|
13
|
+
require 'librarian/chef/manifest_reader'
|
13
14
|
|
14
15
|
module Librarian
|
15
16
|
module Chef
|
@@ -18,14 +19,304 @@ module Librarian
|
|
18
19
|
|
19
20
|
include Helpers::Debug
|
20
21
|
|
22
|
+
class Line
|
23
|
+
|
24
|
+
include Helpers::Debug
|
25
|
+
|
26
|
+
attr_accessor :source, :name
|
27
|
+
private :source=, :name=
|
28
|
+
|
29
|
+
def initialize(source, name)
|
30
|
+
self.source = source
|
31
|
+
self.name = name
|
32
|
+
end
|
33
|
+
|
34
|
+
def install_version!(version, install_path)
|
35
|
+
cache_version_unpacked! version
|
36
|
+
|
37
|
+
if install_path.exist?
|
38
|
+
debug { "Deleting #{relative_path_to(install_path)}" }
|
39
|
+
install_path.rmtree
|
40
|
+
end
|
41
|
+
|
42
|
+
unpacked_path = version_unpacked_cache_path(version)
|
43
|
+
|
44
|
+
debug { "Copying #{relative_path_to(unpacked_path)} to #{relative_path_to(install_path)}" }
|
45
|
+
FileUtils.cp_r(unpacked_path, install_path)
|
46
|
+
end
|
47
|
+
|
48
|
+
def manifests
|
49
|
+
version_uris.map do |version_uri|
|
50
|
+
Manifest.new(source, name, version_uri)
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
def to_version(version_uri)
|
55
|
+
version_uri_metadata(version_uri)["version"]
|
56
|
+
end
|
57
|
+
|
58
|
+
def version_dependencies(version)
|
59
|
+
version_manifest(version)["dependencies"]
|
60
|
+
end
|
61
|
+
|
62
|
+
private
|
63
|
+
|
64
|
+
attr_accessor :metadata_cached
|
65
|
+
alias metadata_cached? metadata_cached
|
66
|
+
|
67
|
+
def environment
|
68
|
+
source.environment
|
69
|
+
end
|
70
|
+
|
71
|
+
def uri
|
72
|
+
@uri ||= URI.parse("#{source.uri}/cookbooks/#{name}")
|
73
|
+
end
|
74
|
+
|
75
|
+
def version_uris
|
76
|
+
metadata["versions"]
|
77
|
+
end
|
78
|
+
|
79
|
+
def version_metadata(version)
|
80
|
+
version_uri = to_version_uri(version)
|
81
|
+
version_uri_metadata(version_uri)
|
82
|
+
end
|
83
|
+
|
84
|
+
def version_uri_metadata(version_uri)
|
85
|
+
@version_uri_metadata ||= { }
|
86
|
+
@version_uri_metadata[version_uri.to_s] ||= begin
|
87
|
+
cache_version_uri_metadata! version_uri
|
88
|
+
parse_local_json(version_uri_metadata_cache_path(version_uri))
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
def version_manifest(version)
|
93
|
+
version_uri = to_version_uri(version)
|
94
|
+
version_uri_manifest(version_uri)
|
95
|
+
end
|
96
|
+
|
97
|
+
def version_uri_manifest(version_uri)
|
98
|
+
@version_uri_manifest ||= { }
|
99
|
+
@version_uri_manifest[version_uri.to_s] ||= begin
|
100
|
+
cache_version_uri_unpacked! version_uri
|
101
|
+
unpacked_path = version_uri_unpacked_cache_path(version_uri)
|
102
|
+
manifest_path = ManifestReader.manifest_path(unpacked_path)
|
103
|
+
ManifestReader.read_manifest(name, manifest_path)
|
104
|
+
end
|
105
|
+
end
|
106
|
+
|
107
|
+
def metadata
|
108
|
+
@metadata ||= begin
|
109
|
+
cache_metadata!
|
110
|
+
parse_local_json(metadata_cache_path)
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
114
|
+
def to_version_uri(version)
|
115
|
+
@to_version_uri ||= { }
|
116
|
+
@to_version_uri[version.to_s] ||= begin
|
117
|
+
cache_version! version
|
118
|
+
version_cache_path(version).read
|
119
|
+
end
|
120
|
+
end
|
121
|
+
|
122
|
+
def metadata_cached!
|
123
|
+
self.metadata_cached = true
|
124
|
+
end
|
125
|
+
|
126
|
+
def cache_path
|
127
|
+
@cache_path ||= source.cache_path.join(name)
|
128
|
+
end
|
129
|
+
|
130
|
+
def metadata_cache_path
|
131
|
+
@metadata_cache_path ||= cache_path.join("metadata.json")
|
132
|
+
end
|
133
|
+
|
134
|
+
def version_cache_path(version)
|
135
|
+
@version_cache_path ||= { }
|
136
|
+
@version_cache_path[version.to_s] ||= begin
|
137
|
+
cache_path.join("version").join(version.to_s)
|
138
|
+
end
|
139
|
+
end
|
140
|
+
|
141
|
+
def version_uri_cache_path(version_uri)
|
142
|
+
@version_uri_cache_path ||= { }
|
143
|
+
@version_uri_cache_path[version_uri.to_s] ||= begin
|
144
|
+
cache_path.join("version-uri").join(hexdigest(version_uri))
|
145
|
+
end
|
146
|
+
end
|
147
|
+
|
148
|
+
def version_metadata_cache_path(version)
|
149
|
+
version_uri = to_version_uri(version)
|
150
|
+
version_uri_metadata_cache_path(version_uri)
|
151
|
+
end
|
152
|
+
|
153
|
+
def version_uri_metadata_cache_path(version_uri)
|
154
|
+
@version_uri_metadata_cache_path ||= { }
|
155
|
+
@version_uri_metadata_cache_path[version_uri.to_s] ||= begin
|
156
|
+
version_uri_cache_path(version_uri).join("metadata.json")
|
157
|
+
end
|
158
|
+
end
|
159
|
+
|
160
|
+
def version_package_cache_path(version)
|
161
|
+
version_uri = to_version_uri(version)
|
162
|
+
version_uri_package_cache_path(version_uri)
|
163
|
+
end
|
164
|
+
|
165
|
+
def version_uri_package_cache_path(version_uri)
|
166
|
+
@version_uri_package_cache_path ||= { }
|
167
|
+
@version_uri_package_cache_path[version_uri.to_s] ||= begin
|
168
|
+
version_uri_cache_path(version_uri).join("package.tar.gz")
|
169
|
+
end
|
170
|
+
end
|
171
|
+
|
172
|
+
def version_unpacked_cache_path(version)
|
173
|
+
version_uri = to_version_uri(version)
|
174
|
+
version_uri_unpacked_cache_path(version_uri)
|
175
|
+
end
|
176
|
+
|
177
|
+
def version_uri_unpacked_cache_path(version_uri)
|
178
|
+
@version_uri_unpacked_cache_path ||= { }
|
179
|
+
@version_uri_unpacked_cache_path[version_uri.to_s] ||= begin
|
180
|
+
version_uri_cache_path(version_uri).join("package")
|
181
|
+
end
|
182
|
+
end
|
183
|
+
|
184
|
+
def cache_metadata!
|
185
|
+
metadata_cached? and return or metadata_cached!
|
186
|
+
cache_remote_json! metadata_cache_path, uri
|
187
|
+
end
|
188
|
+
|
189
|
+
def cache_version_uri_metadata!(version_uri)
|
190
|
+
path = version_uri_metadata_cache_path(version_uri)
|
191
|
+
path.file? and return
|
192
|
+
|
193
|
+
cache_remote_json! path, version_uri
|
194
|
+
end
|
195
|
+
|
196
|
+
def cache_version!(version)
|
197
|
+
path = version_cache_path(version)
|
198
|
+
path.file? and return
|
199
|
+
|
200
|
+
version_uris.each do |version_uri|
|
201
|
+
m = version_uri_metadata(version_uri)
|
202
|
+
v = m["version"]
|
203
|
+
if version.to_s == v
|
204
|
+
write! path, version_uri.to_s
|
205
|
+
break
|
206
|
+
end
|
207
|
+
end
|
208
|
+
end
|
209
|
+
|
210
|
+
def cache_version_package!(version)
|
211
|
+
version_uri = to_version_uri(version)
|
212
|
+
cache_version_uri_package! version_uri
|
213
|
+
end
|
214
|
+
|
215
|
+
def cache_version_uri_package!(version_uri)
|
216
|
+
path = version_uri_package_cache_path(version_uri)
|
217
|
+
path.file? and return
|
218
|
+
|
219
|
+
file_uri = version_uri_metadata(version_uri)["file"]
|
220
|
+
cache_remote_object! path, file_uri
|
221
|
+
end
|
222
|
+
|
223
|
+
def cache_version_unpacked!(version)
|
224
|
+
version_uri = to_version_uri(version)
|
225
|
+
cache_version_uri_unpacked! version_uri
|
226
|
+
end
|
227
|
+
|
228
|
+
def cache_version_uri_unpacked!(version_uri)
|
229
|
+
cache_version_uri_package!(version_uri)
|
230
|
+
|
231
|
+
path = version_uri_unpacked_cache_path(version_uri)
|
232
|
+
path.directory? and return
|
233
|
+
|
234
|
+
package_path = version_uri_package_cache_path(version_uri)
|
235
|
+
unpacked_path = version_uri_unpacked_cache_path(version_uri)
|
236
|
+
temp_path = Pathname(Dir.tmpdir)
|
237
|
+
|
238
|
+
Zlib::GzipReader.open(package_path) do |input|
|
239
|
+
Archive::Tar::Minitar.unpack(input, temp_path.to_s)
|
240
|
+
end
|
241
|
+
FileUtils.move(temp_path.join(name), unpacked_path)
|
242
|
+
end
|
243
|
+
|
244
|
+
def cache_remote_json!(path, uri)
|
245
|
+
path = Pathname(path)
|
246
|
+
uri = to_uri(uri)
|
247
|
+
|
248
|
+
path.dirname.mkpath unless path.dirname.directory?
|
249
|
+
|
250
|
+
debug { "Caching #{uri} to #{path}" }
|
251
|
+
|
252
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
253
|
+
request = Net::HTTP::Get.new(uri.path)
|
254
|
+
response = http.start{|http| http.request(request)}
|
255
|
+
unless Net::HTTPSuccess === response
|
256
|
+
raise Error, "Could not get #{uri} because #{response.code} #{response.message}!"
|
257
|
+
end
|
258
|
+
json = response.body
|
259
|
+
JSON.parse(json) # verify that it's really JSON.
|
260
|
+
write! path, json
|
261
|
+
end
|
262
|
+
|
263
|
+
def cache_remote_object!(path, uri)
|
264
|
+
path = Pathname(path)
|
265
|
+
uri = to_uri(uri)
|
266
|
+
|
267
|
+
path.dirname.mkpath unless path.dirname.directory?
|
268
|
+
|
269
|
+
debug { "Caching #{uri} to #{path}" }
|
270
|
+
|
271
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
272
|
+
request = Net::HTTP::Get.new(uri.path)
|
273
|
+
response = http.start{|http| http.request(request)}
|
274
|
+
unless Net::HTTPSuccess === response
|
275
|
+
raise Error, "Could not get #{uri} because #{response.code} #{response.message}!"
|
276
|
+
end
|
277
|
+
write! path, response.body
|
278
|
+
end
|
279
|
+
|
280
|
+
def write!(path, bytes)
|
281
|
+
path.dirname.mkpath
|
282
|
+
path.open("wb"){|f| f.write(bytes)}
|
283
|
+
end
|
284
|
+
|
285
|
+
def parse_local_json(path)
|
286
|
+
JSON.parse(path.read)
|
287
|
+
end
|
288
|
+
|
289
|
+
def hexdigest(bytes)
|
290
|
+
Digest::MD5.hexdigest(bytes)
|
291
|
+
end
|
292
|
+
|
293
|
+
def to_uri(uri)
|
294
|
+
uri = URI(uri) unless URI === uri
|
295
|
+
uri
|
296
|
+
end
|
297
|
+
|
298
|
+
end
|
299
|
+
|
21
300
|
class << self
|
301
|
+
|
22
302
|
LOCK_NAME = 'SITE'
|
303
|
+
|
23
304
|
def lock_name
|
24
305
|
LOCK_NAME
|
25
306
|
end
|
307
|
+
|
26
308
|
def from_lock_options(environment, options)
|
27
309
|
new(environment, options[:remote], options.reject{|k, v| k == :remote})
|
28
310
|
end
|
311
|
+
|
312
|
+
def from_spec_args(environment, uri, options)
|
313
|
+
recognized_options = []
|
314
|
+
unrecognized_options = options.keys - recognized_options
|
315
|
+
unrecognized_options.empty? or raise Error, "unrecognized options: #{unrecognized_options.join(", ")}"
|
316
|
+
|
317
|
+
new(environment, uri, options)
|
318
|
+
end
|
319
|
+
|
29
320
|
end
|
30
321
|
|
31
322
|
attr_accessor :environment
|
@@ -67,18 +358,26 @@ module Librarian
|
|
67
358
|
def unpin!
|
68
359
|
end
|
69
360
|
|
70
|
-
def cache!(
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
361
|
+
def cache!(names)
|
362
|
+
end
|
363
|
+
|
364
|
+
def install!(manifest)
|
365
|
+
manifest.source == self or raise ArgumentError
|
366
|
+
|
367
|
+
name = manifest.name
|
368
|
+
version = manifest.version
|
369
|
+
install_path = install_path(name)
|
370
|
+
line = line(name)
|
371
|
+
|
372
|
+
debug { "Installing: #{manifest}" }
|
373
|
+
|
374
|
+
line.install_version! version, install_path
|
75
375
|
end
|
76
376
|
|
77
377
|
# NOTE:
|
78
378
|
# Assumes the Opscode Site API responds with versions in reverse sorted order
|
79
|
-
def manifests(
|
80
|
-
|
81
|
-
metadata['versions'].map{|version_uri| Manifest.new(self, dependency.name, version_uri)}
|
379
|
+
def manifests(name)
|
380
|
+
line(name).manifests
|
82
381
|
end
|
83
382
|
|
84
383
|
def manifest(name, version, dependencies)
|
@@ -88,10 +387,6 @@ module Librarian
|
|
88
387
|
manifest
|
89
388
|
end
|
90
389
|
|
91
|
-
def install_path(dependency)
|
92
|
-
environment.install_path.join(dependency.name)
|
93
|
-
end
|
94
|
-
|
95
390
|
def cache_path
|
96
391
|
@cache_path ||= begin
|
97
392
|
dir = Digest::MD5.hexdigest(uri)
|
@@ -99,109 +394,23 @@ module Librarian
|
|
99
394
|
end
|
100
395
|
end
|
101
396
|
|
102
|
-
def
|
103
|
-
|
397
|
+
def install_path(name)
|
398
|
+
environment.install_path.join(name)
|
104
399
|
end
|
105
400
|
|
106
|
-
def
|
107
|
-
|
401
|
+
def fetch_version(name, version_uri)
|
402
|
+
line(name).to_version(version_uri)
|
108
403
|
end
|
109
404
|
|
110
|
-
def
|
111
|
-
|
405
|
+
def fetch_dependencies(name, version, version_uri)
|
406
|
+
line(name).version_dependencies(version).map{|k, v| Dependency.new(k, v, nil)}
|
112
407
|
end
|
113
408
|
|
114
|
-
|
115
|
-
version_cache_path(dependency, version_uri).join("version.json")
|
116
|
-
end
|
117
|
-
|
118
|
-
def version_archive_cache_file(dependency, version_uri)
|
119
|
-
Pathname.new("archive.tgz")
|
120
|
-
end
|
121
|
-
|
122
|
-
def version_archive_cache_path(dependency, version_uri)
|
123
|
-
version_archive_cache_file = version_archive_cache_file(dependency, version_uri)
|
124
|
-
version_cache_path(dependency, version_uri).join(version_archive_cache_file)
|
125
|
-
end
|
126
|
-
|
127
|
-
def version_unpacked_cache_file(dependency, version_uri)
|
128
|
-
Pathname.new(dependency.name)
|
129
|
-
end
|
130
|
-
|
131
|
-
def version_unpacked_cache_path(dependency, version_uri)
|
132
|
-
version_unpacked_cache_file = version_unpacked_cache_file(dependency, version_uri)
|
133
|
-
version_cache_path(dependency, version_uri).join(version_unpacked_cache_file)
|
134
|
-
end
|
135
|
-
|
136
|
-
def version_package_cache_file(dependency, version_uri)
|
137
|
-
Pathname.new("package")
|
138
|
-
end
|
139
|
-
|
140
|
-
def version_package_cache_path(dependency, version_uri)
|
141
|
-
version_package_cache_file = version_package_cache_file(dependency, version_uri)
|
142
|
-
version_cache_path(dependency, version_uri).join(version_package_cache_file)
|
143
|
-
end
|
409
|
+
private
|
144
410
|
|
145
|
-
def
|
146
|
-
|
147
|
-
|
148
|
-
|
149
|
-
def cache_metadata!(dependency)
|
150
|
-
dependency_cache_path = cache_path.join(dependency.name)
|
151
|
-
dependency_cache_path.mkpath
|
152
|
-
metadata_cache_path = metadata_cache_path(dependency)
|
153
|
-
|
154
|
-
caching_metadata(dependency.name) do
|
155
|
-
dep_uri = URI.parse(dependency_uri(dependency))
|
156
|
-
debug { "Caching #{dep_uri}" }
|
157
|
-
http = Net::HTTP.new(dep_uri.host, dep_uri.port)
|
158
|
-
request = Net::HTTP::Get.new(dep_uri.path)
|
159
|
-
response = http.start{|http| http.request(request)}
|
160
|
-
unless Net::HTTPSuccess === response
|
161
|
-
raise Error, "Could not cache #{dependency} from #{dep_uri} because #{response.code} #{response.message}!"
|
162
|
-
end
|
163
|
-
metadata_blob = response.body
|
164
|
-
JSON.parse(metadata_blob) # check that it's JSON
|
165
|
-
metadata_cache_path(dependency).open('wb') do |f|
|
166
|
-
f.write(metadata_blob)
|
167
|
-
end
|
168
|
-
end
|
169
|
-
end
|
170
|
-
|
171
|
-
def caching_metadata(name)
|
172
|
-
_metadata_cache[name] = yield unless _metadata_cache.include?(name)
|
173
|
-
_metadata_cache[name]
|
174
|
-
end
|
175
|
-
|
176
|
-
def cache_version_metadata!(dependency, version_uri)
|
177
|
-
version_cache_path = version_cache_path(dependency, version_uri)
|
178
|
-
unless version_cache_path.exist?
|
179
|
-
version_cache_path.mkpath
|
180
|
-
debug { "Caching #{version_uri}" }
|
181
|
-
version_metadata_blob = Net::HTTP.get(URI.parse(version_uri))
|
182
|
-
JSON.parse(version_metadata_blob) # check that it's JSON
|
183
|
-
version_metadata_cache_path(dependency, version_uri).open('wb') do |f|
|
184
|
-
f.write(version_metadata_blob)
|
185
|
-
end
|
186
|
-
end
|
187
|
-
end
|
188
|
-
|
189
|
-
def cache_version_package!(dependency, version_uri, file_uri)
|
190
|
-
version_archive_cache_path = version_archive_cache_path(dependency, version_uri)
|
191
|
-
unless version_archive_cache_path.exist?
|
192
|
-
version_archive_cache_path.open('wb') do |f|
|
193
|
-
f.write(Net::HTTP.get(URI.parse(file_uri)))
|
194
|
-
end
|
195
|
-
end
|
196
|
-
version_package_cache_path = version_package_cache_path(dependency, version_uri)
|
197
|
-
unless version_package_cache_path.exist?
|
198
|
-
dependency_cache_path = dependency_cache_path(dependency)
|
199
|
-
version_unpacked_temp_path = dependency_cache_path.join(dependency.name)
|
200
|
-
Zlib::GzipReader.open(version_archive_cache_path) do |input|
|
201
|
-
Archive::Tar::Minitar.unpack(input, version_unpacked_temp_path.to_s)
|
202
|
-
end
|
203
|
-
FileUtils.move(version_unpacked_temp_path.join(dependency.name), version_package_cache_path)
|
204
|
-
end
|
411
|
+
def line(name)
|
412
|
+
@line ||= { }
|
413
|
+
@line[name] ||= Line.new(self, name)
|
205
414
|
end
|
206
415
|
|
207
416
|
end
|