librarian-puppet 1.0.9 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +20 -38
- data/lib/librarian/puppet/cli.rb +5 -11
- data/lib/librarian/puppet/dsl.rb +2 -38
- data/lib/librarian/puppet/environment.rb +0 -3
- data/lib/librarian/puppet/extension.rb +18 -19
- data/lib/librarian/puppet/lockfile/parser.rb +3 -5
- data/lib/librarian/puppet/source/forge.rb +10 -11
- data/lib/librarian/puppet/source/forge/repo.rb +3 -11
- data/lib/librarian/puppet/source/forge/repo_v1.rb +1 -12
- data/lib/librarian/puppet/source/forge/repo_v3.rb +4 -5
- data/lib/librarian/puppet/source/githubtarball.rb +1 -1
- data/lib/librarian/puppet/source/githubtarball/repo.rb +9 -32
- data/lib/librarian/puppet/source/local.rb +26 -67
- data/lib/librarian/puppet/source/repo.rb +1 -1
- data/lib/librarian/puppet/templates/Puppetfile +6 -15
- data/lib/librarian/puppet/util.rb +0 -10
- data/lib/librarian/puppet/version.rb +1 -1
- metadata +16 -6
- data/lib/librarian/puppet/action.rb +0 -1
- data/lib/librarian/puppet/action/install.rb +0 -24
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dec4cc59a2c57a5139ded1012aba17a43754bce8
|
4
|
+
data.tar.gz: f93343ec556afd556e21c33ceed2258cd4fa67ba
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 38d1da1a402eaf543f848d0bd9dcc351815f0a40085c1210a3a9d94dab1fc22235d821135087d1c2dc127e396bb30782b0a9d82699e4794b72a0bd77684d118b
|
7
|
+
data.tar.gz: fab30de03b221891662376c9189c41d1d8ff257f75adee91954f4b5b327d11fd7a1a3123ecbaf59a7cc0320281f15928b60efacdc2eee8cd4d90000c5e1f9e5f
|
data/README.md
CHANGED
@@ -9,7 +9,7 @@ librarian-puppet to manage the puppet modules your infrastructure depends on,
|
|
9
9
|
whether the modules come from the [Puppet Forge](https://forge.puppetlabs.com/),
|
10
10
|
Git repositories or a just a path.
|
11
11
|
|
12
|
-
* Librarian-puppet can reuse the dependencies listed in your
|
12
|
+
* Librarian-puppet can reuse the dependencies listed in your Modulefile
|
13
13
|
* Forge modules can be installed from [Puppetlabs Forge](https://forge.puppetlabs.com/) or an internal Forge such as [Pulp](http://www.pulpproject.org/)
|
14
14
|
* Git modules can be installed from a branch, tag or specific commit, optionally using a path inside the repository
|
15
15
|
* Modules can be installed from GitHub using tarballs, without needing Git installed
|
@@ -28,52 +28,42 @@ It is based on [Librarian](https://github.com/applicationsonline/librarian), a
|
|
28
28
|
framework for writing bundlers, which are tools that resolve, fetch, install,
|
29
29
|
and isolate a project's dependencies.
|
30
30
|
|
31
|
-
## Versions
|
32
|
-
|
33
|
-
Librarian-puppet >= 1.1.0 requires Ruby 1.9 and uses the Puppet Forge API v3.
|
34
|
-
Versions < 1.1.0 works on Ruby 1.8.
|
35
|
-
|
36
|
-
See the [Changelog](Changelog.md) for more details.
|
37
|
-
|
38
31
|
## The Puppetfile
|
39
32
|
|
40
|
-
Every Puppet repository that uses Librarian-puppet
|
41
|
-
`Puppetfile
|
42
|
-
|
43
|
-
for which modules your puppet infrastructure repository depends goes in here.
|
33
|
+
Every Puppet repository that uses Librarian-puppet will have a file named
|
34
|
+
`Puppetfile` in the root directory of that repository. The full specification
|
35
|
+
for which modules your puppet infrastructure repository depends goes in here.
|
44
36
|
|
45
|
-
### Simple
|
37
|
+
### Simple Puppetfile
|
46
38
|
|
47
|
-
|
48
|
-
listed in your `metadata.json` or `Modulefile` from the Puppet Forge,
|
49
|
-
as if the Puppetfile contained
|
39
|
+
This Puppetfile will download all the dependencies listed in your Modulefile from the Puppet Forge
|
50
40
|
|
51
|
-
forge "https://
|
41
|
+
forge "https://forge.puppetlabs.com"
|
52
42
|
|
53
|
-
|
43
|
+
modulefile
|
54
44
|
|
55
45
|
|
56
46
|
### Example Puppetfile
|
57
47
|
|
58
48
|
forge "https://forge.puppetlabs.com"
|
59
49
|
|
60
|
-
mod
|
61
|
-
mod
|
50
|
+
mod "puppetlabs/razor"
|
51
|
+
mod "puppetlabs/ntp", "0.0.3"
|
62
52
|
|
63
|
-
mod
|
53
|
+
mod "puppetlabs/apt",
|
64
54
|
:git => "git://github.com/puppetlabs/puppetlabs-apt.git"
|
65
55
|
|
66
|
-
mod
|
56
|
+
mod "puppetlabs/stdlib",
|
67
57
|
:git => "git://github.com/puppetlabs/puppetlabs-stdlib.git"
|
68
58
|
|
69
|
-
mod 'puppetlabs
|
59
|
+
mod 'puppetlabs/apache', '0.6.0',
|
70
60
|
:github_tarball => 'puppetlabs/puppetlabs-apache'
|
71
61
|
|
72
62
|
|
73
63
|
### Recursive module dependency resolution
|
74
64
|
|
75
65
|
When fetching a module all dependencies specified in its
|
76
|
-
`Modulefile
|
66
|
+
`Modulefile` and `Puppetfile` will be resolved and installed.
|
77
67
|
|
78
68
|
### Puppetfile Breakdown
|
79
69
|
|
@@ -83,29 +73,29 @@ This declares that we want to use the official Puppet Labs Forge as our default
|
|
83
73
|
source when pulling down modules. If you run your own local forge, you may
|
84
74
|
want to change this.
|
85
75
|
|
86
|
-
mod
|
76
|
+
mod "puppetlabs/razor"
|
87
77
|
|
88
78
|
Pull in the latest version of the Puppet Labs Razor module from the default
|
89
79
|
source.
|
90
80
|
|
91
|
-
mod
|
81
|
+
mod "puppetlabs/ntp", "0.0.3"
|
92
82
|
|
93
83
|
Pull in version 0.0.3 of the Puppet Labs NTP module from the default source.
|
94
84
|
|
95
|
-
mod
|
85
|
+
mod "puppetlabs/apt",
|
96
86
|
:git => "git://github.com/puppetlabs/puppetlabs-apt.git"
|
97
87
|
|
98
88
|
Our puppet infrastructure repository depends on the `apt` module from the
|
99
89
|
Puppet Labs GitHub repos and checks out the `master` branch.
|
100
90
|
|
101
|
-
mod
|
91
|
+
mod "puppetlabs/apt",
|
102
92
|
:git => "git://github.com/puppetlabs/puppetlabs-apt.git",
|
103
93
|
:ref => '0.0.3'
|
104
94
|
|
105
95
|
Our puppet infrastructure repository depends on the `apt` module from the
|
106
96
|
Puppet Labs GitHub repos and checks out a tag of `0.0.3`.
|
107
97
|
|
108
|
-
mod
|
98
|
+
mod "puppetlabs/apt",
|
109
99
|
:git => "git://github.com/puppetlabs/puppetlabs-apt.git",
|
110
100
|
:ref => 'feature/master/dans_refactor'
|
111
101
|
|
@@ -127,7 +117,7 @@ with many modules in it. If we need a module from such a repository, we can
|
|
127
117
|
use the `:path =>` option here to help Librarian-puppet drill down and find the
|
128
118
|
module subdirectory.
|
129
119
|
|
130
|
-
mod
|
120
|
+
mod "puppetlabs/apt",
|
131
121
|
:git => "git://github.com/fake/puppet-modules.git",
|
132
122
|
:path => "modules/apt"
|
133
123
|
|
@@ -164,14 +154,6 @@ source specified. This command writes the complete resolution into
|
|
164
154
|
`Puppetfile.lock` and then copies all of the fetched modules into your
|
165
155
|
`modules/` directory, overwriting whatever was there before.
|
166
156
|
|
167
|
-
Librarian-puppet support both v1 and v3 of the Puppet Forge API.
|
168
|
-
Specify a specific API version when installing modules:
|
169
|
-
|
170
|
-
$ librarian-puppet install --use-v1-api # this is default
|
171
|
-
$ librarian-puppet install --no-use-v1-api # use the v3 API
|
172
|
-
|
173
|
-
Please note that this does not apply for the official Puppet Forge where v3 is used by default.
|
174
|
-
|
175
157
|
Get an overview of your `Puppetfile.lock` with:
|
176
158
|
|
177
159
|
$ librarian-puppet show
|
data/lib/librarian/puppet/cli.rb
CHANGED
@@ -2,7 +2,6 @@ require 'librarian/helpers'
|
|
2
2
|
|
3
3
|
require 'librarian/cli'
|
4
4
|
require 'librarian/puppet'
|
5
|
-
require 'librarian/puppet/action'
|
6
5
|
|
7
6
|
module Librarian
|
8
7
|
module Puppet
|
@@ -45,9 +44,13 @@ module Librarian
|
|
45
44
|
option "path", :type => :string
|
46
45
|
option "destructive", :type => :boolean, :default => false
|
47
46
|
option "local", :type => :boolean, :default => false
|
48
|
-
option "use-v1-api", :type => :boolean, :default => true
|
49
47
|
def install
|
50
48
|
|
49
|
+
unless File.exist?('Puppetfile')
|
50
|
+
say "Could not find Puppetfile in #{Dir.pwd}", :red
|
51
|
+
exit 1
|
52
|
+
end
|
53
|
+
|
51
54
|
ensure!
|
52
55
|
clean! if options["clean"]
|
53
56
|
unless options["destructive"].nil?
|
@@ -61,7 +64,6 @@ module Librarian
|
|
61
64
|
environment.config_db.local["path"] = options["path"]
|
62
65
|
end
|
63
66
|
|
64
|
-
environment.config_db.local['use-v1-api'] = options['use-v1-api'] ? '1' : nil
|
65
67
|
environment.config_db.local['mode'] = options['local'] ? 'local' : nil
|
66
68
|
|
67
69
|
resolve!
|
@@ -85,14 +87,6 @@ module Librarian
|
|
85
87
|
def version
|
86
88
|
say "librarian-puppet v#{Librarian::Puppet::VERSION}"
|
87
89
|
end
|
88
|
-
|
89
|
-
private
|
90
|
-
|
91
|
-
# override the actions to use our own
|
92
|
-
|
93
|
-
def install!(options = { })
|
94
|
-
Action::Install.new(environment, options).run
|
95
|
-
end
|
96
90
|
end
|
97
91
|
end
|
98
92
|
end
|
data/lib/librarian/puppet/dsl.rb
CHANGED
@@ -17,14 +17,6 @@ module Librarian
|
|
17
17
|
def run(specfile = nil, sources = [])
|
18
18
|
specfile, sources = nil, specfile if specfile.kind_of?(Array) && sources.empty?
|
19
19
|
|
20
|
-
if specfile.kind_of?(Pathname) and !File.exists?(specfile)
|
21
|
-
debug { "Specfile not found, using defaults: #{specfile}" }
|
22
|
-
specfile = Proc.new do
|
23
|
-
forge "http://forge.puppetlabs.com"
|
24
|
-
metadata
|
25
|
-
end
|
26
|
-
end
|
27
|
-
|
28
20
|
Target.new(self).tap do |target|
|
29
21
|
target.precache_sources(sources)
|
30
22
|
debug_named_source_cache("Pre-Cached Sources", target)
|
@@ -38,49 +30,21 @@ module Librarian
|
|
38
30
|
end
|
39
31
|
|
40
32
|
class Receiver < Librarian::Dsl::Receiver
|
41
|
-
attr_reader :specfile
|
33
|
+
attr_reader :specfile
|
42
34
|
|
43
35
|
# save the specfile and call librarian
|
44
36
|
def run(specfile = nil)
|
45
|
-
@working_path = specfile.kind_of?(Pathname) ? specfile.parent : Pathname.new(Dir.pwd)
|
46
37
|
@specfile = specfile
|
47
38
|
super
|
48
39
|
end
|
49
40
|
|
50
41
|
# implement the 'modulefile' syntax for Puppetfile
|
51
42
|
def modulefile
|
52
|
-
|
53
|
-
raise Error, "Modulefile file does not exist: #{f}" unless File.exists?(f)
|
54
|
-
File.read(f).lines.each do |line|
|
43
|
+
File.read(Pathname.new(specfile).parent.join('Modulefile')).lines.each do |line|
|
55
44
|
regexp = /\s*dependency\s+('|")([^'"]+)\1\s*(?:,\s*('|")([^'"]+)\3)?/
|
56
45
|
regexp =~ line && mod($2, $4)
|
57
46
|
end
|
58
47
|
end
|
59
|
-
|
60
|
-
# implement the 'metadata' syntax for Puppetfile
|
61
|
-
def metadata
|
62
|
-
f = working_path.join('metadata.json')
|
63
|
-
unless File.exists?(f)
|
64
|
-
msg = "Metadata file does not exist: #{f}"
|
65
|
-
# try modulefile, in case we don't have a Puppetfile and we are using the default template
|
66
|
-
if File.exists?(modulefile_path)
|
67
|
-
modulefile
|
68
|
-
return
|
69
|
-
else
|
70
|
-
raise Error, msg
|
71
|
-
end
|
72
|
-
end
|
73
|
-
dependencyList = JSON.parse(File.read(f))['dependencies']
|
74
|
-
dependencyList.each do |d|
|
75
|
-
mod(d['name'], d['version_requirement'])
|
76
|
-
end
|
77
|
-
end
|
78
|
-
|
79
|
-
private
|
80
|
-
|
81
|
-
def modulefile_path
|
82
|
-
working_path.join('Modulefile')
|
83
|
-
end
|
84
48
|
end
|
85
49
|
end
|
86
50
|
end
|
@@ -8,22 +8,6 @@ module Librarian
|
|
8
8
|
end
|
9
9
|
|
10
10
|
class Dependency
|
11
|
-
include Librarian::Puppet::Util
|
12
|
-
|
13
|
-
def initialize(name, requirement, source)
|
14
|
-
assert_name_valid! name
|
15
|
-
|
16
|
-
# Issue #235 fail if forge source is not defined
|
17
|
-
raise Error, "forge entry is not defined in Puppetfile" if source.instance_of?(Array) && source.empty?
|
18
|
-
|
19
|
-
# let's settle on provider-module syntax instead of provider/module
|
20
|
-
self.name = normalize_name(name)
|
21
|
-
self.requirement = Requirement.new(requirement)
|
22
|
-
self.source = source
|
23
|
-
|
24
|
-
@manifests = nil
|
25
|
-
end
|
26
|
-
|
27
11
|
class Requirement
|
28
12
|
def initialize(*args)
|
29
13
|
args = initialize_normalize_args(args)
|
@@ -83,9 +67,23 @@ module Librarian
|
|
83
67
|
end
|
84
68
|
end
|
85
69
|
|
86
|
-
|
87
|
-
|
70
|
+
module Action
|
71
|
+
class Install < Base
|
88
72
|
|
73
|
+
private
|
74
|
+
|
75
|
+
def create_install_path
|
76
|
+
install_path.rmtree if install_path.exist? && destructive?
|
77
|
+
install_path.mkpath
|
78
|
+
end
|
79
|
+
|
80
|
+
def destructive?
|
81
|
+
environment.config_db.local['destructive'] == 'true'
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
|
86
|
+
class ManifestSet
|
89
87
|
private
|
90
88
|
|
91
89
|
# Check if module doesn't exist and fail fast
|
@@ -95,7 +93,7 @@ module Librarian
|
|
95
93
|
|
96
94
|
deps = Set.new
|
97
95
|
until names.empty?
|
98
|
-
name =
|
96
|
+
name = names.shift
|
99
97
|
next if deps.include?(name)
|
100
98
|
|
101
99
|
deps << name
|
@@ -107,6 +105,7 @@ module Librarian
|
|
107
105
|
end
|
108
106
|
|
109
107
|
class Manifest
|
108
|
+
|
110
109
|
class PreReleaseVersion
|
111
110
|
|
112
111
|
# Compares pre-release component ids using Semver 2.0.0 spec
|
@@ -5,7 +5,6 @@ require 'librarian/manifest_set'
|
|
5
5
|
module Librarian
|
6
6
|
class Lockfile
|
7
7
|
class Parser
|
8
|
-
include Librarian::Puppet::Util
|
9
8
|
|
10
9
|
def parse(string)
|
11
10
|
string = string.dup
|
@@ -27,8 +26,8 @@ module Librarian
|
|
27
26
|
manifests = {}
|
28
27
|
while lines.first =~ /^ {4}([\w\-\/]+) \((.*)\)$/ # This change allows forward slash
|
29
28
|
lines.shift
|
30
|
-
name
|
31
|
-
manifests[name] = {:version =>
|
29
|
+
name = $1
|
30
|
+
manifests[name] = {:version => $2, :dependencies => {}}
|
32
31
|
while lines.first =~ /^ {6}([\w\-\/]+) \((.*)\)$/
|
33
32
|
lines.shift
|
34
33
|
manifests[name][:dependencies][$1] = $2.split(/,\s*/)
|
@@ -43,10 +42,9 @@ module Librarian
|
|
43
42
|
dependencies = []
|
44
43
|
while lines.first =~ /^ {2}([\w\-\/]+)(?: \((.*)\))?$/ # This change allows forward slash
|
45
44
|
lines.shift
|
46
|
-
name, requirement =
|
45
|
+
name, requirement = $1, $2.split(/,\s*/)
|
47
46
|
dependencies << Dependency.new(name, requirement, manifests_index[name].source)
|
48
47
|
end
|
49
|
-
|
50
48
|
Resolution.new(dependencies, manifests)
|
51
49
|
end
|
52
50
|
|
@@ -1,7 +1,7 @@
|
|
1
1
|
require 'uri'
|
2
2
|
require 'librarian/puppet/util'
|
3
3
|
require 'librarian/puppet/source/forge/repo_v1'
|
4
|
-
|
4
|
+
require 'librarian/puppet/source/forge/repo_v3'
|
5
5
|
|
6
6
|
module Librarian
|
7
7
|
module Puppet
|
@@ -53,10 +53,10 @@ module Librarian
|
|
53
53
|
def initialize(environment, uri, options = {})
|
54
54
|
self.environment = environment
|
55
55
|
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
56
|
+
if uri =~ %r{^http(s)?://forge\.puppetlabs\.com}
|
57
|
+
uri = "https://forgeapi.puppetlabs.com"
|
58
|
+
debug { "Replacing Puppet Forge API URL to use v3 #{uri}. You should update your Puppetfile" }
|
59
|
+
end
|
60
60
|
|
61
61
|
@uri = URI::parse(uri)
|
62
62
|
@cache_path = nil
|
@@ -121,7 +121,7 @@ module Librarian
|
|
121
121
|
end
|
122
122
|
|
123
123
|
def install_path(name)
|
124
|
-
environment.install_path.join(
|
124
|
+
environment.install_path.join(name.split('/').last)
|
125
125
|
end
|
126
126
|
|
127
127
|
def fetch_version(name, version_uri)
|
@@ -148,14 +148,13 @@ module Librarian
|
|
148
148
|
|
149
149
|
def repo(name)
|
150
150
|
@repo ||= {}
|
151
|
-
|
152
151
|
unless @repo[name]
|
153
152
|
# if we are using the official Forge then use API v3, otherwise stick to v1 for now
|
154
|
-
|
155
|
-
|
156
|
-
|
153
|
+
if uri.hostname =~ /\.puppetlabs\.com$/
|
154
|
+
@repo[name] = RepoV3.new(self, name)
|
155
|
+
else
|
157
156
|
@repo[name] = RepoV1.new(self, name)
|
158
|
-
|
157
|
+
end
|
159
158
|
end
|
160
159
|
@repo[name]
|
161
160
|
end
|
@@ -59,7 +59,7 @@ module Librarian
|
|
59
59
|
install_path.rmtree
|
60
60
|
end
|
61
61
|
|
62
|
-
unpacked_path = version_unpacked_cache_path(version).join(
|
62
|
+
unpacked_path = version_unpacked_cache_path(version).join(name.split('/').last)
|
63
63
|
|
64
64
|
unless unpacked_path.exist?
|
65
65
|
raise Error, "#{unpacked_path} does not exist, something went wrong. Try removing it manually"
|
@@ -86,17 +86,9 @@ module Librarian
|
|
86
86
|
|
87
87
|
target = vendored?(name, version) ? vendored_path(name, version).to_s : name
|
88
88
|
|
89
|
-
# can't pass the default v3 forge url (http://forgeapi.puppetlabs.com)
|
90
|
-
# to clients that use the v1 API (https://forge.puppetlabs.com)
|
91
|
-
# nor the other way around
|
89
|
+
# TODO can't pass the default v3 forge url (http://forgeapi.puppetlabs.com) to clients that use the v1 API (https://forge.puppetlabs.com)
|
92
90
|
module_repository = source.to_s
|
93
|
-
|
94
|
-
if Forge.client_api_version() > 1 and module_repository =~ %r{^http(s)?://forge\.puppetlabs\.com}
|
95
|
-
module_repository = "https://forgeapi.puppetlabs.com"
|
96
|
-
warn { "Replacing Puppet Forge API URL to use v3 #{module_repository} as required by your client version #{Librarian::Puppet.puppet_version}" }
|
97
|
-
end
|
98
|
-
|
99
|
-
m = module_repository.match(%r{^http(s)?://forge(api)?\.puppetlabs\.com})
|
91
|
+
m = module_repository.match(%r{^http(s)?://forgeapi\.puppetlabs\.com})
|
100
92
|
if Forge.client_api_version() == 1 and m
|
101
93
|
ssl = m[1]
|
102
94
|
# Puppet 2.7 can't handle the 302 returned by the https url, so stick to http
|
@@ -34,15 +34,11 @@ module Librarian
|
|
34
34
|
private
|
35
35
|
|
36
36
|
# Issue #223 dependencies may be duplicated
|
37
|
-
# and convert organization/modulename to organization-modulename
|
38
37
|
def clear_duplicated_dependencies(data)
|
39
38
|
return nil if data.nil?
|
40
39
|
data.each do |m,versions|
|
41
40
|
versions.each do |v|
|
42
41
|
if v["dependencies"] and !v["dependencies"].empty?
|
43
|
-
# convert organization/modulename to organization-modulename
|
44
|
-
v["dependencies"].each {|d| d[0] = normalize_name(d[0])}
|
45
|
-
|
46
42
|
dependency_names = v["dependencies"].map {|d| d[0]}
|
47
43
|
duplicated = dependency_names.select{ |e| dependency_names.count(e) > 1 }
|
48
44
|
unless duplicated.empty?
|
@@ -57,13 +53,6 @@ module Librarian
|
|
57
53
|
end
|
58
54
|
end
|
59
55
|
end
|
60
|
-
# convert organization/modulename to organization-modulename
|
61
|
-
data.keys.each do |m|
|
62
|
-
if m =~ %r{.*/.*}
|
63
|
-
data[normalize_name(m)] = data[m]
|
64
|
-
data.delete(m)
|
65
|
-
end
|
66
|
-
end
|
67
56
|
data
|
68
57
|
end
|
69
58
|
|
@@ -90,7 +79,7 @@ module Librarian
|
|
90
79
|
def api_call(module_name, version=nil)
|
91
80
|
url = source.uri.clone
|
92
81
|
url.path += "#{'/' if url.path.empty? or url.path[-1] != '/'}api/v1/releases.json"
|
93
|
-
url.query = "module=#{module_name
|
82
|
+
url.query = "module=#{module_name}"
|
94
83
|
url.query += "&version=#{version}" unless version.nil?
|
95
84
|
debug { "Querying Forge API for module #{name}#{" and version #{version}" unless version.nil?}: #{url}" }
|
96
85
|
|
@@ -1,6 +1,7 @@
|
|
1
1
|
require 'librarian/puppet/source/forge/repo'
|
2
2
|
require 'puppet_forge'
|
3
|
-
|
3
|
+
|
4
|
+
PuppetForge.user_agent = "librarian-puppet/#{Librarian::Puppet::VERSION}"
|
4
5
|
|
5
6
|
module Librarian
|
6
7
|
module Puppet
|
@@ -8,8 +9,6 @@ module Librarian
|
|
8
9
|
class Forge
|
9
10
|
class RepoV3 < Librarian::Puppet::Source::Forge::Repo
|
10
11
|
|
11
|
-
PuppetForge.user_agent = "librarian-puppet/#{Librarian::Puppet::VERSION}"
|
12
|
-
|
13
12
|
def get_versions
|
14
13
|
get_module.releases.map{|r| r.version}
|
15
14
|
end
|
@@ -25,7 +24,7 @@ module Librarian
|
|
25
24
|
else
|
26
25
|
# should never get here as we use one repo object for each module (to be changed in the future)
|
27
26
|
debug { "Looking up url for #{name}@#{version}" }
|
28
|
-
release = PuppetForge::Release.find("#{name}-#{version}")
|
27
|
+
release = PuppetForge::Release.find("#{name.sub('/','-')}-#{version}")
|
29
28
|
end
|
30
29
|
"#{source}#{release.file_uri}"
|
31
30
|
end
|
@@ -33,7 +32,7 @@ module Librarian
|
|
33
32
|
private
|
34
33
|
|
35
34
|
def get_module
|
36
|
-
@module ||= PuppetForge::Module.find(name)
|
35
|
+
@module ||= PuppetForge::Module.find(name.sub('/','-'))
|
37
36
|
raise(Error, "Unable to find module '#{name}' on #{source}") unless @module
|
38
37
|
@module
|
39
38
|
end
|
@@ -25,7 +25,7 @@ module Librarian
|
|
25
25
|
all_versions = data.map { |r| r['name'].gsub(/^v/, '') }.sort.reverse
|
26
26
|
|
27
27
|
all_versions.delete_if do |version|
|
28
|
-
version !~ /\A\d
|
28
|
+
version !~ /\A\d\.\d(\.\d.*)?\z/
|
29
29
|
end
|
30
30
|
|
31
31
|
@versions = all_versions.compact
|
@@ -40,11 +40,11 @@ module Librarian
|
|
40
40
|
end
|
41
41
|
|
42
42
|
def install_version!(version, install_path)
|
43
|
-
if environment.local? && !vendored?(
|
43
|
+
if environment.local? && !vendored?(source.uri.to_s, version)
|
44
44
|
raise Error, "Could not find a local copy of #{source.uri} at #{version}."
|
45
45
|
end
|
46
46
|
|
47
|
-
vendor_cache(source.uri.to_s, version) unless vendored?(
|
47
|
+
vendor_cache(source.uri.to_s, version) unless vendored?(source.uri.to_s, version)
|
48
48
|
|
49
49
|
cache_version_unpacked! version
|
50
50
|
|
@@ -62,19 +62,19 @@ module Librarian
|
|
62
62
|
|
63
63
|
path.mkpath
|
64
64
|
|
65
|
-
target = vendored?(
|
65
|
+
target = vendored?(source.uri.to_s, version) ? vendored_path(source.uri.to_s, version) : name
|
66
66
|
|
67
67
|
Librarian::Posix.run!(%W{tar xzf #{target} -C #{path}})
|
68
68
|
end
|
69
69
|
|
70
70
|
def vendor_cache(name, version)
|
71
|
-
clean_up_old_cached_versions(
|
71
|
+
clean_up_old_cached_versions(name)
|
72
72
|
|
73
73
|
url = "https://api.github.com/repos/#{name}/tarball/#{version}"
|
74
|
-
|
74
|
+
url << "?access_token=#{ENV['GITHUB_API_TOKEN']}" if ENV['GITHUB_API_TOKEN']
|
75
75
|
|
76
76
|
environment.vendor!
|
77
|
-
File.open(vendored_path(
|
77
|
+
File.open(vendored_path(name, version).to_s, 'wb') do |f|
|
78
78
|
begin
|
79
79
|
debug { "Downloading <#{url}> to <#{f.path}>" }
|
80
80
|
open(url,
|
@@ -90,30 +90,11 @@ module Librarian
|
|
90
90
|
end
|
91
91
|
|
92
92
|
def clean_up_old_cached_versions(name)
|
93
|
-
Dir["#{environment.vendor_cache}/#{name}*.tar.gz"].each do |old_version|
|
93
|
+
Dir["#{environment.vendor_cache}/#{name.sub('/', '-')}*.tar.gz"].each do |old_version|
|
94
94
|
FileUtils.rm old_version
|
95
95
|
end
|
96
96
|
end
|
97
97
|
|
98
|
-
def token_key_value
|
99
|
-
ENV[TOKEN_KEY]
|
100
|
-
end
|
101
|
-
|
102
|
-
def token_key_nil?
|
103
|
-
token_key_value.nil? || token_key_value.empty?
|
104
|
-
end
|
105
|
-
|
106
|
-
def add_api_token_to_url url
|
107
|
-
if token_key_nil?
|
108
|
-
debug { "#{TOKEN_KEY} environment value is empty or missing" }
|
109
|
-
elsif url.include? "?"
|
110
|
-
url << "&access_token=#{ENV[TOKEN_KEY]}"
|
111
|
-
else
|
112
|
-
url << "?access_token=#{ENV[TOKEN_KEY]}"
|
113
|
-
end
|
114
|
-
url
|
115
|
-
end
|
116
|
-
|
117
98
|
private
|
118
99
|
|
119
100
|
def api_call(path)
|
@@ -121,7 +102,7 @@ module Librarian
|
|
121
102
|
url = "https://api.github.com#{path}?page=1&per_page=100"
|
122
103
|
while true do
|
123
104
|
debug { " Module #{name} getting tags at: #{url}" }
|
124
|
-
|
105
|
+
url << "&access_token=#{ENV[TOKEN_KEY]}" if ENV[TOKEN_KEY]
|
125
106
|
response = http_get(url, :headers => {
|
126
107
|
"User-Agent" => "librarian-puppet v#{Librarian::Puppet::VERSION}"
|
127
108
|
})
|
@@ -161,10 +142,6 @@ module Librarian
|
|
161
142
|
options[:headers].each { |k, v| request.add_field k, v }
|
162
143
|
http.request(request)
|
163
144
|
end
|
164
|
-
|
165
|
-
def vendored_name(name = source.uri.to_s)
|
166
|
-
name.sub('/','-')
|
167
|
-
end
|
168
145
|
end
|
169
146
|
end
|
170
147
|
end
|
@@ -1,5 +1,16 @@
|
|
1
1
|
require 'librarian/puppet/util'
|
2
2
|
|
3
|
+
begin
|
4
|
+
require 'puppet'
|
5
|
+
require 'puppet/module_tool'
|
6
|
+
rescue LoadError
|
7
|
+
$stderr.puts <<-EOF
|
8
|
+
Unable to load puppet, the puppet gem is required for :git and :path source.
|
9
|
+
Install it with: gem install puppet
|
10
|
+
EOF
|
11
|
+
exit 1
|
12
|
+
end
|
13
|
+
|
3
14
|
module Librarian
|
4
15
|
module Puppet
|
5
16
|
module Source
|
@@ -15,11 +26,11 @@ module Librarian
|
|
15
26
|
found_path = found_path(name)
|
16
27
|
raise Error, "Path for #{name} doesn't contain a puppet module" if found_path.nil?
|
17
28
|
|
18
|
-
unless name.include? '/'
|
19
|
-
warn { "Invalid module name '#{name}', you should qualify it with 'ORGANIZATION
|
29
|
+
unless name.include? '/'
|
30
|
+
warn { "Invalid module name '#{name}', you should qualify it with 'ORGANIZATION/#{name}' for resolution to work correctly" }
|
20
31
|
end
|
21
32
|
|
22
|
-
install_path = environment.install_path.join(
|
33
|
+
install_path = environment.install_path.join(name.split('/').last)
|
23
34
|
if install_path.exist?
|
24
35
|
debug { "Deleting #{relative_path_to(install_path)}" }
|
25
36
|
install_path.rmtree
|
@@ -37,60 +48,39 @@ module Librarian
|
|
37
48
|
def fetch_dependencies(name, version, extra)
|
38
49
|
dependencies = Set.new
|
39
50
|
|
51
|
+
if modulefile?
|
52
|
+
evaluate_modulefile(modulefile).dependencies.each do |dependency|
|
53
|
+
dependency_name = dependency.instance_variable_get(:@full_module_name)
|
54
|
+
version = dependency.instance_variable_get(:@version_requirement)
|
55
|
+
gem_requirement = Requirement.new(version).gem_requirement
|
56
|
+
dependencies << Dependency.new(dependency_name, gem_requirement, forge_source)
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
40
60
|
if specfile?
|
41
61
|
spec = environment.dsl(Pathname(specfile))
|
42
62
|
dependencies.merge spec.dependencies
|
43
63
|
end
|
44
64
|
|
45
|
-
parsed_metadata['dependencies'].each do |d|
|
46
|
-
gem_requirement = Requirement.new(d['version_requirement']).gem_requirement
|
47
|
-
new_dependency = Dependency.new(d['name'], gem_requirement, forge_source)
|
48
|
-
# Avoid duplicated dependencies with different sources
|
49
|
-
unless dependencies.find { |spec_dependency| spec_dependency.name == new_dependency.name && spec_dependency.requirement == new_dependency.requirement }
|
50
|
-
dependencies << new_dependency
|
51
|
-
end
|
52
|
-
end
|
53
|
-
|
54
65
|
dependencies
|
55
66
|
end
|
56
67
|
|
57
68
|
def forge_source
|
58
|
-
Forge.from_lock_options(environment, :remote
|
69
|
+
Forge.from_lock_options(environment, :remote=>"http://forge.puppetlabs.com")
|
59
70
|
end
|
60
71
|
|
61
72
|
private
|
62
73
|
|
63
74
|
# Naming this method 'version' causes an exception to be raised.
|
64
75
|
def module_version
|
65
|
-
|
66
|
-
|
67
|
-
else
|
68
|
-
warn { "Module #{to_s} does not have version, defaulting to 0.0.1" }
|
69
|
-
'0.0.1'
|
70
|
-
end
|
71
|
-
end
|
72
|
-
|
73
|
-
def require_puppet
|
74
|
-
begin
|
75
|
-
require 'puppet'
|
76
|
-
require 'puppet/module_tool'
|
77
|
-
rescue LoadError
|
78
|
-
$stderr.puts <<-EOF
|
79
|
-
Unable to load puppet, the puppet gem is required for :git and :path source.
|
80
|
-
Install it with: gem install puppet
|
81
|
-
EOF
|
82
|
-
exit 1
|
83
|
-
end
|
84
|
-
true
|
76
|
+
return '0.0.1' unless modulefile?
|
77
|
+
evaluate_modulefile(modulefile).version
|
85
78
|
end
|
86
79
|
|
87
80
|
def evaluate_modulefile(modulefile)
|
88
|
-
@@require_puppet ||= require_puppet
|
89
|
-
|
90
81
|
metadata = ::Puppet::ModuleTool::Metadata.new
|
91
82
|
begin
|
92
83
|
::Puppet::ModuleTool::ModulefileReader.evaluate(metadata, modulefile)
|
93
|
-
raise SyntaxError, "Missing version" unless metadata.version
|
94
84
|
rescue ArgumentError, SyntaxError => error
|
95
85
|
warn { "Unable to parse #{modulefile}, ignoring: #{error}" }
|
96
86
|
if metadata.respond_to? :version=
|
@@ -102,29 +92,6 @@ module Librarian
|
|
102
92
|
metadata
|
103
93
|
end
|
104
94
|
|
105
|
-
def parsed_metadata
|
106
|
-
@metadata ||= if metadata?
|
107
|
-
JSON.parse(File.read(metadata))
|
108
|
-
elsif modulefile?
|
109
|
-
# translate Modulefile to metadata.json
|
110
|
-
evaluated = evaluate_modulefile(modulefile)
|
111
|
-
{
|
112
|
-
'version' => evaluated.version,
|
113
|
-
'dependencies' => evaluated.dependencies.map do |dependency|
|
114
|
-
{
|
115
|
-
'name' => dependency.instance_variable_get(:@full_module_name),
|
116
|
-
'version_requirement' => dependency.instance_variable_get(:@version_requirement)
|
117
|
-
}
|
118
|
-
end
|
119
|
-
}
|
120
|
-
else
|
121
|
-
{
|
122
|
-
'dependencies' => []
|
123
|
-
}
|
124
|
-
end
|
125
|
-
@metadata
|
126
|
-
end
|
127
|
-
|
128
95
|
def modulefile
|
129
96
|
File.join(filesystem_path, 'Modulefile')
|
130
97
|
end
|
@@ -133,14 +100,6 @@ module Librarian
|
|
133
100
|
File.exists?(modulefile)
|
134
101
|
end
|
135
102
|
|
136
|
-
def metadata
|
137
|
-
File.join(filesystem_path, 'metadata.json')
|
138
|
-
end
|
139
|
-
|
140
|
-
def metadata?
|
141
|
-
File.exists?(metadata)
|
142
|
-
end
|
143
|
-
|
144
103
|
def specfile
|
145
104
|
File.join(filesystem_path, environment.specfile_name)
|
146
105
|
end
|
@@ -1,25 +1,16 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
#^syntax detection
|
3
3
|
|
4
|
-
forge "
|
5
|
-
|
6
|
-
# use dependencies defined in metadata.json
|
7
|
-
metadata
|
4
|
+
forge "http://forge.puppetlabs.com"
|
8
5
|
|
9
6
|
# use dependencies defined in Modulefile
|
10
|
-
|
7
|
+
modulefile
|
11
8
|
|
12
|
-
#
|
13
|
-
# mod 'puppetlabs-stdlib'
|
9
|
+
# mod 'puppetlabs/stdlib'
|
14
10
|
|
15
|
-
#
|
16
|
-
# mod 'puppetlabs-ntp',
|
11
|
+
# mod 'ntp',
|
17
12
|
# :git => 'git://github.com/puppetlabs/puppetlabs-ntp.git'
|
18
13
|
|
19
|
-
#
|
20
|
-
# mod 'puppetlabs-apt',
|
14
|
+
# mod 'apt',
|
21
15
|
# :git => 'https://github.com/puppetlabs/puppetlabs-apt.git',
|
22
|
-
# :ref => '
|
23
|
-
|
24
|
-
# A module from Github pre-packaged tarball
|
25
|
-
# mod 'puppetlabs-apache', '0.6.0', :github_tarball => 'puppetlabs/puppetlabs-apache'
|
16
|
+
# :ref => 'feature/master/dans_refactor'
|
@@ -33,16 +33,6 @@ module Librarian
|
|
33
33
|
new_uri.password = nil
|
34
34
|
new_uri
|
35
35
|
end
|
36
|
-
|
37
|
-
# normalize module name to use organization-module instead of organization/module
|
38
|
-
def normalize_name(name)
|
39
|
-
name.sub('/','-')
|
40
|
-
end
|
41
|
-
|
42
|
-
# get the organization name from organization-module
|
43
|
-
def organization_name(name)
|
44
|
-
name.split('-',2).last
|
45
|
-
end
|
46
36
|
end
|
47
37
|
end
|
48
38
|
end
|
metadata
CHANGED
@@ -1,15 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: librarian-puppet
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tim Sharpe
|
8
|
-
- Carlos Sanchez
|
9
8
|
autorequire:
|
10
9
|
bindir: bin
|
11
10
|
cert_chain: []
|
12
|
-
date: 2014-
|
11
|
+
date: 2014-06-11 00:00:00.000000000 Z
|
13
12
|
dependencies:
|
14
13
|
- !ruby/object:Gem::Dependency
|
15
14
|
name: librarian
|
@@ -25,6 +24,20 @@ dependencies:
|
|
25
24
|
- - '>='
|
26
25
|
- !ruby/object:Gem::Version
|
27
26
|
version: 0.1.2
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: puppet_forge
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - '>='
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - '>='
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
28
41
|
- !ruby/object:Gem::Dependency
|
29
42
|
name: json
|
30
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -143,7 +156,6 @@ description: |-
|
|
143
156
|
a single command.
|
144
157
|
email:
|
145
158
|
- tim@sharpe.id.au
|
146
|
-
- carlos@apache.org
|
147
159
|
executables:
|
148
160
|
- librarian-puppet
|
149
161
|
extensions: []
|
@@ -154,8 +166,6 @@ files:
|
|
154
166
|
- README.md
|
155
167
|
- bin/librarian-puppet
|
156
168
|
- lib/librarian/puppet.rb
|
157
|
-
- lib/librarian/puppet/action.rb
|
158
|
-
- lib/librarian/puppet/action/install.rb
|
159
169
|
- lib/librarian/puppet/cli.rb
|
160
170
|
- lib/librarian/puppet/dsl.rb
|
161
171
|
- lib/librarian/puppet/environment.rb
|
@@ -1 +0,0 @@
|
|
1
|
-
require "librarian/puppet/action/install"
|
@@ -1,24 +0,0 @@
|
|
1
|
-
module Librarian
|
2
|
-
module Puppet
|
3
|
-
module Action
|
4
|
-
class Install < Librarian::Action::Install
|
5
|
-
|
6
|
-
private
|
7
|
-
|
8
|
-
def create_install_path
|
9
|
-
install_path.rmtree if install_path.exist? && destructive?
|
10
|
-
install_path.mkpath
|
11
|
-
end
|
12
|
-
|
13
|
-
def destructive?
|
14
|
-
environment.config_db.local['destructive'] == 'true'
|
15
|
-
end
|
16
|
-
|
17
|
-
def check_specfile
|
18
|
-
# don't fail if Puppetfile doesn't exist as we'll use the Modulefile or metadata.json
|
19
|
-
end
|
20
|
-
|
21
|
-
end
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|