licensed 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +16 -1
- data/README.md +1 -1
- data/docs/sources/manifests.md +124 -3
- data/lib/licensed.rb +0 -29
- data/lib/licensed/cli.rb +1 -2
- data/lib/licensed/dependency.rb +27 -43
- data/lib/licensed/git.rb +6 -0
- data/lib/licensed/license.rb +3 -2
- data/lib/licensed/source/bundler.rb +7 -0
- data/lib/licensed/source/manifest.rb +205 -21
- data/lib/licensed/source/npm.rb +1 -12
- data/lib/licensed/version.rb +1 -1
- data/licensed.gemspec +0 -3
- metadata +2 -44
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a40ec635869b9918fb610cf744adb0ff82a65410
|
4
|
+
data.tar.gz: f810f71a593eb01547b1ffc518b19e3df8b21ca3
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c4b254f5fb6bd2a268adfc06971e82421be382d3f83d3eaca29acd5839c56d8c646c151c1b6702fb636c148ab2fff173171e066da315f1e2bf2ceb47349f8b46
|
7
|
+
data.tar.gz: 9426edecae8edf19eb61374711f3fdb2eae46468c0c0eff1d7db13f21718ac524a7f21bb1dce5f528d02e8f9bee68e043a393ebe513ee8ef6da5ef71073d0923
|
data/CHANGELOG.md
CHANGED
@@ -6,6 +6,21 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
|
|
6
6
|
|
7
7
|
## [Unreleased]
|
8
8
|
|
9
|
+
## 1.3.0 - 2018-07-25
|
10
|
+
### Added
|
11
|
+
- Manifests for the manifest dependency source can be specified using glob patterns in the configuration
|
12
|
+
- Paths to licenses for dependencies from the manifest dependency source can be specified in the configuration
|
13
|
+
- Manifest dependency source looks for license content in C-style comments if a license file isn't found
|
14
|
+
|
15
|
+
## Changes
|
16
|
+
- GitHub is no longer queried to find remote license information
|
17
|
+
- Removed custom logic around determining whether to use the license key from `licensee`
|
18
|
+
- NPM dependency enumeration doesn't use `npm list`
|
19
|
+
- Licensed now tracks content from multiple license files when available
|
20
|
+
|
21
|
+
### Fixed
|
22
|
+
- Fixed regression finding platform-specific ruby gems
|
23
|
+
|
9
24
|
## 1.2.0 - 2018-06-22
|
10
25
|
### Added
|
11
26
|
- Building and packaging distributable exes for licensed releases
|
@@ -45,4 +60,4 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
|
|
45
60
|
|
46
61
|
Initial release :tada:
|
47
62
|
|
48
|
-
[Unreleased]: https://github.com/github/licensed/compare/1.
|
63
|
+
[Unreleased]: https://github.com/github/licensed/compare/1.3.0...HEAD
|
data/README.md
CHANGED
@@ -80,7 +80,7 @@ Dependencies will be automatically detected for
|
|
80
80
|
5. [Go Dep](./docs/sources/dep.md)
|
81
81
|
6. [Manifest lists](./docs/sources/manifests.md)
|
82
82
|
7. [NPM](./docs/sources/npm.md)
|
83
|
-
8. [Pip](./docs/
|
83
|
+
8. [Pip](./docs/sources/pip.md)
|
84
84
|
|
85
85
|
You can disable any of them in the configuration file:
|
86
86
|
|
data/docs/sources/manifests.md
CHANGED
@@ -1,6 +1,16 @@
|
|
1
1
|
# Manifests
|
2
2
|
|
3
|
-
The manifest source can be used when no package managers are available.
|
3
|
+
The manifest source can be used when no package managers are available. The manifest source will be enabled when a manifest file is found or a manifest is configured in the configuration file.
|
4
|
+
|
5
|
+
## Manifest sources
|
6
|
+
|
7
|
+
A dependency file manifest can be specified in two ways - a dependency manifest file or in the licensed configuration file as a set of patterns used to find files.
|
8
|
+
|
9
|
+
Manifests are loaded from (in order)
|
10
|
+
- A manifest file, if found
|
11
|
+
- The licensed configuration file, if a manifest file is not found and the `manifest.dependencies` configuration property exists.
|
12
|
+
|
13
|
+
### Manifest files
|
4
14
|
|
5
15
|
Manifest files are used to match source files with their corresponding packages to find package dependencies. Manifest file paths can be specified in the app configuration with the following setting:
|
6
16
|
```yml
|
@@ -19,8 +29,119 @@ The manifest can be a JSON or YAML file with a single root object and properties
|
|
19
29
|
}
|
20
30
|
```
|
21
31
|
|
22
|
-
File paths are relative to the git repository root.
|
32
|
+
File paths are relative to the git repository root. A metadata file, `path/to/cache/manifest/<package>.txt`, will be created for each package.
|
33
|
+
|
34
|
+
**NOTE** It is the responsibility of the repository owner to maintain the manifest file.
|
35
|
+
|
36
|
+
### Configured manifest
|
37
|
+
|
38
|
+
Manifests can be specified using patterns specified in the configuration file.
|
39
|
+
|
40
|
+
Dependencies are specified at the `manifest.dependencies` configuration property and should map dependency names to one or more patterns of files matching the dependencies.
|
41
|
+
|
42
|
+
In the following example, there are two dependencies, `package` and `nested`, where `nested` is a sub-dependency in the `vendor/package` folder. Using inclusion and exclusion patterns we can specify that files in the `vendor/package/nested` folder belong to the `nested` package, while all other files belong to `package`.
|
43
|
+
|
44
|
+
```yaml
|
45
|
+
manifest:
|
46
|
+
dependencies:
|
47
|
+
package:
|
48
|
+
- "vendor/package/**/*"
|
49
|
+
- "!vendor/package/nested/*"
|
50
|
+
nested: "vendor/package/nested/*"
|
51
|
+
```
|
52
|
+
|
53
|
+
This demonstrates that
|
54
|
+
1. Dependencies can be mapped to a string or an array of strings
|
55
|
+
2. Inclusion and exclusion patterns apply
|
56
|
+
3. Patterns follow a glob-style format
|
57
|
+
|
58
|
+
#### Pattern format
|
59
|
+
|
60
|
+
Patterns are evaluated using [Dir.glob](https://ruby-doc.org/core/Dir.html#method-c-glob) and must follow these rules:
|
61
|
+
1. Patterns are evaluated from the project root directory
|
62
|
+
2. Patterns will only match files that are tracked by Git
|
63
|
+
3. Patterns should follow standard shell glob syntax
|
64
|
+
4. Patterns are evaluated in the order specified - order matters!
|
65
|
+
5. `!` can be appended to any pattern to indicate a negative pattern
|
66
|
+
- Negative patterns exclude matching files from the result set
|
67
|
+
- If a pattern should match files starting with `!`, escape the leading `!` with `\` -> `\!filename`
|
68
|
+
6. Patterns will match dotfiles
|
69
|
+
|
70
|
+
**NOTE** If the first, or only, pattern for a dependency is a negative pattern, it will not affect the set of files matched to a dependency. An inclusion pattern should always be specified before any negative patterns for a dependency.
|
71
|
+
|
72
|
+
#### Restrictions for specifying dependency patterns via configuration
|
73
|
+
|
74
|
+
The following restrictions will raise errors if they are not met when specifying manifest dependency patterns in the configuration file
|
75
|
+
|
76
|
+
1. The dependencies key is specified but is empty
|
77
|
+
2. A file in the project is not attributed to any of the configured dependencies
|
78
|
+
- The manifest by default will track all files in the project to ensure that license metadata updates occur when dependencies change
|
79
|
+
- See [Globally excluding files](#globally-excluding-files) to limit the scope of files that are tracked
|
80
|
+
3. A file in the project is attributed to multiple configured dependencies
|
81
|
+
- All files must be tracked by a single dependency
|
82
|
+
|
83
|
+
#### Globally excluding files
|
84
|
+
|
85
|
+
Tracking project files is needed to make sure that any changes to dependencies does not go unnoticed. What about non-dependency code?
|
86
|
+
|
87
|
+
To reduce friction on changes to project code, global exclusion patterns can be added to the configuration that will cause any matching files to **NOT** be tracked as a dependency of the project.
|
88
|
+
|
89
|
+
Global exclusion patterns follow the same rules as [dependency patterns](#pattern-format) and are set on the `manifest.exclude` property.
|
90
|
+
|
91
|
+
The following example excludes all files that are not under the `vendor` folder.
|
92
|
+
|
93
|
+
```yaml
|
94
|
+
manifest:
|
95
|
+
exclude:
|
96
|
+
- "**/*"
|
97
|
+
- "!vendor/**/*"
|
98
|
+
```
|
99
|
+
|
100
|
+
## Finding license content
|
101
|
+
|
102
|
+
### From common source file directories
|
23
103
|
|
24
104
|
If multiple source files map to a single package and they share a common path under the git repository root, that directory will be used to find license information, if available.
|
25
105
|
|
26
|
-
|
106
|
+
### From source file comments
|
107
|
+
|
108
|
+
When a file containing license content is not found for a group of source files,
|
109
|
+
Licensed will attempt to parse license text from source file comments.
|
110
|
+
|
111
|
+
There are some limitations on this functionality:
|
112
|
+
|
113
|
+
1. Comments MUST contain a copyright statement
|
114
|
+
2. Comments MUST be C-style multiline comments, e.g. `/* comment */`
|
115
|
+
3. Comments SHOULD contain identical indentation for each content line.
|
116
|
+
|
117
|
+
The following examples are all :+1:. Licensed will try to preserve formatting,
|
118
|
+
however for best results comments should not mix tabs and spaces in leading whitespace.
|
119
|
+
```
|
120
|
+
/*
|
121
|
+
<copyright statement>
|
122
|
+
|
123
|
+
<license text>
|
124
|
+
*/
|
125
|
+
|
126
|
+
/* <copyright statement>
|
127
|
+
<license text>
|
128
|
+
*/
|
129
|
+
|
130
|
+
/*
|
131
|
+
* <copyright statement>
|
132
|
+
* <license text>
|
133
|
+
*
|
134
|
+
* <license text>
|
135
|
+
*/
|
136
|
+
```
|
137
|
+
|
138
|
+
### From license files specified in the configuration
|
139
|
+
|
140
|
+
If all else fails, the path to a dependency's license files can be specified manually
|
141
|
+
in the configuration. All paths should be relative to the repository root.
|
142
|
+
|
143
|
+
```yaml
|
144
|
+
manifest:
|
145
|
+
licenses:
|
146
|
+
package: path/to/LICENSE
|
147
|
+
```
|
data/lib/licensed.rb
CHANGED
@@ -17,32 +17,3 @@ require "licensed/command/cache"
|
|
17
17
|
require "licensed/command/status"
|
18
18
|
require "licensed/command/list"
|
19
19
|
require "licensed/ui/shell"
|
20
|
-
require "octokit"
|
21
|
-
|
22
|
-
module Licensed
|
23
|
-
class << self
|
24
|
-
attr_accessor :use_github
|
25
|
-
end
|
26
|
-
|
27
|
-
self.use_github = true
|
28
|
-
|
29
|
-
GITHUB_URL = %r{\Ahttps://github.com/([a-z0-9]+(-[a-z0-9]+)*/(\w|\.|\-)+)}
|
30
|
-
LICENSE_CONTENT_TYPE = "application/vnd.github.drax-preview+json"
|
31
|
-
|
32
|
-
# Load license content from a GitHub url. Returns nil if the url does not point
|
33
|
-
# to a GitHub repository, or if the license content is not found
|
34
|
-
def self.from_github(url)
|
35
|
-
return unless use_github && match = GITHUB_URL.match(url)
|
36
|
-
|
37
|
-
license_url = Octokit::Repository.path(match[1]) + "/license"
|
38
|
-
response = octokit.get license_url, accept: LICENSE_CONTENT_TYPE
|
39
|
-
content = Base64.decode64(response["content"]).force_encoding("UTF-8")
|
40
|
-
Licensee::ProjectFiles::LicenseFile.new(content, response["name"])
|
41
|
-
rescue Octokit::NotFound
|
42
|
-
nil
|
43
|
-
end
|
44
|
-
|
45
|
-
def self.octokit
|
46
|
-
@octokit ||= Octokit::Client.new(access_token: ENV["GITHUB_TOKEN"])
|
47
|
-
end
|
48
|
-
end
|
data/lib/licensed/cli.rb
CHANGED
@@ -9,11 +9,10 @@ module Licensed
|
|
9
9
|
method_option :force, type: :boolean,
|
10
10
|
desc: "Overwrite licenses even if version has not changed."
|
11
11
|
method_option :offline, type: :boolean,
|
12
|
-
desc: "
|
12
|
+
desc: "This option is deprecated and will be removed in the next major release."
|
13
13
|
method_option :config, aliases: "-c", type: :string,
|
14
14
|
desc: "Path to licensed configuration file"
|
15
15
|
def cache
|
16
|
-
Licensed.use_github = false if options[:offline]
|
17
16
|
run Licensed::Command::Cache.new(config), force: options[:force]
|
18
17
|
end
|
19
18
|
|
data/lib/licensed/dependency.rb
CHANGED
@@ -9,17 +9,10 @@ module Licensed
|
|
9
9
|
attr_reader :search_root
|
10
10
|
|
11
11
|
def initialize(path, metadata = {})
|
12
|
-
@path = path
|
13
12
|
@search_root = metadata.delete("search_root")
|
14
|
-
|
15
|
-
# with licensee providing license_file[:dir],
|
16
|
-
# enforcing absolute paths makes life much easier when determining
|
17
|
-
# an absolute file path in notices
|
18
|
-
unless Pathname.new(path).absolute?
|
19
|
-
raise "Dependency path #{path} must be absolute"
|
20
|
-
end
|
21
|
-
|
22
13
|
super metadata
|
14
|
+
|
15
|
+
self.path = path
|
23
16
|
end
|
24
17
|
|
25
18
|
# Returns a Licensee::Projects::FSProject for the dependency path
|
@@ -27,12 +20,24 @@ module Licensed
|
|
27
20
|
@project ||= Licensee::Projects::FSProject.new(path, search_root: search_root, detect_packages: true, detect_readme: true)
|
28
21
|
end
|
29
22
|
|
23
|
+
# Sets the path to source dependency license information
|
24
|
+
def path=(path)
|
25
|
+
# enforcing absolute paths makes life much easier when determining
|
26
|
+
# an absolute file path in #notices
|
27
|
+
unless Pathname.new(path).absolute?
|
28
|
+
raise "Dependency path #{path} must be absolute"
|
29
|
+
end
|
30
|
+
|
31
|
+
@path = path
|
32
|
+
reset_license!
|
33
|
+
end
|
34
|
+
|
30
35
|
# Detects license information and sets it on this dependency object.
|
31
36
|
# After calling `detect_license!``, the license is set at
|
32
37
|
# `dependency["license"]` and legal text is set to `dependency.text`
|
33
38
|
def detect_license!
|
34
39
|
self["license"] = license_key
|
35
|
-
self.text = [license_text, *notices].join("\n" + TEXT_SEPARATOR + "\n").
|
40
|
+
self.text = [license_text, *notices].join("\n" + TEXT_SEPARATOR + "\n").rstrip
|
36
41
|
end
|
37
42
|
|
38
43
|
# Extract legal notices from the dependency source
|
@@ -40,7 +45,7 @@ module Licensed
|
|
40
45
|
local_files.uniq # unique local file paths
|
41
46
|
.sort # sorted by the path
|
42
47
|
.map { |f| File.read(f) } # read the file contents
|
43
|
-
.map(&:
|
48
|
+
.map(&:rstrip) # strip whitespace
|
44
49
|
.select { |t| t.length > 0 } # files with content only
|
45
50
|
end
|
46
51
|
|
@@ -60,46 +65,25 @@ module Licensed
|
|
60
65
|
|
61
66
|
private
|
62
67
|
|
63
|
-
#
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
# Returns the Licensee::LicenseFile, Licensee::PackageManagerFile, or
|
70
|
-
# Licensee::ReadmeFile with a matched license, in that order or nil
|
71
|
-
# if no license file matched a known license
|
72
|
-
def matched_project_file
|
73
|
-
@matched_project_file ||= project.matched_files
|
74
|
-
.select { |f| f.license && !f.license.other? }
|
75
|
-
.first
|
76
|
-
end
|
77
|
-
|
78
|
-
# Returns a Licensee::LicenseFile with the content of the license in the
|
79
|
-
# dependency's repository to account for LICENSE files not being distributed
|
80
|
-
def remote_license_file
|
81
|
-
return @remote_license_file if defined?(@remote_license_file)
|
82
|
-
@remote_license_file = Licensed.from_github(self["homepage"])
|
68
|
+
# Resets all local project and license information
|
69
|
+
def reset_license!
|
70
|
+
@project = nil
|
71
|
+
self.delete("license")
|
72
|
+
self.text = nil
|
83
73
|
end
|
84
74
|
|
85
75
|
# Regardless of the license detected, try to pull the license content
|
86
|
-
# from the local LICENSE, remote LICENSE, or the README, in that order
|
76
|
+
# from the local LICENSE-type files, remote LICENSE, or the README, in that order
|
87
77
|
def license_text
|
88
|
-
|
89
|
-
|
78
|
+
content_files = Array(project.license_files)
|
79
|
+
content_files << project.readme_file if content_files.empty? && project.readme_file
|
80
|
+
content_files.map(&:content).join("\n#{LICENSE_SEPARATOR}\n")
|
90
81
|
end
|
91
82
|
|
92
83
|
# Returns a string representing the project's license
|
93
|
-
# Note, this will be "other" if a license file was found but the license
|
94
|
-
# could not be identified and "none" if no license file was found at all
|
95
84
|
def license_key
|
96
|
-
|
97
|
-
|
98
|
-
elsif project.license_file || remote_license_file
|
99
|
-
"other"
|
100
|
-
else
|
101
|
-
"none"
|
102
|
-
end
|
85
|
+
return "none" unless project.license
|
86
|
+
project.license.key
|
103
87
|
end
|
104
88
|
end
|
105
89
|
end
|
data/lib/licensed/git.rb
CHANGED
@@ -34,6 +34,12 @@ module Licensed
|
|
34
34
|
return unless available? && sha
|
35
35
|
Licensed::Shell.execute("git", "show", "-s", "-1", "--format=%ct", sha)
|
36
36
|
end
|
37
|
+
|
38
|
+
def files
|
39
|
+
return unless available?
|
40
|
+
output = Licensed::Shell.execute("git", "ls-tree", "--full-tree", "-r", "--name-only", "HEAD")
|
41
|
+
output.lines.map(&:strip)
|
42
|
+
end
|
37
43
|
end
|
38
44
|
end
|
39
45
|
end
|
data/lib/licensed/license.rb
CHANGED
@@ -11,6 +11,7 @@ module Licensed
|
|
11
11
|
|
12
12
|
YAML_FRONTMATTER_PATTERN = /\A---\s*\n(.*?\n?)^---\s*$\n?(.*)\z/m
|
13
13
|
TEXT_SEPARATOR = ("-" * 80).freeze
|
14
|
+
LICENSE_SEPARATOR = ("*" * 80).freeze
|
14
15
|
|
15
16
|
# Read an existing license file
|
16
17
|
#
|
@@ -23,7 +24,7 @@ module Licensed
|
|
23
24
|
new(YAML.load(match[1]), match[2])
|
24
25
|
end
|
25
26
|
|
26
|
-
def_delegators :@metadata, :[], :[]
|
27
|
+
def_delegators :@metadata, :[], :[]=, :delete
|
27
28
|
|
28
29
|
# The license text and other legal notices
|
29
30
|
attr_accessor :text
|
@@ -53,7 +54,7 @@ module Licensed
|
|
53
54
|
# if the text didn't contain the separator, the text itself is the entirety
|
54
55
|
# of the license text
|
55
56
|
split = text.split(TEXT_SEPARATOR)
|
56
|
-
split.length > 1 ? split.first.
|
57
|
+
split.length > 1 ? split.first.rstrip : text.rstrip
|
57
58
|
end
|
58
59
|
alias_method :content, :license_text # use license_text for content matching
|
59
60
|
|
@@ -94,6 +94,13 @@ module Licensed
|
|
94
94
|
return spec.source.specs.first
|
95
95
|
end
|
96
96
|
|
97
|
+
# spec.source.specs gives access to specifications with more
|
98
|
+
# information than spec itself, including platform-specific gems.
|
99
|
+
# try to find a specification that matches `spec`
|
100
|
+
if source_spec = spec.source.specs.find { |s| s.name == spec.name && s.version == spec.version }
|
101
|
+
spec = source_spec
|
102
|
+
end
|
103
|
+
|
97
104
|
# look for a specification at the bundler specs path
|
98
105
|
spec_path = ::Bundler.specs_path.join("#{spec.full_name}.gemspec")
|
99
106
|
return unless File.exist?(spec_path.to_s)
|
@@ -13,32 +13,22 @@ module Licensed
|
|
13
13
|
end
|
14
14
|
|
15
15
|
def enabled?
|
16
|
-
File.exist?(manifest_path)
|
16
|
+
File.exist?(manifest_path) || generate_manifest?
|
17
17
|
end
|
18
18
|
|
19
19
|
def dependencies
|
20
20
|
@dependencies ||= packages.map do |package_name, sources|
|
21
|
-
Dependency.new(
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
21
|
+
Licensed::Source::Manifest::Dependency.new(sources,
|
22
|
+
package_license(package_name),
|
23
|
+
{
|
24
|
+
"type" => Manifest.type,
|
25
|
+
"name" => package_name,
|
26
|
+
"version" => package_version(sources)
|
27
|
+
}
|
28
|
+
)
|
26
29
|
end
|
27
30
|
end
|
28
31
|
|
29
|
-
# Returns the top-most directory that is common to all paths in `sources`
|
30
|
-
def sources_license_path(sources)
|
31
|
-
common_prefix = Pathname.common_prefix(*sources).to_path
|
32
|
-
|
33
|
-
# don't allow the repo root to be used as common prefix
|
34
|
-
# the project this is run for should be excluded from the manifest,
|
35
|
-
# or ignored in the config. any license in the root should be ignored.
|
36
|
-
return common_prefix if common_prefix != Licensed::Git.repository_root
|
37
|
-
|
38
|
-
# use the first source file as the license path.
|
39
|
-
sources.first
|
40
|
-
end
|
41
|
-
|
42
32
|
# Returns the latest git SHA available from `sources`
|
43
33
|
def package_version(sources)
|
44
34
|
return if sources.nil? || sources.empty?
|
@@ -48,6 +38,16 @@ module Licensed
|
|
48
38
|
.max_by { |sha| Licensed::Git.commit_date(sha) }
|
49
39
|
end
|
50
40
|
|
41
|
+
# Returns the license path for a package specified in the configuration.
|
42
|
+
def package_license(package_name)
|
43
|
+
license_path = @config.dig("manifest", "licenses", package_name)
|
44
|
+
return unless license_path
|
45
|
+
|
46
|
+
license_path = Licensed::Git.repository_root.join(license_path)
|
47
|
+
return unless license_path.exist?
|
48
|
+
license_path
|
49
|
+
end
|
50
|
+
|
51
51
|
# Returns a map of package names -> array of full source paths found
|
52
52
|
# in the app manifest
|
53
53
|
def packages
|
@@ -58,8 +58,10 @@ module Licensed
|
|
58
58
|
end
|
59
59
|
end
|
60
60
|
|
61
|
-
# Returns parsed manifest data for the app
|
61
|
+
# Returns parsed or generated manifest data for the app
|
62
62
|
def manifest
|
63
|
+
return generate_manifest if generate_manifest?
|
64
|
+
|
63
65
|
case manifest_path.extname.downcase.delete "."
|
64
66
|
when "json"
|
65
67
|
JSON.parse(File.read(manifest_path))
|
@@ -70,11 +72,193 @@ module Licensed
|
|
70
72
|
|
71
73
|
# Returns the manifest location for the app
|
72
74
|
def manifest_path
|
73
|
-
path = @config
|
75
|
+
path = @config.dig("manifest", "path")
|
74
76
|
return Licensed::Git.repository_root.join(path) if path
|
75
77
|
|
76
78
|
@config.cache_path.join("manifest.json")
|
77
79
|
end
|
80
|
+
|
81
|
+
# Returns whether a manifest should be generated automatically
|
82
|
+
def generate_manifest?
|
83
|
+
!File.exist?(manifest_path) && !@config.dig("manifest", "dependencies").nil?
|
84
|
+
end
|
85
|
+
|
86
|
+
# Returns a manifest of files generated automatically based on patterns
|
87
|
+
# set in the licensed configuration file
|
88
|
+
def generate_manifest
|
89
|
+
verify_configured_dependencies!
|
90
|
+
configured_dependencies.each_with_object({}) do |(name, files), hsh|
|
91
|
+
files.each { |f| hsh[f] = name }
|
92
|
+
end
|
93
|
+
end
|
94
|
+
|
95
|
+
# Verify that the licensed configuration file is valid for the current project.
|
96
|
+
# Raises errors for issues found with configuration
|
97
|
+
def verify_configured_dependencies!
|
98
|
+
# verify that dependencies are configured
|
99
|
+
if configured_dependencies.empty?
|
100
|
+
raise "The manifest \"dependencies\" cannot be empty!"
|
101
|
+
end
|
102
|
+
|
103
|
+
# verify all included files match a single configured dependency
|
104
|
+
errors = included_files.map do |file|
|
105
|
+
matches = configured_dependencies.select { |name, files| files.include?(file) }
|
106
|
+
.map { |name, files| name }
|
107
|
+
case matches.size
|
108
|
+
when 0
|
109
|
+
"#{file} did not match a configured dependency"
|
110
|
+
when 1
|
111
|
+
nil
|
112
|
+
else
|
113
|
+
"#{file} matched multiple configured dependencies: #{matches.join(", ")}"
|
114
|
+
end
|
115
|
+
end
|
116
|
+
|
117
|
+
errors.compact!
|
118
|
+
raise errors.join($/) unless errors.empty?
|
119
|
+
end
|
120
|
+
|
121
|
+
# Returns the project dependencies specified from the licensed configuration
|
122
|
+
def configured_dependencies
|
123
|
+
@configured_dependencies ||= begin
|
124
|
+
dependencies = @config.dig("manifest", "dependencies")&.dup || {}
|
125
|
+
|
126
|
+
dependencies.each do |name, patterns|
|
127
|
+
# map glob pattern(s) listed for the dependency to a listing
|
128
|
+
# of files that match the patterns and are not excluded
|
129
|
+
dependencies[name] = files_from_pattern_list(patterns) & included_files
|
130
|
+
end
|
131
|
+
|
132
|
+
dependencies
|
133
|
+
end
|
134
|
+
end
|
135
|
+
|
136
|
+
# Returns the set of project files that are included in dependency evaluation
|
137
|
+
def included_files
|
138
|
+
@sources ||= all_files - files_from_pattern_list(@config.dig("manifest", "exclude"))
|
139
|
+
end
|
140
|
+
|
141
|
+
# Finds and returns all files in the project that match
|
142
|
+
# the glob pattern arguments.
|
143
|
+
def files_from_pattern_list(patterns)
|
144
|
+
return Set.new if patterns.nil? || patterns.empty?
|
145
|
+
|
146
|
+
# evaluate all patterns from the project root
|
147
|
+
Dir.chdir Licensed::Git.repository_root do
|
148
|
+
Array(patterns).reduce(Set.new) do |files, pattern|
|
149
|
+
if pattern.start_with?("!")
|
150
|
+
# if the pattern is an exclusion, remove all matching files
|
151
|
+
# from the result
|
152
|
+
files - Dir.glob(pattern[1..-1], File::FNM_DOTMATCH)
|
153
|
+
else
|
154
|
+
# if the pattern is an inclusion, add all matching files
|
155
|
+
# to the result
|
156
|
+
files + Dir.glob(pattern, File::FNM_DOTMATCH)
|
157
|
+
end
|
158
|
+
end
|
159
|
+
end
|
160
|
+
end
|
161
|
+
|
162
|
+
# Returns all tracked files in the project
|
163
|
+
def all_files
|
164
|
+
# remove files if they are tracked but don't exist on the file system
|
165
|
+
@all_files ||= Set.new(Licensed::Git.files || [])
|
166
|
+
.delete_if { |f| !File.exist?(f) }
|
167
|
+
end
|
168
|
+
|
169
|
+
class Dependency < Licensed::Dependency
|
170
|
+
ANY_EXCEPT_COMMENT_CLOSE_REGEX = /(\*(?!\/)|[^\*])*/m.freeze
|
171
|
+
HEADER_LICENSE_REGEX = /
|
172
|
+
(
|
173
|
+
\/\*
|
174
|
+
#{ANY_EXCEPT_COMMENT_CLOSE_REGEX}#{Licensee::Matchers::Copyright::COPYRIGHT_SYMBOLS}#{ANY_EXCEPT_COMMENT_CLOSE_REGEX}
|
175
|
+
\*\/
|
176
|
+
)
|
177
|
+
/imx.freeze
|
178
|
+
|
179
|
+
def initialize(sources, license_path, metadata = {})
|
180
|
+
@sources = sources
|
181
|
+
license_path ||= sources_license_path(sources)
|
182
|
+
super license_path, metadata
|
183
|
+
end
|
184
|
+
|
185
|
+
# Detects license information and sets it on this dependency object.
|
186
|
+
# After calling `detect_license!``, the license is set at
|
187
|
+
# `dependency["license"]` and legal text is set to `dependency.text`
|
188
|
+
def detect_license!
|
189
|
+
# if no license key is found for the project, try to create a
|
190
|
+
# temporary LICENSE file from unique source file license headers
|
191
|
+
if license_key == "none"
|
192
|
+
tmp_license_file = write_license_from_source_licenses(self.path, @sources)
|
193
|
+
reset_license!
|
194
|
+
end
|
195
|
+
|
196
|
+
super
|
197
|
+
ensure
|
198
|
+
File.delete(tmp_license_file) if tmp_license_file && File.exist?(tmp_license_file)
|
199
|
+
end
|
200
|
+
|
201
|
+
private
|
202
|
+
|
203
|
+
# Returns the top-most directory that is common to all paths in `sources`
|
204
|
+
def sources_license_path(sources)
|
205
|
+
# return the source directory if there is only one source given
|
206
|
+
return source_directory(sources[0]) if sources.size == 1
|
207
|
+
|
208
|
+
common_prefix = Pathname.common_prefix(*sources).to_path
|
209
|
+
|
210
|
+
# don't allow the repo root to be used as common prefix
|
211
|
+
# the project this is run for should be excluded from the manifest,
|
212
|
+
# or ignored in the config. any license in the root should be ignored.
|
213
|
+
return common_prefix if common_prefix != Licensed::Git.repository_root
|
214
|
+
|
215
|
+
# use the first source directory as the license path.
|
216
|
+
source_directory(sources.first)
|
217
|
+
end
|
218
|
+
|
219
|
+
# Returns the directory for the source. Checks whether the source
|
220
|
+
# is a file or a directory
|
221
|
+
def source_directory(source)
|
222
|
+
return File.dirname(source) if File.file?(source)
|
223
|
+
source
|
224
|
+
end
|
225
|
+
|
226
|
+
# Writes any licenses found in source file comments to a LICENSE
|
227
|
+
# file at `dir`
|
228
|
+
# Returns the path to the license file
|
229
|
+
def write_license_from_source_licenses(dir, sources)
|
230
|
+
license_path = File.join(dir, "LICENSE")
|
231
|
+
File.open(license_path, "w") do |f|
|
232
|
+
licenses = source_comment_licenses(sources).uniq
|
233
|
+
f.puts(licenses.join("\n#{LICENSE_SEPARATOR}\n"))
|
234
|
+
end
|
235
|
+
|
236
|
+
license_path
|
237
|
+
end
|
238
|
+
|
239
|
+
# Returns a list of unique licenses parsed from source comments
|
240
|
+
def source_comment_licenses(sources)
|
241
|
+
comments = sources.select { |s| File.file?(s) }.flat_map do |source|
|
242
|
+
content = File.read(source)
|
243
|
+
content.scan(HEADER_LICENSE_REGEX).map { |match| match[0] }
|
244
|
+
end
|
245
|
+
|
246
|
+
comments.map do |comment|
|
247
|
+
# strip leading "*" and whitespace
|
248
|
+
indent = nil
|
249
|
+
comment.lines.map do |line|
|
250
|
+
# find the length of the indent as the number of characters
|
251
|
+
# until the first word character
|
252
|
+
indent ||= line[/\A([^\w]*)\w/, 1]&.size
|
253
|
+
|
254
|
+
# insert newline for each line until a word character is found
|
255
|
+
next "\n" unless indent
|
256
|
+
|
257
|
+
line[/([^\w\r\n]{0,#{indent}})(.*)/m, 2]
|
258
|
+
end.join
|
259
|
+
end
|
260
|
+
end
|
261
|
+
end
|
78
262
|
end
|
79
263
|
end
|
80
264
|
end
|
data/lib/licensed/source/npm.rb
CHANGED
@@ -19,16 +19,10 @@ module Licensed
|
|
19
19
|
def dependencies
|
20
20
|
return @dependencies if defined?(@dependencies)
|
21
21
|
|
22
|
-
locations = {}
|
23
|
-
package_location_command.lines.each do |line|
|
24
|
-
path, id = line.split(":")[0, 2]
|
25
|
-
locations[id] ||= path
|
26
|
-
end
|
27
|
-
|
28
22
|
packages = recursive_dependencies(JSON.parse(package_metadata_command)["dependencies"])
|
29
23
|
|
30
24
|
@dependencies = packages.map do |name, package|
|
31
|
-
path = package["
|
25
|
+
path = package["path"]
|
32
26
|
fail "couldn't locate #{name} under node_modules/" unless path
|
33
27
|
Dependency.new(path, {
|
34
28
|
"type" => NPM.type,
|
@@ -50,11 +44,6 @@ module Licensed
|
|
50
44
|
result
|
51
45
|
end
|
52
46
|
|
53
|
-
# Returns the output from running `npm list` to get package paths
|
54
|
-
def package_location_command
|
55
|
-
npm_list_command("--parseable", "--production", "--long")
|
56
|
-
end
|
57
|
-
|
58
47
|
# Returns the output from running `npm list` to get package metadata
|
59
48
|
def package_metadata_command
|
60
49
|
npm_list_command("--json", "--production", "--long")
|
data/lib/licensed/version.rb
CHANGED
data/licensed.gemspec
CHANGED
@@ -25,15 +25,12 @@ Gem::Specification.new do |spec|
|
|
25
25
|
|
26
26
|
spec.add_dependency "licensee", "~> 9.0"
|
27
27
|
spec.add_dependency "thor", "~>0.19"
|
28
|
-
spec.add_dependency "octokit", "~>4.0"
|
29
28
|
spec.add_dependency "pathname-common_prefix", "~>0.0.1"
|
30
29
|
spec.add_dependency "tomlrb", "~>1.2"
|
31
30
|
spec.add_dependency "bundler", "~> 1.10"
|
32
31
|
|
33
32
|
spec.add_development_dependency "rake", "~> 10.0"
|
34
33
|
spec.add_development_dependency "minitest", "~> 5.8"
|
35
|
-
spec.add_development_dependency "vcr", "~> 2.9"
|
36
|
-
spec.add_development_dependency "webmock", "~> 1.21"
|
37
34
|
spec.add_development_dependency "rubocop", "~> 0.49"
|
38
35
|
spec.add_development_dependency "rubocop-github", "~> 0.6"
|
39
36
|
spec.add_development_dependency "byebug", "~> 10.0.0"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: licensed
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- GitHub
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2018-
|
11
|
+
date: 2018-07-25 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: licensee
|
@@ -38,20 +38,6 @@ dependencies:
|
|
38
38
|
- - "~>"
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '0.19'
|
41
|
-
- !ruby/object:Gem::Dependency
|
42
|
-
name: octokit
|
43
|
-
requirement: !ruby/object:Gem::Requirement
|
44
|
-
requirements:
|
45
|
-
- - "~>"
|
46
|
-
- !ruby/object:Gem::Version
|
47
|
-
version: '4.0'
|
48
|
-
type: :runtime
|
49
|
-
prerelease: false
|
50
|
-
version_requirements: !ruby/object:Gem::Requirement
|
51
|
-
requirements:
|
52
|
-
- - "~>"
|
53
|
-
- !ruby/object:Gem::Version
|
54
|
-
version: '4.0'
|
55
41
|
- !ruby/object:Gem::Dependency
|
56
42
|
name: pathname-common_prefix
|
57
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -122,34 +108,6 @@ dependencies:
|
|
122
108
|
- - "~>"
|
123
109
|
- !ruby/object:Gem::Version
|
124
110
|
version: '5.8'
|
125
|
-
- !ruby/object:Gem::Dependency
|
126
|
-
name: vcr
|
127
|
-
requirement: !ruby/object:Gem::Requirement
|
128
|
-
requirements:
|
129
|
-
- - "~>"
|
130
|
-
- !ruby/object:Gem::Version
|
131
|
-
version: '2.9'
|
132
|
-
type: :development
|
133
|
-
prerelease: false
|
134
|
-
version_requirements: !ruby/object:Gem::Requirement
|
135
|
-
requirements:
|
136
|
-
- - "~>"
|
137
|
-
- !ruby/object:Gem::Version
|
138
|
-
version: '2.9'
|
139
|
-
- !ruby/object:Gem::Dependency
|
140
|
-
name: webmock
|
141
|
-
requirement: !ruby/object:Gem::Requirement
|
142
|
-
requirements:
|
143
|
-
- - "~>"
|
144
|
-
- !ruby/object:Gem::Version
|
145
|
-
version: '1.21'
|
146
|
-
type: :development
|
147
|
-
prerelease: false
|
148
|
-
version_requirements: !ruby/object:Gem::Requirement
|
149
|
-
requirements:
|
150
|
-
- - "~>"
|
151
|
-
- !ruby/object:Gem::Version
|
152
|
-
version: '1.21'
|
153
111
|
- !ruby/object:Gem::Dependency
|
154
112
|
name: rubocop
|
155
113
|
requirement: !ruby/object:Gem::Requirement
|