license_scout 2.2.0 → 2.6.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +26 -5
- data/bin/license_scout +1 -1
- data/lib/license_scout/cli.rb +9 -4
- data/lib/license_scout/collector.rb +1 -1
- data/lib/license_scout/config.rb +22 -2
- data/lib/license_scout/dependency.rb +1 -1
- data/lib/license_scout/dependency_manager.rb +4 -0
- data/lib/license_scout/dependency_manager/base.rb +7 -6
- data/lib/license_scout/dependency_manager/cargo.rb +95 -0
- data/lib/license_scout/dependency_manager/dep.rb +1 -1
- data/lib/license_scout/dependency_manager/glide.rb +1 -1
- data/lib/license_scout/dependency_manager/godep.rb +1 -1
- data/lib/license_scout/dependency_manager/gomod.rb +113 -0
- data/lib/license_scout/dependency_manager/habitat.rb +55 -41
- data/lib/license_scout/exporter/csv.rb +2 -2
- data/lib/license_scout/license.rb +2 -2
- data/lib/license_scout/reporter.rb +1 -1
- data/lib/license_scout/spdx.rb +6 -3
- data/lib/license_scout/version.rb +1 -1
- metadata +5 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f8ff75c46c04b580099b10f59088cde2bbf10e6131e13daf266f8c107faf9629
|
4
|
+
data.tar.gz: c96cbd17eb105ba96b663d37f3dbd8e3b589da387c5339a655fb612a9d975a92
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 30cd04254f9d26f7b613bd3f22274494ad0c62cbd49131684404b4a1d245c6ff439b0a55182dbe6b05caeb2a5bea22b7cd223c830299b57f6019578bc16cd492
|
7
|
+
data.tar.gz: 88ca96ac3db1fb0cd74029a9b7f5724b481d5cbcb0ec7eade95db1a0ae063c093e918d2eabf7c261d39bc99a25b64bb32a64b5bfef83f8b037f6f2e91c0dcfd0
|
data/README.md
CHANGED
@@ -9,11 +9,12 @@ Dependency Type | Supported Dependency Managers
|
|
9
9
|
chef_cookbook | berkshelf
|
10
10
|
erlang | rebar
|
11
11
|
elixir | mix
|
12
|
-
golang | dep, godep, glide
|
12
|
+
golang | modules, dep, godep, glide
|
13
13
|
habitat | habitat
|
14
14
|
nodejs | npm
|
15
15
|
perl | cpan
|
16
16
|
ruby | bundler
|
17
|
+
rust | cargo
|
17
18
|
|
18
19
|
## Installation
|
19
20
|
|
@@ -27,6 +28,8 @@ gem install license_scout
|
|
27
28
|
|
28
29
|
* If you wish to scan for `berkshelf` dependencies, you'll need to manually install the Berkshelf gem in the same Ruby as License Scout
|
29
30
|
* If you wish to scan for `mix` or `rebar` dependencies, you'll need to install Erlang OTP 18.3 or greater.
|
31
|
+
* If you wish to scan for `cargo` dependencies, you'll need to manually install cargo
|
32
|
+
* If you wish to scan for `go mod` dependencies, you'll need to manually install go
|
30
33
|
|
31
34
|
### Habitat
|
32
35
|
|
@@ -179,12 +182,21 @@ license_content | A URL to a file where the raw text of the license can be downl
|
|
179
182
|
|
180
183
|
In addition to including any files Licensee identified as potential license files (but couldn't identify), License Scout will also include the Fallback License you specified in the Dependency Manifest.
|
181
184
|
|
185
|
+
### Searching Nested Subdirectories
|
186
|
+
|
187
|
+
License Scout's default behavior is to only look for dependency manager files in the root of the `directories` that you configure. This default behavior provides greater control over the dependencies that you want to appear in your report. For example, you may not want to enforce license acceptance on an internal-only tool that is included in a project.
|
188
|
+
|
189
|
+
License Scout will also scan subdirectories for all dependency manager files and generate a full report on all dependencies that the project uses. To do this, either specify the `--include-sub-directories` command line flag, or set `include_subdirectories` to true in your configuration file.
|
190
|
+
|
191
|
+
A common use case for this functionality is to run `license_scout` from the root of a project and get a full report for that project.
|
192
|
+
|
193
|
+
```
|
194
|
+
license_scout --include-sub-directories
|
195
|
+
```
|
196
|
+
|
182
197
|
## Habitat Channel Configuration
|
183
198
|
|
184
|
-
By default License Scout searches for Habitat package in the `stable`
|
185
|
-
channel. If your build process publishes packages to another channel
|
186
|
-
by default, you can use the `channel_for_origin` habitat configuration
|
187
|
-
option:
|
199
|
+
By default License Scout searches for Habitat package in the `stable` channel. If your build process publishes packages to another channel by default, you can use the `channel_for_origin` habitat configuration option:
|
188
200
|
|
189
201
|
```yaml
|
190
202
|
habitat:
|
@@ -216,6 +228,7 @@ Format | Description
|
|
216
228
|
Value | Description | Default
|
217
229
|
--- | --- | ---
|
218
230
|
directories | The fully-qualified local paths to the directories you wish to scan | _The current working directory._ |
|
231
|
+
include_subdirectories | Whether or not to include all nested sub-directories of `directories` in the search. | `false` |
|
219
232
|
name | The name you want to give to the scan result. | _The basename of the first directory to be scanned._ |
|
220
233
|
output_directory | The path to the directory where the output JSON file should be saved. | _The current working directory._ |
|
221
234
|
log_level | What log information should be included in STDOUT | `info` |
|
@@ -235,6 +248,14 @@ https://github.com/chef/chef/blob/master/CONTRIBUTING.md
|
|
235
248
|
|
236
249
|
Pull requests in this project are merged when they have two :+1:s from maintainers.
|
237
250
|
|
251
|
+
## GitHub Tokens
|
252
|
+
|
253
|
+
If you wish to scan private GitHub repositories or are hitting API rate limits, [create a GitHub token](https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line) and set it to this environmental variable:
|
254
|
+
|
255
|
+
```
|
256
|
+
OCTOKIT_ACCESS_TOKEN=your_token_value
|
257
|
+
```
|
258
|
+
|
238
259
|
## Maintainers
|
239
260
|
|
240
261
|
- [Dan DeLeo](https://github.com/danielsdeleo)
|
data/bin/license_scout
CHANGED
data/lib/license_scout/cli.rb
CHANGED
@@ -15,9 +15,9 @@
|
|
15
15
|
# limitations under the License.
|
16
16
|
#
|
17
17
|
|
18
|
-
require "zlib" # Temporarily require before rugged to fix https://github.com/prontolabs/pronto/issues/23
|
18
|
+
require "zlib" unless defined?(Zlib) # Temporarily require before rugged to fix https://github.com/prontolabs/pronto/issues/23
|
19
19
|
|
20
|
-
require "mixlib/cli"
|
20
|
+
require "mixlib/cli" unless defined?(Mixlib::CLI)
|
21
21
|
require "license_scout/config"
|
22
22
|
require "license_scout/exporter"
|
23
23
|
require "license_scout/collector"
|
@@ -42,6 +42,11 @@ module LicenseScout
|
|
42
42
|
description: "Comma-separated list of directories to scan",
|
43
43
|
proc: Proc.new { |d| d.split(",") }
|
44
44
|
|
45
|
+
option :include_subdirectories,
|
46
|
+
long: "--include-sub-directories",
|
47
|
+
description: "Include all sub-directories of 'directories' in the analysis",
|
48
|
+
boolean: true
|
49
|
+
|
45
50
|
option :format,
|
46
51
|
long: "--format FORMAT",
|
47
52
|
description: "When exporting a Dependency Manifest, export to this format",
|
@@ -52,7 +57,7 @@ module LicenseScout
|
|
52
57
|
short: "-l LEVEL",
|
53
58
|
long: "--log-level LEVEL",
|
54
59
|
description: "Set the log level",
|
55
|
-
in:
|
60
|
+
in: %i{debug info warn error fatal},
|
56
61
|
default: :info,
|
57
62
|
proc: Proc.new { |l| l.to_sym }
|
58
63
|
|
@@ -80,7 +85,7 @@ module LicenseScout
|
|
80
85
|
|
81
86
|
LicenseScout::Config.config_files.each do |config_file|
|
82
87
|
if config_file =~ /^http/
|
83
|
-
require "open-uri"
|
88
|
+
require "open-uri" unless defined?(OpenURI)
|
84
89
|
|
85
90
|
LicenseScout::Log.info("[cli] Loading config from #{config_file}")
|
86
91
|
|
@@ -57,7 +57,7 @@ module LicenseScout
|
|
57
57
|
end
|
58
58
|
|
59
59
|
def dependency_managers
|
60
|
-
@dependency_managers ||= LicenseScout::Config.
|
60
|
+
@dependency_managers ||= LicenseScout::Config.all_directories.map do |dir|
|
61
61
|
LicenseScout::DependencyManager.implementations.map do |implementation|
|
62
62
|
dep_mgr = implementation.new(File.expand_path(dir))
|
63
63
|
if dep_mgr.detected?
|
data/lib/license_scout/config.rb
CHANGED
@@ -15,8 +15,8 @@
|
|
15
15
|
# limitations under the License.
|
16
16
|
#
|
17
17
|
|
18
|
-
require "mixlib/config"
|
19
|
-
require "tmpdir"
|
18
|
+
require "mixlib/config" unless defined?(Mixlib::Config)
|
19
|
+
require "tmpdir" unless defined?(Dir.mktmpdir)
|
20
20
|
|
21
21
|
require "license_scout/exceptions"
|
22
22
|
require "license_scout/log"
|
@@ -28,6 +28,7 @@ module LicenseScout
|
|
28
28
|
|
29
29
|
# Inputs
|
30
30
|
default :directories, [File.expand_path(Dir.pwd)]
|
31
|
+
default :include_subdirectories, false
|
31
32
|
default :name, File.basename(directories.first)
|
32
33
|
default :config_files, [File.join(File.expand_path(Dir.pwd), ".license_scout.yml")]
|
33
34
|
|
@@ -49,6 +50,7 @@ module LicenseScout
|
|
49
50
|
default :nodejs, []
|
50
51
|
default :perl, []
|
51
52
|
default :ruby, []
|
53
|
+
default :rust, []
|
52
54
|
end
|
53
55
|
|
54
56
|
config_context :fallbacks do
|
@@ -60,6 +62,7 @@ module LicenseScout
|
|
60
62
|
default :nodejs, []
|
61
63
|
default :perl, []
|
62
64
|
default :ruby, []
|
65
|
+
default :rust, []
|
63
66
|
end
|
64
67
|
|
65
68
|
config_context :habitat do
|
@@ -79,6 +82,23 @@ module LicenseScout
|
|
79
82
|
|
80
83
|
class << self
|
81
84
|
|
85
|
+
def all_directories
|
86
|
+
if include_subdirectories
|
87
|
+
new_directories = []
|
88
|
+
|
89
|
+
directories.each do |old_directory|
|
90
|
+
new_directories << old_directory
|
91
|
+
Dir.chdir(old_directory) do
|
92
|
+
new_directories << Dir.glob("**/*").select { |f| File.directory?(f) }.map { |d| File.join(old_directory, d) }
|
93
|
+
end
|
94
|
+
end
|
95
|
+
|
96
|
+
new_directories.flatten.compact
|
97
|
+
else
|
98
|
+
directories
|
99
|
+
end
|
100
|
+
end
|
101
|
+
|
82
102
|
def validate!
|
83
103
|
if !allowed_licenses.empty? && !flagged_licenses.empty?
|
84
104
|
raise LicenseScout::Exceptions::ConfigError.new("You may specify a list of licenses to allow or flag. You may not specify both.")
|
@@ -94,7 +94,7 @@ module LicenseScout
|
|
94
94
|
|
95
95
|
# @return [Boolean] Whether or not this object is equal to another one. Used for Set uniqueness.
|
96
96
|
def eql?(other)
|
97
|
-
other.
|
97
|
+
other.is_a?(self.class) && other.hash == hash
|
98
98
|
end
|
99
99
|
|
100
100
|
# @return [Integer] A hashcode that can be used to idenitfy this object. Used for Set uniqueness.
|
@@ -19,10 +19,12 @@ require "license_scout/dependency_manager/base"
|
|
19
19
|
|
20
20
|
require "license_scout/dependency_manager/berkshelf"
|
21
21
|
require "license_scout/dependency_manager/bundler"
|
22
|
+
require "license_scout/dependency_manager/cargo"
|
22
23
|
require "license_scout/dependency_manager/cpanm"
|
23
24
|
require "license_scout/dependency_manager/dep"
|
24
25
|
require "license_scout/dependency_manager/glide"
|
25
26
|
require "license_scout/dependency_manager/godep"
|
27
|
+
require "license_scout/dependency_manager/gomod"
|
26
28
|
require "license_scout/dependency_manager/habitat"
|
27
29
|
require "license_scout/dependency_manager/mix"
|
28
30
|
require "license_scout/dependency_manager/rebar"
|
@@ -34,10 +36,12 @@ module LicenseScout
|
|
34
36
|
[
|
35
37
|
Berkshelf,
|
36
38
|
Bundler,
|
39
|
+
Cargo,
|
37
40
|
Cpanm,
|
38
41
|
Dep,
|
39
42
|
Glide,
|
40
43
|
Godep,
|
44
|
+
Gomod,
|
41
45
|
Habitat,
|
42
46
|
Mix,
|
43
47
|
Rebar,
|
@@ -20,14 +20,14 @@ require "license_scout/dependency"
|
|
20
20
|
require "license_scout/exceptions"
|
21
21
|
|
22
22
|
require "bundler"
|
23
|
-
require "ffi_yajl"
|
24
|
-
require "net/http"
|
25
|
-
require "mixlib/shellout"
|
26
|
-
require "pathname"
|
23
|
+
require "ffi_yajl" unless defined?(FFI_Yajl)
|
24
|
+
require "net/http" unless defined?(Net::HTTP)
|
25
|
+
require "mixlib/shellout" unless defined?(Mixlib::ShellOut)
|
26
|
+
require "pathname" unless defined?(Pathname)
|
27
27
|
require "psych"
|
28
|
-
require "set"
|
28
|
+
require "set" unless defined?(Set)
|
29
29
|
require "toml-rb"
|
30
|
-
require "yaml"
|
30
|
+
require "yaml" unless defined?(YAML)
|
31
31
|
|
32
32
|
module LicenseScout
|
33
33
|
# The DependencyManager module (or more accurately, implementations of it) are responsible for recognizing
|
@@ -51,6 +51,7 @@ module LicenseScout
|
|
51
51
|
# @example Go's various package managers
|
52
52
|
# Name Reference
|
53
53
|
# -------- -----------------------------------------------
|
54
|
+
# go_mod [`gomod`](https://golang.org/cmd/go/#hdr-The_go_mod_file)
|
54
55
|
# go_dep [`godep`](https://github.com/tools/godep)
|
55
56
|
# go_godep [`dep`](https://github.com/golang/dep)
|
56
57
|
# go_glide [`glide`](https://github.com/Masterminds/glide)
|
@@ -0,0 +1,95 @@
|
|
1
|
+
#
|
2
|
+
# Copyright:: Copyright 2016, Chef Software Inc.
|
3
|
+
# License:: Apache License, Version 2.0
|
4
|
+
#
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
+
# you may not use this file except in compliance with the License.
|
7
|
+
# You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
#
|
17
|
+
|
18
|
+
require "license_scout/dependency_manager/base"
|
19
|
+
|
20
|
+
module LicenseScout
|
21
|
+
module DependencyManager
|
22
|
+
class Cargo < Base
|
23
|
+
def name
|
24
|
+
"rust_cargo"
|
25
|
+
end
|
26
|
+
|
27
|
+
def type
|
28
|
+
"rust"
|
29
|
+
end
|
30
|
+
|
31
|
+
def signature
|
32
|
+
"Cargo and Cargo.lock files"
|
33
|
+
end
|
34
|
+
|
35
|
+
def install_command
|
36
|
+
"cargo build"
|
37
|
+
end
|
38
|
+
|
39
|
+
def detected?
|
40
|
+
File.exist?(cargo_file_path) && File.exist?(cargo_lockfile_path)
|
41
|
+
end
|
42
|
+
|
43
|
+
def dependencies
|
44
|
+
dependency_data.map do |crate_data|
|
45
|
+
dep_name = crate_data["name"]
|
46
|
+
dep_version = crate_data["version"]
|
47
|
+
dep_license = crate_data["license"]
|
48
|
+
|
49
|
+
dependency = new_dependency(dep_name, dep_version, nil)
|
50
|
+
dependency.add_license(dep_license, "https://crates.io/crates/#{dep_name}/#{dep_version}")
|
51
|
+
|
52
|
+
dependency
|
53
|
+
end.compact
|
54
|
+
end
|
55
|
+
|
56
|
+
private
|
57
|
+
|
58
|
+
def dependency_data
|
59
|
+
Dir.chdir(directory) do
|
60
|
+
install_cargo_license_crate
|
61
|
+
|
62
|
+
s = Mixlib::ShellOut.new("cargo-license -d -j")
|
63
|
+
s.run_command
|
64
|
+
s.error!
|
65
|
+
|
66
|
+
json_dep_data = s.stdout
|
67
|
+
FFI_Yajl::Parser.parse(json_dep_data)
|
68
|
+
end
|
69
|
+
end
|
70
|
+
|
71
|
+
def install_cargo_license_crate
|
72
|
+
# Attempt to install cargo-license
|
73
|
+
s = Mixlib::ShellOut.new("cargo install cargo-license")
|
74
|
+
s.run_command
|
75
|
+
|
76
|
+
# If cargo-license is already installed, it will return an error
|
77
|
+
# but we can ignore it
|
78
|
+
# Any other error, however, should halt the process and be returned
|
79
|
+
# to the user
|
80
|
+
if s.stderr != "" && s.stderr !~ /binary `cargo-license` already exists/
|
81
|
+
s.error!
|
82
|
+
end
|
83
|
+
end
|
84
|
+
|
85
|
+
def cargo_file_path
|
86
|
+
File.join(directory, "Cargo.toml")
|
87
|
+
end
|
88
|
+
|
89
|
+
def cargo_lockfile_path
|
90
|
+
File.join(directory, "Cargo.lock")
|
91
|
+
end
|
92
|
+
|
93
|
+
end
|
94
|
+
end
|
95
|
+
end
|
@@ -0,0 +1,113 @@
|
|
1
|
+
#
|
2
|
+
# Copyright:: Copyright 2020, Chef Software Inc.
|
3
|
+
# License:: Apache License, Version 2.0
|
4
|
+
#
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
+
# you may not use this file except in compliance with the License.
|
7
|
+
# You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
#
|
17
|
+
|
18
|
+
require "license_scout/dependency_manager/base"
|
19
|
+
|
20
|
+
module LicenseScout
|
21
|
+
module DependencyManager
|
22
|
+
class Gomod < Base
|
23
|
+
|
24
|
+
def name
|
25
|
+
"golang_modules"
|
26
|
+
end
|
27
|
+
|
28
|
+
def type
|
29
|
+
"golang"
|
30
|
+
end
|
31
|
+
|
32
|
+
def signature
|
33
|
+
"go.sum file"
|
34
|
+
end
|
35
|
+
|
36
|
+
def install_command
|
37
|
+
"go mod download"
|
38
|
+
end
|
39
|
+
|
40
|
+
def detected?
|
41
|
+
File.exist?(go_sum_file)
|
42
|
+
end
|
43
|
+
|
44
|
+
def dependencies
|
45
|
+
go_modules.map do |mod|
|
46
|
+
next if mod["Main"] == true
|
47
|
+
|
48
|
+
dep_name = mod["Path"]
|
49
|
+
dep_version = mod["Version"]
|
50
|
+
dep_path = mod["Dir"]
|
51
|
+
|
52
|
+
new_dependency(dep_name, dep_version, dep_path)
|
53
|
+
end.compact
|
54
|
+
end
|
55
|
+
|
56
|
+
def go_sum_file
|
57
|
+
File.join(directory, "go.sum")
|
58
|
+
end
|
59
|
+
|
60
|
+
def vendor_dir
|
61
|
+
File.join(directory, "vendor")
|
62
|
+
end
|
63
|
+
|
64
|
+
def modules_txt_file
|
65
|
+
File.join(vendor_dir, "modules.txt")
|
66
|
+
end
|
67
|
+
|
68
|
+
def go_modules
|
69
|
+
if vendor_mode
|
70
|
+
GoModulesTxtParser.parse(File.read(modules_txt_file), vendor_dir)
|
71
|
+
else
|
72
|
+
FFI_Yajl::Parser.parse(go_modules_json)
|
73
|
+
end
|
74
|
+
end
|
75
|
+
|
76
|
+
def vendor_mode
|
77
|
+
if @vendor_mode.nil?
|
78
|
+
@vendor_mode = File.directory?(vendor_dir)
|
79
|
+
end
|
80
|
+
@vendor_mode
|
81
|
+
end
|
82
|
+
|
83
|
+
def go_modules_json
|
84
|
+
s = Mixlib::ShellOut.new("go list -m -json all", cwd: directory, environment: LicenseScout::Config.environment)
|
85
|
+
s.run_command
|
86
|
+
s.error!
|
87
|
+
"[" + s.stdout.gsub("}\n{", "},\n{") + "]"
|
88
|
+
end
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
module GoModulesTxtParser
|
93
|
+
# The modules.txt file has lines that look like:
|
94
|
+
#
|
95
|
+
# # gopkg.in/square/go-jose.v2 v2.1.3
|
96
|
+
#
|
97
|
+
# We parse these lines and return something that looks like `go
|
98
|
+
# list -m -json all` output.
|
99
|
+
def self.parse(data, base_path)
|
100
|
+
data.lines.map do |l|
|
101
|
+
if l.start_with?("#")
|
102
|
+
parts = l.split
|
103
|
+
{
|
104
|
+
"Main" => false,
|
105
|
+
"Path" => parts[1],
|
106
|
+
"Version" => parts[2],
|
107
|
+
"Dir" => File.join(base_path, parts[1]),
|
108
|
+
}
|
109
|
+
end
|
110
|
+
end.compact
|
111
|
+
end
|
112
|
+
end
|
113
|
+
end
|
@@ -46,28 +46,38 @@ module LicenseScout
|
|
46
46
|
def dependencies
|
47
47
|
tdeps = Set.new(pkg_deps)
|
48
48
|
|
49
|
-
pkg_deps.
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
o, n, v, r = tdep.split("/")
|
55
|
-
dep_name = "#{o}/#{n}"
|
56
|
-
dep_version = "#{v}-#{r}"
|
57
|
-
|
58
|
-
dependency = new_dependency(dep_name, dep_version, nil)
|
59
|
-
|
60
|
-
license_from_manifest(pkg_info(tdep)["manifest"]).each do |spdx|
|
61
|
-
# We hard code the channel to "unstable" because a package could be
|
62
|
-
# demoted from any given channel except unstable in the future and
|
63
|
-
# we want the url metadata to be stable in order to give end users
|
64
|
-
# the ability to self-audit licenses
|
65
|
-
# tl;dr, we want a permalink not a nowlink
|
66
|
-
dependency.add_license(spdx, "https://bldr.habitat.sh/v1/depot/channels/#{o}/unstable/pkgs/#{n}/#{v}/#{r}")
|
49
|
+
if pkg_deps.any?
|
50
|
+
pkg_deps.each do |pkg_dep|
|
51
|
+
unless pkg_info(pkg_dep).nil?
|
52
|
+
pkg_info(pkg_dep)["tdeps"].each { |dep| tdeps << to_ident(dep) }
|
53
|
+
end
|
67
54
|
end
|
68
55
|
|
69
|
-
|
70
|
-
|
56
|
+
tdeps.delete(nil)
|
57
|
+
|
58
|
+
tdeps.sort.map do |tdep|
|
59
|
+
o, n, v, r = tdep.split("/")
|
60
|
+
dep_name = "#{o}/#{n}"
|
61
|
+
dep_version = "#{v}-#{r}"
|
62
|
+
|
63
|
+
dependency = new_dependency(dep_name, dep_version, nil)
|
64
|
+
|
65
|
+
if pkg_info(tdep).nil?
|
66
|
+
LicenseScout::Log.warn("Could not find information for #{tdep} -- skipping")
|
67
|
+
else
|
68
|
+
license_from_manifest(pkg_info(tdep)["manifest"]).each do |spdx|
|
69
|
+
# We hard code the channel to "unstable" because a package could be
|
70
|
+
# demoted from any given channel except unstable in the future and
|
71
|
+
# we want the url metadata to be stable in order to give end users
|
72
|
+
# the ability to self-audit licenses
|
73
|
+
# tl;dr, we want a permalink not a nowlink
|
74
|
+
dependency.add_license(spdx, "https://bldr.habitat.sh/v1/depot/channels/#{o}/unstable/pkgs/#{n}/#{v}/#{r}")
|
75
|
+
end
|
76
|
+
end
|
77
|
+
|
78
|
+
dependency
|
79
|
+
end.compact
|
80
|
+
end
|
71
81
|
end
|
72
82
|
|
73
83
|
private
|
@@ -86,7 +96,9 @@ module LicenseScout
|
|
86
96
|
pkg_deps = c.stdout.split("\s")
|
87
97
|
|
88
98
|
# Fetch the fully-qualified pkg_ident for each pkg
|
89
|
-
pkg_deps.map
|
99
|
+
pkg_deps.map do |dep|
|
100
|
+
to_ident(pkg_info(dep)["ident"]) unless pkg_info(dep).nil?
|
101
|
+
end
|
90
102
|
end
|
91
103
|
end
|
92
104
|
|
@@ -100,29 +112,31 @@ module LicenseScout
|
|
100
112
|
end
|
101
113
|
|
102
114
|
def pkg_info_with_channel_fallbacks(pkg_ident)
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
if pkg_channel != DEFAULT_CHANNEL
|
114
|
-
LicenseScout::Log.debug("[habitat] Looking for #{pkg_ident} in #{DEFAULT_CHANNEL} channel")
|
115
|
-
info = get_pkg_info(pkg_origin, DEFAULT_CHANNEL, pkg_name, pkg_version, pkg_release)
|
115
|
+
unless pkg_ident.nil?
|
116
|
+
pkg_origin, pkg_name, pkg_version, pkg_release = pkg_ident.split("/")
|
117
|
+
pkg_channel = channel_for_origin(pkg_origin)
|
118
|
+
|
119
|
+
# Channel selection here is similar to the logic that
|
120
|
+
# Habitat uses. First, search in the user-provided channel,
|
121
|
+
# then search in stable, then use unstable IF it is a fully
|
122
|
+
# qualified package
|
123
|
+
info = get_pkg_info(pkg_origin, pkg_channel, pkg_name, pkg_version, pkg_release)
|
116
124
|
return info if info
|
117
|
-
end
|
118
125
|
|
119
|
-
|
120
|
-
|
121
|
-
|
122
|
-
|
123
|
-
|
126
|
+
if pkg_channel != DEFAULT_CHANNEL
|
127
|
+
LicenseScout::Log.debug("[habitat] Looking for #{pkg_ident} in #{DEFAULT_CHANNEL} channel")
|
128
|
+
info = get_pkg_info(pkg_origin, DEFAULT_CHANNEL, pkg_name, pkg_version, pkg_release)
|
129
|
+
return info if info
|
130
|
+
end
|
124
131
|
|
125
|
-
|
132
|
+
if !pkg_version.nil? && !pkg_release.nil?
|
133
|
+
LicenseScout::Log.debug("[habitat] Looking for #{pkg_ident} in #{FALLBACK_CHANNEL_FOR_FQ} channel since it is fully-qualified")
|
134
|
+
info = get_pkg_info(pkg_origin, FALLBACK_CHANNEL_FOR_FQ, pkg_name, pkg_version, pkg_release)
|
135
|
+
return info if info
|
136
|
+
end
|
137
|
+
|
138
|
+
LicenseScout::Log.warn("Could not find information for #{pkg_ident} -- skipping")
|
139
|
+
end
|
126
140
|
end
|
127
141
|
|
128
142
|
def get_pkg_info(origin, channel, name, version, release)
|
@@ -15,7 +15,7 @@
|
|
15
15
|
# limitations under the License.
|
16
16
|
#
|
17
17
|
|
18
|
-
require "csv"
|
18
|
+
require "csv" unless defined?(CSV)
|
19
19
|
|
20
20
|
module LicenseScout
|
21
21
|
class Exporter
|
@@ -65,7 +65,7 @@ module LicenseScout
|
|
65
65
|
(exception_reason.nil? ? "" : exception_reason),
|
66
66
|
id,
|
67
67
|
source,
|
68
|
-
content
|
68
|
+
content,
|
69
69
|
]
|
70
70
|
end
|
71
71
|
end
|
@@ -106,7 +106,7 @@ module LicenseScout
|
|
106
106
|
|
107
107
|
begin
|
108
108
|
LicenseScout::Log.debug("[license] Pulling license content for #{license_id} from #{new_url}")
|
109
|
-
open(new_url).read
|
109
|
+
URI.open(new_url).read
|
110
110
|
rescue RuntimeError => e
|
111
111
|
if e.message =~ /redirection forbidden/
|
112
112
|
m = /redirection forbidden:\s+(.+)\s+->\s+(.+)/.match(e.message)
|
@@ -126,7 +126,7 @@ module LicenseScout
|
|
126
126
|
|
127
127
|
def raw_github_url(url)
|
128
128
|
case url
|
129
|
-
when
|
129
|
+
when %r{github.com/(.+)/blob/(.+)}
|
130
130
|
"https://raw.githubusercontent.com/#{$1}/#{$2}"
|
131
131
|
else
|
132
132
|
url
|
data/lib/license_scout/spdx.rb
CHANGED
@@ -17,7 +17,7 @@
|
|
17
17
|
|
18
18
|
# This library was inspired by (and pulls some logic from) librariesio/spdx
|
19
19
|
|
20
|
-
require "ffi_yajl"
|
20
|
+
require "ffi_yajl" unless defined?(FFI_Yajl)
|
21
21
|
require "fuzzy_match"
|
22
22
|
|
23
23
|
module LicenseScout
|
@@ -32,6 +32,7 @@ module LicenseScout
|
|
32
32
|
def find(license_id, force = false)
|
33
33
|
return license_id if force
|
34
34
|
return nil if license_id.nil? || %w{ NOASSERTION NONE }.include?(license_id)
|
35
|
+
|
35
36
|
lookup(license_id) || find_by_special_case(license_id) || closest(license_id) || license_id
|
36
37
|
end
|
37
38
|
|
@@ -44,12 +45,12 @@ module LicenseScout
|
|
44
45
|
|
45
46
|
# @return [Hash] The SPDX license data in Hash form
|
46
47
|
def licenses
|
47
|
-
@@license_data ||= FFI_Yajl::Parser.parse(File.read(File.expand_path("
|
48
|
+
@@license_data ||= FFI_Yajl::Parser.parse(File.read(File.expand_path("data/licenses.json", __dir__)))["licenses"]
|
48
49
|
end
|
49
50
|
|
50
51
|
# @return [Hash] The SPDX license data in Hash form
|
51
52
|
def exceptions
|
52
|
-
@@license_data ||= FFI_Yajl::Parser.parse(File.read(File.expand_path("
|
53
|
+
@@license_data ||= FFI_Yajl::Parser.parse(File.read(File.expand_path("data/exceptions.json", __dir__)))["exceptions"]
|
53
54
|
end
|
54
55
|
|
55
56
|
def known_ids
|
@@ -71,6 +72,7 @@ module LicenseScout
|
|
71
72
|
def find_by_special_case(license_id)
|
72
73
|
gpl = gpl_match(license_id)
|
73
74
|
return gpl unless gpl.nil?
|
75
|
+
|
74
76
|
lookup(special_cases[license_id.downcase])
|
75
77
|
end
|
76
78
|
|
@@ -81,6 +83,7 @@ module LicenseScout
|
|
81
83
|
def gpl_match(license_id)
|
82
84
|
match = license_id.match(/^(l|a)?gpl-?\s?_?v?(1|2|3)\.?(\d)?(\+)?$/i)
|
83
85
|
return unless match
|
86
|
+
|
84
87
|
lookup("#{match[1]}GPL-#{match[2]}.#{match[3] || 0}#{match[4]}".upcase)
|
85
88
|
end
|
86
89
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: license_scout
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.2
|
4
|
+
version: 2.6.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tom Duffield
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2021-07-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: ffi-yajl
|
@@ -162,10 +162,12 @@ files:
|
|
162
162
|
- lib/license_scout/dependency_manager/base.rb
|
163
163
|
- lib/license_scout/dependency_manager/berkshelf.rb
|
164
164
|
- lib/license_scout/dependency_manager/bundler.rb
|
165
|
+
- lib/license_scout/dependency_manager/cargo.rb
|
165
166
|
- lib/license_scout/dependency_manager/cpanm.rb
|
166
167
|
- lib/license_scout/dependency_manager/dep.rb
|
167
168
|
- lib/license_scout/dependency_manager/glide.rb
|
168
169
|
- lib/license_scout/dependency_manager/godep.rb
|
170
|
+
- lib/license_scout/dependency_manager/gomod.rb
|
169
171
|
- lib/license_scout/dependency_manager/habitat.rb
|
170
172
|
- lib/license_scout/dependency_manager/mix.rb
|
171
173
|
- lib/license_scout/dependency_manager/npm.rb
|
@@ -197,7 +199,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
197
199
|
- !ruby/object:Gem::Version
|
198
200
|
version: '0'
|
199
201
|
requirements: []
|
200
|
-
rubygems_version: 3.
|
202
|
+
rubygems_version: 3.1.4
|
201
203
|
signing_key:
|
202
204
|
specification_version: 4
|
203
205
|
summary: Discovers license files of a project's dependencies.
|