licensed 3.7.4 → 3.8.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1002bceeb85db079486f7cbde9c09638362fc03b287c15822fcc7e4e250a1c6d
4
- data.tar.gz: acb2c626b8efd7573e32efef91a3c68b93c5aeee07580d874d58ce1326ee5aa8
3
+ metadata.gz: a8ae3ae52bbb8b7a7b9bca992046e870a917fa891600ecb81386d1aaeee3c65a
4
+ data.tar.gz: 31988f8d467f49ae8b3ba0765479a9ae5be4e2de4b60f58de6a40ab8578c50e5
5
5
  SHA512:
6
- metadata.gz: fb16460e74fd404c19e3a977c9fc125f91ef2b96f04952aac6aa46da8dc5981d9bcfb556cd2db2503e0d36ce7a78278e57532005e54384069e632d62cd7181de
7
- data.tar.gz: e1d0b1df753a9dcc0c6b488cdbfef416d776edd8e3128363a46722f43be2178f939e6cbef71a645d944049619d058c3df7f291e2b3779f32046f094726ca95f4
6
+ metadata.gz: 6a14c36d7d1c0060114ff1b30176e148ba85f8a211249e8b4964af00a818677a4252ffc88428355a09e8070b765803ddf6a68d35db8ea14cf0253855f6cfd299
7
+ data.tar.gz: f735c4a13b0d8aa00c496e2d7f502e7fd23680865a2530f6262689f3484faa7a34297e05f7db6deb2b539b991cc4e8415816b3ecd10ce8d0d9e89830d277d3d6
data/CHANGELOG.md CHANGED
@@ -6,6 +6,18 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## 3.8.0
10
+
11
+ ### Added
12
+
13
+ - Licensing compliance status checks can now be used without cached files in a repository (https://github.com/github/licensed/pull/560)
14
+
15
+ ## 3.7.5
16
+
17
+ ### Fixed
18
+
19
+ - Python dependency metadata will be correctly parsed from the ouput of `pip show` (https://github.com/github/licensed/pull/555)
20
+
9
21
  ## 3.7.4
10
22
 
11
23
  ### Fixed
@@ -637,4 +649,4 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
637
649
 
638
650
  Initial release :tada:
639
651
 
640
- [Unreleased]: https://github.com/github/licensed/compare/3.7.4...HEAD
652
+ [Unreleased]: https://github.com/github/licensed/compare/3.8.0...HEAD
@@ -1,17 +1,36 @@
1
1
  # `licensed status`
2
2
 
3
- The status command finds all dependencies and checks whether each dependency has a valid cached record.
3
+ The status command finds all dependencies and checks whether each dependency has a valid record. There are two methods for checking dependencies' statuses
4
+
5
+ ## Checking status with metadata loaded from cached files
6
+
7
+ This is the default method for checking the status for dependencies and the recommended method for using licensed. Checking status from cached metadata files will occur when the `--data-source` CLI flag is either unset, or set to `--data-source=files`.
8
+
9
+ When using `licensed status` in this scenario, licensed will only compile a minimal amount of dependency metadata like the dependency name and version to match against cached files. Dependency license and notice texts are loaded from cached files that are created from the [licensed cache](./cache.md) command.
4
10
 
5
11
  A dependency will fail the status checks if:
6
12
 
7
13
  1. No cached record is found
8
14
  2. The cached record's version is different than the current dependency's version
9
15
  3. The cached record's `licenses` data is empty
10
- 4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration.
16
+ 4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration and the dependency has not been marked `reviewed` or `ignored`
11
17
  - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged
12
18
  5. The cached record is flagged for re-review.
13
19
  - This occurs when the record's license text has changed since the record was reviewed.
14
20
 
21
+ ## Checking status with computed metadata
22
+
23
+ Checking status with computed metadata and the licensed configuration will occur when the `--data-source` CLI flag is set to `--data-source=configuration`.
24
+
25
+ When using `licensed status` in this scenario, licensed will compile all dependency metadata and license text to use in status checks. There is no need to run `licensed cache` prior to running `licensed status --data-source=configuration`.
26
+
27
+ A dependency will fail the status checks if:
28
+
29
+ 1. The record's `licenses` data is empty
30
+ 2. The record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration and the dependency has not been marked `reviewed` or `ignored`
31
+ - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged
32
+ - A `reviewed` entry must reference a specific version of the depdency, e.g. `<name>@<version>`. The version identifier must specify a specific dependency version, ranges are not allowed.
33
+
15
34
  ## Options
16
35
 
17
36
  - `--config`/`-c`: the path to the licensed configuration file
@@ -20,6 +39,9 @@ A dependency will fail the status checks if:
20
39
  - default value: not set, all configured sources
21
40
  - `--format`/`-f`: the output format
22
41
  - default value: `yaml`
42
+ - `--data-source`/`-d`: where to find the data source of records for status checks
43
+ - available values: `files`, `configuration`
44
+ - default value: `files`
23
45
  - `--force`: if set, forces all dependency metadata files to be recached
24
46
  - default value: not set
25
47
 
@@ -63,14 +85,16 @@ If the dependency does not include license text but does specify that it uses a
63
85
 
64
86
  *Resolution:* Review the changes to the license text and classification, along with other metadata contained in the cached file for the dependency. If the dependency is still allowable for use in your project, remove the `review_changed_license` key from the cached record file.
65
87
 
66
- ### license needs review
88
+ ### license needs review / dependency needs review
67
89
 
68
90
  *Cause:* A dependency is using a license that is not in the configured [allowed list of licenses][allowed], and the dependency has not been marked [ignored] or [reviewed].
69
91
  *Resolution:* Review the dependency's usage and specified license with someone familiar with OSS licensing and compliance rules to determine whether the dependency is allowable. Some common resolutions:
70
92
 
71
- 1. The dependency's specified license text differed enough from the standard license text that it was not recognized and classified as `other`. If, with human review, the license text is recognizable then update the `license: other` value in the cached metadata file to the correct license.
72
- - An updated classification will persist through version upgrades until the detected license contents have changed. The determination is made by [licensee/licensee](https://github.com/licensee/licensee), the library which this tool uses to detect and classify license contents.
93
+ 1. The dependency's specified license text differed enough from the standard license text that it was not recognized and classified as `other`.
94
+ - This resolution only applies when checking dependency status [with cached metadata files](./#checking-status-with-metadata-loaded-from-cached-files).
95
+ - If the cached license text is recognizable with human review then update the `license: other` value in the cached metadata file to the correct license. An updated classification will persist through version upgrades until the detected license contents have changed. The determination is made by [licensee/licensee](https://github.com/licensee/licensee), the library which this tool uses to detect and classify license contents.
73
96
  1. The dependency might need to be marked as [ignored] or [reviewed] if either of those scenarios are applicable.
97
+ - When checking status [with computed metadata](./#checking-status-with-computed-metadata), a reviewed entry must include both the dependency's name and the version that it was reviewed at e.g. `licensed@3.8.0`
74
98
  1. If the used license should be allowable without review (if your entity has a legal team, they may want to review this assessment), ensure the license SPDX is set as [allowed] in the licensed configuration file.
75
99
 
76
100
  [allowed]: ../configuration/allowed_licenses.md
data/lib/licensed/cli.rb CHANGED
@@ -26,8 +26,11 @@ module Licensed
26
26
  desc: "Individual source(s) to evaluate. Must also be enabled via configuration."
27
27
  method_option :format, aliases: "-f", enum: ["yaml", "json"],
28
28
  desc: "Output format"
29
+ method_option :data_source, aliases: "-d",
30
+ enum: ["files", "configuration"], default: "files",
31
+ desc: "Whether to check compliance status from cached records or the configuration file"
29
32
  def status
30
- run Licensed::Commands::Status.new(config: config), sources: options[:sources], reporter: options[:format]
33
+ run Licensed::Commands::Status.new(config: config), sources: options[:sources], reporter: options[:format], data_source: options[:data_source]
31
34
  end
32
35
 
33
36
  desc "list", "List dependencies"
@@ -29,33 +29,39 @@ module Licensed
29
29
  end
30
30
  end
31
31
 
32
- # Verifies that a cached record exists, is up to date and
33
- # has license data that complies with the licensed configuration.
32
+ # Evaluates a dependency for any compliance errors.
33
+ # Checks a dependency against either a cached metadata record or
34
+ # reviewed entries in the configuration file.
34
35
  #
35
36
  # app - The application configuration for the dependency
36
37
  # source - The dependency source enumerator for the dependency
37
38
  # dependency - An application dependency
38
39
  # report - A report hash for the command to provide extra data for the report output.
39
40
  #
40
- # Returns whether the dependency has a cached record that is compliant
41
+ # Returns whether the dependency is compliant
41
42
  # with the licensed configuration.
42
43
  def evaluate_dependency(app, source, dependency, report)
43
- filename = app.cache_path.join(source.class.type, "#{dependency.name}.#{DependencyRecord::EXTENSION}")
44
- report["filename"] = filename
45
44
  report["version"] = dependency.version
46
45
 
47
- cached_record = cached_record(filename)
48
- if cached_record.nil?
46
+ if data_source == "configuration"
47
+ record = dependency.record
48
+ else
49
+ filename = app.cache_path.join(source.class.type, "#{dependency.name}.#{DependencyRecord::EXTENSION}")
50
+ report["filename"] = filename
51
+ record = cached_record(filename)
52
+ end
53
+
54
+ if record.nil?
49
55
  report["license"] = nil
50
56
  report.errors << "cached dependency record not found"
51
57
  else
52
- report["license"] = cached_record["license"]
53
- report.errors << "cached dependency record out of date" if cached_record["version"] != dependency.version
54
- report.errors << "missing license text" if cached_record.licenses.empty?
55
- if cached_record["review_changed_license"]
58
+ report["license"] = record["license"]
59
+ report.errors << "dependency record out of date" if record["version"] != dependency.version
60
+ report.errors << "missing license text" if record.licenses.empty?
61
+ if record["review_changed_license"]
56
62
  report.errors << "license text has changed and needs re-review. if the new text is ok, remove the `review_changed_license` flag from the cached record"
57
- elsif license_needs_review?(app, cached_record)
58
- report.errors << "license needs review: #{cached_record["license"]}"
63
+ elsif license_needs_review?(app, record)
64
+ report.errors << needs_review_error_message(app, record)
59
65
  end
60
66
  end
61
67
 
@@ -64,19 +70,20 @@ module Licensed
64
70
 
65
71
  # Returns true if a cached record needs further review based on the
66
72
  # record's license(s) and the app's configuration
67
- def license_needs_review?(app, cached_record)
73
+ def license_needs_review?(app, record)
68
74
  # review is not needed if the record is set as reviewed
69
- return false if app.reviewed?(cached_record)
75
+ return false if app.reviewed?(record, match_version: data_source == "configuration")
76
+
70
77
  # review is not needed if the top level license is allowed
71
- return false if app.allowed?(cached_record["license"])
78
+ return false if app.allowed?(record["license"])
72
79
 
73
80
  # the remaining checks are meant to allow records marked as "other"
74
81
  # that have multiple licenses, all of which are allowed
75
82
 
76
83
  # review is needed for non-"other" licenses
77
- return true unless cached_record["license"] == "other"
84
+ return true unless record["license"] == "other"
78
85
 
79
- licenses = cached_record.licenses.map { |license| license_from_text(license.text) }
86
+ licenses = record.licenses.map { |license| license_from_text(license.text) }
80
87
 
81
88
  # review is needed when there is only one license notice
82
89
  # this is a performance optimization for the single license case
@@ -86,6 +93,29 @@ module Licensed
86
93
  licenses.any? { |license| !app.allowed?(license) }
87
94
  end
88
95
 
96
+ def needs_review_error_message(app, record)
97
+ return "license needs review: #{record["license"]}" if data_source == "files"
98
+
99
+ error = "dependency needs review"
100
+
101
+ # look for an unversioned reviewed list match
102
+ if app.reviewed?(record, match_version: false)
103
+ error += ", unversioned 'reviewed' match found: #{record["name"]}"
104
+ end
105
+
106
+ # look for other version matches in reviewed list
107
+ possible_matches = app.reviewed_versions(record)
108
+ if possible_matches.any?
109
+ error += ", possible 'reviewed' matches found at other versions: #{possible_matches.join(", ")}"
110
+ end
111
+
112
+ error
113
+ end
114
+
115
+ def data_source
116
+ options[:data_source] || "files"
117
+ end
118
+
89
119
  def cached_record(filename)
90
120
  return nil unless File.exist?(filename)
91
121
  DependencyRecord.read(filename)
@@ -73,17 +73,18 @@ module Licensed
73
73
  end
74
74
 
75
75
  # Is the given dependency reviewed?
76
- def reviewed?(dependency)
77
- Array(self["reviewed"][dependency["type"]]).any? do |pattern|
78
- File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
79
- end
76
+ def reviewed?(dependency, match_version: false)
77
+ any_list_pattern_matched? self["reviewed"][dependency["type"]], dependency, match_version: match_version
78
+ end
79
+
80
+ # Find all reviewed dependencies that match the provided dependency's name
81
+ def reviewed_versions(dependency)
82
+ similar_list_patterns self["reviewed"][dependency["type"]], dependency
80
83
  end
81
84
 
82
85
  # Is the given dependency ignored?
83
86
  def ignored?(dependency)
84
- Array(self["ignored"][dependency["type"]]).any? do |pattern|
85
- File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
86
- end
87
+ any_list_pattern_matched? self["ignored"][dependency["type"]], dependency
87
88
  end
88
89
 
89
90
  # Is the license of the dependency allowed?
@@ -97,8 +98,10 @@ module Licensed
97
98
  end
98
99
 
99
100
  # Set a dependency as reviewed
100
- def review(dependency)
101
- (self["reviewed"][dependency["type"]] ||= []) << dependency["name"]
101
+ def review(dependency, at_version: false)
102
+ id = dependency["name"]
103
+ id += "@#{dependency["version"]}" if at_version && dependency["version"]
104
+ (self["reviewed"][dependency["type"]] ||= []) << id
102
105
  end
103
106
 
104
107
  # Set a license as explicitly allowed
@@ -108,6 +111,27 @@ module Licensed
108
111
 
109
112
  private
110
113
 
114
+ def any_list_pattern_matched?(list, dependency, match_version: false)
115
+ Array(list).any? do |pattern|
116
+ if match_version
117
+ at_version = "@#{dependency["version"]}"
118
+ pattern, pattern_version = pattern.rpartition(at_version).values_at(0, 1)
119
+ next false if pattern == "" || pattern_version == ""
120
+ end
121
+
122
+ File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
123
+ end
124
+ end
125
+
126
+ def similar_list_patterns(list, dependency)
127
+ Array(list).select do |pattern|
128
+ pattern, version = pattern.rpartition("@").values_at(0, 2)
129
+ next if pattern == "" || version == ""
130
+
131
+ File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
132
+ end
133
+ end
134
+
111
135
  # Returns the cache path for the application based on:
112
136
  # 1. An explicitly set cache path for the application, if set
113
137
  # 2. An inherited shared cache path
@@ -74,11 +74,13 @@ module Licensed
74
74
  # Returns a hash filled with package info parsed from the email-header formatted output
75
75
  # returned by `pip show`
76
76
  def parse_package_info(package_info)
77
- package_info.lines.each_with_object(Hash.new(0)) { |pkg, a|
77
+ package_info.lines.each_with_object(Hash.new(0)) do |pkg, a|
78
+ next if pkg.start_with?(/^\s/)
79
+
78
80
  k, v = pkg.split(":", 2)
79
81
  next if k.nil? || k.empty?
80
82
  a[k.strip] = v&.strip
81
- }
83
+ end
82
84
  end
83
85
 
84
86
  # Returns the output from `pip list --format=json`
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
  module Licensed
3
- VERSION = "3.7.4".freeze
3
+ VERSION = "3.8.0".freeze
4
4
 
5
5
  def self.previous_major_versions
6
6
  major_version = Gem::Version.new(Licensed::VERSION).segments.first
data/licensed.gemspec CHANGED
@@ -36,7 +36,6 @@ Gem::Specification.new do |spec|
36
36
  spec.add_development_dependency "rake", ">= 12.3.3"
37
37
  spec.add_development_dependency "minitest", "~> 5.8"
38
38
  spec.add_development_dependency "mocha", "~> 1.0"
39
- spec.add_development_dependency "rubocop", "~> 1.26", "< 1.27"
40
39
  spec.add_development_dependency "rubocop-github", "~> 0.6"
41
40
  spec.add_development_dependency "byebug", "~> 11.1.3"
42
41
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: licensed
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.7.4
4
+ version: 3.8.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - GitHub
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2022-10-07 00:00:00.000000000 Z
11
+ date: 2022-10-29 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: licensee
@@ -196,26 +196,6 @@ dependencies:
196
196
  - - "~>"
197
197
  - !ruby/object:Gem::Version
198
198
  version: '1.0'
199
- - !ruby/object:Gem::Dependency
200
- name: rubocop
201
- requirement: !ruby/object:Gem::Requirement
202
- requirements:
203
- - - "~>"
204
- - !ruby/object:Gem::Version
205
- version: '1.26'
206
- - - "<"
207
- - !ruby/object:Gem::Version
208
- version: '1.27'
209
- type: :development
210
- prerelease: false
211
- version_requirements: !ruby/object:Gem::Requirement
212
- requirements:
213
- - - "~>"
214
- - !ruby/object:Gem::Version
215
- version: '1.26'
216
- - - "<"
217
- - !ruby/object:Gem::Version
218
- version: '1.27'
219
199
  - !ruby/object:Gem::Dependency
220
200
  name: rubocop-github
221
201
  requirement: !ruby/object:Gem::Requirement