licensed 3.7.5 → 3.9.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2c460f79af039dfcd0059dafa8c1ce9ea096f5fc869c05ed374a61e9d1504520
4
- data.tar.gz: 98729cb3117d5642fe1bb6b81a8b5f9781578751407f0425ea2ed0c5842ae0ac
3
+ metadata.gz: f4df54260766353e4cd56b9ae56ded611ed4d6a312469d43e18ab47b6b9cabde
4
+ data.tar.gz: 8f04c9e9d11bcaf7f47698a174ee1269346e798eaa176e28ac3282de482ef237
5
5
  SHA512:
6
- metadata.gz: 3f5bd011838dc17f8bb74da74b0799da181419718bf96f55ac097af2a077546bba007a1efa2ec8613bd03f345c1f82bf5b97fcad64ebb140ef06a99210ee45db
7
- data.tar.gz: 79b667beb6bde2a62ec1c53c706a5379f761bc9189f2fd6caf4da10e7e2ca51216f44816fc06721c023f81446c7f1a480ca482cc6b77f53af92dcb1943a535b5
6
+ metadata.gz: 0006a278b5b2a7af75ad7634fe11f418f310e7c7506e1ae3cc68bdfca873cdbf74b9bd359d2a9b917c8c79df3a91b589c0c7e8f0b6ad2c03349a81e3bfbc91cd
7
+ data.tar.gz: 0c6275c87fe724a747f0432395b568341c495453f0072aa78e0f1ee48e075ebfbd94de5ef85d3eb46adf7dfb16e203c725a90929be6dfe801bf53a7b55c783e8
data/CHANGELOG.md CHANGED
@@ -6,6 +6,18 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## 3.9.0
10
+
11
+ ### Added
12
+
13
+ - `NOTICE` files can now be generated without cached files in a repository (https://github.com/github/licensed/pull/572)
14
+
15
+ ## 3.8.0
16
+
17
+ ### Added
18
+
19
+ - Licensing compliance status checks can now be used without cached files in a repository (https://github.com/github/licensed/pull/560)
20
+
9
21
  ## 3.7.5
10
22
 
11
23
  ### Fixed
@@ -643,4 +655,4 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
643
655
 
644
656
  Initial release :tada:
645
657
 
646
- [Unreleased]: https://github.com/github/licensed/compare/3.7.5...HEAD
658
+ [Unreleased]: https://github.com/github/licensed/compare/3.9.0...HEAD
@@ -2,7 +2,7 @@
2
2
 
3
3
  Outputs license and notice text for all dependencies in each app into a `NOTICE` file in the app's `cache_path`. If an app uses a shared cache path, the file name will contain the app name as well, e.g. `NOTICE.my_app`.
4
4
 
5
- `NOTICE` file contents are retrieved from cached records, with the assumption that cached records have already been reviewed in a compliance workflow.
5
+ `NOTICE` file contents are retrieved from cached records when the `--computed`/`-l` option is not set, with the assumption that cached records have already been reviewed in a compliance workflow. When the `--computed`/`-l` option is set and a dependency's license is not found, that dependency's license text will be empty in the `NOTICE` file.
6
6
 
7
7
  ## Options
8
8
 
@@ -10,3 +10,5 @@ Outputs license and notice text for all dependencies in each app into a `NOTICE`
10
10
  - default value: `./.licensed.yml`
11
11
  - `--sources`/`-s`: runtime filter on which dependency sources are run. Sources must also be enabled in the licensed configuration file.
12
12
  - default value: not set, all configured sources
13
+ - `--computed`/`-l`: use live computed when generating a `NOTICE` file
14
+ - default value: not set, `NOTICE` file generated from cached records
@@ -1,17 +1,36 @@
1
1
  # `licensed status`
2
2
 
3
- The status command finds all dependencies and checks whether each dependency has a valid cached record.
3
+ The status command finds all dependencies and checks whether each dependency has a valid record. There are two methods for checking dependencies' statuses
4
+
5
+ ## Checking status with metadata loaded from cached files
6
+
7
+ This is the default method for checking the status for dependencies and the recommended method for using licensed. Checking status from cached metadata files will occur when the `--data-source` CLI flag is either unset, or set to `--data-source=files`.
8
+
9
+ When using `licensed status` in this scenario, licensed will only compile a minimal amount of dependency metadata like the dependency name and version to match against cached files. Dependency license and notice texts are loaded from cached files that are created from the [licensed cache](./cache.md) command.
4
10
 
5
11
  A dependency will fail the status checks if:
6
12
 
7
13
  1. No cached record is found
8
14
  2. The cached record's version is different than the current dependency's version
9
15
  3. The cached record's `licenses` data is empty
10
- 4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration.
16
+ 4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration and the dependency has not been marked `reviewed` or `ignored`
11
17
  - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged
12
18
  5. The cached record is flagged for re-review.
13
19
  - This occurs when the record's license text has changed since the record was reviewed.
14
20
 
21
+ ## Checking status with computed metadata
22
+
23
+ Checking status with computed metadata and the licensed configuration will occur when the `--data-source` CLI flag is set to `--data-source=configuration`.
24
+
25
+ When using `licensed status` in this scenario, licensed will compile all dependency metadata and license text to use in status checks. There is no need to run `licensed cache` prior to running `licensed status --data-source=configuration`.
26
+
27
+ A dependency will fail the status checks if:
28
+
29
+ 1. The record's `licenses` data is empty
30
+ 2. The record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration and the dependency has not been marked `reviewed` or `ignored`
31
+ - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged
32
+ - A `reviewed` entry must reference a specific version of the depdency, e.g. `<name>@<version>`. The version identifier must specify a specific dependency version, ranges are not allowed.
33
+
15
34
  ## Options
16
35
 
17
36
  - `--config`/`-c`: the path to the licensed configuration file
@@ -20,6 +39,9 @@ A dependency will fail the status checks if:
20
39
  - default value: not set, all configured sources
21
40
  - `--format`/`-f`: the output format
22
41
  - default value: `yaml`
42
+ - `--data-source`/`-d`: where to find the data source of records for status checks
43
+ - available values: `files`, `configuration`
44
+ - default value: `files`
23
45
  - `--force`: if set, forces all dependency metadata files to be recached
24
46
  - default value: not set
25
47
 
@@ -63,14 +85,16 @@ If the dependency does not include license text but does specify that it uses a
63
85
 
64
86
  *Resolution:* Review the changes to the license text and classification, along with other metadata contained in the cached file for the dependency. If the dependency is still allowable for use in your project, remove the `review_changed_license` key from the cached record file.
65
87
 
66
- ### license needs review
88
+ ### license needs review / dependency needs review
67
89
 
68
90
  *Cause:* A dependency is using a license that is not in the configured [allowed list of licenses][allowed], and the dependency has not been marked [ignored] or [reviewed].
69
91
  *Resolution:* Review the dependency's usage and specified license with someone familiar with OSS licensing and compliance rules to determine whether the dependency is allowable. Some common resolutions:
70
92
 
71
- 1. The dependency's specified license text differed enough from the standard license text that it was not recognized and classified as `other`. If, with human review, the license text is recognizable then update the `license: other` value in the cached metadata file to the correct license.
72
- - An updated classification will persist through version upgrades until the detected license contents have changed. The determination is made by [licensee/licensee](https://github.com/licensee/licensee), the library which this tool uses to detect and classify license contents.
93
+ 1. The dependency's specified license text differed enough from the standard license text that it was not recognized and classified as `other`.
94
+ - This resolution only applies when checking dependency status [with cached metadata files](./#checking-status-with-metadata-loaded-from-cached-files).
95
+ - If the cached license text is recognizable with human review then update the `license: other` value in the cached metadata file to the correct license. An updated classification will persist through version upgrades until the detected license contents have changed. The determination is made by [licensee/licensee](https://github.com/licensee/licensee), the library which this tool uses to detect and classify license contents.
73
96
  1. The dependency might need to be marked as [ignored] or [reviewed] if either of those scenarios are applicable.
97
+ - When checking status [with computed metadata](./#checking-status-with-computed-metadata), a reviewed entry must include both the dependency's name and the version that it was reviewed at e.g. `licensed@3.8.0`
74
98
  1. If the used license should be allowable without review (if your entity has a legal team, they may want to review this assessment), ensure the license SPDX is set as [allowed] in the licensed configuration file.
75
99
 
76
100
  [allowed]: ../configuration/allowed_licenses.md
data/lib/licensed/cli.rb CHANGED
@@ -26,8 +26,11 @@ module Licensed
26
26
  desc: "Individual source(s) to evaluate. Must also be enabled via configuration."
27
27
  method_option :format, aliases: "-f", enum: ["yaml", "json"],
28
28
  desc: "Output format"
29
+ method_option :data_source, aliases: "-d",
30
+ enum: ["files", "configuration"], default: "files",
31
+ desc: "Whether to check compliance status from cached records or the configuration file"
29
32
  def status
30
- run Licensed::Commands::Status.new(config: config), sources: options[:sources], reporter: options[:format]
33
+ run Licensed::Commands::Status.new(config: config), sources: options[:sources], reporter: options[:format], data_source: options[:data_source]
31
34
  end
32
35
 
33
36
  desc "list", "List dependencies"
@@ -43,13 +46,15 @@ module Licensed
43
46
  run Licensed::Commands::List.new(config: config), sources: options[:sources], reporter: options[:format], licenses: options[:licenses]
44
47
  end
45
48
 
46
- desc "notices", "Generate a NOTICE file from cached records"
49
+ desc "notices", "Generate a NOTICE file with dependency data"
47
50
  method_option :config, aliases: "-c", type: :string,
48
51
  desc: "Path to licensed configuration file"
49
52
  method_option :sources, aliases: "-s", type: :array,
50
53
  desc: "Individual source(s) to evaluate. Must also be enabled via configuration."
54
+ method_option :computed, aliases: "-l", type: :boolean,
55
+ desc: "Whether to generate a NOTICE file using computed data or cached records"
51
56
  def notices
52
- run Licensed::Commands::Notices.new(config: config), sources: options[:sources]
57
+ run Licensed::Commands::Notices.new(config: config), sources: options[:sources], computed: options[:computed]
53
58
  end
54
59
 
55
60
  map "-v" => :version
@@ -13,7 +13,7 @@ module Licensed
13
13
 
14
14
  protected
15
15
 
16
- # Load stored dependency record data to add to the notices report.
16
+ # Load a dependency record data and add it to the notices report.
17
17
  #
18
18
  # app - The application configuration for the dependency
19
19
  # source - The dependency source enumerator for the dependency
@@ -22,13 +22,36 @@ module Licensed
22
22
  #
23
23
  # Returns true.
24
24
  def evaluate_dependency(app, source, dependency, report)
25
+ report["record"] =
26
+ if load_dependency_record_from_files
27
+ load_cached_dependency_record(app, source, dependency, report)
28
+ else
29
+ dependency.record
30
+ end
31
+
32
+ true
33
+ end
34
+
35
+ # Loads a dependency record from a cached file.
36
+ #
37
+ # app - The application configuration for the dependency
38
+ # source - The dependency source enumerator for the dependency
39
+ # dependency - An application dependency
40
+ # report - A report hash for the command to provide extra data for the report output.
41
+ #
42
+ # Returns a dependency record or nil if one doesn't exist
43
+ def load_cached_dependency_record(app, source, dependency, report)
25
44
  filename = app.cache_path.join(source.class.type, "#{dependency.name}.#{DependencyRecord::EXTENSION}")
26
- report["cached_record"] = Licensed::DependencyRecord.read(filename)
27
- if !report["cached_record"]
45
+ record = Licensed::DependencyRecord.read(filename)
46
+ if !record
28
47
  report.warnings << "expected cached record not found at #{filename}"
29
48
  end
30
49
 
31
- true
50
+ record
51
+ end
52
+
53
+ def load_dependency_record_from_files
54
+ !options.fetch(:computed, false)
32
55
  end
33
56
  end
34
57
  end
@@ -29,33 +29,39 @@ module Licensed
29
29
  end
30
30
  end
31
31
 
32
- # Verifies that a cached record exists, is up to date and
33
- # has license data that complies with the licensed configuration.
32
+ # Evaluates a dependency for any compliance errors.
33
+ # Checks a dependency against either a cached metadata record or
34
+ # reviewed entries in the configuration file.
34
35
  #
35
36
  # app - The application configuration for the dependency
36
37
  # source - The dependency source enumerator for the dependency
37
38
  # dependency - An application dependency
38
39
  # report - A report hash for the command to provide extra data for the report output.
39
40
  #
40
- # Returns whether the dependency has a cached record that is compliant
41
+ # Returns whether the dependency is compliant
41
42
  # with the licensed configuration.
42
43
  def evaluate_dependency(app, source, dependency, report)
43
- filename = app.cache_path.join(source.class.type, "#{dependency.name}.#{DependencyRecord::EXTENSION}")
44
- report["filename"] = filename
45
44
  report["version"] = dependency.version
46
45
 
47
- cached_record = cached_record(filename)
48
- if cached_record.nil?
46
+ if data_source == "configuration"
47
+ record = dependency.record
48
+ else
49
+ filename = app.cache_path.join(source.class.type, "#{dependency.name}.#{DependencyRecord::EXTENSION}")
50
+ report["filename"] = filename
51
+ record = cached_record(filename)
52
+ end
53
+
54
+ if record.nil?
49
55
  report["license"] = nil
50
56
  report.errors << "cached dependency record not found"
51
57
  else
52
- report["license"] = cached_record["license"]
53
- report.errors << "cached dependency record out of date" if cached_record["version"] != dependency.version
54
- report.errors << "missing license text" if cached_record.licenses.empty?
55
- if cached_record["review_changed_license"]
58
+ report["license"] = record["license"]
59
+ report.errors << "dependency record out of date" if record["version"] != dependency.version
60
+ report.errors << "missing license text" if record.licenses.empty?
61
+ if record["review_changed_license"]
56
62
  report.errors << "license text has changed and needs re-review. if the new text is ok, remove the `review_changed_license` flag from the cached record"
57
- elsif license_needs_review?(app, cached_record)
58
- report.errors << "license needs review: #{cached_record["license"]}"
63
+ elsif license_needs_review?(app, record)
64
+ report.errors << needs_review_error_message(app, record)
59
65
  end
60
66
  end
61
67
 
@@ -64,19 +70,20 @@ module Licensed
64
70
 
65
71
  # Returns true if a cached record needs further review based on the
66
72
  # record's license(s) and the app's configuration
67
- def license_needs_review?(app, cached_record)
73
+ def license_needs_review?(app, record)
68
74
  # review is not needed if the record is set as reviewed
69
- return false if app.reviewed?(cached_record)
75
+ return false if app.reviewed?(record, match_version: data_source == "configuration")
76
+
70
77
  # review is not needed if the top level license is allowed
71
- return false if app.allowed?(cached_record["license"])
78
+ return false if app.allowed?(record["license"])
72
79
 
73
80
  # the remaining checks are meant to allow records marked as "other"
74
81
  # that have multiple licenses, all of which are allowed
75
82
 
76
83
  # review is needed for non-"other" licenses
77
- return true unless cached_record["license"] == "other"
84
+ return true unless record["license"] == "other"
78
85
 
79
- licenses = cached_record.licenses.map { |license| license_from_text(license.text) }
86
+ licenses = record.licenses.map { |license| license_from_text(license.text) }
80
87
 
81
88
  # review is needed when there is only one license notice
82
89
  # this is a performance optimization for the single license case
@@ -86,6 +93,29 @@ module Licensed
86
93
  licenses.any? { |license| !app.allowed?(license) }
87
94
  end
88
95
 
96
+ def needs_review_error_message(app, record)
97
+ return "license needs review: #{record["license"]}" if data_source == "files"
98
+
99
+ error = "dependency needs review"
100
+
101
+ # look for an unversioned reviewed list match
102
+ if app.reviewed?(record, match_version: false)
103
+ error += ", unversioned 'reviewed' match found: #{record["name"]}"
104
+ end
105
+
106
+ # look for other version matches in reviewed list
107
+ possible_matches = app.reviewed_versions(record)
108
+ if possible_matches.any?
109
+ error += ", possible 'reviewed' matches found at other versions: #{possible_matches.join(", ")}"
110
+ end
111
+
112
+ error
113
+ end
114
+
115
+ def data_source
116
+ options[:data_source] || "files"
117
+ end
118
+
89
119
  def cached_record(filename)
90
120
  return nil unless File.exist?(filename)
91
121
  DependencyRecord.read(filename)
@@ -73,17 +73,18 @@ module Licensed
73
73
  end
74
74
 
75
75
  # Is the given dependency reviewed?
76
- def reviewed?(dependency)
77
- Array(self["reviewed"][dependency["type"]]).any? do |pattern|
78
- File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
79
- end
76
+ def reviewed?(dependency, match_version: false)
77
+ any_list_pattern_matched? self["reviewed"][dependency["type"]], dependency, match_version: match_version
78
+ end
79
+
80
+ # Find all reviewed dependencies that match the provided dependency's name
81
+ def reviewed_versions(dependency)
82
+ similar_list_patterns self["reviewed"][dependency["type"]], dependency
80
83
  end
81
84
 
82
85
  # Is the given dependency ignored?
83
86
  def ignored?(dependency)
84
- Array(self["ignored"][dependency["type"]]).any? do |pattern|
85
- File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
86
- end
87
+ any_list_pattern_matched? self["ignored"][dependency["type"]], dependency
87
88
  end
88
89
 
89
90
  # Is the license of the dependency allowed?
@@ -97,8 +98,10 @@ module Licensed
97
98
  end
98
99
 
99
100
  # Set a dependency as reviewed
100
- def review(dependency)
101
- (self["reviewed"][dependency["type"]] ||= []) << dependency["name"]
101
+ def review(dependency, at_version: false)
102
+ id = dependency["name"]
103
+ id += "@#{dependency["version"]}" if at_version && dependency["version"]
104
+ (self["reviewed"][dependency["type"]] ||= []) << id
102
105
  end
103
106
 
104
107
  # Set a license as explicitly allowed
@@ -108,6 +111,27 @@ module Licensed
108
111
 
109
112
  private
110
113
 
114
+ def any_list_pattern_matched?(list, dependency, match_version: false)
115
+ Array(list).any? do |pattern|
116
+ if match_version
117
+ at_version = "@#{dependency["version"]}"
118
+ pattern, pattern_version = pattern.rpartition(at_version).values_at(0, 1)
119
+ next false if pattern == "" || pattern_version == ""
120
+ end
121
+
122
+ File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
123
+ end
124
+ end
125
+
126
+ def similar_list_patterns(list, dependency)
127
+ Array(list).select do |pattern|
128
+ pattern, version = pattern.rpartition("@").values_at(0, 2)
129
+ next if pattern == "" || version == ""
130
+
131
+ File.fnmatch?(pattern, dependency["name"], File::FNM_PATHNAME | File::FNM_CASEFOLD)
132
+ end
133
+ end
134
+
111
135
  # Returns the cache path for the application based on:
112
136
  # 1. An explicitly set cache path for the application, if set
113
137
  # 2. An inherited shared cache path
@@ -54,11 +54,11 @@ module Licensed
54
54
  def notices(report)
55
55
  return unless report.target.is_a?(Licensed::Dependency)
56
56
 
57
- cached_record = report["cached_record"]
58
- return unless cached_record
57
+ record = report["record"]
58
+ return unless record
59
59
 
60
- texts = cached_record.licenses.map(&:text)
61
- cached_record.notices.each do |notice|
60
+ texts = record.licenses.map(&:text)
61
+ record.notices.each do |notice|
62
62
  case notice
63
63
  when Hash
64
64
  texts << notice["text"]
@@ -70,7 +70,7 @@ module Licensed
70
70
  end
71
71
 
72
72
  <<~NOTICE
73
- #{cached_record["name"]}@#{cached_record["version"]}
73
+ #{record["name"]}@#{record["version"]}
74
74
 
75
75
  #{texts.map(&:strip).reject(&:empty?).compact.join(TEXT_SEPARATOR)}
76
76
  NOTICE
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
  module Licensed
3
- VERSION = "3.7.5".freeze
3
+ VERSION = "3.9.0".freeze
4
4
 
5
5
  def self.previous_major_versions
6
6
  major_version = Gem::Version.new(Licensed::VERSION).segments.first
data/licensed.gemspec CHANGED
@@ -35,7 +35,7 @@ Gem::Specification.new do |spec|
35
35
 
36
36
  spec.add_development_dependency "rake", ">= 12.3.3"
37
37
  spec.add_development_dependency "minitest", "~> 5.8"
38
- spec.add_development_dependency "mocha", "~> 1.0"
38
+ spec.add_development_dependency "mocha", "~> 2.0"
39
39
  spec.add_development_dependency "rubocop-github", "~> 0.6"
40
40
  spec.add_development_dependency "byebug", "~> 11.1.3"
41
41
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: licensed
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.7.5
4
+ version: 3.9.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - GitHub
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2022-10-20 00:00:00.000000000 Z
11
+ date: 2022-11-14 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: licensee
@@ -188,14 +188,14 @@ dependencies:
188
188
  requirements:
189
189
  - - "~>"
190
190
  - !ruby/object:Gem::Version
191
- version: '1.0'
191
+ version: '2.0'
192
192
  type: :development
193
193
  prerelease: false
194
194
  version_requirements: !ruby/object:Gem::Requirement
195
195
  requirements:
196
196
  - - "~>"
197
197
  - !ruby/object:Gem::Version
198
- version: '1.0'
198
+ version: '2.0'
199
199
  - !ruby/object:Gem::Dependency
200
200
  name: rubocop-github
201
201
  requirement: !ruby/object:Gem::Requirement