licensed 2.15.2 → 3.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/test.yml +55 -11
  3. data/CHANGELOG.md +56 -1
  4. data/README.md +38 -81
  5. data/docs/adding_a_new_source.md +11 -8
  6. data/docs/commands/README.md +59 -0
  7. data/docs/commands/cache.md +35 -0
  8. data/docs/commands/env.md +10 -0
  9. data/docs/commands/list.md +23 -0
  10. data/docs/commands/migrate.md +10 -0
  11. data/docs/commands/notices.md +12 -0
  12. data/docs/commands/status.md +73 -0
  13. data/docs/commands/version.md +3 -0
  14. data/docs/configuration.md +9 -161
  15. data/docs/configuration/README.md +11 -0
  16. data/docs/configuration/allowed_licenses.md +17 -0
  17. data/docs/configuration/application_name.md +63 -0
  18. data/docs/configuration/application_source.md +64 -0
  19. data/docs/configuration/configuration_root.md +27 -0
  20. data/docs/configuration/configuring_multiple_apps.md +58 -0
  21. data/docs/configuration/dependency_source_enumerators.md +28 -0
  22. data/docs/configuration/ignoring_dependencies.md +19 -0
  23. data/docs/configuration/metadata_cache.md +106 -0
  24. data/docs/configuration/reviewing_dependencies.md +18 -0
  25. data/docs/{migrating_to_newer_versions.md → migrations/v2.md} +1 -1
  26. data/docs/migrations/v3.md +109 -0
  27. data/docs/sources/bundler.md +1 -11
  28. data/docs/sources/swift.md +4 -0
  29. data/lib/licensed.rb +1 -0
  30. data/lib/licensed/cli.rb +6 -3
  31. data/lib/licensed/commands/cache.rb +19 -20
  32. data/lib/licensed/commands/command.rb +104 -72
  33. data/lib/licensed/commands/environment.rb +12 -11
  34. data/lib/licensed/commands/list.rb +0 -19
  35. data/lib/licensed/commands/notices.rb +0 -19
  36. data/lib/licensed/commands/status.rb +13 -15
  37. data/lib/licensed/configuration.rb +105 -12
  38. data/lib/licensed/report.rb +44 -0
  39. data/lib/licensed/reporters/cache_reporter.rb +48 -64
  40. data/lib/licensed/reporters/json_reporter.rb +19 -21
  41. data/lib/licensed/reporters/list_reporter.rb +45 -58
  42. data/lib/licensed/reporters/notices_reporter.rb +33 -46
  43. data/lib/licensed/reporters/reporter.rb +37 -104
  44. data/lib/licensed/reporters/status_reporter.rb +58 -56
  45. data/lib/licensed/reporters/yaml_reporter.rb +19 -21
  46. data/lib/licensed/sources.rb +1 -0
  47. data/lib/licensed/sources/bundler.rb +36 -217
  48. data/lib/licensed/sources/bundler/missing_specification.rb +54 -0
  49. data/lib/licensed/sources/go.rb +1 -1
  50. data/lib/licensed/sources/gradle.rb +2 -2
  51. data/lib/licensed/sources/npm.rb +4 -3
  52. data/lib/licensed/sources/nuget.rb +57 -27
  53. data/lib/licensed/sources/swift.rb +69 -0
  54. data/lib/licensed/version.rb +1 -1
  55. data/script/source-setup/go +1 -1
  56. data/script/source-setup/swift +22 -0
  57. metadata +27 -4
  58. data/docs/commands.md +0 -95
@@ -33,11 +33,12 @@ module Licensed
33
33
 
34
34
  def enumerate_dependencies
35
35
  packages.map do |name, package|
36
- path = package["path"]
36
+ errors = package["problems"] unless package["path"]
37
37
  Dependency.new(
38
38
  name: name,
39
- version: package["version"],
40
- path: path,
39
+ version: package["version"] || package["required"],
40
+ path: package["path"],
41
+ errors: Array(errors),
41
42
  metadata: {
42
43
  "type" => NPM.type,
43
44
  "name" => package["name"],
@@ -164,7 +164,7 @@ module Licensed
164
164
  end
165
165
 
166
166
  def project_assets_file_path
167
- File.join(config.pwd, "project.assets.json")
167
+ File.join(config.pwd, nuget_obj_path, "project.assets.json")
168
168
  end
169
169
 
170
170
  def project_assets_file
@@ -172,6 +172,17 @@ module Licensed
172
172
  @project_assets_file = File.read(project_assets_file_path)
173
173
  end
174
174
 
175
+ def project_assets_json
176
+ @project_assets_json ||= JSON.parse(project_assets_file)
177
+ rescue JSON::ParserError => e
178
+ message = "Licensed was unable to read the project.assets.json file. Error: #{e.message}"
179
+ raise Licensed::Sources::Source::Error, message
180
+ end
181
+
182
+ def nuget_obj_path
183
+ config.dig("nuget", "obj_path") || ""
184
+ end
185
+
175
186
  def enabled?
176
187
  File.exist?(project_assets_file_path)
177
188
  end
@@ -180,32 +191,51 @@ module Licensed
180
191
  # Ideally we'd use `dotnet list package` instead, but its output isn't
181
192
  # easily machine readable and doesn't contain everything we need.
182
193
  def enumerate_dependencies
183
- json = JSON.parse(project_assets_file)
184
- nuget_packages_dir = json["project"]["restore"]["packagesPath"]
185
- json["targets"].each_with_object({}) do |(_, target), dependencies|
186
- target.each do |reference_key, reference|
187
- # Ignore project references
188
- next unless reference["type"] == "package"
189
- package_id_parts = reference_key.partition("/")
190
- name = package_id_parts[0]
191
- version = package_id_parts[-1]
192
- id = "#{name}-#{version}"
193
-
194
- # Already know this package from another target
195
- next if dependencies.key?(id)
196
-
197
- path = File.join(nuget_packages_dir, json["libraries"][reference_key]["path"])
198
- dependencies[id] = NuGetDependency.new(
199
- name: id,
200
- version: version,
201
- path: path,
202
- metadata: {
203
- "type" => NuGet.type,
204
- "name" => name
205
- }
206
- )
207
- end
208
- end.values
194
+ reference_keys.map do |reference_key|
195
+ package_id_parts = reference_key.partition("/")
196
+ name = package_id_parts[0]
197
+ version = package_id_parts[-1]
198
+ id = "#{name}-#{version}"
199
+
200
+ path = full_dependency_path(reference_key)
201
+ error = "Package #{id} path was not found in project.assets.json, or does not exist on disk at any project package folder" if path.nil?
202
+
203
+ NuGetDependency.new(
204
+ name: id,
205
+ version: version,
206
+ path: path,
207
+ errors: Array(error),
208
+ metadata: {
209
+ "type" => NuGet.type,
210
+ "name" => name
211
+ }
212
+ )
213
+ end
214
+ end
215
+
216
+ # Returns a unique set of the package reference keys used across all target groups
217
+ def reference_keys
218
+ all_reference_keys = project_assets_json["targets"].flat_map do |_, references|
219
+ references.select { |key, reference| reference["type"] == "package" }
220
+ .keys
221
+ end
222
+
223
+ Set.new(all_reference_keys)
224
+ end
225
+
226
+ # Returns a dependency's path, if it exists, in one of the project's global or fallback package folders
227
+ def full_dependency_path(reference_key)
228
+ dependency_path = project_assets_json.dig("libraries", reference_key, "path")
229
+ return unless dependency_path
230
+
231
+ nuget_package_dirs = [
232
+ project_assets_json.dig("project", "restore", "packagesPath"),
233
+ *Array(project_assets_json.dig("project", "restore", "fallbackFolders"))
234
+ ].compact
235
+
236
+ nuget_package_dirs.map { |dir| File.join(dir, dependency_path) }
237
+ .select { |path| File.directory?(path) }
238
+ .first
209
239
  end
210
240
  end
211
241
  end
@@ -0,0 +1,69 @@
1
+ # frozen_string_literal: true
2
+ require "json"
3
+ require "pathname"
4
+ require "uri"
5
+
6
+ module Licensed
7
+ module Sources
8
+ class Swift < Source
9
+ def enabled?
10
+ return unless Licensed::Shell.tool_available?("swift") && swift_package?
11
+ File.exist?(package_resolved_file_path)
12
+ end
13
+
14
+ def enumerate_dependencies
15
+ pins.map { |pin|
16
+ name = pin["package"]
17
+ version = pin.dig("state", "version")
18
+ path = dependency_path_for_url(pin["repositoryURL"])
19
+ error = "Unable to determine project path from #{url}" unless path
20
+
21
+ Dependency.new(
22
+ name: name,
23
+ path: path,
24
+ version: version,
25
+ errors: Array(error),
26
+ metadata: {
27
+ "type" => Swift.type,
28
+ "homepage" => homepage_for_url(pin["repositoryURL"])
29
+ }
30
+ )
31
+ }
32
+ end
33
+
34
+ private
35
+
36
+ def pins
37
+ return @pins if defined?(@pins)
38
+
39
+ @pins = begin
40
+ json = JSON.parse(File.read(package_resolved_file_path))
41
+ json.dig("object", "pins")
42
+ rescue => e
43
+ message = "Licensed was unable to read the Package.resolved file. Error: #{e.message}"
44
+ raise Licensed::Sources::Source::Error, message
45
+ end
46
+ end
47
+
48
+ def dependency_path_for_url(url)
49
+ last_path_component = URI(url).path.split("/").last.sub(/\.git$/, "")
50
+ File.join(config.pwd, ".build", "checkouts", last_path_component)
51
+ rescue URI::InvalidURIError
52
+ end
53
+
54
+ def homepage_for_url(url)
55
+ return unless %w{http https}.include?(URI(url).scheme)
56
+ url.sub(/\.git$/, "")
57
+ rescue URI::InvalidURIError
58
+ end
59
+
60
+ def package_resolved_file_path
61
+ File.join(config.pwd, "Package.resolved")
62
+ end
63
+
64
+ def swift_package?
65
+ Licensed::Shell.success?("swift", "package", "describe")
66
+ end
67
+ end
68
+ end
69
+ end
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
  module Licensed
3
- VERSION = "2.15.2".freeze
3
+ VERSION = "3.2.0".freeze
4
4
 
5
5
  def self.previous_major_versions
6
6
  major_version = Gem::Version.new(Licensed::VERSION).segments.first
@@ -25,7 +25,7 @@ if [ "$1" == "-f" ]; then
25
25
  fi
26
26
  fi
27
27
 
28
- (cd src/test && go get)
28
+ (export GO111MODULE=off && cd src/test && go get)
29
29
  if go help mod >/dev/null; then
30
30
  (cd src/modules_test && GO111MODULE=on go mod download)
31
31
  fi
@@ -0,0 +1,22 @@
1
+ #!/bin/bash
2
+ set -e
3
+
4
+ if [ -z "$(which swift)" ]; then
5
+ echo "A local swift installation is required for swift development." >&2
6
+ exit 127
7
+ fi
8
+
9
+ swift --version
10
+
11
+ BASE_PATH="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
12
+ cd $BASE_PATH/test/fixtures/swift
13
+
14
+ if [ "$1" == "-f" ]; then
15
+ find . -not -regex "\.*" \
16
+ -and -not -path "*/Package.swift" \
17
+ -and -not -path "*/Sources*" \
18
+ -and -not -path "*/Tests*" \
19
+ -print0 | xargs -0 rm -rf
20
+ fi
21
+
22
+ swift package resolve
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: licensed
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.15.2
4
+ version: 3.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - GitHub
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2021-04-06 00:00:00.000000000 Z
11
+ date: 2021-08-19 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: licensee
@@ -241,9 +241,27 @@ files:
241
241
  - Rakefile
242
242
  - docker/Dockerfile.build-linux
243
243
  - docs/adding_a_new_source.md
244
- - docs/commands.md
244
+ - docs/commands/README.md
245
+ - docs/commands/cache.md
246
+ - docs/commands/env.md
247
+ - docs/commands/list.md
248
+ - docs/commands/migrate.md
249
+ - docs/commands/notices.md
250
+ - docs/commands/status.md
251
+ - docs/commands/version.md
245
252
  - docs/configuration.md
246
- - docs/migrating_to_newer_versions.md
253
+ - docs/configuration/README.md
254
+ - docs/configuration/allowed_licenses.md
255
+ - docs/configuration/application_name.md
256
+ - docs/configuration/application_source.md
257
+ - docs/configuration/configuration_root.md
258
+ - docs/configuration/configuring_multiple_apps.md
259
+ - docs/configuration/dependency_source_enumerators.md
260
+ - docs/configuration/ignoring_dependencies.md
261
+ - docs/configuration/metadata_cache.md
262
+ - docs/configuration/reviewing_dependencies.md
263
+ - docs/migrations/v2.md
264
+ - docs/migrations/v3.md
247
265
  - docs/packaging.md
248
266
  - docs/reporters.md
249
267
  - docs/sources/bower.md
@@ -261,6 +279,7 @@ files:
261
279
  - docs/sources/pip.md
262
280
  - docs/sources/pipenv.md
263
281
  - docs/sources/stack.md
282
+ - docs/sources/swift.md
264
283
  - docs/sources/yarn.md
265
284
  - exe/licensed
266
285
  - lib/licensed.rb
@@ -278,6 +297,7 @@ files:
278
297
  - lib/licensed/git.rb
279
298
  - lib/licensed/migrations.rb
280
299
  - lib/licensed/migrations/v2.rb
300
+ - lib/licensed/report.rb
281
301
  - lib/licensed/reporters.rb
282
302
  - lib/licensed/reporters/cache_reporter.rb
283
303
  - lib/licensed/reporters/json_reporter.rb
@@ -290,6 +310,7 @@ files:
290
310
  - lib/licensed/sources.rb
291
311
  - lib/licensed/sources/bower.rb
292
312
  - lib/licensed/sources/bundler.rb
313
+ - lib/licensed/sources/bundler/missing_specification.rb
293
314
  - lib/licensed/sources/cabal.rb
294
315
  - lib/licensed/sources/composer.rb
295
316
  - lib/licensed/sources/dep.rb
@@ -304,6 +325,7 @@ files:
304
325
  - lib/licensed/sources/pip.rb
305
326
  - lib/licensed/sources/pipenv.rb
306
327
  - lib/licensed/sources/source.rb
328
+ - lib/licensed/sources/swift.rb
307
329
  - lib/licensed/sources/yarn.rb
308
330
  - lib/licensed/ui/shell.rb
309
331
  - lib/licensed/version.rb
@@ -327,6 +349,7 @@ files:
327
349
  - script/source-setup/nuget
328
350
  - script/source-setup/pip
329
351
  - script/source-setup/pipenv
352
+ - script/source-setup/swift
330
353
  - script/source-setup/yarn
331
354
  - script/test
332
355
  homepage: https://github.com/github/licensed
data/docs/commands.md DELETED
@@ -1,95 +0,0 @@
1
- # Commands
2
-
3
- Run `licensed -h` to see help content for running licensed commands.
4
-
5
- ## `list`
6
-
7
- Running the list command finds the dependencies for all sources in all configured applications. No additional actions are taken on each dependency.
8
-
9
- An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source.
10
-
11
- ## `cache`
12
-
13
- The cache command finds all dependencies and ensures that each dependency has an up-to-date cached record.
14
-
15
- An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source.
16
-
17
- Dependency records will be saved if:
18
- 1. The `force` option is set
19
- 2. No cached record is found
20
- 3. The cached record's version is different than the current dependency's version
21
- - If the cached record's license text contents matches the current dependency's license text then the `license` metadata from the cached record is retained for the new saved record.
22
-
23
- After the cache command is run, any cached records that don't match up to a current application dependency will be deleted.
24
-
25
- ## `status`
26
-
27
- The status command finds all dependencies and checks whether each dependency has a valid cached record.
28
-
29
- An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source.
30
-
31
- A dependency will fail the status checks if:
32
- 1. No cached record is found
33
- 2. The cached record's version is different than the current dependency's version
34
- 3. The cached record's `licenses` data is empty
35
- 4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration.
36
- - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged
37
- 5. The cached record is flagged for re-review.
38
- - This occurs when the record's license text has changed since the record was reviewed.
39
-
40
- ## `notices`
41
-
42
- Outputs license and notice text for all dependencies in each app into a `NOTICE` file in the app's `cache_path`. If an app uses a shared cache path, the file name will contain the app name as well, e.g. `NOTICE.my_app`.
43
-
44
- An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source.
45
-
46
- The `NOTICE` file contents are retrieved from cached records, with the assumption that cached records have already been reviewed in a compliance workflow.
47
-
48
- ## `env`
49
-
50
- Prints the runtime environment used by licensed after loading a configuration file. By default the output is in YAML format, but can be output in JSON using the `--json` flag.
51
-
52
- The output will not be equivalent to configuration input. For example, all paths will be
53
-
54
- ## `version`
55
-
56
- Displays the current licensed version.
57
-
58
- # Adding a new command
59
-
60
- ## Implement new `Command` class
61
-
62
- Licensed commands inherit and override the [`Licensed::Sources::Command`](../lib/licensed/commands/command.rb) class.
63
-
64
- #### Required method overrides
65
- 1. `Licensed::Commands::Command#evaluate_dependency`
66
- - Runs a command execution on an application dependency.
67
-
68
- The `evaluate_dependency` method should contain the specific command logic. This method has access to the application configuration, dependency source enumerator and dependency currently being evaluated as well as a reporting hash to contain information about the command execution.
69
-
70
- #### Optional method overrides
71
-
72
- The following methods break apart the different levels of command execution. Each method wraps lower levels of command execution in a corresponding reporter method.
73
-
74
- 1. `Licensed::Commands::Command#run`
75
- - Runs `run_app` for each application configuration found. Wraps the execution of all applications in `Reporter#report_run`.
76
- 2. `Licensed::Commands::Command#run_app`
77
- - Runs `run_source` for each dependency source enumerator enabled for the application configuration. Wraps the execution of all sources in `Reporter#report_app`.
78
- 3. `Licensed::Commands::Command#run_source`
79
- - Runs `run_dependency` for each dependency found in the source. Wraps the execution of all dependencies in `Reporter#report_source`.
80
- 4. `Licensed::Commands::Command#run_dependency`
81
- - Runs `evaluate_dependency` for the dependency. Wraps the execution of all dependencies in `Reporter#report_dependency`.
82
-
83
- As an example, `Licensed::Commands::Command#run_app` calls `Reporter#report_app` to wrap every call to `Licensed::Commands::Command#run_source`.
84
-
85
- ##### Specifying additional report data
86
-
87
- The `run` methods can be overridden and pass a block to `super` to provide additional reporting data or functionality.
88
-
89
- ```ruby
90
- def run_app(app)
91
- super do |report|
92
- report["my_app_data"] = true
93
- end
94
- end
95
- ```