licensed 2.9.0 → 2.11.1

data/CHANGELOG.md

@@ -6,17 +6,55 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ## [Unreleased]
 
+## 2.11.1
+2020-06-09
+
+### Fixed
+- `notices` command properly reads cached dependency notices contents (https://github.com/github/licensed/pull/283)
+
+## 2.11.0
+2020-06-02
+
+### Added
+- `notices` command to create a `NOTICE` file for each configured app (https://github.com/github/licensed/pull/277)
+
+### Fixed
+- NuGet source no longer crashes on a non-existent dependency path (https://github.com/github/licensed/pull/280)
+- Go source no longer crashes on a non-existent dependency package path (https://github.com/github/licensed/pull/274)
+
+## 2.10.0
+2020-05-15
+
+### Changed
+- NPM source ignores missing peer dependencies (https://github.com/github/licensed/pull/267)
+
+### Added
+- NuGet source (:tada: @zarenner https://github.com/github/licensed/pull/261)
+- Multiple apps can share a single cache location (https://github.com/github/licensed/pull/263)
+
+## 2.9.2
+2020-04-28
+
+### Changed
+- `licensee` minimum version bumped to 9.13.2 (https://github.com/github/licensed/pull/256)
+
+## 2.9.1
+2020-03-24
+
+### Changed
+- relaxed gem version restrictions on Thor (:tada: @eileencodes https://github.com/github/licensed/pull/254)
+
 ## 2.9.0
 2020-03-19
 
-## Added
+### Added
 - Source paths use glob pattern matching (https://github.com/github/licensed/pull/245)
 
-## Fixed
+### Fixed
 - Mix source supports updates to mix.lock format (:tada: @bruce https://github.com/github/licensed/pull/242)
 - Go source supports `go list` format changes in go 1.14 (https://github.com/github/licensed/pull/247)
 
-## Changed
+### Changed
 - `licensed cache` will flag dependencies for re-review when license text changes (https://github.com/github/licensed/pull/248)
 - `licensed status` will raise errors on dependencies that need re-review (https://github.com/github/licensed/pull/248)
 - `licensee` minimum version bumped to 9.13.1 (https://github.com/github/licensed/pull/251)
@@ -280,4 +318,4 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 Initial release :tada:
 
-[Unreleased]: https://github.com/github/licensed/compare/2.9.0...HEAD
+[Unreleased]: https://github.com/github/licensed/compare/2.11.1...HEAD

data/README.md

@@ -24,13 +24,13 @@ See the [migration documentation](./docs/migrating_to_newer_versions.md) for mor
 ### Dependencies
 
 Licensed uses the `libgit2` bindings for Ruby provided by `rugged`. `rugged` requires `cmake` and `pkg-config` which you may need to install before you can install Licensed.
-   
+
    >  Ubuntu
-    
+
     sudo apt-get install cmake pkg-config
-    
+
    >  OS X
-       
+
     brew install cmake pkg-config
 
 ### With a Gemfile
@@ -64,8 +64,10 @@ For system wide usage, install licensed to a location on `$PATH`, e.g. `/usr/loc
 
 - `licensed list`: Output enumerated dependencies only.
 - `licensed cache`: Cache licenses and metadata.
-- `licensed status`: Check status of dependencies' cached licenses. For example:
+- `licensed status`: Check status of dependencies' cached licenses.
+- `licensed notices`: Write a `NOTICE` file for each application configuration.
 - `licensed version`: Show current installed version of Licensed. Aliases: `-v|--version`
+- `licensed env`: Output environment information from the licensed configuration.
 
 See the [commands documentation](./docs/commands.md) for additional documentation, or run `licensed -h` to see all of the current available commands.
 
@@ -102,14 +104,16 @@ Dependencies will be automatically detected for all of the following sources by
 1. [Bundler](./docs/sources/bundler.md)
 1. [Cabal](./docs/sources/cabal.md)
 1. [Composer](./docs/sources/composer.md)
+1. [Git Submodules (git_submodule)](./docs/sources/git_submodule.md)
 1. [Go](./docs/sources/go.md)
 1. [Go Dep (dep)](./docs/sources/dep.md)
+1. [Gradle](./docs/sources/gradle.md)
 1. [Manifest lists (manifests)](./docs/sources/manifests.md)
+1. [Mix](./docs/sources/mix.md)
 1. [NPM](./docs/sources/npm.md)
+1. [NuGet](./docs/sources/nuget.md)
 1. [Pip](./docs/sources/pip.md)
 1. [Pipenv](./docs/sources/pipenv.md)
-1. [Git Submodules (git_submodule)](./docs/sources/git_submodule.md)
-1. [Mix](./docs/sources/mix.md)
 1. [Yarn](./docs/sources/yarn.md)
 
 You can disable any of them in the configuration file:

data/docs/commands.md

@@ -31,6 +31,12 @@ A dependency will fail the status checks if:
 5. The cached record is flagged for re-review.
    - This occurs when the record's license text has changed since the record was reviewed.
 
+## `notices`
+
+Outputs license and notice text for all dependencies in each app into a `NOTICE` file in the app's `cache_path`.  If an app uses a shared cache path, the file name will contain the app name as well, e.g. `NOTICE.my_app`.
+
+The `NOTICE` file contents are retrieved from cached records, with the assumption that cached records have already been reviewed in a compliance workflow.
+
 ## `env`
 
 Prints the runtime environment used by licensed after loading a configuration file.  By default the output is in YAML format, but can be output in JSON using the `--json` flag.

data/docs/configuration.md

@@ -209,6 +209,26 @@ apps:
 
 In this example, the root configuration will contain a default cache path of `.licenses`.  `app1` will inherit this value and append it's name, resulting in a cache path of `.licenses/app1`.
 
+### Sharing caches between apps
+
+Dependency caches can be shared between apps by setting the same cache path on each app.
+
+```yaml
+apps:
+  - source_path: "path/to/app1"
+    cache_path: ".licenses/apps"
+  - source_path: "path/to/app2"
+    cache_path: ".licenses/apps"
+```
+
+When using a source path with a glob pattern, the apps created from the glob pattern can share a dependency by setting an explicit cache path and setting `shared_cache` to true.
+
+```yaml
+source_path: "path/to/apps/*"
+cache_path: ".licenses/apps"
+shared_cache: true
+```
+
 ## Source specific configuration
 
 See the [source documentation](./sources) for details on any source specific configuration.

data/docs/sources/nuget.md

@@ -0,0 +1,14 @@
+# NuGet
+
+The NuGet source will detect ProjectReference-style restored packages by inspecting `project.assets.json` files for dependencies. It requires that `dotnet restore` has already ran on the project.
+
+The source currently expects that `source_path` is set to the `obj` directory containing the `project.assets.json`.
+For example, if your project lives at `foo/foo.proj`, you likely want to set `source_path` to `foo/obj`.
+If in MSBuild you have customized your `obj` paths (e.g. to live outside your source tree), you may need to set `source_path` to something different such as `../obj/foo`.
+
+### Search strategy
+This source looks for licenses:
+1. Specified by SPDX expression via `<license type="expression">` in a package's `.nuspec` (via licensee)
+2. In license files such as `LICENSE.txt`, even if not specified in the `.nuspec` (via licensee)
+3. Specified by filepath via `<license type="file">` in a package's `.nuspec`, even if not a standard license filename.
+4. By downloading and inspecting the contents of `<licenseUrl>` in a package's `.nuspec`, if not found otherwise.

data/lib/licensed/cli.rb

@@ -28,6 +28,13 @@ module Licensed
       run Licensed::Commands::List.new(config: config)
     end
 
+    desc "notices", "Generate a NOTICE file from cached records"
+    method_option :config, aliases: "-c", type: :string,
+      desc: "Path to licensed configuration file"
+    def notices
+      run Licensed::Commands::Notices.new(config: config)
+    end
+
     map "-v" => :version
     map "--version" => :version
     desc "version", "Show Installed Version of Licensed, [-v, --version]"

data/lib/licensed/commands.rb

@@ -6,5 +6,6 @@ module Licensed
     require "licensed/commands/status"
     require "licensed/commands/list"
     require "licensed/commands/environment"
+    require "licensed/commands/notices"
   end
 end

data/lib/licensed/commands/cache.rb

@@ -11,20 +11,39 @@ module Licensed
         Licensed::Reporters::CacheReporter.new
       end
 
+      # Run the command.
+      # Removes any cached records that don't match a current application
+      # dependency.
+      #
+      # options - Options to run the command with
+      #
+      # Returns whether the command was a success
+      def run(**options)
+        begin
+          result = super
+          clear_stale_cached_records if result
+
+          result
+        ensure
+          cache_paths.clear
+          files.clear
+        end
+      end
+
       protected
 
-      # Run the command for all enumerated dependencies found in a dependency source,
+      # Run the command for all enabled sources for an application configuration,
       # recording results in a report.
-      # Removes any cached records that don't match a current application
-      # dependency.
       #
-      # app - The application configuration for the source
-      # source - A dependency source enumerator
+      # app - An application configuration
       #
-      # Returns whether the command succeeded for the dependency source enumerator
-      def run_source(app, source)
+      # Returns whether the command succeeded for the application.
+      def run_app(app)
         result = super
-        clear_stale_cached_records(app, source) if result
+
+        # add the full cache path to the list of cache paths evaluted during this run
+        cache_paths << app.cache_path
+
         result
       end
 
@@ -62,6 +81,9 @@ module Licensed
           report.warnings << "expected dependency path #{dependency.path} does not exist"
         end
 
+        # add the absolute dependency file path to the list of files seen during this licensed run
+        files << filename.to_s
+
         true
       end
 
@@ -86,18 +108,26 @@ module Licensed
 
       # Clean up cached files that dont match current dependencies
       #
-      # app - An application configuration
-      # source - A dependency source enumerator
-      #
       # Returns nothing
-      def clear_stale_cached_records(app, source)
-        names = source.dependencies.map { |dependency| File.join(source.class.type, dependency.name) }
-        Dir.glob(app.cache_path.join(source.class.type, "**/*.#{DependencyRecord::EXTENSION}")).each do |file|
-          file_path = Pathname.new(file)
-          relative_path = file_path.relative_path_from(app.cache_path).to_s
-          FileUtils.rm(file) unless names.include?(relative_path.chomp(".#{DependencyRecord::EXTENSION}"))
+      def clear_stale_cached_records
+        cache_paths.each do |cache_path|
+          Dir.glob(cache_path.join("**/*.#{DependencyRecord::EXTENSION}")).each do |file|
+            next if files.include?(file)
+
+            FileUtils.rm(file)
+          end
         end
       end
+
+      # Set of unique cache paths that are evaluted during the run
+      def cache_paths
+        @cache_paths ||= Set.new
+      end
+
+      # Set of unique absolute file paths of cached records evaluted during the run
+      def files
+        @files ||= Set.new
+      end
     end
   end
 end

data/lib/licensed/commands/notices.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+module Licensed
+  module Commands
+    class Notices < Command
+      # Create a reporter to use during a command run
+      #
+      # options - The options the command was run with
+      #
+      # Raises a Licensed::Reporters::CacheReporter
+      def create_reporter(options)
+        Licensed::Reporters::NoticesReporter.new
+      end
+
+      protected
+
+      # Load stored dependency record data to add to the notices report.
+      #
+      # app - The application configuration for the dependency
+      # source - The dependency source enumerator for the dependency
+      # dependency - An application dependency
+      # report - A report hash for the command to provide extra data for the report output.
+      #
+      # Returns true.
+      def evaluate_dependency(app, source, dependency, report)
+        filename = app.cache_path.join(source.class.type, "#{dependency.name}.#{DependencyRecord::EXTENSION}")
+        report["cached_record"] = Licensed::DependencyRecord.read(filename)
+        if !report["cached_record"]
+          report["warning"] = "expected cached record not found at #{filename}"
+        end
+
+        true
+      end
+    end
+  end
+end

data/lib/licensed/configuration.rb

@@ -108,7 +108,12 @@ module Licensed
     def detect_cache_path(options, inherited_options)
       return options["cache_path"] unless options["cache_path"].to_s.empty?
 
-      cache_path = inherited_options["cache_path"] || DEFAULT_CACHE_PATH
+      # if cache_path and shared_cache are both set in inherited_options,
+      # don't append the app name to the cache path
+      cache_path = inherited_options["cache_path"]
+      return cache_path if cache_path && inherited_options["shared_cache"] == true
+
+      cache_path ||= DEFAULT_CACHE_PATH
       File.join(cache_path, self["name"])
     end
 
@@ -167,7 +172,12 @@ module Licensed
         # will handle configurations that don't have these explicitly set
         dir_name = File.basename(path)
         config["name"] = "#{config["name"]}-#{dir_name}" if config["name"]
-        config["cache_path"] = File.join(config["cache_path"], dir_name) if config["cache_path"]
+
+        # if a cache_path is set and is not marked as shared, append the app name
+        # to the end of the cache path to make a unique cache path for the app
+        if config["cache_path"] && config["shared_cache"] != true
+          config["cache_path"] = File.join(config["cache_path"], dir_name)
+        end
 
         config
       end

data/lib/licensed/dependency.rb

@@ -74,7 +74,7 @@ module Licensed
     def license_contents
       files = matched_files.reject { |f| f == package_file }
                            .group_by(&:content)
-                           .map { |content, files| { "sources" => license_content_sources(files), "text" => content } }
+                           .map { |content, sources| { "sources" => license_content_sources(sources), "text" => content } }
 
       files << generated_license_contents if files.empty?
       files.compact

data/lib/licensed/reporters.rb

@@ -7,5 +7,6 @@ module Licensed
     require "licensed/reporters/list_reporter"
     require "licensed/reporters/json_reporter"
     require "licensed/reporters/yaml_reporter"
+    require "licensed/reporters/notices_reporter"
   end
 end

data/lib/licensed/reporters/cache_reporter.rb

@@ -27,32 +27,32 @@ module Licensed
           shell.info "  #{source.class.type}"
           result = yield report
 
-          warning_reports = report.all_reports.select { |report| report.warnings.any? }.to_a
+          warning_reports = report.all_reports.select { |r| r.warnings.any? }.to_a
           if warning_reports.any?
             shell.newline
             shell.warn "  * Warnings:"
-            warning_reports.each do |report|
-              display_metadata = report.map { |k, v| "#{k}: #{v}" }.join(", ")
+            warning_reports.each do |r|
+              display_metadata = r.map { |k, v| "#{k}: #{v}" }.join(", ")
 
-              shell.warn "    * #{report.name}"
+              shell.warn "    * #{r.name}"
               shell.warn "    #{display_metadata}" unless display_metadata.empty?
-              report.warnings.each do |warning|
+              r.warnings.each do |warning|
                 shell.warn "      - #{warning}"
               end
               shell.newline
             end
           end
 
-          errored_reports = report.all_reports.select { |report| report.errors.any? }.to_a
+          errored_reports = report.all_reports.select { |r| r.errors.any? }.to_a
           if errored_reports.any?
             shell.newline
             shell.error "  * Errors:"
-            errored_reports.each do |report|
-              display_metadata = report.map { |k, v| "#{k}: #{v}" }.join(", ")
+            errored_reports.each do |r|
+              display_metadata = r.map { |k, v| "#{k}: #{v}" }.join(", ")
 
-              shell.error "    * #{report.name}"
+              shell.error "    * #{r.name}"
               shell.error "    #{display_metadata}" unless display_metadata.empty?
-              report.errors.each do |error|
+              r.errors.each do |error|
                 shell.error "      - #{error}"
               end
               shell.newline

data/lib/licensed/reporters/list_reporter.rb

@@ -28,16 +28,16 @@ module Licensed
           shell.info "  #{source.class.type}"
           result = yield report
 
-          errored_reports = report.all_reports.select { |report| report.errors.any? }.to_a
+          errored_reports = report.all_reports.select { |r| r.errors.any? }.to_a
           if errored_reports.any?
             shell.newline
             shell.error "  * Errors:"
-            errored_reports.each do |report|
-              display_metadata = report.map { |k, v| "#{k}: #{v}" }.join(", ")
+            errored_reports.each do |r|
+              display_metadata = r.map { |k, v| "#{k}: #{v}" }.join(", ")
 
-              shell.error "    * #{report.name}"
+              shell.error "    * #{r.name}"
               shell.error "    #{display_metadata}" unless display_metadata.empty?
-              report.errors.each do |error|
+              r.errors.each do |error|
                 shell.error "      - #{error}"
               end
               shell.newline

data/lib/licensed/reporters/notices_reporter.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+module Licensed
+  module Reporters
+    class NoticesReporter < Reporter
+      TEXT_SEPARATOR = "\n\n#{("-" * 5)}\n\n".freeze
+      LICENSE_SEPARATOR = "\n#{("*" * 5)}\n".freeze
+
+      # Reports on an application configuration in a notices command run
+      #
+      # app - An application configuration
+      #
+      # Returns the result of the yielded method
+      # Note - must be called from inside the `report_run` scope
+      def report_app(app)
+        super do |report|
+          filename = app["shared_cache"] ? "NOTICE.#{app["name"]}" : "NOTICE"
+          path = app.cache_path.join(filename)
+          shell.info "Writing notices for #{app["name"]} to #{path}"
+
+          result = yield report
+
+          File.open(path, "w") do |file|
+            file << "THIRD PARTY NOTICES\n"
+            file << LICENSE_SEPARATOR
+            file << report.all_reports
+                          .map { |r| notices(r) }
+                          .compact
+                          .join(LICENSE_SEPARATOR)
+          end
+
+          result
+        end
+      end
+
+      # Reports on a dependency in a notices command run.
+      #
+      # dependency - An application dependency
+      #
+      # Returns the result of the yielded method
+      # Note - must be called from inside the `report_run` scope
+      def report_dependency(dependency)
+        super do |report|
+          result = yield report
+          shell.warn "* #{report["warning"]}" if report["warning"]
+          result
+        end
+      end
+
+      # Returns notices information for a dependency report
+      def notices(report)
+        return unless report.target.is_a?(Licensed::Dependency)
+
+        cached_record = report["cached_record"]
+        return unless cached_record
+
+        texts = cached_record.licenses.map(&:text)
+        cached_record.notices.each do |notice|
+          case notice
+          when Hash
+            texts << notice["text"]
+          when String
+            texts << notice
+          else
+            shell.warn "* unable to parse notices for #{report.target.name}"
+          end
+        end
+
+        <<~NOTICE
+          #{cached_record["name"]}@#{cached_record["version"]}
+
+          #{texts.map(&:strip).reject(&:empty?).compact.join(TEXT_SEPARATOR)}
+        NOTICE
+      end
+    end
+  end
+end