licensed 3.0.1 → 3.2.2

data/docs/commands/status.md

@@ -0,0 +1,74 @@
+# `licensed status`
+
+The status command finds all dependencies and checks whether each dependency has a valid cached record.
+
+A dependency will fail the status checks if:
+
+1. No cached record is found
+2. The cached record's version is different than the current dependency's version
+3. The cached record's `licenses` data is empty
+4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration.
+   - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged
+5. The cached record is flagged for re-review.
+   - This occurs when the record's license text has changed since the record was reviewed.
+
+## Options
+
+- `--config`/`-c`: the path to the licensed configuration file
+   - default value: `./.licensed.yml`
+- `--sources`/`-s`: runtime filter on which dependency sources are run.  Sources must also be enabled in the licensed configuration file.
+   - default value: not set, all configured sources
+- `--format`/`-f`: the output format
+   - default value: `yaml`
+- `--force`: if set, forces all dependency metadata files to be recached
+   - default value: not set
+
+## Reported Data
+
+The following data is reported for each dependency when the YAML or JSON report formats are used
+
+- name: the licensed recognized name for the dependency including the app and source name
+   - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor`
+- allowed: true if the dependency has passed all checks, false otherwise
+- version: the version of the enumerated dependency
+- license: the dependency's SPDX license identifier
+- filename: the full path on disk to the dependency's cached metadata file, if available
+- errors: any error messages from failed status checks, if available
+
+## Status errors and resolutions
+
+### cached dependency record not found
+
+**Cause:** A dependency was found while running `licensed status` that does not have a corresponding cached metadata file
+**Resolution:** Run `licensed cache` to update the metadata cache and create the missing metadata file
+
+### cached dependency record out of date
+
+**Cause:** A dependency was found while running `licensed status` with a different version than is contained in the dependency's cached metadata file
+**Resolution:** Run `licensed cache` to update the out-of-date metadata files
+
+### missing license text
+
+**Cause:** A license determination was made, e.g. from package metadata, but no license text was found.
+**Resolution:** Manually verify whether the dependency includes a file containing license text.  If the dependency code that was downloaded locally does not contain the license text, please check the dependency source at the version listed in the dependency's cached metadata file to see if there is license text that can be used.
+
+If the dependency does not include license text but does specify that it uses a specific license, please copy the standard license text from a [well known source](https://opensource.org/licenses).
+
+### license text has changed and needs re-review. if the new text is ok, remove the `review_changed_license` flag from the cached record
+
+**Cause:** A dependency that is set as [reviewed] in the licensed configuration file has substantially changed and should be re-reviewed.
+**Resolution:** Review the changes to the license text and classification, along with other metadata contained in the cached file for the dependency.  If the dependency is still allowable for use in your project, remove the `review_changed_license` key from the cached record file.
+
+### license needs review
+
+**Cause:** A dependency is using a license that is not in the configured [allowed list of licenses][allowed], and the dependency has not been marked [ignored] or [reviewed].
+**Resolution:** Review the dependency's usage and specified license with someone familiar with OSS licensing and compliance rules to determine whether the dependency is allowable.  Some common resolutions:
+
+1. The dependency's specified license text differed enough from the standard license text that it was not recognized and classified as `other`.  If, with human review, the license text is recognizable then update the `license: other` value in the cached metadata file to the correct license.
+   - An updated classification will persist through version upgrades until the detected license contents have changed.  The determination is made by [licensee/licensee](https://github.com/licensee/licensee), the library which this tool uses to detect and classify license contents.
+1. The dependency might need to be marked as [ignored] or [reviewed] if either of those scenarios are applicable.
+1. If the used license should be allowable without review (if your entity has a legal team, they may want to review this assessment), ensure the license SPDX is set as [allowed] in the licensed configuration file.
+
+[allowed]: ../configuration/allowed_licenses.md
+[ignored]: ../configuration/ignoring_dependencies.md
+[reviewed]: ../configuration/reviewing_dependencies.md

data/docs/commands/version.md

@@ -0,0 +1,3 @@
+# `licensed version`
+
+Displays the current licensed version.  Can also be run with `licensed -v` and `licensed --version`.

data/docs/configuration/README.md

@@ -0,0 +1,11 @@
+# Configuration options
+
+1. [Application source path](./application_source.md)
+1. [Dependency metadata cache](./metadata_cache.md)
+1. [Configuring multiple applications / monorepo support](./configuring_multiple_apps.md)
+1. [Configuration root](./configuration_root.md)
+1. [Application name](./application_name.md)
+1. [Dependency source enumerators](./dependency_source_enumerators.md)
+1. [Allowed licenses](./allowed_licenses.md)
+1. [Ignoring dependencies](./ignoring_dependencies.md)
+1. [Reviewing dependencies](./reviewing_dependencies.md)

data/docs/configuration/allowed_licenses.md

@@ -0,0 +1,17 @@
+# Allowed licenses
+
+**Key**: allowed
+**Default Value**: none
+
+The list of allowed licenses is used with the [status command](../commands/status.md) to detail which licenses are allowable for use in the current project and do not need further review.  If a dependency uses a license that is not included in the allowed list, and the dependency is not on the ignored or reviewed dependency lists, it will be flagged and the status command will fail.
+
+This configuration value accepts an array of lower-cased [open source license SPDX identifiers](https://spdx.org/licenses/).
+
+```yml
+# accepts lowercase SPDX license identifiers
+allowed:
+  - mit
+  - bsd-2-clause
+  - bsd-3-clause
+  - isc
+```

data/docs/configuration/application_name.md

@@ -0,0 +1,63 @@
+# Application name
+
+**Key**: name  
+**Default value**: The directory name of the application's source path
+
+The name of the application is primarily used for organizational and display purposes.  Application names are not included with
+dependency metadata information included in cached files.
+
+```yml
+source_path: path/to/application1
+name: application1
+```
+
+## Dynamically generated application names
+
+### Source path directory name
+
+When not specified, an application's name will be the directory name from the application's source path.
+
+```yml
+# if not explicitly set the name will be inferred from the source path
+source_path: path/to/application1
+# name: application1
+
+# or, use the `directory_name` name generator to explicitly set this behavior
+source_path: path/to/application1
+name:
+  generator: directory_name
+# name: application1
+```
+
+### Relative path from configuration root
+
+Application names can be created from the path from the configuration root to the application source path.
+This can be useful when specifying multiple applications using a glob pattern in a directory structure where directory names
+are not unique.
+
+As an example, given the following directory structure and configuration YML, the resulting application names
+would be `linux.installer`, `windows.installer` and `mac.installer`.  The optional arguments `separator` and `depth` are used
+to better control the resulting application name.
+
+```text
+path
+|_to
+  |_linux
+    |_installer
+  |_windows
+    |_installer
+  |_mac
+    |_installer
+```
+
+```yml
+source_path: '**/installer'
+name:
+  generator: relative_path
+  # separator controls what character separates path parts in the application name
+  # default: "-"
+  separator: '.'
+  # depth controls how many of the path parts are used in the application name
+  # default: 0 / all path parts
+  depth: 2
+```

data/docs/configuration/application_source.md

@@ -0,0 +1,64 @@
+# Application source path
+
+**Key**: source_path  
+**Default value**:
+
+- if the `apps` key is not present, then the current working directory where `licensed` was executed
+- if the `apps` key is present, then `nil`
+
+The source path is the directory in which licensed should run to enumerate dependencies.  This is often dependent
+on the project type, for example the bundler source should be run from the directory containing a `Gemfile` or `gems.rb`
+while the go source should be run from the directory containing an entrypoint function.
+
+The source path is required to run `licensed`.  A default value is available only when the configuration file specifies a single application.
+When multiple applications are configured, each application must specify a source path.
+
+Paths can be given as absolute or relative paths, and can use special path identifiers. If a relative path is given, it will be based on the application's root path.
+
+```yml
+# when apps is not set, a source path does not need to be specified.  it will default to the users current directory
+sources:
+  bundler: true
+
+# ------
+# or a path can be given as either an absolute or relative path
+sources:
+  bundler: true
+source_path: path/to/application1
+
+# ------
+# when apps is set, each application must specify a source_path
+sources:
+  bundler: true
+apps:
+  - source_path: relative/path/to/application1
+  - source_path: /absolute/path/to/application2
+  - source_path: ~/path/from/home/to/application3
+```
+
+## Expanding source paths with glob patterns
+
+The `source_path` property can use one or more glob patterns to share configuration properties across multiple application entrypoints.
+
+For example, there is a common pattern in Go projects to include multiple executable entrypoints under folders in `cmd`.  Using a glob pattern allows users to avoid manually configuring and maintaining multiple licensed application `source_path`s.  Using a glob pattern will also ensure that any new entrypoints matching the pattern are automatically picked up by licensed commands as they are added.
+
+```yml
+sources:
+  go: true
+
+# treat all directories under `cmd` as separate apps
+source_path: cmd/*
+```
+
+In order to better filter the results from glob patterns, the `source_path` property also accepts an array of
+inclusion and exclusion glob patterns similar to gitignore files.  Inclusion patterns will add matching directory
+paths to resulting set of source paths, while exclusion patterns will remove matching directory paths.
+
+```yml
+source_path:
+  - "projects/*" # include by default all directories under "projects"
+  - "!projects/*Test" # exclude all projects ending in "Test"
+```
+
+Glob patterns are syntactic sugar for, and provide the same functionality as, manually specifying multiple `source_path` values.
+See the instructions on [specifying multiple apps](../configuration.md#specifying-multiple-apps) below for additional considerations when using multiple apps.

data/docs/configuration/configuration_root.md

@@ -0,0 +1,27 @@
+# Configuration root
+
+**Key**: root  
+**Default value**:
+
+   1. the root of the local git repository, if run inside a git repository
+   1. the directory that `licensed` is run from
+
+An application's root path is used as the base for any relative configuration paths in the application.
+
+From a configuration file, the root value can be specified as one of the following.  Path string values can contain special path characters.
+
+- a relative path from the configuration file location
+- an absolute path
+- `true` to use the configuration file's directory as the root
+
+When creating a `Licensed::AppConfiguration` manually with a `root` property, the property must be an absolute path - no path expansion will occur.
+
+```yml
+root: path/from/configuration
+# or
+root: /absolute/path/to/root
+# or
+root: ~/path/from/home/to/root
+# or
+root: true
+```

data/docs/configuration/configuring_multiple_apps.md

@@ -0,0 +1,58 @@
+# Configuring multiple application definitions
+
+**Key**: apps
+**Required**: false
+
+The configuration file can specify multiple source paths to enumerate metadata, each with their own configuration by using the `apps` key.
+Each source path and any additional configuration make up an "application".  Root configuration settings are inherited into each application,
+allowing applications to share a common configuration and reducing the overall size of the configuration file.
+
+When the apps key is not given, the root configuration is treated as a single application.
+
+```yml
+apps:
+  # application definition for "go-application"
+  - source_path: path/to/go-application
+    sources:
+      go: true
+    allowed:
+      - mit
+
+  # application definition for "ruby-application"
+  - source_path: path/to/ruby-application
+    sources:
+      bundler: true
+    allowed:
+      - bsd-3-clause
+```
+
+## Inheriting configuration
+
+Applications inherit all root configuration settings.  Inherited settings will be overridden by any configuration set directly on the application definition.
+
+In this example, two apps have been declared.  The first app, with `source_path: path/to/application1`, inherits all configuration settings from the root configuration.  The second app, with `source_path: path/to/application2`, overrides the `sources` configuration and inherits all other settings.
+
+```yml
+sources:
+  go: true
+  bundler: false
+
+ignored:
+  bundler:
+    - some-internal-gem
+
+reviewed:
+  bundler:
+    - bcrypt-ruby
+
+cache_path: 'path/to/cache'
+apps:
+  # inherits all settings from the root configuration
+  - source_path: 'path/to/application1'
+
+  # inherits all settings except for "sources" from the root configuration
+  - source_path: 'path/to/application2'
+    sources:
+      bundler: true
+      go: false
+```

data/docs/configuration/dependency_source_enumerators.md

@@ -0,0 +1,28 @@
+# Specifying dependency sources to use in an application
+
+**Key**: sources  
+**Required**: false  
+**Default value**: All sources enabled
+
+The sources configuration property specifies which sources `licensed` will use to enumerate dependencies.
+By default, `licensed` will try to enumerate dependencies from all sources.  As a result,
+the configuration property should be used to explicitly disable sources rather than to enable a particular source.
+
+This configuration value does not guarantee that a source will enumerate dependencies.  Each
+configured source's `enabled?` method must return true for licensed to pull dependency information.
+
+`licensed` determines which sources will try to enumerate dependencies based on the following rules:
+
+1. If no sources are configured, all sources are enabled
+2. If no sources are set to true, any unconfigured sources are enabled
+3. If any sources are set to true, any unconfigured sources are disabled
+
+```yml
+# all other sources are enabled by default since there are no sources set to true
+sources:
+  bower: false
+
+# all other sources are disabled by default because a source was set to true
+sources:
+  bower: true
+```

data/docs/configuration/ignoring_dependencies.md

@@ -0,0 +1,19 @@
+# Ignoring dependencies
+
+**Key**: ignored
+**Default value**: none
+
+This configuration property is used to fully ignore a dependency during all `licensed` commands.  Any dependency on this list will not
+be enumerated, or have its metadata cached or checked for compliance.  This is intended for dependencies that do not require attribution
+or compliance checking - internal or 1st party dependencies, or dependencies that do not ship with the product such as test frameworks.
+
+The ignored dependency list is organized based on the dependency source type - `bundler`, `go`, etc.  Add a dependency's metadata identifier to the appropriate source type sub-property to cause `licensed` to no longer take action on the dependency.  Glob patterns can be used to identify multiple internal dependencies without having to manage a large list.
+
+```yml
+ignored:
+  bundler:
+    - my-internal-gem
+    - my-first-party-gem
+  go:
+    - github.com/me/my-repo/**/*
+```

data/docs/configuration/metadata_cache.md

@@ -0,0 +1,106 @@
+# Dependency metadata cache
+
+**Key**: cache_path  
+**Default value**: .licenses
+
+The cache path is the root directory where `licensed` will write cached metadata files for each dependency.
+By default, files will be written to a folder under the cache path in a structure like:
+
+```text
+<cache path>
+|_<source name>
+  |_<dependency identifier>.dep.yml
+```
+
+Cache paths can be given as an absolute or relative path and can contain special path characters
+
+```yml
+cache_path: relative/path/to/cache
+# or
+cache_path: /absolute/path/to/cache
+# or
+cache_path: ~/path/from/home/to/cache
+```
+
+## Configuring caches for multiple applications
+
+When multiple applications are specified in a configuration file caches can be shared, inherited and explicitly configured.
+Unless otherwise specified, preference is given based on the locality and intention of the configuration options.
+
+1. explicitly configured cache paths are preferred to inherited or shared cache paths
+2. shared cache paths are preferred to inherited cache paths
+3. cache paths that are not otherwise set by an application will be inherited from the root configuration
+
+### Explicit cache usage for multiple applications
+
+Individual applications in a multi-application configuration can explicitly set cache paths.
+
+```yml
+apps:
+  - source_path: path/to/application1
+    cache_path: path/to/application1/.licenses
+  - source_path: path/to/application2
+    cache_path: path/to/application2/.licenses
+```
+
+### Sharing a single cache for multiple applications
+
+Sharing a cache across multiple applications is possible by setting `cache_path: <path>` and `shared_cache: true` at the root level.
+Individual applications can opt out of sharing a cache by explicitly setting a cache path.
+
+```yml
+shared_cache: true
+cache_path: .cache/shared
+apps:
+  # application1 and application2 will share a cache at .cache/shared
+  - source_path: path/to/application1
+  - source_path: path/to/application2
+  # application3 will use a separate cache at .cache/application3
+  - source_path: path/to/application3
+    cache_path: .cache/application3
+```
+
+This is equivalent to explicitly configuring the same cache for many applications
+
+```yaml
+apps:
+  - source_path: "path/to/application1"
+    cache_path: ".cache/shared"
+  - source_path: "path/to/application2"
+    cache_path: ".cache/shared"
+```
+
+Using the `shared_cache` key is primarily useful when specifying `source_path` as a glob pattern.
+
+```yml
+shared_cache: true
+cache_path: .cache/shared
+source_path: path/to/*
+```
+
+### Inheriting cache usage from the root configuration
+
+When not otherwise specified, applications will inherit a cache path from the root configuration.
+If the root configuration does not explicitly set a cache path value, the default cache path value is used.
+
+Inherited cache paths are treated as the root location for each application's metadata cache.  Each application
+will store metadata in a named subdirectory of the root location to avoid file path clashes between
+applications.
+
+```yml
+# optional.  if not specified, the default value `.licenses` will be used
+cache_path: .cache
+apps:
+  - source_path: path/to/application1
+  - source_path: path/to/application2
+```
+
+```text
+.cache
+|_application1
+  |_<source type>
+    |_<dependency identifier>.dep.yml
+|_application2
+  |_<source type>
+    |_<dependency identifier>.dep.yml
+```