packwerk 1.0.0

data/LICENSE.md

@@ -0,0 +1,7 @@
+Copyright 2020-present, Shopify Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,73 @@
+# Packwerk [![Build Status](https://github.com/Shopify/packwerk/workflows/CI/badge.svg)](https://github.com/Shopify/packwerk/actions?query=workflow%3ACI)
+
+Packwerk is a Ruby gem used to enforce boundaries and modularize Rails applications.
+
+Packwerk can be used to:
+* Combine group of files into packages
+* Define package-level constant visibility (i.e. have publicly accessible constants)
+* Enforce privacy (inbound) and dependency (outbound) boundaries between packages
+* Help existing codebases to become more modular without obstructing development
+
+## Prerequisites
+
+Packwerk needs [Zeitwerk](https://github.com/fxn/zeitwerk) enabled, which comes with Rails 6.
+
+Packwerk supports MRI versions 2.6 and above.
+
+## Demo
+
+Watch a [1-minute video demo](https://drive.google.com/file/d/1D-t1nYduwgpHAP4DHY-EwVwVNFHlwRWj/view?usp=sharing) on how Packwerk works.
+
+[![](./static/packwerk-check-demo.png)](https://drive.google.com/file/d/1D-t1nYduwgpHAP4DHY-EwVwVNFHlwRWj/view?usp=sharing)
+
+## Installation
+
+1. Add this line to your application's Gemfile:
+
+```ruby
+gem 'packwerk'
+```
+
+2. Install the gem
+
+Execute:
+
+    $ bundle install
+
+Or install it yourself as:
+
+    $ gem install packwerk
+
+3. Run `bundle exec packwerk init` to generate the configuration files
+
+## Usage
+
+Read [USAGE.md](USAGE.md) for usage once Packwerk is installed on your project.
+
+## Pronunciation
+
+"Packwerk" is pronounced [[ˈpakvɛʁk]](https://cdn.shopify.com/s/files/1/0258/7469/files/packwerk.mp3).
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `bundle exec rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+## Limitations
+
+With Ruby being a very dynamic language, static analysis tools such as Packwerk are bound to have limitations.
+To reduce the impact of those limitations, Packwerk is designed to avoid false positives (reporting references as violations that are actually fine) at any cost, and we pay the cost by accepting a small number of false negatives (failing to report actual violations).
+
+- Packwerk can only resolve references to constants that are defined in code loaded by the application's Zeitwerk autoloader.
+  This is because we rely on [Zeitwerk's conventions](https://github.com/fxn/zeitwerk#file-structure), and code that is loaded differently (like through an explicit `require`) often doesn't follow these conventions.
+- Method calls and objects passed around the application are completely ignored. Packwerk only cares about static constant references. That said, if you want Packwerk to analyze parameters of a method, you can use [Sorbet](https://sorbet.org/) to define a type signature. Sorbet signatures are pure Ruby code and use constants to express types, and Packwerk understands that.
+- Support for custom Zeitwerk configuration is limited. If [custom ActiveSupport inflections](https://guides.rubyonrails.org/autoloading_and_reloading_constants.html#customizing-inflections) are used, Packwerk will understand that and everything is fine. However, if Zeitwerk is directly configured with [custom Zeitwerk inflections](https://github.com/fxn/zeitwerk#inflection) or to [collapse directories](https://github.com/fxn/zeitwerk#collapsing-directories), _Packwerk will get confused and produce false positives_.
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/Shopify/packwerk.
+
+Read and follow the guidelines in [CONTRIBUTING.md](https://github.com/Shopify/packwerk/blob/main/CONTRIBUTING.md).
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+  t.warning = true
+end
+
+task(default: :test)

data/TROUBLESHOOT.md

@@ -0,0 +1,67 @@
+# Troubleshoot
+
+* [Troubleshooting violations](#Troubleshooting-violations)
+  * [Feedback loop](#Feedback-loop)
+  * [Package Privacy violation](#Package-Privacy-violation)
+    * [Interpreting Privacy violation](#Interpreting-Privacy-violation)
+  * [Package Dependency violation](#Package-Dependency-violation)
+    * [Interpreting Dependency violation](#Interpreting-Dependency-violation)
+
+## Troubleshooting violations
+
+### Feedback loop
+Packwerk can give feedback via continuous integration (CI) if you have it set up, but that is a long feedback cycle. If you want faster feedback, use Packwerk on the command line locally.
+
+You can specify folders or packages in Packwerk commands for a shorter run time:
+
+     bundle exec packwerk check components/your_package
+
+     bundle exec packwerk update components/your_package
+
+_Note: You cannot specify folders or packages for `packwerk validate` because the command runs for the entire application._
+
+![](static/packwerk_check_violation.gif)
+
+### Package Privacy violation
+A constant that is private to its package has been referenced from outside of the package. Constants are declared private in their package’s `package.yml`.
+
+See: [USAGE.md - Enforcing privacy boundary](USAGE.md#Enforcing-privacy-boundary)
+
+#### Interpreting Privacy violation
+
+> /Users/JaneDoe/src/github.com/sample-project/user/app/controllers/labels_controller.rb:170:30
+> Privacy violation: '::Billing::CarrierInvoiceTransaction' is private to 'billing' but referenced from 'user'.
+> Is there a public entrypoint in 'billing/app/public/' that you can use instead?
+>
+> Inference details: 'Billing::CarrierInvoiceTransaction' refers to ::Billing::CarrierInvoiceTransaction which seems to be defined in billing/app/models/billing/carrier_invoice_transaction.rb.
+
+There has been a privacy violation of the package `billing` in the package `user`, through the use of the constant `Billing::CarrierInvoiceTransaction` in the file `user/app/controllers/labels_controller.rb`.
+
+##### Suggestions
+You may be accessing the implementation of a piece of functionality that is supposed to be accessed through a public interface on the package. Try to use the public interface instead. A package’s public interface should be defined in its `app/public` folder and documented.
+
+The functionality you’re looking for may not be intended to be reused across packages at all. If there is no public interface for it but you have a good reason to use it from outside of its package, find the people responsible for the package and discuss a solution with them.
+
+### Package Dependency violation
+A constant defined in a package A is referenced from a package B that doesn’t define a dependency on A. Packages define their dependencies in their `package.yml`.
+
+See: [USAGE.md - Enforcing dependency boundary](USAGE.md#Enforcing-dependency-boundary)
+
+#### Interpreting Dependency violation
+
+> /Users/JaneDoe/src/github.com/sample-project/billing/app/jobs/document_processing_job.rb:48:6
+> Dependency violation: ::Edi::Source belongs to 'edi', but 'billing' does not specify a dependency on 'edi'.
+> Are we missing an abstraction?
+> Is the code making the reference, and the referenced constant, in the right packages?
+>
+> Inference details: 'Edi::Source' refers to ::Edi::Source which seems to be defined in edi/app/models/edi/source.rb.
+
+There has been a dependency violation in the package `billing` to the package `edi`, through the use of the constant `Edi::Source` in the file `billing/app/jobs/document_processing_job.rb`.
+
+##### Suggestions
+If a package declares dependencies, assume that some thought went into that. It is unlikely that the correct fix is to add another dependency.
+Check if the code containing the reference and the code defining the constants are in the right packages. If not, fix that by moving code around.
+
+If a dependency A -> B is not desired, but a dependency B -> A is OK, consider using [dependency inversion](https://www.sandimetz.com/blog/2009/03/21/solid-design-principles).
+
+If you’re getting stuck, find people with context about the two packages involved and ask for their opinion.

data/USAGE.md

@@ -0,0 +1,250 @@
+# Packwerk usage
+
+## Table of Contents
+* [What problem does Packwerk solve?](#What-problem-does-Packwerk-solve?)
+* [What is a package?](#What-is-a-package?)
+  * [Package principles](#Package-principles)
+* [Getting started](#Getting-started)
+* [Setting up the configuration file](#Setting-up-the-configuration-file)
+  * [Inflections](#Inflections)
+* [Validating the package system](#Validating-the-package-system)
+* [Defining packages](#Defining-packages)
+  * [Package metadata](#Package-metadata)
+* [Types of boundary checks](#Types-of-boundary-checks)
+  * [Enforcing privacy boundary](#Enforcing-privacy-boundary)
+    * [Using public folders](#Using-public-folders)
+  * [Enforcing dependency boundary](#Enforcing-dependency-boundary)
+* [Checking for violations](#Checking-for-violations)
+* [Recording existing violations](#Recording-existing-violations)
+  * [Understanding the list of deprecated references](#Understanding-the-list-of-deprecated-references)
+
+## What problem does Packwerk solve?
+Large applications need clear boundaries to avoid turning into a [ball of mud](https://en.wikipedia.org/wiki/Big_ball_of_mud). However, Ruby does not provide a good solution to enforcing boundaries between code.
+
+Packwerk is a gem that can be used to enforce boundaries between groups of code we call packages.
+
+## What is a package?
+A package is a folder containing autoloaded code. To decide whether code belongs together in a package, these are some design best practices:
+
+- We should package things together that have high functional [cohesion](https://en.wikipedia.org/wiki/Cohesion_(computer_science)).
+- Packages should be relatively loosely coupled to each other.
+
+![cohesion](docs/cohesion.png)
+
+### Package principles
+
+Package principles help to guide the organization of classes in a large system. These principles can also be applied to packages in large and complex codebases.
+
+The [package principles](https://en.wikipedia.org/wiki/Package_principles) page on Wikipedia does a good job explaining what well designed packages look like.
+
+## Getting started
+
+After including Packwerk in the Gemfile, you can generate the necessary files to get Packwerk running by executing:
+
+    bundle exec packwerk init
+
+Here is a list of files generated:
+
+| File                        | Location     | Description |
+|-----------------------------|--------------|------------|
+| Packwerk configuration      | packwerk.yml | See [Setting up configuration file](#Setting-up-configuration-file) |
+| Root package                | package.yml  | A package for the root folder |
+| Bin script                  | bin/packwerk | For Rails applications to run Packwerk validation on CI, see [Validating the package system](#Validating-the-package-system) |
+| Validation test             | test/packwerk_validator_test.rb | For Ruby projects to run Packwerk validation using tests, see [Validating the package system](#Validating-the-package-system) |
+| Custom inflections          | configs/inflections.yml | A custom inflections file is only required if you have custom inflections in `inflections.rb`, see [Inflections](#Inflections) |
+
+After that, you may begin creating packages for your application. See [Defining packages](#Defining-packages)
+
+## Setting up the configuration file
+
+Packwerk reads from the `packwerk.yml` configuration file in the root directory. Packwerk will run with the default configuration if any of these settings are not specified.
+
+| Key                  | Default value                      | Description  |
+|----------------------|------------------------------------|--------------|
+| include              | **/*.{rb,rake,erb}                 | list of patterns for folder paths to include |
+| exclude              | {bin,node_modules,script,tmp}/**/* | list of patterns for folder paths to exclude |
+| package_paths        | **/                                | patterns to find package configuration files, see: Defining packages |
+| load_paths           | All application autoload paths     | list of load paths |
+| custom_associations  | N/A                                | list of custom associations, if any |
+
+
+### Inflections
+
+Packwerk requires custom inflections to be defined in `inflections.yml` instead of the traditional `inflections.rb`. This is because Packwerk accounts for custom inflections, such as acronyms, when resolving constants. Additionally, Packwerk interprets Active Record associations as references to constants. For example, `has_many :birds` is reference to the `Birds` constant.
+
+In order to make your custom inflections compatible with Active Support and Packwerk, you must create an `inflections.yml` file and point `ActiveSupport::Inflector` to that file.
+
+In `inflections.rb`, add:
+
+```rb
+ActiveSupport::Inflector.inflections do |inflect|
+  # please add all custom inflections in the file below.
+  Packwerk::Inflections::Custom.new(
+    Rails.root.join("inflections.yml")
+  ).apply_to(inflect)
+end
+```
+
+Next, move your existing custom inflections into `inflections.yml`:
+
+```yaml
+acronym:
+  - 'GraphQL'
+  - 'MRuby'
+  - 'TOS'
+irregular:
+  - ['analysis', 'analyses']
+  - ['reserve', 'reserves']
+uncountable:
+  - 'payment_details'
+```
+
+Any new inflectors should be added to `inflections.yml`.
+
+## Validating the package system
+
+There are some criteria that an application must meet in order to have a valid package system. These criteria include having a valid autoload path cache, package definition files, and application folder structure. The dependency graph within the package system also has to be acyclic.
+
+The package system can be validated through a series of built in validation checks. Currently, the validation checks require the application to be booted either through `spring` or as part of its test suite.
+
+We recommend setting up the package system validation for your Rails application in a CI step (or through a test suite for Ruby projects) separate from `packwerk check`.
+
+If running `packwerk init` generates a `bin/packwerk` script, proceed to run:
+
+    bin/packwerk validate
+
+![](static/packwerk_validate.gif)
+
+If running `packwerk init` on your Ruby project generates `test/packwerk_validator_test.rb`, you can use this test as validation.
+
+## Defining packages
+
+You can create a `package.yml` in any folder to make it a package. The package name is the path to the folder from the project root.
+
+_Note: It is helpful to define a namespace that corresponds to the package name and contains at least all the public constants of the package. This makes it more obvious which package a constant is defined in._
+
+### Package metadata
+Package metadata can be included in the `package.yml`. Metadata won't be validated, and can thus be anything. We recommend including information on ownership and stewardship of the package.
+
+Example:
+```yaml
+    # components/sales/package.yml
+    metadata:
+      stewards:
+      - "@Shopify/sales"
+      slack_channels:
+      - "#sales"
+```
+
+## Types of boundary checks
+
+Packwerk can perform two types of boundary checks: privacy and dependency.
+
+#### Enforcing privacy boundary
+A package's privacy boundary is violated when there is a reference to the package's private constants from a source outside the package.
+
+There are two ways you can enforce privacy for your package:
+
+1. Enforce privacy for all external sources
+
+```yaml
+# components/merchandising/package.yml
+enforce_privacy: true  # will make everything private that is not in
+                        # the components/merchandising/app/public folder
+```
+
+Setting `enforce_privacy` to true will make all references to private constants in your package a violation.
+
+2. Enforce privacy for specific constants
+
+```yaml
+# components/merchandising/package.yml
+enforce_privacy:
+  - "::Merchandising::Product"
+  - "::SomeNamespace"  # enforces privacy for the namespace and
+                       # everything nested in it
+```
+
+It will be a privacy violation when a file outside of the `components/merchandising` package tries to reference `Merchandising::Product`.
+
+##### Using public folders
+You may enforce privacy either way mentioned above and still expose a public API for your package by placing constants in the `app/public` folder. The constants in the public folder will be made available for use by the rest of the application.
+
+#### Enforcing dependency boundary
+A package's dependency boundary is violated whenever it references a constant in some package that has not been declared as a dependency.
+
+Specify `enforce_dependencies: true` to start enforcing the dependencies of a package. The intentional dependencies of the package are specified as a list under a `dependencies:` key.
+
+Example:
+
+```yaml
+# components/shop_identity/package.yml
+enforce_dependencies: true
+dependencies:
+  - components/platform
+```
+
+It will be a dependency violation when `components/shop_identity` tries to reference a constant that is not within `components/platform` or itself.
+
+## Checking for violations
+
+After enforcing the boundary checks for a package, you may execute:
+
+    bundle exec packwerk check
+
+Packwerk will check the entire codebase for any violations.
+
+You can also specify folders or packages for a shorter run time:
+
+    bundle exec packwerk check components/your_package
+
+![](static/packwerk_check.gif)
+
+In order to keep the package system valid at each version of the application, we recommend running `packwerk check` in your CI pipeline.
+
+See: [TROUBLESHOOT.md - Sample violations](TROUBLESHOOT.md#Sample-violations)
+
+## Recording existing violations
+
+For existing codebases, packages are likely to have existing boundary violations.
+
+If so, you will want to stop the bleeding and prevent more violations from occuring. The existing violations in the codebase can be recorded in a [deprecated references list](#Understanding_the_list_of_deprecated_references) by executing:
+
+    bundle exec packwerk update
+
+Similar to `packwerk check`, you may also run `packwerk update` on folders or packages:
+
+    bundle exec packwerk update components/your_package
+
+![](static/packwerk_update.gif)
+
+_Note: Changing dependencies or enabling dependencies will not require a full update of the codebase, only the package that changed. On the other hand, changing or enabling privacy will require a full update of the codebase._
+
+`packwerk update` should only be run to record existing violations and to remove deprecated references that have been worked off. Running `packwerk update` to resolve a violation should be the very last resort.
+
+See: [TROUBLESHOOT.md - Troubleshooting violations](TROUBLESHOOT.md#Troubleshooting_violations)
+
+
+### Understanding the list of deprecated references
+
+The deprecated references list is called `deprecated_references.yml` and can be found in the package folder. The list outlines the constant violations of the package, where the violation is located, and the file defining the violation.
+
+The deprecated references list should not be added to, but worked off over time.
+
+```yaml
+components/merchant:
+  "::Checkouts::Core::CheckoutId":
+    violations:
+    - dependency
+    files:
+    - components/merchant/app/public/merchant/generate_order.rb
+```
+
+Above is an example of a constant violation entry in `deprecated_references.yml`.
+
+* `components/merchant` - package where the constant violation is found
+* `::Checkouts::Core::CheckoutId` - violated constant in question
+* `dependency` - type of violation, either dependency or privacy
+* `components/merchant/app/public/merchant/generate_order.rb` - path to the file containing the violated constant
+
+Violations exist within the package that makes a violating reference. This means privacy violations of your package can be found listed in `deprecated_references.yml` files in the packages with the reference to a private constant.

data/bin/console

@@ -0,0 +1,15 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "packwerk"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/dev.yml

@@ -0,0 +1,32 @@
+name: packwerk
+
+type: ruby
+
+up:
+  - ruby: 2.6.6
+  - bundler
+
+commands:
+  test:
+    run: |
+      if [[ "$*" =~ ":"[0-9]+ ]];
+      then
+        # run test by its line number
+        bundle exec m "$@"
+      elif [[ "$#" -eq 1 && -f "$1" ]];
+      then
+        # run all tests in given file(s)
+        bundle exec rake test TEST="$@"
+      else
+        # run all tests
+        bundle exec rake test
+      fi
+  style: "bundle exec rubocop -D --auto-correct"
+  typecheck:
+    desc: "run Sorbet typechecking"
+    run: "bundle exec srb tc"
+    aliases: ['tc']
+    subcommands:
+      update:
+        desc: "update RBIs for gems"
+        run: "bundle exec tapioca sync -c 'dev typecheck update'"