steep-activesupport-4 1.9.3

data/doc/narrowing.md

@@ -0,0 +1,195 @@
+# Narrowing Implementation
+
+> This is an internal doc for Steep developers. [Narrowing guide](../guides/narrowing/narrowing.md) is for users.
+
+The challenge is Ruby has special type predicate methods that should be supported by type checkers. `#nil?` is used instead of `unless` statement to test if a value is a `nil` or not. `#is_a?` or `#===` are used to confirm if an object is an instance of a class. Negation and equality are implemented as methods -- `#!` and `#==`.
+
+Steep supports those methods by introducing special types for those methods.
+
+```rbs
+# This is not a valid RBS type definition.
+# Steep implements a transformation from valid RBS syntax to those special types.
+module Kernel
+  def nil?: () -> RECEIVER_IS_NIL
+
+  def is_a?: (Class klass) -> RECEIVER_IS_ARG
+end
+```
+
+When type checking a conditional resulted in `RECEIVER_IS_NIL` type, the type checker overrides the type of the expressions inside the *then* and *else* clauses.
+
+```ruby
+x = [1, ""].sample      # The type of `x` is `String | Integer | nil`
+
+unless x.is_a?(String)  # 1. The condition expression has `RECEIVER_IS_NIL`
+  x.upcase              # 2. Steep overrides the type of `x` to `String` in *then* clause
+else
+                        # 3. Steep overrides the type of `x` to `Integer | nil` in *else* clause
+end
+```
+## Logical types
+
+We extend *type* with *logical types* as follows:
+
+```
+type ::= ...
+       | NOT                           # Negation of type of receiver
+       | RECEIVER_IS_NIL               # Receiver is nil when it evaluates to truthy
+       | RECEIVER_IS_NOT_NIL           # Receiver is not nil when it evaluates to truthy
+       | RECEIVER_IS_ARG               # Receiver is an instance of argument when it evaluates to truthy
+       | ARG_IS_RECEIVER               # Argument is an instance of receiver when it evaluates to truthy
+       | ARG_EQUALS_RECEIVER           # Argument is equal to receiver when it evaluates to truthy
+       | ENV(original_type, truthy_env, falsy_env)    # Two type environments for truthy and falsy
+```
+### ENV type
+
+`ENV` looks a bit different from others because it takes arguments. The type is used for `and` and `or`.
+
+Consider the example with local variables `x` and `y` where both of them have type `String?`.
+
+```ruby
+(x && y) && (x + y)   # Parens added for ease of reading
+```
+
+The type of the whole expression is `String?`. When `x` or `y` is `nil`, it evaluates to `nil`. If both of `x` and `y` is a `String`, `x + y` evaluates to `String` because of the definition of `String#+`.
+
+The type narrowing starts with the top expression.
+
+```ruby
+(...) && (x + y)
+```
+
+It immediately type checks the left hand side, but with *conditional mode*. Conditional mode is a special flag that the type checking result should keep as many of the environments as possible.
+
+```ruby
+x && y
+```
+
+Going down again, it gets a typing `x: String?` and `y: String?`. It runs a type narrowing, to obtain a result both of `x` and `y` are `String` for truthy result, both of `x` and `y` are `String?` for falsy result. The two environments should be propagated to the upper node, because the parent node is also `&&` which is a subject of type narrowing. So, it returns an `ENV` type, `String?` for original type, `{ x: String, y: String }` for truthy result, and `{ x: String?, y: String? }` for falsy result.
+
+Going up to the outer `&&` expression. The left hand side has `ENV` type, and then the right hand side is type checked based on the truthy environment (because of the semantics of `&&` expression.) Both `x` and `y` are `String` and it type checks. The type of the whole expression union of `String` and the falsy part of the original type -- `nil`.
+## Union type partition
+
+We introduce *partition* function for union types. It returns a pair of two types -- truthy parts and falsy parts. We need a slightly different variant for *non-nil* partition to support safe-navigation-operator.
+
+```
+Pt(T) -> T? ⨉ T?    # Truthy partition
+Pn(T) -> T? ⨉ T?    # Non-nil partition
+```
+
+Both return a pair of optional types for non-falsy/non-nil types.
+
+```
+Pt(false?)  -> ∅ ⨉ false?
+Pn(false)   -> false ⨉ ∅
+```
+
+Note that we cannot partition non-regular recursive types, but that types are prohibited in RBS.
+## Type environment
+
+Type environment is a mapping from local variables to their types. It's extended in Steep to support overriding type of *pure* expressions.
+
+```
+E ::= ∅
+    | x : T, E       # Type of a local variable -- `x`
+    | p : T, E       # Type of a pure expression -- `p`
+```
+
+Pure expressions are defined recursively as following:
+
+* *Value expressions* are pure
+* Call expressions of *pure methods* with pure arguments are pure
+
+Note that expression purity depends on the result of type checking of the expression because it requires to detect if a method call is *pure* or not.
+## Logic type interpreter
+Logic type interpreter is an additional layer to support type narrowing. It is a function that takes an expression and it's typing, and returns a pair of type environments -- one for the case the value of the expr is *truthy* and another for the case the value is *falsy*.
+
+```
+I(expr : T) -> E ⨉ E
+```
+
+It takes account of assignments to local variables.
+
+```
+I(x = y : String?) -> { x: String, y: String } ⨉ { x: nil, y: nil }
+```
+
+It also calculates the reachability to truthy and falsy results which can be used to detect unreachable branches.
+## Narrowing syntaxes
+### Simple conditionals -- `if`, `while`, `and`, ...
+Type checking those syntaxes are simple. It type checks the condition expression with conditional mode, passes the expression to logic type interpreter, uses the type environments to type check then and else clauses.
+
+Steep also reports unreachable branch issues based on the reachability calculated by logic type interpreter.
+### `case-when`
+The easier case is if `case-when` doesn't have a node just after `case` token.
+
+```ruby
+case
+when x == 1
+  ...
+end
+```
+
+This is the same with simple conditionals.
+
+The difficult case is if `case-when` syntax has a node.
+
+```ruby
+case foo()
+when Integer
+  ...
+when String
+  ...
+end
+```
+
+Ruby uses the `===` operator to test if the value of `foo()` matches the condition, while we don't want to type check `foo()` calls every time. It may be a pure expression, and we can give better type narrowing using the *falsy* results of predecessor `when` clauses.
+
+So, we transform the condition expression given to the logic type interpreter to value form.
+
+```ruby
+__case_when:x01jg9__ = foo()
+if Integer === __case_when:x01jg9__
+  ...
+elsif String === __case_when:x01jg9__
+  ...
+end
+```
+
+It generates a fresh local variable and assigns the expression to it. Uses the variable inside the patterns.
+
+We also need to propagate the type of local variables which are included in the expression.
+
+```ruby
+case x = foo()
+when Integer
+  x.abs      # x should be Integer
+when String
+  x.upcase   # x should be String
+end
+```
+
+The local variable insertion is done at the outermost non-assignment position to support the local variable propagation.
+
+```ruby
+__case_when:x01jg9__ = foo()
+if Integer === (x = __case_when:x01jg9__)
+  ...
+elsif String === (x = __case_when:x01jg9__)
+  ...
+end
+```
+
+The last trick to type check case-when is *pure call* narrowing in the bodies.
+
+```ruby
+__case_when:x01jg9__ = foo()
+if Integer === (x = __case_when:x01jg9__)
+  foo.abs
+elsif String === (x = __case_when:x01jg9__)
+  foo.upcase
+end
+```
+
+To support this, we propagate the type of the fresh local variable to the type of right hand side expression, if it's a pure call.
+

data/doc/shape.md

@@ -0,0 +1,194 @@
+# Shapes
+
+A *shape* is a data structure, which contains the set of available methods and their types, which is associated with a type. Steep uses shapes to type check method calls -- it calculates the shape of the type of the receiver, checks if the called method is defined on the shape and the arguments are compatible with the method, and calculates the return type.
+
+Assume an interface `_Foo` is defined as follows:
+
+```rbs
+interface _Foo
+  def foo: () -> String
+
+  def bar: () -> self
+end
+```
+
+The shape of `_Foo` will be the following:
+
+```
+Shape (_Foo) {
+  foo: () -> String,
+  bar: () -> _Foo
+}
+```
+
+Note that the `self` type in the example is resolved to `_Foo` during shape calculation.
+
+The shape calculation of an object is straightforward. Calculate a `RBS::Definition` of a class singleton/instance, or an interface, and translate the data structure to a `Shape` object. But there are a few things to consider.
+
+## Tuple, record, and proc types
+
+The shape of tuple, record, or proc types are based on their base types -- Array, Hash, or Proc classes --, but with specialized method types.
+
+```
+Shape ([Integer, String]) {
+  []: (0) -> Integer
+    | (1) -> String
+    | (Integer) -> (Integer | String)
+  ...
+}
+```
+
+The specialization is implemented as a part of shape calculation.
+
+## Special methods
+
+Steep recognizes some special methods for type narrowing, including `#is_a?`, `#===`, `#nil?`, ... These methods are defined with normal RBS syntax, but the method types in shapes are transformed to types using logic types.
+
+The shape calculation inserts the specialized methods with these special methods.
+
+## `self` types
+
+There are two cases of `self` types to consider during shape calculation.
+
+1. `self` types included in the shape of a type
+2. `self` types included in given types
+
+### 1. `self` types included in the shape of a type
+
+`self` types may be included in a class or interface definition.
+
+```rbs
+interface _Foo
+  def itself: () -> self
+end
+```
+
+The `self` types included in the shape of `_Foo` type should be resolved to `_Foo` type.
+
+```
+Shape (_Foo) {
+  itself: () -> _Foo
+}
+```
+
+### 2. `self` types included in given types
+
+Unlike `self` types included in definitions, `self` types in given types should be preserved.
+
+```rbs
+interface _Foo[A]
+  def get: () -> A
+end
+```
+
+The shape of `_Foo[self]` has `self` type as its type argument, and we want the `self` type preserved after the shape calculation.
+
+```
+Shape (_Foo[self]) {
+  get: () -> self
+}
+```
+
+We often use `self` types as the return type of a method.
+
+```rbs
+class Foo
+  def foo: () -> self
+end
+```
+
+So, the implementation of `foo` might use `self` node to return `self` type.
+
+```rb
+class Foo
+  def foo
+    # @type var foo: _Foo[self]
+    foo = ...
+    foo.get
+  end
+end
+```
+
+We want the type of `foo.get` to be `self`, not `Foo`, to avoid a type error being detected.
+
+## Shape of `self` types
+
+We also want `self` type if `self` is the type of the shape.
+
+```rb
+class Foo
+  def foo
+    self.itself
+  end
+end
+```
+
+This is a straightforward case, because the type of `self` is `self` itself. Calculate the shape of it, but keep the `self` types in the shape.
+
+```
+Shape (self) {
+  itself: () -> self
+}
+```
+
+If `self` is a union type, or something built with a type constructor, the shape calculation gets complicated.
+
+```rbs
+class Foo
+  def foo: () -> Integer
+end
+
+class Bar
+  def foo: () -> self
+end
+```
+
+What is the expected shape of `self` where the type of `self` is `Foo | Bar`?
+
+The shape of a union type is straightforward. It calculates the shape of each type, and then it calculates a union of the shape.
+
+We do the same for the case with `self` types, but it results in slightly incorrect shapes.
+
+```
+Shape (Foo) {
+  foo: () -> Integer
+}
+
+Shape (Bar) {
+  foo: () -> self   # self is preserved, because the shape of `self` is being calculated
+}
+
+Shape (Foo | Bar) {
+  foo: () -> (Integer | self)
+}
+```
+
+So, the resulting type of `self.foo` where the type of `self` is `Foo | Bar`, would be `Integer | Foo | Bar`. But, actually, it won't be `Foo` because the `self` comes from `Bar`.
+
+This is an incorrect result, but Steep is doing this right now.
+
+## `class` and `instance` types
+
+The shape calculation provides limited support for `class` and `instance` types.
+
+1. `class`/`instance` types from the definition are resolved
+2. `class`/`instance` types in generics type arguments of interfaces/instances are preserved
+3. Shape of `class`/`instance` types are resolved to configuration's `class_type` and `instance_type`, and the translated types are used to calculate the shape
+
+It's different from `self` types except case #2. The relationship between `self`/`class`/`instance` is not trivial in Ruby. All of them might be resolved to any type, which means calculating one from another of them is simply impossible.
+
+## Public methods, private methods
+
+`Shape` objects have a flag of if the shape is for *public* method calls or *private* method calls. Private method call is a form of `foo()` or `self.foo()` -- when the receiver is omitted or `self`. Public method calls are anything else.
+
+The shape calculation starts with *private methods*, and the `Shape#public_shape` method returns another shape that only has *public* methods.
+
+> Note that the private shape calculation is required even on public method calls. This means a possible chance of future optimizations.
+
+## Lazy method type calculation
+
+We rarely need all of the methods available for an object. If we want to type check a method call, we only need the method type of that method. All other methods can be just ignored.
+
+*Lazy method type calculation* is introduced for that case. Instead of calculating the types of all of the methods, it registers a block that computes the method type.
+
+It is implemented in `Steep::Interface::Shape::Entry` and used to make the shape calculation of a union type faster.

data/exe/steep

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+require 'pathname'
+
+$LOAD_PATH << Pathname(__dir__) + "../lib"
+
+require 'steep'
+require 'steep/cli'
+
+begin
+  exit Steep::CLI.new(argv: ARGV.dup, stdout: STDOUT, stderr: STDERR, stdin: STDIN).run
+rescue => exn
+  STDERR.puts exn.inspect
+  exn.backtrace.each do |t|
+    STDERR.puts "  #{t}"
+  end
+  exit 2
+end

data/guides/README.md

@@ -0,0 +1,5 @@
+# Steep Guides
+
+* [Getting Started with Steep in 5 minutes](src/getting-started/getting-started.md)
+* [Using RBS from gem_rbs_collection](src/gem-rbs-collection/gem-rbs-collection.md)
+* [`nil` and Optional types](src/nil-optional/nil-optional.md)

data/guides/src/gem-rbs-collection/gem-rbs-collection.md

@@ -0,0 +1,126 @@
+# Using RBS from gem_rbs_collection
+
+gem_rbs_collection is a repository of type definitions of gems that are managed by the community. You may find the type definition of a gem that ships without RBS type definitions.
+
+To use RBS files from the repository, you can use rbs-collection subcommand. This guide explains how the command works.
+
+## Quick start
+
+Run rbs-collection-init to start setup.
+
+```
+$ rbs collection init
+```
+
+You have to edit your `Gemfile`. Specify `require: false` for gems for which you do not want type definitions.
+
+```ruby
+gem 'rbs', require: false
+gem 'steep', require: false
+gem 'rbs_rails', require: false
+gem 'rbs_protobuf', require: false
+```
+
+Once you save the file, run the install command.
+
+```
+$ rbs collection install
+```
+
+That generates `rbs_collection.lock.yaml` and downloads the RBS files from the git repository.
+
+Note that rbs-collection automatically downloads RBS files of gems included in your Bundler environment. You don't have to write all of the gems in your `Gemfile` in `rbs_collection.yaml`.
+
+Finally, we recommend adding the `rbs_collection.yaml` and `rbs_collection.lock.yaml` to your repository, and ignoring `.gem_rbs_collection` directory.
+
+```
+$ git add rbs_collection.yaml
+$ git add rbs_collection.lock.yaml
+$ echo /.gem_rbs_collection >> .gitignore
+$ git commit -m "Set up rbs-collection"
+```
+
+## Updating RBS files
+
+You may want to run rbs-collection-update to update the contents of `rbs_collection.lock.yaml`, when you add a new gem or some gems are updated.
+
+You also need to run rbs-collection-update when the RBS files in the source repository are updated. The type definitions are updated with bug-fixes or improvements, and you need to update to apply the changes to your app.
+
+## Using rbs-collection with Steep
+
+Steep automatically reads `rb_collection.yaml`. You can use Steep immediately without any modifications.
+
+```
+$ bin/steep project    # Show dependencies detected by Steep
+$ bin/steep check      # Run type check
+```
+
+## Migration
+
+If you have used older versions of Steep or RBS, you may have configured libraries manually.
+
+* You have library calls in `Steepfile`
+* You have git submodules in your git repository to download gem_rbs_collection
+
+These are steps to migrate to rbs-collection.
+
+1. Remove unnecessary library configuration
+2. Set up rbs-collection
+3. Validate the configuration
+4. Delete submodule
+
+### 1. Remove unnecessary library configurations
+
+You may have `#library` calls in your Steepfile, or something equivalent in your scripts. We can group the configured libraries into three groups.
+
+1. Gems that is managed by Bundler
+2. Standard libraries (non-gem libs, default gems, and bundled gems)
+    * 2-1) That is a transitive dependency from libraries in group 1
+    * 2-2) That is included in Gemfile.lock
+    * 2-3) Implicitly installed gems – not included in Gemfile
+
+You can delete library configs of 1, 2-1, and 2-2. So, you have to keep the configurations of libraries in 2-3.
+
+Practically, you can remove all library configs and `#library` calls in `Steepfile`, go step 2, run steep check to test, and restore the configs of libraries if error is detected.
+
+### 2. Set up rbs-collection
+
+See the quick start section above!
+
+### 3. Validate the configuration
+
+Run steep check to validate the configuration.
+
+```
+$ bin/steep check
+```
+
+You may see the `UnknownTypeName` error if some libraries are missing. Or some errors that implies duplication or inconsistency of methods/class/module definitions, that may be caused by loading RBS files of a library twice.
+
+Running steep project may help you too. It shows the source files and library directories recognized by Steep.
+
+```
+$ bin/steep project
+```
+
+Note that the type errors detected may change after migrating to rbs-collection, because it typically includes updating to newer versions of RBS files.
+
+### 4. Delete submodule
+
+After you confirmed everything is working correctly, you can delete the submodule. deinit the submodule, remove the directory using git-rm, and delete $GIT_DIR/modules/<name>/.
+
+## What is the rbs_collection.yaml file?
+
+The file mainly defines three properties – sources, gems and path. Sources is essential when you want to create a new RBS file repository, usually for RBS files of the private gems.
+
+Another trick is ignoring RBS files of type checker toolchain. RBS and Steep ships with their own RBS files. However, these RBS files may be unnecessary for you, unless you are not a type checking toolchain developer. It requires some additional gems and RBS files. So adding ignore: true is recommended for the gems.
+
+It seems like we need to add a feature to skip loading RBS files automatically and use the feature for RBS, Steep, and RBS Rails. It looks weird that the gems section is only used to ignore gems.
+
+## Versions of RBS files in gem_rbs_collection
+
+Gem versions in rbs-collection are relatively loosely managed. If a gem is found but the version is different, rbs-collection simply uses the incorrect version.
+
+It will load RBS files of activerecord/6.1 even if your Gemfile specifies activerecord-7.0.4. This is by design with an assumption that having RBS files with some incompatibility is better than having nothing. We see most APIs are compatible even after major version upgrade. Dropping everything for minor API incompatibilities would not make much sense.
+
+That behavior will change in future versions when we see that the assumption is not reasonable and we have better coverage.

data/guides/src/getting-started/getting-started.md

@@ -0,0 +1,163 @@
+# Getting Started with Steep in 5 minutes
+
+## Installing Steep
+
+Add the lines to Gemfile:
+
+```rb
+group :development do
+  gem "steep", require: false
+end
+```
+
+and install the gems.
+
+```
+$ bundle install
+```
+
+Alternatively, you can install it with the gem command.
+
+```
+$ gem install steep
+```
+
+Execute the following command to confirm if the command is successfully installed.
+
+```
+$ steep version
+$ bundle exec steep version         # If you install with bundler
+```
+
+We omit the `bundle exec` prefix from the following commands. Run commands with the prefix if you installed Steep with Bundler.
+
+## Type checking your first Ruby script
+
+Run the `steep init` command to generate the configuration file, `Steepfile`.
+
+```
+$ steep init
+```
+
+Open `Steepfile` in your text editor, and replace the content with the following lines:
+
+```rb
+target :lib do
+  signature "sig"
+  check "lib"
+end
+```
+
+Type the following Ruby code in your editor, and save it as `lib/hello.rb`.
+
+```rb
+currencies = { US: "$", JP: "¥", UK: "£" }
+country = %w(US JP UK).sample()
+
+puts "Hello! The price is #{currencies[country]}100. 💸"
+```
+
+And type check it with Steep.
+
+```
+$ steep check
+```
+
+The output will report a type error.
+
+```
+# Type checking files:
+
+........................................................F
+
+lib/hello.rb:4:39: [error] Cannot pass a value of type `(::String | nil)` as an argument of type `::Symbol`
+│   (::String | nil) <: ::Symbol
+│     ::String <: ::Symbol
+│       ::Object <: ::Symbol
+│         ::BasicObject <: ::Symbol
+│
+│ Diagnostic ID: Ruby::ArgumentTypeMismatch
+│
+└ puts "Hello! The price is #{currencies[country]}100. 💸"
+                                         ~~~~~~~
+
+Detected 1 problem from 1 file
+```
+
+The error says that the type of the country variable causes a type error. It is expected to be a symbol, but string or `nil` will be given.
+
+Let's see how we can fix the error.
+
+## Fixing the type error
+
+The first step is converting the string value to a symbol. We can add a `#to_sym` call.
+
+```rb
+currencies = { US: "$", JP: "¥", UK: "£" }
+country = %w(US JP UK).sample()
+
+puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+```
+
+The `#to_sym` call will convert a string into a symbol. Does it solve the problem??
+
+```
+$ steep check
+# Type checking files:
+
+........................................................F
+
+lib/hello.rb:4:47: [error] Type `(::String | nil)` does not have method `to_sym`
+│ Diagnostic ID: Ruby::NoMethod
+│
+└ puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+                                                 ~~~~~~
+
+Detected 1 problem from 1 file
+```
+
+It detects another problem. The first error was `Ruby::ArgumentTypeMismatch`, but the new error is `Ruby::NoMethod`. The value of `country` may be `nil`, and it doesn't have the `#to_sym` method.
+
+This would be annoying, but one of the most common sources of a type error. The value of an expression may be `nil` unexpectedly, and using the value of the expression may cause an error.
+
+In this case, the `sample()` call introduces the `nil`. `Array#sample()` returns `nil` when the array is empty. We know the receiver of the `sample` call cannot be `nil`, because it is an array literal. But the type checker doesn't know of it. The source code detects the `Array#sample()` method is called, but it ignores the fact that the receiver cannot be empty.
+
+Instead, we can simply tell the type checker that the value of the country cannot be `nil`.
+
+# Satisfying the type checker by adding a guard
+
+The underlying type system supports flow-sensitive typing similar to TypeScript and Rust. It detects conditional expressions testing the value of a variable and propagates the knowledge that the value cannot be `nil`.
+
+We can fix the type error with an `or` construct.
+
+```rb
+currencies = { US: "$", JP: "¥", UK: "£" }
+country = %w(US JP UK).sample() or raise
+
+puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+```
+
+The change lets the type checking succeed.
+
+```
+$ steep check
+# Type checking files:
+
+.........................................................
+
+No type error detected. 🫖
+```
+
+The `raise` method is called when `sample()` returns `nil`. Steep can reason the possible control flow based on the semantics of or in Ruby:
+
+* The value of `country` is the return value of `sample()` method call
+* The value of `country` may be `nil`
+  * If the value of `country` is `nil`, the right hand side of the or is evaluated
+  * It calls the `raise` method, which results in an exception and jumps to somewhere
+  * So, when the execution continues to the puts line, the value of `country` cannot be `nil`
+
+There are two possibilities of the type of the result of the `sample()` call, `nil` or a string. We humans can reason that we can safely ignore the case of `nil`. But, the type checker cannot. We have to add `or raise` to tell the type checker it can stop considering the case of `nil` safely.
+
+## Next steps
+
+This is a really quick introduction to using Steep. You may have noticed that I haven't explained anything about defining new classes or modules. See the RBS guide for more examples!