steep 1.6.0 → 1.7.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c834be3917dcff04077098f89b0300f7819b0cda30499bd11d34b172627e975f
-  data.tar.gz: d48ce39f9e337ad408047b56a0bb68a133ba6e044b66381c2bdd6974fcf94a69
+  metadata.gz: 6031a713ec97be707b33a667121cba53ba966e30c0949c1070c62ccf4c698596
+  data.tar.gz: 01cd81e06eb1228a4b01070e0f6a12098fe10ee2cfe7620434a629f70e504005
 SHA512:
-  metadata.gz: 1dca3fc949d3df061bf63f913eda7553c6ecb18a8a8e7f270d81ba118ca1e852326bcee836200c377b2791902e395ac112e79961b4bc614f75e6ff6fca1e5697
-  data.tar.gz: 30cdc11cfc25432270751e3e8c492424131055062afb5d430a415a4d2000c55b9614db304b36a051e3f42da5e0cc96335296b499a7eff41ce37d6db191e1ce01
+  metadata.gz: abfac9447afb1c68ba1702738f1ffd67c9b2cddc860a84a4552b963eafd3e9936987c2085d25b565b8834bc70be49bf5271daa28ac02926db12a5d056cc0780b
+  data.tar.gz: '0871a287d3bc466c58a9106def0ed944e682c862f5ab30d849cfd9dfdcbd9aefb49b376791c6a61211f0cfefef000d05f62e7b0ce3a5045faf9f2f9a39eadc0d'

data/.gitignore

@@ -2,7 +2,6 @@
 /.yardoc
 /_yardoc/
 /coverage/
-/doc/
 /pkg/
 /spec/reports/
 /tmp/

data/CHANGELOG.md

@@ -1,5 +1,54 @@
 # CHANGELOG
 
+## 1.7.1 (2024-06-12)
+
+### Type checker core
+
+* Fix subtyping issues ([#1165](https://github.com/soutaro/steep/pull/1165))
+
+## 1.7.0 (2024-06-07)
+
+### Type checker core
+
+* Implement untyped function params ([#1101](https://github.com/soutaro/steep/pull/1101))
+* Fix method call type narrowing with`||` operator ([#1085](https://github.com/soutaro/steep/pull/1085))
+* Refactor `Interface::Builder` ([#1081](https://github.com/soutaro/steep/pull/1081))
+* Fix UnexpectedError with unnamed arguments  ([#1041](https://github.com/soutaro/steep/pull/1041))
+* Abort if subtyping fails to (possible) infinite loop ([#1055](https://github.com/soutaro/steep/pull/1055))
+* Ignore diagnostics by `steep:ignore` comment ([#1034](https://github.com/soutaro/steep/pull/1034))
+* Make defined? accept any type ([#1031](https://github.com/soutaro/steep/pull/1031))
+* Refactor case-when syntax ([#999](https://github.com/soutaro/steep/pull/999))
+* Update `Steep::Subtyping::Check#check` types ([#972](https://github.com/soutaro/steep/pull/972))
+
+### Commandline tool
+
+* Improve wildcard handling in pattern matching for directory paths ([#1121](https://github.com/soutaro/steep/pull/1121))
+* Fix issue with wildcard expansion in FileLoader ([#1113](https://github.com/soutaro/steep/pull/1113))
+* Avoid unnecessary calling `RBS::Location#to_s` on debug log ([#1092](https://github.com/soutaro/steep/pull/1092))
+* Better recovery from collection problems ([#987](https://github.com/soutaro/steep/pull/987))
+* Handle absolute path ([#975](https://github.com/soutaro/steep/pull/975))
+* Default to check the entire current directory when there is no `Steepfile` ([#968](https://github.com/soutaro/steep/pull/968))
+
+### Language server
+
+* Fix variable untyped hover issue in rescue clause ([#1147](https://github.com/soutaro/steep/pull/1147))
+* Handle `$/file/reset` request in interaction worker ([#1122](https://github.com/soutaro/steep/pull/1122))
+* Skip pending UI jobs ([#1035](https://github.com/soutaro/steep/pull/1035))
+* Fix no method error ([#1040](https://github.com/soutaro/steep/pull/1040))
+* Implement completion for comments ([#1039](https://github.com/soutaro/steep/pull/1039))
+* Fix signature help ([#1038](https://github.com/soutaro/steep/pull/1038))
+
+### Miscellaneous
+
+* Revise rake tasks ([#1156](https://github.com/soutaro/steep/pull/1156))
+* Update `steep.gemspec` ([#1155](https://github.com/soutaro/steep/pull/1155))
+* Add `Steep::RakeTask` ([#995](https://github.com/soutaro/steep/pull/995))
+* Stop to distribute sig dir ([#1144](https://github.com/soutaro/steep/pull/1144))
+* Update gem-rbs-collection.md ([#1127](https://github.com/soutaro/steep/pull/1127))
+* Add shape docs ([#1089](https://github.com/soutaro/steep/pull/1089))
+* Ignore smoke tests when packaging gem ([#962](https://github.com/soutaro/steep/pull/962))
+* Fix setup for Ruby 3.3 ([#1000](https://github.com/soutaro/steep/pull/1000))
+
 ## master
 
 ## 1.6.0 (2023-11-09)

data/README.md

@@ -219,6 +219,24 @@ Steep implements some of the Language Server Protocol features.
 
 Other LSP supporting tools may work with Steep where it starts the server as `steep langserver`.
 
+## Rake Tasks
+
+Steep comes with a set of configurable Rake tasks.
+
+```ruby
+# Rakefile
+
+require "steep/rake_task"
+Steep::RakeTask.new do |t|
+  t.check.severity_level = :error
+  t.watch.verbose
+end
+
+task default: [:steep]
+```
+
+Use `bundle exec rake -T` to see all available tasks.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/Rakefile

@@ -188,6 +188,11 @@ NOTES
       milestone = "Steep #{major}.#{minor}.x"
     end
 
+    if pre =~ /dev/
+      puts "🔍 Skipping `Released` tags because `dev` release: `#{pre}`"
+      next
+    end
+
     puts "🔍 Finding pull requests that is associated to milestone `#{milestone}`..."
 
     command = [

data/bin/output_test.rb

@@ -27,6 +27,7 @@ test_dirs.each do |dir|
   end
 
   command = %w(steep check --with-expectations=test_expectations.yml)
+  command << "-j2" if ENV["CI"]
   puts "  command: #{command.join(" ")}"
 
   output, status = Open3.capture2(*command, chdir: dir.to_s)

data/doc/narrowing.md

@@ -0,0 +1,195 @@
+# Narrowing Implementation
+
+> This is an internal doc for Steep developers. [Narrowing guide](../guides/narrowing/narrowing.md) is for users.
+
+The challenge is Ruby has special type predicate methods that should be supported by type checkers. `#nil?` is used instead of `unless` statement to test if a value is a `nil` or not. `#is_a?` or `#===` are used to confirm if an object is an instance of a class. Negation and equality are implemented as methods -- `#!` and `#==`.
+
+Steep supports those methods by introducing special types for those methods.
+
+```rbs
+# This is not a valid RBS type definition.
+# Steep implements a transformation from valid RBS syntax to those special types.
+module Kernel
+  def nil?: () -> RECEIVER_IS_NIL
+
+  def is_a?: (Class klass) -> RECEIVER_IS_ARG
+end
+```
+
+When type checking a conditional resulted in `RECEIVER_IS_NIL` type, the type checker overrides the type of the expressions inside the *then* and *else* clauses.
+
+```ruby
+x = [1, ""].sample      # The type of `x` is `String | Integer | nil`
+
+unless x.is_a?(String)  # 1. The condition expression has `RECEIVER_IS_NIL`
+  x.upcase              # 2. Steep overrides the type of `x` to `String` in *then* clause
+else
+                        # 3. Steep overrides the type of `x` to `Integer | nil` in *else* clause
+end
+```
+## Logical types
+
+We extend *type* with *logical types* as follows:
+
+```
+type ::= ...
+       | NOT                           # Negation of type of receiver
+       | RECEIVER_IS_NIL               # Receiver is nil when it evaluates to truthy
+       | RECEIVER_IS_NOT_NIL           # Receiver is not nil when it evaluates to truthy
+       | RECEIVER_IS_ARG               # Receiver is an instance of argument when it evaluates to truthy
+       | ARG_IS_RECEIVER               # Argument is an instance of receiver when it evaluates to truthy
+       | ARG_EQUALS_RECEIVER           # Argument is equal to receiver when it evaluates to truthy
+       | ENV(original_type, truthy_env, falsy_env)    # Two type environments for truthy and falsy
+```
+### ENV type
+
+`ENV` looks a bit different from others because it takes arguments. The type is used for `and` and `or`.
+
+Consider the example with local variables `x` and `y` where both of them have type `String?`.
+
+```ruby
+(x && y) && (x + y)   # Parens added for ease of reading
+```
+
+The type of the whole expression is `String?`. When `x` or `y` is `nil`, it evaluates to `nil`. If both of `x` and `y` is a `String`, `x + y` evaluates to `String` because of the definition of `String#+`.
+
+The type narrowing starts with the top expression.
+
+```ruby
+(...) && (x + y)
+```
+
+It immediately type checks the left hand side, but with *conditional mode*. Conditional mode is a special flag that the type checking result should keep as many of the environments as possible.
+
+```ruby
+x && y
+```
+
+Going down again, it gets a typing `x: String?` and `y: String?`. It runs a type narrowing, to obtain a result both of `x` and `y` are `String` for truthy result, both of `x` and `y` are `String?` for falsy result. The two environments should be propagated to the upper node, because the parent node is also `&&` which is a subject of type narrowing. So, it returns an `ENV` type, `String?` for original type, `{ x: String, y: String }` for truthy result, and `{ x: String?, y: String? }` for falsy result.
+
+Going up to the outer `&&` expression. The left hand side has `ENV` type, and then the right hand side is type checked based on the truthy environment (because of the semantics of `&&` expression.) Both `x` and `y` are `String` and it type checks. The type of the whole expression union of `String` and the falsy part of the original type -- `nil`.
+## Union type partition
+
+We introduce *partition* function for union types. It returns a pair of two types -- truthy parts and falsy parts. We need a slightly different variant for *non-nil* partition to support safe-navigation-operator.
+
+```
+Pt(T) -> T? ⨉ T?    # Truthy partition
+Pn(T) -> T? ⨉ T?    # Non-nil partition
+```
+
+Both return a pair of optional types for non-falsy/non-nil types.
+
+```
+Pt(false?)  -> ∅ ⨉ false?
+Pn(false)   -> false ⨉ ∅
+```
+
+Note that we cannot partition non-regular recursive types, but that types are prohibited in RBS.
+## Type environment
+
+Type environment is a mapping from local variables to their types. It's extended in Steep to support overriding type of *pure* expressions.
+
+```
+E ::= ∅
+    | x : T, E       # Type of a local variable -- `x`
+    | p : T, E       # Type of a pure expression -- `p`
+```
+
+Pure expressions are defined recursively as following:
+
+* *Value expressions* are pure
+* Call expressions of *pure methods* with pure arguments are pure
+
+Note that expression purity depends on the result of type checking of the expression because it requires to detect if a method call is *pure* or not.
+## Logic type interpreter
+Logic type interpreter is an additional layer to support type narrowing. It is a function that takes an expression and it's typing, and returns a pair of type environments -- one for the case the value of the expr is *truthy* and another for the case the value is *falsy*.
+
+```
+I(expr : T) -> E ⨉ E
+```
+
+It takes account of assignments to local variables.
+
+```
+I(x = y : String?) -> { x: String, y: String } ⨉ { x: nil, y: nil }
+```
+
+It also calculates the reachability to truthy and falsy results which can be used to detect unreachable branches.
+## Narrowing syntaxes
+### Simple conditionals -- `if`, `while`, `and`, ...
+Type checking those syntaxes are simple. It type checks the condition expression with conditional mode, passes the expression to logic type interpreter, uses the type environments to type check then and else clauses.
+
+Steep also reports unreachable branch issues based on the reachability calculated by logic type interpreter.
+### `case-when`
+The easier case is if `case-when` doesn't have a node just after `case` token.
+
+```ruby
+case
+when x == 1
+  ...
+end
+```
+
+This is the same with simple conditionals.
+
+The difficult case is if `case-when` syntax has a node.
+
+```ruby
+case foo()
+when Integer
+  ...
+when String
+  ...
+end
+```
+
+Ruby uses the `===` operator to test if the value of `foo()` matches the condition, while we don't want to type check `foo()` calls every time. It may be a pure expression, and we can give better type narrowing using the *falsy* results of predecessor `when` clauses.
+
+So, we transform the condition expression given to the logic type interpreter to value form.
+
+```ruby
+__case_when:x01jg9__ = foo()
+if Integer === __case_when:x01jg9__
+  ...
+elsif String === __case_when:x01jg9__
+  ...
+end
+```
+
+It generates a fresh local variable and assigns the expression to it. Uses the variable inside the patterns.
+
+We also need to propagate the type of local variables which are included in the expression.
+
+```ruby
+case x = foo()
+when Integer
+  x.abs      # x should be Integer
+when String
+  x.upcase   # x should be String
+end
+```
+
+The local variable insertion is done at the outermost non-assignment position to support the local variable propagation.
+
+```ruby
+__case_when:x01jg9__ = foo()
+if Integer === (x = __case_when:x01jg9__)
+  ...
+elsif String === (x = __case_when:x01jg9__)
+  ...
+end
+```
+
+The last trick to type check case-when is *pure call* narrowing in the bodies.
+
+```ruby
+__case_when:x01jg9__ = foo()
+if Integer === (x = __case_when:x01jg9__)
+  foo.abs
+elsif String === (x = __case_when:x01jg9__)
+  foo.upcase
+end
+```
+
+To support this, we propagate the type of the fresh local variable to the type of right hand side expression, if it's a pure call.
+

data/doc/shape.md

@@ -0,0 +1,194 @@
+# Shapes
+
+A *shape* is a data structure, which contains the set of available methods and their types, which is associated with a type. Steep uses shapes to type check method calls -- it calculates the shape of the type of the receiver, checks if the called method is defined on the shape and the arguments are compatible with the method, and calculates the return type.
+
+Assume an interface `_Foo` is defined as follows:
+
+```rbs
+interface _Foo
+  def foo: () -> String
+
+  def bar: () -> self
+end
+```
+
+The shape of `_Foo` will be the following:
+
+```
+Shape (_Foo) {
+  foo: () -> String,
+  bar: () -> _Foo
+}
+```
+
+Note that the `self` type in the example is resolved to `_Foo` during shape calculation.
+
+The shape calculation of an object is straightforward. Calculate a `RBS::Definition` of a class singleton/instance, or an interface, and translate the data structure to a `Shape` object. But there are a few things to consider.
+
+## Tuple, record, and proc types
+
+The shape of tuple, record, or proc types are based on their base types -- Array, Hash, or Proc classes --, but with specialized method types.
+
+```
+Shape ([Integer, String]) {
+  []: (0) -> Integer
+    | (1) -> String
+    | (Integer) -> (Integer | String)
+  ...
+}
+```
+
+The specialization is implemented as a part of shape calculation.
+
+## Special methods
+
+Steep recognizes some special methods for type narrowing, including `#is_a?`, `#===`, `#nil?`, ... These methods are defined with normal RBS syntax, but the method types in shapes are transformed to types using logic types.
+
+The shape calculation inserts the specialized methods with these special methods.
+
+## `self` types
+
+There are two cases of `self` types to consider during shape calculation.
+
+1. `self` types included in the shape of a type
+2. `self` types included in given types
+
+### 1. `self` types included in the shape of a type
+
+`self` types may be included in a class or interface definition.
+
+```rbs
+interface _Foo
+  def itself: () -> self
+end
+```
+
+The `self` types included in the shape of `_Foo` type should be resolved to `_Foo` type.
+
+```
+Shape (_Foo) {
+  itself: () -> _Foo
+}
+```
+
+### 2. `self` types included in given types
+
+Unlike `self` types included in definitions, `self` types in given types should be preserved.
+
+```rbs
+interface _Foo[A]
+  def get: () -> A
+end
+```
+
+The shape of `_Foo[self]` has `self` type as its type argument, and we want the `self` type preserved after the shape calculation.
+
+```
+Shape (_Foo[self]) {
+  get: () -> self
+}
+```
+
+We often use `self` types as the return type of a method.
+
+```rbs
+class Foo
+  def foo: () -> self
+end
+```
+
+So, the implementation of `foo` might use `self` node to return `self` type.
+
+```rb
+class Foo
+  def foo
+    # @type var foo: _Foo[self]
+    foo = ...
+    foo.get
+  end
+end
+```
+
+We want the type of `foo.get` to be `self`, not `Foo`, to avoid a type error being detected.
+
+## Shape of `self` types
+
+We also want `self` type if `self` is the type of the shape.
+
+```rb
+class Foo
+  def foo
+    self.itself
+  end
+end
+```
+
+This is a straightforward case, because the type of `self` is `self` itself. Calculate the shape of it, but keep the `self` types in the shape.
+
+```
+Shape (self) {
+  itself: () -> self
+}
+```
+
+If `self` is a union type, or something built with a type constructor, the shape calculation gets complicated.
+
+```rbs
+class Foo
+  def foo: () -> Integer
+end
+
+class Bar
+  def foo: () -> self
+end
+```
+
+What is the expected shape of `self` where the type of `self` is `Foo | Bar`?
+
+The shape of a union type is straightforward. It calculates the shape of each type, and then it calculates a union of the shape.
+
+We do the same for the case with `self` types, but it results in slightly incorrect shapes.
+
+```
+Shape (Foo) {
+  foo: () -> Integer
+}
+
+Shape (Bar) {
+  foo: () -> self   # self is preserved, because the shape of `self` is being calculated
+}
+
+Shape (Foo | Bar) {
+  foo: () -> (Integer | self)
+}
+```
+
+So, the resulting type of `self.foo` where the type of `self` is `Foo | Bar`, would be `Integer | Foo | Bar`. But, actually, it won't be `Foo` because the `self` comes from `Bar`.
+
+This is an incorrect result, but Steep is doing this right now.
+
+## `class` and `instance` types
+
+The shape calculation provides limited support for `class` and `instance` types.
+
+1. `class`/`instance` types from the definition are resolved
+2. `class`/`instance` types in generics type arguments of interfaces/instances are preserved
+3. Shape of `class`/`instance` types are resolved to configuration's `class_type` and `instance_type`, and the translated types are used to calculate the shape
+
+It's different from `self` types except case #2. The relationship between `self`/`class`/`instance` is not trivial in Ruby. All of them might be resolved to any type, which means calculating one from another of them is simply impossible.
+
+## Public methods, private methods
+
+`Shape` objects have a flag of if the shape is for *public* method calls or *private* method calls. Private method call is a form of `foo()` or `self.foo()` -- when the receiver is omitted or `self`. Public method calls are anything else.
+
+The shape calculation starts with *private methods*, and the `Shape#public_shape` method returns another shape that only has *public* methods.
+
+> Note that the private shape calculation is required even on public method calls. This means a possible chance of future optimizations.
+
+## Lazy method type calculation
+
+We rarely need all of the methods available for an object. If we want to type check a method call, we only need the method type of that method. All other methods can be just ignored.
+
+*Lazy method type calculation* is introduced for that case. Instead of calculating the types of all of the methods, it registers a block that computes the method type.
+
+It is implemented in `Steep::Interface::Shape::Entry` and used to make the shape calculation of a union type faster.

data/guides/src/gem-rbs-collection/gem-rbs-collection.md

@@ -12,30 +12,13 @@ Run rbs-collection-init to start setup.
 $ rbs collection init
 ```
 
-You have to edit the generated `rbs_collection.yaml` file. Add a few lines under gems section.
-
-```yaml
-# Download sources
-sources:
-  - name: ruby/gem_rbs_collection
-    remote: https://github.com/ruby/gem_rbs_collection.git
-    revision: main
-    repo_dir: gems
-
-# A directory to install the downloaded RBSs
-path: .gem_rbs_collection
-
-gems:
-  # Skip loading rbs gem's RBS.
-  # It's unnecessary if you don't use rbs as a library.
-  - name: rbs
-    ignore: true
-  - name: steep
-    ignore: true
-  - name: rbs_rails     # Add these lines if you use rbs_rails
-    ignore: true
-  - name: rbs_protobuf  # Add these lines if you use rbs_protobuf
-    ignore: true
+You have to edit your `Gemfile`. Specify `require: false` for gems for which you do not want type definitions.
+
+```ruby
+gem 'rbs', require: false
+gem 'steep, require: false
+gem 'rbs_rails', require: false
+gem 'rbs_protobuf', require: false
 ```
 
 Once you save the file, run the install command.

data/guides/src/getting-started/getting-started.md

@@ -16,7 +16,7 @@ and install the gems.
 $ bundle install
 ```
 
-You can install it with the gem command.
+Alternatively, you can install it with the gem command.
 
 ```
 $ gem install steep
@@ -26,14 +26,14 @@ Execute the following command to confirm if the command is successfully installe
 
 ```
 $ steep version
-$ bundle exec steep version         # When you install with bundler
+$ bundle exec steep version         # If you install with bundler
 ```
 
-We omit the `bundle exec` prefix from the following commands. Run commands with the prefix if you install Steep with bundler.
+We omit the `bundle exec` prefix from the following commands. Run commands with the prefix if you installed Steep with Bundler.
 
 ## Type checking your first Ruby script
 
-Run steep init command to generate the configuration file, Steepfile.
+Run the `steep init` command to generate the configuration file, `Steepfile`.
 
 ```
 $ steep init
@@ -54,7 +54,7 @@ Type the following Ruby code in your editor, and save it as `lib/hello.rb`.
 currencies = { US: "$", JP: "¥", UK: "£" }
 country = %w(US JP UK).sample()
 
-puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+puts "Hello! The price is #{currencies[country]}100. 💸"
 ```
 
 And type check it with Steep.
@@ -84,13 +84,13 @@ lib/hello.rb:4:39: [error] Cannot pass a value of type `(::String | nil)` as an
 Detected 1 problem from 1 file
 ```
 
-The error says that the type of the country variable causes a type error. It is expected to be a Symbol, but String or nil will be given.
+The error says that the type of the country variable causes a type error. It is expected to be a symbol, but string or `nil` will be given.
 
 Let's see how we can fix the error.
 
 ## Fixing the type error
 
-The first step is converting the string value to a symbol. We can add to_sym call.
+The first step is converting the string value to a symbol. We can add a `#to_sym` call.
 
 ```rb
 currencies = { US: "$", JP: "¥", UK: "£" }
@@ -128,7 +128,7 @@ Instead, we can simply tell the type checker that the value of the country canno
 
 The underlying type system supports flow-sensitive typing similar to TypeScript and Rust. It detects conditional expressions testing the value of a variable and propagates the knowledge that the value cannot be `nil`.
 
-We can fix the type error with an or construct.
+We can fix the type error with an `or` construct.
 
 ```rb
 currencies = { US: "$", JP: "¥", UK: "£" }
@@ -137,7 +137,7 @@ country = %w(US JP UK).sample() or raise
 puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
 ```
 
-The change let the type checking succeed.
+The change lets the type checking succeed.
 
 ```
 $ steep check
@@ -145,7 +145,7 @@ $ steep check
 
 .........................................................
 
-No type error detected. 🧉
+No type error detected. 🫖
 ```
 
 The `raise` method is called when `sample()` returns `nil`. Steep can reason the possible control flow based on the semantics of or in Ruby:
@@ -161,4 +161,3 @@ There are two possibilities of the type of the result of the `sample()` call, `n
 ## Next steps
 
 This is a really quick introduction to using Steep. You may have noticed that I haven't explained anything about defining new classes or modules. See the RBS guide for more examples!
-