steep-relaxed 1.9.3.3

data/guides/src/gem-rbs-collection/gem-rbs-collection.md

@@ -0,0 +1,126 @@
+# Using RBS from gem_rbs_collection
+
+gem_rbs_collection is a repository of type definitions of gems that are managed by the community. You may find the type definition of a gem that ships without RBS type definitions.
+
+To use RBS files from the repository, you can use rbs-collection subcommand. This guide explains how the command works.
+
+## Quick start
+
+Run rbs-collection-init to start setup.
+
+```
+$ rbs collection init
+```
+
+You have to edit your `Gemfile`. Specify `require: false` for gems for which you do not want type definitions.
+
+```ruby
+gem 'rbs', require: false
+gem 'steep', require: false
+gem 'rbs_rails', require: false
+gem 'rbs_protobuf', require: false
+```
+
+Once you save the file, run the install command.
+
+```
+$ rbs collection install
+```
+
+That generates `rbs_collection.lock.yaml` and downloads the RBS files from the git repository.
+
+Note that rbs-collection automatically downloads RBS files of gems included in your Bundler environment. You don't have to write all of the gems in your `Gemfile` in `rbs_collection.yaml`.
+
+Finally, we recommend adding the `rbs_collection.yaml` and `rbs_collection.lock.yaml` to your repository, and ignoring `.gem_rbs_collection` directory.
+
+```
+$ git add rbs_collection.yaml
+$ git add rbs_collection.lock.yaml
+$ echo /.gem_rbs_collection >> .gitignore
+$ git commit -m "Set up rbs-collection"
+```
+
+## Updating RBS files
+
+You may want to run rbs-collection-update to update the contents of `rbs_collection.lock.yaml`, when you add a new gem or some gems are updated.
+
+You also need to run rbs-collection-update when the RBS files in the source repository are updated. The type definitions are updated with bug-fixes or improvements, and you need to update to apply the changes to your app.
+
+## Using rbs-collection with Steep
+
+Steep automatically reads `rb_collection.yaml`. You can use Steep immediately without any modifications.
+
+```
+$ bin/steep project    # Show dependencies detected by Steep
+$ bin/steep check      # Run type check
+```
+
+## Migration
+
+If you have used older versions of Steep or RBS, you may have configured libraries manually.
+
+* You have library calls in `Steepfile`
+* You have git submodules in your git repository to download gem_rbs_collection
+
+These are steps to migrate to rbs-collection.
+
+1. Remove unnecessary library configuration
+2. Set up rbs-collection
+3. Validate the configuration
+4. Delete submodule
+
+### 1. Remove unnecessary library configurations
+
+You may have `#library` calls in your Steepfile, or something equivalent in your scripts. We can group the configured libraries into three groups.
+
+1. Gems that is managed by Bundler
+2. Standard libraries (non-gem libs, default gems, and bundled gems)
+    * 2-1) That is a transitive dependency from libraries in group 1
+    * 2-2) That is included in Gemfile.lock
+    * 2-3) Implicitly installed gems – not included in Gemfile
+
+You can delete library configs of 1, 2-1, and 2-2. So, you have to keep the configurations of libraries in 2-3.
+
+Practically, you can remove all library configs and `#library` calls in `Steepfile`, go step 2, run steep check to test, and restore the configs of libraries if error is detected.
+
+### 2. Set up rbs-collection
+
+See the quick start section above!
+
+### 3. Validate the configuration
+
+Run steep check to validate the configuration.
+
+```
+$ bin/steep check
+```
+
+You may see the `UnknownTypeName` error if some libraries are missing. Or some errors that implies duplication or inconsistency of methods/class/module definitions, that may be caused by loading RBS files of a library twice.
+
+Running steep project may help you too. It shows the source files and library directories recognized by Steep.
+
+```
+$ bin/steep project
+```
+
+Note that the type errors detected may change after migrating to rbs-collection, because it typically includes updating to newer versions of RBS files.
+
+### 4. Delete submodule
+
+After you confirmed everything is working correctly, you can delete the submodule. deinit the submodule, remove the directory using git-rm, and delete $GIT_DIR/modules/<name>/.
+
+## What is the rbs_collection.yaml file?
+
+The file mainly defines three properties – sources, gems and path. Sources is essential when you want to create a new RBS file repository, usually for RBS files of the private gems.
+
+Another trick is ignoring RBS files of type checker toolchain. RBS and Steep ships with their own RBS files. However, these RBS files may be unnecessary for you, unless you are not a type checking toolchain developer. It requires some additional gems and RBS files. So adding ignore: true is recommended for the gems.
+
+It seems like we need to add a feature to skip loading RBS files automatically and use the feature for RBS, Steep, and RBS Rails. It looks weird that the gems section is only used to ignore gems.
+
+## Versions of RBS files in gem_rbs_collection
+
+Gem versions in rbs-collection are relatively loosely managed. If a gem is found but the version is different, rbs-collection simply uses the incorrect version.
+
+It will load RBS files of activerecord/6.1 even if your Gemfile specifies activerecord-7.0.4. This is by design with an assumption that having RBS files with some incompatibility is better than having nothing. We see most APIs are compatible even after major version upgrade. Dropping everything for minor API incompatibilities would not make much sense.
+
+That behavior will change in future versions when we see that the assumption is not reasonable and we have better coverage.

data/guides/src/getting-started/getting-started.md

@@ -0,0 +1,163 @@
+# Getting Started with Steep in 5 minutes
+
+## Installing Steep
+
+Add the lines to Gemfile:
+
+```rb
+group :development do
+  gem "steep", require: false
+end
+```
+
+and install the gems.
+
+```
+$ bundle install
+```
+
+Alternatively, you can install it with the gem command.
+
+```
+$ gem install steep
+```
+
+Execute the following command to confirm if the command is successfully installed.
+
+```
+$ steep version
+$ bundle exec steep version         # If you install with bundler
+```
+
+We omit the `bundle exec` prefix from the following commands. Run commands with the prefix if you installed Steep with Bundler.
+
+## Type checking your first Ruby script
+
+Run the `steep init` command to generate the configuration file, `Steepfile`.
+
+```
+$ steep init
+```
+
+Open `Steepfile` in your text editor, and replace the content with the following lines:
+
+```rb
+target :lib do
+  signature "sig"
+  check "lib"
+end
+```
+
+Type the following Ruby code in your editor, and save it as `lib/hello.rb`.
+
+```rb
+currencies = { US: "$", JP: "¥", UK: "£" }
+country = %w(US JP UK).sample()
+
+puts "Hello! The price is #{currencies[country]}100. 💸"
+```
+
+And type check it with Steep.
+
+```
+$ steep check
+```
+
+The output will report a type error.
+
+```
+# Type checking files:
+
+........................................................F
+
+lib/hello.rb:4:39: [error] Cannot pass a value of type `(::String | nil)` as an argument of type `::Symbol`
+│   (::String | nil) <: ::Symbol
+│     ::String <: ::Symbol
+│       ::Object <: ::Symbol
+│         ::BasicObject <: ::Symbol
+│
+│ Diagnostic ID: Ruby::ArgumentTypeMismatch
+│
+└ puts "Hello! The price is #{currencies[country]}100. 💸"
+                                         ~~~~~~~
+
+Detected 1 problem from 1 file
+```
+
+The error says that the type of the country variable causes a type error. It is expected to be a symbol, but string or `nil` will be given.
+
+Let's see how we can fix the error.
+
+## Fixing the type error
+
+The first step is converting the string value to a symbol. We can add a `#to_sym` call.
+
+```rb
+currencies = { US: "$", JP: "¥", UK: "£" }
+country = %w(US JP UK).sample()
+
+puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+```
+
+The `#to_sym` call will convert a string into a symbol. Does it solve the problem??
+
+```
+$ steep check
+# Type checking files:
+
+........................................................F
+
+lib/hello.rb:4:47: [error] Type `(::String | nil)` does not have method `to_sym`
+│ Diagnostic ID: Ruby::NoMethod
+│
+└ puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+                                                 ~~~~~~
+
+Detected 1 problem from 1 file
+```
+
+It detects another problem. The first error was `Ruby::ArgumentTypeMismatch`, but the new error is `Ruby::NoMethod`. The value of `country` may be `nil`, and it doesn't have the `#to_sym` method.
+
+This would be annoying, but one of the most common sources of a type error. The value of an expression may be `nil` unexpectedly, and using the value of the expression may cause an error.
+
+In this case, the `sample()` call introduces the `nil`. `Array#sample()` returns `nil` when the array is empty. We know the receiver of the `sample` call cannot be `nil`, because it is an array literal. But the type checker doesn't know of it. The source code detects the `Array#sample()` method is called, but it ignores the fact that the receiver cannot be empty.
+
+Instead, we can simply tell the type checker that the value of the country cannot be `nil`.
+
+# Satisfying the type checker by adding a guard
+
+The underlying type system supports flow-sensitive typing similar to TypeScript and Rust. It detects conditional expressions testing the value of a variable and propagates the knowledge that the value cannot be `nil`.
+
+We can fix the type error with an `or` construct.
+
+```rb
+currencies = { US: "$", JP: "¥", UK: "£" }
+country = %w(US JP UK).sample() or raise
+
+puts "Hello! The price is #{currencies[country.to_sym]}100. 💸"
+```
+
+The change lets the type checking succeed.
+
+```
+$ steep check
+# Type checking files:
+
+.........................................................
+
+No type error detected. 🫖
+```
+
+The `raise` method is called when `sample()` returns `nil`. Steep can reason the possible control flow based on the semantics of or in Ruby:
+
+* The value of `country` is the return value of `sample()` method call
+* The value of `country` may be `nil`
+  * If the value of `country` is `nil`, the right hand side of the or is evaluated
+  * It calls the `raise` method, which results in an exception and jumps to somewhere
+  * So, when the execution continues to the puts line, the value of `country` cannot be `nil`
+
+There are two possibilities of the type of the result of the `sample()` call, `nil` or a string. We humans can reason that we can safely ignore the case of `nil`. But, the type checker cannot. We have to add `or raise` to tell the type checker it can stop considering the case of `nil` safely.
+
+## Next steps
+
+This is a really quick introduction to using Steep. You may have noticed that I haven't explained anything about defining new classes or modules. See the RBS guide for more examples!

data/guides/src/nil-optional/nil-optional.md

@@ -0,0 +1,195 @@
+# nil and Optional Types
+
+`nil`s have been the most common source of the errors you see when testing and after the deployment of your apps.
+
+```
+NoMethodError (undefined method `save!' for nil:NilClass)
+```
+
+Steep/RBS provides *optional types* to help you identify the problems while you are coding.
+
+## Preventing `nil` problems
+
+Technically, there is only one way to prevent `nil` problems – test everytime if the value is `nil` before calling methods with the receiver.
+
+```rb
+if account
+  account.save!
+end
+```
+
+Using the `if` statement is the most popular way to ensure the value is not `nil`. But you can do it with safe-navigation-operators, case-when (or case-in), and `#try` method.
+
+```rb
+account&.save!
+
+case account
+when Account
+  account.save!
+end
+
+account.try {|account| account.save! }
+```
+
+It's simple, but not easy to do in your code.
+
+**You may forget testing.** This happens easily. You don't notice that you forget until you deploy the code into production and it crashes after loading an account that is old and satisfies complicated conditions that leads it to be `nil`.
+
+**You may add redundant guards.** This won't get your app crash, but it will make understanding your code more difficult. It adds unnecessary noise that tells your teammates this can be `nil` in some place, and results in another redundant guard.
+
+The `nil` problems can be solved by a tool that tells you:
+
+* If the value can be `nil` or not, and
+* You forget testing the value before using the value
+
+RBS has a language construct to do this called optional types and Steep implements analysis to let you know if you forget testing the value.
+
+## Optional types
+
+Optional types in RBS are denoted with a suffix `?` – Type?. It means the value of the type may be `nil`.
+
+```
+Integer?            # Integer or nil
+Array[Account]?     # An array of Account or nil
+```
+
+Note that optional types can be included in other types as:
+
+```
+Array[Account?]
+```
+
+The value of the type above is always an array, but the element may be `nil`.
+
+In other words, a non optional type in RBS means the value cannot be `nil`.
+
+```
+Integer            # Integer, cannot be nil
+Array[Account]     # An array, cannot be nil
+```
+
+Let's see how Steep reports errors on optional and non-optional types.
+
+```rb
+account = Account.find(1)
+account.save!
+```
+
+Assume the type of `account` is `Account` (non optional type), the code type checks successfully. There is no chance to be `nil` here. The `save!` method call never results in a `NoMethodError`.
+
+```rb
+account = Account.find_by(email: "soutaro@squareup.com")
+account.save!
+```
+
+Steep reports a `NoMethod` error on the `save!` call. Because the value of the `account` may be `nil`, depending on the actual records in the `accounts` table. You cannot call the `save!` method without checking if the `account` is `nil`.
+
+You cannot assign `nil` to a local variable with non-optional types.
+
+```rb
+# @type var account: Account
+
+account = nil
+account = Account.find_by(email: "soutaro@squareup.com")
+```
+
+Because the type of `account` is declared Account, non-optional type, it cannot be `nil`. And Steep detects a type error if you try to assign `nil`. Same for assigning an optional type value at the last line.
+
+# Unwrapping optional types
+
+There are several ways to unwrap optional types. The most common one is using if.
+
+```rb
+account = Account.find_by(id: 1)
+if account
+  account.save!
+end
+```
+
+The *if* statement tests if `account` is `nil`. Inside the then clause, `account` cannot be `nil`. Then Steep type checks the code.
+
+This works for *else* clause of *unless*.
+
+```rb
+account = Account.find_by(id: 1)
+unless account
+  # Do something
+else
+  account.save!
+end
+```
+
+This also type checks successfully.
+
+Steep supports `nil?` predicate too.
+
+```rb
+unless (account = Account.find_by(id: 1)).nil?
+  account.save!
+end
+```
+
+This assumes the `Account` class doesn't have a custom `nil?` method, but keeps the built-in `nil?` or equivalent.
+
+The last one is using safe-nevigation-navigator. It checks if the receiver is `nil` and calls the method if it is not. Otherwise just evaluates to `nil`.
+
+```rb
+account = Account.find_by(id: 1)
+account&.save!
+```
+
+This is a shorthand for the case you don't do any error handling case if it is `nil`.
+
+## What should I do for the case of `nil`?
+
+There is no universal answer for this question. You may just stop the execution of the method by returning. You may want to insert a new account to ensure the record exists. Raising an exception with a detailed error message will help troubleshooting.
+
+It depends on what the program is expected to do. Steep just checks if accessing `nil` may happen or not. The developers only know how to handle the `nil` cases.
+
+# Handling unwanted `nil`s
+
+When you start using Steep, you may see many unwanted `nil`s. This typically happens when you want to use Array methods, like `first` or `sample`.
+
+```rb
+account = accounts.first
+account.save!
+```
+
+It returns `nil` if the array is empty. Steep cannot detect if the array is empty or not, and it conservatively assumes the return value of the methods may be `nil`. While you know the `account` array is not empty, Steep infer the `first` method may return `nil`.
+
+This is one of the most frequently seen sources of unwanted `nil`s.
+
+## Raising an error
+
+In this case, you have to add an extra code to let Steep unwrap it.
+
+```rb
+account = accounts.first or raise
+account.save!
+```
+
+My recommendation is to raise an exception, `|| raise` or `or raise`. It raises an exception in the case of `nil`, and Steep unwraps the type of the `account` variable.
+
+Exceptions are better than other control flow operators – `return`/`break`/`next`. It doesn't affect the control flow until it actually happens during execution, and the type checking result other than the unwrapping is changed.
+
+An `#raise` call without argument is my favorite. It's short. It's uncommon in the Ruby code and it can tell the readers that something unexpected is happening.
+
+But of course, you can add some message:
+
+```rb
+account = accounts.first or raise("accounts cannot be empty")
+account.save!
+```
+
+## Type assertions
+
+You can also use a type assertion, that is introduced in Steep 1.3.
+
+```rb
+account = accounts.first #: Account
+account.save!
+```
+
+It tells Steep that the right hand side of the assignment is `Account`. That overwrites the type checking algorithm, and the developer is responsible for making sure the value cannot be `nil`.
+
+Note: Nothing happens during the execution. It just works for Steep and Ruby doesn't do any extra type checking on it. I recommend using the `or raise` idiom for most of the cases.

data/lib/steep/annotation_parser.rb

@@ -0,0 +1,199 @@
+module Steep
+  class AnnotationParser
+    VAR_NAME = /[a-z][A-Za-z0-9_]*/
+    METHOD_NAME = Regexp.union(
+      /[^.:\s]+/
+    )
+    CONST_NAME = Regexp.union(
+      /(::)?([A-Z][A-Za-z0-9_]*::)*[A-Z][A-Za-z0-9_]*/
+    )
+    DYNAMIC_NAME = /(self\??\.)?#{METHOD_NAME}/
+    IVAR_NAME = /@[^:\s]+/
+
+    attr_reader :factory
+
+    def initialize(factory:)
+      @factory = factory
+    end
+
+    class SyntaxError < StandardError
+      attr_reader :source
+      attr_reader :location
+
+      def initialize(source:, location:, exn: nil, message: nil)
+        @source = source
+        @location = location
+
+        if exn
+          message =
+            case exn
+            when RBS::ParsingError
+              Diagnostic::Signature::SyntaxError.parser_syntax_error_message(exn)
+            else
+              exn.message
+            end
+        end
+
+        super message
+      end
+    end
+
+    TYPE = /(?<type>.*)/
+    COLON = /\s*:\s*/
+
+    PARAM = /[A-Z][A-Za-z0-9_]*/
+    TYPE_PARAMS = /(\[(?<params>#{PARAM}(,\s*#{PARAM})*)\])?/
+
+    def parse_type(match, name = :type, location:)
+      string = match[name] or raise
+      st, en = match.offset(name)
+      st or raise
+      en or raise
+      loc = RBS::Location.new(location.buffer, location.start_pos + st, location.start_pos + en)
+
+      type =
+        begin
+          RBS::Parser.parse_type(string)
+        rescue RBS::ParsingError => exn
+          raise SyntaxError.new(source: string, location: loc, exn: exn)
+        end or raise
+
+      unless (type.location || raise).source == string.strip
+        raise SyntaxError.new(source: string, location: loc, message: "Failed to parse a type in annotation")
+      end
+
+      factory.type(type)
+    end
+
+    def keyword_subject_type(keyword, name)
+      /@type\s+#{keyword}\s+(?<name>#{name})#{COLON}#{TYPE}/
+    end
+
+    def keyword_and_type(keyword)
+      /@type\s+#{keyword}#{COLON}#{TYPE}/
+    end
+
+    def parse(src, location:)
+      case src
+      when keyword_subject_type("var", VAR_NAME)
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          name = match[:name] or raise
+
+          AST::Annotation::VarType.new(name: name.to_sym,
+                                       type: parse_type(match, location: location),
+                                       location: location)
+        end
+
+      when keyword_subject_type("method", METHOD_NAME)
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          name = match[:name] or raise
+          type = match[:type] or raise
+
+          method_type = factory.method_type(RBS::Parser.parse_method_type(type) || raise)
+
+          AST::Annotation::MethodType.new(name: name.to_sym,
+                                          type: method_type,
+                                          location: location)
+        end
+
+      when keyword_subject_type("const", CONST_NAME)
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          name = match[:name] or raise
+          type = parse_type(match, location: location)
+
+          AST::Annotation::ConstType.new(name: RBS::TypeName.parse(name), type: type, location: location)
+        end
+
+      when keyword_subject_type("ivar", IVAR_NAME)
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          name = match[:name] or raise
+          type = parse_type(match, location: location)
+
+          AST::Annotation::IvarType.new(name: name.to_sym,
+                                         type: type,
+                                         location: location)
+        end
+
+      when keyword_and_type("return")
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type = parse_type(match, location: location)
+          AST::Annotation::ReturnType.new(type: type, location: location)
+        end
+
+      when keyword_and_type("block")
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type = parse_type(match, location: location)
+          AST::Annotation::BlockType.new(type: type, location: location)
+        end
+
+      when keyword_and_type("self")
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type = parse_type(match, location: location)
+          AST::Annotation::SelfType.new(type: type, location: location)
+        end
+
+      when keyword_and_type("instance")
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type = parse_type(match, location: location)
+          AST::Annotation::InstanceType.new(type: type, location: location)
+        end
+
+      when keyword_and_type("module")
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type = parse_type(match, location: location)
+          AST::Annotation::ModuleType.new(type: type, location: location)
+        end
+
+      when keyword_and_type("break")
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type = parse_type(match, location: location)
+
+          AST::Annotation::BreakType.new(type: type, location: location)
+        end
+
+      when /@dynamic\s+(?<names>(#{DYNAMIC_NAME}\s*,\s*)*#{DYNAMIC_NAME})/
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          names = (match[:names] || raise).split(/\s*,\s*/)
+
+          AST::Annotation::Dynamic.new(
+            names: names.map {|name|
+              case
+              when name.delete_prefix!("self.")
+                AST::Annotation::Dynamic::Name.new(name: name.to_sym, kind: :module)
+              when name.delete_prefix!("self?.")
+                AST::Annotation::Dynamic::Name.new(name: name.to_sym, kind: :module_instance)
+              else
+                AST::Annotation::Dynamic::Name.new(name: name.to_sym, kind: :instance)
+              end
+            },
+            location: location
+          )
+        end
+
+      when /@implements\s+(?<name>#{CONST_NAME})#{TYPE_PARAMS}$/
+        Regexp.last_match.yield_self do |match|
+          match or raise
+          type_name = RBS::TypeName.parse(match[:name] || raise)
+          params = match[:params]&.yield_self {|params| params.split(/,/).map {|param| param.strip.to_sym } } || []
+
+          name = AST::Annotation::Implements::Module.new(name: type_name, args: params)
+          AST::Annotation::Implements.new(name: name, location: location)
+        end
+      end
+
+    rescue RBS::ParsingError => exn
+      raise SyntaxError.new(source: src, location: location, exn: exn)
+    end
+  end
+end