steep 1.4.0.dev.1 → 1.4.0.dev.3

data/guides/src/nil-optional/nil-optional.md

@@ -0,0 +1,195 @@
+# nil and Optional Types
+
+`nil`s have been the most common source of the errors you see when testing and after the deployment of your apps.
+
+```
+NoMethodError (undefined method `save!' for nil:NilClass)
+```
+
+Steep/RBS provides *optional types* to help you identify the problems while you are coding.
+
+## Preventing `nil` problems
+
+Technically, there is only one way to prevent `nil` problems – test everytime if the value is `nil` before calling methods with the receiver.
+
+```rb
+if account
+  account.save!
+end
+```
+
+Using the `if` statement is the most popular way to ensure the value is not `nil`. But you can do it with safe-navigation-operators, case-when (or case-in), and `#try` method.
+
+```rb
+account&.save!
+
+case account
+when Account
+  account.save!
+end
+
+account.try {|account| account.save! }
+```
+
+It's simple, but not easy to do in your code.
+
+**You may forget testing.** This happens easily. You don't notice that you forget until you deploy the code into production and it crashes after loading an account that is old and satisfies complicated conditions that leads it to be `nil`.
+
+**You may add redundant guards.** This won't get your app crash, but it will make understanding your code more difficult. It adds unnecessary noise that tells your teammates this can be `nil` in some place, and results in another redundant guard.
+
+The `nil` problems can be solved by a tool that tells you:
+
+* If the value can be `nil` or not, and
+* You forget testing the value before using the value
+
+RBS has a language construct to do this called optional types and Steep implements analysis to let you know if you forget testing the value.
+
+## Optional types
+
+Optional types in RBS are denoted with a suffix `?` – Type?. It means the value of the type may be `nil`.
+
+```
+Integer?            # Integer or nil
+Array[Account]?     # An array of Account or nil
+```
+
+Note that optional types can be included in other types as:
+
+```
+Array[Account?]
+```
+
+The value of the type above is always an array, but the element may be `nil`.
+
+In other words, a non optional type in RBS means the value cannot be `nil`.
+
+```
+Integer            # Integer, cannot be nil
+Array[Account]     # An array, cannot be nil
+```
+
+Let's see how Steep reports errors on optional and non-optional types.
+
+```rb
+account = Account.find(1)
+account.save!
+```
+
+Assume the type of `account` is `Account` (non optional type), the code type checks successfully. There is no chance to be `nil` here. The `save!` method call never results in a `NoMethodError`.
+
+```rb
+account = Account.find_by(email: "soutaro@squareup.com")
+account.save!
+```
+
+Steep reports a `NoMethod` error on the `save!` call. Because the value of the `account` may be `nil`, depending on the actual records in the `accounts` table. You cannot call the `save!` method without checking if the `account` is `nil`.
+
+You cannot assign `nil` to a local variable with non-optional types.
+
+```rb
+# @type var account: Account
+
+account = nil
+account = Account.find_by(email: "soutaro@squareup.com")
+```
+
+Because the type of `account` is declared Account, non-optional type, it cannot be `nil`. And Steep detects a type error if you try to assign `nil`. Same for assigning an optional type value at the last line.
+
+# Unwrapping optional types
+
+There are several ways to unwrap optional types. The most common one is using if.
+
+```rb
+account = Account.find_by(id: 1)
+if account
+  account.save!
+end
+```
+
+The *if* statement tests if `account` is `nil`. Inside the then clause, `account` cannot be `nil`. Then Steep type checks the code.
+
+This works for *else* clause of *unless*.
+
+```rb
+account = Account.find_by(id: 1)
+unless account
+  # Do something
+else
+  account.save!
+end
+```
+
+This also type checks successfully.
+
+Steep supports `nil?` predicate too.
+
+```rb
+unless (account = Account.find_by(id: 1)).nil?
+  account.save!
+end
+```
+
+This assumes the `Account` class doesn't have a custom `nil?` method, but keeps the built-in `nil?` or equivalent.
+
+The last one is using safe-nevigation-navigator. It checks if the receiver is `nil` and calls the method if it is not. Otherwise just evaluates to `nil`.
+
+```rb
+account = Account.find_by(id: 1)
+account&.save!
+```
+
+This is a shorthand for the case you don't do any error handling case if it is `nil`.
+
+## What should I do for the case of `nil`?
+
+There is no universal answer for this question. You may just stop the execution of the method by returning. You may want to insert a new account to ensure the record exists. Raising an exception with a detailed error message will help troubleshooting.
+
+It depends on what the program is expected to do. Steep just checks if accessing `nil` may happen or not. The developers only know how to handle the `nil` cases.
+
+# Handling unwanted `nil`s
+
+When you start using Steep, you may see many unwanted `nil`s. This typically happens when you want to use Array methods, like `first` or `sample`.
+
+```rb
+account = accounts.first
+account.save!
+```
+
+It returns `nil` if the array is empty. Steep cannot detect if the array is empty or not, and it conservatively assumes the return value of the methods may be `nil`. While you know the `account` array is not empty, Steep infer the `first` method may return `nil`.
+
+This is one of the most frequently seen sources of unwanted `nil`s.
+
+## Raising an error
+
+In this case, you have to add an extra code to let Steep unwrap it.
+
+```rb
+account = accounts.first or raise
+account.save!
+```
+
+My recommendation is to raise an exception, `|| raise` or `or raise`. It raises an exception in the case of `nil`, and Steep unwraps the type of the `account` variable.
+
+Exceptions are better than other control flow operators – `return`/`break`/`next`. It doesn't affect the control flow until it actually happens during execution, and the type checking result other than the unwrapping is changed.
+
+An `#raise` call without argument is my favorite. It's short. It's uncommon in the Ruby code and it can tell the readers that something unexpected is happening.
+
+But of course, you can add some message:
+
+```rb
+account = accounts.first or raise("accounts cannot be empty")
+account.save!
+```
+
+## Type assertions
+
+You can also use a type assertion, that is introduced in Steep 1.3.
+
+```rb
+account = accounts.first #: Account
+account.save!
+```
+
+It tells Steep that the right hand side of the assignment is `Account`. That overwrites the type checking algorithm, and the developer is responsible for making sure the value cannot be `nil`.
+
+Note: Nothing happens during the execution. It just works for Steep and Ruby doesn't do any extra type checking on it. I recommend using the `or raise` idiom for most of the cases.

data/lib/steep/annotation_parser.rb

@@ -20,16 +20,19 @@ module Steep
       attr_reader :source
       attr_reader :location
 
-      def initialize(source:, location:, exn: nil)
+      def initialize(source:, location:, exn: nil, message: nil)
         @source = source
         @location = location
 
-        message = case exn
-                  when RBS::ParsingError
-                    Diagnostic::Signature::SyntaxError.parser_syntax_error_message(exn)
-                  when Exception
-                    exn.message
-                  end
+        if exn
+          message =
+            case exn
+            when RBS::ParsingError
+              Diagnostic::Signature::SyntaxError.parser_syntax_error_message(exn)
+            else
+              exn.message
+            end
+        end
 
         super message
       end
@@ -41,8 +44,25 @@ module Steep
     PARAM = /[A-Z][A-Za-z0-9_]*/
     TYPE_PARAMS = /(\[(?<params>#{PARAM}(,\s*#{PARAM})*)\])?/
 
-    def parse_type(string)
-      factory.type(RBS::Parser.parse_type(string))
+    def parse_type(match, name = :type, location:)
+      string = match[name] or raise
+      st, en = match.offset(name)
+      st or raise
+      en or raise
+      loc = RBS::Location.new(location.buffer, location.start_pos + st, location.start_pos + en)
+
+      type =
+        begin
+          RBS::Parser.parse_type(string)
+        rescue RBS::ParsingError => exn
+          raise SyntaxError.new(source: string, location: loc, exn: exn)
+        end or raise
+
+      unless (type.location || raise).source == string.strip
+        raise SyntaxError.new(source: string, location: loc, message: "Failed to parse a type in annotation")
+      end
+
+      factory.type(type)
     end
 
     def keyword_subject_type(keyword, name)
@@ -59,10 +79,9 @@ module Steep
         Regexp.last_match.yield_self do |match|
           match or raise
           name = match[:name] or raise
-          type = match[:type] or raise
 
           AST::Annotation::VarType.new(name: name.to_sym,
-                                       type: parse_type(type),
+                                       type: parse_type(match, location: location),
                                        location: location)
         end
 
@@ -72,7 +91,7 @@ module Steep
           name = match[:name] or raise
           type = match[:type] or raise
 
-          method_type = factory.method_type(RBS::Parser.parse_method_type(type), method_decls: Set[])
+          method_type = factory.method_type(RBS::Parser.parse_method_type(type) || raise, method_decls: Set[])
 
           AST::Annotation::MethodType.new(name: name.to_sym,
                                           type: method_type,
@@ -83,7 +102,7 @@ module Steep
         Regexp.last_match.yield_self do |match|
           match or raise
           name = match[:name] or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
 
           AST::Annotation::ConstType.new(name: TypeName(name), type: type, location: location)
         end
@@ -92,7 +111,7 @@ module Steep
         Regexp.last_match.yield_self do |match|
           match or raise
           name = match[:name] or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
 
           AST::Annotation::IvarType.new(name: name.to_sym,
                                          type: type,
@@ -102,42 +121,43 @@ module Steep
       when keyword_and_type("return")
         Regexp.last_match.yield_self do |match|
           match or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
           AST::Annotation::ReturnType.new(type: type, location: location)
         end
 
       when keyword_and_type("block")
         Regexp.last_match.yield_self do |match|
           match or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
           AST::Annotation::BlockType.new(type: type, location: location)
         end
 
       when keyword_and_type("self")
         Regexp.last_match.yield_self do |match|
           match or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
           AST::Annotation::SelfType.new(type: type, location: location)
         end
 
       when keyword_and_type("instance")
         Regexp.last_match.yield_self do |match|
           match or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
           AST::Annotation::InstanceType.new(type: type, location: location)
         end
 
       when keyword_and_type("module")
         Regexp.last_match.yield_self do |match|
           match or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
           AST::Annotation::ModuleType.new(type: type, location: location)
         end
 
       when keyword_and_type("break")
         Regexp.last_match.yield_self do |match|
           match or raise
-          type = parse_type(match[:type] || raise)
+          type = parse_type(match, location: location)
+
           AST::Annotation::BreakType.new(type: type, location: location)
         end
 

data/lib/steep/ast/types/factory.rb

@@ -370,19 +370,11 @@ module Steep
         end
 
         def module_name?(type_name)
-          if entry = env.class_decls[type_name]
-            entry.is_a?(RBS::Environment::ModuleEntry)
-          else
-            false
-          end
+          env.module_entry(type_name) ? true : false
         end
 
         def class_name?(type_name)
-          if entry = env.class_decls[type_name]
-            entry.is_a?(RBS::Environment::ClassEntry)
-          else
-            false
-          end
+          env.class_entry(type_name) ? true : false
         end
 
         def env
@@ -434,6 +426,60 @@ module Steep
             nil
           end
         end
+
+        def normalize_type(type)
+          case type
+          when AST::Types::Name::Instance
+            AST::Types::Name::Instance.new(
+              name: env.normalize_module_name(type.name),
+              args: type.args.map {|ty| normalize_type(ty) },
+              location: type.location
+            )
+          when AST::Types::Name::Singleton
+            AST::Types::Name::Singleton.new(
+              name: env.normalize_module_name(type.name),
+              location: type.location
+            )
+          when AST::Types::Any, AST::Types::Boolean, AST::Types::Bot, AST::Types::Nil,
+            AST::Types::Top, AST::Types::Void, AST::Types::Literal, AST::Types::Class, AST::Types::Instance,
+            AST::Types::Self, AST::Types::Var, AST::Types::Logic::Base
+            type
+          when AST::Types::Intersection
+            AST::Types::Intersection.build(
+              types: type.types.map {|type| normalize_type(type) },
+              location: type.location
+            )
+          when AST::Types::Union
+            AST::Types::Union.build(
+              types: type.types.map {|type| normalize_type(type) },
+              location: type.location
+            )
+          when AST::Types::Record
+            AST::Types::Record.new(
+              elements: type.elements.transform_values {|type| normalize_type(type) },
+              location: type.location
+            )
+          when AST::Types::Tuple
+            AST::Types::Tuple.new(
+              types: type.types.map {|type| normalize_type(type) },
+              location: type.location
+            )
+          when AST::Types::Proc
+            type.map_type {|type| normalize_type(type) }
+          when AST::Types::Name::Alias
+            AST::Types::Name::Alias.new(
+              name: type.name,
+              args: type.args.map {|ty| normalize_type(ty) },
+              location: type.location
+            )
+          when AST::Types::Name::Interface
+            AST::Types::Name::Interface.new(
+              name: type.name,
+              args: type.args.map {|ty| normalize_type(ty) },
+              location: type.location
+            )
+          end
+        end
       end
     end
   end

data/lib/steep/ast/types/name.rb

@@ -20,6 +20,10 @@ module Steep
           def level
             [0]
           end
+
+          def map_type(&block)
+            self
+          end
         end
 
         class Applying < Base
@@ -87,6 +91,12 @@ module Steep
           def level
             [0] + level_of_children(args)
           end
+
+          def map_type(&block)
+            args = self.args.map(&block)
+
+            _ = self.class.new(name: self.name, args: self.args, location: self.location)
+          end
         end
 
         class Singleton < Base

data/lib/steep/diagnostic/ruby.rb

@@ -37,22 +37,72 @@ module Steep
           when relation.interface?
             nil
           when relation.block?
-            nil
+            "(Blocks are incompatible)"
           when relation.function?
             nil
           when relation.params?
-            nil
+            "(Params are incompatible)"
           end
         end
 
         def detail_lines
-          StringIO.new.tap do |io|
-            result.failure_path&.reverse_each.map do |result|
+          lines = StringIO.new.tap do |io|
+            failure_path = result.failure_path || []
+            failure_path.reverse_each.map do |result|
               relation_message(result.relation)
             end.compact.each.with_index(1) do |message, index|
               io.puts "#{"  " * (index)}#{message}"
             end
           end.string.chomp
+
+          unless lines.empty?
+            lines
+          end
+        end
+      end
+
+      module ResultPrinter2
+        def result_line(result)
+          case result
+          when Subtyping::Result::Failure
+            case result.error
+            when Subtyping::Result::Failure::UnknownPairError
+              nil
+            when Subtyping::Result::Failure::UnsatisfiedConstraints
+              "Unsatisfied constraints: #{result.relation}"
+            when Subtyping::Result::Failure::MethodMissingError
+              "Method `#{result.error.name}` is missing"
+            when Subtyping::Result::Failure::BlockMismatchError
+              "Incomaptible block: #{result.relation}"
+            when Subtyping::Result::Failure::ParameterMismatchError
+              if result.relation.params?
+                "Incompatible arity: #{result.relation.super_type} and #{result.relation.sub_type}"
+              else
+                "Incompatible arity: #{result.relation}"
+              end
+            when Subtyping::Result::Failure::PolyMethodSubtyping
+              "Unsupported polymorphic method comparison: #{result.relation}"
+            when Subtyping::Result::Failure::SelfBindingMismatch
+              "Incompatible block self type: #{result.relation}"
+            end
+          else
+            result.relation.to_s
+          end
+        end
+
+        def detail_lines
+          lines = StringIO.new.tap do |io|
+            failure_path = result.failure_path || []
+            failure_path.reverse_each.filter_map do |result|
+              result_line(result)
+            end.each.with_index(1) do |message, index|
+              io.puts "#{"  " * (index)}#{message}"
+            end
+          end.string.chomp
+
+          unless lines.empty?
+            lines
+          end
         end
       end
 
@@ -783,7 +833,7 @@ module Steep
         end
 
         def header_line
-          "Assertion cannot hold: no relationship between infered type (`#{node_type.to_s}`) and asserted type (`#{assertion_type.to_s}`)"
+          "Assertion cannot hold: no relationship between inferred type (`#{node_type.to_s}`) and asserted type (`#{assertion_type.to_s}`)"
         end
       end
 
@@ -832,6 +882,31 @@ module Steep
         end
       end
 
+      class IncompatibleArgumentForwarding < Base
+        attr_reader :method_name, :params_pair, :block_pair, :result
+
+        def initialize(method_name:, node:, params_pair: nil, block_pair: nil, result:)
+          super(node: node)
+          @method_name = method_name
+          @result = result
+          @params_pair = params_pair
+          @block_pair = block_pair
+        end
+
+        include ResultPrinter2
+
+        def header_line
+          case
+          when params_pair
+            "Cannot forward arguments to `#{method_name}`:"
+          when block_pair
+            "Cannot forward block to `#{method_name}`:"
+          else
+            raise
+          end
+        end
+      end
+
       ALL = ObjectSpace.each_object(Class).with_object([]) do |klass, array|
         if klass < Base
           array << klass

data/lib/steep/diagnostic/signature.rb

@@ -381,6 +381,42 @@ module Steep
         end
       end
 
+      class InconsistentClassModuleAliasError < Base
+        attr_reader :decl
+
+        def initialize(decl:)
+          @decl = decl
+          super(location: decl.location&.[](:old_name))
+        end
+
+        def header_line
+          expected_kind =
+            case decl
+            when RBS::AST::Declarations::ModuleAlias
+              "module"
+            when RBS::AST::Declarations::ClassAlias
+              "class"
+            else
+              raise
+            end
+
+          "A #{expected_kind} `#{decl.new_name}` cannot be an alias of `#{decl.old_name}`"
+        end
+      end
+
+      class CyclicClassAliasDefinitionError < Base
+        attr_reader :decl
+
+        def initialize(decl:)
+          @decl = decl
+          super(location: decl.location&.[](:new_name))
+        end
+
+        def header_line
+          "#{decl.new_name} is a cyclic definition"
+        end
+      end
+
       def self.from_rbs_error(error, factory:)
         case error
         when RBS::ParsingError
@@ -475,6 +511,10 @@ module Steep
           )
         when RBS::InheritModuleError
           Diagnostic::Signature::InheritModuleError.new(error.super_decl)
+        when RBS::InconsistentClassModuleAliasError
+          Diagnostic::Signature::InconsistentClassModuleAliasError.new(decl: error.alias_entry.decl)
+        when RBS::CyclicClassAliasDefinitionError
+          Diagnostic::Signature::CyclicClassAliasDefinitionError.new(decl: error.alias_entry.decl)
         else
           raise error
         end

data/lib/steep/drivers/check.rb

@@ -30,11 +30,11 @@ module Steep
         client_read, server_write = IO.pipe
         server_read, client_write = IO.pipe
 
-        client_reader = LanguageServer::Protocol::Transport::Io::Reader.new(client_read)
-        client_writer = LanguageServer::Protocol::Transport::Io::Writer.new(client_write)
+        client_reader = LSP::Transport::Io::Reader.new(client_read)
+        client_writer = LSP::Transport::Io::Writer.new(client_write)
 
-        server_reader = LanguageServer::Protocol::Transport::Io::Reader.new(server_read)
-        server_writer = LanguageServer::Protocol::Transport::Io::Writer.new(server_write)
+        server_reader = LSP::Transport::Io::Reader.new(server_read)
+        server_writer = LSP::Transport::Io::Writer.new(server_write)
 
         typecheck_workers = Server::WorkerProcess.start_typecheck_workers(
           steepfile: project.steepfile_path,

data/lib/steep/index/rbs_index.rb

@@ -18,7 +18,9 @@ module Steep
             declarations << decl
           when RBS::AST::Declarations::Interface
             declarations << decl
-          when RBS::AST::Declarations::Alias
+          when RBS::AST::Declarations::TypeAlias
+            declarations << decl
+          when RBS::AST::Declarations::ClassAlias, RBS::AST::Declarations::ModuleAlias
             declarations << decl
           else
             raise "Unexpected type declaration: #{decl}"
@@ -41,7 +43,9 @@ module Steep
             references << ref
           when RBS::AST::Declarations::Constant, RBS::AST::Declarations::Global
             references << ref
-          when RBS::AST::Declarations::Alias
+          when RBS::AST::Declarations::TypeAlias
+            references << ref
+          when RBS::AST::Declarations::ClassAlias, RBS::AST::Declarations::ModuleAlias
             references << ref
           else
             raise "Unexpected type reference: #{ref}"
@@ -312,6 +316,11 @@ module Steep
             end
           end
 
+          env.class_alias_decls.each do |name, entry|
+            index.add_type_declaration(name, entry.decl)
+            index.add_type_reference(entry.decl.old_name, entry.decl)
+          end
+
           env.interface_decls.each do |name, decl|
             index.add_type_declaration(name, decl.decl)
 
@@ -320,7 +329,7 @@ module Steep
             end
           end
 
-          env.alias_decls.each do |name, decl|
+          env.type_alias_decls.each do |name, decl|
             index.add_type_declaration(name, decl.decl)
             type_reference decl.decl.type, from: decl.decl
           end

data/lib/steep/index/signature_symbol_provider.rb

@@ -84,7 +84,7 @@ module Steep
                     kind: LSP::Constant::SymbolKind::INTERFACE,
                     container_name: container_name
                   )
-                when RBS::AST::Declarations::Alias
+                when RBS::AST::Declarations::TypeAlias
                   symbols << SymbolInformation.new(
                     name: name,
                     location: decl.location,