steep 1.4.0.dev.2 → 1.4.0.dev.3

data/guides/src/nil-optional/nil-optional.md

@@ -0,0 +1,195 @@
+# nil and Optional Types
+
+`nil`s have been the most common source of the errors you see when testing and after the deployment of your apps.
+
+```
+NoMethodError (undefined method `save!' for nil:NilClass)
+```
+
+Steep/RBS provides *optional types* to help you identify the problems while you are coding.
+
+## Preventing `nil` problems
+
+Technically, there is only one way to prevent `nil` problems – test everytime if the value is `nil` before calling methods with the receiver.
+
+```rb
+if account
+  account.save!
+end
+```
+
+Using the `if` statement is the most popular way to ensure the value is not `nil`. But you can do it with safe-navigation-operators, case-when (or case-in), and `#try` method.
+
+```rb
+account&.save!
+
+case account
+when Account
+  account.save!
+end
+
+account.try {|account| account.save! }
+```
+
+It's simple, but not easy to do in your code.
+
+**You may forget testing.** This happens easily. You don't notice that you forget until you deploy the code into production and it crashes after loading an account that is old and satisfies complicated conditions that leads it to be `nil`.
+
+**You may add redundant guards.** This won't get your app crash, but it will make understanding your code more difficult. It adds unnecessary noise that tells your teammates this can be `nil` in some place, and results in another redundant guard.
+
+The `nil` problems can be solved by a tool that tells you:
+
+* If the value can be `nil` or not, and
+* You forget testing the value before using the value
+
+RBS has a language construct to do this called optional types and Steep implements analysis to let you know if you forget testing the value.
+
+## Optional types
+
+Optional types in RBS are denoted with a suffix `?` – Type?. It means the value of the type may be `nil`.
+
+```
+Integer?            # Integer or nil
+Array[Account]?     # An array of Account or nil
+```
+
+Note that optional types can be included in other types as:
+
+```
+Array[Account?]
+```
+
+The value of the type above is always an array, but the element may be `nil`.
+
+In other words, a non optional type in RBS means the value cannot be `nil`.
+
+```
+Integer            # Integer, cannot be nil
+Array[Account]     # An array, cannot be nil
+```
+
+Let's see how Steep reports errors on optional and non-optional types.
+
+```rb
+account = Account.find(1)
+account.save!
+```
+
+Assume the type of `account` is `Account` (non optional type), the code type checks successfully. There is no chance to be `nil` here. The `save!` method call never results in a `NoMethodError`.
+
+```rb
+account = Account.find_by(email: "soutaro@squareup.com")
+account.save!
+```
+
+Steep reports a `NoMethod` error on the `save!` call. Because the value of the `account` may be `nil`, depending on the actual records in the `accounts` table. You cannot call the `save!` method without checking if the `account` is `nil`.
+
+You cannot assign `nil` to a local variable with non-optional types.
+
+```rb
+# @type var account: Account
+
+account = nil
+account = Account.find_by(email: "soutaro@squareup.com")
+```
+
+Because the type of `account` is declared Account, non-optional type, it cannot be `nil`. And Steep detects a type error if you try to assign `nil`. Same for assigning an optional type value at the last line.
+
+# Unwrapping optional types
+
+There are several ways to unwrap optional types. The most common one is using if.
+
+```rb
+account = Account.find_by(id: 1)
+if account
+  account.save!
+end
+```
+
+The *if* statement tests if `account` is `nil`. Inside the then clause, `account` cannot be `nil`. Then Steep type checks the code.
+
+This works for *else* clause of *unless*.
+
+```rb
+account = Account.find_by(id: 1)
+unless account
+  # Do something
+else
+  account.save!
+end
+```
+
+This also type checks successfully.
+
+Steep supports `nil?` predicate too.
+
+```rb
+unless (account = Account.find_by(id: 1)).nil?
+  account.save!
+end
+```
+
+This assumes the `Account` class doesn't have a custom `nil?` method, but keeps the built-in `nil?` or equivalent.
+
+The last one is using safe-nevigation-navigator. It checks if the receiver is `nil` and calls the method if it is not. Otherwise just evaluates to `nil`.
+
+```rb
+account = Account.find_by(id: 1)
+account&.save!
+```
+
+This is a shorthand for the case you don't do any error handling case if it is `nil`.
+
+## What should I do for the case of `nil`?
+
+There is no universal answer for this question. You may just stop the execution of the method by returning. You may want to insert a new account to ensure the record exists. Raising an exception with a detailed error message will help troubleshooting.
+
+It depends on what the program is expected to do. Steep just checks if accessing `nil` may happen or not. The developers only know how to handle the `nil` cases.
+
+# Handling unwanted `nil`s
+
+When you start using Steep, you may see many unwanted `nil`s. This typically happens when you want to use Array methods, like `first` or `sample`.
+
+```rb
+account = accounts.first
+account.save!
+```
+
+It returns `nil` if the array is empty. Steep cannot detect if the array is empty or not, and it conservatively assumes the return value of the methods may be `nil`. While you know the `account` array is not empty, Steep infer the `first` method may return `nil`.
+
+This is one of the most frequently seen sources of unwanted `nil`s.
+
+## Raising an error
+
+In this case, you have to add an extra code to let Steep unwrap it.
+
+```rb
+account = accounts.first or raise
+account.save!
+```
+
+My recommendation is to raise an exception, `|| raise` or `or raise`. It raises an exception in the case of `nil`, and Steep unwraps the type of the `account` variable.
+
+Exceptions are better than other control flow operators – `return`/`break`/`next`. It doesn't affect the control flow until it actually happens during execution, and the type checking result other than the unwrapping is changed.
+
+An `#raise` call without argument is my favorite. It's short. It's uncommon in the Ruby code and it can tell the readers that something unexpected is happening.
+
+But of course, you can add some message:
+
+```rb
+account = accounts.first or raise("accounts cannot be empty")
+account.save!
+```
+
+## Type assertions
+
+You can also use a type assertion, that is introduced in Steep 1.3.
+
+```rb
+account = accounts.first #: Account
+account.save!
+```
+
+It tells Steep that the right hand side of the assignment is `Account`. That overwrites the type checking algorithm, and the developer is responsible for making sure the value cannot be `nil`.
+
+Note: Nothing happens during the execution. It just works for Steep and Ruby doesn't do any extra type checking on it. I recommend using the `or raise` idiom for most of the cases.

data/lib/steep/diagnostic/ruby.rb

@@ -37,22 +37,72 @@ module Steep
           when relation.interface?
             nil
           when relation.block?
-            nil
+            "(Blocks are incompatible)"
           when relation.function?
             nil
           when relation.params?
-            nil
+            "(Params are incompatible)"
           end
         end
 
         def detail_lines
-          StringIO.new.tap do |io|
-            result.failure_path&.reverse_each.map do |result|
+          lines = StringIO.new.tap do |io|
+            failure_path = result.failure_path || []
+            failure_path.reverse_each.map do |result|
               relation_message(result.relation)
             end.compact.each.with_index(1) do |message, index|
               io.puts "#{"  " * (index)}#{message}"
             end
           end.string.chomp
+
+          unless lines.empty?
+            lines
+          end
+        end
+      end
+
+      module ResultPrinter2
+        def result_line(result)
+          case result
+          when Subtyping::Result::Failure
+            case result.error
+            when Subtyping::Result::Failure::UnknownPairError
+              nil
+            when Subtyping::Result::Failure::UnsatisfiedConstraints
+              "Unsatisfied constraints: #{result.relation}"
+            when Subtyping::Result::Failure::MethodMissingError
+              "Method `#{result.error.name}` is missing"
+            when Subtyping::Result::Failure::BlockMismatchError
+              "Incomaptible block: #{result.relation}"
+            when Subtyping::Result::Failure::ParameterMismatchError
+              if result.relation.params?
+                "Incompatible arity: #{result.relation.super_type} and #{result.relation.sub_type}"
+              else
+                "Incompatible arity: #{result.relation}"
+              end
+            when Subtyping::Result::Failure::PolyMethodSubtyping
+              "Unsupported polymorphic method comparison: #{result.relation}"
+            when Subtyping::Result::Failure::SelfBindingMismatch
+              "Incompatible block self type: #{result.relation}"
+            end
+          else
+            result.relation.to_s
+          end
+        end
+
+        def detail_lines
+          lines = StringIO.new.tap do |io|
+            failure_path = result.failure_path || []
+            failure_path.reverse_each.filter_map do |result|
+              result_line(result)
+            end.each.with_index(1) do |message, index|
+              io.puts "#{"  " * (index)}#{message}"
+            end
+          end.string.chomp
+
+          unless lines.empty?
+            lines
+          end
         end
       end
 
@@ -832,6 +882,31 @@ module Steep
         end
       end
 
+      class IncompatibleArgumentForwarding < Base
+        attr_reader :method_name, :params_pair, :block_pair, :result
+
+        def initialize(method_name:, node:, params_pair: nil, block_pair: nil, result:)
+          super(node: node)
+          @method_name = method_name
+          @result = result
+          @params_pair = params_pair
+          @block_pair = block_pair
+        end
+
+        include ResultPrinter2
+
+        def header_line
+          case
+          when params_pair
+            "Cannot forward arguments to `#{method_name}`:"
+          when block_pair
+            "Cannot forward block to `#{method_name}`:"
+          else
+            raise
+          end
+        end
+      end
+
       ALL = ObjectSpace.each_object(Class).with_object([]) do |klass, array|
         if klass < Base
           array << klass

data/lib/steep/drivers/check.rb

@@ -30,11 +30,11 @@ module Steep
         client_read, server_write = IO.pipe
         server_read, client_write = IO.pipe
 
-        client_reader = LanguageServer::Protocol::Transport::Io::Reader.new(client_read)
-        client_writer = LanguageServer::Protocol::Transport::Io::Writer.new(client_write)
+        client_reader = LSP::Transport::Io::Reader.new(client_read)
+        client_writer = LSP::Transport::Io::Writer.new(client_write)
 
-        server_reader = LanguageServer::Protocol::Transport::Io::Reader.new(server_read)
-        server_writer = LanguageServer::Protocol::Transport::Io::Writer.new(server_write)
+        server_reader = LSP::Transport::Io::Reader.new(server_read)
+        server_writer = LSP::Transport::Io::Writer.new(server_write)
 
         typecheck_workers = Server::WorkerProcess.start_typecheck_workers(
           steepfile: project.steepfile_path,

data/lib/steep/interface/block.rb

@@ -69,6 +69,16 @@ module Steep
         )
       end
 
+      def to_proc_type
+        proc = AST::Types::Proc.new(type: type, self_type: self_type, block: nil)
+
+        if optional?
+          AST::Types::Union.build(types: [proc, AST::Builtin.nil_type])
+        else
+          proc
+        end
+      end
+
       def +(other)
         optional = self.optional? || other.optional?
         type = Function.new(

data/lib/steep/module_helper.rb

@@ -1,21 +1,23 @@
 module Steep
   module ModuleHelper
     def module_name_from_node(parent_node, constant_name)
-      namespace = namespace_from_node(parent_node) or return
-      name = constant_name
-      RBS::TypeName.new(name: name, namespace: namespace)
+      if namespace = namespace_from_node(parent_node)
+        RBS::TypeName.new(name: constant_name, namespace: namespace)
+      end
     end
 
     def namespace_from_node(node)
-      case node&.type
-      when nil
-        RBS::Namespace.empty
-      when :cbase
-        RBS::Namespace.root
-      when :const
-        namespace_from_node(node.children[0])&.yield_self do |parent|
-          parent.append(node.children[1])
+      if node
+        case node.type
+        when :cbase
+          RBS::Namespace.root
+        when :const
+          if parent = namespace_from_node(node.children[0])
+            parent.append(node.children[1])
+          end
         end
+      else
+        RBS::Namespace.empty
       end
     end
   end

data/lib/steep/path_helper.rb

@@ -11,6 +11,10 @@ module Steep
       end
     end
 
+    def to_pathname!(uri, dosish: Gem.win_platform?)
+      to_pathname(uri, dosish: dosish) or raise "Cannot translate a URI to pathname: #{uri}"
+    end
+
     def to_uri(path, dosish: Gem.win_platform?)
       str_path = path.to_s
       if dosish

data/lib/steep/server/interaction_worker.rb

@@ -3,9 +3,9 @@ module Steep
     class InteractionWorker < BaseWorker
       include ChangeBuffer
 
-      ApplyChangeJob = Class.new()
-      HoverJob = Struct.new(:id, :path, :line, :column, keyword_init: true)
-      CompletionJob = Struct.new(:id, :path, :line, :column, :trigger, keyword_init: true)
+      ApplyChangeJob = _ = Class.new()
+      HoverJob = _ = Struct.new(:id, :path, :line, :column, keyword_init: true)
+      CompletionJob = _ = Struct.new(:id, :path, :line, :column, :trigger, keyword_init: true)
 
       LSP = LanguageServer::Protocol
 
@@ -53,7 +53,7 @@ module Steep
         when "textDocument/hover"
           id = request[:id]
 
-          path = project.relative_path(Steep::PathHelper.to_pathname(request[:params][:textDocument][:uri]))
+          path = project.relative_path(Steep::PathHelper.to_pathname!(request[:params][:textDocument][:uri]))
           line = request[:params][:position][:line]+1
           column = request[:params][:position][:character]
 
@@ -63,7 +63,8 @@ module Steep
           id = request[:id]
 
           params = request[:params]
-          path = project.relative_path(Steep::PathHelper.to_pathname(params[:textDocument][:uri]))
+
+          path = project.relative_path(Steep::PathHelper.to_pathname!(params[:textDocument][:uri]))
           line, column = params[:position].yield_self {|hash| [hash[:line]+1, hash[:character]] }
           trigger = params.dig(:context, :triggerCharacter)
 
@@ -104,6 +105,7 @@ module Steep
         Steep.logger.tagged("#response_to_completion") do
           Steep.measure "Generating response" do
             Steep.logger.info "path: #{job.path}, line: #{job.line}, column: #{job.column}, trigger: #{job.trigger}"
+
             case
             when target = project.target_for_source_path(job.path)
               file = service.source_files[job.path] or return
@@ -127,56 +129,78 @@ module Steep
                 items: completion_items
               )
             when (targets = project.targets_for_path(job.path)).is_a?(Array)
-              target = targets[0] or return
-              sig_service = service.signature_services[target.name] #: Services::SignatureService
+              target = targets[0] or raise
+              sig_service = service.signature_services[target.name] or raise
               relative_path = job.path
-              buffer = RBS::Buffer.new(name: relative_path, content: sig_service.files[relative_path].content)
-              pos = buffer.loc_to_pos([job.line, job.column])
-              prefix = buffer.content[0...pos].reverse[/\A[\w\d]*/].reverse
+
+              context = nil #: RBS::Resolver::context
 
               case sig_service.status
               when Steep::Services::SignatureService::SyntaxErrorStatus, Steep::Services::SignatureService::AncestorErrorStatus
-                return
-              end
 
-              sig = sig_service.files[relative_path].signature
-              sig.is_a?(Array) or raise
-              decls = sig[2]
-              locator = RBS::Locator.new(buffer: sig[0], dirs: sig[1], decls: decls)
-
-              (_hd, tail = locator.find2(line: job.line, column: job.column)) or return []
-
-              namespace = []
-              tail.each do |t|
-                case t
-                when RBS::AST::Declarations::Module, RBS::AST::Declarations::Class
-                  namespace << t.name.to_namespace
+                if buffer = sig_service.latest_env.buffers.find {|buf| Pathname(buf.name) == Pathname(relative_path) }
+                  dirs = sig_service.latest_env.signatures[buffer][0]
+                else
+                  dirs = [] #: Array[RBS::AST::Directives::t]
+                end
+              else
+                signature = sig_service.files[relative_path].signature
+                signature.is_a?(Array) or raise
+                buffer, dirs, decls = signature
+
+                locator = RBS::Locator.new(buffer: buffer, dirs: dirs, decls: decls)
+
+                _hd, tail = locator.find2(line: job.line, column: job.column)
+                tail ||= []
+
+                tail.reverse_each do |t|
+                  case t
+                  when RBS::AST::Declarations::Module, RBS::AST::Declarations::Class
+                    if (last_type_name = context&.[](1)).is_a?(RBS::TypeName)
+                      context = [context, last_type_name + t.name]
+                    else
+                      context = [context, t.name.absolute!]
+                    end
+                  end
                 end
               end
-              context = []
 
-              namespace.each do |ns|
-                context.map! { |n| ns + n }
-                context << ns
+              buffer = RBS::Buffer.new(name: relative_path, content: sig_service.files[relative_path].content)
+              prefix = Services::TypeNameCompletion::Prefix.parse(buffer, line: job.line, column: job.column)
+
+              completion = Services::TypeNameCompletion.new(env: sig_service.latest_env, context: context, dirs: dirs)
+              type_names = completion.find_type_names(prefix)
+              prefix_size = prefix ? prefix.size : 0
+
+              completion_items = type_names.map {|type_name|
+                absolute_name, relative_name = completion.resolve_name_in_context(type_name)
+
+                format_completion_item_for_rbs(sig_service, absolute_name, job, relative_name.to_s, prefix_size)
+              }
+
+              ["untyped", "void", "bool", "class", "module", "instance", "nil"].each do |name|
+                completion_items << LanguageServer::Protocol::Interface::CompletionItem.new(
+                  label: name,
+                  detail: "(builtin type)",
+                  text_edit: LanguageServer::Protocol::Interface::TextEdit.new(
+                    range: LanguageServer::Protocol::Interface::Range.new(
+                      start: LanguageServer::Protocol::Interface::Position.new(
+                        line: job.line - 1,
+                        character: job.column - prefix_size
+                      ),
+                      end: LanguageServer::Protocol::Interface::Position.new(
+                        line: job.line - 1,
+                        character: job.column
+                      )
+                    ),
+                    new_text: name
+                  ),
+                  kind: LSP::Constant::CompletionItemKind::KEYWORD,
+                  filter_text: name,
+                  sort_text: "zz__#{name}"
+                )
               end
 
-              context.map!(&:absolute!)
-
-              class_names = sig_service.latest_env.class_decls.keys + sig_service.latest_env.class_alias_decls.keys
-              class_items = class_names.map { |type_name|
-                format_completion_item_for_rbs(sig_service, type_name, context, job, prefix)
-              }.compact
-
-              alias_items = sig_service.latest_env.type_alias_decls.keys.map { |type_name|
-                format_completion_item_for_rbs(sig_service, type_name, context, job, prefix)
-              }.compact
-
-              interface_items = sig_service.latest_env.interface_decls.keys.map {|type_name|
-                format_completion_item_for_rbs(sig_service, type_name, context, job, prefix)
-              }.compact
-
-              completion_items = class_items + alias_items + interface_items
-
               LSP::Interface::CompletionList.new(
                 is_incomplete: false,
                 items: completion_items
@@ -186,11 +210,11 @@ module Steep
         end
       end
 
-      def format_completion_item_for_rbs(sig_service, type_name, context, job, prefix)
+      def format_completion_item_for_rbs(sig_service, type_name, job, complete_text, prefix_size)
         range = LanguageServer::Protocol::Interface::Range.new(
           start: LanguageServer::Protocol::Interface::Position.new(
             line: job.line - 1,
-            character: job.column - prefix.size
+            character: job.column - prefix_size
           ),
           end: LanguageServer::Protocol::Interface::Position.new(
             line: job.line - 1,
@@ -198,64 +222,64 @@ module Steep
           )
         )
 
-        name = relative_name_in_context(type_name, context).to_s
-
-        return unless name.start_with?(prefix)
-
         case type_name.kind
         when :class
           env = sig_service.latest_env #: RBS::Environment
           class_entry = env.module_class_entry(type_name) or raise
 
-          case class_entry
-          when RBS::Environment::ClassEntry, RBS::Environment::ModuleEntry
-            LanguageServer::Protocol::Interface::CompletionItem.new(
-              label: "#{name}",
-              documentation:  format_comment(class_entry.primary.decl.comment),
-              text_edit: LanguageServer::Protocol::Interface::TextEdit.new(
-                range: range,
-                new_text: name
-              ),
-              kind: LSP::Constant::CompletionItemKind::CLASS,
-              insert_text_format: LSP::Constant::InsertTextFormat::SNIPPET
-            )
-          when RBS::Environment::ClassAliasEntry, RBS::Environment::ModuleAliasEntry
-            LanguageServer::Protocol::Interface::CompletionItem.new(
-              label: "#{name}",
-              documentation:  format_comment(class_entry.decl.comment),
-              text_edit: LanguageServer::Protocol::Interface::TextEdit.new(
-                range: range,
-                new_text: name
-              ),
-              kind: LSP::Constant::CompletionItemKind::CLASS,
-              insert_text_format: LSP::Constant::InsertTextFormat::SNIPPET
-            )
-          end
+          comment =
+            case class_entry
+            when RBS::Environment::ClassEntry, RBS::Environment::ModuleEntry
+              class_entry.decls.flat_map {|decl| [decl.decl.comment] }.first
+            when RBS::Environment::ClassAliasEntry, RBS::Environment::ModuleAliasEntry
+              class_entry.decl.comment
+            end
+
+          LanguageServer::Protocol::Interface::CompletionItem.new(
+            label: complete_text,
+            detail: type_name.to_s,
+            documentation:  format_comment(comment),
+            text_edit: LanguageServer::Protocol::Interface::TextEdit.new(
+              range: range,
+              new_text: complete_text
+            ),
+            kind: LSP::Constant::CompletionItemKind::CLASS,
+            insert_text_format: LSP::Constant::InsertTextFormat::SNIPPET,
+            sort_text: complete_text,
+            filter_text: complete_text
+          )
         when :alias
           alias_decl = sig_service.latest_env.type_alias_decls[type_name]&.decl or raise
+
           LanguageServer::Protocol::Interface::CompletionItem.new(
-            label: "#{name}",
+            label: complete_text,
+            detail: type_name.to_s,
             text_edit: LanguageServer::Protocol::Interface::TextEdit.new(
               range: range,
-              new_text: name
+              new_text: complete_text
             ),
             documentation: format_comment(alias_decl.comment),
             # https://github.com/microsoft/vscode-languageserver-node/blob/6d78fc4d25719b231aba64a721a606f58b9e0a5f/client/src/common/client.ts#L624-L650
             kind: LSP::Constant::CompletionItemKind::FIELD,
-            insert_text_format: LSP::Constant::InsertTextFormat::SNIPPET
+            insert_text_format: LSP::Constant::InsertTextFormat::SNIPPET,
+            sort_text: complete_text,
+            filter_text: complete_text
           )
         when :interface
           interface_decl = sig_service.latest_env.interface_decls[type_name]&.decl or raise
 
           LanguageServer::Protocol::Interface::CompletionItem.new(
-            label: "#{name}",
+            label: complete_text,
+            detail: type_name.to_s,
             text_edit: LanguageServer::Protocol::Interface::TextEdit.new(
               range: range,
-              new_text: name
+              new_text: complete_text
             ),
             documentation: format_comment(interface_decl.comment),
             kind: LanguageServer::Protocol::Constant::CompletionItemKind::INTERFACE,
-            insert_text_format: LanguageServer::Protocol::Constant::InsertTextFormat::SNIPPET
+            insert_text_format: LanguageServer::Protocol::Constant::InsertTextFormat::SNIPPET,
+            sort_text: complete_text,
+            filter_text: complete_text
           )
         end
       end
@@ -417,17 +441,6 @@ module Steep
 
         params.join(", ")
       end
-
-      def relative_name_in_context(type_name, context)
-        context.each do |namespace|
-          if (type_name.to_s == namespace.to_type_name.to_s || type_name.namespace.to_s == "::")
-            return RBS::TypeName.new(namespace: RBS::Namespace.empty, name: type_name.name)
-          elsif type_name.to_s.start_with?(namespace.to_s)
-            return TypeName(type_name.to_s.sub(namespace.to_type_name.to_s, '')).relative!
-          end
-        end
-        type_name
-      end
     end
   end
 end