rbs 3.4.3 → 3.5.0.pre.1

data/docs/architecture.md

@@ -0,0 +1,110 @@
+# Architecture
+
+This guide describes the outline of the architecture of RBS library. It helps you to understand the structure and key features of the library to start contributing to the library.
+
+## Bird's Eye View
+
+The goal of the library is simple: Read RBS files and generate the structure of Ruby programs.
+
+```
+RBS files
+  ↓         -- RBS::Parser
+Syntax tree
+  ↓
+Environment
+  ↓        -- Definition builder
+Definition
+```
+
+The input is RBS files. The gem ships with RBS type definitions of Ruby core library and some of the standard libraries. You write RBS files for your applications or gems.
+
+Syntax tree is the next representation. `RBS::Parser` transforms the sequence of characters in RBS files into syntax trees.
+
+Syntax tree objects are loaded to `RBS::Environment`. It collects loaded RBS objects, organizes the definitions, and provides some utilities, like resolving type names and finding the declarations.
+
+`RBS::Definition` is the goal of the transformation steps. It is associated with a class singleton, a class object, or an interface. You can find the list of available methods and their types, instance variables, and class hierarchies.
+
+## Core classes
+
+### Types
+
+Types are defined under `RBS::Types`, like `RBS::Types::ClassInstance` or `RBS::Types::Union`. You will find the definition of each type supported in RBS.
+
+### Parsing RBS files
+
+The RBS source code is loaded into `RBS::Buffer`, and `RBS::Parser` is the parser. The parser is implemented in C extension.
+
+`RBS::Parser` provides three entrypoints.
+
+- `RBS::Parser.parse_method_type` parsers a *method type*. (`[T] (String) { (IO) -> T } -> Array[T]`)
+- `RBS::Parser.parse_type` parses a *type*. (`Hash[Symbol, untyped]`)
+- `RBS::Parser.parse_signature` parses the whole RBS file.
+
+### Environment
+
+RBS AST is loaded to `RBS::Environment` by `RBS::EnvironmentLoader`. `Environment` gives *absolute names* to the declarations, and provides an index from the *absolute name* to their declarations.
+
+Assume we have the following nested RBS declarations:
+
+```rbs
+module Hello
+  class World
+  end
+end
+
+class Hello::World
+end
+```
+
+And the environment organizes the definitions as follows:
+
+- There are two classes `::Hello` and `::Hello::World`
+- It provides a mapping from `::Hello` to it's `module` declaration and `::Hello::World` to it's two `class` declarations
+
+### Definition and DefinitionBuilder
+
+`RBS::Definition` tells you:
+
+- The set of available methods in a class/module/interface
+- The set of instance variables in a class/module
+- The ancestors in a class/module
+
+Definition is constructed for:
+
+- A singleton class of a class/module -- `singleton(String)`, `singleton(Array)`,
+- An instance of a class -- `String`, `Array[T]`, or
+- An interface -- `_ToS`
+
+Note that generic class instances/interfaces are kept generic. We don't have a definition of `Array[String]` but of `Array[T]`.
+
+`DefinitionBuilder` constructs `Definition` of given type names.
+
+- `DefinitionBuilder#build_singleton` returns a definition of singleton classes of given class/module.
+- `DefinitionBuilder#build_instance` returns a definition of instances of given class/module.
+- `DefinitionBuilder#build_interface` returns a definition of interfaces.
+
+It uses `AncestorBuilder` to construct ancestor chains of the type. `MethodBuilder` constructs sets of available methods based on the ancestor chains.
+
+The `#build_singleton` calculates the type of `.new` methods based on the definition of `#initialize` method. This is different from Ruby's implementation -- it reused `Class#new` method but we need the custom implementation to give precise `.new` method type of each class.
+
+#### Working with type aliases
+
+`DefinitionBuilder#expand_alias` and its variants provide one step *unfold* operation of type aliases.
+
+```ruby
+builder.expand_alias2(TypeName("::int"), []) # => returns `::Integer | ::_ToInt`
+```
+
+We don't have *normalize* operation for type aliases, because RBS allows recursive type alias definition, which cannot be *fully* unfolded.
+
+### Other utilities
+
+`RBS::Validator` provides validation of RBS type declaration. It validates that all of the type name references can be resolved, all type applications have correct arity, and so on.
+
+`RBS::Test` provides runtime type checking, which confirms if a Ruby object can have an RBS type. It also provides an integration to existing Ruby code so that we run Ruby code, assuming unit tests, with runtime type checking.
+
+`RBS::UnitTest` provides utilities to help write unit tests for RBS type definitions. Use the tool to make sure your RBS type definition is consistent with implementation.
+
+`RBS::Prototype` is the core of `rbs prototype` feature. It scans Ruby source code or uses reflection features, and it generates the prototype of RBS files.
+
+`RBS::Collection` includes `rbs collection` features.

data/docs/syntax.md

@@ -45,6 +45,7 @@ _literal_ ::= _string-literal_
             | `false`
 
 _proc_ ::= `^` _parameters?_ _self-type-binding?_ _block?_ `->` _type_
+         | `^` `(` `?` `)` `->` _type_                                   # Proc type with untyped parameter
 ```
 
 ### Class instance type
@@ -310,6 +311,7 @@ end
 
 ```markdown
 _method-type_ ::= _parameters?_ _block?_ `->` _type_                # Method type
+                | `(` `?` `)` `->` _type_                           # Method type with untyped parameters
 
 _parameters?_ ::=                   (Empty)
                 | _parameters_      (Parameters)
@@ -333,9 +335,11 @@ _var-name_ ::= /[a-z]\w*/
 _self-type-binding?_ =                              (Empty)
                      | `[` `self` `:` _type_ `]`    (Self type binding)
 
-_block?_ =                                                           (No block)
+_block?_ =                                                            (No block)
          | `{` _parameters_ _self-type-binding?_ `->` _type_ `}`      (Block)
+         | `{` `(` `?` `)` `->` _type_ `}`                            (Block with untyped parameters)
          | `?` `{` _parameters_ _self-type-binding?_ `->` _type_ `}`  (Optional block)
+         | `?` `{` `(` `?` `)` `->` _type_ `}`                        (Optional block with untyped parameters)
 ```
 
 ### Parameters

data/ext/rbs_extension/constants.c

@@ -61,6 +61,7 @@ VALUE RBS_Types_ClassInstance;
 VALUE RBS_Types_ClassSingleton;
 VALUE RBS_Types_Function_Param;
 VALUE RBS_Types_Function;
+VALUE RBS_Types_UntypedFunction;
 VALUE RBS_Types_Interface;
 VALUE RBS_Types_Intersection;
 VALUE RBS_Types_Literal;
@@ -138,6 +139,7 @@ void rbs__init_constants(void) {
   IMPORT_CONSTANT(RBS_Types_ClassInstance, RBS_Types, "ClassInstance");
   IMPORT_CONSTANT(RBS_Types_ClassSingleton, RBS_Types, "ClassSingleton");
   IMPORT_CONSTANT(RBS_Types_Function, RBS_Types, "Function");
+  IMPORT_CONSTANT(RBS_Types_UntypedFunction, RBS_Types, "UntypedFunction");
   IMPORT_CONSTANT(RBS_Types_Function_Param, RBS_Types_Function, "Param");
   IMPORT_CONSTANT(RBS_Types_Interface, RBS_Types, "Interface");
   IMPORT_CONSTANT(RBS_Types_Intersection, RBS_Types, "Intersection");

data/ext/rbs_extension/constants.h

@@ -64,6 +64,7 @@ extern VALUE RBS_Types_ClassInstance;
 extern VALUE RBS_Types_ClassSingleton;
 extern VALUE RBS_Types_Function_Param;
 extern VALUE RBS_Types_Function;
+extern VALUE RBS_Types_UntypedFunction;
 extern VALUE RBS_Types_Interface;
 extern VALUE RBS_Types_Intersection;
 extern VALUE RBS_Types_Literal;

data/ext/rbs_extension/location.c

@@ -1,83 +1,76 @@
 #include "rbs_extension.h"
 
+#define RBS_LOC_REQUIRED_P(loc, i) ((loc)->children->required_p & (1 << (i)))
+#define RBS_LOC_OPTIONAL_P(loc, i) (!RBS_LOC_REQUIRED_P((loc), (i)))
+
 VALUE RBS_Location;
 
-rbs_loc_list *rbs_loc_list_add(rbs_loc_list *list, const ID name, const range r) {
-  rbs_loc_list *new = malloc(sizeof(rbs_loc_list));
-  new->next = list;
-  new->name = name;
-  new->rg = r;
-  return new;
+position rbs_loc_position(int char_pos) {
+  position pos = { 0, char_pos, -1, -1 };
+  return pos;
 }
 
-rbs_loc_list *rbs_loc_list_dup(rbs_loc_list *list) {
-  if (list) {
-    return rbs_loc_list_add(rbs_loc_list_dup(list->next), list->name, list->rg);
-  } else {
-    return NULL;
-  }
+position rbs_loc_position3(int char_pos, int line, int column) {
+  position pos = { 0, char_pos, line, column };
+  return pos;
 }
 
-void rbs_loc_list_free(rbs_loc_list *list) {
-  while (list) {
-    rbs_loc_list *next = list->next;
-    free(list);
-    list = next;
+static void check_children_max(unsigned short n) {
+  size_t max = sizeof(rbs_loc_entry_bitmap) * 8;
+  if (n > max) {
+    rb_raise(rb_eRuntimeError, "Too many children added to location: %d", n);
   }
 }
 
-bool rbs_loc_list_find(const rbs_loc_list *list, ID name, range *rg) {
-  while (list) {
-    if (list->name == name) {
-      *rg = list->rg;
-      return true;
-    }
+void rbs_loc_alloc_children(rbs_loc *loc, unsigned short cap) {
+  check_children_max(cap);
 
-    list = list->next;
-  }
+  size_t s = sizeof(rbs_loc_children) + sizeof(rbs_loc_entry) * cap;
+  loc->children = malloc(s);
 
-  return false;
+  loc->children->len = 0;
+  loc->children->required_p = 0;
+  loc->children->cap = cap;
 }
 
-size_t rbs_loc_list_size(const rbs_loc_list *list) {
-  size_t size = 0;
-
-  while (list) {
-    size += 1;
-    list = list->next;
+static void check_children_cap(rbs_loc *loc) {
+  if (loc->children == NULL) {
+    rbs_loc_alloc_children(loc, 1);
+  } else {
+    if (loc->children->len == loc->children->cap) {
+      check_children_max(loc->children->cap + 1);
+      size_t s = sizeof(rbs_loc_children) + sizeof(rbs_loc_entry) * (++loc->children->cap);
+      loc->children = realloc(loc->children, s);
+    }
   }
-
-  return size;
 }
 
-position rbs_loc_position(int char_pos) {
-  position pos = { 0, char_pos, -1, -1 };
-  return pos;
-}
+void rbs_loc_add_required_child(rbs_loc *loc, ID name, range r) {
+  check_children_cap(loc);
 
-position rbs_loc_position3(int char_pos, int line, int column) {
-  position pos = { 0, char_pos, line, column };
-  return pos;
-}
+  unsigned short i = loc->children->len++;
+  loc->children->entries[i].name = name;
+  loc->children->entries[i].rg = r;
 
-void rbs_loc_add_required_child(rbs_loc *loc, ID name, range r) {
-  loc->requireds = rbs_loc_list_add(loc->requireds, name, r);
+  loc->children->required_p |= 1 << i;
 }
 
 void rbs_loc_add_optional_child(rbs_loc *loc, ID name, range r) {
-  loc->optionals = rbs_loc_list_add(loc->optionals, name, r);
+  check_children_cap(loc);
+
+  unsigned short i = loc->children->len++;
+  loc->children->entries[i].name = name;
+  loc->children->entries[i].rg = r;
 }
 
 void rbs_loc_init(rbs_loc *loc, VALUE buffer, range rg) {
   loc->buffer = buffer;
   loc->rg = rg;
-  loc->optionals = NULL;
-  loc->requireds = NULL;
+  loc->children = NULL;
 }
 
 void rbs_loc_free(rbs_loc *loc) {
-  rbs_loc_list_free(loc->optionals);
-  rbs_loc_list_free(loc->requireds);
+  free(loc->children);
   ruby_xfree(loc);
 }
 
@@ -89,7 +82,11 @@ static void rbs_loc_mark(void *ptr)
 
 static size_t rbs_loc_memsize(const void *ptr) {
   const rbs_loc *loc = ptr;
-  return sizeof(*loc) + (rbs_loc_list_size(loc->optionals) + rbs_loc_list_size(loc->requireds)) * sizeof(rbs_loc_list);
+  if (loc->children == NULL) {
+    return sizeof(rbs_loc);
+  } else {
+    return sizeof(rbs_loc) + sizeof(rbs_loc_children) + sizeof(rbs_loc_entry) * loc->children->cap;
+  }
 }
 
 static rb_data_type_t location_type = {
@@ -130,8 +127,10 @@ static VALUE location_initialize_copy(VALUE self, VALUE other) {
 
   self_loc->buffer = other_loc->buffer;
   self_loc->rg = other_loc->rg;
-  self_loc->requireds = rbs_loc_list_dup(other_loc->requireds);
-  self_loc->optionals = rbs_loc_list_dup(other_loc->optionals);
+  if (other_loc->children != NULL) {
+    rbs_loc_alloc_children(self_loc, other_loc->children->cap);
+    memcpy(self_loc->children, other_loc->children, sizeof(rbs_loc_children) + sizeof(rbs_loc_entry) * other_loc->children->cap);
+  }
 
   return Qnil;
 }
@@ -221,18 +220,19 @@ VALUE rbs_new_location(VALUE buffer, range rg) {
 static VALUE location_aref(VALUE self, VALUE name) {
   rbs_loc *loc = rbs_check_location(self);
 
-  range result;
   ID id = SYM2ID(name);
 
-  if (rbs_loc_list_find(loc->requireds, id, &result)) {
-    return rbs_new_location(loc->buffer, result);
-  }
-
-  if (rbs_loc_list_find(loc->optionals, id, &result)) {
-    if (null_range_p(result)) {
-      return Qnil;
-    } else {
-      return rbs_new_location(loc->buffer, result);
+  if (loc->children != NULL) {
+    for (unsigned short i = 0; i < loc->children->len; i++) {
+      if (loc->children->entries[i].name == id) {
+        range result = loc->children->entries[i].rg;
+
+        if (RBS_LOC_OPTIONAL_P(loc, i) && null_range_p(result)) {
+          return Qnil;
+        } else {
+          return rbs_new_location(loc->buffer, result);
+        }
+      }
     }
   }
 
@@ -244,11 +244,16 @@ static VALUE location_optional_keys(VALUE self) {
   VALUE keys = rb_ary_new();
 
   rbs_loc *loc = rbs_check_location(self);
-  rbs_loc_list *list = loc->optionals;
+  rbs_loc_children *children = loc->children;
+  if (children == NULL) {
+    return keys;
+  }
 
-  while (list) {
-    rb_ary_push(keys, ID2SYM(list->name));
-    list = list->next;
+  for (unsigned short i = 0; i < children->len; i++) {
+    if (RBS_LOC_OPTIONAL_P(loc, i)) {
+      rb_ary_push(keys, ID2SYM(children->entries[i].name));
+
+    }
   }
 
   return keys;
@@ -258,11 +263,15 @@ static VALUE location_required_keys(VALUE self) {
   VALUE keys = rb_ary_new();
 
   rbs_loc *loc = rbs_check_location(self);
-  rbs_loc_list *list = loc->requireds;
+  rbs_loc_children *children = loc->children;
+  if (children == NULL) {
+    return keys;
+  }
 
-  while (list) {
-    rb_ary_push(keys, ID2SYM(list->name));
-    list = list->next;
+  for (unsigned short i = 0; i < children->len; i++) {
+    if (RBS_LOC_REQUIRED_P(loc, i)) {
+      rb_ary_push(keys, ID2SYM(children->entries[i].name));
+    }
   }
 
   return keys;

data/ext/rbs_extension/location.h

@@ -9,17 +9,24 @@
  * */
 extern VALUE RBS_Location;
 
-typedef struct rbs_loc_list {
+typedef struct {
   ID name;
   range rg;
-  struct rbs_loc_list *next;
-} rbs_loc_list;
+} rbs_loc_entry;
+
+typedef unsigned int rbs_loc_entry_bitmap;
+
+typedef struct {
+  unsigned short len;
+  unsigned short cap;
+  rbs_loc_entry_bitmap required_p;
+  rbs_loc_entry entries[0];
+} rbs_loc_children;
 
 typedef struct {
   VALUE buffer;
   range rg;
-  rbs_loc_list *requireds;
-  rbs_loc_list *optionals;
+  rbs_loc_children *children;
 } rbs_loc;
 
 /**
@@ -32,13 +39,24 @@ VALUE rbs_new_location(VALUE buffer, range rg);
  * */
 rbs_loc *rbs_check_location(VALUE location);
 
+/**
+ * Allocate memory for child locations.
+ *
+ * Do not call twice for the same location.
+ * */
+void rbs_loc_alloc_children(rbs_loc *loc, unsigned short cap);
+
 /**
  * Add a required child range with given name.
+ *
+ * Allocate memory for children with rbs_loc_alloc_children before calling this function.
  * */
 void rbs_loc_add_required_child(rbs_loc *loc, ID name, range r);
 
 /**
  * Add an optional child range with given name.
+ *
+ * Allocate memory for children with rbs_loc_alloc_children before calling this function.
  * */
 void rbs_loc_add_optional_child(rbs_loc *loc, ID name, range r);