rbs 4.0.0.dev.4 → 4.0.0

data/docs/aliases.md

@@ -0,0 +1,79 @@
+# Aliases
+
+This document explains module/class aliases and type aliases.
+
+## Module/class alias
+
+Module/class aliases give another name to a module/class.
+This is useful for some syntaxes that has lexical constraints.
+
+```rbs
+class C
+end
+
+class D = C         # ::D is an alias for ::C
+
+class E < D         # ::E inherits from ::D, which is actually ::C
+end
+```
+
+Note that module/class aliases cannot be recursive.
+
+So, we can define a *normalization* of aliased module/class names.
+Normalization follows the chain of alias definitions and resolves them to the original module/class defined with `module`/`class` syntax.
+
+```rbs
+class C
+end
+
+class D = C
+class E = D
+```
+
+`::E` is defined as an alias, and it can be normalized to `::C`.
+
+## Type alias
+
+The biggest difference from module/class alias is that type alias can be recursive.
+
+```rbs
+# cons_cell type is defined recursively
+type cons_cell = nil
+               | [Integer, cons_cell]
+```
+
+This means type aliases *cannot be* normalized generally.
+So, we provide another operation for type alias, `DefinitionBuilder#expand_alias` and its family.
+It substitutes with the immediate right hand side of a type alias.
+
+```
+cons_cell ===> nil | [Integer, cons_cell]                   (expand 1 step)
+          ===> nil | [Integer, nil | [Integer, cons_cell]]  (expand 2 steps)
+          ===> ...                                          (expand will go infinitely)
+```
+
+Note that the namespace of a type alias *can be* normalized, because they are module names.
+
+```rbs
+module M
+  type t = String
+end
+
+module N = M
+```
+
+With the type definition above, a type `::N::t` can be normalized to `::M::t`.
+And then it can be expanded to `::String`.
+
+> [!NOTE]
+> This is something like an *unfold* operation in type theory.
+
+## Type name resolution
+
+Type name resolution in RBS usually rewrites *relative* type names to *absolute* type names.
+`Environment#resolve_type_names` converts all type names in the RBS type definitions, and returns a new `Environment` object.
+
+It also *normalizes* modules names in type names.
+
+- If the type name can be resolved and normalized successfully, the AST has *absolute* type names.
+- If the type name resolution/normalization fails, the AST has *relative* type names.

data/docs/collection.md

@@ -159,9 +159,9 @@ For example:
 # manifest.yaml
 
 dependencies:
-  # If your gem depends on pathname but the gemspec doesn't include pathname,
+  # If your gem depends on logger but the gemspec doesn't include logger,
   # you need to write the following.
-  - name: pathname
+  - name: logger
 ```
 
 If the gem's RBS is managed with [ruby/gem_rbs_collection](https://github.com/ruby/gem_rbs_collection), put it as `gems/GEM_NAME/VERSION/manifest.yaml`.  For example, `gems/activesupport/6.0/manifest.yaml`.
@@ -189,4 +189,4 @@ If the gem's RBS is included in the gem package, put it as `sig/manifest.yaml`.
 ## How it works
 
 `rbs collection` is integrated with Bundler.
-`rbs collection install` command generates `gem_rbs_collection.lock.yaml` from `gem_rbs_collection.yaml` and `Gemfile.lock`. It uses `Gemfile.lock` to detects dependencies.
+`rbs collection install` command generates `rbs_collection.lock.yaml` from `rbs_collection.yaml` and `Gemfile.lock`. It uses `Gemfile.lock` to detects dependencies.

data/docs/config.md

@@ -0,0 +1,171 @@
+# config.yml
+
+`config.yml` is the definition of data structure for parser results -- AST.
+
+It defines the data structure for the parser implementation in C and Rust `node` modules.
+
+## C parser code
+
+```sh
+$ rake templates
+```
+
+Our C parser consists of two components:
+
+1. Pure C Parser
+2. Translator from the pure C AST to Ruby object
+
+`config.yml` defines the AST for pure C parser in `ast.h`/`ast.c` and translator from the C AST to Ruby objects in `ast_translation.h`/`ast_translation.c`.
+
+## `ruby-rbs` crate
+
+```sh
+$ cd rust; cargo build
+```
+
+The `build.rs` in `ruby-rbs` crate defines the data structure derived from `config.yml` definitions under `node` module.
+
+## nodes
+
+`nodes` defines *node* data types in C or Rust.
+
+```yaml
+nodes:
+  - name: RBS::AST::Declarations::Class
+    rust_name: ClassNode
+    fields:
+      - name: name
+        c_type: rbs_type_name
+      - name: type_params
+        c_type: rbs_node_list
+      - name: super_class
+        c_type: rbs_ast_declarations_class_super
+        optional: true  # NULL when no superclass (e.g., `class Foo end` vs `class Foo < Bar end`)
+      - name: members
+        c_type: rbs_node_list
+      - name: annotations
+        c_type: rbs_node_list
+      - name: comment
+        c_type: rbs_ast_comment
+        optional: true  # NULL when no comment precedes the declaration
+```
+
+This defines `rbs_ast_declarations_class` struct so that the parser constructs the AST using the structs.
+
+```c
+typedef struct rbs_ast_declarations_class {
+    rbs_node_t base;
+
+    struct rbs_type_name *name;
+    struct rbs_node_list *type_params;
+    struct rbs_ast_declarations_class_super *super_class; /* Optional */
+    struct rbs_node_list *members;
+    struct rbs_node_list *annotations;
+    struct rbs_ast_comment *comment; /* Optional */
+
+    rbs_location_range keyword_range;     /* Required */
+    rbs_location_range name_range;        /* Required */
+    rbs_location_range end_range;         /* Required */
+    rbs_location_range type_params_range; /* Optional */
+    rbs_location_range lt_range;          /* Optional */
+} rbs_ast_declarations_class_t;
+```
+
+The `rbs_ast_declarations_class` struct is a pure C AST, and `ast_translation.c` defines translation into a Ruby object of `RBS::AST::Declarations::Class` class.
+
+```c
+    case RBS_AST_DECLARATIONS_CLASS: {
+        rbs_ast_declarations_class_t *node = (rbs_ast_declarations_class_t *) instance;
+
+        VALUE h = rb_hash_new();
+        VALUE location = rbs_location_range_to_ruby_location(ctx, node->base.location);
+        rbs_loc *loc = rbs_check_location(location);
+        rbs_loc_legacy_alloc_children(loc, 5);
+        rbs_loc_legacy_add_required_child(loc, rb_intern("keyword"), (rbs_loc_range) { .start = node->keyword_range.start_char, .end = node->keyword_range.end_char });
+        rbs_loc_legacy_add_required_child(loc, rb_intern("name"), (rbs_loc_range) { .start = node->name_range.start_char, .end = node->name_range.end_char });
+        rbs_loc_legacy_add_required_child(loc, rb_intern("end"), (rbs_loc_range) { .start = node->end_range.start_char, .end = node->end_range.end_char });
+        rbs_loc_legacy_add_optional_child(loc, rb_intern("type_params"), (rbs_loc_range) { .start = node->type_params_range.start_char, .end = node->type_params_range.end_char });
+        rbs_loc_legacy_add_optional_child(loc, rb_intern("lt"), (rbs_loc_range) { .start = node->lt_range.start_char, .end = node->lt_range.end_char });
+        rb_hash_aset(h, ID2SYM(rb_intern("location")), location);
+        rb_hash_aset(h, ID2SYM(rb_intern("name")), rbs_struct_to_ruby_value(ctx, (rbs_node_t *) node->name)); // rbs_type_name
+        rb_hash_aset(h, ID2SYM(rb_intern("type_params")), rbs_node_list_to_ruby_array(ctx, node->type_params));
+        rb_hash_aset(h, ID2SYM(rb_intern("super_class")), rbs_struct_to_ruby_value(ctx, (rbs_node_t *) node->super_class)); // rbs_ast_declarations_class_super
+        rb_hash_aset(h, ID2SYM(rb_intern("members")), rbs_node_list_to_ruby_array(ctx, node->members));
+        rb_hash_aset(h, ID2SYM(rb_intern("annotations")), rbs_node_list_to_ruby_array(ctx, node->annotations));
+        rb_hash_aset(h, ID2SYM(rb_intern("comment")), rbs_struct_to_ruby_value(ctx, (rbs_node_t *) node->comment)); // rbs_ast_comment
+
+        rb_funcall(
+            RBS_AST_TypeParam,
+            rb_intern("resolve_variables"),
+            1,
+            rb_hash_lookup(h, ID2SYM(rb_intern("type_params")))
+        );
+        return CLASS_NEW_INSTANCE(
+            RBS_AST_Declarations_Class,
+            1,
+            &h
+        );
+    }
+```
+
+## enums
+
+`enums` defines *enum* data types in C or Rust.
+
+```yaml
+enums:
+  attribute_visibility:
+    optional: true
+    symbols:
+      - unspecified
+      - public
+      - private
+```
+
+For example, the `attribute_visibility` enum is a data type for `visibility` attribute of `attr_reader`, `attr_writer`, and `attr_accessor` definitions.
+The `visibility` attribute can be one of `unspecified`, `public`, and `private`.
+
+### Symbol enums
+
+Enum definition with `symbols:` attribute defines *enum* data that is mapped to Ruby symbols.
+
+```yaml
+enums:
+  attribute_visibility:
+    optional: true
+    symbols:
+      - unspecified
+      - public
+      - private
+```
+
+It defines an `enum` in C AST definition.
+
+```c
+enum RBS_ATTRIBUTE_VISIBILITY_TAG {
+    RBS_ATTRIBUTE_VISIBILITY_TAG_UNSPECIFIED,
+    RBS_ATTRIBUTE_VISIBILITY_TAG_PUBLIC,
+    RBS_ATTRIBUTE_VISIBILITY_TAG_PRIVATE,
+};
+```
+
+The C extension also defines a translation:
+
+```c
+VALUE rbs_attribute_visibility_to_ruby(enum rbs_attribute_visibility value) {
+    switch (value) {
+    case RBS_ATTRIBUTE_VISIBILITY_UNSPECIFIED:
+        return Qnil;
+    case RBS_ATTRIBUTE_VISIBILITY_PUBLIC:
+        return rb_id2sym(rb_intern("public"));
+    case RBS_ATTRIBUTE_VISIBILITY_PRIVATE:
+        return rb_id2sym(rb_intern("private"));
+    default:
+        rb_fatal("unknown enum rbs_attribute_visibility value: %d", value);
+    }
+}
+```
+
+`RBS_ATTRIBUTE_VISIBILITY_PUBLIC` and `RBS_ATTRIBUTE_VISIBILITY_PRIVATE` are translated to Ruby symbols `:public` and `:private` respectively. 
+
+Note that the first `RBS_ATTRIBUTE_VISIBILITY_UNSPECIFIED` is translated to `nil` in Ruby. This is specified by the `optional: true` attribute in YAML. When `optional: true` is set, the first enum value is translated to `nil`.

data/docs/encoding.md

@@ -0,0 +1,56 @@
+# RBS File Encoding
+
+## Best Practice
+
+**Use UTF-8** for both file encoding and your system locale.
+
+## Supported Encodings
+
+RBS parser supports ASCII-compatible encodings (similar to Ruby's script encoding support).
+
+**Examples**: UTF-8, US-ASCII, Shift JIS, EUC-JP, ...
+
+## Unicode Codepoint Symbols
+
+String literal types in RBS can contain Unicode codepoint escape sequences (`\uXXXX`).
+
+When the file encoding is UTF-8, the parser translates Unicode codepoint symbols:
+
+```rbs
+# In UTF-8 encoded files
+
+type t = "\u0123"  # Translated to the actual Unicode character ģ
+type s = "\u3042"  # Translated to the actual Unicode character あ
+```
+
+When the file encoding is not UTF-8, Unicode escape sequences are interpreted literally as the string `\uXXXX`:
+
+```rbs
+# In non-UTF-8 encoded files
+
+type t = "\u0123"  # Remains as the literal string "\u0123"
+```
+
+## Implementation
+
+RBS gem currently doesn't do anything for file encoding. It relies on Ruby's encoding handling, specifically `Encoding.default_external` and `Encoding.default_internal`.
+
+`Encoding.default_external` is the encoding Ruby assumes when it reads external resources like files. The Ruby interpreter sets it based on the locale. `Encoding.default_internal` is the encoding Ruby converts the external resources to. The default is `nil` (no conversion.)
+
+When your locale is set to use `UTF-8` encoding, `default_external` is `Encoding::UTF_8`. So the RBS file content read from the disk will have UTF-8 encoding.
+
+### Parsing non UTF-8 RBS source text
+
+If you want to work with another encoding, ensure the source string has ASCII compatible encoding.
+
+```ruby
+source = '"日本語"'
+RBS::Parser.parse_type(source.encode(Encoding::EUC_JP))  # => Parses successfully
+RBS::Parser.parse_type(source.encode(Encoding::UTF_32))  # => Returns `nil` since UTF-32 is not ASCII compatible
+```
+
+### Specifying file encoding
+
+Currently, RBS doesn't support specifying file encoding directly.
+
+You can use `Encoding.default_external` while the gem loads RBS files from the storage.

data/docs/gem.md

@@ -34,7 +34,6 @@ dependencies:
   - name: json
   - name: logger
   - name: optparse
-  - name: pathname
   - name: rdoc
   - name: tsort
 ```