rubex 0.1.1 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA256:
-  metadata.gz: 742568f609bbfd393f0262b1f8b79089a3e22007053e72714bbb8cc256fd5066
-  data.tar.gz: 7bd08df3a97cdf480bdb4fa5b34ef37553678f97cbd19f53027296148724c66a
+SHA1:
+  metadata.gz: c8859787757dadc4f2cadd8cbb2d227d26ae5f69
+  data.tar.gz: bb028214cc70f7fee50d0fd0c185bd89244eb593
 SHA512:
-  metadata.gz: 32b17fb06897cf4ca018dd61ba27ee58b6f21d885b715c7ab13effd726175465379912c168c21ba0a7ba48ee69f244996fbb680ae5a33a78021cd07745b6473f
-  data.tar.gz: febfc14a8dabb0ab050d592f9d50cfeaaf068007cb0635104876cfa679710fd311e4e71d38f4f8cf3019a6ef78e231fa4638381803899e3f4ee499f19312d41c
+  metadata.gz: 71322d0c3be4a0acedfa74d11789941c43f5cd81613bd771445f6124e48fa84b22bbdb451de731bdc15df1a534715e1628debdc10334cb468773c584bb992746
+  data.tar.gz: 1ec819863e026a14283dfb0d0ffde57df6d29457cb72561f816cc6aa840d29fe01947a1e705550205b5dff30806f1c12873653917ffa1780a9425408dd772d58

data/.gitignore

@@ -53,3 +53,4 @@ lib/rubex/parser.output
 lib/rubex/parser.tab.rb
 /.idea/
 TAGS
+parse.y

data/CONTRIBUTING.md

@@ -31,14 +31,66 @@ Couple of conventions that I followed when writing the code:
   - analyse_statement(local_scope)
   - generate_code(code, local_scope)
 * Expressions contain methods:
-  - analyse_statement(local_scope)
+  - analyse_types(local_scope)
   - c_code(local_scope)
 
-Sometimes it can so happen that an expression can consist simply of a single variable (like `if (a)`) or be a full expression (like `if (a == b)`). In the first case, the `a` is just read as an `IDENTIFIER` by the parser. In the second case, `a == b` is read as an `expr` and is stored in the AST as `Rubex::AST::Expression`.
-
-Now you might be thinking why expressions should also have a `analyse_statement` method, that's just for maintaining uniformity between exprs and stmts (maybe that should be just `analyse`?).
-
-When writing the `<=>` operator under Rubex::DataType classes, I have been forced to assume that `Int` is 32 bits long since I have not yet incorporated a way to figure out the number of bits used by a particular machine for representing an `int`. It will be changed later to make it machine-dependent.
+Sometimes it can so happen that an expression can consist simply of a single variable 
+(like `if (a)`) or be a full expression (like `if (a == b)`). In the first case, the 
+`a` is just read as an `IDENTIFIER` by the parser. In the second case, `a == b` is 
+read as an `expr` and is stored in the AST as `Rubex::AST::Expression`.
+
+When writing the `<=>` operator under Rubex::DataType classes, I have been forced to 
+assume that `Int` is 32 bits long since I have not yet incorporated a way to figure 
+out the number of bits used by a particular machine for representing an `int`. 
+It will be changed later to make it machine-dependent.
+
+## Parser particulars
+
+Kinds of statements that can be defined at the top:
+* classes.
+* attached classes.
+* modules.
+* ruby/c methods.
+* all kinds of exprs. (command call/binary/unary etc.)
+* if stmts.
+* while.
+* for.
+* c bindings.
+* function declaration.
+* struct or union definition.
+* alias statements.
+* assignment statements.
+* variable decl and init.
+* forward declaration.
+
+stmts that can be defined inside other classes:
+* other classes.
+* attached classes.
+* modules.
+* c functions.
+* ruby methods.
+* struct/union definitions.
+* constants.
+* class variables.
+* attr reader/writer/accessor.
+* forward declaration.
+
+stmts that can be defined inside other functions/blocks:
+* EVERYTHING _except_:
+  - classes.
+  - attached classes.
+  - modules.
+  - ruby/c methods.
+  - c bindinds.
+
+## Multi file programs
+
+When specifying mutliple rubex files using rake tasks or the command line, rubex will read
+the name of the main file, find the main file and start compiling that first. Whatever
+
+Since above approach does not really require user to specify all the files at compile time,
+the 'files' option is only useful when the user is only running the Rubex compiler and not
+generating any binaries.
 
 # Important data representations
 
@@ -52,7 +104,8 @@ This consists of a hash that looks like this:
 }
 ```
 
-The `:variables` field might be `nil` in case of a function declaration in which case it is not necessary to specify the name of the variable.
+The `:variables` field might be `nil` in case of a function declaration in which
+case it is not necessary to specify the name of the variable.
 
 The `:variables` key maps to a value that is an Array of Hashes that contains a single Hash:
 ```
@@ -63,7 +116,8 @@ The `:variables` key maps to a value that is an Array of Hashes that contains a
 }
 ```
 
-`identity` can be a Hash in case of a function pointer argument, or a simple String in case its an identifier, or an `ElementRef` if specifying an array of elements.
+`identity` can be a Hash in case of a function pointer argument, or a simple String in case 
+its an identifier, or an `ElementRef` if specifying an array of elements.
 
 If Hash, it will look like this:
 ```
@@ -78,13 +132,23 @@ If Hash, it will look like this:
 
 ## Attach classes
 
+## no_gil block code generation
 
+The generated code for the `no_gil` block has the following components:
+* a function that is namespaced according to the name of the scoping
+    function, class and number of no_gil blocks that occur in the function.
+* all the statements that are present in the no_gil block are present in this function.
+* all the variables that are present in the block are made global and are properly
+    namespaced accoring to above namespacing criteria.
+* code appropriate for `rb_thread_call_without_gvl` is generated and a pointer to the
+  generated function is passed to the function.
+* the new code using the `without_gvl` call is inserted in the place where the `no_gil`
+    block was present.
 
 # Future work
 
 The following features in Rubex need to be implemented or can be made better:
 
-* Ability to have Ruby-style method arguments without parenthesis.
 * Multiline conditionals in the condition of if-elsif statements.
 * Special treatment for VALUE C arrays by marking each element with GC.
 * Checks for return statement. No return statement or wrong return type should raise error/warning.

data/HISTORY.md

@@ -1,3 +1,22 @@
+# v0.1.2 (26 May 2018)
+
+## Bug fixes
+
+* Fixed bug that prevented proper loading of files from CLI.
+
+## Features
+
+* Added `no_gil` block support.
+* Allow creating instance variables.
+* Multiple file Rubex programs using `rubex_require`.
+* Support multiple arguments in `[]` operator.
+* New rake tasks for various options like debug, just compiling, compile + install etc.
+* More CLI options for debug and force.
+
+# v0.1.1
+
+* Bugs fixes and internal code refactoring. Not much in terms of user-facing features.
+
 # v0.1 (17 Sept. 2017)
 
 * First release.

data/README.md

@@ -118,19 +118,54 @@ Notice the only difference between the above Rubex code and Ruby is the specific
 Rubex also takes care of the initial setup and compilation of the C files, so all you need to do is execute a bunch of commands and your extension is up and running!
 
 # Installation
-To build, requires Ruby version >= 2.3.0
+Requires Ruby version >= 2.3.0
 
-The gem as of now has not reached v0.1. However, you can try it out with:
+Install with:
 ```
-git clone https://github.com/v0dro/rubex.git
-cd rubex
-bundle install
-rake install
+gem install rubex
 ```
 
 # Usage
 
-Installing the gem will also install the `rubex` binary. You can now write a Rubex file (with a `.rubex` file extension) and compile it into C code with:
+Installing the gem will also install the `rubex` binary. You can now write a Rubex file (with a `.rubex` file extension) and compile it into C code using the following commands.
+
+## Commands
+
+#### Generate
+
+Create all the necessary files for the C extension.
+
+```
+rubex generate file_name.rubex
+```
+
+```
+Options
+  -f, [--force]            # replace existing files and directories
+  -d, [--dir=DIR]          # specify a directory for generating files
+  -i, [--install]          # automatically run install command after generating Makefile
+  -g, [--debug]            # enable debugging symbols when compiling with GCC
+```
+
+#### Install
+
+Run the `make` utility on generated files.
+
+```
+rubex install path/to/generated/directory
+```
+
+#### Help
+
+Describe available commands or one specific command
+
+```
+rubex help [COMMAND]
+```
+
+## Manual Usage
+If you want to manually generate the files, you can do that with:
+
 ```
 rubex file_name.rubex
 ```
@@ -150,6 +185,15 @@ Give yourself 5 min and go through the [TUTORIAL](TUTORIAL.md). Convert a part o
 
 Read the full Rubex reference in [REFERENCE](REFERENCE.md).
 
+# Differences with Ruby
+
+Although Rubex tries its best to support the Ruby syntax as much as possible, in some cases it is not feasible or necessary to provide full support. Following is a list of differences between Ruby and Rubex syntax:
+
+* All methods in Rubex (including `require` calls) must use round brackets for arguments.
+* No support Ruby blocks.
+* No support for class variables.
+* All methods and functions in Rubex must use the `return` statement for returning values.
+
 # Roadmap
 
 See the [CONTRIBUTING](CONTRIBUTING.md) and the GitHub issue tracker for future features.
@@ -157,5 +201,6 @@ See the [CONTRIBUTING](CONTRIBUTING.md) and the GitHub issue tracker for future
 # Acknowledgements
 
 * The Ruby Association (Japan) for providing the initial funding for this project through the Ruby Association Grant 2016.
-* Koichi Sasada (@ko1) and Kenta Murata (@mrkn) for their support and mentorship throughout this project.
+* Koichi Sasada (@ko1), Kenta Murata (@mrkn) and Naotoshi Seo (@sonots) for their support and mentorship throughout this project.
 * Fukuoka Ruby Award 2017.
+* Tokyo Institute of Technology.

data/REFERENCE.md

@@ -2,59 +2,63 @@
 
 Rubex is a language designed to keep you happy even when writing C extension.
 
-# Table of Contents
-<!-- MarkdownTOC autolink="true" bracket="round" depth="3"-->
+<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-generate-toc again -->
+**Table of Contents**
 
+- [Rubex (RUBy EXtension Language)](#rubex-ruby-extension-language)
+- [Table of Contents](#table-of-contents)
 - [Comments](#comments)
 - [C data types](#c-data-types)
-  - [Primitive data types](#primitive-data-types)
-  - [C struct, union and enum](#c-struct-union-and-enum)
-    - [Forward declarations](#forward-declarations)
-  - [Pointers](#pointers)
+    - [Primitive data types](#primitive-data-types)
+    - [C struct, union and enum](#c-struct-union-and-enum)
+        - [Forward declarations](#forward-declarations)
+    - [Pointers](#pointers)
 - [Ruby Objects](#ruby-objects)
 - [Literals](#literals)
-  - [Integer](#integer)
-  - [Float](#float)
-  - [Character](#character)
-  - [String](#string)
-  - [Ruby Literals](#ruby-literals)
-    - [Ruby Symbol](#ruby-symbol)
-    - [Ruby Array](#ruby-array)
-    - [Ruby Hash](#ruby-hash)
+    - [Integer](#integer)
+    - [Float](#float)
+    - [Character](#character)
+    - [String](#string)
+    - [Ruby Literals](#ruby-literals)
+        - [Ruby Symbol](#ruby-symbol)
+        - [Ruby Array](#ruby-array)
+        - [Ruby Hash](#ruby-hash)
 - [C Functions and Ruby Methods](#c-functions-and-ruby-methods)
 - [Ruby Constants](#ruby-constants)
 - [The print statement](#the-print-statement)
 - [Loops](#loops)
-  - [The while loop](#the-while-loop)
-  - [The for loop](#the-for-loop)
+    - [The while loop](#the-while-loop)
+    - [The for loop](#the-for-loop)
 - [Conditionals](#conditionals)
-  - [Important Note](#important-note)
+    - [Important Note](#important-note)
 - [Interfacing C libraries with lib](#interfacing-c-libraries-with-lib)
-  - [Basic Usage](#basic-usage)
-  - [Supported declarations](#supported-declarations)
-    - [Functions](#functions)
-    - [Variables](#variables)
-    - [Macros](#macros)
-    - [Types](#types)
-    - [Typedefs](#typedefs)
-  - [Linking Libraries](#linking-libraries)
-  - [Ready-to-use C functions](#ready-to-use-c-functions)
-    - [Filename: "rubex/ruby"](#filename-rubexruby)
-    - [Filename: "rubex/ruby/encoding"](#filename-rubexrubyencoding)
-    - [Filename: "rubex/stdlib"](#filename-rubexstdlib)
+    - [Basic Usage](#basic-usage)
+    - [Supported declarations](#supported-declarations)
+        - [Functions](#functions)
+        - [Variables](#variables)
+        - [Macros](#macros)
+        - [Types](#types)
+        - [Typedefs](#typedefs)
+    - [Linking Libraries](#linking-libraries)
+    - [Ready-to-use C functions](#ready-to-use-c-functions)
+        - [Filename: "rubex/ruby"](#filename-rubexruby)
+        - [Filename: "rubex/ruby/encoding"](#filename-rubexrubyencoding)
+        - [Filename: "rubex/stdlib"](#filename-rubexstdlib)
 - [Exception Handling](#exception-handling)
 - ['Attach' Classes](#attach-classes)
-  - [The attach keyword](#the-attach-keyword)
-  - [The data$ variable](#the-data-variable)
-  - [Special C functions in attach classes](#special-c-functions-in-attach-classes)
-    - [deallocate](#deallocate)
-    - [allocate](#allocate)
-    - [memcount](#memcount)
-    - [get_struct](#getstruct)
-    - [gc_mark](#gcmark)
+    - [The attach keyword](#the-attach-keyword)
+    - [The data$ variable](#the-data-variable)
+    - [Special C functions in attach classes](#special-c-functions-in-attach-classes)
+        - [deallocate](#deallocate)
+        - [allocate](#allocate)
+        - [memcount](#memcount)
+        - [get_struct](#getstruct)
+        - [gc_mark](#gcmark)
 - [Typecast](#typecast)
 - [Alias](#alias)
 - [Conversions between Ruby and C data](#conversions-between-ruby-and-c-data)
+- [Releasing the Global Interpreter Lock](#releasing-the-global-interpreter-lock)
+- [Multi-file rubex programs](#multi-file-rubex-programs)
 - [C callbacks](#c-callbacks)
 - [Inline C](#inline-c)
 - [Handling Strings](#handling-strings)
@@ -62,7 +66,7 @@ Rubex is a language designed to keep you happy even when writing C extension.
 - [Differences from Ruby](#differences-from-ruby)
 - [Limitations](#limitations)
 
-<!-- /MarkdownTOC -->
+<!-- markdown-toc end -->
 
 # Comments
 
@@ -640,6 +644,25 @@ The `xfree` function, which is the standard memory freeing function provided by
 
 ### allocate
 
+The `allocate` function is called by the Ruby interpreter before it initializes the class
+using `initialize`. It is here that the C level memory allocations that are assigned to a
+particular class via attached structs should be `xmalloc`'d. The correponding data should be
+freed using `xfree` inside the `deallocate` function that is documented below.
+
+Keep in mind that if you choose to define your own `allocate` function, it must have a return
+type `object`. An example `deallocate` would look like so:
+``` ruby
+struct test
+  int *a
+end
+
+class A attach test
+  cfunc object allocate
+    data$.test.a = <int*>xmalloc(sizeof(int)*100)
+  end
+end
+```
+
 ### memcount
 
 ### get_struct
@@ -681,7 +704,43 @@ end
 
 # Conversions between Ruby and C data
 
-Rubex will implicitly convert most primitive C types like `char`, `int` and `float` to their equivalent Ruby types and vice versa. However, types conversions for user defined types like structs and unions are not supported.
+Rubex will implicitly convert most primitive C types like `char`, `int` and `float`
+to their equivalent Ruby types and vice versa. However, types conversions for user
+defined types like structs and unions are not supported.
+
+# Releasing the Global Interpreter Lock
+
+Rubex can release the GIL a very simple `no_gil` construct. Some care must be taken when
+using other functions inside this block. As a rule you cannot use any Ruby objects
+inside a `no_gil` block. All data handling must be done _only_ by C data types.
+
+Functions that are called inside the `no_gil` block cannot use any Ruby objects either, and
+must be tagged using the `no_gil` keyword. This will ensure that Rubex checks for Ruby
+objects inside these functions and raises an error if you accidentally use Ruby objects. Note
+that these checks are not exhaustive and the user should be careful.
+
+Here's an example of using a `no_gil` block from a Ruby method:
+``` ruby
+cfunc void _work_without_gil(double n) no_gil
+  while n > 0 do
+    n ** 0.5 + 4
+    n -= 1
+  end
+end
+
+def work_without_gil(n)
+  double i = n
+  no_gil
+    _work_without_gil(i)
+  end
+
+  return i
+end
+```
+
+# Multi-file rubex programs
+
+
 
 # C callbacks
 
@@ -729,14 +788,23 @@ end
 
 # Handling Strings
 
-For purposes of optimization and compatibility with C, Rubex makes certain assumptions about strings. When you assign a Ruby object to a `char*`, Rubex will automatically pass a pointer to the C string contained inside the object to the `char*`, thereby increasing the efficiency of operations on the string. The resulting string is a regular `\0` delimited C string.
+For purposes of optimization and compatibility with C, Rubex makes certain 
+assumptions about strings. When you assign a Ruby object to a `char*`, Rubex
+will automatically pass a pointer to the C string contained inside the object
+to the `char*`, thereby increasing the efficiency of operations on the string.
+The resulting string is a regular `\0` delimited C string.
 
-It should be noted that strings MUST use the double quotation syntax (`""`) and not single quotes in all cases. Single quote is reserved for use by C `char` data. For example, to assign a literal value to a C char, you can write `char a = 'b'`, to assign a literal C string to a `char*` array, use `char* a = "hello"`.
+It should be noted that strings MUST use the double quotation syntax (`""`) and
+not single quotes in all cases. Single quote is reserved for use by C `char` data. 
+For example, to assign a literal value to a C char, you can write `char a = 'b'`, 
+to assign a literal C string to a `char*` array, use `char* a = "hello"`.
 
 # Differences from C
 
-* Rubex does not have dereferencing operator (`*`). Instead use `[0]` to access values pointed to by pointers.
-* There is no `->` operator for accessing struct elements from a pointer to a struct. Use the `.` operator directly.
+* Rubex does not have dereferencing operator (`*`). Instead use `[0]` to access
+values pointed to by pointers.
+* There is no `->` operator for accessing struct elements from a pointer to a struct.
+Use the `.` operator directly.
 
 # Differences from Ruby
 
@@ -750,4 +818,4 @@ It should be noted that strings MUST use the double quotation syntax (`""`) and
 Most of the below limitations are temporary and will be taken care of in future versions:
 
 * The `require` statement is currently not supported.
-* Multi-file Rubex programs are not yet possible.
+* Multi-file Rubex programs are not yet possible.