heist 0.1.0 → 0.2.0

data/History.txt

@@ -1,3 +1,20 @@
+=== Version 0.2.0 (2009-04-01)
+
+The 'lists' release
+* Entirely revised to correctly support lists as linked pairs
+* Complete set of R5RS list functions
+* Syntax for dotted pairs and improper lists implemented
+* Rest-args for functions using dot notation
+* Almost-complete R5RS numeric library, including complexes and rationals
+* Some parser bugs regarding literals and quoting fixed
+* Many macro parsing and expansion bugs fixed, esp. concerning nested repeating patterns
+* Macro keywords and collisions with local variables now follow the spec
+* R6RS ellipsis escaping feature -- (... ...) -- implemented
+* All library syntax now implemented as macros, should all support call/cc
+* Ruby data can now be executed as Scheme code
+* Lots of inline documentation for the runtime
+
+
 === Version 0.1.0 (2009-02-25)
 
 First public release

data/Manifest.txt

@@ -1,36 +1,36 @@
 History.txt
 Manifest.txt
-README.txt
 Rakefile
+README.txt
 bin/heist
-lib/heist.rb
 lib/bin_spec.rb
+lib/heist.rb
 lib/repl.rb
+lib/builtin/library.scm
 lib/builtin/primitives.rb
-lib/builtin/syntax.rb
 lib/builtin/syntax.scm
-lib/builtin/library.scm
-lib/parser/scheme.tt
-lib/parser/scheme.rb
 lib/parser/nodes.rb
-lib/runtime/runtime.rb
-lib/runtime/data/expression.rb
-lib/runtime/data/identifier.rb
-lib/runtime/data/list.rb
+lib/parser/ruby.rb
+lib/parser/scheme.rb
+lib/parser/scheme.tt
+lib/runtime/callable/continuation.rb
 lib/runtime/callable/function.rb
+lib/runtime/callable/syntax.rb
 lib/runtime/callable/macro.rb
 lib/runtime/callable/macro/matches.rb
-lib/runtime/callable/macro/splice.rb
+lib/runtime/callable/macro/tree.rb
 lib/runtime/callable/macro/expansion.rb
-lib/runtime/callable/continuation.rb
-lib/runtime/stack.rb
-lib/runtime/stackless.rb
+lib/runtime/data/cons.rb
+lib/runtime/data/expression.rb
+lib/runtime/data/identifier.rb
+lib/runtime/binding.rb
 lib/runtime/frame.rb
+lib/runtime/runtime.rb
 lib/runtime/scope.rb
-lib/runtime/binding.rb
+lib/runtime/stack.rb
+lib/runtime/stackless.rb
 lib/stdlib/benchmark.scm
 lib/stdlib/birdhouse.scm
-test/test_heist.rb
 test/arithmetic.scm
 test/benchmarks.scm
 test/booleans.scm
@@ -40,14 +40,18 @@ test/continuations.scm
 test/define_functions.scm
 test/define_values.scm
 test/delay.scm
+test/equivalence.scm
 test/file_loading.scm
+test/functional.scm
+test/hygienic.scm
 test/let.scm
 test/lib.scm
+test/lists.scm
 test/macro-helpers.scm
 test/macros.scm
-test/hygienic.scm
-test/unhygienic.scm
-test/plt-macros.txt
 test/numbers.scm
+test/plt-macros.txt
+test/test_heist.rb
+test/unhygienic.scm
 test/vars.scm
 

data/README.txt

@@ -33,51 +33,61 @@ languages.
 
 == Features
 
-Heist nominally targets R5RS and is still in the early stages of development
-(at time of writing it is about a month old). The priority at this stage
-is runtime support for features that cannot be directly expressed as
-Scheme functions. The library is therefore rather small but the runtime
-is more advanced than some other toy Schemes I've seen.
+Heist nominally targets R5RS (http://www.schemers.org/Documents/Standards/R5RS/HTML).
+It has good support for many Scheme runtime features such as tail call
+optimisation, macros and first-class continuations, as well as reasonably
+complete numeric and list libraries.
 
 Currently implemented R5RS features include:
 
 * Boolean literals: #t and #f
-* Numeric literals for integers, reals and rationals (though rationals
-  are currently converted to floating point)
+* Numeric literals for integers, reals, rationals and complexes. The
+  latter two are mapped to Ruby's <tt>Rational</tt> and <tt>Complex</tt>
+  objects and Ruby handles all math procedures.
 * String and symbol literals
-* Proper lists and full (quasi)quoting with <tt>'</tt>, <tt>`</tt>,
-  <tt>,</tt>, <tt>,@</tt>
-* Macros: <tt>(syntax-rules)</tt>, <tt>(define-syntax)</tt>,
-  <tt>(let-syntax)</tt>, <tt>(letrec-syntax)</tt>. Supports keywords,
-  lists, pattern variables and ellipses down to arbitrary depth
-* Continuations: <tt>(call-with-current-continuation)</tt>, aliased
-  as <tt>(call/cc)</tt>
+* Proper and improper lists and full (quasi)quoting with <tt>'</tt>,
+  <tt>`</tt>, <tt>,</tt>, <tt>,@</tt>
+* Macros: <tt>syntax-rules</tt>, <tt>define-syntax</tt>,
+  <tt>let-syntax</tt>, <tt>letrec-syntax</tt>. Supports keywords,
+  lists, pattern variables and ellipses down to arbitrary depth, as
+  well as the R6RS ellipsis-escaping feature (<tt>(... ...)</tt>) to
+  better enable macro-writing macros
+* Continuations: <tt>call-with-current-continuation</tt>, aliased
+  as <tt>call/cc</tt>
 * Proper tail recursion
-* Variable assignment: <tt>(define)</tt> and <tt>(set!)</tt>
-* Lambda expressions: <tt>(lambda (x) body)</tt>, including lexical
-  scoping and closures (varargs are currently not supported)
-* Control flow: <tt>(begin)</tt>, <tt>(if)</tt>, <tt>(cond)</tt>,
-  <tt>(case)</tt>
-* Binding constructs: <tt>(let)</tt>, <tt>(let*)</tt>, <tt>(letrec)</tt>
-* Iteration: named <tt>(let)</tt>, <tt>(do)</tt>
-* Delayed evaluation: <tt>(delay)</tt> and <tt>(force)</tt>
-* Logical connectives: <tt>(and)</tt>, <tt>(or)</tt>, <tt>(not)</tt>
-* Type checking: <tt>(boolean?)</tt>, <tt>(number?)</tt>, <tt>(complex?)</tt>,
-  <tt>(real?)</tt>, <tt>(rational?)</tt>, <tt>(integer?)</tt>
-* Comparators: <tt>(eqv?)</tt>, <tt>(=)</tt>, <tt>(<)</tt>, <tt>(<=)</tt>,
-  <tt>(>)</tt>, <tt>(>=)</tt>
-* Numeric library: <tt>(max)</tt>, <tt>(min)</tt>, <tt>(+)</tt>,
-  <tt>(-)</tt>, <tt>(*)</tt>, <tt>(/)</tt>, <tt>(quotient)</tt>,
-  <tt>(remainder)</tt>, <tt>(modulo)</tt>, <tt>(floor)</tt>,
-  <tt>(ceiling)</tt>, <tt>(truncate)</tt>, <tt>(round)</tt>,
-  <tt>(exp)</tt>, <tt>(log)</tt>, <tt>(sin)</tt>, <tt>(cos)</tt>,
-  <tt>(tan)</tt>, <tt>(asin)</tt>, <tt>(acos)</tt>, <tt>(atan)</tt>,
-  <tt>(expt)</tt>, <tt>(sqrt)</tt>, <tt>(zero?)</tt>, <tt>(positive?)</tt>,
-  <tt>(negative?)</tt>, <tt>(odd?)</tt>, <tt>(even?)</tt>, <tt>(abs)</tt>,
-  <tt>(gcd)</tt>, <tt>(lcm)</tt>, <tt>(factorial)</tt>,
-  <tt>(number->string)</tt>
-* String library: <tt>(display)</tt>, <tt>(newline)</tt>,
-  <tt>(string->number)</tt>
+* Variable assignment: <tt>define</tt> and <tt>set!</tt>
+* Lambda expressions: <tt>(lambda (x ...) body ...)</tt>, including lexical
+  scoping and closures. Supports varargs using <tt>(lambda args body ...)</tt>
+  and <tt>(lambda (x y . rest) body ...)</tt> forms.
+* Control flow: <tt>begin</tt>, <tt>if</tt>, <tt>cond</tt>,
+  <tt>case</tt>
+* Binding constructs: <tt>let</tt>, <tt>let*</tt>, <tt>letrec</tt>
+* Iteration: named <tt>let</tt>, <tt>do</tt>
+* Delayed evaluation: <tt>delay</tt> and <tt>force</tt>
+* Logical connectives: <tt>and</tt>, <tt>or</tt>, <tt>not</tt>
+* Type checking: <tt>boolean?</tt>, <tt>number?</tt>, <tt>complex?</tt>,
+  <tt>real?</tt>, <tt>rational?</tt>, <tt>integer?</tt>, <tt>exact?</tt>,
+  <tt>inexact?</tt>, <tt>symbol?</tt>, <tt>string?</tt>, <tt>pair?</tt>,
+  <tt>list?</tt>, <tt>null?</tt>, <tt>procedure?</tt>
+* Comparators: <tt>eq?</tt>, <tt>eqv?</tt>, <tt>equal?</tt>, <tt>=</tt>,
+  <tt><</tt>, <tt><=</tt>, <tt>></tt>, <tt>>=</tt>
+* Numeric library: <tt>max</tt>, <tt>min</tt>, <tt>+</tt>, <tt>-</tt>,
+  <tt>*</tt>, <tt>/</tt>, <tt>quotient</tt>, <tt>remainder</tt>,
+  <tt>modulo</tt>, <tt>numerator</tt>, <tt>denominator</tt>, <tt>floor</tt>,
+  <tt>ceiling</tt>, <tt>truncate</tt>, <tt>round</tt>, <tt>exp</tt>,
+  <tt>log</tt>, <tt>sin</tt>, <tt>cos</tt>, <tt>tan</tt>, <tt>asin</tt>,
+  <tt>acos</tt>, <tt>atan</tt>, <tt>expt</tt>, <tt>sqrt</tt>,
+  <tt>zero?</tt>, <tt>positive?</tt>, <tt>negative?</tt>, <tt>odd?</tt>,
+  <tt>even?</tt>, <tt>abs</tt>, <tt>gcd</tt>, <tt>lcm</tt>,
+  <tt>make-rectangular</tt>, <tt>make-polar</tt>, <tt>real-part</tt>,
+  <tt>imag-part</tt>, <tt>magnitude</tt>, <tt>angle</tt>, <tt>factorial</tt>,
+  <tt>number->string</tt>
+* List library: <tt>cons</tt>, <tt>car</tt>, <tt>cdr</tt>, <tt>set-car!</tt>,
+  <tt>set-cdr!</tt>, <tt>c[ad]{1,4}r</tt>, <tt>list</tt>, <tt>length</tt>,
+  <tt>append</tt>, <tt>reverse</tt>, <tt>list-tail</tt>, <tt>list-ref</tt>,
+  <tt>memq</tt>, <tt>memv</tt>, <tt>member</tt>, <tt>assq</tt>, <tt>assv</tt>,
+  <tt>assoc</tt>, <tt>apply</tt>, <tt>map</tt>, <tt>for-each</tt>
+* String library: <tt>display</tt>, <tt>newline</tt>, <tt>string->number</tt>
 * File loading: <tt>(load "foo.scm")</tt>, paths are relative to the
   current file
 
@@ -120,6 +130,9 @@ To start an REPL session:
 Both these commands accept a number of runtime configuration options;
 run <tt>heist -h</tt> for more information.
 
+
+=== Embed in your Ruby app
+
 To embed Heist in a Ruby application, you need to create an instance of
 the runtime:
 
@@ -146,6 +159,13 @@ compatible; enabling continuations forces eager evaluation. Also
 note that lazy evaluation has a slightly unpredictable effect on the
 use of macros.
 
+To run a Scheme file using the runtime, just call:
+
+  scheme.run("path/to/file.scm")
+
+
+=== Create Scheme functions using Ruby
+
 Every runtime gets its own copy of all the built-in functions. You can add
 your own functions written in Ruby and Heist will make them available as
 Scheme procedures. For example, for running tests I do this in my +setup+
@@ -160,9 +180,28 @@ This lets me write, for example, <tt>(assert-equal 7 (+ 3 4))</tt> in my
 tests. You can place arbitrary Ruby code in your functions, so this is
 an easy way to expose Ruby functionality to the Scheme environment.
 
-To run a Scheme file using the runtime, just call:
 
-  scheme.run("path/to/file.scm")
+=== Run Ruby data as Scheme code
+
+Heist can interpret Scheme code provided as Ruby data, allowing you
+to write and manipulate Lisp-style expressions inline with Ruby code.
+For example, using our <tt>Heist::Runtime</tt> instance from above:
+
+  scheme.exec [:define, [:square, :x], [:*, :x, :x]]
+  
+  scheme.exec [:square, 9]
+  #=> 81
+
+Arrays map directly to Scheme lists, symbols map to identifiers, strings
+and numbers are treated as literals, and any other data is interpreted
+verbatim. So, you can use any piece of Ruby data in these expressions
+as long as the functions you're calling can operate on it.
+
+Due to syntactic limitations, some Scheme syntax cannot be used in this
+interface. Quoting is out, so the expression <tt>'(1 2 3)</tt> would
+need to be written <tt>[:quote, [1, 2, 3]]</tt>. Dotted pairs and
+improper lists are also not available, though you can work around
+this to some extent using <tt>:cons</tt>.
 
 
 == Notes
@@ -184,13 +223,6 @@ is not a boolean literal, a number or a string is considered a valid
 symbol. This means that syntax that should be used for chars and
 vectors will currently parse as symbols.
 
-There are currently no list operations. This may seem like a
-staggering omission but my goal initially was to work through the
-early stages of SICP, for which Heist in its current form is
-adequate. Being something of a simpleton, I wrote the list representation
-using arrays and it will need gutting and converting to linked lists
-before list operations are implemented.
-
 Macros are slightly more liberal than R5RS, in that this is a valid
 macro:
 
@@ -202,10 +234,10 @@ macro:
 You are allowed to use ellipses in infix positions, as long as the
 pattern that follows the ellipsis matches input that the preceeding
 pattern would fail to match. This is, the ellipsis acts as a greedy
-star operator in regular expressions. In the above, <tt>(name value) ...</tt>
-captures input until a non-list expression is met, at which point
-<tt>body ...</tt> takes over. The following expression evaluates
-to <tt>3</tt> using this macro:
+star operator in regular expressions. In the above, <tt>(name value) ... body ...</tt>
+matches zero or more two-element lists followed by zero or more
+expressions of any type. The following expression evaluates to
+<tt>3</tt> using this macro:
 
   (my-let (x 2) (y 3) y)
 

data/lib/builtin/library.scm

@@ -2,11 +2,36 @@
 ; in Scheme should go here. If at all possible, write
 ; builtins in Scheme rather than Ruby.
 
+; (newline)
+; prints a new-line character
+(define (newline)
+  (display "\n"))
+
+; (force)
+; Extracts the value of a promise created using (delay)
+(define (force promise) (promise))
+
+; (call/cc)
+; Alias for (call-with-current-continuation)
+(define call/cc call-with-current-continuation)
+
+; (eq? x y)
+; Currently an alias for (eqv? x y). TODO implement properly
+(define eq? eqv?)
+
 ; (not x)
 ; Boolean inverse of x
 (define (not x)
   (if x #f #t))
 
+; (negate x)
+; Returns a negated form of x, like (not) but also
+; works on functions
+(define (negate proc)
+  (if (procedure? proc)
+      (lambda args (not (apply proc args)))
+      (not proc)))
+
 ; Longhand aliases for boolean constants
 (define true #t)
 (define false #f)
@@ -16,10 +41,18 @@
 (define (boolean? x)
   (or (eqv? x #t) (eqv? x #f)))
 
+;----------------------------------------------------------------
+
+; Numerical functions
+
 ; (number? x)
 ; Returns true iff x is any type of number
 (define number? complex?)
 
+; (inexact? x)
+; Returns true iff x is not an exact number
+(define inexact? (negate exact?))
+
 ; (zero? x)
 ; Returns true iff x is zero
 (define (zero? x)
@@ -45,6 +78,20 @@
 (define (even? x)
   (zero? (remainder x 2)))
 
+; (max arg1 arg2 ...)
+; Returns the maximum value in the list of arguments
+(define (max . values)
+  (foldr (lambda (a b) (if (>= a b) a b))
+         (car values)
+         (cdr values)))
+
+; (min arg1 arg2 ...)
+; Returns the minimum value in the list of arguments
+(define (min . values)
+  (foldr (lambda (a b) (if (<= a b) a b))
+         (car values)
+         (cdr values)))
+
 ; (abs x)
 ; Returns the absolute value of a number
 (define (abs x)
@@ -71,6 +118,29 @@
 
 (define ceiling ceil)
 
+; (make-polar magnitude angle)
+; Returns a new complex number with the given
+; magnitude and angle
+(define (make-polar magnitude angle)
+  (let ([re (* magnitude (cos angle))]
+        [im (* magnitude (sin angle))])
+    (make-rectangular re im)))
+
+; (magnitude z)
+; Returns the magnitude of a complex number
+(define (magnitude z)
+  (let ([re (real-part z)]
+        [im (imag-part z)])
+    (sqrt (+ (* re re) (* im im)))))
+
+; (angle z)
+; Returns the angle a complex number makes with the
+; real axis when plotted in the complex plane
+(define (angle z)
+  (let ([re (real-part z)]
+        [im (imag-part z)])
+    (atan im re)))
+
 ; (factorial x)
 ; Returns factorial of x
 (define (factorial x)
@@ -80,16 +150,144 @@
         (iter (- y 1) (* y acc))))
   (iter x 1))
 
-; (newline)
-; prints a new-line character
-(define (newline)
-  (display "\n"))
+;----------------------------------------------------------------
 
-; (force)
-; Extracts the value of a promise created using (delay)
-(define (force promise) (promise))
+; List/pair functions
 
-; (call/cc)
-; Alias for (call-with-current-continuation)
-(define call/cc call-with-current-continuation)
+; (null? object)
+; Returns true iff object is the empty list
+(define (null? object)
+  (eqv? '() object))
+
+; (list? object)
+; Returns true iff object is a proper list
+(define (list? object)
+  (or (null? object)
+      (and (pair? object)
+           (list? (cdr object)))))
+
+; (list arg ...)
+; Allocates and returns a new list from its arguments
+(define (list . args) args)
+
+; (length object)
+; Returns the length of a proper list
+(define (length object)
+  (define (iter list acc)
+    (if (null? list)
+        acc
+        (iter (cdr list) (+ 1 acc))))
+  (iter object 0))
+
+; (append list ...)
+; Returns a new list formed by concatenating the arguments.
+; The final argument is not copied and the return value of
+; (append) shares structure with it.
+(define (append first . rest)
+  (if (null? first)
+      (apply append rest)
+      (if (null? rest)
+          first
+          (let ([copy (apply list first)])
+            (do ([pair copy (cdr pair)])
+                ((null? (cdr pair))
+                  (set-cdr! pair (apply append rest))))
+            copy))))
+
+; (reverse list)
+; Returns a newly allocated list consisting of the
+; elements of list in reverse order.
+(define (reverse object)
+  (if (null? object)
+      object
+      (append (reverse (cdr object))
+              (list (car object)))))
+
+; (list-tail list k)
+; Returns the sublist of list obtained by omitting the
+; first k elements.
+(define (list-tail list k)
+  (do ([pair list (cdr pair)]
+       [i k (- i 1)])
+      ((zero? i) pair)))
+
+; (list-ref list k)
+; Returns the kth element of list.
+(define (list-ref list k)
+  (car (list-tail list k)))
+
+; (memq obj list)
+; (memv obj list)
+; (member obj list)
+; These procedures return the first sublist of list whose
+; car is obj, where the sublists of list are the non-empty
+; lists returned by (list-tail list k) for k less than the
+; length of list. If obj does not occur in list, then #f
+; (not the empty list) is returned. Memq uses eq? to compare
+; obj with the elements of list, while memv uses eqv? and
+; member uses equal?.
+
+(define (list-transform-search transform)
+  (lambda (predicate)
+    (lambda (object list)
+      (do ([pair list (cdr pair)])
+          ((or (null? pair)
+               (predicate (car (transform pair)) object))
+           (if (null? pair)
+               #f
+               (transform pair)))))))
+
+(define list-search (list-transform-search (lambda (x) x)))
+(define memq   (list-search eq?))
+(define memv   (list-search eqv?))
+(define member (list-search equal?))
+
+; (assq obj alist)
+; (assv obj alist)
+; (assoc obj alist)
+; Alist (for "association list") must be a list of pairs.
+; These procedures find the first pair in alist whose car
+; field is obj, and returns that pair. If no pair in alist
+; has obj as its car, then #f (not the empty list) is
+; returned. Assq uses eq? to compare obj with the car fields
+; of the pairs in alist, while assv uses eqv? and assoc
+; uses equal?.
+
+(define assoc-list-search (list-transform-search car))
+(define assq  (assoc-list-search eq?))
+(define assv  (assoc-list-search eqv?))
+(define assoc (assoc-list-search equal?))
+
+; (map proc list1 list2 ...)
+; Returns a new list formed by applying proc to each member
+; (or set of members) of the given list(s).
+(define (map proc list1 . list2)
+  (if (null? list1)
+      list1
+      (if (null? list2)
+          (cons (proc (car list1))
+                (map proc (cdr list1)))
+          (let* ([all (cons list1 list2)]
+                 [args (map car all)]
+                 [rest (map cdr all)])
+            (cons (apply proc args)
+                  (apply map (cons proc rest)))))))
+
+; (for-each proc list1 list2 ...)
+; Calls proc once for each member of list1, passing each
+; member (or set of members if more than one list given)
+; as arguments to proc.
+(define (for-each proc list1 . list2)
+  (do ([pair list1 (cdr pair)]
+       [others list2 (map cdr others)])
+      ((null? pair) '())
+    (apply proc (cons (car pair)
+                      (map car others)))))
+
+; (foldr proc value list)
+(define (foldr proc value list)
+  (if (null? list)
+      value
+      (proc (car list)
+            (foldr proc value (cdr list)))))