nydp 0.0.6 → 0.0.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 56ae5257ab6792eeaef112c3e81991dae7c43ce1
-  data.tar.gz: 327b5770bc8f1c442202d340f5392288e58efdbe
+  metadata.gz: 1cb1c4582106367e04bc3ac5163db7401782331c
+  data.tar.gz: 3c8be4535a704cdbe02674796f4068d6f079880c
 SHA512:
-  metadata.gz: 65b2df58e4d78c39e8ea8aacb1e3e0298269b5f2b03f6d9fb11e3ce44bf45012cd35b7b397a74de46d6e10f36a1b64be179c9efae28d5ca709959c50c9574e2b
-  data.tar.gz: 9e787875c9011054dbaa9b416b84ef162f798bea1f5c33fde9a834322fc54e8435d537a37ce7a7e62e1972db95103aca67a3336b2e727bf7da07b94953af34f8
+  metadata.gz: 7523046a3da0fb43634736645674566d77c10a3178b911765d849503c9b66cdb32e9b17810365795bddc7f2bf5fe655d24c1e2f10b6253054987cdfdbe5a6a16
+  data.tar.gz: 82f1f13fb1e3badf6f80c8a9ca6077019b01c0f8e571c61b1e1ade4ba1c640c4c8945f9e3d77442d69e77382e5ecee60587d51b9010ff03ec10ee4bb422664c8

data/.gitattributes

@@ -0,0 +1 @@
+*.nydp linguist-language=NYDP

data/README.md

@@ -1,35 +1,252 @@
-# Nydp
+# NYDP
+
+NYDP is a new LISP dialect, much inspired by [Arc](http://arclanguage.org/), and implemented in Ruby.
 
 NYDP is "Not Your Daddy's Parentheses", a reference to [Xkcd 297](http://xkcd.com/297/) (itself a reference
 to Star Wars), as well as to the meme [Not Your Daddy's Q](http://tvtropes.org/pmwiki/pmwiki.php/Main/NotYourDaddysX), where Q is a modern,
 improved Q unlike the Q your daddy used. "NYDP" also shamelessly piggypacks on the
 catchiness and popularity of the [NYPD](https://en.wikipedia.org/wiki/NYPD_Blue) abbreviation ("New York Police Department",
-for those who have no interest in popular US TV or authoritarian politics).
+for those who have no interest in popular US politics or TV).
+
+We do not wish to suggest by "Not Your Daddy's Parentheses" that Common Lisp, Scheme, Racket, Arc, Clojure or your favourite other lisp are somehow old-fashioned, inferior, or in need of improvement in any way.
+
+The goal of NYDP is to allow untrusted users run sandboxed server-side scripts. By default, NYDP provides no system access :
+
+* no file functions
+* no network functions
+* no IO other than $stdin and $stdout
+* no process functions
+* no threading functions
+* no ruby calls
+
+[Peruse NYDP's features here](lib/lisp/tests) in the `tests` directory.
+
+## Running
+
+#### Get a REPL :
+
+```Shell
+$ bundle exec bin/nydp
+welcome to nydp
+^D to exit
+nydp >
+```
+
+The REPL uses the readline library so you can use up- and down-arrows to navigate history.
+
+#### Invoking from Ruby
+
+Suppose you want to invoke the function named `question` with some arguments. Do this:
+
+```ruby
+ns     = Nydp.build_nydp     # keep this for later re-use, it's expensive to set up
 
-Macro-expansion is not built-in to the interpreter; however, the compiler will invoke 'pre-compile before compiling, passing
-the expression to compile as an argument. You can override 'pre-compile to transform the expression in any way you wish. By default,
-nydp provides an implementation of 'pre-compile that performs macro-expansion.
+answer = Nydp.apply_function ns, :question, :life, ("The Universe" and everything())
 
+==> 42
 ```
+
+`ns` is just a plain old ruby hash, mapping ruby symbols to nydp symbols for quick lookup at nydp compile-time. The nydp symbols maintain the values of global variables, including all builtin functions and any other functions defined using `def`.
+
+You can maintain multiple `ns` instances without mutual interference. In other words, assigning global variables while one `ns` is in scope will not affect the values of variables in any other `ns` (unless you've specifically arranged it to be so by duplicating namespaces or some such sorcery).
+
+
+## Different from Arc :
+
+#### 1. Macro-expansion runs in lisp
+
+After parsing its input, NYDP passes the result as an argument to the `pre-compile` function. This is where things get a little bit circular: initially, `pre-compile` is a builtin function that just returns its argument. `pre-compile` bootstraps itself into existence in [boot.nydp](lib/lisp/boot.nydp).
+
+You can override `pre-compile` to transform the expression in any way you wish. By default, the `boot.nydp` implementation of `pre-compile` performs macro-expansion.
+
+
+
+```lisp
 (def pre-compile (expr)
   (map pre-compile
     (if (mac-names (car expr))
         (pre-compile (mac-expand (car expr) (cdr expr)))
         expr)))
+
+(mac yoyo (thing) `(do-yoyo ,thing))
+
+nydp > (pre-compile '(yoyo 42))
+
+==> (do-yoyo 42)
+```
+
+
+#### 2. Special symbol syntax
+
+The parser detects syntax embedded in smybol names and emits a form whose first element names the syntax used. Here's an example:
+
+```lisp
+
+nydp > (parse "x.y")
+
+==> (dot-syntax x y)
+
+nydp > (parse "$x x$ x$x $x$ $$")
+
+==> (dollar-syntax || x)     ; '|| is the empty symbol.
+==> (dollar-syntax x ||)
+==> (dollar-syntax x x)
+==> (dollar-syntax || x ||)
+==> (dollar-syntax || || ||)
+
+nydp > (parse "!foo")
+
+==> (bang-syntax || foo)
+
+nydp > (parse "!x.$y")
+
+==> (bang-syntax || (dot-syntax x (dollar-syntax || y)))
+
+```
+
+Nydp provides macros for some but not all possible special syntax
+
+```lisp
+nydp > (pre-compile 'x.y)
+
+==> (hash-get x 'y) ; 'dot-syntax is a macro that expands to perform hash lookups
+
+nydp > (pre-compile 'x.y.z)
+
+==> (hash-get (hash-get x 'y) 'z)
+
+
+nydp > (pre-compile '!eq?)
+
+==> (fn args (no (apply eq? args)))
+
+nydp > (pre-compile '(!eq? a b))
+
+==> ((fn args (no (apply eq? args))) a b) ; equivalent to (no (eq? a b))
 ```
 
-; blah blah
+Look for `SYMBOL_OPERATORS` in [parser.rb](lib/nydp/parser.rb) to see which syntax is recognised and in which order. The order of these definitions defines special-syntax-operator precedence.
+
+#### 3. Special list syntax
+
+The parser detects alternative list delimiters
+
+```lisp
+nydp > (parse "{ a 1 b 2 }")
+
+==> (brace-list a 1 b 2)
 
 ```
-  ==> (comment "blah blah")
 
-  ==> (mac comment (txt) nil)
+`brace-list` is a macro that expands to create a hash literal. It assumes every (2n+1)th items are literal symbol keys, and every (2(n+1))th item is the corresponding value which is evaluated at run time.
+
+```lisp
+
+nydp > { a 1 b (author-name) }
+
+==> {a=>1, b=>"conanite"}
+
+```
+
+
+
+#### 4. Sensible, nestable string interpolation
+
+The parser detects lisp code inside strings. When this happens, instead of emitting a string literal, the parser emits a form whose car is the symbol `string-pieces`.
+
+```lisp
+nydp > (parse "\"foo\"")
+
+==> "foo"
+
+nydp > (let bar "Mister Nice Guy" "hello, ~bar")
+
+==> hello, Mister Nice Guy
+
+; this is a more tricky example because we need to make a string with an interpolation token in it
+
+nydp > (let s (joinstr "" "\"hello, " '~ "world\"") (parse s))
+
+==> (string-pieces "hello, " world "") ; "hello, ", followed by the interpolation 'world, followed by the empty string after 'world
+
+nydp > (def also (str) "\nAND ALSO, ~str")
+nydp > (with (a 1 b 2)
+         (p "Consider ~a : the first thing,
+              ~(also "Consider ~b : the second thing,
+              ~(also "Consider ~(+ a b), the third (and final) thing")")"))
+
+==> Consider 1 : the first thing,
+==> AND ALSO, Consider 2 : the second thing,
+==> AND ALSO, Consider 3, the third (and final) thing
 ```
 
-We do not wish to suggest by "Not Your Daddy's Parentheses" that Common Lisp,
-Scheme, Racket, Arc, Clojure or your favourite other lisp are somehow
-old-fashioned, inferior, or in need of improvement in any way.
+By default, `string-pieces` is a function that just concatenates the string value of its arguments. You can redefine it as a macro to perform more fun stuff, or you can detect it within another macro to do extra-special stuff with it.
+
+
+#### 5. No continuations.
+
+Sorry. While technically possible ... why bother?
+
+#### 6. No argument destructuring
 
+However, this doesn't need to be built-in, it can be done with macros alone.
+
+
+## Besides that, what can Nydp do?
+
+#### 1. Functions and variables exist in the same namespace.
+#### 2. Macros are maintained in a hash called 'macs in the main namespace.
+#### 3. General [tail call elimination](https://en.wikipedia.org/wiki/Tail_call) allowing recursion without stack overflow in some cases.
+#### 4. 'if like Arc:
+
+```lisp
+(if a b c d e) ; equivalent to ruby :
+```
+
+```ruby
+if    a
+      b
+elsif c
+      d
+else  e
+```
+
+#### 5. Lexically scoped, but with macros to define dynamic variables backed by ruby threadlocals.
+
+```lisp
+nydp> (dynamic foo)
+
+nydp> (def do-something () (+ (foo) 1))
+
+nydp> (w/foo 99 (do-something))
+
+==> 100
+
+nydp> (foo)
+
+==> nil
+```
+
+#### 6. Basic error handling
+
+```lisp
+nydp> (on-err (p "error")
+        (ensure (p "make sure this happens")
+          (/ 1 0)))
+
+make sure this happens
+error
+```
+
+#### 7 Intercept comments
+
+```lisp
+nydp > (parse "; blah blah")
+
+==> (comment "blah blah")
+
+By default, `comment` is a macro that expands to nil. If you have a better idea, go for it. (doc-comments for example)
+```
 
 ## Installation
 
@@ -45,9 +262,6 @@ Or install it yourself as:
 
     $ gem install nydp
 
-## Usage
-
-TODO: Write usage instructions here
 
 ## Contributing
 

data/lib/lisp/bm.nydp

@@ -0,0 +1,18 @@
+(def bm-pythag ()
+  (for i 1 100
+    (for j 1 100
+      (sqrt (+ (* i i) (* j j))))))
+
+(def bmf (f n)
+  (for b 1 n (f)))
+
+(def bm (f n)
+  (let time (millisecs)
+    (bmf f n)
+    (let elapsed (- (millisecs) time)
+      (p "took: ~elapsed ms")
+      (p "~n iterations, ~(/ elapsed n) ms per iteration")))
+  nil)
+
+(def rbs ()
+  (bm bm-pythag 20))

data/lib/lisp/boot.nydp

@@ -1,12 +1,17 @@
 ; -*- lisp -*-
 
+;;
+;; Acknowledgements to Paul Graham. Some nydp features defined in this file (including,
+;; but not limited to, 'do, 'rfn, 'loop, 'for) are stolen directly from arc.arc
+;;
+
 (assign last-cons (fn (xs)
                       (cond (pair? (cdr xs))
                             (last-cons (cdr xs))
                             xs)))
 
 
-(assign append-list! (fn (list-1 list-2)
+(assign append-list (fn (list-1 list-2)
                          (cdr-set (last-cons list-1) list-2)
                          list-1))
 
@@ -15,7 +20,7 @@
 (assign caar  (fn (arg) (car (car arg))))
 (assign cadr  (fn (arg) (car (cdr arg))))
 (assign cddr  (fn (arg) (cdr (cdr arg))))
-(assign   no  (fn (arg) (eq? arg nil)))
+(assign   no  (fn (arg) (cond arg nil t)))
 (assign just  (fn (arg) arg))
 (assign pargs (fn args (apply p args) (last args)))
 
@@ -145,6 +150,14 @@
                        `(cond ,(car args) ,(cadr args)))
                  (car args))))
 
+(mac bang-syntax (pfx . rest)
+  (if (no (eq? pfx '||))
+      (error "Irregular ! syntax: got prefix ~(inspect pfx) in ~(joinstr "!" (cons pfx rest))"))
+  (if (cdr rest)
+      (error "Irregular ! syntax: got suffix ~(inspect (cdr rest)) in ~(joinstr "!" (cons pfx rest))"))
+  `(fn args
+     (no (apply ,(car rest) args))))
+
 (mac and args
   (if args
       (if (cdr args)
@@ -215,13 +228,12 @@
                (flattenize things))
           acc))
 
-(def joinstr (txt things)
-     (if (no (pair? things))
-         (error "joinstr : 'things is a %%(type-of things) expected a list : %%(inspect things)"))
-     (apply +
-            (to-string (car things))
-            (flatten (zip (map (fn (_) txt) (cdr things))
-                          (map to-string (cdr things))))))
+(def joinstr (txt . things)
+     (let joinables (flatten things)
+       (apply +
+              (to-string (car joinables))
+              (flatten (zip (map (fn (_) txt) (cdr joinables))
+                            (map to-string (cdr joinables)))))))
 
 (let uniq-counter 0
      (def uniq (prefix)
@@ -263,6 +275,7 @@
               (iso (cdr x) (cdr y)))))
 
 (def isa   (type   obj) (eq? (type-of obj) type))
+(def sym? (arg) (isa 'symbol arg))
 (mac just (arg) arg)
 (def quotify (arg) `(quote ,arg))
 
@@ -317,5 +330,51 @@
 (mac = (name value)
   (if (isa 'symbol name)
       `(assign ,name ,value)
-    (caris 'dot-syntax name)
+      (caris 'dot-syntax name)
       (dot-syntax-assignment (cdr name) value)))
+
+(def brace-list-hash-key (k)
+  (if (isa 'symbol k)      `(quote ,k)
+      (caris 'unquote k)   (cadr k)
+      k))
+
+(def brace-list-build-hash (args)
+  (w/uniq hash
+    (let mappings (pairs args)
+      `(let ,hash (hash)
+            ,@(map (fn (m) `(hash-set ,hash ,(brace-list-hash-key (car m)) ,(cadr m))) mappings)
+            ,hash))))
+
+(mac brace-list args
+  (if (no (cdr args))
+      (car args)
+      (brace-list-build-hash args)))
+
+(mac on-err (handler . body)
+  `(handle-error (fn (err) ,handler)
+                 (fn ()    ,@body)))
+
+(mac ensure (protection . body)
+  `(ensuring (fn () ,protection)
+             (fn ()    ,@body)))
+
+(mac rfn (name parms . body)
+  `(let ,name nil
+     (assign ,name (fn ,parms ,@body))))
+
+(mac afn (parms . body)
+  `(rfn self ,parms ,@body))
+
+(mac loop (start test update . body)
+  (w/uniq (gfn gparm)
+    `(do ,start
+         ((rfn ,gfn (,gparm)
+            (if ,gparm
+                (do ,@body ,update (,gfn ,test))))
+          ,test))))
+
+(mac for (v init max . body)
+  (w/uniq (gi gm)
+    `(with (,v nil ,gi ,init ,gm (+ ,max 1))
+       (loop (assign ,v ,gi) (< ,v ,gm) (assign ,v (+ ,v 1))
+         ,@body))))

data/lib/lisp/tests/boot-tests.nydp

@@ -20,8 +20,39 @@
 (make-plus +seven 7)
 (make-plus +eleven 11)
 
+;
+; another contrived example to check deeply nested lexical scoping
+;
+(let test-a0 "a0"
+  (def test-foo (f0 f1)
+    (with (w0 "w0" w1 "w1" w2 "w2" w3 "w3")
+      (let f (fn (x0) (joinstr " " test-a0 x0 f0 x0 f1))
+        (map f (list w0 w1 w2 w3))))))
+
 (register-test
  '(suite "Boot Tests"
+         (suite "hashtables"
+                ("build a hash table from brace-list syntax"
+                 (let hsh { foo 1 bar 2 }
+                      (list 'foo hsh.foo 'bar hsh.bar))
+                 (foo 1 bar 2))
+
+                ("single-item brace list is just the thing itself"
+                 (let zi 10 "finds the ~{zi}th item")
+                 "finds the 10th item")
+
+                ("unquotes hash keys"
+                 (with (zi 'foo chi 'bar yi 'grr)
+                      (let hsh { ,zi 10 ,chi 11 ,yi 12 }
+                           (list zi hsh.foo chi hsh.bar yi hsh.grr)))
+                 (foo 10 bar 11 grr 12))
+
+                ("allows literal and invocation hash keys"
+                 (with (zi "hello" chi "world")
+                      (let hsh { (joinstr " " zi chi) 10 "yesterday" 11 }
+                           (list "hello world" (hash-get hsh "hello world") "yesterday" (hash-get hsh "yesterday"))))
+                 ("hello world" 10 "yesterday" 11)))
+
          (suite "list management"
                 ("'pair breaks a list into pairs"
                  (pairs '(1 a 2 b 3 c))
@@ -43,6 +74,18 @@
                  (joinstr "" '("foo" "bar" "bax"))
                  "foobarbax")
 
+                ("joins separate elements into a string"
+                 (joinstr "/" "foo" "bar" "bax")
+                 "foo/bar/bax")
+
+                ("joins a single thing"
+                 (joinstr "/" "foo")
+                 "foo")
+
+                ("joins nested and separate elements into a string"
+                 (joinstr "/" "foo" "bar" '(twiddle diddle) "bax")
+                 "foo/bar/twiddle/diddle/bax")
+
                 ("joins elements into a string"
                  (joinstr " - " '(1 2 3))
                  "1 - 2 - 3")
@@ -54,9 +97,30 @@
          (suite "map"
                 ("maps a function over a list of numbers"
                  (map (fn (x) (* x x)) '(1 2 3))
-                 (1 4 9)))
+                 (1 4 9))
+
+                ("maps a string join function over a list of strings"
+                 (test-foo "x" "y")
+                 ("a0 w0 x w0 y" "a0 w1 x w1 y" "a0 w2 x w2 y" "a0 w3 x w3 y"))
 
          (suite "pre-compile"
+                (suite "bang-syntax"
+                       ("expansion"
+                        (pre-compile '(!eq? a b))
+                        ((fn args (no (apply eq? args))) a b))
+
+                       ("bang-syntax for 'eq?"
+                        (!eq? 1 2)
+                        t)
+
+                       ("bang-syntax for 'caris"
+                        (!caris 'foo '(foo bar))
+                        nil)
+
+                       ("bang-syntax for 'caris"
+                        (!caris 'foo '(zozo foo bar))
+                        t))
+
                 ("expands 'let"
                  (do
                      (def x+3*z (x y)