rapydscript-ns 0.9.0 → 0.9.2

package/.agignore

@@ -1 +1 @@
-lib/*
+lib/*

package/.github/workflows/ci.yml

@@ -1,38 +1,38 @@
-name: CI
-on: [push, pull_request]
-env:
-    CI: 'true'
-
-jobs:
-    test:
-        name: Test on ${{ matrix.os }} LANG ${{ matrix.lang }}
-        runs-on: ${{ matrix.os }}
-        env:
-          LANG: ${{ matrix.lang }}
-          LC_ALL: ${{ matrix.lang }}
-        strategy:
-            matrix:
-                include:
-                    - { os: ubuntu-latest, lang: en_US.UTF-8 }
-                    - { os: ubuntu-latest, lang: de_DE.UTF-8}
-                    - { os: ubuntu-latest, lang: hi_IN.UTF-8 }
-
-                    - { os: macos-latest, lang: en_US.UTF-8 }
-                    - { os: windows-latest, lang: en_US.UTF-8 }
-
-        steps:
-            - name: Checkout source code
-              uses: actions/checkout@master
-              with:
-                fetch-depth: 10
-
-            - name: Set up Node ${{ matrix.node }}
-              uses: actions/setup-node@master
-
-            - name: Install deps
-              run:
-                npm install --no-optional
-
-            - name: Test
-              run:
-                npm test
+name: CI
+on: [push, pull_request]
+env:
+    CI: 'true'
+
+jobs:
+    test:
+        name: Test on ${{ matrix.os }} LANG ${{ matrix.lang }}
+        runs-on: ${{ matrix.os }}
+        env:
+          LANG: ${{ matrix.lang }}
+          LC_ALL: ${{ matrix.lang }}
+        strategy:
+            matrix:
+                include:
+                    - { os: ubuntu-latest, lang: en_US.UTF-8 }
+                    - { os: ubuntu-latest, lang: de_DE.UTF-8}
+                    - { os: ubuntu-latest, lang: hi_IN.UTF-8 }
+
+                    - { os: macos-latest, lang: en_US.UTF-8 }
+                    - { os: windows-latest, lang: en_US.UTF-8 }
+
+        steps:
+            - name: Checkout source code
+              uses: actions/checkout@master
+              with:
+                fetch-depth: 10
+
+            - name: Set up Node ${{ matrix.node }}
+              uses: actions/setup-node@master
+
+            - name: Install deps
+              run:
+                npm install --no-optional
+
+            - name: Test
+              run:
+                npm test

package/=template.pyj

@@ -1,5 +1,5 @@
-# vim:fileencoding=utf-8
-# License: BSD Copyright: %YEAR%, %USER% <%MAIL%>
-from __python__ import hash_literals
-
-%HERE%
+# vim:fileencoding=utf-8
+# License: BSD Copyright: %YEAR%, %USER% <%MAIL%>
+from __python__ import hash_literals
+
+%HERE%

package/CHANGELOG.md

@@ -1,3 +1,13 @@
+version 0.9.2
+=======================
+  * Added the contextlib standard library
+  * Added the datetime standard library
+  * Added the io standard library
+  * Added the json standard library
+  * Added support for the `debugger` keyword 
+  * Added TypeScript definitions for compiler.js
+  * Simplified ExtraBuiltins interface
+
 version 0.9.0
 =======================
   * All optional Python compatibility features are now enabled by default. See Python Feature Coverage from the README

package/HACKING.md

@@ -1,103 +1,103 @@
-Hacking the RapydScript compiler
-=================================
-
-The RapydScript compiler is written in RapydScript itself and uses the
-RapydScript import system to modularize its code. The compiler source code is
-in the `src` directory. The compiled compiler is by default in the `release`
-directory. 
-
-In order to start hacking on the compiler, run the command
-
-```
-bin/rapydscript self --complete --test
-```
-
-This will generate a build of the compiler in the `dev` directory. Now, the
-rapydscript command will automatically use this build, rather than the one in
-release. If you want to go back to the release build, simply delete the `dev`
-directory.
-
-
-Code organization
--------------------
-
-The way the compiler works, given some RapydScript source code:
-
-* The source code is lexed into a stream of tokens (`src/tokenzier.pyj`)
-
-* The tokens are parsed into a Abstract Syntax Tree (`src/parse.pyj and src/ast.pyj`)
-
-* During parsing any import statement are resolved (this is different from
-  python, where imports happen at run-time, not compile time).
-
-* The Abstract Syntax Tree is transformed into the output JavaScript (`src/output/*.pyj`)
-
-* Various bits of functionality in RapydScript depend upon the *Base Library*
-  (`src/baselib*.pyj`). This includes things like the basic container types
-  (list/set/dict) string functions such as `str.format()`, etc. The baselib
-  is automatically inserted into the start of the output JavaScript.
-
-The RapydScript standard library can be found in `src/lib`. The various tools,
-such as the linter, gettext support, the REPL, etc. are in the `tools`
-directory.
-
-Tests
---------
-
-The tests are in the test directory and can be run using the command:
-```
-rapydscript test
-```
-
-You can run individual test files by providing the name of the file, as
-
-```
-rapydscript test classes
-```
-
-Modifying the compiler
--------------------------
-
-Edit the files in the `src` directory to make your changes, then use the
-`./try` script to test them.  This script will compile an updated version of
-the compiler with your changes, if any, and use it to run the snippet of code
-you pass to it.
-
-For example:
-
-```
-./try 'print("Hello world")'
-```
-
-will compile `print ("Hello world")` and show you the output on stdout. You can
-tell it to omit the baselib, so you can focus on the output, with the `-m`
-switch, like this:
-
-```
-./try -m 'print("Hello world")'
-```
-
-You can also have it not print out the JavaScript, instead directly executing the output
-JavaScript with the `-x` switch, like this
-
-```
-./try -x 'print("Hello world")'
-```
-
-If you want to test longer sections of code, you can use the `-f` switch to
-pass in the path to a RapydScript file to compile, like this:
-
-```
-./try -f myfile.pyj
-```
-
-Once you are happy with your changes, you can build the compiler and run the
-test suite, all with a single command:
-
-```
-./build
-```
-
-This will build the compiler with the updated version of itself and then run
-the test suite. If all test pass you can commit your changes and send a pull
-request :)
+Hacking the RapydScript compiler
+=================================
+
+The RapydScript compiler is written in RapydScript itself and uses the
+RapydScript import system to modularize its code. The compiler source code is
+in the `src` directory. The compiled compiler is by default in the `release`
+directory. 
+
+In order to start hacking on the compiler, run the command
+
+```
+bin/rapydscript self --complete --test
+```
+
+This will generate a build of the compiler in the `dev` directory. Now, the
+rapydscript command will automatically use this build, rather than the one in
+release. If you want to go back to the release build, simply delete the `dev`
+directory.
+
+
+Code organization
+-------------------
+
+The way the compiler works, given some RapydScript source code:
+
+* The source code is lexed into a stream of tokens (`src/tokenzier.pyj`)
+
+* The tokens are parsed into a Abstract Syntax Tree (`src/parse.pyj and src/ast.pyj`)
+
+* During parsing any import statement are resolved (this is different from
+  python, where imports happen at run-time, not compile time).
+
+* The Abstract Syntax Tree is transformed into the output JavaScript (`src/output/*.pyj`)
+
+* Various bits of functionality in RapydScript depend upon the *Base Library*
+  (`src/baselib*.pyj`). This includes things like the basic container types
+  (list/set/dict) string functions such as `str.format()`, etc. The baselib
+  is automatically inserted into the start of the output JavaScript.
+
+The RapydScript standard library can be found in `src/lib`. The various tools,
+such as the linter, gettext support, the REPL, etc. are in the `tools`
+directory.
+
+Tests
+--------
+
+The tests are in the test directory and can be run using the command:
+```
+rapydscript test
+```
+
+You can run individual test files by providing the name of the file, as
+
+```
+rapydscript test classes
+```
+
+Modifying the compiler
+-------------------------
+
+Edit the files in the `src` directory to make your changes, then use the
+`./try` script to test them.  This script will compile an updated version of
+the compiler with your changes, if any, and use it to run the snippet of code
+you pass to it.
+
+For example:
+
+```
+./try 'print("Hello world")'
+```
+
+will compile `print ("Hello world")` and show you the output on stdout. You can
+tell it to omit the baselib, so you can focus on the output, with the `-m`
+switch, like this:
+
+```
+./try -m 'print("Hello world")'
+```
+
+You can also have it not print out the JavaScript, instead directly executing the output
+JavaScript with the `-x` switch, like this
+
+```
+./try -x 'print("Hello world")'
+```
+
+If you want to test longer sections of code, you can use the `-f` switch to
+pass in the path to a RapydScript file to compile, like this:
+
+```
+./try -f myfile.pyj
+```
+
+Once you are happy with your changes, you can build the compiler and run the
+test suite, all with a single command:
+
+```
+./build
+```
+
+This will build the compiler with the updated version of itself and then run
+the test suite. If all test pass you can commit your changes and send a pull
+request :)

package/LICENSE

@@ -1,24 +1,24 @@
-Copyright (c) 2015-, Kovid Goyal <kovid@kovidgoyal.net>
-Copyright (c) 2013-2014, Alexander Tsepkov <atsepkov@pyjeon.com>
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-
-* Redistributions of source code must retain the above copyright notice, this
-  list of conditions and the following disclaimer.
-
-* Redistributions in binary form must reproduce the above copyright notice,
-  this list of conditions and the following disclaimer in the documentation
-  and/or other materials provided with the distribution.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
-FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+Copyright (c) 2015-, Kovid Goyal <kovid@kovidgoyal.net>
+Copyright (c) 2013-2014, Alexander Tsepkov <atsepkov@pyjeon.com>
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

package/README.md

@@ -3348,6 +3348,7 @@ One of Python's main strengths is the number of libraries available to the devel
 	itertools           # count, cycle, repeat, accumulate, chain, compress, dropwhile, filterfalse,
 	                    # groupby, islice, pairwise, starmap, takewhile, zip_longest,
 	                    # product, permutations, combinations, combinations_with_replacement
+	io                  # StringIO (in-memory text stream), BytesIO (in-memory binary stream)
 
 For the most part, the logic implemented in these libraries functions identically to the Python versions. I'd be happy to include more libraries, if other members of the community want them. However, unlike most other Python-to-JavaScript compilers, RapydScript doesn't need them to be complete since there are already tons of available JavaScript libraries that it can use natively.
 
@@ -3687,7 +3688,7 @@ when the editor is torn down.
 | `stdlibFiles` | `{name: source}` | `{}` | Like `virtualFiles` but treated as stdlib — always available and never produce bad-import warnings. |
 | `dtsFiles` | `[{name, content}]` | `[]` | TypeScript `.d.ts` files loaded at startup. |
 | `loadDts` | `(name) => Promise<string>` | — | Async callback for lazy-loading `.d.ts` content on demand. |
-| `extraBuiltins` | `{name: true}` | `{}` | Extra global names that suppress undefined-symbol warnings. |
+| `extraBuiltins` | `string[]` | `[]` | Extra global names that suppress undefined-symbol warnings. |
 | `pythonFlags` | string | — | Comma-separated Python flags to enable globally (e.g. `"dict_literals,overload_getitem"`). See [Python Flags](#python-flags) above. |
 
 ### Runtime API
@@ -4016,7 +4017,7 @@ This restores the original RapydScript behavior: plain JS objects for `{}`, no o
 
 ---
 
-### Not Supported
+### Python Features That Are Not Supported
 
 | Feature                               | Notes                                                                                   |
 |---------------------------------------|-----------------------------------------------------------------------------------------|
@@ -4050,13 +4051,13 @@ Modules with a `src/lib/` implementation available are marked ✅. All others ar
 | `dataclasses` | ✅           | `@dataclass`, `field()`, `asdict()`, `astuple()`, `replace()`, `fields()`, `is_dataclass()`, `MISSING` in `src/lib/dataclasses.pyj`; `frozen=True`, `order=True`, inheritance supported; note: `field()` first positional arg is the default value (JS reserved word `default` cannot be used as a kwarg) |
 | `enum`        | ✅           | `Enum` base class in `src/lib/enum.pyj`; `.name`, `.value`, iteration, `isinstance` checks; `IntEnum`/`Flag` not available |
 | `abc`         | ✅           | `ABC`, `@abstractmethod`, `Protocol`, `@runtime_checkable`, `ABCMeta` (informational), `get_cache_token()` in `src/lib/abc.pyj`; abstract method enforcement via `__init__` guard; `ABC.register()` for virtual subclasses with isinstance support; `Symbol.hasInstance` enables structural isinstance for `@runtime_checkable` protocols; `ABCMeta` metaclass not usable (no metaclass support), use `ABC` base class instead |
-| `contextlib`  | ❌           | `contextmanager`, `suppress`, `ExitStack`, `asynccontextmanager` not available                |
+| `contextlib`  | ✅           | `AbstractContextManager`, `@contextmanager`, `closing`, `nullcontext`, `suppress`, `ExitStack` in `src/lib/contextlib.pyj`; generator-based context managers via `@contextmanager`; `asynccontextmanager` not available |
+| `datetime`    | ✅           | `date`, `time`, `datetime`, `timedelta`, `MINYEAR`, `MAXYEAR` in `src/lib/datetime.pyj`; construction, `isoformat()`, `fromisoformat()`, `today()`/`now()`, `combine()`, arithmetic (via `__add__`/`__sub__`; operators need `overload_operators`), comparisons, `replace()`, `strftime()` (%Y %m %d %H %M %S %f %A %a %B %b %j %p %I %%); no tzinfo/timezone support |
+| `json`        | ✅           | `dumps()`, `loads()`, `dump()`, `load()`, `JSONDecodeError` in `src/lib/json.pyj`; `indent`, `sort_keys`, `separators`, `dflt` (Python's `default`) callback, `object_hook`, `object_pairs_hook`, `parse_float`, `parse_int` supported; backed by the JS `JSON` global; note: `default` is a JS reserved word — use `dflt=` instead |
+| `io`          | ✅           | `StringIO`, `BytesIO`, `UnsupportedOperation` in `src/lib/io.pyj`; `read([size])`, `readline([size])`, `readlines([hint])`, `write()`, `writelines()`, `seek(pos[, whence])`, `tell()`, `truncate([pos])`, `getvalue()`, `close()`, `closed`; context manager support; `readable()`, `writable()`, `seekable()` return True; `newline` parameter accepted for API compatibility |
 | `string`      | ❌           | Character constants, `Template`, `Formatter` not available                                    |
-| `json`        | ❌           | No Python wrapper; JS `JSON.parse` / `JSON.stringify` work directly via verbatim JS           |
-| `datetime`    | ❌           | `date`, `time`, `datetime`, `timedelta` not available                                         |
 | `inspect`     | ❌           | `signature`, `getmembers`, `isfunction` etc. not available                                    |
 | `asyncio`     | ❌           | Event loop, `gather`, `sleep`, `Queue`, `Task` wrappers not available; use `async`/`await`    |
-| `io`          | ❌           | `StringIO`, `BytesIO` not available                                                           |
 | `struct`      | ❌           | Binary packing/unpacking not available                                                        |
 | `hashlib`     | ❌           | MD5, SHA-256 etc. not available; use Web Crypto API via verbatim JS                           |
 | `hmac`        | ❌           | Keyed hashing not available                                                                   |

package/TODO.md

@@ -15,7 +15,122 @@
 - examples of using js libraries in rapydscript in readme?
 
 
-I would like you to add support for [`vars()` / `locals()` / `globals()`  ] to rapydscript. It should have the same syntax as the Python implementation, and be transpiled into equivalent javascript. Please ensure with unit tests that it transpiles and the output JS runs correctly, and that the language service correctly handles it in parsed code. Please make sure it works in the web-repl too. Please also update the README if it has any outdated info about this, and the PYTHON_FEATURE_COVERAGE report. Please also add a simple example to the bottom of the TODO document using this feature (make no other changes to that file).
+I would like you to add support for [the python io.stringIO and io.bytesio library ] to rapydscript. It should have the same syntax as the Python implementation, and be transpiled into equivalent javascript. Please ensure with unit tests that it transpiles and the output JS runs correctly, and that the language service correctly handles it in parsed code. Please make sure it works in the web-repl too. Please also update the README if it has any outdated info about this, and the PYTHON_FEATURE_COVERAGE report. Please also add a simple example to the bottom of the TODO document using this feature (make no other changes to that file).
 
 
+```py
+# Over-engineered RapydScript hacking script — Python features on parade
 
+class HackAction:
+    """Enum-like action namespace."""
+    WEAKEN = 'weaken'
+    GROW   = 'grow'
+    HACK   = 'hack'
+
+    @staticmethod
+    def all():
+        return [HackAction.WEAKEN, HackAction.GROW, HackAction.HACK]
+
+
+class ServerConfig:
+    """Data-class-style target config with property + fluent builder."""
+
+    def __init__(self, name, money_factor=1.0, sec_buffer=0.0):
+        self._name        = name
+        self.money_factor = money_factor
+        self.sec_buffer   = sec_buffer
+        self._tags        = {'auto', 'hack'}        # set literal
+
+    @property
+    def name(self):
+        return self._name
+
+    def __repr__(self):
+        return "ServerConfig(name={}, tags={})".format(
+            repr(self._name), sorted(list(self._tags))
+        )
+
+    def add_tags(self, *tags):                      # *args + fluent return
+        for t in tags:
+            self._tags.add(t)
+        return self
+
+
+def decide_action(ns, target, money_thresh, sec_thresh):
+    """next() + generator expression to pick highest-priority action."""
+    checks = [
+        (ns.getServerSecurityLevel(target)  > sec_thresh,   HackAction.WEAKEN),
+        (ns.getServerMoneyAvailable(target) < money_thresh, HackAction.GROW),
+    ]
+    return next((action for cond, action in checks if cond), HackAction.HACK)
+
+
+async def breach(ns, target, exploits):
+    """Apply available exploits via (exe, fn) tuple list, nuke, return summary."""
+    # list of (exe filename, bound ns method) tuples
+    exe_methods = [
+        ('BruteSSH.exe',  ns.brutessh),
+        ('FTPCrack.exe',  ns.ftpcrack),
+        ('relaySMTP.exe', ns.relaysmtp),
+    ]
+    applied = [fn for exe, fn in exe_methods if ns.fileExists(exe, "home")]
+    missing = [exe for exe, fn in exe_methods if not ns.fileExists(exe, "home")]
+
+    if missing:
+        ns.tprint("Missing: {}".format(', '.join(missing)))
+
+    for fn in applied:
+        fn(target)
+    ns.nuke(target)
+
+    return {'applied': len(applied), 'total': len(exploits),
+            'rooted': ns.hasRootAccess(target)}
+
+
+async def main(ns):
+    config = ServerConfig("n00dles").add_tags('target', 'beginner')  # fluent
+    ns.tprint(repr(config))                                           # __repr__
+
+    target       = config.name                                        # @property
+    money_thresh = ns.getServerMaxMoney(target)         * config.money_factor
+    sec_thresh   = ns.getServerMinSecurityLevel(target) + config.sec_buffer
+
+    def by_name_len(p):                             # named key fn (required for
+        return len(p[0])                            # anonymous fns in call args)
+    exploits = sorted([
+        ('brutessh',  'BruteSSH.exe'),
+        ('ftpcrack',  'FTPCrack.exe'),
+        ('relaysmtp', 'relaySMTP.exe'),
+    ], key=by_name_len)
+
+    result = await breach(ns, target, exploits)
+    ns.tprint("Breach: {applied}/{total} applied, rooted={rooted}".format(**result))
+    assert result['rooted'], "Root access denied on {}!".format(target)
+
+    dispatch = {                                    # dict literal
+        'weaken': ns.weaken,
+        'grow':   ns.grow,
+        'hack':   ns.hack,
+    }
+    stats = {a: 0 for a in HackAction.all()}        # dict comprehension
+
+    cycle = 0
+    while True:
+        cycle += 1
+        action = decide_action(ns, target, money_thresh, sec_thresh)
+        stats[action] += 1
+
+        if cycle % 10 is 0:                         # periodic summary
+            counts  = [stats[a] for a in HackAction.all()]
+            summary = ', '.join(["{}: {}".format(a, v)
+                                 for a, v in zip(HackAction.all(), counts)])
+            ns.tprint("Cycle {} | {}  any={} all={}".format(
+                cycle, summary,
+                any([v > 0 for v in counts]),       # any()
+                all([v > 0 for v in counts])        # all()
+            ))
+
+        await dispatch[action](target)
+```
+
+`ρσ_unpack is not defined`

package/add-toc-to-readme

@@ -1,2 +1,2 @@
-#!/bin/sh
-exec node_modules/doctoc/doctoc.js --title '**Contents**' README.md
+#!/bin/sh
+exec node_modules/doctoc/doctoc.js --title '**Contents**' README.md