@lewin671/python-vm 0.1.1 → 0.1.3

package/README.md

@@ -1,93 +1,94 @@
 # @lewin671/python-vm
 
-A Python compiler and interpreter implemented in TypeScript. The compiler currently packages the AST into a bytecode container that the VM executes.
+[![License](https://img.shields.io/github/license/Lewin671/python-compiler-ts)](https://github.com/Lewin671/python-compiler-ts/blob/main/LICENSE)
+[![NPM Version](https://img.shields.io/npm/v/@lewin671/python-vm)](https://www.npmjs.com/package/@lewin671/python-vm)
+
+A high-performance Python compiler and Virtual Machine (VM) implemented entirely in TypeScript. This project aims to provide a robust, Python-compliant execution environment within the JavaScript ecosystem, featuring a complete compilation pipeline from source code to bytecode.
 
 [简体中文](README_zh-CN.md)
 
-## Features
+## 🚀 Key Highlights
 
-- [x] CLI entry point for running `.py` files
-- [x] Lexer with indentation handling, numbers, strings (including f-strings), keywords, and operators
-- [x] Parser that builds ASTs for statements and expressions (functions, classes, loops, comprehensions, exceptions, and more)
-- [x] Bytecode compiler scaffold that passes ASTs to the VM
-- [x] AST-based virtual machine with scopes, control flow, functions, classes, generators, context managers, and exceptions
-- [x] Python data structures including lists, tuples, dicts, sets, slicing, and comprehensions
-- [x] Built-ins:
-  - Type/conversion: int, float, str, bool, list, tuple, set, type, isinstance
-  - Iteration: range, enumerate, zip, sorted, reversed, map, filter, next
-  - Math/util: abs, round, sum, min, max
-  - I/O: print, open
-- [x] Example scripts and Vitest suite that compare output against system Python
+- **Advanced Compilation Pipeline**: Moves beyond simple interpretation by implementing a multi-stage pipeline: Source → Tokens → AST → Control Flow Graph (CFG) → Linear Bytecode → VM.
+- **Python-Strict Semantics**: Carefully implemented data structures (`PyDict`, `PySet`, `PyList`) that strictly follow Python's rules for equality, hashing, and numeric types (including BigInt for arbitrary-precision integers and NaN handling).
+- **Comprehensive Language Support**:
+  - **Core**: Full support for functions, classes, closures, and decorators.
+  - **Modern Features**: Includes `match` statements (Structural Pattern Matching), `with` statements (Context Managers), and `try/except/finally` blocks.
+  - **Control Flow**: Robust handling of generators (`yield`), list/dict/set comprehensions, and nested scopes (`global`, `nonlocal`).
+- **Production-Ready Tooling**: Includes a high-fidelity Lexer with indentation/dedentation logic, a recursive descent Parser, and a stack-based VM.
 
-## Getting Started
+## 🛠 Features
 
-### Requirements
+### Compiler & VM
+- [x] **Lexer**: Handles complex Python indentation, f-strings, and multi-line literals.
+- [x] **Parser**: Generates a typed AST supporting a wide subset of Python 3.10+ syntax.
+- [x] **CFG Builder**: Optimizes code structures into a Control Flow Graph before bytecode generation.
+- [x] **Bytecode Virtual Machine**: A stack-based execution engine with local/global scope management.
+- [x] **Exception System**: Full traceback support and Python-compliant exception hierarchy.
 
-- Node.js 18+
-- npm
-- Python 3 available on your PATH for tests (set `PYTHON=python3` if needed)
+### Standard Library & Built-ins
+- **Data Types**: `int`, `float`, `str`, `bool`, `list`, `tuple`, `dict`, `set`, `None`.
+- **Iteration**: `range`, `enumerate`, `zip`, `reversed`, `map`, `filter`, `sorted`.
+- **Utilities**: `abs`, `round`, `sum`, `min`, `max`, `isinstance`, `type`, `print`, `open`, `next`.
 
-### Install
+## 📦 Installation
 
 ```bash
-npm install
+npm install @lewin671/python-vm
 ```
 
-### Build
+## 📖 Usage
 
-```bash
-npm run build
-```
+### Running via CLI
 
-### Test
-
-```bash
-npm test
-```
-
-If the tests cannot find Python, set the environment variable before running them, for example: `export PYTHON=python3`.
-
-### Run
+After cloning the repository, you can run Python files directly:
 
 ```bash
 npm run build
 npm start -- examples/hello.py
 ```
 
-Or run directly:
+### AOT Compilation (Ahead-Of-Time)
+
+You can compile Python source files to a compressed binary bytecode format (`.pyc`) and execute them later. This avoids parsing overhead at runtime.
 
 ```bash
-node dist/index.js examples/hello.py
-```
+# 1. Compile source to bytecode
+npm start -- compile examples/hello.py hello.pyc
 
-### Use in TypeScript
+# 2. Run the bytecode directly
+npm start -- run hello.pyc
+```
 
-After `npm run build`, you can call the compiler from TypeScript. When consuming the package, import from `@lewin671/python-vm`. For local development in this repo, import from `./dist` instead.
+### Using in your TypeScript project
 
 ```ts
 import { PythonCompiler } from '@lewin671/python-vm';
 
 const compiler = new PythonCompiler();
-const result = compiler.run('print("Hello from TypeScript")');
 
-console.log(result);
-```
+// Execute code directly
+const result = compiler.run(`
+def greet(name):
+    return f"Hello, {name}!"
 
-## Project Structure
+result = [greet(x) for x in ["World", "TypeScript"]]
+print(result)
+`);
 
+// Or run a file
+// compiler.runFile('./script.py');
 ```
-@lewin671/python-vm/
-├── dist/                # Compiled output
-├── examples/            # Sample Python programs used by tests
-├── src/
-│   ├── compiler.ts      # Public PythonCompiler API
-│   ├── compiler_module/ # Bytecode compiler scaffold
-│   ├── index.ts         # CLI entry + exports
-│   ├── lexer/           # Tokenizer
-│   ├── parser/          # AST parser
-│   ├── types/           # Tokens, AST, bytecode types
-│   └── vm/              # AST interpreter (virtual machine)
-├── tests/               # Vitest suites comparing against CPython
-├── package.json
-└── tsconfig.json
+
+## 🧪 Testing and Correctness
+
+Correctness is a top priority. The project includes an extensive test suite using **Vitest** that compares the VM output against the system's CPython interpreter for parity.
+
+```bash
+# Run all tests (requires Python 3 installed locally)
+npm test
 ```
+
+## ⚖️ License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

package/README_zh-CN.md

@@ -1,93 +1,94 @@
 # @lewin671/python-vm
 
-一个使用 TypeScript 实现的 Python 编译器与解释器。目前编译器会将 AST 打包到字节码容器中，由虚拟机执行。
+[![License](https://img.shields.io/github/license/Lewin671/python-compiler-ts)](https://github.com/Lewin671/python-compiler-ts/blob/main/LICENSE)
+[![NPM Version](https://img.shields.io/npm/v/@lewin671/python-vm)](https://www.npmjs.com/package/@lewin671/python-vm)
+
+一个完全使用 TypeScript 实现的高性能 Python 编译器与虚拟机 (VM)。本项目旨在为 JavaScript 生态提供一个鲁棒且符合 Python 语义的执行环境，包含从源码到字节码的完整编译流水线。
 
 [English](README.md)
 
-## 功能特性
+## 🚀 项目亮点
 
-- [x] CLI 入口，用于运行 `.py` 文件
-- [x] 具备缩进处理、数字/字符串（含 f-string）、关键字与运算符的词法分析器
-- [x] 语法分析器生成 AST，覆盖函数、类、循环、推导式、异常等语句与表达式
-- [x] 字节码编译器框架，将 AST 交给虚拟机执行
-- [x] 基于 AST 的虚拟机，支持作用域、控制流、函数、类、生成器、上下文管理器与异常
-- [x] Python 数据结构支持：list、tuple、dict、set、切片与推导式
-- [x] 内置函数：
-  - 类型/转换：int、float、str、bool、list、tuple、set、type、isinstance
-  - 迭代相关：range、enumerate、zip、sorted、reversed、map、filter、next
-  - 数值/工具：abs、round、sum、min、max
-  - 输入输出：print、open
-- [x] 示例脚本与 Vitest 测试，输出对比系统 Python
+- **先进的编译流水线**：超越简单的解释执行，实现了多阶段编译流程：源码 → Token 流 → AST (抽象语法树) → CFG (控制流图) → 线性字节码 → 虚拟机执行。
+- **严格的 Python 语义**：精心实现的数据结构（`PyDict`, `PySet`, `PyList`）严格遵循 Python 关于等值性、哈希计算以及数值类型的规则（支持大整数 BigInt 和完整的 NaN 处理逻辑）。
+- **全面的语言支持**：
+  - **核心语法**：完整支持函数、类、闭包及装饰器。
+  - **现代特性**：包括 `match` 语句（结构化模式匹配）、`with` 语句（上下文管理器）以及 `try/except/finally` 异常处理块。
+  - **控制流**：支持生成器 (`yield`)、列表/字典/集合推导式，以及复杂的嵌套作用域（`global`, `nonlocal`）。
+- **工程化实现**：配备了支持缩进逻辑的高精度词法分析器 (Lexer)、递归下降语法分析器 (Parser) 以及一个高效的基于栈的虚拟机 (VM)。
 
-## 快速开始
+## 🛠 功能特性
 
-### 环境要求
+### 编译器与虚拟机
+- [x] **词法分析器 (Lexer)**：处理复杂的 Python 缩进/反缩进、f-strings 以及多行字面量。
+- [x] **语法分析器 (Parser)**：生成的类型化 AST 覆盖了 Python 3.10+ 的绝大部分常用语法。
+- [x] **控制流图构建器 (CFG Builder)**：在生成字节码前优化代码结构。
+- [x] **字节码虚拟机**：支持局部/全局作用域管理、调用栈以及高效的指令集执行。
+- [x] **异常系统**：提供完整的 Traceback 支持和符合 Python 规范的异常层级。
 
-- Node.js 18+
-- npm
-- Python 3（运行测试需要，确保在 PATH 中；必要时设置 `PYTHON=python3`）
+### 标准库与内置函数
+- **数据类型**：`int` (任意精度), `float`, `str`, `bool`, `list`, `tuple`, `dict`, `set`, `None`。
+- **迭代工具**：`range`, `enumerate`, `zip`, `reversed`, `map`, `filter`, `sorted`。
+- **内置工具**：`abs`, `round`, `sum`, `min`, `max`, `isinstance`, `type`, `print`, `open`, `next`。
 
-### 安装依赖
+## 📦 安装使用
 
 ```bash
-npm install
+npm install @lewin671/python-vm
 ```
 
-### 构建
+## 📖 使用指南
 
-```bash
-npm run build
-```
+### 通过命令行运行
 
-### 测试
-
-```bash
-npm test
-```
-
-如果测试无法找到 Python，可先设置环境变量，例如：`export PYTHON=python3`。
-
-### 运行
+克隆仓库后，您可以直接运行 Python 文件：
 
 ```bash
 npm run build
 npm start -- examples/hello.py
 ```
 
-或直接运行：
+### AOT 预编译 (Ahead-Of-Time)
+
+您可以将 Python 源代码编译为压缩的二进制字节码格式 (`.pyc`)，并在后续直接加载执行，从而避免运行时的解析开销。
 
 ```bash
-node dist/index.js examples/hello.py
-```
+# 1. 将源码编译为字节码
+npm start -- compile examples/hello.py hello.pyc
 
-### 在 TypeScript 中调用
+# 2. 直接运行字节码文件
+npm start -- run hello.pyc
+```
 
-运行 `npm run build` 后，可以在 TypeScript 中这样调用。作为依赖使用时请从 `@lewin671/python-vm` 导入，本仓库本地开发时则从 `./dist` 导入。
+### 在 TypeScript 项目中调用
 
 ```ts
 import { PythonCompiler } from '@lewin671/python-vm';
 
 const compiler = new PythonCompiler();
-const result = compiler.run('print("Hello from TypeScript")');
 
-console.log(result);
-```
+// 直接执行代码
+const result = compiler.run(`
+def greet(name):
+    return f"Hello, {name}!"
 
-## 项目结构
+result = [greet(x) for x in ["World", "TypeScript"]]
+print(result)
+`);
 
+// 或者运行文件
+// compiler.runFile('./script.py');
 ```
-@lewin671/python-vm/
-├── dist/                # 编译输出
-├── examples/            # 测试用 Python 示例
-├── src/
-│   ├── compiler.ts      # PythonCompiler 公共 API
-│   ├── compiler_module/ # 字节码编译器框架
-│   ├── index.ts         # CLI 入口 + 导出
-│   ├── lexer/           # 词法分析器
-│   ├── parser/          # 语法分析器
-│   ├── types/           # Token/AST/字节码类型
-│   └── vm/              # AST 解释执行器
-├── tests/               # 与 CPython 对比的 Vitest 测试
-├── package.json
-└── tsconfig.json
+
+## 🧪 测试与正确性
+
+正确性是本项目的核心指标。我们使用 **Vitest** 构建了庞大的测试套件，通过将虚拟机的输出与系统原生的 CPython 解释器进行对比，确保执行结果的一致性。
+
+```bash
+# 运行所有测试（需要本地安装 Python 3）
+npm test
 ```
+
+## ⚖️ 开源协议
+
+本项目基于 MIT 协议开源 - 详见 [LICENSE](LICENSE) 文件。

package/dist/common/string-token.d.ts

@@ -0,0 +1,5 @@
+export declare const parseStringToken: (tokenValue: string) => {
+    value: string;
+    isFString: boolean;
+};
+//# sourceMappingURL=string-token.d.ts.map

package/dist/common/string-token.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"string-token.d.ts","sourceRoot":"","sources":["../../src/common/string-token.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,gBAAgB,GAAI,YAAY,MAAM,KAAG;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,OAAO,CAAA;CAiBxF,CAAC"}

package/dist/common/string-token.js

@@ -0,0 +1,23 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.parseStringToken = void 0;
+const parseStringToken = (tokenValue) => {
+    let raw = tokenValue;
+    let isFString = false;
+    if (raw.startsWith('f') || raw.startsWith('F')) {
+        isFString = true;
+        raw = raw.slice(1);
+    }
+    const quote = raw[0];
+    if (raw.startsWith(quote.repeat(3))) {
+        const inner = raw.slice(3, -3);
+        return { value: inner, isFString };
+    }
+    const inner = raw.slice(1, -1);
+    return {
+        value: inner.replace(/\\n/g, '\n').replace(/\\t/g, '\t').replace(/\\"/g, '"').replace(/\\'/g, "'"),
+        isFString,
+    };
+};
+exports.parseStringToken = parseStringToken;
+//# sourceMappingURL=string-token.js.map

package/dist/common/string-token.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"string-token.js","sourceRoot":"","sources":["../../src/common/string-token.ts"],"names":[],"mappings":";;;AAAO,MAAM,gBAAgB,GAAG,CAAC,UAAkB,EAAyC,EAAE;IAC5F,IAAI,GAAG,GAAG,UAAU,CAAC;IACrB,IAAI,SAAS,GAAG,KAAK,CAAC;IACtB,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,SAAS,GAAG,IAAI,CAAC;QACjB,GAAG,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;IACrB,CAAC;IACD,MAAM,KAAK,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IACrB,IAAI,GAAG,CAAC,UAAU,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QACpC,MAAM,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QAC/B,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC;IACrC,CAAC;IACD,MAAM,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;IAC/B,OAAO;QACL,KAAK,EAAE,KAAK,CAAC,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC;QAClG,SAAS;KACV,CAAC;AACJ,CAAC,CAAC;AAjBW,QAAA,gBAAgB,oBAiB3B"}

package/dist/compiler/cfg_builder.d.ts

@@ -0,0 +1,48 @@
+import { ASTNode, CFG } from '../types';
+export declare class CFGBuilder {
+    private blocks;
+    private currentBlock;
+    private blockIdCounter;
+    private constants;
+    private names;
+    private varnames;
+    private argcount;
+    private globalVars;
+    private nonlocalVars;
+    private loopStack;
+    constructor(argcount?: number, initialVarnames?: string[]);
+    private createBlock;
+    private addInstruction;
+    private getConstantIndex;
+    private getNameIndex;
+    private getVarIndex;
+    build(ast: ASTNode): CFG;
+    getConstants(): any[];
+    getNames(): string[];
+    getVarnames(): string[];
+    getArgcount(): number;
+    getGlobals(): string[];
+    getNonlocals(): string[];
+    private visit;
+    private addBinaryOperation;
+    private addInplaceOperation;
+    private addCompareOperation;
+    private visitTarget;
+    private visitIf;
+    private visitWhile;
+    private visitFor;
+    private visitFunctionDef;
+    private containsYield;
+    private findAssignedVariables;
+    private extractAssignedNames;
+    private visitLambda;
+    private addUnaryOperation;
+    private visitBoolOp;
+    private visitIfExpression;
+    private visitTry;
+    private visitWith;
+    private processWithItems;
+    private visitMatch;
+    private compilePattern;
+}
+//# sourceMappingURL=cfg_builder.d.ts.map

package/dist/compiler/cfg_builder.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cfg_builder.d.ts","sourceRoot":"","sources":["../../src/compiler/cfg_builder.ts"],"names":[],"mappings":"AAAA,OAAO,EACH,OAAO,EAKP,GAAG,EAcN,MAAM,UAAU,CAAC;AAKlB,qBAAa,UAAU;IACnB,OAAO,CAAC,MAAM,CAAoB;IAClC,OAAO,CAAC,YAAY,CAAa;IACjC,OAAO,CAAC,cAAc,CAAK;IAE3B,OAAO,CAAC,SAAS,CAAiB;IAClC,OAAO,CAAC,KAAK,CAAgB;IAC7B,OAAO,CAAC,QAAQ,CAAgB;IAChC,OAAO,CAAC,QAAQ,CAAa;IAC7B,OAAO,CAAC,UAAU,CAA0B;IAC5C,OAAO,CAAC,YAAY,CAA0B;IAI9C,OAAO,CAAC,SAAS,CAIT;gBAEI,QAAQ,GAAE,MAAU,EAAE,eAAe,GAAE,MAAM,EAAO;IAMhE,OAAO,CAAC,WAAW;IASnB,OAAO,CAAC,cAAc;IAItB,OAAO,CAAC,gBAAgB;IAOxB,OAAO,CAAC,YAAY;IAOpB,OAAO,CAAC,WAAW;IAOnB,KAAK,CAAC,GAAG,EAAE,OAAO,GAAG,GAAG;IAUxB,YAAY;IACZ,QAAQ;IACR,WAAW;IACX,WAAW;IACX,UAAU;IACV,YAAY;IAEZ,OAAO,CAAC,KAAK;IAieb,OAAO,CAAC,kBAAkB;IAkB1B,OAAO,CAAC,mBAAmB;IAkB3B,OAAO,CAAC,mBAAmB;IAkB3B,OAAO,CAAC,WAAW;IAmFnB,OAAO,CAAC,OAAO;IAkDf,OAAO,CAAC,UAAU;IAsClB,OAAO,CAAC,QAAQ;IA6ChB,OAAO,CAAC,gBAAgB;IA4DxB,OAAO,CAAC,aAAa;IAiBrB,OAAO,CAAC,qBAAqB;IAkE7B,OAAO,CAAC,oBAAoB;IAc5B,OAAO,CAAC,WAAW;IAkCnB,OAAO,CAAC,iBAAiB;IAUzB,OAAO,CAAC,WAAW;IAsCnB,OAAO,CAAC,iBAAiB;IA6BzB,OAAO,CAAC,QAAQ;IAiKhB,OAAO,CAAC,SAAS;IAOjB,OAAO,CAAC,gBAAgB;IA4ExB,OAAO,CAAC,UAAU;IAoElB,OAAO,CAAC,cAAc;CAsJzB"}