pylgen-core 0.5.4__tar.gz

pylgen_core-0.5.4/LICENSE

@@ -0,0 +1,30 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, YonyUk
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"IMPORTANT: The source code generated by (or with) this tool (i.e., the lexer, parser, AST classes, and evaluation/codegen methods produced from the user's specification) is NOT considered a derivative work of this repository. The user retains all copyright, intellectual property, and license choice rights over the generated code. This license (BSD 3-Clause) applies ONLY to the PyLGEN's source code and its supporting files, not to the output produced by it."

pylgen_core-0.5.4/MANIFEST.in

@@ -0,0 +1,9 @@
+include *.pyx *.c *.py *.pxd
+recursive-include pylgen/common *.c *.pyx *.py *.pxd
+recursive-include pylgen/automaton *.c *.pyx *.py *.pxd
+recursive-include pylgen/grammar *.c *.pyx *.py *.pxd
+recursive-include pylgen/regex *.c *.pyx *.py *.pxd
+recursive-include pylgen/parser *.c *.pyx *.py *.pxd
+recursive-include pylgen/lexer *.c *.pyx *.py *.pxd
+recursive-include pylgen/analysis *.c *.pyx *.py *.pxd
+include requirements.txt

pylgen_core-0.5.4/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: pylgen-core
+Version: 0.5.4
+Summary: From prototype to production: a Python-native compiler framework that brings the Dragon Book to life in Python, with clarity throughout.
+Author-email: YonyUk <yonatanjoseguerraperez@gmail.com>
+License: BSD 3-Clause License
+        
+        Copyright (c) 2026, YonyUk
+        
+        Redistribution and use in source and binary forms, with or without
+        modification, are permitted provided that the following conditions are met:
+        
+        1. Redistributions of source code must retain the above copyright notice, this
+           list of conditions and the following disclaimer.
+        
+        2. Redistributions in binary form must reproduce the above copyright notice,
+           this list of conditions and the following disclaimer in the documentation
+           and/or other materials provided with the distribution.
+        
+        3. Neither the name of the copyright holder nor the names of its
+           contributors may be used to endorse or promote products derived from
+           this software without specific prior written permission.
+        
+        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+        
+        "IMPORTANT: The source code generated by (or with) this tool (i.e., the lexer, parser, AST classes, and evaluation/codegen methods produced from the user's specification) is NOT considered a derivative work of this repository. The user retains all copyright, intellectual property, and license choice rights over the generated code. This license (BSD 3-Clause) applies ONLY to the PyLGEN's source code and its supporting files, not to the output produced by it."
+Project-URL: Homepage, https://github.com/yonyuk/pylgen
+Project-URL: Repository, https://github.com/yonyuk/pylgen
+Project-URL: Bug Tracker, https://github.com/yonyuk/pylgen/issues
+Project-URL: Documentation, https://pylgen.readthedocs.io/
+Keywords: parser,lalr,lexer,compiler,grammar,attributed-grammar,automaton,regex,language-tool,pylgen,visualization,high-performs
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Compilers
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3 :: Only
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: colorama==0.4.6
+Requires-Dist: decorator==5.2.1
+Requires-Dist: Jinja2==3.1.6
+Requires-Dist: jsonpickle==4.1.1
+Requires-Dist: packaging==26.2
+Requires-Dist: pyparsing==3.3.2
+Requires-Dist: PyYAML==6.0.3
+Provides-Extra: visual
+Requires-Dist: matplotlib==3.10.9; extra == "visual"
+Requires-Dist: networkx==3.6.1; extra == "visual"
+Requires-Dist: pyvis==0.3.2; extra == "visual"
+Provides-Extra: cython
+Requires-Dist: build==1.5.0; extra == "cython"
+Requires-Dist: Cython==3.2.4; extra == "cython"
+Requires-Dist: setuptools==82.0.1; extra == "cython"
+Requires-Dist: wheel==0.47.0; extra == "cython"
+Provides-Extra: test
+Requires-Dist: pytest==9.0.3; extra == "test"
+Provides-Extra: all
+Requires-Dist: colorama==0.4.6; extra == "all"
+Requires-Dist: decorator==5.2.1; extra == "all"
+Requires-Dist: Jinja2==3.1.6; extra == "all"
+Requires-Dist: jsonpickle==4.1.1; extra == "all"
+Requires-Dist: packaging==26.2; extra == "all"
+Requires-Dist: pyparsing==3.3.2; extra == "all"
+Requires-Dist: PyYAML==6.0.3; extra == "all"
+Requires-Dist: matplotlib==3.10.9; extra == "all"
+Requires-Dist: networkx==3.6.1; extra == "all"
+Requires-Dist: pyvis==0.3.2; extra == "all"
+Requires-Dist: build==1.5.0; extra == "all"
+Requires-Dist: Cython==3.2.4; extra == "all"
+Requires-Dist: setuptools==82.0.1; extra == "all"
+Requires-Dist: wheel==0.47.0; extra == "all"
+Requires-Dist: pytest==9.0.3; extra == "all"
+Dynamic: license-file
+
+# PyLGEN
+![PyPI - 0.3.5](https://img.shields.io/pypi/v/pylgen)
+![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/yonyuk/pylgen/ci.yml)
+![PyPI - 3.8](https://img.shields.io/pypi/pyversions/pylgen)
+![PyPI - LICENSE](https://img.shields.io/pypi/l/pylgen)
+
+![Codecov](https://img.shields.io/codecov/c/github/yonyuk/pylgen)
+
+*From prototype to production: a **Python-native compiler framework** that brings the "**Dragon Book**" to life in Python, with clarity throughout.*
+
+PyLGEN gives you **complete control** over every stage of language processing. Build interpreters and compilers from scratch without leaving the Python ecosystem. Prototype rapidly in pure Python, then compile to native speed with Cython for production workloads.
+
+> [!note]
+> **Cython compilation** requires a **C** compiler installed on system to compile the code
+
+## Why PyLGEN?
+
+ - **Full pipeline control**: You own every step: lexer, parser, AST, semantic analysis, and evaluation.
+
+ - **Dual‑nature design**: Pure Python for development and debugging; Cython for near‑C performance.
+
+ - **Built for scale**: Handles 2‑million‑line inputs with ~2.5x faster parsing than Lark + `lark_cython`.
+
+ - **Python ecosystem integration**: Leverage NumPy, SciPy, or any library from within your language.
+
+## 📊 Benchmark at a Glance
+
+| | **Lark + `lark_cython`** | **PyLGEN** |
+| :---: | :---: | :---: |
+| **Parsing (2M lines, 40 MB)** | 151.9 s | 61.1 s |
+| **AST Construction** | Separate pass | Integrated |
+| **Full Interpreter** | — | ~65 s (incl. semantic checks + eval) |
+| **Peak Memory** | ~4 GB | ~887 MB |
+
+> PyLGEN is **not just a parser**, it's a complete, production‑ready interpreter framework that outperforms industry‑standard tools.
+
+## :rocket: Minimal Example
+
+```python
+from pylgen.lexer import Lexer
+from pylgen.grammar import AttributedGrammar
+from pylgen.parser import ParserBuilder, ParserType
+
+# 1. Define tokens & lexer
+lexer = Lexer(mapping_function, r'\s+')
+lexer[0, 'NUMBER'] = r'\d+'
+lexer[1, 'PLUS']   = r'\+'
+
+# 2. Define grammar with AST builders
+G = AttributedGrammar(start=Symbol('E'))
+G[E] += (E, plus, T), binary_reductor
+G[E] += (T,),        single_reductor
+
+# 3. Build the parser
+parser = ParserBuilder.build_parser_from_attributed(G, ParserType.LALR1)
+
+# 4. Parse, analyse, execute
+ast = parser.parse(lexer.tokens)
+# ... your semantic visitors & evaluator ...
+```
+
+## 🧩 Architecture
+
+```mermaid
+flowchart LR
+    Source["Source Code"] --> Lexer["Lexer"]
+    Lexer --> Parser["Parser LALR(1)"]
+    Parser --> AST["AST"]
+    AST --> Visitors["Visitors"]
+    Visitors --> Result["Result / Errors"] 
+```
+
+| **`Module`** | **Purpose** |
+| :---: | :---: |
+| **`common`** | Core types: `Symbol`, `AST`, `Token`, `ASTListView` |
+| **`automaton`** | Finite automata(`DFA`/`NFA`), determinization, **Hopcroft minimisation** |
+| **`regex`** | **Full regex engine** -> **automata conversion** |
+| **`lexer`** | **Regex‑based tokenisation** with priority and validation |
+| **`grammar`** | **CFG and attributed grammar** with reducers |
+| **`parser`** | **LALR(1) parser generation** and runtime |
+| **`analysis`** | **Visitor pattern**, **traversal strategies**, **context management** |
+| **`visual`** | Interactive **HTML visualisation** of ASTs, parse trees, automata and parsing tables |
+
+> **All modules are fully usable from Python and Cython**, prototype in Python, ship in C.
+
+# ⚡ Quick Start
+
+```bash
+pip install pylgen
+```
+
+Then build your interpreter step by step, following the complete tutorial in the [documentation](https://pylgen.readthedocs.io/en/latest/section-1/example-1-first-approach/).
+
+## 📖 Learn More
+
+ - **Step‑by‑step tutorial**: Build a **full arithmetic REPL** from scratch.
+
+ - **VecLang case study**: Production‑grade language with vectors, functions, and slicing.
+
+ - **Deep‑dive API tour**: Understand every module inside out.
+
+[Read the full documentation →](https://pylgen.readthedocs.io)
+
+**PyLGEN**: Where compiler theory meets Python pragmatism.

pylgen_core-0.5.4/pylgen/analysis/__init__.py

@@ -0,0 +1,5 @@
+from .error_type import ErrorType
+from .error import Error,LexicalError,SemanticError,SyntaxError,RuntimeError
+from .lexical import LexicalRule
+from .visitor import ASTChildrenSelector,ASTVisitor,ASTWalker
+from .context import Context