dbrownell-parserlib 0.8.2__tar.gz → 0.8.4__tar.gz

{dbrownell_parserlib-0.8.2 → dbrownell_parserlib-0.8.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: dbrownell-parserlib
-Version: 0.8.2
+Version: 0.8.4
 Summary: Functionality useful when creating parsers.
 Author: Dave Brownell
 Author-email: Dave Brownell <github@DavidBrownell.com>
@@ -47,10 +47,158 @@ Description-Content-Type: text/markdown
 - [License](#license)
 
 ## Overview
-TODO: Complete this section
+`dbrownell_ParserLib` is a Python library that provides utility functionality for creating parsers, with a primary focus on [ANTLR](https://antlr.org) (Another Tool for Language Recognition) integration. The library simplifies the process of building parsers that produce strongly-typed abstract syntax trees (ASTs) with comprehensive source location tracking.
+
+### Key Features
+- **ANTLR Integration**: Streamlined wrapper around ANTLR for Python, handling grammar compilation and parser generation
+- **AST Representation**: Expression-based AST nodes with automatic parent-child relationship management
+- **Source Location Tracking**: Every AST node tracks its source file location (filename, line, column ranges) for detailed error reporting
+- **Visitor Pattern**: Flexible visitor pattern implementation with fine-grained traversal control
+- **Error Handling**: Rich error objects that associate messages with source locations and extract Python exception tracebacks
+- **Multi-threaded Parsing**: Built-in support for parsing multiple files concurrently
+- **Workspace Support**: Parse entire directory trees with customizable file filtering
+- **Whitespace Handling**: Mixins for both insignificant and significant whitespace grammars (e.g., Python-like indentation)
 
 ### How to use `dbrownell_ParserLib`
-TODO: Complete this section
+
+The typical workflow for creating a parser with this library involves the following steps. For detailed examples, see the test files in the repository, particularly the end-to-end tests that demonstrate complete parsing workflows.
+
+
+#### 1. Define an ANTLR Grammar
+Create a `.g4` grammar file defining your language syntax:
+
+```antlr
+grammar Calculator;
+
+expr : expr ('*'|'/') expr   # BinaryOp
+     | expr ('+'|'-') expr   # BinaryOp
+     | INT                   # Number
+     ;
+
+INT : [0-9]+ ;
+WS : [ \t\n\r]+ -> skip ;
+```
+
+#### 2. Build the Grammar
+Use `BuildAntlrGrammar` to generate the lexer and parser:
+
+```python
+from pathlib import Path
+
+from dbrownell_ParserLib import BuildAntlrGrammar
+
+
+BuildAntlrGrammar(
+    dm,  # DoneManager instance
+    Path("Calculator.g4"),
+    Path("output_dir"),
+)
+```
+
+#### 3. Create Custom Visitor
+Implement a visitor that converts ANTLR parse trees to Expression objects:
+
+```python
+from dataclasses import dataclass
+
+from dbrownell_ParserLib import (
+    Expression,
+    InsignificantWhitespaceAntlrVisitorMixin,
+    TerminalExpression,
+)
+
+from CalculatorVisitor import CalculatorVisitor as GeneratedVisitor
+
+
+@dataclass(eq=False)
+class BinaryExpression(Expression):
+    left: Expression
+    operator: TerminalExpression[str]
+    right: Expression
+
+    def _GenerateAcceptDetails(self):
+        yield "left", self.left
+        yield "operator", self.operator
+        yield "right", self.right
+
+
+class CalculatorVisitor(InsignificantWhitespaceAntlrVisitorMixin, GeneratedVisitor):
+    def visitExpr(self, ctx):
+        children = self.GetChildren(ctx)
+        assert len(children) == 2, children
+
+        self._stack.append(
+            BinaryExpression(
+                self.CreateRegion(ctx),
+                children[0],
+                TerminalExpression[str](self.CreateRegion(ctx.children[1]), ctx.children[1].getText()),
+
+                children[1],
+            )
+        )
+
+    def visitNumber(self, ctx):
+        self._stack.append(
+            TerminalExpression(self.CreateRegion(ctx), int(ctx.getText()))
+        )
+```
+
+#### 4. Create Parser Instance
+Use `CreateAntlrParser` to build a callable parser:
+
+```python
+from dbrownell_ParserLib import CreateAntlrParser
+
+
+parser = CreateAntlrParser(
+    CalculatorLexer,
+    CalculatorParser,
+    CalculatorVisitor,
+    lambda p: p.expr(),  # Entry point rule
+)
+```
+
+#### 5. Parse Files
+Parse single files, multiple files, or entire workspaces:
+
+```python
+from dbrownell_ParserLib import Error
+
+
+# Parse a single file
+results = parser(dm, Path("input.txt"), None)
+
+# Parse multiple files
+results = parser(dm, [Path("file1.txt"), Path("file2.txt")], None)
+
+# Check results
+for filepath, result in results.items():
+    if isinstance(result, Error):
+        print(f"Parse error in {filepath}: {result}")
+    else:
+        # result is the visitor containing the parsed AST
+        ast = result._stack[0]
+        # Process the AST
+```
+
+#### 6. Traverse the AST
+Use the visitor pattern to process your AST:
+
+```python
+from contextlib import contextmanager
+
+from dbrownell_ParserLib import ExpressionVisitorHelper, VisitResult
+
+
+class EvaluatorVisitor(ExpressionVisitorHelper):
+    @contextmanager
+    def OnBinaryExpression(self, expression):
+        # Process binary expressions
+        yield VisitResult.Continue
+
+
+ast.Accept(EvaluatorVisitor())
+```
 
 <!-- Content below this delimiter will be copied to the generated README.md file. DO NOT REMOVE THIS COMMENT, as it will cause regeneration to fail. -->
 

{dbrownell_parserlib-0.8.2 → dbrownell_parserlib-0.8.4}/README.md

@@ -25,10 +25,158 @@
 - [License](#license)
 
 ## Overview
-TODO: Complete this section
+`dbrownell_ParserLib` is a Python library that provides utility functionality for creating parsers, with a primary focus on [ANTLR](https://antlr.org) (Another Tool for Language Recognition) integration. The library simplifies the process of building parsers that produce strongly-typed abstract syntax trees (ASTs) with comprehensive source location tracking.
+
+### Key Features
+- **ANTLR Integration**: Streamlined wrapper around ANTLR for Python, handling grammar compilation and parser generation
+- **AST Representation**: Expression-based AST nodes with automatic parent-child relationship management
+- **Source Location Tracking**: Every AST node tracks its source file location (filename, line, column ranges) for detailed error reporting
+- **Visitor Pattern**: Flexible visitor pattern implementation with fine-grained traversal control
+- **Error Handling**: Rich error objects that associate messages with source locations and extract Python exception tracebacks
+- **Multi-threaded Parsing**: Built-in support for parsing multiple files concurrently
+- **Workspace Support**: Parse entire directory trees with customizable file filtering
+- **Whitespace Handling**: Mixins for both insignificant and significant whitespace grammars (e.g., Python-like indentation)
 
 ### How to use `dbrownell_ParserLib`
-TODO: Complete this section
+
+The typical workflow for creating a parser with this library involves the following steps. For detailed examples, see the test files in the repository, particularly the end-to-end tests that demonstrate complete parsing workflows.
+
+
+#### 1. Define an ANTLR Grammar
+Create a `.g4` grammar file defining your language syntax:
+
+```antlr
+grammar Calculator;
+
+expr : expr ('*'|'/') expr   # BinaryOp
+     | expr ('+'|'-') expr   # BinaryOp
+     | INT                   # Number
+     ;
+
+INT : [0-9]+ ;
+WS : [ \t\n\r]+ -> skip ;
+```
+
+#### 2. Build the Grammar
+Use `BuildAntlrGrammar` to generate the lexer and parser:
+
+```python
+from pathlib import Path
+
+from dbrownell_ParserLib import BuildAntlrGrammar
+
+
+BuildAntlrGrammar(
+    dm,  # DoneManager instance
+    Path("Calculator.g4"),
+    Path("output_dir"),
+)
+```
+
+#### 3. Create Custom Visitor
+Implement a visitor that converts ANTLR parse trees to Expression objects:
+
+```python
+from dataclasses import dataclass
+
+from dbrownell_ParserLib import (
+    Expression,
+    InsignificantWhitespaceAntlrVisitorMixin,
+    TerminalExpression,
+)
+
+from CalculatorVisitor import CalculatorVisitor as GeneratedVisitor
+
+
+@dataclass(eq=False)
+class BinaryExpression(Expression):
+    left: Expression
+    operator: TerminalExpression[str]
+    right: Expression
+
+    def _GenerateAcceptDetails(self):
+        yield "left", self.left
+        yield "operator", self.operator
+        yield "right", self.right
+
+
+class CalculatorVisitor(InsignificantWhitespaceAntlrVisitorMixin, GeneratedVisitor):
+    def visitExpr(self, ctx):
+        children = self.GetChildren(ctx)
+        assert len(children) == 2, children
+
+        self._stack.append(
+            BinaryExpression(
+                self.CreateRegion(ctx),
+                children[0],
+                TerminalExpression[str](self.CreateRegion(ctx.children[1]), ctx.children[1].getText()),
+
+                children[1],
+            )
+        )
+
+    def visitNumber(self, ctx):
+        self._stack.append(
+            TerminalExpression(self.CreateRegion(ctx), int(ctx.getText()))
+        )
+```
+
+#### 4. Create Parser Instance
+Use `CreateAntlrParser` to build a callable parser:
+
+```python
+from dbrownell_ParserLib import CreateAntlrParser
+
+
+parser = CreateAntlrParser(
+    CalculatorLexer,
+    CalculatorParser,
+    CalculatorVisitor,
+    lambda p: p.expr(),  # Entry point rule
+)
+```
+
+#### 5. Parse Files
+Parse single files, multiple files, or entire workspaces:
+
+```python
+from dbrownell_ParserLib import Error
+
+
+# Parse a single file
+results = parser(dm, Path("input.txt"), None)
+
+# Parse multiple files
+results = parser(dm, [Path("file1.txt"), Path("file2.txt")], None)
+
+# Check results
+for filepath, result in results.items():
+    if isinstance(result, Error):
+        print(f"Parse error in {filepath}: {result}")
+    else:
+        # result is the visitor containing the parsed AST
+        ast = result._stack[0]
+        # Process the AST
+```
+
+#### 6. Traverse the AST
+Use the visitor pattern to process your AST:
+
+```python
+from contextlib import contextmanager
+
+from dbrownell_ParserLib import ExpressionVisitorHelper, VisitResult
+
+
+class EvaluatorVisitor(ExpressionVisitorHelper):
+    @contextmanager
+    def OnBinaryExpression(self, expression):
+        # Process binary expressions
+        yield VisitResult.Continue
+
+
+ast.Accept(EvaluatorVisitor())
+```
 
 <!-- Content below this delimiter will be copied to the generated README.md file. DO NOT REMOVE THIS COMMENT, as it will cause regeneration to fail. -->
 

{dbrownell_parserlib-0.8.2 → dbrownell_parserlib-0.8.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dbrownell-parserlib"
-version = "0.8.2"
+version = "0.8.4"
 #          ^^^^^
 # Wheel names will be generated according to this value. Do not manually modify this value; instead
 # update it according to committed changes by running this command from the root of the repository: