vex-ast 0.1.0__py3-none-any.whl → 0.2.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- vex_ast/README.md +51 -0
- vex_ast/READMEAPI.md +318 -0
- vex_ast/ast/README.md +87 -0
- vex_ast/parser/README.md +47 -0
- vex_ast/registry/README.md +29 -0
- vex_ast/types/README.md +26 -0
- vex_ast/utils/README.md +39 -0
- vex_ast/visitors/README.md +49 -0
- {vex_ast-0.1.0.dist-info → vex_ast-0.2.0.dist-info}/METADATA +9 -11
- {vex_ast-0.1.0.dist-info → vex_ast-0.2.0.dist-info}/RECORD +13 -4
- vex_ast-0.2.0.dist-info/licenses/LICENSE +1 -0
- {vex_ast-0.1.0.dist-info → vex_ast-0.2.0.dist-info}/WHEEL +0 -0
- {vex_ast-0.1.0.dist-info → vex_ast-0.2.0.dist-info}/top_level.txt +0 -0
vex_ast/README.md
ADDED
@@ -0,0 +1,51 @@
|
|
1
|
+
VEX AST Package (vex_ast)
|
2
|
+
|
3
|
+
This directory is the root of the vex_ast Python package. It orchestrates the different components of the VEX AST generation and processing system.
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
The vex_ast package provides a unified interface for parsing VEX V5 Python code into an Abstract Syntax Tree (AST) and tools for working with that AST.
|
8
|
+
|
9
|
+
Structure
|
10
|
+
|
11
|
+
The package is organized into several sub-packages:
|
12
|
+
|
13
|
+
ast/: Contains the definitions for all AST node types, representing the structure of the parsed code. See vex_ast/ast/README.md.
|
14
|
+
|
15
|
+
parser/: Includes the parsing logic responsible for converting source code text into an AST instance. See vex_ast/parser/README.md.
|
16
|
+
|
17
|
+
visitors/: Provides implementations of the Visitor pattern for traversing, analyzing, or transforming the AST. See vex_ast/visitors/README.md.
|
18
|
+
|
19
|
+
utils/: Contains utility classes and functions, primarily for error handling and source location tracking. See vex_ast/utils/README.md.
|
20
|
+
|
21
|
+
Core Exports
|
22
|
+
|
23
|
+
The main components are exposed through the vex_ast/__init__.py file, making them easily accessible:
|
24
|
+
|
25
|
+
parse_string(source, ...): Parses Python code from a string.
|
26
|
+
|
27
|
+
parse_file(filepath, ...): Parses Python code from a file.
|
28
|
+
|
29
|
+
Program: The root node type of the generated AST.
|
30
|
+
|
31
|
+
PrintVisitor: A visitor to generate a string representation of the AST.
|
32
|
+
|
33
|
+
NodeCounter: A visitor to count nodes in the AST.
|
34
|
+
|
35
|
+
VariableCollector: A visitor to collect variable names used in the AST.
|
36
|
+
|
37
|
+
ErrorHandler: Class for managing errors during parsing and processing.
|
38
|
+
|
39
|
+
VexSyntaxError, VexAstError: Custom exception types.
|
40
|
+
|
41
|
+
Workflow
|
42
|
+
|
43
|
+
The typical workflow involves:
|
44
|
+
|
45
|
+
Using parse_string or parse_file from this package to generate an AST (Program object) from source code.
|
46
|
+
|
47
|
+
Instantiating one or more visitors from vex_ast.visitors (or custom ones).
|
48
|
+
|
49
|
+
Calling the visit method of the visitor with the root Program node to perform analysis, transformation, or other operations on the AST.
|
50
|
+
|
51
|
+
Using the ErrorHandler to manage and inspect any errors encountered during the process.
|
vex_ast/READMEAPI.md
ADDED
@@ -0,0 +1,318 @@
|
|
1
|
+
# VEX AST (`vex_ast`) - Public API Reference
|
2
|
+
|
3
|
+
This document describes the main functions and classes exposed by the `vex_ast` package, intended for users who want to parse VEX V5 Python code and interact with the resulting Abstract Syntax Tree (AST).
|
4
|
+
|
5
|
+
## Overview
|
6
|
+
|
7
|
+
The core workflow typically involves:
|
8
|
+
|
9
|
+
1. Parsing source code (from a string or file) using `parse_string` or `parse_file`. This requires an optional `ErrorHandler` instance.
|
10
|
+
2. Receiving a `Program` object, which is the root of the generated AST.
|
11
|
+
3. Instantiating one or more `AstVisitor` subclasses (like `PrintVisitor`, `NodeCounter`, `VariableCollector`, or custom ones).
|
12
|
+
4. Calling the visitor's `visit()` method with the `Program` node to traverse the AST and perform actions.
|
13
|
+
5. Checking the `ErrorHandler` instance for any reported errors during parsing or visiting.
|
14
|
+
|
15
|
+
---
|
16
|
+
|
17
|
+
## Parsing Functions
|
18
|
+
|
19
|
+
These functions are the entry points for converting source code into an AST.
|
20
|
+
|
21
|
+
### `parse_string`
|
22
|
+
|
23
|
+
Parses VEX V5 Python code provided as a string.
|
24
|
+
|
25
|
+
**Signature:**
|
26
|
+
|
27
|
+
```python
|
28
|
+
def parse_string(
|
29
|
+
source: str,
|
30
|
+
filename: str = "<string>",
|
31
|
+
error_handler: Optional[ErrorHandler] = None
|
32
|
+
) -> Program:
|
33
|
+
|
34
|
+
|
35
|
+
Arguments:
|
36
|
+
|
37
|
+
source (str): The string containing the Python code to parse.
|
38
|
+
|
39
|
+
filename (str, optional): The name to associate with the source code, used in error messages and source locations. Defaults to "<string>".
|
40
|
+
|
41
|
+
error_handler (Optional[ErrorHandler], optional): An instance of ErrorHandler to collect parsing errors. If None, a default ErrorHandler is created internally (which will raise VexSyntaxError on the first syntax error). It's recommended to provide your own handler to manage errors more flexibly.
|
42
|
+
|
43
|
+
Returns:
|
44
|
+
|
45
|
+
Program: The root node of the generated Abstract Syntax Tree.
|
46
|
+
|
47
|
+
Raises:
|
48
|
+
|
49
|
+
VexSyntaxError: If a syntax error is encountered during parsing and the error_handler is configured to raise errors immediately (or if no handler is provided).
|
50
|
+
|
51
|
+
VexAstError: For other internal parsing or AST conversion issues if the error_handler is configured to raise errors.
|
52
|
+
|
53
|
+
parse_file
|
54
|
+
|
55
|
+
Parses VEX V5 Python code read from a specified file.
|
56
|
+
|
57
|
+
Signature:
|
58
|
+
|
59
|
+
def parse_file(
|
60
|
+
filepath: str,
|
61
|
+
error_handler: Optional[ErrorHandler] = None
|
62
|
+
) -> Program:
|
63
|
+
IGNORE_WHEN_COPYING_START
|
64
|
+
content_copy
|
65
|
+
download
|
66
|
+
Use code with caution.
|
67
|
+
Python
|
68
|
+
IGNORE_WHEN_COPYING_END
|
69
|
+
|
70
|
+
Arguments:
|
71
|
+
|
72
|
+
filepath (str): The path to the Python file to be parsed.
|
73
|
+
|
74
|
+
error_handler (Optional[ErrorHandler], optional): An instance of ErrorHandler to collect parsing errors. See parse_string for details.
|
75
|
+
|
76
|
+
Returns:
|
77
|
+
|
78
|
+
Program: The root node of the generated Abstract Syntax Tree.
|
79
|
+
|
80
|
+
Raises:
|
81
|
+
|
82
|
+
FileNotFoundError: If the specified filepath does not exist.
|
83
|
+
|
84
|
+
IOError: If there is an error reading the file.
|
85
|
+
|
86
|
+
VexSyntaxError: If a syntax error is encountered (see parse_string).
|
87
|
+
|
88
|
+
VexAstError: For other internal parsing errors (see parse_string).
|
89
|
+
|
90
|
+
Core AST Node
|
91
|
+
Program
|
92
|
+
|
93
|
+
Represents the root node of the entire Abstract Syntax Tree.
|
94
|
+
|
95
|
+
Description:
|
96
|
+
|
97
|
+
The Program node is the object returned by parse_string and parse_file. It serves as the entry point for traversing the AST using visitors.
|
98
|
+
|
99
|
+
Key Attribute:
|
100
|
+
|
101
|
+
body (List[IStatement]): A list containing the top-level statements (like function definitions, assignments, expression statements) found in the parsed code.
|
102
|
+
|
103
|
+
Usage:
|
104
|
+
|
105
|
+
You typically don't instantiate Program directly. You receive it from a parsing function and pass it to the visit() method of an AstVisitor.
|
106
|
+
|
107
|
+
ast_root: Program = parse_string("x = 1")
|
108
|
+
printer = PrintVisitor()
|
109
|
+
printer.visit(ast_root) # Pass the Program node to the visitor
|
110
|
+
IGNORE_WHEN_COPYING_START
|
111
|
+
content_copy
|
112
|
+
download
|
113
|
+
Use code with caution.
|
114
|
+
Python
|
115
|
+
IGNORE_WHEN_COPYING_END
|
116
|
+
Standard Visitors
|
117
|
+
|
118
|
+
Visitors provide mechanisms to traverse and process the AST. You instantiate a visitor and then call its visit method on an AST node (usually the Program root).
|
119
|
+
|
120
|
+
PrintVisitor
|
121
|
+
|
122
|
+
Generates a formatted, indented string representation of the AST structure. Useful for debugging and inspection.
|
123
|
+
|
124
|
+
Instantiation:
|
125
|
+
|
126
|
+
printer = PrintVisitor()
|
127
|
+
IGNORE_WHEN_COPYING_START
|
128
|
+
content_copy
|
129
|
+
download
|
130
|
+
Use code with caution.
|
131
|
+
Python
|
132
|
+
IGNORE_WHEN_COPYING_END
|
133
|
+
|
134
|
+
Core Method:
|
135
|
+
|
136
|
+
visit(node: IAstNode) -> str: Traverses the AST starting from node (typically the Program root) and returns a multi-line string representing the tree.
|
137
|
+
|
138
|
+
Example:
|
139
|
+
|
140
|
+
ast_root = parse_string("y = a + 5")
|
141
|
+
visitor = PrintVisitor()
|
142
|
+
ast_string_representation = visitor.visit(ast_root)
|
143
|
+
print(ast_string_representation)
|
144
|
+
IGNORE_WHEN_COPYING_START
|
145
|
+
content_copy
|
146
|
+
download
|
147
|
+
Use code with caution.
|
148
|
+
Python
|
149
|
+
IGNORE_WHEN_COPYING_END
|
150
|
+
NodeCounter
|
151
|
+
|
152
|
+
Counts the number of nodes in the AST.
|
153
|
+
|
154
|
+
Instantiation:
|
155
|
+
|
156
|
+
counter = NodeCounter()
|
157
|
+
IGNORE_WHEN_COPYING_START
|
158
|
+
content_copy
|
159
|
+
download
|
160
|
+
Use code with caution.
|
161
|
+
Python
|
162
|
+
IGNORE_WHEN_COPYING_END
|
163
|
+
|
164
|
+
Core Method:
|
165
|
+
|
166
|
+
visit(node: IAstNode) -> int: Traverses the AST starting from node and returns the total count of nodes visited.
|
167
|
+
|
168
|
+
Additional Attribute:
|
169
|
+
|
170
|
+
counts_by_type (Dict[str, int]): After calling visit, this dictionary holds the counts for each specific node type encountered (e.g., {'Assignment': 1, 'BinaryOperation': 1, ...}).
|
171
|
+
|
172
|
+
Example:
|
173
|
+
|
174
|
+
ast_root = parse_string("def f():\n return 10")
|
175
|
+
visitor = NodeCounter()
|
176
|
+
total_nodes = visitor.visit(ast_root)
|
177
|
+
print(f"Total nodes: {total_nodes}")
|
178
|
+
print(f"Nodes by type: {visitor.counts_by_type}")
|
179
|
+
IGNORE_WHEN_COPYING_START
|
180
|
+
content_copy
|
181
|
+
download
|
182
|
+
Use code with caution.
|
183
|
+
Python
|
184
|
+
IGNORE_WHEN_COPYING_END
|
185
|
+
VariableCollector
|
186
|
+
|
187
|
+
Collects the names of all variables referenced (read from) in the AST. Does not include variables only assigned to or function/class names being defined.
|
188
|
+
|
189
|
+
Instantiation:
|
190
|
+
|
191
|
+
collector = VariableCollector()
|
192
|
+
IGNORE_WHEN_COPYING_START
|
193
|
+
content_copy
|
194
|
+
download
|
195
|
+
Use code with caution.
|
196
|
+
Python
|
197
|
+
IGNORE_WHEN_COPYING_END
|
198
|
+
|
199
|
+
Core Method:
|
200
|
+
|
201
|
+
visit(node: IAstNode) -> Set[str]: Traverses the AST starting from node and returns a set containing the names of all referenced variables (identifiers used in a Load context).
|
202
|
+
|
203
|
+
Example:
|
204
|
+
|
205
|
+
code = """
|
206
|
+
x = 10
|
207
|
+
y = x + z
|
208
|
+
def my_func(p):
|
209
|
+
q = p
|
210
|
+
"""
|
211
|
+
ast_root = parse_string(code)
|
212
|
+
visitor = VariableCollector()
|
213
|
+
referenced_vars = visitor.visit(ast_root)
|
214
|
+
print(f"Referenced variables: {referenced_vars}") # Output: {'x', 'z', 'p'}
|
215
|
+
IGNORE_WHEN_COPYING_START
|
216
|
+
content_copy
|
217
|
+
download
|
218
|
+
Use code with caution.
|
219
|
+
Python
|
220
|
+
IGNORE_WHEN_COPYING_END
|
221
|
+
Error Handling
|
222
|
+
|
223
|
+
Components for managing and reporting errors during parsing and processing.
|
224
|
+
|
225
|
+
ErrorHandler
|
226
|
+
|
227
|
+
Manages the collection and reporting of errors.
|
228
|
+
|
229
|
+
Instantiation:
|
230
|
+
|
231
|
+
# Option 1: Collect errors, don't raise exceptions immediately
|
232
|
+
handler = ErrorHandler(raise_on_error=False)
|
233
|
+
|
234
|
+
# Option 2: Raise VexSyntaxError/VexAstError on the first error
|
235
|
+
handler = ErrorHandler(raise_on_error=True) # Default behavior if omitted
|
236
|
+
IGNORE_WHEN_COPYING_START
|
237
|
+
content_copy
|
238
|
+
download
|
239
|
+
Use code with caution.
|
240
|
+
Python
|
241
|
+
IGNORE_WHEN_COPYING_END
|
242
|
+
|
243
|
+
Key Methods:
|
244
|
+
|
245
|
+
get_errors() -> List[Error]: Returns a list of all Error objects collected so far.
|
246
|
+
|
247
|
+
has_errors() -> bool: Returns True if any errors have been collected, False otherwise. Useful after parsing with raise_on_error=False.
|
248
|
+
|
249
|
+
clear_errors() -> None: Removes all collected errors.
|
250
|
+
|
251
|
+
add_observer(observer: ErrorObserver) -> None: (Advanced) Registers a custom object to be notified whenever an error is added.
|
252
|
+
|
253
|
+
remove_observer(observer: ErrorObserver) -> None: (Advanced) Unregisters an observer.
|
254
|
+
|
255
|
+
Usage:
|
256
|
+
|
257
|
+
Typically, you create an ErrorHandler instance and pass it to parse_string or parse_file.
|
258
|
+
|
259
|
+
error_handler = ErrorHandler(raise_on_error=False)
|
260
|
+
try:
|
261
|
+
ast = parse_string("x = 1 + ", error_handler=error_handler)
|
262
|
+
if error_handler.has_errors():
|
263
|
+
print("Parsing completed with non-fatal errors:")
|
264
|
+
for err in error_handler.get_errors():
|
265
|
+
print(f"- {err}")
|
266
|
+
else:
|
267
|
+
print("Parsing successful.")
|
268
|
+
# Proceed with AST processing...
|
269
|
+
except VexAstError as e:
|
270
|
+
# This shouldn't happen if raise_on_error=False, but good practice
|
271
|
+
print(f"Unexpected VexAstError: {e}")
|
272
|
+
IGNORE_WHEN_COPYING_START
|
273
|
+
content_copy
|
274
|
+
download
|
275
|
+
Use code with caution.
|
276
|
+
Python
|
277
|
+
IGNORE_WHEN_COPYING_END
|
278
|
+
ErrorType
|
279
|
+
|
280
|
+
An Enum used within Error objects to categorize the type of error.
|
281
|
+
|
282
|
+
Values:
|
283
|
+
|
284
|
+
LEXER_ERROR
|
285
|
+
|
286
|
+
PARSER_ERROR
|
287
|
+
|
288
|
+
TYPE_ERROR
|
289
|
+
|
290
|
+
SEMANTIC_ERROR
|
291
|
+
|
292
|
+
INTERNAL_ERROR
|
293
|
+
|
294
|
+
Usage:
|
295
|
+
|
296
|
+
You typically check the error_type attribute of Error objects retrieved from ErrorHandler.get_errors().
|
297
|
+
|
298
|
+
VexSyntaxError
|
299
|
+
|
300
|
+
Exception raised specifically for syntax errors encountered during parsing when the ErrorHandler is configured to raise errors.
|
301
|
+
|
302
|
+
Inheritance: VexSyntaxError -> VexAstError -> Exception
|
303
|
+
|
304
|
+
Attribute:
|
305
|
+
|
306
|
+
location (Optional[SourceLocation]): May contain the location (line, column) where the syntax error occurred.
|
307
|
+
|
308
|
+
VexAstError
|
309
|
+
|
310
|
+
The base exception class for all errors originating from the vex_ast library. VexSyntaxError is a subclass.
|
311
|
+
|
312
|
+
This reference covers the main components needed to use the vex_ast library effectively for parsing and basic AST interaction. For more advanced use cases, you might need to delve into the specific AST node types in vex_ast.ast or create custom visitors inheriting from vex_ast.visitors.AstVisitor.
|
313
|
+
|
314
|
+
IGNORE_WHEN_COPYING_START
|
315
|
+
content_copy
|
316
|
+
download
|
317
|
+
Use code with caution.
|
318
|
+
IGNORE_WHEN_COPYING_END
|
vex_ast/ast/README.md
ADDED
@@ -0,0 +1,87 @@
|
|
1
|
+
AST Node Definitions (vex_ast.ast)
|
2
|
+
|
3
|
+
This directory defines the structure and types of nodes used in the Abstract Syntax Tree (AST) representation of VEX V5 Python code.
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
The AST provides a structured, hierarchical representation of the source code, abstracting away the specific syntax details. This structure is the foundation for analysis, transformation, and interpretation of the code.
|
8
|
+
|
9
|
+
Core Concepts
|
10
|
+
|
11
|
+
Interfaces (interfaces.py): Defines the fundamental protocols (IAstNode, IExpression, IStatement, ILiteral, IVisitor) that all nodes and visitors adhere to. This ensures a consistent structure.
|
12
|
+
|
13
|
+
Base Classes (core.py): Provides abstract base classes (AstNode, Expression, Statement) that implement common functionality like location tracking and the basic accept method for the Visitor pattern. Program is the root node type.
|
14
|
+
|
15
|
+
Visitor Pattern: Each node implements an accept(visitor) method, allowing external IVisitor objects to operate on the AST without modifying the node classes themselves.
|
16
|
+
|
17
|
+
Child Nodes: Each node provides a get_children() method to facilitate traversal of the tree.
|
18
|
+
|
19
|
+
Source Location: Nodes can store optional SourceLocation information (from vex_ast.utils) linking them back to the original code.
|
20
|
+
|
21
|
+
Node Categories
|
22
|
+
|
23
|
+
The AST nodes are organized into logical categories:
|
24
|
+
|
25
|
+
Expressions (expressions.py): Represent code constructs that evaluate to a value.
|
26
|
+
|
27
|
+
Identifier: Variable names, function names.
|
28
|
+
|
29
|
+
VariableReference: Usage of a variable.
|
30
|
+
|
31
|
+
AttributeAccess: Accessing attributes (e.g., motor.spin).
|
32
|
+
|
33
|
+
BinaryOperation: Operations like +, -, *, /, ==, and, or.
|
34
|
+
|
35
|
+
UnaryOperation: Operations like - (negation), not.
|
36
|
+
|
37
|
+
FunctionCall: Calling functions or methods.
|
38
|
+
|
39
|
+
KeywordArgument: Named arguments in function calls (e.g., speed=50).
|
40
|
+
|
41
|
+
Literals (literals.py): Represent constant values.
|
42
|
+
|
43
|
+
NumberLiteral: Integers and floats.
|
44
|
+
|
45
|
+
StringLiteral: Text strings.
|
46
|
+
|
47
|
+
BooleanLiteral: True or False.
|
48
|
+
|
49
|
+
NoneLiteral: The None value.
|
50
|
+
|
51
|
+
Statements (statements.py): Represent actions or control flow constructs.
|
52
|
+
|
53
|
+
ExpressionStatement: An expression used on its own line (e.g., a function call).
|
54
|
+
|
55
|
+
Assignment: Assigning a value to a variable (=).
|
56
|
+
|
57
|
+
IfStatement: Conditional execution (if/elif/else).
|
58
|
+
|
59
|
+
WhileLoop: Looping based on a condition.
|
60
|
+
|
61
|
+
ForLoop: Iterating over a sequence.
|
62
|
+
|
63
|
+
FunctionDefinition: Defining a function (def).
|
64
|
+
|
65
|
+
Argument: Parameters in a function definition.
|
66
|
+
|
67
|
+
ReturnStatement: Returning a value from a function.
|
68
|
+
|
69
|
+
BreakStatement, ContinueStatement: Loop control.
|
70
|
+
|
71
|
+
Operators (operators.py): Defines Operator enums used within BinaryOperation and UnaryOperation nodes. Includes mappings from Python operators.
|
72
|
+
|
73
|
+
VEX-Specific Nodes (vex_nodes.py): Subclasses of FunctionCall tailored to represent common VEX API patterns.
|
74
|
+
|
75
|
+
VexAPICall: Base class for VEX calls.
|
76
|
+
|
77
|
+
MotorControl: Calls related to motors (e.g., motor.spin).
|
78
|
+
|
79
|
+
SensorReading: Calls related to sensors (e.g., sensor.value).
|
80
|
+
|
81
|
+
TimingControl: Calls like wait.
|
82
|
+
|
83
|
+
DisplayOutput: Calls related to screen output (e.g., brain.screen.print).
|
84
|
+
|
85
|
+
Usage
|
86
|
+
|
87
|
+
These node classes are typically instantiated by the parser (vex_ast.parser) during the code parsing process. Visitors (vex_ast.visitors) then interact with these node objects to perform tasks.
|
vex_ast/parser/README.md
ADDED
@@ -0,0 +1,47 @@
|
|
1
|
+
Code Parser (vex_ast.parser)
|
2
|
+
|
3
|
+
This directory contains the components responsible for parsing VEX V5 Python source code and constructing the Abstract Syntax Tree (AST).
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
The parser's role is to take raw source code (as a string or from a file) and transform it into the structured AST representation defined in vex_ast.ast. It handles syntax rules, identifies different code constructs, and builds the corresponding tree of nodes.
|
8
|
+
|
9
|
+
Key Components
|
10
|
+
|
11
|
+
Interfaces (interfaces.py):
|
12
|
+
|
13
|
+
IParser: Protocol defining the essential parse() method that all parser implementations must provide.
|
14
|
+
|
15
|
+
BaseParser: An abstract base class providing common functionality, such as integrating with the ErrorHandler.
|
16
|
+
|
17
|
+
Node Factory (factory.py):
|
18
|
+
|
19
|
+
NodeFactory: Implements the Factory pattern. It provides methods (create_identifier, create_binary_operation, etc.) to instantiate specific AST node types defined in vex_ast.ast. This decouples the main parsing logic from the details of node creation and allows for easier management of node instantiation, including attaching source locations and potentially handling errors during creation.
|
20
|
+
|
21
|
+
Python Parser (python_parser.py):
|
22
|
+
|
23
|
+
PythonParser: The concrete implementation of IParser. It leverages Python's built-in ast module to parse the input Python code into a standard Python AST.
|
24
|
+
|
25
|
+
It then traverses the Python AST, using the NodeFactory to convert the standard Python nodes (ast.Assign, ast.Call, ast.BinOp, etc.) into the corresponding custom VEX AST nodes (Assignment, FunctionCall, BinaryOperation, etc.).
|
26
|
+
|
27
|
+
Provides convenience functions parse_string and parse_file.
|
28
|
+
|
29
|
+
Workflow
|
30
|
+
|
31
|
+
The user calls parse_string or parse_file.
|
32
|
+
|
33
|
+
An instance of PythonParser is created with the source code and an optional ErrorHandler.
|
34
|
+
|
35
|
+
The PythonParser uses Python's ast.parse to generate a standard Python AST.
|
36
|
+
|
37
|
+
The PythonParser walks through the Python AST. For each Python ast node, it determines the corresponding VEX AST node type.
|
38
|
+
|
39
|
+
It calls the appropriate create_... method on its internal NodeFactory instance, passing the necessary components (sub-expressions, statements, names, values) converted recursively.
|
40
|
+
|
41
|
+
The NodeFactory creates the VEX AST node, potentially adding source location information extracted from the original Python ast node.
|
42
|
+
|
43
|
+
This process continues recursively until the entire Python AST is converted into the custom VEX AST (Program node).
|
44
|
+
|
45
|
+
Any syntax errors during Python parsing or unsupported constructs during conversion are reported via the ErrorHandler.
|
46
|
+
|
47
|
+
The final Program node representing the VEX AST is returned.
|
@@ -0,0 +1,29 @@
|
|
1
|
+
# VEX AST Registry Package (vex_ast.registry)
|
2
|
+
|
3
|
+
This package manages the registration and organization of VEX V5 Robot Python language features, including functions, objects, and other elements.
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
The registry provides a centralized system for defining and accessing information about VEX-specific language constructs. This information is used by the parser, type checker, and other tools to understand and process VEX code.
|
8
|
+
|
9
|
+
Structure
|
10
|
+
|
11
|
+
The package is organized into several modules:
|
12
|
+
|
13
|
+
* `categories.py`: Defines categories for organizing registry entries.
|
14
|
+
* `language_map.py`: Maps VEX language features to their Python equivalents.
|
15
|
+
* `registry.py`: Implements the core registry logic.
|
16
|
+
* `signature.py`: Defines the structure for function signatures.
|
17
|
+
* `simulation_behavior.py`: Specifies how VEX features should behave in a simulation environment.
|
18
|
+
* `validation.py`: Provides validation rules for registry entries.
|
19
|
+
* `functions/`: Contains submodules for specific function categories (e.g., drivetrain, motor, sensors).
|
20
|
+
|
21
|
+
Key Concepts
|
22
|
+
|
23
|
+
* Registration: VEX language features are registered with the registry, providing metadata about their syntax, semantics, and simulation behavior.
|
24
|
+
* Organization: The registry organizes features into categories and subcategories for easy access and management.
|
25
|
+
* Validation: Registry entries are validated to ensure consistency and correctness.
|
26
|
+
|
27
|
+
Usage
|
28
|
+
|
29
|
+
The registry is used internally by the VEX AST parser and other tools. It is not typically accessed directly by users.
|
vex_ast/types/README.md
ADDED
@@ -0,0 +1,26 @@
|
|
1
|
+
# VEX AST Types Package (vex_ast.types)
|
2
|
+
|
3
|
+
This package defines the type system used for the VEX V5 Robot Python language.
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
The type system provides a way to represent and reason about the types of values in VEX code. This information is used for type checking, code optimization, and other purposes.
|
8
|
+
|
9
|
+
Structure
|
10
|
+
|
11
|
+
The package is organized into several modules:
|
12
|
+
|
13
|
+
* `base.py`: Defines base classes for types.
|
14
|
+
* `enums.py`: Defines enums for different type categories.
|
15
|
+
* `objects.py`: Defines classes for representing objects.
|
16
|
+
* `primitives.py`: Defines classes for representing primitive types (e.g., int, float, string, boolean).
|
17
|
+
* `type_checker.py`: Implements the type checking logic.
|
18
|
+
|
19
|
+
Key Concepts
|
20
|
+
|
21
|
+
* Types: Represent the kind of value that a variable or expression can have.
|
22
|
+
* Type checking: The process of verifying that the types in a program are consistent.
|
23
|
+
|
24
|
+
Usage
|
25
|
+
|
26
|
+
The type system is used internally by the VEX AST parser, type checker, and other tools. It is not typically accessed directly by users.
|
vex_ast/utils/README.md
ADDED
@@ -0,0 +1,39 @@
|
|
1
|
+
Utilities (vex_ast.utils)
|
2
|
+
|
3
|
+
This directory contains utility modules that provide supporting functionality for the VEX AST parser and other components.
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
These modules encapsulate common tasks and data structures needed across the vex_ast package, promoting code reuse and separation of concerns.
|
8
|
+
|
9
|
+
Modules
|
10
|
+
|
11
|
+
Error Handling (errors.py):
|
12
|
+
|
13
|
+
ErrorType: An enum defining categories of errors (e.g., PARSER_ERROR, SEMANTIC_ERROR).
|
14
|
+
|
15
|
+
Error: A class representing a single error instance, containing the type, message, optional source location, and optional suggestion.
|
16
|
+
|
17
|
+
ErrorHandler: A central class for collecting and managing errors encountered during parsing or AST processing. It allows registering observers and can optionally raise exceptions immediately upon error detection.
|
18
|
+
|
19
|
+
VexAstError, VexSyntaxError: Custom exception classes for AST-related errors, with VexSyntaxError specifically for parsing issues.
|
20
|
+
|
21
|
+
Importance: Provides a robust way to handle and report problems found in the source code or during AST processing, crucial for user feedback and debugging.
|
22
|
+
|
23
|
+
Source Location (source_location.py):
|
24
|
+
|
25
|
+
SourceLocation: A dataclass representing a position or span within the original source code file. It includes line and column numbers (and optionally end line/column and filename).
|
26
|
+
|
27
|
+
Importance: Allows AST nodes and errors to be linked back to their origin in the source code, which is essential for accurate error reporting, debugging tools, and source mapping.
|
28
|
+
|
29
|
+
Usage
|
30
|
+
|
31
|
+
These utilities are primarily used internally by other parts of the vex_ast package:
|
32
|
+
|
33
|
+
The PythonParser uses the ErrorHandler to report syntax errors or issues during AST conversion. It uses SourceLocation to tag generated AST nodes with their origin.
|
34
|
+
|
35
|
+
AST Node classes store SourceLocation objects.
|
36
|
+
|
37
|
+
Visitors might use the ErrorHandler if they perform analysis that detects semantic errors.
|
38
|
+
|
39
|
+
Error messages often include the string representation of a SourceLocation.
|
@@ -0,0 +1,49 @@
|
|
1
|
+
AST Visitors (vex_ast.visitors)
|
2
|
+
|
3
|
+
This directory provides implementations of the Visitor design pattern for traversing and operating on the VEX Abstract Syntax Tree (AST).
|
4
|
+
|
5
|
+
Purpose
|
6
|
+
|
7
|
+
Visitors allow you to perform operations on the AST without modifying the AST node classes themselves. They provide a clean and extensible way to implement various forms of analysis, transformation, or code generation based on the AST structure.
|
8
|
+
|
9
|
+
Core Concepts
|
10
|
+
|
11
|
+
Visitor Pattern: The core idea is double dispatch. You call visitor.visit(node), which in turn calls node.accept(visitor). The node's accept method then calls the specific visit_NodeType(node) method on the visitor corresponding to its own type.
|
12
|
+
|
13
|
+
Base Visitor (base.py):
|
14
|
+
|
15
|
+
AstVisitor[T_VisitorResult]: An abstract generic base class for all visitors. It defines the main visit entry point and provides default visit_NodeType methods that typically delegate to a generic_visit method. Subclasses must implement generic_visit and can override specific visit_NodeType methods for custom behavior.
|
16
|
+
|
17
|
+
Traversal: The generic_visit method in concrete visitors often iterates through the node's children (obtained via node.get_children()) and calls self.visit(child) recursively to traverse the tree.
|
18
|
+
|
19
|
+
Provided Visitors
|
20
|
+
|
21
|
+
Printer (printer.py):
|
22
|
+
|
23
|
+
PrintVisitor: Traverses the AST and generates a formatted, indented string representation of the tree structure, including node types, attributes, and source locations (if available). Useful for debugging and understanding the AST structure.
|
24
|
+
|
25
|
+
Analyzers (analyzer.py):
|
26
|
+
|
27
|
+
NodeCounter: Traverses the AST and counts the total number of nodes, optionally keeping track of counts per node type.
|
28
|
+
|
29
|
+
VariableCollector: Traverses the AST and collects the names of all variables referenced (VariableReference nodes).
|
30
|
+
|
31
|
+
Usage
|
32
|
+
|
33
|
+
Instantiate a Visitor: Create an instance of the desired visitor class (e.g., printer = PrintVisitor()).
|
34
|
+
|
35
|
+
Parse Code: Obtain the root node (Program object) of the AST using vex_ast.parse_string or vex_ast.parse_file.
|
36
|
+
|
37
|
+
Start Visitation: Call the visitor's visit method with the root node (e.g., result = printer.visit(ast_root)).
|
38
|
+
|
39
|
+
Process Result: The visit method will return a result whose type depends on the specific visitor (T_VisitorResult). For PrintVisitor, it's a string; for NodeCounter, an integer; for VariableCollector, a set of strings.
|
40
|
+
|
41
|
+
Extensibility
|
42
|
+
|
43
|
+
You can create custom visitors to perform specific tasks (e.g., type checking, code optimization, simulation execution) by:
|
44
|
+
|
45
|
+
Creating a new class that inherits from AstVisitor[YourResultType].
|
46
|
+
|
47
|
+
Implementing the generic_visit method to define default behavior and traversal logic.
|
48
|
+
|
49
|
+
Overriding specific visit_NodeType methods for nodes where specialized logic is required.
|
@@ -1,13 +1,14 @@
|
|
1
1
|
Metadata-Version: 2.4
|
2
2
|
Name: vex_ast
|
3
|
-
Version: 0.
|
3
|
+
Version: 0.2.0
|
4
4
|
Summary: A Python package for generating Abstract Syntax Trees for VEX V5 code.
|
5
|
-
Home-page: https://github.com/
|
5
|
+
Home-page: https://github.com/heartx2/vex_ast
|
6
6
|
Author: Chaze
|
7
|
-
Author-email: chazelexander@
|
7
|
+
Author-email: Chaze <chazelexander@gmail.com>
|
8
|
+
License: HX2's Vex AST © 2025 by charkwayteowy is licensed under CC BY-NC 4.0
|
9
|
+
Project-URL: Repository, https://github.com/heartx2/vex_ast
|
8
10
|
Classifier: Development Status :: 3 - Alpha
|
9
11
|
Classifier: Intended Audience :: Developers
|
10
|
-
Classifier: License :: OSI Approved :: MIT License
|
11
12
|
Classifier: Programming Language :: Python :: 3
|
12
13
|
Classifier: Programming Language :: Python :: 3.8
|
13
14
|
Classifier: Programming Language :: Python :: 3.9
|
@@ -17,14 +18,11 @@ Classifier: Topic :: Software Development :: Compilers
|
|
17
18
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
18
19
|
Requires-Python: >=3.8
|
19
20
|
Description-Content-Type: text/markdown
|
21
|
+
License-File: LICENSE
|
20
22
|
Dynamic: author
|
21
|
-
Dynamic: author-email
|
22
|
-
Dynamic: classifier
|
23
|
-
Dynamic: description
|
24
|
-
Dynamic: description-content-type
|
25
23
|
Dynamic: home-page
|
24
|
+
Dynamic: license-file
|
26
25
|
Dynamic: requires-python
|
27
|
-
Dynamic: summary
|
28
26
|
|
29
27
|
# VEX AST Generator
|
30
28
|
|
@@ -60,7 +58,7 @@ The core library is within the `vex_ast` directory:
|
|
60
58
|
|
61
59
|
1. **Clone the repository:**
|
62
60
|
```bash
|
63
|
-
git clone https://github.com/
|
61
|
+
git clone https://github.com/heartx2/vex_ast # Replace with actual URL
|
64
62
|
cd vex_ast
|
65
63
|
```
|
66
64
|
|
@@ -173,4 +171,4 @@ Contributions are welcome! Please follow the established coding standards and en
|
|
173
171
|
|
174
172
|
## License
|
175
173
|
|
176
|
-
|
174
|
+
HX2's Vex AST © 2025 by charkwayteowy is licensed under CC BY-NC 4.0
|
@@ -1,4 +1,7 @@
|
|
1
|
+
vex_ast/README.md,sha256=tRaq6n2Ab4IahHSo3utW2imVuTrJu_7zaelgF-lGBYo,2007
|
2
|
+
vex_ast/READMEAPI.md,sha256=hGn4IMT9NnI_LW0kd36BdKKXdIhYFLbQcgbVj1EWJps,9107
|
1
3
|
vex_ast/__init__.py,sha256=YPsS_1ab9-G3WYmyxQobDA1U8y5KJkj2qxm_p_s9Oow,1724
|
4
|
+
vex_ast/ast/README.md,sha256=7IQDHEXmKyJ8zfJNwa3pMGTDZKVPMFD0YeHT06ATRyM,3205
|
2
5
|
vex_ast/ast/__init__.py,sha256=3ZtTiNeBPWOYgNWmxThn0h7UQx8Z0bHbK6hRHCUuILQ,2435
|
3
6
|
vex_ast/ast/core.py,sha256=q-15q5VfGIVPAmFsSiWeFAZ55MPsXfCRa21rrt2hmlM,2670
|
4
7
|
vex_ast/ast/expressions.py,sha256=b0cDs239wvygCIL9s0W8uuyuPeMZz3DwxQ0wmBta_XM,7758
|
@@ -9,11 +12,13 @@ vex_ast/ast/operators.py,sha256=I-yWvhsrz-OxmBZs5zIss_GTZF5S-nwcSmIzvAVtddM,3160
|
|
9
12
|
vex_ast/ast/statements.py,sha256=OWRthjYGmTuNozYAHjh_Enp5t-hR1PphtPnFg31FeWw,11841
|
10
13
|
vex_ast/ast/validators.py,sha256=lhEm1dt-nzMiaUan_AWEOm0NiwXN7wYNkLCl1c3drqc,4455
|
11
14
|
vex_ast/ast/vex_nodes.py,sha256=kVil-ApFIeZ_BoeqYppfVRPSTg1KFUOcvt1vCE6QRgs,10148
|
15
|
+
vex_ast/parser/README.md,sha256=P1qq_97skpgluLDpNu9V_pAdkuF9StjkzOEXJJYpEuM,2565
|
12
16
|
vex_ast/parser/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
13
17
|
vex_ast/parser/factory.py,sha256=gaje0jOPu5lpndgX9c3VuQ_q-FJuFz9yC5y2bD69lL4,8770
|
14
18
|
vex_ast/parser/interfaces.py,sha256=Ttc0bD_5X420ZCT9MLUf_wE1aZLWLkaJRQqCwBy9wIs,956
|
15
19
|
vex_ast/parser/python_parser.py,sha256=e1owk-c6Rb4Bt5SYcgB-urZzqsX5xC9tzcK1sB8gkr4,29687
|
16
20
|
vex_ast/parser/strategies.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
21
|
+
vex_ast/registry/README.md,sha256=J8xEXEscEZhWa6wmn05kjE395AjrV7p2h7nq4SXI5uE,1535
|
17
22
|
vex_ast/registry/__init__.py,sha256=dXT8OS7uLyBSdQKZDWrtbANzgSWcg9HUQDA1EaLOvjQ,1188
|
18
23
|
vex_ast/registry/api.py,sha256=yyW-TsCIULTzePAoyMvEqD_RNlkFSb4PLpiSkw6ueQk,5460
|
19
24
|
vex_ast/registry/categories.py,sha256=y5tnCzHQ9PPg9hOzOMgyE1ZAhcsjgeFSrgqdJrdqHEg,4885
|
@@ -26,16 +31,20 @@ vex_ast/serialization/__init__.py,sha256=qPTEiMjU8hpVxNH5z4MY7Yj60AxuFurjXZStjhW
|
|
26
31
|
vex_ast/serialization/json_deserializer.py,sha256=m_xiEUEoFXGfFylK3M8BVs7F2EieQJ9wbUa_ZTpkHtQ,10523
|
27
32
|
vex_ast/serialization/json_serializer.py,sha256=YvMRUpqXjtbxlIvathEIywUqbH3Ne6GSRODfFB0QCGM,4465
|
28
33
|
vex_ast/serialization/schema.py,sha256=zpKujT7u3Qw2kNbwjV09lNzeQAsCxvMx-uAdaRMEstA,14431
|
34
|
+
vex_ast/types/README.md,sha256=Wd3jBShiNXNc3iJ69Qps5_0mBq8QVEd_Gz5OachfS34,1029
|
35
|
+
vex_ast/utils/README.md,sha256=Y9RJMQTqQbpjVkvYJmpeRihD1zfW9PhNL_LgoDJ84ak,1945
|
29
36
|
vex_ast/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
30
37
|
vex_ast/utils/errors.py,sha256=uq_jyu-YasifXjKSpPbVxup-XFf0XSHmp7A0Zd87TUY,3850
|
31
38
|
vex_ast/utils/source_location.py,sha256=r857ypqUBqex2y_R0RzZo7pz5di9qtEeWFFQ4HwzYQE,1297
|
32
39
|
vex_ast/utils/type_definitions.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
40
|
+
vex_ast/visitors/README.md,sha256=BKDj8LMuBlCrzgSSLmCkbET7WIqgxe-oCkqQbqXekrE,2725
|
33
41
|
vex_ast/visitors/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
34
42
|
vex_ast/visitors/analyzer.py,sha256=cBT5PRbtfQ_Dt1qiHN-wdO5KnC2yl-Ul5RasXg9geHY,3668
|
35
43
|
vex_ast/visitors/base.py,sha256=Ph0taTIVMHV_QjXfDd0ipZnCXVJErkyRtPpfwfzo-kY,4859
|
36
44
|
vex_ast/visitors/printer.py,sha256=CUY_73hxm7MoC-63JeIXaVYnZ8QqtfMqbek2R2HMEmE,5411
|
37
45
|
vex_ast/visitors/transformer.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
38
|
-
vex_ast-0.
|
39
|
-
vex_ast-0.
|
40
|
-
vex_ast-0.
|
41
|
-
vex_ast-0.
|
46
|
+
vex_ast-0.2.0.dist-info/licenses/LICENSE,sha256=IOSlfCuxGv4OAg421BRDKVi16RZ7-5kCMJ4B16r4kuc,69
|
47
|
+
vex_ast-0.2.0.dist-info/METADATA,sha256=xGeh73jQORTXs2-uS-qgiH3aW-Pdsk31J1WW-tKQREE,5295
|
48
|
+
vex_ast-0.2.0.dist-info/WHEEL,sha256=pxyMxgL8-pra_rKaQ4drOZAegBVuX-G_4nRHjjgWbmo,91
|
49
|
+
vex_ast-0.2.0.dist-info/top_level.txt,sha256=MoZGrpKgNUDiqL9gWp4q3wMw3q93XPEEjmBNPJQcNAs,8
|
50
|
+
vex_ast-0.2.0.dist-info/RECORD,,
|
@@ -0,0 +1 @@
|
|
1
|
+
HX2's Vex AST © 2025 by charkwayteowy is licensed under CC BY-NC 4.0
|
File without changes
|
File without changes
|