flowquery 1.0.17 → 1.0.18

package/flowquery-py/CONTRIBUTING.md

@@ -0,0 +1,127 @@
+# Contributing to FlowQuery Python
+
+This guide covers setting up a development environment and contributing to the Python implementation of FlowQuery.
+
+## Development Setup
+
+### From Source
+
+```bash
+git clone https://github.com/microsoft/FlowQuery.git
+cd FlowQuery/flowquery-py
+pip install -e .
+```
+
+### With Development Dependencies
+
+```bash
+pip install -e ".[dev]"
+```
+
+### Using Conda (Recommended)
+
+**Windows (PowerShell):**
+
+```powershell
+cd flowquery-py
+.\setup_env.ps1
+conda activate flowquery
+```
+
+**Linux/macOS:**
+
+```bash
+cd flowquery-py
+chmod +x setup_env.sh
+./setup_env.sh
+conda activate flowquery
+```
+
+The setup scripts automatically:
+
+1. Read the Python version from `pyproject.toml`
+2. Create a conda environment named `flowquery`
+3. Install the package with all dev dependencies
+
+## Requirements
+
+- Python 3.10+ (defined in `pyproject.toml`)
+- pytest (for running tests)
+- pytest-asyncio (for async test support)
+- aiohttp (for HTTP requests)
+
+All dependencies are managed in `pyproject.toml`.
+
+## Running Tests
+
+```bash
+pytest tests/
+```
+
+## Project Structure
+
+```
+flowquery-py/
+├── pyproject.toml       # Dependencies & project config (single source of truth)
+├── setup_env.ps1        # Windows conda setup script
+├── setup_env.sh         # Linux/macOS conda setup script
+├── README.md
+├── src/
+│   ├── __init__.py          # Main package entry point
+│   ├── extensibility.py     # Public API for custom functions
+│   ├── compute/
+│   │   └── runner.py        # Query execution engine
+│   ├── graph/
+│   │   ├── node.py          # Graph node representation
+│   │   ├── relationship.py  # Graph relationship representation
+│   │   ├── pattern.py       # Pattern matching
+│   │   └── database.py      # In-memory graph database
+│   ├── io/
+│   │   └── command_line.py  # Interactive REPL
+│   ├── parsing/
+│   │   ├── parser.py        # Main parser
+│   │   ├── ast_node.py      # AST node base class
+│   │   ├── expressions/     # Expression types (numbers, strings, operators)
+│   │   ├── functions/       # Built-in and custom functions
+│   │   ├── operations/      # Query operations (WITH, RETURN, UNWIND, etc.)
+│   │   ├── components/      # LOAD clause components
+│   │   ├── data_structures/ # Arrays, objects, lookups
+│   │   └── logic/           # CASE/WHEN/THEN/ELSE
+│   ├── tokenization/
+│   │   ├── tokenizer.py     # Lexer
+│   │   ├── token.py         # Token class
+│   │   └── ...              # Token types and mappers
+│   └── utils/
+│       ├── string_utils.py  # String manipulation utilities
+│       └── object_utils.py  # Object utilities
+└── tests/
+    ├── test_extensibility.py
+    ├── compute/
+    │   └── test_runner.py
+    ├── graph/
+    │   ├── test_create.py
+    │   ├── test_data.py
+    │   └── test_match.py
+    ├── parsing/
+    │   ├── test_parser.py
+    │   ├── test_context.py
+    │   └── test_expression.py
+    └── tokenization/
+        ├── test_tokenizer.py
+        ├── test_token_mapper.py
+        └── test_trie.py
+```
+
+## Code Style
+
+- Follow PEP 8 guidelines
+- Use type hints where appropriate
+- Write docstrings for public APIs
+
+## Submitting Changes
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Run tests: `pytest tests/`
+5. Submit a pull request

package/flowquery-py/README.md

@@ -1,68 +1,24 @@
-# FlowQuery Python Implementation
+# FlowQuery
 
-This is the Python implementation of FlowQuery, a declarative query language for data processing pipelines.
+A declarative query language for data processing pipelines.
 
 ## Installation
 
-### From Source
-
 ```bash
-git clone https://github.com/microsoft/FlowQuery.git
-cd FlowQuery/flowquery-py
-pip install -e .
-```
-
-### With Development Dependencies
-
-```bash
-pip install -e ".[dev]"
+pip install flowquery
 ```
 
 ## Quick Start
 
 ### Command Line Interface
 
-After installation, you can start the interactive REPL:
+Start the interactive REPL:
 
 ```bash
 flowquery
 ```
 
-### Using Conda (Alternative)
-
-**Windows (PowerShell):**
-
-```powershell
-cd flowquery-py
-.\setup_env.ps1
-conda activate flowquery
-```
-
-**Linux/macOS:**
-
-```bash
-cd flowquery-py
-chmod +x setup_env.sh
-./setup_env.sh
-conda activate flowquery
-```
-
-The setup scripts automatically:
-
-1. Read the Python version from `pyproject.toml`
-2. Create a conda environment named `flowquery`
-3. Install the package with all dev dependencies
-
-## Requirements
-
-- Python 3.10+ (defined in `pyproject.toml`)
-- pytest (for running tests)
-- pytest-asyncio (for async test support)
-- aiohttp (for HTTP requests)
-
-All dependencies are managed in `pyproject.toml`.
-
-## Programmatic Usage
+### Programmatic Usage
 
 ```python
 import asyncio
@@ -73,66 +29,6 @@ asyncio.run(runner.run())
 print(runner.results)  # [{'result': 2}]
 ```
 
-## Running Tests
-
-```bash
-pytest tests/
-```
-
-## Project Structure
-
-```
-flowquery-py/
-├── pyproject.toml       # Dependencies & project config (single source of truth)
-├── setup_env.ps1        # Windows conda setup script
-├── setup_env.sh         # Linux/macOS conda setup script
-├── README.md
-├── src/
-│   ├── __init__.py          # Main package entry point
-│   ├── extensibility.py     # Public API for custom functions
-│   ├── compute/
-│   │   └── runner.py        # Query execution engine
-│   ├── graph/
-│   │   ├── node.py          # Graph node representation
-│   │   ├── relationship.py  # Graph relationship representation
-│   │   ├── pattern.py       # Pattern matching
-│   │   └── database.py      # In-memory graph database
-│   ├── io/
-│   │   └── command_line.py  # Interactive REPL
-│   ├── parsing/
-│   │   ├── parser.py        # Main parser
-│   │   ├── ast_node.py      # AST node base class
-│   │   ├── expressions/     # Expression types (numbers, strings, operators)
-│   │   ├── functions/       # Built-in and custom functions
-│   │   ├── operations/      # Query operations (WITH, RETURN, UNWIND, etc.)
-│   │   ├── components/      # LOAD clause components
-│   │   ├── data_structures/ # Arrays, objects, lookups
-│   │   └── logic/           # CASE/WHEN/THEN/ELSE
-│   ├── tokenization/
-│   │   ├── tokenizer.py     # Lexer
-│   │   ├── token.py         # Token class
-│   │   └── ...              # Token types and mappers
-│   └── utils/
-│       ├── string_utils.py  # String manipulation utilities
-│       └── object_utils.py  # Object utilities
-└── tests/
-    ├── test_extensibility.py
-    ├── compute/
-    │   └── test_runner.py
-    ├── graph/
-    │   ├── test_create.py
-    │   ├── test_data.py
-    │   └── test_match.py
-    ├── parsing/
-    │   ├── test_parser.py
-    │   ├── test_context.py
-    │   └── test_expression.py
-    └── tokenization/
-        ├── test_tokenizer.py
-        ├── test_token_mapper.py
-        └── test_trie.py
-```
-
 ## Creating Custom Functions
 
 ```python
@@ -155,12 +51,17 @@ class UpperCase(Function):
         return str(self.get_children()[0].value()).upper()
 ```
 
+## Documentation
+
+- [Full Documentation](https://github.com/microsoft/FlowQuery)
+- [Contributing Guide](https://github.com/microsoft/FlowQuery/blob/main/flowquery-py/CONTRIBUTING.md)
+
 ## License
 
-MIT License - see [LICENSE](LICENSE) for details.
+MIT License - see [LICENSE](https://github.com/microsoft/FlowQuery/blob/main/LICENSE) for details.
 
 ## Links
 
-- [Homepage](https://github.com/microsoft/FlowQuery/flowquery-py)
-- [Repository](https://github.com/microsoft/FlowQuery/flowquery-py)
+- [Homepage](https://github.com/microsoft/FlowQuery)
+- [Repository](https://github.com/microsoft/FlowQuery)
 - [Issues](https://github.com/microsoft/FlowQuery/issues)

package/flowquery-py/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "flowquery"
-version = "1.0.0"
+version = "1.0.7"
 description = "A declarative query language for data processing pipelines"
 readme = "README.md"
 requires-python = ">=3.10"

package/flowquery-py/src/graph/pattern_expression.py

@@ -20,11 +20,14 @@ class PatternExpression(Pattern):
         self._evaluation: bool = False
 
     def add_element(self, element) -> None:
-        """Add an element to the pattern, ensuring it starts with a NodeReference."""
-        if len(self._chain) == 0 and not isinstance(element, NodeReference):
-            raise ValueError("PatternExpression must start with a NodeReference")
         super().add_element(element)
 
+    def verify(self) -> None:
+        if(len(self._chain) == 0):
+            raise ValueError("PatternExpression cannot be empty")
+        if not(any(isinstance(el, NodeReference) for el in self._chain if isinstance(el, ASTNode))):
+            raise ValueError("PatternExpression must contain at least one NodeReference")
+
     @property
     def identifier(self):
         return None

package/flowquery-py/src/io/command_line.py

@@ -1,5 +1,6 @@
 """Interactive command-line interface for FlowQuery."""
 
+import argparse
 import asyncio
 from typing import Optional
 
@@ -15,8 +16,26 @@ class CommandLine:
     Example:
         cli = CommandLine()
         cli.loop()  # Starts interactive mode
+        
+        # Or execute a single query:
+        cli.execute("load json from 'https://example.com/data' as d return d")
     """
 
+    def execute(self, query: str) -> None:
+        """Execute a single FlowQuery statement and print results.
+        
+        Args:
+            query: The FlowQuery statement to execute.
+        """
+        # Remove the termination semicolon if present
+        query = query.strip().rstrip(";")
+        
+        try:
+            runner = Runner(query)
+            asyncio.run(self._execute(runner))
+        except Exception as e:
+            print(f"Error: {e}")
+
     def loop(self) -> None:
         """Starts the interactive command loop.
         
@@ -63,5 +82,28 @@ class CommandLine:
 
 
 def main() -> None:
-    """Entry point for the flowquery CLI command."""
-    CommandLine().loop()
+    """Entry point for the flowquery CLI command.
+    
+    Usage:
+        flowquery              # Start interactive mode
+        flowquery -c "query"   # Execute a single query
+        flowquery --command "query"
+    """
+    parser = argparse.ArgumentParser(
+        description="FlowQuery - A declarative query language for data processing pipelines",
+        prog="flowquery"
+    )
+    parser.add_argument(
+        "-c", "--command",
+        type=str,
+        metavar="QUERY",
+        help="Execute a FlowQuery statement and exit"
+    )
+    
+    args = parser.parse_args()
+    cli = CommandLine()
+    
+    if args.command:
+        cli.execute(args.command)
+    else:
+        cli.loop()

package/flowquery-py/src/parsing/base_parser.py

@@ -69,7 +69,7 @@ class BaseParser:
             The current token, or EOF if at the end
         """
         if self._token_index >= len(self._tokens):
-            return Token.EOF
+            return Token.EOF()
         return self._tokens[self._token_index]
 
     @property
@@ -80,5 +80,5 @@ class BaseParser:
             The previous token, or EOF if at the beginning
         """
         if self._token_index - 1 < 0:
-            return Token.EOF
+            return Token.EOF()
         return self._tokens[self._token_index - 1]

package/flowquery-py/src/parsing/operations/load.py

@@ -96,6 +96,12 @@ class Load(Operation):
             headers = options.pop("headers", {})
             body = options.pop("body", None)
             
+            # Set Accept-Encoding to support common compression formats
+            # Note: brotli (br) is excluded due to API incompatibility between
+            # aiohttp 3.13+ and the brotli package's Decompressor.decompress() method
+            if "Accept-Encoding" not in headers:
+                headers["Accept-Encoding"] = "gzip, deflate"
+            
             async with session.request(
                 method, 
                 self.from_, 

package/flowquery-py/src/parsing/parser.py

@@ -426,8 +426,6 @@ class Parser(BaseParser):
         node = self._parse_node()
         if node is None:
             raise ValueError("Expected node definition")
-        if not isinstance(node, NodeReference):
-            raise ValueError("PatternExpression must start with a NodeReference")
         pattern.add_element(node)
         while True:
             relationship = self._parse_relationship()
@@ -440,6 +438,7 @@ class Parser(BaseParser):
             if node is None:
                 raise ValueError("Expected target node definition")
             pattern.add_element(node)
+        pattern.verify()
         return pattern
 
     def _parse_node(self) -> Optional[Node]:
@@ -606,7 +605,7 @@ class Parser(BaseParser):
                 if func is not None:
                     lookup = self._parse_lookup(func)
                     expression.add_node(lookup)
-            elif self.token.is_left_parenthesis() and self.peek() is not None and self.peek().is_identifier():
+            elif self.token.is_left_parenthesis() and self.peek() is not None and (self.peek().is_identifier() or self.peek().is_colon() or self.peek().is_right_parenthesis()):
                 # Possible graph pattern expression
                 pattern = self._parse_pattern_expression()
                 if pattern is not None:
@@ -795,9 +794,9 @@ class Parser(BaseParser):
         # Lookahead: identifier ( identifier in
         if not self.ahead([
             Token.IDENTIFIER(""),
-            Token.LEFT_PARENTHESIS,
+            Token.LEFT_PARENTHESIS(),
             Token.IDENTIFIER(""),
-            Token.IN,
+            Token.IN(),
         ]):
             return None
         if self.token.value is None: