feldera 0.129.0__tar.gz → 0.131.0__tar.gz

{feldera-0.129.0 → feldera-0.131.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: feldera
-Version: 0.129.0
+Version: 0.131.0
 Summary: The feldera python client
 Author-email: Feldera Team <dev@feldera.com>
 License: MIT
@@ -59,6 +59,12 @@ source .venv/activate
 uv pip install .
 ```
 
+You also have to install the `pytest` module:
+
+```bash
+python3 -m pip install pytest
+```
+
 ## Documentation
 
 The Python SDK documentation is available at
@@ -78,73 +84,6 @@ make html
 
 To clean the build, run `make clean`.
 
-## Testing
-
-To run unit tests:
-
-```bash
-cd python && python3 -m pytest tests/
-```
-
-- This will detect and run all test files that match the pattern `test_*.py` or
-  `*_test.py`.
-- By default, the tests expect a running Feldera instance at `http://localhost:8080`.
-  To override the default endpoint, set the `FELDERA_HOST` environment variable.
-
-To run tests from a specific file:
-
-```bash
-(cd python && python3 -m pytest ./tests/path-to-file.py)
-```
-
-To run a specific test:
-
-```bash
-uv run python -m pytest tests/test_shared_pipeline.py::TestPipeline::test_adhoc_query_hash -v
-```
-
-#### Running All Tests
-
-The tests validate end-to-end correctness of SQL functionality.  To
-run the tests use:
-
-```bash
-cd python
-PYTHONPATH=`pwd` ./tests/run-all-tests.sh
-```
-
-### Reducing Compilation Cycles
-
-To reduce redundant compilation cycles during testing:
-
-* **Inherit from `SharedTestPipeline`** instead of `unittest.TestCase`.
-* **Define DDLs** (e.g., `CREATE TABLE`, `CREATE VIEW`) in the **docstring** of each test method.
-  * All DDLs from all test functions in the class are combined and compiled into a single pipeline.
-  * If a table or view is already defined in one test, it can be used directly in others without redefinition.
-  * Ensure that all table and view names are unique within the class.
-* Use `@enterprise_only` on tests that require Enterprise features. Their DDLs will be skipped on OSS builds.
-* Use `self.set_runtime_config(...)` to override the default pipeline config.
-  * Reset it at the end using `self.reset_runtime_config()`.
-* Access the shared pipeline via `self.pipeline`.
-
-#### Example
-
-```python
-from tests.shared_test_pipeline import SharedTestPipeline
-
-class TestAverage(SharedTestPipeline):
-    def test_average(self):
-        """
-        CREATE TABLE students(id INT, name STRING);
-        CREATE MATERIALIZED VIEW v AS SELECT * FROM students;
-        """
-        ...
-        self.pipeline.start()
-        self.pipeline.input_pandas("students", df)
-        self.pipeline.wait_for_completion(True)
-        ...
-```
-
 ## Linting and formatting
 
 Use [Ruff] to run the lint checks that will be executed by the

feldera-0.131.0/README.md

@@ -0,0 +1,79 @@
+# Feldera Python SDK
+
+Feldera Python is the Feldera SDK for Python developers.
+
+## Installation
+
+```bash
+uv pip install feldera
+```
+
+### Installing from Github
+
+```bash
+uv pip install git+https://github.com/feldera/feldera#subdirectory=python
+```
+
+Similarly, to install from a specific branch:
+
+```bash
+uv pip install git+https://github.com/feldera/feldera@{BRANCH_NAME}#subdirectory=python
+```
+
+Replace `{BRANCH_NAME}` with the name of the branch you want to install from.
+
+### Installing from Local Directory
+
+If you have cloned the Feldera repo, you can install the python SDK as follows:
+
+```bash
+# the Feldera Python SDK is present inside the python/ directory
+cd python
+# If you don't have a virtual environment, create one
+uv venv
+source .venv/activate
+# Install the SDK in editable mode
+uv pip install .
+```
+
+You also have to install the `pytest` module:
+
+```bash
+python3 -m pip install pytest
+```
+
+## Documentation
+
+The Python SDK documentation is available at
+[Feldera Python SDK Docs](https://docs.feldera.com/python).
+
+To build the html documentation run:
+
+Ensure that you have sphinx installed. If not, install it using `uv pip install sphinx`.
+
+Then run the following commands:
+
+```bash
+cd docs
+sphinx-apidoc -o . ../feldera
+make html
+```
+
+To clean the build, run `make clean`.
+
+## Linting and formatting
+
+Use [Ruff] to run the lint checks that will be executed by the
+precommit hook when a PR is submitted:
+
+```bash
+ruff check python/
+```
+
+To reformat the code in the same way as the precommit hook:
+
+```bash
+ruff format
+```
+
+[Ruff]: https://github.com/astral-sh/ruff

{feldera-0.129.0 → feldera-0.131.0}/feldera.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: feldera
-Version: 0.129.0
+Version: 0.131.0
 Summary: The feldera python client
 Author-email: Feldera Team <dev@feldera.com>
 License: MIT
@@ -59,6 +59,12 @@ source .venv/activate
 uv pip install .
 ```
 
+You also have to install the `pytest` module:
+
+```bash
+python3 -m pip install pytest
+```
+
 ## Documentation
 
 The Python SDK documentation is available at
@@ -78,73 +84,6 @@ make html
 
 To clean the build, run `make clean`.
 
-## Testing
-
-To run unit tests:
-
-```bash
-cd python && python3 -m pytest tests/
-```
-
-- This will detect and run all test files that match the pattern `test_*.py` or
-  `*_test.py`.
-- By default, the tests expect a running Feldera instance at `http://localhost:8080`.
-  To override the default endpoint, set the `FELDERA_HOST` environment variable.
-
-To run tests from a specific file:
-
-```bash
-(cd python && python3 -m pytest ./tests/path-to-file.py)
-```
-
-To run a specific test:
-
-```bash
-uv run python -m pytest tests/test_shared_pipeline.py::TestPipeline::test_adhoc_query_hash -v
-```
-
-#### Running All Tests
-
-The tests validate end-to-end correctness of SQL functionality.  To
-run the tests use:
-
-```bash
-cd python
-PYTHONPATH=`pwd` ./tests/run-all-tests.sh
-```
-
-### Reducing Compilation Cycles
-
-To reduce redundant compilation cycles during testing:
-
-* **Inherit from `SharedTestPipeline`** instead of `unittest.TestCase`.
-* **Define DDLs** (e.g., `CREATE TABLE`, `CREATE VIEW`) in the **docstring** of each test method.
-  * All DDLs from all test functions in the class are combined and compiled into a single pipeline.
-  * If a table or view is already defined in one test, it can be used directly in others without redefinition.
-  * Ensure that all table and view names are unique within the class.
-* Use `@enterprise_only` on tests that require Enterprise features. Their DDLs will be skipped on OSS builds.
-* Use `self.set_runtime_config(...)` to override the default pipeline config.
-  * Reset it at the end using `self.reset_runtime_config()`.
-* Access the shared pipeline via `self.pipeline`.
-
-#### Example
-
-```python
-from tests.shared_test_pipeline import SharedTestPipeline
-
-class TestAverage(SharedTestPipeline):
-    def test_average(self):
-        """
-        CREATE TABLE students(id INT, name STRING);
-        CREATE MATERIALIZED VIEW v AS SELECT * FROM students;
-        """
-        ...
-        self.pipeline.start()
-        self.pipeline.input_pandas("students", df)
-        self.pipeline.wait_for_completion(True)
-        ...
-```
-
 ## Linting and formatting
 
 Use [Ruff] to run the lint checks that will be executed by the

{feldera-0.129.0 → feldera-0.131.0}/feldera.egg-info/SOURCES.txt

@@ -24,9 +24,4 @@ feldera/rest/feldera_config.py
 feldera/rest/pipeline.py
 feldera/rest/sql_table.py
 feldera/rest/sql_view.py
-tests/test_checkpoint_sync.py
-tests/test_issue4457.py
-tests/test_pipeline_builder.py
-tests/test_shared_pipeline.py
-tests/test_shared_pipeline_stress.py
-tests/test_udf.py
+tests/test_uda.py

{feldera-0.129.0 → feldera-0.131.0}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 name = "feldera"
 readme = "README.md"
 description = "The feldera python client"
-version = "0.129.0"
+version = "0.131.0"
 license = { text = "MIT" }
 requires-python = ">=3.10"
 authors = [

feldera-0.131.0/tests/test_uda.py

@@ -0,0 +1,283 @@
+import unittest
+
+from feldera import PipelineBuilder
+from tests import TEST_CLIENT
+
+
+class TestUDA(unittest.TestCase):
+    def test_local(self):
+        sql = """
+CREATE LINEAR AGGREGATE I128_SUM(s BINARY(16)) RETURNS BINARY(16);
+CREATE TABLE T(x BINARY(16));
+CREATE MATERIALIZED VIEW V AS SELECT I128_SUM(x) AS S, COUNT(*) AS C FROM T;
+        """
+
+        toml = """
+i256 = { version = "0.2.2", features = ["num-traits"] }
+num-traits = "0.2.19"
+        """
+
+        udfs = """
+use i256::I256;
+use feldera_sqllib::*;
+use crate::{AddAssignByRef, AddByRef, HasZero, MulByRef, SizeOf, Tup3};
+use derive_more::Add;
+use num_traits::Zero;
+use rkyv::Fallible;
+use std::ops::{Add, AddAssign};
+
+#[derive(Add, Clone, Debug, Default, PartialOrd, Ord, Eq, PartialEq, Hash)]
+pub struct I256Wrapper {
+    pub data: I256,
+}
+
+impl SizeOf for I256Wrapper {
+    fn size_of_children(&self, context: &mut size_of::Context) {}
+}
+
+impl From<[u8; 32]> for I256Wrapper {
+    fn from(value: [u8; 32]) -> Self {
+        Self { data: I256::from_be_bytes(value) }
+    }
+}
+
+impl From<&[u8]> for I256Wrapper {
+    fn from(value: &[u8]) -> Self {
+        let mut padded = [0u8; 32];
+        // If original value is negative, pad with sign
+        if value[0] & 0x80 != 0 {
+            padded.fill(0xff);
+        }
+        let len = value.len();
+        if len > 32 {
+            panic!("Slice larger than target");
+        }
+        padded[32-len..].copy_from_slice(&value[..len]);
+        Self { data: I256::from_be_bytes(padded) }
+    }
+}
+
+impl MulByRef<Weight> for I256Wrapper {
+    type Output = Self;
+
+    fn mul_by_ref(&self, other: &Weight) -> Self::Output {
+        println!("Mul {:?} by {}", self, other);
+        Self {
+            data: self.data.checked_mul_i64(*other)
+                .expect("Overflow during multiplication"),
+        }
+    }
+}
+
+impl HasZero for I256Wrapper {
+    fn zero() -> Self {
+        Self { data: I256::zero() }
+    }
+
+    fn is_zero(&self) -> bool {
+        self.data.is_zero()
+    }
+}
+
+impl AddByRef for I256Wrapper {
+    fn add_by_ref(&self, other: &Self) -> Self {
+        Self { data: self.data.add(other.data) }
+    }
+}
+
+impl AddAssignByRef<Self> for I256Wrapper {
+    fn add_assign_by_ref(&mut self, other: &Self) {
+        self.data += other.data
+    }
+}
+
+#[repr(C)]
+#[derive(Debug, Copy, Clone, PartialOrd, Ord, Eq, PartialEq)]
+pub struct ArchivedI256Wrapper {
+    pub bytes: [u8; 32],
+}
+
+impl rkyv::Archive for I256Wrapper {
+    type Archived = ArchivedI256Wrapper;
+    type Resolver = ();
+
+    #[inline]
+    unsafe fn resolve(&self, pos: usize, _: Self::Resolver, out: *mut Self::Archived) {
+        out.write(ArchivedI256Wrapper {
+            bytes: self.data.to_be_bytes(),
+        });
+    }
+}
+
+impl<S: Fallible + ?Sized> rkyv::Serialize<S> for I256Wrapper {
+    #[inline]
+    fn serialize(&self, serializer: &mut S) -> Result<Self::Resolver, S::Error> {
+        Ok(())
+    }
+}
+
+impl<D: Fallible + ?Sized> rkyv::Deserialize<I256Wrapper, D> for ArchivedI256Wrapper {
+    #[inline]
+    fn deserialize(&self, _: &mut D) -> Result<I256Wrapper, D::Error> {
+        Ok(I256Wrapper::from(self.bytes))
+    }
+}
+
+pub type i128_sum_accumulator_type = Tup3<I256Wrapper, i64, i64>;
+
+pub fn i128_sum_map(val: Option<ByteArray>) -> i128_sum_accumulator_type {
+    match val {
+        None => Tup3::new(I256Wrapper::zero(), 0, 1),
+        Some(val) => Tup3::new(
+           I256Wrapper::from(val.as_slice()),
+           1,
+           1,
+        ),
+    }
+}
+
+pub fn i128_sum_post(val: i128_sum_accumulator_type) -> Option<ByteArray> {
+    if val.1 == 0 {
+       None
+    } else {
+       // Check for overflow
+       if val.0.data < I256::from(i128::MIN) || val.0.data > I256::from(i128::MAX) {
+           panic!("Result of aggregation {} does not fit in 128 bits", val.0.data);
+       }
+       Some(ByteArray::new(&val.0.data.to_be_bytes()[16..]))
+    }
+}
+        """
+
+        pipeline = PipelineBuilder(
+            TEST_CLIENT, name="test_uda", sql=sql, udf_rust=udfs, udf_toml=toml
+        ).create_or_replace()
+
+        pipeline.start()
+        pipeline.input_json(
+            "t",
+            [
+                {
+                    "insert": {
+                        "x": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
+                    }
+                }
+            ],
+            update_format="insert_delete",
+        )
+        pipeline.wait_for_idle()
+        output = list(pipeline.query("SELECT * FROM V;"))
+        assert output == [{"s": "00000000000000000000000000000001", "c": 1}]
+
+        # Insert -1
+        pipeline.input_json(
+            "t",
+            [
+                {
+                    "insert": {
+                        "x": [
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                        ],
+                    }
+                }
+            ],
+            update_format="insert_delete",
+        )
+        pipeline.wait_for_idle()
+        output = list(pipeline.query("SELECT * FROM V;"))
+        assert output == [{"s": "00000000000000000000000000000000", "c": 2}]
+
+        pipeline.input_json(
+            "t",
+            [
+                {
+                    "insert": {
+                        "x": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2],
+                    }
+                }
+            ],
+            update_format="insert_delete",
+        )
+        output = list(pipeline.query("SELECT * FROM V;"))
+        assert output == [{"s": "00000000000000000000000000000002", "c": 3}]
+
+        pipeline.input_json(
+            "t",
+            [
+                {
+                    "insert": {
+                        "x": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3],
+                    }
+                }
+            ],
+            update_format="insert_delete",
+        )
+        output = list(pipeline.query("SELECT * FROM V;"))
+        assert output == [{"s": "00000000000000000000000000000005", "c": 4}]
+
+        pipeline.input_json(
+            "t",
+            [
+                {
+                    "delete": {
+                        "x": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
+                    }
+                },
+                {
+                    "delete": {
+                        "x": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2],
+                    }
+                },
+                {
+                    "delete": {
+                        "x": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3],
+                    }
+                },
+                {
+                    "delete": {
+                        "x": [
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            255,
+                            1,
+                        ],
+                    }
+                },
+            ],
+            update_format="insert_delete",
+        )
+        output = list(pipeline.query("SELECT * FROM V;"))
+        assert output == [{"s": None, "c": 0}]
+
+        pipeline.stop(force=True)
+
+
+if __name__ == "__main__":
+    unittest.main()

feldera-0.129.0/README.md

@@ -1,140 +0,0 @@
-# Feldera Python SDK
-
-Feldera Python is the Feldera SDK for Python developers.
-
-## Installation
-
-```bash
-uv pip install feldera
-```
-
-### Installing from Github
-
-```bash
-uv pip install git+https://github.com/feldera/feldera#subdirectory=python
-```
-
-Similarly, to install from a specific branch:
-
-```bash
-uv pip install git+https://github.com/feldera/feldera@{BRANCH_NAME}#subdirectory=python
-```
-
-Replace `{BRANCH_NAME}` with the name of the branch you want to install from.
-
-### Installing from Local Directory
-
-If you have cloned the Feldera repo, you can install the python SDK as follows:
-
-```bash
-# the Feldera Python SDK is present inside the python/ directory
-cd python
-# If you don't have a virtual environment, create one
-uv venv
-source .venv/activate
-# Install the SDK in editable mode
-uv pip install .
-```
-
-## Documentation
-
-The Python SDK documentation is available at
-[Feldera Python SDK Docs](https://docs.feldera.com/python).
-
-To build the html documentation run:
-
-Ensure that you have sphinx installed. If not, install it using `uv pip install sphinx`.
-
-Then run the following commands:
-
-```bash
-cd docs
-sphinx-apidoc -o . ../feldera
-make html
-```
-
-To clean the build, run `make clean`.
-
-## Testing
-
-To run unit tests:
-
-```bash
-cd python && python3 -m pytest tests/
-```
-
-- This will detect and run all test files that match the pattern `test_*.py` or
-  `*_test.py`.
-- By default, the tests expect a running Feldera instance at `http://localhost:8080`.
-  To override the default endpoint, set the `FELDERA_HOST` environment variable.
-
-To run tests from a specific file:
-
-```bash
-(cd python && python3 -m pytest ./tests/path-to-file.py)
-```
-
-To run a specific test:
-
-```bash
-uv run python -m pytest tests/test_shared_pipeline.py::TestPipeline::test_adhoc_query_hash -v
-```
-
-#### Running All Tests
-
-The tests validate end-to-end correctness of SQL functionality.  To
-run the tests use:
-
-```bash
-cd python
-PYTHONPATH=`pwd` ./tests/run-all-tests.sh
-```
-
-### Reducing Compilation Cycles
-
-To reduce redundant compilation cycles during testing:
-
-* **Inherit from `SharedTestPipeline`** instead of `unittest.TestCase`.
-* **Define DDLs** (e.g., `CREATE TABLE`, `CREATE VIEW`) in the **docstring** of each test method.
-  * All DDLs from all test functions in the class are combined and compiled into a single pipeline.
-  * If a table or view is already defined in one test, it can be used directly in others without redefinition.
-  * Ensure that all table and view names are unique within the class.
-* Use `@enterprise_only` on tests that require Enterprise features. Their DDLs will be skipped on OSS builds.
-* Use `self.set_runtime_config(...)` to override the default pipeline config.
-  * Reset it at the end using `self.reset_runtime_config()`.
-* Access the shared pipeline via `self.pipeline`.
-
-#### Example
-
-```python
-from tests.shared_test_pipeline import SharedTestPipeline
-
-class TestAverage(SharedTestPipeline):
-    def test_average(self):
-        """
-        CREATE TABLE students(id INT, name STRING);
-        CREATE MATERIALIZED VIEW v AS SELECT * FROM students;
-        """
-        ...
-        self.pipeline.start()
-        self.pipeline.input_pandas("students", df)
-        self.pipeline.wait_for_completion(True)
-        ...
-```
-
-## Linting and formatting
-
-Use [Ruff] to run the lint checks that will be executed by the
-precommit hook when a PR is submitted:
-
-```bash
-ruff check python/
-```
-
-To reformat the code in the same way as the precommit hook:
-
-```bash
-ruff format
-```
-
-[Ruff]: https://github.com/astral-sh/ruff