datafun-streaming 0.1.0__tar.gz → 0.3.0__tar.gz

{datafun_streaming-0.1.0 → datafun_streaming-0.3.0}/CHANGELOG.md

@@ -13,6 +13,34 @@ and this project adheres to **[Semantic Versioning](https://semver.org/spec/v2.0
 
 ---
 
+## [0.3.0] - 2026-05-08
+
+### Added
+
+- `datafun_streaming.storage.duckdb_sql` - pure SQL string builder functions
+  (`build_create_table_sql`, `build_clear_table_sql`, `build_insert_sql`)
+  with no database connection required, fully testable in isolation
+- Tests for all three SQL builder functions in `tests/test_duckdb_sql.py`
+
+### Changed
+
+- `upsert_row` now requires caller-supplied `allowed_tables: frozenset[str]`
+  parameter - removes the module-level placeholder allowlist and gives
+  callers full control over which tables are permitted
+- Removed `_ALLOWED_TABLE_NAMES` placeholder constant from `duckdb_utils.py`
+- pytest `minversion` updated to `9.0`
+
+---
+
+## [0.2.0] - 2026-05-08
+
+### Changed
+
+- updated README.md
+- added `npx markdownlint-cli2 --fix` to pre-commit
+
+---
+
 ## [0.1.0] - 2026-05-08
 
 ### Added
@@ -62,7 +90,7 @@ Follow these steps exactly when creating a new release.
 1.2. CHANGELOG.md: add section, move unreleased entries, update links
 1.3. pyproject.toml - update [tool.hatch.version].fallback-version (near the end)
 
-### Task 2. Sync Version and Validate
+### Task 2. Validate
 
 ```shell
 uv sync --extra dev --extra docs --upgrade
@@ -72,7 +100,6 @@ uv run python -m pyright
 uv run python -m pytest
 uv run python -m zensical build
 uvx validate-pyproject[all] pyproject.toml
-
 uv build
 uv run python -m twine check dist/\*
 ```
@@ -101,7 +128,9 @@ git push origin :refs/tags/vX.Z.Y
 
 ## Links
 
-[Unreleased]: https://github.com/denisecase/datafun-streaming/compare/v0.1.0...HEAD
+[Unreleased]: https://github.com/denisecase/datafun-streaming/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/denisecase/datafun-streaming/releases/tag/v0.3.0
+[0.2.0]: https://github.com/denisecase/datafun-streaming/releases/tag/v0.2.0
 [0.1.0]: https://github.com/denisecase/datafun-streaming/releases/tag/v0.1.0
 
 <!-- markdownlint-enable MD024 -->

{datafun_streaming-0.1.0 → datafun_streaming-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: datafun-streaming
-Version: 0.1.0
+Version: 0.3.0
 Summary: Utilities for streaming data analytics with Kafka and DuckDB.
 Project-URL: Homepage, https://github.com/denisecase/datafun-streaming
 Project-URL: Repository, https://github.com/denisecase/datafun-streaming
@@ -43,7 +43,7 @@ Description-Content-Type: text/markdown
 [![PyPI](https://img.shields.io/pypi/v/datafun-streaming?logo=pypi&label=pypi)](https://pypi.org/project/datafun-streaming/)
 [![Docs Site](https://img.shields.io/badge/docs-site-blue?logo=github)](https://denisecase.github.io/datafun-streaming/)
 [![Repo](https://img.shields.io/badge/repo-GitHub-black?logo=github)](https://github.com/denisecase/datafun-streaming)
-[![Python 3.15+](https://img.shields.io/badge/python-3.15%2B-blue?logo=python)](https://github.com/denisecase/datafun-streaming/blob/main/pyproject.toml)
+[![Python 3.14+](https://img.shields.io/badge/python-3.14%2B-blue?logo=python)](https://github.com/denisecase/datafun-streaming/blob/main/pyproject.toml)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
 [![CI](https://github.com/denisecase/datafun-streaming/actions/workflows/ci-python-zensical.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/ci-python-zensical.yml)
@@ -101,68 +101,3 @@ git push -u origin main
 ```
 
 </details>
-
-## Notes
-
-- Use the **UP ARROW** and **DOWN ARROW** in the terminal to scroll through past commands.
-- Use `CTRL+f` to find (and replace) text within a file.
-- You do not need to add to or modify `tests/`. They are provided for example only.
-- Many files are silent helpers. Explore as you like, but nothing is required.
-- You do NOT not to understand everything; understanding builds naturally over time.
-
-## Troubleshooting >>> or
-
-If you see something like this in your terminal: `>>>` or `...`
-You accidentally started Python interactive mode.
-It happens.
-Press `Ctrl+c` (both keys together) or `Ctrl+Z` then `Enter` on Windows.
-
-## Example Output
-
-```shell
-| INFO | P01 | ========================
-| INFO | P01 | START main()
-| INFO | P01 | ========================
-| INFO | P01 | ROOT_DIR = .
-| INFO | P01 | DATA_DIR = data
-| INFO | P01 | OUTPUT_CSV = data\sales.csv
-| INFO | P01 | Streaming 3 sales to C:\Repos\streaming\datafun-streaming\data\sales.csv ...
-| INFO | P01 | Watch each sale arrive. Press CTRL+C to stop early.
-
-| INFO | P01 | (1, 81.87, 'Backpack', 'East')
-| INFO | P01 | Generated formatted multi-line SUMMARY string.
-| INFO | P01 | Returning the str to the calling function.
-| INFO | P01 |
-    Descriptive Statistics for Streaming Sales Amounts ($):
-        Count of sales   : 1
-        Minimum sale     : $81.87
-        Maximum sale     : $81.87
-        Average sale     : $81.87
-        Standard deviation: $0.00
-
-| INFO | P01 | (2, 101.58, 'Water Bottle', 'North')
-| INFO | P01 | Generated formatted multi-line SUMMARY string.
-| INFO | P01 | Returning the str to the calling function.
-| INFO | P01 |
-    Descriptive Statistics for Streaming Sales Amounts ($):
-        Count of sales   : 2
-        Minimum sale     : $81.87
-        Maximum sale     : $101.58
-        Average sale     : $91.72
-        Standard deviation: $13.94
-
-| INFO | P01 | (3, 27.15, 'Running Shoes', 'East')
-| INFO | P01 | Generated formatted multi-line SUMMARY string.
-| INFO | P01 | Returning the str to the calling function.
-| INFO | P01 |
-    Descriptive Statistics for Streaming Sales Amounts ($):
-        Count of sales   : 3
-        Minimum sale     : $27.15
-        Maximum sale     : $101.58
-        Average sale     : $70.20
-        Standard deviation: $38.56
-
-| INFO | P01 | ========================
-| INFO | P01 | Producer executed successfully!
-| INFO | P01 | ========================
-```

datafun_streaming-0.3.0/README.md

@@ -0,0 +1,64 @@
+# datafun-streaming
+
+[![Adaptive Interfaces](https://img.shields.io/badge/adaptive--interfaces-schema-blue?logo=github)](https://github.com/denisecase)
+[![PyPI](https://img.shields.io/pypi/v/datafun-streaming?logo=pypi&label=pypi)](https://pypi.org/project/datafun-streaming/)
+[![Docs Site](https://img.shields.io/badge/docs-site-blue?logo=github)](https://denisecase.github.io/datafun-streaming/)
+[![Repo](https://img.shields.io/badge/repo-GitHub-black?logo=github)](https://github.com/denisecase/datafun-streaming)
+[![Python 3.14+](https://img.shields.io/badge/python-3.14%2B-blue?logo=python)](https://github.com/denisecase/datafun-streaming/blob/main/pyproject.toml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+[![CI](https://github.com/denisecase/datafun-streaming/actions/workflows/ci-python-zensical.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/ci-python-zensical.yml)
+[![Docs](https://github.com/denisecase/datafun-streaming/actions/workflows/deploy-zensical.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/deploy-zensical.yml)
+[![Links](https://github.com/denisecase/datafun-streaming/actions/workflows/links.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/links.yml)
+
+> Shared Python utilities for Kafka, DuckDB, validation, stats, and visualization
+> across streaming data analytics projects.
+
+## Command Reference
+
+<details>
+<summary>Show command reference</summary>
+
+### In a machine terminal
+
+Open a machine terminal where you want the project:
+
+```shell
+git clone https://github.com/denisecase/datafun-streaming
+
+cd datafun-streaming
+code .
+```
+
+### In a VS Code terminal
+
+```shell
+# reset uv cache only after suspected cache corruption or strange dependency errors
+# uv cache clean
+
+uv self update
+uv python pin 3.14
+uv sync --extra dev --extra docs --upgrade
+
+uvx pre-commit install
+
+git add -A
+uvx pre-commit run --all-files
+# repeat if changes were made
+git add -A
+uvx pre-commit run --all-files
+
+# do chores
+uv run python -m ruff format .
+uv run python -m ruff check . --fix
+uv run python -m pyright
+uv run python -m pytest
+uv run python -m zensical build
+
+# save progress
+git add -A
+git commit -m "update"
+git push -u origin main
+```
+
+</details>

{datafun_streaming-0.1.0 → datafun_streaming-0.3.0}/pyproject.toml

@@ -110,7 +110,7 @@ reportCallIssue = "none"
 
 [tool.pytest.ini_options]
 # WHY: Consistent test discovery and coverage visibility.
-minversion = "7.0"
+minversion = "9.0"
 testpaths = ["tests"]
 addopts = "--cov=datafun_streaming --cov-report=term-missing --cov-fail-under=50"
 
@@ -226,5 +226,5 @@ packages = ["src/datafun_streaming"]  # REQ.PACKAGES: Discovery rooted at src/.
 [tool.hatch.version]
 # WHY: Version derived from git tags at build time, no manual source file edits.
 source = "vcs"
-fallback-version = "0.1.0"              # Used when no git tags present (fresh clone, CI).
+fallback-version = "0.3.0"              # Used when no git tags present (fresh clone, CI).
 tag-pattern = "^(?:v)?(?P<version>\\d+\\.\\d+\\.\\d+)$"

{datafun_streaming-0.1.0 → datafun_streaming-0.3.0}/src/datafun_streaming/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.1.0'
-__version_tuple__ = version_tuple = (0, 1, 0)
+__version__ = version = '0.3.0'
+__version_tuple__ = version_tuple = (0, 3, 0)
 
 __commit_id__ = commit_id = None

datafun_streaming-0.3.0/src/datafun_streaming/storage/duckdb_sql.py

@@ -0,0 +1,83 @@
+"""src/datafun_streaming/storage/duckdb_sql.py.
+
+Pure SQL string builders for DuckDB streaming tables.
+
+These functions build SQL statements as strings.
+They require no database connection and have no side effects.
+All use module-level constant table names, never raw user input.
+
+Author: Denise Case
+Date: 2026-05
+"""
+
+# === EXPORTS ===
+
+__all__ = [
+    "build_create_table_sql",
+    "build_clear_table_sql",
+    "build_insert_sql",
+]
+
+
+# === DEFINE SQL BUILDER FUNCTIONS ===
+
+
+def build_create_table_sql(table_name: str, fieldnames: list[str]) -> str:
+    """Build a DuckDB CREATE TABLE IF NOT EXISTS statement.
+
+    All columns are declared as VARCHAR.
+    Use this to initialize tables before writing records.
+
+    Arguments:
+        table_name: The table to create.
+        fieldnames: The field names to include as VARCHAR columns.
+
+    Returns:
+        A CREATE TABLE IF NOT EXISTS SQL string.
+
+    Example:
+        sql = build_create_table_sql("sales", ["order_id", "region_id"])
+        connection.execute(sql)
+    """
+    columns = ", ".join(f"{field} VARCHAR" for field in fieldnames)
+    return f"CREATE TABLE IF NOT EXISTS {table_name} ({columns})"  # noqa: S608 - caller responsible for validated table name
+
+
+def build_clear_table_sql(table_name: str) -> str:
+    """Build a DuckDB DELETE statement to clear all rows from a table.
+
+    Use this at the start of a consumer run to reset the table.
+
+    Arguments:
+        table_name: The table to clear.
+
+    Returns:
+        A DELETE FROM SQL string.
+
+    Example:
+        sql = build_clear_table_sql("sales")
+        connection.execute(sql)
+    """
+    return f"DELETE FROM {table_name}"  # noqa: S608 - caller responsible for validated table name
+
+
+def build_insert_sql(table_name: str, fieldnames: list[str]) -> str:
+    """Build a DuckDB INSERT statement with ? parameter placeholders.
+
+    The returned SQL expects one ? per field, passed as a list of values
+    to connection.execute().
+
+    Arguments:
+        table_name: The table to insert into.
+        fieldnames: The field names to insert.
+
+    Returns:
+        An INSERT INTO SQL string with ? placeholders.
+
+    Example:
+        sql = build_insert_sql("sales", ["order_id", "region_id"])
+        connection.execute(sql, ["S001", "US-MO"])
+    """
+    columns = ", ".join(fieldnames)
+    placeholders = ", ".join("?" for _ in fieldnames)
+    return f"INSERT INTO {table_name} ({columns}) VALUES ({placeholders})"  # noqa: S608 - caller responsible for validated table name

{datafun_streaming-0.1.0 → datafun_streaming-0.3.0}/src/datafun_streaming/storage/duckdb_utils.py

@@ -45,16 +45,6 @@ _DUCKDB_TYPE_MAP: dict[type, str] = {
     bool: "BOOLEAN",
 }
 
-# === DECLARE SQL-SAFE TABLE NAMES ===
-
-_ALLOWED_TABLE_NAMES: frozenset[str] = frozenset(
-    {
-        "valid_table",
-        "rejected_table",
-    }
-)
-
-
 # === DEFINE FUNCTIONS ===
 
 

datafun_streaming-0.1.0/README.md

@@ -1,129 +0,0 @@
-# datafun-streaming
-
-[![Adaptive Interfaces](https://img.shields.io/badge/adaptive--interfaces-schema-blue?logo=github)](https://github.com/denisecase)
-[![PyPI](https://img.shields.io/pypi/v/datafun-streaming?logo=pypi&label=pypi)](https://pypi.org/project/datafun-streaming/)
-[![Docs Site](https://img.shields.io/badge/docs-site-blue?logo=github)](https://denisecase.github.io/datafun-streaming/)
-[![Repo](https://img.shields.io/badge/repo-GitHub-black?logo=github)](https://github.com/denisecase/datafun-streaming)
-[![Python 3.15+](https://img.shields.io/badge/python-3.15%2B-blue?logo=python)](https://github.com/denisecase/datafun-streaming/blob/main/pyproject.toml)
-[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
-
-[![CI](https://github.com/denisecase/datafun-streaming/actions/workflows/ci-python-zensical.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/ci-python-zensical.yml)
-[![Docs](https://github.com/denisecase/datafun-streaming/actions/workflows/deploy-zensical.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/deploy-zensical.yml)
-[![Links](https://github.com/denisecase/datafun-streaming/actions/workflows/links.yml/badge.svg?branch=main)](https://github.com/denisecase/datafun-streaming/actions/workflows/links.yml)
-
-> Shared Python utilities for Kafka, DuckDB, validation, stats, and visualization
-> across streaming data analytics projects.
-
-## Command Reference
-
-<details>
-<summary>Show command reference</summary>
-
-### In a machine terminal
-
-Open a machine terminal where you want the project:
-
-```shell
-git clone https://github.com/denisecase/datafun-streaming
-
-cd datafun-streaming
-code .
-```
-
-### In a VS Code terminal
-
-```shell
-# reset uv cache only after suspected cache corruption or strange dependency errors
-# uv cache clean
-
-uv self update
-uv python pin 3.14
-uv sync --extra dev --extra docs --upgrade
-
-uvx pre-commit install
-
-git add -A
-uvx pre-commit run --all-files
-# repeat if changes were made
-git add -A
-uvx pre-commit run --all-files
-
-# do chores
-uv run python -m ruff format .
-uv run python -m ruff check . --fix
-uv run python -m pyright
-uv run python -m pytest
-uv run python -m zensical build
-
-# save progress
-git add -A
-git commit -m "update"
-git push -u origin main
-```
-
-</details>
-
-## Notes
-
-- Use the **UP ARROW** and **DOWN ARROW** in the terminal to scroll through past commands.
-- Use `CTRL+f` to find (and replace) text within a file.
-- You do not need to add to or modify `tests/`. They are provided for example only.
-- Many files are silent helpers. Explore as you like, but nothing is required.
-- You do NOT not to understand everything; understanding builds naturally over time.
-
-## Troubleshooting >>> or
-
-If you see something like this in your terminal: `>>>` or `...`
-You accidentally started Python interactive mode.
-It happens.
-Press `Ctrl+c` (both keys together) or `Ctrl+Z` then `Enter` on Windows.
-
-## Example Output
-
-```shell
-| INFO | P01 | ========================
-| INFO | P01 | START main()
-| INFO | P01 | ========================
-| INFO | P01 | ROOT_DIR = .
-| INFO | P01 | DATA_DIR = data
-| INFO | P01 | OUTPUT_CSV = data\sales.csv
-| INFO | P01 | Streaming 3 sales to C:\Repos\streaming\datafun-streaming\data\sales.csv ...
-| INFO | P01 | Watch each sale arrive. Press CTRL+C to stop early.
-
-| INFO | P01 | (1, 81.87, 'Backpack', 'East')
-| INFO | P01 | Generated formatted multi-line SUMMARY string.
-| INFO | P01 | Returning the str to the calling function.
-| INFO | P01 |
-    Descriptive Statistics for Streaming Sales Amounts ($):
-        Count of sales   : 1
-        Minimum sale     : $81.87
-        Maximum sale     : $81.87
-        Average sale     : $81.87
-        Standard deviation: $0.00
-
-| INFO | P01 | (2, 101.58, 'Water Bottle', 'North')
-| INFO | P01 | Generated formatted multi-line SUMMARY string.
-| INFO | P01 | Returning the str to the calling function.
-| INFO | P01 |
-    Descriptive Statistics for Streaming Sales Amounts ($):
-        Count of sales   : 2
-        Minimum sale     : $81.87
-        Maximum sale     : $101.58
-        Average sale     : $91.72
-        Standard deviation: $13.94
-
-| INFO | P01 | (3, 27.15, 'Running Shoes', 'East')
-| INFO | P01 | Generated formatted multi-line SUMMARY string.
-| INFO | P01 | Returning the str to the calling function.
-| INFO | P01 |
-    Descriptive Statistics for Streaming Sales Amounts ($):
-        Count of sales   : 3
-        Minimum sale     : $27.15
-        Maximum sale     : $101.58
-        Average sale     : $70.20
-        Standard deviation: $38.56
-
-| INFO | P01 | ========================
-| INFO | P01 | Producer executed successfully!
-| INFO | P01 | ========================
-```