beaver-db 0.14.0__tar.gz → 0.16.0__tar.gz

{beaver_db-0.14.0/beaver_db.egg-info → beaver_db-0.16.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: beaver-db
-Version: 0.14.0
+Version: 0.16.0
 Summary: Fast, embedded, and multi-modal DB based on SQLite for AI-powered applications.
 Requires-Python: >=3.13
 Description-Content-Type: text/markdown
@@ -19,24 +19,26 @@ A fast, single-file, multi-modal database for Python, built with the standard `s
 
 `beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
 
-  - **Minimalistic**: Uses only Python's standard libraries (`sqlite3`) and `numpy`/`faiss-cpu`.
-  - **Schemaless**: Flexible data storage without rigid schemas across all modalities.
-  - **Synchronous, Multi-Process, and Thread-Safe**: Designed for simplicity and safety in multi-threaded and multi-process environments.
-  - **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
-  - **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications. Vector search is accelerated with a high-performance, persistent `faiss` index.
-  - **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
+- **Minimalistic**: Uses only Python's standard libraries (`sqlite3`) and `numpy`/`faiss-cpu`.
+- **Schemaless**: Flexible data storage without rigid schemas across all modalities.
+- **Synchronous, Multi-Process, and Thread-Safe**: Designed for simplicity and safety in multi-threaded and multi-process environments.
+- **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
+- **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications. Vector search is accelerated with a high-performance, persistent `faiss` index.
+- **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
 
 ## Core Features
 
-  - **Sync/Async High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture. Sync by default, but with an `as_async` wrapper for async applications.
-  - **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
-  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
-  - **Persistent Priority Queue**: A high-performance, persistent queue that always returns the item with the highest priority, perfect for task management.
-  - **Simple Blob Storage**: A dictionary-like interface for storing medium-sized binary files (like PDFs or images) directly in the database, ensuring transactional integrity with your other data.
-  - **High-Performance Vector Storage & Search**: Store vector embeddings and perform fast, crash-safe approximate nearest neighbor searches using a `faiss`-based hybrid index.
-  - **Full-Text and Fuzzy Search**: Automatically index and search through document metadata using SQLite's powerful FTS5 engine, enhanced with optional fuzzy search for typo-tolerant matching.
-  - **Knowledge Graph**: Create relationships between documents and traverse the graph to find neighbors or perform multi-hop walks.
-  - **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+- **Sync/Async High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture. Sync by default, but with an `as_async` wrapper for async applications.
+- **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
+- **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
+- **Persistent Priority Queue**: A high-performance, persistent priority queue perfect for task orchestration across multiple processes. Also with optional async support.
+- **Time-Indexed Log for Monitoring**: A specialized data structure for structured, time-series logs. Query historical data by time range or create a live, aggregated view of the most recent events for real-time dashboards.
+- **Simple Blob Storage**: A dictionary-like interface for storing medium-sized binary files (like PDFs or images) directly in the database, ensuring transactional integrity with your other data.
+- **High-Performance Vector Storage & Search**: Store vector embeddings and perform fast, crash-safe approximate nearest neighbor searches using a `faiss`-based hybrid index.
+- **Full-Text and Fuzzy Search**: Automatically index and search through document metadata using SQLite's powerful FTS5 engine, enhanced with optional fuzzy search for typo-tolerant matching.
+- **Knowledge Graph**: Create relationships between documents and traverse the graph to find neighbors or perform multi-hop walks.
+- **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+- **Optional Type-Safety:** Although the database is schemaless, you can use a minimalistic typing system for automatic serialization and deserialization that is Pydantic-compatible out of the box.
 
 ## How Beaver is Implemented
 
@@ -219,6 +221,29 @@ attachments.put(
 avatar = attachments.get("user_123_avatar.png")
 ```
 
+### 8. Real-time Application Monitoring
+
+Use the **time-indexed log** to monitor your application's health in real-time. The `live()` method provides a continuously updating, aggregated view of your log data, perfect for building simple dashboards directly in your terminal.
+
+```python
+from datetime import timedelta
+import statistics
+
+logs = db.log("system_metrics")
+
+def summarize(window):
+    values = [log.get("value", 0) for log in window]
+    return {"mean": statistics.mean(values), "count": len(values)}
+
+live_summary = logs.live(
+    window_duration=timedelta(seconds=10),
+    sampling_period=timedelta(seconds=1),
+    aggregator=summarize
+)
+
+for summary in live_summary:
+    print(f"Live Stats (10s window): Count={summary['count']}, Mean={summary['mean']:.2f}")
+```
 
 ## Type-Safe Data Models
 
@@ -260,21 +285,24 @@ Basically everywhere you can store or get some object in BeaverDB, you can use a
 
 For more in-depth examples, check out the scripts in the `examples/` directory:
 
-  - [`examples/async_pubsub.py`](examples/async_pubsub.py): A demonstration of the asynchronous wrapper for the publish/subscribe system.
-  - [`examples/blobs.py`](examples/blobs.py): Demonstrates how to store and retrieve binary data in the database.
-  - [`examples/cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
-  - [`examples/fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
-  - [`examples/fuzzy.py`](examples/fuzzy.py): Demonstrates fuzzy search capabilities for text search.
-  - [`examples/general_test.py`](examples/general_test.py): A general-purpose test to run all operations randomly which allows testing long-running processes and synchronicity issues.
-  - [`examples/graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
-  - [`examples/kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
-  - [`examples/list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
-  - [`examples/publisher.py`](examples/publisher.py) and [`examples/subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
-  - [`examples/pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
-  - [`examples/queue.py`](examples/queue.py): A practical example of using the persistent priority queue for task management.
-  - [`examples/rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
-  - [`examples/stress_vectors.py`](examples/stress_vectors.py): A stress test for the vector search functionality.
-  - [`examples/vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
+  - [`async_pubsub.py`](examples/async_pubsub.py): A demonstration of the asynchronous wrapper for the publish/subscribe system.
+  - [`blobs.py`](examples/blobs.py): Demonstrates how to store and retrieve binary data in the database.
+  - [`cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
+  - [`fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
+  - [`fuzzy.py`](examples/fuzzy.py): Demonstrates fuzzy search capabilities for text search.
+  - [`general_test.py`](examples/general_test.py): A general-purpose test to run all operations randomly which allows testing long-running processes and synchronicity issues.
+  - [`graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
+  - [`kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
+  - [`list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
+  - [`pqueue.py`](examples/pqueue.py): A practical example of using the persistent priority queue for task management.
+  - [`producer_consumer.py`](examples/producer_consumer.py): A demonstration of the distributed task queue system in a multi-process environment.
+  - [`publisher.py`](examples/publisher.py) and [`subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
+  - [`pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
+  - [`rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
+  - [`stress_vectors.py`](examples/stress_vectors.py): A stress test for the vector search functionality.
+  - [`textual_chat.py`](examples/textual_chat.py): A chat application built with `textual` and `beaver` to illustrate the use of several primitives (lists, dicts, and channels) at the same time.
+  - [`type_hints.py](examples/type_hints.py): Shows how to use type hints with `beaver` to get better IDE support and type safety.
+  - [`vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
 
 ## Roadmap
 
@@ -283,7 +311,6 @@ For more in-depth examples, check out the scripts in the `examples/` directory:
 These are some of the features and improvements planned for future releases:
 
   - **Async API**: Extend the async support with on-demand wrappers for all features besides channels.
-  - **Type Hints**: Extend type hints for channels and documents.
 
 Check out the [roadmap](roadmap.md) for a detailed list of upcoming features and design ideas.
 

{beaver_db-0.14.0 → beaver_db-0.16.0}/README.md

@@ -8,24 +8,26 @@ A fast, single-file, multi-modal database for Python, built with the standard `s
 
 `beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
 
-  - **Minimalistic**: Uses only Python's standard libraries (`sqlite3`) and `numpy`/`faiss-cpu`.
-  - **Schemaless**: Flexible data storage without rigid schemas across all modalities.
-  - **Synchronous, Multi-Process, and Thread-Safe**: Designed for simplicity and safety in multi-threaded and multi-process environments.
-  - **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
-  - **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications. Vector search is accelerated with a high-performance, persistent `faiss` index.
-  - **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
+- **Minimalistic**: Uses only Python's standard libraries (`sqlite3`) and `numpy`/`faiss-cpu`.
+- **Schemaless**: Flexible data storage without rigid schemas across all modalities.
+- **Synchronous, Multi-Process, and Thread-Safe**: Designed for simplicity and safety in multi-threaded and multi-process environments.
+- **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
+- **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications. Vector search is accelerated with a high-performance, persistent `faiss` index.
+- **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
 
 ## Core Features
 
-  - **Sync/Async High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture. Sync by default, but with an `as_async` wrapper for async applications.
-  - **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
-  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
-  - **Persistent Priority Queue**: A high-performance, persistent queue that always returns the item with the highest priority, perfect for task management.
-  - **Simple Blob Storage**: A dictionary-like interface for storing medium-sized binary files (like PDFs or images) directly in the database, ensuring transactional integrity with your other data.
-  - **High-Performance Vector Storage & Search**: Store vector embeddings and perform fast, crash-safe approximate nearest neighbor searches using a `faiss`-based hybrid index.
-  - **Full-Text and Fuzzy Search**: Automatically index and search through document metadata using SQLite's powerful FTS5 engine, enhanced with optional fuzzy search for typo-tolerant matching.
-  - **Knowledge Graph**: Create relationships between documents and traverse the graph to find neighbors or perform multi-hop walks.
-  - **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+- **Sync/Async High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture. Sync by default, but with an `as_async` wrapper for async applications.
+- **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
+- **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
+- **Persistent Priority Queue**: A high-performance, persistent priority queue perfect for task orchestration across multiple processes. Also with optional async support.
+- **Time-Indexed Log for Monitoring**: A specialized data structure for structured, time-series logs. Query historical data by time range or create a live, aggregated view of the most recent events for real-time dashboards.
+- **Simple Blob Storage**: A dictionary-like interface for storing medium-sized binary files (like PDFs or images) directly in the database, ensuring transactional integrity with your other data.
+- **High-Performance Vector Storage & Search**: Store vector embeddings and perform fast, crash-safe approximate nearest neighbor searches using a `faiss`-based hybrid index.
+- **Full-Text and Fuzzy Search**: Automatically index and search through document metadata using SQLite's powerful FTS5 engine, enhanced with optional fuzzy search for typo-tolerant matching.
+- **Knowledge Graph**: Create relationships between documents and traverse the graph to find neighbors or perform multi-hop walks.
+- **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+- **Optional Type-Safety:** Although the database is schemaless, you can use a minimalistic typing system for automatic serialization and deserialization that is Pydantic-compatible out of the box.
 
 ## How Beaver is Implemented
 
@@ -208,6 +210,29 @@ attachments.put(
 avatar = attachments.get("user_123_avatar.png")
 ```
 
+### 8. Real-time Application Monitoring
+
+Use the **time-indexed log** to monitor your application's health in real-time. The `live()` method provides a continuously updating, aggregated view of your log data, perfect for building simple dashboards directly in your terminal.
+
+```python
+from datetime import timedelta
+import statistics
+
+logs = db.log("system_metrics")
+
+def summarize(window):
+    values = [log.get("value", 0) for log in window]
+    return {"mean": statistics.mean(values), "count": len(values)}
+
+live_summary = logs.live(
+    window_duration=timedelta(seconds=10),
+    sampling_period=timedelta(seconds=1),
+    aggregator=summarize
+)
+
+for summary in live_summary:
+    print(f"Live Stats (10s window): Count={summary['count']}, Mean={summary['mean']:.2f}")
+```
 
 ## Type-Safe Data Models
 
@@ -249,21 +274,24 @@ Basically everywhere you can store or get some object in BeaverDB, you can use a
 
 For more in-depth examples, check out the scripts in the `examples/` directory:
 
-  - [`examples/async_pubsub.py`](examples/async_pubsub.py): A demonstration of the asynchronous wrapper for the publish/subscribe system.
-  - [`examples/blobs.py`](examples/blobs.py): Demonstrates how to store and retrieve binary data in the database.
-  - [`examples/cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
-  - [`examples/fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
-  - [`examples/fuzzy.py`](examples/fuzzy.py): Demonstrates fuzzy search capabilities for text search.
-  - [`examples/general_test.py`](examples/general_test.py): A general-purpose test to run all operations randomly which allows testing long-running processes and synchronicity issues.
-  - [`examples/graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
-  - [`examples/kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
-  - [`examples/list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
-  - [`examples/publisher.py`](examples/publisher.py) and [`examples/subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
-  - [`examples/pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
-  - [`examples/queue.py`](examples/queue.py): A practical example of using the persistent priority queue for task management.
-  - [`examples/rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
-  - [`examples/stress_vectors.py`](examples/stress_vectors.py): A stress test for the vector search functionality.
-  - [`examples/vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
+  - [`async_pubsub.py`](examples/async_pubsub.py): A demonstration of the asynchronous wrapper for the publish/subscribe system.
+  - [`blobs.py`](examples/blobs.py): Demonstrates how to store and retrieve binary data in the database.
+  - [`cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
+  - [`fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
+  - [`fuzzy.py`](examples/fuzzy.py): Demonstrates fuzzy search capabilities for text search.
+  - [`general_test.py`](examples/general_test.py): A general-purpose test to run all operations randomly which allows testing long-running processes and synchronicity issues.
+  - [`graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
+  - [`kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
+  - [`list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
+  - [`pqueue.py`](examples/pqueue.py): A practical example of using the persistent priority queue for task management.
+  - [`producer_consumer.py`](examples/producer_consumer.py): A demonstration of the distributed task queue system in a multi-process environment.
+  - [`publisher.py`](examples/publisher.py) and [`subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
+  - [`pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
+  - [`rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
+  - [`stress_vectors.py`](examples/stress_vectors.py): A stress test for the vector search functionality.
+  - [`textual_chat.py`](examples/textual_chat.py): A chat application built with `textual` and `beaver` to illustrate the use of several primitives (lists, dicts, and channels) at the same time.
+  - [`type_hints.py](examples/type_hints.py): Shows how to use type hints with `beaver` to get better IDE support and type safety.
+  - [`vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
 
 ## Roadmap
 
@@ -272,7 +300,6 @@ For more in-depth examples, check out the scripts in the `examples/` directory:
 These are some of the features and improvements planned for future releases:
 
   - **Async API**: Extend the async support with on-demand wrappers for all features besides channels.
-  - **Type Hints**: Extend type hints for channels and documents.
 
 Check out the [roadmap](roadmap.md) for a detailed list of upcoming features and design ideas.
 

{beaver_db-0.14.0 → beaver_db-0.16.0}/beaver/core.py

@@ -8,6 +8,7 @@ from .channels import ChannelManager
 from .collections import CollectionManager, Document
 from .dicts import DictManager
 from .lists import ListManager
+from .logs import LogManager
 from .queues import QueueManager
 
 
@@ -51,11 +52,31 @@ class BeaverDB:
             self._create_edges_table()
             self._create_fts_table()
             self._create_list_table()
+            self._create_logs_table()
             self._create_priority_queue_table()
             self._create_pubsub_table()
             self._create_trigrams_table()
             self._create_versions_table()
 
+    def _create_logs_table(self):
+        """Creates the table for time-indexed logs."""
+        self._conn.execute(
+            """
+            CREATE TABLE IF NOT EXISTS beaver_logs (
+                log_name TEXT NOT NULL,
+                timestamp REAL NOT NULL,
+                data TEXT NOT NULL,
+                PRIMARY KEY (log_name, timestamp)
+            )
+            """
+        )
+        self._conn.execute(
+            """
+            CREATE INDEX IF NOT EXISTS idx_logs_timestamp
+            ON beaver_logs (log_name, timestamp)
+            """
+        )
+
     def _create_blobs_table(self):
         """Creates the table for storing named blobs."""
         self._conn.execute(
@@ -343,3 +364,16 @@ class BeaverDB:
             raise TypeError("Blob store name must be a non-empty string.")
 
         return BlobManager(name, self._conn, model)
+
+    def log[T](self, name: str, model: type[T] | None = None) -> LogManager[T]:
+        """
+        Returns a wrapper for interacting with a named, time-indexed log.
+        If model is defined, it should be a type used for automatic (de)serialization.
+        """
+        if not isinstance(name, str) or not name:
+            raise TypeError("Log name must be a non-empty string.")
+
+        if model and not isinstance(model, JsonSerializable):
+            raise TypeError("The model parameter must be a JsonSerializable class.")
+
+        return LogManager(name, self._conn, self._db_path, model)

beaver_db-0.16.0/beaver/logs.py

@@ -0,0 +1,278 @@
+import asyncio
+import collections
+import json
+import sqlite3
+import threading
+import time
+from datetime import datetime, timedelta, timezone
+from queue import Empty, Queue
+from typing import Any, AsyncIterator, Callable, Iterator, Type, TypeVar
+
+from .types import JsonSerializable
+
+
+# A special message object used to signal the iterator to gracefully shut down.
+_SHUTDOWN_SENTINEL = object()
+
+
+class LiveIterator[T,R]:
+    """
+    A thread-safe, blocking iterator that yields aggregated results from a
+    rolling window of log data.
+    """
+
+    def __init__(
+        self,
+        db_path: str,
+        log_name: str,
+        window: timedelta,
+        period: timedelta,
+        aggregator: Callable[[list[T]], R],
+        deserializer: Callable[[str], T],
+    ):
+        self._db_path = db_path
+        self._log_name = log_name
+        self._window_duration_seconds = window.total_seconds()
+        self._sampling_period_seconds = period.total_seconds()
+        self._aggregator = aggregator
+        self._deserializer = deserializer
+        self._queue: Queue = Queue()
+        self._stop_event = threading.Event()
+        self._thread: threading.Thread | None = None
+
+    def _polling_loop(self):
+        """The main loop for the background thread that queries and aggregates data."""
+        # Each thread needs its own database connection.
+        thread_conn = sqlite3.connect(self._db_path, check_same_thread=False)
+        thread_conn.row_factory = sqlite3.Row
+
+        window_deque: collections.deque[tuple[float, T]] = collections.deque()
+        last_seen_timestamp = 0.0
+
+        # --- Initial window population ---
+        now = datetime.now(timezone.utc).timestamp()
+        start_time = now - self._window_duration_seconds
+        cursor = thread_conn.cursor()
+        cursor.execute(
+            "SELECT timestamp, data FROM beaver_logs WHERE log_name = ? AND timestamp >= ? ORDER BY timestamp ASC",
+            (self._log_name, start_time),
+        )
+        for row in cursor:
+            ts, data_str = row
+            window_deque.append((ts, self._deserializer(data_str)))
+            last_seen_timestamp = max(last_seen_timestamp, ts)
+
+        # Yield the first result
+        try:
+            initial_result = self._aggregator([item[1] for item in window_deque])
+            self._queue.put(initial_result)
+        except Exception as e:
+            # Propagate aggregator errors to the main thread
+            self._queue.put(e)
+
+        # --- Continuous polling loop ---
+        while not self._stop_event.is_set():
+            time.sleep(self._sampling_period_seconds)
+
+            # Fetch only new data since the last check
+            cursor.execute(
+                "SELECT timestamp, data FROM beaver_logs WHERE log_name = ? AND timestamp > ? ORDER BY timestamp ASC",
+                (self._log_name, last_seen_timestamp),
+            )
+            for row in cursor:
+                ts, data_str = row
+                window_deque.append((ts, self._deserializer(data_str)))
+                last_seen_timestamp = max(last_seen_timestamp, ts)
+
+            # Evict old data from the left of the deque
+            now = datetime.now(timezone.utc).timestamp()
+            eviction_time = now - self._window_duration_seconds
+            while window_deque and window_deque[0][0] < eviction_time:
+                window_deque.popleft()
+
+            # Run aggregator and yield the new result
+            try:
+                new_result = self._aggregator([item[1] for item in window_deque])
+                self._queue.put(new_result)
+            except Exception as e:
+                self._queue.put(e)
+
+        thread_conn.close()
+
+    def __iter__(self) -> "LiveIterator[T,R]":
+        self._thread = threading.Thread(target=self._polling_loop, daemon=True)
+        self._thread.start()
+        return self
+
+    def __next__(self) -> R:
+        result = self._queue.get()
+        if result is _SHUTDOWN_SENTINEL:
+            raise StopIteration
+        if isinstance(result, Exception):
+            # If the background thread put an exception in the queue, re-raise it
+            raise result
+        return result
+
+    def close(self):
+        """Stops the background polling thread."""
+        self._stop_event.set()
+        self._queue.put(_SHUTDOWN_SENTINEL)
+        if self._thread:
+            self._thread.join()
+
+
+class AsyncLiveIterator[T,R]:
+    """An async wrapper for the LiveIterator."""
+
+    def __init__(self, sync_iterator: LiveIterator[T,R]):
+        self._sync_iterator = sync_iterator
+
+    async def __anext__(self) -> R:
+        try:
+            return await asyncio.to_thread(self._sync_iterator.__next__)
+        except StopIteration:
+            raise StopAsyncIteration
+
+    def __aiter__(self) -> "AsyncLiveIterator[T,R]":
+        # The synchronous iterator's __iter__ method starts the thread.
+        # This is non-blocking, so it's safe to call directly.
+        self._sync_iterator.__iter__()
+        return self
+
+    def close(self):
+        self._sync_iterator.close()
+
+
+class AsyncLogManager[T]:
+    """An async-compatible wrapper for the LogManager."""
+
+    def __init__(self, sync_manager: "LogManager[T]"):
+        self._sync_manager = sync_manager
+
+    async def log(self, data: T, timestamp: datetime | None = None) -> None:
+        """Asynchronously adds a new entry to the log."""
+        await asyncio.to_thread(self._sync_manager.log, data, timestamp)
+
+    async def range(self, start: datetime, end: datetime) -> list[T]:
+        """Asynchronously retrieves all log entries within a specific time window."""
+        return await asyncio.to_thread(self._sync_manager.range, start, end)
+
+    def live[R](
+        self,
+        window: timedelta,
+        period: timedelta,
+        aggregator: Callable[[list[T]], R],
+    ) -> AsyncIterator[R]:
+        """Returns an async, infinite iterator for real-time log analysis."""
+        sync_iterator = self._sync_manager.live(
+            window, period, aggregator
+        )
+        return AsyncLiveIterator(sync_iterator)
+
+
+class LogManager[T]:
+    """
+    A wrapper for interacting with a named, time-indexed log, providing
+    type-safe and async-compatible methods.
+    """
+
+    def __init__(
+        self,
+        name: str,
+        conn: sqlite3.Connection,
+        db_path: str,
+        model: Type[T] | None = None,
+    ):
+        self._name = name
+        self._conn = conn
+        self._db_path = db_path
+        self._model = model
+
+    def _serialize(self, value: T) -> str:
+        """Serializes the given value to a JSON string."""
+        if isinstance(value, JsonSerializable):
+            return value.model_dump_json()
+
+        return json.dumps(value)
+
+    def _deserialize(self, value: str) -> T:
+        """Deserializes a JSON string into the specified model or a generic object."""
+        if self._model:
+            return self._model.model_validate_json(value)
+
+        return json.loads(value)
+
+    def log(self, data: T, timestamp: datetime | None = None) -> None:
+        """
+        Adds a new entry to the log.
+
+        Args:
+            data: The JSON-serializable data to store. If a model is used, this
+                  should be an instance of that model.
+            timestamp: A timezone-naive datetime object. If not provided,
+                       `datetime.now()` is used.
+        """
+        ts = timestamp or datetime.now(timezone.utc)
+        ts_float = ts.timestamp()
+
+        with self._conn:
+            self._conn.execute(
+                "INSERT INTO beaver_logs (log_name, timestamp, data) VALUES (?, ?, ?)",
+                (self._name, ts_float, self._serialize(data)),
+            )
+
+    def range(self, start: datetime, end: datetime) -> list[T]:
+        """
+        Retrieves all log entries within a specific time window.
+
+        Args:
+            start: The start of the time range (inclusive).
+            end: The end of the time range (inclusive).
+
+        Returns:
+            A list of log entries, deserialized into the specified model if provided.
+        """
+        start_ts = start.timestamp()
+        end_ts = end.timestamp()
+
+        cursor = self._conn.cursor()
+        cursor.execute(
+            "SELECT data FROM beaver_logs WHERE log_name = ? AND timestamp BETWEEN ? AND ? ORDER BY timestamp ASC",
+            (self._name, start_ts, end_ts),
+        )
+        return [self._deserialize(row["data"]) for row in cursor.fetchall()]
+
+    def live[R](
+        self,
+        window: timedelta,
+        period: timedelta,
+        aggregator: Callable[[list[T]], R],
+    ) -> Iterator[R]:
+        """
+        Returns a blocking, infinite iterator for real-time log analysis.
+
+        This maintains a sliding window of log entries and yields the result
+        of an aggregator function at specified intervals.
+
+        Args:
+            window: The duration of the sliding window (e.g., `timedelta(minutes=5)`).
+            period: The interval at which to update and yield a new result
+                             (e.g., `timedelta(seconds=10)`).
+            aggregator: A function that takes a list of log entries (the window) and
+                        returns a single, aggregated result.
+
+        Returns:
+            An iterator that yields the results of the aggregator.
+        """
+        return LiveIterator(
+            db_path=self._db_path,
+            log_name=self._name,
+            window=window,
+            period=period,
+            aggregator=aggregator,
+            deserializer=self._deserialize,
+        )
+
+    def as_async(self) -> AsyncLogManager[T]:
+        """Returns an async-compatible version of the log manager."""
+        return AsyncLogManager(self)

{beaver_db-0.14.0 → beaver_db-0.16.0}/beaver/queues.py

@@ -1,3 +1,4 @@
+import asyncio
 import json
 import sqlite3
 import time
@@ -14,8 +15,34 @@ class QueueItem[T](NamedTuple):
     data: T
 
 
+class AsyncQueueManager[T]:
+    """An async wrapper for the producer-consumer priority queue."""
+
+    def __init__(self, queue: "QueueManager[T]"):
+        self._queue = queue
+
+    async def put(self, data: T, priority: float):
+        """Asynchronously adds an item to the queue with a specific priority."""
+        await asyncio.to_thread(self._queue.put, data, priority)
+
+    @overload
+    async def get(self, block: Literal[True] = True, timeout: float | None = None) -> QueueItem[T]: ...
+    @overload
+    async def get(self, block: Literal[False]) -> QueueItem[T]: ...
+
+    async def get(self, block: bool = True, timeout: float | None = None) -> QueueItem[T]:
+        """
+        Asynchronously and atomically retrieves the highest-priority item.
+        This method will run the synchronous blocking logic in a separate thread.
+        """
+        return await asyncio.to_thread(self._queue.get, block=block, timeout=timeout)
+
+
 class QueueManager[T]:
-    """A wrapper providing a Pythonic interface to a persistent priority queue."""
+    """
+    A wrapper providing a Pythonic interface to a persistent, multi-process
+    producer-consumer priority queue.
+    """
 
     def __init__(self, name: str, conn: sqlite3.Connection, model: Type[T] | None = None):
         self._name = name
@@ -50,19 +77,13 @@ class QueueManager[T]:
                 (self._name, priority, time.time(), self._serialize(data)),
             )
 
-    @overload
-    def get(self, safe:Literal[True]) -> QueueItem[T] | None: ...
-    @overload
-    def get(self) -> QueueItem[T]: ...
-
-    def get(self, safe:bool=False) -> QueueItem[T] | None:
+    def _get_item_atomically(self) -> QueueItem[T] | None:
         """
-        Atomically retrieves and removes the highest-priority item from the queue.
-        If the queue is empty, returns None if safe is True, otherwise (the default) raises IndexError.
+        Performs a single, atomic attempt to retrieve and remove the
+        highest-priority item from the queue. Returns None if the queue is empty.
         """
         with self._conn:
             cursor = self._conn.cursor()
-            # The compound index on (queue_name, priority, timestamp) makes this query efficient.
             cursor.execute(
                 """
                 SELECT rowid, priority, timestamp, data
@@ -76,19 +97,61 @@ class QueueManager[T]:
             result = cursor.fetchone()
 
             if result is None:
-                if safe:
-                    return None
-                else:
-                    raise IndexError("No item available.")
+                return None
 
             rowid, priority, timestamp, data = result
-            # Delete the retrieved item to ensure it's processed only once.
             cursor.execute("DELETE FROM beaver_priority_queues WHERE rowid = ?", (rowid,))
 
             return QueueItem(
                 priority=priority, timestamp=timestamp, data=self._deserialize(data)
             )
 
+    @overload
+    def get(self, block: Literal[True] = True, timeout: float | None = None) -> QueueItem[T]: ...
+    @overload
+    def get(self, block: Literal[False]) -> QueueItem[T]: ...
+
+    def get(self, block: bool = True, timeout: float | None = None) -> QueueItem[T]:
+        """
+        Atomically retrieves and removes the highest-priority item from the queue.
+
+        This method is designed for producer-consumer patterns and can block
+        until an item becomes available.
+
+        Args:
+            block: If True (default), the method will wait until an item is available.
+            timeout: If `block` is True, this specifies the maximum number of seconds
+                     to wait. If the timeout is reached, `TimeoutError` is raised.
+
+        Returns:
+            A `QueueItem` containing the retrieved data.
+
+        Raises:
+            IndexError: If `block` is False and the queue is empty.
+            TimeoutError: If `block` is True and the timeout expires.
+        """
+        if not block:
+            item = self._get_item_atomically()
+            if item is None:
+                raise IndexError("get from an empty queue.")
+            return item
+
+        start_time = time.time()
+        while True:
+            item = self._get_item_atomically()
+            if item is not None:
+                return item
+
+            if timeout is not None and (time.time() - start_time) > timeout:
+                raise TimeoutError("Timeout expired while waiting for an item.")
+
+            # Sleep for a short interval to avoid busy-waiting and consuming CPU.
+            time.sleep(0.1)
+
+    def as_async(self) -> "AsyncQueueManager[T]":
+        """Returns an async version of the queue manager."""
+        return AsyncQueueManager(self)
+
     def __len__(self) -> int:
         """Returns the current number of items in the queue."""
         cursor = self._conn.cursor()

{beaver_db-0.14.0 → beaver_db-0.16.0/beaver_db.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: beaver-db
-Version: 0.14.0
+Version: 0.16.0
 Summary: Fast, embedded, and multi-modal DB based on SQLite for AI-powered applications.
 Requires-Python: >=3.13
 Description-Content-Type: text/markdown
@@ -19,24 +19,26 @@ A fast, single-file, multi-modal database for Python, built with the standard `s
 
 `beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
 
-  - **Minimalistic**: Uses only Python's standard libraries (`sqlite3`) and `numpy`/`faiss-cpu`.
-  - **Schemaless**: Flexible data storage without rigid schemas across all modalities.
-  - **Synchronous, Multi-Process, and Thread-Safe**: Designed for simplicity and safety in multi-threaded and multi-process environments.
-  - **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
-  - **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications. Vector search is accelerated with a high-performance, persistent `faiss` index.
-  - **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
+- **Minimalistic**: Uses only Python's standard libraries (`sqlite3`) and `numpy`/`faiss-cpu`.
+- **Schemaless**: Flexible data storage without rigid schemas across all modalities.
+- **Synchronous, Multi-Process, and Thread-Safe**: Designed for simplicity and safety in multi-threaded and multi-process environments.
+- **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
+- **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications. Vector search is accelerated with a high-performance, persistent `faiss` index.
+- **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
 
 ## Core Features
 
-  - **Sync/Async High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture. Sync by default, but with an `as_async` wrapper for async applications.
-  - **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
-  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
-  - **Persistent Priority Queue**: A high-performance, persistent queue that always returns the item with the highest priority, perfect for task management.
-  - **Simple Blob Storage**: A dictionary-like interface for storing medium-sized binary files (like PDFs or images) directly in the database, ensuring transactional integrity with your other data.
-  - **High-Performance Vector Storage & Search**: Store vector embeddings and perform fast, crash-safe approximate nearest neighbor searches using a `faiss`-based hybrid index.
-  - **Full-Text and Fuzzy Search**: Automatically index and search through document metadata using SQLite's powerful FTS5 engine, enhanced with optional fuzzy search for typo-tolerant matching.
-  - **Knowledge Graph**: Create relationships between documents and traverse the graph to find neighbors or perform multi-hop walks.
-  - **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+- **Sync/Async High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture. Sync by default, but with an `as_async` wrapper for async applications.
+- **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
+- **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
+- **Persistent Priority Queue**: A high-performance, persistent priority queue perfect for task orchestration across multiple processes. Also with optional async support.
+- **Time-Indexed Log for Monitoring**: A specialized data structure for structured, time-series logs. Query historical data by time range or create a live, aggregated view of the most recent events for real-time dashboards.
+- **Simple Blob Storage**: A dictionary-like interface for storing medium-sized binary files (like PDFs or images) directly in the database, ensuring transactional integrity with your other data.
+- **High-Performance Vector Storage & Search**: Store vector embeddings and perform fast, crash-safe approximate nearest neighbor searches using a `faiss`-based hybrid index.
+- **Full-Text and Fuzzy Search**: Automatically index and search through document metadata using SQLite's powerful FTS5 engine, enhanced with optional fuzzy search for typo-tolerant matching.
+- **Knowledge Graph**: Create relationships between documents and traverse the graph to find neighbors or perform multi-hop walks.
+- **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+- **Optional Type-Safety:** Although the database is schemaless, you can use a minimalistic typing system for automatic serialization and deserialization that is Pydantic-compatible out of the box.
 
 ## How Beaver is Implemented
 
@@ -219,6 +221,29 @@ attachments.put(
 avatar = attachments.get("user_123_avatar.png")
 ```
 
+### 8. Real-time Application Monitoring
+
+Use the **time-indexed log** to monitor your application's health in real-time. The `live()` method provides a continuously updating, aggregated view of your log data, perfect for building simple dashboards directly in your terminal.
+
+```python
+from datetime import timedelta
+import statistics
+
+logs = db.log("system_metrics")
+
+def summarize(window):
+    values = [log.get("value", 0) for log in window]
+    return {"mean": statistics.mean(values), "count": len(values)}
+
+live_summary = logs.live(
+    window_duration=timedelta(seconds=10),
+    sampling_period=timedelta(seconds=1),
+    aggregator=summarize
+)
+
+for summary in live_summary:
+    print(f"Live Stats (10s window): Count={summary['count']}, Mean={summary['mean']:.2f}")
+```
 
 ## Type-Safe Data Models
 
@@ -260,21 +285,24 @@ Basically everywhere you can store or get some object in BeaverDB, you can use a
 
 For more in-depth examples, check out the scripts in the `examples/` directory:
 
-  - [`examples/async_pubsub.py`](examples/async_pubsub.py): A demonstration of the asynchronous wrapper for the publish/subscribe system.
-  - [`examples/blobs.py`](examples/blobs.py): Demonstrates how to store and retrieve binary data in the database.
-  - [`examples/cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
-  - [`examples/fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
-  - [`examples/fuzzy.py`](examples/fuzzy.py): Demonstrates fuzzy search capabilities for text search.
-  - [`examples/general_test.py`](examples/general_test.py): A general-purpose test to run all operations randomly which allows testing long-running processes and synchronicity issues.
-  - [`examples/graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
-  - [`examples/kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
-  - [`examples/list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
-  - [`examples/publisher.py`](examples/publisher.py) and [`examples/subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
-  - [`examples/pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
-  - [`examples/queue.py`](examples/queue.py): A practical example of using the persistent priority queue for task management.
-  - [`examples/rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
-  - [`examples/stress_vectors.py`](examples/stress_vectors.py): A stress test for the vector search functionality.
-  - [`examples/vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
+  - [`async_pubsub.py`](examples/async_pubsub.py): A demonstration of the asynchronous wrapper for the publish/subscribe system.
+  - [`blobs.py`](examples/blobs.py): Demonstrates how to store and retrieve binary data in the database.
+  - [`cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
+  - [`fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
+  - [`fuzzy.py`](examples/fuzzy.py): Demonstrates fuzzy search capabilities for text search.
+  - [`general_test.py`](examples/general_test.py): A general-purpose test to run all operations randomly which allows testing long-running processes and synchronicity issues.
+  - [`graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
+  - [`kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
+  - [`list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
+  - [`pqueue.py`](examples/pqueue.py): A practical example of using the persistent priority queue for task management.
+  - [`producer_consumer.py`](examples/producer_consumer.py): A demonstration of the distributed task queue system in a multi-process environment.
+  - [`publisher.py`](examples/publisher.py) and [`subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
+  - [`pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
+  - [`rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
+  - [`stress_vectors.py`](examples/stress_vectors.py): A stress test for the vector search functionality.
+  - [`textual_chat.py`](examples/textual_chat.py): A chat application built with `textual` and `beaver` to illustrate the use of several primitives (lists, dicts, and channels) at the same time.
+  - [`type_hints.py](examples/type_hints.py): Shows how to use type hints with `beaver` to get better IDE support and type safety.
+  - [`vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
 
 ## Roadmap
 
@@ -283,7 +311,6 @@ For more in-depth examples, check out the scripts in the `examples/` directory:
 These are some of the features and improvements planned for future releases:
 
   - **Async API**: Extend the async support with on-demand wrappers for all features besides channels.
-  - **Type Hints**: Extend type hints for channels and documents.
 
 Check out the [roadmap](roadmap.md) for a detailed list of upcoming features and design ideas.
 

{beaver_db-0.14.0 → beaver_db-0.16.0}/beaver_db.egg-info/SOURCES.txt

@@ -8,6 +8,7 @@ beaver/collections.py
 beaver/core.py
 beaver/dicts.py
 beaver/lists.py
+beaver/logs.py
 beaver/queues.py
 beaver/types.py
 beaver/vectors.py

{beaver_db-0.14.0 → beaver_db-0.16.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "beaver-db"
-version = "0.14.0"
+version = "0.16.0"
 description = "Fast, embedded, and multi-modal DB based on SQLite for AI-powered applications."
 readme = "README.md"
 requires-python = ">=3.13"
@@ -11,3 +11,8 @@ dependencies = [
 
 [tool.hatch.build.targets.wheel]
 packages = ["beaver"]
+
+[dependency-groups]
+dev = [
+    "textual>=6.1.0",
+]