beaver-db 0.8.0__tar.gz → 0.9.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of beaver-db might be problematic. Click here for more details.

@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: beaver-db
3
- Version: 0.8.0
3
+ Version: 0.9.0
4
4
  Summary: Fast, embedded, and multi-modal DB based on SQLite for AI-powered applications.
5
5
  Requires-Python: >=3.13
6
6
  Description-Content-Type: text/markdown
@@ -9,6 +9,10 @@ Requires-Dist: numpy>=2.3.3
9
9
  Requires-Dist: scipy>=1.16.2
10
10
  Dynamic: license-file
11
11
 
12
+ I've updated the README to highlight the new high-efficiency, thread-safe, and process-safe pub/sub system. I've also added an example of how you can use it to build real-time, event-driven applications.
13
+
14
+ Here are the changes:
15
+
12
16
  # beaver 🦫
13
17
 
14
18
  A fast, single-file, multi-modal database for Python, built with the standard `sqlite3` library.
@@ -27,7 +31,7 @@ A fast, single-file, multi-modal database for Python, built with the standard `s
27
31
 
28
32
  ## Core Features
29
33
 
30
- - **Synchronous Pub/Sub**: A simple, thread-safe, Redis-like publish-subscribe system for real-time messaging.
34
+ - **High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture.
31
35
  - **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
32
36
  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
33
37
  - **Persistent Priority Queue**: A high-performance, persistent queue that always returns the item with the highest priority, perfect for task management.
@@ -163,19 +167,36 @@ if response is None:
163
167
  api_cache.set("weather_new_york", response, ttl_seconds=3600)
164
168
  ```
165
169
 
170
+ ### 6. Real-time Event-Driven Systems
171
+
172
+ Use the **high-efficiency pub/sub system** to build applications where different components react to events in real-time. This is perfect for decoupled systems, real-time UIs, or monitoring services.
173
+
174
+ ```python
175
+ # In one process or thread (e.g., a monitoring service)
176
+ system_events = db.channel("system_events")
177
+ system_events.publish({"event": "user_login", "user_id": "alice"})
178
+
179
+ # In another process or thread (e.g., a UI updater or logger)
180
+ with db.channel("system_events").subscribe() as listener:
181
+ for message in listener.listen():
182
+ print(f"Event received: {message}")
183
+ # >> Event received: {'event': 'user_login', 'user_id': 'alice'}
184
+ ```
185
+
166
186
  ## More Examples
167
187
 
168
188
  For more in-depth examples, check out the scripts in the `examples/` directory:
169
189
 
170
- - [`examples/kvstore.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/kvstore.py%5D\(https://www.google.com/search%3Fq%3Dexamples/kvstore.py\)): A comprehensive demo of the namespaced dictionary feature.
171
- - [`examples/list.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/list.py%5D\(https://www.google.com/search%3Fq%3Dexamples/list.py\)): Shows the full capabilities of the persistent list, including slicing and in-place updates.
172
- - [`examples/queue.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/queue.py%5D\(https://www.google.com/search%3Fq%3Dexamples/queue.py\)): A practical example of using the persistent priority queue for task management.
173
- - [`examples/vector.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/vector.py%5D\(https://www.google.com/search%3Fq%3Dexamples/vector.py\)): Demonstrates how to index and search vector embeddings, including upserts.
174
- - [`examples/fts.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/fts.py%5D\(https://www.google.com/search%3Fq%3Dexamples/fts.py\)): A detailed look at full-text search, including targeted searches on specific metadata fields.
175
- - [`examples/graph.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/graph.py%5D\(https://www.google.com/search%3Fq%3Dexamples/graph.py\)): Shows how to create relationships between documents and perform multi-hop graph traversals.
176
- - [`examples/pubsub.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/pubsub.py%5D\(https://www.google.com/search%3Fq%3Dexamples/pubsub.py\)): A demonstration of the synchronous, thread-safe publish/subscribe system.
177
- - [`examples/cache.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/cache.py%5D\(https://www.google.com/search%3Fq%3Dexamples/cache.py\)): A practical example of using a dictionary with TTL as a cache for API calls.
178
- - [`examples/rerank.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/rerank.py%5D\(https://www.google.com/search%3Fq%3Dexamples/rerank.py\)): Shows how to combine results from vector and text search for more refined results.
190
+ - [`examples/kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
191
+ - [`examples/list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
192
+ - [`examples/queue.py`](examples/queue.py): A practical example of using the persistent priority queue for task management.
193
+ - [`examples/vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
194
+ - [`examples/fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
195
+ - [`examples/graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
196
+ - [`examples/pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
197
+ - [`examples/publisher.py`](examples/publisher.py) and [`examples/subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
198
+ - [`examples/cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
199
+ - [`examples/rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
179
200
 
180
201
  ## Roadmap
181
202
 
@@ -183,10 +204,9 @@ These are some of the features and improvements planned for future releases:
183
204
 
184
205
  - **Fuzzy search**: Implement fuzzy matching capabilities for text search.
185
206
  - **Faster ANN**: Explore integrating more advanced ANN libraries like `faiss` for improved vector search performance.
186
- - **Improved Pub/Sub**: Fan-out implementation with a more Pythonic API.
187
207
  - **Async API**: Comprehensive async support with on-demand wrappers for all collections.
188
208
 
189
- Check out the [roadmap](https://www.google.com/search?q=roadmap.md) for a detailed list of upcoming features and design ideas.
209
+ Check out the [roadmap](roadmap.md) for a detailed list of upcoming features and design ideas.
190
210
 
191
211
  ## License
192
212
 
@@ -1,3 +1,7 @@
1
+ I've updated the README to highlight the new high-efficiency, thread-safe, and process-safe pub/sub system. I've also added an example of how you can use it to build real-time, event-driven applications.
2
+
3
+ Here are the changes:
4
+
1
5
  # beaver 🦫
2
6
 
3
7
  A fast, single-file, multi-modal database for Python, built with the standard `sqlite3` library.
@@ -16,7 +20,7 @@ A fast, single-file, multi-modal database for Python, built with the standard `s
16
20
 
17
21
  ## Core Features
18
22
 
19
- - **Synchronous Pub/Sub**: A simple, thread-safe, Redis-like publish-subscribe system for real-time messaging.
23
+ - **High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture.
20
24
  - **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
21
25
  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
22
26
  - **Persistent Priority Queue**: A high-performance, persistent queue that always returns the item with the highest priority, perfect for task management.
@@ -152,19 +156,36 @@ if response is None:
152
156
  api_cache.set("weather_new_york", response, ttl_seconds=3600)
153
157
  ```
154
158
 
159
+ ### 6. Real-time Event-Driven Systems
160
+
161
+ Use the **high-efficiency pub/sub system** to build applications where different components react to events in real-time. This is perfect for decoupled systems, real-time UIs, or monitoring services.
162
+
163
+ ```python
164
+ # In one process or thread (e.g., a monitoring service)
165
+ system_events = db.channel("system_events")
166
+ system_events.publish({"event": "user_login", "user_id": "alice"})
167
+
168
+ # In another process or thread (e.g., a UI updater or logger)
169
+ with db.channel("system_events").subscribe() as listener:
170
+ for message in listener.listen():
171
+ print(f"Event received: {message}")
172
+ # >> Event received: {'event': 'user_login', 'user_id': 'alice'}
173
+ ```
174
+
155
175
  ## More Examples
156
176
 
157
177
  For more in-depth examples, check out the scripts in the `examples/` directory:
158
178
 
159
- - [`examples/kvstore.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/kvstore.py%5D\(https://www.google.com/search%3Fq%3Dexamples/kvstore.py\)): A comprehensive demo of the namespaced dictionary feature.
160
- - [`examples/list.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/list.py%5D\(https://www.google.com/search%3Fq%3Dexamples/list.py\)): Shows the full capabilities of the persistent list, including slicing and in-place updates.
161
- - [`examples/queue.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/queue.py%5D\(https://www.google.com/search%3Fq%3Dexamples/queue.py\)): A practical example of using the persistent priority queue for task management.
162
- - [`examples/vector.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/vector.py%5D\(https://www.google.com/search%3Fq%3Dexamples/vector.py\)): Demonstrates how to index and search vector embeddings, including upserts.
163
- - [`examples/fts.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/fts.py%5D\(https://www.google.com/search%3Fq%3Dexamples/fts.py\)): A detailed look at full-text search, including targeted searches on specific metadata fields.
164
- - [`examples/graph.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/graph.py%5D\(https://www.google.com/search%3Fq%3Dexamples/graph.py\)): Shows how to create relationships between documents and perform multi-hop graph traversals.
165
- - [`examples/pubsub.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/pubsub.py%5D\(https://www.google.com/search%3Fq%3Dexamples/pubsub.py\)): A demonstration of the synchronous, thread-safe publish/subscribe system.
166
- - [`examples/cache.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/cache.py%5D\(https://www.google.com/search%3Fq%3Dexamples/cache.py\)): A practical example of using a dictionary with TTL as a cache for API calls.
167
- - [`examples/rerank.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/rerank.py%5D\(https://www.google.com/search%3Fq%3Dexamples/rerank.py\)): Shows how to combine results from vector and text search for more refined results.
179
+ - [`examples/kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
180
+ - [`examples/list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
181
+ - [`examples/queue.py`](examples/queue.py): A practical example of using the persistent priority queue for task management.
182
+ - [`examples/vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
183
+ - [`examples/fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
184
+ - [`examples/graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
185
+ - [`examples/pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
186
+ - [`examples/publisher.py`](examples/publisher.py) and [`examples/subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
187
+ - [`examples/cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
188
+ - [`examples/rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
168
189
 
169
190
  ## Roadmap
170
191
 
@@ -172,10 +193,9 @@ These are some of the features and improvements planned for future releases:
172
193
 
173
194
  - **Fuzzy search**: Implement fuzzy matching capabilities for text search.
174
195
  - **Faster ANN**: Explore integrating more advanced ANN libraries like `faiss` for improved vector search performance.
175
- - **Improved Pub/Sub**: Fan-out implementation with a more Pythonic API.
176
196
  - **Async API**: Comprehensive async support with on-demand wrappers for all collections.
177
197
 
178
- Check out the [roadmap](https://www.google.com/search?q=roadmap.md) for a detailed list of upcoming features and design ideas.
198
+ Check out the [roadmap](roadmap.md) for a detailed list of upcoming features and design ideas.
179
199
 
180
200
  ## License
181
201
 
@@ -0,0 +1,185 @@
1
+ import json
2
+ import sqlite3
3
+ import threading
4
+ import time
5
+ from queue import Empty, Queue
6
+ from typing import Any, Iterator, Set
7
+
8
+ # A special message object used to signal the listener to gracefully shut down.
9
+ _SHUTDOWN_SENTINEL = object()
10
+
11
+
12
+ class Subscriber:
13
+ """
14
+ A thread-safe message receiver for a specific channel subscription.
15
+
16
+ This object is designed to be used as a context manager (`with` statement).
17
+ It holds a dedicated in-memory queue that receives messages from the
18
+ channel's central polling thread, ensuring that a slow listener does not
19
+ impact others.
20
+ """
21
+
22
+ def __init__(self, channel: "ChannelManager"):
23
+ self._channel = channel
24
+ self._queue: Queue = Queue()
25
+
26
+ def __enter__(self) -> "Subscriber":
27
+ """Registers the listener's queue with the channel to start receiving messages."""
28
+ self._channel._register(self._queue)
29
+ return self
30
+
31
+ def __exit__(self, exc_type, exc_val, exc_tb):
32
+ """Unregisters the listener's queue from the channel to stop receiving messages."""
33
+ self._channel._unregister(self._queue)
34
+
35
+ def listen(self, timeout: float | None = None) -> Iterator[Any]:
36
+ """
37
+ Returns a blocking iterator that yields messages as they arrive.
38
+
39
+ This method pulls messages from the listener's dedicated, thread-safe
40
+ in-memory queue. It performs no database operations itself.
41
+
42
+ Args:
43
+ timeout: If provided, the iterator will raise `queue.Empty` if no message is
44
+ received within this many seconds.
45
+ """
46
+ while True:
47
+ try:
48
+ msg = self._queue.get(timeout=timeout)
49
+
50
+ if msg is _SHUTDOWN_SENTINEL:
51
+ break
52
+
53
+ yield msg
54
+ except Empty:
55
+ raise TimeoutError(f"Timeout {timeout}s expired.")
56
+
57
+
58
+ class ChannelManager:
59
+ """
60
+ The central hub for a named pub/sub channel.
61
+
62
+ This object manages all active listeners for the channel and runs a single,
63
+ efficient background thread to poll the database for new messages. It then
64
+ "fans out" these messages to all subscribed listeners.
65
+ """
66
+
67
+ def __init__(
68
+ self,
69
+ name: str,
70
+ conn: sqlite3.Connection,
71
+ db_path: str,
72
+ poll_interval: float = 0.1,
73
+ ):
74
+ self._name = name
75
+ self._conn = conn
76
+ self._db_path = db_path
77
+ self._poll_interval = poll_interval
78
+ self._listeners: Set[Queue] = set()
79
+ self._lock = threading.Lock()
80
+ self._polling_thread: threading.Thread | None = None
81
+ self._stop_event = threading.Event()
82
+
83
+ def _register(self, queue: Queue):
84
+ """Adds a listener's queue and starts the poller if it's the first one."""
85
+
86
+ with self._lock:
87
+ self._listeners.add(queue)
88
+ # If the polling thread isn't running, start it.
89
+ if self._polling_thread is None or not self._polling_thread.is_alive():
90
+ self._start_polling()
91
+
92
+ def _unregister(self, queue: Queue):
93
+ """Removes a listener's queue and stops the poller if it's the last one."""
94
+
95
+ with self._lock:
96
+ self._listeners.discard(queue)
97
+ # If there are no more listeners, stop the polling thread to save resources.
98
+ if not self._listeners:
99
+ self._stop_polling()
100
+
101
+ def _start_polling(self):
102
+ """Starts the background polling thread."""
103
+ self._stop_event.clear()
104
+ self._polling_thread = threading.Thread(target=self._polling_loop, daemon=True)
105
+ self._polling_thread.start()
106
+
107
+ def _stop_polling(self):
108
+ """Signals the background polling thread to stop."""
109
+ if self._polling_thread and self._polling_thread.is_alive():
110
+ self._stop_event.set()
111
+ self._polling_thread.join()
112
+ self._polling_thread = None
113
+
114
+ def close(self):
115
+ """Reliable close this channel and removes listeners."""
116
+ self._stop_polling()
117
+
118
+ with self._lock:
119
+ for listener in self._listeners:
120
+ listener.put(_SHUTDOWN_SENTINEL)
121
+
122
+ self._listeners.clear()
123
+
124
+ def _polling_loop(self):
125
+ """
126
+ The main loop for the background thread.
127
+
128
+ This function polls the database for new messages and fans them out
129
+ to all registered listener queues.
130
+ """
131
+ # A separate SQLite connection is required for each thread.
132
+ thread_conn = sqlite3.connect(self._db_path, check_same_thread=False)
133
+ thread_conn.row_factory = sqlite3.Row
134
+
135
+ # The poller starts listening for messages from this moment forward.
136
+ last_seen_timestamp = time.time()
137
+
138
+
139
+ while not self._stop_event.is_set():
140
+ cursor = thread_conn.cursor()
141
+ cursor.execute(
142
+ "SELECT timestamp, message_payload FROM beaver_pubsub_log WHERE channel_name = ? AND timestamp > ? ORDER BY timestamp ASC",
143
+ (self._name, last_seen_timestamp),
144
+ )
145
+ messages = cursor.fetchall()
146
+ cursor.close()
147
+
148
+ if messages:
149
+ # Update the timestamp to the last message we've seen.
150
+ last_seen_timestamp = messages[-1]["timestamp"]
151
+
152
+ # The "fan-out": Push messages to all active listener queues.
153
+ # This block is locked to prevent modification of the listeners set
154
+ # while we are iterating over it.
155
+ with self._lock:
156
+ for queue in self._listeners:
157
+ for row in messages:
158
+ queue.put(json.loads(row["message_payload"]))
159
+
160
+ # Wait for the poll interval before checking for new messages again.
161
+ time.sleep(self._poll_interval)
162
+
163
+ thread_conn.close()
164
+
165
+ def subscribe(self) -> Subscriber:
166
+ """Creates a new subscription, returning a Listener context manager."""
167
+ return Subscriber(self)
168
+
169
+ def publish(self, payload: Any):
170
+ """
171
+ Publishes a JSON-serializable message to the channel.
172
+
173
+ This is a synchronous operation that performs a fast, atomic INSERT
174
+ into the database's pub/sub log.
175
+ """
176
+ try:
177
+ json_payload = json.dumps(payload)
178
+ except TypeError as e:
179
+ raise TypeError("Message payload must be JSON-serializable.") from e
180
+
181
+ with self._conn:
182
+ self._conn.execute(
183
+ "INSERT INTO beaver_pubsub_log (timestamp, channel_name, message_payload) VALUES (?, ?, ?)",
184
+ (time.time(), self._name, json_payload),
185
+ )
@@ -1,7 +1,5 @@
1
- import json
2
1
  import sqlite3
3
- import time
4
- from typing import Any
2
+ import threading
5
3
 
6
4
  from .dicts import DictManager
7
5
  from .lists import ListManager
@@ -29,6 +27,8 @@ class BeaverDB:
29
27
  self._conn.execute("PRAGMA journal_mode=WAL;")
30
28
  self._conn.row_factory = sqlite3.Row
31
29
  self._create_all_tables()
30
+ self._channels: dict[str, ChannelManager] = {}
31
+ self._channels_lock = threading.Lock()
32
32
 
33
33
  def _create_all_tables(self):
34
34
  """Initializes all required tables in the database file."""
@@ -170,6 +170,10 @@ class BeaverDB:
170
170
  def close(self):
171
171
  """Closes the database connection."""
172
172
  if self._conn:
173
+ # Cleanly shut down any active polling threads before closing
174
+ with self._channels_lock:
175
+ for channel in self._channels.values():
176
+ channel.close()
173
177
  self._conn.close()
174
178
 
175
179
  # --- Factory and Passthrough Methods ---
@@ -178,41 +182,39 @@ class BeaverDB:
178
182
  """Returns a wrapper object for interacting with a named dictionary."""
179
183
  if not isinstance(name, str) or not name:
180
184
  raise TypeError("Dictionary name must be a non-empty string.")
185
+
181
186
  return DictManager(name, self._conn)
182
187
 
183
188
  def list(self, name: str) -> ListManager:
184
189
  """Returns a wrapper object for interacting with a named list."""
185
190
  if not isinstance(name, str) or not name:
186
191
  raise TypeError("List name must be a non-empty string.")
192
+
187
193
  return ListManager(name, self._conn)
188
194
 
189
195
  def queue(self, name: str) -> QueueManager:
190
196
  """Returns a wrapper object for interacting with a persistent priority queue."""
191
197
  if not isinstance(name, str) or not name:
192
198
  raise TypeError("Queue name must be a non-empty string.")
199
+
193
200
  return QueueManager(name, self._conn)
194
201
 
195
202
  def collection(self, name: str) -> CollectionManager:
196
203
  """Returns a wrapper for interacting with a document collection."""
197
204
  if not isinstance(name, str) or not name:
198
205
  raise TypeError("Collection name must be a non-empty string.")
206
+
199
207
  return CollectionManager(name, self._conn)
200
208
 
201
- def publish(self, channel_name: str, payload: Any):
202
- """Publishes a JSON-serializable message to a channel. This is synchronous."""
203
- if not isinstance(channel_name, str) or not channel_name:
209
+ def channel(self, name: str) -> ChannelManager:
210
+ """
211
+ Returns a singleton Channel instance for high-efficiency pub/sub.
212
+ """
213
+ if not isinstance(name, str) or not name:
204
214
  raise ValueError("Channel name must be a non-empty string.")
205
- try:
206
- json_payload = json.dumps(payload)
207
- except TypeError as e:
208
- raise TypeError("Message payload must be JSON-serializable.") from e
209
-
210
- with self._conn:
211
- self._conn.execute(
212
- "INSERT INTO beaver_pubsub_log (timestamp, channel_name, message_payload) VALUES (?, ?, ?)",
213
- (time.time(), channel_name, json_payload),
214
- )
215
215
 
216
- def subscribe(self, channel_name: str) -> ChannelManager:
217
- """Subscribes to a channel, returning a synchronous iterator."""
218
- return ChannelManager(self._conn, channel_name)
216
+ # Use a thread-safe lock to ensure only one Channel object is created per name.
217
+ with self._channels_lock:
218
+ if name not in self._channels:
219
+ self._channels[name] = ChannelManager(name, self._conn, self._db_path)
220
+ return self._channels[name]
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: beaver-db
3
- Version: 0.8.0
3
+ Version: 0.9.0
4
4
  Summary: Fast, embedded, and multi-modal DB based on SQLite for AI-powered applications.
5
5
  Requires-Python: >=3.13
6
6
  Description-Content-Type: text/markdown
@@ -9,6 +9,10 @@ Requires-Dist: numpy>=2.3.3
9
9
  Requires-Dist: scipy>=1.16.2
10
10
  Dynamic: license-file
11
11
 
12
+ I've updated the README to highlight the new high-efficiency, thread-safe, and process-safe pub/sub system. I've also added an example of how you can use it to build real-time, event-driven applications.
13
+
14
+ Here are the changes:
15
+
12
16
  # beaver 🦫
13
17
 
14
18
  A fast, single-file, multi-modal database for Python, built with the standard `sqlite3` library.
@@ -27,7 +31,7 @@ A fast, single-file, multi-modal database for Python, built with the standard `s
27
31
 
28
32
  ## Core Features
29
33
 
30
- - **Synchronous Pub/Sub**: A simple, thread-safe, Redis-like publish-subscribe system for real-time messaging.
34
+ - **High-Efficiency Pub/Sub**: A powerful, thread and process-safe publish-subscribe system for real-time messaging with a fan-out architecture.
31
35
  - **Namespaced Key-Value Dictionaries**: A Pythonic, dictionary-like interface for storing any JSON-serializable object within separate namespaces with optional TTL for cache implementations.
32
36
  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
33
37
  - **Persistent Priority Queue**: A high-performance, persistent queue that always returns the item with the highest priority, perfect for task management.
@@ -163,19 +167,36 @@ if response is None:
163
167
  api_cache.set("weather_new_york", response, ttl_seconds=3600)
164
168
  ```
165
169
 
170
+ ### 6. Real-time Event-Driven Systems
171
+
172
+ Use the **high-efficiency pub/sub system** to build applications where different components react to events in real-time. This is perfect for decoupled systems, real-time UIs, or monitoring services.
173
+
174
+ ```python
175
+ # In one process or thread (e.g., a monitoring service)
176
+ system_events = db.channel("system_events")
177
+ system_events.publish({"event": "user_login", "user_id": "alice"})
178
+
179
+ # In another process or thread (e.g., a UI updater or logger)
180
+ with db.channel("system_events").subscribe() as listener:
181
+ for message in listener.listen():
182
+ print(f"Event received: {message}")
183
+ # >> Event received: {'event': 'user_login', 'user_id': 'alice'}
184
+ ```
185
+
166
186
  ## More Examples
167
187
 
168
188
  For more in-depth examples, check out the scripts in the `examples/` directory:
169
189
 
170
- - [`examples/kvstore.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/kvstore.py%5D\(https://www.google.com/search%3Fq%3Dexamples/kvstore.py\)): A comprehensive demo of the namespaced dictionary feature.
171
- - [`examples/list.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/list.py%5D\(https://www.google.com/search%3Fq%3Dexamples/list.py\)): Shows the full capabilities of the persistent list, including slicing and in-place updates.
172
- - [`examples/queue.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/queue.py%5D\(https://www.google.com/search%3Fq%3Dexamples/queue.py\)): A practical example of using the persistent priority queue for task management.
173
- - [`examples/vector.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/vector.py%5D\(https://www.google.com/search%3Fq%3Dexamples/vector.py\)): Demonstrates how to index and search vector embeddings, including upserts.
174
- - [`examples/fts.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/fts.py%5D\(https://www.google.com/search%3Fq%3Dexamples/fts.py\)): A detailed look at full-text search, including targeted searches on specific metadata fields.
175
- - [`examples/graph.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/graph.py%5D\(https://www.google.com/search%3Fq%3Dexamples/graph.py\)): Shows how to create relationships between documents and perform multi-hop graph traversals.
176
- - [`examples/pubsub.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/pubsub.py%5D\(https://www.google.com/search%3Fq%3Dexamples/pubsub.py\)): A demonstration of the synchronous, thread-safe publish/subscribe system.
177
- - [`examples/cache.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/cache.py%5D\(https://www.google.com/search%3Fq%3Dexamples/cache.py\)): A practical example of using a dictionary with TTL as a cache for API calls.
178
- - [`examples/rerank.py`](https://www.google.com/search?q=%5Bhttps://www.google.com/search%3Fq%3Dexamples/rerank.py%5D\(https://www.google.com/search%3Fq%3Dexamples/rerank.py\)): Shows how to combine results from vector and text search for more refined results.
190
+ - [`examples/kvstore.py`](examples/kvstore.py): A comprehensive demo of the namespaced dictionary feature.
191
+ - [`examples/list.py`](examples/list.py): Shows the full capabilities of the persistent list, including slicing and in-place updates.
192
+ - [`examples/queue.py`](examples/queue.py): A practical example of using the persistent priority queue for task management.
193
+ - [`examples/vector.py`](examples/vector.py): Demonstrates how to index and search vector embeddings, including upserts.
194
+ - [`examples/fts.py`](examples/fts.py): A detailed look at full-text search, including targeted searches on specific metadata fields.
195
+ - [`examples/graph.py`](examples/graph.py): Shows how to create relationships between documents and perform multi-hop graph traversals.
196
+ - [`examples/pubsub.py`](examples/pubsub.py): A demonstration of the synchronous, thread-safe publish/subscribe system in a single process.
197
+ - [`examples/publisher.py`](examples/publisher.py) and [`examples/subscriber.py`](examples/subscriber.py): A pair of examples demonstrating inter-process message passing with the publish/subscribe system.
198
+ - [`examples/cache.py`](examples/cache.py): A practical example of using a dictionary with TTL as a cache for API calls.
199
+ - [`examples/rerank.py`](examples/rerank.py): Shows how to combine results from vector and text search for more refined results.
179
200
 
180
201
  ## Roadmap
181
202
 
@@ -183,10 +204,9 @@ These are some of the features and improvements planned for future releases:
183
204
 
184
205
  - **Fuzzy search**: Implement fuzzy matching capabilities for text search.
185
206
  - **Faster ANN**: Explore integrating more advanced ANN libraries like `faiss` for improved vector search performance.
186
- - **Improved Pub/Sub**: Fan-out implementation with a more Pythonic API.
187
207
  - **Async API**: Comprehensive async support with on-demand wrappers for all collections.
188
208
 
189
- Check out the [roadmap](https://www.google.com/search?q=roadmap.md) for a detailed list of upcoming features and design ideas.
209
+ Check out the [roadmap](roadmap.md) for a detailed list of upcoming features and design ideas.
190
210
 
191
211
  ## License
192
212
 
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "beaver-db"
3
- version = "0.8.0"
3
+ version = "0.9.0"
4
4
  description = "Fast, embedded, and multi-modal DB based on SQLite for AI-powered applications."
5
5
  readme = "README.md"
6
6
  requires-python = ">=3.13"
@@ -1,54 +0,0 @@
1
- import json
2
- import sqlite3
3
- import time
4
- from typing import Any, Iterator
5
-
6
-
7
- class ChannelManager(Iterator):
8
- """
9
- A synchronous, blocking iterator that polls a channel for new messages.
10
- """
11
-
12
- def __init__(
13
- self, conn: sqlite3.Connection, channel_name: str, poll_interval: float = 0.1
14
- ):
15
- """
16
- Initializes the synchronous subscriber.
17
-
18
- Args:
19
- conn: The SQLite database connection.
20
- channel_name: The name of the channel to subscribe to.
21
- poll_interval: The time in seconds to wait between polling for new messages.
22
- """
23
- self._conn = conn
24
- self._channel = channel_name
25
- self._poll_interval = poll_interval
26
- self._last_seen_timestamp = time.time()
27
-
28
- def __iter__(self) -> "ChannelManager":
29
- """Returns the iterator object itself."""
30
- return self
31
-
32
- def __next__(self) -> Any:
33
- """
34
- Blocks until a new message is available on the channel and returns it.
35
- This polling mechanism is simple but can introduce a slight latency
36
- equivalent to the poll_interval.
37
- """
38
- while True:
39
- # Fetch the next available message from the database
40
- cursor = self._conn.cursor()
41
- cursor.execute(
42
- "SELECT timestamp, message_payload FROM beaver_pubsub_log WHERE channel_name = ? AND timestamp > ? ORDER BY timestamp ASC LIMIT 1",
43
- (self._channel, self._last_seen_timestamp),
44
- )
45
- result = cursor.fetchone()
46
- cursor.close()
47
-
48
- if result:
49
- # If a message is found, update the timestamp and return the payload
50
- self._last_seen_timestamp = result["timestamp"]
51
- return json.loads(result["message_payload"])
52
- else:
53
- # If no new messages, wait for the poll interval before trying again
54
- time.sleep(self._poll_interval)
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes