collections-cache 0.2.7.20250303__tar.gz → 0.2.9.20250303__tar.gz

collections_cache-0.2.9.20250303/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.1
+Name: collections-cache
+Version: 0.2.9.20250303
+Summary: collections-cache is a Python package for managing data collections across multiple SQLite databases. It allows efficient storage, retrieval, and updating of key-value pairs, supporting various data types serialized with pickle. The package uses parallel processing for fast access and manipulation of large collections.
+License: MIT
+Author: Luiz-Trindade
+Author-email: luiz.gabriel.m.trindade@gmail.com
+Requires-Python: >=3.12,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Description-Content-Type: text/markdown
+
+# collections-cache 🚀
+
+`collections-cache` is a fast and scalable key–value caching solution built on top of SQLite. It allows you to store, update, and retrieve data using unique keys, and it supports complex Python data types (thanks to `pickle`). Designed to harness the power of multiple CPU cores, the library shards data across multiple SQLite databases, enabling impressive performance scaling.
+
+---
+
+## Features ✨
+
+- **Multiple SQLite Databases**: Distributes your data across several databases to optimize I/O and take advantage of multi-core systems.
+- **Key–Value Store**: Simple and intuitive interface for storing and retrieving data.
+- **Supports Complex Data Types**: Serialize and store lists, dictionaries, objects, and more using `pickle`.
+- **Parallel Processing**: Uses Python’s `multiprocessing` and `concurrent.futures` modules to perform operations in parallel.
+- **Efficient Data Retrieval**: Caches all keys in memory for super-fast lookups.
+- **Cross-Platform**: Runs on Linux, macOS, and Windows.
+- **Performance Scaling**: Benchmarks show near-linear scaling with the number of real CPU cores.
+
+---
+
+## Installation 📦
+
+Use [Poetry](https://python-poetry.org/) to install and manage dependencies:
+
+1. Clone the repository:
+
+    ```bash
+    git clone https://github.com/Luiz-Trindade/collections_cache.git
+    cd collections-cache
+    ```
+
+2. Install the package with Poetry:
+
+    ```bash
+    poetry install
+    ```
+
+---
+
+## Usage ⚙️
+
+Simply import and start using the main class, `Collection_Cache`, to interact with your collection:
+
+### Basic Example
+
+```python
+from collections_cache import Collection_Cache
+
+# Create a new collection named "STORE"
+cache = Collection_Cache("STORE")
+
+# Set a key-value pair
+cache.set_key("products", ["apple", "orange", "onion"])
+
+# Retrieve the value by key
+products = cache.get_key("products")
+print(products)  # Output: ['apple', 'orange', 'onion']
+```
+
+### Bulk Insertion Example
+
+For faster insertions, accumulate your data and use `set_multi_keys`:
+
+```python
+from collections_cache import Collection_Cache
+from random import uniform, randint
+from time import time
+
+cache = Collection_Cache("web_cache")
+insertions = 100_000
+data = {}
+
+# Generate data
+for i in range(insertions):
+    key = str(uniform(0.0, 100.0))
+    value = "some text :)" * randint(1, 100)
+    data[key] = value
+
+# Bulk insert keys using multi-threaded execution
+cache.set_multi_keys(data)
+
+print(f"Inserted {len(cache.keys())} keys successfully!")
+```
+
+---
+
+## API Overview 📚
+
+- **`set_key(key, value)`**: Stores a key–value pair. Updates the value if the key already exists.
+- **`set_multi_keys(key_and_value)`**: (Experimental) Inserts multiple key–value pairs in parallel.
+- **`get_key(key)`**: Retrieves the value associated with a given key.
+- **`delete_key(key)`**: Removes a key and its corresponding value.
+- **`keys()`**: Returns a list of all stored keys.
+- **`export_to_json()`**: (Future feature) Exports your collection to a JSON file.
+
+---
+
+## Performance Benchmark 📊
+
+On a machine with 4 real CPU cores, benchmarks indicate around **781 insertions per second**. The library is designed to scale nearly linearly with the number of real cores. For example:
+- **6 cores**: ~1,171 insertions per second.
+- **16 cores**: ~3,125 insertions per second.
+- **128 cores**: ~25,000 insertions per second (theoretically).
+
+*Note: Actual performance will depend on disk I/O, SQLite contention, and system architecture.*
+
+---
+
+## Development & Contributing 👩‍💻👨‍💻
+
+To contribute or run tests:
+
+1. Install development dependencies:
+
+    ```bash
+    poetry install --dev
+    ```
+
+2. Run tests using:
+
+    ```bash
+    poetry run pytest
+    ```
+
+Feel free to submit issues, pull requests, or feature suggestions. Your contributions help make `collections-cache` even better!
+
+---
+
+## License 📄
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+
+---
+
+## Acknowledgements 🙌
+
+- Inspired by the need for efficient, multi-core caching with SQLite.
+- Created by Luiz Trindade.
+- Thanks to all contributors and users who provide feedback to keep improving the library!
+
+---
+
+Give `collections-cache` a try and let it power your high-performance caching needs! 🚀
+

collections_cache-0.2.9.20250303/README.md

@@ -0,0 +1,141 @@
+# collections-cache 🚀
+
+`collections-cache` is a fast and scalable key–value caching solution built on top of SQLite. It allows you to store, update, and retrieve data using unique keys, and it supports complex Python data types (thanks to `pickle`). Designed to harness the power of multiple CPU cores, the library shards data across multiple SQLite databases, enabling impressive performance scaling.
+
+---
+
+## Features ✨
+
+- **Multiple SQLite Databases**: Distributes your data across several databases to optimize I/O and take advantage of multi-core systems.
+- **Key–Value Store**: Simple and intuitive interface for storing and retrieving data.
+- **Supports Complex Data Types**: Serialize and store lists, dictionaries, objects, and more using `pickle`.
+- **Parallel Processing**: Uses Python’s `multiprocessing` and `concurrent.futures` modules to perform operations in parallel.
+- **Efficient Data Retrieval**: Caches all keys in memory for super-fast lookups.
+- **Cross-Platform**: Runs on Linux, macOS, and Windows.
+- **Performance Scaling**: Benchmarks show near-linear scaling with the number of real CPU cores.
+
+---
+
+## Installation 📦
+
+Use [Poetry](https://python-poetry.org/) to install and manage dependencies:
+
+1. Clone the repository:
+
+    ```bash
+    git clone https://github.com/Luiz-Trindade/collections_cache.git
+    cd collections-cache
+    ```
+
+2. Install the package with Poetry:
+
+    ```bash
+    poetry install
+    ```
+
+---
+
+## Usage ⚙️
+
+Simply import and start using the main class, `Collection_Cache`, to interact with your collection:
+
+### Basic Example
+
+```python
+from collections_cache import Collection_Cache
+
+# Create a new collection named "STORE"
+cache = Collection_Cache("STORE")
+
+# Set a key-value pair
+cache.set_key("products", ["apple", "orange", "onion"])
+
+# Retrieve the value by key
+products = cache.get_key("products")
+print(products)  # Output: ['apple', 'orange', 'onion']
+```
+
+### Bulk Insertion Example
+
+For faster insertions, accumulate your data and use `set_multi_keys`:
+
+```python
+from collections_cache import Collection_Cache
+from random import uniform, randint
+from time import time
+
+cache = Collection_Cache("web_cache")
+insertions = 100_000
+data = {}
+
+# Generate data
+for i in range(insertions):
+    key = str(uniform(0.0, 100.0))
+    value = "some text :)" * randint(1, 100)
+    data[key] = value
+
+# Bulk insert keys using multi-threaded execution
+cache.set_multi_keys(data)
+
+print(f"Inserted {len(cache.keys())} keys successfully!")
+```
+
+---
+
+## API Overview 📚
+
+- **`set_key(key, value)`**: Stores a key–value pair. Updates the value if the key already exists.
+- **`set_multi_keys(key_and_value)`**: (Experimental) Inserts multiple key–value pairs in parallel.
+- **`get_key(key)`**: Retrieves the value associated with a given key.
+- **`delete_key(key)`**: Removes a key and its corresponding value.
+- **`keys()`**: Returns a list of all stored keys.
+- **`export_to_json()`**: (Future feature) Exports your collection to a JSON file.
+
+---
+
+## Performance Benchmark 📊
+
+On a machine with 4 real CPU cores, benchmarks indicate around **781 insertions per second**. The library is designed to scale nearly linearly with the number of real cores. For example:
+- **6 cores**: ~1,171 insertions per second.
+- **16 cores**: ~3,125 insertions per second.
+- **128 cores**: ~25,000 insertions per second (theoretically).
+
+*Note: Actual performance will depend on disk I/O, SQLite contention, and system architecture.*
+
+---
+
+## Development & Contributing 👩‍💻👨‍💻
+
+To contribute or run tests:
+
+1. Install development dependencies:
+
+    ```bash
+    poetry install --dev
+    ```
+
+2. Run tests using:
+
+    ```bash
+    poetry run pytest
+    ```
+
+Feel free to submit issues, pull requests, or feature suggestions. Your contributions help make `collections-cache` even better!
+
+---
+
+## License 📄
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+
+---
+
+## Acknowledgements 🙌
+
+- Inspired by the need for efficient, multi-core caching with SQLite.
+- Created by Luiz Trindade.
+- Thanks to all contributors and users who provide feedback to keep improving the library!
+
+---
+
+Give `collections-cache` a try and let it power your high-performance caching needs! 🚀

{collections_cache-0.2.7.20250303 → collections_cache-0.2.9.20250303}/collections_cache/collections_cache.py

@@ -15,6 +15,8 @@ class Collection_Cache:
         self.collection_dir         = path.join("./Collections", self.collection_name)
         self.databases_list         = []
         self.keys_databases         = {}
+        self.temp_keys_values       = {}
+        self.size_limit             = 200
 
         # Init methods
         self.create_collection()
@@ -66,13 +68,22 @@ class Collection_Cache:
         conn.close()
         return keys
 
-    '''def set_key(self, key: str, value: any):
-        """Used to store values and associate a value with a key."""
-        t = Thread_Exec(target=self.set_key_exec, args=(key, value))
-        t.start()
-        t.join()'''
+    # Experimental
+    def verify_size_of_temp_queue(self, type_of_operation: str):
+        if type_of_operation == "set_key" and len(self.temp_keys_values) >= self.size_limit:
+            self.set_multi_keys(self.temp_keys_values)
+            self.temp_keys_values = {}
+        else:
+            self.set_multi_keys(self.temp_keys_values)
+            self.temp_keys_values = {}
 
+    # Experimental
     def set_key(self, key: str, value: any):
+        """Used to store values and associate a value with a key."""
+        self.temp_keys_values[key]  = value
+        self.verify_size_of_temp_queue("set_key")
+
+    def set_key_exec(self, key: str, value: any):
         """Used to store values and associate a value with a key."""
         if key not in self.keys_databases:
             database_to_insert = choice(self.databases_list)
@@ -95,22 +106,20 @@ class Collection_Cache:
 
     def set_multi_keys(self, keys_and_values: dict[str, any]):
         """Experimental. Set multiple keys and values at the same time."""
-
         with Thread(self.cpu_cores) as thread:
-            thread.map(lambda kv: self.set_key(kv[0], kv[1]), keys_and_values.items())
+            thread.map(lambda kv: self.set_key_exec(kv[0], kv[1]), keys_and_values.items())
 
     def add_to_keys_database(self, key, database):
         self.keys_databases[key] = database
 
     def delete_to_keys_database(self, key):
         """Removes the key from the dictionary of stored keys"""
-
         if key in self.keys_databases:
             del self.keys_databases[key]
 
     def get_key(self, key: str):
         """Used to obtain the value stored by the key"""
-
+        self.verify_size_of_temp_queue("get_key")
         try:
             database_to_search = self.keys_databases[key]
             conn = sqlite3.connect(database_to_search)
@@ -122,11 +131,10 @@ class Collection_Cache:
             return pickle.loads(result[0][0])
 
         except Exception as error:
-            return error
+            return None
 
     def delete_key(self, key: str):
         """Used to delete the value stored by the key"""
-
         try:
             database_to_delete = self.keys_databases[key]
             conn = sqlite3.connect(database_to_delete)

{collections_cache-0.2.7.20250303 → collections_cache-0.2.9.20250303}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "collections-cache"
-version = "0.2.7.20250303"
+version = "0.2.9.20250303"
 description = "collections-cache is a Python package for managing data collections across multiple SQLite databases. It allows efficient storage, retrieval, and updating of key-value pairs, supporting various data types serialized with pickle. The package uses parallel processing for fast access and manipulation of large collections."
 authors = ["Luiz-Trindade <luiz.gabriel.m.trindade@gmail.com>"]
 license = "MIT"

collections_cache-0.2.7.20250303/PKG-INFO

@@ -1,96 +0,0 @@
-Metadata-Version: 2.1
-Name: collections-cache
-Version: 0.2.7.20250303
-Summary: collections-cache is a Python package for managing data collections across multiple SQLite databases. It allows efficient storage, retrieval, and updating of key-value pairs, supporting various data types serialized with pickle. The package uses parallel processing for fast access and manipulation of large collections.
-License: MIT
-Author: Luiz-Trindade
-Author-email: luiz.gabriel.m.trindade@gmail.com
-Requires-Python: >=3.12,<4.0
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.12
-Description-Content-Type: text/markdown
-
-# collections-cache  
-
-`collections-cache` is a simple and efficient caching solution built with SQLite databases. It allows storing, updating, and retrieving data using unique keys while supporting complex data types through `pickle`. Designed to scale across multiple CPU cores, it distributes data across multiple SQLite databases for improved performance.  
-
-## Features  
-
-- **Multiple SQLite databases**: Distributes data across multiple databases for better scalability.  
-- **Key-value store**: Stores data as key-value pairs.  
-- **Supports complex data types**: Data is serialized using `pickle`, allowing you to store lists, dictionaries, and other Python objects.  
-- **Parallel processing**: Utilizes Python’s `multiprocessing` module to handle large collections in parallel across multiple CPU cores.  
-- **Efficient data retrieval**: Retrieves stored data efficiently based on the key.  
-- **Cross-platform**: Works on Linux, macOS, and Windows.  
-
-## Installation  
-
-To install the `collections-cache` package, use [Poetry](https://python-poetry.org/) to manage dependencies.  
-
-1. Clone the repository:  
-
-    ```bash
-    git clone https://github.com/Luiz-Trindade/collections_cache.git
-    cd collections-cache
-    ```  
-
-2. Install the package with Poetry:  
-
-    ```bash
-    poetry install
-    ```  
-
-## Usage  
-
-To use the `collections-cache` package, import the main class `Collection_Cache` and interact with your collection.  
-
-### Example  
-
-```python
-from collections_cache import Collection_Cache
-
-# Create a new collection
-cache = Collection_Cache("STORE")
-
-# Set a key-value pair
-cache.set_key("products", ["apple", "orange", "onion"])
-
-# Get the value by key
-products = cache.get_key("products")
-print(products)  # Output: ['apple', 'orange', 'onion']
-```  
-
-### Methods  
-
-- **`set_key(key, value)`**: Stores a key-value pair in the cache. If the key already exists, its value is updated.
-- **`set_multi_keys(key_and_value)`**: Stores a multi key-value pair in the cache. If the key already exists, its value is updated.  
-- **`get_key(key)`**: Retrieves the value associated with a key.  
-- **`delete_key(key)`**: Removes an existing key from the cache.  
-- **`keys`***: Returns all stored keys.
-
-## Development  
-
-To contribute or run tests:  
-
-1. Install development dependencies:  
-
-    ```bash
-    poetry install --dev
-    ```  
-
-2. Run tests:  
-
-    ```bash
-    poetry run pytest
-    ```  
-
-## License  
-
-This project is licensed under the MIT License – see the [LICENSE](LICENSE) file for details.  
-
-## Acknowledgements  
-
-- This package was created to demonstrate how to work with SQLite, `pickle`, and Python's `multiprocessing` module.  
-- Created by: Luiz Trindade.
-

collections_cache-0.2.7.20250303/README.md

@@ -1,82 +0,0 @@
-# collections-cache  
-
-`collections-cache` is a simple and efficient caching solution built with SQLite databases. It allows storing, updating, and retrieving data using unique keys while supporting complex data types through `pickle`. Designed to scale across multiple CPU cores, it distributes data across multiple SQLite databases for improved performance.  
-
-## Features  
-
-- **Multiple SQLite databases**: Distributes data across multiple databases for better scalability.  
-- **Key-value store**: Stores data as key-value pairs.  
-- **Supports complex data types**: Data is serialized using `pickle`, allowing you to store lists, dictionaries, and other Python objects.  
-- **Parallel processing**: Utilizes Python’s `multiprocessing` module to handle large collections in parallel across multiple CPU cores.  
-- **Efficient data retrieval**: Retrieves stored data efficiently based on the key.  
-- **Cross-platform**: Works on Linux, macOS, and Windows.  
-
-## Installation  
-
-To install the `collections-cache` package, use [Poetry](https://python-poetry.org/) to manage dependencies.  
-
-1. Clone the repository:  
-
-    ```bash
-    git clone https://github.com/Luiz-Trindade/collections_cache.git
-    cd collections-cache
-    ```  
-
-2. Install the package with Poetry:  
-
-    ```bash
-    poetry install
-    ```  
-
-## Usage  
-
-To use the `collections-cache` package, import the main class `Collection_Cache` and interact with your collection.  
-
-### Example  
-
-```python
-from collections_cache import Collection_Cache
-
-# Create a new collection
-cache = Collection_Cache("STORE")
-
-# Set a key-value pair
-cache.set_key("products", ["apple", "orange", "onion"])
-
-# Get the value by key
-products = cache.get_key("products")
-print(products)  # Output: ['apple', 'orange', 'onion']
-```  
-
-### Methods  
-
-- **`set_key(key, value)`**: Stores a key-value pair in the cache. If the key already exists, its value is updated.
-- **`set_multi_keys(key_and_value)`**: Stores a multi key-value pair in the cache. If the key already exists, its value is updated.  
-- **`get_key(key)`**: Retrieves the value associated with a key.  
-- **`delete_key(key)`**: Removes an existing key from the cache.  
-- **`keys`***: Returns all stored keys.
-
-## Development  
-
-To contribute or run tests:  
-
-1. Install development dependencies:  
-
-    ```bash
-    poetry install --dev
-    ```  
-
-2. Run tests:  
-
-    ```bash
-    poetry run pytest
-    ```  
-
-## License  
-
-This project is licensed under the MIT License – see the [LICENSE](LICENSE) file for details.  
-
-## Acknowledgements  
-
-- This package was created to demonstrate how to work with SQLite, `pickle`, and Python's `multiprocessing` module.  
-- Created by: Luiz Trindade.