redis-allocator 0.0.1__tar.gz

redis_allocator-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Invoker Bot
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

redis_allocator-0.0.1/PKG-INFO

@@ -0,0 +1,229 @@
+Metadata-Version: 2.4
+Name: redis-allocator
+Version: 0.0.1
+Summary: Redis-based resource allocation system.
+Home-page: https://github.com/invoker-bot/RedisAllocator-python
+Author: Invoker Bot
+Author-email: invoker-bot@outlook.com
+License: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.9
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: redis>=5.0.0
+Provides-Extra: test
+Requires-Dist: pytest>=7.4.3; extra == "test"
+Requires-Dist: pytest-cov>=4.1.0; extra == "test"
+Requires-Dist: pytest-mock>=3.12.0; extra == "test"
+Requires-Dist: fakeredis[lua]>=2.20.1; extra == "test"
+Requires-Dist: flake8>=6.1.0; extra == "test"
+Requires-Dist: freezegun>=1.4.0; extra == "test"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# RedisAllocator
+
+## Project Overview
+
+RedisAllocator is an efficient Redis-based distributed memory allocation system. This system simulates traditional memory allocation mechanisms but implements them in a distributed environment, using Redis as the underlying storage and coordination tool.
+
+> **Note**: Currently, RedisAllocator only supports single Redis instance deployments. For Redis cluster environments, we recommend using RedLock for distributed locking operations.
+
+### Core Features
+
+- **Distributed Locking**: Provides robust distributed locking mechanisms to ensure data consistency in concurrent environments
+- **Resource Allocation**: Implements a distributed resource allocation system with support for:
+  - Priority-based distribution
+  - Soft binding
+  - Garbage collection
+  - Health checking
+- **Task Management**: Implements a distributed task queue system for efficient task processing across multiple workers
+- **Object Allocation**: Supports allocation of resources with priority-based distribution and soft binding
+- **Health Checking**: Monitors the health of distributed instances and automatically handles unhealthy resources
+- **Garbage Collection**: Automatically identifies and reclaims unused resources, optimizing memory usage
+
+
+## Installation
+
+```bash
+pip install redis-allocator
+```
+
+## Quick Start
+
+### Using RedisLock for Distributed Locking
+
+```python
+from redis import Redis
+from redis_allocator import RedisLock
+
+# Initialize Redis client
+redis = Redis(host='localhost', port=6379)
+
+# Create a RedisLock instance
+lock = RedisLock(redis, "myapp", "resource-lock")
+
+# Acquire a lock
+if lock.lock("resource-123", timeout=60):
+    try:
+        # Perform operations with the locked resource
+        print("Resource locked successfully")
+    finally:
+        # Release the lock when done
+        lock.unlock("resource-123")
+```
+
+### Using RedisAllocator for Resource Management
+
+```python
+from redis import Redis
+from redis_allocator import RedisAllocator
+
+# Initialize Redis client
+redis = Redis(host='localhost', port=6379)
+
+# Create a RedisAllocator instance
+allocator = RedisAllocator(
+    redis, 
+    prefix='myapp',
+    suffix='allocator',
+    shared=False  # Whether resources can be shared
+)
+
+# Add resources to the pool
+allocator.extend(['resource-1', 'resource-2', 'resource-3'])
+
+# Allocate a resource key (returns only the key)
+key = allocator.malloc_key(timeout=120)
+if key:
+    try:
+        # Use the allocated resource
+        print(f"Allocated resource: {key}")
+    finally:
+        # Free the resource when done
+        allocator.free_keys(key)
+
+# Allocate a resource with object (returns a RedisAllocatorObject)
+allocated_obj = allocator.malloc(timeout=120)
+if allocated_obj:
+    try:
+        # The key is available as a property
+        print(f"Allocated resource: {allocated_obj.key}")
+        
+        # Update the resource's lock timeout
+        allocated_obj.update(timeout=60)
+    finally:
+        # Free the resource when done
+        allocator.free(allocated_obj)
+
+# Using soft binding (associates a name with a resource)
+allocator.update_soft_bind("worker-1", "resource-1")
+# Later...
+allocator.unbind_soft_bind("worker-1")
+
+# Garbage collection (reclaims unused resources)
+allocator.gc(count=10)  # Check 10 items for cleanup
+```
+
+### Using RedisTaskQueue for Distributed Task Processing
+
+```python
+from redis import Redis
+from redis_allocator import RedisTaskQueue, TaskExecutePolicy
+import json
+
+# Initialize Redis client
+redis = Redis(host='localhost', port=6379)
+
+# Process tasks in a worker
+def process_task(task):
+    # Process the task (task is a RedisTask object)
+    # You can access task.id, task.name, task.params
+    # You can update progress with task.update(current, total)
+    return json.dumps({"result": "processed"})
+
+
+# Create a task queue
+task_queue = RedisTaskQueue(redis, "myapp", task_fn=process_task)
+
+# Submit a task with query method
+result = task_queue.query(
+    id="task-123",
+    name="example-task",
+    params={"input": "data"},
+    timeout=300,  # Optional timeout in seconds
+    policy=TaskExecutePolicy.Auto,  # Execution policy
+    once=False  # Whether to delete the result after getting it
+)
+
+# Start listening for tasks
+task_queue.listen(
+    names=["example-task"],  # List of task names to listen for
+    workers=128,  # Number of worker threads
+    event=None  # Optional event to signal when to stop listening
+)
+```
+
+## Modules
+
+RedisAllocator consists of several modules, each providing specific functionality:
+
+- **lock.py**: Provides `RedisLock` and `RedisLockPool` for distributed locking mechanisms
+- **task_queue.py**: Implements `RedisTaskQueue` for distributed task processing
+- **allocator.py**: Contains `RedisAllocator` and `RedisThreadHealthChecker` for resource allocation
+
+
+## Roadmap
+
+### Phase 1 (Completed)
+- [x] Distributed lock mechanism implementation
+- [x] Task queue processing system
+- [x] Resource allocation and management
+- [x] Basic health checking and monitoring
+- [x] Object allocation with serialization
+- [x] Unit tests for core components
+
+### Phase 2 (In Progress)
+- [ ] Advanced sharding implementation
+- [ ] Performance optimization and benchmarking
+- [ ] Documentation improvement
+- [ ] Enhanced error handling and recovery
+
+### Phase 3 (Planned)
+- [ ] Advanced garbage collection strategies
+- [ ] Redis cluster support
+- [ ] Fault recovery mechanisms
+- [ ] Automated resource scaling
+
+### Phase 4 (Future)
+- [ ] API stability and backward compatibility
+- [ ] Performance monitoring and tuning tools
+- [ ] Advanced features (transaction support, data compression, etc.)
+- [ ] Production environment validation and case studies
+
+## Contributing
+
+Contributions and suggestions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for more information.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Contact
+
+For questions or suggestions, please contact us through GitHub Issues.

redis_allocator-0.0.1/README.md

@@ -0,0 +1,191 @@
+# RedisAllocator
+
+## Project Overview
+
+RedisAllocator is an efficient Redis-based distributed memory allocation system. This system simulates traditional memory allocation mechanisms but implements them in a distributed environment, using Redis as the underlying storage and coordination tool.
+
+> **Note**: Currently, RedisAllocator only supports single Redis instance deployments. For Redis cluster environments, we recommend using RedLock for distributed locking operations.
+
+### Core Features
+
+- **Distributed Locking**: Provides robust distributed locking mechanisms to ensure data consistency in concurrent environments
+- **Resource Allocation**: Implements a distributed resource allocation system with support for:
+  - Priority-based distribution
+  - Soft binding
+  - Garbage collection
+  - Health checking
+- **Task Management**: Implements a distributed task queue system for efficient task processing across multiple workers
+- **Object Allocation**: Supports allocation of resources with priority-based distribution and soft binding
+- **Health Checking**: Monitors the health of distributed instances and automatically handles unhealthy resources
+- **Garbage Collection**: Automatically identifies and reclaims unused resources, optimizing memory usage
+
+
+## Installation
+
+```bash
+pip install redis-allocator
+```
+
+## Quick Start
+
+### Using RedisLock for Distributed Locking
+
+```python
+from redis import Redis
+from redis_allocator import RedisLock
+
+# Initialize Redis client
+redis = Redis(host='localhost', port=6379)
+
+# Create a RedisLock instance
+lock = RedisLock(redis, "myapp", "resource-lock")
+
+# Acquire a lock
+if lock.lock("resource-123", timeout=60):
+    try:
+        # Perform operations with the locked resource
+        print("Resource locked successfully")
+    finally:
+        # Release the lock when done
+        lock.unlock("resource-123")
+```
+
+### Using RedisAllocator for Resource Management
+
+```python
+from redis import Redis
+from redis_allocator import RedisAllocator
+
+# Initialize Redis client
+redis = Redis(host='localhost', port=6379)
+
+# Create a RedisAllocator instance
+allocator = RedisAllocator(
+    redis, 
+    prefix='myapp',
+    suffix='allocator',
+    shared=False  # Whether resources can be shared
+)
+
+# Add resources to the pool
+allocator.extend(['resource-1', 'resource-2', 'resource-3'])
+
+# Allocate a resource key (returns only the key)
+key = allocator.malloc_key(timeout=120)
+if key:
+    try:
+        # Use the allocated resource
+        print(f"Allocated resource: {key}")
+    finally:
+        # Free the resource when done
+        allocator.free_keys(key)
+
+# Allocate a resource with object (returns a RedisAllocatorObject)
+allocated_obj = allocator.malloc(timeout=120)
+if allocated_obj:
+    try:
+        # The key is available as a property
+        print(f"Allocated resource: {allocated_obj.key}")
+        
+        # Update the resource's lock timeout
+        allocated_obj.update(timeout=60)
+    finally:
+        # Free the resource when done
+        allocator.free(allocated_obj)
+
+# Using soft binding (associates a name with a resource)
+allocator.update_soft_bind("worker-1", "resource-1")
+# Later...
+allocator.unbind_soft_bind("worker-1")
+
+# Garbage collection (reclaims unused resources)
+allocator.gc(count=10)  # Check 10 items for cleanup
+```
+
+### Using RedisTaskQueue for Distributed Task Processing
+
+```python
+from redis import Redis
+from redis_allocator import RedisTaskQueue, TaskExecutePolicy
+import json
+
+# Initialize Redis client
+redis = Redis(host='localhost', port=6379)
+
+# Process tasks in a worker
+def process_task(task):
+    # Process the task (task is a RedisTask object)
+    # You can access task.id, task.name, task.params
+    # You can update progress with task.update(current, total)
+    return json.dumps({"result": "processed"})
+
+
+# Create a task queue
+task_queue = RedisTaskQueue(redis, "myapp", task_fn=process_task)
+
+# Submit a task with query method
+result = task_queue.query(
+    id="task-123",
+    name="example-task",
+    params={"input": "data"},
+    timeout=300,  # Optional timeout in seconds
+    policy=TaskExecutePolicy.Auto,  # Execution policy
+    once=False  # Whether to delete the result after getting it
+)
+
+# Start listening for tasks
+task_queue.listen(
+    names=["example-task"],  # List of task names to listen for
+    workers=128,  # Number of worker threads
+    event=None  # Optional event to signal when to stop listening
+)
+```
+
+## Modules
+
+RedisAllocator consists of several modules, each providing specific functionality:
+
+- **lock.py**: Provides `RedisLock` and `RedisLockPool` for distributed locking mechanisms
+- **task_queue.py**: Implements `RedisTaskQueue` for distributed task processing
+- **allocator.py**: Contains `RedisAllocator` and `RedisThreadHealthChecker` for resource allocation
+
+
+## Roadmap
+
+### Phase 1 (Completed)
+- [x] Distributed lock mechanism implementation
+- [x] Task queue processing system
+- [x] Resource allocation and management
+- [x] Basic health checking and monitoring
+- [x] Object allocation with serialization
+- [x] Unit tests for core components
+
+### Phase 2 (In Progress)
+- [ ] Advanced sharding implementation
+- [ ] Performance optimization and benchmarking
+- [ ] Documentation improvement
+- [ ] Enhanced error handling and recovery
+
+### Phase 3 (Planned)
+- [ ] Advanced garbage collection strategies
+- [ ] Redis cluster support
+- [ ] Fault recovery mechanisms
+- [ ] Automated resource scaling
+
+### Phase 4 (Future)
+- [ ] API stability and backward compatibility
+- [ ] Performance monitoring and tuning tools
+- [ ] Advanced features (transaction support, data compression, etc.)
+- [ ] Production environment validation and case studies
+
+## Contributing
+
+Contributions and suggestions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for more information.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Contact
+
+For questions or suggestions, please contact us through GitHub Issues.

redis_allocator-0.0.1/pyproject.toml

@@ -0,0 +1,11 @@
+[build-system]
+requires = ['setuptools>=42']
+build-backend = 'setuptools.build_meta'
+
+[tool.commitizen]
+name = "cz_conventional_commits"
+version = "0.1.0"
+tag_format = "v$version"
+
+[tool.commitizen.version_files]
+"setup.py" = { search = "__version__ = '", replace = "__version__ = '" }

redis_allocator-0.0.1/redis_allocator/__init__.py

@@ -0,0 +1,28 @@
+"""RedisAllocator package for distributed memory allocation using Redis.
+
+This package provides efficient, Redis-based distributed memory allocation
+services that simulate traditional memory allocation mechanisms in a
+distributed environment.
+"""
+
+from redis_allocator.lock import (RedisLock, RedisLockPool, LockStatus,
+                                  BaseLock, BaseLockPool, ThreadLock, ThreadLockPool)
+from redis_allocator.task_queue import TaskExecutePolicy, RedisTask, RedisTaskQueue
+from redis_allocator.allocator import RedisAllocator
+
+
+__version__ = '0.0.1'
+
+__all__ = [
+    'RedisLock',
+    'RedisLockPool',
+    'LockStatus',
+    'BaseLock',
+    'BaseLockPool',
+    'ThreadLock',
+    'ThreadLockPool',
+    'TaskExecutePolicy',
+    'RedisTask',
+    'RedisTaskQueue',
+    'RedisAllocator',
+]