kycli 0.2.3__tar.gz → 0.4.0__tar.gz

{kycli-0.2.3/kycli.egg-info → kycli-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kycli
-Version: 0.2.3
+Version: 0.4.0
 Summary: **kycli** is a high-performance Python CLI toolkit built with Cython for speed.
 Author: Balakrishna Maduru
 Author-email: balakrishnamaduru@gmail.com
@@ -62,6 +62,16 @@ Install the latest version from PyPI:
 pip install kycli
 ```
 
+### Validate The Install
+
+Run the end-to-end command matrix from the repo root:
+
+```bash
+bash scripts/validate_install.sh
+```
+
+The validator auto-resolves a usable Python executable, prefers the active `python3` runtime, and runs stateful sections in isolated home directories so queue, profile, and policy checks do not interfere with each other.
+
 ---
 
 ## 💻 CLI Command Reference
@@ -86,6 +96,17 @@ pip install kycli
 | **`kyd`** | Delete Key | `kyd host` |
 | **`kypush`**| Push to List | `kypush logs "error"` |
 | **`kyrem`** | Remove from List | `kyrem tags "old"` |
+### 🧱 Queues & Stacks Operations
+| Command | Description | Example |
+| :--- | :--- | :--- |
+| **`kypush`** | Push to Queue/Stack | `kypush "task" --priority 10` |
+| **`kypop`** | Pop item (Atomic) | `kypop` |
+| **`kypeek`** | Peek next item | `kypeek` |
+| **`kycount`** | Count items | `kycount` |
+| **`kyclear`** | Clear implementation | `kyclear` |
+| **`kyws`** | Create Typed WS | `kyws create q --type queue` |
+| **`kyack`** | Ack leased item | `kyack <receipt_id>` |
+| **`kynack`** | Requeue leased item | `kynack <receipt_id> --delay 30s` |
 
 ### 🔍 Search & Utility
 | Command | Description | Example |
@@ -106,6 +127,13 @@ pip install kycli
 | **`kyc`** | Execute Command | `kyc hello` |
 | **`kyco`** | Compact DB | `kyco 7` |
 | **`kyrotate`** | Rotate Master Key | `kyrotate --new-key "newpass" --old-key "oldpass" --backup` |
+| **`kyttl`** | Workspace TTL policy | `kyttl set 1h` |
+| **`kyprofile`** | Manage profiles | `kyprofile save dev` |
+| **`kyacl`** | Workspace ACL/policy | `kyacl readonly on` |
+| **`kyaudit`** | Export audit history | `kyaudit export audit.json json` |
+| **`kystats`** | Show workspace stats | `kystats --json` |
+| **`kybackup`** | Create/restore backup | `kybackup snapshot.db` |
+| **`kymetrics`** | Local metrics endpoint | `kymetrics 8765` |
 
 ---
 
@@ -129,7 +157,6 @@ kyws
 
 #### `kymv` — Move Key
 Moves a key (and its value) from the current workspace to another.
-- **Safety**: Asks for confirmation if the key exists in the target.
 ```bash
 kymv my_api_key project_beta
 # Result: ✅ Moved 'my_api_key' to 'project_beta'.
@@ -137,16 +164,87 @@ kymv my_api_key project_beta
 
 #### `kydrop` — Delete Workspace
 Permanently deletes an entire workspace and its database file.
-- **Safety**: Requires explicit confirmation (`y/N`) in CLI.
-- **Restriction**: You cannot drop the currently active workspace.
 ```bash
 kydrop temp_test
 # Result: ⚠️ DANGER: Are you sure you want to PERMANENTLY delete workspace 'temp_test'? (y/N): y
 # Result: ✅ Workspace 'temp_test' deleted.
 ```
 
----
 
+## 🧭 Advanced Features
+
+### Queue Reliability
+- **Batch Queue Ops**: `kypush --file` and `kypop --n 100` for throughput-oriented flows.
+- **Delayed Jobs**: `kypush --delay 30s` schedules visibility in queue workspaces.
+- **Visibility Timeout**: `kypop --lease 30s` with `kyack` and `kynack` supports retry flows.
+- **Workspace TTL Policies**: `kyttl set/get` applies default TTL behavior per workspace.
+
+### Usability & DX
+- **Config Profiles**: `kyprofile use prod` switches saved CLI defaults.
+- **Output Formatting**: `--json` and `--pretty` are available on structured read paths.
+- **Interactive TUI**: `kyshell` provides an interactive terminal workflow on top of the same engine.
+
+### Security & Compliance
+- **Workspace ACLs**: `kyacl readonly on|off|status` supports write protection.
+- **Workspace Access Keys**: `kyacl key set|get|clear` gates writes with `KYCLI_ACCESS_KEY`.
+- **Audit Export**: `kyaudit export` supports JSON or CSV output with optional time filters.
+
+### Observability
+- **Stats Command**: `kystats` exposes workspace type, counts, TTL usage, archive size, and DB size.
+- **Metrics Endpoint**: `kymetrics` starts a local HTTP endpoint on `127.0.0.1`.
+
+### Data Management
+- **Namespace/Prefix Views**: `kyws view <prefix>` filters large keyspaces.
+- **Backup/Restore**: `kybackup` creates and restores encrypted snapshots.
+
+### Queue Reliability
+```bash
+# Push from a file
+kypush --file tasks.txt
+
+# Delay visibility for 30 seconds
+kypush "email:user_123" --delay 30s
+
+# Lease one item for 30 seconds
+kypop --lease 30s --json
+
+# Acknowledge or requeue
+kyack <receipt_id>
+kynack <receipt_id> --delay 15s
+```
+
+### Workspace Policies
+```bash
+# Set a default TTL for all new values in this workspace
+kyttl set 1h
+kyttl get
+
+# Enable read-only mode
+kyacl readonly on
+
+# Protect a workspace with an access key
+kyacl key set my-secret
+KYCLI_ACCESS_KEY=my-secret kys protected value
+```
+
+### Profiles, Stats, and Backups
+```bash
+kyprofile save dev
+kyprofile list
+kyprofile use dev
+
+kystats --json
+kyaudit export audit.json json --since "2026-01-01 00:00:00"
+kybackup snapshot.db
+kybackup restore snapshot.db
+kymetrics 8765
+```
+
+### Maintenance & Reliability
+- **Lock Management**: Retry with exponential backoff for "Database is locked" in multi-process scenarios.
+- **Activity Logs**: Background logging via Python `logging` module.
+- **Atomic Rename Exports**: Write-to-temp-then-rename for export safety.
+- **Compression**: Optional large-value compression for storage efficiency.
 ### `kyh` — The Help Center
 Shows the available commands and basic usage instructions.
 ```bash
@@ -170,6 +268,40 @@ kys session_id "data" --ttl 1h
 # Result: ✅ Saved: session_id (New) (Expires in 1 hour)
 ```
 
+### 🧱 Typed Workspaces: Queues & Stacks
+Optimize your workspace for specific data structures with strict concurrency guarantees (`BEGIN IMMEDIATE` locking).
+
+#### 1. Queue (FIFO)
+```bash
+# Create
+kyws create jobs --type queue
+kyuse jobs
+
+# Push & Pop (Atomic)
+kypush "job_1"
+kypop
+# Result: job_1
+```
+
+#### 2. Stack (LIFO)
+```bash
+kyws create undo_log --type stack
+kypush "cmd1"
+kypush "cmd2"
+kypop
+# Result: cmd2
+```
+
+#### 3. Priority Queue
+```bash
+kyws create triage --type priority_queue
+kypush "low" --priority 1
+kypush "high" --priority 100
+kypop
+# Result: high
+```
+See [docs/QUEUES_STACKS.md](docs/QUEUES_STACKS.md) for full details.
+
 ### 📂 Advanced JSONPath & Dot-Notation
 `kycli` allows you to treat your key-value store like a document database. You can query and update deep nested structures without retrieving the entire object.
 
@@ -186,10 +318,13 @@ kyg logs[0:5]
 ```
 
 #### Atomic Patching (Partial Updates):
-Instead of rewriting a large JSON object, you can update just one field.
+Instead of rewriting a large JSON object, you can update just one field. Use
+`kypatch` (not `kys`) for this — `kys` always treats dots in a key as a literal
+flat key name (e.g. for namespacing, like `kys ns.alpha 1`), while `kypatch`
+treats them as a path into the nested object.
 ```bash
 # Update just the 'age' field inside the 'user' object
-kys user.profile.age 25
+kypatch user.profile.age 25
 ```
 
 ### 📦 Collection Operations (Lists & Sets)
@@ -466,6 +601,24 @@ with Kycore() as kv:
     kv.save_many(items, ttl="1h")
     # Result: ⚡ Atomic transaction per batch (extremely fast)
 
+# 5. Queues & Stacks (API)
+use `set_type` to separate queues from KV stores.
+```python
+from kycli import Kycore
+
+# Initialize and set type
+with Kycore("jobs.db") as q:
+    q.set_type("queue")
+    
+    # Push (Atomic & Durable)
+    q.push("task_1")
+    q.push({"id": 2, "action": "email"})
+    
+    # Pop (Thread-Safe)
+    item = q.pop()
+    print(item) # task_1
+```
+
 # 6. Maintenance & PITR
 with Kycore() as kv:
     # Cleanup history older than 30 days
@@ -544,8 +697,9 @@ async def fetch_config(key: str, db: Kycore = Depends(get_db)):
 
 ## 🏗 Architecture & Internal Safety
 
-- **SQLite Engine**: Running in `WAL` (Write-Ahead Logging) mode for concurrent reads/writes.
-- **Atomic Operations**: Exports use a "temp-file then rename" strategy to prevent corruption.
+- **Encrypted Single-File Storage**: Each workspace is a single AES-GCM-encrypted file. `kycli` decrypts it into an in-memory SQLite engine on open and re-serializes the whole workspace back to disk on every write.
+- **Atomic Writes**: Every persist (workspace file, exports, audit export) uses a "temp-file then rename" strategy — a concurrent reader always sees a complete file, never a partial/corrupted one.
+- **Cross-Process Write Safety**: Writes from independent `kycli` processes sharing one workspace file are serialized via an OS-level advisory lock (`flock`, POSIX). A process reloads the latest on-disk state immediately before mutating it, so sibling processes' writes are never silently overwritten. *(Windows: atomic writes still prevent corruption, but cross-process mutual exclusion requires POSIX `flock` and is not yet supported.)*
 - **Data Integrity**: Keys are automatically lowercased and stripped to prevent duplicate-but-slightly-different keys.
 - **Auto-Purge Policy**: Deleted keys are moved to an **Archive** table and automatically purged after **15 days** to keep the database size optimized.
 - **Embedded C**: Core operations are written in Cython, binding directly to native library pointers.
@@ -556,7 +710,7 @@ async def fetch_config(key: str, db: Kycore = Depends(get_db)):
 
 Want to test the speed on your own hardware?
 ```bash
-PYTHONPATH=. python3 tests/integration/benchmark.py
+PYTHONPATH=. python3 tests/performance/kycli_benchmark.py
 ```
 
  ---

{kycli-0.2.3 → kycli-0.4.0}/README.md

@@ -36,6 +36,16 @@ Install the latest version from PyPI:
 pip install kycli
 ```
 
+### Validate The Install
+
+Run the end-to-end command matrix from the repo root:
+
+```bash
+bash scripts/validate_install.sh
+```
+
+The validator auto-resolves a usable Python executable, prefers the active `python3` runtime, and runs stateful sections in isolated home directories so queue, profile, and policy checks do not interfere with each other.
+
 ---
 
 ## 💻 CLI Command Reference
@@ -60,6 +70,17 @@ pip install kycli
 | **`kyd`** | Delete Key | `kyd host` |
 | **`kypush`**| Push to List | `kypush logs "error"` |
 | **`kyrem`** | Remove from List | `kyrem tags "old"` |
+### 🧱 Queues & Stacks Operations
+| Command | Description | Example |
+| :--- | :--- | :--- |
+| **`kypush`** | Push to Queue/Stack | `kypush "task" --priority 10` |
+| **`kypop`** | Pop item (Atomic) | `kypop` |
+| **`kypeek`** | Peek next item | `kypeek` |
+| **`kycount`** | Count items | `kycount` |
+| **`kyclear`** | Clear implementation | `kyclear` |
+| **`kyws`** | Create Typed WS | `kyws create q --type queue` |
+| **`kyack`** | Ack leased item | `kyack <receipt_id>` |
+| **`kynack`** | Requeue leased item | `kynack <receipt_id> --delay 30s` |
 
 ### 🔍 Search & Utility
 | Command | Description | Example |
@@ -80,6 +101,13 @@ pip install kycli
 | **`kyc`** | Execute Command | `kyc hello` |
 | **`kyco`** | Compact DB | `kyco 7` |
 | **`kyrotate`** | Rotate Master Key | `kyrotate --new-key "newpass" --old-key "oldpass" --backup` |
+| **`kyttl`** | Workspace TTL policy | `kyttl set 1h` |
+| **`kyprofile`** | Manage profiles | `kyprofile save dev` |
+| **`kyacl`** | Workspace ACL/policy | `kyacl readonly on` |
+| **`kyaudit`** | Export audit history | `kyaudit export audit.json json` |
+| **`kystats`** | Show workspace stats | `kystats --json` |
+| **`kybackup`** | Create/restore backup | `kybackup snapshot.db` |
+| **`kymetrics`** | Local metrics endpoint | `kymetrics 8765` |
 
 ---
 
@@ -103,7 +131,6 @@ kyws
 
 #### `kymv` — Move Key
 Moves a key (and its value) from the current workspace to another.
-- **Safety**: Asks for confirmation if the key exists in the target.
 ```bash
 kymv my_api_key project_beta
 # Result: ✅ Moved 'my_api_key' to 'project_beta'.
@@ -111,16 +138,87 @@ kymv my_api_key project_beta
 
 #### `kydrop` — Delete Workspace
 Permanently deletes an entire workspace and its database file.
-- **Safety**: Requires explicit confirmation (`y/N`) in CLI.
-- **Restriction**: You cannot drop the currently active workspace.
 ```bash
 kydrop temp_test
 # Result: ⚠️ DANGER: Are you sure you want to PERMANENTLY delete workspace 'temp_test'? (y/N): y
 # Result: ✅ Workspace 'temp_test' deleted.
 ```
 
----
 
+## 🧭 Advanced Features
+
+### Queue Reliability
+- **Batch Queue Ops**: `kypush --file` and `kypop --n 100` for throughput-oriented flows.
+- **Delayed Jobs**: `kypush --delay 30s` schedules visibility in queue workspaces.
+- **Visibility Timeout**: `kypop --lease 30s` with `kyack` and `kynack` supports retry flows.
+- **Workspace TTL Policies**: `kyttl set/get` applies default TTL behavior per workspace.
+
+### Usability & DX
+- **Config Profiles**: `kyprofile use prod` switches saved CLI defaults.
+- **Output Formatting**: `--json` and `--pretty` are available on structured read paths.
+- **Interactive TUI**: `kyshell` provides an interactive terminal workflow on top of the same engine.
+
+### Security & Compliance
+- **Workspace ACLs**: `kyacl readonly on|off|status` supports write protection.
+- **Workspace Access Keys**: `kyacl key set|get|clear` gates writes with `KYCLI_ACCESS_KEY`.
+- **Audit Export**: `kyaudit export` supports JSON or CSV output with optional time filters.
+
+### Observability
+- **Stats Command**: `kystats` exposes workspace type, counts, TTL usage, archive size, and DB size.
+- **Metrics Endpoint**: `kymetrics` starts a local HTTP endpoint on `127.0.0.1`.
+
+### Data Management
+- **Namespace/Prefix Views**: `kyws view <prefix>` filters large keyspaces.
+- **Backup/Restore**: `kybackup` creates and restores encrypted snapshots.
+
+### Queue Reliability
+```bash
+# Push from a file
+kypush --file tasks.txt
+
+# Delay visibility for 30 seconds
+kypush "email:user_123" --delay 30s
+
+# Lease one item for 30 seconds
+kypop --lease 30s --json
+
+# Acknowledge or requeue
+kyack <receipt_id>
+kynack <receipt_id> --delay 15s
+```
+
+### Workspace Policies
+```bash
+# Set a default TTL for all new values in this workspace
+kyttl set 1h
+kyttl get
+
+# Enable read-only mode
+kyacl readonly on
+
+# Protect a workspace with an access key
+kyacl key set my-secret
+KYCLI_ACCESS_KEY=my-secret kys protected value
+```
+
+### Profiles, Stats, and Backups
+```bash
+kyprofile save dev
+kyprofile list
+kyprofile use dev
+
+kystats --json
+kyaudit export audit.json json --since "2026-01-01 00:00:00"
+kybackup snapshot.db
+kybackup restore snapshot.db
+kymetrics 8765
+```
+
+### Maintenance & Reliability
+- **Lock Management**: Retry with exponential backoff for "Database is locked" in multi-process scenarios.
+- **Activity Logs**: Background logging via Python `logging` module.
+- **Atomic Rename Exports**: Write-to-temp-then-rename for export safety.
+- **Compression**: Optional large-value compression for storage efficiency.
 ### `kyh` — The Help Center
 Shows the available commands and basic usage instructions.
 ```bash
@@ -144,6 +242,40 @@ kys session_id "data" --ttl 1h
 # Result: ✅ Saved: session_id (New) (Expires in 1 hour)
 ```
 
+### 🧱 Typed Workspaces: Queues & Stacks
+Optimize your workspace for specific data structures with strict concurrency guarantees (`BEGIN IMMEDIATE` locking).
+
+#### 1. Queue (FIFO)
+```bash
+# Create
+kyws create jobs --type queue
+kyuse jobs
+
+# Push & Pop (Atomic)
+kypush "job_1"
+kypop
+# Result: job_1
+```
+
+#### 2. Stack (LIFO)
+```bash
+kyws create undo_log --type stack
+kypush "cmd1"
+kypush "cmd2"
+kypop
+# Result: cmd2
+```
+
+#### 3. Priority Queue
+```bash
+kyws create triage --type priority_queue
+kypush "low" --priority 1
+kypush "high" --priority 100
+kypop
+# Result: high
+```
+See [docs/QUEUES_STACKS.md](docs/QUEUES_STACKS.md) for full details.
+
 ### 📂 Advanced JSONPath & Dot-Notation
 `kycli` allows you to treat your key-value store like a document database. You can query and update deep nested structures without retrieving the entire object.
 
@@ -160,10 +292,13 @@ kyg logs[0:5]
 ```
 
 #### Atomic Patching (Partial Updates):
-Instead of rewriting a large JSON object, you can update just one field.
+Instead of rewriting a large JSON object, you can update just one field. Use
+`kypatch` (not `kys`) for this — `kys` always treats dots in a key as a literal
+flat key name (e.g. for namespacing, like `kys ns.alpha 1`), while `kypatch`
+treats them as a path into the nested object.
 ```bash
 # Update just the 'age' field inside the 'user' object
-kys user.profile.age 25
+kypatch user.profile.age 25
 ```
 
 ### 📦 Collection Operations (Lists & Sets)
@@ -440,6 +575,24 @@ with Kycore() as kv:
     kv.save_many(items, ttl="1h")
     # Result: ⚡ Atomic transaction per batch (extremely fast)
 
+# 5. Queues & Stacks (API)
+use `set_type` to separate queues from KV stores.
+```python
+from kycli import Kycore
+
+# Initialize and set type
+with Kycore("jobs.db") as q:
+    q.set_type("queue")
+    
+    # Push (Atomic & Durable)
+    q.push("task_1")
+    q.push({"id": 2, "action": "email"})
+    
+    # Pop (Thread-Safe)
+    item = q.pop()
+    print(item) # task_1
+```
+
 # 6. Maintenance & PITR
 with Kycore() as kv:
     # Cleanup history older than 30 days
@@ -518,8 +671,9 @@ async def fetch_config(key: str, db: Kycore = Depends(get_db)):
 
 ## 🏗 Architecture & Internal Safety
 
-- **SQLite Engine**: Running in `WAL` (Write-Ahead Logging) mode for concurrent reads/writes.
-- **Atomic Operations**: Exports use a "temp-file then rename" strategy to prevent corruption.
+- **Encrypted Single-File Storage**: Each workspace is a single AES-GCM-encrypted file. `kycli` decrypts it into an in-memory SQLite engine on open and re-serializes the whole workspace back to disk on every write.
+- **Atomic Writes**: Every persist (workspace file, exports, audit export) uses a "temp-file then rename" strategy — a concurrent reader always sees a complete file, never a partial/corrupted one.
+- **Cross-Process Write Safety**: Writes from independent `kycli` processes sharing one workspace file are serialized via an OS-level advisory lock (`flock`, POSIX). A process reloads the latest on-disk state immediately before mutating it, so sibling processes' writes are never silently overwritten. *(Windows: atomic writes still prevent corruption, but cross-process mutual exclusion requires POSIX `flock` and is not yet supported.)*
 - **Data Integrity**: Keys are automatically lowercased and stripped to prevent duplicate-but-slightly-different keys.
 - **Auto-Purge Policy**: Deleted keys are moved to an **Archive** table and automatically purged after **15 days** to keep the database size optimized.
 - **Embedded C**: Core operations are written in Cython, binding directly to native library pointers.
@@ -530,7 +684,7 @@ async def fetch_config(key: str, db: Kycore = Depends(get_db)):
 
 Want to test the speed on your own hardware?
 ```bash
-PYTHONPATH=. python3 tests/integration/benchmark.py
+PYTHONPATH=. python3 tests/performance/kycli_benchmark.py
 ```
 
  ---

{kycli-0.2.3 → kycli-0.4.0}/TODO.md

@@ -37,15 +37,31 @@ This list tracks the progress of implementing high-performance and robust enhanc
 - [x] **100% Code Coverage Maintenance**
 ## Phase 6: Maintenance & Refinement ⚙️
 - [x] **Modular Refactoring** (Core engines split)
-- [ ] **Lock Management**: Retry with exponential backoff for "Database is locked" in multi-process scenarios.
-- [ ] **Activity Logs**: Background logging via Python `logging` module.
-- [ ] **Atomic Rename Exports**: Write-to-temp-then-rename for export safety.
-- [ ] **Compression**: Optional MsgPack/Zstd for large values.
+- [x] **Lock Management**: Cross-process `flock` mutual exclusion + reload-on-write around every mutating operation, with atomic temp-file+rename persistence (no more lost updates or file corruption from concurrent `kycli` processes sharing a workspace).
+- [x] **Activity Logs**: Background logging via Python `logging` module (`kycli/logging_utils.py`, writes to `~/.kycli/kycli.log`).
+- [x] **Atomic Rename Exports**: Write-to-temp-then-rename for export safety (`kye`/`kyaudit export`, and now the internal workspace persistence itself).
+- [x] **Compression**: Transparent zlib compression for values above a size threshold (workspace-configurable).
+
 ## Phase 7: Community & Branding 🤝
 - [x] **Community Guidelines** (COC, Contributing, Security)
 - [x] **Issue Templates**
 - [x] **Official Branding/Logo**
 - [x] **GitHub Repository Cleanup**
 
+## Phase 8: Roadmap (Planned Features) 🧭
+- [ ] **Batch Queue Ops**: `kypush --file` and `kypop --n 100` for throughput.
+- [ ] **Delayed Jobs**: `kypush --delay 30s` with scheduled dequeue.
+- [ ] **Visibility Timeout**: `kypop --lease 30s` + `kyack`/`kynack` for retry flows.
+- [ ] **Workspace TTL Policies**: Default TTL per workspace + `kyttl set/get`.
+- [ ] **Interactive CLI Prompts**: Fuzzy key search + history in non-TUI mode.
+- [ ] **Config Profiles**: `kyprofile use prod` to switch db path, master key, defaults.
+- [ ] **Output Formatting**: `--json` everywhere; `--pretty` for tables.
+- [ ] **Key ACLs / Scopes**: Read-only mode, write lock, per-workspace access key.
+- [ ] **Audit Export**: `kyaudit export` with time filters.
+- [ ] **Stats Command**: `kystats` for size, counts, TTL expirations, queue depth.
+- [ ] **Metrics Endpoint**: Optional local HTTP for queue depth + ops/sec.
+- [ ] **Namespace/Prefix Views**: `kyws view <prefix>` for large stores.
+- [ ] **Backup/Restore**: `kybackup` with encryption and versioned snapshots.
+
 ---
 *Optimized for Performance by Antigravity*