datavault-cli 1.0.0__tar.gz

datavault_cli-1.0.0/PKG-INFO

@@ -0,0 +1,178 @@
+Metadata-Version: 2.4
+Name: datavault-cli
+Version: 1.0.0
+Summary: Cryptographic provenance and version control for AI training datasets
+Home-page: https://github.com/thiminhkhuedao/DataVault
+Author: Ry Nguyen
+Author-email: thiminhkhuedao@gmail.com
+Keywords: data provenance versioning AI machine-learning CSV sha256
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Version Control
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: requires-python
+Dynamic: summary
+
+# DataVault
+
+**Cryptographic provenance and version control for AI training datasets.**
+
+Git tracks code. DataVault tracks data — with SHA-256 tamper detection, row-level CSV diffing, cryptographic hash chains, and multi-user signing workflows.
+
+Zero dependencies. Pure Python stdlib.
+
+---
+
+## Why not Git or DVC?
+
+| | Git | DVC | DataVault |
+|---|---|---|---|
+| Tracks file versions | ✓ | ✓ | ✓ |
+| Works on large files | ✗ | ✓ | ✓ |
+| Row-level CSV diff | ✗ | ✗ | ✓ |
+| Tamper detection (SHA-256) | partial | ✗ | ✓ |
+| Cryptographic hash chain | ✓ | ✗ | ✓ |
+| Multi-user signing & approval | ✗ | ✗ | ✓ |
+| Zero dependencies | ✗ | ✗ | ✓ |
+
+---
+
+## Quickstart
+
+```bash
+# 1. Clone the repo
+git clone https://github.com/thiminhkhuedao/DataVault.git
+cd DataVault
+
+# 2. Run the full demo (no setup needed)
+python test_datavault.py
+```
+
+That's it. No pip install, no config, no database. Just Python 3.6+.
+
+---
+
+## Usage
+
+```bash
+# Initialize a project in any folder
+python datavault.py init my-project
+
+# Track a dataset
+python datavault.py add training_data.csv "raw data from source"
+
+# After editing the file, commit the change
+python datavault.py commit training_data.csv "removed 42 duplicate rows"
+
+# View full history
+python datavault.py log training_data.csv
+
+# Prove the file hasn't been tampered with
+python datavault.py verify training_data.csv
+
+# See which rows changed between versions (CSV only)
+python datavault.py rowdiff training_data.csv v1 v3
+
+# Verify the full cryptographic chain
+python datavault.py chain training_data.csv
+
+# Sign a version
+python datavault.py sign training_data.csv v3 alice reviewer
+python datavault.py sign training_data.csv v3 bob approver
+python datavault.py approve training_data.csv v3
+
+# Open the web dashboard
+python datavault.py dashboard
+
+# Export a provenance report (HTML → print as PDF)
+python datavault.py export
+```
+
+---
+
+## All commands
+
+| Command | Description |
+|---|---|
+| `init <name>` | Start a new project |
+| `add <file> "msg"` | Start tracking a file |
+| `commit <file> "msg"` | Save a new version |
+| `log <file>` | Show version history |
+| `status` | Show all tracked files |
+| `verify <file>` | Check for tampering |
+| `diff <file> v1 v2` | Text diff between versions |
+| `checkout <file> v1` | Restore a version |
+| `tag <file> v2 "label"` | Label a version |
+| `rowlog <file>` | Row change stats (CSV) |
+| `rowdiff <file> v1 v2` | Which rows changed (CSV) |
+| `rowhistory <file> col val` | History of one specific row |
+| `chain <file>` | Verify cryptographic chain |
+| `chainshow <file>` | Display chain visually |
+| `sign <file> v <name> [role]` | Sign a version |
+| `signatures <file> [v]` | List or verify signatures |
+| `approve <file> v` | Check approval status |
+| `export` | Export HTML/PDF report |
+| `dashboard` | Open web dashboard |
+
+---
+
+## How it works
+
+**SHA-256 hashing** — every version gets a 64-character fingerprint. Change one character anywhere in the file, the fingerprint changes completely. `verify` catches any modification made outside DataVault.
+
+**Hash chain** — each version's hash includes the previous version's hash, like a blockchain. You cannot delete or reorder history without breaking the chain. `chain` verifies the full sequence back to genesis.
+
+**Row-level tracking** — for CSV files, DataVault tracks individual rows, not just the whole file. `rowdiff v1 v3` shows exactly which rows were added, deleted, or moved between any two versions.
+
+**Multi-user signing** — reviewers and approvers sign specific versions. Signatures are cryptographically tied to the file hash — if the file changes after signing, the signature is invalidated automatically.
+
+**Pi Bridge** — optional module that connects DataVault to a Raspberry Pi cluster, automatically committing hourly power and temperature readings as tracked CSV files with anomaly detection and auto-tagging.
+
+---
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `datavault.py` | CLI entrypoint — the only file you run |
+| `vault_core.py` | Core engine — hashing, versioning, storage |
+| `row_tracker.py` | Row-level CSV diffing |
+| `chain.py` | Cryptographic hash chain |
+| `signing.py` | Multi-user signing and approval |
+| `dashboard.py` | Local web dashboard (port 5000) |
+| `pi_bridge.py` | Auto-commit sensor data from Raspberry Pi |
+| `migrate.py` | Backfill snapshots for existing versions |
+| `test_datavault.py` | Full demo — run this first |
+
+---
+
+## Requirements
+
+- Python 3.6+
+- Zero external dependencies — pure standard library
+
+---
+
+## Why this matters
+
+When an AI system makes a wrong decision in a medical, legal, or financial context, regulators will ask: where did the training data come from? Who modified it? Which version was used? Today there is no standard tool that answers these questions reliably.
+
+DataVault is a working prototype of that provenance layer.
+
+---
+
+## Copyright
+
+Copyright © 2026 Ry Nguyen. All rights reserved.
+
+This project and its source code are public for portfolio viewing purposes only.
+No permission is granted to copy, distribute, modify, or use this code for any other purpose.

datavault_cli-1.0.0/README.md

@@ -0,0 +1,154 @@
+# DataVault
+
+**Cryptographic provenance and version control for AI training datasets.**
+
+Git tracks code. DataVault tracks data — with SHA-256 tamper detection, row-level CSV diffing, cryptographic hash chains, and multi-user signing workflows.
+
+Zero dependencies. Pure Python stdlib.
+
+---
+
+## Why not Git or DVC?
+
+| | Git | DVC | DataVault |
+|---|---|---|---|
+| Tracks file versions | ✓ | ✓ | ✓ |
+| Works on large files | ✗ | ✓ | ✓ |
+| Row-level CSV diff | ✗ | ✗ | ✓ |
+| Tamper detection (SHA-256) | partial | ✗ | ✓ |
+| Cryptographic hash chain | ✓ | ✗ | ✓ |
+| Multi-user signing & approval | ✗ | ✗ | ✓ |
+| Zero dependencies | ✗ | ✗ | ✓ |
+
+---
+
+## Quickstart
+
+```bash
+# 1. Clone the repo
+git clone https://github.com/thiminhkhuedao/DataVault.git
+cd DataVault
+
+# 2. Run the full demo (no setup needed)
+python test_datavault.py
+```
+
+That's it. No pip install, no config, no database. Just Python 3.6+.
+
+---
+
+## Usage
+
+```bash
+# Initialize a project in any folder
+python datavault.py init my-project
+
+# Track a dataset
+python datavault.py add training_data.csv "raw data from source"
+
+# After editing the file, commit the change
+python datavault.py commit training_data.csv "removed 42 duplicate rows"
+
+# View full history
+python datavault.py log training_data.csv
+
+# Prove the file hasn't been tampered with
+python datavault.py verify training_data.csv
+
+# See which rows changed between versions (CSV only)
+python datavault.py rowdiff training_data.csv v1 v3
+
+# Verify the full cryptographic chain
+python datavault.py chain training_data.csv
+
+# Sign a version
+python datavault.py sign training_data.csv v3 alice reviewer
+python datavault.py sign training_data.csv v3 bob approver
+python datavault.py approve training_data.csv v3
+
+# Open the web dashboard
+python datavault.py dashboard
+
+# Export a provenance report (HTML → print as PDF)
+python datavault.py export
+```
+
+---
+
+## All commands
+
+| Command | Description |
+|---|---|
+| `init <name>` | Start a new project |
+| `add <file> "msg"` | Start tracking a file |
+| `commit <file> "msg"` | Save a new version |
+| `log <file>` | Show version history |
+| `status` | Show all tracked files |
+| `verify <file>` | Check for tampering |
+| `diff <file> v1 v2` | Text diff between versions |
+| `checkout <file> v1` | Restore a version |
+| `tag <file> v2 "label"` | Label a version |
+| `rowlog <file>` | Row change stats (CSV) |
+| `rowdiff <file> v1 v2` | Which rows changed (CSV) |
+| `rowhistory <file> col val` | History of one specific row |
+| `chain <file>` | Verify cryptographic chain |
+| `chainshow <file>` | Display chain visually |
+| `sign <file> v <name> [role]` | Sign a version |
+| `signatures <file> [v]` | List or verify signatures |
+| `approve <file> v` | Check approval status |
+| `export` | Export HTML/PDF report |
+| `dashboard` | Open web dashboard |
+
+---
+
+## How it works
+
+**SHA-256 hashing** — every version gets a 64-character fingerprint. Change one character anywhere in the file, the fingerprint changes completely. `verify` catches any modification made outside DataVault.
+
+**Hash chain** — each version's hash includes the previous version's hash, like a blockchain. You cannot delete or reorder history without breaking the chain. `chain` verifies the full sequence back to genesis.
+
+**Row-level tracking** — for CSV files, DataVault tracks individual rows, not just the whole file. `rowdiff v1 v3` shows exactly which rows were added, deleted, or moved between any two versions.
+
+**Multi-user signing** — reviewers and approvers sign specific versions. Signatures are cryptographically tied to the file hash — if the file changes after signing, the signature is invalidated automatically.
+
+**Pi Bridge** — optional module that connects DataVault to a Raspberry Pi cluster, automatically committing hourly power and temperature readings as tracked CSV files with anomaly detection and auto-tagging.
+
+---
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `datavault.py` | CLI entrypoint — the only file you run |
+| `vault_core.py` | Core engine — hashing, versioning, storage |
+| `row_tracker.py` | Row-level CSV diffing |
+| `chain.py` | Cryptographic hash chain |
+| `signing.py` | Multi-user signing and approval |
+| `dashboard.py` | Local web dashboard (port 5000) |
+| `pi_bridge.py` | Auto-commit sensor data from Raspberry Pi |
+| `migrate.py` | Backfill snapshots for existing versions |
+| `test_datavault.py` | Full demo — run this first |
+
+---
+
+## Requirements
+
+- Python 3.6+
+- Zero external dependencies — pure standard library
+
+---
+
+## Why this matters
+
+When an AI system makes a wrong decision in a medical, legal, or financial context, regulators will ask: where did the training data come from? Who modified it? Which version was used? Today there is no standard tool that answers these questions reliably.
+
+DataVault is a working prototype of that provenance layer.
+
+---
+
+## Copyright
+
+Copyright © 2026 Ry Nguyen. All rights reserved.
+
+This project and its source code are public for portfolio viewing purposes only.
+No permission is granted to copy, distribute, modify, or use this code for any other purpose.

datavault_cli-1.0.0/chain.py

@@ -0,0 +1,173 @@
+import hashlib
+import json
+import os
+
+HISTORY_FILE = ".datavault/history.json"
+CHAIN_FILE   = ".datavault/chain.json"
+
+
+def compute_chain_hash(file_hash, prev_chain_hash, version_id, timestamp, author, message):
+    chain_input = "|".join([file_hash, prev_chain_hash, version_id, timestamp, author, message])
+    return hashlib.sha256(chain_input.encode()).hexdigest()
+
+
+def build_chain(filename):
+    if not os.path.exists(HISTORY_FILE):
+        return False, "No DataVault project found."
+    with open(HISTORY_FILE, "r", encoding="utf-8") as f:
+        history = json.load(f)
+    if filename not in history.get("files", {}):
+        return False, f"'{filename}' is not tracked."
+
+    versions        = history["files"][filename]["versions"]
+    prev_chain_hash = "GENESIS"
+    chain           = []
+
+    for v in versions:
+        chain_hash = compute_chain_hash(
+            v["hash"], prev_chain_hash, v["version_id"],
+            v["timestamp"], v.get("author", "unknown"), v["message"]
+        )
+        chain.append({
+            "version_id":      v["version_id"],
+            "file_hash":       v["hash"],
+            "prev_chain_hash": prev_chain_hash,
+            "chain_hash":      chain_hash,
+            "timestamp":       v["timestamp"],
+            "author":          v.get("author", "unknown"),
+            "message":         v["message"]
+        })
+        v["chain_hash"]      = chain_hash
+        v["prev_chain_hash"] = prev_chain_hash
+        prev_chain_hash      = chain_hash
+
+    with open(HISTORY_FILE, "w", encoding="utf-8") as f:
+        json.dump(history, f, indent=2)
+
+    chain_data = _load_chain()
+    chain_data["files"][filename] = chain
+    _save_chain(chain_data)
+    return True, chain
+
+
+def verify_chain(filename):
+    if not os.path.exists(HISTORY_FILE):
+        return False, "No DataVault project found."
+    with open(HISTORY_FILE, "r", encoding="utf-8") as f:
+        history = json.load(f)
+    if filename not in history.get("files", {}):
+        return False, f"'{filename}' is not tracked."
+
+    versions = history["files"][filename]["versions"]
+    if not any("chain_hash" in v for v in versions):
+        return False, f"No chain data for '{filename}'.\nRun: python datavault.py chainbuild {filename}"
+
+    prev_chain_hash = "GENESIS"
+    results         = []
+    all_valid       = True
+
+    for v in versions:
+        stored = v.get("chain_hash")
+        if not stored:
+            results.append({"version_id": v["version_id"], "valid": False, "reason": "no chain hash — run chainbuild"})
+            all_valid = False
+            continue
+
+        expected = compute_chain_hash(
+            v["hash"], prev_chain_hash, v["version_id"],
+            v["timestamp"], v.get("author", "unknown"), v["message"]
+        )
+        valid = (expected == stored)
+        if not valid:
+            all_valid = False
+
+        results.append({
+            "version_id": v["version_id"],
+            "valid":      valid,
+            "stored":     stored[:24],
+            "expected":   expected[:24],
+            "prev":       prev_chain_hash[:24] if prev_chain_hash != "GENESIS" else "GENESIS",
+            "timestamp":  v["timestamp"][:19].replace("T", " "),
+            "message":    v["message"]
+        })
+        prev_chain_hash = stored
+
+    return all_valid, results
+
+
+def format_chain_verification(filename, results):
+    lines = [f"\nChain verification: '{filename}'\n" + "─"*55]
+    for r in results:
+        if r["valid"]:
+            lines.append(
+                f"\n  ✓ {r['version_id']}  |  {r.get('timestamp','')}\n"
+                f"    chain: {r.get('stored','?')}…\n"
+                f"    prev:  {r.get('prev','?')}…\n"
+                f"    \"{r.get('message','')}\""
+            )
+        else:
+            lines.append(
+                f"\n  ✗ {r['version_id']}  CHAIN BROKEN\n"
+                f"    stored:   {r.get('stored','?')}…\n"
+                f"    expected: {r.get('expected','?')}…\n"
+                f"    reason:   {r.get('reason','hash mismatch')}"
+            )
+    lines.append(f"\n{'─'*55}")
+    if all(r["valid"] for r in results):
+        lines.append(
+            f"  ✓ CHAIN INTACT — {len(results)} version(s) verified\n"
+            f"  Complete history is authentic and unmodified.\n"
+            f"  No versions were deleted, reordered, or tampered with."
+        )
+    else:
+        broken = [r["version_id"] for r in results if not r["valid"]]
+        lines.append(
+            f"  ✗ CHAIN BROKEN at: {', '.join(broken)}\n"
+            f"  History has been tampered with."
+        )
+    return "\n".join(lines)
+
+
+def display_chain(filename):
+    if not os.path.exists(HISTORY_FILE):
+        return False, "No DataVault project found."
+    with open(HISTORY_FILE, "r", encoding="utf-8") as f:
+        history = json.load(f)
+    if filename not in history.get("files", {}):
+        return False, f"'{filename}' is not tracked."
+
+    versions = history["files"][filename]["versions"]
+    lines    = [f"\nHash chain: '{filename}'\n" + "─"*55]
+    lines.append("  (Each block's hash includes the previous block's hash)")
+    lines.append("  (Changing any block invalidates all blocks after it)\n")
+
+    for i, v in enumerate(versions):
+        chain_hash = v.get("chain_hash", "not computed")[:20]
+        file_hash  = v["hash"][:20]
+        prev_hash  = v.get("prev_chain_hash", "GENESIS")
+        if prev_hash != "GENESIS":
+            prev_hash = prev_hash[:20]
+        lines.append(f"  ┌─────────────────────────────────────┐")
+        lines.append(f"  │  {v['version_id']}  {v['timestamp'][:10]}           │")
+        lines.append(f"  │  file hash:  {file_hash}…  │")
+        lines.append(f"  │  prev hash:  {prev_hash}…  │")
+        lines.append(f"  │  chain hash: {chain_hash}…  │")
+        lines.append(f"  │  \"{v['message'][:30]}\"")
+        lines.append(f"  └──────────────────┬──────────────────┘")
+        if i < len(versions) - 1:
+            lines.append(f"                     │")
+            lines.append(f"                     ▼")
+
+    return True, "\n".join(lines)
+
+
+def _load_chain():
+    if os.path.exists(CHAIN_FILE):
+        with open(CHAIN_FILE, "r", encoding="utf-8") as f:
+            return json.load(f)
+    return {"files": {}}
+
+
+def _save_chain(data):
+    with open(CHAIN_FILE, "w", encoding="utf-8") as f:
+        json.dump(data, f, indent=2)