datavault-cli 1.0.0__py3-none-any.whl

datavault_cli-1.0.0.dist-info/METADATA

@@ -0,0 +1,178 @@
+Metadata-Version: 2.4
+Name: datavault-cli
+Version: 1.0.0
+Summary: Cryptographic provenance and version control for AI training datasets
+Home-page: https://github.com/thiminhkhuedao/DataVault
+Author: Ry Nguyen
+Author-email: thiminhkhuedao@gmail.com
+Keywords: data provenance versioning AI machine-learning CSV sha256
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Version Control
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: requires-python
+Dynamic: summary
+
+# DataVault
+
+**Cryptographic provenance and version control for AI training datasets.**
+
+Git tracks code. DataVault tracks data — with SHA-256 tamper detection, row-level CSV diffing, cryptographic hash chains, and multi-user signing workflows.
+
+Zero dependencies. Pure Python stdlib.
+
+---
+
+## Why not Git or DVC?
+
+| | Git | DVC | DataVault |
+|---|---|---|---|
+| Tracks file versions | ✓ | ✓ | ✓ |
+| Works on large files | ✗ | ✓ | ✓ |
+| Row-level CSV diff | ✗ | ✗ | ✓ |
+| Tamper detection (SHA-256) | partial | ✗ | ✓ |
+| Cryptographic hash chain | ✓ | ✗ | ✓ |
+| Multi-user signing & approval | ✗ | ✗ | ✓ |
+| Zero dependencies | ✗ | ✗ | ✓ |
+
+---
+
+## Quickstart
+
+```bash
+# 1. Clone the repo
+git clone https://github.com/thiminhkhuedao/DataVault.git
+cd DataVault
+
+# 2. Run the full demo (no setup needed)
+python test_datavault.py
+```
+
+That's it. No pip install, no config, no database. Just Python 3.6+.
+
+---
+
+## Usage
+
+```bash
+# Initialize a project in any folder
+python datavault.py init my-project
+
+# Track a dataset
+python datavault.py add training_data.csv "raw data from source"
+
+# After editing the file, commit the change
+python datavault.py commit training_data.csv "removed 42 duplicate rows"
+
+# View full history
+python datavault.py log training_data.csv
+
+# Prove the file hasn't been tampered with
+python datavault.py verify training_data.csv
+
+# See which rows changed between versions (CSV only)
+python datavault.py rowdiff training_data.csv v1 v3
+
+# Verify the full cryptographic chain
+python datavault.py chain training_data.csv
+
+# Sign a version
+python datavault.py sign training_data.csv v3 alice reviewer
+python datavault.py sign training_data.csv v3 bob approver
+python datavault.py approve training_data.csv v3
+
+# Open the web dashboard
+python datavault.py dashboard
+
+# Export a provenance report (HTML → print as PDF)
+python datavault.py export
+```
+
+---
+
+## All commands
+
+| Command | Description |
+|---|---|
+| `init <name>` | Start a new project |
+| `add <file> "msg"` | Start tracking a file |
+| `commit <file> "msg"` | Save a new version |
+| `log <file>` | Show version history |
+| `status` | Show all tracked files |
+| `verify <file>` | Check for tampering |
+| `diff <file> v1 v2` | Text diff between versions |
+| `checkout <file> v1` | Restore a version |
+| `tag <file> v2 "label"` | Label a version |
+| `rowlog <file>` | Row change stats (CSV) |
+| `rowdiff <file> v1 v2` | Which rows changed (CSV) |
+| `rowhistory <file> col val` | History of one specific row |
+| `chain <file>` | Verify cryptographic chain |
+| `chainshow <file>` | Display chain visually |
+| `sign <file> v <name> [role]` | Sign a version |
+| `signatures <file> [v]` | List or verify signatures |
+| `approve <file> v` | Check approval status |
+| `export` | Export HTML/PDF report |
+| `dashboard` | Open web dashboard |
+
+---
+
+## How it works
+
+**SHA-256 hashing** — every version gets a 64-character fingerprint. Change one character anywhere in the file, the fingerprint changes completely. `verify` catches any modification made outside DataVault.
+
+**Hash chain** — each version's hash includes the previous version's hash, like a blockchain. You cannot delete or reorder history without breaking the chain. `chain` verifies the full sequence back to genesis.
+
+**Row-level tracking** — for CSV files, DataVault tracks individual rows, not just the whole file. `rowdiff v1 v3` shows exactly which rows were added, deleted, or moved between any two versions.
+
+**Multi-user signing** — reviewers and approvers sign specific versions. Signatures are cryptographically tied to the file hash — if the file changes after signing, the signature is invalidated automatically.
+
+**Pi Bridge** — optional module that connects DataVault to a Raspberry Pi cluster, automatically committing hourly power and temperature readings as tracked CSV files with anomaly detection and auto-tagging.
+
+---
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `datavault.py` | CLI entrypoint — the only file you run |
+| `vault_core.py` | Core engine — hashing, versioning, storage |
+| `row_tracker.py` | Row-level CSV diffing |
+| `chain.py` | Cryptographic hash chain |
+| `signing.py` | Multi-user signing and approval |
+| `dashboard.py` | Local web dashboard (port 5000) |
+| `pi_bridge.py` | Auto-commit sensor data from Raspberry Pi |
+| `migrate.py` | Backfill snapshots for existing versions |
+| `test_datavault.py` | Full demo — run this first |
+
+---
+
+## Requirements
+
+- Python 3.6+
+- Zero external dependencies — pure standard library
+
+---
+
+## Why this matters
+
+When an AI system makes a wrong decision in a medical, legal, or financial context, regulators will ask: where did the training data come from? Who modified it? Which version was used? Today there is no standard tool that answers these questions reliably.
+
+DataVault is a working prototype of that provenance layer.
+
+---
+
+## Copyright
+
+Copyright © 2026 Ry Nguyen. All rights reserved.
+
+This project and its source code are public for portfolio viewing purposes only.
+No permission is granted to copy, distribute, modify, or use this code for any other purpose.

datavault_cli-1.0.0.dist-info/RECORD

@@ -0,0 +1,13 @@
+chain.py,sha256=bW3zTN5-ICcLqu-0e0518xQpqOzjYURnYFSYn8yuLb0,6735
+dashboard.py,sha256=koVXnGwqZ7UwXSTf7uCY-UKgyWJQc5dFdSQdRQH8ef0,10136
+datavault.py,sha256=Y-N9EwaokH26bmzNOQlZ1S2ijkxqQj3eG3owG1Dxh3Q,8760
+migrate.py,sha256=8A0kMzqR2Rii0icPwVlz2TCG2qdnIHRVOOzEPT-Vyd4,2535
+pi_bridge.py,sha256=d6aZZsFzNREYAcM_vnFUO1wFfBkxgNTmVH1oGm46YyE,8449
+row_tracker.py,sha256=gHFYWzieFt_SGHb-VZF5osnkAH3a0HKVSX5nNas5G5A,7680
+signing.py,sha256=hPAY2OjjfKg3Xgin1jJL9_DizOB24KPBI4oLuvIiuTc,7013
+vault_core.py,sha256=F0dGThUZUPCBwVYNLD8zqIJSnVAFix86kTHSIyQ8AP0,16697
+datavault_cli-1.0.0.dist-info/METADATA,sha256=V9j_J_RLbASyC1qfnYaEZix8q4NKbhz-VYpIXGM8B5s,6124
+datavault_cli-1.0.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+datavault_cli-1.0.0.dist-info/entry_points.txt,sha256=1yk981ex3zxgdjSm4V8myMyEurtZ4toi-1Xx9SNw26E,45
+datavault_cli-1.0.0.dist-info/top_level.txt,sha256=TU0TmxIQy0xiZE0KyOfET3S4vkUuoKSY3y1Um8SypsY,75
+datavault_cli-1.0.0.dist-info/RECORD,,

datavault_cli-1.0.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

datavault_cli-1.0.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+datavault = datavault:main

datavault_cli-1.0.0.dist-info/top_level.txt

@@ -0,0 +1,8 @@
+chain
+dashboard
+datavault
+migrate
+pi_bridge
+row_tracker
+signing
+vault_core

migrate.py

@@ -0,0 +1,70 @@
+# ============================================================
+# migrate.py — backfill row snapshots + hash chain
+#
+# Run once: python migrate.py
+# ============================================================
+
+import json
+import os
+import shutil
+import row_tracker
+import chain as chain_module
+
+VAULT_DIR    = ".datavault"
+HISTORY_FILE = ".datavault/history.json"
+VERSIONS_DIR = ".datavault/versions"
+
+def migrate():
+    if not os.path.exists(HISTORY_FILE):
+        print("No DataVault project found.")
+        return
+
+    with open(HISTORY_FILE, "r") as f:
+        history = json.load(f)
+
+    print("DataVault migration — backfilling row snapshots + hash chain\n")
+
+    for filename, data in history["files"].items():
+        versions = data["versions"]
+        print(f"File: {filename} ({len(versions)} version(s))")
+
+        for v in versions:
+            version_id = v["version_id"]
+
+            # Find the stored version file in .datavault/versions/
+            safe_name    = filename.replace("/", "_").replace("\\", "_")
+            stored_path  = os.path.join(VERSIONS_DIR, f"{safe_name}__{version_id}")
+
+            if not os.path.exists(stored_path):
+                print(f"  {version_id}: stored file not found, skipping")
+                continue
+
+            # Generate row snapshot from stored file (not current file)
+            if filename.lower().endswith(".csv"):
+                snapshot, headers = row_tracker.build_row_snapshot(stored_path)
+                if snapshot:
+                    row_tracker.save_row_snapshot(filename, version_id, snapshot, headers)
+                    print(f"  {version_id}: row snapshot created ({len(snapshot)} rows)")
+                else:
+                    print(f"  {version_id}: empty CSV, skipped")
+            else:
+                print(f"  {version_id}: not a CSV, skipping row snapshot")
+
+        # Build hash chain for this file
+        success, result = chain_module.build_chain(filename)
+        if success:
+            print(f"  Chain built ({len(result)} blocks)")
+        else:
+            print(f"  Chain failed: {result}")
+
+        print()
+
+    print("Migration complete.")
+    print("\nNow try:")
+    print("  python datavault.py rowlog training_data.csv")
+    print("  python datavault.py rowdiff training_data.csv v1 v3")
+    print("  python datavault.py chain training_data.csv")
+    print("  python datavault.py chainshow training_data.csv")
+
+if __name__ == "__main__":
+    migrate()

pi_bridge.py

@@ -0,0 +1,214 @@
+#!/usr/bin/env python3
+import os
+import sys
+import csv
+import json
+import time
+import argparse
+import subprocess
+from datetime import datetime, timedelta
+from collections import deque
+
+EXPORT_INTERVAL_SECONDS = 3600
+EXPORT_DIR              = "pi_exports"
+DATAVAULT_PROJECT       = "pi-datacenter"
+POWER_SPIKE_THRESHOLD   = 5.5
+TEMP_DANGER_THRESHOLD   = 75.0
+
+readings_buffer = deque(maxlen=3600)
+
+
+def simulated_reading():
+    import random, math
+    t          = time.time()
+    base_power = 3.5 + math.sin(t / 300) * 0.8
+    spike      = random.random() < 0.02
+    power      = base_power + (2.0 if spike else 0) + random.gauss(0, 0.1)
+    cpu_temp   = 45 + power * 4 + random.gauss(0, 1.5)
+    sink_temp  = cpu_temp - 15 + random.gauss(0, 0.5)
+    teg_power  = max(0, (cpu_temp - sink_temp) * 0.008)
+    return {
+        "timestamp":       datetime.now().isoformat(),
+        "pi_power_W":      round(power, 4),
+        "teg_power_W":     round(teg_power, 4),
+        "cpu_temp_C":      round(cpu_temp, 2),
+        "heatsink_temp_C": round(sink_temp, 2),
+        "thermal_delta_C": round(cpu_temp - sink_temp, 2),
+        "fan_speed_pct":   min(100, max(20, int((cpu_temp - 35) * 3))),
+        "efficiency_pct":  round((teg_power / power) * 100, 3) if power > 0 else 0,
+        "anomaly":         spike
+    }
+
+
+def real_reading():
+    state_file = "/tmp/datavault_pi_state.json"
+    if not os.path.exists(state_file):
+        return None
+    try:
+        with open(state_file, "r") as f:
+            state = json.load(f)
+        state["timestamp"] = datetime.now().isoformat()
+        state["anomaly"]   = (
+            state.get("pi_power_W", 0) > POWER_SPIKE_THRESHOLD or
+            state.get("cpu_temp_C", 0) > TEMP_DANGER_THRESHOLD
+        )
+        return state
+    except Exception:
+        return None
+
+
+def run_datavault(command_args):
+    result = subprocess.run(
+        [sys.executable, "datavault.py"] + command_args,
+        capture_output=True, text=True,
+        cwd=os.path.dirname(os.path.abspath(__file__))
+    )
+    return result.returncode == 0, result.stdout + result.stderr
+
+
+def ensure_datavault_initialized():
+    if not os.path.exists(".datavault"):
+        success, output = run_datavault(["init", DATAVAULT_PROJECT])
+        if success:
+            print(f"[DataVault] Initialized project '{DATAVAULT_PROJECT}'")
+        else:
+            print(f"[DataVault] Init failed: {output}")
+            return False
+    return True
+
+
+def export_hourly_csv(readings, export_time):
+    os.makedirs(EXPORT_DIR, exist_ok=True)
+    if not readings:
+        return None
+    filename = f"pi_power_{export_time.strftime('%Y-%m-%d_%H-%M')}.csv"
+    filepath = os.path.join(EXPORT_DIR, filename)
+    with open(filepath, "w", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(f, fieldnames=list(readings[0].keys()))
+        writer.writeheader()
+        writer.writerows(readings)
+    return filepath
+
+
+def compute_hourly_stats(readings):
+    if not readings:
+        return {}
+    powers    = [r["pi_power_W"]  for r in readings if r.get("pi_power_W")]
+    temps     = [r["cpu_temp_C"]  for r in readings if r.get("cpu_temp_C")]
+    teg       = [r["teg_power_W"] for r in readings if r.get("teg_power_W")]
+    anomalies = sum(1 for r in readings if r.get("anomaly"))
+    def avg(lst): return round(sum(lst)/len(lst), 3) if lst else 0
+    total_Wh  = avg(powers) * (len(readings) / 3600)
+    teg_Wh    = avg(teg)    * (len(readings) / 3600)
+    return {
+        "readings":          len(readings),
+        "avg_power_W":       avg(powers),
+        "max_power_W":       round(max(powers), 3) if powers else 0,
+        "avg_cpu_temp_C":    avg(temps),
+        "max_cpu_temp_C":    round(max(temps), 3) if temps else 0,
+        "total_energy_Wh":   round(total_Wh, 6),
+        "total_teg_Wh":      round(teg_Wh, 6),
+        "heat_recovery_pct": round((teg_Wh/total_Wh)*100, 3) if total_Wh > 0 else 0,
+        "anomalies":         anomalies
+    }
+
+
+def commit_hourly_export(readings, export_time):
+    filepath = export_hourly_csv(list(readings), export_time)
+    if not filepath:
+        print("[Bridge] No readings to export")
+        return
+
+    stats        = compute_hourly_stats(list(readings))
+    hour_str     = export_time.strftime("%Y-%m-%d %H:00")
+    anomaly_note = f" ⚠ {stats['anomalies']} anomalies" if stats.get("anomalies") else ""
+    message      = (
+        f"Pi power data {hour_str} | "
+        f"avg:{stats['avg_power_W']}W max:{stats['max_power_W']}W "
+        f"temp:{stats['avg_cpu_temp_C']}C "
+        f"TEG:{stats['heat_recovery_pct']}% recovered"
+        f"{anomaly_note}"
+    )
+
+    history_file    = ".datavault/history.json"
+    csv_filename    = os.path.basename(filepath)
+    already_tracked = False
+    if os.path.exists(history_file):
+        with open(history_file) as f:
+            already_tracked = csv_filename in json.load(f).get("files", {})
+
+    action  = "commit" if already_tracked else "add"
+    success, output = run_datavault([action, filepath, message])
+
+    if success:
+        print(f"[DataVault] {'Committed' if already_tracked else 'Added'}: {csv_filename}")
+        print(f"            {message}")
+        if stats.get("anomalies", 0) > 0:
+            with open(history_file) as f:
+                hist = json.load(f)
+            latest_v = hist["files"][csv_filename]["versions"][-1]["version_id"]
+            run_datavault(["tag", filepath, latest_v, f"ANOMALY: {stats['anomalies']} power/temp events detected"])
+            print(f"[DataVault] Tagged {latest_v} with anomaly warning")
+        _, output2 = run_datavault(["chain", csv_filename])
+        print(f"[DataVault] Chain: {'✓ intact' if 'CHAIN INTACT' in output2 else '✗ BROKEN'}")
+    else:
+        print(f"[DataVault] Failed: {output}")
+
+    return filepath, stats
+
+
+def main(simulate=False):
+    print("\n" + "="*55)
+    print("   PI BRIDGE — DataVault ↔ Mini Data Center")
+    print("="*55)
+    print(f"  Mode:   {'SIMULATED' if simulate else 'REAL (Raspberry Pi)'}")
+    print(f"  Export: every {EXPORT_INTERVAL_SECONDS//60} minutes")
+    print(f"\n  Press Ctrl+C to stop\n")
+
+    if not ensure_datavault_initialized():
+        sys.exit(1)
+
+    next_export   = datetime.now() + timedelta(seconds=EXPORT_INTERVAL_SECONDS)
+    reading_fn    = simulated_reading if simulate else real_reading
+    reading_count = 0
+
+    try:
+        while True:
+            reading = reading_fn()
+            if reading:
+                readings_buffer.append(reading)
+                reading_count += 1
+                if reading_count % 10 == 0:
+                    r = reading
+                    flag = " ⚠" if r.get("anomaly") else ""
+                    print(f"  [{datetime.now().strftime('%H:%M:%S')}] power:{r['pi_power_W']:.2f}W temp:{r['cpu_temp_C']:.1f}C TEG:{r['teg_power_W']:.4f}W fan:{r['fan_speed_pct']}%{flag}")
+
+            now = datetime.now()
+            if now >= next_export:
+                print(f"\n[{now.strftime('%H:%M:%S')}] Hourly export triggered...")
+                commit_hourly_export(readings_buffer, now)
+                next_export = now + timedelta(seconds=EXPORT_INTERVAL_SECONDS)
+                print(f"[Bridge] Next export in {EXPORT_INTERVAL_SECONDS//60} minutes\n")
+
+            time.sleep(1)
+
+    except KeyboardInterrupt:
+        print(f"\n\nBridge stopped. Readings: {reading_count} | Buffer: {len(readings_buffer)}")
+        if len(readings_buffer) > 10:
+            print("Running final export...")
+            commit_hourly_export(readings_buffer, datetime.now())
+        print("\nDataVault status:")
+        subprocess.run([sys.executable, "datavault.py", "status"], cwd=os.path.dirname(os.path.abspath(__file__)))
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Pi Bridge — connects mini data center to DataVault")
+    parser.add_argument("--simulate", action="store_true", help="Simulation mode (no real Pi sensors)")
+    parser.add_argument("--fast",     action="store_true", help="Export every 60s instead of 60min (testing)")
+    args = parser.parse_args()
+
+    if args.fast:
+        EXPORT_INTERVAL_SECONDS = 60
+        print("[Bridge] Fast mode: exporting every 60 seconds")
+
+    main(simulate=args.simulate)