pydorky 2.1.8

package/todo/01-core-infrastructure.md

@@ -0,0 +1,84 @@
+# Core Infrastructure
+
+Core HTTP service, CLI foundation, and Node version alignment.
+
+## P0: HTTP Service Foundation
+
+### Upload & Download
+- [ ] Finalize OpenAPI spec (multipart form, JSON responses, error codes)
+- [ ] Implement `POST /artifacts` endpoint with streaming support
+- [ ] Implement `GET /artifacts/{id}` endpoint with streaming download
+- [ ] Add HTTP 201 Created response for uploads
+- [ ] Add HTTP 200 OK for downloads
+- [ ] Handle large file uploads (>100MB) efficiently
+
+### Metadata Operations
+- [ ] Implement `GET /artifacts/{id}/metadata` endpoint
+- [ ] Implement `PATCH /artifacts/{id}/metadata` endpoint
+- [ ] Support custom key-value metadata
+- [ ] Store `content_hash`, `file_size`, `last_updated`, `uploaded_by`
+
+### Idempotency
+- [ ] Add idempotency key support in upload endpoint
+- [ ] Store idempotency key → artifact ID mapping
+- [ ] Return cached result if same key uploaded twice
+- [ ] Auto-clean old idempotency entries (TTL)
+
+### Health & Status
+- [ ] Add `GET /health` endpoint
+- [ ] Add `GET /status` with service diagnostics
+- [ ] Return storage backend status
+
+## P0: Node Version & CI
+
+### Node Version Management
+- [ ] Add `.nvmrc` file with `16` in repo root
+- [ ] Add `engines.node` to `package.json` (`>=16`)
+- [ ] Update devcontainer image to Node 16 base
+- [ ] Document Node version requirement in `README.md` and `docs/doc#1  get-started`
+
+### CI/CD Alignment
+- [ ] Ensure publish workflow runs on Node 16
+- [ ] Add CI matrix job for Node 16 + Node 20 testing
+- [ ] Run integration tests on both versions
+- [ ] Plan migration steps to Node 20 (documented)
+
+## P1: CLI Foundation
+
+### Commit/Stage/Push Workflow
+- [ ] Implement `dorky init` — scaffold project config
+- [ ] Implement `dorky status` — show staged/changed files
+- [ ] Implement `dorky stage <path>` — add file to staging
+- [ ] Implement `dorky unstage <path>` — remove from staging
+- [ ] Implement `dorky commit <message>` — create commit with staged files
+- [ ] Implement `dorky push` — upload commits to remote bucket
+- [ ] Implement `dorky pull` — download latest from bucket
+
+### Configuration
+- [ ] Add `.dorkyrc` project config file support
+- [ ] Support environment variables for bucket URL, credentials
+- [ ] Store local state in `.dorky/` directory
+
+### Error Handling
+- [ ] Add `--verbose` flag for debugging
+- [ ] Add `--help` for all commands
+- [ ] Return clear error messages with actionable hints
+
+## P2: Testing & Documentation
+
+### Testing
+- [ ] Add unit tests for upload/download logic
+- [ ] Add integration tests with local server
+- [ ] Add E2E tests for CLI workflow
+- [ ] Test with various file sizes (small, medium, large >1GB)
+
+### Documentation
+- [ ] Document OpenAPI spec with examples
+- [ ] Add CLI command reference to README
+- [ ] Add architecture diagram
+- [ ] Add troubleshooting guide
+
+---
+
+**Status**: Core service + CLI in dev, Node alignment pending
+**Next**: Finalize OpenAPI spec, implement upload/download endpoints

package/todo/02-storage-providers.md

@@ -0,0 +1,104 @@
+# Storage Providers (BYOB)
+
+Bring-your-own-bucket support for AWS S3, Google Cloud Storage, Azure, and local backends.
+
+## P0: Existing Providers
+
+### AWS S3
+- [x] Basic S3 client integration exists
+- [ ] Verify streaming upload works (multipart)
+- [ ] Verify streaming download works
+- [ ] Add region configuration support
+- [ ] Test with various file sizes
+- [ ] Add S3 access error handling
+
+### Google Cloud Storage (GCS)
+- [x] Basic GCS integration exists
+- [ ] Verify streaming upload works
+- [ ] Verify streaming download works
+- [ ] Add project ID configuration
+- [ ] Test bucket access and permissions
+- [ ] Add GCS-specific error handling
+
+### Google Drive
+- [x] Integration exists (documented in docs)
+- [ ] Test with OAuth 2.0 flow
+- [ ] Verify file sync and versioning
+
+## P1: New Providers
+
+### Azure Blob Storage
+- [ ] Implement Azure Blob Storage client
+- [ ] Add connection string support
+- [ ] Implement streaming upload (block blobs)
+- [ ] Implement streaming download
+- [ ] Add container creation support
+- [ ] Test with various blob sizes
+
+### MinIO / S3-Compatible
+- [ ] Implement S3-compatible endpoint support
+- [ ] Allow custom endpoint URL configuration
+- [ ] Test with MinIO local instance
+- [ ] Test with other S3-compatible services (DigitalOcean Spaces, etc.)
+
+### Local Filesystem (Dev/Testing)
+- [ ] Implement local filesystem backend
+- [ ] Use `.dorky-data/` directory
+- [ ] Support all operations (upload, download, metadata, delete)
+- [ ] Add for easy local testing without cloud credentials
+
+## P2: Bucket Configuration & Management
+
+### Bucket Provisioning
+- [ ] Implement `dorky bucket create` command
+- [ ] Auto-detect cloud provider (S3, GCS, Azure)
+- [ ] Create bucket with proper permissions
+- [ ] Store credentials securely
+
+### Bucket Operations
+- [ ] Implement `dorky bucket list` — list all configured buckets
+- [ ] Implement `dorky bucket delete` — remove bucket reference
+- [ ] Implement `dorky bucket validate` — test connection and permissions
+- [ ] Add bucket selection for multi-bucket projects
+
+### Credentials Management
+- [ ] Support AWS IAM credentials (env vars or ~/.aws/credentials)
+- [ ] Support GCS service account keys
+- [ ] Support Azure connection strings
+- [ ] Add `dorky config credentials` command
+
+## P2: Hierarchical Sync
+
+### Folder Structure
+- [ ] Create hierarchical folder structure on bucket (e.g., `projects/myproject/data/`)
+- [ ] Map local file structure to bucket paths
+- [ ] Support nested directories at arbitrary depth
+
+### Sync Operations
+- [ ] Implement `dorky sync` — two-way sync of local/remote
+- [ ] Implement `dorky upload-folder <path>` — upload entire directory
+- [ ] Implement `dorky download-folder <path>` — download entire directory
+- [ ] Support `.dorkyignore` patterns (like `.gitignore`)
+
+### Incremental Sync
+- [ ] Track synced files and timestamps
+- [ ] Only sync changed files on subsequent runs
+- [ ] Add `--force` flag to re-sync everything
+- [ ] Detect conflicts and prompt user
+
+## P3: Advanced Features
+
+### Multi-Cloud
+- [ ] Support multiple buckets across different cloud providers
+- [ ] Implement replication across providers
+- [ ] Add failover logic
+
+### Performance
+- [ ] Add concurrent uploads to multiple buckets (backup)
+- [ ] Add bandwidth limiting for uploads/downloads
+- [ ] Cache bucket listings for faster operations
+
+---
+
+**Status**: S3, GCS, Google Drive exist; Azure/MinIO/local pending
+**Next**: Test existing providers, implement Azure + local filesystem

package/todo/03-compression-formats.md

@@ -0,0 +1,94 @@
+# Compression & Data Formats
+
+Compress artifacts during upload and support data format conversions (CSV to Parquet, etc.).
+
+## P1: Core Compression
+
+### Gzip
+- [ ] Implement gzip compression in upload flow
+- [ ] Store `Content-Encoding: gzip` in metadata
+- [ ] Auto-decompress on download
+- [ ] Add `--compress=gzip` CLI flag
+- [ ] Add compression level configuration (1-9)
+
+### LZ4 (Fast Compression)
+- [ ] Add LZ4 compression support (faster than gzip)
+- [ ] Store compression type in metadata
+- [ ] Add `--compress=lz4` CLI flag
+- [ ] Test performance vs gzip
+
+### Brotli
+- [ ] Add Brotli compression support (better ratio)
+- [ ] Add `--compress=brotli` CLI flag
+- [ ] Test performance vs gzip/lz4
+
+### Decompression on Download
+- [ ] Auto-detect compression from metadata
+- [ ] Decompress transparently during download
+- [ ] Support partial decompression (streaming)
+- [ ] Add `--no-decompress` flag to keep compressed
+
+## P1: Compression Configuration
+
+### Project-Level Config
+- [ ] Add compression preference in `.dorkyrc`
+- [ ] Support per-file pattern compression (e.g., `*.json` → gzip)
+- [ ] Add `dorky config compression` command
+
+### Metadata Tracking
+- [ ] Store `compression_type` in metadata
+- [ ] Store `compressed_size` alongside `file_size`
+- [ ] Store `compression_ratio` for reporting
+- [ ] Add `--show-compression` flag to display stats
+
+## P2: Data Format Conversions
+
+### CSV to Parquet
+- [ ] Create Python worker service for conversions
+- [ ] Implement CSV → Parquet conversion
+- [ ] Infer schema from CSV headers
+- [ ] Support schema config file (JSON)
+- [ ] Add `dorky convert csv-to-parquet <input> <output>` command
+
+### JSON Compression
+- [ ] Minify JSON before compression
+- [ ] Remove whitespace and comments
+- [ ] Add `--minify-json` flag
+- [ ] Store original schema separately for reconstruction
+
+### Format Detection
+- [ ] Auto-detect file type on upload (.json, .csv, .parquet, etc.)
+- [ ] Suggest optimal compression based on file type
+- [ ] Add `--format` hint flag for ambiguous files
+
+## P2: Data Quality & Validation
+
+### Integrity Checking
+- [ ] Store SHA256 hash of original file in metadata
+- [ ] Verify hash on download before decompression
+- [ ] Add `--verify` flag for integrity check
+- [ ] Report any corrupted files
+
+### Schema Validation
+- [ ] Optional schema validation for structured formats
+- [ ] Store schema in `.dorkyschema` or metadata
+- [ ] Validate before uploading
+- [ ] Warn on schema changes
+
+## P3: Advanced Conversions
+
+### Other Formats
+- [ ] Parquet → Arrow IPC
+- [ ] JSON → MessagePack
+- [ ] Binary format conversions
+- [ ] Add `dorky convert list` to show available conversions
+
+### Streaming Conversions
+- [ ] Stream large CSV → Parquet without buffering
+- [ ] Support incremental conversions
+- [ ] Allow on-demand conversion (download → convert → return)
+
+---
+
+**Status**: Compression infrastructure pending, format conversions planned
+**Next**: Implement gzip/lz4, add compression config, create Python conversion service

package/todo/04-python-client.md

@@ -0,0 +1,126 @@
+# Python Client (PyPI)
+
+Lightweight Python package on PyPI that mirrors Node client feature set.
+
+## P0: Core Blocking Client
+
+### Basic Operations
+- [x] Scaffold `python-client/` with `pyproject.toml`
+- [x] Implement `DorkyClient.upload()` (blocking)
+- [x] Implement `DorkyClient.download()` (blocking)
+- [ ] Add `DorkyClient.get_metadata()` method
+- [ ] Add `DorkyClient.delete()` method
+- [ ] Verify streaming for large files (no buffering)
+
+### Idempotency & Conflict Handling
+- [ ] Add idempotency key support in `upload()`
+- [ ] Handle 409 Conflict responses
+- [ ] Add retry logic with exponential backoff
+- [ ] Support `--force` option to overwrite
+
+### Error Handling
+- [ ] Catch and wrap HTTP errors
+- [ ] Provide clear error messages
+- [ ] Add retry-after handling (429 Rate Limited)
+- [ ] Add timeout configuration
+
+## P1: Async Client
+
+### Async Implementation
+- [ ] Create `AsyncDorkyClient` using `httpx`
+- [ ] Mirror all sync methods in async version
+- [ ] Add async `upload()`, `download()`, `get_metadata()`, `delete()`
+- [ ] Add connection pooling and session reuse
+- [ ] Support async context manager (`async with`)
+
+### Testing
+- [ ] Add integration tests for async client
+- [ ] Test concurrent uploads/downloads
+- [ ] Verify no blocking calls in async code
+
+## P1: CLI Wrapper
+
+### CLI Commands
+- [ ] Add `dorky-py` or `dorky` CLI entry point
+- [ ] Implement `dorky-py commit <message>` command
+- [ ] Implement `dorky-py stage <path>` command
+- [ ] Implement `dorky-py push` command
+- [ ] Implement `dorky-py pull` command
+- [ ] Implement `dorky-py download <artifact_id> <dest>` command
+- [ ] Implement `dorky-py status` command
+
+### Configuration
+- [ ] Read from `.dorkyrc` in project root
+- [ ] Support environment variables (DORKY_URL, DORKY_BUCKET, etc.)
+- [ ] Add `dorky-py config` command to view/edit settings
+- [ ] Support `~/.dorky/config` for user defaults
+
+### Progress & Output
+- [ ] Show progress bar for uploads/downloads
+- [ ] Add `--quiet` / `-q` flag
+- [ ] Add `--verbose` / `-v` flag for debugging
+- [ ] Pretty-print metadata and file listings
+
+## P1: PyPI Publishing
+
+### Package Setup
+- [ ] Finalize `pyproject.toml` with metadata
+- [ ] Add PyPI classifiers (Python 3.7+)
+- [ ] Set up build script (uses setuptools/wheel)
+- [ ] Test local install: `pip install -e .`
+
+### Publishing Workflow
+- [ ] Create TestPyPI account
+- [ ] Publish to TestPyPI first
+- [ ] Test install from TestPyPI: `pip install --index-url https://test.pypi.org/simple/ dorky-client`
+- [ ] Add GitHub Actions workflow for PyPI publish
+- [ ] Publish to PyPI (production)
+
+### Version Management
+- [ ] Follow semantic versioning (MAJOR.MINOR.PATCH)
+- [ ] Update version in `pyproject.toml` for each release
+- [ ] Tag releases in git (e.g., `py-v0.1.0`)
+- [ ] Auto-generate changelog from commits
+
+## P2: Compression & Format Support
+
+### Compression
+- [ ] Add `upload(compress='gzip')` parameter
+- [ ] Support `compress='lz4'`, `compress='brotli'`
+- [ ] Auto-decompress downloads based on metadata
+- [ ] Add `--compress` flag to CLI
+
+### Data Formats
+- [ ] Add optional `pyarrow` dependency for Parquet
+- [ ] Support `upload(..., format='parquet')`
+- [ ] Add format conversion helpers
+- [ ] Make heavy dependencies optional (`dorky[parquet]`, `dorky[async]`)
+
+## P2: Optional Extras
+
+### Package Extras
+- [ ] `dorky[parquet]` — includes `pyarrow` for Parquet support
+- [ ] `dorky[async]` — includes `httpx` for async client
+- [ ] `dorky[all]` — includes all optional dependencies
+- [ ] Document in README
+
+### Security Extras
+- [ ] `dorky[encrypt]` — includes `cryptography` for encryption
+- [ ] Support encrypted upload/download with client-side keys
+
+## P3: Advanced Features
+
+### Batch Operations
+- [ ] Implement `upload_batch(files: List[str])` for multiple files
+- [ ] Implement `download_batch(ids: List[str], dest_dir: str)`
+- [ ] Support concurrent batch operations with asyncio
+
+### Streaming & Chunking
+- [ ] Expose raw streaming download (generator/iterator)
+- [ ] Support range requests for partial downloads
+- [ ] Add chunk size configuration
+
+---
+
+**Status**: Core client exists, async/CLI/publishing pending
+**Next**: Add async client, CLI commands, publish to TestPyPI

package/todo/05-metadata-versioning.md

@@ -0,0 +1,116 @@
+# Metadata & Versioning
+
+Artifact versioning, conflict detection, and complete change history.
+
+## P1: Enhanced Metadata
+
+### Standard Metadata Fields
+- [ ] Implement `last_updated` timestamp (UTC ISO 8601)
+- [ ] Implement `content_hash` (SHA256 hex string)
+- [ ] Implement `uploaded_by` (user/email from config)
+- [ ] Implement `file_size` (bytes)
+- [ ] Implement `compressed_size` (bytes, if compressed)
+- [ ] Implement `compression_type` (gzip, lz4, brotli, or null)
+
+### Custom Metadata
+- [ ] Support user-defined key-value metadata
+- [ ] Store custom metadata in artifact metadata section
+- [ ] Merge custom + standard metadata in response
+- [ ] Document metadata schema in OpenAPI
+
+### Metadata Update
+- [ ] Implement `PATCH /artifacts/{id}/metadata` endpoint
+- [ ] Allow updating custom metadata without re-uploading file
+- [ ] Store update timestamp and updater info
+- [ ] Maintain update history
+
+## P1: Artifact Versioning
+
+### Version Storage
+- [ ] Store multiple versions of same artifact
+- [ ] Use version ID (integer or UUID) for each version
+- [ ] Track version creation timestamp and uploader
+- [ ] Keep version metadata (size, hash, compression)
+
+### Version Endpoints
+- [ ] Implement `GET /artifacts/{id}/versions` — list all versions
+- [ ] Implement `GET /artifacts/{id}?version={v}` — get specific version
+- [ ] Implement `GET /artifacts/{id}/versions/{v}/metadata` — version metadata
+- [ ] Implement `DELETE /artifacts/{id}/versions/{v}` — delete version (keep data)
+
+### CLI Commands
+- [ ] Add `dorky history <artifact_id>` — show version history
+- [ ] Add `dorky show <artifact_id> --version=<v>` — view specific version
+- [ ] Add `dorky checkout <artifact_id> --version=<v>` — download version
+- [ ] Add `dorky info <artifact_id>` — show current + version info
+
+## P1: Conflict Detection & Resolution
+
+### Race Condition Detection
+- [ ] Detect concurrent uploads of same file
+- [ ] Implement optimistic locking using ETags
+- [ ] Return HTTP 409 Conflict if updated concurrently
+- [ ] Include conflict details in error response
+
+### Conflict Resolution Strategies
+- [ ] `--force` flag to overwrite
+- [ ] `--merge` strategy for mergeable files
+- [ ] `--abort` flag to fail on conflicts
+- [ ] Show conflict info (who, when, hash diff)
+
+### Merge Conflicts for Structured Data
+- [ ] Detect schema mismatches
+- [ ] Support JSON merge (deep merge)
+- [ ] Support CSV/Parquet schema evolution
+- [ ] Warn on incompatible changes
+
+## P2: Time Travel & Diffs
+
+### Historical Queries
+- [ ] `dorky log <artifact_id>` — show commit-like history
+- [ ] `dorky diff <id> --version=<v1> --version=<v2>` — compare versions
+- [ ] `dorky blame <artifact_id>` — show line-by-line history (for text)
+
+### Diff Implementation
+- [ ] For text files: line-by-line diff (unified format)
+- [ ] For JSON: semantic diff (structure aware)
+- [ ] For Parquet: schema + row count diff
+- [ ] For binary: byte-level diff or hash comparison
+
+### Rollback
+- [ ] `dorky rollback <artifact_id> --version=<v>` — restore version
+- [ ] Creates new version (doesn't delete old)
+- [ ] Mark as rollback in metadata/history
+
+## P2: Retention & Cleanup
+
+### Version Retention Policies
+- [ ] Keep last N versions of each artifact
+- [ ] Auto-delete old versions after TTL (configurable)
+- [ ] Support explicit version deletion
+- [ ] Warn before deleting versions
+
+### Garbage Collection
+- [ ] Implement cleanup of orphaned blob data
+- [ ] Run GC on schedule (configurable)
+- [ ] Track storage usage and report
+- [ ] Add `dorky storage stats` command
+
+## P3: Audit & Compliance
+
+### Audit Log
+- [ ] Log all mutations (upload, update, delete)
+- [ ] Store user, timestamp, action, artifact ID
+- [ ] Implement `GET /audit-log` endpoint (admin only)
+- [ ] Support audit log export
+
+### Compliance Features
+- [ ] Lock versions for immutability (compliance)
+- [ ] Add `--immutable` flag to upload
+- [ ] Prevent deletion of locked versions
+- [ ] Generate compliance reports
+
+---
+
+**Status**: Standard metadata planned, versioning in design, conflict detection pending
+**Next**: Implement standard metadata fields, version storage, conflict detection

package/todo/06-performance-concurrency.md

@@ -0,0 +1,130 @@
+# Performance & Concurrency
+
+Parallel uploads, streaming, incremental updates, and caching for high performance.
+
+## P1: Parallel Upload Support
+
+### Worker Threads
+- [ ] Use Node `worker_threads` for compression
+- [ ] Spawn `2 * num_cores - 1` worker threads
+- [ ] Queue files/chunks for compression
+- [ ] Collect compressed chunks and upload
+
+### Chunked Upload (Multipart)
+- [ ] Split large files into chunks (e.g., 5MB each)
+- [ ] Upload chunks in parallel
+- [ ] Use cloud provider multipart APIs (S3 MultipartUpload, GCS resumable)
+- [ ] Implement retry logic per chunk
+- [ ] Reconstruct file on server side
+
+### Progress Reporting
+- [ ] Show upload progress (bytes/total, percentage)
+- [ ] Display ETA based on current speed
+- [ ] Update progress in real-time
+- [ ] Support `--no-progress` flag
+
+## P1: Streaming & Chunking Download
+
+### Streaming Download
+- [ ] Stream response directly to file (no buffering)
+- [ ] Support range requests (`Range` header)
+- [ ] Allow partial download resume on network error
+- [ ] Show download progress with ETA
+
+### Chunk-based Download
+- [ ] For very large files, download in parallel chunks
+- [ ] Use range requests to fetch chunks concurrently
+- [ ] Reconstruct file from chunks in correct order
+- [ ] Verify integrity per chunk
+
+### Resume on Failure
+- [ ] Store partially downloaded files
+- [ ] Resume from last byte on retry
+- [ ] Clean up stale partial downloads
+
+## P2: Incremental / Delta Updates
+
+### File Change Detection
+- [ ] Compute file hash on upload and download
+- [ ] Detect changes by comparing hashes
+- [ ] Only upload changed files in batch operations
+
+### Rsync-like Delta Sync
+- [ ] Implement rolling hash algorithm (rsync)
+- [ ] Compute delta between local and remote versions
+- [ ] Upload only differences (not whole file)
+- [ ] Reconstruct file on server side
+- [ ] Add `--incremental` flag to upload
+
+### Partial File Updates
+- [ ] Support append-only updates (logs)
+- [ ] Support range writes for binary files
+- [ ] Track which ranges have been written
+- [ ] Merge multiple partial writes
+
+## P2: Caching
+
+### Local Artifact Cache
+- [ ] Cache downloaded artifacts in `~/.dorky/cache/`
+- [ ] Use artifact ID as cache key
+- [ ] Check cache before downloading
+- [ ] Validate cache using content_hash from metadata
+
+### Cache Invalidation
+- [ ] Invalidate cache if remote file changes (hash mismatch)
+- [ ] Support TTL-based expiration (configurable)
+- [ ] Manual cache clear: `dorky cache clear`
+- [ ] Cache statistics: `dorky cache stats`
+
+### Metadata Caching
+- [ ] Cache artifact metadata to avoid repeated GET requests
+- [ ] Use `Last-Modified` / `ETag` headers for validation
+- [ ] Respect `Cache-Control` headers from server
+
+## P2: Connection Pooling & Optimization
+
+### HTTP Session Reuse
+- [ ] Reuse HTTP connections across operations
+- [ ] Implement connection pool with size limit
+- [ ] Keep-alive to reduce connection overhead
+- [ ] Graceful shutdown of idle connections
+
+### Request Batching
+- [ ] Batch metadata queries in single request
+- [ ] Use GraphQL-like query language for complex operations
+- [ ] Reduce round-trips for multi-operation workflows
+
+## P3: Advanced Performance
+
+### Bandwidth Limiting
+- [ ] Add `--bandwidth-limit=<bytes/sec>` flag
+- [ ] Throttle uploads to prevent network saturation
+- [ ] Support upload/download speed limits
+
+### Priority Queue
+- [ ] Prioritize recent/important files in batch uploads
+- [ ] Support `--priority=<high|normal|low>` flag
+- [ ] Pre-warm cache with priority files
+
+### Load Balancing
+- [ ] Support multiple server replicas
+- [ ] Distribute uploads across replicas
+- [ ] Health check and failover
+
+## P3: Monitoring & Metrics
+
+### Performance Metrics
+- [ ] Track upload/download speed (bytes/sec)
+- [ ] Record compression ratio and time
+- [ ] Monitor worker thread utilization
+- [ ] Add `--metrics` flag to capture timing data
+
+### Diagnostics
+- [ ] Log slow operations (threshold configurable)
+- [ ] Report bottlenecks (network, CPU, disk)
+- [ ] Add `dorky profile <command>` for profiling
+
+---
+
+**Status**: Streaming partially exists, parallel/chunking/caching pending
+**Next**: Implement multipart/chunked upload, local caching, worker threads for compression