pydorky 2.1.8

package/todo/07-security-encryption.md

@@ -0,0 +1,114 @@
+# Security & Encryption
+
+Client-side encryption, key management, and secure authentication.
+
+## P1: Client-Side Encryption at Rest
+
+### AES-256 Encryption
+- [ ] Implement AES-256-GCM encryption
+- [ ] Use `crypto` module in Node.js
+- [ ] Support encryption in Python with `cryptography`
+- [ ] Encrypt before upload, decrypt after download
+
+### Key Management (User-Provided)
+- [ ] Allow user to provide encryption key (password or raw key)
+- [ ] Derive key from password using PBKDF2 (cost factor 100k+)
+- [ ] Store key securely (not in config, in key ring)
+- [ ] Add `--encrypt-key <path>` flag to CLI
+
+### Encryption Metadata
+- [ ] Store encryption algorithm and parameters in metadata
+- [ ] Store salt for key derivation
+- [ ] Store IV for GCM in metadata (safe to store)
+- [ ] Prevent accidental download without decryption
+
+### CLI Support
+- [ ] Add `--encrypt` flag to upload command
+- [ ] Add `--decrypt` flag to download command
+- [ ] Prompt for password if not provided
+- [ ] Support keychain integration (macOS Keychain, Windows Credential Manager)
+
+## P1: Selective File Encryption
+
+### Pattern-Based Encryption
+- [ ] Create `.dorkyencrypt` file (like `.gitignore`)
+- [ ] Support patterns: `*.key`, `secrets/*`, `*.pem`, etc.
+- [ ] Encrypt matching files automatically on upload
+- [ ] Decrypt on download transparently
+
+### Obfuscation Mode (Optional)
+- [ ] Implement faster obfuscation (XOR, Base64) for non-sensitive data
+- [ ] Use `--obfuscate` flag (less secure, faster)
+- [ ] Document security limitations clearly
+
+## P2: KMS Integration
+
+### AWS KMS
+- [ ] Use AWS KMS for key management
+- [ ] Generate data keys from KMS master key
+- [ ] Encrypt/decrypt using AWS SDK
+- [ ] Support assume role for cross-account KMS
+- [ ] Add `--kms-key-id=<arn>` flag
+
+### Google Cloud KMS
+- [ ] Use Google Cloud KMS
+- [ ] Similar flow to AWS KMS
+- [ ] Support project ID and key ring configuration
+- [ ] Add `--gcp-kms-key=<resource-name>` flag
+
+### Azure Key Vault
+- [ ] Use Azure Key Vault for key management
+- [ ] Support managed identity authentication
+- [ ] Add `--azure-key-vault-url=<url>` flag
+
+### KMS Metadata
+- [ ] Store which KMS was used in metadata
+- [ ] Store wrapped key in metadata
+- [ ] Support key rotation with rekeying
+
+## P2: Authentication & Authorization
+
+### API Key Support
+- [ ] Generate API keys for programmatic access
+- [ ] Hash keys server-side (not plaintext)
+- [ ] Support key expiration and rotation
+- [ ] Add `dorky auth generate-key` command
+
+### JWT Tokens
+- [ ] Support JWT tokens for authentication
+- [ ] Issue tokens with short TTL (1 hour)
+- [ ] Refresh tokens for longer sessions
+- [ ] Add claims for user ID, project, scopes
+
+### Role-Based Access Control (RBAC)
+- [ ] Define roles: admin, writer, reader
+- [ ] Enforce at API level (who can upload, delete, etc.)
+- [ ] Support resource-level permissions (bucket, artifact, folder)
+- [ ] Audit role changes
+
+## P3: Advanced Security
+
+### Public Key Cryptography
+- [ ] Support RSA or ECDH for key agreement
+- [ ] Allow sharing encrypted files with specific users
+- [ ] Implement key exchange protocol
+
+### Multi-User Encryption
+- [ ] Encrypt file once, share with multiple users
+- [ ] Each user has encrypted copy of data key
+- [ ] Support adding/removing users from access list
+
+### Compliance Features
+- [ ] Support FIPS 140-2 mode (restricted algorithms)
+- [ ] Audit all encryption operations
+- [ ] Log key access and rotation
+
+### Secret Rotation
+- [ ] Auto-rotate encryption keys on schedule
+- [ ] Re-encrypt existing files with new key
+- [ ] Track which key was used for each file
+
+---
+
+**Status**: Encryption design ready, implementation pending
+**Next**: Implement AES-256-GCM, user key management, selective encryption patterns

package/todo/08-developer-experience.md

@@ -0,0 +1,175 @@
+# Developer Experience
+
+Documentation, configuration, testing, and error handling.
+
+## P0: Documentation
+
+### README Updates
+- [ ] Update main README with current architecture (HTTP service + thin clients)
+- [ ] Add architecture diagram (ASCII or SVG)
+- [ ] Add quick start guide for Node and Python
+- [ ] Add cloud provider setup instructions (S3, GCS, Azure)
+- [ ] Add troubleshooting section
+
+### API Documentation
+- [ ] Generate API docs from OpenAPI spec (Swagger UI)
+- [ ] Host Swagger UI at `/api/docs`
+- [ ] Add cURL examples for each endpoint
+- [ ] Document error codes and response formats
+
+### CLI Documentation
+- [ ] Document all CLI commands and flags
+- [ ] Add `dorky --help` output
+- [ ] Create `docs/cli-reference.md`
+- [ ] Add usage examples for common workflows
+
+### Guides
+- [ ] Create `docs/guides/` directory
+- [ ] Add guide: "Using S3 with Dorky"
+- [ ] Add guide: "Data compression strategies"
+- [ ] Add guide: "Python integration"
+- [ ] Add guide: "GitHub Actions integration"
+
+### Architecture Document
+- [ ] Document system design and decisions
+- [ ] Explain HTTP service role
+- [ ] Explain client library architecture
+- [ ] Diagram: local CLI → HTTP service → cloud storage
+
+## P1: Configuration Management
+
+### Project Config File (`.dorkyrc`)
+- [ ] TOML or YAML format
+- [ ] Store bucket URL, credentials, compression settings
+- [ ] Support environment variable overrides
+- [ ] Example `.dorkyrc.example` in repo
+
+### User Configuration
+- [ ] Support `~/.dorky/config` for user defaults
+- [ ] Support `~/.dorky/credentials` for secrets
+- [ ] Add `dorky config set <key> <value>` command
+- [ ] Add `dorky config get <key>` command
+- [ ] Add `dorky config list` command
+
+### Environment Variables
+- [ ] `DORKY_URL` — service base URL
+- [ ] `DORKY_BUCKET_URL` — cloud bucket URL
+- [ ] `DORKY_ACCESS_KEY` — cloud credentials
+- [ ] `DORKY_SECRET_KEY` — cloud credentials
+- [ ] `DORKY_COMPRESS` — default compression type
+- [ ] Document all variables in README
+
+### Init Command
+- [ ] `dorky init` — interactive setup wizard
+- [ ] Detect cloud provider from credentials
+- [ ] Create `.dorkyrc` with initial config
+- [ ] Test bucket connection
+- [ ] Offer to add `.dorkyrc` to `.gitignore`
+
+## P1: Testing & CI
+
+### Unit Tests
+- [ ] Test upload/download logic
+- [ ] Test compression/decompression
+- [ ] Test metadata operations
+- [ ] Test error handling and retries
+- [ ] Target 80%+ code coverage
+
+### Integration Tests
+- [ ] Test against local HTTP server
+- [ ] Test all storage backends (S3, GCS, local)
+- [ ] Test CLI commands end-to-end
+- [ ] Test compression options
+- [ ] Test various file sizes (small, large, >1GB)
+
+### E2E Tests
+- [ ] Test full workflow: init → stage → commit → push → pull
+- [ ] Test multi-file operations
+- [ ] Test concurrent uploads
+- [ ] Test error recovery (network interruption, etc.)
+
+### Test Infrastructure
+- [ ] Use Jest for Node tests
+- [ ] Use pytest for Python tests
+- [ ] Use Docker for consistent test environment
+- [ ] Add CI workflows (GitHub Actions)
+- [ ] Run tests on Node 16 + 20
+- [ ] Run Python tests on 3.8, 3.9, 3.10, 3.11
+
+### CI Workflows
+- [ ] Lint (ESLint for Node, pylint/black for Python)
+- [ ] Type check (TypeScript, mypy)
+- [ ] Unit tests on every PR
+- [ ] Integration tests on main branch
+- [ ] E2E tests before release
+
+## P2: Error Handling & Diagnostics
+
+### Error Messages
+- [ ] Clear, actionable error messages
+- [ ] Suggest fixes when possible
+- [ ] Include error codes (e.g., `E001: Invalid bucket URL`)
+- [ ] Don't expose internal stack traces to users
+
+### Verbose Mode
+- [ ] `--verbose` / `-v` flag for detailed output
+- [ ] Log all HTTP requests/responses
+- [ ] Show timing information
+- [ ] Include diagnostics (network, disk, permissions)
+
+### Debug Mode
+- [ ] `--debug` flag for maximum verbosity
+- [ ] Dump request/response bodies
+- [ ] Show internal state
+- [ ] Enable profiling
+
+### Logging
+- [ ] Structured logging (JSON format option)
+- [ ] Log to file: `~/.dorky/logs/`
+- [ ] Log rotation (keep last 10 files)
+- [ ] Log level configuration (info, debug, trace)
+
+### Diagnostics Command
+- [ ] `dorky diagnose` — check system health
+- [ ] Verify bucket connectivity
+- [ ] Check permissions
+- [ ] Verify file system space
+- [ ] Test network speed
+- [ ] Report diagnostics bundle for issue reporting
+
+## P2: IDE Integration
+
+### VS Code Extension
+- [ ] Create VS Code extension for Dorky
+- [ ] File explorer integration
+- [ ] Show sync status icon
+- [ ] Quick actions: upload file, download artifact
+- [ ] Status bar: connection status, last sync time
+
+### Editor Integration
+- [ ] Context menu: "Upload to Dorky", "Sync with Dorky"
+- [ ] Sync status in file decorations (✓, !, ↑, ↓)
+- [ ] Preview artifact metadata
+- [ ] Show file history (versions)
+
+## P3: Advanced Developer Tools
+
+### GitHub Integration
+- [ ] GitHub Action: Upload artifacts from CI/CD
+- [ ] GitHub Action: Download artifacts from Dorky
+- [ ] Example workflows in `docs/examples/`
+
+### SDK & Libraries
+- [ ] TypeScript SDK with full type definitions
+- [ ] Python SDK with type hints (mypy compatible)
+- [ ] Generate SDKs from OpenAPI spec
+
+### API Playground
+- [ ] Interactive API explorer (Swagger UI)
+- [ ] Example code generation (cURL, Python, Node, etc.)
+- [ ] Request/response history
+
+---
+
+**Status**: Documentation started, config/testing/diagnostics pending
+**Next**: Update README with architecture, create `.dorkyrc` support, add unit/integration tests

package/todo/README.md

@@ -0,0 +1,37 @@
+# Pydorky Feature Implementation Roadmap
+
+This folder tracks features to implement, organized by priority and category.
+Each file represents a feature area with actionable tasks.
+
+## Priority Legend
+- **P0**: Critical / blocking other work
+- **P1**: High priority / near-term
+- **P2**: Medium priority / planned
+- **P3**: Nice-to-have / backlog
+
+## Status Legend
+- [ ] Not started
+- [~] In progress
+- [x] Completed
+
+## Feature Areas
+1. **Core Infrastructure** — HTTP service, CLI foundation, Node version alignment
+2. **Storage Providers** — Cloud integrations (S3, GCS, Azure), BYOB support
+3. **Compression & Formats** — gzip, lz4, brotli, Parquet conversions
+4. **Python Client** — PyPI package, async/sync clients, CLI parity
+5. **Metadata & Versioning** — Artifact versioning, conflict detection, time travel
+6. **Performance & Concurrency** — Parallel uploads, streaming, incremental updates
+7. **Security & Encryption** — Client-side encryption, KMS integration
+8. **Developer Experience** — Docs, config, testing, error handling
+
+## Key Milestones
+- [ ] Phase 1: Core HTTP service + Node CLI (P0)
+- [ ] Phase 2: Python client + storage providers (P0-P1)
+- [ ] Phase 3: Compression + metadata versioning (P1-P2)
+- [ ] Phase 4: Performance optimization (P2)
+- [ ] Phase 5: Security & encryption (P2-P3)
+
+---
+
+**Last Updated**: 2025-12-28
+**Status**: Roadmap created, ready for implementation

package/web-app/README.md

@@ -0,0 +1,70 @@
+# Getting Started with Create React App
+
+This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
+
+## Available Scripts
+
+In the project directory, you can run:
+
+### `npm start`
+
+Runs the app in the development mode.\
+Open [http://localhost:3000](http://localhost:3000) to view it in your browser.
+
+The page will reload when you make changes.\
+You may also see any lint errors in the console.
+
+### `npm test`
+
+Launches the test runner in the interactive watch mode.\
+See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
+
+### `npm run build`
+
+Builds the app for production to the `build` folder.\
+It correctly bundles React in production mode and optimizes the build for the best performance.
+
+The build is minified and the filenames include the hashes.\
+Your app is ready to be deployed!
+
+See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
+
+### `npm run eject`
+
+**Note: this is a one-way operation. Once you `eject`, you can't go back!**
+
+If you aren't satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
+
+Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you're on your own.
+
+You don't have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn't feel obligated to use this feature. However we understand that this tool wouldn't be useful if you couldn't customize it when you are ready for it.
+
+## Learn More
+
+You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
+
+To learn React, check out the [React documentation](https://reactjs.org/).
+
+### Code Splitting
+
+This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting)
+
+### Analyzing the Bundle Size
+
+This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size)
+
+### Making a Progressive Web App
+
+This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app)
+
+### Advanced Configuration
+
+This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration)
+
+### Deployment
+
+This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment)
+
+### `npm run build` fails to minify
+
+This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify)