@mc1global/opencode-jarvis 0.13.0 → 0.13.1

package/.opencode/skills/env-manager/reference.md

@@ -0,0 +1,717 @@
+# ENV Manager - Complete Technical Reference
+
+## Masking Algorithm
+
+### Sensitive Key Detection
+
+All keys are checked against these patterns (case-insensitive):
+
+```python
+SENSITIVE_PATTERNS = {
+    'TOKEN': r'.*TOKEN.*',        # JWT, API tokens, etc.
+    'KEY': r'.*KEY.*',             # API keys, encryption keys
+    'SECRET': r'.*SECRET.*',       # Shared secrets
+    'PASSWORD': r'.*PASSWORD.*',   # Database/service passwords
+    'PASS': r'.*PASS.*',           # Short password refs
+    'API': r'.*API.*',             # API credentials
+    'CREDENTIALS': r'.*CREDENTIALS.*',  # Credential bundles
+    'AUTH': r'.*AUTH.*',           # Authentication data
+}
+```
+
+### Pattern Matching Examples
+
+| Key Name | Pattern | Detected |
+|----------|---------|----------|
+| `TELEGRAM_BOT_TOKEN` | `*TOKEN*` | ✅ Yes |
+| `api_key_prod` | `*KEY*` | ✅ Yes |
+| `db_password` | `*PASSWORD*` | ✅ Yes |
+| `JWT_SECRET` | `*SECRET*` | ✅ Yes |
+| `DEBUG_MODE` | None | ❌ No |
+| `LOG_LEVEL` | None | ❌ No |
+| `admin_pass` | `*PASS*` | ✅ Yes |
+| `AWS_ACCESS_KEY_ID` | `*KEY*` | ✅ Yes |
+
+### Masking Strategies
+
+#### Strategy 1: Tokens/Keys (Long Values)
+
+**Detection**: Value length >= 16 characters and key matches TOKEN/KEY pattern
+
+**Algorithm**:
+```python
+def mask_token(value: str) -> str:
+    if len(value) <= 8:
+        return "***...***"
+
+    first_4 = value[:4]
+    last_4 = value[-4:]
+    return f"{first_4}...{last_4}"
+```
+
+**Examples**:
+```
+7890123456789ABCDEFGHIjklmnopqrstu_VWXYZabcDef → 7890...cDef
+sk_live_1234567890abcdef1234567890abcdef → sk_l...cdef
+```
+
+#### Strategy 2: Passwords/Secrets
+
+**Detection**: Key matches PASSWORD, PASS, SECRET patterns
+
+**Algorithm**:
+```python
+def mask_password(value: str) -> str:
+    return "***...***"
+```
+
+**Examples**:
+```
+MySecurePassword123! → ***...***
+secret_key_value → ***...***
+P@ssw0rd!Complex → ***...***
+```
+
+#### Strategy 3: IDs/Numbers
+
+**Detection**: Value is numeric or key matches ID pattern
+
+**Algorithm**:
+```python
+def mask_id(value: str) -> str:
+    if len(value) <= 6:
+        return "***...***"
+
+    first_3 = value[:3]
+    last_3 = value[-3:]
+    return f"{first_3}...{last_3}"
+```
+
+**Examples**:
+```
+123456789 → 123...789
+987654321 → 987...321
+12345 → ***...***
+```
+
+#### Strategy 4: URLs/Connection Strings
+
+**Detection**: Value contains `://` or key matches DATABASE pattern
+
+**Algorithm**:
+```python
+def mask_url(value: str) -> str:
+    # Preserve protocol and structure
+    # Mask credentials and details
+    if '://' in value:
+        protocol, rest = value.split('://', 1)
+
+        if '@' in rest:
+            creds, host_part = rest.rsplit('@', 1)
+            masked_creds = mask_credentials(creds)
+            return f"{protocol}://{masked_creds}@{mask_host(host_part)}"
+
+        return f"{protocol}://{mask_host(rest)}"
+
+    return "***...***"
+```
+
+**Examples**:
+```
+postgresql://user:password@localhost:5432/database
+→ post***://u***:p***@loc***:5432/d***
+
+mongodb://admin:secret123@server.com:27017/db
+→ mongo***://a***:s***@ser***:27017/d***
+
+https://api.example.com:8080/secure/endpoint?token=xyz
+→ https***://a***:8080/s***?t***=xyz
+```
+
+#### Strategy 5: Emails
+
+**Detection**: Value contains `@` symbol
+
+**Algorithm**:
+```python
+def mask_email(value: str) -> str:
+    if '@' not in value:
+        return value
+
+    local, domain = value.rsplit('@', 1)
+    if len(local) <= 2:
+        return f"***@{domain}"
+
+    first = local[0]
+    last = local[-1]
+    masked_local = f"{first}***{last}"
+
+    return f"{masked_local}@{domain}"
+```
+
+**Examples**:
+```
+admin@company.com → a***@company.com
+service.account@example.org → s***t@example.org
+a@b.com → ***@b.com
+```
+
+### Masking Algorithm Pseudocode
+
+```python
+def mask_value(key: str, value: str) -> str:
+    """
+    Mask sensitive values based on key pattern and value characteristics.
+
+    Precedence:
+    1. Check if key is sensitive
+    2. Select masking strategy based on value type/length
+    3. Apply masking strategy
+    4. Return masked value
+    """
+
+    if not is_sensitive(key):
+        return value
+
+    # Strategy selection
+    if is_email(value):
+        return mask_email(value)
+
+    if is_url(value):
+        return mask_url(value)
+
+    if is_password_key(key):
+        return mask_password(value)  # Always hide
+
+    if is_numeric(value):
+        return mask_id(value)
+
+    if is_token_or_key(key) and len(value) >= 16:
+        return mask_token(value)
+
+    # Default for other sensitive keys
+    return "***...***"
+```
+
+## Sensitive Pattern Matching Rules
+
+### Priority Order
+
+1. **Explicit matches** (exact word): `PASSWORD`, `SECRET`, `TOKEN`
+2. **Substring matches**: Any key containing sensitive word
+3. **Case-insensitive**: `password`, `PASSWORD`, `Password` all match
+4. **Partial word matches**: `PASS` matches `PASSWORD`, `PASSPORT`, `PASSKEY`
+
+### Pattern Expansion
+
+```python
+# These all match PASSWORD pattern:
+ADMIN_PASSWORD
+db_password
+PASSWORD_HASH
+PASSWORD_SALT
+old_password
+test_password
+user_password
+api_password_prod
+```
+
+### Non-Matching Examples
+
+```python
+# These don't match (not sensitive):
+PASSWORD_RESET_URL    # URL is not secret
+PASSWORD_POLICY       # Policy is not secret
+PASSWORD_MIN_LENGTH   # Configuration, not secret
+PASSWORD_EXPIRY_DAYS  # Configuration, not secret
+DISPLAY_PASSWORD_FIELD  # UI flag, not secret
+```
+
+## File Format Specification
+
+### Valid .env Syntax
+
+```env
+# Single comment
+## Group header comment
+KEY=value
+KEY="value with spaces"
+KEY='single quoted value'
+KEY=value_with_special_chars-123
+
+# Multi-line not supported (continuation lines not preserved)
+# Empty lines preserved
+```
+
+### Key Name Rules
+
+- **Allowed characters**: `A-Z`, `a-z`, `0-9`, `_`
+- **Starting character**: Must be letter or underscore
+- **Length**: No practical limit
+- **Case-sensitive**: `DEBUG` and `debug` are different keys
+
+### Value Rules
+
+- **No escaping required** unless value contains newlines/quotes
+- **Spaces allowed**: No quoting needed for trailing/leading spaces in values
+- **Special characters**: All allowed
+- **Unicode**: Full UTF-8 support
+- **Quotes**: Optional for most values, required only if needed
+
+### Valid Examples
+
+```env
+# Simple values
+DEBUG=true
+PORT=3000
+NAME=MyApp
+
+# Values with spaces
+DESCRIPTION=This is a long description
+TITLE="My Application Title"
+
+# URLs
+DATABASE_URL=postgresql://localhost:5432/mydb
+API_ENDPOINT=https://api.example.com/v1
+
+# Complex values
+PARAMS=key1=value1&key2=value2
+FLAGS=-v -d --enable-cache
+
+# Unicode
+APP_NAME=Jarvis 🤖 Trading
+EMOJI=😀🎉🚀
+```
+
+### Invalid Examples
+
+```env
+# Missing =
+KEY value
+
+# Only key
+KEY
+
+# Duplicate keys (error - use latest)
+DEBUG=true
+DEBUG=false
+
+# Invalid key names
+123KEY=value    # Can't start with number
+KEY-NAME=value  # Dashes not allowed
+KEY NAME=value  # Spaces not allowed
+```
+
+## Backup Specification
+
+### Automatic Backup Format
+
+- **Filename**: `{original_name}.backup`
+- **Location**: Same directory as original
+- **Content**: Complete copy of original .env
+- **Overwrite**: Latest backup overwrites previous (only 1 backup kept)
+
+### Manual Backup Format
+
+- **Filename**: User-specified via `--output`
+- **Timestamp**: Optional (e.g., `.env.2025-11-16.14-30-45`)
+- **Multiple**: Can keep multiple backups with different names
+
+### Backup Recovery
+
+```python
+def restore_backup(original_path: Path, backup_path: Path) -> None:
+    """Restore from backup by copying."""
+    shutil.copy2(backup_path, original_path)
+```
+
+## Validation Checks
+
+### Syntax Validation
+
+1. **Line parsing**: Each line is KEY=VALUE format
+2. **Key format**: Valid identifier (A-Z, a-z, 0-9, _)
+3. **Duplicate keys**: Error if same key appears multiple times
+4. **Encoding**: UTF-8 valid encoding
+5. **Line endings**: Consistent (CRLF or LF)
+
+### Logical Validation
+
+1. **Empty values allowed**: `KEY=` is valid
+2. **Whitespace handling**: Trimmed appropriately
+3. **Comment preservation**: Comments don't interfere with validation
+4. **Quoted values**: Can contain = signs safely
+
+### Return Format
+
+```python
+{
+    "valid": bool,
+    "variables": int,
+    "sensitive_keys": int,
+    "issues": [
+        {
+            "line": int,
+            "type": str,  # "syntax_error", "duplicate", "encoding"
+            "description": str,
+            "context": str,
+        }
+    ],
+    "warnings": [
+        {
+            "type": str,  # "line_length", "unusual_format"
+            "description": str,
+        }
+    ]
+}
+```
+
+## Command Implementation Details
+
+### get
+
+**Implementation**:
+1. Parse .env file
+2. Find KEY (case-sensitive)
+3. Extract VALUE
+4. Check if KEY is sensitive
+5. Apply masking if sensitive
+6. Output formatted result
+
+**Error cases**:
+- KEY not found → List similar keys
+- File not found → Create suggestion
+- Permission denied → Suggest chmod
+
+### set
+
+**Implementation**:
+1. Create backup (.env.backup)
+2. Parse existing .env (if exists)
+3. Find KEY (update) or append (new)
+4. Write updated file
+5. Validate syntax
+6. Return success with masked value
+
+**Error cases**:
+- File locked → Retry or suggest
+- Permission denied → Suggest chmod
+- Invalid value → Syntax check
+
+### add
+
+**Implementation**:
+1. Check if KEY exists (error if yes)
+2. Parse .env file
+3. Append KEY=VALUE
+4. Write file
+5. Validate syntax
+6. Return success
+
+**Error cases**:
+- KEY exists → Suggest use set
+- Syntax error in new value → Show error
+
+### remove
+
+**Implementation**:
+1. Parse .env file
+2. Find KEY
+3. Create backup (.env.backup)
+4. Remove line
+5. Write file
+6. Validate syntax
+7. Return success
+
+**Error cases**:
+- KEY not found → Error with suggestions
+
+### list
+
+**Implementation**:
+1. Parse .env file
+2. For each KEY=VALUE:
+   - Check if KEY is sensitive
+   - Apply masking if needed
+   - Format for output
+3. Preserve comment structure
+4. Display grouped by comments
+5. Return formatted list
+
+**Options**:
+- `--group`: Filter by group header
+- `--sensitive`: Show only sensitive keys
+- `--sort`: Sort by name or original order
+
+### validate
+
+**Implementation**:
+1. Parse file line by line
+2. Check syntax each line
+3. Detect duplicates
+4. Check encoding
+5. Generate report
+6. Optional: Auto-fix with `--fix`
+
+**Return**: Detailed report with issues/warnings
+
+### add-comment
+
+**Implementation**:
+1. Parse .env file
+2. Find insertion point (--before or --at-end)
+3. Insert comment line: `# TEXT`
+4. Write file
+5. Return success
+
+**Format**: Single `#` for comments, `##` for group headers
+
+### add-group
+
+**Implementation**:
+1. Parse .env file
+2. Insert group header: `## GROUP_NAME`
+3. Optional: Indent following variables
+4. Write file
+5. Return success
+
+**Format**: `## Group Name` (double hash for headers)
+
+### backup
+
+**Implementation**:
+1. Verify source file exists
+2. Count variables
+3. Copy file to backup location
+4. Set backup as read-only (chmod 400)
+5. Return success with variable count
+
+**Security**: Backup is read-only to prevent accidental modification
+
+## Error Messages
+
+### Error Codes and Messages
+
+| Code | Message | Cause | Solution |
+|------|---------|-------|----------|
+| ERR_NOT_FOUND | Key '{key}' not found | Key doesn't exist | Check key name spelling |
+| ERR_EXISTS | Key '{key}' already exists | Key duplicate in add | Use set to update |
+| ERR_SYNTAX | Invalid .env syntax on line {line} | Malformed line | Use validate to check |
+| ERR_PERMISSION | Permission denied | File permissions issue | chmod 600 .env |
+| ERR_ENCODING | Invalid UTF-8 encoding | File encoding issue | Convert to UTF-8 |
+| ERR_DUPLICATE | Duplicate key '{key}' on lines {l1}, {l2} | Multiple same keys | Remove duplicate |
+
+## Integration Points
+
+### System Integration
+
+```bash
+# Use in shell scripts
+source <(python scripts/managers/env_manager.py export --file=.env)
+
+# Use in Python
+import subprocess
+result = subprocess.run(
+    ['python', 'scripts/managers/env_manager.py', 'get', 'KEY', '--file=.env'],
+    capture_output=True, text=True
+)
+value = result.stdout.strip()
+```
+
+### Framework Integration
+
+1. **file-manager**: Get correct .env path
+2. **git-repo-manager**: Backup .env before commits
+3. **context-memory**: Log .env changes
+4. **workspace-manager**: Manage per-workspace .env files
+
+## Performance Characteristics
+
+### Time Complexity
+
+| Operation | Time | Notes |
+|-----------|------|-------|
+| get | O(n) | Linear scan for key |
+| set | O(n) | Parse + write |
+| add | O(n) | Parse + append |
+| remove | O(n) | Parse + rewrite |
+| list | O(n) | Full scan with masking |
+| validate | O(n) | Line-by-line check |
+
+### Space Complexity
+
+| Operation | Space | Notes |
+|-----------|-------|-------|
+| get | O(1) | Single value returned |
+| set | O(n) | Full file in memory |
+| add | O(n) | Full file in memory |
+| remove | O(n) | Full file in memory |
+| list | O(n) | Full file + masking map |
+
+### Real-World Performance
+
+```
+Operation | File Size | Time | Memory
+----------|-----------|------|--------
+get       | 1KB       | 5ms  | 1MB
+set       | 1KB       | 15ms | 2MB
+add       | 1KB       | 15ms | 2MB
+remove    | 1KB       | 15ms | 2MB
+list      | 1KB       | 10ms | 2MB
+validate  | 1KB       | 8ms  | 2MB
+
+(File size 1KB ≈ 20-50 variables)
+```
+
+## Security Considerations
+
+### Data Exposure Prevention
+
+1. **Console Output**: All output masked unless explicitly requested
+2. **Logging**: Never log unmasked values
+3. **Shell History**: Masked values prevent credential leakage
+4. **Terminal Copy**: Copied text shows masked values
+5. **Error Messages**: Even errors show masked values
+
+### Backup Security
+
+1. **File Permissions**: Backups are read-only (600)
+2. **Location**: Same directory as original (keep with code)
+3. **Cleanup**: Old backups should be manually deleted
+4. **Never commit**: Add *.backup to .gitignore
+
+### Access Control
+
+No special access control in script:
+- Relies on OS file permissions
+- Ensure .env is 600 (user read/write only)
+- Ensure .env not world-readable
+
+## Unicode Handling
+
+### Supported Encodings
+
+- **Default**: UTF-8
+- **Fallback**: UTF-8 with error handling
+- **Output**: Terminal encoding (usually UTF-8)
+
+### Examples
+
+```env
+# All valid
+APP_NAME=Jarvis 🤖 Trading
+EMOJI=😀🎉🚀
+CHINESE=你好世界
+ARABIC=مرحبا بالعالم
+RUSSIAN=Привет мир
+```
+
+### Error Handling
+
+If UTF-8 decoding fails:
+- Attempt with 'replace' mode
+- Warn user about encoding
+- Show hex representation in error
+
+## Python Implementation Notes
+
+### Dependencies
+
+- **Only stdlib**: No external dependencies
+- **Python 3.11+**: Required for type hints
+- **Built-in modules**: `argparse`, `pathlib`, `re`, `shutil`, `json`
+
+### Type Hints
+
+```python
+from typing import Dict, Optional, Tuple, List
+
+def get_value(env_file: Path, key: str, mask: bool = True) -> Optional[str]:
+    """Get value from .env file."""
+    ...
+
+def is_sensitive(key: str) -> bool:
+    """Check if key is sensitive."""
+    ...
+
+def mask_value(value: str, key: str) -> str:
+    """Mask sensitive values."""
+    ...
+```
+
+### Error Handling
+
+All functions raise explicit exceptions:
+
+```python
+class EnvError(Exception):
+    """Base exception for env-manager."""
+    pass
+
+class KeyNotFoundError(EnvError):
+    """Key not found in .env file."""
+    pass
+
+class KeyExistsError(EnvError):
+    """Key already exists (for add operation)."""
+    pass
+
+class SyntaxError(EnvError):
+    """Invalid .env syntax."""
+    pass
+```
+
+## Thread Safety
+
+**Not thread-safe**: Operations may conflict if multiple processes access same .env file simultaneously.
+
+**Solution**: Use file locks or external coordination for concurrent access.
+
+## Future Enhancements
+
+Potential improvements (not in v1):
+
+1. **Encryption**: Encrypt sensitive values on disk
+2. **Versioning**: Track changes with git integration
+3. **Rotation**: Auto-rotate credentials on schedule
+4. **Secrets manager**: Integration with HashiCorp Vault, AWS Secrets Manager
+5. **File locking**: Prevent concurrent modifications
+6. **Compression**: Auto-compress backup files
+7. **Diff**: Show differences between .env files
+8. **Merge**: Merge multiple .env files safely
+
+## Appendix: Masking Examples
+
+### Complete Masking Examples
+
+```
+TELEGRAM_BOT_TOKEN=5890123:ABCDefGHIjklMNOpqrstu_VWXYZabcDef
+→ TELEGRAM_BOT_TOKEN=5890...cDef
+
+API_KEY=sk_live_1234567890abcdef1234567890abcdef
+→ API_KEY=sk_l...cdef
+
+DATABASE_PASSWORD=MySecurePassword123!@#
+→ DATABASE_PASSWORD=***...***
+
+AWS_ACCOUNT_ID=123456789012
+→ AWS_ACCOUNT_ID=123...012
+
+ADMIN_EMAIL=admin.user@company.com
+→ ADMIN_EMAIL=a***r@company.com
+
+DATABASE_URL=postgresql://admin:secret@prod-db.company.com:5432/mydb
+→ DATABASE_URL=post***://a***:s***@pro***:5432/m***
+
+WEBHOOK_URL=https://api.example.com:8080/webhook?token=xyz123
+→ WEBHOOK_URL=https***://a***:8080/w***?t***=xyz123
+
+JWT_SECRET=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
+→ JWT_SECRET=eyJh...XCJ9
+
+REDIS_PASSWORD=p@ssw0rd!complex$value
+→ REDIS_PASSWORD=***...***
+
+USER_ID=987654321
+→ USER_ID=987...321
+```