proagents 1.0.0

package/proagents/ai-training/training-data.md

@@ -0,0 +1,337 @@
+# Training Data Management
+
+Manage data used for project-specific AI training and learning.
+
+---
+
+## Data Sources
+
+### Source Types
+
+```yaml
+training:
+  sources:
+    # Codebase
+    codebase:
+      enabled: true
+      paths:
+        - "src/**"
+        - "lib/**"
+      exclude:
+        - "**/*.test.*"
+        - "**/node_modules/**"
+
+    # Documentation
+    documentation:
+      enabled: true
+      paths:
+        - "docs/**"
+        - "README.md"
+        - "*.md"
+
+    # Git history
+    git:
+      enabled: true
+      include:
+        - "commit_messages"
+        - "pr_descriptions"
+        - "code_reviews"
+      depth: 1000  # Last 1000 commits
+
+    # User interactions
+    interactions:
+      enabled: true
+      include:
+        - "corrections"
+        - "acceptances"
+        - "feedback"
+```
+
+---
+
+## Data Collection
+
+### What Gets Collected
+
+```yaml
+training:
+  collection:
+    # Code patterns
+    code:
+      - naming_conventions
+      - code_structure
+      - common_patterns
+      - import_styles
+      - error_handling
+
+    # Documentation
+    docs:
+      - writing_style
+      - structure
+      - terminology
+
+    # User behavior
+    behavior:
+      - corrections
+      - preferences
+      - workflow_patterns
+
+    # Domain knowledge
+    domain:
+      - business_terms
+      - entities
+      - relationships
+```
+
+### Collection Configuration
+
+```yaml
+training:
+  collection:
+    # Frequency
+    schedule:
+      full_collection: "weekly"
+      incremental: "on_change"
+
+    # Limits
+    limits:
+      max_files: 10000
+      max_file_size: "1MB"
+      max_total_size: "100MB"
+
+    # Quality filters
+    filters:
+      min_file_age: "1d"  # Avoid collecting unstable code
+      exclude_generated: true
+      exclude_vendor: true
+```
+
+---
+
+## Data Processing
+
+### Processing Pipeline
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                  Data Processing Pipeline                    │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│  Raw Data                                                   │
+│       │                                                     │
+│       ▼                                                     │
+│  ┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐    │
+│  │  Clean  │──►│ Extract │──►│Normalize│──►│  Store  │    │
+│  └─────────┘   └─────────┘   └─────────┘   └─────────┘    │
+│       │             │             │             │          │
+│       ▼             ▼             ▼             ▼          │
+│  Remove       Extract        Standardize    Index for      │
+│  secrets      patterns       formats        retrieval      │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Processing Configuration
+
+```yaml
+training:
+  processing:
+    # Cleaning
+    cleaning:
+      remove_secrets: true
+      remove_pii: true
+      remove_comments: false
+      normalize_whitespace: true
+
+    # Extraction
+    extraction:
+      extract_patterns: true
+      extract_terminology: true
+      extract_relationships: true
+
+    # Normalization
+    normalization:
+      standardize_naming: false  # Preserve actual naming
+      standardize_structure: true
+```
+
+---
+
+## Data Storage
+
+### Storage Structure
+
+```
+.proagents/training-data/
+├── raw/                    # Raw collected data
+│   ├── codebase/
+│   ├── documentation/
+│   └── interactions/
+├── processed/              # Processed data
+│   ├── patterns.json
+│   ├── terminology.json
+│   └── relationships.json
+├── models/                 # Trained models/embeddings
+│   └── project-model.json
+└── metadata.json          # Collection metadata
+```
+
+### Storage Configuration
+
+```yaml
+training:
+  storage:
+    path: ".proagents/training-data"
+
+    # Compression
+    compress: true
+
+    # Encryption
+    encrypt: false
+    encryption_key_env: "TRAINING_DATA_KEY"
+
+    # Retention
+    retention:
+      raw_data: "30d"
+      processed_data: "1y"
+      models: "forever"
+
+    # Size limits
+    max_size: "500MB"
+    cleanup_on_limit: true
+```
+
+---
+
+## Data Quality
+
+### Quality Metrics
+
+```bash
+proagents training quality
+
+# Output:
+┌─────────────────────────────────────────────────────────────┐
+│ Training Data Quality                                       │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│ Total Data Points: 15,432                                   │
+│                                                             │
+│ Quality Scores:                                             │
+│ ├── Completeness: 92%                                      │
+│ ├── Consistency: 87%                                       │
+│ ├── Accuracy: 94%                                          │
+│ └── Freshness: 98%                                         │
+│                                                             │
+│ Issues:                                                     │
+│ ├── Outdated patterns: 45 (from deleted files)            │
+│ ├── Conflicting examples: 12                               │
+│ └── Missing context: 23                                    │
+│                                                             │
+│ Recommendation: Run 'proagents training refresh'           │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Quality Rules
+
+```yaml
+training:
+  quality:
+    rules:
+      # Minimum examples
+      min_examples_per_pattern: 3
+
+      # Consistency check
+      consistency_threshold: 0.80
+
+      # Freshness
+      max_data_age: "90d"
+
+      # Validation
+      validate_on_collection: true
+```
+
+---
+
+## Data Commands
+
+```bash
+# Collect training data
+proagents training collect
+
+# Process collected data
+proagents training process
+
+# View data statistics
+proagents training stats
+
+# Validate data quality
+proagents training validate
+
+# Refresh stale data
+proagents training refresh
+
+# Export data
+proagents training export --output training-data.tar.gz
+
+# Import data
+proagents training import training-data.tar.gz
+
+# Clear all data
+proagents training clear
+```
+
+---
+
+## Data Privacy
+
+### Privacy Controls
+
+```yaml
+training:
+  privacy:
+    # What to exclude
+    exclude:
+      - "**/*.env"
+      - "**/secrets/**"
+      - "**/credentials/**"
+      - "**/*password*"
+      - "**/*token*"
+      - "**/*key*"
+
+    # PII handling
+    pii:
+      detect: true
+      action: "redact"  # redact, exclude, hash
+
+    # Sensitive patterns
+    sensitive_patterns:
+      - "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Z|a-z]{2,}\\b"  # Email
+      - "\\b\\d{3}-\\d{2}-\\d{4}\\b"  # SSN
+      - "\\b\\d{16}\\b"  # Credit card
+```
+
+### Data Deletion
+
+```bash
+# Delete all training data
+proagents training clear --all
+
+# Delete specific category
+proagents training clear --category interactions
+
+# GDPR-compliant deletion
+proagents training clear --gdpr --user user_id
+```
+
+---
+
+## Best Practices
+
+1. **Regular Collection**: Keep training data fresh
+2. **Quality First**: Validate data quality regularly
+3. **Privacy Always**: Never collect sensitive data
+4. **Clean Up**: Remove outdated patterns
+5. **Backup**: Export data before major changes
+6. **Version**: Track data versions for reproducibility

package/proagents/ai-training/user-preferences.md

@@ -0,0 +1,346 @@
+# User Preferences Learning
+
+Track and learn individual user preferences for personalized AI assistance.
+
+---
+
+## Preference Categories
+
+### Communication Style
+
+```yaml
+user_preferences:
+  communication:
+    # Detail level
+    verbosity:
+      learned: "concise"  # concise, balanced, detailed
+      confidence: 0.89
+
+    # Explanation style
+    explanations:
+      prefer_examples: true
+      prefer_analogies: false
+      code_first: true
+
+    # Response format
+    format:
+      prefer_markdown: true
+      prefer_code_blocks: true
+      prefer_bullet_points: true
+```
+
+### Coding Preferences
+
+```yaml
+user_preferences:
+  coding:
+    # Language preferences
+    languages:
+      primary: "typescript"
+      frameworks: ["react", "nextjs"]
+
+    # Style preferences
+    style:
+      prefer_functional: true
+      prefer_immutable: true
+      prefer_explicit_types: true
+
+    # Patterns
+    patterns:
+      prefer_hooks: true
+      prefer_composition: true
+      avoid_class_components: true
+```
+
+### Workflow Preferences
+
+```yaml
+user_preferences:
+  workflow:
+    # Checkpoints
+    checkpoints:
+      usually_skips: ["after_requirements"]
+      always_reviews: ["after_design", "before_deployment"]
+
+    # Approvals
+    approvals:
+      prefers_quick_approval: false
+      detail_level: "high"
+
+    # Output
+    output:
+      prefers_file_output: false
+      prefers_inline_code: true
+```
+
+---
+
+## Learning Process
+
+### Implicit Learning
+
+```yaml
+preference_learning:
+  implicit:
+    # Learn from behavior
+    signals:
+      # Verbosity preference
+      - signal: "user_scrolls_past_explanation"
+        learn: "prefers_concise"
+        weight: 0.3
+
+      - signal: "user_asks_for_more_detail"
+        learn: "prefers_detailed"
+        weight: 0.5
+
+      # Code style
+      - signal: "user_modifies_to_functional"
+        learn: "prefers_functional"
+        weight: 0.7
+
+      # Format preference
+      - signal: "user_copies_code_block"
+        learn: "prefers_code_blocks"
+        weight: 0.2
+```
+
+### Explicit Learning
+
+```bash
+# Set preference explicitly
+proagents preferences set verbosity concise
+
+# Answer preference questions
+proagents preferences setup
+
+# Output:
+┌─────────────────────────────────────────────────────────────┐
+│ Preference Setup                                            │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│ How detailed should explanations be?                       │
+│ [1] Concise - Just the essentials                          │
+│ [2] Balanced - Key details with context                    │
+│ [3] Detailed - Full explanations                           │
+│                                                             │
+│ > 1                                                         │
+│                                                             │
+│ Do you prefer code examples over written explanations?     │
+│ [Y/N] > Y                                                   │
+│                                                             │
+│ ✓ Preferences saved                                        │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## Preference Storage
+
+### Storage Location
+
+```
+.proagents/user-preferences/
+├── preferences.json       # Main preferences
+├── history.json          # Preference change history
+└── learning.json         # Learning signals
+```
+
+### Preference File
+
+```json
+{
+  "user_id": "user_abc123",
+  "preferences": {
+    "communication": {
+      "verbosity": {
+        "value": "concise",
+        "source": "explicit",
+        "confidence": 1.0,
+        "set_at": "2024-01-15T10:00:00Z"
+      },
+      "code_first": {
+        "value": true,
+        "source": "learned",
+        "confidence": 0.85,
+        "learned_at": "2024-01-10T15:30:00Z"
+      }
+    },
+    "coding": {
+      "prefer_functional": {
+        "value": true,
+        "source": "learned",
+        "confidence": 0.92
+      }
+    }
+  },
+  "last_updated": "2024-01-15T10:30:00Z"
+}
+```
+
+---
+
+## Applying Preferences
+
+### Automatic Application
+
+```yaml
+preference_learning:
+  application:
+    # Confidence threshold
+    apply_when_confidence: 0.70
+
+    # Override user's explicit settings?
+    learned_overrides_explicit: false
+
+    # Per-category application
+    categories:
+      communication:
+        auto_apply: true
+      coding:
+        auto_apply: true
+        confirm_first: false
+      workflow:
+        auto_apply: true
+```
+
+### Example Application
+
+```
+Without Preferences:
+AI: "Here's a detailed explanation of how React hooks work..."
+    [500 words of explanation]
+    [code example]
+
+With Learned Preferences (concise, code_first):
+AI: "Here's how to use the hook:"
+    [code example]
+    "Key points: [brief bullet points]"
+```
+
+---
+
+## Preference Commands
+
+```bash
+# View current preferences
+proagents preferences show
+
+# Output:
+┌─────────────────────────────────────────────────────────────┐
+│ Your Preferences                                            │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│ Communication:                                              │
+│ ├── Verbosity: Concise (explicit)                          │
+│ ├── Code first: Yes (learned, 85%)                         │
+│ └── Format: Markdown with code blocks                      │
+│                                                             │
+│ Coding:                                                     │
+│ ├── Style: Functional (learned, 92%)                       │
+│ ├── Types: Explicit TypeScript                             │
+│ └── Patterns: Hooks, composition                           │
+│                                                             │
+│ Workflow:                                                   │
+│ ├── Skip checkpoints: after_requirements                   │
+│ └── Always review: design, deployment                      │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+
+# Reset preferences
+proagents preferences reset
+
+# Reset specific category
+proagents preferences reset --category coding
+
+# Export preferences
+proagents preferences export > my-preferences.json
+
+# Import preferences
+proagents preferences import my-preferences.json
+```
+
+---
+
+## Team Preferences
+
+### Shared Defaults
+
+```yaml
+# Team-level preferences
+team_preferences:
+  defaults:
+    verbosity: "balanced"
+    code_style: "functional"
+
+  # Allow individual override
+  allow_override: true
+
+  # Share learning across team
+  share_learning: false
+```
+
+### New User Defaults
+
+```yaml
+preference_learning:
+  new_users:
+    # Start with team defaults
+    use_team_defaults: true
+
+    # Or start fresh
+    start_fresh: false
+
+    # Prompt for initial setup
+    prompt_setup: true
+```
+
+---
+
+## Privacy
+
+### Privacy Settings
+
+```yaml
+preference_learning:
+  privacy:
+    # What to store
+    store:
+      explicit_preferences: true
+      learned_preferences: true
+      learning_signals: false  # Don't store raw signals
+
+    # Anonymization
+    anonymize: true
+
+    # Retention
+    retention: "1y"
+
+    # Export/delete
+    allow_export: true
+    allow_delete: true
+```
+
+### Delete Preferences
+
+```bash
+# Delete all preferences
+proagents preferences delete --all
+
+# Delete specific category
+proagents preferences delete --category communication
+
+# GDPR export
+proagents preferences export --gdpr
+```
+
+---
+
+## Best Practices
+
+1. **Start Explicit**: Set key preferences explicitly
+2. **Let It Learn**: Allow learning for fine-tuning
+3. **Review Periodically**: Check learned preferences
+4. **Reset If Wrong**: Reset if preferences drift
+5. **Team Alignment**: Consider team defaults
+6. **Privacy First**: Review privacy settings