@bostonuniversity/buwp-local 0.5.3 → 0.6.1

package/docs/ROADMAP.md

@@ -0,0 +1,237 @@
+# Roadmap
+
+Strategic direction and development priorities for buwp-local.
+
+## Release History
+
+### ✅ v0.5.x - Keychain & Credential Management (Complete)
+
+**Status:** Shipped  
+
+**Key Features:**
+- macOS Keychain integration for secure credential storage
+- Automatic hex decoding for legacy multiline credentials
+- Credential validation on startup with interactive setup
+- Multi-project support with isolated Docker volumes
+- Smart initialization for plugins, themes, and mu-plugins
+
+**Result:** Ready for production use by small development teams.
+
+---
+
+## v0.6.0 - Quality & Robustness
+**Status:** Shipped  
+**Focus:** Foundation improvements before team rollout
+
+### Features in Development
+
+1. **Credential Validation** ✅ (Complete)
+   - Validates credentials before starting containers
+   - Offers interactive setup if credentials missing
+   - Clear error messages and guidance
+
+2. **Documentation Consolidation** ✅ (Complete)
+   - Reorganized docs in `/docs` directory
+   - Comprehensive guides for all user levels
+   - Architecture documentation for contributors
+
+3. **Robust /etc/hosts Detection** ✅ (Complete)
+   - Detects if hostname exists in `/etc/hosts` during start
+   - Provides copy-paste sudo command if missing
+   - Smart messaging (only shows once per project)
+   - Non-blocking (user can continue without adding)
+
+### Success Criteria
+- ✅ Zero setup confusion for new users
+- ✅ All common workflows documented
+- ✅ Hostname setup guidance clear and actionable
+
+**Status:** All v0.6.0 features complete. Ready for initial user rollout.
+
+---
+
+## Lessons Learned: /etc/hosts Management
+
+### What We Built (v0.6.0)
+The detection-only approach proved to be the right choice:
+- **Non-intrusive** - Checks once, shows clear instructions
+- **No sudo required** - Detection runs without elevated permissions
+- **Smart persistence** - Doesn't nag on every start
+- **User control** - Provides copy-paste command, user decides when to run
+
+### Future Consideration: Automatic Management
+
+**Library Identified:** [`hostile`](https://www.npmjs.com/package/hostile) npm package
+- ~500k weekly downloads, well-maintained
+- Cross-platform (macOS, Linux, Windows)
+- Programmatic API: `hostile.set('127.0.0.1', 'hostname')`
+- Sync and async methods available
+
+**Why We're Not Implementing It Now:**
+
+1. **Requires sudo** - Password prompt interrupts smooth startup flow
+2. **Security sensitivity** - Modifying system files needs user trust
+3. **Current solution works** - Detection + copy-paste is clear and effective
+4. **Premature optimization** - Need real user feedback first
+
+**If We Revisit (v0.8.0+):**
+
+Implementation would be straightforward:
+```javascript
+import hostile from 'hostile';
+
+// Add hostname (requires sudo, prompts for password)
+hostile.set('127.0.0.1', config.hostname, (err) => {
+  if (err) {
+    // Fall back to manual instructions
+  }
+});
+
+// Remove on destroy
+hostile.remove('127.0.0.1', config.hostname);
+```
+
+**Decision criteria for future:**
+- Do >50% of users request automatic management?
+- Is sudo prompt acceptable to most users?
+- Would opt-in flag (`autoManageHosts: true`) address concerns?
+- Does it meaningfully improve onboarding over current approach?
+
+**Current recommendation:** Gather feedback from Stage 1 users (2-3 developers) in December 2024. If hostname setup proves to be a friction point, revisit automatic management in Q1 2025.
+
+---
+
+## Incremental Functional Improvements: v0.6.1
+**Status:** Currently Ongoing
+**Focus:** Enhancements based on initial user experience
+
+- **Container Registry Assistance**
+  - Guide users on setting up access to private registries (ghcr.io)
+  - Automatic check for registry login or existing image on `start`
+
+- **Basic docs on existing Xdebug features**
+  - Quickstart guide for enabling and using Xdebug in containers
+
+- **Volume Mapping pattern guide**
+  - Documentation on different volume mapping strategies for various development workflows
+
+## Next Phase: v0.7.0 - Developer Experience
+
+**Status:** Planned  
+**Timeline:** After team onboarding feedback  
+**Focus:** Ease of use and visibility
+
+### Potential Features
+
+- **Security Checks**
+  - Check database access on db port (e.g. `localhost:3306`)
+  - Consider more stringent default database passwords
+  - The database can have restricted content in it, so we need to ensure that users are aware of this and take appropriate measures.
+
+- **Xdebug Integration**
+  - Command to help generate Xdebug configuration for IDEs (VSCode, Zed)
+  - Documentation on usage patterns
+
+- **Improved Windows and Linux support**
+  - Multiplatform /etc/hosts hostname management
+  - Evaluate credential storage solutions for non-macOS platforms (https://www.npmjs.com/package/keytar)
+
+- **Project Status & Listing**
+  - Central tracking of all buwp-local projects in `~/.buwp-local/projects.json`
+  - View all running projects: `buwp-local list`
+  - Quick status checks: `buwp-local status`
+
+- **Health Checks**
+  - Verify services are running properly
+  - Database connectivity validation
+  - Clear diagnostics on failures
+
+- **Improved Error Messages**
+  - Docker startup failures → actionable solutions
+  - Credential issues → clear next steps
+  - Port conflicts → suggest alternatives
+
+- **Multi project experience**
+  - There is a problem when starting a new project when an existing project exists in docker but is stopped. When starting the new project, docker first starts the container for the stopped project for unknown reasons. If the new project uses the same ports, this causes conflicts. Need to investigate and resolve, projects should be isolated and not interfere with each other.
+
+- **Docker Volume management assistant**
+  - listing and cleanup of unused volumes
+
+### Prioritization
+Will be informed by feedback from initial 2-3 users and actual pain points observed during rollout.
+
+---
+
+## Future Phases: v0.8.0+
+
+**Status:** Conceptual  
+**Timeline:** TBD based on team feedback
+
+### Potential Features (Lower Priority)
+
+- **SSL Certificate Generation** - Local HTTPS with mkcert
+- **Database Backup/Restore** - Simplified snapshots and recovery
+- **Performance Monitoring** - Real-time resource usage tracking
+- **Team Configuration Sync** - Share project configurations across team
+- **Cross-Platform Support** - Windows WSL2 and Linux credential storage
+
+**Note:** Automatic `/etc/hosts` management deferred pending user feedback. See "Lessons Learned" section above for details on the `hostile` library approach.
+
+---
+
+## Roadmap by User Stage
+
+### Stage 1: Initial Users
+**Users:** 1-3 developers  
+**Goal:** Validate core functionality  
+**Release:** v0.6.0  
+**Focus:** Robustness, clear setup, good documentation
+
+### Stage 2: Team Rollout
+**Users:** 10-15 developers  
+**Goal:** Find and fix real-world issues  
+**Release:** v0.7.0+  
+**Focus:** Developer experience, error handling
+
+### Stage 3: Broader Adoption
+**Users:** 20+ developers  
+**Goal:** Self-service onboarding  
+**Release:** v1.0.0+  
+**Focus:** Advanced features, automation
+
+---
+
+## Technical Debt
+
+### Testing
+- Unit tests for core modules (config, keychain, docker-compose)
+- Integration tests for Docker operations
+- E2E tests for full workflows
+
+### Documentation
+- Troubleshooting guide for common issues
+- FAQ section
+- Video tutorials for setup
+
+### Quality
+- Standardized help for all CLI commands
+
+---
+
+## Decision Framework
+
+Features will be prioritized based on:
+
+1. **User Feedback** - What's actually blocking users?
+2. **Adoption Impact** - Does it help onboard new users?
+3. **Implementation Effort** - Can it be done quickly?
+4. **Maintenance Burden** - Will it create ongoing support overhead?
+
+---
+
+## Technical Details
+
+For detailed information about planned features, implementation approaches, and architecture decisions, see:
+
+- **[ARCHITECTURE.md](ARCHITECTURE.md)** - Planned Features section
+- **[CHANGELOG.md](CHANGELOG.md)** - Version history and release notes

package/lib/commands/destroy.js

@@ -85,7 +85,7 @@ function confirmDestroy(projectName) {
     console.log(chalk.yellow(`This will destroy project: ${chalk.bold(projectName)}`));
     console.log(chalk.yellow('  - Stop all containers'));
     console.log(chalk.yellow('  - Remove all containers'));
-    console.log(chalk.yellow('  - Delete all volumes (including database data)\n'));
+    console.log(chalk.yellow('  - Delete all volumes (including database data) (except maybe not, please fix)\n'));
 
     rl.question(chalk.red('Are you sure you want to continue? (yes/no): '), (answer) => {
       rl.close();

package/lib/commands/init.js

@@ -295,7 +295,8 @@ async function initCommand(options) {
   // Show next steps
   console.log(chalk.cyan('📝 Next steps:'));
   console.log(chalk.gray('  1. Check keychain credentials or add .env.local with your credentials'));
-  console.log(chalk.gray(`  2. Add to /etc/hosts: 127.0.0.1 ${answers.hostname}`));
+  console.log(chalk.gray('  2. Add to /etc/hosts with this command:'));
+  console.log(chalk.cyan(`     echo "127.0.0.1 ${answers.hostname}" | sudo tee -a /etc/hosts\n`));
   
   if (answers.projectType === 'sandbox') {
     console.log(chalk.gray('  3. Edit .buwp-local.json and add volume mappings to the mappings array'));

package/lib/commands/keychain.js

@@ -635,7 +635,7 @@ function showHelp() {
   console.log('  # Interactive setup (prompts for each credential)');
   console.log('  buwp-local keychain setup\n');
   console.log('  # Bulk import from JSON file');
-  console.log('  buwp-local keychain setup --file .buwp-credentials.json\n');
+  console.log('  buwp-local keychain setup --file buwp-local-credentials.json\n');
   console.log('  # Interactive prompt for single-line credential');
   console.log('  buwp-local keychain set WORDPRESS_DB_PASSWORD\n');
   console.log('  # Set single-line credential directly');

package/lib/commands/start.js

@@ -6,8 +6,164 @@ import chalk from 'chalk';
 import { execSync } from 'child_process';
 import path from 'path';
 import fs from 'fs';
+import prompts from 'prompts';
 import { loadConfig, validateConfig, ENV_FILE_NAME, loadKeychainCredentials, createSecureTempEnvFile, secureDeleteTempEnvFile } from '../config.js';
 import { generateComposeFile } from '../compose-generator.js';
+import keychainCommand from './keychain.js';
+
+/**
+ * Required credentials that must be present for WordPress to function
+ */
+const REQUIRED_CREDENTIALS = [
+  'WORDPRESS_DB_PASSWORD',
+  'DB_ROOT_PASSWORD'
+];
+
+/**
+ * Check if hostname exists in /etc/hosts
+ * @param {string} hostname - Hostname to check
+ * @returns {object} { found: boolean, error?: string }
+ */
+function checkHostsFile(hostname) {
+  try {
+    const hostsPath = '/etc/hosts';
+    const content = fs.readFileSync(hostsPath, 'utf8');
+    const lines = content.split('\n');
+    
+    // Check each line for hostname
+    for (const line of lines) {
+      // Skip comments and empty lines
+      if (line.trim().startsWith('#') || !line.trim()) continue;
+      
+      // Parse: "127.0.0.1 hostname.local" or "127.0.0.1\thostname.local"
+      const parts = line.trim().split(/\s+/);
+      if (parts.length >= 2) {
+        // Check if hostname appears in any position after IP
+        const hostnames = parts.slice(1);
+        if (hostnames.includes(hostname)) {
+          return { found: true };
+        }
+      }
+    }
+    
+    return { found: false };
+  } catch (error) {
+    // /etc/hosts not readable (unlikely on macOS)
+    return { found: false, error: error.message };
+  }
+}
+
+/**
+ * Check if we've already shown the hosts warning for this project
+ * @param {string} projectPath - Project directory path
+ * @returns {boolean} true if warning already shown
+ */
+function hasShownHostsWarning(projectPath) {
+  const warningFile = path.join(projectPath, '.buwp-local', '.hosts-warning-shown');
+  return fs.existsSync(warningFile);
+}
+
+/**
+ * Mark that we've shown the hosts warning for this project
+ * @param {string} projectPath - Project directory path
+ */
+function markHostsWarningShown(projectPath) {
+  const dir = path.join(projectPath, '.buwp-local');
+  fs.mkdirSync(dir, { recursive: true });
+  const warningFile = path.join(dir, '.hosts-warning-shown');
+  fs.writeFileSync(warningFile, Date.now().toString());
+}
+
+/**
+ * Check if required credentials are available in keychain or .env.local
+ * @param {object} keychainCreds - Credentials loaded from keychain
+ * @param {string} envFilePath - Path to .env.local file
+ * @returns {object} { isValid, missing }
+ */
+function validateCredentials(keychainCreds, envFilePath) {
+  const missing = [];
+  
+  for (const key of REQUIRED_CREDENTIALS) {
+    // Check if credential exists in keychain
+    if (keychainCreds[key]) {
+      continue; // Found in keychain
+    }
+    
+    // Check if credential exists in .env.local
+    if (fs.existsSync(envFilePath)) {
+      const envContent = fs.readFileSync(envFilePath, 'utf8');
+      if (envContent.includes(`${key}=`)) {
+        continue; // Found in .env.local
+      }
+    }
+    
+    // Credential not found anywhere
+    missing.push(key);
+  }
+  
+  return {
+    isValid: missing.length === 0,
+    missing
+  };
+}
+
+/**
+ * Prompt user to set up missing credentials
+ * @param {string[]} missingCreds - List of missing credential keys
+ * @returns {Promise<boolean>} true if user wants to set up, false otherwise
+ */
+async function promptCredentialSetup(missingCreds) {
+  console.log(chalk.yellow('\n⚠️  Missing Required Credentials\n'));
+  console.log(chalk.gray('The following credentials are not configured:\n'));
+  missingCreds.forEach(key => {
+    console.log(chalk.gray(`  - ${key}`));
+  });
+  console.log('');
+  
+  console.log(chalk.cyan('You can set up credentials in two ways:\n'));
+  console.log(chalk.white('  1. Use macOS Keychain (recommended):'));
+  console.log(chalk.gray('     npx buwp-local keychain setup\n'));
+  console.log(chalk.white('  2. Create .env.local file in your project:'));
+  console.log(chalk.gray('     cp .env.local.example .env.local'));
+  console.log(chalk.gray('     # Edit .env.local with your credentials\n'));
+  
+  const { shouldSetup } = await prompts({
+    type: 'confirm',
+    name: 'shouldSetup',
+    message: 'Would you like to set up credentials now using Keychain?',
+    initial: true
+  });
+  
+  return shouldSetup;
+}
+
+// Check if image is accessible
+async function checkImageAccess(imageName) {
+  try {
+    // Try to inspect image locally
+    execSync(`docker image inspect ${imageName}`, { stdio: 'pipe' });
+    return true; // Image exists locally
+  } catch {
+    // Image doesn't exist, try to pull
+    try {
+      console.log(chalk.gray(`Checking access to ${imageName}...`));
+      execSync(`docker pull ${imageName}`, { stdio: 'pipe' });
+      return true;
+    } catch (pullError) {
+      if (pullError.message.includes('unauthorized') || 
+          pullError.message.includes('denied')) {
+        console.log(chalk.yellow('\n⚠️  Cannot access container image\n'));
+        console.log('The BU WordPress image requires GitHub Packages authentication.\n');
+        console.log('Run this command to authenticate:\n');
+        console.log(chalk.cyan('  docker login ghcr.io\n'));
+        console.log('You will need a GitHub personal access token with "read:packages" scope.');
+        console.log('See: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry\n');
+        return false;
+      }
+      throw pullError; // Other error, let it surface
+    }
+  }
+}
 
 async function startCommand(options) {
   console.log(chalk.blue('🚀 Starting BU WordPress local environment...\n'));
@@ -43,6 +199,80 @@ async function startCommand(options) {
       process.exit(1);
     }
 
+    // Load and validate credentials early
+    const envFilePath = path.join(projectPath, ENV_FILE_NAME);
+    const keychainCredentials = loadKeychainCredentials();
+    const credentialValidation = validateCredentials(keychainCredentials, envFilePath);
+    
+    if (!credentialValidation.isValid) {
+      const shouldSetup = await promptCredentialSetup(credentialValidation.missing);
+      
+      if (shouldSetup) {
+        // Run keychain setup command
+        console.log(chalk.gray('\nLaunching Keychain Setup...\n'));
+        await keychainCommand('setup', [], {});
+        
+        // Reload credentials after setup
+        const reloadedCreds = loadKeychainCredentials();
+        const revalidation = validateCredentials(reloadedCreds, envFilePath);
+        
+        if (!revalidation.isValid) {
+          console.log(chalk.red('\n❌ Setup incomplete. Required credentials still missing:\n'));
+          revalidation.missing.forEach(key => {
+            console.log(chalk.red(`  - ${key}`));
+          });
+          console.log(chalk.gray('\nPlease complete setup or add credentials to .env.local\n'));
+          process.exit(1);
+        }
+      } else {
+        console.log(chalk.red('\n❌ Cannot start without required credentials.\n'));
+        process.exit(1);
+      }
+    } else {
+      console.log(chalk.green('✓ Required credentials validated\n'));
+    }
+
+    // Check image accessibility
+    console.log(chalk.gray('Checking container image access...'));
+    const imageAccessible = await checkImageAccess(config.image);
+
+    if (!imageAccessible) {
+      console.log(chalk.red('\n❌ Cannot proceed without access to container image.'));
+      console.log(chalk.gray('Please authenticate with GitHub Packages and try again.\n'));
+      process.exit(1);
+    }
+
+    console.log(chalk.green('✓ Container image accessible\n'));
+
+    // Check /etc/hosts for hostname entry
+    if (!hasShownHostsWarning(projectPath)) {
+      const hostsCheck = checkHostsFile(config.hostname);
+      
+      if (!hostsCheck.found) {
+        console.log(chalk.yellow('⚠️  Hostname not found in /etc/hosts\n'));
+        console.log(`Your site won't be accessible at ${chalk.cyan(`http://${config.hostname}`)}`);
+        console.log('until you add this entry:\n');
+        console.log(chalk.green(`  127.0.0.1 ${config.hostname}\n`));
+        console.log('Run this command to add it (copy and paste, and then enter your password):\n');
+        console.log(chalk.cyan(`  echo "127.0.0.1 ${config.hostname}" | sudo tee -a /etc/hosts\n`));
+        
+        const { continueStart } = await prompts({
+          type: 'confirm',
+          name: 'continueStart',
+          message: 'Continue starting containers?',
+          initial: true
+        });
+        
+        if (!continueStart) {
+          console.log(chalk.gray('\nStart cancelled. Add hostname to /etc/hosts and try again.'));
+          process.exit(0);
+        }
+        
+        // Mark warning as shown so we don't annoy user on every start
+        markHostsWarningShown(projectPath);
+      }
+    }
+
     // Generate docker-compose.yml
     console.log(chalk.gray('Generating docker-compose.yml...'));
     const composePath = generateComposeFile(config, projectPath);
@@ -63,12 +293,12 @@ async function startCommand(options) {
     
     // Load keychain credentials and create secure temp env file if available
     let tempEnvPath = null;
-    const keychainCredentials = loadKeychainCredentials();
-    const keychainCredCount = Object.keys(keychainCredentials).length;
+    const finalKeychainCredentials = loadKeychainCredentials();
+    const keychainCredCount = Object.keys(finalKeychainCredentials).length;
     
     if (keychainCredCount > 0) {
       try {
-        tempEnvPath = createSecureTempEnvFile(keychainCredentials, projectName);
+        tempEnvPath = createSecureTempEnvFile(finalKeychainCredentials, projectName);
         console.log(chalk.gray(`✓ Loaded ${keychainCredCount} credentials from keychain\n`));
       } catch (err) {
         console.warn(chalk.yellow(`⚠️  Could not load keychain credentials: ${err.message}`));
@@ -76,7 +306,6 @@ async function startCommand(options) {
     }
     
     // Check if .env.local exists and build env-file flags
-    const envFilePath = path.join(projectPath, ENV_FILE_NAME);
     const envFileFlag = fs.existsSync(envFilePath) ? `--env-file ${envFilePath}` : '';
     const tempEnvFileFlag = tempEnvPath ? `--env-file ${tempEnvPath}` : '';
     
@@ -111,10 +340,6 @@ async function startCommand(options) {
     console.log(chalk.white('  buwp-local stop    - Stop environment'));
     console.log(chalk.white('  buwp-local destroy - Remove environment\n'));
 
-    // Reminder about /etc/hosts
-    console.log(chalk.yellow('⚠️  Remember to add this to your /etc/hosts file:'));
-    console.log(chalk.white(`  127.0.0.1 ${config.hostname}\n`));
-
   } catch (err) {
     console.error(chalk.red('\n❌ Error:'), err.message);
     process.exit(1);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bostonuniversity/buwp-local",
-  "version": "0.5.3",
+  "version": "0.6.1",
   "description": "Local WordPress development environment for Boston University projects",
   "type": "module",
   "main": "lib/index.js",

package/readme.md

@@ -2,6 +2,16 @@
 
 This repository contains resources and instructions for setting up a local WordPress development environment for Boston University projects. It uses the BU WordPress container image and provides the additional resources needed to run it locally with Docker.
 
+The package can be installed in a specific repo for development of that one package, or standalone for more general use, mapping local code into the container as needed.
+
+> **Why buwp-local over VM sandboxes?**
+> 
+> Traditional VM sandboxes require **monthly rebuilds that wipe your development code**. With buwp-local's Docker architecture, your code lives on your local filesystem while only WordPress core updates. This means:
+> - ✅ **Keep your work** - No more monthly rebuild cycles that erase local changes
+> - ✅ **Update independently** - Pull WordPress updates on your schedule, not a global calendar
+> - ✅ **Instant rollback** - Switch between WordPress versions without losing work
+> 
+> Learn more: [Migration from VM Sandboxes](docs/MIGRATION_FROM_VM.md)
 
 ## Quickstart for plugin or theme development
 
@@ -84,3 +94,19 @@ Your local WordPress site should now be accessible at the hostname you configure
     ```
 
     This will download the latest snapshot from the specified source and import it into your local WordPress environment.
+
+## Documentation
+
+- 📘 [Getting Started Guide](docs/GETTING_STARTED.md)
+- 📖 [Command Reference](docs/COMMANDS.md)
+- 🔐 [Credential Management](docs/CREDENTIALS.md)
+- 🏗️ [Architecture](docs/ARCHITECTURE.md) (for contributors)
+
+## Features
+
+- ✅ One-time credential setup with macOS Keychain
+- ✅ Isolated environments for multiple projects
+- ✅ Pre-configured BU infrastructure (Shibboleth, S3, Redis)
+- ✅ Smart initialization for plugins, themes, and mu-plugins
+- ✅ Volume mapping for live code sync
+- ✅ Xdebug support for step debugging