workspace-architect 1.3.0

package/assets/instructions/performance-optimization.instructions.md

@@ -0,0 +1,420 @@
+---
+applyTo: '*'
+description: 'The most comprehensive, practical, and engineer-authored performance optimization instructions for all languages, frameworks, and stacks. Covers frontend, backend, and database best practices with actionable guidance, scenario-based checklists, troubleshooting, and pro tips.'
+---
+
+# Performance Optimization Best Practices
+
+## Introduction
+
+Performance isn't just a buzzword—it's the difference between a product people love and one they abandon. I've seen firsthand how a slow app can frustrate users, rack up cloud bills, and even lose customers. This guide is a living collection of the most effective, real-world performance practices I've used and reviewed, covering frontend, backend, and database layers, as well as advanced topics. Use it as a reference, a checklist, and a source of inspiration for building fast, efficient, and scalable software.
+
+---
+
+## General Principles
+
+- **Measure First, Optimize Second:** Always profile and measure before optimizing. Use benchmarks, profilers, and monitoring tools to identify real bottlenecks. Guessing is the enemy of performance.
+  - *Pro Tip:* Use tools like Chrome DevTools, Lighthouse, New Relic, Datadog, Py-Spy, or your language's built-in profilers.
+- **Optimize for the Common Case:** Focus on optimizing code paths that are most frequently executed. Don't waste time on rare edge cases unless they're critical.
+- **Avoid Premature Optimization:** Write clear, maintainable code first; optimize only when necessary. Premature optimization can make code harder to read and maintain.
+- **Minimize Resource Usage:** Use memory, CPU, network, and disk resources efficiently. Always ask: "Can this be done with less?"
+- **Prefer Simplicity:** Simple algorithms and data structures are often faster and easier to optimize. Don't over-engineer.
+- **Document Performance Assumptions:** Clearly comment on any code that is performance-critical or has non-obvious optimizations. Future maintainers (including you) will thank you.
+- **Understand the Platform:** Know the performance characteristics of your language, framework, and runtime. What's fast in Python may be slow in JavaScript, and vice versa.
+- **Automate Performance Testing:** Integrate performance tests and benchmarks into your CI/CD pipeline. Catch regressions early.
+- **Set Performance Budgets:** Define acceptable limits for load time, memory usage, API latency, etc. Enforce them with automated checks.
+
+---
+
+## Frontend Performance
+
+### Rendering and DOM
+- **Minimize DOM Manipulations:** Batch updates where possible. Frequent DOM changes are expensive.
+  - *Anti-pattern:* Updating the DOM in a loop. Instead, build a document fragment and append it once.
+- **Virtual DOM Frameworks:** Use React, Vue, or similar efficiently—avoid unnecessary re-renders.
+  - *React Example:* Use `React.memo`, `useMemo`, and `useCallback` to prevent unnecessary renders.
+- **Keys in Lists:** Always use stable keys in lists to help virtual DOM diffing. Avoid using array indices as keys unless the list is static.
+- **Avoid Inline Styles:** Inline styles can trigger layout thrashing. Prefer CSS classes.
+- **CSS Animations:** Use CSS transitions/animations over JavaScript for smoother, GPU-accelerated effects.
+- **Defer Non-Critical Rendering:** Use `requestIdleCallback` or similar to defer work until the browser is idle.
+
+### Asset Optimization
+- **Image Compression:** Use tools like ImageOptim, Squoosh, or TinyPNG. Prefer modern formats (WebP, AVIF) for web delivery.
+- **SVGs for Icons:** SVGs scale well and are often smaller than PNGs for simple graphics.
+- **Minification and Bundling:** Use Webpack, Rollup, or esbuild to bundle and minify JS/CSS. Enable tree-shaking to remove dead code.
+- **Cache Headers:** Set long-lived cache headers for static assets. Use cache busting for updates.
+- **Lazy Loading:** Use `loading="lazy"` for images, and dynamic imports for JS modules/components.
+- **Font Optimization:** Use only the character sets you need. Subset fonts and use `font-display: swap`.
+
+### Network Optimization
+- **Reduce HTTP Requests:** Combine files, use image sprites, and inline critical CSS.
+- **HTTP/2 and HTTP/3:** Enable these protocols for multiplexing and lower latency.
+- **Client-Side Caching:** Use Service Workers, IndexedDB, and localStorage for offline and repeat visits.
+- **CDNs:** Serve static assets from a CDN close to your users. Use multiple CDNs for redundancy.
+- **Defer/Async Scripts:** Use `defer` or `async` for non-critical JS to avoid blocking rendering.
+- **Preload and Prefetch:** Use `<link rel="preload">` and `<link rel="prefetch">` for critical resources.
+
+### JavaScript Performance
+- **Avoid Blocking the Main Thread:** Offload heavy computation to Web Workers.
+- **Debounce/Throttle Events:** For scroll, resize, and input events, use debounce/throttle to limit handler frequency.
+- **Memory Leaks:** Clean up event listeners, intervals, and DOM references. Use browser dev tools to check for detached nodes.
+- **Efficient Data Structures:** Use Maps/Sets for lookups, TypedArrays for numeric data.
+- **Avoid Global Variables:** Globals can cause memory leaks and unpredictable performance.
+- **Avoid Deep Object Cloning:** Use shallow copies or libraries like lodash's `cloneDeep` only when necessary.
+
+### Accessibility and Performance
+- **Accessible Components:** Ensure ARIA updates are not excessive. Use semantic HTML for both accessibility and performance.
+- **Screen Reader Performance:** Avoid rapid DOM updates that can overwhelm assistive tech.
+
+### Framework-Specific Tips
+#### React
+- Use `React.memo`, `useMemo`, and `useCallback` to avoid unnecessary renders.
+- Split large components and use code-splitting (`React.lazy`, `Suspense`).
+- Avoid anonymous functions in render; they create new references on every render.
+- Use `ErrorBoundary` to catch and handle errors gracefully.
+- Profile with React DevTools Profiler.
+
+#### Angular
+- Use OnPush change detection for components that don't need frequent updates.
+- Avoid complex expressions in templates; move logic to the component class.
+- Use `trackBy` in `ngFor` for efficient list rendering.
+- Lazy load modules and components with the Angular Router.
+- Profile with Angular DevTools.
+
+#### Vue
+- Use computed properties over methods in templates for caching.
+- Use `v-show` vs `v-if` appropriately (`v-show` is better for toggling visibility frequently).
+- Lazy load components and routes with Vue Router.
+- Profile with Vue Devtools.
+
+### Common Frontend Pitfalls
+- Loading large JS bundles on initial page load.
+- Not compressing images or using outdated formats.
+- Failing to clean up event listeners, causing memory leaks.
+- Overusing third-party libraries for simple tasks.
+- Ignoring mobile performance (test on real devices!).
+
+### Frontend Troubleshooting
+- Use Chrome DevTools' Performance tab to record and analyze slow frames.
+- Use Lighthouse to audit performance and get actionable suggestions.
+- Use WebPageTest for real-world load testing.
+- Monitor Core Web Vitals (LCP, FID, CLS) for user-centric metrics.
+
+---
+
+## Backend Performance
+
+### Algorithm and Data Structure Optimization
+- **Choose the Right Data Structure:** Arrays for sequential access, hash maps for fast lookups, trees for hierarchical data, etc.
+- **Efficient Algorithms:** Use binary search, quicksort, or hash-based algorithms where appropriate.
+- **Avoid O(n^2) or Worse:** Profile nested loops and recursive calls. Refactor to reduce complexity.
+- **Batch Processing:** Process data in batches to reduce overhead (e.g., bulk database inserts).
+- **Streaming:** Use streaming APIs for large data sets to avoid loading everything into memory.
+
+### Concurrency and Parallelism
+- **Asynchronous I/O:** Use async/await, callbacks, or event loops to avoid blocking threads.
+- **Thread/Worker Pools:** Use pools to manage concurrency and avoid resource exhaustion.
+- **Avoid Race Conditions:** Use locks, semaphores, or atomic operations where needed.
+- **Bulk Operations:** Batch network/database calls to reduce round trips.
+- **Backpressure:** Implement backpressure in queues and pipelines to avoid overload.
+
+### Caching
+- **Cache Expensive Computations:** Use in-memory caches (Redis, Memcached) for hot data.
+- **Cache Invalidation:** Use time-based (TTL), event-based, or manual invalidation. Stale cache is worse than no cache.
+- **Distributed Caching:** For multi-server setups, use distributed caches and be aware of consistency issues.
+- **Cache Stampede Protection:** Use locks or request coalescing to prevent thundering herd problems.
+- **Don't Cache Everything:** Some data is too volatile or sensitive to cache.
+
+### API and Network
+- **Minimize Payloads:** Use JSON, compress responses (gzip, Brotli), and avoid sending unnecessary data.
+- **Pagination:** Always paginate large result sets. Use cursors for real-time data.
+- **Rate Limiting:** Protect APIs from abuse and overload.
+- **Connection Pooling:** Reuse connections for databases and external services.
+- **Protocol Choice:** Use HTTP/2, gRPC, or WebSockets for high-throughput, low-latency communication.
+
+### Logging and Monitoring
+- **Minimize Logging in Hot Paths:** Excessive logging can slow down critical code.
+- **Structured Logging:** Use JSON or key-value logs for easier parsing and analysis.
+- **Monitor Everything:** Latency, throughput, error rates, resource usage. Use Prometheus, Grafana, Datadog, or similar.
+- **Alerting:** Set up alerts for performance regressions and resource exhaustion.
+
+### Language/Framework-Specific Tips
+#### Node.js
+- Use asynchronous APIs; avoid blocking the event loop (e.g., never use `fs.readFileSync` in production).
+- Use clustering or worker threads for CPU-bound tasks.
+- Limit concurrent open connections to avoid resource exhaustion.
+- Use streams for large file or network data processing.
+- Profile with `clinic.js`, `node --inspect`, or Chrome DevTools.
+
+#### Python
+- Use built-in data structures (`dict`, `set`, `deque`) for speed.
+- Profile with `cProfile`, `line_profiler`, or `Py-Spy`.
+- Use `multiprocessing` or `asyncio` for parallelism.
+- Avoid GIL bottlenecks in CPU-bound code; use C extensions or subprocesses.
+- Use `lru_cache` for memoization.
+
+#### Java
+- Use efficient collections (`ArrayList`, `HashMap`, etc.).
+- Profile with VisualVM, JProfiler, or YourKit.
+- Use thread pools (`Executors`) for concurrency.
+- Tune JVM options for heap and garbage collection (`-Xmx`, `-Xms`, `-XX:+UseG1GC`).
+- Use `CompletableFuture` for async programming.
+
+#### .NET
+- Use `async/await` for I/O-bound operations.
+- Use `Span<T>` and `Memory<T>` for efficient memory access.
+- Profile with dotTrace, Visual Studio Profiler, or PerfView.
+- Pool objects and connections where appropriate.
+- Use `IAsyncEnumerable<T>` for streaming data.
+
+### Common Backend Pitfalls
+- Synchronous/blocking I/O in web servers.
+- Not using connection pooling for databases.
+- Over-caching or caching sensitive/volatile data.
+- Ignoring error handling in async code.
+- Not monitoring or alerting on performance regressions.
+
+### Backend Troubleshooting
+- Use flame graphs to visualize CPU usage.
+- Use distributed tracing (OpenTelemetry, Jaeger, Zipkin) to track request latency across services.
+- Use heap dumps and memory profilers to find leaks.
+- Log slow queries and API calls for analysis.
+
+---
+
+## Database Performance
+
+### Query Optimization
+- **Indexes:** Use indexes on columns that are frequently queried, filtered, or joined. Monitor index usage and drop unused indexes.
+- **Avoid SELECT *:** Select only the columns you need. Reduces I/O and memory usage.
+- **Parameterized Queries:** Prevent SQL injection and improve plan caching.
+- **Query Plans:** Analyze and optimize query execution plans. Use `EXPLAIN` in SQL databases.
+- **Avoid N+1 Queries:** Use joins or batch queries to avoid repeated queries in loops.
+- **Limit Result Sets:** Use `LIMIT`/`OFFSET` or cursors for large tables.
+
+### Schema Design
+- **Normalization:** Normalize to reduce redundancy, but denormalize for read-heavy workloads if needed.
+- **Data Types:** Use the most efficient data types and set appropriate constraints.
+- **Partitioning:** Partition large tables for scalability and manageability.
+- **Archiving:** Regularly archive or purge old data to keep tables small and fast.
+- **Foreign Keys:** Use them for data integrity, but be aware of performance trade-offs in high-write scenarios.
+
+### Transactions
+- **Short Transactions:** Keep transactions as short as possible to reduce lock contention.
+- **Isolation Levels:** Use the lowest isolation level that meets your consistency needs.
+- **Avoid Long-Running Transactions:** They can block other operations and increase deadlocks.
+
+### Caching and Replication
+- **Read Replicas:** Use for scaling read-heavy workloads. Monitor replication lag.
+- **Cache Query Results:** Use Redis or Memcached for frequently accessed queries.
+- **Write-Through/Write-Behind:** Choose the right strategy for your consistency needs.
+- **Sharding:** Distribute data across multiple servers for scalability.
+
+### NoSQL Databases
+- **Design for Access Patterns:** Model your data for the queries you need.
+- **Avoid Hot Partitions:** Distribute writes/reads evenly.
+- **Unbounded Growth:** Watch for unbounded arrays or documents.
+- **Sharding and Replication:** Use for scalability and availability.
+- **Consistency Models:** Understand eventual vs strong consistency and choose appropriately.
+
+### Common Database Pitfalls
+- Missing or unused indexes.
+- SELECT * in production queries.
+- Not monitoring slow queries.
+- Ignoring replication lag.
+- Not archiving old data.
+
+### Database Troubleshooting
+- Use slow query logs to identify bottlenecks.
+- Use `EXPLAIN` to analyze query plans.
+- Monitor cache hit/miss ratios.
+- Use database-specific monitoring tools (pg_stat_statements, MySQL Performance Schema).
+
+---
+
+## Code Review Checklist for Performance
+
+- [ ] Are there any obvious algorithmic inefficiencies (O(n^2) or worse)?
+- [ ] Are data structures appropriate for their use?
+- [ ] Are there unnecessary computations or repeated work?
+- [ ] Is caching used where appropriate, and is invalidation handled correctly?
+- [ ] Are database queries optimized, indexed, and free of N+1 issues?
+- [ ] Are large payloads paginated, streamed, or chunked?
+- [ ] Are there any memory leaks or unbounded resource usage?
+- [ ] Are network requests minimized, batched, and retried on failure?
+- [ ] Are assets optimized, compressed, and served efficiently?
+- [ ] Are there any blocking operations in hot paths?
+- [ ] Is logging in hot paths minimized and structured?
+- [ ] Are performance-critical code paths documented and tested?
+- [ ] Are there automated tests or benchmarks for performance-sensitive code?
+- [ ] Are there alerts for performance regressions?
+- [ ] Are there any anti-patterns (e.g., SELECT *, blocking I/O, global variables)?
+
+---
+
+## Advanced Topics
+
+### Profiling and Benchmarking
+- **Profilers:** Use language-specific profilers (Chrome DevTools, Py-Spy, VisualVM, dotTrace, etc.) to identify bottlenecks.
+- **Microbenchmarks:** Write microbenchmarks for critical code paths. Use `benchmark.js`, `pytest-benchmark`, or JMH for Java.
+- **A/B Testing:** Measure real-world impact of optimizations with A/B or canary releases.
+- **Continuous Performance Testing:** Integrate performance tests into CI/CD. Use tools like k6, Gatling, or Locust.
+
+### Memory Management
+- **Resource Cleanup:** Always release resources (files, sockets, DB connections) promptly.
+- **Object Pooling:** Use for frequently created/destroyed objects (e.g., DB connections, threads).
+- **Heap Monitoring:** Monitor heap usage and garbage collection. Tune GC settings for your workload.
+- **Memory Leaks:** Use leak detection tools (Valgrind, LeakCanary, Chrome DevTools).
+
+### Scalability
+- **Horizontal Scaling:** Design stateless services, use sharding/partitioning, and load balancers.
+- **Auto-Scaling:** Use cloud auto-scaling groups and set sensible thresholds.
+- **Bottleneck Analysis:** Identify and address single points of failure.
+- **Distributed Systems:** Use idempotent operations, retries, and circuit breakers.
+
+### Security and Performance
+- **Efficient Crypto:** Use hardware-accelerated and well-maintained cryptographic libraries.
+- **Validation:** Validate inputs efficiently; avoid regexes in hot paths.
+- **Rate Limiting:** Protect against DoS without harming legitimate users.
+
+### Mobile Performance
+- **Startup Time:** Lazy load features, defer heavy work, and minimize initial bundle size.
+- **Image/Asset Optimization:** Use responsive images and compress assets for mobile bandwidth.
+- **Efficient Storage:** Use SQLite, Realm, or platform-optimized storage.
+- **Profiling:** Use Android Profiler, Instruments (iOS), or Firebase Performance Monitoring.
+
+### Cloud and Serverless
+- **Cold Starts:** Minimize dependencies and keep functions warm.
+- **Resource Allocation:** Tune memory/CPU for serverless functions.
+- **Managed Services:** Use managed caching, queues, and DBs for scalability.
+- **Cost Optimization:** Monitor and optimize for cloud cost as a performance metric.
+
+---
+
+## Practical Examples
+
+### Example 1: Debouncing User Input in JavaScript
+```javascript
+// BAD: Triggers API call on every keystroke
+input.addEventListener('input', (e) => {
+  fetch(`/search?q=${e.target.value}`);
+});
+
+// GOOD: Debounce API calls
+let timeout;
+input.addEventListener('input', (e) => {
+  clearTimeout(timeout);
+  timeout = setTimeout(() => {
+    fetch(`/search?q=${e.target.value}`);
+  }, 300);
+});
+```
+
+### Example 2: Efficient SQL Query
+```sql
+-- BAD: Selects all columns and does not use an index
+SELECT * FROM users WHERE email = 'user@example.com';
+
+-- GOOD: Selects only needed columns and uses an index
+SELECT id, name FROM users WHERE email = 'user@example.com';
+```
+
+### Example 3: Caching Expensive Computation in Python
+```python
+# BAD: Recomputes result every time
+result = expensive_function(x)
+
+# GOOD: Cache result
+from functools import lru_cache
+
+@lru_cache(maxsize=128)
+def expensive_function(x):
+    ...
+result = expensive_function(x)
+```
+
+### Example 4: Lazy Loading Images in HTML
+```html
+<!-- BAD: Loads all images immediately -->
+<img src="large-image.jpg" />
+
+<!-- GOOD: Lazy loads images -->
+<img src="large-image.jpg" loading="lazy" />
+```
+
+### Example 5: Asynchronous I/O in Node.js
+```javascript
+// BAD: Blocking file read
+const data = fs.readFileSync('file.txt');
+
+// GOOD: Non-blocking file read
+fs.readFile('file.txt', (err, data) => {
+  if (err) throw err;
+  // process data
+});
+```
+
+### Example 6: Profiling a Python Function
+```python
+import cProfile
+import pstats
+
+def slow_function():
+    ...
+
+cProfile.run('slow_function()', 'profile.stats')
+p = pstats.Stats('profile.stats')
+p.sort_stats('cumulative').print_stats(10)
+```
+
+### Example 7: Using Redis for Caching in Node.js
+```javascript
+const redis = require('redis');
+const client = redis.createClient();
+
+function getCachedData(key, fetchFunction) {
+  return new Promise((resolve, reject) => {
+    client.get(key, (err, data) => {
+      if (data) return resolve(JSON.parse(data));
+      fetchFunction().then(result => {
+        client.setex(key, 3600, JSON.stringify(result));
+        resolve(result);
+      });
+    });
+  });
+}
+```
+
+---
+
+## References and Further Reading
+- [Google Web Fundamentals: Performance](https://web.dev/performance/)
+- [MDN Web Docs: Performance](https://developer.mozilla.org/en-US/docs/Web/Performance)
+- [OWASP: Performance Testing](https://owasp.org/www-project-performance-testing/)
+- [Microsoft Performance Best Practices](https://learn.microsoft.com/en-us/azure/architecture/best-practices/performance)
+- [PostgreSQL Performance Optimization](https://wiki.postgresql.org/wiki/Performance_Optimization)
+- [MySQL Performance Tuning](https://dev.mysql.com/doc/refman/8.0/en/optimization.html)
+- [Node.js Performance Best Practices](https://nodejs.org/en/docs/guides/simple-profiling/)
+- [Python Performance Tips](https://docs.python.org/3/library/profile.html)
+- [Java Performance Tuning](https://www.oracle.com/java/technologies/javase/performance.html)
+- [.NET Performance Guide](https://learn.microsoft.com/en-us/dotnet/standard/performance/)
+- [WebPageTest](https://www.webpagetest.org/)
+- [Lighthouse](https://developers.google.com/web/tools/lighthouse)
+- [Prometheus](https://prometheus.io/)
+- [Grafana](https://grafana.com/)
+- [k6 Load Testing](https://k6.io/)
+- [Gatling](https://gatling.io/)
+- [Locust](https://locust.io/)
+- [OpenTelemetry](https://opentelemetry.io/)
+- [Jaeger](https://www.jaegertracing.io/)
+- [Zipkin](https://zipkin.io/)
+
+---
+
+## Conclusion
+
+Performance optimization is an ongoing process. Always measure, profile, and iterate. Use these best practices, checklists, and troubleshooting tips to guide your development and code reviews for high-performance, scalable, and efficient software. If you have new tips or lessons learned, add them here—let's keep this guide growing!
+
+---
+
+<!-- End of Performance Optimization Instructions -->