kollabor 0.4.9__py3-none-any.whl → 0.4.15__py3-none-any.whl

agents/research/performance-profiling.md

@@ -0,0 +1,1047 @@
+<!-- Performance Profiling skill - identify bottlenecks and performance issues -->
+
+performance-profiling mode: MEASURE AND REPORT ONLY
+
+when this skill is active, you follow performance investigation discipline.
+this is a comprehensive guide to finding performance bottlenecks.
+you DO NOT implement optimizations - you report findings for the coder agent.
+
+
+PHASE 0: PROFILING TOOLKIT VERIFICATION
+
+before conducting ANY performance analysis, verify your tools are ready.
+
+
+check for python profilers
+
+  <terminal>python -c "import cProfile; print('cProfile available')"</terminal>
+  <terminal>python -c "import pstats; print('pstats available')"</terminal>
+
+if not available (should be in stdlib):
+  <terminal>python -m pip install --upgrade pip --quiet</terminal>
+
+
+check for advanced profilers
+
+  <terminal>python -c "import line_profiler; print('line_profiler installed')" 2>/dev/null || echo "line_profiler not installed"</terminal>
+  <terminal>python -c "import memory_profiler; print('memory_profiler installed')" 2>/dev/null || echo "memory_profiler not installed"</terminal>
+  <terminal>python -c "import py-spy; print('py-spy installed')" 2>/dev/null || echo "py-spy not installed"</terminal>
+
+if not installed:
+  <terminal>pip install line_profiler memory_profiler py-spy --quiet</terminal>
+
+verify installation:
+  <terminal>kernprof --version 2>/dev/null || echo "kernprof needs install"</terminal>
+  <terminal>mprof --version 2>/dev/null || echo "mprof needs install"</terminal>
+
+
+check for visualization tools
+
+  <terminal>python -c "import snakeviz; print('snakeviz installed')" 2>/dev/null || echo "snakeviz not installed"</terminal>
+  <terminal>python -c "import tuna; print('tuna installed')" 2>/dev/null || echo "tuna not installed"</terminal>
+
+if not installed:
+  <terminal>pip install snakeviz tuna --quiet</terminal>
+
+
+check for system monitoring tools
+
+  <terminal>which time 2>/dev/null || echo "time not found"</terminal>
+  <terminal>which ps 2>/dev/null || echo "ps not found"</terminal>
+  <terminal>which top 2>/dev/null || echo "top not found"</terminal>
+  <terminal>which htop 2>/dev/null || echo "htop not found"</terminal>
+
+these help monitor resource usage during profiling.
+
+
+check project structure
+
+  <terminal>ls -la</terminal>
+  <terminal>find . -name "*.py" -type f | head -20</terminal>
+  <terminal>grep -r "if __name__" --include="*.py" . 2>/dev/null | head -10</terminal>
+
+identify:
+  - main entry points
+  - long-running functions
+  - data processing modules
+  - API endpoints
+
+
+verify baseline measurements
+
+  <terminal>time python -c "print('python works')"</terminal>
+
+if python not responding quickly, investigate system issues first.
+
+
+PHASE 1: ESTABLISHING BASELINE METRICS
+
+you cannot improve what you do not measure.
+
+
+identify performance-critical paths
+
+  <read><file>main.py</file></read>
+
+look for:
+  - main loops
+  - request handlers
+  - data processing pipelines
+  - file I/O operations
+  - network calls
+  - heavy computations
+
+document these as primary profiling targets.
+
+
+measure execution time
+
+  <terminal>time python main.py 2>&1 | tee /tmp/baseline_time.txt</terminal>
+
+record:
+  - real (wall clock) time
+  - user CPU time
+  - system CPU time
+
+baseline establishes what "normal" performance looks like.
+
+
+measure memory baseline
+
+  <terminal>python -c "import psutil; print(f'Memory: {psutil.Process().memory_info().rss / 1024 / 1024:.1f} MB')"</terminal>
+
+record baseline memory usage before profiling.
+
+
+measure with python timeit
+
+  <terminal>python -m timeit "import main; main.main()" 2>&1 | tee /tmp/timeit_results.txt</terminal>
+
+timeit runs code multiple times for statistical accuracy.
+
+
+create reproducible workload
+
+for consistent profiling, you need consistent input.
+
+identify test data:
+  <terminal>find . -name "test*.json" -o -name "test*.csv" -o -name "fixtures*" 2>/dev/null | head -10</terminal>
+
+use the same workload for all profiling runs.
+
+
+PHASE 2: CPU PROFILING WITH CPROFILE
+
+
+basic cProfile usage
+
+  <terminal>python -m cProfile -o /tmp/profile.stats main.py 2>&1</terminal>
+
+this creates a binary statistics file.
+
+
+view cProfile output in text
+
+  <terminal>python -m pstats /tmp/profile.stats</terminal>
+
+in pstats interactive mode:
+  - stats 10              # show top 10 functions
+  - strip                 # strip directory names
+  - sort cumulative       # sort by cumulative time
+  - sort time             # sort by own time
+  - callers functionName  # show who calls a function
+  - callees functionName  # show what function calls
+
+
+view sorted by different metrics
+
+  <terminal>python -c "import pstats; p = pstats.Stats('/tmp/profile.stats'); p.sort_stats('cumulative').print_stats(20)"</terminal>
+
+metrics to sort by:
+  - cumulative: total time including subcalls
+  - time: time in function excluding subcalls
+  - calls: number of calls
+  - filename: file name
+
+
+profile with time output
+
+  <terminal>python -m cProfile -s time main.py 2>&1 | tee /tmp/profile_sorted.txt</terminal>
+
+  <terminal>python -m cProfile -s cumulative main.py 2>&1 | tee /tmp/profile_cumulative.txt</terminal>
+
+
+profile specific function
+
+  <read><file>path/to/module.py</file></read>
+
+create test script:
+  <terminal>python -c "import cProfile; from module import function; cProfile.run('function()', 'function_profile.stats')"</terminal>
+
+
+profile with context manager
+
+  <read><file>path/to/module.py</file></read>
+
+look for existing profiling code:
+  <terminal>grep -rn "cProfile\|pstats" --include="*.py" . 2>/dev/null</terminal>
+
+
+PHASE 3: INTERPRETING CPROFILE OUTPUT
+
+
+understanding the columns
+
+ncalls:   number of calls
+tottime:  total time in function (excluding subcalls)
+percall:  tottime / ncalls
+cumtime:  cumulative time (including subcalls)
+percall:  cumtime / ncalls
+filename:lineno(function): location
+
+key insights:
+  - high cumtime + low tottime = function calls slow sub-functions
+  - high tottime = function itself is slow
+  - high ncalls = function called many times (optimization target)
+
+
+identifying bottlenecks
+
+look for:
+  [1] functions with high cumulative time
+      these are the slowest paths through code
+
+  [2] functions called many times
+      optimization candidates: memoization, caching
+
+  [3] functions with high self-time
+      the computation itself is slow
+
+  <terminal>python -c "import pstats; p = pstats.Stats('/tmp/profile.stats'); p.sort_stats('cumulative').print_stats(30)"</terminal>
+
+
+identifying hot loops
+
+  <terminal>python -c "import pstats; p = pstats.Stats('/tmp/profile.stats'); p.sort_stats('calls').print_stats(30)"</terminal>
+
+many calls to same function = hot loop candidate.
+
+
+finding unexpected calls
+
+  <terminal>python -c "import pstats; p = pstats.Stats('/tmp/profile.stats'); p.print_callers('function_name')"</terminal>
+
+discover who is calling expensive functions unexpectedly.
+
+
+PHASE 4: LINE PROFILING WITH LINE_PROFILER
+
+
+install line_profiler
+
+  <terminal>pip install line_profiler --quiet</terminal>
+
+verify:
+  <terminal>kernprof --version</terminal>
+
+
+decorate functions for line profiling
+
+  <read><file>path/to/module.py</file></read>
+
+add @profile decorator to functions:
+  <terminal>grep -rn "@profile" --include="*.py" . 2>/dev/null | head -10</terminal>
+
+if decorators exist, they are for line_profiler.
+
+
+run line profiler
+
+  <terminal>kernprof -l -v main.py 2>&1 | tee /tmp/line_profile.txt</terminal>
+
+output shows time per line of code.
+
+
+understanding line profiler output
+
+  Line #      Hits         Time  Per Hit   % Time  Line Contents
+  ================================================================
+       1                                           @profile
+       2                                           def process(data):
+       3      1000          50      0.0      5.0      result = []
+       4   1000000       10000      0.0     50.0      for item in data:
+       5   1000000        8000      0.0     40.0          result.append(transform(item))
+       6      1000         200      0.2      5.0      return result
+
+key metrics:
+  - Hits: how many times line executed
+  - Time: total time spent on this line
+  - Per Hit: average time per execution
+  - % Time: percentage of total function time
+
+identify slow lines by high % Time.
+
+
+profile specific functions only
+
+  <terminal>kernprof -l -v -b main.py 2>&1 | tee /tmp/line_profile_specific.txt</terminal>
+
+or create test script:
+  <terminal>python -m line_profiler module.py::function_name 2>&1</terminal>
+
+
+PHASE 5: MEMORY PROFILING
+
+
+memory usage with memory_profiler
+
+  <terminal>pip install memory_profiler --quiet</terminal>
+
+verify:
+  <terminal>mprof --version</terminal>
+
+
+profile memory line by line
+
+add @profile decorator:
+  <read><file>path/to/module.py</file></read>
+
+check for existing @profile decorators:
+  <terminal>grep -rn "^@profile" --include="*.py" . 2>/dev/null</terminal>
+
+
+run memory profiler
+
+  <terminal>python -m memory_profiler main.py 2>&1 | tee /tmp/memory_profile.txt</terminal>
+
+
+understanding memory profiler output
+
+  Line #    Mem usage    Increment  Occurrences   Line Contents
+  =================================================================
+   1     50.0 MiB     50.0 MiB           1   @profile
+   2     50.0 MiB      0.0 MiB           1   def process():
+   3     50.0 MiB      0.0 MiB           1       data = []
+   4    150.0 MiB    100.0 MiB       10000       for i in range(10000):
+   5    150.0 MiB      0.0 MiB       10000           data.append(large_object(i))
+   6     50.0 MiB   -100.0 MiB           1       return data
+
+key metrics:
+  - Mem usage: total memory at that line
+  - Increment: memory change on that line
+  - Occurrences: how many times executed
+
+identify memory allocations by large increments.
+
+
+track memory over time
+
+  <terminal>mprof run main.py 2>&1</terminal>
+  <terminal>mprof plot --output /tmp/memory_plot.png</terminal>
+
+this shows memory usage timeline.
+
+
+find memory leaks
+
+run mprof for extended duration:
+  <terminal>mprof run --interval 0.1 --python python main.py 2>&1</terminal>
+  <terminal>mprof plot</terminal>
+
+look for:
+  - continuous memory growth
+  - memory not released after operations
+  - baseline increase over time
+
+
+compare memory before/after
+
+  <terminal>python -m memory_tracer --output /tmp/memory_trace.json main.py 2>&1</terminal>
+
+or use memory_profiler with timestamps.
+
+
+PHASE 6: PROFILING SPECIFIC PATTERNS
+
+
+profiling I/O bound operations
+
+  <terminal>grep -rn "open(.*r\|\.read(\|\.write(" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+I/O issues:
+  - many small file reads
+  - no buffering
+  - synchronous I/O in loops
+  - unnecessary file operations
+
+profile I/O specifically:
+  <terminal>python -m cProfile -s time main.py 2>&1 | grep -E "(read|write|open)"</terminal>
+
+
+profiling network operations
+
+  <terminal>grep -rn "requests\.\|urllib\|aiohttp\|httpx" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+network issues:
+  - no connection pooling
+  - requests in loops
+  - no timeouts configured
+  - sequential vs parallel requests
+
+identify network hotspots:
+  <terminal>python -c "import cProfile; import pstats; p = pstats.Stats('profile.stats'); p.print_stats(); p.sort_stats('cumulative'); p.print_stats('request|connect')" 2>&1</terminal>
+
+
+profiling database queries
+
+  <terminal>grep -rn "\.execute\|\.query\|\.fetchall\|\.fetchone" --include="*.py" . 2>/dev/null | head -30</terminal>
+
+database issues:
+  - N+1 query patterns
+  - missing indexes
+  - fetching too much data
+  - no query result caching
+  - queries in loops
+
+find N+1 patterns:
+  <terminal>grep -rn "for.*:" --include="*.py" . 2>/dev/null | xargs -I{} grep -l "execute\|query" {} 2>/dev/null | head -10</terminal>
+
+
+profiling data processing
+
+  <terminal>grep -rn "for.*in.*:\|while.*:" --include="*.py" . 2>/dev/null | head -30</terminal>
+
+data processing issues:
+  - nested loops (O(n^2) or worse)
+  - repeated calculations
+  - no memoization
+  - wrong data structure choice
+  - list instead of set for membership
+
+identify algorithmic complexity:
+  <read><file>path/to/algorithm.py</file></read>
+
+look for:
+  - nested loops
+  - repeated function calls in loops
+  - growing lists without preallocation
+
+
+profiling async code
+
+  <terminal>grep -rn "async def\|await " --include="*.py" . 2>/dev/null | head -30</terminal>
+
+async issues:
+  - not using await properly
+  - blocking calls in async functions
+  - sequential awaits instead of gather
+  - no async/await where beneficial
+
+profile async with cProfile:
+  <terminal>python -m cProfile -o async_profile.stats -m asyncio.run main() 2>&1</terminal>
+
+
+PHASE 7: SYSTEM RESOURCE PROFILING
+
+
+CPU profiling with py-spy
+
+  <terminal>pip install py-spy --quiet</terminal>
+
+spy on running process:
+  <terminal>py-spy top --pid $(pgrep -f "python main.py")</terminal>
+
+live flame graph:
+  <terminal>py-spy record --pid $(pgrep -f "python main.py") --output /tmp/py_spy.svg --duration 30</terminal>
+
+
+memory sampling with py-spy
+
+  <terminal>py-spy record --pid $(pgrep -f "python main.py") --output /tmp/py_spy_memory.svg --format memory --duration 30</terminal>
+
+
+system monitoring during profiling
+
+  <terminal>while true; do ps aux | grep python | head -5; sleep 1; done</terminal>
+
+track CPU and memory over time.
+
+
+using /usr/bin/time for detailed metrics
+
+  <terminal>/usr/bin/time -v python main.py 2>&1 | tee /tmp/detailed_time.txt</terminal>
+
+provides:
+  - maximum resident set size
+  - page faults
+  - context switches
+  - CPU usage percentage
+
+
+PHASE 8: PROFILING WEB APPLICATIONS
+
+
+profile Flask applications
+
+  <terminal>grep -rn "from flask import\|import flask" --include="*.py" . 2>/dev/null | head -5</terminal>
+
+if Flask exists:
+  <terminal>python -c "from app import app; app.run(profile=True)"</terminal>
+
+or use Werkzeug profiler:
+  <terminal>python -m werkzeug.serving --profile app:app</terminal>
+
+
+profile FastAPI applications
+
+  <terminal>grep -rn "from fastapi import\|import fastapi" --include="*.py" . 2>/dev/null | head -5</terminal>
+
+if FastAPI exists, use middleware:
+  <read><file>path/to/main.py</file></read>
+
+look for existing profiling middleware.
+
+
+profile Django applications
+
+  <terminal>grep -rn "from django import\|import django" --include="*.py" . 2>/dev/null | head -5</terminal>
+
+Django debug toolbar for profiling:
+  <terminal>grep -rn "debug_toolbar" --include="*.py" . 2>/dev/null</terminal>
+
+
+PHASE 9: VISUALIZING PROFILE DATA
+
+
+using snakeviz
+
+  <terminal>pip install snakeviz --quiet</terminal>
+
+view profile:
+  <terminal>snakeviz /tmp/profile.stats</terminal>
+
+opens interactive visualization in browser.
+
+
+using tuna
+
+  <terminal>pip install tuna --quiet</terminal>
+
+visualize:
+  <terminal>tuna /tmp/profile.stats</terminal>
+
+creates interactive icicle plot.
+
+
+creating flame graphs
+
+  <terminal>pip install flameprof --quiet</terminal>
+
+  <terminal>flameprof /tmp/profile.stats > /tmp/flamegraph.svg</terminal>
+
+view flamegraph in browser.
+
+
+PHASE 10: COMPARATIVE PROFILING
+
+
+compare before/after
+
+  <terminal>python -m cProfile -o before.stats main.py</terminal>
+  <terminal>python -m cProfile -o after.stats main_modified.py</terminal>
+
+compare:
+  <terminal>python -c "import pstats; p1 = pstats.Stats('before.stats'); p2 = pstats.Stats('after.stats'); print('Before:', p1.total_calls); print('After:', p2.total_calls)"</terminal>
+
+
+statistically significant measurements
+
+  <terminal>for i in {1..10}; do time python main.py; done 2>&1 | tee /tmp/timing_samples.txt</terminal>
+
+analyze:
+  - calculate mean
+  - calculate standard deviation
+  - identify outliers
+
+
+PHASE 11: INTERPRETING RESULTS
+
+
+classify performance issues
+
+category 1: algorithmic complexity
+  - O(n^2) or worse
+  - nested loops over large datasets
+  - repeated work without memoization
+
+category 2: I/O bound
+  - slow file operations
+  - network latency
+  - database queries
+  - disk I/O
+
+category 3: memory issues
+  - memory leaks
+  - excessive allocations
+  - large data structures
+  - garbage collection pauses
+
+category 4: concurrency issues
+  - underutilized CPU
+  - blocking operations
+  - lack of parallelization
+
+category 5: framework/overhead
+  - abstraction layers
+  - unnecessary serialization
+  - reflection/dynamic code
+
+
+identify quick wins
+
+common quick wins:
+  - caching computed values
+  - using set/list for O(1) lookup
+  - batch I/O operations
+  - pre-allocating lists
+  - using generators instead of lists
+
+
+identify complex fixes
+
+requires more thought:
+  - algorithm redesign
+  - data structure changes
+  - architecture modifications
+  - introducing concurrency
+
+
+PHASE 12: PERFORMANCE REPORT TEMPLATE
+
+
+performance analysis report template
+
+  executive summary:
+    - application: [name]
+    - date: [date]
+    - baseline performance: [metrics]
+    - primary bottleneck: [what]
+
+  methodology:
+    - profiling tools used
+    - workload description
+    - measurement approach
+    - limitations
+
+  findings:
+
+    bottleneck 1: [name]
+      category: [cpu|memory|i/o|network|algorithm]
+      severity: [critical|high|medium|low]
+      location:
+        file: [path]
+        function: [name]
+        lines: [range]
+
+      evidence:
+        - cProfile output showing hot function
+        - percentage of total time
+        - comparison to baseline
+
+      impact:
+        - current performance
+        - user experience impact
+        - resource cost
+
+      recommendations:
+        - specific optimization approach
+        - expected improvement
+        - complexity estimate
+
+    [repeat for each bottleneck...]
+
+  recommendations summary:
+    prioritized by impact/effort:
+      [1] quick wins with high impact
+      [2] medium effort, medium impact
+      [3] complex fixes for long-term benefit
+
+  appendix:
+    - full profile output
+    - charts/graphs
+    - detailed metrics
+
+
+PHASE 13: COMMON PERFORMANCE PATTERNS
+
+
+pattern 1: nested loops over large data
+
+  symptoms:
+    - cProfile shows high cumulative time in nested function
+    - O(n^2) complexity visible in code
+    - performance degrades quadratically with input size
+
+  detection:
+    <terminal>grep -rn "for.*in.*:" --include="*.py" -A2 . 2>/dev/null | grep "for.*in.*:" | head -20</terminal>
+
+  look for: for within for with large datasets.
+
+
+pattern 2: repeated expensive computation
+
+  symptoms:
+    - function called many times with same inputs
+    - high ncalls in cProfile
+    - result could be cached
+
+  detection:
+    <terminal>python -c "import pstats; p = pstats.Stats('/tmp/profile.stats'); p.sort_stats('calls').print_stats(20)"</terminal>
+
+  look for: same function name repeated with high ncalls.
+
+
+pattern 3: list membership on large lists
+
+  symptoms:
+    - "if item in large_list" with large list
+    - O(n) lookup repeated many times
+
+  detection:
+    <terminal>grep -rn "in.*\[.*\]" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+  look for: membership tests on list literals.
+
+
+pattern 4: string concatenation in loops
+
+  symptoms:
+    - building string with += in loop
+    - O(n^2) due to string immutability
+
+  detection:
+    <terminal>grep -rn "for.*:.*+=" --include="*.py" . 2>/dev/null | grep -E "(str|s\\b)\\s*\\+=" | head -20</terminal>
+
+  look for: string concatenation in loop bodies.
+
+
+pattern 5: unnecessary I/O
+
+  symptoms:
+    - file opened/closed repeatedly
+    - same file read multiple times
+    - many small read/write operations
+
+  detection:
+    <terminal>grep -rn "with open\|open(" --include="*.py" . 2>/dev/null | wc -l</terminal>
+    <terminal>grep -rn "for.*:.*open\|while.*:.*open" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+
+pattern 6: missing index hints
+
+  symptoms:
+    - database queries slow
+    - full table scans
+
+  detection:
+    <terminal>grep -rn "\.execute\|\.query" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+  examine query patterns for unindexed lookups.
+
+
+pattern 7: synchronous I/O blocking
+
+  symptoms:
+    - waiting for I/O
+    - no concurrency during I/O operations
+
+  detection:
+    <terminal>grep -rn "requests\.get\|urllib\.request\|urlopen" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+  look for: synchronous HTTP calls in potentially parallelizable code.
+
+
+pattern 8: large object copies
+
+  symptoms:
+    - unexpected memory allocations
+    - high memory usage
+
+  detection:
+    <terminal>grep -rn "\.copy(\|list(data)\|dict(data)" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+  look for: unnecessary copying of large data structures.
+
+
+pattern 9: inefficient data structure
+
+  symptoms:
+    - operation slower than expected
+    - wrong complexity for access pattern
+
+  detection:
+    <terminal>grep -rn "\\[.*\\].*\\[.*\\]" --include="*.py" . 2>/dev/null | head -20</terminal>
+
+  look for: nested list access that could be dict or set.
+
+
+pattern 10:过早优化
+
+  wait, that's the opposite problem. trust profiling, not intuition.
+
+
+PHASE 14: PROFILING CHECKLIST
+
+
+before profiling
+
+  [ ] have clear performance goal
+  [ ] establish baseline metrics
+  [ ] identify critical path
+  [ ] prepare reproducible workload
+  [ ] verify tools are installed
+
+
+during profiling
+
+  [ ] use consistent workload
+  [ ] run multiple iterations
+  [ ] profile realistic scenarios
+  [ ] measure both CPU and memory
+  [ ] record environment details
+
+
+analysis phase
+
+  [ ] identify top time consumers
+  [ ] identify memory allocations
+  [ ] classify bottlenecks by type
+  [ ] prioritize by impact
+  [ ] verify findings are reproducible
+
+
+reporting phase
+
+  [ ] document methodology
+  [ ] provide specific file/line references
+  [ ] include evidence (screenshots, logs)
+  [ ] classify severity
+  [ ] suggest remediation approaches
+
+
+PHASE 15: PERFORMANCE PROFILING RULES
+
+
+while this skill is active, these rules are MANDATORY:
+
+  [1] NEVER optimize without profiling first
+      intuition is often wrong about performance
+      measure, then optimize
+
+  [2] ALWAYS establish a baseline
+      you cannot improve what you cannot measure
+      record before/after metrics
+
+  [3] profile realistic workloads
+      synthetic benchmarks may mislead
+      use production-like data and scenarios
+
+  [4] focus on hot paths
+      80/20 rule: 20% of code accounts for 80% of time
+      optimize the critical path first
+
+  [5] measure twice, cut once
+      verify findings with multiple approaches
+      cross-check with different tools
+
+  [6] consider both time and space
+      fastest solution may use too much memory
+      balance trade-offs
+
+  [7] account for measurement overhead
+      profiling itself affects performance
+      understand tool overhead
+
+  [8] document findings thoroughly
+      file paths, line numbers, function names
+      include evidence and metrics
+
+  [9] prioritize by impact
+      what matters most to users?
+      what costs the most resources?
+
+  [10] recommend, don't implement
+      this is a research skill
+      provide detailed guidance for coder agent
+
+
+PHASE 16: WORKFLOW GUIDE
+
+
+step 1: understand the problem
+
+  [ ] what is the performance complaint?
+  [ ] what is acceptable performance?
+  [ ] when is the slowness noticed?
+  [ ] what are the usage patterns?
+
+
+step 2: establish baseline
+
+  [ ] measure current performance
+  [ ] identify critical path
+  [ ] document system specs
+  [ ] record resource usage
+
+
+step 3: profile
+
+  [ ] run cProfile for CPU analysis
+  [ ] run memory_profiler for memory
+  [ ] identify hot functions
+  [ ] identify memory allocations
+  [ ] classify bottleneck types
+
+
+step 4: analyze
+
+  [ ] review hot path code
+  [ ] identify root cause
+  [ ] research optimization approaches
+  [ ] estimate improvement potential
+
+
+step 5: report
+
+  [ ] document findings
+  [ ] provide specific recommendations
+  [ ] include before/after comparison potential
+  [ ] prioritize by impact/effort
+
+
+step 6: validate recommendations
+
+  [ ] ensure suggestions are actionable
+  [ ] verify approach is sound
+  [ ] check for side effects
+  [ ] estimate improvement range
+
+
+PHASE 17: INTERPRETING COMMON METRICS
+
+
+wall clock time (real)
+
+  what it measures: total elapsed time
+  what it indicates: user-perceived performance
+  factors: CPU + I/O + waiting
+
+
+user CPU time
+
+  what it measures: CPU time in user mode
+  what it indicates: application computation
+  factors: algorithmic complexity, code efficiency
+
+
+system CPU time
+
+  what it measures: CPU time in kernel mode
+  what it indicates: system calls, I/O
+  factors: file operations, network, context switches
+
+
+memory usage (RSS)
+
+  what it measures: resident set size
+  what it indicates: physical memory used
+  factors: data structures, allocations
+
+
+memory increment
+
+  what it measures: change in memory
+  what it indicates: allocation at specific point
+  factors: large objects, caching
+
+
+hit count
+
+  what it measures: how many times executed
+  what it indicates: loop iterations, function calls
+  factors: algorithm, input size
+
+
+PHASE 18: TOOL REFERENCE
+
+
+cProfile quick reference
+
+  basic profile:
+    python -m cProfile -o output.stats script.py
+
+  sorted output:
+    python -m cProfile -s cumulative script.py
+
+  interactive analysis:
+    python -m pstats output.stats
+
+  useful pstats commands:
+    stats [n]           # show top n
+    strip               # remove path
+    sort <metric>       # cumulative|time|calls
+    callers <name>      # who calls function
+    callees <name>      # what function calls
+
+
+line_profiler quick reference
+
+  add @profile decorator to function
+  run: kernprof -l -v script.py
+
+  or use module:
+    python -m line_profiler script.py
+
+
+memory_profiler quick reference
+
+  add @profile decorator
+  run: python -m memory_profiler script.py
+
+  mprof commands:
+    mprof run script.py
+    mprof plot
+    mprof clean
+
+
+py-spy quick reference
+
+  sudo py-spy top --pid <pid>
+  sudo py-spy record --pid <pid> --output output.svg
+  sudo py-spy dump --pid <pid>
+
+
+FINAL REMINDERS
+
+
+profiling reveals truth
+
+your code is not slow where you think it is.
+only data can tell you where to focus.
+trust the profiler, not your intuition.
+
+
+performance is a feature
+
+slow applications frustrate users.
+efficient applications scale better.
+your analysis enables better software.
+
+
+measure what matters
+
+profile realistic scenarios.
+measure user-visible latency.
+optimize the critical path.
+
+find the bottlenecks before the users do.