@luquimbo/bi-superpowers 1.0.0

package/library/examples/hr-analytics/README.md

@@ -0,0 +1,371 @@
+# HR Analytics - Complete Example
+
+A comprehensive HR analytics solution for workforce planning, turnover analysis, and employee metrics.
+
+---
+
+## Overview
+
+This example demonstrates building a complete HR analytics solution using:
+
+- **Employee-centric data model** with demographics, positions, and history
+- **DAX measures** for headcount, turnover, tenure, and diversity metrics
+- **Time-based analysis** for trend tracking and forecasting
+- **Organizational hierarchy** support for drill-down reporting
+
+---
+
+## Business Requirements
+
+### Stakeholders
+- CHRO: Workforce strategy and executive metrics
+- HR Business Partners: Department-level insights
+- Talent Acquisition: Hiring pipeline and time-to-fill
+- Compensation Team: Pay equity and benchmarking
+- People Analytics: Deep-dive analysis
+
+### Key Questions
+1. What is our current headcount and how is it trending?
+2. What's our turnover rate and what's driving attrition?
+3. How diverse is our workforce across levels?
+4. What's our average tenure and where are the risks?
+5. How effective is our recruiting pipeline?
+
+---
+
+## Data Model
+
+### Star Schema Design
+
+```
+                    ┌─────────────────┐
+                    │   DimDate       │
+                    │─────────────────│
+                    │ DateKey (PK)    │
+                    │ Date            │
+                    │ Year            │
+                    │ Month           │
+                    │ Quarter         │
+                    └────────┬────────┘
+                             │
+┌─────────────────┐          │          ┌─────────────────┐
+│  DimEmployee    │          │          │  DimDepartment  │
+│─────────────────│          │          │─────────────────│
+│ EmployeeKey(PK) │          │          │ DepartmentKey   │
+│ EmployeeID      │          │          │ DepartmentName  │
+│ FirstName       │          │          │ Division        │
+│ LastName        │          │          │ CostCenter      │
+│ Gender          │          │          │ Manager         │
+│ Ethnicity       │          │          └────────┬────────┘
+│ BirthDate       │          │                   │
+│ HireDate        │          │                   │
+│ TermDate        │     ┌────┴─────────────────┐ │
+│ IsActive        │     │   FactEmployeeSnap   │ │
+└────────┬────────┘     │──────────────────────│ │
+         │              │ SnapshotKey (PK)     │ │
+         └─────────────>│ DateKey (FK)         │<┘
+                        │ EmployeeKey (FK)     │
+                        │ DepartmentKey (FK)   │
+                        │ JobKey (FK)          │
+                        │ LocationKey (FK)     │
+                        │ Salary               │
+                        │ Bonus                │
+                        │ PerformanceRating    │
+                        │ EngagementScore      │
+                        └──────────┬───────────┘
+                                   │
+                    ┌──────────────┴──────────────┐
+                    │                             │
+           ┌────────┴────────┐           ┌───────┴────────┐
+           │    DimJob       │           │  DimLocation   │
+           │─────────────────│           │────────────────│
+           │ JobKey (PK)     │           │ LocationKey    │
+           │ JobTitle        │           │ LocationName   │
+           │ JobFamily       │           │ City           │
+           │ JobLevel        │           │ State          │
+           │ IsManagement    │           │ Country        │
+           │ PayGrade        │           │ Region         │
+           └─────────────────┘           └────────────────┘
+```
+
+### Key Design Decisions
+
+1. **Snapshot Fact Table** - Monthly snapshots capture point-in-time state
+2. **Employee Dimension** - Contains hire/term dates for tenure calculations
+3. **Slowly Changing Dimensions** - Track job and department changes over time
+4. **Separate Job Dimension** - Enables job family and level analysis
+
+---
+
+## Key Measures
+
+### Headcount Metrics
+
+```dax
+// Current active headcount
+Headcount =
+CALCULATE(
+    DISTINCTCOUNT(FactEmployeeSnap[EmployeeKey]),
+    DimEmployee[IsActive] = TRUE()
+)
+
+// Headcount at end of period
+Headcount EOP =
+CALCULATE(
+    [Headcount],
+    LASTDATE(DimDate[Date])
+)
+
+// Average headcount for period
+Headcount Avg =
+AVERAGEX(
+    VALUES(DimDate[Date]),
+    [Headcount EOP]
+)
+
+// Full-Time Equivalent (if tracking FTE)
+FTE =
+CALCULATE(
+    SUM(FactEmployeeSnap[FTEFactor]),
+    DimEmployee[IsActive] = TRUE()
+)
+```
+
+### Turnover Metrics
+
+```dax
+// Terminations in period
+Terminations =
+CALCULATE(
+    COUNTROWS(DimEmployee),
+    DimEmployee[TermDate] <> BLANK(),
+    DimEmployee[TermDate] >= MIN(DimDate[Date]),
+    DimEmployee[TermDate] <= MAX(DimDate[Date])
+)
+
+// Voluntary terminations
+Voluntary Terms =
+CALCULATE(
+    [Terminations],
+    DimEmployee[TermType] = "Voluntary"
+)
+
+// Turnover Rate (annualized)
+Turnover Rate =
+VAR _Terms = [Terminations]
+VAR _AvgHC = [Headcount Avg]
+VAR _Months = DISTINCTCOUNT(DimDate[YearMonth])
+RETURN
+    DIVIDE(_Terms, _AvgHC) * (12 / _Months)
+
+// Voluntary Turnover Rate
+Voluntary Turnover Rate =
+VAR _VolTerms = [Voluntary Terms]
+VAR _AvgHC = [Headcount Avg]
+VAR _Months = DISTINCTCOUNT(DimDate[YearMonth])
+RETURN
+    DIVIDE(_VolTerms, _AvgHC) * (12 / _Months)
+
+// Retention Rate
+Retention Rate =
+1 - [Turnover Rate]
+```
+
+### Hiring Metrics
+
+```dax
+// New hires in period
+New Hires =
+CALCULATE(
+    COUNTROWS(DimEmployee),
+    DimEmployee[HireDate] >= MIN(DimDate[Date]),
+    DimEmployee[HireDate] <= MAX(DimDate[Date])
+)
+
+// Net change in headcount
+Net Headcount Change =
+[New Hires] - [Terminations]
+
+// Hiring Rate
+Hiring Rate =
+DIVIDE([New Hires], [Headcount Avg])
+
+// 90-Day Turnover (early attrition)
+90 Day Turnover =
+CALCULATE(
+    COUNTROWS(DimEmployee),
+    DimEmployee[TermDate] <> BLANK(),
+    DimEmployee[TermDate] >= MIN(DimDate[Date]),
+    DimEmployee[TermDate] <= MAX(DimDate[Date]),
+    DATEDIFF(DimEmployee[HireDate], DimEmployee[TermDate], DAY) <= 90
+)
+```
+
+### Tenure Metrics
+
+```dax
+// Average tenure (years)
+Avg Tenure =
+AVERAGEX(
+    FILTER(
+        DimEmployee,
+        DimEmployee[IsActive] = TRUE()
+    ),
+    DATEDIFF(DimEmployee[HireDate], TODAY(), YEAR)
+)
+
+// Tenure distribution
+Tenure Bucket =
+SWITCH(
+    TRUE(),
+    [Tenure Years] < 1, "< 1 Year",
+    [Tenure Years] < 2, "1-2 Years",
+    [Tenure Years] < 5, "2-5 Years",
+    [Tenure Years] < 10, "5-10 Years",
+    ">= 10 Years"
+)
+
+// Flight risk (high performers with long tenure)
+Flight Risk Count =
+CALCULATE(
+    [Headcount],
+    FactEmployeeSnap[PerformanceRating] >= 4,
+    DimEmployee[TenureYears] >= 5,
+    FactEmployeeSnap[EngagementScore] < 3.5
+)
+```
+
+### Diversity Metrics
+
+```dax
+// Gender diversity
+Female % =
+DIVIDE(
+    CALCULATE([Headcount], DimEmployee[Gender] = "Female"),
+    [Headcount]
+)
+
+// Gender diversity in management
+Female Managers % =
+DIVIDE(
+    CALCULATE([Headcount], DimEmployee[Gender] = "Female", DimJob[IsManagement] = TRUE()),
+    CALCULATE([Headcount], DimJob[IsManagement] = TRUE())
+)
+
+// Diversity index (simplified)
+Diversity Index =
+VAR _Total = [Headcount]
+VAR _Groups =
+    ADDCOLUMNS(
+        VALUES(DimEmployee[Ethnicity]),
+        "@Count", [Headcount]
+    )
+VAR _SumSquares =
+    SUMX(_Groups, POWER(DIVIDE([@Count], _Total), 2))
+RETURN
+    1 - _SumSquares  // Higher = more diverse
+```
+
+See [measures.dax](./measures.dax) for the complete measure library (40+ measures).
+
+---
+
+## Report Pages
+
+### 1. Executive Dashboard
+
+| Visual | Content |
+|--------|---------|
+| KPI Cards | Headcount, Turnover Rate, Avg Tenure, Diversity % |
+| Line Chart | Headcount trend over 12 months |
+| Stacked Bar | Hires vs Terms by month |
+| Donut | Headcount by Division |
+
+### 2. Turnover Analysis
+
+| Visual | Content |
+|--------|---------|
+| KPI Cards | Turnover Rate, Voluntary %, Regrettable % |
+| Waterfall | Headcount changes breakdown |
+| Bar Chart | Turnover by Department |
+| Table | Top term reasons with counts |
+| Heat Map | Turnover by Tenure × Job Level |
+
+### 3. Workforce Composition
+
+| Visual | Content |
+|--------|---------|
+| Tree Map | Headcount by Division > Department |
+| Bar Chart | Headcount by Job Level |
+| Stacked Bar | Demographics breakdown |
+| Map | Headcount by Location |
+
+### 4. Tenure & Risk
+
+| Visual | Content |
+|--------|---------|
+| Histogram | Tenure distribution |
+| Scatter | Tenure vs Performance |
+| Table | Flight risk employees |
+| Line Chart | Avg tenure trend |
+
+### 5. Compensation Analysis
+
+| Visual | Content |
+|--------|---------|
+| Box Plot | Salary distribution by level |
+| Scatter | Salary vs Performance |
+| Table | Compa-ratio by job family |
+| Bar | Pay equity by gender |
+
+---
+
+## Implementation Checklist
+
+### Data Model
+- [ ] Create DimDate with calendar attributes
+- [ ] Create DimEmployee with demographics and dates
+- [ ] Create DimDepartment with hierarchy
+- [ ] Create DimJob with job family/level
+- [ ] Create DimLocation with geography
+- [ ] Load FactEmployeeSnap (monthly snapshots)
+- [ ] Set up relationships
+- [ ] Mark DimDate as Date Table
+
+### Measures
+- [ ] Create headcount measures
+- [ ] Create turnover/retention measures
+- [ ] Create hiring measures
+- [ ] Create tenure measures
+- [ ] Create diversity measures
+- [ ] Create compensation measures
+- [ ] Organize in display folders
+
+### Reports
+- [ ] Executive Dashboard
+- [ ] Turnover Analysis with drill-through
+- [ ] Workforce Composition
+- [ ] Tenure & Risk Analysis
+- [ ] Compensation Analysis
+- [ ] Apply theme and RLS
+
+---
+
+## Files in This Example
+
+| File | Description |
+|------|-------------|
+| [README.md](./README.md) | This overview |
+| [data-model.md](./data-model.md) | Complete table definitions |
+| [measures.dax](./measures.dax) | 40+ DAX measures |
+
+---
+
+## HR Analytics Best Practices
+
+1. **Use snapshots** - Point-in-time data is critical for trend analysis
+2. **Annualize rates** - Turnover should always be annualized for comparison
+3. **Segment analysis** - Always allow drill-down by org hierarchy
+4. **Protect PII** - Use RLS and hide sensitive columns
+5. **Define "turnover"** - Distinguish voluntary, involuntary, regrettable
+6. **Benchmark** - Industry benchmarks add context to your metrics

package/library/examples/hr-analytics/data-model.md

@@ -0,0 +1,315 @@
+# HR Analytics - Data Model
+
+Complete table definitions for the HR analytics star schema.
+
+---
+
+## Fact Table
+
+### FactEmployeeSnap
+
+Monthly snapshot of employee state - one row per employee per month.
+
+| Column | Data Type | Description |
+|--------|-----------|-------------|
+| SnapshotKey | Integer | Surrogate primary key |
+| DateKey | Text | FK to DimDate (YYYYMM01 - first of month) |
+| EmployeeKey | Integer | FK to DimEmployee |
+| DepartmentKey | Integer | FK to DimDepartment |
+| JobKey | Integer | FK to DimJob |
+| LocationKey | Integer | FK to DimLocation |
+| ManagerKey | Integer | FK to DimEmployee (manager) |
+| Salary | Currency | Annual base salary |
+| Bonus | Currency | Annual bonus target |
+| FTEFactor | Decimal | Full-time equivalent (1.0 = full-time) |
+| PerformanceRating | Integer | Last performance rating (1-5) |
+| EngagementScore | Decimal | Last engagement survey score (1-5) |
+| CompaRatio | Decimal | Salary / midpoint of pay range |
+
+**Grain:** One row per Employee per Month (end of month snapshot).
+
+**Relationships:**
+- FactEmployeeSnap[DateKey] → DimDate[DateKey] (Many:1)
+- FactEmployeeSnap[EmployeeKey] → DimEmployee[EmployeeKey] (Many:1)
+- FactEmployeeSnap[DepartmentKey] → DimDepartment[DepartmentKey] (Many:1)
+- FactEmployeeSnap[JobKey] → DimJob[JobKey] (Many:1)
+- FactEmployeeSnap[LocationKey] → DimLocation[LocationKey] (Many:1)
+- FactEmployeeSnap[ManagerKey] → DimEmployee[EmployeeKey] (Many:1, inactive)
+
+---
+
+## Dimension Tables
+
+### DimEmployee
+
+| Column | Data Type | Description | Example |
+|--------|-----------|-------------|---------|
+| EmployeeKey | Integer | Surrogate PK | 10001 |
+| EmployeeID | Text | Business key (badge #) | "E-12345" |
+| FirstName | Text | First name | "John" |
+| LastName | Text | Last name | "Smith" |
+| FullName | Text | Display name | "Smith, John" |
+| Email | Text | Work email | "jsmith@company.com" |
+| Gender | Text | Gender identity | "Male" |
+| Ethnicity | Text | Self-identified ethnicity | "Asian" |
+| BirthDate | Date | Date of birth | 1985-06-15 |
+| HireDate | Date | Original hire date | 2018-03-01 |
+| TermDate | Date | Termination date (null if active) | NULL |
+| TermType | Text | Termination type | "Voluntary" |
+| TermReason | Text | Termination reason | "Career Opportunity" |
+| IsActive | Boolean | Currently employed | TRUE |
+| AgeYears | Integer | Current age (calculated) | 39 |
+| TenureYears | Decimal | Years of service (calculated) | 6.8 |
+| TenureBucket | Text | Tenure range | "5-10 Years" |
+| Generation | Text | Generational cohort | "Millennial" |
+
+**Termination Types:**
+- Voluntary: Resignation, retirement
+- Involuntary: Performance, restructuring, misconduct
+- Other: Death, disability
+
+**Calculated Columns:**
+
+```dax
+// Age
+AgeYears =
+DATEDIFF(DimEmployee[BirthDate], TODAY(), YEAR)
+
+// Tenure
+TenureYears =
+VAR _EndDate = IF(ISBLANK(DimEmployee[TermDate]), TODAY(), DimEmployee[TermDate])
+RETURN
+    DATEDIFF(DimEmployee[HireDate], _EndDate, DAY) / 365.25
+
+// Tenure Bucket
+TenureBucket =
+SWITCH(
+    TRUE(),
+    DimEmployee[TenureYears] < 1, "< 1 Year",
+    DimEmployee[TenureYears] < 2, "1-2 Years",
+    DimEmployee[TenureYears] < 5, "2-5 Years",
+    DimEmployee[TenureYears] < 10, "5-10 Years",
+    ">= 10 Years"
+)
+
+// Generation
+Generation =
+VAR _BirthYear = YEAR(DimEmployee[BirthDate])
+RETURN
+    SWITCH(
+        TRUE(),
+        _BirthYear >= 1997, "Gen Z",
+        _BirthYear >= 1981, "Millennial",
+        _BirthYear >= 1965, "Gen X",
+        _BirthYear >= 1946, "Boomer",
+        "Silent"
+    )
+```
+
+---
+
+### DimDepartment
+
+| Column | Data Type | Description | Example |
+|--------|-----------|-------------|---------|
+| DepartmentKey | Integer | Surrogate PK | 201 |
+| DepartmentCode | Text | Business key | "DEPT-ENG-001" |
+| DepartmentName | Text | Department name | "Software Engineering" |
+| Division | Text | Division | "Technology" |
+| BusinessUnit | Text | Business unit | "Product Development" |
+| CostCenter | Text | Cost center code | "CC-4500" |
+| DepartmentHead | Text | Department leader | "Jane Doe" |
+| ParentDepartmentKey | Integer | For hierarchy | 200 |
+| Level | Integer | Hierarchy level | 3 |
+| IsActive | Boolean | Currently active | TRUE |
+
+**Hierarchies:**
+- Division > BusinessUnit > DepartmentName
+
+---
+
+### DimJob
+
+| Column | Data Type | Description | Example |
+|--------|-----------|-------------|---------|
+| JobKey | Integer | Surrogate PK | 301 |
+| JobCode | Text | Job code | "JOB-SWE-003" |
+| JobTitle | Text | Job title | "Senior Software Engineer" |
+| JobFamily | Text | Job family | "Engineering" |
+| JobSubFamily | Text | Sub-family | "Software Development" |
+| JobLevel | Integer | Level (1-10) | 6 |
+| JobLevelName | Text | Level name | "Senior Individual Contributor" |
+| IsManagement | Boolean | People manager | FALSE |
+| IsExempt | Boolean | FLSA exempt | TRUE |
+| PayGrade | Text | Pay grade | "G6" |
+| PayRangeMin | Currency | Grade minimum | 120000 |
+| PayRangeMid | Currency | Grade midpoint | 150000 |
+| PayRangeMax | Currency | Grade maximum | 180000 |
+
+**Job Levels:**
+
+| Level | LevelName | IsManagement |
+|-------|-----------|--------------|
+| 1-3 | Entry Level | FALSE |
+| 4-5 | Mid Level | FALSE |
+| 6-7 | Senior Individual Contributor | FALSE |
+| 5 | Team Lead | TRUE |
+| 6 | Manager | TRUE |
+| 7 | Senior Manager | TRUE |
+| 8 | Director | TRUE |
+| 9 | VP | TRUE |
+| 10 | C-Suite | TRUE |
+
+---
+
+### DimLocation
+
+| Column | Data Type | Description | Example |
+|--------|-----------|-------------|---------|
+| LocationKey | Integer | Surrogate PK | 101 |
+| LocationCode | Text | Location code | "LOC-SF-001" |
+| LocationName | Text | Location name | "San Francisco HQ" |
+| Address | Text | Street address | "100 Main Street" |
+| City | Text | City | "San Francisco" |
+| State | Text | State/Province | "CA" |
+| Country | Text | Country | "USA" |
+| Region | Text | Geographic region | "North America" |
+| Latitude | Decimal | Latitude | 37.7749 |
+| Longitude | Decimal | Longitude | -122.4194 |
+| IsRemote | Boolean | Remote/virtual location | FALSE |
+| Timezone | Text | Timezone | "America/Los_Angeles" |
+
+---
+
+### DimDate
+
+| Column | Data Type | Description | Example |
+|--------|-----------|-------------|---------|
+| DateKey | Text | Primary key (YYYYMMDD) | "20240131" |
+| Date | Date | Full date | 2024-01-31 |
+| Year | Integer | Calendar year | 2024 |
+| Quarter | Text | Quarter | "Q1" |
+| Month | Integer | Month number | 1 |
+| MonthName | Text | Month name | "January" |
+| MonthYear | Text | Month-Year label | "Jan 2024" |
+| YearMonth | Text | Year-Month (sortable) | "2024-01" |
+| WeekOfYear | Integer | Week number | 5 |
+| DayOfWeek | Integer | Day of week (1=Mon) | 3 |
+| IsWeekend | Boolean | Weekend flag | FALSE |
+| IsMonthEnd | Boolean | Last day of month | TRUE |
+| IsQuarterEnd | Boolean | Last day of quarter | FALSE |
+| IsYearEnd | Boolean | Last day of year | FALSE |
+
+---
+
+## Sample Data
+
+### DimEmployee (Selected Rows)
+
+| EmployeeKey | EmployeeID | FullName | Gender | HireDate | TermDate | IsActive |
+|-------------|------------|----------|--------|----------|----------|----------|
+| 10001 | E-001 | Smith, John | Male | 2018-03-01 | NULL | TRUE |
+| 10002 | E-002 | Johnson, Sarah | Female | 2019-06-15 | NULL | TRUE |
+| 10003 | E-003 | Williams, Mike | Male | 2017-01-10 | 2023-08-31 | FALSE |
+| 10004 | E-004 | Brown, Emily | Female | 2021-09-01 | NULL | TRUE |
+| 10005 | E-005 | Davis, Alex | Non-Binary | 2022-02-14 | NULL | TRUE |
+
+### FactEmployeeSnap (Selected Rows)
+
+| SnapshotKey | DateKey | EmployeeKey | DepartmentKey | JobKey | Salary | PerformanceRating |
+|-------------|---------|-------------|---------------|--------|--------|-------------------|
+| 1 | 20240101 | 10001 | 201 | 301 | 145000 | 4 |
+| 2 | 20240101 | 10002 | 202 | 305 | 125000 | 5 |
+| 3 | 20240101 | 10004 | 201 | 302 | 95000 | 3 |
+| 4 | 20240201 | 10001 | 201 | 301 | 145000 | 4 |
+| 5 | 20240201 | 10002 | 202 | 305 | 130000 | 5 |
+
+---
+
+## Relationship Diagram
+
+```
+DimDate[DateKey] ──────────────────────────────────────┐
+                                                       │
+DimEmployee[EmployeeKey] ──────────────────────────┐   │
+                                                   │   │
+DimDepartment[DepartmentKey] ────────────────────┐ │   │
+                                                 │ │   │
+DimJob[JobKey] ────────────────────────────────┐ │ │   │
+                                               │ │ │   │
+DimLocation[LocationKey] ────────────────────┐ │ │ │   │
+                                             │ │ │ │   │
+                                             ▼ ▼ ▼ ▼   ▼
+                                         ┌──────────────┐
+                                         │FactEmployee  │
+                                         │    Snap      │
+                                         └──────────────┘
+
+All relationships: Many-to-One, Single direction
+ManagerKey relationship: Inactive (activate with USERELATIONSHIP)
+```
+
+---
+
+## Data Security (RLS)
+
+### Row-Level Security Rules
+
+```dax
+// HR Business Partners see their divisions only
+[Division] = USERPRINCIPALNAME() HRBP Division Lookup
+
+// Managers see their direct reports only
+[ManagerKey] IN
+    CALCULATETABLE(
+        VALUES(FactEmployeeSnap[EmployeeKey]),
+        DimEmployee[Email] = USERPRINCIPALNAME()
+    )
+
+// Sensitive columns hidden for non-HR users
+// - Salary, Bonus, PerformanceRating, EngagementScore
+```
+
+---
+
+## Power Query - Loading Employee Snapshots
+
+```m
+let
+    // Get current month end
+    CurrentMonthEnd = Date.EndOfMonth(DateTime.LocalNow()),
+
+    // Load from HRIS
+    Source = Sql.Database("hris-server", "HRDatabase"),
+    Employees = Source{[Schema="dbo", Item="vw_EmployeeSnapshot"]}[Data],
+
+    // Filter to active or recently termed
+    Filtered = Table.SelectRows(Employees, each
+        [TermDate] = null or [TermDate] >= Date.AddMonths(CurrentMonthEnd, -24)
+    ),
+
+    // Add snapshot date key
+    WithDateKey = Table.AddColumn(Filtered, "DateKey",
+        each Date.ToText(Date.StartOfMonth([SnapshotDate]), "yyyyMMdd")
+    ),
+
+    // Add calculated tenure
+    WithTenure = Table.AddColumn(WithDateKey, "TenureYears",
+        each Duration.TotalDays(
+            (if [TermDate] = null then DateTime.LocalNow() else [TermDate]) - [HireDate]
+        ) / 365.25
+    )
+in
+    WithTenure
+```
+
+---
+
+## Performance Tips
+
+1. **Monthly snapshots only** - Don't store daily; monthly is sufficient for HR
+2. **Pre-calculate tenure** - Avoid complex DATEDIFF in measures
+3. **Use integer keys** - Faster than employee IDs for joins
+4. **Partition by year** - Enable incremental refresh
+5. **Apply RLS early** - Filter before aggregation for performance
+6. **Limit history** - 2-3 years of snapshots is typically enough