orangeslice 1.7.2 → 1.7.3

package/docs/b2b-docs/B2B_NORMALIZED_VS_DENORMALIZED.md

@@ -0,0 +1,952 @@
+# B2B Database: Normalized vs Denormalized Table Selection Guide
+
+Comprehensive test results and decision matrix for choosing between normalized tables (`linkedin_profile`, `linkedin_company`) and denormalized views (`lkd_profile`, `lkd_company`).
+
+---
+
+## Executive Summary
+
+### Profile Tables (`linkedin_profile` vs `lkd_profile`)
+
+| Scenario                                                         | Winner       | Speed Advantage  |
+| ---------------------------------------------------------------- | ------------ | ---------------- |
+| **Simple PK/slug lookups**                                       | Normalized   | 3-6x faster      |
+| **Simple text/ILIKE (common terms)**                             | Normalized   | 2-20x faster     |
+| **Batch PK lookups (≤20 IDs)**                                   | Normalized   | 3x faster        |
+| **Large result sets (5000+ rows)**                               | Normalized   | 10-17x faster    |
+| **Indexed filters (updated_at, jobs_count, industry_id)**        | Normalized   | 3-8x faster      |
+| **Org/company name search (GIN indexed)**                        | Normalized   | 20x faster       |
+| **Country filter + headline search**                             | Denormalized | Works vs TIMEOUT |
+| **Multi-filter combinations (2+ filters)**                       | Denormalized | 2-3x faster      |
+| **Rare/uncommon headline terms**                                 | Denormalized | 2-2.5x faster    |
+| **Multiple skills array searches**                               | Denormalized | 2-3x faster      |
+| **Numeric + text combos (follower + headline)**                  | Denormalized | 1.7-2.5x faster  |
+| **Complex regex patterns**                                       | Denormalized | 1.4-2x faster    |
+| **Summary text searches**                                        | Denormalized | 1.6x faster      |
+| **Name searches (first + last)**                                 | Denormalized | 1.5x faster      |
+| **Getting enriched nested data (seniority, job_function, etc.)** | Denormalized | Only option      |
+
+### Company Tables (`linkedin_company` vs `lkd_company`) - Updated Jan 2026
+
+| Scenario                                 | Winner       | Speed Advantage           |
+| ---------------------------------------- | ------------ | ------------------------- |
+| **Slug lookups via `key64()`**           | Normalized   | 5-12x faster (4ms)        |
+| **Domain/ticker/universal_name lookups** | Normalized   | Only option (indexed)     |
+| **Single country filter**                | Normalized   | 20x faster (9ms vs 228ms) |
+| **Industry via industry_code**           | Normalized   | 136x faster (2ms)         |
+| **Nested data via JOINs (1 ID)**         | Normalized   | 7-19x faster              |
+| **Aggregations (GROUP BY)**              | Normalized   | Works vs TIMEOUT          |
+| **Simple slug lookup (no key64)**        | Denormalized | Only option (21-47ms)     |
+| **Compound queries (2+ filters)**        | Denormalized | 1.5-2.4x faster           |
+| **Funding + employee compound** ⚠️       | Denormalized | **17x faster** (70ms)     |
+| **Country + description combinations**   | Denormalized | 2.4x faster               |
+| **Description ILIKE (rare terms)**       | Denormalized | 1.5x faster               |
+| **Specialties filtering**                | Denormalized | 2.2x faster               |
+| **Pre-formatted nested JSON**            | Denormalized | Convenience               |
+
+**Key Insights** (from 40+ comprehensive tests):
+
+1. **Text Selectivity Crossover**: Common terms (CEO, engineer) favor normalized (2.4-5.7x faster); rare terms (kubernetes, blockchain) favor denormalized (2-2.8x faster)
+2. **Filter Count Scaling**: 1 filter → normalized usually wins; 3+ filters → denormalized wins (2-4x faster)
+3. **Filter Location**: Text filter on company side → normalized (company table smaller); text filter on profile side → denormalized (profile table huge)
+4. **Cross-Table Pattern**: Profile text + company constraint → denormalized (13-93x faster, normalized often times out)
+5. **Companies**: Use `key64()` for slug lookups; `lkd_company` lacks domain/ticker columns
+
+See [B2B_GENERALIZATION_RULES.md](./B2B_GENERALIZATION_RULES.md) for the complete decision matrix.
+
+---
+
+## Critical Finding: Country Filtering
+
+The most significant discovery is that **country filtering behaves completely differently** between the two tables:
+
+| Query Pattern                                      | `linkedin_profile` | `lkd_profile` | Result                  |
+| -------------------------------------------------- | ------------------ | ------------- | ----------------------- |
+| `location_country_code = 'US'` + headline ILIKE    | **TIMEOUT (30s+)** | -             | Fails                   |
+| `country_iso = 'US'` + headline ILIKE              | -                  | **16-22s**    | Works                   |
+| `location_country_code = 'US'` + common term (CTO) | 47ms               | 104ms         | Both work               |
+| `country_iso = 'US'` + rare term (iOS developer)   | TIMEOUT            | **16s**       | Only denormalized works |
+
+**Rule**: For country-filtered searches with uncommon headline terms, you **must** use `lkd_profile` with `country_iso`.
+
+```sql
+-- ✅ WORKS: US-based iOS developers
+SELECT profile_id, first_name, headline, locality
+FROM lkd_profile
+WHERE country_iso = 'US'
+  AND headline ~* '(\miOS\M|\mSwift\M).*(developer|engineer)'
+LIMIT 100
+
+-- ❌ TIMEOUT: Same query on normalized table
+SELECT id, first_name, headline, location_name
+FROM linkedin_profile
+WHERE location_country_code = 'US'
+  AND headline ILIKE '%iOS developer%'
+LIMIT 100
+```
+
+---
+
+## Compound Query Performance
+
+The key pattern: **compound queries scale better on denormalized views**.
+
+| # of Filters        | Normalized | Denormalized | Winner              |
+| ------------------- | ---------- | ------------ | ------------------- |
+| 1 filter (indexed)  | **Faster** | Slower       | Normalized          |
+| 1 filter (headline) | Faster     | Slower       | Normalized          |
+| 2 filters           | Similar    | Similar      | Tie                 |
+| 3+ filters          | **Slower** | **Faster**   | Denormalized (2-3x) |
+
+### Example: Triple Filter Performance
+
+```sql
+-- lkd_profile: 5.8s ✅
+SELECT first_name, headline, connection_count
+FROM lkd_profile
+WHERE connection_count > 500
+  AND 'Python' = ANY(skills)
+  AND headline ILIKE '%engineer%'
+LIMIT 100
+
+-- linkedin_profile: 18.2s (3.1x slower)
+SELECT first_name, headline, connections
+FROM linkedin_profile
+WHERE connections > 500
+  AND 'Python' = ANY(skills)
+  AND headline ILIKE '%engineer%'
+LIMIT 100
+```
+
+---
+
+## Test Results: Profile Queries
+
+### 1. Primary Key Lookups
+
+| Query Type              | Normalized | Denormalized | Winner            |
+| ----------------------- | ---------- | ------------ | ----------------- |
+| Single ID lookup        | **0.12ms** | 0.66ms       | Normalized (5.5x) |
+| Batch 5 IDs             | **1.84ms** | 1.15ms       | Tie               |
+| Batch 20 IDs            | **3.46ms** | 11.44ms      | Normalized (3.3x) |
+| Slug lookup (with JOIN) | **0.18ms** | timeout      | Normalized        |
+
+```sql
+-- RECOMMENDED: Normalized PK lookup
+SELECT id, formatted_name, title, org, headline
+FROM linkedin_profile WHERE id = ?
+
+-- AVOID: Denormalized for simple lookups
+SELECT profile_id, name, title, company_name, headline
+FROM lkd_profile WHERE profile_id = ?
+```
+
+### 2. Text/ILIKE Searches
+
+| Query Type                         | Normalized | Denormalized | Winner            |
+| ---------------------------------- | ---------- | ------------ | ----------------- |
+| Title ILIKE (LIMIT 10)             | **24.5ms** | 66.7ms       | Normalized (2.7x) |
+| Headline multi-term                | **99.2ms** | 116.4ms      | Normalized (1.2x) |
+| Org/company_name ILIKE             | **86.7ms** | 1,710ms      | Normalized (20x)  |
+| Compound filter (location + title) | timeout    | 283.7ms      | Denormalized\*    |
+
+\*Note: Normalized query timed out on this particular compound filter
+
+```sql
+-- RECOMMENDED: Normalized text search
+SELECT id, formatted_name, title
+FROM linkedin_profile
+WHERE title ILIKE '%software engineer%' LIMIT 10
+
+-- AVOID for search: Denormalized
+SELECT profile_id, name, title
+FROM lkd_profile
+WHERE title ILIKE '%software engineer%' LIMIT 10
+```
+
+### 3. Getting Nested Data (Experience, Education)
+
+| Query Type                                 | Normalized (with JOINs) | Denormalized | Winner              |
+| ------------------------------------------ | ----------------------- | ------------ | ------------------- |
+| Profile + positions (1 ID)                 | **0.33ms**              | 6.42ms       | Normalized (19x)    |
+| Profile + education (1 ID)                 | **1.22ms**              | 0.66ms       | Denormalized (1.8x) |
+| Profile + positions + education (1 ID)     | **0.27ms**              | 4.58ms       | Normalized (17x)    |
+| Batch 5 IDs + positions                    | **0.85ms**              | 6.36ms       | Normalized (7.5x)   |
+| Full profile (positions, education, certs) | **0.32ms**              | 6.28ms       | Normalized (20x)    |
+
+```sql
+-- RECOMMENDED: Normalized with JOINs for nested data
+SELECT lp.id, lp.formatted_name,
+       pp.title as job_title, pp.company_name, pp.start_date
+FROM linkedin_profile lp
+LEFT JOIN linkedin_profile_position3 pp ON pp.linkedin_profile_id = lp.id
+WHERE lp.id = ?
+ORDER BY pp.start_date DESC NULLS LAST
+
+-- USE ONLY when you need enriched fields (seniority, job_function, etc.)
+SELECT profile_id, name, experience, education
+FROM lkd_profile WHERE profile_id = ?
+```
+
+### 4. Filtering on Nested Data
+
+| Query Type                                | Normalized | Denormalized | Winner                |
+| ----------------------------------------- | ---------- | ------------ | --------------------- |
+| EXISTS check on positions (known profile) | 1,106.6ms  | **2.84ms**   | Denormalized (390x)\* |
+| Filter positions by title                 | **17.6ms** | 1,582ms      | Normalized (90x)      |
+| COUNT positions > 5 (HAVING)              | **0.99ms** | 94ms         | Normalized (95x)      |
+| jsonb_array_length filter                 | N/A        | 94ms         | Normalized preferred  |
+
+\*Note: Denormalized wins when starting from a known profile_id and checking JSON fields. But loses badly for scanning/searching within JSON arrays.
+
+```sql
+-- RECOMMENDED: Normalized for filtering on nested data
+SELECT DISTINCT pp.linkedin_profile_id
+FROM linkedin_profile_position3 pp
+WHERE pp.title ILIKE '%software engineer%'
+LIMIT 10
+
+-- AVOID: JSON array filtering is very slow
+SELECT profile_id, name
+FROM lkd_profile
+WHERE EXISTS (
+  SELECT 1 FROM jsonb_array_elements(experience::jsonb) e
+  WHERE (e->>'title') ILIKE '%software engineer%'
+) LIMIT 10
+```
+
+### 5. Extended Comparison Tests (2025)
+
+Comprehensive tests across 23 query patterns:
+
+| Test                       | Query Pattern                 | `lkd_profile` | `linkedin_profile` | Winner       | Speedup |
+| -------------------------- | ----------------------------- | ------------- | ------------------ | ------------ | ------- |
+| Slug lookup                | slug = 'x'                    | TIMEOUT       | **7ms**            | Normalized   | ∞       |
+| Profile ID lookup          | id = X                        | 82ms          | **33ms**           | Normalized   | 2.5x    |
+| updated_at filter          | updated_at > date             | 91ms          | **12ms**           | Normalized   | 7.5x    |
+| jobs_count filter          | jobs_count > 10               | 228ms         | **53ms**           | Normalized   | 4.3x    |
+| Large results (5000)       | headline ILIKE                | 5.5s          | **324ms**          | Normalized   | 17x     |
+| **Name search**            | first + last                  | **2.3s**      | 3.5s               | Denormalized | 1.5x    |
+| **Follower + headline**    | num > X AND headline          | **1.8s**      | 3.1s               | Denormalized | 1.7x    |
+| **Multiple skills (3)**    | skill1 AND skill2 AND skill3  | **2.5s**      | 7s                 | Denormalized | 2.8x    |
+| **Skills + headline**      | skill AND headline ILIKE      | **7s**        | 9.7s               | Denormalized | 1.4x    |
+| **Rare term (blockchain)** | headline ILIKE '%blockchain%' | **1.4s**      | 3.4s               | Denormalized | 2.4x    |
+| **Rare term (kubernetes)** | headline ILIKE '%kubernetes%' | **1.6s**      | 3.1s               | Denormalized | 1.9x    |
+| **Summary search**         | summary ILIKE '%startup%'     | **612ms**     | 966ms              | Denormalized | 1.6x    |
+| **Connection + skills**    | connections > X AND skill     | **1.6s**      | 3.8s               | Denormalized | 2.4x    |
+| **Complex regex**          | headline ~\* '(a\|b\|c\|d)'   | **335ms**     | 465ms              | Denormalized | 1.4x    |
+| **Triple filter**          | conn + skill + headline       | **5.8s**      | 18.2s              | Denormalized | 3.1x    |
+| **Rare term + follower**   | tensorflow + follower > X     | **9.2s**      | 23s                | Denormalized | 2.5x    |
+| **Word boundary regex**    | headline ~\* '\mAI\M'         | **227ms**     | 480ms              | Denormalized | 2.1x    |
+| **Headline + locality**    | headline + city ILIKE         | **746ms**     | 1.95s              | Denormalized | 2.6x    |
+
+---
+
+## Test Results: Company Queries (January 2026)
+
+> **Note**: `lkd_company` is a VIEW (not a materialized view or table), so it has no indexes and relies on the underlying table indexes.
+
+### 5. Primary Key & Slug Lookups
+
+| Query Type                      | Normalized | Denormalized | Winner              |
+| ------------------------------- | ---------- | ------------ | ------------------- |
+| Single ID lookup                | **3-4ms**  | 4-19ms       | Normalized (varies) |
+| Batch 5 IDs                     | 4ms        | 4ms          | Tie                 |
+| Batch 20 IDs                    | 5ms        | 4ms          | Tie                 |
+| Slug via raw text               | TIMEOUT    | TIMEOUT      | Both fail           |
+| Slug via `key64()` + subquery   | **4ms**    | N/A          | Normalized          |
+| Slug direct on lkd_company      | N/A        | **21-47ms**  | Denormalized only   |
+| Domain lookup (indexed)         | **3ms**    | N/A          | Normalized only     |
+| universal_name lookup (indexed) | **2ms**    | N/A          | Normalized only     |
+| Ticker lookup (indexed)         | **5ms**    | N/A          | Normalized only     |
+
+**Critical: Slug Lookup Strategy**
+
+```sql
+-- ✅ FASTEST: Use key64() for indexed slug lookup
+SELECT id, company_name FROM linkedin_company
+WHERE id = (
+  SELECT linkedin_company_id FROM linkedin_company_slug
+  WHERE slug_key64 = key64('google') LIMIT 1
+)
+-- Result: 4ms
+
+-- 🟡 WORKS: Denormalized direct (slower but simple)
+SELECT linkedin_company_id, name FROM lkd_company WHERE slug = 'google'
+-- Result: 21-47ms
+
+-- ❌ TIMEOUT: Raw slug lookup without key64()
+SELECT linkedin_company_id FROM linkedin_company_slug WHERE slug = 'google'
+-- Result: 30s+ TIMEOUT
+```
+
+### 6. Text Searches
+
+| Query Type                       | Normalized | Denormalized | Winner              |
+| -------------------------------- | ---------- | ------------ | ------------------- |
+| Company name ILIKE               | **691ms**  | 872ms        | Normalized (1.3x)   |
+| Description ILIKE (AI)           | 396ms      | **337ms**    | Denormalized (1.2x) |
+| Description ILIKE (blockchain)   | 380ms      | **372ms**    | Tie                 |
+| Description ILIKE (ML)           | 413ms      | **276ms**    | Denormalized (1.5x) |
+| Company headline ILIKE           | **36ms**   | 42ms         | Tie                 |
+| Regex multiple keywords (AI\|ML) | **128ms**  | 158ms        | Normalized (1.2x)   |
+| Specialties array contains       | 25.9s      | **11.6s**    | Denormalized (2.2x) |
+
+### 7. Getting Nested Data (Funding, Locations, Industries)
+
+| Query Type                 | Normalized (JOINs) | Denormalized | Winner            |
+| -------------------------- | ------------------ | ------------ | ----------------- |
+| Company + funding (1 ID)   | **9ms**            | 67ms         | Normalized (7.4x) |
+| Company + locations (1 ID) | **6ms**            | 11ms         | Normalized (1.8x) |
+| Company + industry (1 ID)  | **3ms**            | 50ms         | Normalized (17x)  |
+| Full company all nested    | **3ms**            | 58ms         | Normalized (19x)  |
+| Batch 5 IDs + industry     | **3ms**            | 53ms         | Normalized (18x)  |
+
+### 8. Filtering on Nested Company Data
+
+| Query Type                      | Normalized | Denormalized | Winner                |
+| ------------------------------- | ---------- | ------------ | --------------------- |
+| EXISTS funding check            | **111ms**  | 191ms        | Normalized (1.7x)     |
+| Funding + Employee compound     | 1,183ms    | **70ms**     | Denormalized (17x) ⚠️ |
+| Industry name via JSON ILIKE    | N/A        | 272ms        | Denormalized only     |
+| Industry via industry_code JOIN | **2ms**    | N/A          | Normalized only       |
+
+**⚠️ Critical Finding**: Funding + Employee compound queries are **17x faster** on denormalized.
+
+### 9. Numeric Filters
+
+| Query Type                    | Normalized | Denormalized | Winner            |
+| ----------------------------- | ---------- | ------------ | ----------------- |
+| employee_count > 1000         | 105ms      | **81ms**     | Denormalized      |
+| employee_count BETWEEN 50-200 | **10ms**   | 11ms         | Tie               |
+| follower_count > 100000       | **209ms**  | 283ms        | Normalized (1.4x) |
+| founded > 2020                | **7ms**    | 6ms          | Tie               |
+| founded + employee compound   | 82ms       | 89ms         | Tie               |
+| updated_at > date (indexed)   | **3ms**    | 4ms          | Normalized        |
+
+### 10. Country Filtering
+
+| Query Type                | Normalized | Denormalized | Winner              |
+| ------------------------- | ---------- | ------------ | ------------------- |
+| country_iso only          | **9-11ms** | 228ms        | Normalized (20x)    |
+| country_code only         | **9ms**    | N/A          | Normalized only     |
+| country_iso + employee    | 250ms      | **158ms**    | Denormalized (1.6x) |
+| country_iso + description | 322ms      | **135ms**    | Denormalized (2.4x) |
+
+### 11. Compound Query Performance
+
+| # of Filters                        | Normalized | Denormalized | Winner              |
+| ----------------------------------- | ---------- | ------------ | ------------------- |
+| 1 filter (indexed: country, domain) | **Faster** | Slower       | Normalized (9-20x)  |
+| 1 filter (text: description)        | Similar    | Similar      | Tie                 |
+| 2 filters (employee + name)         | 243ms      | **229ms**    | Tie                 |
+| 2 filters (employee + description)  | 162ms      | **84ms**     | Denormalized (1.9x) |
+| 3 filters (country+emp+desc)        | 797ms      | **372ms**    | Denormalized (2.1x) |
+| 4 filters (country+emp+foll+desc)   | 158ms      | **108ms**    | Denormalized (1.5x) |
+| Double text (name + description)    | 176ms      | **148ms**    | Denormalized (1.2x) |
+| Locality + description              | **357ms**  | 434ms        | Normalized (1.2x)   |
+
+### 12. Large Result Sets & Aggregations
+
+| Query Type                   | Normalized | Denormalized | Winner             |
+| ---------------------------- | ---------- | ------------ | ------------------ |
+| 1,000 rows (employee filter) | 16ms       | 17ms         | Tie                |
+| 5,000 rows (employee filter) | **20ms**   | 27ms         | Normalized (1.35x) |
+| 10,000 rows (no filter)      | 30ms       | 29ms         | Tie                |
+| COUNT with filter            | **20s**    | 21s          | Both slow          |
+| GROUP BY country             | **26s**    | TIMEOUT      | Normalized         |
+
+---
+
+## Decision Matrix
+
+### When to Use `linkedin_profile` (Normalized)
+
+✅ **Always use for:**
+
+- Slug lookups (via `key64()` function) - **7ms vs TIMEOUT**
+- Profile ID lookups - **2.5x faster**
+- `updated_at` filtering - **7.5x faster** (indexed)
+- `jobs_count` filtering - **4.3x faster**
+- Org/company name search - **20x faster** (GIN indexed)
+- Title field searches - **14x faster**
+- Large result sets (5000+ rows) - **17x faster**
+- JOINs with position/education tables for filtering
+- COUNT, GROUP BY, aggregations
+- EXISTS checks across many profiles
+
+### When to Use `lkd_profile` (Denormalized)
+
+✅ **Use for:**
+
+- **Country filtering with uncommon terms** - `country_iso = 'US'` works where `location_country_code` times out
+- **Multi-filter combinations (2+ filters)** - 2-3x faster
+- **Multiple skills array searches** - 2.8x faster
+- **Rare/uncommon headline terms** (blockchain, kubernetes, tensorflow) - 2-2.5x faster
+- **Numeric + text combos** (follower count + headline) - 1.7-2.5x faster
+- **Complex regex patterns** (multi-option, word boundaries) - 1.4-2x faster
+- **Summary text searches** - 1.6x faster
+- **Name searches** (first + last name) - 1.5x faster
+- **Headline + locality combinations** - 2.6x faster
+- Enriched experience data (seniority, job_function, employment_type, academic_qualification, inferred_location)
+- Building API responses that need complete profile with all nested arrays
+- You have a known profile_id and want all nested data in one query
+- Checking JSON fields on a single known entity
+
+❌ **Never use for:**
+
+- Slug lookups (use normalized with `key64()`)
+- Searching/filtering within JSON arrays (experience titles, etc.)
+- Aggregations or counting
+- Large result sets without filters
+
+### When to Use `linkedin_company` (Normalized)
+
+✅ **Always use for:**
+
+- **Slug lookups via `key64()`** - 4ms vs 21-47ms (5-12x faster)
+- **Domain lookups** - only option (indexed, 3ms)
+- **Ticker lookups** - only option (indexed, 5ms)
+- **Universal name lookups** - only option (indexed, 2ms)
+- **Single country filter** - 9ms vs 228ms (20x faster)
+- **Updated_at filtering** - indexed (3ms)
+- **Industry filtering via industry_code** - 2ms vs 272ms (136x faster)
+- **Aggregations (GROUP BY)** - 26s vs TIMEOUT
+- **Complex JOINs** (company → positions → profiles)
+- **Getting nested data by known ID** - JOINs are 7-19x faster
+- **EXISTS checks on funding** - 1.7x faster
+
+### When to Use `lkd_company` (Denormalized)
+
+✅ **Use for:**
+
+- **Compound queries (2+ filters)** - 1.5-2.4x faster
+- **Funding + employee compound** - 17x faster (70ms vs 1,183ms) ⚠️
+- **Country + description combinations** - 2.4x faster
+- **Country + employee combinations** - 1.6x faster
+- **Description searches (rare terms)** - 1.5x faster for ML/AI terms
+- **Specialties filtering** - 2.2x faster (11.6s vs 25.9s)
+- **Simple slug lookups** without `key64()` - works (21-47ms) vs TIMEOUT
+- Pre-formatted `industries`, `locations`, `crunchbase_funding` JSON
+- Building API responses with complete company data in one query
+
+❌ **Never use for:**
+
+- Domain/ticker/universal_name lookups (columns don't exist)
+- Single indexed filter lookups (country_iso, industry_code) - 20x slower
+- Aggregations (COUNT, GROUP BY) - TIMEOUT
+- Complex JOINs with other tables
+
+---
+
+## Cross-Table Query Performance (Profile + Company)
+
+**Critical Finding**: When combining profile text filters (headline, skills) with company constraints, **denormalized JOINs are 20-93x faster** and often the only option that completes.
+
+### Performance Comparison
+
+| Query Pattern              | Normalized Multi-JOIN | Denormalized JOIN | Winner             |
+| -------------------------- | --------------------- | ----------------- | ------------------ |
+| Headline + company size    | 20,205ms              | **217ms**         | Denormalized (93x) |
+| Multi-skill + company size | 28,173ms              | **1,281ms**       | Denormalized (22x) |
+| Skill + company industry   | TIMEOUT               | **3,553ms**       | Denormalized (∞)   |
+| Senior engineers + company | 4,535ms               | **196ms**         | Denormalized (23x) |
+| Company ID → employees     | **48ms**              | 279ms             | Normalized (5.8x)  |
+| Company name (org) search  | **274ms**             | 8,600ms           | Normalized (31x)   |
+
+### Rules for Cross-Table Queries
+
+1. **Company name search** → Always use `linkedin_profile.org` (GIN indexed, 68x faster)
+2. **Headline/skill + company constraint** → Always use `lkd_profile JOIN lkd_company` (normalized times out)
+3. **Company-first lookups** → Use normalized (5-8x faster)
+4. **Multi-filter profile + company** → Denormalized is often the only option
+
+### Example: Engineers at Large Companies
+
+```sql
+-- ❌ SLOW: Normalized multi-JOIN (20 seconds)
+SELECT lp.id, lp.headline, lc.company_name
+FROM linkedin_profile lp
+JOIN linkedin_profile_position3 pos ON pos.linkedin_profile_id = lp.id
+JOIN linkedin_company lc ON lc.id = pos.linkedin_company_id
+WHERE pos.end_date IS NULL
+  AND lp.headline ILIKE '%engineer%'
+  AND lc.employee_count > 1000
+LIMIT 50
+
+-- ✅ FAST: Denormalized JOIN (217ms - 93x faster)
+SELECT lkd.profile_id, lkd.headline, lkdc.name
+FROM lkd_profile lkd
+JOIN lkd_company lkdc ON lkdc.linkedin_company_id = lkd.linkedin_company_id
+WHERE lkd.headline ILIKE '%engineer%'
+  AND lkdc.employee_count > 1000
+LIMIT 50
+```
+
+See [B2B_CROSS_TABLE_TEST_FINDINGS.md](./B2B_CROSS_TABLE_TEST_FINDINGS.md) for full test results.
+
+---
+
+## Query Patterns & Recommendations
+
+### Pattern 1: Find Profiles with Specific Experience
+
+```sql
+-- ✅ CORRECT: Use normalized tables
+SELECT DISTINCT lp.id, lp.formatted_name, pp.title, pp.company_name
+FROM linkedin_profile lp
+JOIN linkedin_profile_position3 pp ON pp.linkedin_profile_id = lp.id
+WHERE pp.title ILIKE '%product manager%'
+  AND pp.linkedin_company_id IN (SELECT id FROM linkedin_company WHERE employee_count > 1000)
+LIMIT 100
+
+-- ❌ WRONG: Don't search JSON arrays
+SELECT profile_id, name FROM lkd_profile
+WHERE EXISTS (SELECT 1 FROM jsonb_array_elements(experience::jsonb) e
+              WHERE (e->>'title') ILIKE '%product manager%')
+```
+
+### Pattern 2: Get Complete Profile for Display
+
+```sql
+-- ✅ Option A: Denormalized (when you need enriched fields)
+SELECT profile_id, name, title, company_name, headline,
+       experience, education, certifications, skills
+FROM lkd_profile WHERE profile_id = ?
+
+-- ✅ Option B: Normalized with multiple queries (faster total time)
+SELECT * FROM linkedin_profile WHERE id = ?;
+SELECT * FROM linkedin_profile_position3 WHERE linkedin_profile_id = ? ORDER BY start_date DESC;
+SELECT * FROM linkedin_profile_education2 WHERE linkedin_profile_id = ?;
+```
+
+### Pattern 3: Find Companies with Funding
+
+```sql
+-- ✅ CORRECT: Use normalized with EXISTS
+SELECT lc.id, lc.company_name, lc.employee_count
+FROM linkedin_company lc
+WHERE EXISTS (SELECT 1 FROM linkedin_crunchbase_funding cf WHERE cf.linkedin_company_id = lc.id)
+  AND lc.employee_count BETWEEN 50 AND 500
+LIMIT 100
+
+-- ❌ WRONG: Don't filter on JSON
+SELECT linkedin_company_id, name FROM lkd_company
+WHERE crunchbase_funding IS NOT NULL AND crunchbase_funding::text != 'null'
+```
+
+### Pattern 4: Company Lookup with Full Data
+
+```sql
+-- ✅ CORRECT: Denormalized for complete company
+SELECT linkedin_company_id, name, description, employee_count,
+       industries, locations, crunchbase_funding
+FROM lkd_company WHERE linkedin_company_id = ?
+
+-- OR: Normalized with JOINs (faster but more code)
+SELECT lc.*,
+       array_agg(DISTINCT ca.address) as addresses,
+       array_agg(DISTINCT cf.round_name) as funding_rounds
+FROM linkedin_company lc
+LEFT JOIN linkedin_company_address2 ca ON ca.linkedin_company_id = lc.id
+LEFT JOIN linkedin_crunchbase_funding cf ON cf.linkedin_company_id = lc.id
+WHERE lc.id = ?
+GROUP BY lc.id
+```
+
+---
+
+## Summary: The Golden Rules
+
+### Profile Rules
+
+1. **For indexed lookups (slug, ID, updated_at)**: Always use normalized tables
+2. **For country + headline filtering**: Use `lkd_profile` with `country_iso` (normalized times out)
+3. **For single filters on common terms**: Use normalized tables (faster)
+4. **For multi-filter combinations (2+ filters)**: Use denormalized views (2-3x faster)
+5. **For rare/uncommon headline terms**: Use denormalized views (2-2.5x faster)
+6. **For multiple skills searches**: Use denormalized views (2.8x faster)
+7. **For nested data**: Normalized with JOINs unless you need enriched fields
+8. **For aggregations**: Always normalized
+9. **Never**: Filter on JSON array contents in denormalized views
+
+### Company Rules (Updated Jan 2026)
+
+1. **For slug lookups**: Use `key64()` with normalized (4ms) or denormalized direct (21-47ms)
+2. **For domain/ticker/universal_name lookups**: Normalized only (indexed, 2-5ms)
+3. **For single indexed filters (country_iso, industry_code)**: Normalized (20-136x faster)
+4. **For compound queries (2+ filters)**: Denormalized (1.5-2.4x faster)
+5. **For funding + other filters**: Denormalized (17x faster) ⚠️
+6. **For nested data by known ID**: Normalized JOINs (7-19x faster)
+7. **For aggregations (COUNT, GROUP BY)**: Normalized only (denormalized TIMEOUTs)
+8. **For pre-formatted JSON response**: Denormalized (convenience)
+
+### Cross-Table Rules (Profile + Company Queries)
+
+1. **Company name search**: Always use `linkedin_profile.org` (GIN indexed, 68x faster)
+2. **Headline/skill + company constraint**: Use `lkd_profile JOIN lkd_company` (20-93x faster)
+3. **Company-first lookup → employees**: Use normalized (5-8x faster)
+4. **Multi-filter profile + company**: Denormalized JOIN is often the only option that works
+
+### Quick Decision Flowchart
+
+```
+Query has indexed lookup (slug, ID, updated_at)?
+  └─ YES → Use linkedin_profile (normalized)
+
+Query needs profile + company data together?
+  └─ YES:
+       Searching by company name?
+         └─ YES → Use linkedin_profile.org (GIN indexed, 68x faster)
+       Profile text filter (headline/skill) + company constraint?
+         └─ YES → Use lkd_profile JOIN lkd_company (20-93x faster)
+       Company-first lookup?
+         └─ YES → Use normalized JOINs (5-8x faster)
+
+Query has country filter + uncommon headline term?
+  └─ YES → Use lkd_profile (denormalized) - normalized will TIMEOUT
+
+Query has 3+ combined filters?
+  └─ YES → Use lkd_profile (denormalized) - 2-3x faster
+
+Query searches rare headline terms (iOS, kubernetes, blockchain)?
+  └─ YES → Use lkd_profile (denormalized) - 2-2.5x faster
+
+Query has multiple skills?
+  └─ YES → Use lkd_profile (denormalized) - 2.8x faster
+
+Query needs large result set (5000+)?
+  └─ YES → Use linkedin_profile (normalized) - 17x faster
+
+Default:
+  └─ Use linkedin_profile (normalized)
+```
+
+---
+
+## Common Query Patterns
+
+### Pattern 5: Country-Filtered Headline Search (NEW)
+
+```sql
+-- ✅ CORRECT: Use lkd_profile for country + uncommon term
+SELECT profile_id, first_name, last_name, headline, locality, url
+FROM lkd_profile
+WHERE country_iso = 'US'
+  AND headline ~* '(\miOS\M|\mSwift\M).*(developer|engineer)'
+LIMIT 100
+-- Result: 20s, 100 rows ✅
+
+-- ❌ WRONG: linkedin_profile times out
+SELECT id, first_name, last_name, headline, location_name
+FROM linkedin_profile
+WHERE location_country_code = 'US'
+  AND headline ILIKE '%iOS developer%'
+LIMIT 100
+-- Result: TIMEOUT after 30s ❌
+```
+
+### Pattern 6: Multi-Skill Search
+
+```sql
+-- ✅ CORRECT: Use lkd_profile for multiple skills (2.8x faster)
+SELECT profile_id, first_name, headline, skills[1:5]
+FROM lkd_profile
+WHERE 'Python' = ANY(skills)
+  AND 'SQL' = ANY(skills)
+  AND 'Data Analysis' = ANY(skills)
+LIMIT 50
+-- Result: 2.5s
+
+-- 🟡 WORKS but slower: linkedin_profile
+SELECT id, first_name, headline, skills[1:5]
+FROM linkedin_profile
+WHERE 'Python' = ANY(skills)
+  AND 'SQL' = ANY(skills)
+  AND 'Data Analysis' = ANY(skills)
+LIMIT 50
+-- Result: 7s (2.8x slower)
+```
+
+### Pattern 7: Rare Headline Term Search
+
+```sql
+-- ✅ FASTER: Use lkd_profile for rare terms (2.4x faster)
+SELECT profile_id, first_name, headline
+FROM lkd_profile
+WHERE headline ILIKE '%blockchain%'
+LIMIT 100
+-- Result: 1.4s
+
+-- 🟡 WORKS but slower: linkedin_profile
+SELECT id, first_name, headline
+FROM linkedin_profile
+WHERE headline ILIKE '%blockchain%'
+LIMIT 100
+-- Result: 3.4s (2.4x slower)
+```
+
+### Pattern 8: Triple Filter Combination
+
+```sql
+-- ✅ MUCH FASTER: Use lkd_profile for 3+ filters (3.1x faster)
+SELECT profile_id, first_name, headline, connection_count
+FROM lkd_profile
+WHERE connection_count > 500
+  AND 'Python' = ANY(skills)
+  AND headline ILIKE '%engineer%'
+LIMIT 100
+-- Result: 5.8s
+
+-- 🟡 WORKS but much slower: linkedin_profile
+SELECT id, first_name, headline, connections
+FROM linkedin_profile
+WHERE connections > 500
+  AND 'Python' = ANY(skills)
+  AND headline ILIKE '%engineer%'
+LIMIT 100
+-- Result: 18.2s (3.1x slower)
+```
+
+---
+
+## Company Query Patterns (Updated Jan 2026)
+
+### Pattern 9: Company Slug Lookup
+
+```sql
+-- ✅ FASTEST: Use key64() for indexed lookup (4ms)
+SELECT id, company_name, employee_count
+FROM linkedin_company
+WHERE id = (
+  SELECT linkedin_company_id FROM linkedin_company_slug
+  WHERE slug_key64 = key64('google') LIMIT 1
+)
+
+-- 🟡 WORKS: Denormalized direct (21-47ms) - simpler but 5-12x slower
+SELECT linkedin_company_id, name, employee_count
+FROM lkd_company WHERE slug = 'google'
+
+-- ❌ TIMEOUT: Raw slug without key64()
+SELECT linkedin_company_id FROM linkedin_company_slug WHERE slug = 'google'
+-- Result: 30s+ TIMEOUT
+```
+
+### Pattern 10: Company Domain/Ticker Lookup
+
+```sql
+-- ✅ CORRECT: Only normalized has these indexed columns
+SELECT id, company_name, domain FROM linkedin_company WHERE domain = 'google.com'
+-- Result: 3ms
+
+SELECT id, company_name, ticker FROM linkedin_company WHERE ticker = 'GOOGL'
+-- Result: 5ms
+
+SELECT id, company_name FROM linkedin_company WHERE universal_name = 'google'
+-- Result: 2ms
+
+-- ❌ WRONG: lkd_company does NOT have domain/ticker columns
+```
+
+### Pattern 11: Company Compound Filters (2+ filters)
+
+```sql
+-- ✅ FASTER: Use lkd_company for compound queries (1.9-2.4x faster)
+SELECT linkedin_company_id, name
+FROM lkd_company
+WHERE country_iso = 'US'
+  AND employee_count > 100
+  AND description ILIKE '%software%'
+LIMIT 50
+-- Result: 135-372ms
+
+-- 🟡 WORKS but slower: linkedin_company
+SELECT id, company_name
+FROM linkedin_company
+WHERE country_iso = 'US'
+  AND employee_count > 100
+  AND description ILIKE '%software%'
+LIMIT 50
+-- Result: 247-797ms (2.1x slower)
+```
+
+### Pattern 12: Companies with Funding + Filters ⚠️
+
+```sql
+-- ✅ MUCH FASTER: Use lkd_company for funding + other filters (17x faster!)
+SELECT linkedin_company_id, name, crunchbase_funding
+FROM lkd_company
+WHERE employee_count > 50
+  AND crunchbase_funding IS NOT NULL
+  AND crunchbase_funding::text != 'null'
+  AND crunchbase_funding::text != '[]'
+LIMIT 50
+-- Result: 70ms
+
+-- ❌ VERY SLOW: Normalized with EXISTS
+SELECT lc.id, lc.company_name
+FROM linkedin_company lc
+WHERE lc.employee_count > 50
+  AND EXISTS (SELECT 1 FROM linkedin_crunchbase_funding cf WHERE cf.linkedin_company_id = lc.id)
+LIMIT 50
+-- Result: 1,183ms (17x slower)
+```
+
+### Pattern 13: Company Industry Lookup
+
+```sql
+-- ✅ FASTEST: Use normalized with industry_code (2ms)
+SELECT id, company_name, industry_code
+FROM linkedin_company
+WHERE industry_code = 4
+LIMIT 100
+
+-- 🟡 ALTERNATIVE: Denormalized JSON search (272ms) - 136x slower
+SELECT linkedin_company_id, name
+FROM lkd_company
+WHERE industries::text ILIKE '%Software Development%'
+LIMIT 100
+```
+
+### Pattern 14: Company Aggregations
+
+```sql
+-- ✅ CORRECT: Only use normalized for aggregations
+SELECT country_iso, COUNT(*)
+FROM linkedin_company
+WHERE country_iso IS NOT NULL
+GROUP BY country_iso
+ORDER BY COUNT(*) DESC
+LIMIT 10
+-- Result: 26s (slow but works)
+
+-- ❌ TIMEOUT: Denormalized for aggregations
+SELECT country_iso, COUNT(*) FROM lkd_company ...
+-- Result: TIMEOUT (30s+)
+```
+
+---
+
+## Appendix: Fields Only Available in Denormalized Views
+
+### `lkd_profile` Exclusive Fields
+
+**Key columns for filtering:**
+
+| Column          | Type | Notes                                                            |
+| --------------- | ---- | ---------------------------------------------------------------- |
+| `country_iso`   | text | **Fast country filter** (use instead of `location_country_code`) |
+| `country_name`  | text | Full country name                                                |
+| `industry_name` | text | Denormalized industry name                                       |
+| `url`           | text | Full profile URL                                                 |
+
+**JSON fields with enriched data:**
+
+- `experience` - JSON array with enriched fields per position:
+   - `seniority[]` - Array of {id, seniority} objects
+   - `job_function[]` - Array of {id, job_function} objects
+   - `employment_type[]` - Array of {id, job_employment_type} objects
+   - `academic_qualification[]` - Array of {id, academic_qualification} objects
+   - `inferred_location` - Geocoded {latitude, longitude, formatted_address, country_iso, admin_district, locality}
+- `education` - JSON array of education records
+- `certifications` - JSON array
+- `courses` - JSON array
+- `projects` - JSON array
+- `volunteering` - JSON array
+- `patents` - JSON array
+- `awards` - JSON array
+- `publications` - JSON array
+- `languages` - JSON array
+- `recommendations` - JSON array
+- `test_scores` - JSON array
+- `articles` - JSON array
+
+### `lkd_company` Exclusive Fields
+
+- `industries[]` - Pre-joined industry data with {id, name, primary}
+- `locations[]` - Pre-joined with inferred_location geocoding
+- `crunchbase_funding[]` - Formatted funding rounds with URLs
+- `naics_codes[]` - Pre-joined NAICS code data
+- `inferred_location` - Geocoded primary location
+
+---
+
+## Appendix: Column Comparison
+
+### Key Filtering Columns
+
+| Purpose         | `linkedin_profile`      | `lkd_profile`                   | Notes                                       |
+| --------------- | ----------------------- | ------------------------------- | ------------------------------------------- |
+| Profile ID      | `id`                    | `profile_id`                    | Both indexed                                |
+| Country code    | `location_country_code` | `country_iso`                   | **lkd_profile works with headline filters** |
+| Location string | `location_name`         | `locality`                      | Similar performance                         |
+| Current company | `org`                   | `company_name`                  | `org` has GIN index                         |
+| Current title   | `title`                 | `title`                         | Similar                                     |
+| Headline        | `headline`              | `headline`                      | Similar                                     |
+| Skills          | `skills` (array)        | `skills` (array)                | lkd_profile faster for multi-skill          |
+| Connections     | `connections`           | `connection_count`              | Similar                                     |
+| Followers       | `num_followers`         | `follower_count`                | Similar                                     |
+| Industry        | `linkedin_industry_id`  | `industry_id` + `industry_name` | lkd_profile has name                        |
+| Updated         | `updated_at`            | `updated_at`                    | linkedin_profile indexed                    |
+
+### Indexes Available
+
+**`linkedin_profile` indexes:**
+
+- `linkedin_profile_pkey` - Primary key on `id`
+- `ix_linkedin_profile_org_tsv` - GIN on `org` (full-text)
+- `linkedin_profile_updated_at_idx` - on `updated_at`
+- `ix_linkedin_profile_linkedin_user_id` - on `linkedin_user_id`
+
+**`lkd_profile`:** No indexes (denormalized view), but optimized for compound queries
+
+### Company Column Comparison
+
+| Purpose        | `linkedin_company`   | `lkd_company`         | Notes                               |
+| -------------- | -------------------- | --------------------- | ----------------------------------- |
+| Company ID     | `id`                 | `linkedin_company_id` | Both fast for lookups               |
+| Name           | `company_name`       | `name`                | Similar                             |
+| Slug           | N/A (use slug table) | `slug`                | **lkd_company has it directly**     |
+| Domain         | `domain`             | N/A                   | **Only normalized has domain**      |
+| Ticker         | `ticker`             | `ticker`              | Both have it                        |
+| Universal name | `universal_name`     | N/A                   | **Only normalized (indexed)**       |
+| Country        | `country_iso`        | `country_iso`         | Normalized faster for single filter |
+| Locality       | `locality`           | `locality`            | Similar                             |
+| Employee count | `employee_count`     | `employee_count`      | Similar                             |
+| Follower count | `follower_count`     | `follower_count`      | Similar                             |
+| Founded        | `founded`            | `founded_year`        | Similar                             |
+| Industry       | `industry_code`      | `industries` (JSON)   | Normalized indexed (136x faster)    |
+| Funding        | N/A (use JOIN)       | `crunchbase_funding`  | JSON pre-formatted                  |
+| Locations      | N/A (use JOIN)       | `locations` (JSON)    | JSON pre-formatted                  |
+
+### Company Indexes Available
+
+**`linkedin_company` indexes:**
+
+- `linkedin_company_pkey` - Primary key on `id`
+- `ix_linkedin_company_domain` - on `domain`
+- `ix_linkedin_company_ticker` - on `ticker`
+- `linkedin_company_universal_name_ix` - on `universal_name`
+- `ix_linkedin_company_tsv` - GIN on company_name + universal_name (full-text)
+- `ix_linkedin_company_company_id` - on `company_id`
+- `ix_linkedin_company_max_snapshot_id` - on `max_snapshot_id`
+
+**`linkedin_company_slug` indexes:**
+
+- `linkedin_company_slug_pk` - Primary key
+- `linkedin_company_slug_slug_key64_uniq` - on `slug_key64` (**use with key64() function**)
+- `linkedin_company_slug_linkedin_company_id_ix` - on `linkedin_company_id`
+
+**`lkd_company`:** No indexes (it's a VIEW, not a materialized view)
+
+---
+
+## Test Methodology (Jan 2026)
+
+Tests were conducted against a live B2B database via `http://165.22.151.131:3000/query` endpoint.
+
+- Each query was run multiple times for consistency verification
+- Timings include network latency (remote server)
+- Results show `duration_ms` as reported by the database
+- TIMEOUT = 30+ seconds (query cancelled)
+- Tests covered: ID lookups, slug lookups, text searches, compound filters, nested data, aggregations