orangeslice 1.0.0

package/docs/B2B_SQL_COMPREHENSIVE_TEST_FINDINGS.md

@@ -0,0 +1,301 @@
+# B2B SQL Comprehensive Test Findings
+
+**Test Date:** January 14, 2026  
+**Test Script:** `scripts/b2b-sql-comprehensive-queries.sh`  
+**Total Queries:** 80  
+**Runtime:** ~7 minutes
+
+## Executive Summary
+
+Ran 80 comprehensive SQL queries against the B2B database to validate documented query patterns and discover performance characteristics. Key findings include several query patterns that timeout, sparse data in some fields, and company lookup gotchas.
+
+---
+
+## ⚡ Fast Indexed Lookups (< 100ms) - All Working
+
+| Query                               | Expected | Actual | Status                        |
+| ----------------------------------- | -------- | ------ | ----------------------------- |
+| Company by domain                   | ~10ms    | ~638ms | ⚠️ Returns multiple companies |
+| Company by universal_name           | ~36ms    | ~18ms  | ✅ Faster than expected       |
+| Company by slug using key64()       | ~25ms    | ~20ms  | ✅                            |
+| Profile by slug using key64()       | ~25ms    | ~21ms  | ✅                            |
+| People at company by ID             | ~35ms    | ~23ms  | ✅                            |
+| Title ILIKE after company_id filter | ~35-65ms | ~21ms  | ✅ Faster than expected       |
+| Title regex after company_id filter | ~30ms    | ~84ms  | ✅ Slightly slower            |
+
+### Key Finding: Multiple Companies Share Domains
+
+Query #1 returned **28 companies** with domain `stripe.com`, including unrelated businesses that use Stripe for payments. Always filter by `employee_count DESC` to find the main company.
+
+```sql
+-- ✅ CORRECT: Order by employee_count to get main company
+SELECT lc.id, lc.company_name, lc.employee_count
+FROM linkedin_company lc
+WHERE lc.domain = 'stripe.com'
+ORDER BY employee_count DESC NULLS LAST
+LIMIT 1
+-- Returns: id = 2135371, company_name = "Stripe", employee_count = 13809
+```
+
+---
+
+## ⚠️ Company Lookup Gotcha: Similar Names
+
+**Query #2** (`universal_name = 'anthropic'`) returned the **wrong company**:
+
+| Field          | Returned                      | Expected            |
+| -------------- | ----------------------------- | ------------------- |
+| id             | 7936402                       | (Anthropic AI's ID) |
+| company_name   | Anthropic                     | Anthropic           |
+| domain         | anthropic.io                  | anthropic.com       |
+| employee_count | 3                             | ~3,500              |
+| description    | "early stage investment fund" | AI research company |
+
+**Lesson:** Similar company names exist. Use **domain lookup** for well-known companies, or verify results by employee count and description.
+
+---
+
+## ✅ Description Search with industry_code (200-800ms)
+
+All queries with `industry_code IN (4, 6, 96)` filter performed well:
+
+| Query | Description                    | Duration |
+| ----- | ------------------------------ | -------- |
+| #8    | AI video companies             | ~385ms   |
+| #9    | Personalization infrastructure | ~1,938ms |
+| #10   | AI platforms for sales/RevOps  | ~128ms   |
+| #11   | Developer API companies        | ~677ms   |
+
+---
+
+## ✅ Top-of-Funnel Queries (5000 results)
+
+| Query | Pattern                            | Duration | Results |
+| ----- | ---------------------------------- | -------- | ------- |
+| #12   | ILIKE OR + industry_code           | ~300ms   | 5,000   |
+| #13   | Indexed filters only               | ~770ms   | 5,000   |
+| #16   | Headline regex                     | ~800ms   | 5,000   |
+| #17   | Word boundaries for short keywords | ~480ms   | 2,000   |
+
+---
+
+## ✅ Funding Queries (19-55ms)
+
+All funding queries performed excellently:
+
+| Query | Description               | Duration |
+| ----- | ------------------------- | -------- |
+| #20   | Recently funded (2024+)   | ~28ms    |
+| #21   | Series A companies        | ~34ms    |
+| #22   | Companies funded by a]16h | ~55ms    |
+| #23   | Seed stage with amount    | ~19ms    |
+
+---
+
+## ✅ Industry Reference Lookups (~15-27ms)
+
+Query #24 revealed actual industry names:
+
+| Code   | Name                              | Company Count |
+| ------ | --------------------------------- | ------------- |
+| 48     | Construction                      | 1,959,385     |
+| 44     | Real Estate                       | 1,796,025     |
+| **96** | **IT Services and IT Consulting** | **1,786,164** |
+| 27     | Retail                            | 1,507,617     |
+| 80     | Advertising Services              | 1,331,885     |
+
+**Note:** Industry code 96 is "IT Services and IT Consulting", not "Internet" as previously documented.
+
+---
+
+## ✅ Employee Growth Metrics (~25-126ms)
+
+| Query | Description               | Duration |
+| ----- | ------------------------- | -------- |
+| #26   | Fast-growing (>50% 12mo)  | ~41ms    |
+| #27   | Hypergrowth (>20% 3mo)    | ~126ms   |
+| #28   | Growth with LinkedIn data | ~25ms    |
+
+---
+
+## 🚀 Subquery Pattern - Mixed Results
+
+| Query | Keyword        | Duration     | Status     |
+| ----- | -------------- | ------------ | ---------- |
+| #29   | "AI" founders  | ~5,302ms     | ✅ Works   |
+| #30   | "fintech" CTOs | **30,017ms** | ❌ TIMEOUT |
+
+**Finding:** Subquery pattern works for **common keywords** (AI, SaaS) but may **timeout for rare keywords** (fintech).
+
+---
+
+## ✅ 3-Table and 4-Table Joins
+
+| Query | Description                                | Duration     | Status     |
+| ----- | ------------------------------------------ | ------------ | ---------- |
+| #31   | Engineers at tech companies                | ~232ms       | ✅         |
+| #32   | Decision makers at AI video companies      | ~2,472ms     | ✅         |
+| #33   | Engineers at Series A startups             | ~44ms        | ✅         |
+| #34   | VPs Engineering at Series B                | ~209ms       | ✅         |
+| #35   | CTOs at Series A-C AI companies + ORDER BY | **30,076ms** | ❌ TIMEOUT |
+
+**Finding:** 4-table joins work well with regex + industry_code, but **adding ORDER BY causes timeout**.
+
+---
+
+## ❌ Education Queries - Large Company Filter Timeout
+
+| Query | Description               | Duration     | Status     |
+| ----- | ------------------------- | ------------ | ---------- |
+| #36   | Stanford alumni at Google | **30,118ms** | ❌ TIMEOUT |
+| #37   | Recent MBA graduates      | ~165ms       | ✅         |
+| #38   | CS degree holders         | ~41ms        | ✅         |
+
+**Finding:** Education + large company filter (Google has 1441 as company_id) causes timeout. Filter by school only.
+
+---
+
+## ✅ Skills, Certifications, Articles
+
+| Query | Description                     | Duration         |
+| ----- | ------------------------------- | ---------------- |
+| #39   | Python + ML skills              | ~979ms           |
+| #40   | Full-stack (React + Node)       | ~22,172ms (slow) |
+| #41   | AWS certified                   | ~47ms            |
+| #42   | Google Cloud certified          | ~92ms            |
+| #43   | AI thought leaders (>100 likes) | ~651ms           |
+| #44   | Prolific writers (5+ articles)  | ~137ms           |
+
+---
+
+## ✅ Job Postings
+
+| Query | Description                       | Duration     | Status     |
+| ----- | --------------------------------- | ------------ | ---------- |
+| #45   | Jobs at Stripe by company_id      | ~18ms        | ✅         |
+| #46   | High-paying engineering (>200k)   | ~122ms       | ✅         |
+| #47   | Remote jobs 2024 + location ILIKE | **30,016ms** | ❌ TIMEOUT |
+
+**Finding:** Job queries by company_id are fast. Location ILIKE with date filter times out.
+
+---
+
+## ❌ Company Posts - Content ILIKE Timeout
+
+| Query | Description                | Duration     | Status     |
+| ----- | -------------------------- | ------------ | ---------- |
+| #48   | Companies posting about AI | **30,132ms** | ❌ TIMEOUT |
+| #49   | Viral posts (>1000 likes)  | ~263ms       | ✅         |
+
+**Finding:** Content ILIKE on company posts times out. Filter by likes/engagement works.
+
+---
+
+## ❌ Sparse/Unavailable Data
+
+| Query | Description                      | Duration  | Results | Status           |
+| ----- | -------------------------------- | --------- | ------- | ---------------- |
+| #50   | Companies with ML/AI specialties | ~27,307ms | 0       | ❌ Sparse        |
+| #52   | Companies with has_careers=true  | ~28,281ms | 0       | ❌ Not populated |
+| #63   | Work modality options            | ~17ms     | 0       | ❌ Empty table   |
+| #64   | Country reference (tech hubs)    | ~18ms     | 0       | ❌ Query issue   |
+| #73   | Multilingual professionals       | ~67ms     | 0       | ❌ Sparse        |
+| #78   | Profiles with GitHub links       | ~19ms     | 0       | ❌ Sparse        |
+
+---
+
+## ❌ Alumni Queries - Complex Joins Timeout
+
+| Query | Description                         | Duration     | Status     |
+| ----- | ----------------------------------- | ------------ | ---------- |
+| #54   | Ex-Google at startups               | **30,077ms** | ❌ TIMEOUT |
+| #55   | Former employees at Stripe (simple) | ~104ms       | ✅         |
+
+**Finding:** Simple alumni queries (former at ONE company) work. Complex "ex-X now at Y" queries timeout.
+
+---
+
+## ✅ Aggregations
+
+| Query | Description                   | Duration |
+| ----- | ----------------------------- | -------- |
+| #56   | Role distribution at OpenAI   | ~5,761ms |
+| #57   | Average funding by round type | ~334ms   |
+
+---
+
+## ❌ Influencer/Ranking Queries Timeout
+
+| Query | Description                | Duration     | Status         |
+| ----- | -------------------------- | ------------ | -------------- |
+| #58   | LinkedIn influencers       | **30,017ms** | ❌ Not indexed |
+| #59   | High follower count (>50k) | ~3,963ms     | ✅             |
+| #68   | Fortune 500 companies      | **30,014ms** | ❌ Not indexed |
+| #69   | Inc Magazine ranked        | **30,265ms** | ❌ Not indexed |
+
+**Finding:** Boolean `influencer = true` and rank columns on `company` table are not indexed.
+
+---
+
+## ✅ Other Working Queries
+
+| Query | Description                     | Duration     |
+| ----- | ------------------------------- | ------------ | ---------------------- |
+| #53   | Companies with multiple offices | ~1,034ms     |
+| #70   | Tech/education volunteers       | ~137ms       |
+| #71   | Innovation/excellence awards    | ~45ms        |
+| #72   | Highly recommended (>20)        | ~691ms       |
+| #74   | Patent holders                  | ~42ms        |
+| #75   | AI/ML patent holders            | ~1,919ms     |
+| #76   | Published researchers           | ~40ms        |
+| #77   | People with AI/ML projects      | ~46ms        |
+| #79   | Full company enrichment         | ~29ms        |
+| #80   | Full profile enrichment         | **30,024ms** | ❌ (specific ID issue) |
+
+---
+
+## Summary of Query Patterns
+
+### ✅ SAFE Patterns (Use These)
+
+1. **Indexed lookups**: domain, universal_name, slug_key64, linkedin_company_id
+2. **Title ILIKE after company_id filter**: ~21ms
+3. **Description ILIKE with industry_code**: ~128-677ms
+4. **Funding joins**: ~19-55ms
+5. **Simple alumni queries** (former at ONE company): ~104ms
+6. **3-table joins with industry_code**: ~232-2,472ms
+7. **4-table joins with regex + industry_code (NO ORDER BY)**: ~44-209ms
+8. **Headline regex for top-of-funnel**: ~800ms for 5000
+
+### ❌ AVOID Patterns (Will Timeout)
+
+1. **Subquery pattern with rare keywords** (fintech, niche terms)
+2. **4-table joins with ORDER BY**
+3. **Education + large company filter** (Stanford at Google)
+4. **Complex alumni queries** (ex-X now at startups)
+5. **Job posting location ILIKE + date filter**
+6. **Company posts content ILIKE**
+7. **Boolean filters**: `influencer = true`
+8. **Rank filters**: `rank_fortune IS NOT NULL`
+9. **ILIKE AND ILIKE** (two conditions on same field)
+10.   **ORDER BY with text search**
+
+### ⚠️ Sparse Data (May Return 0 Results)
+
+- `specialties` array
+- `has_careers` boolean
+- `linkedin_work_modality` table
+- Language proficiency joins
+- URL resources (GitHub links)
+
+---
+
+## Recommendations
+
+1. **Always filter by employee_count** when looking up companies by domain
+2. **Verify company identity** by domain, not just universal_name
+3. **Use broader keywords** in subquery patterns (avoid rare terms)
+4. **Skip ORDER BY** in 4-table joins
+5. **Split complex alumni queries** into multiple simpler queries
+6. **Use likes/engagement filters** for company posts, not content ILIKE
+7. **Avoid sparse fields** unless specifically needed