orangeslice 1.6.1 → 1.7.0

package/docs/B2B_DATABASE.md

@@ -1,314 +0,0 @@
-# B2B Database Guide
-
-A comprehensive B2B data enrichment database with LinkedIn profiles, companies, job postings, and funding data.
-
-## Database Scale
-
-| Table                         | Estimated Rows   |
-| ----------------------------- | ---------------- |
-| `linkedin_profile`            | **1.15 billion** |
-| `linkedin_profile_position3`  | **2.6 billion**  |
-| `linkedin_job`                | **1.48 billion** |
-| `linkedin_profile_education2` | **965 million**  |
-| `linkedin_profile_slug`       | **1.14 billion** |
-| `person`                      | **1.32 billion** |
-
----
-
-## Access Permissions
-
-### ✅ Tables WITH Access (48 tables)
-
-| Category            | Tables                                                                                                                                                                                                                                       |
-| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **Core LinkedIn**   | `linkedin_profile`, `linkedin_company`, `linkedin_job`, `linkedin_profile_position3`, `linkedin_profile_education2`                                                                                                                          |
-| **Slugs**           | `linkedin_profile_slug`, `linkedin_company_slug`                                                                                                                                                                                             |
-| **Funding**         | `linkedin_crunchbase_funding`                                                                                                                                                                                                                |
-| **Reference**       | `linkedin_industry`, `linkedin_school`, `linkedin_language`, `linkedin_specialty`, `linkedin_work_modality`                                                                                                                                  |
-| **Profile Details** | `linkedin_profile_certification`, `linkedin_profile_award`, `linkedin_profile_project`, `linkedin_profile_volunteer_experience`, `linkedin_profile_recommendation2`, `linkedin_profile_test_scores`, `linkedin_profile_language_proficiency` |
-| **Company Details** | `linkedin_company_address2`, `linkedin_company_post`                                                                                                                                                                                         |
-| **Content**         | `linkedin_article`, `linkedin_patent`, `linkedin_project`, `linkedin_publication2`                                                                                                                                                           |
-| **Jobs**            | `job_title`, `job_function`, `job_seniority`, `job_employment_type`, `job_academic_qualification`                                                                                                                                            |
-| **Geography**       | `country`, `locality`, `naics_code`                                                                                                                                                                                                          |
-| **Other**           | `company`, `company_type`, `person`, `language_proficiency`                                                                                                                                                                                  |
-
-### ❌ Tables WITHOUT Access (84 tables)
-
-| Category              | Tables (Permission Denied)                                        |
-| --------------------- | ----------------------------------------------------------------- |
-| **Contact Data**      | `email_address`, `email_address_linkedin_profile`, `phone_number` |
-| **Company Reference** | `company_size`, `company_location`, `company_country`             |
-| **Geography**         | `linkedin_geo`                                                    |
-| **Indeed**            | All `indeed_*` tables                                             |
-| **Domain/Web**        | `domain`, `host`, `domain_traffic_estimate`, `web_tag`            |
-| **App Store**         | `ios_sdk`, `play_store_sdk`, `itunes_*`, `play_store_*`           |
-
----
-
-## Top Industries
-
-| ID  | Industry                             | Companies |
-| --- | ------------------------------------ | --------- |
-| 48  | Construction                         | 1.96M     |
-| 44  | Real Estate                          | 1.80M     |
-| 96  | IT Services and IT Consulting        | 1.79M     |
-| 27  | Retail                               | 1.51M     |
-| 80  | Advertising Services                 | 1.33M     |
-| 11  | Business Consulting and Services     | 1.32M     |
-| 4   | **Software Development**             | 1.19M     |
-| 6   | Technology, Information and Internet | 617K      |
-
----
-
-## GTM Query Examples
-
-### 🏢 Company Enrichment
-
-**Find company by universal_name (FAST ~300ms)**
-
-```sql
-SELECT id, company_name, domain, website, employee_count,
-       locality, country_code, description
-FROM linkedin_company
-WHERE universal_name = 'stripe';
-```
-
-**Find company by domain (FAST ~500ms)**
-
-```sql
-SELECT id, company_name, universal_name, employee_count, locality
-FROM linkedin_company
-WHERE domain = 'openai.com';
-```
-
-**Full company enrichment with industry**
-
-```sql
-SELECT lc.company_name, lc.domain, lc.employee_count, lc.locality,
-       lc.description, li.name as industry_name
-FROM linkedin_company lc
-LEFT JOIN linkedin_industry li ON li.id = lc.industry_code
-WHERE lc.universal_name = 'openai';
-```
-
-### 👤 Profile Lookup
-
-**Find profile by LinkedIn slug (FAST ~400ms - use key64)**
-
-```sql
-SELECT lp.first_name, lp.last_name, lp.headline, lp.location_name
-FROM linkedin_profile lp
-JOIN linkedin_profile_slug lps ON lps.linkedin_profile_id = lp.id
-WHERE lps.slug_key64 = key64('satyanadella');
-```
-
-**Find company by LinkedIn slug (FAST ~400ms - use key64)**
-
-```sql
-SELECT lc.id, lc.company_name, lc.domain, lc.employee_count
-FROM linkedin_company lc
-JOIN linkedin_company_slug lcs ON lcs.linkedin_company_id = lc.id
-WHERE lcs.slug_key64 = key64('meta');
-```
-
-### 👥 Find Employees at a Company
-
-**Current employees at a company**
-
-```sql
-SELECT lp.first_name, lp.last_name, lp.headline, pos.title
-FROM linkedin_profile lp
-JOIN linkedin_profile_position3 pos ON pos.linkedin_profile_id = lp.id
-WHERE pos.linkedin_company_id = 2135371  -- Stripe's ID
-  AND pos.end_date IS NULL
-LIMIT 100;
-```
-
-**Headcount by role at a company**
-
-```sql
-SELECT pos.title, COUNT(*) as count
-FROM linkedin_profile_position3 pos
-WHERE pos.linkedin_company_id = 11130470  -- OpenAI
-  AND pos.end_date IS NULL
-GROUP BY pos.title
-ORDER BY count DESC
-LIMIT 20;
-```
-
-### 🎯 Find Decision Makers
-
-**VPs of Sales at mid-to-large companies**
-
-```sql
-SELECT lp.first_name, lp.last_name, lp.headline,
-       pos.title, lc.company_name, lc.employee_count
-FROM linkedin_profile lp
-JOIN linkedin_profile_position3 pos ON pos.linkedin_profile_id = lp.id
-JOIN linkedin_company lc ON lc.id = pos.linkedin_company_id
-WHERE pos.title ILIKE '%vp%sales%'
-  AND pos.end_date IS NULL
-  AND lc.employee_count > 50
-LIMIT 20;
-```
-
-**C-suite at mid-size software companies (~10 seconds)**
-
-```sql
-SELECT lp.first_name, lp.last_name, pos.title,
-       lc.company_name, lc.employee_count
-FROM linkedin_profile lp
-JOIN linkedin_profile_position3 pos ON pos.linkedin_profile_id = lp.id
-JOIN linkedin_company lc ON lc.id = pos.linkedin_company_id
-WHERE pos.end_date IS NULL
-  AND lc.employee_count BETWEEN 100 AND 500
-  AND lc.industry_code = 4  -- Software Development
-  AND (pos.title ILIKE 'ceo%' OR pos.title ILIKE 'cto%'
-       OR pos.title ILIKE 'cfo%' OR pos.title ILIKE 'chief%')
-LIMIT 20;
-```
-
-### 🔍 Find People by Skills
-
-```sql
-SELECT lp.first_name, lp.last_name, lp.headline,
-       lp.skills[1:5] as top_skills, lp.location_name
-FROM linkedin_profile lp
-WHERE 'Python' = ANY(lp.skills)
-  AND 'Machine Learning' = ANY(lp.skills)
-LIMIT 20;
-```
-
-### 🎓 Find Alumni
-
-```sql
-SELECT lp.first_name, lp.last_name, lp.headline,
-       edu.school_name, edu.degree
-FROM linkedin_profile lp
-JOIN linkedin_profile_education2 edu ON edu.linkedin_profile_id = lp.id
-WHERE edu.school_name ILIKE '%stanford%'
-LIMIT 20;
-```
-
-### 📍 Find People by Location
-
-```sql
-SELECT lp.first_name, lp.last_name, lp.headline, lp.location_name
-FROM linkedin_profile lp
-WHERE lp.location_name ILIKE '%san francisco%'
-LIMIT 20;
-```
-
-### 💼 Job Postings
-
-**Recent jobs at a company with salary info**
-
-```sql
-SELECT lj.title, lj.location, lj.salary_range,
-       lj.salary_min, lj.salary_max, lj.posted_date, lj.applicants
-FROM linkedin_job lj
-WHERE lj.linkedin_company_id = 2135371  -- Stripe
-ORDER BY lj.posted_date DESC NULLS LAST
-LIMIT 20;
-```
-
-### 💰 Companies with Funding
-
-```sql
-SELECT lc.company_name, lc.domain, lc.employee_count,
-       cf.round_name, cf.round_date, cf.round_amount,
-       cf.investor_names[1:3] as top_investors
-FROM linkedin_company lc
-JOIN linkedin_crunchbase_funding cf ON cf.linkedin_company_id = lc.id
-WHERE cf.round_date >= '2024-01-01'
-ORDER BY cf.round_date DESC
-LIMIT 20;
-```
-
----
-
-## Performance Guide
-
-### ✅ Fast Queries (indexed lookups)
-
-| Lookup Method                      | Speed  | Example                            |
-| ---------------------------------- | ------ | ---------------------------------- |
-| Company by `universal_name`        | ~300ms | `WHERE universal_name = 'stripe'`  |
-| Company by `domain`                | ~500ms | `WHERE domain = 'stripe.com'`      |
-| Company by `id`                    | ~300ms | `WHERE id = 2135371`               |
-| Profile/Company by `slug_key64`    | ~400ms | `WHERE slug_key64 = key64('meta')` |
-| Positions by `linkedin_company_id` | ~500ms | `WHERE linkedin_company_id = X`    |
-| Positions by `linkedin_profile_id` | ~500ms | `WHERE linkedin_profile_id = X`    |
-| Jobs by `linkedin_company_id`      | ~1s    | `WHERE linkedin_company_id = X`    |
-| Education by `linkedin_profile_id` | ~500ms | `WHERE linkedin_profile_id = X`    |
-
-### ⚠️ Slow Queries (full table scans)
-
-| Query Type                          | Speed     | Reason                   |
-| ----------------------------------- | --------- | ------------------------ |
-| Company by `company_name ILIKE`     | Timeout   | No index on company_name |
-| Profile by `slug` (without key64)   | Timeout   | No index on raw slug     |
-| Profile by `headline ILIKE`         | Very slow | Full text scan           |
-| Aggregate queries across industries | 60s+      | Large table joins        |
-| Profile by `location_name ILIKE`    | Slow      | No index                 |
-
-### 🔑 Key Functions
-
-```sql
--- Convert slug to indexed key64 for fast lookups
-key64('stripe')  -- Returns bigint for index lookup
-
--- Example: Fast profile lookup
-WHERE slug_key64 = key64('satyanadella')
-
--- Example: Fast company lookup
-WHERE slug_key64 = key64('meta')
-```
-
----
-
-## Common Gotchas
-
-1. **Use `company_name` not `name`** - The column is `company_name` on linkedin_company
-2. **Use `slug_key64` for slug lookups** - Raw `slug` column is not indexed
-3. **Use `linkedin_profile_position3`** - position2 is legacy
-4. **Use `industry_code`** - Not `linkedin_industry_id` on linkedin_company
-5. **NULL `end_date` = current position** - Check for current employees
-6. **Email/phone tables are restricted** - Cannot access contact data directly
-7. **Always use LIMIT** - Tables are massive
-8. **Use `ILIKE` for case-insensitive** - Names vary in casing
-
----
-
-## Schema Exploration
-
-```sql
--- List accessible tables
-SELECT table_name
-FROM information_schema.table_privileges
-WHERE grantee = 'jzt2be9botwq' AND privilege_type = 'SELECT'
-ORDER BY table_name;
-
--- Describe a table
-SELECT column_name, data_type, is_nullable
-FROM information_schema.columns
-WHERE table_name = 'linkedin_profile';
-
--- Check table indexes
-SELECT indexname, indexdef
-FROM pg_indexes
-WHERE tablename = 'linkedin_company';
-
--- Check table row counts
-SELECT relname AS table_name, reltuples::bigint AS estimated_rows
-FROM pg_class
-WHERE relkind = 'r' AND relnamespace = 'public'::regnamespace
-ORDER BY reltuples DESC
-LIMIT 25;
-```
-
----
-
-## Related Documentation
-
-- **[B2B_SCHEMA.md](./B2B_SCHEMA.md)** - Complete schema reference with all columns and indexes