orangeslice 1.0.0

package/docs/B2B_GENERALIZATION_RULES.md

@@ -0,0 +1,220 @@
+# B2B Database: Generalized Rules for Table Selection
+
+Based on 40+ comprehensive tests across different filter types, selectivity levels, and cross-table patterns.
+
+---
+
+## The Core Rules
+
+### Rule 1: Single Filter Type Determines Winner
+
+| Filter Type              | Normalized | Denormalized | Winner              |
+| ------------------------ | ---------- | ------------ | ------------------- |
+| **ID lookup**            | 4-6ms      | 12-31ms      | Normalized (3-8x)   |
+| **NULL check**           | 5ms        | 18ms         | Normalized (3.6x)   |
+| **Numeric range**        | 19ms       | 187ms        | Normalized (10x)    |
+| **Numeric comparison**   | 115ms      | 354ms        | Normalized (3x)     |
+| **Single skill array**   | 209ms      | 152ms        | Denormalized (1.4x) |
+| **updated_at** (indexed) | 4ms        | 14ms         | Normalized (3.5x)   |
+
+**Generalization**: For single filters, normalized wins on indexed columns and numeric filters; denormalized wins on array filters.
+
+---
+
+### Rule 2: Text Selectivity is the Key Factor
+
+| Headline Term            | Normalized | Denormalized | Winner                  |
+| ------------------------ | ---------- | ------------ | ----------------------- |
+| **CEO** (very common)    | 55ms       | 131ms        | Normalized (2.4x)       |
+| **engineer** (common)    | 24ms       | 136ms        | Normalized (5.7x)       |
+| **devops** (uncommon)    | 409ms      | 256ms        | **Denormalized (1.6x)** |
+| **tensorflow** (rare)    | 28,000ms   | 9,873ms      | **Denormalized (2.8x)** |
+| **solidity** (very rare) | 27,246ms   | 12,829ms     | **Denormalized (2.1x)** |
+
+**Crossover Point**: Around "uncommon" selectivity level.
+
+**Generalization**:
+
+- **Common terms** (CEO, engineer, manager, developer) → Use normalized
+- **Uncommon/rare terms** (devops, kubernetes, blockchain, tensorflow) → Use denormalized
+
+---
+
+### Rule 3: Filter Count Matters
+
+| # of Filters | Pattern                 | Normalized | Denormalized | Winner                  |
+| ------------ | ----------------------- | ---------- | ------------ | ----------------------- |
+| 1            | Numeric only            | 19-115ms   | 187-354ms    | Normalized (3-10x)      |
+| 1            | Text (common)           | 24-55ms    | 131-136ms    | Normalized (2.4-5.7x)   |
+| 1            | Text (rare)             | 27-28s     | 9-13s        | Denormalized (2-2.8x)   |
+| 2            | Numeric + text (common) | 119ms      | 603ms        | Normalized (5x)         |
+| 2            | Numeric + text (rare)   | 10,643ms   | 4,024ms      | **Denormalized (2.6x)** |
+| 2            | Array + text            | 2,968ms    | 1,037ms      | **Denormalized (2.9x)** |
+| 3            | Array + numeric + text  | 6,948ms    | 3,257ms      | **Denormalized (2.1x)** |
+| 3            | Company triple filter   | 6,568ms    | 1,486ms      | **Denormalized (4.4x)** |
+
+**Generalization**:
+
+- **1 filter**: Normalized usually wins (except rare text or arrays)
+- **2 filters**: Depends on selectivity and filter types
+- **3+ filters**: Denormalized usually wins
+
+---
+
+### Rule 4: Filter Location in Cross-Table Queries
+
+| Text Filter Location         | Normalized | Denormalized   | Winner                   |
+| ---------------------------- | ---------- | -------------- | ------------------------ |
+| **Company description only** | 125ms      | 4,166-15,435ms | **Normalized (33-123x)** |
+| **Profile headline only**    | TIMEOUT    | 602-709ms      | **Denormalized (∞)**     |
+| **Both profile + company**   | 702ms      | 3,440ms        | **Normalized (4.9x)**    |
+
+**Generalization**:
+
+- Text filter on **company** side → Normalized wins (company table smaller)
+- Text filter on **profile** side → Denormalized wins (profile table huge)
+- Text filter on **both** → Normalized wins if company filter is selective
+
+---
+
+### Rule 5: LIMIT Size Impact
+
+| Term Type             | LIMIT 10                  | LIMIT 100                   | LIMIT 500-1000                  |
+| --------------------- | ------------------------- | --------------------------- | ------------------------------- |
+| **Common** (engineer) | N: 24ms, D: 42ms (N 1.8x) | N: 24ms, D: 136ms (N 5.7x)  | N: 80ms, D: 865ms (N 10.8x)     |
+| **Rare** (blockchain) | N: 234ms, D: 241ms (tie)  | N: 713ms, D: 384ms (D 1.9x) | N: 3,725ms, D: 2,806ms (D 1.3x) |
+
+**Generalization**:
+
+- **Common terms**: Normalized scales better with larger LIMITs
+- **Rare terms**: Denormalized maintains advantage across all LIMITs
+
+---
+
+### Rule 6: Cross-Table JOIN Patterns
+
+| Pattern                          | Normalized | Denormalized | Winner                 |
+| -------------------------------- | ---------- | ------------ | ---------------------- |
+| Company ID → employees           | 48ms       | 279ms        | Normalized (5.8x)      |
+| Company name (org) search        | 274ms      | 8,600ms      | Normalized (31x)       |
+| Profile headline + company size  | 20,205ms   | 217ms        | **Denormalized (93x)** |
+| Profile skill + company size     | 7,168ms    | 536ms        | **Denormalized (13x)** |
+| Profile skill + company industry | TIMEOUT    | 3,553ms      | **Denormalized (∞)**   |
+| Profile text + company text      | 702ms      | 3,440ms      | Normalized (4.9x)      |
+
+**Generalization**:
+
+- **Company-first lookups**: Normalized wins
+- **Profile text + company constraint**: Denormalized wins (dramatically)
+- **Both have text filters**: Normalized may win if company filter is selective
+
+---
+
+## The Decision Matrix
+
+### Profile-Only Queries
+
+```
+Single filter?
+├─ ID/slug/updated_at (indexed) → Normalized
+├─ Numeric (connections, followers) → Normalized
+├─ Single skill array → Denormalized (slight)
+├─ Headline (common term) → Normalized
+└─ Headline (rare term) → Denormalized
+
+Multiple filters?
+├─ All numeric/indexed → Normalized
+├─ Contains rare headline term → Denormalized
+├─ Contains multiple skills → Denormalized
+└─ 3+ filters → Denormalized
+```
+
+### Company-Only Queries
+
+```
+Single filter?
+├─ ID/slug (indexed) → Normalized
+├─ employee_count > high threshold → Denormalized (odd case)
+├─ employee_count range → Normalized or tie
+└─ Description ILIKE → Normalized or tie
+
+Multiple filters?
+├─ 2 filters → Usually tie or slight normalized advantage
+└─ 3+ filters → Denormalized
+```
+
+### Cross-Table Queries (Profile + Company)
+
+```
+Text filter location?
+├─ Only on company → Normalized (company table smaller)
+├─ Only on profile → Denormalized (profile table huge)
+└─ Both → Depends on company filter selectivity
+
+Filter type combination?
+├─ Company-first (ID/name lookup) → Normalized
+├─ Profile text + company numeric → Denormalized
+├─ Profile skill + company constraint → Denormalized
+└─ Profile numeric + company text → Normalized (if company selective)
+```
+
+---
+
+## Quick Reference Table
+
+| Scenario                       | Use This                | Speedup     |
+| ------------------------------ | ----------------------- | ----------- |
+| ID lookup                      | Normalized              | 3-8x        |
+| Slug lookup                    | Normalized (with key64) | ∞           |
+| updated_at filter              | Normalized              | 3.5x        |
+| Common headline term           | Normalized              | 2.4-5.7x    |
+| **Rare headline term**         | Denormalized            | 2-2.8x      |
+| **Multi-skill search**         | Denormalized            | 1.4-2.9x    |
+| **3+ combined filters**        | Denormalized            | 2-4x        |
+| Company name via org           | Normalized (GIN)        | 31-68x      |
+| Company description search     | Normalized              | 1.2x or tie |
+| **Profile headline + company** | Denormalized            | 13-93x      |
+| Company text + profile         | Normalized              | 4.9x        |
+
+---
+
+## The Three Key Insights
+
+### 1. Text Selectivity Crossover
+
+Common terms have high early-termination probability → Normalized wins.
+Rare terms require full table scan → Denormalized wins (smaller row size).
+
+### 2. Filter Count Scaling
+
+More filters = more full-table scan work → Denormalized's smaller row size advantage compounds.
+
+### 3. Table Size Asymmetry
+
+- `linkedin_profile`: ~1.15 billion rows
+- `linkedin_company`: ~millions of rows (100-1000x smaller)
+
+Text filter on the **smaller table** (company) is fast for normalized.
+Text filter on the **larger table** (profile) favors denormalized.
+
+---
+
+## Summary: When to Use Each
+
+### Use Normalized When:
+
+1. Single indexed filter (ID, slug, updated_at)
+2. Company-first queries
+3. Common headline terms (CEO, engineer, manager)
+4. Text filter is on company side (not profile)
+5. Large LIMIT with common terms
+6. Aggregations (COUNT, GROUP BY)
+
+### Use Denormalized When:
+
+1. Rare headline terms (kubernetes, blockchain, tensorflow)
+2. Multiple skills in query
+3. 3+ combined filters
+4. Profile text filter + company constraint
+5. Country filter + headline filter
+6. Complex regex patterns on headline

package/docs/B2B_NLP_QUERY_MAPPINGS.md

@@ -0,0 +1,240 @@
+# B2B Database NLP Test Queries
+
+Natural language queries for testing the B2B database agent.
+
+---
+
+## Company Lookups
+
+1. Find the company with domain stripe.com
+2. Get company info for Anthropic
+3. Look up the company OpenAI
+4. What company has the website openai.com?
+5. Show me details about Stripe
+
+## People at Companies
+
+6. Show me people who work at Stripe
+7. Find engineers at Stripe
+8. Find sales and account executives at OpenAI
+9. Who are the software engineers at Google?
+10.   Find product managers at Meta
+
+## Company Description Searches
+
+11. Find AI video companies
+12. Find companies building personalization infrastructure
+13. Find AI platforms for sales and RevOps
+14. Find companies with developer APIs
+15. Find companies that do data integration
+16. Find cybersecurity startups
+17. Find companies building workflow automation tools
+
+## Top-of-Funnel Company Lists
+
+18. Find SaaS platform software companies
+19. List all tech companies
+20. Find B2B software companies
+21. Show me internet companies
+
+## People by Headline/Title
+
+22. Find people working in AI
+23. Find CTOs at AI startups
+24. Find founders, CEOs, CTOs, VPs, and directors
+25. Find C-suite executives
+26. Find VP of Sales professionals
+27. Find data scientists
+28. Find product managers in tech
+
+## Complex Company Searches
+
+29. Find SaaS companies with usage-based pricing
+30. Find companies with video APIs or SDKs
+31. Find companies doing real-time analytics
+32. Find enterprise security companies
+
+## Funding Queries
+
+33. Find recently funded companies
+34. Find Series A software companies
+35. Find companies funded by Andreessen Horowitz
+36. Find seed stage startups funded recently
+37. Find Series B companies in fintech
+38. Find companies that raised over $50M
+39. Show me YC-backed companies
+
+## Industry Lookups
+
+40. What are the top industries by company count?
+41. What industry is OpenAI in?
+42. List all industry categories
+
+## Employee Growth
+
+43. Find fast-growing companies with over 50% growth
+44. Find hypergrowth companies
+45. Find companies that doubled headcount this year
+46. Show me the fastest growing startups
+
+## Targeted Role + Company Searches
+
+47. Find founders at AI companies
+48. Find CTOs at fintech companies
+49. Find VPs of Engineering at SaaS companies
+50. Find sales leaders at Series A startups
+51. Find marketing executives at e-commerce companies
+
+## People at Company Types
+
+52. Find engineers at tech companies
+53. Find decision makers at AI video companies
+54. Find recruiters at fast-growing startups
+55. Find designers at consumer apps
+
+## Role + Funding Stage
+
+56. Find engineers at Series A startups
+57. Find VPs of Engineering at Series B companies
+58. Find CTOs at recently funded AI companies
+59. Find product managers at seed stage companies
+60. Find sales reps at Series C companies
+
+## Education
+
+61. Find Stanford alumni who work at Google
+62. Find people who got their MBA recently
+63. Find people with Computer Science degrees
+64. Find MIT graduates in tech
+65. Find Harvard Business School alumni
+
+## Skills
+
+66. Find people with Python and Machine Learning skills
+67. Find full-stack developers with React and Node.js
+68. Find data engineers with SQL and Spark
+69. Find iOS developers with Swift experience
+70. Find people who know Kubernetes
+
+## Certifications
+
+71. Find AWS certified professionals
+72. Find Google Cloud certified people
+73. Find PMP certified project managers
+74. Find Salesforce certified admins
+
+## Thought Leaders & Content
+
+75. Find AI thought leaders with popular articles
+76. Find people who write LinkedIn articles frequently
+77. Find tech influencers
+78. Find people writing about machine learning
+
+## Job Postings
+
+79. Show me jobs at Stripe
+80. Find high-paying engineering jobs over 200k
+81. Find remote jobs posted recently
+82. Find senior engineering roles at startups
+83. Show me product manager openings
+
+## Company Social Activity
+
+84. Find companies posting about AI
+85. Find viral company posts with high engagement
+86. Find companies announcing new products
+
+## Company Attributes
+
+87. Find companies specializing in ML or AI
+88. Find public companies with stock tickers
+89. Find companies with active career pages
+90. Find companies with multiple office locations
+
+## Alumni Searches
+
+91. Find ex-Google employees who now work at startups
+92. Find former Stripe engineers
+93. Find Meta alumni at AI companies
+94. Find people who left Amazon for startups
+
+## Analytics & Aggregations
+
+95. What roles are most common at OpenAI?
+96. What is the average funding by round type?
+97. How many engineers does Stripe have?
+98. What's the headcount breakdown at Google?
+
+## Influencers
+
+99. Find LinkedIn influencers
+100. Find people with over 50,000 followers
+101. Find well-connected professionals
+
+## Reference Data
+
+102. What are the company types?
+103. What are the job seniority levels?
+104. What are the employment types?
+105. List countries in North America
+
+## Geography
+
+106. Find tech companies in the UK
+107. Find tech companies in the Bay Area
+108. Find tech workers in New York
+109. Find startups in Austin
+110. Find AI companies in London
+
+## Ranked Companies
+
+111. Show me Fortune 500 companies
+112. Show me Inc Magazine ranked companies
+113. Find Forbes Cloud 100 companies
+
+## Volunteers & Awards
+
+114. Find people who volunteer in education
+115. Find people with innovation awards
+116. Find award-winning engineers
+
+## Recommendations & Languages
+
+117. Find highly recommended people
+118. Find multilingual professionals
+119. Find people who speak Mandarin and English
+
+## Patents & Publications
+
+120. Find patent holders
+121. Find people with AI patents
+122. Find published researchers
+123. Find people with machine learning patents
+
+## Projects & Links
+
+124. Find people with AI/ML projects
+125. Find people with GitHub profiles
+126. Find developers with open source contributions
+
+## Full Enrichment
+
+127. Get full company profile for Stripe
+128. Get complete info on this person
+129. Enrich this company with funding data
+130. Get all details about OpenAI
+
+---
+
+## Complex Multi-Criteria Queries
+
+131. Find CTOs at Series A AI companies in San Francisco
+132. Find Stanford MBAs who are now founders at fintech startups
+133. Find ex-Google engineers at YC companies
+134. Find AWS certified developers at fast-growing startups
+135. Find VPs of Sales at recently funded B2B SaaS companies
+136. Find people with ML skills who work at Series B companies
+137. Find founders at AI companies who went to Stanford
+138. Find CTOs at healthcare tech companies with recent funding
+139. Find engineers at remote-first companies with Python skills
+140. Find product managers at e-commerce companies in New York