linkedin-automation-cli 1.0.0

package/docs/superpowers/specs/2026-03-14-company-profile-extraction-design.md

@@ -0,0 +1,371 @@
+# Company Profile Extraction Feature
+
+**Date:** 2026-03-14
+**Status:** Draft
+**Author:** Claude
+
+## Overview
+
+Add a new `company info` command to extract structured data from LinkedIn company profile pages using Playwright selectors.
+
+## Command Interface
+
+```bash
+linkedin-cli company info <url> [--headless] [--debug]
+```
+
+### Arguments
+
+| Argument | Required | Description |
+|----------|----------|-------------|
+| `url` | Yes | LinkedIn company URL (e.g., `https://www.linkedin.com/company/openai/`) |
+| `--headless` | No | Run browser in headless mode (default: false, falls back to config) |
+| `--debug` | No | Enable debug mode with screenshots and HTML snapshots |
+
+### URL Validation
+
+Valid pattern: `^https:\/\/www\.linkedin\.com\/company\/[^\/]+\/?$`
+
+Requires HTTPS for security. Note: The existing `connection.ts` pattern allows HTTP, but we enforce HTTPS here for new commands.
+
+### Example Usage
+
+```bash
+# Extract company info (uses config headless setting)
+linkedin-cli company info https://www.linkedin.com/company/openai/
+
+# Force headless mode
+linkedin-cli company info https://www.linkedin.com/company/openai/ --headless
+
+# Debug mode for troubleshooting
+linkedin-cli company info https://www.linkedin.com/company/openai/ --debug
+```
+
+## Output Format
+
+JSON to stdout with the following structure:
+
+```json
+{
+  "name": "OpenAI",
+  "linkedin_url": "https://www.linkedin.com/company/openai/",
+  "website": "https://openai.com",
+  "industry": "Research Services",
+  "company_size": "1,001-5,000 employees",
+  "headquarters": "San Francisco, California",
+  "founded": "2015",
+  "specialties": ["Artificial Intelligence", "Machine Learning", "Research"],
+  "type": "Privately Held",
+  "follower_count": 2500000
+}
+```
+
+### Field Specifications
+
+| Field | Type | Source Location | Nullable |
+|-------|------|-----------------|----------|
+| `name` | string | Top card heading | No |
+| `linkedin_url` | string | Input URL | No |
+| `website` | string \| null | About section | Yes |
+| `industry` | string \| null | About section | Yes |
+| `company_size` | string \| null | About section | Yes |
+| `headquarters` | string \| null | About section | Yes |
+| `founded` | string \| null | About section | Yes |
+| `specialties` | string[] \| null | About section | Yes |
+| `type` | string \| null | About section | Yes |
+| `follower_count` | number \| null | Sidebar/top card | Yes |
+
+## Architecture
+
+### File Structure
+
+```
+src/
+├── cli/
+│   ├── index.ts             # Add: export { registerCompanyCommands }
+│   └── company.ts           # New: Command registration
+├── linkedin/
+│   ├── selectors.ts         # Modify: Add company section to SELECTORS
+│   └── company-extractor.ts # New: CompanyExtractor class
+└── types/
+    └── index.ts             # Add: CompanyProfile interface
+```
+
+### Components
+
+#### 1. `src/cli/company.ts`
+
+Command registration following existing patterns (`connection.ts`, `messages.ts`):
+
+- Validates company URL pattern
+- Launches browser via `BrowserController` with headless from options or config
+- **No auth required** - company pages are public (just dismiss authwall popup)
+- Calls `CompanyExtractor.extract()`
+- Logs action via `getAuditLogger()`
+- Outputs JSON to stdout
+- Handles errors with appropriate exit codes
+
+**Browser instantiation pattern:**
+```typescript
+browser = new BrowserController({
+  headless: options.headless !== undefined ? options.headless : config.getValue('headless'),
+  debug: options.debug,
+});
+```
+
+**Browser cleanup pattern:**
+```typescript
+try {
+  // ... extraction logic
+  await browser.close();
+} catch (error) {
+  // Always close browser on error
+  if (browser) await browser.close();
+  throw error;
+}
+```
+
+#### 2. `src/linkedin/company-extractor.ts`
+
+Main extraction logic:
+
+```typescript
+export interface CompanyProfile {
+  name: string;
+  linkedin_url: string;
+  website: string | null;
+  industry: string | null;
+  company_size: string | null;
+  headquarters: string | null;
+  founded: string | null;
+  specialties: string[] | null;
+  type: string | null;
+  follower_count: number | null;
+}
+
+export class CompanyExtractor {
+  constructor(private page: Page) {}
+
+  async extract(url: string): Promise<CompanyProfile>;
+}
+```
+
+### Data Flow
+
+```
+User Input (URL)
+      │
+      ▼
+┌─────────────────┐
+│  URL Validation │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│ Browser Launch  │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│ Navigate to URL │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│Dismiss Authwall │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│ Extract Fields  │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│  JSON Output    │
+└─────────────────┘
+```
+
+**Note:** No authentication required. LinkedIn company pages are public. An authwall popup may appear but can be dismissed.
+
+## Selector Strategy
+
+LinkedIn company pages store data in the About section as key-value pairs. The extractor uses multi-layer fallback selectors for resilience, matching the existing `SELECTORS` pattern in `selectors.ts`.
+
+### COMPANY_SELECTORS Constant
+
+Add nested `company` section to existing `SELECTORS` in `src/linkedin/selectors.ts`:
+
+```typescript
+// Add to existing SELECTORS object
+company: {
+  name: [
+    'h1.text-heading-xlarge',
+    '.org-top-card-primary-content h1',
+    'section[data-testid="about-us"] h1',
+  ],
+  website: [
+    'a[data-testid="website-link"]',
+    '.org-about-us-module__website a',
+    'a[href*="company"][target="_blank"]',
+  ],
+  industry: [
+    'dt:has-text("Industry") + dd',
+    '.org-about-company-module__dl dt:has-text("Industry") + dd',
+  ],
+  company_size: [
+    'dt:has-text("Company size") + dd',
+    'dt:has-text("Employees") + dd',
+    '.org-about-company-module__dl dt:has-text("Company size") + dd',
+  ],
+  headquarters: [
+    'dt:has-text("Headquarters") + dd',
+    '.org-about-company-module__dl dt:has-text("Headquarters") + dd',
+  ],
+  founded: [
+    'dt:has-text("Founded") + dd',
+    '.org-about-company-module__dl dt:has-text("Founded") + dd',
+  ],
+  specialties: [
+    'dt:has-text("Specialties") + dd',
+    '.org-about-company-module__dl dt:has-text("Specialties") + dd',
+  ],
+  type: [
+    'dt:has-text("Company type") + dd',
+    'dt:has-text("Type") + dd',
+    '.org-about-company-module__dl dt:has-text("Company type") + dd',
+  ],
+  follower_count: [
+    '.org-top-card-primary-content__followers-count',
+    '.org-top-card-module__followers-count',
+    'span:has-text("followers")',
+  ],
+}
+```
+
+This follows the existing nested structure (`login`, `messages`, `connection`) for consistency.
+
+### Fallback Strategy
+
+1. Try each selector in array order
+2. Use `page.locator(selector).first()` to handle multiple matches
+3. Return `null` if no selector matches
+
+## Parsing Rules
+
+### Follower Count Parsing
+
+Convert string formats to number:
+
+| Input | Output |
+|-------|--------|
+| `"2.5M followers"` | `2500000` |
+| `"12K followers"` | `12000` |
+| `"1,234 followers"` | `1234` |
+| `"500+ followers"` | `500` |
+
+Algorithm:
+1. Extract numeric part: `/([\d,.]+)\s*([KMkmb]?)/`
+2. Remove commas
+3. Apply multiplier: K/k = 1000, M/m = 1000000
+4. Return integer
+
+### Specialties Parsing
+
+Split by comma, trim whitespace:
+
+| Input | Output |
+|-------|--------|
+| `"AI, ML, Research"` | `["AI", "ML", "Research"]` |
+| `"Artificial Intelligence"` | `["Artificial Intelligence"]` |
+
+## Error Handling
+
+| Error | Exit Code | Message |
+|-------|-----------|---------|
+| Invalid URL | 1 | "Invalid company URL. Expected format: https://www.linkedin.com/company/<name>/" |
+| Navigation timeout | 1 | "Failed to load company page (timeout: 30000ms)" |
+| Extraction failure | 1 | "Failed to extract company data: <reason>" |
+| Success | 0 | JSON output |
+
+## Popup Dismissal
+
+LinkedIn may show an authwall popup on company pages. The extractor should dismiss it:
+
+```typescript
+const DISMISS_SELECTORS = [
+  'button[aria-label="Dismiss"]',
+  'button[aria-label="Close"]',
+  '.artdeco-modal__dismiss',
+  'button.artdeco-modal__dismiss',
+];
+
+async function dismissAuthwall(page: Page): Promise<void> {
+  for (const selector of DISMISS_SELECTORS) {
+    try {
+      const btn = page.locator(selector).first();
+      if (await btn.isVisible({ timeout: 1000 })) {
+        await btn.click();
+        await page.waitForTimeout(500);
+        return;
+      }
+    } catch {
+      // Try next selector
+    }
+  }
+}
+```
+
+### Timeout Configuration
+
+- Navigation timeout: 30000ms (matches existing `page.goto()` calls)
+- Selector wait timeout: 5000ms per field
+
+## Audit Logging
+
+Log extraction attempts using existing `getAuditLogger()`:
+
+```typescript
+// On success
+logger.log('company.info', { url, fieldsExtracted: Object.keys(profile).filter(k => profile[k] !== null) }, true);
+
+// On failure
+logger.log('company.info', { url, error: error.message }, false);
+```
+
+## Testing Strategy
+
+### Unit Tests
+
+- URL validation (valid/invalid patterns)
+- Selector fallback logic
+- Follower count parsing (various formats)
+- Specialties parsing (comma-separated string → array)
+
+### Integration Tests
+
+- Mock Playwright page with sample HTML
+- Test extraction with fixture data
+
+### Manual Testing
+
+```bash
+# Test with real company pages
+linkedin-cli company info https://www.linkedin.com/company/openai/
+linkedin-cli company info https://www.linkedin.com/company/microsoft/
+```
+
+## Dependencies
+
+- Existing: `BrowserController`, `getConfig()`
+- No new external dependencies required
+- **No auth required** - company pages are public
+
+## Implementation Notes
+
+1. Reuse existing browser infrastructure (no auth needed)
+2. Follow existing command patterns from `connection.ts`
+3. Add `company` section to existing `SELECTORS` in `selectors.ts`
+4. Dismiss authwall popup before extraction
+5. Use `page.waitForSelector()` with timeout for reliability
+6. Implement robust parsing for follower count with K/M suffixes