search-talent 0.1.0__tar.gz

search_talent-0.1.0/PKG-INFO

@@ -0,0 +1,195 @@
+Metadata-Version: 2.4
+Name: search-talent
+Version: 0.1.0
+Summary: LinkedIn Recruiter 搜索爬虫 & JSON 转 Excel 工具
+Author-email: Chandler <275737875@qq.com>
+License: MIT
+Keywords: linkedin,recruiter,scraper,search,talent
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
+Classifier: Topic :: Office/Business
+Requires-Python: >=3.9
+Requires-Dist: beautifulsoup4>=4.12.0
+Requires-Dist: browser-dog>=0.1.0
+Requires-Dist: openpyxl>=3.0.0
+Requires-Dist: pandas>=1.5.0
+Requires-Dist: selenium>=4.0.0
+Requires-Dist: typer>=0.9.0
+Description-Content-Type: text/markdown
+
+# search-talent
+
+[![PyPI version](https://img.shields.io/pypi/v/search-talent.svg)](https://pypi.org/project/search-talent/)
+[![Python](https://img.shields.io/pypi/pyversions/search-talent.svg)](https://pypi.org/project/search-talent/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+基于 Selenium 的 LinkedIn Recruiter 搜索爬虫工具，可自动抓取候选人信息并导出为 Excel 文件。
+
+## 功能特性
+
+- **自动爬取**：模拟真人滚动行为，智能加载 LinkedIn Recruiter 搜索结果页的所有候选人信息。
+- **结构化数据提取**：解析候选人姓名、头衔、地点、工作经历、教育背景、行业、关系度等字段。
+- **断点续爬**：已保存的页面 JSON 文件会自动跳过，支持中断后继续爬取。
+- **Excel 导出**：一键将 JSON 数据整理并导出为格式化的 `.xlsx` 文件，驼峰命名字段自动拆分，方便阅读。
+- **CLI 驱动**：安装后直接通过终端命令 `search-talent` 运行，无需编写额外脚本。
+
+## 安装
+
+### 通过 pip 安装（推荐）
+
+```bash
+pip install search-talent
+```
+
+### 从源码安装
+
+```bash
+git clone https://github.com/your-username/search-talent.git
+cd search-talent
+pip install .
+```
+
+### 依赖项
+
+安装时会自动安装以下依赖：
+
+| 依赖 | 用途 |
+|------|------|
+| `typer` | CLI 框架 |
+| `selenium` | 浏览器自动化 |
+| `beautifulsoup4` | HTML 解析 |
+| `pandas` | 数据处理 |
+| `openpyxl` | Excel 文件读写（pandas 后端） |
+| `browser-dog` | LinkedIn 登录与 Cookie 管理 |
+
+> **注意**：运行爬虫需要本机已安装 Chrome/Chromium 浏览器及对应版本的 ChromeDriver。
+
+## 前置准备
+
+### Cookie 文件
+
+本工具通过 Cookie 注入方式登录 LinkedIn Recruiter，你需要提前准备好一个 `cookie.json` 文件。
+
+文件内容为 JSON 数组格式，每条记录是一个 Cookie 对象：
+
+```json
+[
+  {
+    "domain": ".linkedin.com",
+    "name": "li_at",
+    "value": "YOUR_COOKIE_VALUE",
+    "path": "/",
+    "secure": true,
+    "httpOnly": true
+  }
+]
+```
+
+> **提示**：可使用浏览器扩展（如 EditThisCookie）导出已登录的 LinkedIn Cookie。
+
+## CLI 使用指南
+
+安装完成后，可在终端直接运行 `search-talent` 命令。
+
+### 查看帮助
+
+```bash
+search-talent --help
+```
+
+### `scrape` 命令 - 爬取 LinkedIn 数据
+
+从 LinkedIn Recruiter 搜索结果页爬取候选人信息，并保存为 JSON 文件。
+
+```bash
+search-talent scrape \
+  --start-url "https://www.linkedin.com/talent/search?q=..." \
+  --search-keyword "python-developer" \
+  --folder ./output
+```
+
+**参数说明：**
+
+| 参数 | 缩写 | 说明 | 默认值 |
+|------|------|------|--------|
+| `--start-url` | `-s` | LinkedIn Recruiter 搜索结果 URL（必填） | — |
+| `--search-keyword` | `-k` | 搜索关键词，用于 JSON 文件命名（必填） | — |
+| `--folder` | `-f` | 保存 JSON 文件的本地目录（必填） | — |
+| `--cookie` | `-c` | Cookie JSON 文件路径 | `cookie.json` |
+| `--scroll-times` | `-n` | 页面滚动次数 | `3` |
+| `--max-pages` | `-m` | 最大爬取页数（不指定则不限制） | `None` |
+| `--headless` | — | 以无头模式运行浏览器 | `False` |
+
+**示例：**
+
+```bash
+# 爬取前 5 页，使用自定义 Cookie 文件
+search-talent scrape \
+  -s "https://www.linkedin.com/talent/search?q=engineer&location=China" \
+  -k "engineer-china" \
+  -f ./data/engineer-china \
+  -c ~/linkedin_cookie.json \
+  -m 5
+
+# 无头模式运行
+search-talent scrape \
+  -s "https://www.linkedin.com/talent/search?q=..." \
+  -k "my-search" \
+  -f ./data \
+  --headless
+```
+
+### `export` 命令 - 导出为 Excel
+
+将 `scrape` 命令生成的 JSON 文件合并导出为一份 Excel 文件。
+
+```bash
+search-talent export \
+  --input ./output \
+  --output candidates.xlsx
+```
+
+**参数说明：**
+
+| 参数 | 缩写 | 说明 | 默认值 |
+|------|------|------|--------|
+| `--input` | `-i` | 包含 JSON 文件的输入目录（必填） | — |
+| `--output` | `-o` | 输出 Excel 文件路径（必填） | — |
+
+**示例：**
+
+```bash
+search-talent export -i ./data/engineer-china -o engineer-china.xlsx
+```
+
+## 项目结构
+
+```
+search-talent/
+├── pyproject.toml          # 项目配置与构建定义
+├── README.md
+└── src/
+    └── search_talent/
+        ├── __init__.py     # 包入口，版本信息
+        ├── scraper.py      # LinkedIn 爬虫核心模块
+        ├── reader.py       # JSON 读取与 Excel 导出模块
+        └── cli.py          # Typer CLI 入口
+```
+
+## 注意事项
+
+- **合规使用**：请遵守 LinkedIn 的使用条款，合理控制爬取频率和数量，避免账号被封禁。
+- **Cookie 有效期**：LinkedIn Cookie 会过期，若登录失败请重新导出 Cookie 文件。
+- **反爬机制**：工具内置了随机等待、拟人滚动等反检测策略，但仍建议控制爬取速度。
+- **ChromeDriver 版本**：请确保本机 ChromeDriver 版本与 Chrome 浏览器版本匹配。
+
+## 许可证
+
+[MIT License](https://opensource.org/licenses/MIT)

search_talent-0.1.0/README.md

@@ -0,0 +1,169 @@
+# search-talent
+
+[![PyPI version](https://img.shields.io/pypi/v/search-talent.svg)](https://pypi.org/project/search-talent/)
+[![Python](https://img.shields.io/pypi/pyversions/search-talent.svg)](https://pypi.org/project/search-talent/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+基于 Selenium 的 LinkedIn Recruiter 搜索爬虫工具，可自动抓取候选人信息并导出为 Excel 文件。
+
+## 功能特性
+
+- **自动爬取**：模拟真人滚动行为，智能加载 LinkedIn Recruiter 搜索结果页的所有候选人信息。
+- **结构化数据提取**：解析候选人姓名、头衔、地点、工作经历、教育背景、行业、关系度等字段。
+- **断点续爬**：已保存的页面 JSON 文件会自动跳过，支持中断后继续爬取。
+- **Excel 导出**：一键将 JSON 数据整理并导出为格式化的 `.xlsx` 文件，驼峰命名字段自动拆分，方便阅读。
+- **CLI 驱动**：安装后直接通过终端命令 `search-talent` 运行，无需编写额外脚本。
+
+## 安装
+
+### 通过 pip 安装（推荐）
+
+```bash
+pip install search-talent
+```
+
+### 从源码安装
+
+```bash
+git clone https://github.com/your-username/search-talent.git
+cd search-talent
+pip install .
+```
+
+### 依赖项
+
+安装时会自动安装以下依赖：
+
+| 依赖 | 用途 |
+|------|------|
+| `typer` | CLI 框架 |
+| `selenium` | 浏览器自动化 |
+| `beautifulsoup4` | HTML 解析 |
+| `pandas` | 数据处理 |
+| `openpyxl` | Excel 文件读写（pandas 后端） |
+| `browser-dog` | LinkedIn 登录与 Cookie 管理 |
+
+> **注意**：运行爬虫需要本机已安装 Chrome/Chromium 浏览器及对应版本的 ChromeDriver。
+
+## 前置准备
+
+### Cookie 文件
+
+本工具通过 Cookie 注入方式登录 LinkedIn Recruiter，你需要提前准备好一个 `cookie.json` 文件。
+
+文件内容为 JSON 数组格式，每条记录是一个 Cookie 对象：
+
+```json
+[
+  {
+    "domain": ".linkedin.com",
+    "name": "li_at",
+    "value": "YOUR_COOKIE_VALUE",
+    "path": "/",
+    "secure": true,
+    "httpOnly": true
+  }
+]
+```
+
+> **提示**：可使用浏览器扩展（如 EditThisCookie）导出已登录的 LinkedIn Cookie。
+
+## CLI 使用指南
+
+安装完成后，可在终端直接运行 `search-talent` 命令。
+
+### 查看帮助
+
+```bash
+search-talent --help
+```
+
+### `scrape` 命令 - 爬取 LinkedIn 数据
+
+从 LinkedIn Recruiter 搜索结果页爬取候选人信息，并保存为 JSON 文件。
+
+```bash
+search-talent scrape \
+  --start-url "https://www.linkedin.com/talent/search?q=..." \
+  --search-keyword "python-developer" \
+  --folder ./output
+```
+
+**参数说明：**
+
+| 参数 | 缩写 | 说明 | 默认值 |
+|------|------|------|--------|
+| `--start-url` | `-s` | LinkedIn Recruiter 搜索结果 URL（必填） | — |
+| `--search-keyword` | `-k` | 搜索关键词，用于 JSON 文件命名（必填） | — |
+| `--folder` | `-f` | 保存 JSON 文件的本地目录（必填） | — |
+| `--cookie` | `-c` | Cookie JSON 文件路径 | `cookie.json` |
+| `--scroll-times` | `-n` | 页面滚动次数 | `3` |
+| `--max-pages` | `-m` | 最大爬取页数（不指定则不限制） | `None` |
+| `--headless` | — | 以无头模式运行浏览器 | `False` |
+
+**示例：**
+
+```bash
+# 爬取前 5 页，使用自定义 Cookie 文件
+search-talent scrape \
+  -s "https://www.linkedin.com/talent/search?q=engineer&location=China" \
+  -k "engineer-china" \
+  -f ./data/engineer-china \
+  -c ~/linkedin_cookie.json \
+  -m 5
+
+# 无头模式运行
+search-talent scrape \
+  -s "https://www.linkedin.com/talent/search?q=..." \
+  -k "my-search" \
+  -f ./data \
+  --headless
+```
+
+### `export` 命令 - 导出为 Excel
+
+将 `scrape` 命令生成的 JSON 文件合并导出为一份 Excel 文件。
+
+```bash
+search-talent export \
+  --input ./output \
+  --output candidates.xlsx
+```
+
+**参数说明：**
+
+| 参数 | 缩写 | 说明 | 默认值 |
+|------|------|------|--------|
+| `--input` | `-i` | 包含 JSON 文件的输入目录（必填） | — |
+| `--output` | `-o` | 输出 Excel 文件路径（必填） | — |
+
+**示例：**
+
+```bash
+search-talent export -i ./data/engineer-china -o engineer-china.xlsx
+```
+
+## 项目结构
+
+```
+search-talent/
+├── pyproject.toml          # 项目配置与构建定义
+├── README.md
+└── src/
+    └── search_talent/
+        ├── __init__.py     # 包入口，版本信息
+        ├── scraper.py      # LinkedIn 爬虫核心模块
+        ├── reader.py       # JSON 读取与 Excel 导出模块
+        └── cli.py          # Typer CLI 入口
+```
+
+## 注意事项
+
+- **合规使用**：请遵守 LinkedIn 的使用条款，合理控制爬取频率和数量，避免账号被封禁。
+- **Cookie 有效期**：LinkedIn Cookie 会过期，若登录失败请重新导出 Cookie 文件。
+- **反爬机制**：工具内置了随机等待、拟人滚动等反检测策略，但仍建议控制爬取速度。
+- **ChromeDriver 版本**：请确保本机 ChromeDriver 版本与 Chrome 浏览器版本匹配。
+
+## 许可证
+
+[MIT License](https://opensource.org/licenses/MIT)

search_talent-0.1.0/pyproject.toml

@@ -0,0 +1,42 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/search_talent"]
+
+[project]
+name = "search-talent"
+version = "0.1.0"
+description = "LinkedIn Recruiter 搜索爬虫 & JSON 转 Excel 工具"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [
+    { name = "Chandler", email = "275737875@qq.com" },
+]
+keywords = ["linkedin", "recruiter", "scraper", "talent", "search"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Internet :: WWW/HTTP :: Browsers",
+    "Topic :: Office/Business",
+]
+dependencies = [
+    "typer>=0.9.0",
+    "selenium>=4.0.0",
+    "beautifulsoup4>=4.12.0",
+    "pandas>=1.5.0",
+    "openpyxl>=3.0.0",
+    "browser-dog>=0.1.0",
+]
+
+[project.scripts]
+search-talent = "search_talent.cli:app"
+

search_talent-0.1.0/src/search_talent/__init__.py

@@ -0,0 +1,3 @@
+"""search-talent: LinkedIn Recruiter 搜索爬虫 & JSON 转 Excel 工具"""
+
+__version__ = "0.1.0"

search_talent-0.1.0/src/search_talent/cli.py

@@ -0,0 +1,69 @@
+"""Talent Search CLI - 基于 Typer 的命令行工具"""
+
+from typing import Optional
+
+import typer
+
+from .scraper import (
+    LinkedInProfileParser,
+    LinkedInPageParser,
+    SeleniumPageNavigator,
+    JsonDataSaver,
+    LinkedInScraper,
+    login,
+)
+from .reader import read_json_files_and_save_to_excel
+
+app = typer.Typer(
+    name="talent-search",
+    help="LinkedIn Recruiter 搜索爬虫 & JSON 转 Excel 工具",
+    add_completion=False,
+)
+
+
+@app.command()
+def scrape(
+    start_url: str = typer.Option(..., "--start-url", "-s", help="LinkedIn search URL"),
+    search_keyword: str = typer.Option(..., "--search-keyword", "-k", help="Search keyword for file naming"),
+    folder: str = typer.Option(..., "--folder", "-f", help="Local folder to save scraped data"),
+    cookie_file: str = typer.Option("cookie.json", "--cookie", "-c", help="Cookie JSON file path"),
+    scroll_times: int = typer.Option(3, "--scroll-times", "-n", help="Number of times to scroll the page"),
+    max_pages: Optional[int] = typer.Option(None, "--max-pages", "-m", help="Maximum pages to scrape (None=unlimited)"),
+    headless: bool = typer.Option(False, "--headless", help="Run browser in headless mode"),
+):
+    """Scrape LinkedIn Recruiter search results and save as JSON files."""
+    profile_parser = LinkedInProfileParser()
+    page_parser = LinkedInPageParser(profile_parser)
+    navigator = SeleniumPageNavigator()
+    data_saver = JsonDataSaver()
+
+    driver = login(cookie_file=cookie_file, headless=headless)
+
+    scraper = LinkedInScraper(driver, page_parser, navigator, data_saver)
+    scraper.scrape(
+        start_url=start_url,
+        search_keyword=search_keyword,
+        folder=folder,
+        scroll_times=scroll_times,
+        max_pages=max_pages,
+    )
+
+    typer.echo(f"Scraping completed. Data saved to: {folder}")
+
+
+@app.command()
+def export(
+    input: str = typer.Option(..., "--input", "-i", help="Input folder containing JSON files"),
+    output: str = typer.Option(..., "--output", "-o", help="Output Excel file path (e.g. output.xlsx)"),
+):
+    """Read JSON files from folder and export to Excel."""
+    try:
+        read_json_files_and_save_to_excel(input, output)
+        typer.echo(f"Export completed: {output}")
+    except Exception as e:
+        typer.echo(f"Error: {e}", err=True)
+        raise typer.Exit(code=1)
+
+
+if __name__ == "__main__":
+    app()

search_talent-0.1.0/src/search_talent/reader.py

@@ -0,0 +1,101 @@
+"""JSON 文件读取与 Excel 导出模块"""
+
+import json
+import os
+import re
+from typing import Optional
+
+import pandas as pd
+
+
+def split_camel_case(word: str) -> str:
+    """将驼峰命名字符串拆分为用空格分隔的单词。
+
+    例如: 'DigitalSalesStrategy' -> 'Digital Sales Strategy'
+    """
+    return re.sub(r"(?<=[a-z])(?=[A-Z])|(?<=[A-Z])(?=[A-Z][a-z])", " ", word)
+
+
+def split_compound_words(text: str) -> str:
+    """识别字符串中的驼峰式复合词，并将其拆分为带空格的单词序列，其余部分原样保留。"""
+    pattern = r"[A-Z][a-z]+(?:[A-Z][a-z]+)+"
+
+    def replacer(match):
+        return split_camel_case(match.group(0))
+
+    return re.sub(pattern, replacer, text)
+
+
+def col_format(exps: list) -> str:
+    """格式化 experiences / education 列表为可读字符串"""
+    exp_list = []
+    for exp in exps:
+        exp_format = exp["title_company"].replace("\u00b7", "  (") + ")"
+        exp_format_str = split_compound_words(exp_format)
+        exp_list.append(exp_format_str)
+    return "\n".join(exp_list)
+
+
+def read_json_files_and_save_to_excel(folder_path: str, output_excel_path: str) -> None:
+    """读取指定文件夹下所有 JSON 文件（每个是 list），保存为 Excel 文件。
+
+    参数:
+        folder_path: JSON 文件所在文件夹路径
+        output_excel_path: 输出的 Excel 文件路径
+    """
+    if not os.path.exists(folder_path):
+        raise FileNotFoundError(f"文件夹不存在: {folder_path}")
+
+    all_data = []
+
+    for filename in os.listdir(folder_path):
+        file_path = os.path.join(folder_path, filename)
+
+        if not (os.path.isfile(file_path) and filename.lower().endswith(".json")):
+            continue
+
+        try:
+            with open(file_path, "r", encoding="utf-8") as f:
+                data = json.load(f)
+
+            if not isinstance(data, list):
+                print(f"跳过: {filename}，内容不是 list。")
+                continue
+
+            for item in data:
+                if isinstance(item, dict):
+                    item["source_file"] = filename
+                else:
+                    item = {"value": item, "source_file": filename}
+
+                exp = col_format(item["experiences"])
+                edu = col_format(item["education"])
+                item["experiences"] = exp
+                item["education"] = edu
+
+                title = split_compound_words(item["headline"])
+                item["headline"] = title
+                all_data.append(item)
+
+            print(f"已读取: {filename} -> {len(data)} 个元素")
+
+        except Exception as e:
+            print(f"读取失败: {filename}, 错误: {e}")
+
+    if all_data:
+        df = pd.DataFrame(all_data)
+        columns_order = [
+            "name",
+            "headline",
+            "location",
+            "experiences",
+            "education",
+            "profile_url",
+            "industry",
+            "source_file",
+        ]
+        df = df[columns_order]
+        df.to_excel(output_excel_path, index=False)
+        print(f"\n所有数据已成功保存至: {output_excel_path}")
+    else:
+        print("没有有效数据可保存。")

search_talent-0.1.0/src/search_talent/scraper.py

@@ -0,0 +1,386 @@
+"""LinkedIn Recruiter 搜索爬虫模块"""
+
+from abc import ABC, abstractmethod
+import json
+import os
+import random
+import re
+import time
+from typing import Dict, List, Optional
+
+from bs4 import BeautifulSoup
+from selenium.webdriver.remote.webdriver import WebDriver
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.support import expected_conditions as EC
+from selenium.common.exceptions import TimeoutException
+from selenium.webdriver.common.by import By
+
+from browser_dog.browser import BrowserDog
+
+
+# ===================== 登录 =====================
+
+def login(cookie_file: str = "cookie.json", headless: bool = False) -> WebDriver:
+    """使用 BrowserDog 登录 LinkedIn Talent"""
+    dog = BrowserDog(
+        cookies_json=cookie_file,
+        base_url="https://www.linkedin.com/talent/home",
+        headless=headless,
+    )
+
+    if dog.detect_block_or_captcha():
+        raise RuntimeError("Login failed or blocked!")
+
+    driver = dog.get_driver()
+    driver.maximize_window()
+    print("正在打开 LinkedIn Talent 首页...")
+    driver.get("https://www.linkedin.com/talent/home")
+    dog.long_wait()
+    return driver
+
+
+# ===================== 1. 解析器接口 =====================
+
+class ProfileParser(ABC):
+    """个人资料解析器抽象接口"""
+
+    @abstractmethod
+    def parse(self, soup: BeautifulSoup) -> Dict:
+        """解析单个个人资料卡片"""
+        pass
+
+
+class LinkedInProfileParser(ProfileParser):
+    """LinkedIn 个人资料卡片解析器"""
+
+    def parse(self, soup: BeautifulSoup) -> Dict:
+        data = {
+            "name": self._extract_name(soup),
+            "profile_url": self._extract_profile_url(soup),
+            "avatar_url": self._extract_avatar_url(soup),
+            "headline": self._extract_headline(soup),
+            "location": self._extract_location(soup),
+            "industry": self._extract_industry(soup),
+            "connection_degree": self._extract_connection_degree(soup),
+            "experiences": self._extract_experiences(soup),
+            "education": self._extract_education(soup),
+            "decorations": self._extract_decorations(soup),
+        }
+        return data
+
+    def _extract_name(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one('[data-test-row-lockup-full-name] a')
+        return elem.get_text(strip=True) if elem else None
+
+    def _extract_profile_url(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one('[data-test-row-lockup-full-name] a')
+        return elem.get("href") if elem else None
+
+    def _extract_avatar_url(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one("[data-test-lockup-image]")
+        return elem.get("src") if elem else None
+
+    def _extract_headline(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one("[data-test-row-lockup-headline]")
+        return elem.get_text(strip=True) if elem else None
+
+    def _extract_location(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one("[data-test-row-lockup-location]")
+        return elem.get_text(strip=True) if elem else None
+
+    def _extract_industry(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one("[data-test-current-employer-industry]")
+        return elem.get_text(strip=True) if elem else None
+
+    def _extract_connection_degree(self, soup: BeautifulSoup) -> Optional[str]:
+        elem = soup.select_one('.artdeco-entity-lockup__badge span[aria-hidden="true"]')
+        if elem:
+            text = elem.get_text(strip=True)
+            return re.sub(r"^·\s*", "", text).strip()
+        badge = soup.select_one(".artdeco-entity-lockup__badge")
+        if badge:
+            text = badge.get_text(strip=True)
+            match = re.search(r"(\d+(?:st|nd|rd|th))", text)
+            return match.group(1) if match else text
+        return None
+
+    def _extract_experiences(self, soup: BeautifulSoup) -> List[Dict]:
+        return self._extract_history_group(soup, "Experience")
+
+    def _extract_education(self, soup: BeautifulSoup) -> List[Dict]:
+        return self._extract_history_group(soup, "Education")
+
+    def _extract_history_group(self, soup: BeautifulSoup, group_title: str) -> List[Dict]:
+        groups = soup.select(".history-group")
+        for group in groups:
+            title_elem = group.select_one(".history-group__term")
+            if title_elem and group_title in title_elem.get_text():
+                return [self._parse_history_item(li) for li in group.select(".history-group__list-items li")]
+        return []
+
+    def _parse_history_item(self, li: BeautifulSoup) -> Dict:
+        main_text = li.get_text(strip=True)
+        date_elem = li.select_one(".description-entry__date-duration")
+        date_range = None
+        if date_elem:
+            times = date_elem.find_all("time")
+            if len(times) == 2:
+                date_range = f"{times[0].get_text(strip=True)} – {times[1].get_text(strip=True)}"
+            elif len(times) == 1:
+                date_range = times[0].get_text(strip=True)
+        return {"title": main_text, "date_range": date_range}
+
+    def _extract_decorations(self, soup: BeautifulSoup) -> Dict:
+        decorations: Dict[str, List[str]] = {"interest": [], "activity": []}
+        for row in soup.select(".decorations__row"):
+            label = row.select_one(".decorations__row-title")
+            if not label:
+                continue
+            label_text = label.get_text(strip=True).lower()
+            triggers = row.select(".base-decoration__trigger-text")
+            if label_text == "interest":
+                decorations["interest"] = [t.get_text(strip=True) for t in triggers]
+            elif label_text == "activity":
+                decorations["activity"] = [t.get_text(strip=True) for t in triggers]
+        return decorations
+
+
+# ===================== 2. 页面解析器 =====================
+
+class PageParser(ABC):
+    """页面解析器抽象接口"""
+
+    @abstractmethod
+    def parse_profiles(self, page_source: str) -> List[Dict]:
+        """从页面源码中解析所有个人资料"""
+        pass
+
+
+class LinkedInPageParser(PageParser):
+    """LinkedIn 搜索结果页面解析器"""
+
+    def __init__(self, profile_parser: ProfileParser):
+        self.profile_parser = profile_parser
+
+    def parse_profiles(self, page_source: str) -> List[Dict]:
+        soup = BeautifulSoup(page_source, "html.parser")
+        profile_items = soup.select("li.profile-list__border-bottom")
+        return [self.profile_parser.parse(item) for item in profile_items]
+
+
+# ===================== 3. 页面导航接口 =====================
+
+class PageNavigator(ABC):
+    """页面导航抽象接口"""
+
+    @abstractmethod
+    def scroll_and_wait(self, driver: WebDriver, **kwargs) -> None:
+        """滚动页面并等待"""
+        pass
+
+    @abstractmethod
+    def click_next(self, driver: WebDriver, timeout: int = 10) -> bool:
+        """点击下一页按钮，返回是否成功"""
+        pass
+
+
+class SeleniumPageNavigator(PageNavigator):
+    """基于 Selenium 的页面导航实现"""
+
+    def scroll_and_wait(
+        self,
+        driver: WebDriver,
+        max_scroll_rounds: int = 30,
+        height_stable_count: int = 3,
+        min_wait: float = 1.0,
+        max_wait: float = 3.0,
+        scroll_step_ratio: tuple = (0.3, 0.8),
+    ) -> None:
+        """动态滚动加载页面所有内容"""
+        print("开始智能滚动加载（动态检测内容）...")
+
+        viewport_height = driver.execute_script("return window.innerHeight")
+
+        def get_total_height() -> int:
+            return driver.execute_script("return document.body.scrollHeight")
+
+        def get_scroll_y() -> int:
+            return driver.execute_script("return window.scrollY")
+
+        def scroll_to(y: int) -> None:
+            driver.execute_script(f"window.scrollTo(0, {y});")
+
+        def scroll_by(delta: int) -> None:
+            driver.execute_script(f"window.scrollBy(0, {delta});")
+
+        def random_wait() -> None:
+            time.sleep(random.uniform(min_wait, max_wait))
+
+        last_height = get_total_height()
+        stable_rounds = 0
+        current_round = 0
+        current_pos = get_scroll_y()
+
+        while current_round < max_scroll_rounds and stable_rounds < height_stable_count:
+            step = random.uniform(*scroll_step_ratio) * viewport_height
+            target_y = min(current_pos + step, last_height)
+
+            if last_height - target_y < viewport_height * 0.2:
+                target_y = last_height
+
+            scroll_to(target_y)
+            random_wait()
+
+            current_pos = get_scroll_y()
+            new_height = get_total_height()
+
+            if new_height > last_height:
+                print(f"轮次 {current_round + 1}: 高度从 {last_height} 增加到 {new_height}")
+                last_height = new_height
+                stable_rounds = 0
+                if random.random() < 0.3:
+                    back_distance = random.uniform(0.1, 0.3) * viewport_height
+                    scroll_by(-back_distance)
+                    random_wait()
+                    scroll_by(back_distance)
+                    random_wait()
+            else:
+                stable_rounds += 1
+                print(f"轮次 {current_round + 1}: 高度未变化 ({new_height})，稳定计数 {stable_rounds}")
+
+            if (last_height - current_pos) < viewport_height * 0.1 and stable_rounds < height_stable_count:
+                print("接近底部但高度未变，强制滚动到底部并等待加载...")
+                scroll_to(last_height)
+                random_wait()
+                new_height = get_total_height()
+                if new_height > last_height:
+                    last_height = new_height
+                    stable_rounds = 0
+                    current_pos = get_scroll_y()
+
+            current_round += 1
+
+        scroll_to(get_total_height())
+        time.sleep(random.uniform(1.5, 3.0))
+
+        if random.random() < 0.5:
+            print("最终回滚到中部并再次滚动到底部...")
+            scroll_to(get_total_height() // 2)
+            time.sleep(random.uniform(0.5, 1.5))
+            scroll_to(get_total_height())
+            time.sleep(random.uniform(1.0, 2.0))
+
+        print("智能滚动加载完成，页面所有内容已尝试加载。")
+
+    def click_next(self, driver: WebDriver, timeout: int = 10) -> bool:
+        try:
+            next_btn = WebDriverWait(driver, timeout).until(
+                EC.element_to_be_clickable((By.CSS_SELECTOR, "[data-test-pagination-next]"))
+            )
+            if next_btn.get_attribute("aria-disabled") == "true" or next_btn.get_attribute("disabled") is not None:
+                print("Next 按钮已禁用，没有更多页面。")
+                return False
+            driver.execute_script("arguments[0].scrollIntoView({block: 'center'});", next_btn)
+            time.sleep(0.5)
+            next_btn.click()
+            time.sleep(5)
+            print("已点击 Next 按钮")
+            time.sleep(3)
+            return True
+        except TimeoutException:
+            print("未找到 Next 按钮或按钮不可点击，到达最后一页。")
+            return False
+        except Exception as e:
+            print(f"点击 Next 按钮时出错: {e}")
+            return False
+
+
+# ===================== 4. 数据持久化接口 =====================
+
+class DataSaver(ABC):
+    """数据保存抽象接口"""
+
+    @abstractmethod
+    def save(self, data: List[Dict], page_num: int, search_keyword: str = "", folder: Optional[str] = None) -> None:
+        pass
+
+
+class JsonDataSaver(DataSaver):
+    """JSON 格式保存实现"""
+
+    def save(self, data: List[Dict], page_num: int, search_keyword: str = "", folder: Optional[str] = None) -> None:
+        filename = f"{search_keyword}_page_{page_num}.json" if search_keyword else f"profiles_page_{page_num}.json"
+        if folder:
+            os.makedirs(folder, exist_ok=True)
+            filepath = os.path.join(folder, filename)
+        else:
+            filepath = filename
+        with open(filepath, "w", encoding="utf-8") as f:
+            json.dump(data, f, ensure_ascii=False, indent=2)
+        print(f"已保存 {len(data)} 条个人资料到 {filepath}")
+
+
+# ===================== 5. 爬虫核心 =====================
+
+class LinkedInScraper:
+    """LinkedIn 搜索结果爬虫，协调各组件完成爬取任务"""
+
+    def __init__(
+        self,
+        driver: WebDriver,
+        page_parser: PageParser,
+        navigator: PageNavigator,
+        data_saver: DataSaver,
+    ):
+        self.driver = driver
+        self.page_parser = page_parser
+        self.navigator = navigator
+        self.data_saver = data_saver
+
+    def _check_json_list_length(self, page_num: int, search_keyword: str, folder: Optional[str]) -> int:
+        filename = f"{search_keyword}_page_{page_num}.json" if search_keyword else f"profiles_page_{page_num}.json"
+        if folder:
+            os.makedirs(folder, exist_ok=True)
+            filepath = os.path.join(folder, filename)
+        else:
+            filepath = filename
+
+        if os.path.exists(filepath):
+            print(f"文件 {filepath} 存在，跳过...")
+            return 1
+        else:
+            print(f"文件 {filepath} 不存在，返回长度 0。")
+            return 0
+
+    def scrape(
+        self,
+        start_url: str,
+        search_keyword: str = "",
+        folder: Optional[str] = None,
+        scroll_times: int = 100,
+        max_pages: Optional[int] = None,
+    ) -> None:
+        """执行爬取"""
+        page_num = 1
+        self.driver.get(start_url)
+        time.sleep(10)
+        json_list_length = self._check_json_list_length(page_num, search_keyword, folder)
+
+        if json_list_length != 1:
+            self.navigator.scroll_and_wait(self.driver)
+
+        self._process_current_page(page_num, search_keyword, folder)
+
+        while (max_pages is None or page_num < max_pages) and self.navigator.click_next(self.driver):
+            page_num += 1
+            self._process_current_page(page_num, search_keyword, folder)
+
+    def _process_current_page(self, page_num: int, search_keyword: str, folder: Optional[str]) -> None:
+        json_list_length = self._check_json_list_length(page_num, search_keyword, folder)
+        if json_list_length == 1:
+            return
+
+        self.navigator.scroll_and_wait(self.driver)
+        page_source = self.driver.page_source
+        profiles = self.page_parser.parse_profiles(page_source)
+        self.data_saver.save(profiles, page_num, search_keyword, folder)