@lyra_wang/chrome-frequent-pages 0.0.0-stage → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +188 -2
- package/bin/cli.js +13 -0
- package/config.example.json +35 -0
- package/inbox-deliver-mac.py +66 -0
- package/install.sh +39 -0
- package/main.py +417 -0
- package/package.json +38 -4
- package/run-if-due.sh +58 -0
- package/table_snapshot.py +46 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Wang Ying (LyraWang)
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,189 @@
|
|
|
1
|
-
#
|
|
1
|
+
# chrome-frequent-pages
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**把 Chrome 里高频访问的网页收录进飞书「常用网页」表,并生成可导入 Chrome 的书签 HTML。**
|
|
4
|
+
|
|
5
|
+
只读 Chrome History(复制到临时目录后再读,不锁 Chrome),归一化 URL 后按
|
|
6
|
+
「域名 + 路径」聚合访问次数,过滤登录页 / 搜索页 / 本地地址。
|
|
7
|
+
|
|
8
|
+
## 安装(npm,推荐)
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
npm install -g --prefix ~/.local @lyra_wang/chrome-frequent-pages
|
|
12
|
+
chrome-frequent-pages --dry-run
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## 用法(仓库目录直接跑)
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
# 配置(默认 ~/.config/chrome-frequent-pages/config.json)
|
|
19
|
+
cp config.example.json ~/.config/chrome-frequent-pages/config.json
|
|
20
|
+
|
|
21
|
+
python3 main.py --dry-run # 只看结果
|
|
22
|
+
python3 main.py --write # 写/更新飞书表
|
|
23
|
+
python3 main.py --html ~/Desktop/常用网页.html # 导出可导入 Charter 的书签 HTML
|
|
24
|
+
python3 main.py --days 30 --min-visits 5 # 临时调窗口与阈值
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## 统计窗口
|
|
28
|
+
|
|
29
|
+
`days` 决定"最近多少天内访问过的页面才参与统计":
|
|
30
|
+
|
|
31
|
+
| 值 | 效果 |
|
|
32
|
+
|---|---|
|
|
33
|
+
| 7 | 只看最近一周(最灵敏,候选最少)|
|
|
34
|
+
| **10** | **当前设置** |
|
|
35
|
+
| 30 | 一个月 |
|
|
36
|
+
| 90 | 一个季度(旧默认)|
|
|
37
|
+
|
|
38
|
+
改法:编辑 `~/.config/chrome-frequent-pages/config.json` 的 `days`,
|
|
39
|
+
或临时用 `python3 main.py --days 7 --dry-run`。
|
|
40
|
+
|
|
41
|
+
## 配置字段
|
|
42
|
+
|
|
43
|
+
| 字段 | 说明 |
|
|
44
|
+
|---|---|
|
|
45
|
+
| `chrome_profile_dir` | Chrome profile 目录(默认 `~/Library/Application Support/Google/Chrome/Default`)|
|
|
46
|
+
| `days` | 统计窗口(可配置;当前设为 **10 天**)|
|
|
47
|
+
| `min_visits` | 最少访问次数(当前设为 **5**)|
|
|
48
|
+
| `limit` | 最多收录条数(默认 60,`0` = 不限)|
|
|
49
|
+
| `exclude_hosts` | 排除的域名(邮箱、登录等)|
|
|
50
|
+
| `base_token` / `table_id` | 目标飞书表坐标 |
|
|
51
|
+
| `identity` | `user`(默认)或 `bot` |
|
|
52
|
+
| `ignored_urls_file` | 忽略清单(一行一个 URL);表里删掉的行会写进这里,避免下次重新收录 |
|
|
53
|
+
| `rules.drop_hosts` | 整站丢弃(娱乐视频站等)|
|
|
54
|
+
| `rules.drop_url_contains` | URL 含指定片段即丢弃(开发者后台、PR/Releases、OAuth、支付等)|
|
|
55
|
+
| `rules.root_only_hosts` | 只保留站点根域名(如 `github.com` 只留首页)|
|
|
56
|
+
| `rules.drop_titles` | 标题命中即丢(失效/无权限文档)|
|
|
57
|
+
| `rules.keep_urls` | 白名单,优先级最高 |
|
|
58
|
+
| `rules.one_per_host` | 每个域名只保留访问最多的一页 |
|
|
59
|
+
|
|
60
|
+
## 规则(0.4.0:根域名级)
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
最近 10 天 → 按「根域名」加总访问次数 → 超过 5 次 → 只排除影视娱乐与本地页 → 一行一个根域名
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
| 项 | 当前值 |
|
|
67
|
+
|---|---|
|
|
68
|
+
| `days` | 10 |
|
|
69
|
+
| `min_visits` | 5(**严格超过**,即 6 次起)|
|
|
70
|
+
| 统计口径 | 该根域名下**所有页面**的访问次数之和(不是单页)|
|
|
71
|
+
| 协议 | 只统计 `https://` |
|
|
72
|
+
| 排除 | 影视娱乐(yfsp / iyf / yifan / tv.cctv)+ 本地页(localhost / 127.0.0.1 / file)|
|
|
73
|
+
| 输出 | `https://<根域名>/` 一行;标题取 `domain_labels`,缺省用域名 |
|
|
74
|
+
| 黑名单 | 无(只有影视娱乐与本地页被排除)|
|
|
75
|
+
|
|
76
|
+
## 表结构(域名级)
|
|
77
|
+
|
|
78
|
+
0.3.1 起写入适配「一行一个域名」的表:
|
|
79
|
+
|
|
80
|
+
| 字段 | 写入内容 |
|
|
81
|
+
|---|---|
|
|
82
|
+
| `域名` | host(判重键)|
|
|
83
|
+
| `标题` | 取 `domain_labels` 里的中文说明;没有配置时退回域名 |
|
|
84
|
+
| `访问次数` | 该域名所有页面的访问次数之和 |
|
|
85
|
+
| `状态` | `已收录` |
|
|
86
|
+
|
|
87
|
+
`收录时间`、`分类` 由表自己维护,脚本不写。
|
|
88
|
+
|
|
89
|
+
## 写入策略
|
|
90
|
+
|
|
91
|
+
- 以 `URL` 去重:已存在则更新访问次数/最近访问,不存在才新建;
|
|
92
|
+
- 同一页面不同 query/fragment 会归一化成同一条(飞书 `/base/<token>`、`/wiki/<token>` 按 token 聚合);
|
|
93
|
+
- 状态默认「已收录」,方便你之后标「已加入书签」或「忽略」;
|
|
94
|
+
- 忽略清单:`~/.config/chrome-frequent-pages/ignored_urls.txt`,一行一个 URL,命中即跳过;
|
|
95
|
+
在表里删掉的页面记得同步进这个文件(或让工具写)。
|
|
96
|
+
|
|
97
|
+
## 书签替换(安全流程)
|
|
98
|
+
|
|
99
|
+
1. 先备份 Chrome 当前书签:
|
|
100
|
+
`cp "~/Library/Application Support/Google/Chrome/Default/Bookmarks" ~/Desktop/Bookmarks-backup-$(date +%F).json`
|
|
101
|
+
2. 跑 `--html` 生成新书签 HTML;
|
|
102
|
+
3. Chrome → 书签管理器 → 删除旧书签 → 导入 HTML;
|
|
103
|
+
4. 不建议直接改 `Bookmarks` JSON:Chrome 运行时会覆盖,写坏会丢全部书签。
|
|
104
|
+
|
|
105
|
+
## 定时(可选)
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
bash install.sh # 生成 ~/.local/bin/chrome-frequent-pages 并注册每周一 09:30 的 launchd 任务
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## 中文标签(domain_labels)
|
|
112
|
+
|
|
113
|
+
```json
|
|
114
|
+
"domain_labels": {
|
|
115
|
+
"meeting.bamamei.online": "组队会议分析",
|
|
116
|
+
"github.com": "GitHub"
|
|
117
|
+
}
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
表里的 `标题` 列与导出的书签名称都优先用这里的文字;没配置的域名退回域名本身。
|
|
121
|
+
|
|
122
|
+
## 书签:按分类子文件夹导出(0.4.1)
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
python3 main.py --bookmarks-from-table ~/Desktop/常用网页书签.html
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
按表里的 `分类` 列生成**同级**文件夹(都在书签栏一层):
|
|
129
|
+
|
|
130
|
+
```
|
|
131
|
+
书签栏
|
|
132
|
+
├── 常用网页 (全部 27 条,平铺)
|
|
133
|
+
├── 工作 (9 条)
|
|
134
|
+
├── 学习 (3 条)
|
|
135
|
+
└── 工具 (15 条)
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
没填分类的域名会进「其他」。
|
|
139
|
+
|
|
140
|
+
## 书签:按「域名」导出
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
python3 main.py --bookmarks-from-table ~/Desktop/常用网页书签.html
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
读取飞书表每个唯一「域名」,生成 `https://<域名>/` 书签 HTML(Chrome 书签管理器导入)。
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## 开机触发(每周一次)+ 投递到收件箱
|
|
151
|
+
|
|
152
|
+
数据在本机 Chrome 里,所以执行必须在 Mac 上。定时不用时钟,用**开机作为触发点**:
|
|
153
|
+
|
|
154
|
+
- `run-if-due.sh`:launchd 的 `RunAtLoad` 调用;读状态文件判断本周是否已跑,
|
|
155
|
+
没跑则执行 `chrome-frequent-pages --write`,跑完写状态(失败不写,下次开机重试)。
|
|
156
|
+
- `inbox-deliver-mac.py`:把结果写进飞书「任务收件箱」,DSH 收件箱面板会显示为「待读」。
|
|
157
|
+
- `table_snapshot.py`:执行前后各取一次「常用网页」表快照,diff 出本周**新增了哪些页面**,
|
|
158
|
+
摘要里带上「标题(域名)→ 书签「分类」」;未分类的页面导出书签时归入「其他」。
|
|
159
|
+
|
|
160
|
+
### 安装
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
npm i -g @lyra_wang/chrome-frequent-pages@0.5.2 # 或 npm i -g file:$PWD
|
|
164
|
+
bash "$(dirname "$(readlink -f "$(command -v chrome-frequent-pages)")")/../install.sh"
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
`install.sh` 会把 `main.py` / `run-if-due.sh` / `inbox-deliver-mac.py` / `table_snapshot.py`
|
|
168
|
+
复制到 `~/.local/share/chrome-frequent-pages/`,并注册 `RunAtLoad` 的 LaunchAgent(无定时时刻)。
|
|
169
|
+
|
|
170
|
+
### 必需的 macOS 权限(否则读不到 Chrome 历史)
|
|
171
|
+
|
|
172
|
+
「完全磁盘访问权限」里需要加入三项(后台进程按二进制逐个鉴权):
|
|
173
|
+
|
|
174
|
+
| 加入项 | 作用 |
|
|
175
|
+
|---|---|
|
|
176
|
+
| `/bin/bash` | 启动脚本 |
|
|
177
|
+
| `/usr/local/bin/node` | npm CLI 需要 |
|
|
178
|
+
| `/usr/bin/python3` | 实际读 History 的进程 |
|
|
179
|
+
|
|
180
|
+
### 行为
|
|
181
|
+
|
|
182
|
+
| 场景 | 结果 |
|
|
183
|
+
|---|---|
|
|
184
|
+
| 本周第一次开机/登录 | 执行一次 |
|
|
185
|
+
| 同周再次开机 | 状态文件命中,跳过 |
|
|
186
|
+
| 执行失败 | 不写状态,下次开机重试;失败结果也会写收件箱 |
|
|
187
|
+
|
|
188
|
+
状态文件:`~/.config/chrome-frequent-pages/last-run.json`(`{"week":"2026-43","at":"..."}`)
|
|
189
|
+
日志:`~/.config/chrome-frequent-pages/run-if-due.log`
|
package/bin/cli.js
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// npm bin shim: run the Python collector that ships in this package.
|
|
3
|
+
import { spawn } from 'node:child_process';
|
|
4
|
+
import { dirname, join } from 'node:path';
|
|
5
|
+
import { fileURLToPath } from 'node:url';
|
|
6
|
+
|
|
7
|
+
const here = dirname(fileURLToPath(import.meta.url));
|
|
8
|
+
const script = join(here, '..', 'main.py');
|
|
9
|
+
const child = spawn('/usr/bin/python3', [script, ...process.argv.slice(2)], { stdio: 'inherit' });
|
|
10
|
+
child.on('exit', (code, signal) => {
|
|
11
|
+
if (signal) process.kill(process.pid, signal);
|
|
12
|
+
else process.exit(code ?? 0);
|
|
13
|
+
});
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
{
|
|
2
|
+
"chrome_profile_dir": "~/Library/Application Support/Google/Chrome/Default",
|
|
3
|
+
"days": 10,
|
|
4
|
+
"min_visits": 5,
|
|
5
|
+
"limit": 60,
|
|
6
|
+
"exclude_hosts": [
|
|
7
|
+
"mail.google.com",
|
|
8
|
+
"accounts.google.com",
|
|
9
|
+
"localhost",
|
|
10
|
+
"127.0.0.1"
|
|
11
|
+
],
|
|
12
|
+
"base_token": "<feishu-base-token>",
|
|
13
|
+
"table_id": "<table-id>",
|
|
14
|
+
"identity": "user",
|
|
15
|
+
"rules": {
|
|
16
|
+
"drop_hosts": [
|
|
17
|
+
"yfsp.tv",
|
|
18
|
+
"iyf.tv",
|
|
19
|
+
"yifan.tv",
|
|
20
|
+
"tv.cctv.com"
|
|
21
|
+
],
|
|
22
|
+
"drop_url_contains": [],
|
|
23
|
+
"root_only_hosts": [],
|
|
24
|
+
"drop_titles": [],
|
|
25
|
+
"keep_urls": [],
|
|
26
|
+
"one_per_host": false
|
|
27
|
+
},
|
|
28
|
+
"ignored_urls_file": "",
|
|
29
|
+
"domain_labels": {
|
|
30
|
+
"meeting.bamamei.online": "组队会议分析",
|
|
31
|
+
"eos.douyin.com": "抖音生活服务 · 直播",
|
|
32
|
+
"github.com": "GitHub",
|
|
33
|
+
"platform.deepseek.com": "DeepSeek 开放平台"
|
|
34
|
+
}
|
|
35
|
+
}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
# -*- coding: utf-8 -*-
|
|
3
|
+
"""把一条任务结果写进飞书「任务收件箱」(Mac 侧,使用本机 lark-cli 用户身份)。
|
|
4
|
+
|
|
5
|
+
用法:
|
|
6
|
+
inbox-deliver-mac.py --task chrome-frequent-pages --result 成功 --summary "本周新增 3 条"
|
|
7
|
+
"""
|
|
8
|
+
import argparse, json, subprocess, sys
|
|
9
|
+
from datetime import datetime
|
|
10
|
+
|
|
11
|
+
BASE = "NeujbbOfcaPEvMsPIxscaJImn9d"
|
|
12
|
+
INBOX = "tbl9LghPBTjFLH4N"
|
|
13
|
+
TASKS = "tbl4k7ggY1paqoUn"
|
|
14
|
+
LARK = "lark-cli"
|
|
15
|
+
|
|
16
|
+
def lark(args):
|
|
17
|
+
cmd = [LARK] + args + ["--as", "user", "--format", "json"]
|
|
18
|
+
res = subprocess.run(cmd, capture_output=True, text=True, timeout=90)
|
|
19
|
+
try:
|
|
20
|
+
payload = json.loads(res.stdout or "{}")
|
|
21
|
+
except Exception:
|
|
22
|
+
payload = {}
|
|
23
|
+
if payload.get("ok") is False:
|
|
24
|
+
raise SystemExit("lark-cli failed: " + json.dumps(payload.get("error", payload), ensure_ascii=False))
|
|
25
|
+
if res.returncode != 0:
|
|
26
|
+
raise SystemExit("lark-cli exited %s: %s" % (res.returncode, (res.stderr or "")[:300]))
|
|
27
|
+
return payload
|
|
28
|
+
|
|
29
|
+
def task_record_id(name):
|
|
30
|
+
payload = lark(["base", "+record-list", "--base-token", BASE, "--table-id", TASKS, "--limit", "200"])
|
|
31
|
+
meta = payload.get("data") or {}
|
|
32
|
+
fields = meta.get("fields") or []
|
|
33
|
+
rows = meta.get("data") or []
|
|
34
|
+
ids = meta.get("record_id_list") or []
|
|
35
|
+
if "任务名称" not in fields:
|
|
36
|
+
return ""
|
|
37
|
+
idx = fields.index("任务名称")
|
|
38
|
+
for rid, row in zip(ids, rows):
|
|
39
|
+
if str(row[idx]).strip() == name:
|
|
40
|
+
return str(rid)
|
|
41
|
+
return ""
|
|
42
|
+
|
|
43
|
+
def main():
|
|
44
|
+
ap = argparse.ArgumentParser()
|
|
45
|
+
ap.add_argument("--task", required=True)
|
|
46
|
+
ap.add_argument("--result", required=True, choices=["成功", "失败", "超时"])
|
|
47
|
+
ap.add_argument("--summary", required=True)
|
|
48
|
+
args = ap.parse_args()
|
|
49
|
+
summary = args.summary.strip()
|
|
50
|
+
if len(summary) > 300:
|
|
51
|
+
summary = summary[:299] + "…"
|
|
52
|
+
record = {
|
|
53
|
+
"结果摘要": summary,
|
|
54
|
+
"结果": [args.result],
|
|
55
|
+
"状态": ["未读"],
|
|
56
|
+
"执行任务": int(datetime.now().timestamp() * 1000),
|
|
57
|
+
}
|
|
58
|
+
rid = task_record_id(args.task)
|
|
59
|
+
if rid:
|
|
60
|
+
record["关联任务"] = [rid]
|
|
61
|
+
lark(["base", "+record-batch-create", "--base-token", BASE, "--table-id", INBOX,
|
|
62
|
+
"--json", json.dumps({"create_records": [record]}, ensure_ascii=False)])
|
|
63
|
+
print(json.dumps({"ok": True, "task": args.task, "linked": bool(rid)}, ensure_ascii=False))
|
|
64
|
+
|
|
65
|
+
if __name__ == "__main__":
|
|
66
|
+
main()
|
package/install.sh
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
set -eu
|
|
3
|
+
HERE="$(cd "$(dirname "$0")" && pwd)"
|
|
4
|
+
BIN_DIR="$HOME/.local/bin"
|
|
5
|
+
DATA_DIR="$HOME/.local/share/chrome-frequent-pages"
|
|
6
|
+
CONF_DIR="$HOME/.config/chrome-frequent-pages"
|
|
7
|
+
mkdir -p "$BIN_DIR" "$DATA_DIR" "$CONF_DIR"
|
|
8
|
+
|
|
9
|
+
# launchd 读不了 ~/Documents(TCC),脚本要放到不受保护的位置
|
|
10
|
+
cp "$HERE/main.py" "$HERE/run-if-due.sh" "$HERE/inbox-deliver-mac.py" "$HERE/table_snapshot.py" "$DATA_DIR/"
|
|
11
|
+
chmod +x "$DATA_DIR/main.py" "$DATA_DIR/run-if-due.sh" "$DATA_DIR/inbox-deliver-mac.py" "$DATA_DIR/table_snapshot.py"
|
|
12
|
+
ln -sf "$DATA_DIR/main.py" "$BIN_DIR/chrome-frequent-pages"
|
|
13
|
+
[ -f "$CONF_DIR/config.json" ] || cp "$HERE/config.example.json" "$CONF_DIR/config.json"
|
|
14
|
+
|
|
15
|
+
PLIST="$HOME/Library/LaunchAgents/com.lyrawang.chrome-frequent-pages.plist"
|
|
16
|
+
cat > "$PLIST" <<EOF
|
|
17
|
+
<?xml version="1.0" encoding="UTF-8"?>
|
|
18
|
+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
|
19
|
+
<plist version="1.0"><dict>
|
|
20
|
+
<key>Label</key><string>com.lyrawang.chrome-frequent-pages</string>
|
|
21
|
+
<key>ProgramArguments</key><array>
|
|
22
|
+
<string>/bin/bash</string><string>$DATA_DIR/run-if-due.sh</string>
|
|
23
|
+
</array>
|
|
24
|
+
<key>RunAtLoad</key><true/>
|
|
25
|
+
<key>StandardOutPath</key><string>$CONF_DIR/launchd.out.log</string>
|
|
26
|
+
<key>StandardErrorPath</key><string>$CONF_DIR/launchd.err.log</string>
|
|
27
|
+
<key>ProcessType</key><string>Background</string>
|
|
28
|
+
</dict></plist>
|
|
29
|
+
EOF
|
|
30
|
+
launchctl bootout "gui/$(id -u)/com.lyrawang.chrome-frequent-pages" 2>/dev/null || true
|
|
31
|
+
launchctl bootstrap "gui/$(id -u)" "$PLIST" 2>/dev/null || launchctl load "$PLIST" 2>/dev/null || true
|
|
32
|
+
echo "✓ 已安装:~/.local/bin/chrome-frequent-pages"
|
|
33
|
+
echo "✓ 已注册开机触发(launchd RunAtLoad,每周首次开机跑一次)"
|
|
34
|
+
cat <<'NOTE'
|
|
35
|
+
|
|
36
|
+
⚠️ 首次安装需在「系统设置 → 隐私与安全性 → 完全磁盘访问权限」中加入:
|
|
37
|
+
/bin/bash、/usr/local/bin/node、/usr/bin/python3
|
|
38
|
+
否则读不到 Chrome History(会写一条「失败」摘要到收件箱)。
|
|
39
|
+
NOTE
|
package/main.py
ADDED
|
@@ -0,0 +1,417 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""把 Chrome 的高频访问页面收录进飞书「常用网页」表,并可导出书签 HTML。
|
|
3
|
+
|
|
4
|
+
只读 Chrome History(复制到临时目录后再读,不锁 Chrome),归一化 URL 后
|
|
5
|
+
按域名 + 路径聚合访问次数,过滤噪声与隐私站点。
|
|
6
|
+
|
|
7
|
+
用法:
|
|
8
|
+
python3 main.py --dry-run # 只看结果,不写表
|
|
9
|
+
python3 main.py --write # 写/更新飞书表
|
|
10
|
+
python3 main.py --html ~/Desktop/常用网页.html # 导出可导入 Chrome 的书签 HTML
|
|
11
|
+
"""
|
|
12
|
+
import argparse
|
|
13
|
+
import html
|
|
14
|
+
import json
|
|
15
|
+
import os
|
|
16
|
+
import shutil
|
|
17
|
+
import sqlite3
|
|
18
|
+
import subprocess
|
|
19
|
+
import sys
|
|
20
|
+
import tempfile
|
|
21
|
+
import urllib.parse
|
|
22
|
+
from collections import defaultdict
|
|
23
|
+
from datetime import datetime, timedelta, timezone
|
|
24
|
+
|
|
25
|
+
HOME = os.path.expanduser("~")
|
|
26
|
+
CONFIG_PATH = os.environ.get("CHROME_PAGES_CONFIG") or os.path.join(HOME, ".config", "chrome-frequent-pages", "config.json")
|
|
27
|
+
DEFAULT_CONFIG = {
|
|
28
|
+
"chrome_profile_dir": os.path.join(HOME, "Library", "Application Support", "Google", "Chrome", "Default"),
|
|
29
|
+
"days": 90,
|
|
30
|
+
"min_visits": 8,
|
|
31
|
+
"limit": 60,
|
|
32
|
+
"exclude_hosts": ["mail.google.com", "accounts.google.com", "localhost", "127.0.0.1"],
|
|
33
|
+
"base_token": "",
|
|
34
|
+
"table_id": "",
|
|
35
|
+
"identity": "user",
|
|
36
|
+
"ignored_urls_file": os.path.join(HOME, ".config", "chrome-frequent-pages", "ignored_urls.txt"),
|
|
37
|
+
"domain_labels": {},
|
|
38
|
+
"rules": {
|
|
39
|
+
# 只保留一条:娱乐影视整站丢弃;其余精细规则已按需求移除
|
|
40
|
+
"drop_hosts": ["yfsp.tv", "iyf.tv", "yifan.tv", "tv.cctv.com"],
|
|
41
|
+
"drop_url_contains": [],
|
|
42
|
+
"root_only_hosts": [],
|
|
43
|
+
"drop_titles": [],
|
|
44
|
+
"keep_urls": [],
|
|
45
|
+
"one_per_host": True,
|
|
46
|
+
},
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
# chrome time: microseconds since 1601-01-01
|
|
50
|
+
CHROME_EPOCH_OFFSET = 11644473600
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def load_config():
|
|
54
|
+
cfg = dict(DEFAULT_CONFIG)
|
|
55
|
+
if os.path.exists(CONFIG_PATH):
|
|
56
|
+
with open(CONFIG_PATH, encoding="utf-8") as fh:
|
|
57
|
+
cfg.update(json.load(fh))
|
|
58
|
+
else:
|
|
59
|
+
print(f"[提示] 没有配置文件,使用默认值;可用 CHROME_PAGES_CONFIG 或 {CONFIG_PATH} 覆盖", file=sys.stderr)
|
|
60
|
+
for key in ("chrome_profile_dir",):
|
|
61
|
+
if isinstance(cfg.get(key), str):
|
|
62
|
+
cfg[key] = os.path.expanduser(cfg[key])
|
|
63
|
+
return cfg
|
|
64
|
+
|
|
65
|
+
|
|
66
|
+
def copy_history(profile_dir):
|
|
67
|
+
if not os.path.isdir(profile_dir):
|
|
68
|
+
raise SystemExit(f"找不到 Chrome profile:{profile_dir}")
|
|
69
|
+
tmp = tempfile.mkdtemp(prefix="chrome-history-")
|
|
70
|
+
for name in ("History", "History-wal", "History-shm"):
|
|
71
|
+
src = os.path.join(profile_dir, name)
|
|
72
|
+
if os.path.exists(src):
|
|
73
|
+
shutil.copy2(src, os.path.join(tmp, name))
|
|
74
|
+
target = os.path.join(tmp, "History")
|
|
75
|
+
if not os.path.exists(target):
|
|
76
|
+
raise SystemExit("复制 History 失败")
|
|
77
|
+
return tmp, target
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def normalize(url):
|
|
81
|
+
parsed = urllib.parse.urlsplit(url)
|
|
82
|
+
host = (parsed.hostname or "").lower()
|
|
83
|
+
if host.startswith("www."):
|
|
84
|
+
host = host[4:]
|
|
85
|
+
path = parsed.path.rstrip("/")
|
|
86
|
+
# 飞书文档/表格按 token 聚合:只认路径开头的 /base/<token> 之类,
|
|
87
|
+
# 避免把 open.feishu.cn/document/.../docs/bitable-v1 这种开发文档误折叠。
|
|
88
|
+
import re as _re
|
|
89
|
+
match = _re.match(r"^/(base|wiki|docx|sheets|docs|mindnote|slides)/([A-Za-z0-9]+)", path)
|
|
90
|
+
if match:
|
|
91
|
+
return host, f"/{match.group(1)}/{match.group(2)}"
|
|
92
|
+
return host, path or "/"
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
def is_noise(url, host, exclude_hosts):
|
|
96
|
+
# 只看 https;http、file://、localhost 等本地页面一律不看
|
|
97
|
+
if not url.startswith("https://"):
|
|
98
|
+
return True
|
|
99
|
+
if not host:
|
|
100
|
+
return True
|
|
101
|
+
if host in exclude_hosts:
|
|
102
|
+
return True
|
|
103
|
+
low = url.lower()
|
|
104
|
+
for bad in ("chrome://", "file://", "chrome-extension://", "/search?q=", "login", "signin", "accounts."):
|
|
105
|
+
if bad in low:
|
|
106
|
+
return True
|
|
107
|
+
return False
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
def load_ignored(cfg):
|
|
111
|
+
path = cfg.get("ignored_urls_file") or ""
|
|
112
|
+
if not path:
|
|
113
|
+
return set()
|
|
114
|
+
path = os.path.expanduser(path)
|
|
115
|
+
if not os.path.exists(path):
|
|
116
|
+
return set()
|
|
117
|
+
with open(path, encoding="utf-8") as fh:
|
|
118
|
+
return {line.strip() for line in fh if line.strip()}
|
|
119
|
+
|
|
120
|
+
|
|
121
|
+
def apply_rules(url, host, title, rules):
|
|
122
|
+
"""Return None to drop, or the url to keep (root_only may rewrite it)."""
|
|
123
|
+
keep_urls = set(rules.get("keep_urls") or [])
|
|
124
|
+
if url in keep_urls:
|
|
125
|
+
return url
|
|
126
|
+
low = url.lower()
|
|
127
|
+
for needle in (rules.get("drop_url_contains") or []):
|
|
128
|
+
if needle.lower() in low:
|
|
129
|
+
return None
|
|
130
|
+
for dropped in (rules.get("drop_hosts") or []):
|
|
131
|
+
if host == dropped.lower() or host.endswith("." + dropped.lower()):
|
|
132
|
+
return None
|
|
133
|
+
clean_title = (title or "").strip()
|
|
134
|
+
for dead in (rules.get("drop_titles") or []):
|
|
135
|
+
if clean_title == dead:
|
|
136
|
+
return None
|
|
137
|
+
root_only = {h.lower() for h in (rules.get("root_only_hosts") or [])}
|
|
138
|
+
if host in root_only:
|
|
139
|
+
from urllib.parse import urlsplit
|
|
140
|
+
path = urlsplit(url).path or "/"
|
|
141
|
+
if path.rstrip("/") != "":
|
|
142
|
+
return None
|
|
143
|
+
return "https://" + host + "/"
|
|
144
|
+
return url
|
|
145
|
+
|
|
146
|
+
|
|
147
|
+
def collect(cfg):
|
|
148
|
+
ignored = load_ignored(cfg)
|
|
149
|
+
rules = cfg.get("rules") or {}
|
|
150
|
+
tmp, db = copy_history(cfg["chrome_profile_dir"])
|
|
151
|
+
try:
|
|
152
|
+
conn = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
|
|
153
|
+
cutoff_us = int((datetime.now().timestamp() - cfg["days"] * 86400 + CHROME_EPOCH_OFFSET) * 1_000_000)
|
|
154
|
+
rows = conn.execute(
|
|
155
|
+
"SELECT url, title, visit_count, last_visit_time FROM urls WHERE last_visit_time > ? AND url LIKE 'http%'",
|
|
156
|
+
(cutoff_us,),
|
|
157
|
+
).fetchall()
|
|
158
|
+
conn.close()
|
|
159
|
+
finally:
|
|
160
|
+
shutil.rmtree(tmp, ignore_errors=True)
|
|
161
|
+
|
|
162
|
+
# 按根域名聚合:访问次数 = 该域名下所有页面之和
|
|
163
|
+
aggregated = {}
|
|
164
|
+
for url, title, visits, last_us in rows:
|
|
165
|
+
host, path = normalize(url)
|
|
166
|
+
if is_noise(url, host, set(cfg["exclude_hosts"])):
|
|
167
|
+
continue
|
|
168
|
+
kept = apply_rules(url, host, title, rules)
|
|
169
|
+
if kept is None:
|
|
170
|
+
continue
|
|
171
|
+
key = host
|
|
172
|
+
last_ts = last_us / 1_000_000 - CHROME_EPOCH_OFFSET
|
|
173
|
+
entry = aggregated.get(key)
|
|
174
|
+
if entry is None:
|
|
175
|
+
aggregated[key] = {
|
|
176
|
+
"key": key,
|
|
177
|
+
# 输出根域名,不带路径
|
|
178
|
+
"url": f"https://{host}/",
|
|
179
|
+
"title": (title or "").strip(),
|
|
180
|
+
"host": host,
|
|
181
|
+
"visits": visits,
|
|
182
|
+
"last": last_ts,
|
|
183
|
+
}
|
|
184
|
+
else:
|
|
185
|
+
entry["visits"] += visits
|
|
186
|
+
if last_ts > entry["last"]:
|
|
187
|
+
entry["last"] = last_ts
|
|
188
|
+
if not entry["title"] and title:
|
|
189
|
+
entry["title"] = title.strip()
|
|
190
|
+
|
|
191
|
+
# 规则:根域名访问次数「超过」min_visits(严格大于)
|
|
192
|
+
items = [e for e in aggregated.values() if e["visits"] > cfg["min_visits"] and e["url"] not in ignored]
|
|
193
|
+
items.sort(key=lambda e: (e["visits"], e["last"]), reverse=True)
|
|
194
|
+
if False and rules.get("one_per_host"):
|
|
195
|
+
seen_hosts = set()
|
|
196
|
+
collapsed = []
|
|
197
|
+
for item in items:
|
|
198
|
+
if item["host"] in seen_hosts:
|
|
199
|
+
continue
|
|
200
|
+
seen_hosts.add(item["host"])
|
|
201
|
+
collapsed.append(item)
|
|
202
|
+
items = collapsed
|
|
203
|
+
if cfg["limit"]:
|
|
204
|
+
items = items[: cfg["limit"]]
|
|
205
|
+
labels = cfg.get("domain_labels") or {}
|
|
206
|
+
for item in items:
|
|
207
|
+
item["last_iso"] = datetime.fromtimestamp(item["last"]).strftime("%Y-%m-%d %H:%M")
|
|
208
|
+
item["title"] = labels.get(item["host"]) or item["host"]
|
|
209
|
+
return items
|
|
210
|
+
|
|
211
|
+
|
|
212
|
+
def lark(args):
|
|
213
|
+
result = subprocess.run(["lark-cli"] + args + ["--as", "user", "--format", "json"], capture_output=True, text=True, timeout=120)
|
|
214
|
+
if result.returncode != 0:
|
|
215
|
+
raise SystemExit("lark-cli 失败:" + (result.stderr or result.stdout).strip()[:400])
|
|
216
|
+
payload = json.loads(result.stdout or "{}")
|
|
217
|
+
if payload.get("ok") is False:
|
|
218
|
+
err = payload.get("error") or {}
|
|
219
|
+
raise SystemExit("lark-cli 返回失败:" + str(err.get("message") or err)[:400])
|
|
220
|
+
return payload
|
|
221
|
+
|
|
222
|
+
|
|
223
|
+
def existing_domains(cfg):
|
|
224
|
+
# 表结构是「一行一个域名」;读失败必须中止,避免重复插入。
|
|
225
|
+
data = lark(["base", "+record-list", "--base-token", cfg["base_token"], "--table-id", cfg["table_id"], "--limit", "200"])
|
|
226
|
+
meta = data.get("data") or {}
|
|
227
|
+
cols = meta.get("fields") or []
|
|
228
|
+
rows = meta.get("data") or []
|
|
229
|
+
ids = meta.get("record_id_list") or []
|
|
230
|
+
if "域名" not in cols:
|
|
231
|
+
raise SystemExit("表里没有「域名」列;请检查表结构")
|
|
232
|
+
idx = {name: i for i, name in enumerate(cols)}
|
|
233
|
+
out = {}
|
|
234
|
+
for rid, row in zip(ids, rows):
|
|
235
|
+
host = str(row[idx["域名"]] or "").strip()
|
|
236
|
+
if host:
|
|
237
|
+
out[host] = rid
|
|
238
|
+
return out
|
|
239
|
+
|
|
240
|
+
|
|
241
|
+
def aggregate_by_host(items, labels=None):
|
|
242
|
+
"""把页面级结果压成域名级:访问次数求和,标题用中文标签(缺省退回域名)。"""
|
|
243
|
+
labels = labels or {}
|
|
244
|
+
grouped = {}
|
|
245
|
+
for item in items:
|
|
246
|
+
host = item["host"]
|
|
247
|
+
rec = grouped.get(host)
|
|
248
|
+
if rec is None:
|
|
249
|
+
grouped[host] = {
|
|
250
|
+
"域名": host,
|
|
251
|
+
"访问次数": item["visits"],
|
|
252
|
+
"标题": labels.get(host) or host,
|
|
253
|
+
"_top": item["visits"],
|
|
254
|
+
}
|
|
255
|
+
else:
|
|
256
|
+
rec["访问次数"] += item["visits"]
|
|
257
|
+
if item["visits"] > rec["_top"]:
|
|
258
|
+
rec["_top"] = item["visits"]
|
|
259
|
+
rec["标题"] = labels.get(host) or host
|
|
260
|
+
rows = sorted(grouped.values(), key=lambda r: r["访问次数"], reverse=True)
|
|
261
|
+
for row in rows:
|
|
262
|
+
row.pop("_top", None)
|
|
263
|
+
return rows
|
|
264
|
+
|
|
265
|
+
|
|
266
|
+
def write_table(cfg, items):
|
|
267
|
+
if not cfg["base_token"] or not cfg["table_id"]:
|
|
268
|
+
raise SystemExit("配置里缺 base_token / table_id")
|
|
269
|
+
known = existing_domains(cfg)
|
|
270
|
+
create, update = [], []
|
|
271
|
+
for row in aggregate_by_host(items, cfg.get("domain_labels") or {}):
|
|
272
|
+
record = {
|
|
273
|
+
"域名": row["域名"],
|
|
274
|
+
"标题": row["标题"],
|
|
275
|
+
"访问次数": row["访问次数"],
|
|
276
|
+
"状态": ["已收录"],
|
|
277
|
+
}
|
|
278
|
+
if row["域名"] in known:
|
|
279
|
+
update.append((known[row["域名"]], record))
|
|
280
|
+
else:
|
|
281
|
+
create.append(record)
|
|
282
|
+
|
|
283
|
+
if create:
|
|
284
|
+
lark(["base", "+record-batch-create", "--base-token", cfg["base_token"], "--table-id", cfg["table_id"],
|
|
285
|
+
"--json", json.dumps({"create_records": create}, ensure_ascii=False)])
|
|
286
|
+
for record_id, record in update:
|
|
287
|
+
lark(["base", "+record-upsert", "--base-token", cfg["base_token"], "--table-id", cfg["table_id"],
|
|
288
|
+
"--record-id", record_id, "--json", json.dumps(record, ensure_ascii=False)])
|
|
289
|
+
return len(create), len(update)
|
|
290
|
+
|
|
291
|
+
|
|
292
|
+
CATEGORY_ORDER = ["工作", "学习", "工具", "内容", "其他"]
|
|
293
|
+
|
|
294
|
+
|
|
295
|
+
def bookmarks_from_table(cfg, path):
|
|
296
|
+
"""按飞书表导出书签:常用网页/<分类>/<域名书签>。"""
|
|
297
|
+
data = lark(["base", "+record-list", "--base-token", cfg["base_token"], "--table-id", cfg["table_id"], "--limit", "200"])
|
|
298
|
+
meta = data.get("data") or {}
|
|
299
|
+
cols = meta.get("fields") or []
|
|
300
|
+
rows = meta.get("data") or []
|
|
301
|
+
if "域名" not in cols:
|
|
302
|
+
raise SystemExit("表里没有「域名」列")
|
|
303
|
+
idx = {name: i for i, name in enumerate(cols)}
|
|
304
|
+
labels = cfg.get("domain_labels") or {}
|
|
305
|
+
groups = {}
|
|
306
|
+
for row in rows:
|
|
307
|
+
host = str(row[idx["域名"]] or "").strip()
|
|
308
|
+
if not host:
|
|
309
|
+
continue
|
|
310
|
+
cat = "其他"
|
|
311
|
+
if "分类" in idx:
|
|
312
|
+
raw = row[idx["分类"]]
|
|
313
|
+
if isinstance(raw, list) and raw:
|
|
314
|
+
cat = str(raw[0])
|
|
315
|
+
elif isinstance(raw, str) and raw.strip():
|
|
316
|
+
cat = raw.strip()
|
|
317
|
+
groups.setdefault(cat, []).append({
|
|
318
|
+
"host": host,
|
|
319
|
+
"name": str(row[idx["标题"]]).strip() if "标题" in idx and row[idx["标题"]] else labels.get(host) or host,
|
|
320
|
+
})
|
|
321
|
+
|
|
322
|
+
lines = [
|
|
323
|
+
"<!DOCTYPE NETSCAPE-Bookmark-file-1>",
|
|
324
|
+
'<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">',
|
|
325
|
+
"<TITLE>Bookmarks</TITLE>",
|
|
326
|
+
"<H1>Bookmarks</H1>",
|
|
327
|
+
"<DL><p>",
|
|
328
|
+
]
|
|
329
|
+
ordered = [c for c in CATEGORY_ORDER if c in groups] + sorted(c for c in groups if c not in CATEGORY_ORDER)
|
|
330
|
+
|
|
331
|
+
def emit_folder(name, entries):
|
|
332
|
+
lines.append(f" <DT><H3>{html.escape(name)}</H3>")
|
|
333
|
+
lines.append(" <DL><p>")
|
|
334
|
+
for entry in sorted(entries, key=lambda e: e["name"]):
|
|
335
|
+
lines.append(f' <DT><A HREF="https://{html.escape(entry["host"])}/">{html.escape(entry["name"])}</A>')
|
|
336
|
+
lines.append(" </DL><p>")
|
|
337
|
+
|
|
338
|
+
# 同级结构:常用网页(全部) + 各分类文件夹,都在书签栏同一层
|
|
339
|
+
emit_folder("常用网页", [e for cat in ordered for e in groups[cat]])
|
|
340
|
+
for cat in ordered:
|
|
341
|
+
emit_folder(cat, groups[cat])
|
|
342
|
+
lines.append("</DL><p>")
|
|
343
|
+
with open(os.path.expanduser(path), "w", encoding="utf-8") as fh:
|
|
344
|
+
fh.write("\n".join(lines) + "\n")
|
|
345
|
+
return ordered, groups
|
|
346
|
+
|
|
347
|
+
|
|
348
|
+
def write_html(items, path):
|
|
349
|
+
lines = [
|
|
350
|
+
"<!DOCTYPE NETSCAPE-Bookmark-file-1>",
|
|
351
|
+
'<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">',
|
|
352
|
+
"<TITLE>Bookmarks</TITLE>",
|
|
353
|
+
"<H1>Bookmarks</H1>",
|
|
354
|
+
"<DL><p>",
|
|
355
|
+
' <DT><H3>常用网页(chrome-frequent-pages)</H3>',
|
|
356
|
+
" <DL><p>",
|
|
357
|
+
]
|
|
358
|
+
for item in items:
|
|
359
|
+
title = html.escape(item["title"] or item["host"])
|
|
360
|
+
lines.append(f' <DT><A HREF="{html.escape(item["url"])}">{title}</A>')
|
|
361
|
+
lines += [" </DL><p>", "</DL><p>"]
|
|
362
|
+
with open(os.path.expanduser(path), "w", encoding="utf-8") as fh:
|
|
363
|
+
fh.write("\n".join(lines) + "\n")
|
|
364
|
+
|
|
365
|
+
|
|
366
|
+
def print_table(items):
|
|
367
|
+
print(f"{'访问':>6} {'最近访问':<16} 标题 / URL")
|
|
368
|
+
print("-" * 78)
|
|
369
|
+
for item in items:
|
|
370
|
+
print(f"{item['visits']:>6} {item['last_iso']:<16} {item['title'][:30]}")
|
|
371
|
+
print(f"{'':>6} {'':<16} {item['url']}")
|
|
372
|
+
|
|
373
|
+
|
|
374
|
+
def main():
|
|
375
|
+
parser = argparse.ArgumentParser()
|
|
376
|
+
parser.add_argument("--dry-run", action="store_true", help="只打印结果")
|
|
377
|
+
parser.add_argument("--write", action="store_true", help="写入飞书表")
|
|
378
|
+
parser.add_argument("--html", metavar="PATH", help="导出可导入 Chrome 的书签 HTML(按页面链接)")
|
|
379
|
+
parser.add_argument("--bookmarks-from-table", metavar="PATH", help="按飞书表的「域名」列导出书签 HTML")
|
|
380
|
+
parser.add_argument("--limit", type=int)
|
|
381
|
+
parser.add_argument("--days", type=int)
|
|
382
|
+
parser.add_argument("--min-visits", type=int)
|
|
383
|
+
args = parser.parse_args()
|
|
384
|
+
|
|
385
|
+
cfg = load_config()
|
|
386
|
+
if args.bookmarks_from_table:
|
|
387
|
+
ordered, groups = bookmarks_from_table(cfg, args.bookmarks_from_table)
|
|
388
|
+
total = sum(len(v) for v in groups.values())
|
|
389
|
+
print(f"已导出分类书签:{os.path.expanduser(args.bookmarks_from_table)}({total} 个域名 / {len(ordered)} 个分类)")
|
|
390
|
+
for cat in ordered:
|
|
391
|
+
print(f" [{cat}]")
|
|
392
|
+
for entry in sorted(groups[cat], key=lambda e: e["name"]):
|
|
393
|
+
print(" -", entry["name"], "->", "https://" + entry["host"] + "/")
|
|
394
|
+
return 0
|
|
395
|
+
for key in ("limit", "days", "min_visits"):
|
|
396
|
+
if getattr(args, key) is not None:
|
|
397
|
+
cfg[key] = getattr(args, key)
|
|
398
|
+
|
|
399
|
+
items = collect(cfg)
|
|
400
|
+
if not items:
|
|
401
|
+
print("没有满足条件的页面(可调低 --min-visits 或调大 --days)")
|
|
402
|
+
return 0
|
|
403
|
+
print(f"共 {len(items)} 个高频页面(窗口 {cfg['days']} 天,最少 {cfg['min_visits']} 次访问)\n")
|
|
404
|
+
print_table(items)
|
|
405
|
+
if args.html:
|
|
406
|
+
write_html(items, args.html)
|
|
407
|
+
print(f"\n已导出书签 HTML:{os.path.expanduser(args.html)}")
|
|
408
|
+
if args.write:
|
|
409
|
+
created, updated = write_table(cfg, items)
|
|
410
|
+
print(f"\n飞书写入完成:新增 {created} 条,更新 {updated} 条")
|
|
411
|
+
if not args.write and not args.html:
|
|
412
|
+
print("\n(dry-run:没有写表;加 --write 写飞书,--html <路径> 导出书签)")
|
|
413
|
+
return 0
|
|
414
|
+
|
|
415
|
+
|
|
416
|
+
if __name__ == "__main__":
|
|
417
|
+
sys.exit(main())
|
package/package.json
CHANGED
|
@@ -1,6 +1,40 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@lyra_wang/chrome-frequent-pages",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.5.2",
|
|
4
|
+
"description": "Chrome 高频网页 → 飞书「常用网页」表 + 按分类导出 Chrome 书签(根域名 / 10 天 / >5 次)",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"type": "module",
|
|
7
|
+
"keywords": [
|
|
8
|
+
"chrome",
|
|
9
|
+
"history",
|
|
10
|
+
"bookmarks",
|
|
11
|
+
"feishu",
|
|
12
|
+
"lark",
|
|
13
|
+
"cli"
|
|
14
|
+
],
|
|
15
|
+
"repository": {
|
|
16
|
+
"type": "git",
|
|
17
|
+
"url": "git+https://github.com/LyraWang6688/ai-capability-store.git",
|
|
18
|
+
"directory": "external-capabilities/cli/chrome-frequent-pages"
|
|
19
|
+
},
|
|
20
|
+
"bin": {
|
|
21
|
+
"chrome-frequent-pages": "bin/cli.js"
|
|
22
|
+
},
|
|
23
|
+
"files": [
|
|
24
|
+
"main.py",
|
|
25
|
+
"run-if-due.sh",
|
|
26
|
+
"inbox-deliver-mac.py",
|
|
27
|
+
"table_snapshot.py",
|
|
28
|
+
"bin/",
|
|
29
|
+
"config.example.json",
|
|
30
|
+
"README.md",
|
|
31
|
+
"LICENSE",
|
|
32
|
+
"install.sh"
|
|
33
|
+
],
|
|
34
|
+
"publishConfig": {
|
|
35
|
+
"access": "public"
|
|
36
|
+
},
|
|
37
|
+
"engines": {
|
|
38
|
+
"node": ">=20"
|
|
39
|
+
}
|
|
40
|
+
}
|
package/run-if-due.sh
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# 开机触发:每周只跑一次(ISO 周去重)。由 launchd 的 RunAtLoad 调用。
|
|
3
|
+
set -u
|
|
4
|
+
STATE_DIR="$HOME/.config/chrome-frequent-pages"
|
|
5
|
+
STATE="$STATE_DIR/last-run.json"
|
|
6
|
+
LOG="$STATE_DIR/run-if-due.log"
|
|
7
|
+
HERE="$(cd "$(dirname "$0")" && pwd)"
|
|
8
|
+
export PATH="$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
|
|
9
|
+
WEEK="$(date +%G-%V)"
|
|
10
|
+
mkdir -p "$STATE_DIR"
|
|
11
|
+
log() { echo "$(date '+%F %T') $*" >> "$LOG"; }
|
|
12
|
+
|
|
13
|
+
LAST=""
|
|
14
|
+
if [ -f "$STATE" ]; then
|
|
15
|
+
LAST="$(python3 -c 'import json,sys;print(json.load(open(sys.argv[1])).get("week",""))' "$STATE" 2>/dev/null || true)"
|
|
16
|
+
fi
|
|
17
|
+
if [ "$LAST" = "$WEEK" ]; then
|
|
18
|
+
log "skip: week $WEEK already ran"
|
|
19
|
+
exit 0
|
|
20
|
+
fi
|
|
21
|
+
|
|
22
|
+
SNAP_BEFORE="$(python3 "$HERE/table_snapshot.py" 2>/dev/null || echo '{}')"
|
|
23
|
+
OUT="$(chrome-frequent-pages --write 2>&1)"
|
|
24
|
+
CODE=$?
|
|
25
|
+
if [ $CODE -ne 0 ]; then
|
|
26
|
+
log "run failed (exit $CODE): $(echo "$OUT" | tail -3 | tr '\n' ' ')"
|
|
27
|
+
python3 "$HERE/inbox-deliver-mac.py" --task chrome-frequent-pages --result 失败 \
|
|
28
|
+
--summary "chrome-frequent-pages 执行失败:$(echo "$OUT" | tail -1 | cut -c1-70)" >> "$LOG" 2>&1 || true
|
|
29
|
+
exit 1
|
|
30
|
+
fi
|
|
31
|
+
|
|
32
|
+
LINE="$(echo "$OUT" | grep -o '飞书写入完成:.*' | tail -1)"
|
|
33
|
+
[ -z "$LINE" ] && LINE="$(echo "$OUT" | tail -1)"
|
|
34
|
+
SNAP_AFTER="$(python3 "$HERE/table_snapshot.py" 2>/dev/null || echo '{}')"
|
|
35
|
+
DETAIL="$(python3 - "$SNAP_BEFORE" "$SNAP_AFTER" <<'PYD'
|
|
36
|
+
import json, sys
|
|
37
|
+
before = json.loads(sys.argv[1] or "{}")
|
|
38
|
+
after = json.loads(sys.argv[2] or "{}")
|
|
39
|
+
new = [(h, v) for h, v in after.items() if h not in before]
|
|
40
|
+
if not new:
|
|
41
|
+
print("本周无新增页面")
|
|
42
|
+
else:
|
|
43
|
+
parts = []
|
|
44
|
+
for host, v in new[:5]:
|
|
45
|
+
cat = v.get("cat") or "其他"
|
|
46
|
+
tag = "未分类→其他" if cat == "其他" else cat
|
|
47
|
+
parts.append(f"{v.get('title') or host}({host})→ 书签「{tag}」")
|
|
48
|
+
more = "" if len(new) <= 5 else f" 等 {len(new)} 条"
|
|
49
|
+
print("新增:" + ";".join(parts) + more)
|
|
50
|
+
PYD
|
|
51
|
+
)"
|
|
52
|
+
python3 "$HERE/inbox-deliver-mac.py" --task chrome-frequent-pages --result 成功 \
|
|
53
|
+
--summary "$DETAIL;$LINE" >> "$LOG" 2>&1 || log "inbox deliver failed"
|
|
54
|
+
python3 - "$STATE" "$WEEK" <<'PY'
|
|
55
|
+
import json, sys, datetime
|
|
56
|
+
json.dump({"week": sys.argv[2], "at": datetime.datetime.now().isoformat(timespec="seconds")}, open(sys.argv[1], "w"), ensure_ascii=False)
|
|
57
|
+
PY
|
|
58
|
+
log "done week=$WEEK result=$LINE"
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
# -*- coding: utf-8 -*-
|
|
3
|
+
"""导出「常用网页」表的 域名 -> {标题, 分类} 快照,供 run-if-due.sh 做前后 diff。"""
|
|
4
|
+
import json, subprocess, sys
|
|
5
|
+
|
|
6
|
+
BASE = "ZNjDb9TrpalLuCsw0macCeVdnld"
|
|
7
|
+
TABLE = "tblDejemWsgqKSA2"
|
|
8
|
+
|
|
9
|
+
def main():
|
|
10
|
+
out = subprocess.run(
|
|
11
|
+
["lark-cli", "base", "+record-list", "--base-token", BASE, "--table-id", TABLE,
|
|
12
|
+
"--limit", "200", "--as", "user", "--format", "json"],
|
|
13
|
+
capture_output=True, text=True, timeout=120)
|
|
14
|
+
try:
|
|
15
|
+
data = json.loads(out.stdout)
|
|
16
|
+
except Exception:
|
|
17
|
+
print("{}")
|
|
18
|
+
return 1
|
|
19
|
+
meta = (data.get("data") or {})
|
|
20
|
+
cols = meta.get("fields") or []
|
|
21
|
+
rows = meta.get("data") or []
|
|
22
|
+
if "域名" not in cols:
|
|
23
|
+
print("{}")
|
|
24
|
+
return 1
|
|
25
|
+
idx = {name: i for i, name in enumerate(cols)}
|
|
26
|
+
snap = {}
|
|
27
|
+
for row in rows:
|
|
28
|
+
host = str(row[idx["域名"]] or "").strip()
|
|
29
|
+
if not host:
|
|
30
|
+
continue
|
|
31
|
+
cat = "其他"
|
|
32
|
+
if "分类" in idx:
|
|
33
|
+
raw = row[idx["分类"]]
|
|
34
|
+
if isinstance(raw, list) and raw:
|
|
35
|
+
cat = str(raw[0])
|
|
36
|
+
elif isinstance(raw, str) and raw.strip():
|
|
37
|
+
cat = raw.strip()
|
|
38
|
+
title = ""
|
|
39
|
+
if "标题" in idx and row[idx["标题"]]:
|
|
40
|
+
title = str(row[idx["标题"]]).strip()
|
|
41
|
+
snap[host] = {"title": title or host, "cat": cat}
|
|
42
|
+
print(json.dumps(snap, ensure_ascii=False))
|
|
43
|
+
return 0
|
|
44
|
+
|
|
45
|
+
if __name__ == "__main__":
|
|
46
|
+
sys.exit(main())
|