ghprobe 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +164 -0
- package/LICENSE +674 -0
- package/README.en.md +582 -0
- package/README.md +575 -3
- package/bin/ghprobe.js +282 -0
- package/config.default.yaml +140 -0
- package/install.cmd +8 -0
- package/install.ps1 +119 -0
- package/package.json +51 -4
- package/src/config.js +64 -0
- package/src/doh.js +89 -0
- package/src/health.js +350 -0
- package/src/hosts.js +332 -0
- package/src/net.js +47 -0
- package/src/pool.js +174 -0
- package/src/probe.js +76 -0
- package/src/prober.js +212 -0
- package/src/ranker.js +30 -0
- package/src/server.js +56 -0
package/README.en.md
ADDED
|
@@ -0,0 +1,582 @@
|
|
|
1
|
+
# ghprobe
|
|
2
|
+
|
|
3
|
+
[](https://github.com/zmw-zmw/ghprobe/actions/workflows/ci.yml)
|
|
4
|
+
[](https://www.npmjs.com/package/ghprobe)
|
|
5
|
+
[](LICENSE)
|
|
6
|
+
[](package.json)
|
|
7
|
+
[](README.en.md#requirements-and-faq)
|
|
8
|
+
|
|
9
|
+
English | [简体中文](README.md)
|
|
10
|
+
|
|
11
|
+
**A Windows command-line tool that speeds up direct connections to GitHub: it measures the IPs that actually work on *your* network right now, writes them into your hosts file, and swaps them out automatically when they die. No proxy, no system proxy changes, no traffic hijacking.**
|
|
12
|
+
|
|
13
|
+
The problem it solves is concrete: GitHub direct connections are sometimes fast, sometimes dead, `git clone` / `git push` hang or time out, and the hosts list you copied off the internet is a static snapshot that nobody maintains — when an IP dies, nobody notices. ghprobe moves "finding a usable IP" onto your own machine, automates it, and gives it self-healing.
|
|
14
|
+
|
|
15
|
+
- **Windows** only (writing the system hosts file, UAC elevation, and registering a scheduled task are all Windows-only capabilities)
|
|
16
|
+
- Touches **the hosts file only** — installs no proxy / driver / service
|
|
17
|
+
- Requires **Node.js ≥ 18**
|
|
18
|
+
- License **GPL-3.0-only**, author **zmw**, repository <https://github.com/zmw-zmw/ghprobe>
|
|
19
|
+
|
|
20
|
+
## Quick start
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npm install -g ghprobe
|
|
24
|
+
ghprobe install # probe once + write hosts + register the per-minute health check (a single UAC prompt)
|
|
25
|
+
ghprobe uninstall # stop: remove the scheduled task and restore hosts
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Ready to use right away — you never touch the command line again. Full commands and configuration are in section 5, "Usage".
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## 1. What it is
|
|
33
|
+
|
|
34
|
+
### What it actually does
|
|
35
|
+
|
|
36
|
+
In one sentence: **ghprobe uses local probing to pick the fastest, most usable IP for each GitHub-related domain, and writes a self-maintaining hosts block.**
|
|
37
|
+
|
|
38
|
+
A full round is: confirm the machine is online (offline → skip) → resolve candidate IPs via local DoH + the built-in pool (auto-refreshed from the official meta once a day) → run a "three-stage probe" on every candidate → rank by latency → write the best IP plus one spare per domain into hosts. After that, a scheduled task health-checks periodically and re-ranks to swap in fresh IPs when the old ones die.
|
|
39
|
+
|
|
40
|
+
It provides **no** proxy, tunnel, traffic forwarding, or protocol evasion, and it does **not** try to "break through" anything; as long as the path is reachable in the first place, it does three things: **prefer good IPs, reject false-alive nodes, and replace IPs automatically when they fail.**
|
|
41
|
+
|
|
42
|
+
### Core technical points
|
|
43
|
+
|
|
44
|
+
**① Local DoH resolution, bypassing system-DNS poisoning**
|
|
45
|
+
|
|
46
|
+
Candidates do not come from system DNS (which may be poisoned); they come over DoH (RFC 8484 wire format, with no UDP/53 path at all). The defaults are endpoints verified reachable from mainland China: Alibaba DNS (`223.5.5.5` / `223.6.6.6`) and DNSPod (`1.12.12.12`).
|
|
47
|
+
|
|
48
|
+
Key detail: **DoH endpoints are reached by IP directly, with SNI validation.** If reaching a DoH endpoint itself required resolving its hostname first, you would fall into the deadlock of "resolving the DoH hostname requires resolving first." So the config stores the endpoint's IP, and a `sni` field carries the service hostname for certificate validation (RFC 6066 forbids using an IP as SNI), for example:
|
|
49
|
+
|
|
50
|
+
```yaml
|
|
51
|
+
doh:
|
|
52
|
+
- { name: alidns1, host: 223.5.5.5, sni: dns.alidns.com, path: /dns-query }
|
|
53
|
+
- { name: alidns2, host: 223.6.6.6, sni: dns.alidns.com, path: /dns-query }
|
|
54
|
+
- { name: dnspod, host: 1.12.12.12, sni: doh.pub, path: /dns-query }
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Multiple endpoints are queried concurrently; successful answers are unioned and de-duplicated, then a built-in IP pool adds more candidates.
|
|
58
|
+
|
|
59
|
+
**② Three-stage probe: only an IP that passes all three is "alive"**
|
|
60
|
+
|
|
61
|
+
For each `(domain, candidate IP)`:
|
|
62
|
+
|
|
63
|
+
1. **TCP RTT** — open a connection on port 443 and measure baseline latency.
|
|
64
|
+
2. **TLS handshake + certificate SAN validation** — `rejectUnauthorized: true`, with `servername` set to the target domain (SNI). The certificate must contain the target domain; DNS poisoning cannot forge a valid certificate.
|
|
65
|
+
3. **HTTP application-layer final check** — send a real `GET` with `Host: <domain>` on a real business path, read back a small prefix of the body and validate: is the status code in the allow list, does the body contain error signatures, does it contain features the domain must have, is the 3xx redirect target within the whitelist.
|
|
66
|
+
|
|
67
|
+
**Why is stage 3 mandatory?** Because stage 2's certificate check cannot stop "false-alive IPs on shared CDNs." Example: a shared Fastly node such as `185.199.x.x` will return a **perfectly valid** certificate for `github.com` (the SAN really does match), yet the HTTP body is `Fastly error: unknown domain` — that IP does not serve `github.com` at all. A certificate-only check would treat such a "false-alive IP" as a good node and write it into hosts, so hosts looks successfully written while GitHub becomes *less* reachable. Stage 3 does the final judgment using the **real business path** and the **response content**, which is what filters it out (for the same reason, `codeload.github.com` is probed on the archive-download path, not the root, because the root may 301 back to the main site).
|
|
68
|
+
|
|
69
|
+
**③ The candidate pool follows GitHub's official ranges daily, and gives up gracefully when it can't**
|
|
70
|
+
|
|
71
|
+
The pool isn't just a static list hard-coded in config. Once a day (`poolUpdate.intervalHours`) ghprobe pulls the latest ranges from GitHub's official `api.github.com/meta`, expands them into candidate IPs and caches them locally for the `auto` pool. So even if GitHub's Azure ranges (`20.x` / `4.x`) shift, the pool keeps up by itself — of the 24 built-in Azure IPs, 4 are already no longer in the official web range.
|
|
72
|
+
|
|
73
|
+
Two details:
|
|
74
|
+
|
|
75
|
+
- **Pulling meta requires direct-IP + SNI**: `api.github.com`'s DNS is poisoned in China so a plain request gets no response — and this step happens precisely when there is no working IP yet, so it can only use the configured **seed pool** (`ipPools.seed`, GitHub's own `140.82.112.0/20`, unchanged for years) as the direct entry point. Seeds are tried **in parallel**; the first success returns immediately and the remaining in-flight requests are aborted.
|
|
76
|
+
- **No retry when it fails**: that API is unauthenticated and limited to **60 requests/hour**, so success and failure **share the same cooldown** (24h by default). If `api.github.com` is blocked at the SNI layer (no IP swap can fix that), retrying is pointless and just burns the rate limit — ghprobe simply carries on with the existing pool + DoH.
|
|
77
|
+
|
|
78
|
+
**④ No proxy**
|
|
79
|
+
|
|
80
|
+
It writes the hosts file, and nothing else. No proxy software, no system proxy settings changed, no drivers, no traffic hijacking. Uninstalling just strips ghprobe's own block out of hosts, leaving nothing behind.
|
|
81
|
+
|
|
82
|
+
**⑤ Fault layering (this is the project's value proposition)**
|
|
83
|
+
|
|
84
|
+
"GitHub is unreachable" has causes at different layers, and the fix differs per layer:
|
|
85
|
+
|
|
86
|
+
| Fault layer | Symptom | Fixable by "changing the IP"? |
|
|
87
|
+
| --- | --- | --- |
|
|
88
|
+
| **DNS poisoning** | resolves to a wrong / unreachable IP | ✅ Yes. Just switch to a correct IP |
|
|
89
|
+
| **SNI hostname blacklist** | the hostname itself is blocked — the SNI in the TLS handshake carries the blocked domain | ❌ No. No IP will help; only **changing the domain** works |
|
|
90
|
+
| **IP blackhole** | entire IP ranges are dropped | ✅ Yes. Switch to a working range |
|
|
91
|
+
|
|
92
|
+
**Hosts-only** tools/tutorials handle only the first layer and are helpless against the second (e.g. `raw.githubusercontent.com` being precisely RST). ghprobe handles the second layer with a dedicated **fallback chain** (see section 6), treating the "changing the IP can't fix it" case separately.
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## 2. Pain points it solves
|
|
97
|
+
|
|
98
|
+
**Pain point 1: GitHub won't open, images won't load, `git clone` is slow**
|
|
99
|
+
|
|
100
|
+
`git clone` stalls at `Receiving objects`, `git push` times out, pages spin, and every image in a README is a broken icon — same repo, same network, works in the morning, dead at night. Because the usable IPs for each GitHub domain drift with region, ISP, and time of day, and the address system DNS hands you is often slow or broken.
|
|
101
|
+
|
|
102
|
+
**Pain point 2: the hosts list you copied is a static snapshot**
|
|
103
|
+
|
|
104
|
+
The "GitHub hosts accelerator" list you find online is a snapshot someone measured at one point in time. When an IP dies, nobody knows, and it never updates itself. You apply it once, it works for a few days, then it stops working and you go hunting for a new list — which is really just manually maintaining a table that is doomed to go stale.
|
|
105
|
+
|
|
106
|
+
**Pain point 3: many tools want a proxy / system proxy change / driver install**
|
|
107
|
+
|
|
108
|
+
To speed up GitHub you are asked to install a pile of things: a proxy client, a system-proxy toggle, a TUN/TAP driver, a background service… heavy, dirty, and it may change how your browser, IDE, and other software reach the network. You just want `git clone` to be a bit faster; you don't want the whole system's network path rewritten.
|
|
109
|
+
|
|
110
|
+
**Pain point 4: picking IPs by hand with `ping` / `curl` is guesswork — and the "alive" one may be fake**
|
|
111
|
+
|
|
112
|
+
A manual `ping` only proves an IP responds; it does not prove that the IP actually serves GitHub. A shared CDN node will hand you a valid TLS certificate but tell you `unknown domain` at the HTTP layer — your ping looks "fine," yet after writing it into hosts GitHub gets *worse*. Picking IPs by feel walks straight into this trap.
|
|
113
|
+
|
|
114
|
+
**Pain point 5: GitHub suddenly breaks after boot and you have to go dig up IPs yourself**
|
|
115
|
+
|
|
116
|
+
IPs usually die while you are not looking. With no automatic check, you only find out when "it stops working one day," and then you go dig manually. ghprobe self-heals on a schedule: it sits silently, and the moment a written IP fails it re-ranks and swaps in a new one — without you watching.
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## 3. How it differs from similar projects
|
|
121
|
+
|
|
122
|
+
GitHub acceleration already has mature projects. They fall into three groups — here is where ghprobe stands, and what each approach costs:
|
|
123
|
+
|
|
124
|
+
| | Static hosts snapshots<br>(e.g. GitHub520, ineo6/hosts) | Local proxy<br>(e.g. dev-sidecar) | **ghprobe** |
|
|
125
|
+
| --- | --- | --- | --- |
|
|
126
|
+
| How it works | The maintainer probes once and ships one hosts file to everyone | Install a local proxy that forwards requests to acceleration endpoints | Probes **your own network** and writes hosts only |
|
|
127
|
+
| Anything to install? | No | Yes — proxy / certificate, plus a **resident process** | No (just a scheduled task that runs once a minute and exits in seconds) |
|
|
128
|
+
| Changes the system proxy? | No | **Yes** | No |
|
|
129
|
+
| Where the IPs come from | Someone else's measurement — not necessarily best for you | N/A | DoH + the three-stage probe, run **on your machine** |
|
|
130
|
+
| When an IP dies | Wait for the maintainer to update | N/A | **Swapped automatically** (per-minute sentinel) |
|
|
131
|
+
| Can it spot "false-alive" IPs? | No | N/A | Yes — gate 3 validates against a real business path |
|
|
132
|
+
| Platform | Cross-platform | Windows / macOS / Linux | Windows |
|
|
133
|
+
|
|
134
|
+
The costs of each approach are real, not theoretical:
|
|
135
|
+
|
|
136
|
+
- **Static snapshots rot.** The IPs were measured at one point in time, from one vantage point — not necessarily best for *your* ISP; and once the maintainer stops updating, it is dead. `ineo6/hosts` content is frozen at 2023-03-08; `GitHub520`'s README says its server expires 2026-12-31 and asks for 831 CNY/year in sponsorship.
|
|
137
|
+
- **A local proxy has a system-level cost.** Proxy, certificate, resident process — it rewrites the network path for everything, when all you wanted was a faster `git clone`. `dev-sidecar`'s own README records that an early version, if you rebooted with it running, left you with no network at all.
|
|
138
|
+
- **A hosts manager only solves "how to edit"** — you still have to find the IPs yourself.
|
|
139
|
+
|
|
140
|
+
ghprobe's trade-off: **it only does "pick good IPs + maintain them", and never touches the system network path.** The cost is Windows-only, plus one UAC prompt at `install` time — after that a highest-privilege scheduled task runs silently.
|
|
141
|
+
|
|
142
|
+
### Limits: what it cannot fix
|
|
143
|
+
|
|
144
|
+
"GitHub is unreachable" is not one problem — it is four problems at four different layers. ghprobe only handles the first three:
|
|
145
|
+
|
|
146
|
+
| Fault layer | Symptom | Fixable by changing the IP? | ghprobe |
|
|
147
|
+
| --- | --- | --- | --- |
|
|
148
|
+
| **DNS poisoning** | resolves to a wrong / unreachable IP | Yes | Yes — DoH bypasses system DNS |
|
|
149
|
+
| **IP blackhole / churn** | connection timeout (`ERR_CONNECTION_TIMED_OUT`) | Yes | Yes — probing + per-minute self-healing |
|
|
150
|
+
| **False-alive IP** | TCP/TLS fine, HTTP returns 4xx | Yes | Yes — gate 3 validates the real business path |
|
|
151
|
+
| **ClientHello fingerprint blocking** | connection reset (`ERR_CONNECTION_RESET`) | **No** | **No** — see below |
|
|
152
|
+
|
|
153
|
+
The last layer needs its own explanation, because it is the most easily misunderstood. Measured (same IP `185.199.110.133`, same SNI `raw.githubusercontent.com`, same moment, 20 trials each):
|
|
154
|
+
|
|
155
|
+
| Client | Handshake success |
|
|
156
|
+
| --- | --- |
|
|
157
|
+
| `openssl s_client` | **20/20** |
|
|
158
|
+
| Node.js `tls.connect` | 5/20 |
|
|
159
|
+
| `curl` | 3/20 |
|
|
160
|
+
|
|
161
|
+
**Same IP, same hostname, only the client changes — and the success rate drops from 100% to 15%.** So the block keys on the **shape of the TLS ClientHello** (extension list, order, length) — not on the hostname itself, and not on the IP. Two consequences:
|
|
162
|
+
|
|
163
|
+
1. **Changing the IP cannot fix it** — only changing the domain can (which is why there is a fallback chain: `raw.githubusercontent.com -> cdn.jsdelivr.net`).
|
|
164
|
+
2. **This tool's probe results do not fully represent your experience** — ghprobe is Node, and its TLS fingerprint differs from your browser's. Measured: `raw.githubusercontent.com` opens fine in a browser, while the Node probe gets `ECONNRESET` on every candidate. In that case the tool writes `# blocked` (not `# absent`), meaning **inconclusive** rather than "unreachable":
|
|
165
|
+
|
|
166
|
+
```text
|
|
167
|
+
# blocked raw.githubusercontent.com 2026-10-11T10:27:09.367Z
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
`# absent` = probed, genuinely no usable IP. `# blocked` = the local probe client got fingerprinted; your browser may well work.
|
|
171
|
+
|
|
172
|
+
**Fixing the fourth layer** requires packet-level techniques (TTL manipulation, decoy packets, TCP-layer desync), which need a kernel driver (tools like GoodbyeDPI). That directly conflicts with this project's "no driver, no system network path changes" trade-off, so it is **deliberately out of scope**. ghprobe's job is: **given the network you already have, pick the right IPs and keep them working.**
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
## 4. How it works
|
|
177
|
+
|
|
178
|
+
### Overall flow
|
|
179
|
+
|
|
180
|
+
```
|
|
181
|
+
┌─────────────────────────────────────┐
|
|
182
|
+
│ config: domains + DoH endpoints + │
|
|
183
|
+
│ IP pool │
|
|
184
|
+
└──────────────────┬──────────────────┘
|
|
185
|
+
│
|
|
186
|
+
┌────────────────▼────────────────┐
|
|
187
|
+
│ Local network check ──no net──▶ │ ← all candidates would
|
|
188
|
+
│ skip this round │ be dead anyway
|
|
189
|
+
└────────────────┬────────────────┘
|
|
190
|
+
│online
|
|
191
|
+
┌────────────────▼────────────────┐
|
|
192
|
+
│ (once a day) refresh pool from │ ← no retry on failure;
|
|
193
|
+
│ official meta │ wait for next cycle
|
|
194
|
+
└────────────────┬────────────────┘
|
|
195
|
+
│
|
|
196
|
+
┌────────────────▼────────────────┐
|
|
197
|
+
│ Local DoH federation (concurrent,│ ← bypasses
|
|
198
|
+
│ IP direct-connect) + IP pool │ system-DNS poisoning
|
|
199
|
+
│ → candidate IP list │
|
|
200
|
+
└────────────────┬────────────────┘
|
|
201
|
+
│
|
|
202
|
+
┌────────────────▼────────────────┐
|
|
203
|
+
│ Three-stage probe │
|
|
204
|
+
│ ① TCP RTT │
|
|
205
|
+
│ ② TLS handshake + cert SAN │ ← blocks DNS poisoning
|
|
206
|
+
│ ③ HTTP application-layer check │ ← blocks shared-CDN fakes
|
|
207
|
+
└────────────────┬────────────────┘
|
|
208
|
+
│
|
|
209
|
+
all three passed? ──no──▶ mark failed, drop
|
|
210
|
+
│yes
|
|
211
|
+
┌────────────────▼────────────────┐
|
|
212
|
+
│ rank by score ascending, take │
|
|
213
|
+
│ the best per domain │
|
|
214
|
+
└────────────────┬────────────────┘
|
|
215
|
+
│
|
|
216
|
+
┌────────────────▼────────────────┐
|
|
217
|
+
│ live IP for primary domain? ──no──▶ probe fallback domains │ ← change domain to beat SNI blocking
|
|
218
|
+
└────────────────┬────────────────┘
|
|
219
|
+
│
|
|
220
|
+
┌────────────────▼────────────────┐
|
|
221
|
+
│ build hosts block (best + spare)│
|
|
222
|
+
└────────────────┬────────────────┘
|
|
223
|
+
│
|
|
224
|
+
┌────────────────▼────────────────┐
|
|
225
|
+
│ elevated write to hosts + flush │
|
|
226
|
+
│ DNS cache; all-dead guard runs │
|
|
227
|
+
│ before writing │
|
|
228
|
+
└─────────────────────────────────┘
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
### How the three command paths relate
|
|
232
|
+
|
|
233
|
+
There is only one core pipeline (DoH → three-stage probe → ranking), used through three command paths:
|
|
234
|
+
|
|
235
|
+
- **`ghprobe probe`** — run one full probe, **without writing hosts** (add `--apply` to write). Good for first seeing what your network can measure.
|
|
236
|
+
- **`ghprobe health`** — health check: **only tests the IPs already written into hosts**. If everything is healthy it exits silently and touches nothing; only on failure does it trigger a full re-rank to swap IPs. This is what the scheduled task actually calls.
|
|
237
|
+
- **`ghprobe install`** — one-shot enable: runs a conservative health-check-style probe first, then in **a single UAC** both writes hosts and registers the scheduled task (which calls `health --apply` on a timer).
|
|
238
|
+
|
|
239
|
+
All three share the same rules when writing hosts (same block format, same all-dead guard), so a manual `probe --apply` and the scheduled `health --apply` never clobber each other.
|
|
240
|
+
|
|
241
|
+
### Scheduling model: a sentinel, with a full re-rank only on failure
|
|
242
|
+
|
|
243
|
+
A fixed "full re-run every N hours" is a mismatch: wasteful when the network is fine, too slow when it isn't. ghprobe uses **tiers** instead:
|
|
244
|
+
|
|
245
|
+
- **Sentinel tier (default every 1 minute)**: one `health` run probes only the **already-written IPs** in the hosts block (with `probe.concurrency` concurrency), finishing in under 2 seconds. If all are healthy it exits silently without touching hosts.
|
|
246
|
+
- **Heavy tier (triggered only on failure)**: a full probe (DoH re-resolution + all candidates through the three stages) runs only once an existing IP is confirmed dead.
|
|
247
|
+
|
|
248
|
+
The sentinel interval comes from `schedule.healthMin`, default **1 minute** (also the lower bound for a Windows Task Scheduler repetition interval). It is the **worst-case wait** for "IP dies → swapped out."
|
|
249
|
+
|
|
250
|
+
### Anti-jitter and safety design
|
|
251
|
+
|
|
252
|
+
- **All-fail predicate (only a total outage counts as network failure)**: the network is judged to be at fault (blackhole window) only when **all written IPs fail** and there are ≥ 2 entries; then nothing is written that round. As long as **any single entry is healthy**, the path is up, so the failing ones are just stale/dead IPs and a re-rank is triggered. (The earlier "majority failed → give up" predicate was proven to misfire: its denominator counted only written entries and was often as low as 3–4, so "3 stale + 1 healthy" was misjudged as an outage — not fixing when it should.)
|
|
253
|
+
- **All-dead guard**: before writing, if the new block has **no valid IP at all** while hosts still has old entries, the write is **refused and the old entries are kept**. This prevents a network blip of a few tens of seconds from wiping mappings that were working. `probe --apply` and `health --apply` share this guard.
|
|
254
|
+
- **Partial update + IP stickiness**: after a full re-rank, only domains that **actually differ** from the current hosts are updated; tiny score jitter on a healthy domain won't trigger an IP change (each change costs a DNS cache flush and invalidates connection pools).
|
|
255
|
+
- **Duplicate-entry fallback**: each domain gets its best IP **plus one spare** by default. Windows returns *all* same-name entries and clients try them one by one, so if the first IP happens to die mid-jitter, the browser / `git` automatically falls through to the spare — turning a "down for up to a minute" into "one extra connect timeout". Count is controlled by `hosts.maxEntriesPerDomain` (default 2; set 1 for the old single-entry behavior).
|
|
256
|
+
- **`# absent` cooldown**: domains present in config but absent from hosts (typically SNI-blocked domains that even a full run can't save) are recorded as a `# absent <domain> <timestamp>` comment inside the hosts block, and are not retried for `schedule.absentCooldownHours` (default 1 hour), so the heavy tier doesn't fire uselessly every minute.
|
|
257
|
+
- **Local offline → skip entirely**: each round starts with a local network check — ① is there an enabled, non-loopback interface; ② can we TCP-connect to a domestic baseline target (default `223.5.5.5:443`). If the machine is offline it **does not probe, does not refresh the pool, does not write hosts** — it only logs `action=skip reason=本地无网络连接`. Rationale: offline, every candidate is necessarily dead, so a round is pure waste, and the log gets flooded with "stuck / all-dead", mixing "this machine has no network" with "GitHub is blocked" and hiding the real fault. The baseline target is deliberately **not GitHub** — it only answers "can this machine get out", independent of whether GitHub is reachable.
|
|
258
|
+
- **Candidate pool refreshed daily**: pull the latest network ranges from GitHub's official `api.github.com/meta`, expand them into candidate IPs, cache them locally and feed the `auto` pool. Why: GitHub's Azure ranges change and the built-in `azure` pool goes stale (4 of 24 measured IPs were already gone). That API is unauthenticated and limited to **60 requests/hour**, so the interval can only be large (24h by default). If it gets blocked at the SNI layer (no IP swap can fix that), **a failure waits a full cycle too** — retrying is pointless there, so ghprobe just keeps working with the existing pool + DoH. Manual trigger: `ghprobe update-pool`, which ignores the interval.
|
|
259
|
+
- **Stateless**: the `# absent` cooldown table is encoded directly in the hosts block comments — it survives process restarts and cross-process runs with no extra state file.
|
|
260
|
+
- **Cross-process write lock**: `probe --apply` and the scheduled task may write hosts concurrently, so a lock file serializes them and avoids read-modify-write clobbering; if the lock can't be acquired the round is skipped and the next health check writes it.
|
|
261
|
+
|
|
262
|
+
### The hosts block looks like this
|
|
263
|
+
|
|
264
|
+
```text
|
|
265
|
+
# === ghprobe begin ===
|
|
266
|
+
# generated 2026-10-11T00:00:00.000Z by ghprobe
|
|
267
|
+
140.82.112.3 github.com
|
|
268
|
+
20.27.177.113 github.com
|
|
269
|
+
20.27.177.113 codeload.github.com
|
|
270
|
+
185.199.108.133 raw.githubusercontent.com
|
|
271
|
+
185.199.111.133 raw.githubusercontent.com
|
|
272
|
+
# absent gist.githubusercontent.com 2026-10-11T00:00:00.000Z
|
|
273
|
+
# === ghprobe end ===
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
Everything between `# === ghprobe begin/end ===` is owned by ghprobe; repeated writes only replace what's inside the block. **Lines outside the block (your own hosts entries and anything a third-party tool wrote) are never touched.**
|
|
277
|
+
|
|
278
|
+
A domain may appear on several lines (2 by default): **the first line is the current best IP, the rest are spares**. Windows returns all same-name entries and curl / browsers try them one by one — so if the first IP happens to be dead, the client falls through to a spare instead of waiting for the next health check. Controlled by `hosts.maxEntriesPerDomain`.
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
## 5. Usage
|
|
283
|
+
|
|
284
|
+
> Platform: **Windows**; prerequisite: **Node.js ≥ 18**. Run all commands in **PowerShell or cmd**.
|
|
285
|
+
|
|
286
|
+
### Option 1: install from npm (recommended)
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
npm install -g ghprobe
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
Then two commands are all it takes:
|
|
293
|
+
|
|
294
|
+
```bash
|
|
295
|
+
ghprobe install # start: probe + write hosts + register the scheduled task (a single UAC prompt)
|
|
296
|
+
ghprobe uninstall # stop: remove the scheduled task and restore hosts
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
**What `ghprobe install` does:**
|
|
300
|
+
|
|
301
|
+
1. **Generates the user config file** `%LOCALAPPDATA%\ghprobe\config.yaml` (it **does not overwrite** an existing one, so your edits survive) — candidate pools, timeouts, scheduling interval and the local network check all live there, ready to edit.
|
|
302
|
+
2. Runs a **conservative health-check-style probe** first: it tests only the IPs already written into hosts, keeping the healthy ones and re-ranking only the confirmed-dead ones (so domains whose current IP is still healthy but weren't captured this round are not misrecorded as missing). If it detects **no local network**, this step is skipped automatically (everything would be dead anyway) but the scheduled task **is still registered**, and takes effect once you're back online.
|
|
303
|
+
3. Prompts for **a single UAC**, then in the same elevated session does two things:
|
|
304
|
+
- **Writes hosts once**: backs up the current hosts to `hosts.ghprobe-bak`, writes the ghprobe block, verifies the block marker exists, rolls back automatically on failure, and flushes the DNS cache on success.
|
|
305
|
+
- **Registers the Windows scheduled task named `ghprobe`**: it runs **silently at highest privileges (RunLevel=Highest)** (so later writes never prompt UAC again) and runs `health --apply` on the `schedule.healthMin` interval (default 1 minute). The task action is launched through a hidden-window wrapper, so it is **windowless** and does not disturb you.
|
|
306
|
+
|
|
307
|
+
It's ready to use right away; you never touch the command line again.
|
|
308
|
+
|
|
309
|
+
**What `ghprobe uninstall` does:**
|
|
310
|
+
|
|
311
|
+
1. **Restores hosts**: **strips** the whole ghprobe block and writes it back — it removes only ghprobe's own lines and **keeps third-party entries untouched** (it deliberately does *not* overwrite from `hosts.ghprobe-bak`: that backup is a "before write" snapshot, and overwriting from it would also roll back entries later added by other software), then flushes the DNS cache.
|
|
312
|
+
2. **Removes the scheduled task**: `Unregister-ScheduledTask ghprobe`; idempotent when the task doesn't exist.
|
|
313
|
+
|
|
314
|
+
The two steps are independent — a failure in either does not affect the other, and the process exits non-zero on failure.
|
|
315
|
+
|
|
316
|
+
> Published on npm: <https://www.npmjs.com/package/ghprobe>. You can also install from source — run `npm install -g .` inside the repository directory.
|
|
317
|
+
|
|
318
|
+
### Option 2: clone and one-click deploy
|
|
319
|
+
|
|
320
|
+
```bash
|
|
321
|
+
git clone https://github.com/zmw-zmw/ghprobe
|
|
322
|
+
cd ghprobe
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
Then double-click `install.cmd`, or run in PowerShell:
|
|
326
|
+
|
|
327
|
+
```powershell
|
|
328
|
+
.\install.ps1
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
`install.ps1` will: self-elevate (**a single UAC prompt in total**) → check the Node version (≥ 18) → install dependencies → register the scheduled task → write hosts once immediately → **self-check and print the result back to the current window**.
|
|
332
|
+
|
|
333
|
+
Available switches:
|
|
334
|
+
|
|
335
|
+
```powershell
|
|
336
|
+
.\install.ps1 # deploy: register the scheduled task + write hosts once now
|
|
337
|
+
.\install.ps1 -SkipProbe # register the scheduled task only; let the first health check write hosts
|
|
338
|
+
.\install.ps1 -Uninstall # uninstall: remove the scheduled task + restore hosts
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
> `install.ps1` also supports `-LogFile <path>` to set the deploy log path (default `%TEMP%\ghprobe-deploy.log`); `-Inner` is used internally for self-elevation and should not be passed manually.
|
|
342
|
+
> `install.cmd` is just a cmd wrapper around `install.ps1` (equivalent, double-click to run).
|
|
343
|
+
|
|
344
|
+
### Other commands
|
|
345
|
+
|
|
346
|
+
```bash
|
|
347
|
+
# Run one probe round and print results (does not write hosts)
|
|
348
|
+
ghprobe probe
|
|
349
|
+
ghprobe probe -j # output JSON (for scripting)
|
|
350
|
+
ghprobe probe --apply # probe then write to system hosts (UAC prompt; automatic backup)
|
|
351
|
+
|
|
352
|
+
# Only build and print the hosts block; you decide whether to write it
|
|
353
|
+
ghprobe hosts
|
|
354
|
+
|
|
355
|
+
# Run a health check manually (does not write hosts by default; only previews on failure)
|
|
356
|
+
ghprobe health
|
|
357
|
+
ghprobe health --apply # write to system hosts when a problem is detected (UAC prompt)
|
|
358
|
+
|
|
359
|
+
# Refresh the candidate pool from GitHub's official api.github.com/meta (once a day is enough)
|
|
360
|
+
# That API is limited to 60 requests/hour unauthenticated, so don't call it often.
|
|
361
|
+
# health also does this on schedule, so you rarely need it manually.
|
|
362
|
+
ghprobe update-pool
|
|
363
|
+
|
|
364
|
+
# Print the paths of the user config file and the pool cache (the config file is editable)
|
|
365
|
+
ghprobe config-path
|
|
366
|
+
|
|
367
|
+
# Local HTTP API (default 127.0.0.1:8930)
|
|
368
|
+
ghprobe serve
|
|
369
|
+
# POST /api/probe trigger one probe round, return a summary
|
|
370
|
+
# GET /api/status latest result
|
|
371
|
+
# GET /api/hosts current generated hosts block
|
|
372
|
+
# GET /api/config effective config
|
|
373
|
+
|
|
374
|
+
# Every command supports -c / --config for a user config override
|
|
375
|
+
ghprobe probe -c my.yaml
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
### Where the config file lives
|
|
379
|
+
|
|
380
|
+
`ghprobe install` generates a **user config file** at:
|
|
381
|
+
|
|
382
|
+
```
|
|
383
|
+
%LOCALAPPDATA%\ghprobe\config.yaml
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
(`ghprobe config-path` prints this path at any time.)
|
|
387
|
+
|
|
388
|
+
It **takes priority over the built-in default config** and can be edited directly: candidate pools, timeouts, scheduling interval, target domains, local network check — all live there. The file ships with full comments (in Chinese) explaining every key. Save it and the next `health` round picks it up — no reinstall needed.
|
|
389
|
+
|
|
390
|
+
> Why not in the package directory: with a global npm install the package lives under `node_modules`, and npm upgrades/reinstalls **replace it entirely**, wiping your edits.
|
|
391
|
+
>
|
|
392
|
+
> If a user config already exists, `install` **will not overwrite it** (it only copies the built-in default when the file is missing).
|
|
393
|
+
|
|
394
|
+
### Configuration override
|
|
395
|
+
|
|
396
|
+
Besides editing the user config file directly, you can also pass a YAML file containing **only the parts you want to change** via `-c my.yaml`: object properties (such as `timeouts` / `probe` / `serve` / `schedule`) are **merged recursively**, so you only write the keys you want to change.
|
|
397
|
+
|
|
398
|
+
Priority: the file given by `-c` > `%LOCALAPPDATA%\ghprobe\config.yaml` > the built-in `config.default.yaml`.
|
|
399
|
+
|
|
400
|
+
```yaml
|
|
401
|
+
# Example 1: change only the TLS timeout to 3s; dohMs / httpMs keep their defaults
|
|
402
|
+
timeouts:
|
|
403
|
+
tlsMs: 3000
|
|
404
|
+
|
|
405
|
+
# Example 2: change the sentinel interval to 5 minutes
|
|
406
|
+
schedule:
|
|
407
|
+
healthMin: 5
|
|
408
|
+
|
|
409
|
+
# Example 3: use different DoH endpoints (note: host is an IP, sni is the service domain)
|
|
410
|
+
doh:
|
|
411
|
+
- { name: alidns1, host: 223.5.5.5, sni: dns.alidns.com, path: /dns-query }
|
|
412
|
+
|
|
413
|
+
# Example 4: adjust the target domain list
|
|
414
|
+
domains:
|
|
415
|
+
- github.com
|
|
416
|
+
- raw.githubusercontent.com
|
|
417
|
+
|
|
418
|
+
# Example 5: turn off the local network check (run even when offline; rarely needed)
|
|
419
|
+
networkCheck:
|
|
420
|
+
enabled: false
|
|
421
|
+
```
|
|
422
|
+
|
|
423
|
+
The full default config lives in `config.default.yaml` in the repo (that file is the source of truth for key names, defaults and comments). The main parts:
|
|
424
|
+
|
|
425
|
+
```yaml
|
|
426
|
+
ipPools:
|
|
427
|
+
seed: [...] # the most stable entry point (GitHub's own range); keep it — it's also the direct entry for pulling meta
|
|
428
|
+
fastly: [185.199.108.133, 185.199.109.133, 185.199.110.133, 185.199.111.133]
|
|
429
|
+
azure: [...] # goes stale; the auto pool maintained by poolUpdate below covers it
|
|
430
|
+
|
|
431
|
+
domainPools:
|
|
432
|
+
github.com: [auto, seed, azure]
|
|
433
|
+
codeload.github.com: [auto, seed, azure]
|
|
434
|
+
objects.githubusercontent.com: [fastly]
|
|
435
|
+
raw.githubusercontent.com: [fastly]
|
|
436
|
+
gist.githubusercontent.com: [fastly]
|
|
437
|
+
avatars.githubusercontent.com: [fastly]
|
|
438
|
+
|
|
439
|
+
poolUpdate:
|
|
440
|
+
enabled: true
|
|
441
|
+
intervalHours: 24 # success and failure share one cooldown (a failed pull waits a full cycle, no retry storm)
|
|
442
|
+
source: "https://api.github.com/meta"
|
|
443
|
+
host: api.github.com
|
|
444
|
+
sections: [web, api]
|
|
445
|
+
maxIps: 48
|
|
446
|
+
perBigNet: 4 # how many IPs to sample per big range (/20 holds 4096 addresses; never expand fully)
|
|
447
|
+
timeoutMs: 8000
|
|
448
|
+
|
|
449
|
+
networkCheck:
|
|
450
|
+
enabled: true
|
|
451
|
+
probe: { host: 223.5.5.5, port: 443, timeoutMs: 2000 }
|
|
452
|
+
|
|
453
|
+
fallbacks:
|
|
454
|
+
raw.githubusercontent.com: [cdn.jsdelivr.net]
|
|
455
|
+
|
|
456
|
+
timeouts: { dohMs: 5000, tcpMs: 2500, tlsMs: 6000, httpMs: 6000 }
|
|
457
|
+
probe: { maxCandidates: 8, maxPoolIps: 32, maxPerDomain: 4, concurrency: 10 }
|
|
458
|
+
hosts: { maxEntriesPerDomain: 2 }
|
|
459
|
+
serve: { host: 127.0.0.1, port: 8930 }
|
|
460
|
+
schedule:
|
|
461
|
+
healthMin: 1 # sentinel interval (minutes): test written IPs only, full re-rank on failure
|
|
462
|
+
absentCooldownHours: 1 # missing-domain cooldown (hours): SNI-blocked domains can't be saved even by a full run
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
### Uninstall / rollback and logs
|
|
466
|
+
|
|
467
|
+
```bash
|
|
468
|
+
ghprobe uninstall # remove the scheduled task + restore hosts (removes only its own lines)
|
|
469
|
+
```
|
|
470
|
+
|
|
471
|
+
If you deployed via `install.ps1`, you can also use `.\install.ps1 -Uninstall`.
|
|
472
|
+
|
|
473
|
+
- **Manual hosts rollback**: before each write the file is backed up to `C:\Windows\System32\drivers\etc\hosts.ghprobe-bak`; but `ghprobe uninstall` is preferred (it strips by block and won't harm third-party entries added later).
|
|
474
|
+
- **Health-check log**: `%LOCALAPPDATA%\ghprobe\ghprobe.log`. The scheduled task runs silently and both stdout/stderr are discarded, so each round's result is appended here (`action` / `reason` / `changed`) so you can investigate if something breaks.
|
|
475
|
+
- **User config**: `%LOCALAPPDATA%\ghprobe\config.yaml` (editable — see the previous section).
|
|
476
|
+
- **Pool cache**: `%LOCALAPPDATA%\ghprobe\pool.cache.json` (maintained by `update-pool`; don't edit by hand).
|
|
477
|
+
- **Deploy log**: with `install.ps1`, written to `%TEMP%\ghprobe-deploy.log` by default.
|
|
478
|
+
|
|
479
|
+
---
|
|
480
|
+
|
|
481
|
+
## 6. About "why is cdn.jsdelivr.net in there?"
|
|
482
|
+
|
|
483
|
+
**`cdn.jsdelivr.net` is not a domestic mirror.** It is the jsDelivr public CDN (running on Cloudflare). It appears in hosts for exactly one reason: the **fallback chain**.
|
|
484
|
+
|
|
485
|
+
One fault type is the **SNI hostname blacklist** — what is blocked is the hostname itself. The SNI in the TLS handshake carries that domain, so **no IP will help**; the only way out is to **change the domain**. jsDelivr maps GitHub files directly:
|
|
486
|
+
|
|
487
|
+
```text
|
|
488
|
+
raw.githubusercontent.com/<user>/<repo>/<ref>/<path>
|
|
489
|
+
cdn.jsdelivr.net/gh/<user>/<repo>@<ref>/<path> <- same content
|
|
490
|
+
```
|
|
491
|
+
|
|
492
|
+
So when `raw.githubusercontent.com` is SNI-blocked and all primary candidates are dead, ghprobe falls back to it.
|
|
493
|
+
|
|
494
|
+
Caveats:
|
|
495
|
+
|
|
496
|
+
- By default only `raw.githubusercontent.com` has a fallback; `github.com` / `codeload.github.com` never use one.
|
|
497
|
+
- It fires only when the primary candidates are **entirely dead** — any live IP and the fallback domain is never probed.
|
|
498
|
+
- What gets written is the `cdn.jsdelivr.net` IP — this rescues "change the domain," not "change the IP."
|
|
499
|
+
|
|
500
|
+
To remove it, empty `fallbacks` (`--config` merges recursively, so an empty object works):
|
|
501
|
+
|
|
502
|
+
```yaml
|
|
503
|
+
fallbacks: {}
|
|
504
|
+
```
|
|
505
|
+
|
|
506
|
+
```bash
|
|
507
|
+
ghprobe install -c my.yaml
|
|
508
|
+
```
|
|
509
|
+
|
|
510
|
+
The cost: once `raw.githubusercontent.com` is SNI-blocked, that class of files has no alternative entry point.
|
|
511
|
+
|
|
512
|
+
---
|
|
513
|
+
|
|
514
|
+
## 7. Requirements and FAQ
|
|
515
|
+
|
|
516
|
+
### Requirements
|
|
517
|
+
|
|
518
|
+
- **OS: Windows.** Writing the system hosts file, UAC elevation, and registering a scheduled task are all Windows-only capabilities.
|
|
519
|
+
(The core logic — probing / ranking / DoH / HTTP API — is cross-platform and CI runs it on ubuntu; but on Linux/macOS, `probe` only shows results and **will not and cannot** write the system hosts file.)
|
|
520
|
+
- **Node.js ≥ 18.**
|
|
521
|
+
- First install needs administrator rights (one UAC prompt).
|
|
522
|
+
|
|
523
|
+
### FAQ
|
|
524
|
+
|
|
525
|
+
**Q: Why does it need administrator rights?**
|
|
526
|
+
The system hosts file (`C:\Windows\System32\drivers\etc\hosts`) is writable only by administrators, and registering a highest-privilege scheduled task also requires it. `ghprobe install` combines "write hosts" and "register the task" into **a single UAC prompt**, so you only see one.
|
|
527
|
+
|
|
528
|
+
**Q: Why can't it always fix things?**
|
|
529
|
+
The tool can only find IPs usable **on your current network at the current moment**. If the problem is an SNI hostname blacklist (which changing the IP can't fix) or an IP blackhole, every candidate may be unusable. In that case it hits the fallback chain (if configured) or the all-dead guard — it **keeps the old IPs and never writes an empty block**, and never forces in a wrong IP.
|
|
530
|
+
|
|
531
|
+
**Q: Will it affect other software?**
|
|
532
|
+
No. It **only writes the hosts file** — no proxy, no system-proxy changes, no traffic hijacking. It operates only on the block between `# === ghprobe begin/end ===` that belongs to it, so **your existing hosts entries and third-party entries are never modified**; uninstall likewise strips only its own block.
|
|
533
|
+
|
|
534
|
+
**Q: How many IPs per domain?**
|
|
535
|
+
Two by default (`hosts.maxEntriesPerDomain`): the best IP plus one spare. Windows returns all same-name entries and clients try them in order, so the spare is a **real fallback** — if the first IP is dead, the client retries the second instead of failing outright. Set it to 1 for the old single-entry behavior.
|
|
536
|
+
|
|
537
|
+
**Q: What if an IP dies between two health checks?**
|
|
538
|
+
The client falls through to the spare entry, so the cost is one extra connect timeout rather than a hard failure. Without a spare, the worst-case interruption would be about `schedule.healthMin` (1 minute by default).
|
|
539
|
+
|
|
540
|
+
**Q: Does the scheduled task run while the screen is locked?**
|
|
541
|
+
The task registered by `install` uses `LogonType = Interactive`, so it won't fire while locked / logged off / not logged in; it resumes on the interval after logon.
|
|
542
|
+
|
|
543
|
+
### Disclaimer
|
|
544
|
+
|
|
545
|
+
This tool performs DNS preference selection and connectivity probing only. It does **not** provide, include, or assist with any proxy, tunnel, traffic forwarding, or network-control evasion capability. You are solely responsible for any consequences of using this tool. Please comply with the laws and regulations of your jurisdiction.
|
|
546
|
+
|
|
547
|
+
---
|
|
548
|
+
|
|
549
|
+
## Changelog
|
|
550
|
+
|
|
551
|
+
See [CHANGELOG.md](CHANGELOG.md).
|
|
552
|
+
|
|
553
|
+
## License
|
|
554
|
+
|
|
555
|
+
**ghprobe** is released under the **GNU General Public License v3.0**, licensed as **GPL-3.0-only** (SPDX identifier: `GPL-3.0-only`).
|
|
556
|
+
|
|
557
|
+
Note: this is **not** `GPL-3.0-or-later`. The project **deliberately** accepts only GPL v3 and *not* "or any later version" — an intentional version lock (for example, so that a future GPL v4's terms are never automatically applied to this work). Please keep the same version lock when redistributing.
|
|
558
|
+
|
|
559
|
+
```
|
|
560
|
+
ghprobe — GitHub direct-connect optimizer: local DoH resolution + TLS-verified probing + hosts generation (no proxy)
|
|
561
|
+
Copyright (C) 2026 zmw
|
|
562
|
+
|
|
563
|
+
This program is free software: you can redistribute it and/or modify
|
|
564
|
+
it under the terms of the GNU General Public License as published by
|
|
565
|
+
the Free Software Foundation, version 3 of the License.
|
|
566
|
+
|
|
567
|
+
This program is distributed in the hope that it will be useful,
|
|
568
|
+
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
569
|
+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
570
|
+
GNU General Public License for more details.
|
|
571
|
+
|
|
572
|
+
You should have received a copy of the GNU General Public License
|
|
573
|
+
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
574
|
+
```
|
|
575
|
+
|
|
576
|
+
You are free to run, study, modify, and redistribute this software, subject to the GPL-3.0 conditions:
|
|
577
|
+
|
|
578
|
+
- **Copyleft**: when distributing this software or a derivative work you must provide the complete corresponding source and license the derivative under the **same license**.
|
|
579
|
+
- **Preserve notices**: you may not remove the copyright, license, or disclaimer notices.
|
|
580
|
+
- **No warranty**: the author provides no express or implied warranty (see GPL-3.0 sections 15 and 16).
|
|
581
|
+
|
|
582
|
+
The full license text is in [`LICENSE`](LICENSE) at the repository root. Repository: <https://github.com/zmw-zmw/ghprobe>.
|