mtproto-checker 0.1.2 → 0.3.0

package/README.md

@@ -1,198 +1,249 @@
-# MTProto Checker
+# 🔍 mtproto-checker
 
-Telegram MTProto proxy checker powered by TDLib. It runs a real `testProxy`
-handshake through each proxy, so a successful result is closer to "works in
-Telegram" than a plain TCP/TLS port check.
+Telegram MTProto proxy health checker powered by **TDLib**. Performs a real `testProxy` handshake through each proxy — the same protocol path tdesktop uses. If it says ✅, the proxy **actually works** in Telegram.
 
-## Features
+## ⚡ Features
 
-- Parses `tg://proxy` and `https://t.me/proxy` links.
-- Loads proxies from direct links, remote lists, local files, or `stdin`.
-- De-duplicates by `server:port:secret`.
-- Supports hex and base64url secrets, including Fake-TLS SNI extraction.
-- Checks proxies concurrently and can re-check only successful proxies across
-  multiple iterations.
-- Works as a CLI, HTTP API server, or CommonJS library.
+- 🤝 Real MTProto handshake (not just a port scan)
+- 📡 Check from remote URLs, local files, or single proxy links
+- 🔄 Multi-iteration filtering — only survivors advance
+- 🌐 Built-in HTTP API server with Basic Auth
+- 🧹 Auto de-duplication by `server:port:secret`
+- 📊 Sorted output: working first, fastest on top
+- 🔐 Fake-TLS SNI extraction from `ee`-prefixed secrets
 
-## Requirements
-
-- Node.js 18+
-- Telegram API credentials: `TG_API_ID` and `TG_API_HASH`
-
-Get credentials from [my.telegram.org](https://my.telegram.org). No Telegram
-login or phone number is required; credentials are only used to initialize TDLib.
-
-## Install
+## 📦 Install
 
 ```bash
 npm install mtproto-checker
 ```
 
-For local development:
+Or clone locally:
 
 ```bash
-git clone git@github.com:Tar4s/mtproto-checker.git
+git clone https://github.com/Tar4s/mtproto-checker.git
 cd mtproto-checker
 npm install
 ```
 
-GitHub Packages users can install the scoped package:
+GitHub Packages (scoped):
 
 ```bash
 npm config set @tar4s:registry https://npm.pkg.github.com
 npm install @tar4s/mtproto-checker
 ```
 
-Private GitHub Packages require a token with `read:packages`.
+> **Requirements:** Node.js ≥ 18 · `TG_API_ID` + `TG_API_HASH` from [my.telegram.org](https://my.telegram.org)
+> No phone login needed — credentials only initialize TDLib.
 
-## CLI
+## 🔑 Environment Variables
+
+| Variable | Required | Description |
+|----------|:--------:|-------------|
+| `TG_API_ID` | ✅ | Telegram API ID |
+| `TG_API_HASH` | ✅ | Telegram API Hash |
+| `CHECK_AUTH_USER` | 🌐 | HTTP Basic Auth username (server mode) |
+| `CHECK_AUTH_PASSWORD` | 🌐 | HTTP Basic Auth password (server mode) |
+| `PORT` | ❌ | Server port (default `3080`) |
+
+## 🚀 CLI Usage
 
 ```bash
-TG_API_ID=12345 TG_API_HASH=abcdef npx mtproto-checker --sources urls.txt
+TG_API_ID=12345 TG_API_HASH=abcdef node check.js [sources] [options]
 ```
 
-Common inputs:
+### Input Methods
 
 ```bash
-# One direct proxy link
-TG_API_ID=12345 TG_API_HASH=abcdef node check.js \
-  --proxy 'tg://proxy?server=1.2.3.4&port=443&secret=...'
+# Single proxy link
+node check.js --proxy "tg://proxy?server=1.2.3.4&port=443&secret=ee..."
+
+# Remote URLs (positional or --url flag, repeatable)
+node check.js https://example.com/proxies.txt
+node check.js --url URL1 --url URL2
 
-# One or more remote proxy-list URLs
-TG_API_ID=12345 TG_API_HASH=abcdef node check.js \
-  --url https://example.com/proxies.txt \
-  --url https://example.com/more.txt
+# File with source URLs (one per line, # comments ok)
+node check.js --sources urls.txt
 
-# Source URL file, local proxy file, or stdin
-TG_API_ID=12345 TG_API_HASH=abcdef node check.js --sources urls.txt
-TG_API_ID=12345 TG_API_HASH=abcdef node check.js proxies.txt
-cat proxies.txt | TG_API_ID=12345 TG_API_HASH=abcdef node check.js
+# Local proxy file
+node check.js ./my-proxies.txt
+
+# Stdin
+cat proxies.txt | node check.js
 ```
 
-Options:
+### ⚙️ Options
+
+| Flag | Default | Description |
+|------|:-------:|-------------|
+| `--proxy <link>` | — | Check one proxy link directly |
+| `--url <url>` | — | Add a source URL (repeatable) |
+| `--sources <file>` | — | File of source URLs |
+| `--dc <1-5>` | `2` | Data center for `testProxy` |
+| `--timeout <sec>` | `10` | Per-proxy timeout |
+| `--concurrency <n>` | `30` | Parallel checks (lower = more accurate ms) |
+| `--iterations <n>` | `1` | Re-check rounds; only working proxies advance |
+| `--out <prefix>` | `result` | Output prefix → `.json` + `.txt` |
 
-| Option | Default | Description |
-| --- | ---: | --- |
-| `--proxy <link>` | none | Check one `tg://proxy` or `https://t.me/proxy` link directly. |
-| `--url <url>` | none | Add a remote proxy-list URL. Repeatable. |
-| `--sources <file>` | none | Read remote source URLs from a file. |
-| `--dc <1-5>` | `2` | Telegram data center used for `testProxy`. |
-| `--timeout <sec>` | `10` | Per-proxy TDLib timeout. |
-| `--concurrency <n>` | `30` | Parallel proxy checks. |
-| `--iterations <n>` | `1` | Re-check only successful proxies for `n` rounds. |
-| `--out <prefix>` | `result` | Writes `<prefix>.json` and `<prefix>.txt`. |
+### 📄 Output Files
 
-Input files may contain blank lines and `#` comments. Only MTProto proxy links
-are checked; `tg://socks` links are ignored.
+| File | Content |
+|------|---------|
+| `result.json` | Full report: server, port, SNI, latency, error, link |
+| `result.txt` | Working proxy links only, fastest first |
 
-## HTTP API
+## 🌐 HTTP API Server
 
-Running `node check.js` without arguments starts the server on `PORT` or `3080`.
-Basic auth is required.
+Start with **no arguments**:
 
 ```bash
-TG_API_ID=12345 \
-TG_API_HASH=abcdef \
-CHECK_AUTH_USER=admin \
-CHECK_AUTH_PASSWORD=secret \
+TG_API_ID=12345 TG_API_HASH=abcdef \
+CHECK_AUTH_USER=admin CHECK_AUTH_PASSWORD=secret \
 node check.js
 ```
 
-Endpoint:
-
-```http
-POST /check
-Content-Type: application/json
-Authorization: Basic ...
-
-{
-  "url": "https://example.com/proxies.txt",
-  "iterations": 3,
-  "concurrency": 10
-}
+```
+[mtproto-checker] ⚡ HTTP server listening on http://localhost:3080
+[mtproto-checker]   POST /check (Basic auth: admin:***)
 ```
 
-`url` accepts either a remote `http(s)` proxy list or one direct `tg://proxy` /
-`https://t.me/proxy` link. `iterations` defaults to `1`; `concurrency` defaults
-to `30`.
-
-Example:
+### `POST /check`
 
 ```bash
-curl -u admin:secret \
-  -H 'content-type: application/json' \
-  -d '{"url":"tg://proxy?server=1.2.3.4&port=443&secret=...","iterations":3,"concurrency":10}' \
-  http://127.0.0.1:3080/check
+curl -u admin:secret http://localhost:3080/check \
+  -H "Content-Type: application/json" \
+  -d '{"url": "https://example.com/proxies.txt", "iterations": 2, "concurrency": 20}'
 ```
 
-Response shape:
+**Request body:**
+
+| Field | Type | Default | Description |
+|-------|------|:-------:|-------------|
+| `url` | string | — | Proxy list URL or single `tg://proxy` link |
+| `iterations` | int | `1` | Check rounds |
+| `concurrency` | int | `30` | Parallel checks |
+
+**Response:**
 
 ```json
 {
   "url": "https://example.com/proxies.txt",
-  "iterations": 3,
-  "concurrency": 10,
-  "count": 1,
-  "working": 1,
+  "iterations": 2,
+  "concurrency": 20,
+  "count": 150,
+  "working": 42,
   "results": [
-    {
-      "server": "1.2.3.4",
-      "port": 443,
-      "sni": "example.com",
-      "ok": true,
-      "ms": 841,
-      "error": null,
-      "link": "tg://proxy?server=1.2.3.4&port=443&secret=..."
-    }
+    { "server": "1.2.3.4", "port": 443, "sni": "example.com", "ok": true, "ms": 312, "error": null, "link": "tg://proxy?..." }
   ]
 }
 ```
 
-## Library Usage
+**Error codes:** `400` bad request · `401` unauthorized · `404` wrong endpoint · `405` wrong method · `502` upstream fetch failed
+
+## 📚 Library API
 
 ```js
-const { checkRequestUrl, checkProxiesFromUrls } = require('mtproto-checker')
+const { checkProxyLink, checkProxiesFromURIs, startServer } = require('mtproto-checker')
+```
 
-const opts = {
-  apiId: Number(process.env.TG_API_ID),
-  apiHash: process.env.TG_API_HASH,
-  concurrency: 30,
-  iterations: 2
-}
+### `checkProxyLink(link, opts)` → `Promise<Array>`
 
-const direct = await checkRequestUrl('tg://proxy?server=1.2.3.4&port=443&secret=...', opts)
-const fromLists = await checkProxiesFromUrls(['https://example.com/proxies.txt'], opts)
+Check a single `tg://proxy` or `https://t.me/proxy` link.
 
-console.log(direct, fromLists)
+```js
+const results = await checkProxyLink(
+  'tg://proxy?server=1.2.3.4&port=443&secret=ee...',
+  { apiId: 12345, apiHash: 'abcdef' }
+)
 ```
 
-Main exports:
+```
+[mtproto-checker] Checking 1.2.3.4:443 [example.com]...
+[mtproto-checker] ✓ 312ms
+```
+
+### `checkProxiesFromURIs(uris, opts)` → `Promise<Array>`
+
+Check proxies from remote URLs, local files, or both. Auto-detects type per entry.
+
+```js
+// Remote
+const results = await checkProxiesFromURIs(
+  'https://example.com/proxies.txt',
+  { apiId: 12345, apiHash: 'abcdef' }
+)
+
+// Local
+const results = await checkProxiesFromURIs('./proxies.txt', opts)
+
+// Mix
+const results = await checkProxiesFromURIs([
+  'https://example.com/list1.txt',
+  './local-list.txt',
+  'https://example.com/list2.txt'
+], { apiId: 12345, apiHash: 'abcdef', iterations: 2, concurrency: 20 })
+```
+
+```
+[mtproto-checker] Loading 3 source(s)...
+  ↓ https://example.com/list1.txt
+  ◈ ./local-list.txt
+  ↓ https://example.com/list2.txt
+[mtproto-checker] Checking 150 proxies (dc=2, timeout=10s, concurrency=30, iterations=2)...
+
+  [  1/150] ✓   312ms  1.2.3.4:443 [example.com]
+  [  2/150] ✗ Timeout  5.6.7.8:443
+  ...
 
-- `checkRequestUrl(url, opts)`
-- `checkProxiesFromUrls(urls, opts)`
-- `loadProxiesFromUrls(urls)`
-- `checkProxies(proxies, opts)`
-- `mergeProxies(texts)`
-- `parseLink(line)`
-- `normalizeSecret(secret)`
-- `faketlsSni(hexSecret)`
+[mtproto-checker] Done: 42/150 working.
+```
+
+### `startServer(opts)` → `Promise<http.Server>`
+
+Start the HTTP API server programmatically.
+
+```js
+const server = await startServer({
+  apiId: 12345,
+  apiHash: 'abcdef',
+  user: 'admin',
+  password: 'secret',
+  port: 8080
+})
+```
+
+All fields are optional — falls back to env vars if omitted.
+
+### `opts` Reference
 
-## Output
+| Key | Type | Default | Description |
+|-----|------|:-------:|-------------|
+| `apiId` | number | — | Telegram API ID |
+| `apiHash` | string | — | Telegram API Hash |
+| `dc` | number | `2` | Data center (1–5) |
+| `timeout` | number | `10` | Timeout in seconds |
+| `concurrency` | number | `30` | Parallel checks |
+| `iterations` | number | `1` | Check rounds |
+| `onProgress` | function | — | `(proxy, res, index, total) => void` |
 
-CLI runs write:
+## 🧩 Proxy Link Formats
 
-- `result.json` or `<out>.json`: full report for the final completed round.
-- `result.txt` or `<out>.txt`: working proxy links, fastest first.
+```
+tg://proxy?server=1.2.3.4&port=443&secret=ee...
+https://t.me/proxy?server=1.2.3.4&port=443&secret=ee...
+```
 
-Use `--out fresh` to write `fresh.json` and `fresh.txt`.
+Secrets: hex (`ee...`, `dd...`), plain hex, or base64url — auto-detected. `tg://socks` links are ignored.
 
-## Troubleshooting
+## 🛠 Troubleshooting
 
-- `Set TG_API_ID and TG_API_HASH`: export both credentials or prefix the command.
-- Noisy latency: reduce `--concurrency`.
-- Need only stable proxies: increase `--iterations`.
-- Temporary TDLib files are created in `.proxy-checker-td/` and removed after the run.
+| Problem | Fix |
+|---------|-----|
+| `Set TG_API_ID and TG_API_HASH` | Export both env vars |
+| Noisy latency | Reduce `--concurrency` |
+| Need stable proxies only | Increase `--iterations` |
+| TDLib leftover files | `.proxy-checker-td/` is auto-cleaned after each run |
 
-## License
+## 📜 License
 
-No license file is currently included.
+ISC

package/check.js

@@ -415,6 +415,92 @@ async function checkSingleUrl(url, opts) {
   return checker(proxies, opts)
 }
 
+/**
+ * Load proxies from a local file path.
+ * @param {string} filePath - path to a text file with proxy links (one per line)
+ * @returns {Array} de-duplicated parsed proxies
+ */
+function loadProxiesFromFile(filePath) {
+  const text = fs.readFileSync(filePath, 'utf8')
+  return mergeProxies([text])
+}
+
+/**
+ * Check proxies from URIs — supports both remote URLs (http/https) and local
+ * file paths. Detects type automatically per entry.
+ * @param {string|string[]} uris - URL(s) or file path(s) with proxy links
+ * @param {{apiId: number, apiHash: string, dc?: number, timeout?: number, concurrency?: number, iterations?: number, onProgress?: Function}} opts
+ * @returns {Promise<Array<{proxy: object, ok: boolean, ms: number, error: string|null}>>}
+ */
+async function checkProxiesFromURIs(uris, opts) {
+  const list = Array.isArray(uris) ? uris : [uris]
+  console.error(`[mtproto-checker] Loading ${list.length} source(s)...`)
+  const directProxies = []
+  const texts = await Promise.all(list.map(uri => {
+    const parsed = parseLink(uri)
+    if (parsed) {
+      console.error(`  ⚡ ${parsed.server}:${parsed.port}`)
+      directProxies.push(parsed)
+      return Promise.resolve('')
+    }
+    if (/^https?:\/\//i.test(uri)) {
+      console.error(`  ↓ ${uri}`)
+      return fetchText(uri).catch(err => { console.error(`  ✗ Skipping ${uri}: ${err.message}`); return '' })
+    }
+    console.error(`  ◈ ${uri}`)
+    return Promise.resolve(fs.readFileSync(uri, 'utf8'))
+  }))
+  const fromTexts = mergeProxies(texts)
+  const seen = new Set(fromTexts.map(p => `${p.server}:${p.port}:${p.secret}`))
+  const proxies = [...fromTexts]
+  for (const p of directProxies) {
+    const key = `${p.server}:${p.port}:${p.secret}`
+    if (seen.has(key)) continue
+    seen.add(key)
+    proxies.push(p)
+  }
+  if (proxies.length === 0) {
+    console.error('[mtproto-checker] No valid proxy links found.')
+    return []
+  }
+  const iterations = opts.iterations ?? 1
+  console.error(`[mtproto-checker] Checking ${proxies.length} proxies (dc=${opts.dc ?? 2}, timeout=${opts.timeout ?? 10}s, concurrency=${opts.concurrency ?? 30}, iterations=${iterations})...\n`)
+  const results = await runIterativeChecks(proxies, iterations, (batch, iteration) => {
+    if (iterations > 1) console.error(`[mtproto-checker] Iteration ${iteration}/${iterations}: ${batch.length} proxies\n`)
+    return checkProxies(batch, {
+      ...opts,
+      onProgress: (proxy, res, index, total) => {
+        const tag = res.ok ? `✓ ${String(res.ms).padStart(5)}ms` : `✗ ${res.error}`
+        const sni = proxy.sni ? ` [${proxy.sni}]` : ''
+        console.error(`  [${String(index + 1).padStart(3)}/${total}] ${tag}  ${proxy.server}:${proxy.port}${sni}`)
+        if (opts.onProgress) opts.onProgress(proxy, res, index, total)
+      }
+    })
+  })
+  const working = results.filter(c => c.ok).length
+  console.error(`\n[mtproto-checker] Done: ${working}/${proxies.length} working.`)
+  return results
+}
+
+/**
+ * Check a single tg://proxy or t.me/proxy link via real MTProto handshake.
+ * @param {string} link - proxy link (tg://proxy?... or https://t.me/proxy?...)
+ * @param {{apiId: number, apiHash: string, dc?: number, timeout?: number, iterations?: number, onProgress?: Function}} opts
+ * @returns {Promise<Array<{proxy: object, ok: boolean, ms: number, error: string|null}>>}
+ * @throws if the link is not a valid MTProto proxy link
+ */
+async function checkProxyLink(link, opts) {
+  const proxy = parseLink(link)
+  if (!proxy) throw new Error('Invalid proxy link. Expected tg://proxy?... or https://t.me/proxy?...')
+  const sni = proxy.sni ? ` [${proxy.sni}]` : ''
+  console.error(`[mtproto-checker] Checking ${proxy.server}:${proxy.port}${sni}...`)
+  const results = await runIterativeChecks([proxy], opts.iterations ?? 1, batch => checkProxies(batch, opts))
+  const r = results[0]
+  if (r)
+    console.error(`[mtproto-checker] ${r.ok ? `✓ ${r.ms}ms` : `✗ ${r.error}`}`)
+  return results
+}
+
 async function loadSingleUrlProxies(url, opts = {}) {
   const directProxy = parseLink(url)
   if (directProxy) return [directProxy]
@@ -574,9 +660,10 @@ function createServer({ auth, checkUrl, logger = console.error }) {
       }
 
       const body = await readJsonBody(req)
-      if (!body || typeof body.url !== 'string' || body.url.trim() === '') {
+      const uris = body.url || body.urls || body.uri || body.uris
+      if (!uris || (typeof uris === 'string' && uris.trim() === '') || (Array.isArray(uris) && uris.length === 0)) {
         statusCode = 400
-        jsonResponse(res, statusCode, { error: 'Request body must include url' })
+        jsonResponse(res, statusCode, { error: 'Request body must include url (string or array)' })
         return
       }
       const iterations = body.iterations === undefined ? 1 : body.iterations
@@ -592,10 +679,10 @@ function createServer({ auth, checkUrl, logger = console.error }) {
         return
       }
 
-      const url = body.url.trim()
+      const input = Array.isArray(uris) ? uris.map(u => u.trim()) : [uris.trim()]
       let results
       try {
-        results = await checkUrl(url, { iterations, concurrency })
+        results = await checkUrl(input, { iterations, concurrency })
       } catch (err) {
         statusCode = 502
         jsonResponse(res, statusCode, {
@@ -604,15 +691,14 @@ function createServer({ auth, checkUrl, logger = console.error }) {
         })
         return
       }
-      const report = toReport(results)
       statusCode = 200
       jsonResponse(res, statusCode, {
-        url,
+        uris: input,
         iterations,
         concurrency,
-        count: report.length,
-        working: report.filter(item => item.ok).length,
-        results: report
+        count: results.length,
+        working: results.filter(item => item.ok).length,
+        results
       })
     } catch (err) {
       statusCode = err.statusCode || 500
@@ -629,20 +715,26 @@ function shouldStartServer(argv) {
   return argv.length === 0
 }
 
-async function startServer(env = process.env) {
-  const apiId = parseInt(env.TG_API_ID, 10)
-  const apiHash = env.TG_API_HASH
-  const user = env.CHECK_AUTH_USER
-  const password = env.CHECK_AUTH_PASSWORD
-  const port = parseInt(env.PORT || '3080', 10)
+/**
+ * Start the HTTP API server for proxy checking.
+ * @param {{apiId?: number, apiHash?: string, user?: string, password?: string, port?: number}} opts
+ *   All fields fall back to environment variables if omitted.
+ * @returns {Promise<http.Server>} the listening server instance
+ */
+async function startServer(opts = {}) {
+  const apiId = opts.apiId ?? parseInt(process.env.TG_API_ID, 10)
+  const apiHash = opts.apiHash ?? process.env.TG_API_HASH
+  const user = opts.user ?? process.env.CHECK_AUTH_USER
+  const password = opts.password ?? process.env.CHECK_AUTH_PASSWORD
+  const port = opts.port ?? parseInt(process.env.PORT || '3080', 10)
 
   if (!apiId || !apiHash) throw new Error('Set TG_API_ID and TG_API_HASH (get them at https://my.telegram.org).')
   if (!user || !password) throw new Error('Set CHECK_AUTH_USER and CHECK_AUTH_PASSWORD for HTTP Basic auth.')
-  if (!Number.isInteger(port) || port < 1 || port > 65535) throw new Error('PORT must be a valid TCP port.')
+  if (!Number.isInteger(port) || port < 1 || port > 65535) throw new Error('port must be a valid TCP port (1-65535).')
 
   const server = createServer({
     auth: { user, password },
-    checkUrl: async (url, requestOpts) => checkRequestUrl(url, {
+    checkUrl: async (url, requestOpts) => checkProxiesFromURIs(url, {
       apiId,
       apiHash,
       iterations: requestOpts.iterations,
@@ -658,7 +750,8 @@ async function startServer(env = process.env) {
     })
   })
 
-  console.error(`MTProto checker HTTP server listening on :${port}`)
+  console.error(`[mtproto-checker] ⚡ HTTP server listening on http://localhost:${port}`)
+  console.error(`[mtproto-checker]   POST /check (Basic auth: ${user}:***)`)
   return server
 }
 
@@ -725,7 +818,8 @@ async function main() {
   process.exit(0)
 }
 
-module.exports = { checkRequestUrl, checkSingleUrl, configureTdlibOnce, createServer, parseArgs, resolveInputProxies, checkProxiesFromUrls, loadProxiesFromUrls, checkProxies, runIterativeChecks, mergeProxies, parseLink, normalizeSecret, faketlsSni, shouldStartServer, startServer }
+module.exports = { checkProxyLink, checkProxiesFromURIs, startServer }
+module.exports._internals = { checkRequestUrl, checkSingleUrl, configureTdlibOnce, createServer, parseArgs, resolveInputProxies, runIterativeChecks, shouldStartServer, parseLink, normalizeSecret, faketlsSni, mergeProxies, loadProxiesFromFile }
 
 if (require.main === module) (shouldStartServer(process.argv.slice(2)) ? startServer() : main()).catch(err => {
   console.error(err)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mtproto-checker",
-  "version": "0.1.2",
+  "version": "0.3.0",
   "description": "Check Telegram MTProto proxies via a real TDLib handshake (testProxy), like tdesktop does",
   "main": "check.js",
   "exports": {

package/urls.txt

@@ -1,4 +0,0 @@
-https://raw.githubusercontent.com/SoliSpirit/mtproto/refs/heads/master/all_proxies.txt
-https://raw.githubusercontent.com/kort0881/telegram-proxy-collector/refs/heads/main/proxy_all.txt
-https://raw.githubusercontent.com/Surfboardv2ray/TGProto/refs/heads/main/proxies.txt
-https://raw.githubusercontent.com/Therealwh/MTPproxyLIST/refs/heads/main/verified/proxy_all_tme_verified.txt