dsh-dl-server 0.0.0-stage → 0.1.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jestions
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,120 @@
-# Temporary Holding Version
+# dsh-dl-server
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+English | [中文](README.zh.md)
+
+**DLS** — DSH Download Server
+A persistent download queue for DeepSeek Harness.
+
+> Downloads should not live inside a session.
+
+---
+
+## Install
+
+### From the UI (recommended)
+
+```
+Settings → Plugins → Install a third-party plugin → Add plugin
+```
+
+Type **`dsh-dl-server`**, confirm, then **restart DSH once**.
+
+### From the command line
+
+```sh
+dsh plugin add dsh-dl-server
+```
+
+> `dsh` must be on your PATH. The desktop build ships one at
+> `…\DeepSeek Harness\resources\runtime\cli\bin\dsh.cmd`; if you have never added it, use the UI path.
+
+**Requires DSH 0.2.1-rc.5 or newer.**
+**aria2 is optional** — install it for 16-way segmented downloading, or skip it and the plugin falls back to a single-stream transfer. Same features, less speed.
+
+---
+
+## Why
+
+Most DSH downloaders are one tool function that blocks until the file lands. They make downloads **fast**; they don't solve three things:
+
+- the download dies when the session or the turn ends
+- there is no queue, no pause, no resume
+- nothing survives a restart
+
+DLS is the **queue layer**, not the transport layer.
+
+```
+Jobs live in DSH, not in a session.
+```
+
+A download started in one session is visible to every other session — progress, destination, everything. But only the session that started it can pause, resume, or cancel it.
+
+---
+
+## The seven tools
+
+| Tool | What it does |
+| --- | --- |
+| `download_enqueue` | Queue a job, **return immediately** with a job id |
+| `download_status` | Inspect the queue or a single job (`J#1042`) |
+| `download_wait` | Block until a job reaches a terminal state, with a timeout |
+| `download_pause` | Pause, **keeping the bytes already on disk** |
+| `download_resume` | Continue from the last checkpoint |
+| `download_cancel` | Cancel, optionally dropping the partial file |
+| `download_cleanup` | Drop records (the default scope leaves downloaded files alone) |
+
+## Example
+
+```
+You:   download these two PDFs into D:\Papers and tell me when they're done
+```
+
+The agent calls `download_enqueue` twice and gets `J#10` / `J#11` back, then `download_wait`.
+
+Because enqueue returns immediately, it can fire off ten downloads without waiting for any of them.
+
+---
+
+## Measured
+
+Same machine, same file:
+
+| Setup | Throughput |
+| --- | --- |
+| TUNA mirror · single connection | 43.98 MB/s |
+| TUNA mirror · 16 threads (another plugin) | 54.20 MB/s |
+| TUNA mirror · **DLS** | **61.08 MB/s** (saturates a 500 Mbps link) |
+| HuggingFace · single-stream fallback | 1.00 MB/s |
+| HuggingFace · **DLS** | **6.96 MB/s** (7×) |
+
+The last row is the point: **on a per-connection-throttled host, segmentation beats bandwidth.**
+
+---
+
+## Behaviour worth knowing
+
+- **Ownership is per session.** Any session on the machine can *see* every job; only the owner can mutate one.
+- **`cleanup(scope="all")` deletes files.** The default `allWithoutFiles` only drops records.
+- **Checkpoints live in DSH storage**, not as lock files dropped into your home directory — the plugin does not depend on `TMPDIR` or the working directory.
+- **When aria2 is unavailable** the transport degrades to a single Node stream. No features lost, some speed lost.
+
+---
+
+## Docs
+
+- [SPEC.md](SPEC.md) — the 565-line design doc: comparison with four sibling plugins, what was borrowed from PCL2, and the v1 trade-offs
+- [CONTRIBUTING.md](CONTRIBUTING.md) — layout, how to run the tests, what not to touch
+- [SECURITY.md](SECURITY.md) — how to report a vulnerability, and what counts as one
+
+## Roadmap
+
+```
+v1    queue + persistence + seven tools + aria2 transport   ← released
+v1.1  disk headroom / silent failures / session fairness / mirrors
+v2    shell interception / torrents / own segmented engine / content addressing
+v3    user panel + callbacks into the agent
+```
+
+## License
+
+MIT

package/README.zh.md

@@ -0,0 +1,120 @@
+# dsh-dl-server
+
+[English](README.md) | 中文
+
+**DLS** — DSH Download Server
+大肥鱼自己的下载服务器插件
+
+> 下载不该活在会话里。
+
+---
+
+## 装
+
+### 方式一：界面（推荐）
+
+```
+设置 → 插件 → 安装第三方插件 → 添加插件
+```
+
+输入框里填 **`dsh-dl-server`**，确认，然后**重启一次 DSH**。
+
+### 方式二：命令行
+
+```sh
+dsh plugin add dsh-dl-server
+```
+
+> `dsh` 需要先在 PATH 里。桌面版自带的那份在
+> `…\DeepSeek Harness\resources\runtime\cli\bin\dsh.cmd`，没加过 PATH 的话用方式一。
+
+**要求 DSH 0.2.1-rc.5 或以上。**
+**aria2 可选** —— 装了走 16 线程分段，没装走单流兜底，一样能下，只是慢一些。
+
+---
+
+## 它解决什么
+
+现有的 DSH 下载插件几乎都是「一个工具函数 + 阻塞等到文件落地」。它们让下载**变快**，但没解决三件事：
+
+- 会话或回合结束，下载就断了
+- 不能排队、不能暂停恢复
+- 进程重启，什么都没了
+
+DLS 做的是**队列层**，不是传输层。
+
+```
+任务活在 DSH 里，不活在会话里
+```
+
+一个会话发起的下载，其他会话都能看见进度和落盘路径；但只有发起它的那个会话能暂停、续传、取消。
+
+---
+
+## 七个工具
+
+| 工具 | 作用 |
+| --- | --- |
+| `download_enqueue` | 入队，**立刻返回** jobId，不等下载 |
+| `download_status` | 看队列和单个任务的进度（`J#1042` 这种 id） |
+| `download_wait` | 等某个任务到终态，带超时 |
+| `download_pause` | 暂停，**保留已下字节** |
+| `download_resume` | 从断点接着下 |
+| `download_cancel` | 取消，可选删掉半截文件 |
+| `download_cleanup` | 清记录（默认不动已下载的文件） |
+
+## 用它
+
+```
+你： 把这两个 PDF 下到 D:\Papers，下完告诉我
+```
+
+agent 会调 `download_enqueue` 两次拿到 `J#10` / `J#11`，然后 `download_wait`。
+
+**入队就返回**，所以它可以一次发十个，不用等。
+
+---
+
+## 实测
+
+同一台机器，同一个文件：
+
+| 场景 | 结果 |
+| --- | --- |
+| 清华 TUNA 源 · 单连接 | 43.98 MB/s |
+| 清华 TUNA 源 · 16 线程（其他插件） | 54.20 MB/s |
+| 清华 TUNA 源 · **DLS** | **61.08 MB/s**（跑满 500M 链路） |
+| HuggingFace · 单流兜底 | 1.00 MB/s |
+| HuggingFace · **DLS** | **6.96 MB/s**（7 倍） |
+
+最后一个说明问题：**被限速的源上，分段比带宽更值钱。**
+
+---
+
+## 已知行为（不是 bug）
+
+- **归属按会话**：同一台机器上任何会话都能**看见**全部任务，但只有 owner 能改。
+- **`cleanup(scope="all")` 会删文件**；默认的 `allWithoutFiles` 只删记录。
+- **断点落在 DSH 的存储里**，不是自己往用户目录丢锁文件 —— 所以不看 TMPDIR 或 cwd 的脸色。
+- **aria2 不可用时自动降级**到 Node 单流，功能不减，速度降。
+
+---
+
+## 文档
+
+- [SPEC.md](SPEC.md) —— 565 行设计说明，含四个同类插件的对比、从 PCL2 借了什么、v1 的取舍
+- [CONTRIBUTING.md](CONTRIBUTING.md) —— 代码布局、怎么跑测试、哪些地方别乱动
+- [SECURITY.md](SECURITY.md) —— 漏洞怎么报，什么算漏洞什么不算
+
+## 路线
+
+```
+v1    队列 + 持久化 + 七个工具 + aria2 传输      ← 已发布
+v1.1  磁盘余量 / 静默失败 / 会话公平 / 镜像表
+v2    拦截钩子 / 种子 / 自研分段 / 内容寻址
+v3    用户面板 + 操作回传
+```
+
+## 许可
+
+MIT

package/cordis.patch.yml

@@ -0,0 +1,16 @@
+# dsh-dl-server bundle patch
+- insert:
+    - id: dsh-dl-server
+      name: 'dsh-dl-server'
+      config:
+        maxConcurrentJobs: 3
+        connectionsPerJob: 16
+        aria2Path: ''
+        aria2Port: 16801
+        speedLimitKbps: 0
+        proxy: ''
+        defaultDownloadDir: ''
+        checkpointSeconds: 30
+        autoResumeOnSessionReopen: true
+        autoResumeOnHostRestart: true
+        keepCompletedRecords: 200

package/lib/bootlog.js

@@ -0,0 +1,31 @@
+import fs from 'node:fs'
+import os from 'node:os'
+import path from 'node:path'
+
+/**
+ * Minimal boot trace. Overwritten every boot, so it always describes the latest
+ * start. A support report is "send me %TEMP%/dls-boot.log".
+ */
+export const BOOT_LOG = path.join(os.tmpdir(), 'dls-boot.log')
+
+let started = false
+
+/** @param {string} step @param {string} [detail] */
+export function mark(step, detail) {
+  try {
+    if (!started) {
+      fs.writeFileSync(BOOT_LOG, `dsh-dl-server boot trace\n`)
+      started = true
+    }
+    const line = `${new Date().toISOString()}  ${step}${detail ? `  ${detail}` : ''}`
+    fs.appendFileSync(BOOT_LOG, `${line}\n`)
+  } catch {
+    /* tracing must never break the plugin */
+  }
+}
+
+/** @param {string} step @param {unknown} error */
+export function markFailure(step, error) {
+  const stack = error instanceof Error ? (error.stack ?? error.message) : String(error)
+  mark(`FAILED at ${step}`, stack.split('\n').slice(0, 6).join(' | '))
+}

package/lib/domain.js

@@ -0,0 +1,51 @@
+import { z } from 'zod'
+import { defineDomain, domainTable } from '@deepseek-ai/dsh-storage-domain'
+
+const JobRecordSchema = z.object({
+  id: z.string(),
+  seq: z.number().int().nonnegative(),
+
+  ownerWorkspace: z.string(),
+  ownerSession: z.string(),
+  origin: z.string().optional(),
+
+  urls: z.array(z.string()).min(1),
+  destPath: z.string(),
+
+  state: z.enum(['queued', 'running', 'paused', 'completed', 'failed', 'canceled']),
+  pauseReason: z.enum(['user', 'session_closed', 'host_restart']).optional(),
+
+  createdAt: z.number(),
+  updatedAt: z.number(),
+
+  bytesTotal: z.number(),
+  bytesCheckpoint: z.number(),
+
+  attempts: z.number().int().nonnegative(),
+  lastError: z.string().optional(),
+
+  expectSha256: z.string().optional(),
+  finalBytes: z.number().optional(),
+})
+
+/**
+ * The persistent queue domain.
+ *
+ * per-record: one corrupt job must not make the whole queue unreadable.
+ * backup-and-skip: move the bad document aside and keep booting.
+ * global.nextSeq: the global counter behind J# ids.
+ */
+export const JobDomain = defineDomain({
+  // Domain names must match /^[a-z][a-z0-9_]*$/ — hyphens are rejected at load.
+  name: 'dsh_dl_jobs',
+  version: 1,
+  layout: 'per-record',
+  invalidRecords: 'backup-and-skip',
+  tables: {
+    jobs: domainTable(JobRecordSchema),
+  },
+  global: {
+    initial: { nextSeq: 1 },
+    schema: z.object({ nextSeq: z.number().int().positive() }),
+  },
+})

package/lib/index.js

@@ -0,0 +1,92 @@
+import { JobDomain } from './domain.js'
+import { Queue } from './queue.js'
+import { Transport } from './transport.js'
+import { registerTools } from './tools.js'
+import { mark, markFailure } from './bootlog.js'
+
+export const name = 'dsh-dl-server'
+export const inject = ['tools', 'storageDomain']
+
+const DEFAULTS = Object.freeze({
+  maxConcurrentJobs: 3,
+  connectionsPerJob: 16,
+  aria2Path: '',
+  aria2Port: 16801,
+  speedLimitKbps: 0,
+  proxy: '',
+  defaultDownloadDir: '',
+  checkpointSeconds: 30,
+  autoResumeOnSessionReopen: true,
+  autoResumeOnHostRestart: true,
+  keepCompletedRecords: 200,
+})
+
+/**
+ * Use the host logger when one is available so INFO lines never land on stdout.
+ * @param {any} ctx
+ */
+function makeLog(ctx) {
+  try {
+    const factory = ctx?.logger
+    if (typeof factory === 'function') {
+      const logger = factory('dsh-dl-server')
+      if (logger && typeof logger.info === 'function') {
+        return (level, message) => {
+          const fn = logger[level] ?? logger.info
+          fn.call(logger, message)
+        }
+      }
+    }
+  } catch {
+    /* fall through to the console fallback */
+  }
+  return (level, message) => {
+    // DLS_DEBUG=1 surfaces INFO lines on stderr; useful when no host logger is wired.
+    if (process.env.DLS_DEBUG === '1' || level === 'warn' || level === 'error') {
+      console.error(`[dsh-dl-server] ${message}`)
+    }
+  }
+}
+
+/**
+ * @param {import('@deepseek-ai/cordis').Context} ctx
+ * @param {Partial<typeof DEFAULTS>} [rawConfig]
+ */
+export async function apply(ctx, rawConfig) {
+  mark('apply enter')
+
+  const config = { ...DEFAULTS, ...(rawConfig ?? {}) }
+  const log = makeLog(ctx)
+
+  try {
+    mark('opening domain')
+    const domain = await ctx.storageDomain.open(JobDomain)
+    mark('domain open')
+
+    const transport = new Transport(config, log)
+    await transport.init()
+    mark('transport ready', `mode=${transport.mode}`)
+
+    const queue = new Queue({ domain, transport, config, log })
+    await queue.recover()
+    mark('queue ready')
+
+    registerTools({ ctx, queue, config })
+    // The tools themselves register inside the deferred ctx.inject callback;
+    // this mark only records that the subscription was installed.
+    mark('registerTools dispatched')
+
+    ctx.effect(() => async () => {
+      await queue.close()
+      await transport.dispose()
+      await domain.close()
+    })
+    mark('effect installed')
+
+    log('info', `ready (transport: ${transport.mode}, concurrency: ${config.maxConcurrentJobs})`)
+    mark('apply done')
+  } catch (error) {
+    markFailure('apply', error)
+    throw error
+  }
+}