pyattacker 0.3.2__tar.gz → 0.3.3__tar.gz

pyattacker-0.3.3/.github/workflows/docs.yml

@@ -0,0 +1,55 @@
+name: Documentation
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: docs-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  build:
+    name: Build documentation
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
+        with:
+          enable-cache: true
+          python-version: '3.11'
+          cache-suffix: docs
+      - name: Install dependencies
+        run: uv sync --locked --group docs
+      - name: Build site and check links
+        run: uv run --group docs mkdocs build --strict
+      - name: Check bilingual documentation and executable examples
+        run: uv run pytest tests/test_docs_i18n.py tests/test_docs_examples.py tests/test_tutorial.py
+      - name: Upload Pages artifact
+        if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
+        uses: actions/upload-pages-artifact@v4
+        with:
+          path: site
+
+  deploy:
+    name: Deploy documentation
+    if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Configure Pages
+        uses: actions/configure-pages@v5
+      - name: Deploy Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

{pyattacker-0.3.2 → pyattacker-0.3.3}/.gitignore

@@ -15,3 +15,5 @@ runs/
 .pyattacker/
 
 .scratch/
+
+site/

{pyattacker-0.3.2 → pyattacker-0.3.3}/CHANGELOG.md

@@ -6,6 +6,45 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.3.3] — 2026-09-21
+
+### Fixed
+
+* **Dashboard HTML injection** (#78): render event and card values as text rather than
+  interpolating untrusted values into HTML.
+* **Atomic write-behind batches** (#79): SQLite commits attempts and events in one transaction
+  and assigns IDs only after commit. Failed batches remain recoverable; legacy stores stop
+  accepting fact writes after an uncertain failure. Suite eviction and shutdown preserve
+  failure boundaries and close remaining resources.
+* **Suite query correctness and cost** (#80): apply experiment/run filters before handoff
+  limits, support task limits consistently across built-in stores, and select recent events
+  using timestamp indexes. Counts and grouped statistics avoid decoding full history while
+  preserving historical run outcomes. Read-only older databases remain supported.
+* Loopback HTTP tests bypass environment proxies (#76).
+
+### Added
+
+* **Bounded pipeline admission** (#81): `max_admitted` limits queued, executing and delayed
+  pipelines together, while due retries retain their admission. Configure it through Runner,
+  the run configuration or `--max-admitted`; Suite members share one Runner budget. Live
+  statistics and persisted run configuration expose the effective limit.
+* **Bilingual documentation website** (#82): tutorials, Python API and CLI references at
+  <https://hazer-bjtu.github.io/pyattacker/>, with search, strict link checks and automated
+  GitHub Pages publishing. The site tracks main and identifies itself as development documentation.
+
+### Changed
+
+* Admission is bounded by default to `4 * concurrency`; a positive `max_admitted` overrides
+  this. Long retry backoffs can now pause input consumption. This bounds scheduler state
+  counts, not payload bytes or total memory; MemoryStore still retains accumulated data.
+* `tasks(limit=0)` returns an empty list; negative and non-integer task limits are rejected.
+  Suite timestamp indexes add disk/write and first-writer migration cost. Exact duration
+  percentiles still read and sort matching pipeline durations.
+* English and Chinese export documentation now covers the `results` row type (#77), and
+  streaming-memory claims describe admission and store retention separately (#81).
+* Development and documentation lockfile downloads use official PyPI URLs for portable CI
+  installation, without changing artifact hashes or adding library runtime dependencies (#82).
+
 ## [0.3.2] — 2026-09-21
 
 ### Changed
@@ -613,6 +652,7 @@ hard limit; asyncio tasks only (wrap blocking code with `asyncio.to_thread`); on
 balance is statistical; a merged report is a union, not a sum; a pipeline is a linear chain; the HTTP endpoint
 is unauthenticated.
 
+[0.3.3]: https://github.com/Hazer-BJTU/pyattacker/releases/tag/v0.3.3
 [0.3.0]: https://github.com/Hazer-BJTU/pyattacker/releases/tag/v0.3.0
 [0.2.0]: https://github.com/Hazer-BJTU/pyattacker/releases/tag/v0.2.0
 [0.1.1]: https://github.com/Hazer-BJTU/pyattacker/releases/tag/v0.1.1

{pyattacker-0.3.2 → pyattacker-0.3.3}/PKG-INFO

@@ -1,12 +1,12 @@
 Metadata-Version: 2.5
 Name: pyattacker
-Version: 0.3.2
+Version: 0.3.3
 Summary: Artifact-centric, resumable async task orchestration: pipeline / task / resource pool.
 Project-URL: Homepage, https://github.com/Hazer-BJTU/pyattacker
 Project-URL: Repository, https://github.com/Hazer-BJTU/pyattacker
 Project-URL: Issues, https://github.com/Hazer-BJTU/pyattacker/issues
 Project-URL: Changelog, https://github.com/Hazer-BJTU/pyattacker/blob/main/CHANGELOG.md
-Project-URL: Documentation, https://github.com/Hazer-BJTU/pyattacker/blob/main/docs/tutorial.md
+Project-URL: Documentation, https://hazer-bjtu.github.io/pyattacker/
 Project-URL: Design, https://github.com/Hazer-BJTU/pyattacker/blob/main/docs/design.md
 Author: pyattacker contributors
 License: MIT
@@ -34,6 +34,8 @@ Description-Content-Type: text/markdown
 
 **English** | [简体中文](README.zh-CN.md)
 
+[Documentation](https://hazer-bjtu.github.io/pyattacker/)
+
 > Run tens of thousands of independent tasks to completion — resumably, observably, and without
 > reimplementing endpoint pools, retries and "which rows already ran" for the fifth time.
 
@@ -119,7 +121,7 @@ async def fetch(row: dict, ctx) -> dict:
 
 
 def dataset_rows():
-    """Your dataset. A generator works too: `map` streams it, so memory stays O(concurrency)."""
+    """Your dataset. `map` streams generators; max_admitted bounds unfinished pipelines."""
     return [{"q": f"question-{i}"} for i in range(4)]
 
 
@@ -135,6 +137,11 @@ with Runner(store="runs/qa.db", pools=[pool], concurrency=8) as runner:
     report.export_jsonl("runs/qa.jsonl")
 ```
 
+Streaming input uses `max_admitted` (default: `4 * concurrency`) to bound queued,
+running and retry-delayed pipelines together. This is a count limit, not a byte
+limit; task payloads, source buffers and Store retention also consume memory.
+The default `MemoryStore` retains results as data grows. See [RunConfig](docs/reference.md#runconfig).
+
 `ctx.acquire()` falls back to the task's declared `resource`, and `lease.report(ok=...)` feeds the pool's
 health and quota accounting.
 
@@ -524,6 +531,11 @@ every known tradeoff with its reason.
 
 ## Status
 
+**0.3.3 — reliability, bounded admission and online documentation.** Atomic write-behind batches,
+correct and faster Suite queries, dashboard HTML injection fixes, and a configurable
+`max_admitted` limit (default: `4 * concurrency`). The bilingual documentation site is now available.
+See the [release notes](https://github.com/Hazer-BJTU/pyattacker/blob/main/CHANGELOG.md#033--2026-09-21).
+
 **0.3.2 — experiment suites and supported control flow.** Independent experiments can share resource
 pools and global concurrency, with unified run/resume/report commands and optional per-experiment output
 directories. Durable run histories preserve earlier outcomes after resume. Handoffs, forward jumps,

{pyattacker-0.3.2 → pyattacker-0.3.3}/README.md

@@ -9,6 +9,8 @@
 
 **English** | [简体中文](README.zh-CN.md)
 
+[Documentation](https://hazer-bjtu.github.io/pyattacker/)
+
 > Run tens of thousands of independent tasks to completion — resumably, observably, and without
 > reimplementing endpoint pools, retries and "which rows already ran" for the fifth time.
 
@@ -94,7 +96,7 @@ async def fetch(row: dict, ctx) -> dict:
 
 
 def dataset_rows():
-    """Your dataset. A generator works too: `map` streams it, so memory stays O(concurrency)."""
+    """Your dataset. `map` streams generators; max_admitted bounds unfinished pipelines."""
     return [{"q": f"question-{i}"} for i in range(4)]
 
 
@@ -110,6 +112,11 @@ with Runner(store="runs/qa.db", pools=[pool], concurrency=8) as runner:
     report.export_jsonl("runs/qa.jsonl")
 ```
 
+Streaming input uses `max_admitted` (default: `4 * concurrency`) to bound queued,
+running and retry-delayed pipelines together. This is a count limit, not a byte
+limit; task payloads, source buffers and Store retention also consume memory.
+The default `MemoryStore` retains results as data grows. See [RunConfig](docs/reference.md#runconfig).
+
 `ctx.acquire()` falls back to the task's declared `resource`, and `lease.report(ok=...)` feeds the pool's
 health and quota accounting.
 
@@ -499,6 +506,11 @@ every known tradeoff with its reason.
 
 ## Status
 
+**0.3.3 — reliability, bounded admission and online documentation.** Atomic write-behind batches,
+correct and faster Suite queries, dashboard HTML injection fixes, and a configurable
+`max_admitted` limit (default: `4 * concurrency`). The bilingual documentation site is now available.
+See the [release notes](https://github.com/Hazer-BJTU/pyattacker/blob/main/CHANGELOG.md#033--2026-09-21).
+
 **0.3.2 — experiment suites and supported control flow.** Independent experiments can share resource
 pools and global concurrency, with unified run/resume/report commands and optional per-experiment output
 directories. Durable run histories preserve earlier outcomes after resume. Handoffs, forward jumps,

{pyattacker-0.3.2 → pyattacker-0.3.3}/README.zh-CN.md

@@ -9,6 +9,8 @@
 
 [English](README.md) | **简体中文**
 
+[在线文档](https://hazer-bjtu.github.io/pyattacker/zh-CN/)
+
 > 把几万条独立任务一口气跑完——崩了能续、跑着能看，而且不用第五次重写端点池、重试逻辑和"哪些行已经跑过"。
 
 ## 它解决什么问题
@@ -76,7 +78,7 @@ async def fetch(row: dict, ctx) -> dict:
 
 
 def dataset_rows():
-    """Your dataset. A generator works too: `map` streams it, so memory stays O(concurrency)."""
+    """Your dataset. `map` streams generators; max_admitted bounds unfinished pipelines."""
     return [{"q": f"question-{i}"} for i in range(4)]
 
 
@@ -92,6 +94,11 @@ with Runner(store="runs/qa.db", pools=[pool], concurrency=8) as runner:
     report.export_jsonl("runs/qa.jsonl")
 ```
 
+流式输入通过 `max_admitted`（默认 `4 * concurrency`）限制排队、执行和等待重试的
+流水线总数。这是数量上限，不是字节上限；任务载荷、源缓冲和 Store 保留数据也占用内存。
+默认 `MemoryStore` 会随数据量增长保留结果。详见 [RunConfig](docs/zh-CN/reference.md#runconfig)。
+
+
 `ctx.acquire()` 不指定参数时，自动用任务声明的 `resource`。`lease.report(ok=...)` 把健康和配额信息喂回资源池。
 
 真正的网络调用写在哪？——你自己写，等框架跑通了再加：
@@ -393,6 +400,10 @@ uv run pyattacker run -c examples/qa_eval.yaml --limit 40
 
 ## 当前状态
 
+**0.3.3——可靠性、准入上限与在线文档。** 提供原子 write-behind 批次、正确且更快的 Suite 查询、
+仪表盘 HTML 注入修复，以及可配置的 `max_admitted` 上限（默认 `4 * concurrency`）。
+双语文档站现已上线。详见[发布说明](https://github.com/Hazer-BJTU/pyattacker/blob/main/CHANGELOG.md#033--2026-09-21)。
+
 **0.3.2——组合实验与正式控制流特性。** 独立子实验可共享资源池和总并发，统一运行、恢复和查看结果，
 并可按实验分目录存储。持久化的运行历史在恢复后仍保留此前的结果。交接、正向跳转、回退和全量重试
 作为正式公共特性维护，通过 `control` 显式声明后启用。

{pyattacker-0.3.2 → pyattacker-0.3.3}/docs/cli.md

@@ -46,6 +46,7 @@ pyattacker run -c config.yaml [options]
 | `--limit N` | run only the first N pipelines (a smoke test over a real dataset) |
 | `--store PATH` | override `run.store`. Used verbatim with `--shard`; otherwise a shard suffix is appended |
 | `--concurrency N` | override `run.concurrency` — attempts in flight, not pipelines alive |
+| `--max-admitted N` | override `run.max_admitted`: queued + running + delayed pipelines per Runner (positive integer; default `4 * concurrency`) |
 | `--journal {full,summary}` | `full` (default) stores artifact payloads, which is what makes resume work at task granularity; `summary` keeps digests only |
 | `--label TEXT` | a label recorded on the run, for telling runs apart later |
 | `--resume` | skip finished pipelines, restart failed ones at their checkpoint |
@@ -303,7 +304,7 @@ source: { kind: jsonl, path: data.jsonl, limit: 100, key_field: id, repeats: 1 }
 ```
 
 The `run:` block accepts exactly the fields the CLI maps onto `RunConfig`: `store`, `journal`,
-`concurrency`, `label`, `heartbeat_s`, `grace_s`, `stale_after_s`, `strict_leases`,
+`concurrency`, `max_admitted`, `label`, `heartbeat_s`, `grace_s`, `stale_after_s`, `strict_leases`,
 `stop_after_failures`, `stop_after_s`, `max_handoffs`, `retry_succeeded`, `fresh_restart`, `seed`, `notes`,
 `write_behind`,
 `write_batch`, `flush_interval`, `artifact_backend` and `meta`. Anything else is a config error, not a
@@ -397,7 +398,7 @@ Python examples, `HistoryArtifact` included.
 * [`docs/tutorial.md`](tutorial.md) — the guided path, with runnable programs
 * [`docs/reference.md`](reference.md) — every class and function the SDK exposes
 * [`docs/design.md`](design.md) — why the CLI has these commands and no others
-* [`README.md`](../README.md) — the compact tour
+* [`README.md`](https://github.com/Hazer-BJTU/pyattacker/blob/main/README.md) — the compact tour
 
 ## Suite commands
 

{pyattacker-0.3.2 → pyattacker-0.3.3}/docs/design.md

@@ -2,7 +2,7 @@
 
 **English** | [简体中文](zh-CN/design.md)
 
-> Version: 0.3.2 — M0–M5 complete, including the supported control flow of §4.8 (forward handoffs and
+> Version: 0.3.3 — M0–M5 complete, including the supported control flow of §4.8 (forward handoffs and
 > backward traversal), enabled through explicit declarations. See "Implemented / Left for later" in Section 9.
 > In one sentence: **an async task orchestration framework centered on the artifact, using the pipeline as the unit of completion, and the resource pool as the only shared surface.**
 > It does not touch the network, does not do reduction, and does not do DAG scheduling — it is only responsible for "running tens of thousands of mutually independent pipelines to completion, reliably, recoverably, and observably".
@@ -318,6 +318,11 @@ short, and with a fake clock the pump advances virtual time instantly (so tests
 `pipelines`, `tasks`, `artifacts` — always go straight through, because a checkpoint that is not durable yet
 is not a checkpoint. A `SIGKILL` can therefore lose the last batch of history while every checkpoint stays
 intact; `--no-write-behind` trades throughput for an immediate commit per attempt.
+SQLite commits a whole attempts/events batch in one transaction, publishing row IDs only after success.
+A failed batch rolls back and remains buffered for retry. Suite stores batch per database, not across
+all child databases. Legacy stores without atomic `write_facts` acknowledge one row at a time; after
+an exception, the wrapper blocks automatic replay because that call may already have committed.
+See [fact batch capability](reference.md#fact-batches-and-failure-recovery) for the recovery contract.
 
 ### 4.6 Monitoring: Operational State and Application Reports
 
@@ -655,8 +660,9 @@ digests and metadata (saves space, at the cost that intermediate artifacts canno
 rerun the whole pipeline, and it leaves a `pipeline.checkpoint_missing` event behind).
 
 **Write policy**: all store methods are synchronous — this keeps critical writes such as "record a running row
-before the attempt starts" from being interrupted by cancellation. Batched writes / write-behind merging is a
-later optimization and does not affect the interface.
+before the attempt starts" from being interrupted by cancellation. Write-behind combines append-only
+attempts/events through the optional atomic `write_facts` capability; state writes remain synchronous
+and the required `Store` interface is unchanged.
 
 **Read policy**: the list queries above may materialize their result — a report wants a list. A whole-kind
 read that must stay bounded (an export of a large store) goes through the optional paged-iteration
@@ -710,7 +716,7 @@ pool = Pool("apis", [Resource.create("llm", capacity=4,
 template = pipeline("qa_eval", fetch | ask | judge | metrics, tags={"bench": "mmlu"})
 
 with Runner(store="runs/qa.db", pools=[pool], concurrency=64) as runner:
-    report = runner.run(template.map(dataset_rows, repeats=3))   # a generator, streaming, memory O(concurrency)
+    report = runner.run(template.map(dataset_rows, repeats=3))   # streams seeds under the max_admitted budget
     print(report.summary())
     report.export_jsonl("runs/qa.jsonl")
 ```
@@ -791,7 +797,8 @@ children can never fight over one file. Each child also gets `PYATACKER_SHARD=I/
 can record its own provenance.
 
 **Row shapes** (`--rows`) for whatever consumes the results: `pipelines` (nested, the default), `tasks`,
-`attempts` (with the retry `decision`), `events`, `artifacts`. **Formats** (`--format`): `jsonl`, `json`, `csv`.
+`attempts` (with the retry `decision`), `events`, `artifacts`, and `results` (one compact row per pipeline,
+with its key, repeat, status, final payload and error; failed/interrupted rows are retained). **Formats** (`--format`): `jsonl`, `json`, `csv`.
 CSV takes its header from the first `header_rows` rows and folds anything introduced later into an `extra`
 column, which keeps memory flat without silently dropping fields. Every kind is exported in full — `events`
 used to stop at the newest 100 000 rows — and read in bounded batches; see
@@ -873,7 +880,7 @@ src/pyattacker/
   monitor.py      snapshot rendering + watch
   shard.py        deterministic shard assignment + per-shard store paths (M3)
   merge.py        join N shard stores: de-duplicate by pipeline_id, recompute statistics (M3)
-  export.py       row shapes (pipelines/tasks/attempts/events/artifacts) and formats (jsonl/json/csv) (M3)
+  export.py       row shapes (pipelines/tasks/attempts/events/artifacts/results) and formats (jsonl/json/csv) (M3)
   declarative.py  YAML/TOML → pools + pipeline + source (including ${ENV} expansion)
   plugins.py      entry-point discovery for tasks/algorithms/codecs/stores (M4)
   backends.py     where artifact payloads live: inline / content-addressed files / null (M4)

pyattacker-0.3.3/docs/documentation.md

@@ -0,0 +1,49 @@
+# Maintaining the documentation
+
+**English** | [简体中文](zh-CN/documentation.md)
+
+The website uses the Markdown files in `docs/` directly. The Python API reference is currently
+curated Markdown; automatic API extraction is not part of this first site migration.
+
+## Preview locally
+
+From the repository root, install the locked documentation dependencies and run the server:
+
+```bash
+uv sync --locked --group docs
+uv run --group docs mkdocs serve
+```
+
+Open the address printed by MkDocs (normally `http://127.0.0.1:8000/pyattacker/`).
+The `/pyattacker/` prefix matches GitHub Pages, so relative links can be checked locally.
+
+## Check a change
+
+```bash
+uv run --group docs mkdocs build --strict
+uv run pytest tests/test_docs_i18n.py tests/test_docs_examples.py tests/test_tutorial.py
+```
+
+The generated `site/` directory is ignored by Git. Keep dependencies in the `docs` dependency
+group and commit `uv.lock` updates. They are not library runtime dependencies.
+Add new pages to `mkdocs.yml`, with a Chinese counterpart and reciprocal language links.
+Extend the document pairs in `tests/test_docs_i18n.py`; keep runnable code blocks identical
+between languages. Existing tutorial/reference markers must remain so the tests execute examples.
+Links to files outside `docs/` should use full GitHub URLs; Markdown pages inside `docs/` use
+relative links. Strict builds check page links and anchors, but do not check external URL availability.
+
+## Publish
+
+The repository's Pages source is **GitHub Actions**. The Documentation workflow builds on PRs,
+pushes to `main`, and manual runs. PRs only validate; pushes to `main` or manual runs on `main`
+upload the site and deploy through the `github-pages` environment. No custom domain is required.
+
+The site is published at <https://hazer-bjtu.github.io/pyattacker/>. Check the repository's Actions
+tab if a build or deployment fails. Environment protection rules may require an approval before
+deployment. The deployment job alone receives Pages and OIDC write permissions.
+
+## Version policy
+
+The first site tracks `main` and displays a development-version notice on every page.
+It is not a frozen reference for the latest package release. A later versioned publishing setup
+can add a stable release alongside development documentation.

pyattacker-0.3.3/docs/hooks.py

@@ -0,0 +1,15 @@
+"""Preserve the repository's GitHub-style heading links in the rendered website."""
+
+
+def github_slug(text: str, separator: str) -> str:
+    # Markdown has already stripped inline markup before calling the TOC slugger.
+    # Keep Unicode and repeated spaces: 'Step 1 — task' becomes 'step-1--task'.
+    return "".join(
+        char if char.isalnum() or char in "-_" else separator if char.isspace() else ""
+        for char in text.lower()
+    )
+
+
+def on_config(config):
+    config.mdx_configs["toc"]["slugify"] = github_slug
+    return config

pyattacker-0.3.3/docs/index.md

@@ -0,0 +1,48 @@
+# pyattacker
+
+**English** | [简体中文](zh-CN/index.md)
+
+Build resumable Python workflows for model evaluations and other dataset-driven tasks.
+Combine task chains, shared resource pools, retries and durable checkpoints under one Runner.
+
+[Start the tutorial](tutorial.md){ .md-button .md-button--primary }
+[Python API reference](reference.md){ .md-button }
+
+## Start here
+
+Install Python 3.11 or later, then install the library:
+
+```bash
+python -m pip install pyattacker
+pyattacker demo
+```
+
+For YAML configurations, install `pyattacker[yaml]`. The base library has no runtime dependencies.
+This site follows the `main` branch; your installed release may not yet include every feature shown.
+To try the development version, clone the repository and follow the [tutorial setup](tutorial.md#step-0--install-and-sanity-check).
+
+## Choose a path
+
+| Your goal | Read |
+| --- | --- |
+| Write a task and run a dataset | [Step-by-step tutorial](tutorial.md) |
+| Share endpoints, retry failures and resume work | [Resources, retry and recovery tutorial](tutorial.md#find-what-you-need) |
+| Compare several experiments under one Runner | [Experiment suites](suites.md) |
+| Look up Python interfaces and behavior | [API reference](reference.md) |
+| Run from JSON, TOML or YAML | [CLI and configuration](cli.md) |
+| Understand checkpoints and scheduling guarantees | [Architecture](design.md) |
+| Compare resource scheduling strategies | [Benchmarking](benchmark.md) |
+
+## Know what is retained
+
+`Runner` defaults to an in-memory store. Choose a file-backed store to retain checkpoints across
+processes. `concurrency` limits executing attempts; `max_admitted` bounds queued, executing and
+delayed pipelines together. Payloads, input buffers and accumulated store data also consume memory.
+See [RunConfig](reference.md#runconfig) before scaling a workload.
+
+## Explore and contribute
+
+Browse the [runnable examples](https://github.com/Hazer-BJTU/pyattacker/tree/main/examples),
+[release history](https://github.com/Hazer-BJTU/pyattacker/blob/main/CHANGELOG.md), or
+[report an issue](https://github.com/Hazer-BJTU/pyattacker/issues).
+For local previews and publishing, see [Maintaining the documentation](documentation.md).

pyattacker-0.3.3/docs/overrides/main.html

@@ -0,0 +1,8 @@
+{% extends "base.html" %}
+{% block announce %}
+  {% if page and page.file.src_uri.startswith('zh-CN/') %}
+    开发版文档 · 对应 main 分支，可能包含尚未发布的功能。
+  {% else %}
+    Development documentation · Tracks main and may include unreleased features.
+  {% endif %}
+{% endblock %}

{pyattacker-0.3.2 → pyattacker-0.3.3}/docs/reference.md

@@ -19,7 +19,7 @@ shaped this way.
 * **Defaults matter here.** Two in particular surprise people: `Retrying(max_attempts=1)` means *no retries*
   unless you ask, and `Runner(store=":memory:")` means nothing is persisted unless you give it a path.
 * **Some examples are complete programs.** A block whose first line is `# reference/<name>.py` is written out
-  and executed by [`tests/test_docs_examples.py`](../tests/test_docs_examples.py) on every test run — the
+  and executed by [`tests/test_docs_examples.py`](https://github.com/Hazer-BJTU/pyattacker/blob/main/tests/test_docs_examples.py) on every test run — the
   README and the CLI document share that contract. Blocks without a marker are fragments on purpose.
 
 ## Contents
@@ -330,7 +330,7 @@ runner.run(template.map(rows))                              # one pipeline per r
 runner.run(template.map(rows, repeats=5))                   # pass@5
 runner.run(template.map(rows, key_of=lambda r: r["qid"]))   # ids from your dataset's primary key
 
-def stream():                                                # memory stays O(concurrency)
+def stream():                                                # streams seeds; max_admitted bounds unfinished pipelines
     with open("dataset.jsonl") as fh:
         for line in fh:
             yield json.loads(line)
@@ -760,7 +760,27 @@ live = runner.stats()                                             # while runnin
 print(live["in_flight_pipelines"], live["delayed_pipelines"])      # in flight vs parked in backoff
 ```
 
-`run()` accepts any iterator of `PipelineSpec`, so a generator keeps memory flat regardless of dataset size.
+`run()` reads a `PipelineSpec` iterator only while admission capacity is available.
+`max_admitted=None` (default) resolves to `4 * concurrency`; a positive integer sets
+an explicit limit on queued, executing and delayed pipelines **together**. Retries
+retain their admission and can return to workers while new input is paused. Lower
+limits reduce memory held by unfinished work, but can leave workers idle during
+backoff; a limit below concurrency also limits useful parallelism. Suite members
+share one budget per Runner; shard processes each have their own budget.
+
+This bounds the number of unfinished scheduler states, not total bytes or process
+RSS. Memory also includes each retained seed/current value, source-side buffering,
+worker references, write-behind buffers, caches and reporting queries. The default
+`MemoryStore` retains accumulated records and artifact payloads as the run grows;
+use a file-backed store and a streaming source for large datasets. A pre-built input
+list or unbounded task payload remains outside the admission guarantee.
+
+`runner.stats()` exposes `admitted_pipelines` (admitted minus terminal) and the
+effective `max_admitted`; the run's persisted config records that effective limit.
+Stop, cancellation and run budgets also interrupt admission waits. As before,
+synchronous input iteration and blocking task code must return control before the
+async scheduler can respond.
+
 
 ### `RunConfig`
 
@@ -771,6 +791,7 @@ Everything that shapes one run. Pass a `RunConfig`, or pass its fields as keywor
 | `store` | `":memory:"` | `":memory:"`, a SQLite path, a plugin URI, or an open store instance |
 | `journal` | `"full"` | `"full"` keeps artifact payloads (**required for task-level resume**); `"summary"` keeps only metadata |
 | `concurrency` | `16` | max attempts in flight; a pipeline parked in a retry backoff does not hold a slot |
+| `max_admitted` | `None` | queued + executing + delayed pipeline limit; auto = `4 * concurrency`, otherwise a positive integer |
 | `label` | `""` | a label recorded on the run |
 | `run_id` | `None` | explicit run id; default is timestamp + digest |
 | `resume` | `False` | mark pipelines abandoned by dead runs as resumable before scheduling |
@@ -1404,6 +1425,37 @@ with closing(open_store("runs/qa.db")) as store:      # reopen a finished run, a
 `spec` can be `":memory:"`, a path, a plugin URI (`"s3://bucket/runs.db"`), or an already-open store.
 Reading a live store while a run writes to it is supported — WAL allows one writer and many readers.
 
+### Fact batches and failure recovery
+
+`pyattacker.store.FactBatchStore` is an optional capability, separate from the required `Store`
+protocol: `write_facts(attempts, events)` atomically commits both sequences and assigns their IDs
+only after success. On exception no batch rows may be committed and input IDs must be unchanged.
+The capability owns its transaction; SQLite rejects a call inside an existing transaction without
+committing or rolling back the caller's work. An empty batch performs no transaction.
+
+`WriteBehindStore` uses this capability when available. SQLite implements it, including the SQLite
+stores wrapped by either Suite layout. A failed atomic batch stays buffered and can be flushed again.
+In a split Suite, a successful child batch stays committed if a later child's batch fails; retry only
+writes the remaining batches. Eviction flushes before removing a child from the connection cache,
+so a failed flush leaves that child and its pending batch available for retry. Checkpoint, artifact,
+and task-state writes still bypass the buffer; handoff/visit transactions keep their existing boundaries.
+
+A legacy store without `write_facts` is supported through individual `record_attempt`/`emit_event`
+calls. Each normally returned call removes that confirmed record from the buffer. If a call raises,
+the original exception propagates and the failing record plus its unattempted suffix stay buffered.
+Because the failing call might have committed before raising, this wrapper then refuses further
+fact writes and flush-dependent reads/flushes with `StoreUnavailable`, chaining the original error.
+`buffer_stats()["flush_blocked"]` reports this condition; `pending` includes the uncertain record.
+Reconcile persisted records with the original submitted facts before constructing another wrapper;
+do not blindly replay the pending suffix. There is no automatic exactly-once guarantee for an
+ambiguous legacy write. Custom stores that can guarantee atomic rollback should implement the optional
+capability instead. SQLite subclasses that customize fact writes must also customize `write_facts`
+or disable it with `write_facts = None` to keep using their individual write overrides.
+
+Closing always releases the underlying store even when flush raises; the error remains visible.
+Suite close attempts every child, the catalog and the ownership lock, and propagates the first error.
+After a failed close, reopen/reconcile as needed; do not retry a flush on the closed connection.
+
 ### Tables and readers
 
 | Table | One row per | Read with |
@@ -1888,6 +1940,7 @@ One row per kind, in the order `limit` truncates:
 | `attempts` | attempt, including each retry `decision` | `attempt_id` (write order) |
 | `events` | structured event | `event_id` (write order, oldest first) |
 | `artifacts` | artifact, intermediate ones included | pipeline order, then `seq`, then `artifact_id` |
+| `results` | pipeline result: key, repeat, state, final payload and error; failed/interrupted rows included | `created_at`, then `pipeline_id` |
 
 `limit` means the same thing for every kind — it counts rows of that kind, including `artifacts`,
 where it used to count pipelines:
@@ -2030,7 +2083,7 @@ print(PLUGINS.errors())        # {name: reason} for everything that failed to lo
 Built-ins resolve first, and a plugin that raises on import is recorded rather than propagated — so a broken
 plugin cannot take a run down, and it cannot fail silently either. `pyattacker plugins` prints the same
 information. `PluginRegistry` is the type; `PLUGINS` is the process-wide instance. A complete worked package
-is in [`examples/plugin_package/`](../examples/plugin_package/README.md).
+is in [`examples/plugin_package/`](https://github.com/Hazer-BJTU/pyattacker/blob/main/examples/plugin_package/README.md).
 
 ---
 
@@ -2177,7 +2230,7 @@ runner.run(template.map(jsonl_source("dataset.jsonl", limit=500)))
 
 An application can publish its latest experiment values while a run is active. Pyattacker stores
 and displays the values; the application computes them. For a runnable accuracy example, see
-[`examples/live_metrics.py`](../examples/live_metrics.py).
+[`examples/live_metrics.py`](https://github.com/Hazer-BJTU/pyattacker/blob/main/examples/live_metrics.py).
 
 ```python
 runner.report_metric("evaluated", completed, label="Evaluated")
@@ -2273,7 +2326,7 @@ command line.
 | [`docs/tutorial.md`](tutorial.md) | the guided path: seventeen runnable steps |
 | [`docs/cli.md`](cli.md) | commands, flags, exit codes, config file format |
 | [`docs/design.md`](design.md) | the model, the invariants, and the tradeoffs behind these APIs |
-| [`examples/`](../examples) | complete programs, including a measured comparison of pipeline shapes |
+| [`examples/`](https://github.com/Hazer-BJTU/pyattacker/tree/main/examples) | complete programs, including a measured comparison of pipeline shapes |
 
 ## Suite API
 

{pyattacker-0.3.2 → pyattacker-0.3.3}/docs/releasing.md

@@ -3,7 +3,7 @@
 **English** | [简体中文](zh-CN/releasing.md)
 
 Written for maintainers with push access. Publishing is automated: you push a tag, and
-[`.github/workflows/release.yml`](../.github/workflows/release.yml) does the rest.
+[`.github/workflows/release.yml`](https://github.com/Hazer-BJTU/pyattacker/blob/main/.github/workflows/release.yml) does the rest.
 
 ## One-time setup: trusted publishing
 
@@ -47,7 +47,8 @@ cannot be borrowed by a branch push.
 release notes. No section means the release still publishes, with a placeholder and a warning.
 
 **2. Bump the version in two places** — they must agree, and the workflow refuses the release if the tag
-disagrees with either:
+disagrees with either. Also update the version headers in `docs/design.md` and
+`docs/zh-CN/design.md`, and the current-status summaries in both READMEs:
 
 ```bash
 # pyproject.toml:  version = "X.Y.Z"