@tiangong-ai/cli 0.0.1 → 0.0.4

package/AGENTS.md

@@ -15,8 +15,8 @@ checkPaths:
   - .docpact/config.yaml
   - docs/agents/**
   - src/**
-lastReviewedAt: 2026-05-08
-lastReviewedCommit: 2f4144ea41116de344f7fce1f2ef919823b09bff
+lastReviewedAt: 2026-05-10
+lastReviewedCommit: f45edb6f41dd767911b3788eb8bc75e9bcb8ba33
 ---
 
 # Tiangong AI CLI Contract
@@ -38,6 +38,10 @@ This repository owns the Tiangong AI command-line interface.
 
 - `tiangong-ai doctor`
 - `tiangong-ai kb ingest`
+- `tiangong-ai kb ingest bulk`
+- `tiangong-ai kb ingest jobs`
+- `tiangong-ai kb ingest resume`
+- `tiangong-ai kb ingest export`
 - `tiangong-ai kb collections`
 - `tiangong-ai kb status`
 
@@ -60,7 +64,7 @@ aggregated local quality gate.
 
 ## Release
 
-GitHub Actions publishes npm releases through `.github/workflows/npm-publish.yml`.
+GitHub Actions publishes npm releases through `.github/workflows/publish.yml`.
 The workflow uses npm Trusted Publishing through GitHub OIDC and runs npm lint,
 test, coverage, and pack checks before publishing.
 

package/README.md

@@ -55,12 +55,85 @@ Upload a folder recursively with local checkpointing:
 tiangong-ai kb ingest upload /path/to/folder --recursive --concurrency 3 --retries 3
 ```
 
+Bulk scan a large folder and emit a structural JSON summary:
+
+```bash
+tiangong-ai kb ingest bulk scan /path/to/folder --json
+```
+
+Dry-run a layered metadata map against a folder and collection schema:
+
+```bash
+tiangong-ai kb ingest bulk dry-run /path/to/folder \
+  --collection-path /course/thu_humanities \
+  --metadata-map metadata-map.yaml \
+  --json
+```
+
+The same dry-run is also available through the skill-facing alias:
+
+```bash
+tiangong-ai kb ingest metadata dry-run /path/to/folder \
+  --collection-path /course/thu_humanities \
+  --metadata-map metadata-map.yaml \
+  --json
+```
+
+Run a resumable sliding-window bulk ingest:
+
+```bash
+tiangong-ai kb ingest bulk /path/to/folder \
+  --collection-path /course/thu_humanities \
+  --metadata-map metadata-map.yaml \
+  --window-size 100 \
+  --top-up-max 50 \
+  --upload-concurrency 4 \
+  --poll-interval 30
+```
+
+`tiangong-ai kb ingest bulk run /path/to/folder` is accepted as an explicit
+alias for wrappers that want a verb before the folder path.
+
+Bulk ingest uses SQLite as its checkpoint source. By default, job files are
+stored under the OS app-data directory:
+
+- macOS: `~/Library/Application Support/tiangong-ai/kb-ingest/jobs/<job-id>.sqlite`
+- Linux: `~/.local/share/tiangong-ai/kb-ingest/jobs/<job-id>.sqlite`
+- Windows: `%APPDATA%/tiangong-ai/kb-ingest/jobs/<job-id>.sqlite`
+
+Use `--state /path/to/job.sqlite` to override the checkpoint path. Bulk jobs do
+not use `.tiangong-kb-ingest-manifest.jsonl` as their checkpoint.
+
+Bulk ingest always creates 300dpi-normalized ingest copies for `.docx` files
+and creates PDF split parts when a PDF exceeds the active upload limit. Derived
+files stay under `.tiangong-kb-ingest-derived` by default, and that directory is
+excluded from future bulk scans. DOCX copies keep the original logical path for
+metadata-map evaluation, and the generated DOCX copy is uploaded even when
+normalization does not materially reduce file size. PDF split parts keep the
+original logical parent directory. Upload metadata remains the user/business
+metadata produced by the metadata map.
+
+Manage bulk jobs:
+
+```bash
+tiangong-ai kb ingest jobs
+tiangong-ai kb ingest status <job-id>
+tiangong-ai kb ingest resume <job-id>
+tiangong-ai kb ingest export <job-id> --format csv
+```
+
 List uploadable collections:
 
 ```bash
 tiangong-ai kb collections list --capability upload
 ```
 
+Resolve a collection and include the effective metadata schema:
+
+```bash
+tiangong-ai kb collections schema --collection-path /course/thu_humanities --json
+```
+
 Check document status:
 
 ```bash
@@ -70,7 +143,11 @@ tiangong-ai kb ingest status <document-id>
 ## Boundary
 
 The CLI is a thin local client. It sends bearer-token requests to the Tiangong
-KB ingest API and records local manifests for batch recovery. The backend owns
+KB ingest API and records local checkpoints for batch recovery. Non-bulk upload
+still uses a JSONL manifest. Bulk ingest uses SQLite and releases sliding-window
+capacity only when document status is `completed` and both `opensearchIndexed`
+and `pineconeIndexed` are true. If the status API does not return those index
+flags yet, the file remains in `waiting_for_index_flags`. The backend owns
 authorization, collection permissions, duplicate detection, NAS raw writes,
 parse queueing, and status transitions.
 
@@ -85,7 +162,7 @@ docpact validate-config --root . --strict
 
 ## Release
 
-Publishing is handled by GitHub Actions in `.github/workflows/npm-publish.yml`.
+Publishing is handled by GitHub Actions in `.github/workflows/publish.yml`.
 Push a `v*` tag that matches `package.json` version. The workflow publishes
 `@tiangong-ai/cli` to npm through npm Trusted Publishing after lint, tests,
 coverage, version availability, and a package dry run pass.

package/dist/cli.d.ts

@@ -1,6 +1,8 @@
 export declare const DEFAULT_API_BASE_URL = "https://thuenv.tiangong.world:7300";
 export declare const DEFAULT_API_PATH_PREFIX = "/api/v1/kb";
-type Output = Pick<NodeJS.WriteStream, "write">;
+interface Output {
+    write(text: string): unknown;
+}
 export interface CliIO {
     env: NodeJS.ProcessEnv;
     stdout: Output;