unstract-cli 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- unstract_cli-0.1.0/.claude/skills/bump-client-pins/SKILL.md +108 -0
- unstract_cli-0.1.0/.github/workflows/ci.yml +74 -0
- unstract_cli-0.1.0/.github/workflows/release.yml +226 -0
- unstract_cli-0.1.0/.gitignore +10 -0
- unstract_cli-0.1.0/LICENSE +21 -0
- unstract_cli-0.1.0/PKG-INFO +194 -0
- unstract_cli-0.1.0/README.md +161 -0
- unstract_cli-0.1.0/install.sh +42 -0
- unstract_cli-0.1.0/pyproject.toml +80 -0
- unstract_cli-0.1.0/src/unstract_cli/__init__.py +3 -0
- unstract_cli-0.1.0/src/unstract_cli/__main__.py +100 -0
- unstract_cli-0.1.0/src/unstract_cli/app.py +321 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/__init__.py +0 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/clone_cmd.py +297 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/common.py +114 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/config_cmd.py +551 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/docstudio_cmd.py +298 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/platform_cmd.py +692 -0
- unstract_cli-0.1.0/src/unstract_cli/commands/whisper_cmd.py +419 -0
- unstract_cli-0.1.0/src/unstract_cli/config.py +747 -0
- unstract_cli-0.1.0/src/unstract_cli/core/__init__.py +0 -0
- unstract_cli-0.1.0/src/unstract_cli/core/clients.py +417 -0
- unstract_cli-0.1.0/src/unstract_cli/core/discover.py +189 -0
- unstract_cli-0.1.0/src/unstract_cli/core/errors.py +443 -0
- unstract_cli-0.1.0/src/unstract_cli/core/output.py +431 -0
- unstract_cli-0.1.0/src/unstract_cli/core/overlay.py +50 -0
- unstract_cli-0.1.0/src/unstract_cli/core/params.py +518 -0
- unstract_cli-0.1.0/src/unstract_cli/core/platform.py +102 -0
- unstract_cli-0.1.0/src/unstract_cli/core/poll.py +432 -0
- unstract_cli-0.1.0/src/unstract_cli/overlay.toml +7 -0
- unstract_cli-0.1.0/src/unstract_cli/specs/README.md +28 -0
- unstract_cli-0.1.0/src/unstract_cli/specs/docstudio.json +1077 -0
- unstract_cli-0.1.0/src/unstract_cli/specs/llmwhisperer.json +2709 -0
- unstract_cli-0.1.0/src/unstract_cli/specs/provenance.json +16 -0
- unstract_cli-0.1.0/tests/__init__.py +0 -0
- unstract_cli-0.1.0/tests/conftest.py +81 -0
- unstract_cli-0.1.0/tests/derived_flags.json +500 -0
- unstract_cli-0.1.0/tests/test_cli.py +412 -0
- unstract_cli-0.1.0/tests/test_clients.py +227 -0
- unstract_cli-0.1.0/tests/test_commands.py +3784 -0
- unstract_cli-0.1.0/tests/test_config.py +642 -0
- unstract_cli-0.1.0/tests/test_contract.py +154 -0
- unstract_cli-0.1.0/tests/test_discover.py +553 -0
- unstract_cli-0.1.0/tests/test_errors.py +245 -0
- unstract_cli-0.1.0/tests/test_output.py +229 -0
- unstract_cli-0.1.0/tests/test_params.py +313 -0
- unstract_cli-0.1.0/tests/test_platform.py +143 -0
- unstract_cli-0.1.0/tests/test_poll.py +649 -0
- unstract_cli-0.1.0/tests/test_specs.py +44 -0
- unstract_cli-0.1.0/tests/test_workflows.py +168 -0
- unstract_cli-0.1.0/uv.lock +433 -0
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: bump-client-pins
|
|
3
|
+
description: Bump the exact `unstract-client` / `llmwhisperer-client` pins, re-sync the vendored specs to match, and cut a CLI release. Use whenever a new version of either client is released, when `tests/test_specs.py` or `tests/test_contract.py` fails, when the CLI is missing a flag for an endpoint the API already has, or when someone asks to "bump the client", "update the pins", "refresh the specs", or "release the CLI". Reach for this even when the request sounds like a plain dependency bump — the pins, the vendored specs and `provenance.json` have to move together or the CLI derives flags the pinned client cannot carry.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Bumping the client pins
|
|
7
|
+
|
|
8
|
+
The CLI derives its flags and help text from two published clients and from
|
|
9
|
+
vendored copies of the specs those clients were generated from. That makes a pin
|
|
10
|
+
bump three coupled edits, not one: the pin, the spec, and the provenance record.
|
|
11
|
+
Move one without the others and the tests say so — which is the point of them.
|
|
12
|
+
|
|
13
|
+
## The pieces
|
|
14
|
+
|
|
15
|
+
| Thing | Where |
|
|
16
|
+
|---|---|
|
|
17
|
+
| Exact pins | `pyproject.toml`, `[project].dependencies` |
|
|
18
|
+
| Vendored specs | `src/unstract_cli/specs/{docstudio,llmwhisperer}.json` |
|
|
19
|
+
| Provenance | `src/unstract_cli/specs/provenance.json` (client pin, upstream repo, commit, sha256) |
|
|
20
|
+
| Coherence tests | `tests/test_specs.py`, `tests/test_contract.py`, `tests/derived_flags.json` |
|
|
21
|
+
| Release | `.github/workflows/release.yml`, `workflow_dispatch` |
|
|
22
|
+
|
|
23
|
+
`src/unstract_cli/specs/README.md` explains the vendoring rule in place; read it
|
|
24
|
+
if any of the below is unclear.
|
|
25
|
+
|
|
26
|
+
## The sequence
|
|
27
|
+
|
|
28
|
+
1. **Move the pins** in `pyproject.toml` to the released versions you are
|
|
29
|
+
upgrading to. They are exact (`==`) on purpose: the CLI's published surface is
|
|
30
|
+
derived from these clients, so a client that moves reshapes the CLI.
|
|
31
|
+
|
|
32
|
+
2. **Relock:** `uv lock` then `uv sync --extra dev --python 3.12`. CI installs
|
|
33
|
+
from `uv.lock`, not from a fresh resolve, so a lockfile left behind means the
|
|
34
|
+
gate tests a dependency set nobody ships.
|
|
35
|
+
|
|
36
|
+
3. **Re-sync each vendored spec from the commit the pinned client was generated
|
|
37
|
+
from.** The chain is: the client repo's release tag → its `tools/gen_sdk.sh`,
|
|
38
|
+
which records the upstream service repo, path and revision the spec was copied
|
|
39
|
+
from → the spec file committed in that client at that tag. Copy that file here
|
|
40
|
+
byte-for-byte. Copying from anywhere else — upstream `main`, a newer service
|
|
41
|
+
commit — is what `tests/test_contract.py` guards: a spec parameter the pinned
|
|
42
|
+
client has no argument for cannot become a flag.
|
|
43
|
+
|
|
44
|
+
4. **Update `provenance.json`** for each spec you moved. Set `client` to the
|
|
45
|
+
exact pin you wrote in `pyproject.toml` (`unstract-client==X.Y.Z`); the
|
|
46
|
+
tests compare the two, so a pin moved without its spec fails here. Check
|
|
47
|
+
`repo`, `path` and `commit` against what that client's `tools/gen_sdk.sh`
|
|
48
|
+
records, because an upstream that moved its spec file leaves `repo` and
|
|
49
|
+
`path` stale and the tests cannot see it: they check the pin, the `sha256`
|
|
50
|
+
and the entry names, nothing about where the file came from. The `sha256`
|
|
51
|
+
is of the file you just wrote (`sha256sum src/unstract_cli/specs/<file>`).
|
|
52
|
+
This record is what lets the next person tell a current copy from a stale
|
|
53
|
+
one.
|
|
54
|
+
|
|
55
|
+
5. **Run the tests:** `uv run pytest -q`.
|
|
56
|
+
|
|
57
|
+
- `test_specs.py` fails if a vendored file stops matching its recorded
|
|
58
|
+
sha256, if its `client` no longer equals the pin in `pyproject.toml`, or if
|
|
59
|
+
a spec has no provenance entry. It is the cheap check that steps 1, 3 and
|
|
60
|
+
4 actually agree.
|
|
61
|
+
- `test_contract.py` fails if a spec parameter the pinned client cannot accept
|
|
62
|
+
would have become a flag, and separately if the derived flags stop matching
|
|
63
|
+
`tests/derived_flags.json`.
|
|
64
|
+
|
|
65
|
+
6. **If `derived_flags.json` fails, read the difference before refreshing it.**
|
|
66
|
+
The failure names the flags that moved. A flag missing from the new set is a
|
|
67
|
+
flag the CLI has stopped offering; a narrowed choice or changed type is a value
|
|
68
|
+
the CLI used to take and now rejects. Once you have decided the change is
|
|
69
|
+
intended, refresh it deliberately:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
UNSTRACT_CLI_REFRESH_FLAG_SNAPSHOT=1 uv run pytest -q tests/test_contract.py
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
and commit the snapshot in the same PR, so the diff shows what the CLI's
|
|
76
|
+
surface gained or lost.
|
|
77
|
+
|
|
78
|
+
7. **Lint:** `uv run ruff check . && uv run ruff format --check .` — the release
|
|
79
|
+
run repeats exactly this, so a failure here is a failure there.
|
|
80
|
+
|
|
81
|
+
## Versioning and release
|
|
82
|
+
|
|
83
|
+
Choose the bump by what changed for CLI users: **minor** for new or changed
|
|
84
|
+
flags and commands, **patch** for fixes that leave the surface identical.
|
|
85
|
+
|
|
86
|
+
Do not bump `__version__` in `src/unstract_cli/__init__.py` in your PR. The
|
|
87
|
+
in-repo value names the last released version; `release.yml` reads it, applies
|
|
88
|
+
the bump chosen at dispatch and commits the result itself.
|
|
89
|
+
|
|
90
|
+
Release by dispatching **Release Tag and Publish Package** on `main`:
|
|
91
|
+
|
|
92
|
+
- `version_bump: none` publishes the version already in the repo — what the
|
|
93
|
+
first release of a version needs.
|
|
94
|
+
- `pre_release: true` publishes `<next-version>rcN` and deliberately leaves the
|
|
95
|
+
committed version alone, counting N up from the rc tags already published for
|
|
96
|
+
that target. To promote to stable, dispatch again with it off **and the same
|
|
97
|
+
`version_bump`**: the workflow recomputes the target from that input every
|
|
98
|
+
time, so a different bump publishes a different version than the one the rc
|
|
99
|
+
tested.
|
|
100
|
+
- It publishes to PyPI **before** it tags and releases, because publishing is the
|
|
101
|
+
only step that cannot be undone: a failure before it leaves nothing to
|
|
102
|
+
unpublish, and one after it is retried by hand against a live artifact.
|
|
103
|
+
|
|
104
|
+
## Upstream
|
|
105
|
+
|
|
106
|
+
If a client pin is missing an endpoint the service already offers, the fix is in
|
|
107
|
+
that client, not here — see the `spec-upgrade` skill in `unstract-python-client`
|
|
108
|
+
and `llm-whisperer-python-client`. Bump the pin here once it is released.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
name: ci
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
pull_request:
|
|
5
|
+
push:
|
|
6
|
+
branches: [main]
|
|
7
|
+
|
|
8
|
+
jobs:
|
|
9
|
+
# Offline by design: no network, no credentials, sub-second. Live round trips
|
|
10
|
+
# are a manual pre-release step, not a per-PR gate.
|
|
11
|
+
test:
|
|
12
|
+
runs-on: ubuntu-latest
|
|
13
|
+
steps:
|
|
14
|
+
- uses: actions/checkout@v4
|
|
15
|
+
- uses: astral-sh/setup-uv@v6
|
|
16
|
+
with:
|
|
17
|
+
version: "0.6.14"
|
|
18
|
+
enable-cache: true
|
|
19
|
+
# Synced from uv.lock rather than resolved fresh, so the gate tests the
|
|
20
|
+
# dependency set an install actually gets.
|
|
21
|
+
- run: uv sync --extra dev --python 3.12
|
|
22
|
+
- run: uv run ruff check .
|
|
23
|
+
- run: uv run ruff format --check .
|
|
24
|
+
- run: uv run pytest -q
|
|
25
|
+
# `uv run` resolves from uv.lock, whose click sits in the middle of the
|
|
26
|
+
# range the pin allows. Both ends are what an install in the wild gets --
|
|
27
|
+
# the floor exactly, not the newest release that satisfies it -- and
|
|
28
|
+
# discovery is a published contract read out of Click's own objects, so
|
|
29
|
+
# both ends are run and their answers compared.
|
|
30
|
+
- run: uv pip install 'click==8.1.0'
|
|
31
|
+
- run: .venv/bin/python -m pytest -q
|
|
32
|
+
- run: .venv/bin/python -m unstract_cli -o json --discover full > floor.json
|
|
33
|
+
- run: uv pip install -U 'click>=8.1,<9'
|
|
34
|
+
- run: .venv/bin/python -m pytest -q
|
|
35
|
+
- run: .venv/bin/python -m unstract_cli -o json --discover full > latest.json
|
|
36
|
+
- run: diff floor.json latest.json
|
|
37
|
+
|
|
38
|
+
# The wheel rather than the checkout: packaging and entry points are declared
|
|
39
|
+
# in `pyproject.toml` and are not exercised by running the tests in place.
|
|
40
|
+
package:
|
|
41
|
+
runs-on: ubuntu-latest
|
|
42
|
+
env:
|
|
43
|
+
VIRTUAL_ENV: /tmp/smoke
|
|
44
|
+
steps:
|
|
45
|
+
- uses: actions/checkout@v4
|
|
46
|
+
- uses: astral-sh/setup-uv@v6
|
|
47
|
+
with:
|
|
48
|
+
version: "0.6.14"
|
|
49
|
+
enable-cache: true
|
|
50
|
+
- run: uv build --out-dir dist
|
|
51
|
+
- run: uv venv --python 3.12 /tmp/smoke
|
|
52
|
+
- run: uv pip install dist/*.whl
|
|
53
|
+
- name: Both entry points answer
|
|
54
|
+
run: |
|
|
55
|
+
/tmp/smoke/bin/unstract --version
|
|
56
|
+
/tmp/smoke/bin/unstract-cli --version
|
|
57
|
+
/tmp/smoke/bin/unstract --help > /dev/null
|
|
58
|
+
- name: The published surfaces render from an installed wheel
|
|
59
|
+
run: |
|
|
60
|
+
/tmp/smoke/bin/unstract -o json --discover full > discover.json
|
|
61
|
+
python -c "import json; d = json.load(open('discover.json')); assert d['ok'] and d['data']['commands']"
|
|
62
|
+
# A usage error still renders the envelope and still exits 2.
|
|
63
|
+
set +e
|
|
64
|
+
/tmp/smoke/bin/unstract -o json whisper status > usage.json
|
|
65
|
+
code=$?
|
|
66
|
+
set -e
|
|
67
|
+
test "$code" = "2"
|
|
68
|
+
python -c "import json; d = json.load(open('usage.json')); assert d['ok'] is False and d['error']['exit_code'] == 2"
|
|
69
|
+
- name: A fresh install can write its own config
|
|
70
|
+
env:
|
|
71
|
+
UNSTRACT_CONFIG: /tmp/smoke-config.toml
|
|
72
|
+
run: |
|
|
73
|
+
/tmp/smoke/bin/unstract config init < /dev/null
|
|
74
|
+
test -s /tmp/smoke-config.toml
|
|
@@ -0,0 +1,226 @@
|
|
|
1
|
+
name: Release Tag and Publish Package
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
inputs:
|
|
6
|
+
version_bump:
|
|
7
|
+
description: "Version bump type."
|
|
8
|
+
required: true
|
|
9
|
+
default: "patch"
|
|
10
|
+
type: choice
|
|
11
|
+
options:
|
|
12
|
+
- patch
|
|
13
|
+
- minor
|
|
14
|
+
- major
|
|
15
|
+
pre_release:
|
|
16
|
+
description: "Publish a release candidate (`<version>rcN`) instead of the version itself. Dispatch again with the same bump and this off to promote it to stable."
|
|
17
|
+
required: false
|
|
18
|
+
default: false
|
|
19
|
+
type: boolean
|
|
20
|
+
release_notes:
|
|
21
|
+
description: "Release notes (optional)"
|
|
22
|
+
required: false
|
|
23
|
+
type: string
|
|
24
|
+
|
|
25
|
+
concurrency:
|
|
26
|
+
group: release
|
|
27
|
+
cancel-in-progress: false
|
|
28
|
+
|
|
29
|
+
jobs:
|
|
30
|
+
release-and-publish:
|
|
31
|
+
runs-on: ubuntu-latest
|
|
32
|
+
# Dispatchable from any branch by anyone who can dispatch it, and a
|
|
33
|
+
# candidate reaches PyPI without a commit to review: the published artefact
|
|
34
|
+
# must come from the reviewed branch.
|
|
35
|
+
if: github.ref == 'refs/heads/main'
|
|
36
|
+
permissions:
|
|
37
|
+
contents: write
|
|
38
|
+
# Publishing is by PyPI Trusted Publisher, so there is no API token.
|
|
39
|
+
id-token: write
|
|
40
|
+
steps:
|
|
41
|
+
# Checked before the checkout, so a missing one fails on the reason
|
|
42
|
+
# rather than on a token that renders empty three steps later.
|
|
43
|
+
- name: Check the release credentials are visible to this repository
|
|
44
|
+
env:
|
|
45
|
+
APP_CLIENT_ID: ${{ vars.PUSH_TO_MAIN_APP_CLIENT_ID }}
|
|
46
|
+
APP_PRIVATE_KEY: ${{ secrets.PUSH_TO_MAIN_APP_PRIVATE_KEY }}
|
|
47
|
+
run: |
|
|
48
|
+
missing=""
|
|
49
|
+
[ -n "$APP_CLIENT_ID" ] || missing="$missing vars.PUSH_TO_MAIN_APP_CLIENT_ID"
|
|
50
|
+
[ -n "$APP_PRIVATE_KEY" ] || missing="$missing secrets.PUSH_TO_MAIN_APP_PRIVATE_KEY"
|
|
51
|
+
if [ -n "$missing" ]; then
|
|
52
|
+
echo "Missing:$missing" >&2
|
|
53
|
+
echo "An organisation-level variable or secret also has to list this repository." >&2
|
|
54
|
+
exit 1
|
|
55
|
+
fi
|
|
56
|
+
|
|
57
|
+
- name: Generate GitHub App Token
|
|
58
|
+
id: generate-token
|
|
59
|
+
uses: actions/create-github-app-token@v3
|
|
60
|
+
with:
|
|
61
|
+
client-id: ${{ vars.PUSH_TO_MAIN_APP_CLIENT_ID }}
|
|
62
|
+
private-key: ${{ secrets.PUSH_TO_MAIN_APP_PRIVATE_KEY }}
|
|
63
|
+
owner: Zipstack
|
|
64
|
+
repositories: |
|
|
65
|
+
unstract-cli
|
|
66
|
+
|
|
67
|
+
- uses: actions/checkout@v4
|
|
68
|
+
with:
|
|
69
|
+
token: ${{ steps.generate-token.outputs.token }}
|
|
70
|
+
fetch-depth: 0
|
|
71
|
+
|
|
72
|
+
- name: Configure Git
|
|
73
|
+
run: |
|
|
74
|
+
git config --global user.name "github-actions[bot]"
|
|
75
|
+
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
|
76
|
+
|
|
77
|
+
- uses: actions/setup-python@v5
|
|
78
|
+
with:
|
|
79
|
+
python-version: "3.12"
|
|
80
|
+
|
|
81
|
+
- uses: astral-sh/setup-uv@v6
|
|
82
|
+
with:
|
|
83
|
+
version: "0.6.14"
|
|
84
|
+
enable-cache: true
|
|
85
|
+
|
|
86
|
+
# The same install as ci.yml, so what the release run lints and tests is
|
|
87
|
+
# what the PR gate lints and tests.
|
|
88
|
+
- run: uv sync --extra dev --python 3.12
|
|
89
|
+
|
|
90
|
+
# Staged locally only: nothing is committed, tagged or released until the
|
|
91
|
+
# checks and the build have passed, so a failure leaves main untouched.
|
|
92
|
+
- name: Compute new version
|
|
93
|
+
id: version
|
|
94
|
+
run: |
|
|
95
|
+
VERSION_FILE=src/unstract_cli/__init__.py
|
|
96
|
+
CURRENT_VERSION=$(sed -nE 's/^__version__ = "(.*)"/\1/p' "$VERSION_FILE")
|
|
97
|
+
echo "Current version: $CURRENT_VERSION"
|
|
98
|
+
|
|
99
|
+
IFS='.' read -r MAJOR MINOR PATCH <<< "$CURRENT_VERSION"
|
|
100
|
+
case "${{ github.event.inputs.version_bump }}" in
|
|
101
|
+
major) MAJOR=$((MAJOR + 1)); MINOR=0; PATCH=0 ;;
|
|
102
|
+
minor) MINOR=$((MINOR + 1)); PATCH=0 ;;
|
|
103
|
+
patch) PATCH=$((PATCH + 1)) ;;
|
|
104
|
+
esac
|
|
105
|
+
NEXT_VERSION="$MAJOR.$MINOR.$PATCH"
|
|
106
|
+
|
|
107
|
+
# A pre-release is a candidate for NEXT_VERSION, not a version of its
|
|
108
|
+
# own, so it never moves the committed one: the file keeps naming the
|
|
109
|
+
# last stable release, and repeat dispatches count up from the rc tags
|
|
110
|
+
# already published for that target.
|
|
111
|
+
if [ "${{ github.event.inputs.pre_release }}" = "true" ]; then
|
|
112
|
+
HIGHEST_RC=$(git tag -l "v${NEXT_VERSION}rc*" \
|
|
113
|
+
| sed -nE "s/^v${NEXT_VERSION}rc([0-9]+)$/\1/p" | sort -n | tail -1)
|
|
114
|
+
NEW_VERSION="${NEXT_VERSION}rc$(( ${HIGHEST_RC:-0} + 1 ))"
|
|
115
|
+
else
|
|
116
|
+
NEW_VERSION="$NEXT_VERSION"
|
|
117
|
+
fi
|
|
118
|
+
|
|
119
|
+
echo "New version: $NEW_VERSION"
|
|
120
|
+
echo "version=$NEW_VERSION" >> "$GITHUB_OUTPUT"
|
|
121
|
+
|
|
122
|
+
sed -i "s/^__version__ = \".*\"/__version__ = \"$NEW_VERSION\"/" "$VERSION_FILE"
|
|
123
|
+
|
|
124
|
+
if git rev-parse -q --verify "refs/tags/v$NEW_VERSION" >/dev/null; then
|
|
125
|
+
echo "Tag v$NEW_VERSION already exists. Exiting..."
|
|
126
|
+
exit 1
|
|
127
|
+
fi
|
|
128
|
+
|
|
129
|
+
- name: Verify version update
|
|
130
|
+
run: |
|
|
131
|
+
BUILT_VERSION=$(uv run python -c "import unstract_cli; print(unstract_cli.__version__)")
|
|
132
|
+
echo "Package version: $BUILT_VERSION"
|
|
133
|
+
echo "Target version: ${{ steps.version.outputs.version }}"
|
|
134
|
+
if [ "$BUILT_VERSION" != "${{ steps.version.outputs.version }}" ]; then
|
|
135
|
+
echo "Version mismatch! Exiting..."
|
|
136
|
+
exit 1
|
|
137
|
+
fi
|
|
138
|
+
|
|
139
|
+
- name: Run linting
|
|
140
|
+
run: |
|
|
141
|
+
uv run ruff check .
|
|
142
|
+
uv run ruff format --check .
|
|
143
|
+
|
|
144
|
+
- name: Run tests
|
|
145
|
+
run: uv run pytest -q
|
|
146
|
+
|
|
147
|
+
- name: Build package
|
|
148
|
+
run: uv build
|
|
149
|
+
|
|
150
|
+
# Publishing is the one step that cannot be undone, so it runs last, and
|
|
151
|
+
# the release is created as a draft so a publish that fails leaves
|
|
152
|
+
# nothing public. Until then the bump commit exists only in the tag, so
|
|
153
|
+
# a failed publish leaves main untouched.
|
|
154
|
+
- name: Commit version bump and create draft release
|
|
155
|
+
env:
|
|
156
|
+
RELEASE_NOTES: ${{ github.event.inputs.release_notes }}
|
|
157
|
+
GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
|
|
158
|
+
run: |
|
|
159
|
+
NEW_VERSION="${{ steps.version.outputs.version }}"
|
|
160
|
+
|
|
161
|
+
# A pre-release leaves the committed version alone.
|
|
162
|
+
if [ "${{ github.event.inputs.pre_release }}" = "true" ]; then
|
|
163
|
+
git checkout -- src/unstract_cli/__init__.py
|
|
164
|
+
elif ! git diff --quiet; then
|
|
165
|
+
git add src/unstract_cli/__init__.py
|
|
166
|
+
git commit -m "chore: bump version to $NEW_VERSION [skip ci]"
|
|
167
|
+
fi
|
|
168
|
+
|
|
169
|
+
git tag "v$NEW_VERSION"
|
|
170
|
+
git push origin "v$NEW_VERSION"
|
|
171
|
+
|
|
172
|
+
if [ -z "$RELEASE_NOTES" ]; then
|
|
173
|
+
gh release create "v$NEW_VERSION" \
|
|
174
|
+
--title "Release v$NEW_VERSION" \
|
|
175
|
+
--generate-notes \
|
|
176
|
+
--draft \
|
|
177
|
+
${{ github.event.inputs.pre_release == 'true' && '--prerelease' || '' }}
|
|
178
|
+
else
|
|
179
|
+
gh release create "v$NEW_VERSION" \
|
|
180
|
+
--title "Release v$NEW_VERSION" \
|
|
181
|
+
--notes "$RELEASE_NOTES" \
|
|
182
|
+
--generate-notes \
|
|
183
|
+
--draft \
|
|
184
|
+
${{ github.event.inputs.pre_release == 'true' && '--prerelease' || '' }}
|
|
185
|
+
fi
|
|
186
|
+
|
|
187
|
+
echo "Created draft release v$NEW_VERSION"
|
|
188
|
+
|
|
189
|
+
- name: Publish to PyPI
|
|
190
|
+
run: uv publish
|
|
191
|
+
|
|
192
|
+
# Everything the run created before the publish, so a rerun after the
|
|
193
|
+
# cause is fixed starts clean rather than on the tag guard. Each removal
|
|
194
|
+
# tolerates its target being absent: the failure may have come partway
|
|
195
|
+
# through creating them.
|
|
196
|
+
- name: Remove the tag and draft release a failed publish leaves behind
|
|
197
|
+
if: failure() && steps.version.outcome == 'success'
|
|
198
|
+
env:
|
|
199
|
+
GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
|
|
200
|
+
run: |
|
|
201
|
+
NEW_VERSION="${{ steps.version.outputs.version }}"
|
|
202
|
+
# The wheel and the sdist upload separately, so a failed publish can
|
|
203
|
+
# still have put the version on PyPI; then the tag has to stay.
|
|
204
|
+
if curl -fsS -o /dev/null "https://pypi.org/pypi/unstract-cli/$NEW_VERSION/json"; then
|
|
205
|
+
echo "PyPI serves $NEW_VERSION; keeping the tag and draft release." >&2
|
|
206
|
+
exit 0
|
|
207
|
+
fi
|
|
208
|
+
gh release delete "v$NEW_VERSION" --yes || true
|
|
209
|
+
git push origin ":refs/tags/v$NEW_VERSION" || true
|
|
210
|
+
|
|
211
|
+
# If this fails the published version is on PyPI and in the tag but not
|
|
212
|
+
# on main; `git push origin v<version>:main` finishes the job by hand.
|
|
213
|
+
- name: Push version bump
|
|
214
|
+
if: github.event.inputs.pre_release != 'true'
|
|
215
|
+
run: git push origin main
|
|
216
|
+
|
|
217
|
+
- name: Publish release
|
|
218
|
+
env:
|
|
219
|
+
GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
|
|
220
|
+
run: gh release edit "v${{ steps.version.outputs.version }}" --draft=false
|
|
221
|
+
|
|
222
|
+
- name: Success message
|
|
223
|
+
run: |
|
|
224
|
+
echo "Published ${{ steps.version.outputs.version }} to PyPI with uv publish using Trusted Publishers"
|
|
225
|
+
echo "Release: https://github.com/${{ github.repository }}/releases/tag/v${{ steps.version.outputs.version }}"
|
|
226
|
+
echo "PyPI: https://pypi.org/project/unstract-cli/${{ steps.version.outputs.version }}/"
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Zipstack
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: unstract-cli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Unified, LLM-friendly CLI for the Unstract suite of products
|
|
5
|
+
Project-URL: Homepage, https://unstract.com
|
|
6
|
+
Project-URL: Documentation, https://docs.unstract.com/unstract/unstract_platform/cli/unstract_cli/
|
|
7
|
+
Project-URL: Repository, https://github.com/Zipstack/unstract-cli
|
|
8
|
+
Project-URL: Issues, https://github.com/Zipstack/unstract-cli/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/Zipstack/unstract-cli/releases
|
|
10
|
+
License: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Environment :: Console
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
16
|
+
Classifier: Operating System :: MacOS :: MacOS X
|
|
17
|
+
Classifier: Operating System :: Microsoft :: Windows
|
|
18
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
19
|
+
Classifier: Programming Language :: Python
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Topic :: Utilities
|
|
22
|
+
Requires-Python: >=3.12
|
|
23
|
+
Requires-Dist: click<9,>=8.1
|
|
24
|
+
Requires-Dist: llmwhisperer-client==2.9.0
|
|
25
|
+
Requires-Dist: requests>=2.32.3
|
|
26
|
+
Requires-Dist: tomli-w>=1.0
|
|
27
|
+
Requires-Dist: unstract-client==1.7.0
|
|
28
|
+
Provides-Extra: dev
|
|
29
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
30
|
+
Requires-Dist: pyyaml>=6.0; extra == 'dev'
|
|
31
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
32
|
+
Description-Content-Type: text/markdown
|
|
33
|
+
|
|
34
|
+
# unstract-cli
|
|
35
|
+
[](https://pypi.org/project/unstract-cli/)
|
|
36
|
+
[
|
|
37
|
+
](https://pypi.org/project/unstract-cli/)
|
|
38
|
+
[](https://pypi.org/project/unstract-cli/)
|
|
39
|
+
|
|
40
|
+
`unstract` runs [LLMWhisperer](https://docs.unstract.com/llmwhisperer/) text
|
|
41
|
+
extraction and [Unstract](https://docs.unstract.com/unstract/) API deployments
|
|
42
|
+
from the terminal. Pass `-o json` and every command prints one JSON envelope, so a
|
|
43
|
+
shell script or a coding agent can drive it.
|
|
44
|
+
|
|
45
|
+
Full reference: <https://docs.unstract.com/unstract/unstract_platform/cli/unstract_cli/>
|
|
46
|
+
|
|
47
|
+
## Install
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
curl -LsSf https://raw.githubusercontent.com/Zipstack/unstract-cli/main/install.sh | sh
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
The installer fetches [`uv`](https://docs.astral.sh/uv/) if it is missing and
|
|
54
|
+
installs the CLI with it; `uv` brings its own Python. With `uv` or `pip`
|
|
55
|
+
already there:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
uv tool install unstract-cli # or: pip install unstract-cli
|
|
59
|
+
unstract --version
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Get your keys
|
|
63
|
+
|
|
64
|
+
| Key | Where it is minted | What it does |
|
|
65
|
+
| --- | --- | --- |
|
|
66
|
+
| **Platform key** | An organisation admin, under **Settings → Platform API Keys** in the Unstract UI | Platform related operations and to identify the organization |
|
|
67
|
+
| **Deployment key** | The API deployment's own page in the Unstract UI or an organisation admin mints one under **Settings → Global API Deployment Keys** | Runs deployments (`deployment run`, `deployment status`) |
|
|
68
|
+
| **LLMWhisperer key** | The LLMWhisperer console | Extracts text (`whisper …`) |
|
|
69
|
+
|
|
70
|
+
## Set up
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
unstract auth login
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
A wizard asks for each product's URL (Enter keeps the cloud host; self-hosted,
|
|
77
|
+
type your own) and API keys, then writes `~/.unstract/config.toml`. Then verify:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
unstract auth whoami # which organisation the platform key belongs to
|
|
81
|
+
unstract config doctor --probe # where each setting resolved from, keys checked
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Usage
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
# Extract text from a document (path or URL); waits for the result
|
|
88
|
+
unstract whisper extract invoice.pdf -o raw > invoice.txt
|
|
89
|
+
|
|
90
|
+
# What deployments can I run?
|
|
91
|
+
unstract docstudio deployment ls
|
|
92
|
+
|
|
93
|
+
# Run one and wait for the structured result
|
|
94
|
+
unstract docstudio deployment run invoice-parser invoice.pdf
|
|
95
|
+
|
|
96
|
+
# Long job: submit, then check later
|
|
97
|
+
unstract docstudio deployment run invoice-parser invoice.pdf --no-wait
|
|
98
|
+
unstract docstudio deployment status invoice-parser <execution_id>
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
`--help` on any command lists its options; `unstract --discover full` prints
|
|
102
|
+
the whole command tree, every flag and the exit-code table as JSON.
|
|
103
|
+
|
|
104
|
+
## Configuration
|
|
105
|
+
|
|
106
|
+
`~/.unstract/config.toml`, or `$UNSTRACT_CONFIG`, or `--config`, or a
|
|
107
|
+
project-local `.unstract.toml` is found by upward search. Here's an example config that uses environment variables for the API keys.
|
|
108
|
+
|
|
109
|
+
```toml
|
|
110
|
+
default_profile = "cloud-us"
|
|
111
|
+
|
|
112
|
+
[profiles.cloud-us.docstudio]
|
|
113
|
+
base_url = "https://us-central.unstract.com"
|
|
114
|
+
org_id = "org_ABC123"
|
|
115
|
+
platform_key = "env:UNSTRACT_PLATFORM_KEY"
|
|
116
|
+
api_key = "env:UNSTRACT_DEPLOYMENT_KEY"
|
|
117
|
+
|
|
118
|
+
[profiles.cloud-us.llmwhisperer]
|
|
119
|
+
base_url = "https://llmwhisperer-api.us-central.unstract.com/api/v2"
|
|
120
|
+
api_key = "env:LLMWHISPERER_API_KEY"
|
|
121
|
+
|
|
122
|
+
# Only for a deployment whose key differs from the profile's.
|
|
123
|
+
[profiles.cloud-us.deployments."invoice-parser"]
|
|
124
|
+
api_key = "env:INVOICE_PARSER_KEY"
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
Every setting resolves **flag > environment > profile > built-in default**.
|
|
128
|
+
`auth login` writes keys literally; `env:VAR_NAME` keeps them out of the file.
|
|
129
|
+
The connection flags on each group (`--base-url`, `--api-key`, `--org-id`,
|
|
130
|
+
`--platform-key`) override the profile for one invocation without writing
|
|
131
|
+
anything. `config doctor` reports where each setting resolved from without
|
|
132
|
+
echoing a value, and exits non-zero when one of its checks fails.
|
|
133
|
+
|
|
134
|
+
### Environment variables
|
|
135
|
+
|
|
136
|
+
The same settings without a file, for CI, containers and agents:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
export UNSTRACT_PLATFORM_KEY=... # auth whoami, deployment ls
|
|
140
|
+
export UNSTRACT_DEPLOYMENT_KEY=... # deployment run / status
|
|
141
|
+
export UNSTRACT_ORG_ID=... # the organisation id auth whoami reports
|
|
142
|
+
export LLMWHISPERER_API_KEY=... # whisper …
|
|
143
|
+
export UNSTRACT_BASE_URL=... # self-hosted only
|
|
144
|
+
export LLMWHISPERER_BASE_URL=... # self-hosted only
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
`auth login` also takes each key as a flag (`--platform-key`, `--deployment-key`,
|
|
148
|
+
`--llmwhisperer-key`; `-` reads it from stdin) and the host as `--base-url`, so
|
|
149
|
+
it runs without a terminal too.
|
|
150
|
+
|
|
151
|
+
## Output for scripts and agents
|
|
152
|
+
|
|
153
|
+
**Parsing anything? Pass `-o json`.** stdout then carries exactly one envelope,
|
|
154
|
+
on success and on failure alike, and diagnostics go to stderr:
|
|
155
|
+
|
|
156
|
+
```json
|
|
157
|
+
{"ok": true, "data": {...}, "error": null, "meta": {"contract_version": 1}}
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Ignore fields you do not recognise; refuse a `meta.contract_version` above the
|
|
161
|
+
one you were written against. `-o raw` prints one field unwrapped. When a
|
|
162
|
+
coding agent is driving (detected from the environment it sets) json is the
|
|
163
|
+
default; `--agent yes|no` forces that, and an explicit `-o` wins over both.
|
|
164
|
+
|
|
165
|
+
Failures exit non-zero with a stable code:
|
|
166
|
+
|
|
167
|
+
| Code | Meaning |
|
|
168
|
+
|------|---------|
|
|
169
|
+
| 0 | success |
|
|
170
|
+
| 1 | generic failure |
|
|
171
|
+
| 2 | usage error |
|
|
172
|
+
| 3 | authentication failed |
|
|
173
|
+
| 4 | not found |
|
|
174
|
+
| 5 | validation failed, including a run in which a document failed |
|
|
175
|
+
| 6 | rate limited |
|
|
176
|
+
| 7 | timed out; resume with the job handle in the error payload, do not resubmit |
|
|
177
|
+
| 8 | server error |
|
|
178
|
+
| 9 | result already consumed (one-shot read); pass `--save` next time |
|
|
179
|
+
| 10 | result read but not saved; it is in `error.details` |
|
|
180
|
+
| 130 | interrupted (128 + SIGINT) — the user stopped it, not a failure |
|
|
181
|
+
|
|
182
|
+
## Development
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
uv sync --extra dev
|
|
186
|
+
uv run pytest
|
|
187
|
+
uv run ruff check .
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
## Questions and Feedback
|
|
191
|
+
|
|
192
|
+
On Slack, [join great conversations](https://join-slack.unstract.com/) around LLMs, their ecosystem and leveraging them to automate the previously unautomatable!
|
|
193
|
+
|
|
194
|
+
[Unstract Cloud](https://unstract.com/): Signup and Try!
|