unstract-cli 0.1.0rc1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. unstract_cli-0.1.0rc1/.claude/skills/bump-client-pins/SKILL.md +108 -0
  2. unstract_cli-0.1.0rc1/.github/workflows/ci.yml +74 -0
  3. unstract_cli-0.1.0rc1/.github/workflows/release.yml +226 -0
  4. unstract_cli-0.1.0rc1/.gitignore +10 -0
  5. unstract_cli-0.1.0rc1/PKG-INFO +197 -0
  6. unstract_cli-0.1.0rc1/README.md +181 -0
  7. unstract_cli-0.1.0rc1/install.sh +42 -0
  8. unstract_cli-0.1.0rc1/pyproject.toml +60 -0
  9. unstract_cli-0.1.0rc1/src/unstract_cli/__init__.py +3 -0
  10. unstract_cli-0.1.0rc1/src/unstract_cli/__main__.py +100 -0
  11. unstract_cli-0.1.0rc1/src/unstract_cli/app.py +321 -0
  12. unstract_cli-0.1.0rc1/src/unstract_cli/commands/__init__.py +0 -0
  13. unstract_cli-0.1.0rc1/src/unstract_cli/commands/clone_cmd.py +297 -0
  14. unstract_cli-0.1.0rc1/src/unstract_cli/commands/common.py +114 -0
  15. unstract_cli-0.1.0rc1/src/unstract_cli/commands/config_cmd.py +534 -0
  16. unstract_cli-0.1.0rc1/src/unstract_cli/commands/docstudio_cmd.py +298 -0
  17. unstract_cli-0.1.0rc1/src/unstract_cli/commands/platform_cmd.py +659 -0
  18. unstract_cli-0.1.0rc1/src/unstract_cli/commands/whisper_cmd.py +419 -0
  19. unstract_cli-0.1.0rc1/src/unstract_cli/config.py +745 -0
  20. unstract_cli-0.1.0rc1/src/unstract_cli/core/__init__.py +0 -0
  21. unstract_cli-0.1.0rc1/src/unstract_cli/core/clients.py +417 -0
  22. unstract_cli-0.1.0rc1/src/unstract_cli/core/discover.py +189 -0
  23. unstract_cli-0.1.0rc1/src/unstract_cli/core/errors.py +443 -0
  24. unstract_cli-0.1.0rc1/src/unstract_cli/core/output.py +431 -0
  25. unstract_cli-0.1.0rc1/src/unstract_cli/core/overlay.py +50 -0
  26. unstract_cli-0.1.0rc1/src/unstract_cli/core/params.py +518 -0
  27. unstract_cli-0.1.0rc1/src/unstract_cli/core/platform.py +102 -0
  28. unstract_cli-0.1.0rc1/src/unstract_cli/core/poll.py +432 -0
  29. unstract_cli-0.1.0rc1/src/unstract_cli/overlay.toml +7 -0
  30. unstract_cli-0.1.0rc1/src/unstract_cli/specs/README.md +28 -0
  31. unstract_cli-0.1.0rc1/src/unstract_cli/specs/docstudio.json +1077 -0
  32. unstract_cli-0.1.0rc1/src/unstract_cli/specs/llmwhisperer.json +2709 -0
  33. unstract_cli-0.1.0rc1/src/unstract_cli/specs/provenance.json +16 -0
  34. unstract_cli-0.1.0rc1/tests/__init__.py +0 -0
  35. unstract_cli-0.1.0rc1/tests/conftest.py +81 -0
  36. unstract_cli-0.1.0rc1/tests/derived_flags.json +500 -0
  37. unstract_cli-0.1.0rc1/tests/test_cli.py +401 -0
  38. unstract_cli-0.1.0rc1/tests/test_clients.py +227 -0
  39. unstract_cli-0.1.0rc1/tests/test_commands.py +3657 -0
  40. unstract_cli-0.1.0rc1/tests/test_config.py +629 -0
  41. unstract_cli-0.1.0rc1/tests/test_contract.py +154 -0
  42. unstract_cli-0.1.0rc1/tests/test_discover.py +553 -0
  43. unstract_cli-0.1.0rc1/tests/test_errors.py +245 -0
  44. unstract_cli-0.1.0rc1/tests/test_output.py +229 -0
  45. unstract_cli-0.1.0rc1/tests/test_params.py +313 -0
  46. unstract_cli-0.1.0rc1/tests/test_platform.py +143 -0
  47. unstract_cli-0.1.0rc1/tests/test_poll.py +649 -0
  48. unstract_cli-0.1.0rc1/tests/test_specs.py +44 -0
  49. unstract_cli-0.1.0rc1/tests/test_workflows.py +168 -0
  50. unstract_cli-0.1.0rc1/uv.lock +433 -0
@@ -0,0 +1,108 @@
1
+ ---
2
+ name: bump-client-pins
3
+ description: Bump the exact `unstract-client` / `llmwhisperer-client` pins, re-sync the vendored specs to match, and cut a CLI release. Use whenever a new version of either client is released, when `tests/test_specs.py` or `tests/test_contract.py` fails, when the CLI is missing a flag for an endpoint the API already has, or when someone asks to "bump the client", "update the pins", "refresh the specs", or "release the CLI". Reach for this even when the request sounds like a plain dependency bump — the pins, the vendored specs and `provenance.json` have to move together or the CLI derives flags the pinned client cannot carry.
4
+ ---
5
+
6
+ # Bumping the client pins
7
+
8
+ The CLI derives its flags and help text from two published clients and from
9
+ vendored copies of the specs those clients were generated from. That makes a pin
10
+ bump three coupled edits, not one: the pin, the spec, and the provenance record.
11
+ Move one without the others and the tests say so — which is the point of them.
12
+
13
+ ## The pieces
14
+
15
+ | Thing | Where |
16
+ |---|---|
17
+ | Exact pins | `pyproject.toml`, `[project].dependencies` |
18
+ | Vendored specs | `src/unstract_cli/specs/{docstudio,llmwhisperer}.json` |
19
+ | Provenance | `src/unstract_cli/specs/provenance.json` (client pin, upstream repo, commit, sha256) |
20
+ | Coherence tests | `tests/test_specs.py`, `tests/test_contract.py`, `tests/derived_flags.json` |
21
+ | Release | `.github/workflows/release.yml`, `workflow_dispatch` |
22
+
23
+ `src/unstract_cli/specs/README.md` explains the vendoring rule in place; read it
24
+ if any of the below is unclear.
25
+
26
+ ## The sequence
27
+
28
+ 1. **Move the pins** in `pyproject.toml` to the released versions you are
29
+ upgrading to. They are exact (`==`) on purpose: the CLI's published surface is
30
+ derived from these clients, so a client that moves reshapes the CLI.
31
+
32
+ 2. **Relock:** `uv lock` then `uv sync --extra dev --python 3.12`. CI installs
33
+ from `uv.lock`, not from a fresh resolve, so a lockfile left behind means the
34
+ gate tests a dependency set nobody ships.
35
+
36
+ 3. **Re-sync each vendored spec from the commit the pinned client was generated
37
+ from.** The chain is: the client repo's release tag → its `tools/gen_sdk.sh`,
38
+ which records the upstream service repo, path and revision the spec was copied
39
+ from → the spec file committed in that client at that tag. Copy that file here
40
+ byte-for-byte. Copying from anywhere else — upstream `main`, a newer service
41
+ commit — is what `tests/test_contract.py` guards: a spec parameter the pinned
42
+ client has no argument for cannot become a flag.
43
+
44
+ 4. **Update `provenance.json`** for each spec you moved. Set `client` to the
45
+ exact pin you wrote in `pyproject.toml` (`unstract-client==X.Y.Z`); the
46
+ tests compare the two, so a pin moved without its spec fails here. Check
47
+ `repo`, `path` and `commit` against what that client's `tools/gen_sdk.sh`
48
+ records, because an upstream that moved its spec file leaves `repo` and
49
+ `path` stale and the tests cannot see it: they check the pin, the `sha256`
50
+ and the entry names, nothing about where the file came from. The `sha256`
51
+ is of the file you just wrote (`sha256sum src/unstract_cli/specs/<file>`).
52
+ This record is what lets the next person tell a current copy from a stale
53
+ one.
54
+
55
+ 5. **Run the tests:** `uv run pytest -q`.
56
+
57
+ - `test_specs.py` fails if a vendored file stops matching its recorded
58
+ sha256, if its `client` no longer equals the pin in `pyproject.toml`, or if
59
+ a spec has no provenance entry. It is the cheap check that steps 1, 3 and
60
+ 4 actually agree.
61
+ - `test_contract.py` fails if a spec parameter the pinned client cannot accept
62
+ would have become a flag, and separately if the derived flags stop matching
63
+ `tests/derived_flags.json`.
64
+
65
+ 6. **If `derived_flags.json` fails, read the difference before refreshing it.**
66
+ The failure names the flags that moved. A flag missing from the new set is a
67
+ flag the CLI has stopped offering; a narrowed choice or changed type is a value
68
+ the CLI used to take and now rejects. Once you have decided the change is
69
+ intended, refresh it deliberately:
70
+
71
+ ```bash
72
+ UNSTRACT_CLI_REFRESH_FLAG_SNAPSHOT=1 uv run pytest -q tests/test_contract.py
73
+ ```
74
+
75
+ and commit the snapshot in the same PR, so the diff shows what the CLI's
76
+ surface gained or lost.
77
+
78
+ 7. **Lint:** `uv run ruff check . && uv run ruff format --check .` — the release
79
+ run repeats exactly this, so a failure here is a failure there.
80
+
81
+ ## Versioning and release
82
+
83
+ Choose the bump by what changed for CLI users: **minor** for new or changed
84
+ flags and commands, **patch** for fixes that leave the surface identical.
85
+
86
+ Do not bump `__version__` in `src/unstract_cli/__init__.py` in your PR. The
87
+ in-repo value names the last released version; `release.yml` reads it, applies
88
+ the bump chosen at dispatch and commits the result itself.
89
+
90
+ Release by dispatching **Release Tag and Publish Package** on `main`:
91
+
92
+ - `version_bump: none` publishes the version already in the repo — what the
93
+ first release of a version needs.
94
+ - `pre_release: true` publishes `<next-version>rcN` and deliberately leaves the
95
+ committed version alone, counting N up from the rc tags already published for
96
+ that target. To promote to stable, dispatch again with it off **and the same
97
+ `version_bump`**: the workflow recomputes the target from that input every
98
+ time, so a different bump publishes a different version than the one the rc
99
+ tested.
100
+ - It publishes to PyPI **before** it tags and releases, because publishing is the
101
+ only step that cannot be undone: a failure before it leaves nothing to
102
+ unpublish, and one after it is retried by hand against a live artifact.
103
+
104
+ ## Upstream
105
+
106
+ If a client pin is missing an endpoint the service already offers, the fix is in
107
+ that client, not here — see the `spec-upgrade` skill in `unstract-python-client`
108
+ and `llm-whisperer-python-client`. Bump the pin here once it is released.
@@ -0,0 +1,74 @@
1
+ name: ci
2
+
3
+ on:
4
+ pull_request:
5
+ push:
6
+ branches: [main]
7
+
8
+ jobs:
9
+ # Offline by design: no network, no credentials, sub-second. Live round trips
10
+ # are a manual pre-release step, not a per-PR gate.
11
+ test:
12
+ runs-on: ubuntu-latest
13
+ steps:
14
+ - uses: actions/checkout@v4
15
+ - uses: astral-sh/setup-uv@v6
16
+ with:
17
+ version: "0.6.14"
18
+ enable-cache: true
19
+ # Synced from uv.lock rather than resolved fresh, so the gate tests the
20
+ # dependency set an install actually gets.
21
+ - run: uv sync --extra dev --python 3.12
22
+ - run: uv run ruff check .
23
+ - run: uv run ruff format --check .
24
+ - run: uv run pytest -q
25
+ # `uv run` resolves from uv.lock, whose click sits in the middle of the
26
+ # range the pin allows. Both ends are what an install in the wild gets --
27
+ # the floor exactly, not the newest release that satisfies it -- and
28
+ # discovery is a published contract read out of Click's own objects, so
29
+ # both ends are run and their answers compared.
30
+ - run: uv pip install 'click==8.1.0'
31
+ - run: .venv/bin/python -m pytest -q
32
+ - run: .venv/bin/python -m unstract_cli -o json --discover full > floor.json
33
+ - run: uv pip install -U 'click>=8.1,<9'
34
+ - run: .venv/bin/python -m pytest -q
35
+ - run: .venv/bin/python -m unstract_cli -o json --discover full > latest.json
36
+ - run: diff floor.json latest.json
37
+
38
+ # The wheel rather than the checkout: packaging and entry points are declared
39
+ # in `pyproject.toml` and are not exercised by running the tests in place.
40
+ package:
41
+ runs-on: ubuntu-latest
42
+ env:
43
+ VIRTUAL_ENV: /tmp/smoke
44
+ steps:
45
+ - uses: actions/checkout@v4
46
+ - uses: astral-sh/setup-uv@v6
47
+ with:
48
+ version: "0.6.14"
49
+ enable-cache: true
50
+ - run: uv build --out-dir dist
51
+ - run: uv venv --python 3.12 /tmp/smoke
52
+ - run: uv pip install dist/*.whl
53
+ - name: Both entry points answer
54
+ run: |
55
+ /tmp/smoke/bin/unstract --version
56
+ /tmp/smoke/bin/unstract-cli --version
57
+ /tmp/smoke/bin/unstract --help > /dev/null
58
+ - name: The published surfaces render from an installed wheel
59
+ run: |
60
+ /tmp/smoke/bin/unstract -o json --discover full > discover.json
61
+ python -c "import json; d = json.load(open('discover.json')); assert d['ok'] and d['data']['commands']"
62
+ # A usage error still renders the envelope and still exits 2.
63
+ set +e
64
+ /tmp/smoke/bin/unstract -o json whisper status > usage.json
65
+ code=$?
66
+ set -e
67
+ test "$code" = "2"
68
+ python -c "import json; d = json.load(open('usage.json')); assert d['ok'] is False and d['error']['exit_code'] == 2"
69
+ - name: A fresh install can write its own config
70
+ env:
71
+ UNSTRACT_CONFIG: /tmp/smoke-config.toml
72
+ run: |
73
+ /tmp/smoke/bin/unstract config init < /dev/null
74
+ test -s /tmp/smoke-config.toml
@@ -0,0 +1,226 @@
1
+ name: Release Tag and Publish Package
2
+
3
+ on:
4
+ workflow_dispatch:
5
+ inputs:
6
+ version_bump:
7
+ description: "Version bump type."
8
+ required: true
9
+ default: "patch"
10
+ type: choice
11
+ options:
12
+ - patch
13
+ - minor
14
+ - major
15
+ pre_release:
16
+ description: "Publish a release candidate (`<version>rcN`) instead of the version itself. Dispatch again with the same bump and this off to promote it to stable."
17
+ required: false
18
+ default: false
19
+ type: boolean
20
+ release_notes:
21
+ description: "Release notes (optional)"
22
+ required: false
23
+ type: string
24
+
25
+ concurrency:
26
+ group: release
27
+ cancel-in-progress: false
28
+
29
+ jobs:
30
+ release-and-publish:
31
+ runs-on: ubuntu-latest
32
+ # Dispatchable from any branch by anyone who can dispatch it, and a
33
+ # candidate reaches PyPI without a commit to review: the published artefact
34
+ # must come from the reviewed branch.
35
+ if: github.ref == 'refs/heads/main'
36
+ permissions:
37
+ contents: write
38
+ # Publishing is by PyPI Trusted Publisher, so there is no API token.
39
+ id-token: write
40
+ steps:
41
+ # Checked before the checkout, so a missing one fails on the reason
42
+ # rather than on a token that renders empty three steps later.
43
+ - name: Check the release credentials are visible to this repository
44
+ env:
45
+ APP_CLIENT_ID: ${{ vars.PUSH_TO_MAIN_APP_CLIENT_ID }}
46
+ APP_PRIVATE_KEY: ${{ secrets.PUSH_TO_MAIN_APP_PRIVATE_KEY }}
47
+ run: |
48
+ missing=""
49
+ [ -n "$APP_CLIENT_ID" ] || missing="$missing vars.PUSH_TO_MAIN_APP_CLIENT_ID"
50
+ [ -n "$APP_PRIVATE_KEY" ] || missing="$missing secrets.PUSH_TO_MAIN_APP_PRIVATE_KEY"
51
+ if [ -n "$missing" ]; then
52
+ echo "Missing:$missing" >&2
53
+ echo "An organisation-level variable or secret also has to list this repository." >&2
54
+ exit 1
55
+ fi
56
+
57
+ - name: Generate GitHub App Token
58
+ id: generate-token
59
+ uses: actions/create-github-app-token@v3
60
+ with:
61
+ client-id: ${{ vars.PUSH_TO_MAIN_APP_CLIENT_ID }}
62
+ private-key: ${{ secrets.PUSH_TO_MAIN_APP_PRIVATE_KEY }}
63
+ owner: Zipstack
64
+ repositories: |
65
+ unstract-cli
66
+
67
+ - uses: actions/checkout@v4
68
+ with:
69
+ token: ${{ steps.generate-token.outputs.token }}
70
+ fetch-depth: 0
71
+
72
+ - name: Configure Git
73
+ run: |
74
+ git config --global user.name "github-actions[bot]"
75
+ git config --global user.email "github-actions[bot]@users.noreply.github.com"
76
+
77
+ - uses: actions/setup-python@v5
78
+ with:
79
+ python-version: "3.12"
80
+
81
+ - uses: astral-sh/setup-uv@v6
82
+ with:
83
+ version: "0.6.14"
84
+ enable-cache: true
85
+
86
+ # The same install as ci.yml, so what the release run lints and tests is
87
+ # what the PR gate lints and tests.
88
+ - run: uv sync --extra dev --python 3.12
89
+
90
+ # Staged locally only: nothing is committed, tagged or released until the
91
+ # checks and the build have passed, so a failure leaves main untouched.
92
+ - name: Compute new version
93
+ id: version
94
+ run: |
95
+ VERSION_FILE=src/unstract_cli/__init__.py
96
+ CURRENT_VERSION=$(sed -nE 's/^__version__ = "(.*)"/\1/p' "$VERSION_FILE")
97
+ echo "Current version: $CURRENT_VERSION"
98
+
99
+ IFS='.' read -r MAJOR MINOR PATCH <<< "$CURRENT_VERSION"
100
+ case "${{ github.event.inputs.version_bump }}" in
101
+ major) MAJOR=$((MAJOR + 1)); MINOR=0; PATCH=0 ;;
102
+ minor) MINOR=$((MINOR + 1)); PATCH=0 ;;
103
+ patch) PATCH=$((PATCH + 1)) ;;
104
+ esac
105
+ NEXT_VERSION="$MAJOR.$MINOR.$PATCH"
106
+
107
+ # A pre-release is a candidate for NEXT_VERSION, not a version of its
108
+ # own, so it never moves the committed one: the file keeps naming the
109
+ # last stable release, and repeat dispatches count up from the rc tags
110
+ # already published for that target.
111
+ if [ "${{ github.event.inputs.pre_release }}" = "true" ]; then
112
+ HIGHEST_RC=$(git tag -l "v${NEXT_VERSION}rc*" \
113
+ | sed -nE "s/^v${NEXT_VERSION}rc([0-9]+)$/\1/p" | sort -n | tail -1)
114
+ NEW_VERSION="${NEXT_VERSION}rc$(( ${HIGHEST_RC:-0} + 1 ))"
115
+ else
116
+ NEW_VERSION="$NEXT_VERSION"
117
+ fi
118
+
119
+ echo "New version: $NEW_VERSION"
120
+ echo "version=$NEW_VERSION" >> "$GITHUB_OUTPUT"
121
+
122
+ sed -i "s/^__version__ = \".*\"/__version__ = \"$NEW_VERSION\"/" "$VERSION_FILE"
123
+
124
+ if git rev-parse -q --verify "refs/tags/v$NEW_VERSION" >/dev/null; then
125
+ echo "Tag v$NEW_VERSION already exists. Exiting..."
126
+ exit 1
127
+ fi
128
+
129
+ - name: Verify version update
130
+ run: |
131
+ BUILT_VERSION=$(uv run python -c "import unstract_cli; print(unstract_cli.__version__)")
132
+ echo "Package version: $BUILT_VERSION"
133
+ echo "Target version: ${{ steps.version.outputs.version }}"
134
+ if [ "$BUILT_VERSION" != "${{ steps.version.outputs.version }}" ]; then
135
+ echo "Version mismatch! Exiting..."
136
+ exit 1
137
+ fi
138
+
139
+ - name: Run linting
140
+ run: |
141
+ uv run ruff check .
142
+ uv run ruff format --check .
143
+
144
+ - name: Run tests
145
+ run: uv run pytest -q
146
+
147
+ - name: Build package
148
+ run: uv build
149
+
150
+ # Publishing is the one step that cannot be undone, so it runs last, and
151
+ # the release is created as a draft so a publish that fails leaves
152
+ # nothing public. Until then the bump commit exists only in the tag, so
153
+ # a failed publish leaves main untouched.
154
+ - name: Commit version bump and create draft release
155
+ env:
156
+ RELEASE_NOTES: ${{ github.event.inputs.release_notes }}
157
+ GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
158
+ run: |
159
+ NEW_VERSION="${{ steps.version.outputs.version }}"
160
+
161
+ # A pre-release leaves the committed version alone.
162
+ if [ "${{ github.event.inputs.pre_release }}" = "true" ]; then
163
+ git checkout -- src/unstract_cli/__init__.py
164
+ elif ! git diff --quiet; then
165
+ git add src/unstract_cli/__init__.py
166
+ git commit -m "chore: bump version to $NEW_VERSION [skip ci]"
167
+ fi
168
+
169
+ git tag "v$NEW_VERSION"
170
+ git push origin "v$NEW_VERSION"
171
+
172
+ if [ -z "$RELEASE_NOTES" ]; then
173
+ gh release create "v$NEW_VERSION" \
174
+ --title "Release v$NEW_VERSION" \
175
+ --generate-notes \
176
+ --draft \
177
+ ${{ github.event.inputs.pre_release == 'true' && '--prerelease' || '' }}
178
+ else
179
+ gh release create "v$NEW_VERSION" \
180
+ --title "Release v$NEW_VERSION" \
181
+ --notes "$RELEASE_NOTES" \
182
+ --generate-notes \
183
+ --draft \
184
+ ${{ github.event.inputs.pre_release == 'true' && '--prerelease' || '' }}
185
+ fi
186
+
187
+ echo "Created draft release v$NEW_VERSION"
188
+
189
+ - name: Publish to PyPI
190
+ run: uv publish
191
+
192
+ # Everything the run created before the publish, so a rerun after the
193
+ # cause is fixed starts clean rather than on the tag guard. Each removal
194
+ # tolerates its target being absent: the failure may have come partway
195
+ # through creating them.
196
+ - name: Remove the tag and draft release a failed publish leaves behind
197
+ if: failure() && steps.version.outcome == 'success'
198
+ env:
199
+ GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
200
+ run: |
201
+ NEW_VERSION="${{ steps.version.outputs.version }}"
202
+ # The wheel and the sdist upload separately, so a failed publish can
203
+ # still have put the version on PyPI; then the tag has to stay.
204
+ if curl -fsS -o /dev/null "https://pypi.org/pypi/unstract-cli/$NEW_VERSION/json"; then
205
+ echo "PyPI serves $NEW_VERSION; keeping the tag and draft release." >&2
206
+ exit 0
207
+ fi
208
+ gh release delete "v$NEW_VERSION" --yes || true
209
+ git push origin ":refs/tags/v$NEW_VERSION" || true
210
+
211
+ # If this fails the published version is on PyPI and in the tag but not
212
+ # on main; `git push origin v<version>:main` finishes the job by hand.
213
+ - name: Push version bump
214
+ if: github.event.inputs.pre_release != 'true'
215
+ run: git push origin main
216
+
217
+ - name: Publish release
218
+ env:
219
+ GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
220
+ run: gh release edit "v${{ steps.version.outputs.version }}" --draft=false
221
+
222
+ - name: Success message
223
+ run: |
224
+ echo "Published ${{ steps.version.outputs.version }} to PyPI with uv publish using Trusted Publishers"
225
+ echo "Release: https://github.com/${{ github.repository }}/releases/tag/v${{ steps.version.outputs.version }}"
226
+ echo "PyPI: https://pypi.org/project/unstract-cli/${{ steps.version.outputs.version }}/"
@@ -0,0 +1,10 @@
1
+ .venv/
2
+ __pycache__/
3
+ *.egg-info/
4
+ .pytest_cache/
5
+ .ruff_cache/
6
+ dist/
7
+ build/
8
+ .coverage
9
+ .coverage.*
10
+ htmlcov/
@@ -0,0 +1,197 @@
1
+ Metadata-Version: 2.5
2
+ Name: unstract-cli
3
+ Version: 0.1.0rc1
4
+ Summary: Unified, LLM-friendly CLI for the Unstract suite of products
5
+ Requires-Python: >=3.12
6
+ Requires-Dist: click<9,>=8.1
7
+ Requires-Dist: llmwhisperer-client==2.9.0
8
+ Requires-Dist: requests>=2.32.3
9
+ Requires-Dist: tomli-w>=1.0
10
+ Requires-Dist: unstract-client==1.7.0
11
+ Provides-Extra: dev
12
+ Requires-Dist: pytest>=8.0; extra == 'dev'
13
+ Requires-Dist: pyyaml>=6.0; extra == 'dev'
14
+ Requires-Dist: ruff>=0.6; extra == 'dev'
15
+ Description-Content-Type: text/markdown
16
+
17
+ # unstract-cli
18
+
19
+ `unstract` — one CLI for the Unstract suite: extract a document with
20
+ LLMWhisperer, run it through a Document Studio API deployment, get structured
21
+ JSON back. It also clones one organization's resources into another.
22
+
23
+ ```bash
24
+ curl -LsSf https://raw.githubusercontent.com/Zipstack/unstract-cli/main/install.sh | sh
25
+ unstract auth login # asks for your keys, checks them, stores them
26
+ unstract docstudio deployment ls # what can I run?
27
+ ```
28
+
29
+ For an agent or CI, no prompts and no file — the environment is the profile:
30
+
31
+ ```bash
32
+ export UNSTRACT_ORG_ID=... UNSTRACT_DEPLOYMENT_KEY=... LLMWHISPERER_API_KEY=...
33
+ unstract -o json whisper extract ./doc.pdf
34
+ unstract -o json docstudio deployment run invoice-parser ./doc.pdf
35
+ ```
36
+
37
+ The installer fetches `uv` if it is missing and installs the CLI with it; `uv`
38
+ brings its own Python, so nothing on the machine has to match. Already have
39
+ `uv`? `uv tool install git+https://github.com/Zipstack/unstract-cli` is the same
40
+ thing. Set `UNSTRACT_CLI_SOURCE` to install a branch or a local checkout
41
+ instead.
42
+
43
+ Or run it without installing: `uvx --from git+https://github.com/Zipstack/unstract-cli unstract --discover groups`.
44
+
45
+ ## Output
46
+
47
+ `unstract` prints a table by default — in a terminal and in a pipe alike, so
48
+ what you see while trying something is what a script sees running it.
49
+
50
+ **Parsing anything? Pass `-o json`.** stdout then carries exactly one envelope,
51
+ on success and on failure alike:
52
+
53
+ ```json
54
+ {"ok": true, "data": {...}, "error": null, "meta": {"contract_version": 1}}
55
+ ```
56
+
57
+ `-o json` output depends on nothing but the command and its arguments — not the
58
+ terminal, not the config, not the environment. `-o raw` prints one field
59
+ unwrapped, for piping a document's text somewhere else. Diagnostics, warnings
60
+ and progress always go to stderr.
61
+
62
+ Consuming the JSON: ignore fields you do not recognise, and refuse a
63
+ `meta.contract_version` above the one you were written against. `unstract
64
+ --discover full` publishes the whole contract alongside every command and flag.
65
+
66
+ If a coding agent is driving (detected from the environment it sets), the
67
+ *default* becomes json. `--agent yes|no` forces that either way, and an explicit
68
+ `-o` always wins over both.
69
+
70
+ Failures exit non-zero with a stable code. The codes are this CLI's own
71
+ convention, not a service's — they are the `ExitCode` enum in
72
+ `core/errors.py`, and `--discover full` publishes the table so a caller does not
73
+ have to copy it:
74
+
75
+ | Code | Meaning |
76
+ |------|---------|
77
+ | 0 | success |
78
+ | 1 | generic failure |
79
+ | 2 | usage error |
80
+ | 3 | authentication failed |
81
+ | 4 | not found |
82
+ | 5 | validation failed — also a completed run in which a document failed; the full result, successful documents included, is in `error.details` |
83
+ | 6 | rate limited |
84
+ | 7 | timed out (the job handle is in the error payload — resume, do not resubmit) |
85
+ | 8 | server error |
86
+ | 9 | result already consumed (one-shot read; use `--save` next time) |
87
+ | 10 | the result was read but could not be saved — it is in `error.details` |
88
+ | 130 | interrupted (128 + SIGINT) — the user stopped it, not a failure |
89
+
90
+ ## Credentials
91
+
92
+ Three keys, each for one job:
93
+
94
+ - **LLMWhisperer key** — extracts text (`whisper …`). Minted in the LLMWhisperer
95
+ console.
96
+ - **Deployment key** — runs deployments (`deployment run`, `deployment status`).
97
+ Shown on the API deployment's own page in the Unstract UI; one an
98
+ organisation admin mints under **Settings → Global API Deployment Keys**
99
+ covers every deployment in the organisation.
100
+ - **Platform key** — identifies the organisation and lists what is in it
101
+ (`auth whoami`, `deployment ls`). Minted by an organisation admin under
102
+ **Settings → Platform API Keys**.
103
+
104
+ `auth login` takes whichever of the three you have, checks the two it can
105
+ (`whoami` for the platform key, the usage endpoint for the LLMWhisperer key; a
106
+ deployment key has nothing side-effect-free to call and is stored as given) and
107
+ writes them to one profile. Run it again to rotate a key. A login that stores a
108
+ different host drops the profile's other keys rather than leave them beside a
109
+ server that never accepted them: it asks first, or without a terminal fails
110
+ until `--force`. Without a terminal pass them as flags — `--platform-key`, `--deployment-key`, `--llmwhisperer-key`,
111
+ any one of them `-` to read from stdin.
112
+
113
+ ## Configuration
114
+
115
+ `~/.unstract/config.toml`, or a project-local `.unstract.toml` found by upward
116
+ search, or `$UNSTRACT_CONFIG`, or `--config`. Every setting resolves
117
+ **flag > env > profile > built-in default**, and the CLI is fully usable with no
118
+ config file at all. The flag tier is the connection options on each product
119
+ group — `--base-url`, `--api-key`, `--org-id` and `--platform-key` on
120
+ `docstudio`, `--base-url`/`--api-key` on `whisper`, `--base-url`/`--platform-key`
121
+ on `auth` — which override the profile for that one invocation without writing
122
+ anything.
123
+
124
+ ```toml
125
+ default_profile = "cloud-us"
126
+
127
+ [profiles.cloud-us.llmwhisperer]
128
+ base_url = "https://llmwhisperer-api.us-central.unstract.com/api/v2"
129
+ api_key = "env:LLMWHISPERER_API_KEY"
130
+
131
+ [profiles.cloud-us.docstudio]
132
+ base_url = "https://us-central.unstract.com"
133
+ org_id = "org_ABC123"
134
+ api_key = "env:UNSTRACT_DEPLOYMENT_KEY"
135
+ platform_key = "env:UNSTRACT_PLATFORM_KEY"
136
+
137
+ # Only for a deployment whose key differs from the one above.
138
+ [profiles.cloud-us.deployments."invoice-parser"]
139
+ api_key = "env:INVOICE_PARSER_KEY"
140
+ ```
141
+
142
+ `deployment run` and `deployment status` take the API name as `deployment ls`
143
+ prints it. `ls` itself authenticates with the platform key and refuses
144
+ `--api-key`, which on `docstudio` means a deployment key. The key for a run resolves `--api-key` > `$UNSTRACT_DEPLOYMENT_KEY` >
145
+ the deployment's own entry > the profile's `api_key`, so most profiles need no
146
+ `deployments` section at all; `config set docstudio api_key <key> --deployment
147
+ <api_name>` writes one. `org_id` lives on the `docstudio` block — `auth login`
148
+ and `auth whoami` write the one the platform key resolves there. `config init`
149
+ writes this shape minus `platform_key` and the `deployments` entry — both are
150
+ the exception, not the starting point — plus a `cloud-eu` profile and an
151
+ `onprem-example` shape to copy for a self-hosted install; only the *active*
152
+ profile is ever resolved.
153
+
154
+ Either form works for a credential. `auth login` writes keys literally, having
155
+ checked them at the moment it writes. `env:VAR_NAME` indirection — what `config
156
+ init` writes and what the example above uses — keeps the secret out of the file,
157
+ so it stays safe to copy or commit; that is the form for a shared machine or a
158
+ CI checkout. Either way the file is created `0600`, and `config doctor` warns
159
+ when its mode is wider than that.
160
+
161
+ `unstract config doctor` reports where each setting resolved from — including
162
+ whether an `env:` reference is actually set in the current process — without
163
+ echoing any value. `--probe` also checks the two keys that can be
164
+ checked — the platform key and the LLMWhisperer key, the same two `auth login`
165
+ checks — and, with a platform key, warns about a `deployments` entry the
166
+ organisation no longer has. It exits non-zero when one of its own checks failed, so a setup script can
167
+ branch on it.
168
+
169
+ A project-local `.unstract.toml` **found by upward search** may not supply a
170
+ key or `base_url`. Those are ignored, with a warning; everything else in it —
171
+ profile selection, `org_id` — applies as usual. A checkout you did not write is
172
+ not trusted to name the host your key is sent to. Name the file explicitly
173
+ (`--config` or `$UNSTRACT_CONFIG`) and it is honoured in full.
174
+
175
+ What that protects is the key and the host, not the routing: `org_id` and
176
+ profile selection stay repo-controllable by design, so a project file can still
177
+ decide *which* organisation a command runs against on a host you trust. Read
178
+ one before you run inside a checkout you did not write.
179
+
180
+ `clone` is the exception, and it is an operator command: a human moving one
181
+ organisation's resources into another, holding two admin Platform keys. It is
182
+ not part of the document-processing path the rest of this CLI wraps, so an agent
183
+ serving a user request should not reach for it unasked. It talks to two
184
+ deployments at once, which no single profile describes, so it takes both
185
+ endpoints as flags and both keys from `UNSTRACT_SRC_PLATFORM_KEY` /
186
+ `UNSTRACT_TGT_PLATFORM_KEY` — two keys for two organisations, so it reads
187
+ neither the profile's `platform_key` nor `$UNSTRACT_PLATFORM_KEY`. It exits 0
188
+ when nothing failed, which is not the same as everything having moved: oversize
189
+ and unsupported documents are skipped by design, and `data.skipped` counts them.
190
+
191
+ ## Development
192
+
193
+ ```bash
194
+ uv venv && uv pip install -e '.[dev]'
195
+ uv run pytest # offline; no network, no credentials
196
+ uv run ruff check .
197
+ ```