evalmarket-cli 0.1.0__tar.gz → 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- evalmarket_cli-0.1.1/PKG-INFO +19 -0
- evalmarket_cli-0.1.1/README.md +12 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/_version.py +1 -1
- evalmarket_cli-0.1.0/PKG-INFO +0 -69
- evalmarket_cli-0.1.0/README.md +0 -62
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/.gitignore +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/pyproject.toml +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/__init__.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/__main__.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/__init__.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/client.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/experiments.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/importer.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/models.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/resources.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/braintrust/validation.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/cli.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/entrypoint.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/json_data.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/__init__.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/importer.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/models.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/playground.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/prompts.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/resources.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/src/evalmarket/langsmith/validation.py +0 -0
- {evalmarket_cli-0.1.0 → evalmarket_cli-0.1.1}/tests/test_installed.py +0 -0
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: evalmarket-cli
|
|
3
|
+
Version: 0.1.1
|
|
4
|
+
Summary: Import purchased EvalMarket evaluations into your evaluation platform
|
|
5
|
+
Requires-Python: >=3.10
|
|
6
|
+
Description-Content-Type: text/markdown
|
|
7
|
+
|
|
8
|
+
# EvalMarket CLI
|
|
9
|
+
|
|
10
|
+
Import an extracted EvalMarket evaluation into your own Braintrust project or LangSmith workspace. The CLI uses the platform files supplied with the evaluation and verifies the imported resources.
|
|
11
|
+
|
|
12
|
+
Install or upgrade in a Python 3.10+ virtual environment:
|
|
13
|
+
|
|
14
|
+
```sh
|
|
15
|
+
python -m pip install --upgrade evalmarket-cli
|
|
16
|
+
evalmarket --version
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Run `evalmarket import braintrust` or `evalmarket import langchain` from your extracted evaluation folder. For platform-specific prerequisites, see the [Braintrust guide](https://docs.evalmarket.ai/buyers/platform-integrations/braintrust/) or the [LangSmith guide](https://docs.evalmarket.ai/buyers/platform-integrations/langsmith/).
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# EvalMarket CLI
|
|
2
|
+
|
|
3
|
+
Import an extracted EvalMarket evaluation into your own Braintrust project or LangSmith workspace. The CLI uses the platform files supplied with the evaluation and verifies the imported resources.
|
|
4
|
+
|
|
5
|
+
Install or upgrade in a Python 3.10+ virtual environment:
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
python -m pip install --upgrade evalmarket-cli
|
|
9
|
+
evalmarket --version
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Run `evalmarket import braintrust` or `evalmarket import langchain` from your extracted evaluation folder. For platform-specific prerequisites, see the [Braintrust guide](https://docs.evalmarket.ai/buyers/platform-integrations/braintrust/) or the [LangSmith guide](https://docs.evalmarket.ai/buyers/platform-integrations/langsmith/).
|
evalmarket_cli-0.1.0/PKG-INFO
DELETED
|
@@ -1,69 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.5
|
|
2
|
-
Name: evalmarket-cli
|
|
3
|
-
Version: 0.1.0
|
|
4
|
-
Summary: Import purchased EvalMarket evaluations into your evaluation platform
|
|
5
|
-
Requires-Python: >=3.10
|
|
6
|
-
Description-Content-Type: text/markdown
|
|
7
|
-
|
|
8
|
-
# EvalMarket Python importer
|
|
9
|
-
|
|
10
|
-
Customers install the versioned Python wheel from the marketplace's [CLI download page](https://www.evalmarket.ai/import-guide/cli). This page and the wheel ship together with the website. Python 3.10+ is required. The distribution has no Python runtime dependencies. LangSmith imports also need the official LangSmith CLI on PATH and its authentication/profile setup. Braintrust uses a local API key.
|
|
11
|
-
|
|
12
|
-
Download the wheel into a folder outside the extracted evaluation. In an activated Python virtual environment, run these commands from the wheel's folder:
|
|
13
|
-
|
|
14
|
-
```sh
|
|
15
|
-
python -I -m pip install --no-index --no-deps --upgrade ./evalmarket_cli-0.1.0-py3-none-any.whl
|
|
16
|
-
evalmarket --version
|
|
17
|
-
evalmarket import /path/to/braintrust
|
|
18
|
-
evalmarket import /path/to/langchain --profile my-profile
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
For upgrades, download the newer wheel and repeat the install with its exact filename in the same environment. Existing extracted evaluation folders can be reused. The package has not been published to a registry.
|
|
22
|
-
|
|
23
|
-
For development in the API environment, run `uv sync --locked` in `apps/api`,
|
|
24
|
-
then `uv run evalmarket import --help`.
|
|
25
|
-
The module entry point is `python -I -m evalmarket` in an installed environment.
|
|
26
|
-
|
|
27
|
-
The CLI preserves existing model selection, confirmation, readback verification,
|
|
28
|
-
partial-failure handling, and explicit experiment/Playground opt-in. It imports
|
|
29
|
-
an extracted platform folder. It does not download purchases or unpack ZIPs.
|
|
30
|
-
|
|
31
|
-
## Source layout
|
|
32
|
-
|
|
33
|
-
```text
|
|
34
|
-
src/evalmarket/
|
|
35
|
-
cli.py Argument parsing and error presentation
|
|
36
|
-
entrypoint.py Isolated process startup
|
|
37
|
-
json_data.py Shared file-size limit, strict JSON parsing and readback comparisons
|
|
38
|
-
braintrust/
|
|
39
|
-
validation.py Pure native package validation shared with the API
|
|
40
|
-
client.py HTTP, pagination, experiment-start acknowledgement
|
|
41
|
-
resources.py Resource identifiers and model labels
|
|
42
|
-
models.py Account model discovery and selection
|
|
43
|
-
importer.py Import orchestration and readback
|
|
44
|
-
experiments.py Explicit optional experiment start
|
|
45
|
-
langsmith/
|
|
46
|
-
validation.py Pure prompt-manifest validation shared with the API
|
|
47
|
-
resources.py Resource identifiers
|
|
48
|
-
models.py Saved judge-model selection
|
|
49
|
-
importer.py Native CLI import orchestration and authentication
|
|
50
|
-
prompts.py Private answering prompt creation and readback
|
|
51
|
-
playground.py Explicit optional browser handoff
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
Use explicit module imports and keep pure validators independent of platform
|
|
55
|
-
clients. A new platform should have its own modules and explicit CLI routing;
|
|
56
|
-
there is no plugin framework or stable Python SDK contract in this release.
|
|
57
|
-
The version in `_version.py` drives distribution metadata and `--version`.
|
|
58
|
-
|
|
59
|
-
## Build and verify
|
|
60
|
-
|
|
61
|
-
From the repository root, run:
|
|
62
|
-
|
|
63
|
-
```sh
|
|
64
|
-
uv build --no-sources packages/evalmarket --out-dir dist
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
Install the resulting wheel in a clean environment. Run `packages/evalmarket/tests/test_installed.py` with its Python interpreter. CI tests the installed CLI on Windows and Linux with Python 3.10 and 3.13. API tests exercise the provider workflows, ZIP delivery, and shared validators.
|
|
68
|
-
|
|
69
|
-
For a new customer release, bump the version, build the wheel and source distribution, and copy the reviewed wheel to `apps/app/public/downloads/` under its original filename. Update the download page and verify the public wheel before API cutover. Obtain release authorization before publication or deployment.
|
evalmarket_cli-0.1.0/README.md
DELETED
|
@@ -1,62 +0,0 @@
|
|
|
1
|
-
# EvalMarket Python importer
|
|
2
|
-
|
|
3
|
-
Customers install the versioned Python wheel from the marketplace's [CLI download page](https://www.evalmarket.ai/import-guide/cli). This page and the wheel ship together with the website. Python 3.10+ is required. The distribution has no Python runtime dependencies. LangSmith imports also need the official LangSmith CLI on PATH and its authentication/profile setup. Braintrust uses a local API key.
|
|
4
|
-
|
|
5
|
-
Download the wheel into a folder outside the extracted evaluation. In an activated Python virtual environment, run these commands from the wheel's folder:
|
|
6
|
-
|
|
7
|
-
```sh
|
|
8
|
-
python -I -m pip install --no-index --no-deps --upgrade ./evalmarket_cli-0.1.0-py3-none-any.whl
|
|
9
|
-
evalmarket --version
|
|
10
|
-
evalmarket import /path/to/braintrust
|
|
11
|
-
evalmarket import /path/to/langchain --profile my-profile
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
For upgrades, download the newer wheel and repeat the install with its exact filename in the same environment. Existing extracted evaluation folders can be reused. The package has not been published to a registry.
|
|
15
|
-
|
|
16
|
-
For development in the API environment, run `uv sync --locked` in `apps/api`,
|
|
17
|
-
then `uv run evalmarket import --help`.
|
|
18
|
-
The module entry point is `python -I -m evalmarket` in an installed environment.
|
|
19
|
-
|
|
20
|
-
The CLI preserves existing model selection, confirmation, readback verification,
|
|
21
|
-
partial-failure handling, and explicit experiment/Playground opt-in. It imports
|
|
22
|
-
an extracted platform folder. It does not download purchases or unpack ZIPs.
|
|
23
|
-
|
|
24
|
-
## Source layout
|
|
25
|
-
|
|
26
|
-
```text
|
|
27
|
-
src/evalmarket/
|
|
28
|
-
cli.py Argument parsing and error presentation
|
|
29
|
-
entrypoint.py Isolated process startup
|
|
30
|
-
json_data.py Shared file-size limit, strict JSON parsing and readback comparisons
|
|
31
|
-
braintrust/
|
|
32
|
-
validation.py Pure native package validation shared with the API
|
|
33
|
-
client.py HTTP, pagination, experiment-start acknowledgement
|
|
34
|
-
resources.py Resource identifiers and model labels
|
|
35
|
-
models.py Account model discovery and selection
|
|
36
|
-
importer.py Import orchestration and readback
|
|
37
|
-
experiments.py Explicit optional experiment start
|
|
38
|
-
langsmith/
|
|
39
|
-
validation.py Pure prompt-manifest validation shared with the API
|
|
40
|
-
resources.py Resource identifiers
|
|
41
|
-
models.py Saved judge-model selection
|
|
42
|
-
importer.py Native CLI import orchestration and authentication
|
|
43
|
-
prompts.py Private answering prompt creation and readback
|
|
44
|
-
playground.py Explicit optional browser handoff
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
Use explicit module imports and keep pure validators independent of platform
|
|
48
|
-
clients. A new platform should have its own modules and explicit CLI routing;
|
|
49
|
-
there is no plugin framework or stable Python SDK contract in this release.
|
|
50
|
-
The version in `_version.py` drives distribution metadata and `--version`.
|
|
51
|
-
|
|
52
|
-
## Build and verify
|
|
53
|
-
|
|
54
|
-
From the repository root, run:
|
|
55
|
-
|
|
56
|
-
```sh
|
|
57
|
-
uv build --no-sources packages/evalmarket --out-dir dist
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
Install the resulting wheel in a clean environment. Run `packages/evalmarket/tests/test_installed.py` with its Python interpreter. CI tests the installed CLI on Windows and Linux with Python 3.10 and 3.13. API tests exercise the provider workflows, ZIP delivery, and shared validators.
|
|
61
|
-
|
|
62
|
-
For a new customer release, bump the version, build the wheel and source distribution, and copy the reviewed wheel to `apps/app/public/downloads/` under its original filename. Update the download page and verify the public wheel before API cutover. Obtain release authorization before publication or deployment.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|