haliosai-cli 2.0.0__tar.gz → 2.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- haliosai_cli-2.0.1/PKG-INFO +138 -0
- haliosai_cli-2.0.1/README.md +101 -0
- haliosai_cli-2.0.1/halios_cli/_version.py +1 -0
- haliosai_cli-2.0.1/haliosai_cli.egg-info/PKG-INFO +138 -0
- haliosai_cli-2.0.0/PKG-INFO +0 -101
- haliosai_cli-2.0.0/README.md +0 -64
- haliosai_cli-2.0.0/halios_cli/_version.py +0 -1
- haliosai_cli-2.0.0/haliosai_cli.egg-info/PKG-INFO +0 -101
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/LICENSE +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/MANIFEST.in +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/__init__.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_auth.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_eval.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_optimize.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_project.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_scenario.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_support.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/cli_trace.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/py.typed +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/schemas/__init__.py +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/schemas/eval.schema.json +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/halios_cli/schemas/scenarios.schema.json +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/haliosai_cli.egg-info/SOURCES.txt +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/haliosai_cli.egg-info/dependency_links.txt +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/haliosai_cli.egg-info/entry_points.txt +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/haliosai_cli.egg-info/requires.txt +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/haliosai_cli.egg-info/top_level.txt +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/pyproject.toml +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/setup.cfg +0 -0
- {haliosai_cli-2.0.0 → haliosai_cli-2.0.1}/tests/test_cli_surface.py +0 -0
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: haliosai-cli
|
|
3
|
+
Version: 2.0.1
|
|
4
|
+
Summary: Halios CLI for coding-agent evaluations, guardrails, and OpenTelemetry evidence
|
|
5
|
+
Author-email: HaliosLabs <support@halios.ai>
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Project-URL: Homepage, https://halios.ai
|
|
8
|
+
Project-URL: Documentation, https://docs.halios.ai
|
|
9
|
+
Project-URL: Repository, https://github.com/HaliosAI/halios
|
|
10
|
+
Project-URL: Changelog, https://github.com/HaliosAI/halios/releases
|
|
11
|
+
Keywords: ai,agents,evaluation,guardrails,opentelemetry,cli
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Environment :: Console
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Classifier: Topic :: Software Development :: Testing
|
|
22
|
+
Requires-Python: >=3.10
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Requires-Dist: httpx<1,>=0.27
|
|
26
|
+
Requires-Dist: jsonschema<5,>=4.18
|
|
27
|
+
Requires-Dist: PyYAML<7,>=6.0
|
|
28
|
+
Requires-Dist: rich<15,>=13
|
|
29
|
+
Requires-Dist: typer<1,>=0.12
|
|
30
|
+
Requires-Dist: tomli>=2.0; python_version < "3.11"
|
|
31
|
+
Provides-Extra: dev
|
|
32
|
+
Requires-Dist: build>=1.2; extra == "dev"
|
|
33
|
+
Requires-Dist: pytest>=8; extra == "dev"
|
|
34
|
+
Requires-Dist: ruff>=0.9; extra == "dev"
|
|
35
|
+
Requires-Dist: twine>=6; extra == "dev"
|
|
36
|
+
Dynamic: license-file
|
|
37
|
+
|
|
38
|
+
# Halios
|
|
39
|
+
|
|
40
|
+
[](https://pypi.org/project/haliosai-cli/)
|
|
41
|
+
[](LICENSE)
|
|
42
|
+
|
|
43
|
+
**Your pair programmer for AI agent evaluations.**
|
|
44
|
+
|
|
45
|
+
Halios helps you build reliable AI agents in minutes. Add the Halios skill to **Codex**, **Claude Code**, **Cursor**, or your favorite coding agent, and create eval suites, run multi-turn simulations, investigate failures, and gate pull requests directly from your repository.
|
|
46
|
+
|
|
47
|
+
Halios provides the evaluation framework, developer tools, and hosted evaluation runtime underneath.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Quickstart
|
|
52
|
+
|
|
53
|
+
### 1. Install the Agent Skill
|
|
54
|
+
|
|
55
|
+
Add the Halios skill to your coding agent environment:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
npx skills add HaliosAI/halios --skill halios
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
*(Works with Codex, Claude Code, Cursor, GitHub Copilot, Gemini CLI, OpenCode, and any harness supporting the open Agent Skills format).*
|
|
62
|
+
|
|
63
|
+
### 2. Ask Your Coding Agent
|
|
64
|
+
|
|
65
|
+
Prompt your agent to set up evaluations for your project:
|
|
66
|
+
|
|
67
|
+
```text
|
|
68
|
+
"Set up evals for this agent: inspect the repository, create realistic test scenarios and checks, and run a baseline."
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Your agent will inspect your application entrypoint, configure standard OpenTelemetry, draft scenarios in `.halios/`, and run baseline evaluations.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
### Standalone CLI Installation
|
|
76
|
+
|
|
77
|
+
If you prefer to drive evaluations directly from the command line or CI:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
# Recommended: Install with uv tool
|
|
81
|
+
uv tool install 'haliosai-cli>=2.0.0'
|
|
82
|
+
|
|
83
|
+
# Or install with pipx
|
|
84
|
+
pipx install 'haliosai-cli>=2.0.0'
|
|
85
|
+
|
|
86
|
+
# See available commands and usage
|
|
87
|
+
halios --help
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## How It Works
|
|
93
|
+
|
|
94
|
+
1. **Inspect & Connect**: Your coding agent inspects your agent's tools, policies, and runtime, then configures standard OpenTelemetry export.
|
|
95
|
+
2. **Author Scenarios & Checks**: Test cases, rubrics, and failure criteria are stored directly in your repository (`.halios/scenarios.yml` and `.halios/eval.yml`). You own the evaluation suite.
|
|
96
|
+
3. **Simulate Multi-Turn Trajectories**: Halios runs fresh simulation passes against your agent across edge cases, tool dependencies, and user personas.
|
|
97
|
+
4. **Investigate & Fix Failures**: Pinpoint hallucinated parameters, broken tool handoffs, or policy violations from complete trace evidence.
|
|
98
|
+
5. **Gate Pull Requests**: Run protected checks in CI to block regressions before merging to production.
|
|
99
|
+
6. **Monitor Production**: Evaluate production traces using the same checks, and turn real-world failures into new regression test scenarios.
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Key Principles
|
|
104
|
+
|
|
105
|
+
- **You own the evaluation suite**: Scenarios and checks are clean YAML files stored in your Git repository.
|
|
106
|
+
- **Fresh simulations, not static replays**: Halios tests real agent execution across multiple turns, rather than replaying outdated completions.
|
|
107
|
+
- **Framework agnostic**: Works with any agent architecture—OpenAI Agents SDK, LangChain, LlamaIndex, PydanticAI, or custom workflows.
|
|
108
|
+
- **Stock OpenTelemetry**: Applications emit standard OpenTelemetry GenAI spans; no proprietary vendor lock-in in your production runtime.
|
|
109
|
+
- **Unified local, CI, and production loop**: The same checks run during local development, gate merge requests in CI, and monitor live production traces.
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Resources
|
|
114
|
+
|
|
115
|
+
- **Website**: [halios.ai](https://halios.ai)
|
|
116
|
+
- **Documentation**: [docs.halios.ai](https://docs.halios.ai)
|
|
117
|
+
- **Python SDK**: [github.com/HaliosAI/haliosai-python-sdk](https://github.com/HaliosAI/haliosai-python-sdk)
|
|
118
|
+
- **Public Skill Source**: [`skills/halios/`](skills/halios/)
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## Development
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
# Clone and install locally in editable mode
|
|
126
|
+
git clone https://github.com/HaliosAI/halios.git
|
|
127
|
+
cd halios
|
|
128
|
+
python -m pip install -e '.[dev]'
|
|
129
|
+
|
|
130
|
+
# Run the test suite
|
|
131
|
+
python -m pytest -q
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## License
|
|
137
|
+
|
|
138
|
+
[Apache 2.0](LICENSE) © Anomalytica Inc. 2026
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# Halios
|
|
2
|
+
|
|
3
|
+
[](https://pypi.org/project/haliosai-cli/)
|
|
4
|
+
[](LICENSE)
|
|
5
|
+
|
|
6
|
+
**Your pair programmer for AI agent evaluations.**
|
|
7
|
+
|
|
8
|
+
Halios helps you build reliable AI agents in minutes. Add the Halios skill to **Codex**, **Claude Code**, **Cursor**, or your favorite coding agent, and create eval suites, run multi-turn simulations, investigate failures, and gate pull requests directly from your repository.
|
|
9
|
+
|
|
10
|
+
Halios provides the evaluation framework, developer tools, and hosted evaluation runtime underneath.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Quickstart
|
|
15
|
+
|
|
16
|
+
### 1. Install the Agent Skill
|
|
17
|
+
|
|
18
|
+
Add the Halios skill to your coding agent environment:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
npx skills add HaliosAI/halios --skill halios
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
*(Works with Codex, Claude Code, Cursor, GitHub Copilot, Gemini CLI, OpenCode, and any harness supporting the open Agent Skills format).*
|
|
25
|
+
|
|
26
|
+
### 2. Ask Your Coding Agent
|
|
27
|
+
|
|
28
|
+
Prompt your agent to set up evaluations for your project:
|
|
29
|
+
|
|
30
|
+
```text
|
|
31
|
+
"Set up evals for this agent: inspect the repository, create realistic test scenarios and checks, and run a baseline."
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Your agent will inspect your application entrypoint, configure standard OpenTelemetry, draft scenarios in `.halios/`, and run baseline evaluations.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
### Standalone CLI Installation
|
|
39
|
+
|
|
40
|
+
If you prefer to drive evaluations directly from the command line or CI:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
# Recommended: Install with uv tool
|
|
44
|
+
uv tool install 'haliosai-cli>=2.0.0'
|
|
45
|
+
|
|
46
|
+
# Or install with pipx
|
|
47
|
+
pipx install 'haliosai-cli>=2.0.0'
|
|
48
|
+
|
|
49
|
+
# See available commands and usage
|
|
50
|
+
halios --help
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## How It Works
|
|
56
|
+
|
|
57
|
+
1. **Inspect & Connect**: Your coding agent inspects your agent's tools, policies, and runtime, then configures standard OpenTelemetry export.
|
|
58
|
+
2. **Author Scenarios & Checks**: Test cases, rubrics, and failure criteria are stored directly in your repository (`.halios/scenarios.yml` and `.halios/eval.yml`). You own the evaluation suite.
|
|
59
|
+
3. **Simulate Multi-Turn Trajectories**: Halios runs fresh simulation passes against your agent across edge cases, tool dependencies, and user personas.
|
|
60
|
+
4. **Investigate & Fix Failures**: Pinpoint hallucinated parameters, broken tool handoffs, or policy violations from complete trace evidence.
|
|
61
|
+
5. **Gate Pull Requests**: Run protected checks in CI to block regressions before merging to production.
|
|
62
|
+
6. **Monitor Production**: Evaluate production traces using the same checks, and turn real-world failures into new regression test scenarios.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## Key Principles
|
|
67
|
+
|
|
68
|
+
- **You own the evaluation suite**: Scenarios and checks are clean YAML files stored in your Git repository.
|
|
69
|
+
- **Fresh simulations, not static replays**: Halios tests real agent execution across multiple turns, rather than replaying outdated completions.
|
|
70
|
+
- **Framework agnostic**: Works with any agent architecture—OpenAI Agents SDK, LangChain, LlamaIndex, PydanticAI, or custom workflows.
|
|
71
|
+
- **Stock OpenTelemetry**: Applications emit standard OpenTelemetry GenAI spans; no proprietary vendor lock-in in your production runtime.
|
|
72
|
+
- **Unified local, CI, and production loop**: The same checks run during local development, gate merge requests in CI, and monitor live production traces.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Resources
|
|
77
|
+
|
|
78
|
+
- **Website**: [halios.ai](https://halios.ai)
|
|
79
|
+
- **Documentation**: [docs.halios.ai](https://docs.halios.ai)
|
|
80
|
+
- **Python SDK**: [github.com/HaliosAI/haliosai-python-sdk](https://github.com/HaliosAI/haliosai-python-sdk)
|
|
81
|
+
- **Public Skill Source**: [`skills/halios/`](skills/halios/)
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## Development
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
# Clone and install locally in editable mode
|
|
89
|
+
git clone https://github.com/HaliosAI/halios.git
|
|
90
|
+
cd halios
|
|
91
|
+
python -m pip install -e '.[dev]'
|
|
92
|
+
|
|
93
|
+
# Run the test suite
|
|
94
|
+
python -m pytest -q
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## License
|
|
100
|
+
|
|
101
|
+
[Apache 2.0](LICENSE) © Anomalytica Inc. 2026
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
__version__ = "2.0.1"
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: haliosai-cli
|
|
3
|
+
Version: 2.0.1
|
|
4
|
+
Summary: Halios CLI for coding-agent evaluations, guardrails, and OpenTelemetry evidence
|
|
5
|
+
Author-email: HaliosLabs <support@halios.ai>
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Project-URL: Homepage, https://halios.ai
|
|
8
|
+
Project-URL: Documentation, https://docs.halios.ai
|
|
9
|
+
Project-URL: Repository, https://github.com/HaliosAI/halios
|
|
10
|
+
Project-URL: Changelog, https://github.com/HaliosAI/halios/releases
|
|
11
|
+
Keywords: ai,agents,evaluation,guardrails,opentelemetry,cli
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Environment :: Console
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Classifier: Topic :: Software Development :: Testing
|
|
22
|
+
Requires-Python: >=3.10
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Requires-Dist: httpx<1,>=0.27
|
|
26
|
+
Requires-Dist: jsonschema<5,>=4.18
|
|
27
|
+
Requires-Dist: PyYAML<7,>=6.0
|
|
28
|
+
Requires-Dist: rich<15,>=13
|
|
29
|
+
Requires-Dist: typer<1,>=0.12
|
|
30
|
+
Requires-Dist: tomli>=2.0; python_version < "3.11"
|
|
31
|
+
Provides-Extra: dev
|
|
32
|
+
Requires-Dist: build>=1.2; extra == "dev"
|
|
33
|
+
Requires-Dist: pytest>=8; extra == "dev"
|
|
34
|
+
Requires-Dist: ruff>=0.9; extra == "dev"
|
|
35
|
+
Requires-Dist: twine>=6; extra == "dev"
|
|
36
|
+
Dynamic: license-file
|
|
37
|
+
|
|
38
|
+
# Halios
|
|
39
|
+
|
|
40
|
+
[](https://pypi.org/project/haliosai-cli/)
|
|
41
|
+
[](LICENSE)
|
|
42
|
+
|
|
43
|
+
**Your pair programmer for AI agent evaluations.**
|
|
44
|
+
|
|
45
|
+
Halios helps you build reliable AI agents in minutes. Add the Halios skill to **Codex**, **Claude Code**, **Cursor**, or your favorite coding agent, and create eval suites, run multi-turn simulations, investigate failures, and gate pull requests directly from your repository.
|
|
46
|
+
|
|
47
|
+
Halios provides the evaluation framework, developer tools, and hosted evaluation runtime underneath.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Quickstart
|
|
52
|
+
|
|
53
|
+
### 1. Install the Agent Skill
|
|
54
|
+
|
|
55
|
+
Add the Halios skill to your coding agent environment:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
npx skills add HaliosAI/halios --skill halios
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
*(Works with Codex, Claude Code, Cursor, GitHub Copilot, Gemini CLI, OpenCode, and any harness supporting the open Agent Skills format).*
|
|
62
|
+
|
|
63
|
+
### 2. Ask Your Coding Agent
|
|
64
|
+
|
|
65
|
+
Prompt your agent to set up evaluations for your project:
|
|
66
|
+
|
|
67
|
+
```text
|
|
68
|
+
"Set up evals for this agent: inspect the repository, create realistic test scenarios and checks, and run a baseline."
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Your agent will inspect your application entrypoint, configure standard OpenTelemetry, draft scenarios in `.halios/`, and run baseline evaluations.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
### Standalone CLI Installation
|
|
76
|
+
|
|
77
|
+
If you prefer to drive evaluations directly from the command line or CI:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
# Recommended: Install with uv tool
|
|
81
|
+
uv tool install 'haliosai-cli>=2.0.0'
|
|
82
|
+
|
|
83
|
+
# Or install with pipx
|
|
84
|
+
pipx install 'haliosai-cli>=2.0.0'
|
|
85
|
+
|
|
86
|
+
# See available commands and usage
|
|
87
|
+
halios --help
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## How It Works
|
|
93
|
+
|
|
94
|
+
1. **Inspect & Connect**: Your coding agent inspects your agent's tools, policies, and runtime, then configures standard OpenTelemetry export.
|
|
95
|
+
2. **Author Scenarios & Checks**: Test cases, rubrics, and failure criteria are stored directly in your repository (`.halios/scenarios.yml` and `.halios/eval.yml`). You own the evaluation suite.
|
|
96
|
+
3. **Simulate Multi-Turn Trajectories**: Halios runs fresh simulation passes against your agent across edge cases, tool dependencies, and user personas.
|
|
97
|
+
4. **Investigate & Fix Failures**: Pinpoint hallucinated parameters, broken tool handoffs, or policy violations from complete trace evidence.
|
|
98
|
+
5. **Gate Pull Requests**: Run protected checks in CI to block regressions before merging to production.
|
|
99
|
+
6. **Monitor Production**: Evaluate production traces using the same checks, and turn real-world failures into new regression test scenarios.
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Key Principles
|
|
104
|
+
|
|
105
|
+
- **You own the evaluation suite**: Scenarios and checks are clean YAML files stored in your Git repository.
|
|
106
|
+
- **Fresh simulations, not static replays**: Halios tests real agent execution across multiple turns, rather than replaying outdated completions.
|
|
107
|
+
- **Framework agnostic**: Works with any agent architecture—OpenAI Agents SDK, LangChain, LlamaIndex, PydanticAI, or custom workflows.
|
|
108
|
+
- **Stock OpenTelemetry**: Applications emit standard OpenTelemetry GenAI spans; no proprietary vendor lock-in in your production runtime.
|
|
109
|
+
- **Unified local, CI, and production loop**: The same checks run during local development, gate merge requests in CI, and monitor live production traces.
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Resources
|
|
114
|
+
|
|
115
|
+
- **Website**: [halios.ai](https://halios.ai)
|
|
116
|
+
- **Documentation**: [docs.halios.ai](https://docs.halios.ai)
|
|
117
|
+
- **Python SDK**: [github.com/HaliosAI/haliosai-python-sdk](https://github.com/HaliosAI/haliosai-python-sdk)
|
|
118
|
+
- **Public Skill Source**: [`skills/halios/`](skills/halios/)
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## Development
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
# Clone and install locally in editable mode
|
|
126
|
+
git clone https://github.com/HaliosAI/halios.git
|
|
127
|
+
cd halios
|
|
128
|
+
python -m pip install -e '.[dev]'
|
|
129
|
+
|
|
130
|
+
# Run the test suite
|
|
131
|
+
python -m pytest -q
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## License
|
|
137
|
+
|
|
138
|
+
[Apache 2.0](LICENSE) © Anomalytica Inc. 2026
|
haliosai_cli-2.0.0/PKG-INFO
DELETED
|
@@ -1,101 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: haliosai-cli
|
|
3
|
-
Version: 2.0.0
|
|
4
|
-
Summary: Halios CLI for coding-agent evaluations, guardrails, and OpenTelemetry evidence
|
|
5
|
-
Author-email: HaliosLabs <support@halios.ai>
|
|
6
|
-
License-Expression: Apache-2.0
|
|
7
|
-
Project-URL: Homepage, https://halios.ai
|
|
8
|
-
Project-URL: Documentation, https://docs.halios.ai
|
|
9
|
-
Project-URL: Repository, https://github.com/HaliosAI/halios
|
|
10
|
-
Project-URL: Changelog, https://github.com/HaliosAI/halios/releases
|
|
11
|
-
Keywords: ai,agents,evaluation,guardrails,opentelemetry,cli
|
|
12
|
-
Classifier: Development Status :: 4 - Beta
|
|
13
|
-
Classifier: Environment :: Console
|
|
14
|
-
Classifier: Intended Audience :: Developers
|
|
15
|
-
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
-
Classifier: Topic :: Software Development :: Testing
|
|
22
|
-
Requires-Python: >=3.10
|
|
23
|
-
Description-Content-Type: text/markdown
|
|
24
|
-
License-File: LICENSE
|
|
25
|
-
Requires-Dist: httpx<1,>=0.27
|
|
26
|
-
Requires-Dist: jsonschema<5,>=4.18
|
|
27
|
-
Requires-Dist: PyYAML<7,>=6.0
|
|
28
|
-
Requires-Dist: rich<15,>=13
|
|
29
|
-
Requires-Dist: typer<1,>=0.12
|
|
30
|
-
Requires-Dist: tomli>=2.0; python_version < "3.11"
|
|
31
|
-
Provides-Extra: dev
|
|
32
|
-
Requires-Dist: build>=1.2; extra == "dev"
|
|
33
|
-
Requires-Dist: pytest>=8; extra == "dev"
|
|
34
|
-
Requires-Dist: ruff>=0.9; extra == "dev"
|
|
35
|
-
Requires-Dist: twine>=6; extra == "dev"
|
|
36
|
-
Dynamic: license-file
|
|
37
|
-
|
|
38
|
-
# Halios CLI and Agent Skill
|
|
39
|
-
|
|
40
|
-
The `halios` command is the machine-facing client used by the Halios Agent Skill and CI. It owns
|
|
41
|
-
repository setup, evaluation-suite authoring, fresh scenario execution, evidence inspection, and
|
|
42
|
-
prompt-optimization handoffs. Applications export their runtime telemetry with stock
|
|
43
|
-
OpenTelemetry; the CLI is not an application tracing SDK.
|
|
44
|
-
|
|
45
|
-
```bash
|
|
46
|
-
uv tool install 'haliosai-cli>=2.0.0'
|
|
47
|
-
halios --version
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
Use `pipx install 'haliosai-cli>=2.0.0'` when `uv` is unavailable. Install the cross-harness skill
|
|
51
|
-
separately:
|
|
52
|
-
|
|
53
|
-
```bash
|
|
54
|
-
npx skills add HaliosAI/halios --skill halios
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
The stable project workflow is:
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
halios auth login
|
|
61
|
-
halios project init --agent support-assistant --command "python tests/halios_adapter.py"
|
|
62
|
-
halios eval review --json
|
|
63
|
-
halios project configure --json
|
|
64
|
-
halios project check --json
|
|
65
|
-
halios eval run -k 3 --json
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
Prompt optimization is driven by the coding agent and the normal immutable eval workflow:
|
|
69
|
-
|
|
70
|
-
```bash
|
|
71
|
-
halios optimize start --baseline-run <run-id> --prompt-file path/to/system-prompt.txt --json
|
|
72
|
-
# make one bounded prompt edit from the returned guidance, then run the unchanged suite
|
|
73
|
-
halios eval run -k 3 --json
|
|
74
|
-
halios optimize record <optimization-run-id> \
|
|
75
|
-
--evaluation-run <candidate-eval-run-id> \
|
|
76
|
-
--prompt-file path/to/system-prompt.txt \
|
|
77
|
-
--json
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
Credentials are stored outside the repository. The repository contains only `.halios/config.toml`,
|
|
81
|
-
`.halios/eval.yml`, and `.halios/scenarios.yml`.
|
|
82
|
-
|
|
83
|
-
In non-interactive CI, use `HALIOS_API_KEY` for control-plane commands and the separate
|
|
84
|
-
agent-scoped `HALIOS_OTLP_TOKEN` for adapter trace export. Only trusted default-branch jobs should
|
|
85
|
-
receive `HALIOS_CI_PUBLISH_TOKEN`. The jsonl adapter is used by local/CI simulation only; deployed
|
|
86
|
-
staging and production applications export their own stock OpenTelemetry directly to Halios.
|
|
87
|
-
|
|
88
|
-
See the repository's
|
|
89
|
-
[`skills/halios`](https://github.com/HaliosAI/halios/tree/main/skills/halios) workflow for
|
|
90
|
-
coding-agent setup and
|
|
91
|
-
[`OTEL_SCHEMA.md`](https://github.com/HaliosAI/haliosai-python-sdk/blob/main/OTEL_SCHEMA.md)
|
|
92
|
-
for the telemetry accepted by Halios.
|
|
93
|
-
|
|
94
|
-
## Development
|
|
95
|
-
|
|
96
|
-
```bash
|
|
97
|
-
python -m pip install -e '.[dev]'
|
|
98
|
-
python -m pytest -q
|
|
99
|
-
```
|
|
100
|
-
|
|
101
|
-
See [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution and release process.
|
haliosai_cli-2.0.0/README.md
DELETED
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
# Halios CLI and Agent Skill
|
|
2
|
-
|
|
3
|
-
The `halios` command is the machine-facing client used by the Halios Agent Skill and CI. It owns
|
|
4
|
-
repository setup, evaluation-suite authoring, fresh scenario execution, evidence inspection, and
|
|
5
|
-
prompt-optimization handoffs. Applications export their runtime telemetry with stock
|
|
6
|
-
OpenTelemetry; the CLI is not an application tracing SDK.
|
|
7
|
-
|
|
8
|
-
```bash
|
|
9
|
-
uv tool install 'haliosai-cli>=2.0.0'
|
|
10
|
-
halios --version
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
Use `pipx install 'haliosai-cli>=2.0.0'` when `uv` is unavailable. Install the cross-harness skill
|
|
14
|
-
separately:
|
|
15
|
-
|
|
16
|
-
```bash
|
|
17
|
-
npx skills add HaliosAI/halios --skill halios
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
The stable project workflow is:
|
|
21
|
-
|
|
22
|
-
```bash
|
|
23
|
-
halios auth login
|
|
24
|
-
halios project init --agent support-assistant --command "python tests/halios_adapter.py"
|
|
25
|
-
halios eval review --json
|
|
26
|
-
halios project configure --json
|
|
27
|
-
halios project check --json
|
|
28
|
-
halios eval run -k 3 --json
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
Prompt optimization is driven by the coding agent and the normal immutable eval workflow:
|
|
32
|
-
|
|
33
|
-
```bash
|
|
34
|
-
halios optimize start --baseline-run <run-id> --prompt-file path/to/system-prompt.txt --json
|
|
35
|
-
# make one bounded prompt edit from the returned guidance, then run the unchanged suite
|
|
36
|
-
halios eval run -k 3 --json
|
|
37
|
-
halios optimize record <optimization-run-id> \
|
|
38
|
-
--evaluation-run <candidate-eval-run-id> \
|
|
39
|
-
--prompt-file path/to/system-prompt.txt \
|
|
40
|
-
--json
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
Credentials are stored outside the repository. The repository contains only `.halios/config.toml`,
|
|
44
|
-
`.halios/eval.yml`, and `.halios/scenarios.yml`.
|
|
45
|
-
|
|
46
|
-
In non-interactive CI, use `HALIOS_API_KEY` for control-plane commands and the separate
|
|
47
|
-
agent-scoped `HALIOS_OTLP_TOKEN` for adapter trace export. Only trusted default-branch jobs should
|
|
48
|
-
receive `HALIOS_CI_PUBLISH_TOKEN`. The jsonl adapter is used by local/CI simulation only; deployed
|
|
49
|
-
staging and production applications export their own stock OpenTelemetry directly to Halios.
|
|
50
|
-
|
|
51
|
-
See the repository's
|
|
52
|
-
[`skills/halios`](https://github.com/HaliosAI/halios/tree/main/skills/halios) workflow for
|
|
53
|
-
coding-agent setup and
|
|
54
|
-
[`OTEL_SCHEMA.md`](https://github.com/HaliosAI/haliosai-python-sdk/blob/main/OTEL_SCHEMA.md)
|
|
55
|
-
for the telemetry accepted by Halios.
|
|
56
|
-
|
|
57
|
-
## Development
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
python -m pip install -e '.[dev]'
|
|
61
|
-
python -m pytest -q
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
See [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution and release process.
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
__version__ = "2.0.0"
|
|
@@ -1,101 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: haliosai-cli
|
|
3
|
-
Version: 2.0.0
|
|
4
|
-
Summary: Halios CLI for coding-agent evaluations, guardrails, and OpenTelemetry evidence
|
|
5
|
-
Author-email: HaliosLabs <support@halios.ai>
|
|
6
|
-
License-Expression: Apache-2.0
|
|
7
|
-
Project-URL: Homepage, https://halios.ai
|
|
8
|
-
Project-URL: Documentation, https://docs.halios.ai
|
|
9
|
-
Project-URL: Repository, https://github.com/HaliosAI/halios
|
|
10
|
-
Project-URL: Changelog, https://github.com/HaliosAI/halios/releases
|
|
11
|
-
Keywords: ai,agents,evaluation,guardrails,opentelemetry,cli
|
|
12
|
-
Classifier: Development Status :: 4 - Beta
|
|
13
|
-
Classifier: Environment :: Console
|
|
14
|
-
Classifier: Intended Audience :: Developers
|
|
15
|
-
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
-
Classifier: Topic :: Software Development :: Testing
|
|
22
|
-
Requires-Python: >=3.10
|
|
23
|
-
Description-Content-Type: text/markdown
|
|
24
|
-
License-File: LICENSE
|
|
25
|
-
Requires-Dist: httpx<1,>=0.27
|
|
26
|
-
Requires-Dist: jsonschema<5,>=4.18
|
|
27
|
-
Requires-Dist: PyYAML<7,>=6.0
|
|
28
|
-
Requires-Dist: rich<15,>=13
|
|
29
|
-
Requires-Dist: typer<1,>=0.12
|
|
30
|
-
Requires-Dist: tomli>=2.0; python_version < "3.11"
|
|
31
|
-
Provides-Extra: dev
|
|
32
|
-
Requires-Dist: build>=1.2; extra == "dev"
|
|
33
|
-
Requires-Dist: pytest>=8; extra == "dev"
|
|
34
|
-
Requires-Dist: ruff>=0.9; extra == "dev"
|
|
35
|
-
Requires-Dist: twine>=6; extra == "dev"
|
|
36
|
-
Dynamic: license-file
|
|
37
|
-
|
|
38
|
-
# Halios CLI and Agent Skill
|
|
39
|
-
|
|
40
|
-
The `halios` command is the machine-facing client used by the Halios Agent Skill and CI. It owns
|
|
41
|
-
repository setup, evaluation-suite authoring, fresh scenario execution, evidence inspection, and
|
|
42
|
-
prompt-optimization handoffs. Applications export their runtime telemetry with stock
|
|
43
|
-
OpenTelemetry; the CLI is not an application tracing SDK.
|
|
44
|
-
|
|
45
|
-
```bash
|
|
46
|
-
uv tool install 'haliosai-cli>=2.0.0'
|
|
47
|
-
halios --version
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
Use `pipx install 'haliosai-cli>=2.0.0'` when `uv` is unavailable. Install the cross-harness skill
|
|
51
|
-
separately:
|
|
52
|
-
|
|
53
|
-
```bash
|
|
54
|
-
npx skills add HaliosAI/halios --skill halios
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
The stable project workflow is:
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
halios auth login
|
|
61
|
-
halios project init --agent support-assistant --command "python tests/halios_adapter.py"
|
|
62
|
-
halios eval review --json
|
|
63
|
-
halios project configure --json
|
|
64
|
-
halios project check --json
|
|
65
|
-
halios eval run -k 3 --json
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
Prompt optimization is driven by the coding agent and the normal immutable eval workflow:
|
|
69
|
-
|
|
70
|
-
```bash
|
|
71
|
-
halios optimize start --baseline-run <run-id> --prompt-file path/to/system-prompt.txt --json
|
|
72
|
-
# make one bounded prompt edit from the returned guidance, then run the unchanged suite
|
|
73
|
-
halios eval run -k 3 --json
|
|
74
|
-
halios optimize record <optimization-run-id> \
|
|
75
|
-
--evaluation-run <candidate-eval-run-id> \
|
|
76
|
-
--prompt-file path/to/system-prompt.txt \
|
|
77
|
-
--json
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
Credentials are stored outside the repository. The repository contains only `.halios/config.toml`,
|
|
81
|
-
`.halios/eval.yml`, and `.halios/scenarios.yml`.
|
|
82
|
-
|
|
83
|
-
In non-interactive CI, use `HALIOS_API_KEY` for control-plane commands and the separate
|
|
84
|
-
agent-scoped `HALIOS_OTLP_TOKEN` for adapter trace export. Only trusted default-branch jobs should
|
|
85
|
-
receive `HALIOS_CI_PUBLISH_TOKEN`. The jsonl adapter is used by local/CI simulation only; deployed
|
|
86
|
-
staging and production applications export their own stock OpenTelemetry directly to Halios.
|
|
87
|
-
|
|
88
|
-
See the repository's
|
|
89
|
-
[`skills/halios`](https://github.com/HaliosAI/halios/tree/main/skills/halios) workflow for
|
|
90
|
-
coding-agent setup and
|
|
91
|
-
[`OTEL_SCHEMA.md`](https://github.com/HaliosAI/haliosai-python-sdk/blob/main/OTEL_SCHEMA.md)
|
|
92
|
-
for the telemetry accepted by Halios.
|
|
93
|
-
|
|
94
|
-
## Development
|
|
95
|
-
|
|
96
|
-
```bash
|
|
97
|
-
python -m pip install -e '.[dev]'
|
|
98
|
-
python -m pytest -q
|
|
99
|
-
```
|
|
100
|
-
|
|
101
|
-
See [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution and release process.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|