cit-ai-model-simulator 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cit_ai_model_simulator-0.1.0/CHANGELOG.md +9 -0
- cit_ai_model_simulator-0.1.0/MANIFEST.in +8 -0
- cit_ai_model_simulator-0.1.0/PKG-INFO +107 -0
- cit_ai_model_simulator-0.1.0/README.md +87 -0
- cit_ai_model_simulator-0.1.0/TECHNICAL_SPECIFICATION.md +908 -0
- cit_ai_model_simulator-0.1.0/pyproject.toml +44 -0
- cit_ai_model_simulator-0.1.0/setup.cfg +4 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/__init__.py +6 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/__main__.py +6 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/api.py +518 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/cli.py +325 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/engine.py +586 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/evidence.py +126 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/scenario.py +296 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/scenarios/__init__.py +2 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/scenarios/lab01-baseline.yaml +76 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator/schemas/scenario-v1.json +105 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator.egg-info/PKG-INFO +107 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator.egg-info/SOURCES.txt +25 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator.egg-info/dependency_links.txt +1 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator.egg-info/entry_points.txt +2 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator.egg-info/requires.txt +2 -0
- cit_ai_model_simulator-0.1.0/src/cit_ai_model_simulator.egg-info/top_level.txt +1 -0
- cit_ai_model_simulator-0.1.0/tests/test_api.py +228 -0
- cit_ai_model_simulator-0.1.0/tests/test_cli.py +41 -0
- cit_ai_model_simulator-0.1.0/tests/test_engine.py +140 -0
- cit_ai_model_simulator-0.1.0/tests/test_scenario.py +79 -0
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0 - Unreleased
|
|
4
|
+
|
|
5
|
+
- Implement the M0/M1 CIT 495 simulator baseline.
|
|
6
|
+
- Add deterministic health, model discovery, synchronous chat, and SSE streaming.
|
|
7
|
+
- Add session creation, reset, approximate token accounting, simulated timing, and JSONL evidence.
|
|
8
|
+
- Add scenario validation, preview, self-test, reset, and evidence CLI operations.
|
|
9
|
+
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
include README.md
|
|
2
|
+
include CHANGELOG.md
|
|
3
|
+
include TECHNICAL_SPECIFICATION.md
|
|
4
|
+
recursive-include src/cit_ai_model_simulator/scenarios *.yaml
|
|
5
|
+
recursive-include src/cit_ai_model_simulator/schemas *.json
|
|
6
|
+
recursive-include tests *.py
|
|
7
|
+
global-exclude __pycache__ *.py[cod]
|
|
8
|
+
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: cit-ai-model-simulator
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Deterministic local model-service simulator for CIT courses
|
|
5
|
+
Author: Jeffrey Myers II, M.S.
|
|
6
|
+
Project-URL: Documentation, https://pypi.org/project/cit-ai-model-simulator/
|
|
7
|
+
Keywords: education,simulation,llama.cpp,openai-compatible,testing
|
|
8
|
+
Classifier: Development Status :: 3 - Alpha
|
|
9
|
+
Classifier: Environment :: Console
|
|
10
|
+
Classifier: Intended Audience :: Education
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
14
|
+
Classifier: Topic :: Education
|
|
15
|
+
Classifier: Topic :: Software Development :: Testing
|
|
16
|
+
Requires-Python: >=3.10
|
|
17
|
+
Description-Content-Type: text/markdown
|
|
18
|
+
Requires-Dist: jsonschema<5,>=4.23
|
|
19
|
+
Requires-Dist: PyYAML<7,>=6.0.2
|
|
20
|
+
|
|
21
|
+
# CIT AI Model Simulator
|
|
22
|
+
|
|
23
|
+
The CIT AI Model Simulator is a deterministic local service for CIT courses. It gives students who cannot run an approved local model a hardware-neutral fallback, and it gives students and instructors reproducible conditions for testing model-client behavior. It presents the small OpenAI-compatible HTTP surface used by the course without claiming to be an AI model or a complete `llama.cpp` replacement.
|
|
24
|
+
|
|
25
|
+
Responses come from versioned scenario state graphs rather than unrestricted text generation. The simulator labels its identity and all simulated timing and token evidence, listens only on the local computer by default, does not execute model-requested tools, and writes append-only evidence logs for debugging and course work.
|
|
26
|
+
|
|
27
|
+
This initial `0.1.0` implementation targets the specification's M0/M1 Week 1 baseline. Later-lab capabilities such as structured-output faults, tool orchestration, context shifting, compaction, protected validation, and evaluation reports remain deliberately deferred.
|
|
28
|
+
|
|
29
|
+
## Install for development
|
|
30
|
+
|
|
31
|
+
```console
|
|
32
|
+
python -m venv .venv-dev
|
|
33
|
+
.venv-dev\Scripts\python -m pip install -e .
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
On macOS or Linux, use `.venv-dev/bin/python` instead.
|
|
37
|
+
|
|
38
|
+
## Start the simulator
|
|
39
|
+
|
|
40
|
+
```console
|
|
41
|
+
cit-simulator serve
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The default service URL is `http://127.0.0.1:8081`. Startup output identifies the backend, scenario, model alias, URL, and evidence directory.
|
|
45
|
+
|
|
46
|
+
Useful commands:
|
|
47
|
+
|
|
48
|
+
```console
|
|
49
|
+
cit-simulator --help
|
|
50
|
+
cit-simulator validate
|
|
51
|
+
cit-simulator preview
|
|
52
|
+
cit-simulator test
|
|
53
|
+
cit-simulator serve --port 8081 --acceleration 20
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Week 1 API
|
|
57
|
+
|
|
58
|
+
Learner-facing endpoints:
|
|
59
|
+
|
|
60
|
+
- `GET /health`
|
|
61
|
+
- `GET /v1/health`
|
|
62
|
+
- `GET /v1/models`
|
|
63
|
+
- `POST /v1/chat/completions`
|
|
64
|
+
- `POST /v1/chat/completions/input_tokens`
|
|
65
|
+
|
|
66
|
+
Simulator control endpoints:
|
|
67
|
+
|
|
68
|
+
- `GET /sim/v1/info`
|
|
69
|
+
- `POST /sim/v1/sessions`
|
|
70
|
+
- `GET /sim/v1/sessions/{id}`
|
|
71
|
+
- `POST /sim/v1/sessions/{id}/reset`
|
|
72
|
+
- `GET /sim/v1/sessions/{id}/evidence`
|
|
73
|
+
|
|
74
|
+
Create a session before a reproducible run:
|
|
75
|
+
|
|
76
|
+
```console
|
|
77
|
+
curl -X POST http://127.0.0.1:8081/sim/v1/sessions \
|
|
78
|
+
-H "Content-Type: application/json" \
|
|
79
|
+
-d "{\"seed\": 49501, \"attempt_number\": 1}"
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Pass the returned session ID in the `X-CIT-Sim-Session` request header. A chat request without that header receives a newly created session ID in the response header, which is convenient for simple compatibility checks but should not be used for graded multi-request attempts.
|
|
83
|
+
|
|
84
|
+
## Scenario data
|
|
85
|
+
|
|
86
|
+
The built-in `lab01-baseline` scenario is YAML data packaged separately from the HTTP adapter and engine. Supply another scenario with:
|
|
87
|
+
|
|
88
|
+
```console
|
|
89
|
+
cit-simulator serve --scenario path/to/scenario.yaml
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Validate it before use:
|
|
93
|
+
|
|
94
|
+
```console
|
|
95
|
+
cit-simulator validate path/to/scenario.yaml
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## Evidence and privacy
|
|
99
|
+
|
|
100
|
+
The server writes one append-only JSON Lines file per session beneath the selected evidence directory. Logs contain structural summaries, counts, identifiers, and hashes rather than raw prompts or credentials. Evidence fields explicitly label the backend as `simulator`, token counts as approximate, and timing as simulated.
|
|
101
|
+
|
|
102
|
+
Do not treat simulator TTFT, TPS, or token estimates as measurements of a student's hardware or of a real model.
|
|
103
|
+
|
|
104
|
+
## Specification
|
|
105
|
+
|
|
106
|
+
The architecture and behavioral authority is [TECHNICAL_SPECIFICATION.md](TECHNICAL_SPECIFICATION.md). The implementation must not silently weaken its observable contract. Features beyond M1 are future milestones unless the changelog states otherwise.
|
|
107
|
+
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# CIT AI Model Simulator
|
|
2
|
+
|
|
3
|
+
The CIT AI Model Simulator is a deterministic local service for CIT courses. It gives students who cannot run an approved local model a hardware-neutral fallback, and it gives students and instructors reproducible conditions for testing model-client behavior. It presents the small OpenAI-compatible HTTP surface used by the course without claiming to be an AI model or a complete `llama.cpp` replacement.
|
|
4
|
+
|
|
5
|
+
Responses come from versioned scenario state graphs rather than unrestricted text generation. The simulator labels its identity and all simulated timing and token evidence, listens only on the local computer by default, does not execute model-requested tools, and writes append-only evidence logs for debugging and course work.
|
|
6
|
+
|
|
7
|
+
This initial `0.1.0` implementation targets the specification's M0/M1 Week 1 baseline. Later-lab capabilities such as structured-output faults, tool orchestration, context shifting, compaction, protected validation, and evaluation reports remain deliberately deferred.
|
|
8
|
+
|
|
9
|
+
## Install for development
|
|
10
|
+
|
|
11
|
+
```console
|
|
12
|
+
python -m venv .venv-dev
|
|
13
|
+
.venv-dev\Scripts\python -m pip install -e .
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
On macOS or Linux, use `.venv-dev/bin/python` instead.
|
|
17
|
+
|
|
18
|
+
## Start the simulator
|
|
19
|
+
|
|
20
|
+
```console
|
|
21
|
+
cit-simulator serve
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
The default service URL is `http://127.0.0.1:8081`. Startup output identifies the backend, scenario, model alias, URL, and evidence directory.
|
|
25
|
+
|
|
26
|
+
Useful commands:
|
|
27
|
+
|
|
28
|
+
```console
|
|
29
|
+
cit-simulator --help
|
|
30
|
+
cit-simulator validate
|
|
31
|
+
cit-simulator preview
|
|
32
|
+
cit-simulator test
|
|
33
|
+
cit-simulator serve --port 8081 --acceleration 20
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Week 1 API
|
|
37
|
+
|
|
38
|
+
Learner-facing endpoints:
|
|
39
|
+
|
|
40
|
+
- `GET /health`
|
|
41
|
+
- `GET /v1/health`
|
|
42
|
+
- `GET /v1/models`
|
|
43
|
+
- `POST /v1/chat/completions`
|
|
44
|
+
- `POST /v1/chat/completions/input_tokens`
|
|
45
|
+
|
|
46
|
+
Simulator control endpoints:
|
|
47
|
+
|
|
48
|
+
- `GET /sim/v1/info`
|
|
49
|
+
- `POST /sim/v1/sessions`
|
|
50
|
+
- `GET /sim/v1/sessions/{id}`
|
|
51
|
+
- `POST /sim/v1/sessions/{id}/reset`
|
|
52
|
+
- `GET /sim/v1/sessions/{id}/evidence`
|
|
53
|
+
|
|
54
|
+
Create a session before a reproducible run:
|
|
55
|
+
|
|
56
|
+
```console
|
|
57
|
+
curl -X POST http://127.0.0.1:8081/sim/v1/sessions \
|
|
58
|
+
-H "Content-Type: application/json" \
|
|
59
|
+
-d "{\"seed\": 49501, \"attempt_number\": 1}"
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Pass the returned session ID in the `X-CIT-Sim-Session` request header. A chat request without that header receives a newly created session ID in the response header, which is convenient for simple compatibility checks but should not be used for graded multi-request attempts.
|
|
63
|
+
|
|
64
|
+
## Scenario data
|
|
65
|
+
|
|
66
|
+
The built-in `lab01-baseline` scenario is YAML data packaged separately from the HTTP adapter and engine. Supply another scenario with:
|
|
67
|
+
|
|
68
|
+
```console
|
|
69
|
+
cit-simulator serve --scenario path/to/scenario.yaml
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Validate it before use:
|
|
73
|
+
|
|
74
|
+
```console
|
|
75
|
+
cit-simulator validate path/to/scenario.yaml
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Evidence and privacy
|
|
79
|
+
|
|
80
|
+
The server writes one append-only JSON Lines file per session beneath the selected evidence directory. Logs contain structural summaries, counts, identifiers, and hashes rather than raw prompts or credentials. Evidence fields explicitly label the backend as `simulator`, token counts as approximate, and timing as simulated.
|
|
81
|
+
|
|
82
|
+
Do not treat simulator TTFT, TPS, or token estimates as measurements of a student's hardware or of a real model.
|
|
83
|
+
|
|
84
|
+
## Specification
|
|
85
|
+
|
|
86
|
+
The architecture and behavioral authority is [TECHNICAL_SPECIFICATION.md](TECHNICAL_SPECIFICATION.md). The implementation must not silently weaken its observable contract. Features beyond M1 are future milestones unless the changelog states otherwise.
|
|
87
|
+
|