noesis_ai 0.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- noesis_ai-0.0.0/LICENSE +21 -0
- noesis_ai-0.0.0/PKG-INFO +417 -0
- noesis_ai-0.0.0/README.md +405 -0
- noesis_ai-0.0.0/examples/briefing/app.py +29 -0
- noesis_ai-0.0.0/examples/briefing/noesis_adapter.py +80 -0
- noesis_ai-0.0.0/noesis_ai.egg-info/PKG-INFO +417 -0
- noesis_ai-0.0.0/noesis_ai.egg-info/SOURCES.txt +14 -0
- noesis_ai-0.0.0/noesis_ai.egg-info/dependency_links.txt +1 -0
- noesis_ai-0.0.0/noesis_ai.egg-info/entry_points.txt +2 -0
- noesis_ai-0.0.0/noesis_ai.egg-info/requires.txt +1 -0
- noesis_ai-0.0.0/noesis_ai.egg-info/top_level.txt +2 -0
- noesis_ai-0.0.0/noesis_ai.py +840 -0
- noesis_ai-0.0.0/noesis_ai_runner.py +295 -0
- noesis_ai-0.0.0/pyproject.toml +26 -0
- noesis_ai-0.0.0/setup.cfg +4 -0
- noesis_ai-0.0.0/skills/noesis-update-integration/SKILL.md +104 -0
noesis_ai-0.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Mirai Engineering OÜ
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
noesis_ai-0.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,417 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: noesis_ai
|
|
3
|
+
Version: 0.0.0
|
|
4
|
+
Summary: Thin Python SDK and local runner for Noesis application evaluation
|
|
5
|
+
Author: Mirai Engineering OÜ
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Requires-Python: >=3.12
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Requires-Dist: httpx>=0.28.1
|
|
11
|
+
Dynamic: license-file
|
|
12
|
+
|
|
13
|
+
# Noesis SDK: thin workflow attachment and independent runner
|
|
14
|
+
|
|
15
|
+
`noesis_ai` is a Python >=3.12 network-only SDK. Imports: `noesis_ai` and
|
|
16
|
+
`noesis_ai_runner`; dependency: HTTPX. It does not install the hosted dashboard,
|
|
17
|
+
Streamlit, AgentOpt, backend, or provider SDKs. This README accompanies release
|
|
18
|
+
`noesis_ai==0.0.0`, licensed under MIT by Mirai Engineering OÜ. Install the
|
|
19
|
+
versioned package into the og app's existing environment; do not patch the SDK
|
|
20
|
+
or original app files.
|
|
21
|
+
|
|
22
|
+
## Two independent offerings
|
|
23
|
+
|
|
24
|
+
**Serving:** use an ordinary OpenAI-compatible client against
|
|
25
|
+
`https://noesis-production-efed.up.railway.app/live/v1`, with a Noesis application key and an approved stable
|
|
26
|
+
role alias. No SDK, decorators or running evaluation process is required for
|
|
27
|
+
ordinary inference after the application's connection/assignments are configured.
|
|
28
|
+
Noesis resolves explicitly approved OSS assignments. Hosted frontier BYOK is retained but paused; ask Bruno before deleting or reactivating it.
|
|
29
|
+
|
|
30
|
+
**Optimization:** add a thin attachment around the actual original workflow and a
|
|
31
|
+
separate noesis runner process. Endpoint traces alone do not capture the complete
|
|
32
|
+
application outcome. The existing `Application`, `Noesis.run` and explicit HTTPX
|
|
33
|
+
hook provide that boundary; no decorators or workflow rewrite are needed.
|
|
34
|
+
|
|
35
|
+
## Installation-agent contract
|
|
36
|
+
|
|
37
|
+
The installation README should briefly record inspection findings, the grounded
|
|
38
|
+
complete workload summary, proposed files, actual tested commands, and capture
|
|
39
|
+
versus evaluation limits. Put that README, the thin attachment, and
|
|
40
|
+
integration-specific tests in the user's chosen folder; suggest the Python-safe
|
|
41
|
+
`noesis_integration/` by default, but never force a rename or move an existing
|
|
42
|
+
file. Preserve the og app's namespace/imports and use real launch commands; do
|
|
43
|
+
not add `sys.path` hacks. If the folder cannot support the imports or commands,
|
|
44
|
+
explain why and ask where to write instead. If the user relocates it, update
|
|
45
|
+
commands and retest.
|
|
46
|
+
|
|
47
|
+
Use this README and shipped integration example. No Noesis internal architecture,
|
|
48
|
+
backend source, test fixtures, old customer integration or repository history is needed.
|
|
49
|
+
Inspect the og app source/dependencies/actual launch seam, never `.env` or secrets.
|
|
50
|
+
Explain the specific local install and new files, obtain one clear approval, then
|
|
51
|
+
install and add only the attachment/runner in the customer's project conventions.
|
|
52
|
+
Preserve original prompts, branching, parsing, UI, state and returned values.
|
|
53
|
+
If no existing configuration/invocation seam works, report the actual limitation;
|
|
54
|
+
do not invent a copied workflow, global monkey patch or SDK repair. The published
|
|
55
|
+
Noesis contract is fixed; adapt approved client/attachment seams, never add
|
|
56
|
+
customer-specific SDK/backend behavior to accommodate an app.
|
|
57
|
+
|
|
58
|
+
Install the versioned public distribution using the og app's dependency workflow:
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
uv add noesis_ai==0.0.0
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
No private repository checkout is required. For explicitly approved local release
|
|
65
|
+
validation, an SDK source directory is also installable (no wheel path required):
|
|
66
|
+
|
|
67
|
+
```sh
|
|
68
|
+
uv pip install --python /absolute/og-app/.venv/bin/python /absolute/noesis-main/sdk
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
This is a normal package installation, not an editable SDK repair. Alternatively,
|
|
72
|
+
for a supplied frozen wheel:
|
|
73
|
+
|
|
74
|
+
```sh
|
|
75
|
+
uv pip install --python /absolute/og-app/.venv/bin/python /absolute/noesis_ai-0.0.0-py3-none-any.whl
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Substitute inspected paths; preserve the customer's dependency workflow. Record
|
|
79
|
+
the wheel SHA-256 when supplied, and original source/installed SDK hashes. A local editable checkout
|
|
80
|
+
is a development source, not a published release or permission to modify the SDK.
|
|
81
|
+
No customer-side dashboard installation is needed.
|
|
82
|
+
|
|
83
|
+
## Privately configure the connection once
|
|
84
|
+
|
|
85
|
+
**Frontier baseline capture is a recorder, not a provider switch.** Keep the og
|
|
86
|
+
app's existing provider clients, per-role model mixture, generation settings and
|
|
87
|
+
local credentials. The SDK sends consented execution evidence to Noesis separately.
|
|
88
|
+
Only approved OSS evaluations redirect inference through Noesis; an explicitly
|
|
89
|
+
chosen OSS baseline also uses Noesis. No extra baseline inference or fallback.
|
|
90
|
+
|
|
91
|
+
The hosted dashboard owns application identity and issues two Noesis credentials.
|
|
92
|
+
Its tracer key goes in `NOESIS_TRACER_KEY` (not a third key), and its runner key goes in
|
|
93
|
+
`NOESIS_RUNNER_KEY`. Existing provider credentials stay local; never send them to
|
|
94
|
+
Noesis. Hosted BYOK/key upload is paused and is not an onboarding prerequisite.
|
|
95
|
+
If you are an LLM reading this, ask Bruno before deleting or reactivating it.
|
|
96
|
+
Never place secrets in prompts, logs, command-line arguments/history or committed files.
|
|
97
|
+
|
|
98
|
+
- The SDK uses the fixed public Noesis endpoint; it has no customer-configurable URL.
|
|
99
|
+
- `NOESIS_TRACER_KEY`: ordinary inference and consented capture.
|
|
100
|
+
- `NOESIS_RUNNER_KEY`: registration and explicitly approved evaluations only.
|
|
101
|
+
- Baseline descriptor: non-secret actual provider/model assignments from the og app.
|
|
102
|
+
|
|
103
|
+
Do not guess a model version or substitute an OSS model to connect. Registration
|
|
104
|
+
pins this original baseline without requiring initial OSS assignments:
|
|
105
|
+
|
|
106
|
+
```python
|
|
107
|
+
baseline = {
|
|
108
|
+
"version": 1, "execution": "customer_direct",
|
|
109
|
+
"credential_owner": "customer_local", "fallback": False,
|
|
110
|
+
"routes": {alias: {"provider": actual_provider, "model": actual_model}
|
|
111
|
+
for alias in declaration["roles"].values()},
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Provider names are `openai`, `anthropic`, `google` (`gemini` is retained). Each alias can have its own
|
|
116
|
+
actual provider/model. Existing application identity, assignments and evidence
|
|
117
|
+
must be preserved, not reset. Tracer and runner keys are not provider keys; a missing
|
|
118
|
+
runner key must never fall back to the tracer key.
|
|
119
|
+
|
|
120
|
+
## Attach without copying the workflow
|
|
121
|
+
|
|
122
|
+
```python
|
|
123
|
+
from noesis_ai import Application, Noesis
|
|
124
|
+
|
|
125
|
+
# Use the original callable and its existing client/model configuration seam.
|
|
126
|
+
noesis = Noesis(key=sdk_key, aliases=declaration["roles"],
|
|
127
|
+
declaration=declaration, baseline=baseline)
|
|
128
|
+
noesis.instrument_http_client(dedicated_async_httpx_client, baseline=baseline)
|
|
129
|
+
application = Application(invoke_original, project=project_outputs,
|
|
130
|
+
declaration=declaration, execution=execution)
|
|
131
|
+
result = await noesis.run(application, original_input)
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
For frontier baselines the dedicated HTTPX clients keep their original provider
|
|
135
|
+
URLs and local credentials, with redirects disabled. Instrument each provider client
|
|
136
|
+
through an existing approved seam; do not consolidate a model mixture into a new
|
|
137
|
+
workflow. An OpenAI `AsyncOpenAI` client can receive one via `http_client=...`.
|
|
138
|
+
The SDK's fixed URL is separately used for control/evidence; only approved trial
|
|
139
|
+
calls use that URL plus `/live/v1/` with runner claims and no provider credentials.
|
|
140
|
+
The hook links calls without buffering streamed delivery or repairing settings. Avoid automatic retry behavior
|
|
141
|
+
changes; retries belong to the og app and count as real attempts.
|
|
142
|
+
|
|
143
|
+
`invoke_original` is async and accepts one JSON value. It calls the original
|
|
144
|
+
workflow with its original preparation and logic, configuring stable aliases via
|
|
145
|
+
an existing seam. `project` is optional, synchronous and deterministic: select
|
|
146
|
+
real final values into the declared review fields, not a new evaluator or an LLM
|
|
147
|
+
projection. Original JSON input/output is retained separately and the original
|
|
148
|
+
result is returned unchanged.
|
|
149
|
+
|
|
150
|
+
The declaration is exactly `{input, roles, outputs}`: `input={"type":"text"}`;
|
|
151
|
+
1–3 named roles mapped to distinct aliases sharing one prefix; 1–3 named output
|
|
152
|
+
fields with `kind` label/number/text. Labels may declare `config.choices`; numbers
|
|
153
|
+
require `config.tolerance` and optional min/max; text has no declaration config.
|
|
154
|
+
Structured original results are projected, not discarded. Review instructions
|
|
155
|
+
belong in the hosted dashboard, not a customer `eval_fn`.
|
|
156
|
+
|
|
157
|
+
Default execution is one call per role. Repeated calls require
|
|
158
|
+
`execution={"version":1,"coverage":{"kind":"fixed_alias_counts","counts":{alias:count}}}`,
|
|
159
|
+
positive counts totaling <=32. Variable/unbounded tool loops are not automatically
|
|
160
|
+
replayable. All model work, stream consumption and tool work must be awaited before
|
|
161
|
+
returning the final result. Detached work is unsupported.
|
|
162
|
+
|
|
163
|
+
Capture must run inside the actual og app execution process. Normal og-app usage
|
|
164
|
+
and capture are the same original workflow with explicit consent; the separate
|
|
165
|
+
local-evaluation runner is needed later to execute approved optimization trials
|
|
166
|
+
and is not a remote source upload. Wire the existing
|
|
167
|
+
UI/endpoint injection seam to the wrapped callable, without copying its renderer.
|
|
168
|
+
A separately wrapped function that the UI never calls is not integration. An
|
|
169
|
+
adjacent noesis runner cannot observe an untouched process.
|
|
170
|
+
|
|
171
|
+
## Independent noesis runner
|
|
172
|
+
|
|
173
|
+
### Short project CLI
|
|
174
|
+
|
|
175
|
+
The installation agent records a minimal, non-secret `noesis.toml` inside the
|
|
176
|
+
integration folder (for example `noesis_integration/noesis.toml`):
|
|
177
|
+
|
|
178
|
+
```toml
|
|
179
|
+
[noesis]
|
|
180
|
+
entrypoint = "noesis_integration.main_with_noesis:create_runner"
|
|
181
|
+
# Optional existing executable attachment module, required for `noesis tracer`.
|
|
182
|
+
tracer_module = "noesis_integration.main_with_noesis"
|
|
183
|
+
# Paths are relative to this config; keep the og app root as working directory.
|
|
184
|
+
project_dir = ".."
|
|
185
|
+
# Optional private file at the og app root; loaded by uv, never parsed by the SDK.
|
|
186
|
+
env_file = "../.env"
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
From the og app repository root, use `uv run noesis register` for metadata registration only,
|
|
190
|
+
`uv run noesis tracer` to launch your existing og-app-with-sdk attachment,
|
|
191
|
+
or `uv run noesis runner` only after explicit approval to execute approved jobs.
|
|
192
|
+
The tracer shortcut is equivalent to launching the configured module with Python;
|
|
193
|
+
it does not run evaluations, enable capture or grant consent. Grant consent and
|
|
194
|
+
click **Start tracing** in the noesis dashboard separately. Normal application
|
|
195
|
+
inference still uses your local provider credentials and billing. Missing or invalid
|
|
196
|
+
`tracer_module` fails before launch; existing register/runner configs need no change.
|
|
197
|
+
An activated project environment also supports `noesis register` / `noesis tracer` / `noesis runner`. The
|
|
198
|
+
CLI finds a single `noesis.toml` in the current directory or an immediate child
|
|
199
|
+
folder; it does not recursively search or climb parent directories. Multiple
|
|
200
|
+
configs fail closed; use `--config PATH` for an explicit or deeper location.
|
|
201
|
+
Bad configuration fails closed without printing values. An omitted `env_file`
|
|
202
|
+
uses the inherited environment or the factory's approved loader. When present,
|
|
203
|
+
its dotenv loading is delegated to the existing `uv run --env-file` mechanism;
|
|
204
|
+
the SDK does not parse dotenv files or expose secrets. The recorded config
|
|
205
|
+
contains an optional relative `project_dir` determining the factory's working
|
|
206
|
+
directory (default: config directory, preserving existing root-level configs).
|
|
207
|
+
Both `project_dir` and `env_file` resolve relative to the config. The installed interpreter
|
|
208
|
+
runs the existing module runner without dependency syncing or recursive CLI startup.
|
|
209
|
+
The legacy
|
|
210
|
+
`python -m noesis_ai_runner module:factory --register-only` command remains
|
|
211
|
+
supported.
|
|
212
|
+
|
|
213
|
+
|
|
214
|
+
The attachment exposes an async context-manager factory, e.g. `create_runner`,
|
|
215
|
+
yielding `(Noesis, Application)` and owning both HTTP-client lifetimes. It binds
|
|
216
|
+
the same original callable, declaration/projection and execution counts, using
|
|
217
|
+
`NOESIS_RUNNER_KEY` instead of the tracer key.
|
|
218
|
+
|
|
219
|
+
The dashboard creates the application and both one-time keys first. Installation
|
|
220
|
+
explains that the tracer key records consented activity while the runner key
|
|
221
|
+
connects configuration and approved local evaluations; provider keys stay local.
|
|
222
|
+
It asks for the og app repository root, a user-chosen integration-folder location
|
|
223
|
+
(suggesting the Python-safe `<repo-root>/noesis_integration/`), and the user's
|
|
224
|
+
preferred environment loader.
|
|
225
|
+
Default: private Git-ignored `.env`, explicitly loaded at each launch; no repeated
|
|
226
|
+
exports. Never read or print credential values, overwrite existing private files,
|
|
227
|
+
or put keys in chat, command arguments or version control.
|
|
228
|
+
|
|
229
|
+
After the user approves the installation proposal and offline checks pass, run
|
|
230
|
+
metadata registration only, from the actual attachment's importable directory:
|
|
231
|
+
|
|
232
|
+
```sh
|
|
233
|
+
uv run --env-file .env python -m noesis_ai_runner package.attachment:create_runner --register-only
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
Replace `.env` loading with the customer's existing mechanism if preferred. The
|
|
237
|
+
command requires an accepted `ready_for_capture=true` receipt, closes its factory,
|
|
238
|
+
and exits without polling, workflow invocation or inference. A rejected/missing
|
|
239
|
+
readiness receipt means setup is incomplete, even if registration was accepted.
|
|
240
|
+
The factory must not execute work on import or entry. Registration sends the
|
|
241
|
+
actual declaration/baseline metadata, not provider keys or application source.
|
|
242
|
+
|
|
243
|
+
Only later, after explicit evaluation-execution approval:
|
|
244
|
+
|
|
245
|
+
```sh
|
|
246
|
+
uv run --env-file .env python -m noesis_ai_runner package.attachment:create_runner
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
The runner registers the unchanged declaration/baseline before polling; repeated
|
|
250
|
+
registration is idempotent. It invokes the original workflow under temporary
|
|
251
|
+
server-approved OSS assignments. Starting polling can execute previously approved
|
|
252
|
+
jobs: obtain approval and isolate state, tools, writes, caches, billing side
|
|
253
|
+
effects and external submissions first. No automatic replay after a claimed
|
|
254
|
+
execution or ambiguous result delivery. Stop with Ctrl+C. AgentOpt and approved judge
|
|
255
|
+
judging stay server-side; optimization never activates an assignment.
|
|
256
|
+
|
|
257
|
+
Claimed result delivery requires a successful HTTP response with `accepted: true`.
|
|
258
|
+
A recognized `409` with the top-level code `execution_evidence_rejected` permits
|
|
259
|
+
one same-claim, metadata-only `RunnerResultRejected` infrastructure-error report;
|
|
260
|
+
then the runner stops. This does not resend outputs or execute the app again.
|
|
261
|
+
Unknown/uncoded rejections, invalid acknowledgements and ambiguous network errors
|
|
262
|
+
stop without this fallback. Human-readable error wording never controls recovery.
|
|
263
|
+
Deploy the compatible backend before updating the local SDK/runner; an older
|
|
264
|
+
backend's uncoded rejection stops the new runner but may still reach the backend
|
|
265
|
+
job deadline. A backend deployment alone does not update an installed SDK.
|
|
266
|
+
|
|
267
|
+
## Protocol and evidence limits
|
|
268
|
+
|
|
269
|
+
The public interface follows OpenAI Chat Completions, not every API in
|
|
270
|
+
https://developers.openai.com/api/reference/overview . Supported protocol paths
|
|
271
|
+
are normal JSON and optional SSE, function tools/results, supported JSON/JSON
|
|
272
|
+
Schema output, and text/image input with text/tool output. Native Anthropic or
|
|
273
|
+
Google client protocols are not accepted just by changing their base URL. Noesis
|
|
274
|
+
owns upstream adaptation; unsupported model/feature combinations fail explicitly.
|
|
275
|
+
Tool execution remains in the og app. No silent parameter removal or provider
|
|
276
|
+
substitution. Model/account/deployment features require actual provider support.
|
|
277
|
+
|
|
278
|
+
The backend retains one canonical trace per call, incremental tool/content deltas,
|
|
279
|
+
finish reasons, usage and settings. Missing usage/prices remain unknown. Refusal,
|
|
280
|
+
truncation, malformed streams, incomplete tools and cancellation are not valid
|
|
281
|
+
quality evidence. Image serving/capture does not establish multimodal quality:
|
|
282
|
+
the text-only judge rejects image-bearing text evaluations. Audio/video, file
|
|
283
|
+
uploads, generated image/audio, Responses and Realtime are outside this release.
|
|
284
|
+
|
|
285
|
+
Application capture is opt-in and bounded: finite JSON, <=100 KB per SDK value,
|
|
286
|
+
<=10 KB display input and <=2 MB backend streamed evidence. Unsupported or oversized
|
|
287
|
+
capture must not become invented evidence. Capture failures warn/abort enrollment
|
|
288
|
+
without substituting the served answer; evaluation failures fail closed. Clients
|
|
289
|
+
must disable redirects to prevent an authenticated inference request escaping the
|
|
290
|
+
validated destination. This is backend proxy routing, not an HTTP redirect.
|
|
291
|
+
|
|
292
|
+
Direct observations are customer-SDK evidence, not server-attested inference.
|
|
293
|
+
Consume streams and await all model/tool work before the original workflow returns.
|
|
294
|
+
Incomplete observation invalidates capture, not the original served result. Usage
|
|
295
|
+
is observed; frontier dollar cost stays unknown without trustworthy rate provenance.
|
|
296
|
+
Native provider protocols and arbitrary HTTP destinations are not supported just by
|
|
297
|
+
changing a URL.
|
|
298
|
+
|
|
299
|
+
## Versioned whole-outcome review
|
|
300
|
+
|
|
301
|
+
A Phase 1 declaration may use `version: 2`, JSON input, and an explicit outcome view of
|
|
302
|
+
`application_output` or deterministic `outputs`. The original application input/output remain
|
|
303
|
+
bounded authoritative evidence; the display excerpt is never replayed. Reviews may contain
|
|
304
|
+
nested objects, lists, scalars, booleans, and null, with optional bounded additional instructions.
|
|
305
|
+
The backend stores one versioned `outcome` specification and sends one approved judge request
|
|
306
|
+
containing the original input, reviewed reference, selected view, and complete candidate outcome.
|
|
307
|
+
Legacy flat declarations and field scoring retain their existing semantics. Legacy execution
|
|
308
|
+
version 1 retains fixed alias counts totaling at most 32 calls. New whole-outcome declarations
|
|
309
|
+
have application-specific roles without the former three-role ceiling. For loops, branches,
|
|
310
|
+
skipped roles and app-owned retries, explicitly configure approved execution bounds:
|
|
311
|
+
|
|
312
|
+
```python
|
|
313
|
+
execution = {"version": 2,
|
|
314
|
+
"coverage": {"kind": "bounded_attempts", "max_attempts": 40},
|
|
315
|
+
"max_wall_seconds": 120, "max_output_tokens": 4096}
|
|
316
|
+
app = Application(invoke, declaration=declaration, execution=execution)
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
These are illustrative allowances, not defaults or a dollar budget. Actual attempts can be
|
|
320
|
+
zero or vary between baseline and candidate paths. Every admitted attempt must reconcile
|
|
321
|
+
with terminal evidence before successful closure; recovered failures remain attributable,
|
|
322
|
+
and missing usage remains unknown cost. Output bounds reject, never silently clamp requests.
|
|
323
|
+
Policy integer bounds must be browser-safe JSON integers; workflow execution totals use exact
|
|
324
|
+
arithmetic and decimal text when beyond that range. The global 100-execution ceiling is gone,
|
|
325
|
+
but the exact full-comparison count still requires approval. A larger role space grows
|
|
326
|
+
Cartesian comparison cost; lifting a ceiling does not solve that. Leveraging AgentOpt's
|
|
327
|
+
existing efficient selection capabilities is later work, not a search change in this phase.
|
|
328
|
+
Use this updated checkout's SDK for the new contract; SDK extraction/publication is separate.
|
|
329
|
+
|
|
330
|
+
Preview checks the complete baseline/reference judge messages against the declared context
|
|
331
|
+
using a conservative UTF-8 byte bound, including chat framing and output reserve; unknown
|
|
332
|
+
limits block whole-outcome optimization. Each actual candidate payload is checked again before
|
|
333
|
+
judging. Nothing is silently truncated, but conservative checks can reject a payload that an
|
|
334
|
+
exact tokenizer could fit, and a candidate can exceed context despite its baseline fitting.
|
|
335
|
+
The dashboard fails closed for integers outside JavaScript's safe range rather than silently
|
|
336
|
+
rounding reviewed references. Original bounded JSON remains preserved by the backend.
|
|
337
|
+
|
|
338
|
+
## Acceptance and honest handoff
|
|
339
|
+
|
|
340
|
+
First verify offline with synthetic inputs/transports: installed module resolution,
|
|
341
|
+
projection, original-result preservation, invocation isolation and cleanup. Record
|
|
342
|
+
exact working directories and real environment-loading capture/runner commands,
|
|
343
|
+
not final placeholders. Verify registration/readiness before declaring installation
|
|
344
|
+
complete. First explain that a new, independent attachment file stays on the user's
|
|
345
|
+
machine and connects the dashboard by sending configuration metadata, not original
|
|
346
|
+
application source or provider keys. Later consented capture transmits execution
|
|
347
|
+
evidence; the separate evaluation runner executes the original application locally.
|
|
348
|
+
Use two clear handoff sections: “Start using and tracing your application now” and
|
|
349
|
+
“Later: run optimization experiments”. Explain each process before its command.
|
|
350
|
+
Then say “Open a new terminal”, give the actual `cd` and capture command, and point
|
|
351
|
+
to the information folder for details. The evaluation runner is not needed for
|
|
352
|
+
normal usage or baseline tracing and must not start until separately approved. State offline
|
|
353
|
+
wiring, registration readiness and live capture separately. If live capture was
|
|
354
|
+
not separately authorized, say so; offer one bounded synthetic-input smoke test
|
|
355
|
+
with explicit consent and customer-owned provider billing. Diagnose application,
|
|
356
|
+
provider and capture failures separately; never rewrite customer prompts/parsers
|
|
357
|
+
to make setup appear successful.
|
|
358
|
+
Quality examples and review are required before optimization, not a prerequisite
|
|
359
|
+
for local installation. Registration, baseline configured, capture consent, actual
|
|
360
|
+
complete captured executions and runner polling are different facts.
|
|
361
|
+
|
|
362
|
+
Paid baseline/candidate/judge inference, real data upload, hosted runner polling,
|
|
363
|
+
deployment and serving activation require separate explicit approval. Synthetic
|
|
364
|
+
checks prove wiring, not live provider compatibility, maintained quality or savings.
|
|
365
|
+
The existing landing-page installation prompt is the customer entrypoint; this
|
|
366
|
+
README and shipped examples are its SDK-side documentation, not another setup flow.
|
|
367
|
+
|
|
368
|
+
Inspection approval does not authorize Git staging, untracking a private file,
|
|
369
|
+
changing the index, editing `.gitignore`, or rewriting history. Report tracked
|
|
370
|
+
private configuration without reading its values and obtain specific approval for
|
|
371
|
+
any such Git changes. Never automatically untrack `.env` as a security cleanup.
|
|
372
|
+
|
|
373
|
+
## Updating an existing integration
|
|
374
|
+
|
|
375
|
+
Keep the full initial installation prompt for the first connection. For later
|
|
376
|
+
application changes, the shared maintenance skill is bundled at
|
|
377
|
+
`skills/noesis-update-integration/SKILL.md` in this SDK checkout and installed at
|
|
378
|
+
`<python-prefix>/share/noesis_ai/skills/noesis-update-integration/SKILL.md`.
|
|
379
|
+
The installation agent proposes and copies it into the current coding agent's
|
|
380
|
+
appropriate project-local skill directory, without overwriting existing rules or
|
|
381
|
+
granting permissions. If native discovery is unavailable, it records a readable
|
|
382
|
+
local copy and an explicit-read prompt instead. No separate per-agent SDK is needed.
|
|
383
|
+
|
|
384
|
+
Future skills repository: `<NOESIS_SKILLS_REPOSITORY_URL>` — **placeholder only**;
|
|
385
|
+
do not fetch it or guess a replacement. Use the approved bundled SDK snapshot now.
|
|
386
|
+
The integration README must record the actual skill source/version/destination,
|
|
387
|
+
verify the copied contents, and give a copyable update prompt with real paths.
|
|
388
|
+
|
|
389
|
+
Use the update skill when LLM calls, prompts, parsing, branching, aliases, baseline
|
|
390
|
+
configuration or SDK compatibility change. The agent inspects the relevant og app
|
|
391
|
+
context, proposes minimal integration edits and tests, and explains registration,
|
|
392
|
+
capture/review and evaluation requirements. It does not repeat account/key setup.
|
|
393
|
+
|
|
394
|
+
Initial integrations set `configuration_revision=1` on their capture and runner
|
|
395
|
+
`Noesis(...)` clients. Material approved workflow, alias, baseline or execution-policy
|
|
396
|
+
changes use exactly the next revision; the runner registers `Application.execution`.
|
|
397
|
+
Identical registration remains idempotent, including during active capture. A new
|
|
398
|
+
revision requires capture/operations/claims to be idle, preserves old configuration
|
|
399
|
+
and evidence, clears only the active-dataset selection, and requires fresh
|
|
400
|
+
capture/review before optimization. Same-label prompt/workflow changes still need
|
|
401
|
+
this deliberate update; there is no automatic application-source change detector.
|
|
402
|
+
|
|
403
|
+
Updates conflicting with saved serving labels/baseline are rejected rather than
|
|
404
|
+
changing production routing. Legacy backends may not support revision updates;
|
|
405
|
+
the skill must explain that blocker rather than bypass locks. Registration
|
|
406
|
+
acceptance alone does not establish live capture or validate old optimization
|
|
407
|
+
results for a changed workflow. Credentials, capture, polling, inference and
|
|
408
|
+
serving remain separately approved; verify CLI/backend versions before use.
|
|
409
|
+
|
|
410
|
+
## Release verification
|
|
411
|
+
|
|
412
|
+
For a reproducible installation, retain the resolved `noesis_ai` version and
|
|
413
|
+
artifact hashes in the og app's dependency lock. The installed modules are
|
|
414
|
+
`noesis_ai` and `noesis_ai_runner`; the console command is `noesis`.
|
|
415
|
+
Do not install the unrelated `noesis` distribution. Package installation and
|
|
416
|
+
imports never register an application, start polling, run inference or activate
|
|
417
|
+
serving. Those operations retain their separate approval requirements.
|