noesis_ai 0.0.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- noesis_ai-0.0.0.data/data/share/noesis_ai/README.md +405 -0
- noesis_ai-0.0.0.data/data/share/noesis_ai/examples/briefing/app.py +29 -0
- noesis_ai-0.0.0.data/data/share/noesis_ai/examples/briefing/noesis_adapter.py +80 -0
- noesis_ai-0.0.0.data/data/share/noesis_ai/skills/noesis-update-integration/SKILL.md +104 -0
- noesis_ai-0.0.0.dist-info/METADATA +417 -0
- noesis_ai-0.0.0.dist-info/RECORD +12 -0
- noesis_ai-0.0.0.dist-info/WHEEL +5 -0
- noesis_ai-0.0.0.dist-info/entry_points.txt +2 -0
- noesis_ai-0.0.0.dist-info/licenses/LICENSE +21 -0
- noesis_ai-0.0.0.dist-info/top_level.txt +2 -0
- noesis_ai.py +840 -0
- noesis_ai_runner.py +295 -0
|
@@ -0,0 +1,405 @@
|
|
|
1
|
+
# Noesis SDK: thin workflow attachment and independent runner
|
|
2
|
+
|
|
3
|
+
`noesis_ai` is a Python >=3.12 network-only SDK. Imports: `noesis_ai` and
|
|
4
|
+
`noesis_ai_runner`; dependency: HTTPX. It does not install the hosted dashboard,
|
|
5
|
+
Streamlit, AgentOpt, backend, or provider SDKs. This README accompanies release
|
|
6
|
+
`noesis_ai==0.0.0`, licensed under MIT by Mirai Engineering OÜ. Install the
|
|
7
|
+
versioned package into the og app's existing environment; do not patch the SDK
|
|
8
|
+
or original app files.
|
|
9
|
+
|
|
10
|
+
## Two independent offerings
|
|
11
|
+
|
|
12
|
+
**Serving:** use an ordinary OpenAI-compatible client against
|
|
13
|
+
`https://noesis-production-efed.up.railway.app/live/v1`, with a Noesis application key and an approved stable
|
|
14
|
+
role alias. No SDK, decorators or running evaluation process is required for
|
|
15
|
+
ordinary inference after the application's connection/assignments are configured.
|
|
16
|
+
Noesis resolves explicitly approved OSS assignments. Hosted frontier BYOK is retained but paused; ask Bruno before deleting or reactivating it.
|
|
17
|
+
|
|
18
|
+
**Optimization:** add a thin attachment around the actual original workflow and a
|
|
19
|
+
separate noesis runner process. Endpoint traces alone do not capture the complete
|
|
20
|
+
application outcome. The existing `Application`, `Noesis.run` and explicit HTTPX
|
|
21
|
+
hook provide that boundary; no decorators or workflow rewrite are needed.
|
|
22
|
+
|
|
23
|
+
## Installation-agent contract
|
|
24
|
+
|
|
25
|
+
The installation README should briefly record inspection findings, the grounded
|
|
26
|
+
complete workload summary, proposed files, actual tested commands, and capture
|
|
27
|
+
versus evaluation limits. Put that README, the thin attachment, and
|
|
28
|
+
integration-specific tests in the user's chosen folder; suggest the Python-safe
|
|
29
|
+
`noesis_integration/` by default, but never force a rename or move an existing
|
|
30
|
+
file. Preserve the og app's namespace/imports and use real launch commands; do
|
|
31
|
+
not add `sys.path` hacks. If the folder cannot support the imports or commands,
|
|
32
|
+
explain why and ask where to write instead. If the user relocates it, update
|
|
33
|
+
commands and retest.
|
|
34
|
+
|
|
35
|
+
Use this README and shipped integration example. No Noesis internal architecture,
|
|
36
|
+
backend source, test fixtures, old customer integration or repository history is needed.
|
|
37
|
+
Inspect the og app source/dependencies/actual launch seam, never `.env` or secrets.
|
|
38
|
+
Explain the specific local install and new files, obtain one clear approval, then
|
|
39
|
+
install and add only the attachment/runner in the customer's project conventions.
|
|
40
|
+
Preserve original prompts, branching, parsing, UI, state and returned values.
|
|
41
|
+
If no existing configuration/invocation seam works, report the actual limitation;
|
|
42
|
+
do not invent a copied workflow, global monkey patch or SDK repair. The published
|
|
43
|
+
Noesis contract is fixed; adapt approved client/attachment seams, never add
|
|
44
|
+
customer-specific SDK/backend behavior to accommodate an app.
|
|
45
|
+
|
|
46
|
+
Install the versioned public distribution using the og app's dependency workflow:
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
uv add noesis_ai==0.0.0
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
No private repository checkout is required. For explicitly approved local release
|
|
53
|
+
validation, an SDK source directory is also installable (no wheel path required):
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
uv pip install --python /absolute/og-app/.venv/bin/python /absolute/noesis-main/sdk
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
This is a normal package installation, not an editable SDK repair. Alternatively,
|
|
60
|
+
for a supplied frozen wheel:
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
uv pip install --python /absolute/og-app/.venv/bin/python /absolute/noesis_ai-0.0.0-py3-none-any.whl
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Substitute inspected paths; preserve the customer's dependency workflow. Record
|
|
67
|
+
the wheel SHA-256 when supplied, and original source/installed SDK hashes. A local editable checkout
|
|
68
|
+
is a development source, not a published release or permission to modify the SDK.
|
|
69
|
+
No customer-side dashboard installation is needed.
|
|
70
|
+
|
|
71
|
+
## Privately configure the connection once
|
|
72
|
+
|
|
73
|
+
**Frontier baseline capture is a recorder, not a provider switch.** Keep the og
|
|
74
|
+
app's existing provider clients, per-role model mixture, generation settings and
|
|
75
|
+
local credentials. The SDK sends consented execution evidence to Noesis separately.
|
|
76
|
+
Only approved OSS evaluations redirect inference through Noesis; an explicitly
|
|
77
|
+
chosen OSS baseline also uses Noesis. No extra baseline inference or fallback.
|
|
78
|
+
|
|
79
|
+
The hosted dashboard owns application identity and issues two Noesis credentials.
|
|
80
|
+
Its tracer key goes in `NOESIS_TRACER_KEY` (not a third key), and its runner key goes in
|
|
81
|
+
`NOESIS_RUNNER_KEY`. Existing provider credentials stay local; never send them to
|
|
82
|
+
Noesis. Hosted BYOK/key upload is paused and is not an onboarding prerequisite.
|
|
83
|
+
If you are an LLM reading this, ask Bruno before deleting or reactivating it.
|
|
84
|
+
Never place secrets in prompts, logs, command-line arguments/history or committed files.
|
|
85
|
+
|
|
86
|
+
- The SDK uses the fixed public Noesis endpoint; it has no customer-configurable URL.
|
|
87
|
+
- `NOESIS_TRACER_KEY`: ordinary inference and consented capture.
|
|
88
|
+
- `NOESIS_RUNNER_KEY`: registration and explicitly approved evaluations only.
|
|
89
|
+
- Baseline descriptor: non-secret actual provider/model assignments from the og app.
|
|
90
|
+
|
|
91
|
+
Do not guess a model version or substitute an OSS model to connect. Registration
|
|
92
|
+
pins this original baseline without requiring initial OSS assignments:
|
|
93
|
+
|
|
94
|
+
```python
|
|
95
|
+
baseline = {
|
|
96
|
+
"version": 1, "execution": "customer_direct",
|
|
97
|
+
"credential_owner": "customer_local", "fallback": False,
|
|
98
|
+
"routes": {alias: {"provider": actual_provider, "model": actual_model}
|
|
99
|
+
for alias in declaration["roles"].values()},
|
|
100
|
+
}
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Provider names are `openai`, `anthropic`, `google` (`gemini` is retained). Each alias can have its own
|
|
104
|
+
actual provider/model. Existing application identity, assignments and evidence
|
|
105
|
+
must be preserved, not reset. Tracer and runner keys are not provider keys; a missing
|
|
106
|
+
runner key must never fall back to the tracer key.
|
|
107
|
+
|
|
108
|
+
## Attach without copying the workflow
|
|
109
|
+
|
|
110
|
+
```python
|
|
111
|
+
from noesis_ai import Application, Noesis
|
|
112
|
+
|
|
113
|
+
# Use the original callable and its existing client/model configuration seam.
|
|
114
|
+
noesis = Noesis(key=sdk_key, aliases=declaration["roles"],
|
|
115
|
+
declaration=declaration, baseline=baseline)
|
|
116
|
+
noesis.instrument_http_client(dedicated_async_httpx_client, baseline=baseline)
|
|
117
|
+
application = Application(invoke_original, project=project_outputs,
|
|
118
|
+
declaration=declaration, execution=execution)
|
|
119
|
+
result = await noesis.run(application, original_input)
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
For frontier baselines the dedicated HTTPX clients keep their original provider
|
|
123
|
+
URLs and local credentials, with redirects disabled. Instrument each provider client
|
|
124
|
+
through an existing approved seam; do not consolidate a model mixture into a new
|
|
125
|
+
workflow. An OpenAI `AsyncOpenAI` client can receive one via `http_client=...`.
|
|
126
|
+
The SDK's fixed URL is separately used for control/evidence; only approved trial
|
|
127
|
+
calls use that URL plus `/live/v1/` with runner claims and no provider credentials.
|
|
128
|
+
The hook links calls without buffering streamed delivery or repairing settings. Avoid automatic retry behavior
|
|
129
|
+
changes; retries belong to the og app and count as real attempts.
|
|
130
|
+
|
|
131
|
+
`invoke_original` is async and accepts one JSON value. It calls the original
|
|
132
|
+
workflow with its original preparation and logic, configuring stable aliases via
|
|
133
|
+
an existing seam. `project` is optional, synchronous and deterministic: select
|
|
134
|
+
real final values into the declared review fields, not a new evaluator or an LLM
|
|
135
|
+
projection. Original JSON input/output is retained separately and the original
|
|
136
|
+
result is returned unchanged.
|
|
137
|
+
|
|
138
|
+
The declaration is exactly `{input, roles, outputs}`: `input={"type":"text"}`;
|
|
139
|
+
1–3 named roles mapped to distinct aliases sharing one prefix; 1–3 named output
|
|
140
|
+
fields with `kind` label/number/text. Labels may declare `config.choices`; numbers
|
|
141
|
+
require `config.tolerance` and optional min/max; text has no declaration config.
|
|
142
|
+
Structured original results are projected, not discarded. Review instructions
|
|
143
|
+
belong in the hosted dashboard, not a customer `eval_fn`.
|
|
144
|
+
|
|
145
|
+
Default execution is one call per role. Repeated calls require
|
|
146
|
+
`execution={"version":1,"coverage":{"kind":"fixed_alias_counts","counts":{alias:count}}}`,
|
|
147
|
+
positive counts totaling <=32. Variable/unbounded tool loops are not automatically
|
|
148
|
+
replayable. All model work, stream consumption and tool work must be awaited before
|
|
149
|
+
returning the final result. Detached work is unsupported.
|
|
150
|
+
|
|
151
|
+
Capture must run inside the actual og app execution process. Normal og-app usage
|
|
152
|
+
and capture are the same original workflow with explicit consent; the separate
|
|
153
|
+
local-evaluation runner is needed later to execute approved optimization trials
|
|
154
|
+
and is not a remote source upload. Wire the existing
|
|
155
|
+
UI/endpoint injection seam to the wrapped callable, without copying its renderer.
|
|
156
|
+
A separately wrapped function that the UI never calls is not integration. An
|
|
157
|
+
adjacent noesis runner cannot observe an untouched process.
|
|
158
|
+
|
|
159
|
+
## Independent noesis runner
|
|
160
|
+
|
|
161
|
+
### Short project CLI
|
|
162
|
+
|
|
163
|
+
The installation agent records a minimal, non-secret `noesis.toml` inside the
|
|
164
|
+
integration folder (for example `noesis_integration/noesis.toml`):
|
|
165
|
+
|
|
166
|
+
```toml
|
|
167
|
+
[noesis]
|
|
168
|
+
entrypoint = "noesis_integration.main_with_noesis:create_runner"
|
|
169
|
+
# Optional existing executable attachment module, required for `noesis tracer`.
|
|
170
|
+
tracer_module = "noesis_integration.main_with_noesis"
|
|
171
|
+
# Paths are relative to this config; keep the og app root as working directory.
|
|
172
|
+
project_dir = ".."
|
|
173
|
+
# Optional private file at the og app root; loaded by uv, never parsed by the SDK.
|
|
174
|
+
env_file = "../.env"
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
From the og app repository root, use `uv run noesis register` for metadata registration only,
|
|
178
|
+
`uv run noesis tracer` to launch your existing og-app-with-sdk attachment,
|
|
179
|
+
or `uv run noesis runner` only after explicit approval to execute approved jobs.
|
|
180
|
+
The tracer shortcut is equivalent to launching the configured module with Python;
|
|
181
|
+
it does not run evaluations, enable capture or grant consent. Grant consent and
|
|
182
|
+
click **Start tracing** in the noesis dashboard separately. Normal application
|
|
183
|
+
inference still uses your local provider credentials and billing. Missing or invalid
|
|
184
|
+
`tracer_module` fails before launch; existing register/runner configs need no change.
|
|
185
|
+
An activated project environment also supports `noesis register` / `noesis tracer` / `noesis runner`. The
|
|
186
|
+
CLI finds a single `noesis.toml` in the current directory or an immediate child
|
|
187
|
+
folder; it does not recursively search or climb parent directories. Multiple
|
|
188
|
+
configs fail closed; use `--config PATH` for an explicit or deeper location.
|
|
189
|
+
Bad configuration fails closed without printing values. An omitted `env_file`
|
|
190
|
+
uses the inherited environment or the factory's approved loader. When present,
|
|
191
|
+
its dotenv loading is delegated to the existing `uv run --env-file` mechanism;
|
|
192
|
+
the SDK does not parse dotenv files or expose secrets. The recorded config
|
|
193
|
+
contains an optional relative `project_dir` determining the factory's working
|
|
194
|
+
directory (default: config directory, preserving existing root-level configs).
|
|
195
|
+
Both `project_dir` and `env_file` resolve relative to the config. The installed interpreter
|
|
196
|
+
runs the existing module runner without dependency syncing or recursive CLI startup.
|
|
197
|
+
The legacy
|
|
198
|
+
`python -m noesis_ai_runner module:factory --register-only` command remains
|
|
199
|
+
supported.
|
|
200
|
+
|
|
201
|
+
|
|
202
|
+
The attachment exposes an async context-manager factory, e.g. `create_runner`,
|
|
203
|
+
yielding `(Noesis, Application)` and owning both HTTP-client lifetimes. It binds
|
|
204
|
+
the same original callable, declaration/projection and execution counts, using
|
|
205
|
+
`NOESIS_RUNNER_KEY` instead of the tracer key.
|
|
206
|
+
|
|
207
|
+
The dashboard creates the application and both one-time keys first. Installation
|
|
208
|
+
explains that the tracer key records consented activity while the runner key
|
|
209
|
+
connects configuration and approved local evaluations; provider keys stay local.
|
|
210
|
+
It asks for the og app repository root, a user-chosen integration-folder location
|
|
211
|
+
(suggesting the Python-safe `<repo-root>/noesis_integration/`), and the user's
|
|
212
|
+
preferred environment loader.
|
|
213
|
+
Default: private Git-ignored `.env`, explicitly loaded at each launch; no repeated
|
|
214
|
+
exports. Never read or print credential values, overwrite existing private files,
|
|
215
|
+
or put keys in chat, command arguments or version control.
|
|
216
|
+
|
|
217
|
+
After the user approves the installation proposal and offline checks pass, run
|
|
218
|
+
metadata registration only, from the actual attachment's importable directory:
|
|
219
|
+
|
|
220
|
+
```sh
|
|
221
|
+
uv run --env-file .env python -m noesis_ai_runner package.attachment:create_runner --register-only
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Replace `.env` loading with the customer's existing mechanism if preferred. The
|
|
225
|
+
command requires an accepted `ready_for_capture=true` receipt, closes its factory,
|
|
226
|
+
and exits without polling, workflow invocation or inference. A rejected/missing
|
|
227
|
+
readiness receipt means setup is incomplete, even if registration was accepted.
|
|
228
|
+
The factory must not execute work on import or entry. Registration sends the
|
|
229
|
+
actual declaration/baseline metadata, not provider keys or application source.
|
|
230
|
+
|
|
231
|
+
Only later, after explicit evaluation-execution approval:
|
|
232
|
+
|
|
233
|
+
```sh
|
|
234
|
+
uv run --env-file .env python -m noesis_ai_runner package.attachment:create_runner
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
The runner registers the unchanged declaration/baseline before polling; repeated
|
|
238
|
+
registration is idempotent. It invokes the original workflow under temporary
|
|
239
|
+
server-approved OSS assignments. Starting polling can execute previously approved
|
|
240
|
+
jobs: obtain approval and isolate state, tools, writes, caches, billing side
|
|
241
|
+
effects and external submissions first. No automatic replay after a claimed
|
|
242
|
+
execution or ambiguous result delivery. Stop with Ctrl+C. AgentOpt and approved judge
|
|
243
|
+
judging stay server-side; optimization never activates an assignment.
|
|
244
|
+
|
|
245
|
+
Claimed result delivery requires a successful HTTP response with `accepted: true`.
|
|
246
|
+
A recognized `409` with the top-level code `execution_evidence_rejected` permits
|
|
247
|
+
one same-claim, metadata-only `RunnerResultRejected` infrastructure-error report;
|
|
248
|
+
then the runner stops. This does not resend outputs or execute the app again.
|
|
249
|
+
Unknown/uncoded rejections, invalid acknowledgements and ambiguous network errors
|
|
250
|
+
stop without this fallback. Human-readable error wording never controls recovery.
|
|
251
|
+
Deploy the compatible backend before updating the local SDK/runner; an older
|
|
252
|
+
backend's uncoded rejection stops the new runner but may still reach the backend
|
|
253
|
+
job deadline. A backend deployment alone does not update an installed SDK.
|
|
254
|
+
|
|
255
|
+
## Protocol and evidence limits
|
|
256
|
+
|
|
257
|
+
The public interface follows OpenAI Chat Completions, not every API in
|
|
258
|
+
https://developers.openai.com/api/reference/overview . Supported protocol paths
|
|
259
|
+
are normal JSON and optional SSE, function tools/results, supported JSON/JSON
|
|
260
|
+
Schema output, and text/image input with text/tool output. Native Anthropic or
|
|
261
|
+
Google client protocols are not accepted just by changing their base URL. Noesis
|
|
262
|
+
owns upstream adaptation; unsupported model/feature combinations fail explicitly.
|
|
263
|
+
Tool execution remains in the og app. No silent parameter removal or provider
|
|
264
|
+
substitution. Model/account/deployment features require actual provider support.
|
|
265
|
+
|
|
266
|
+
The backend retains one canonical trace per call, incremental tool/content deltas,
|
|
267
|
+
finish reasons, usage and settings. Missing usage/prices remain unknown. Refusal,
|
|
268
|
+
truncation, malformed streams, incomplete tools and cancellation are not valid
|
|
269
|
+
quality evidence. Image serving/capture does not establish multimodal quality:
|
|
270
|
+
the text-only judge rejects image-bearing text evaluations. Audio/video, file
|
|
271
|
+
uploads, generated image/audio, Responses and Realtime are outside this release.
|
|
272
|
+
|
|
273
|
+
Application capture is opt-in and bounded: finite JSON, <=100 KB per SDK value,
|
|
274
|
+
<=10 KB display input and <=2 MB backend streamed evidence. Unsupported or oversized
|
|
275
|
+
capture must not become invented evidence. Capture failures warn/abort enrollment
|
|
276
|
+
without substituting the served answer; evaluation failures fail closed. Clients
|
|
277
|
+
must disable redirects to prevent an authenticated inference request escaping the
|
|
278
|
+
validated destination. This is backend proxy routing, not an HTTP redirect.
|
|
279
|
+
|
|
280
|
+
Direct observations are customer-SDK evidence, not server-attested inference.
|
|
281
|
+
Consume streams and await all model/tool work before the original workflow returns.
|
|
282
|
+
Incomplete observation invalidates capture, not the original served result. Usage
|
|
283
|
+
is observed; frontier dollar cost stays unknown without trustworthy rate provenance.
|
|
284
|
+
Native provider protocols and arbitrary HTTP destinations are not supported just by
|
|
285
|
+
changing a URL.
|
|
286
|
+
|
|
287
|
+
## Versioned whole-outcome review
|
|
288
|
+
|
|
289
|
+
A Phase 1 declaration may use `version: 2`, JSON input, and an explicit outcome view of
|
|
290
|
+
`application_output` or deterministic `outputs`. The original application input/output remain
|
|
291
|
+
bounded authoritative evidence; the display excerpt is never replayed. Reviews may contain
|
|
292
|
+
nested objects, lists, scalars, booleans, and null, with optional bounded additional instructions.
|
|
293
|
+
The backend stores one versioned `outcome` specification and sends one approved judge request
|
|
294
|
+
containing the original input, reviewed reference, selected view, and complete candidate outcome.
|
|
295
|
+
Legacy flat declarations and field scoring retain their existing semantics. Legacy execution
|
|
296
|
+
version 1 retains fixed alias counts totaling at most 32 calls. New whole-outcome declarations
|
|
297
|
+
have application-specific roles without the former three-role ceiling. For loops, branches,
|
|
298
|
+
skipped roles and app-owned retries, explicitly configure approved execution bounds:
|
|
299
|
+
|
|
300
|
+
```python
|
|
301
|
+
execution = {"version": 2,
|
|
302
|
+
"coverage": {"kind": "bounded_attempts", "max_attempts": 40},
|
|
303
|
+
"max_wall_seconds": 120, "max_output_tokens": 4096}
|
|
304
|
+
app = Application(invoke, declaration=declaration, execution=execution)
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
These are illustrative allowances, not defaults or a dollar budget. Actual attempts can be
|
|
308
|
+
zero or vary between baseline and candidate paths. Every admitted attempt must reconcile
|
|
309
|
+
with terminal evidence before successful closure; recovered failures remain attributable,
|
|
310
|
+
and missing usage remains unknown cost. Output bounds reject, never silently clamp requests.
|
|
311
|
+
Policy integer bounds must be browser-safe JSON integers; workflow execution totals use exact
|
|
312
|
+
arithmetic and decimal text when beyond that range. The global 100-execution ceiling is gone,
|
|
313
|
+
but the exact full-comparison count still requires approval. A larger role space grows
|
|
314
|
+
Cartesian comparison cost; lifting a ceiling does not solve that. Leveraging AgentOpt's
|
|
315
|
+
existing efficient selection capabilities is later work, not a search change in this phase.
|
|
316
|
+
Use this updated checkout's SDK for the new contract; SDK extraction/publication is separate.
|
|
317
|
+
|
|
318
|
+
Preview checks the complete baseline/reference judge messages against the declared context
|
|
319
|
+
using a conservative UTF-8 byte bound, including chat framing and output reserve; unknown
|
|
320
|
+
limits block whole-outcome optimization. Each actual candidate payload is checked again before
|
|
321
|
+
judging. Nothing is silently truncated, but conservative checks can reject a payload that an
|
|
322
|
+
exact tokenizer could fit, and a candidate can exceed context despite its baseline fitting.
|
|
323
|
+
The dashboard fails closed for integers outside JavaScript's safe range rather than silently
|
|
324
|
+
rounding reviewed references. Original bounded JSON remains preserved by the backend.
|
|
325
|
+
|
|
326
|
+
## Acceptance and honest handoff
|
|
327
|
+
|
|
328
|
+
First verify offline with synthetic inputs/transports: installed module resolution,
|
|
329
|
+
projection, original-result preservation, invocation isolation and cleanup. Record
|
|
330
|
+
exact working directories and real environment-loading capture/runner commands,
|
|
331
|
+
not final placeholders. Verify registration/readiness before declaring installation
|
|
332
|
+
complete. First explain that a new, independent attachment file stays on the user's
|
|
333
|
+
machine and connects the dashboard by sending configuration metadata, not original
|
|
334
|
+
application source or provider keys. Later consented capture transmits execution
|
|
335
|
+
evidence; the separate evaluation runner executes the original application locally.
|
|
336
|
+
Use two clear handoff sections: “Start using and tracing your application now” and
|
|
337
|
+
“Later: run optimization experiments”. Explain each process before its command.
|
|
338
|
+
Then say “Open a new terminal”, give the actual `cd` and capture command, and point
|
|
339
|
+
to the information folder for details. The evaluation runner is not needed for
|
|
340
|
+
normal usage or baseline tracing and must not start until separately approved. State offline
|
|
341
|
+
wiring, registration readiness and live capture separately. If live capture was
|
|
342
|
+
not separately authorized, say so; offer one bounded synthetic-input smoke test
|
|
343
|
+
with explicit consent and customer-owned provider billing. Diagnose application,
|
|
344
|
+
provider and capture failures separately; never rewrite customer prompts/parsers
|
|
345
|
+
to make setup appear successful.
|
|
346
|
+
Quality examples and review are required before optimization, not a prerequisite
|
|
347
|
+
for local installation. Registration, baseline configured, capture consent, actual
|
|
348
|
+
complete captured executions and runner polling are different facts.
|
|
349
|
+
|
|
350
|
+
Paid baseline/candidate/judge inference, real data upload, hosted runner polling,
|
|
351
|
+
deployment and serving activation require separate explicit approval. Synthetic
|
|
352
|
+
checks prove wiring, not live provider compatibility, maintained quality or savings.
|
|
353
|
+
The existing landing-page installation prompt is the customer entrypoint; this
|
|
354
|
+
README and shipped examples are its SDK-side documentation, not another setup flow.
|
|
355
|
+
|
|
356
|
+
Inspection approval does not authorize Git staging, untracking a private file,
|
|
357
|
+
changing the index, editing `.gitignore`, or rewriting history. Report tracked
|
|
358
|
+
private configuration without reading its values and obtain specific approval for
|
|
359
|
+
any such Git changes. Never automatically untrack `.env` as a security cleanup.
|
|
360
|
+
|
|
361
|
+
## Updating an existing integration
|
|
362
|
+
|
|
363
|
+
Keep the full initial installation prompt for the first connection. For later
|
|
364
|
+
application changes, the shared maintenance skill is bundled at
|
|
365
|
+
`skills/noesis-update-integration/SKILL.md` in this SDK checkout and installed at
|
|
366
|
+
`<python-prefix>/share/noesis_ai/skills/noesis-update-integration/SKILL.md`.
|
|
367
|
+
The installation agent proposes and copies it into the current coding agent's
|
|
368
|
+
appropriate project-local skill directory, without overwriting existing rules or
|
|
369
|
+
granting permissions. If native discovery is unavailable, it records a readable
|
|
370
|
+
local copy and an explicit-read prompt instead. No separate per-agent SDK is needed.
|
|
371
|
+
|
|
372
|
+
Future skills repository: `<NOESIS_SKILLS_REPOSITORY_URL>` — **placeholder only**;
|
|
373
|
+
do not fetch it or guess a replacement. Use the approved bundled SDK snapshot now.
|
|
374
|
+
The integration README must record the actual skill source/version/destination,
|
|
375
|
+
verify the copied contents, and give a copyable update prompt with real paths.
|
|
376
|
+
|
|
377
|
+
Use the update skill when LLM calls, prompts, parsing, branching, aliases, baseline
|
|
378
|
+
configuration or SDK compatibility change. The agent inspects the relevant og app
|
|
379
|
+
context, proposes minimal integration edits and tests, and explains registration,
|
|
380
|
+
capture/review and evaluation requirements. It does not repeat account/key setup.
|
|
381
|
+
|
|
382
|
+
Initial integrations set `configuration_revision=1` on their capture and runner
|
|
383
|
+
`Noesis(...)` clients. Material approved workflow, alias, baseline or execution-policy
|
|
384
|
+
changes use exactly the next revision; the runner registers `Application.execution`.
|
|
385
|
+
Identical registration remains idempotent, including during active capture. A new
|
|
386
|
+
revision requires capture/operations/claims to be idle, preserves old configuration
|
|
387
|
+
and evidence, clears only the active-dataset selection, and requires fresh
|
|
388
|
+
capture/review before optimization. Same-label prompt/workflow changes still need
|
|
389
|
+
this deliberate update; there is no automatic application-source change detector.
|
|
390
|
+
|
|
391
|
+
Updates conflicting with saved serving labels/baseline are rejected rather than
|
|
392
|
+
changing production routing. Legacy backends may not support revision updates;
|
|
393
|
+
the skill must explain that blocker rather than bypass locks. Registration
|
|
394
|
+
acceptance alone does not establish live capture or validate old optimization
|
|
395
|
+
results for a changed workflow. Credentials, capture, polling, inference and
|
|
396
|
+
serving remain separately approved; verify CLI/backend versions before use.
|
|
397
|
+
|
|
398
|
+
## Release verification
|
|
399
|
+
|
|
400
|
+
For a reproducible installation, retain the resolved `noesis_ai` version and
|
|
401
|
+
artifact hashes in the og app's dependency lock. The installed modules are
|
|
402
|
+
`noesis_ai` and `noesis_ai_runner`; the console command is `noesis`.
|
|
403
|
+
Do not install the unrelated `noesis` distribution. Package installation and
|
|
404
|
+
imports never register an application, start polling, run inference or activate
|
|
405
|
+
serving. Those operations retain their separate approval requirements.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
"""Independent briefing workflow using an ordinary AsyncOpenAI-compatible client.
|
|
2
|
+
|
|
3
|
+
No Noesis imports or callable conventions. The caller supplies its normal client
|
|
4
|
+
and model configuration. Tests substitute inference, not this business logic.
|
|
5
|
+
"""
|
|
6
|
+
import json
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
async def build_brief(company, channels, *, llm, research_model, writer_model):
|
|
10
|
+
async def ask(model, instruction, data):
|
|
11
|
+
response = await llm.chat.completions.create(
|
|
12
|
+
model=model, max_tokens=256,
|
|
13
|
+
messages=[{"role": "system", "content": instruction},
|
|
14
|
+
{"role": "user", "content": json.dumps(data, ensure_ascii=False)}],
|
|
15
|
+
)
|
|
16
|
+
choice = response.choices[0]
|
|
17
|
+
if choice.finish_reason != "stop" or not choice.message.content:
|
|
18
|
+
raise ValueError("A complete briefing response is required")
|
|
19
|
+
return choice.message.content
|
|
20
|
+
|
|
21
|
+
assessment = json.loads(await ask(research_model, "Assess company: return JSON category and score.", company))
|
|
22
|
+
checks = await ask(research_model, "Check the assessment against company facts.",
|
|
23
|
+
{"company": company, "assessment": assessment})
|
|
24
|
+
draft = await ask(writer_model, "Draft a company brief for the requested channels.",
|
|
25
|
+
{"company": company, "channels": channels, "assessment": assessment, "checks": checks})
|
|
26
|
+
final = await ask(writer_model, "Revise the brief using the fact check.",
|
|
27
|
+
{"draft": draft, "checks": checks, "channels": channels})
|
|
28
|
+
return {"assessment": assessment, "research": {"checks": checks},
|
|
29
|
+
"delivery": {"channels": channels, "draft": draft, "text": final}}
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
"""Explicit attachment to the independent briefing app; no evaluation code."""
|
|
2
|
+
import os
|
|
3
|
+
import json
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
from contextlib import asynccontextmanager
|
|
6
|
+
from typing import Any, cast
|
|
7
|
+
|
|
8
|
+
import httpx
|
|
9
|
+
from openai import AsyncOpenAI
|
|
10
|
+
from noesis_ai import Application, Noesis
|
|
11
|
+
|
|
12
|
+
if __package__:
|
|
13
|
+
from .app import build_brief
|
|
14
|
+
else:
|
|
15
|
+
from app import build_brief
|
|
16
|
+
|
|
17
|
+
DECLARATION = {
|
|
18
|
+
"input": {"type": "text"},
|
|
19
|
+
"roles": {"research": "brief/research", "writer": "brief/write"},
|
|
20
|
+
"outputs": [
|
|
21
|
+
{"name": "category", "kind": "label", "config": {"choices": ["promising", "uncertain"]}},
|
|
22
|
+
{"name": "score", "kind": "number", "config": {"tolerance": 1, "min": 0, "max": 10}},
|
|
23
|
+
{"name": "text", "kind": "text"},
|
|
24
|
+
],
|
|
25
|
+
}
|
|
26
|
+
EXECUTION = {"version": 1, "coverage": {"kind": "fixed_alias_counts",
|
|
27
|
+
"counts": {"brief/research": 2, "brief/write": 2}}}
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
def make_application(noesis, client):
|
|
31
|
+
async def invoke(payload):
|
|
32
|
+
return await build_brief(payload["company"], payload["channels"], llm=client,
|
|
33
|
+
research_model=noesis.aliases["research"], writer_model=noesis.aliases["writer"])
|
|
34
|
+
|
|
35
|
+
def project(result):
|
|
36
|
+
return {"category": result["assessment"]["category"], "score": result["assessment"]["score"],
|
|
37
|
+
"text": result["delivery"]["text"]}
|
|
38
|
+
|
|
39
|
+
return Application(invoke, project=project, declaration=DECLARATION, execution=EXECUTION)
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
@asynccontextmanager
|
|
43
|
+
async def create_adapter(noesis=None, *, provider_transport=None, provider_key=None):
|
|
44
|
+
# Capture uses the SDK credential. Hosted runner startup is explicit below.
|
|
45
|
+
if noesis is None:
|
|
46
|
+
baseline = json.loads(Path(os.environ["NOESIS_BASELINE_FILE"]).read_text())
|
|
47
|
+
noesis = Noesis(key=os.environ["NOESIS_TRACER_KEY"], aliases=DECLARATION["roles"],
|
|
48
|
+
declaration=DECLARATION, baseline=baseline)
|
|
49
|
+
key = noesis.key
|
|
50
|
+
async with noesis:
|
|
51
|
+
direct = (noesis.baseline or {}).get("execution") == "customer_direct"
|
|
52
|
+
if direct:
|
|
53
|
+
# This example's original client is Anthropic-compatible. Other apps
|
|
54
|
+
# instrument their own original client factories, not a copied workflow.
|
|
55
|
+
if {p["provider"] for p in noesis.baseline["routes"].values()} != {"anthropic"}:
|
|
56
|
+
raise ValueError("Use the original provider client for this baseline")
|
|
57
|
+
key = provider_key or os.environ["ANTHROPIC_API_KEY"]
|
|
58
|
+
http = httpx.AsyncClient(base_url="https://api.anthropic.com/v1/",
|
|
59
|
+
transport=provider_transport, follow_redirects=False)
|
|
60
|
+
noesis.instrument_http_client(http, baseline=noesis.baseline)
|
|
61
|
+
else:
|
|
62
|
+
http = httpx.AsyncClient(base_url=noesis.url + "/live/v1/",
|
|
63
|
+
event_hooks={"request": [noesis.request_hook]}, follow_redirects=False)
|
|
64
|
+
async with http:
|
|
65
|
+
# Installed OpenAI annotations name httpx2; its injected HTTPX client
|
|
66
|
+
# interface is exercised by the real HTTP subprocess acceptance test.
|
|
67
|
+
async with AsyncOpenAI(api_key=key, base_url=http.base_url, http_client=cast(Any, http),
|
|
68
|
+
max_retries=0) as client:
|
|
69
|
+
yield noesis, make_application(noesis, client)
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
@asynccontextmanager
|
|
73
|
+
async def create_runner():
|
|
74
|
+
"""Use only the runner credential and an explicit server-supported baseline."""
|
|
75
|
+
baseline = json.loads(Path(os.environ["NOESIS_BASELINE_FILE"]).read_text())
|
|
76
|
+
if not isinstance(baseline, dict):
|
|
77
|
+
raise ValueError("Hosted runner requires a baseline descriptor")
|
|
78
|
+
noesis = Noesis(key=os.environ["NOESIS_RUNNER_KEY"], aliases=DECLARATION["roles"], declaration=DECLARATION, baseline=baseline)
|
|
79
|
+
async with create_adapter(noesis) as value:
|
|
80
|
+
yield value
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: noesis-update-integration
|
|
3
|
+
description: Update an existing Noesis integration after changes to the original application's LLM workflow, prompts, parsing, model roles, baseline configuration or SDK. Inspect the relevant application context, propose minimal adapter changes, verify offline and explain registration/capture/evaluation requirements without changing serving or running paid work.
|
|
4
|
+
metadata:
|
|
5
|
+
version: "0.1.0"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Update an existing Noesis integration
|
|
9
|
+
|
|
10
|
+
Use this skill for an existing installation, not to repeat initial onboarding. The full Noesis installation prompt remains the initial entrypoint.
|
|
11
|
+
|
|
12
|
+
## Ownership and limits
|
|
13
|
+
|
|
14
|
+
- The **og app** owns its workflow, prompts, parsing, tools, branching, history and provider configuration. Do not copy or rewrite it to fit Noesis.
|
|
15
|
+
- The local integration connects the actual og app through the installed SDK. The **noesis runner** executes only explicitly approved evaluations; the **noesis dashboard** controls capture, review, optimization and explicit activation.
|
|
16
|
+
- Reuse the installed SDK and existing integration factory. Do not patch installed SDK files, add customer-specific backend behavior, install another SDK or create a second integration without approval.
|
|
17
|
+
- Do not read secret values, `.env` contents or credential stores; print keys; change Git tracking; reset applications; rotate credentials; or delete evidence. Ask before commands that load the user's private configuration. Do not upload application source to Noesis.
|
|
18
|
+
- Inspection approval is not approval for edits, credential use, data uploads, capture, polling, inference, optimization, deployment or serving changes. Obtain the specific approval needed before each such action. Never use operator-funded frontier credentials or fallback.
|
|
19
|
+
|
|
20
|
+
## 1. Find the existing installation and relevant changes
|
|
21
|
+
|
|
22
|
+
Read the integration README, actual local integration/factory, relevant og app source and dependency declarations, and the SDK documentation referenced by that installation. Inspect relevant source changes/history when available; do not assume a complete Git diff or a particular framework exists. Ask which feature changed if context is missing.
|
|
23
|
+
|
|
24
|
+
Confirm the existing application identity using non-secret metadata. Preserve its keys and historical evidence; do not create a new application as a shortcut. Record the installed SDK version/source and inspect its supported interface before proposing commands. Never install the unrelated PyPI package named `noesis`.
|
|
25
|
+
|
|
26
|
+
The supported runner factory is an async context manager yielding `(Noesis, Application)` and owning resource cleanup. Importing/entering it must not launch the UI, execute the workload or start polling. Its declaration, baseline and execution configuration must agree with the actual adapter and approved job.
|
|
27
|
+
|
|
28
|
+
Understand the affected execution path, not only the integration file: input/output meaning, role selection, prompts, parsing, tools, branching, retry/call counts and provider settings. Do not claim unchanged behavior merely because aliases or JSON fields are unchanged.
|
|
29
|
+
|
|
30
|
+
## 2. Explain aliases and provider ownership
|
|
31
|
+
|
|
32
|
+
Aliases such as `acme/task1` identify model roles, not concrete provider models or final output fields. Preserve existing names when their meaning is unchanged. Adding/removing/renaming a role requires coordinated local declaration and server configuration changes; changing an approved model assignment for an unchanged role is a different operation.
|
|
33
|
+
|
|
34
|
+
For a customer-direct frontier baseline, the local SDK translates aliases to the existing provider model and keeps provider credentials/billing local. Anthropic/OpenAI/Google do not interpret Noesis aliases themselves. A dashboard assignment does not reconfigure that local provider connection.
|
|
35
|
+
|
|
36
|
+
During approved evaluation, temporary Noesis OSS assignments are isolated from ordinary responses and provider credentials are removed from redirected requests. With explicitly configured Noesis serving, the backend resolves application-scoped aliases to approved model assignments. Never automatically switch a provider, base URL, local client, fallback or production assignment. Hosted frontier BYOK remains paused.
|
|
37
|
+
|
|
38
|
+
## 3. Classify the impact and propose a minimal update
|
|
39
|
+
|
|
40
|
+
Before editing, report:
|
|
41
|
+
|
|
42
|
+
- What changed and which relevant paths were inspected; uncertainties remain explicit.
|
|
43
|
+
- Proposed integration/README/test edits, if any.
|
|
44
|
+
- Alias/declaration and baseline provider/model impact.
|
|
45
|
+
- Input/output projection, execution limits and call-topology impact.
|
|
46
|
+
- Whether original captures, reviewed references and previous optimization still describe this workflow.
|
|
47
|
+
- Required offline tests and whether registration, new capture/review or a new approved comparison is needed.
|
|
48
|
+
|
|
49
|
+
Use these distinctions:
|
|
50
|
+
|
|
51
|
+
| Change | Required treatment |
|
|
52
|
+
| --- | --- |
|
|
53
|
+
| Cosmetic/non-LLM change | Explain why no integration update is needed, if supported by inspection. |
|
|
54
|
+
| Integration repair, same configuration | Propose the repair and focused tests; identical registration can be idempotent. |
|
|
55
|
+
| Added/removed/renamed aliases or changed pinned baseline | Requires a supported explicit configuration update, not blind registration. |
|
|
56
|
+
| Changed prompts/parsing/tools/branching/model calls, even with the same aliases | Reassess execution bounds, reference meaning and applicability of old results; material changes require fresh capture/review before new optimization. |
|
|
57
|
+
|
|
58
|
+
Ask for approval of the concrete local changes and checks. Do not silently increase execution limits or spend, reinterpret old evidence, change rubric/expected answers, or rewrite existing observations to make an update pass.
|
|
59
|
+
|
|
60
|
+
## 4. Update and verify only the approved local integration
|
|
61
|
+
|
|
62
|
+
Reuse the original callable and existing client/model seam. Keep integration code thin and understandable; update the actual README paths, launch instructions and integration-specific tests. Do not rename an existing integration file as incidental cleanup.
|
|
63
|
+
|
|
64
|
+
Run only relevant offline tests using neutral mocked transports or the customer's existing local tests where safe. Verify original outputs/settings are preserved, correct alias binding, execution bounds, lazy factory startup and cleanup. Do not import a customer UI or run inference merely to test registration.
|
|
65
|
+
|
|
66
|
+
Classify a failed check as an integration, SDK/backend capability or application problem. Fix only the approved scope. Stop and report unsupported capability; never weaken tests or fabricate evidence.
|
|
67
|
+
|
|
68
|
+
## 5. Register supported configuration without bypassing conflicts
|
|
69
|
+
|
|
70
|
+
Registration submits declaration/baseline metadata; it does not inspect source code or certify a new feature. Identical registration is idempotent. A successful receipt does not prove live capture, current-workflow compatibility or optimization quality.
|
|
71
|
+
|
|
72
|
+
For this SDK/backend contract, a material approved update uses the next `configuration_revision` (exactly current + 1), shared by capture and runner `Noesis(...)` clients. This also covers material prompt/workflow changes with unchanged aliases: registration cannot infer them from source. The runner registers the adapter's execution policy. New revisions preserve configuration history and require fresh capture/review before new optimization; old approvals cannot be reused.
|
|
73
|
+
|
|
74
|
+
Read the previous receipt/README and any safe registered-revision error before proposing the next revision. Never blindly bump it to bypass a conflict. Stop active capture and wait for running operations/claims to finish before an approved update. If changed labels/baseline conflict with saved serving assignments, stop for separate serving review; registration must not change routing. Legacy backends may still lock all post-baseline declaration changes. Verify supported versions and stop on unsupported transitions. Do not clear history, overwrite observations, recreate an application or retry a changed declaration as unchanged.
|
|
75
|
+
|
|
76
|
+
Use the actual CLI supported by the installed SDK and recorded configuration. This SDK provides `uv run noesis register` (or `noesis register` in the activated project environment), using the non-secret `noesis.toml` created during initial installation. Keep it with the integration files; for an immediate child folder, `project_dir = ".."` preserves the og app root as working directory and `env_file = "../.env"` refers to an approved root private file. Run commands from the og app root; the CLI finds one root/immediate-child config, rejects ambiguity, and supports explicit `--config PATH` for deeper folders. Preserve existing root-level configs unless relocation is approved. For normal use and tracing, the updated SDK also provides `uv run noesis tracer`: configure `tracer_module` as the existing capture-enabled attachment's executable module in the same config. It reuses `project_dir` and `env_file`; it never starts the noesis runner, changes provider selection, grants consent or enables capture. Dashboard consent and Start tracing remain separate. Normal inference uses local customer-owned keys and billing. Do not assume an older installed SDK includes it. Verify each shortcut before use; obtain approval before updating the SDK or changing the attachment/config. For legacy SDKs, the documented module runner supports `--register-only`; verify the version and use the actual recorded factory/environment loader. Never leave unresolved placeholders in the handoff.
|
|
77
|
+
|
|
78
|
+
Obtain approval before a metadata-only registration command that loads credentials. Require `accepted=true` and `ready_for_capture=true` before claiming capture readiness. Registration-only exits without polling or invoking the application. Never omit its registration-only option as a retry strategy.
|
|
79
|
+
|
|
80
|
+
For a mismatch, name only the non-secret expected/actual configuration and give the next action, for example:
|
|
81
|
+
|
|
82
|
+
> Your local labels differ from the registered labels. Did you modify your application or labels? Complete this integration update before registering. The current backend may require a supported configuration transition; re-running registration alone cannot resolve the conflict.
|
|
83
|
+
|
|
84
|
+
## 6. Explain the next capture/evaluation step
|
|
85
|
+
|
|
86
|
+
Preserve old declarations, observations, snapshots, references, runs, approvals and pricing provenance. Historical results describe their recorded workflow, not automatically the changed one. Do not reuse an old approval for a changed configuration.
|
|
87
|
+
|
|
88
|
+
When new examples are required, explain that original answers are observations, not guaranteed correct references. Include representative normal requests, difficult cases and genuine incorrect-but-complete outputs; correct reviewed expectations without modifying original outputs. Do not manufacture bad baseline outputs or use only failures. Incomplete execution evidence is not a usable quality score.
|
|
89
|
+
|
|
90
|
+
Use the same approved judge/rubric configuration for baseline and candidates. Domain-specific criteria may help, but vague, contradictory or overly restrictive instructions can worsen evaluation; more instructions are not inherently better. Rubric drafts from a coding agent require customer review. Do not generate a new rubric or change criteria as an incidental repair.
|
|
91
|
+
|
|
92
|
+
Capture/upload, judge calls, optimization and runner polling each require their corresponding explicit consent/count/cost approval. Runner startup can execute previously approved jobs. Recommendations and serving activation remain separate; do not activate even a completed evaluated combination as part of maintenance.
|
|
93
|
+
|
|
94
|
+
## 7. Finish with facts, not a blanket success claim
|
|
95
|
+
|
|
96
|
+
Update the existing integration README and give a concise handoff:
|
|
97
|
+
|
|
98
|
+
- Files changed and offline checks actually passed.
|
|
99
|
+
- Configuration/declaration impact and any unresolved mismatch.
|
|
100
|
+
- Whether metadata registration was attempted/accepted and capture readiness confirmed.
|
|
101
|
+
- Whether fresh capture, review and evaluation are required; state what has not run.
|
|
102
|
+
- One concrete next action using the real local paths/commands and required approval.
|
|
103
|
+
|
|
104
|
+
If blocked, lead with the blocker. Do not say “optimization ready” solely because registration succeeded. Skills are instructions, not guaranteed change detection: a user can change the app without invoking them, and metadata equality cannot prove semantic equivalence.
|