loopiter 0.2.0a1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- loopiter-0.2.0a1/.gitignore +17 -0
- loopiter-0.2.0a1/LICENSE +21 -0
- loopiter-0.2.0a1/PKG-INFO +295 -0
- loopiter-0.2.0a1/README.md +268 -0
- loopiter-0.2.0a1/examples/reviewed_loop.py +260 -0
- loopiter-0.2.0a1/pyproject.toml +48 -0
- loopiter-0.2.0a1/src/loopiter/__init__.py +16 -0
- loopiter-0.2.0a1/src/loopiter/_validation.py +335 -0
- loopiter-0.2.0a1/src/loopiter/analysis.py +261 -0
- loopiter-0.2.0a1/src/loopiter/client.py +637 -0
- loopiter-0.2.0a1/src/loopiter/migrations/001-python-store.sql +25 -0
- loopiter-0.2.0a1/src/loopiter/postgres.py +161 -0
- loopiter-0.2.0a1/src/loopiter/py.typed +0 -0
- loopiter-0.2.0a1/src/loopiter/store.py +143 -0
- loopiter-0.2.0a1/src/loopiter/testing.py +95 -0
- loopiter-0.2.0a1/tests/postgres_worker.py +28 -0
- loopiter-0.2.0a1/tests/test_analysis.py +157 -0
- loopiter-0.2.0a1/tests/test_core.py +511 -0
- loopiter-0.2.0a1/tests/test_docs.py +55 -0
- loopiter-0.2.0a1/tests/test_example.py +32 -0
- loopiter-0.2.0a1/tests/test_postgres.py +142 -0
loopiter-0.2.0a1/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Loopiter contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,295 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: loopiter
|
|
3
|
+
Version: 0.2.0a1
|
|
4
|
+
Summary: Lightweight, review-first feedback loops for Python AI applications.
|
|
5
|
+
Project-URL: Homepage, https://loopiter.co
|
|
6
|
+
Project-URL: Documentation, https://loopiter.docs.buildwithfern.com/get-started/python-quickstart
|
|
7
|
+
Project-URL: Repository, https://github.com/SanaanKhalid/loopiter
|
|
8
|
+
Project-URL: Issues, https://github.com/SanaanKhalid/loopiter/issues
|
|
9
|
+
Author: Loopiter contributors
|
|
10
|
+
License-Expression: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Framework :: AsyncIO
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Provides-Extra: dev
|
|
21
|
+
Requires-Dist: build<2,>=1.2; extra == 'dev'
|
|
22
|
+
Requires-Dist: ruff<1,>=0.12; extra == 'dev'
|
|
23
|
+
Requires-Dist: twine<7,>=6; extra == 'dev'
|
|
24
|
+
Provides-Extra: postgres
|
|
25
|
+
Requires-Dist: psycopg[binary,pool]<4,>=3.2; extra == 'postgres'
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
|
|
28
|
+
# Loopiter for Python
|
|
29
|
+
|
|
30
|
+
Native, async Python SDK for **reviewed, evidence-driven AI improvements**.
|
|
31
|
+
No Node.js subprocess, hosted service, model dependency, telemetry, or core runtime
|
|
32
|
+
dependencies. Python **3.11+**, MIT. Python alpha version: **0.2.0a1** (PEP 440).
|
|
33
|
+
|
|
34
|
+
## Install the Python alpha
|
|
35
|
+
|
|
36
|
+
```sh
|
|
37
|
+
python -m pip install 'loopiter==0.2.0a1'
|
|
38
|
+
# Optional PostgreSQL adapter:
|
|
39
|
+
python -m pip install 'loopiter[postgres]==0.2.0a1'
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Use an explicit version for this prerelease. The Node.js SDK is distributed separately
|
|
43
|
+
on npm; installing either SDK does not install the other.
|
|
44
|
+
|
|
45
|
+
## Run the offline example from source
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
git clone https://github.com/SanaanKhalid/loopiter.git
|
|
49
|
+
cd loopiter
|
|
50
|
+
python3.11 -m venv .venv
|
|
51
|
+
source .venv/bin/activate
|
|
52
|
+
python -m pip install ./python
|
|
53
|
+
python python/examples/reviewed_loop.py
|
|
54
|
+
python python/examples/reviewed_loop.py --reject
|
|
55
|
+
python python/examples/reviewed_loop.py --interrupt
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Use an installed Python 3.11 or newer. The example runs offline, prints measured
|
|
59
|
+
fixture metrics, and demonstrates explicit approval, a real change to its in-process
|
|
60
|
+
classifier, rollback and interrupted-apply recovery. **Simulated fixtures, not LLM
|
|
61
|
+
performance evidence.** Its in-memory deployment registry is not production storage.
|
|
62
|
+
|
|
63
|
+
## Capture feedback
|
|
64
|
+
|
|
65
|
+
```python
|
|
66
|
+
import asyncio
|
|
67
|
+
from loopiter import FeedbackLoop, InMemoryStore
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
async def main():
|
|
71
|
+
loop = FeedbackLoop(store=InMemoryStore(), namespace="support/dev")
|
|
72
|
+
execution = await loop.record_execution(
|
|
73
|
+
id="request-123",
|
|
74
|
+
kind="prediction",
|
|
75
|
+
episode_id="ticket-123",
|
|
76
|
+
input={"text": "Please explain this charge."},
|
|
77
|
+
output={"label": "other"},
|
|
78
|
+
artifacts={"model": "your-model-version", "prompt": "intent-v1"},
|
|
79
|
+
metadata={"intent": "billing"},
|
|
80
|
+
)
|
|
81
|
+
await loop.record_signal(
|
|
82
|
+
id="review-123",
|
|
83
|
+
execution_id=execution["id"],
|
|
84
|
+
kind="correction",
|
|
85
|
+
name="verified_correct",
|
|
86
|
+
value=False,
|
|
87
|
+
correction={"label": "billing"},
|
|
88
|
+
source="authorized-reviewer",
|
|
89
|
+
confidence=1,
|
|
90
|
+
)
|
|
91
|
+
findings = await loop.analyze(
|
|
92
|
+
dimensions=["metadata.intent"],
|
|
93
|
+
minimum_support=1,
|
|
94
|
+
minimum_scored_count=1,
|
|
95
|
+
)
|
|
96
|
+
print(findings)
|
|
97
|
+
await loop.close()
|
|
98
|
+
|
|
99
|
+
|
|
100
|
+
if __name__ == "__main__":
|
|
101
|
+
asyncio.run(main())
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
All records are detached plain dictionaries using **snake_case**; access IDs with
|
|
105
|
+
`record["id"]`. Use `await` in an existing async application instead of nesting
|
|
106
|
+
`asyncio.run`. Inputs must be plain JSON; timestamps are ISO UTC strings ending in
|
|
107
|
+
`Z`. Unknown fields, NaN/infinity, non-string keys, cycles, NUL/lone-surrogate strings and missing safety fields
|
|
108
|
+
fail closed. Use explicit IDs to retry identical sanitized input idempotently;
|
|
109
|
+
conflicting input raises `LoopiterError(code="conflict")`. Defaults omitted on the
|
|
110
|
+
first request must remain omitted on retries. Revisions protect explicit updates.
|
|
111
|
+
|
|
112
|
+
Namespaces are integrity scopes, **not authentication**. Verify correction sources,
|
|
113
|
+
select authorized namespaces, and sanitize sensitive content in your application.
|
|
114
|
+
An async `sanitize(value)` hook runs before persistence and must return valid JSON;
|
|
115
|
+
failure never falls back to unsanitized input. Defaults: 256 KiB per input and
|
|
116
|
+
120 seconds per async callback. Sanitizers must be deterministic for retries and
|
|
117
|
+
must preserve deployment identity fields. Loopiter sends no data anywhere except
|
|
118
|
+
your explicitly supplied store and callbacks.
|
|
119
|
+
|
|
120
|
+
## PostgreSQL (optional)
|
|
121
|
+
|
|
122
|
+
```sh
|
|
123
|
+
python -m pip install 'loopiter[postgres]==0.2.0a1'
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Review the packaged versioned migration before running it with an authorized setup
|
|
127
|
+
identity. Constructors never execute migrations. Request-time credentials do not
|
|
128
|
+
need schema-creation permissions. For example, during **explicit setup only**:
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
import asyncio
|
|
132
|
+
import os
|
|
133
|
+
from psycopg import AsyncConnection
|
|
134
|
+
from loopiter.postgres import migration_sql
|
|
135
|
+
|
|
136
|
+
|
|
137
|
+
async def setup():
|
|
138
|
+
# SQL contains its own BEGIN/COMMIT; execute on a standalone autocommit connection.
|
|
139
|
+
async with await AsyncConnection.connect(os.environ["DATABASE_URL"], autocommit=True) as conn:
|
|
140
|
+
await conn.execute(migration_sql())
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
if __name__ == "__main__":
|
|
144
|
+
asyncio.run(setup())
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Runtime application:
|
|
148
|
+
|
|
149
|
+
```python
|
|
150
|
+
import asyncio
|
|
151
|
+
import os
|
|
152
|
+
from psycopg_pool import AsyncConnectionPool
|
|
153
|
+
from loopiter import FeedbackLoop
|
|
154
|
+
from loopiter.postgres import PostgresStore
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
async def main():
|
|
158
|
+
async with AsyncConnectionPool(os.environ["DATABASE_URL"], open=False) as pool:
|
|
159
|
+
await pool.wait()
|
|
160
|
+
loop = FeedbackLoop(store=PostgresStore(pool), namespace="support/dev")
|
|
161
|
+
await loop.record_execution(kind="prediction", input={"text": "Synthetic check"})
|
|
162
|
+
await loop.close() # Does not close your pool; its context manager does.
|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
if __name__ == "__main__":
|
|
166
|
+
asyncio.run(main())
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Use TLS, timeouts, least-privilege DB credentials, backups and restore drills. The
|
|
170
|
+
adapter uses one checked-out connection per transaction and namespace advisory
|
|
171
|
+
locking across independent clients/processes. Follow [psycopg's async pool lifecycle](https://www.psycopg.org/psycopg3/docs/advanced/pool.html).
|
|
172
|
+
No transaction is held during evaluation or deployment callbacks. Avoid nested
|
|
173
|
+
store transactions, spawned tasks within a transaction, and direct SQL record
|
|
174
|
+
mutations. These bypass or interfere with the adapter's guarantees.
|
|
175
|
+
|
|
176
|
+
Python uses contract **v1**, `loopiter_python_records`, separate migration/version
|
|
177
|
+
tables and lock keys. Node uses its existing contract v2 and tables. **The languages
|
|
178
|
+
do not share records, hashes, active pointers, or deployment locks.** They can use
|
|
179
|
+
the same PostgreSQL database, but must not independently control the same external
|
|
180
|
+
target. Choose one language as lifecycle owner and communicate through your own
|
|
181
|
+
application API if both languages participate in a single workflow. This alpha
|
|
182
|
+
does not provide a wire protocol, cross-language migration, or a shared service.
|
|
183
|
+
|
|
184
|
+
## Evaluate, approve, deploy
|
|
185
|
+
|
|
186
|
+
1. Analyze scored evidence with `await loop.analyze(dimensions=[...])`.
|
|
187
|
+
2. Create an immutable proposal with `await loop.create_candidate(target={"kind":
|
|
188
|
+
"prompt", "key": "support"}, proposed_change={...}, evidence={...}, risk="low")`.
|
|
189
|
+
Your application proposes it, using its own model or deterministic logic.
|
|
190
|
+
3. `evaluate_candidate(id, evaluator, evaluator_name=..., version=..., dataset_hash=...)`
|
|
191
|
+
calls your async `evaluator(candidate, cancellation_event)` outside the transaction.
|
|
192
|
+
It must return `{"passed": bool, "metrics": {"name": finite_number}}`. Your evaluator
|
|
193
|
+
must enforce independent holdout/guardrail thresholds; Loopiter does not invent them.
|
|
194
|
+
4. After human review, call `approve_candidate(id, actor=..., evaluation_id=...)` with
|
|
195
|
+
the exact latest passing evaluation's ID. Failed or stale evaluations cannot approve.
|
|
196
|
+
5. Explicitly call `deploy_candidate(id, adapter, expected_artifact_version=...)`.
|
|
197
|
+
Omit the initial version only when your infrastructure really has no existing artifact.
|
|
198
|
+
6. Use `rollback_candidate(id, adapter)` to restore its explicit predecessor/version.
|
|
199
|
+
|
|
200
|
+
Candidate IDs are insert-idempotency keys, not automatic semantic deduplication. For
|
|
201
|
+
proposal deduplication choose a stable ID derived with `fingerprint({...})` from the
|
|
202
|
+
target, proposed change, evidence fingerprint, evaluator version and dataset hash.
|
|
203
|
+
Changed evidence/evaluator versions should produce a new ID. No callback runs
|
|
204
|
+
automatically after feedback capture. No Python controller or unattended auto-apply
|
|
205
|
+
ships in this first alpha.
|
|
206
|
+
|
|
207
|
+
## Deployment and recovery contract
|
|
208
|
+
|
|
209
|
+
`DeploymentAdapter` exposes async `apply(request)`, `inspect(request)`, and
|
|
210
|
+
`rollback(request)`. Each request contains `attempt`, `candidate`,
|
|
211
|
+
`restore_candidate` (or `None`), stable `idempotency_key`, and a cooperative
|
|
212
|
+
`cancellation` event. Your adapter must durably record attempt IDs, compare the
|
|
213
|
+
expected artifact version atomically, implement real rollback, and fence late calls.
|
|
214
|
+
|
|
215
|
+
Apply and rollback return:
|
|
216
|
+
|
|
217
|
+
```python
|
|
218
|
+
receipt = {
|
|
219
|
+
"attempt_id": request["attempt"]["id"],
|
|
220
|
+
"artifact_version": "new-version", # Or None only when rollback removes an artifact.
|
|
221
|
+
"previous_artifact_version": "old-version", # Or None when genuinely absent.
|
|
222
|
+
}
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
Inspection returns `{"status": "applied", "receipt": receipt}`, `{"status": "unknown"}`,
|
|
226
|
+
or `{"status": "not_applied"}`. **not_applied means fenced**: an earlier operation
|
|
227
|
+
cannot still complete later. A temporary absence or HTTP timeout is not proof.
|
|
228
|
+
|
|
229
|
+
Loopiter persists the attempt before the external change, then atomically finalizes
|
|
230
|
+
receipt, lifecycle events, candidate states and the active pointer. Ambiguous outcomes
|
|
231
|
+
stay pending and block the target. On `LoopiterError` with code `deployment_pending`,
|
|
232
|
+
use `error.attempt_id` with `reconcile_deployment(attempt_id, adapter)`. On process
|
|
233
|
+
restart or task cancellation, paginate `loop.list("attempts")` and reconcile pending
|
|
234
|
+
attempts. Reconciliation only inspects; it never blindly repeats apply or rollback.
|
|
235
|
+
An unknown inspection stays pending. Repeated rollback cannot revive a rolled-back
|
|
236
|
+
version. This is not universal exactly-once execution.
|
|
237
|
+
|
|
238
|
+
Cancel a running SDK operation with `task.cancel()`. Cancellation/timeout discards
|
|
239
|
+
late callback results; it cannot sandbox code, kill a process or undo an external
|
|
240
|
+
change. Async callbacks must cooperate and must not block the event loop. Database
|
|
241
|
+
deadlines remain application-owned. `deployments_enabled=lambda: False` blocks new
|
|
242
|
+
applies, but rollback/reconciliation remain available. Pending attempts and lifecycle
|
|
243
|
+
events are available through paginated `list`; no logger, daemon, or scheduler is installed.
|
|
244
|
+
|
|
245
|
+
## Analysis semantics
|
|
246
|
+
|
|
247
|
+
Separate `execution_window` and `observation_window` (inclusive `from`, exclusive `to`)
|
|
248
|
+
allow a later outcome to score an older execution. Include older executions explicitly
|
|
249
|
+
when narrowing the cohort. Default scoring recognizes numeric/boolean values and
|
|
250
|
+
success/failure words. Supply a finite-valued synchronous `score(signal)` and a
|
|
251
|
+
`scoring_version` for custom semantics. Conflicting corrections remain separate
|
|
252
|
+
evidence; they are never silently promoted to trusted labels.
|
|
253
|
+
|
|
254
|
+
Episodes are scoring units where available; otherwise executions are units. Signals
|
|
255
|
+
deduplicate by ID within a unit; one episode outcome is not counted once per turn.
|
|
256
|
+
Zero-confidence or unscored signals do not add effective support or recurrence.
|
|
257
|
+
Counts are descriptive, not calibrated probabilities. Evidence includes a versioned
|
|
258
|
+
fingerprint of scored values, execution revisions, baseline and scoring version.
|
|
259
|
+
Default maximum is 20,000 executions and 20,000 signals per analysis; exceeding it
|
|
260
|
+
raises an explicit error. Pages max at 1,000. Analysis materializes that bounded
|
|
261
|
+
snapshot in memory; it is not a streaming warehouse engine.
|
|
262
|
+
|
|
263
|
+
## Scope and release checks
|
|
264
|
+
|
|
265
|
+
Python includes capture, updates, structured analysis, manual candidate lifecycle,
|
|
266
|
+
timeouts/cancellation, events, namespace deletion, an in-memory dev store, optional
|
|
267
|
+
PostgreSQL 16/17 adapter, conformance tests and an offline end-to-end example.
|
|
268
|
+
The Node controller/experimental auto-apply, JsonFileStore/legacy migration CLI,
|
|
269
|
+
and OpenAI/Azure classification starter remain **Node-only**. Neither SDK automatically
|
|
270
|
+
fine-tunes a model. Application callbacks, database permissions and deployment
|
|
271
|
+
fencing remain application responsibilities.
|
|
272
|
+
|
|
273
|
+
From the repository root:
|
|
274
|
+
|
|
275
|
+
```sh
|
|
276
|
+
python -m pip install './python[dev,postgres]'
|
|
277
|
+
python -m unittest discover -s python/tests -v
|
|
278
|
+
ruff check python scripts/python-package-smoke.py
|
|
279
|
+
ruff format --check python scripts/python-package-smoke.py
|
|
280
|
+
python -m build python
|
|
281
|
+
twine check python/dist/*
|
|
282
|
+
python scripts/python-package-smoke.py
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
Set `LOOPITER_PYTHON_TEST_DATABASE_URL` only to a disposable PostgreSQL database;
|
|
286
|
+
integration tests explicitly migrate and delete only their randomly named test
|
|
287
|
+
namespaces. CI runs Python 3.11–3.14 against PostgreSQL 16/17. Maintainers publish
|
|
288
|
+
through the manual `python-publish.yml` workflow with an exact version/commit,
|
|
289
|
+
all validation jobs passing, and the protected `pypi` environment approved. The
|
|
290
|
+
uploaded wheel/sdist are the same artifacts installed in clean-consumer tests.
|
|
291
|
+
Trusted Publishing uses short-lived GitHub identity; no release credentials belong
|
|
292
|
+
in this repository. See the [release procedure](https://github.com/SanaanKhalid/loopiter/blob/main/docs/python-release.md).
|
|
293
|
+
|
|
294
|
+
[Security reporting](https://github.com/SanaanKhalid/loopiter/security/advisories/new)
|
|
295
|
+
· [Canonical docs](https://loopiter.docs.buildwithfern.com/get-started/overview)
|
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
# Loopiter for Python
|
|
2
|
+
|
|
3
|
+
Native, async Python SDK for **reviewed, evidence-driven AI improvements**.
|
|
4
|
+
No Node.js subprocess, hosted service, model dependency, telemetry, or core runtime
|
|
5
|
+
dependencies. Python **3.11+**, MIT. Python alpha version: **0.2.0a1** (PEP 440).
|
|
6
|
+
|
|
7
|
+
## Install the Python alpha
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
python -m pip install 'loopiter==0.2.0a1'
|
|
11
|
+
# Optional PostgreSQL adapter:
|
|
12
|
+
python -m pip install 'loopiter[postgres]==0.2.0a1'
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Use an explicit version for this prerelease. The Node.js SDK is distributed separately
|
|
16
|
+
on npm; installing either SDK does not install the other.
|
|
17
|
+
|
|
18
|
+
## Run the offline example from source
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
git clone https://github.com/SanaanKhalid/loopiter.git
|
|
22
|
+
cd loopiter
|
|
23
|
+
python3.11 -m venv .venv
|
|
24
|
+
source .venv/bin/activate
|
|
25
|
+
python -m pip install ./python
|
|
26
|
+
python python/examples/reviewed_loop.py
|
|
27
|
+
python python/examples/reviewed_loop.py --reject
|
|
28
|
+
python python/examples/reviewed_loop.py --interrupt
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Use an installed Python 3.11 or newer. The example runs offline, prints measured
|
|
32
|
+
fixture metrics, and demonstrates explicit approval, a real change to its in-process
|
|
33
|
+
classifier, rollback and interrupted-apply recovery. **Simulated fixtures, not LLM
|
|
34
|
+
performance evidence.** Its in-memory deployment registry is not production storage.
|
|
35
|
+
|
|
36
|
+
## Capture feedback
|
|
37
|
+
|
|
38
|
+
```python
|
|
39
|
+
import asyncio
|
|
40
|
+
from loopiter import FeedbackLoop, InMemoryStore
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
async def main():
|
|
44
|
+
loop = FeedbackLoop(store=InMemoryStore(), namespace="support/dev")
|
|
45
|
+
execution = await loop.record_execution(
|
|
46
|
+
id="request-123",
|
|
47
|
+
kind="prediction",
|
|
48
|
+
episode_id="ticket-123",
|
|
49
|
+
input={"text": "Please explain this charge."},
|
|
50
|
+
output={"label": "other"},
|
|
51
|
+
artifacts={"model": "your-model-version", "prompt": "intent-v1"},
|
|
52
|
+
metadata={"intent": "billing"},
|
|
53
|
+
)
|
|
54
|
+
await loop.record_signal(
|
|
55
|
+
id="review-123",
|
|
56
|
+
execution_id=execution["id"],
|
|
57
|
+
kind="correction",
|
|
58
|
+
name="verified_correct",
|
|
59
|
+
value=False,
|
|
60
|
+
correction={"label": "billing"},
|
|
61
|
+
source="authorized-reviewer",
|
|
62
|
+
confidence=1,
|
|
63
|
+
)
|
|
64
|
+
findings = await loop.analyze(
|
|
65
|
+
dimensions=["metadata.intent"],
|
|
66
|
+
minimum_support=1,
|
|
67
|
+
minimum_scored_count=1,
|
|
68
|
+
)
|
|
69
|
+
print(findings)
|
|
70
|
+
await loop.close()
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
if __name__ == "__main__":
|
|
74
|
+
asyncio.run(main())
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
All records are detached plain dictionaries using **snake_case**; access IDs with
|
|
78
|
+
`record["id"]`. Use `await` in an existing async application instead of nesting
|
|
79
|
+
`asyncio.run`. Inputs must be plain JSON; timestamps are ISO UTC strings ending in
|
|
80
|
+
`Z`. Unknown fields, NaN/infinity, non-string keys, cycles, NUL/lone-surrogate strings and missing safety fields
|
|
81
|
+
fail closed. Use explicit IDs to retry identical sanitized input idempotently;
|
|
82
|
+
conflicting input raises `LoopiterError(code="conflict")`. Defaults omitted on the
|
|
83
|
+
first request must remain omitted on retries. Revisions protect explicit updates.
|
|
84
|
+
|
|
85
|
+
Namespaces are integrity scopes, **not authentication**. Verify correction sources,
|
|
86
|
+
select authorized namespaces, and sanitize sensitive content in your application.
|
|
87
|
+
An async `sanitize(value)` hook runs before persistence and must return valid JSON;
|
|
88
|
+
failure never falls back to unsanitized input. Defaults: 256 KiB per input and
|
|
89
|
+
120 seconds per async callback. Sanitizers must be deterministic for retries and
|
|
90
|
+
must preserve deployment identity fields. Loopiter sends no data anywhere except
|
|
91
|
+
your explicitly supplied store and callbacks.
|
|
92
|
+
|
|
93
|
+
## PostgreSQL (optional)
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
python -m pip install 'loopiter[postgres]==0.2.0a1'
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Review the packaged versioned migration before running it with an authorized setup
|
|
100
|
+
identity. Constructors never execute migrations. Request-time credentials do not
|
|
101
|
+
need schema-creation permissions. For example, during **explicit setup only**:
|
|
102
|
+
|
|
103
|
+
```python
|
|
104
|
+
import asyncio
|
|
105
|
+
import os
|
|
106
|
+
from psycopg import AsyncConnection
|
|
107
|
+
from loopiter.postgres import migration_sql
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
async def setup():
|
|
111
|
+
# SQL contains its own BEGIN/COMMIT; execute on a standalone autocommit connection.
|
|
112
|
+
async with await AsyncConnection.connect(os.environ["DATABASE_URL"], autocommit=True) as conn:
|
|
113
|
+
await conn.execute(migration_sql())
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
if __name__ == "__main__":
|
|
117
|
+
asyncio.run(setup())
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Runtime application:
|
|
121
|
+
|
|
122
|
+
```python
|
|
123
|
+
import asyncio
|
|
124
|
+
import os
|
|
125
|
+
from psycopg_pool import AsyncConnectionPool
|
|
126
|
+
from loopiter import FeedbackLoop
|
|
127
|
+
from loopiter.postgres import PostgresStore
|
|
128
|
+
|
|
129
|
+
|
|
130
|
+
async def main():
|
|
131
|
+
async with AsyncConnectionPool(os.environ["DATABASE_URL"], open=False) as pool:
|
|
132
|
+
await pool.wait()
|
|
133
|
+
loop = FeedbackLoop(store=PostgresStore(pool), namespace="support/dev")
|
|
134
|
+
await loop.record_execution(kind="prediction", input={"text": "Synthetic check"})
|
|
135
|
+
await loop.close() # Does not close your pool; its context manager does.
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
if __name__ == "__main__":
|
|
139
|
+
asyncio.run(main())
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Use TLS, timeouts, least-privilege DB credentials, backups and restore drills. The
|
|
143
|
+
adapter uses one checked-out connection per transaction and namespace advisory
|
|
144
|
+
locking across independent clients/processes. Follow [psycopg's async pool lifecycle](https://www.psycopg.org/psycopg3/docs/advanced/pool.html).
|
|
145
|
+
No transaction is held during evaluation or deployment callbacks. Avoid nested
|
|
146
|
+
store transactions, spawned tasks within a transaction, and direct SQL record
|
|
147
|
+
mutations. These bypass or interfere with the adapter's guarantees.
|
|
148
|
+
|
|
149
|
+
Python uses contract **v1**, `loopiter_python_records`, separate migration/version
|
|
150
|
+
tables and lock keys. Node uses its existing contract v2 and tables. **The languages
|
|
151
|
+
do not share records, hashes, active pointers, or deployment locks.** They can use
|
|
152
|
+
the same PostgreSQL database, but must not independently control the same external
|
|
153
|
+
target. Choose one language as lifecycle owner and communicate through your own
|
|
154
|
+
application API if both languages participate in a single workflow. This alpha
|
|
155
|
+
does not provide a wire protocol, cross-language migration, or a shared service.
|
|
156
|
+
|
|
157
|
+
## Evaluate, approve, deploy
|
|
158
|
+
|
|
159
|
+
1. Analyze scored evidence with `await loop.analyze(dimensions=[...])`.
|
|
160
|
+
2. Create an immutable proposal with `await loop.create_candidate(target={"kind":
|
|
161
|
+
"prompt", "key": "support"}, proposed_change={...}, evidence={...}, risk="low")`.
|
|
162
|
+
Your application proposes it, using its own model or deterministic logic.
|
|
163
|
+
3. `evaluate_candidate(id, evaluator, evaluator_name=..., version=..., dataset_hash=...)`
|
|
164
|
+
calls your async `evaluator(candidate, cancellation_event)` outside the transaction.
|
|
165
|
+
It must return `{"passed": bool, "metrics": {"name": finite_number}}`. Your evaluator
|
|
166
|
+
must enforce independent holdout/guardrail thresholds; Loopiter does not invent them.
|
|
167
|
+
4. After human review, call `approve_candidate(id, actor=..., evaluation_id=...)` with
|
|
168
|
+
the exact latest passing evaluation's ID. Failed or stale evaluations cannot approve.
|
|
169
|
+
5. Explicitly call `deploy_candidate(id, adapter, expected_artifact_version=...)`.
|
|
170
|
+
Omit the initial version only when your infrastructure really has no existing artifact.
|
|
171
|
+
6. Use `rollback_candidate(id, adapter)` to restore its explicit predecessor/version.
|
|
172
|
+
|
|
173
|
+
Candidate IDs are insert-idempotency keys, not automatic semantic deduplication. For
|
|
174
|
+
proposal deduplication choose a stable ID derived with `fingerprint({...})` from the
|
|
175
|
+
target, proposed change, evidence fingerprint, evaluator version and dataset hash.
|
|
176
|
+
Changed evidence/evaluator versions should produce a new ID. No callback runs
|
|
177
|
+
automatically after feedback capture. No Python controller or unattended auto-apply
|
|
178
|
+
ships in this first alpha.
|
|
179
|
+
|
|
180
|
+
## Deployment and recovery contract
|
|
181
|
+
|
|
182
|
+
`DeploymentAdapter` exposes async `apply(request)`, `inspect(request)`, and
|
|
183
|
+
`rollback(request)`. Each request contains `attempt`, `candidate`,
|
|
184
|
+
`restore_candidate` (or `None`), stable `idempotency_key`, and a cooperative
|
|
185
|
+
`cancellation` event. Your adapter must durably record attempt IDs, compare the
|
|
186
|
+
expected artifact version atomically, implement real rollback, and fence late calls.
|
|
187
|
+
|
|
188
|
+
Apply and rollback return:
|
|
189
|
+
|
|
190
|
+
```python
|
|
191
|
+
receipt = {
|
|
192
|
+
"attempt_id": request["attempt"]["id"],
|
|
193
|
+
"artifact_version": "new-version", # Or None only when rollback removes an artifact.
|
|
194
|
+
"previous_artifact_version": "old-version", # Or None when genuinely absent.
|
|
195
|
+
}
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Inspection returns `{"status": "applied", "receipt": receipt}`, `{"status": "unknown"}`,
|
|
199
|
+
or `{"status": "not_applied"}`. **not_applied means fenced**: an earlier operation
|
|
200
|
+
cannot still complete later. A temporary absence or HTTP timeout is not proof.
|
|
201
|
+
|
|
202
|
+
Loopiter persists the attempt before the external change, then atomically finalizes
|
|
203
|
+
receipt, lifecycle events, candidate states and the active pointer. Ambiguous outcomes
|
|
204
|
+
stay pending and block the target. On `LoopiterError` with code `deployment_pending`,
|
|
205
|
+
use `error.attempt_id` with `reconcile_deployment(attempt_id, adapter)`. On process
|
|
206
|
+
restart or task cancellation, paginate `loop.list("attempts")` and reconcile pending
|
|
207
|
+
attempts. Reconciliation only inspects; it never blindly repeats apply or rollback.
|
|
208
|
+
An unknown inspection stays pending. Repeated rollback cannot revive a rolled-back
|
|
209
|
+
version. This is not universal exactly-once execution.
|
|
210
|
+
|
|
211
|
+
Cancel a running SDK operation with `task.cancel()`. Cancellation/timeout discards
|
|
212
|
+
late callback results; it cannot sandbox code, kill a process or undo an external
|
|
213
|
+
change. Async callbacks must cooperate and must not block the event loop. Database
|
|
214
|
+
deadlines remain application-owned. `deployments_enabled=lambda: False` blocks new
|
|
215
|
+
applies, but rollback/reconciliation remain available. Pending attempts and lifecycle
|
|
216
|
+
events are available through paginated `list`; no logger, daemon, or scheduler is installed.
|
|
217
|
+
|
|
218
|
+
## Analysis semantics
|
|
219
|
+
|
|
220
|
+
Separate `execution_window` and `observation_window` (inclusive `from`, exclusive `to`)
|
|
221
|
+
allow a later outcome to score an older execution. Include older executions explicitly
|
|
222
|
+
when narrowing the cohort. Default scoring recognizes numeric/boolean values and
|
|
223
|
+
success/failure words. Supply a finite-valued synchronous `score(signal)` and a
|
|
224
|
+
`scoring_version` for custom semantics. Conflicting corrections remain separate
|
|
225
|
+
evidence; they are never silently promoted to trusted labels.
|
|
226
|
+
|
|
227
|
+
Episodes are scoring units where available; otherwise executions are units. Signals
|
|
228
|
+
deduplicate by ID within a unit; one episode outcome is not counted once per turn.
|
|
229
|
+
Zero-confidence or unscored signals do not add effective support or recurrence.
|
|
230
|
+
Counts are descriptive, not calibrated probabilities. Evidence includes a versioned
|
|
231
|
+
fingerprint of scored values, execution revisions, baseline and scoring version.
|
|
232
|
+
Default maximum is 20,000 executions and 20,000 signals per analysis; exceeding it
|
|
233
|
+
raises an explicit error. Pages max at 1,000. Analysis materializes that bounded
|
|
234
|
+
snapshot in memory; it is not a streaming warehouse engine.
|
|
235
|
+
|
|
236
|
+
## Scope and release checks
|
|
237
|
+
|
|
238
|
+
Python includes capture, updates, structured analysis, manual candidate lifecycle,
|
|
239
|
+
timeouts/cancellation, events, namespace deletion, an in-memory dev store, optional
|
|
240
|
+
PostgreSQL 16/17 adapter, conformance tests and an offline end-to-end example.
|
|
241
|
+
The Node controller/experimental auto-apply, JsonFileStore/legacy migration CLI,
|
|
242
|
+
and OpenAI/Azure classification starter remain **Node-only**. Neither SDK automatically
|
|
243
|
+
fine-tunes a model. Application callbacks, database permissions and deployment
|
|
244
|
+
fencing remain application responsibilities.
|
|
245
|
+
|
|
246
|
+
From the repository root:
|
|
247
|
+
|
|
248
|
+
```sh
|
|
249
|
+
python -m pip install './python[dev,postgres]'
|
|
250
|
+
python -m unittest discover -s python/tests -v
|
|
251
|
+
ruff check python scripts/python-package-smoke.py
|
|
252
|
+
ruff format --check python scripts/python-package-smoke.py
|
|
253
|
+
python -m build python
|
|
254
|
+
twine check python/dist/*
|
|
255
|
+
python scripts/python-package-smoke.py
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
Set `LOOPITER_PYTHON_TEST_DATABASE_URL` only to a disposable PostgreSQL database;
|
|
259
|
+
integration tests explicitly migrate and delete only their randomly named test
|
|
260
|
+
namespaces. CI runs Python 3.11–3.14 against PostgreSQL 16/17. Maintainers publish
|
|
261
|
+
through the manual `python-publish.yml` workflow with an exact version/commit,
|
|
262
|
+
all validation jobs passing, and the protected `pypi` environment approved. The
|
|
263
|
+
uploaded wheel/sdist are the same artifacts installed in clean-consumer tests.
|
|
264
|
+
Trusted Publishing uses short-lived GitHub identity; no release credentials belong
|
|
265
|
+
in this repository. See the [release procedure](https://github.com/SanaanKhalid/loopiter/blob/main/docs/python-release.md).
|
|
266
|
+
|
|
267
|
+
[Security reporting](https://github.com/SanaanKhalid/loopiter/security/advisories/new)
|
|
268
|
+
· [Canonical docs](https://loopiter.docs.buildwithfern.com/get-started/overview)
|