percolate-core 0.1.0__tar.gz

percolate_core-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.env

percolate_core-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Percolate
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

percolate_core-0.1.0/PKG-INFO

@@ -0,0 +1,195 @@
+Metadata-Version: 2.4
+Name: percolate-core
+Version: 0.1.0
+Summary: Percolate core: the worker, Content Server and Agent Runtime for a Postgres-native stack
+Project-URL: Homepage, https://github.com/percolating-sirsh/percolate-core
+Project-URL: Repository, https://github.com/percolating-sirsh/percolate-core
+Project-URL: Specs, https://github.com/percolating-sirsh/p8-subsystems
+Author: Percolate
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agents,pgvector,postgres,rag,workflow
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Requires-Dist: asyncpg>=0.30
+Requires-Dist: httpx>=0.28
+Requires-Dist: typer>=0.15
+Provides-Extra: agent
+Requires-Dist: fastapi>=0.115; extra == 'agent'
+Requires-Dist: mcp>=1.2; extra == 'agent'
+Requires-Dist: pydantic-ai-slim[openai]>=0.4; extra == 'agent'
+Requires-Dist: pydantic>=2.9; extra == 'agent'
+Requires-Dist: pyyaml>=6.0; extra == 'agent'
+Requires-Dist: rich>=13.9; extra == 'agent'
+Requires-Dist: uvicorn>=0.34; extra == 'agent'
+Provides-Extra: all
+Requires-Dist: boto3>=1.35; extra == 'all'
+Requires-Dist: fastapi>=0.115; extra == 'all'
+Requires-Dist: mcp>=1.2; extra == 'all'
+Requires-Dist: pydantic-ai-slim[openai]>=0.4; extra == 'all'
+Requires-Dist: pydantic>=2.9; extra == 'all'
+Requires-Dist: pyyaml>=6.0; extra == 'all'
+Requires-Dist: rich>=13.9; extra == 'all'
+Requires-Dist: uvicorn>=0.34; extra == 'all'
+Provides-Extra: content
+Requires-Dist: boto3>=1.35; extra == 'content'
+Requires-Dist: fastapi>=0.115; extra == 'content'
+Requires-Dist: uvicorn>=0.34; extra == 'content'
+Description-Content-Type: text/markdown
+
+# p8
+
+The processes that sit in front of the database. `../../INSTALL.md` covers the
+database itself; nothing here is required to use it.
+
+```bash
+pip install p8                 # the worker -- asyncpg, httpx, typer
+pip install 'p8[content]'      # + Content Server (boto3, fastapi)
+pip install 'p8[agent]'        # + Agent Runtime (pydantic-ai, mcp)
+pip install 'p8[all]'
+```
+
+```bash
+p8 worker --queue http    # claim and execute tasks
+p8 content serve          # uploads, scraping, http_call execution
+p8 agent serve            # agents, streaming, delegation
+```
+
+---
+
+## Layout
+
+| Module | Extra | What it is |
+|---|---|---|
+| `p8.core` | — | connecting **as the caller**, configuration, credential resolution |
+| `p8.worker` | — | the step loop and the `@handler` registry |
+| `p8.content` | `content` | the Content Server |
+| `p8.agentic` | `agent` | the Agent Runtime |
+
+**One distribution rather than three.** Three packages means a version matrix
+(`p8content 0.3` requiring `p8core >=0.2,<0.3`) resolved forever, for services
+that release together and are written by the same people. Splitting later is
+mechanical; merging two that have drifted is not.
+
+**The base install stays small on purpose.** The common case is someone writing
+their own worker, and they should not pull boto3 and pydantic-ai to do it. The
+CLI imports each subpackage lazily, so `p8 worker` runs without either
+installed and `p8 content serve` fails with the extra to install rather than an
+`ImportError`.
+
+---
+
+## `p8.core` — the one that matters
+
+It decides **whose** RLS applies, and it exists because that logic was
+previously written three times on two different database drivers.
+
+```python
+from p8.core import as_caller
+
+async with as_caller(claims) as conn:      # claims = the VERIFIED JWT payload
+    rows = await conn.fetch("select * from content.resources")
+```
+
+Every service connects as a low-privilege role and sets the caller's claims
+**per transaction**, exactly as PostgREST does. A service that queried as
+*itself* would bypass every policy in the collection — not by exploiting
+anything, just by never presenting an identity for the policies to filter on.
+
+Transaction-local (`set_config(..., true)`) is not a detail: an unregistered
+GUC left at session scope survives into the next transaction on a pooled
+connection, so the following request would inherit the previous caller's
+identity.
+
+`as_service()` exists for work with genuinely no user behind it — a scheduled
+poll, a reconciliation sweep. Deliberately a separate function rather than
+`as_caller(None)`, so "this query has no user" is something someone wrote down.
+
+---
+
+## Writing your own worker
+
+Most steps need no worker from you:
+
+| kind | who runs it |
+|---|---|
+| `sql` / `p8ql` | **nobody** — executes inside Postgres |
+| `http_call` | the built-in handler |
+| `timer` / `signal` / `decision` / `sub_workflow` | the engine |
+| `work` | **you** |
+
+When you do need one, it is this loop with a handler registered — not a
+different program:
+
+```python
+from p8.worker import handler, run
+
+@handler("transcode")
+async def transcode(spec, ctx):
+    return {"duration": await ffmpeg(ctx["run_input"]["file_key"])}
+
+run(queue="media")
+```
+
+`ctx` comes from `workflow.get_task_context()`: `run_input`, the accumulated
+`context` (so a later step reads an earlier step's output), `task_input`,
+`step_key`, and `trace_id`/`span_id`.
+
+**The worker holds no table grants.** Every interaction is a `SECURITY DEFINER`
+function call — `claim_task`, `get_task_context`, `complete_task`, `fail_task`
+— which is why "bring your own worker" is safe to offer: a compromised worker
+can claim and complete tasks, and nothing else.
+
+**Raise `TerminalError` for what will not get better.** A bad argument, a
+missing credential, a 404. Anything else is retried with backoff. The worker is
+the only thing that knows what a failure means, so it decides and the engine
+honours the verdict.
+
+---
+
+## Configuration
+
+Environment only. Credentials by **reference**, never by value:
+`credential_ref: "LLM_API_KEY"` on a task names a variable this process
+resolves, so `workflow.tasks` stays inspectable and replayable.
+
+| | Used by |
+|---|---|
+| `P8_DSN` | all |
+| `P8_JWT_SECRET` | services verifying bearer tokens (same secret PostgREST uses) |
+| `P8_QUEUE`, `P8_WORKER_ID`, `P8_POLL_SECONDS` | worker |
+| `P8_S3_ENDPOINT`, `P8_S3_KEY`, `P8_S3_SECRET`, `P8_BUCKET` | content |
+
+---
+
+## Deployment
+
+One image, three entrypoints — they share `p8.core`, so three images would be
+three builds of the same base and three tags to keep in step.
+
+```yaml
+content: { image: p8/runtime:0.1, command: ["p8","content","serve"] }
+agent:   { image: p8/runtime:0.1, command: ["p8","agent","serve"] }
+worker:  { image: p8/runtime:0.1, command: ["p8","worker","--queue","http"] }
+```
+
+See [`../../PACKAGING.md`](../../PACKAGING.md) for the reasoning and the
+remaining work.
+
+---
+
+## Status
+
+- `p8.core`, `p8.worker`, `p8.content` — built, and exercised against a live
+  PG19 instance with MinIO.
+- `p8.agentic` — moved from `experiments/agentic-runtime` unchanged (it uses
+  relative imports throughout, so the move needed no edits). Its own tests have
+  not been re-run under the new namespace.
+- No Dockerfile yet. No service-surface entries in `../../surface.sql`, which
+  by `meta/skills/spec-driven-development` §7 should exist **before** the
+  endpoints they describe.

percolate_core-0.1.0/README.md

@@ -0,0 +1,151 @@
+# p8
+
+The processes that sit in front of the database. `../../INSTALL.md` covers the
+database itself; nothing here is required to use it.
+
+```bash
+pip install p8                 # the worker -- asyncpg, httpx, typer
+pip install 'p8[content]'      # + Content Server (boto3, fastapi)
+pip install 'p8[agent]'        # + Agent Runtime (pydantic-ai, mcp)
+pip install 'p8[all]'
+```
+
+```bash
+p8 worker --queue http    # claim and execute tasks
+p8 content serve          # uploads, scraping, http_call execution
+p8 agent serve            # agents, streaming, delegation
+```
+
+---
+
+## Layout
+
+| Module | Extra | What it is |
+|---|---|---|
+| `p8.core` | — | connecting **as the caller**, configuration, credential resolution |
+| `p8.worker` | — | the step loop and the `@handler` registry |
+| `p8.content` | `content` | the Content Server |
+| `p8.agentic` | `agent` | the Agent Runtime |
+
+**One distribution rather than three.** Three packages means a version matrix
+(`p8content 0.3` requiring `p8core >=0.2,<0.3`) resolved forever, for services
+that release together and are written by the same people. Splitting later is
+mechanical; merging two that have drifted is not.
+
+**The base install stays small on purpose.** The common case is someone writing
+their own worker, and they should not pull boto3 and pydantic-ai to do it. The
+CLI imports each subpackage lazily, so `p8 worker` runs without either
+installed and `p8 content serve` fails with the extra to install rather than an
+`ImportError`.
+
+---
+
+## `p8.core` — the one that matters
+
+It decides **whose** RLS applies, and it exists because that logic was
+previously written three times on two different database drivers.
+
+```python
+from p8.core import as_caller
+
+async with as_caller(claims) as conn:      # claims = the VERIFIED JWT payload
+    rows = await conn.fetch("select * from content.resources")
+```
+
+Every service connects as a low-privilege role and sets the caller's claims
+**per transaction**, exactly as PostgREST does. A service that queried as
+*itself* would bypass every policy in the collection — not by exploiting
+anything, just by never presenting an identity for the policies to filter on.
+
+Transaction-local (`set_config(..., true)`) is not a detail: an unregistered
+GUC left at session scope survives into the next transaction on a pooled
+connection, so the following request would inherit the previous caller's
+identity.
+
+`as_service()` exists for work with genuinely no user behind it — a scheduled
+poll, a reconciliation sweep. Deliberately a separate function rather than
+`as_caller(None)`, so "this query has no user" is something someone wrote down.
+
+---
+
+## Writing your own worker
+
+Most steps need no worker from you:
+
+| kind | who runs it |
+|---|---|
+| `sql` / `p8ql` | **nobody** — executes inside Postgres |
+| `http_call` | the built-in handler |
+| `timer` / `signal` / `decision` / `sub_workflow` | the engine |
+| `work` | **you** |
+
+When you do need one, it is this loop with a handler registered — not a
+different program:
+
+```python
+from p8.worker import handler, run
+
+@handler("transcode")
+async def transcode(spec, ctx):
+    return {"duration": await ffmpeg(ctx["run_input"]["file_key"])}
+
+run(queue="media")
+```
+
+`ctx` comes from `workflow.get_task_context()`: `run_input`, the accumulated
+`context` (so a later step reads an earlier step's output), `task_input`,
+`step_key`, and `trace_id`/`span_id`.
+
+**The worker holds no table grants.** Every interaction is a `SECURITY DEFINER`
+function call — `claim_task`, `get_task_context`, `complete_task`, `fail_task`
+— which is why "bring your own worker" is safe to offer: a compromised worker
+can claim and complete tasks, and nothing else.
+
+**Raise `TerminalError` for what will not get better.** A bad argument, a
+missing credential, a 404. Anything else is retried with backoff. The worker is
+the only thing that knows what a failure means, so it decides and the engine
+honours the verdict.
+
+---
+
+## Configuration
+
+Environment only. Credentials by **reference**, never by value:
+`credential_ref: "LLM_API_KEY"` on a task names a variable this process
+resolves, so `workflow.tasks` stays inspectable and replayable.
+
+| | Used by |
+|---|---|
+| `P8_DSN` | all |
+| `P8_JWT_SECRET` | services verifying bearer tokens (same secret PostgREST uses) |
+| `P8_QUEUE`, `P8_WORKER_ID`, `P8_POLL_SECONDS` | worker |
+| `P8_S3_ENDPOINT`, `P8_S3_KEY`, `P8_S3_SECRET`, `P8_BUCKET` | content |
+
+---
+
+## Deployment
+
+One image, three entrypoints — they share `p8.core`, so three images would be
+three builds of the same base and three tags to keep in step.
+
+```yaml
+content: { image: p8/runtime:0.1, command: ["p8","content","serve"] }
+agent:   { image: p8/runtime:0.1, command: ["p8","agent","serve"] }
+worker:  { image: p8/runtime:0.1, command: ["p8","worker","--queue","http"] }
+```
+
+See [`../../PACKAGING.md`](../../PACKAGING.md) for the reasoning and the
+remaining work.
+
+---
+
+## Status
+
+- `p8.core`, `p8.worker`, `p8.content` — built, and exercised against a live
+  PG19 instance with MinIO.
+- `p8.agentic` — moved from `experiments/agentic-runtime` unchanged (it uses
+  relative imports throughout, so the move needed no edits). Its own tests have
+  not been re-run under the new namespace.
+- No Dockerfile yet. No service-surface entries in `../../surface.sql`, which
+  by `meta/skills/spec-driven-development` §7 should exist **before** the
+  endpoints they describe.

percolate_core-0.1.0/percolate_core/__init__.py

@@ -0,0 +1,13 @@
+"""percolate-core — Postgres-native subsystems.
+
+The database is the system; these are the processes that sit in front of it.
+Everything here connects through the same RLS-scoped roles and holds no table
+grants: every interaction is a SECURITY DEFINER function call.
+
+    percolate_core.core      connecting as the caller, configuration   (always)
+    percolate_core.worker    the step loop and @handler registry       (always)
+    percolate_core.content   the Content Server                        [content]
+    percolate_core.agentic   the Agent Runtime                         [agent]
+"""
+
+__version__ = "0.1.0"

percolate_core-0.1.0/percolate_core/agentic/__init__.py

@@ -0,0 +1,48 @@
+"""p8agentic — the agent runtime speced in ``specs/agentic``.
+
+Agents are rows in Postgres, tools are MCP/OpenAPI services reached over the
+network, a turn's rows are written through RLS as an ordinary ``authenticated``
+caller, AG-UI events stream out over ``NOTIFY``, and delegation is an MCP call
+into this runtime's own agent gateway. ``dev/verify.py`` asserts all of that
+against a live LLM.
+
+Layout, mapped onto ``specs/agentic/brief.md``:
+
+- ``contracts``  — the ontology as pydantic types (brief §7, ontology.md), plus
+  the two wire protocols (brief §5.1 item 2).
+- ``db``         — a small asyncpg client and the ``Repository`` protocol every
+  read/write in this runtime goes through (brief §5: "through the same functions
+  and RLS any other client would go through, no privileged backdoor").
+- ``tools``      — tool-server resolution and invocation: MCP is the canonical
+  shape, OpenAPI is an adapter onto it (brief §5.2). Zero built-in tools.
+- ``runtime``    — the four components of brief §5.1, one module each, plus the
+  chained action (§4) and citation (§5.3) mechanisms.
+- ``sequencer``  — the ordered steps of one agent turn, and the mapping from
+  ``specs/agentic/sequence.md``'s phase gates onto demos that prove them.
+- ``fakes``      — pydantic-ai fake models and an in-memory repository, so the
+  whole path above runs with no Postgres and no LLM credentials.
+- ``cli``        — the operator surface. FastAPI wraps this later (brief §4:
+  streaming is the one piece that structurally can't be plain PostgREST).
+"""
+
+__version__ = "0.0.1"
+
+
+def build_fingerprint() -> str:
+    """A short hash of this package's source on disk.
+
+    Exposed by every long-running surface (`/_build`) so a caller can tell
+    whether the process it is talking to is running the code in the working
+    tree. Three separate debugging sessions in this module's history began with
+    a confusing failure whose cause was a server started before the fix — a
+    stale process fails in the shape of a bug, and that is worth fifteen lines
+    to make impossible.
+    """
+    import hashlib
+    from pathlib import Path
+
+    digest = hashlib.sha256()
+    for path in sorted(Path(__file__).parent.rglob("*.py")):
+        digest.update(path.name.encode())
+        digest.update(str(path.stat().st_mtime_ns).encode())
+    return digest.hexdigest()[:12]