graph-agents-cli 0.3.1__py3-none-any.whl

graph_agents_cli/skills/data/graph-agents-cli-langgraph-code/references/langchain-models.md

@@ -0,0 +1,124 @@
+# LangChain models: `init_chat_model` and provider switching
+
+The agent never names a provider in code. `app/app_utils/model.py` reads the environment and calls
+`init_chat_model`; the same function builds the judge from `JUDGE_*`.
+
+## Provider table
+
+| `MODEL_PROVIDER` | LangChain provider | Key variable | Extra variables | Notes |
+|---|---|---|---|---|
+| `openai` | `openai` | `OPENAI_API_KEY` | | default under `-y` |
+| `anthropic` | `anthropic` | `ANTHROPIC_API_KEY` | | |
+| `gemini` | `google_genai` | `GOOGLE_API_KEY` | | the AI Studio API-key path only |
+| `openai-compatible` | `openai` with `base_url` | `MODEL_API_KEY` (sent as the OpenAI key) | `OPENAI_BASE_URL` | Ollama, vLLM, TGI, OpenRouter, any OpenAI-compatible server |
+| `fake` | the template's deterministic `FakeChatModel` (`app/app_utils/model.py`) | none | | tests and CI only; never offered by `create`; replies depend only on the input (greeting, a request mentioning a bound tool -> a call of it, judge prompt -> JSON verdict) |
+
+Default model names per provider are recorded by `create` in `.env` and the manifest
+(`create_params.model`). They are placeholders to verify at release time; do not replace them
+from memory.
+
+## `get_model()` shape (as `app/app_utils/model.py` implements it)
+
+```python
+from langchain.chat_models import init_chat_model  # imported lazily, inside build_model
+
+PROVIDER_TO_LANGCHAIN = {
+    "openai": "openai",
+    "anthropic": "anthropic",
+    "gemini": "google_genai",
+    "openai-compatible": "openai",
+}
+PROVIDER_KEY_VARS = {
+    "openai": "OPENAI_API_KEY",
+    "anthropic": "ANTHROPIC_API_KEY",
+    "gemini": "GOOGLE_API_KEY",
+    "openai-compatible": "MODEL_API_KEY",
+}
+
+
+def model_settings(*, judge=False) -> dict:
+    # MODEL_PROVIDER (default "openai"), MODEL_NAME, OPENAI_BASE_URL, the provider's key variable;
+    # with judge=True each is overridden by JUDGE_MODEL_PROVIDER / JUDGE_MODEL_NAME / JUDGE_BASE_URL /
+    # JUDGE_API_KEY when set. Read at CALL time, so tests and the eval judge runner can set them late.
+    ...
+
+
+def build_model(provider, name, *, base_url=None, api_key=None, **kwargs):
+    if provider == "fake":
+        return FakeChatModel(**kwargs)
+    # unknown provider -> ValueError; empty MODEL_NAME -> ValueError;
+    # openai-compatible without OPENAI_BASE_URL -> ValueError
+    return init_chat_model(
+        name, model_provider=PROVIDER_TO_LANGCHAIN[provider], api_key=..., base_url=..., **kwargs
+    )
+
+
+def get_model(**kwargs):  # the agent's model
+    return build_model(**model_settings(), **kwargs)
+
+
+def get_judge_model(**kwargs):  # the judge; JUDGE_* with the agent's values as defaults
+    return build_model(**model_settings(judge=True), **kwargs)
+
+
+def model_label(*, judge=False) -> str:  # "<provider>/<model>", recorded in run records and traces
+    ...
+```
+
+`get_judge_model()` takes no provider or model arguments; `eval grade --judge-provider/--judge-model`
+reach it through `JUDGE_MODEL_PROVIDER` / `JUDGE_MODEL_NAME` in the judge runner's environment.
+
+## OpenAI-API models: reasoning effort and the Responses API
+
+For `openai` and `openai-compatible` only (set for another provider, the app refuses to start):
+
+| Variable | Values | Effect |
+|---|---|---|
+| `MODEL_REASONING_EFFORT` | `none`, `minimal`, `low`, `medium`, `high`, `xhigh`; unset = the model's default | `reasoning_effort` on Chat Completions, `reasoning.effort` on the Responses API |
+| `MODEL_USE_RESPONSES_API` | `true`, `false`; unset = langchain-openai chooses | `true`: every request to `/v1/responses`; `false`: `/v1/chat/completions` |
+
+`build_model` passes them to `init_chat_model` as `reasoning_effort` and `use_responses_api`
+(an explicit argument wins). Unset, langchain-openai uses Chat Completions except for the
+models its own list says need the Responses API. A model that refuses function tools with a
+reasoning effort on Chat Completions fails its first call with a 400 naming `/v1/responses`:
+set `MODEL_USE_RESPONSES_API=true` (or `MODEL_REASONING_EFFORT=none`). Keep it `false` or unset
+for an OpenAI-compatible server without `/v1/responses`. The judge reads
+`JUDGE_REASONING_EFFORT` / `JUDGE_USE_RESPONSES_API`, defaulting to the agent's when the judge
+is an OpenAI-API model too. Tools, streaming and token usage work on both APIs.
+
+## Switching providers
+
+Edit `.env` (locally) and the chart `env:` plus the Secret (deployed):
+
+```bash
+# hosted -> on-network vLLM
+MODEL_PROVIDER=openai-compatible
+MODEL_NAME=Qwen/Qwen2.5-14B-Instruct
+OPENAI_BASE_URL=http://vllm.models.svc:8000/v1
+MODEL_API_KEY=none
+```
+
+Then update the manifest's `secrets.keys` if the provider key variable changed (the allow-list
+names the variable `secrets apply` exports), and re-run `secrets apply` and `deploy --restart`.
+`scaffold enhance` can rewrite `create_params.model_provider` and the allow-list for you.
+
+## Tool calling on open models
+
+`create_agent` and `ToolNode` depend on native function calling. Use Llama 3.1+, Qwen 2.5+, or
+Mistral families served with tool-call parsing enabled (vLLM: `--enable-auto-tool-choice
+--tool-call-parser <parser>`; Ollama: a model tagged with tools). Small or older models loop or
+emit malformed calls; the eval gate (`expect.tool_calls`) catches this.
+
+## Judge model
+
+The judge is a chat model like any other, built from `JUDGE_MODEL_PROVIDER`, `JUDGE_MODEL_NAME`,
+`JUDGE_BASE_URL`, and `JUDGE_API_KEY`, defaulting to the agent's configuration. In the
+disconnected profile it points at an on-network OpenAI-compatible server, possibly the same one.
+`eval_config.yaml` `judge: {provider, model}` overrides the environment for a run.
+
+## Egress
+
+Selecting a hosted provider sends prompts, tool results, and any context the agent assembles to
+that provider. That is a policy decision the consuming project makes explicitly (and publishes a
+privacy notice for) before connecting a hosted model. Local hosting alone does not make data local
+if traces or evals are exported elsewhere; see `/graph-agents-cli-observability`.

graph_agents_cli/skills/data/graph-agents-cli-langgraph-code/references/langgraph.md

@@ -0,0 +1,235 @@
+# LangGraph patterns used by the template
+
+Patterns below are the ones the template and its tests rely on. For anything else, fetch the
+upstream LangGraph docs; do not guess API names from memory.
+
+## `create_agent` (default)
+
+```python
+from langchain.agents import create_agent
+from app.app_utils.model import get_model
+from app.tools import TOOLS
+
+graph = create_agent(model=get_model(), tools=TOOLS, system_prompt=SYSTEM_PROMPT)
+```
+
+A ReAct loop: model -> tool calls -> tool node -> model until no tool call remains. State is
+`MessagesState` (`{"messages": [...]}`). Unbound; the app binds the checkpointer.
+
+## Explicit `StateGraph`
+
+```python
+from typing import Annotated, TypedDict
+from langchain_core.messages import AnyMessage
+from langgraph.graph import StateGraph, START, END, MessagesState, add_messages
+from langgraph.prebuilt import ToolNode, tools_condition
+
+
+class State(MessagesState):
+    site_id: str | None  # extra keys beside messages
+
+
+def call_model(state: State):
+    model = get_model().bind_tools(TOOLS)
+    return {"messages": [model.invoke([SystemMessage(SYSTEM_PROMPT), *state["messages"]])]}
+
+
+builder = StateGraph(State)
+builder.add_node("model", call_model)
+builder.add_node("tools", ToolNode(TOOLS))
+builder.add_edge(START, "model")
+builder.add_conditional_edges("model", tools_condition, {"tools": "tools", END: END})
+builder.add_edge("tools", "model")
+
+graph = builder.compile()  # still unbound
+```
+
+Use an explicit graph for fixed stages (classify -> retrieve -> answer), branching by state, or a
+human-approval node. Keep the export `graph`.
+
+## Tools
+
+```python
+from langchain_core.tools import tool
+
+API_CALLS = [
+    {"api": "sites", "method": "GET", "operation_id": "getSite", "path": "/sites/{site_id}"},
+    {"api": "sites", "method": "PATCH", "operation_id": "renameSite", "path": "/sites/{site_id}"},
+]
+
+
+@tool
+async def get_site(site_id: str) -> str:
+    """Return the site record for SITE_ID."""
+    ...
+```
+
+- The docstring is the model's description of the tool; write it for the model.
+- Typed parameters become the schema; keep them simple (str, int, bool, small models).
+- Return strings or JSON-serialisable data; long payloads bloat the context and the trace.
+- Tools that call an external API use `get_client("<api>")` from `app_utils.api_client` and
+  declare `API_CALLS`.
+- Tools that need the caller's identity read it from the run context the app sets
+  (`runtime: ToolRuntime[Any]` parameter, `runtime.context`: `principal_id`, `roles`,
+  `attributes`; or `current_caller(runtime.context)` from `app_utils.api_client`); never from a
+  global. Parameterise `ToolRuntime`: the bare form makes pydantic warn with the run context
+  (which may hold credentials) on every call.
+- Tool results are untrusted data: write tools act only on ids the user named
+  (`require_user_mentioned`) and on the caller's own records (`require_owner`); see section 2a
+  of the skill.
+
+## Checkpointers and `thread_id`
+
+The app (not `agent.py`) does:
+
+```python
+# app/app_utils/checkpointer.py
+def get_checkpointer():
+    if os.environ.get("CHECKPOINTER", "memory") == "postgres":
+        from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver
+
+        saver = AsyncPostgresSaver.from_conn_string(os.environ["POSTGRES_DSN"])
+        await saver.setup()
+        return saver
+    from langgraph.checkpoint.memory import InMemorySaver
+
+    return InMemorySaver()
+
+
+# app/fast_api_app.py
+runnable = graph.with_config(...)  # or graph rebuilt with checkpointer=get_checkpointer()
+config = {"configurable": {"thread_id": thread_id, "principal": principal}}
+```
+
+- `thread_id` selects the checkpoint namespace; each `/chat` call on the same id continues the
+  conversation.
+- Under `memory`, state lives only in the process. Under `postgres`, it survives restarts and is
+  shared across replicas.
+- Under `langgraph-server`, the server creates and persists threads from `DATABASE_URI`; the app's
+  `/chat` route uses the server's thread.
+
+## Streaming
+
+```python
+async for event in runnable.astream_events(
+    {"messages": [HumanMessage(text)]}, config, version="v2"
+):
+    kind = event["event"]
+    if kind == "on_chat_model_stream":
+        yield sse("message.delta", {"text": event["data"]["chunk"].content})
+    elif kind == "on_tool_start":
+        yield sse(
+            "tool.call",
+            {"id": event["run_id"], "name": event["name"], "args": event["data"].get("input")},
+        )
+    elif kind == "on_tool_end":
+        yield sse(
+            "tool.result",
+            {
+                "id": event["run_id"],
+                "name": event["name"],
+                "result": str(event["data"].get("output")),
+                "is_error": False,
+            },
+        )
+```
+
+The app applies `TRACE_CAPTURE` before emitting `args` and `result` to a non-owner. Nodes should
+not `print` or log prompt text; the telemetry layer handles capture.
+
+## Human-in-the-loop (interrupts)
+
+For API calls, use the policy's approval gate: `graph-agents-cli api approval NAME ...` makes the
+API client itself call `interrupt()` before a gated call, `/chat` ends with `message.end` status
+`awaiting_approval`, and `POST /threads/{thread_id}/approvals/{approval_id}` resumes the run (see
+SKILL.md section 5). Nothing in the graph changes.
+
+The general LangGraph pattern is below for reference. **The scaffolded chat API does not expose
+your own interrupts:** there is no status for a graph paused elsewhere and no `metadata.resume`
+request convention, so a node that calls `interrupt()` outside the API client stalls the `/chat`
+stream. Use such interrupts only under `playground --graph` (LangGraph Studio).
+
+```python
+from langgraph.types import interrupt, Command
+
+
+def confirm_action(state: State):
+    decision = interrupt(
+        {"question": "Approve closing incident?", "incident_id": state["incident_id"]}
+    )
+    if decision != "approve":
+        return {"messages": [AIMessage("Cancelled.")]}
+    return {"approved": True}
+```
+
+- Requires a checkpointer (any); the thread pauses at the interrupt and is resumed with
+  `graph.invoke(Command(resume="approve"), config)` on the same `thread_id`. On resume the
+  interrupted node runs again from its start (`interrupt()` then returns the resume value), so
+  the code before it must be safe to repeat.
+- Not wired to `/chat` outside the API client's approval gate: the app does not report such a
+  paused graph and does not translate a request `metadata.resume` into `Command(resume=...)`. Adding that is a scaffolding change to
+  `app/app_utils/chat.py` and `fast_api_app.py`, which `upgrade` 3-way merges; ask before doing it.
+- `interrupt_before=["tools"]` at compile time pauses before every tool call; same caveat.
+
+## Subgraphs
+
+```python
+sub = StateGraph(SubState)
+...
+sub_graph = sub.compile()  # unbound
+
+
+def run_sub(state: State):
+    result = sub_graph.invoke({"query": state["messages"][-1].content})
+    return {"messages": [AIMessage(result["answer"])]}
+
+
+builder.add_node(
+    "research", run_sub
+)  # or builder.add_node("research", sub_graph) when schemas share keys
+```
+
+Subgraphs inherit the parent's checkpointer and `thread_id`; interrupts inside a subgraph surface
+through the parent.
+
+## Testing with the `fake` provider
+
+```python
+# tests/unit/conftest.py
+import pytest
+
+
+@pytest.fixture(autouse=True)
+def fake_model(monkeypatch):
+    monkeypatch.setenv("MODEL_PROVIDER", "fake")
+    monkeypatch.setenv("MODEL_NAME", "fake")
+    monkeypatch.setenv("CHECKPOINTER", "memory")
+    monkeypatch.setenv("AUTH_POLICY", "shared-bearer")
+    monkeypatch.setenv("API_KEY", "test-key")
+```
+
+`get_model()` returns the template's deterministic `FakeChatModel` (`app/app_utils/model.py`):
+`hi` -> `Hello! How can I help you today?`; a request that mentions a bound tool (its name, or a
+distinctive word of it: `What is the weather in Paris?` with `get_weather` bound) -> a call of that
+tool with every required argument filled (text arguments get the request's subject, `Paris`), then
+`Here is what I found: <tool result>`; a prompt mentioning "score" and "json" -> the JSON judge
+verdict `{"score": 5, ...}`; anything else -> `I am a fake model. I can use these tools: ... You
+said: <text>`. There is no scripted-response list or `FAKE_MODEL_RESPONSES` variable. The
+template's `tests/conftest.py` keeps `.env` and the shell's app settings out of the tests and
+offers `use_test_tools(tool, ...)`, which serves the graph with test-only tools, so plumbing tests
+never depend on the project's own tools. Use it to test:
+
+- the SSE mapping (`/chat` returns `message.start`, deltas, `tool.call`/`tool.result`,
+  `message.end`);
+- tool routing (with a test-only tool) and the middleware turning `ApiPolicyError` / `ApiCallError`
+  into a tool error;
+- policy enforcement (401 without the bearer, 403 on a foreign thread under a per-user policy
+  such as `jwt` or an implemented `custom`).
+
+Never test model behaviour here; that is `graph-agents-cli eval`.
+
+## Prompts and graph modules
+
+`app/prompts/**` and `app/graph/**` are reserved agent-code directories: `upgrade` never touches
+them. Put system prompts in `app/prompts/` as Python constants or text files loaded at import;
+put a large explicit graph in `app/graph/` and import it from `agent.py`.