@netlify/agent-runner-cli 1.68.0 → 1.69.0

package/dist/skills/netlify-inference/SKILL.md

@@ -0,0 +1,510 @@
+---
+name: netlify-inference
+description: Use Netlify AI Gateway for AI inference in Netlify Functions and server-side code. Use when adding AI/LLM features with OpenAI, Anthropic, or Google Gemini without managing API keys.
+---
+
+# Netlify AI Gateway
+
+Zero-config AI inference for Netlify projects. Netlify automatically injects environment variables so official
+provider SDKs work without API keys or configuration.
+
+## How It Works
+
+Netlify sets provider-specific environment variables in all compute contexts (Functions, Edge Functions, server-side
+framework code). Official SDKs auto-detect these variables, so a default constructor like `new OpenAI()` works
+out of the box. Requests are proxied through AI Gateway and billed to your Netlify account credits.
+
+Netlify **never overrides** environment variables you have already set. The check is per-provider: if you set your own
+`OPENAI_API_KEY`, Netlify will not set `OPENAI_API_KEY` or `OPENAI_BASE_URL`, but will still inject Anthropic and
+Gemini variables independently.
+
+AI Gateway requires a **credit-based plan** (Free, Personal, or Pro). It is not available on legacy pricing plans.
+
+### Environment Variables
+
+| Provider | Variables |
+|----------|-----------|
+| **Anthropic** | `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL` |
+| **OpenAI** | `OPENAI_API_KEY`, `OPENAI_BASE_URL` |
+| **Google Gemini** | `GEMINI_API_KEY`, `GOOGLE_GEMINI_BASE_URL` |
+| **Gateway (always set)** | `NETLIFY_AI_GATEWAY_KEY`, `NETLIFY_AI_GATEWAY_BASE_URL` |
+
+## Quick Start
+
+Install the SDK for your provider and use a zero-config constructor. No API key or base URL needed.
+
+### Anthropic
+
+```bash
+npm install @anthropic-ai/sdk
+```
+
+```typescript
+import Anthropic from '@anthropic-ai/sdk'
+
+const anthropic = new Anthropic()
+
+const message = await anthropic.messages.create({
+  model: 'claude-opus-4-6',
+  max_tokens: 1024,
+  messages: [{ role: 'user', content: 'Hello!' }],
+})
+
+console.log(message.content[0].text)
+```
+
+### OpenAI
+
+```bash
+npm install openai
+```
+
+```typescript
+import OpenAI from 'openai'
+
+const openai = new OpenAI()
+
+const completion = await openai.chat.completions.create({
+  model: 'gpt-5.2',
+  messages: [{ role: 'user', content: 'Hello!' }],
+})
+
+console.log(completion.choices[0].message.content)
+```
+
+OpenAI also supports the newer Responses API:
+
+```typescript
+import OpenAI from 'openai'
+
+const openai = new OpenAI()
+
+const response = await openai.responses.create({
+  model: 'gpt-5.2',
+  input: [{ role: 'user', content: 'Hello!' }],
+})
+
+console.log(response.output_text)
+```
+
+### Google Gemini
+
+```bash
+npm install @google/genai
+```
+
+```typescript
+import { GoogleGenAI } from '@google/genai'
+
+const ai = new GoogleGenAI({})
+
+const response = await ai.models.generateContent({
+  model: 'gemini-3-flash-preview',
+  contents: 'Hello!',
+})
+
+console.log(response.text)
+```
+
+## Streaming
+
+### Anthropic Streaming
+
+```typescript
+import Anthropic from '@anthropic-ai/sdk'
+
+const anthropic = new Anthropic()
+
+const stream = await anthropic.messages.create({
+  model: 'claude-opus-4-6',
+  max_tokens: 1024,
+  messages: [{ role: 'user', content: 'Tell me a story.' }],
+  stream: true,
+})
+
+for await (const event of stream) {
+  if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
+    process.stdout.write(event.delta.text)
+  }
+}
+```
+
+### OpenAI Streaming
+
+```typescript
+import OpenAI from 'openai'
+
+const openai = new OpenAI()
+
+const stream = await openai.chat.completions.create({
+  model: 'gpt-5.2',
+  messages: [{ role: 'user', content: 'Tell me a story.' }],
+  stream: true,
+})
+
+for await (const chunk of stream) {
+  if (chunk.choices[0]?.delta?.content) {
+    process.stdout.write(chunk.choices[0].delta.content)
+  }
+}
+```
+
+### Gemini Streaming
+
+```typescript
+import { GoogleGenAI } from '@google/genai'
+
+const ai = new GoogleGenAI({})
+
+const stream = await ai.models.generateContentStream({
+  model: 'gemini-3-flash-preview',
+  contents: 'Tell me a story.',
+})
+
+for await (const chunk of stream) {
+  if (chunk.text) {
+    process.stdout.write(chunk.text)
+  }
+}
+```
+
+## Image Generation
+
+### OpenAI (gpt-image-1)
+
+```typescript
+import OpenAI from 'openai'
+
+const openai = new OpenAI()
+
+const result = await openai.images.generate({
+  model: 'gpt-image-1',
+  prompt: 'A cute otter in a river',
+})
+
+const imageBase64 = result.data[0].b64_json
+```
+
+Image generation is also available via the Responses API with the `image_generation` tool:
+
+```typescript
+const response = await openai.responses.create({
+  model: 'gpt-4o',
+  input: 'Create a simple logo',
+  tools: [{ type: 'image_generation' }],
+})
+
+for (const item of response.output) {
+  if (item.type === 'image_generation_call') {
+    const imageBase64 = item.result.data
+  }
+}
+```
+
+### Google Gemini (Imagen)
+
+```typescript
+import { GoogleGenAI } from '@google/genai'
+
+const ai = new GoogleGenAI({})
+
+const response = await ai.models.generateImages({
+  model: 'imagen-4.0-generate-001',
+  prompt: 'A cute otter in a river',
+  config: { numberOfImages: 1 },
+})
+
+for (const image of response.generatedImages) {
+  const imageBytes = image.image.imageBytes
+}
+```
+
+## Netlify Functions
+
+Full example of a Netlify Function using AI Gateway:
+
+```typescript
+// netlify/functions/chat.mts
+import type { Context } from '@netlify/functions'
+import Anthropic from '@anthropic-ai/sdk'
+
+const anthropic = new Anthropic()
+
+export default async (req: Request, context: Context) => {
+  const { prompt } = await req.json()
+
+  const message = await anthropic.messages.create({
+    model: 'claude-opus-4-6',
+    max_tokens: 1024,
+    messages: [{ role: 'user', content: prompt }],
+  })
+
+  return Response.json({ response: message.content[0].text })
+}
+
+export const config = {
+  path: '/api/chat',
+}
+```
+
+For streaming responses from a function:
+
+```typescript
+// netlify/functions/stream.mts
+import type { Context } from '@netlify/functions'
+import OpenAI from 'openai'
+
+const openai = new OpenAI()
+
+export default async (req: Request, context: Context) => {
+  const { prompt } = await req.json()
+
+  const stream = await openai.chat.completions.create({
+    model: 'gpt-5.2',
+    messages: [{ role: 'user', content: prompt }],
+    stream: true,
+  })
+
+  return new Response(
+    new ReadableStream({
+      async start(controller) {
+        for await (const chunk of stream) {
+          const text = chunk.choices[0]?.delta?.content
+          if (text) {
+            controller.enqueue(new TextEncoder().encode(text))
+          }
+        }
+        controller.close()
+      },
+    }),
+    { headers: { 'Content-Type': 'text/plain; charset=utf-8' } },
+  )
+}
+
+export const config = {
+  path: '/api/stream',
+}
+```
+
+## Framework Server-Side Code
+
+AI Gateway env vars are available in any server-side context, not just Netlify Functions.
+
+### Next.js Route Handler
+
+```typescript
+// app/api/chat/route.ts
+import OpenAI from 'openai'
+
+const openai = new OpenAI()
+
+export async function POST(request: Request) {
+  const { prompt } = await request.json()
+
+  const completion = await openai.chat.completions.create({
+    model: 'gpt-5.2',
+    messages: [{ role: 'user', content: prompt }],
+  })
+
+  return Response.json({ response: completion.choices[0].message.content })
+}
+```
+
+This also works in server components, API routes in Astro, Remix loaders/actions, and SvelteKit server routes.
+
+## REST API / Direct Fetch
+
+For HTTP-level control, use the gateway variables directly:
+
+```typescript
+const response = await fetch(`${process.env.NETLIFY_AI_GATEWAY_BASE_URL}/anthropic/v1/messages`, {
+  method: 'POST',
+  headers: {
+    'Content-Type': 'application/json',
+    'Authorization': `Bearer ${process.env.NETLIFY_AI_GATEWAY_KEY}`,
+    'anthropic-version': '2023-06-01',
+  },
+  body: JSON.stringify({
+    model: 'claude-opus-4-6',
+    max_tokens: 1024,
+    messages: [{ role: 'user', content: 'Hello!' }],
+  }),
+})
+
+const data = await response.json()
+```
+
+Or using the provider-specific variables with `fetch`:
+
+```typescript
+const response = await fetch(`${process.env.ANTHROPIC_BASE_URL}/v1/messages`, {
+  method: 'POST',
+  headers: {
+    'Content-Type': 'application/json',
+    'x-api-key': process.env.ANTHROPIC_API_KEY,
+    'anthropic-version': '2023-06-01',
+  },
+  body: JSON.stringify({
+    model: 'claude-opus-4-6',
+    max_tokens: 1024,
+    messages: [{ role: 'user', content: 'Hello!' }],
+  }),
+})
+```
+
+## Local Development
+
+### Netlify CLI
+
+Run `netlify dev` to start a local development server with AI Gateway env vars injected automatically.
+
+```bash
+netlify dev
+```
+
+### Vite Plugin
+
+Alternatively, use the `@netlify/vite-plugin` for Vite-based projects:
+
+```bash
+npm install @netlify/vite-plugin
+```
+
+```javascript
+// vite.config.js
+import { defineConfig } from 'vite'
+import netlify from '@netlify/vite-plugin'
+
+export default defineConfig({
+  plugins: [netlify()],
+})
+```
+
+Then run your normal dev command (`npm run dev`).
+
+**Requirement:** The site must have at least one production deploy before AI Gateway env vars become available locally.
+
+## Available Models
+
+Key models per provider. For the latest list, see https://docs.netlify.com/build/ai-gateway/overview/.
+
+### Anthropic
+
+- `claude-opus-4-6`
+- `claude-opus-4-5-20251101`
+- `claude-opus-4-1-20250805`
+- `claude-sonnet-4-5-20250929`
+- `claude-sonnet-4-20250514`
+- `claude-haiku-4-5-20251001`
+- `claude-3-7-sonnet-20250219`
+- `claude-3-5-haiku-20241022`
+
+### OpenAI
+
+- `gpt-5.2`
+- `gpt-5.1`
+- `gpt-5.1-codex`
+- `gpt-5`
+- `gpt-5-mini`
+- `gpt-5-nano`
+- `gpt-4.1`
+- `gpt-4.1-mini`
+- `gpt-4.1-nano`
+- `gpt-4o`
+- `gpt-4o-mini`
+- `o3`
+- `o3-mini`
+- `o4-mini`
+- `gpt-image-1` (image generation)
+- `codex-mini-latest`
+
+### Google Gemini
+
+- `gemini-3-pro-preview`
+- `gemini-3-flash-preview`
+- `gemini-2.5-pro`
+- `gemini-2.5-flash`
+- `gemini-2.5-flash-lite`
+- `gemini-2.0-flash`
+- `gemini-2.0-flash-lite`
+- `imagen-4.0-generate-001` (image generation)
+- `veo-3.0-generate-preview` (video generation)
+
+## Rate Limits
+
+Tokens per minute (TPM) are scoped per **account** across all projects. Both input and output tokens count.
+For Anthropic, cached tokens are excluded; for other providers, cached tokens are included.
+For the latest limits, see https://docs.netlify.com/build/ai-gateway/overview/.
+
+| Model | Free | Personal | Pro |
+|-------|------|----------|-----|
+| claude-sonnet-4-5-20250929 | 18,000 | 90,000 | 180,000 |
+| gpt-5 | 18,000 | 90,000 | 180,000 |
+| gpt-4o-mini | 250,000 | 500,000 | 750,000 |
+| gemini-2.5-pro | 24,000 | 120,000 | 240,000 |
+
+Set up rate limiting rules on your functions to prevent abuse from client-side callers.
+
+## Limitations
+
+- **Context window:** Max 200k tokens per request
+- **Batch inference:** Not supported
+- **Custom headers:** Cannot send custom request headers to providers
+- **Prompt caching:** Anthropic supports 5-minute ephemeral caching only. Gemini caching is not supported. OpenAI sets `prompt_cache_key` per-account automatically.
+- **Priority processing (OpenAI):** Not supported
+- **Production deploy required:** At least one production deploy must exist before AI Gateway activates
+
+## Disabling AI Gateway
+
+To prevent Netlify from injecting any AI-related environment variables, disable AI features in the Netlify UI under
+**Project configuration > Build & deploy > Build with AI > Manage AI features**.
+
+## Monitoring Usage
+
+View AI Gateway token usage and costs in the Netlify UI under **Team settings > Billing > Usage**. Each request's
+token consumption is tracked per model and converted to Netlify credits.
+
+## Common Errors & Solutions
+
+### "API key not found" or env var is undefined
+
+**Cause:** AI Gateway env vars are not injected.
+
+**Fix:**
+
+1. Ensure the site has at least one production deploy
+2. Use `netlify dev` (not a bare `npm run dev`) for local development
+3. Check that you are reading the env var in server-side code, not client-side browser code
+
+### Rate limit exceeded (429)
+
+**Cause:** Account hit the tokens-per-minute limit for that model.
+
+**Fix:**
+
+1. Wait briefly and retry — TPM limits reset each minute
+2. Switch to a smaller/cheaper model (e.g. `gpt-4.1-mini` instead of `gpt-4.1`)
+3. Reduce prompt length or `max_tokens`
+4. Upgrade plan tier for higher limits
+
+### Model not available
+
+**Cause:** The requested model is not supported through AI Gateway.
+
+**Fix:**
+
+1. Check the available models list above
+2. Use the exact model ID string (e.g. `claude-sonnet-4-5-20250929`, not `claude-sonnet`)
+
+## Packages
+
+```bash
+# Anthropic
+npm install @anthropic-ai/sdk
+
+# OpenAI
+npm install openai
+
+# Google Gemini
+npm install @google/genai
+```

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@netlify/agent-runner-cli",
   "type": "module",
-  "version": "1.68.0",
+  "version": "1.69.0",
   "description": "CLI tool for running Netlify agents",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",