npm - ima2-gen - Versions diffs - 3.20.0 → 3.21.0 - Mend

ima2-gen 3.20.0 → 3.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (1856) hide show

package/node_modules/ai/docs/05-ai-sdk-rsc/index.mdx DELETED Viewed

@@ -1,63 +0,0 @@
----
-title: AI SDK RSC
-description: Learn about AI SDK RSC.
-collapsed: true
----
-# AI SDK RSC
-<Note type="warning">
-  AI SDK RSC is currently experimental. We recommend using [AI SDK
-  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
-  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
-</Note>
-<IndexCards
-  cards={[
-    {
-      title: 'Overview',
-      description: 'Learn about AI SDK RSC.',
-      href: '/docs/ai-sdk-rsc/overview',
-    },
-    {
-      title: 'Streaming React Components',
-      description: 'Learn how to stream React components.',
-      href: '/docs/ai-sdk-rsc/streaming-react-components',
-    },
-    {
-      title: 'Managing Generative UI State',
-      description: 'Learn how to manage generative UI state.',
-      href: '/docs/ai-sdk-rsc/generative-ui-state',
-    },
-    {
-      title: 'Saving and Restoring States',
-      description: 'Learn how to save and restore states.',
-      href: '/docs/ai-sdk-rsc/saving-and-restoring-states',
-    },
-    {
-      title: 'Multi-step Interfaces',
-      description: 'Learn how to build multi-step interfaces.',
-      href: '/docs/ai-sdk-rsc/multistep-interfaces',
-    },
-    {
-      title: 'Streaming Values',
-      description: 'Learn how to stream values with AI SDK RSC.',
-      href: '/docs/ai-sdk-rsc/streaming-values',
-    },
-    {
-      title: 'Handling Loading State',
-      description: 'Learn how to handle loading state.',
-      href: '/docs/ai-sdk-rsc/loading-state',
-    },
-    {
-      title: 'Error Handling',
-      description: 'Learn how to handle errors.',
-      href: '/docs/ai-sdk-rsc/error-handling',
-    },
-    {
-      title: 'Authentication',
-      description: 'Learn how to authenticate users.',
-      href: '/docs/ai-sdk-rsc/authentication',
-    },
-  ]}
-/>

package/node_modules/ai/docs/06-advanced/01-prompt-engineering.mdx DELETED Viewed

@@ -1,96 +0,0 @@
----
-title: Prompt Engineering
-description: Learn how to engineer prompts for LLMs with the AI SDK
----
-# Prompt Engineering
-## What is a Large Language Model (LLM)?
-A Large Language Model is essentially a prediction engine that takes a sequence of words as input and aims to predict the most likely sequence to follow. It does this by assigning probabilities to potential next sequences and then selecting one. The model continues to generate sequences until it meets a specified stopping criterion.
-These models learn by training on massive text corpuses, which means they will be better suited to some use cases than others. For example, a model trained on GitHub data would understand the probabilities of sequences in source code particularly well. However, it's crucial to understand that the generated sequences, while often seeming plausible, can sometimes be random and not grounded in reality. As these models become more accurate, many surprising abilities and applications emerge.
-## What is a prompt?
-Prompts are the starting points for LLMs. They are the inputs that trigger the model to generate text. The scope of prompt engineering involves not just crafting these prompts but also understanding related concepts such as hidden prompts, tokens, token limits, and the potential for prompt hacking, which includes phenomena like jailbreaks and leaks.
-## Why is prompt engineering needed?
-Prompt engineering currently plays a pivotal role in shaping the responses of LLMs. It allows us to tweak the model to respond more effectively to a broader range of queries. This includes the use of techniques like semantic search, command grammars, and the ReActive model architecture. The performance, context window, and cost of LLMs varies between models and model providers which adds further constraints to the mix. For example, the GPT-4 model is more expensive than GPT-3.5-turbo and significantly slower, but it can also be more effective at certain tasks. And so, like many things in software engineering, there is a trade-offs between cost and performance.
-To assist with comparing and tweaking LLMs, we've built an AI playground that allows you to compare the performance of different models side-by-side online. When you're ready, you can even generate code with the AI SDK to quickly use your prompt and your selected model into your own applications.
-## Example: Build a Slogan Generator
-### Start with an instruction
-Imagine you want to build a slogan generator for marketing campaigns. Creating catchy slogans isn't always straightforward!
-First, you'll need a prompt that makes it clear what you want. Let's start with an instruction. Submit this prompt to generate your first completion.
-<InlinePrompt initialInput="Create a slogan for a coffee shop." />
-Not bad! Now, try making your instruction more specific.
-<InlinePrompt initialInput="Create a slogan for an organic coffee shop." />
-Introducing a single descriptive term to our prompt influences the completion. Essentially, crafting your prompt is the means by which you "instruct" or "program" the model.
-### Include examples
-Clear instructions are key for quality outcomes, but that might not always be enough. Let's try to enhance your instruction further.
-<InlinePrompt initialInput="Create three slogans for a coffee shop with live music." />
-These slogans are fine, but could be even better. It appears the model overlooked the 'live' part in our prompt. Let's change it slightly to generate more appropriate suggestions.
-Often, it's beneficial to both demonstrate and tell the model your requirements. Incorporating examples in your prompt can aid in conveying patterns or subtleties. Test this prompt that carries a few examples.
-<InlinePrompt
-  initialInput={`Create three slogans for a business with unique features.
-Business: Bookstore with cats
-Slogans: "Purr-fect Pages", "Books and Whiskers", "Novels and Nuzzles"
-Business: Gym with rock climbing
-Slogans: "Peak Performance", "Reach New Heights", "Climb Your Way Fit"
-Business: Coffee shop with live music
-Slogans:`}
-/>
-Great! Incorporating examples of expected output for a certain input prompted the model to generate the kind of names we aimed for.
-### Tweak your settings
-Apart from designing prompts, you can influence completions by tweaking model settings. A crucial setting is the **temperature**.
-You might have seen that the same prompt, when repeated, yielded the same or nearly the same completions. This happens when your temperature is at 0.
-Attempt to re-submit the identical prompt a few times with temperature set to 1.
-<InlinePrompt
-  initialInput={`Create three slogans for a business with unique features.
-Business: Bookstore with cats
-Slogans: "Purr-fect Pages", "Books and Whiskers", "Novels and Nuzzles"
-Business: Gym with rock climbing
-Slogans: "Peak Performance", "Reach New Heights", "Climb Your Way Fit"
-Business: Coffee shop with live music
-Slogans:`}
-showTemp={true}
-initialTemperature={1}
-/>
-Notice the difference? With a temperature above 0, the same prompt delivers varied completions each time.
-Keep in mind that the model forecasts the text most likely to follow the preceding text. Temperature, a value from 0 to 1, essentially governs the model's confidence level in making these predictions. A lower temperature implies lesser risks, leading to more precise and deterministic completions. A higher temperature yields a broader range of completions.
-For your slogan generator, you might want a large pool of name suggestions. A moderate temperature of 0.6 should serve well.
-## Recommended Resources
-Prompt Engineering is evolving rapidly, with new methods and research papers surfacing every week. Here are some resources that we've found useful for learning about and experimenting with prompt engineering:
-- [The Vercel AI Playground](/playground)
-- [Brex Prompt Engineering](https://github.com/brexhq/prompt-engineering)
-- [Prompt Engineering Guide by Dair AI](https://www.promptingguide.ai/)

package/node_modules/ai/docs/06-advanced/02-stopping-streams.mdx DELETED Viewed

@@ -1,184 +0,0 @@
----
-title: Stopping Streams
-description: Learn how to cancel streams with the AI SDK
----
-# Stopping Streams
-Canceling ongoing streams is often needed.
-For example, users might want to stop a stream when they realize that the response is not what they want.
-The different parts of the AI SDK support canceling streams in different ways.
-## AI SDK Core
-The AI SDK functions have an `abortSignal` argument that you can use to cancel a stream.
-You would use this if you want to cancel a stream from the server side to the LLM API, e.g. by
-forwarding the `abortSignal` from the request.
-```tsx highlight="10,11,12-16"
-import { streamText } from 'ai';
-__PROVIDER_IMPORT__;
-export async function POST(req: Request) {
-  const { prompt } = await req.json();
-  const result = streamText({
-    model: __MODEL__,
-    prompt,
-    // forward the abort signal:
-    abortSignal: req.signal,
-    onAbort: ({ steps }) => {
-      // Handle cleanup when stream is aborted
-      console.log('Stream aborted after', steps.length, 'steps');
-      // Persist partial results to database
-    },
-  });
-  return result.toTextStreamResponse();
-}
-```
-## AI SDK UI
-The hooks, e.g. `useChat` or `useCompletion`, provide a `stop` helper function that can be used to cancel a stream.
-This will cancel the stream from the client side to the server.
-<Note type="warning">
-  Stream abort functionality is not compatible with stream resumption. If you're
-  using `resume: true` in `useChat`, the abort functionality will break the
-  resumption mechanism. Choose either abort or resume functionality, but not
-  both.
-</Note>
-```tsx file="app/page.tsx" highlight="9,18-20"
-'use client';
-import { useCompletion } from '@ai-sdk/react';
-export default function Chat() {
-  const { input, completion, stop, status, handleSubmit, handleInputChange } =
-    useCompletion();
-  return (
-    <div>
-      {(status === 'submitted' || status === 'streaming') && (
-        <button type="button" onClick={() => stop()}>
-          Stop
-        </button>
-      )}
-      {completion}
-      <form onSubmit={handleSubmit}>
-        <input value={input} onChange={handleInputChange} />
-      </form>
-    </div>
-  );
-}
-```
-## Handling stream abort cleanup
-When streams are aborted, you may need to perform cleanup operations such as persisting partial results or cleaning up resources. The `onAbort` callback provides a way to handle these scenarios on the server side.
-Unlike `onFinish`, which is called when a stream completes normally, `onAbort` is specifically called when a stream is aborted via `AbortSignal`. This distinction allows you to handle normal completion and aborted streams differently.
-<Note>
-  For UI message streams (`toUIMessageStreamResponse`), the `onFinish` callback
-  also receives an `isAborted` parameter that indicates whether the stream was
-  aborted. This allows you to handle both completion and abort scenarios in a
-  single callback.
-</Note>
-```tsx highlight="8-12"
-import { streamText } from 'ai';
-__PROVIDER_IMPORT__;
-const result = streamText({
-  model: __MODEL__,
-  prompt: 'Write a long story...',
-  abortSignal: controller.signal,
-  onAbort: ({ steps }) => {
-    // Called when stream is aborted - persist partial results
-    await savePartialResults(steps);
-    await logAbortEvent(steps.length);
-  },
-  onFinish: ({ steps, totalUsage }) => {
-    // Called when stream completes normally
-    await saveFinalResults(steps, totalUsage);
-  },
-});
-```
-The `onAbort` callback receives:
-- `steps`: Array of all completed steps before the abort occurred
-This is particularly useful for:
-- Persisting partial conversation history to database
-- Saving partial progress for later continuation
-- Cleaning up server-side resources or connections
-- Logging abort events for analytics
-You can also handle abort events directly in the stream using the `abort` stream part:
-```tsx highlight="8-12"
-for await (const part of result.fullStream) {
-  switch (part.type) {
-    case 'text-delta':
-      // Handle text delta content
-      break;
-    case 'abort':
-      // Handle abort event directly in stream
-      console.log('Stream was aborted');
-      break;
-    // ... other cases
-  }
-}
-```
-## UI Message Streams
-When using `toUIMessageStreamResponse`, you need to handle stream abortion slightly differently. The `onFinish` callback receives an `isAborted` parameter, and you should pass the `consumeStream` function to ensure proper abort handling:
-```tsx highlight="5,19,20-24,26"
-import { openai } from '@ai-sdk/openai';
-import {
-  consumeStream,
-  convertToModelMessages,
-  streamText,
-  UIMessage,
-} from 'ai';
-__PROVIDER_IMPORT__;
-export async function POST(req: Request) {
-  const { messages }: { messages: UIMessage[] } = await req.json();
-  const result = streamText({
-    model: __MODEL__,
-    messages: await convertToModelMessages(messages),
-    abortSignal: req.signal,
-  });
-  return result.toUIMessageStreamResponse({
-    onFinish: async ({ isAborted }) => {
-      if (isAborted) {
-        console.log('Stream was aborted');
-        // Handle abort-specific cleanup
-      } else {
-        console.log('Stream completed normally');
-        // Handle normal completion
-      }
-    },
-    consumeSseStream: consumeStream,
-  });
-}
-```
-The `consumeStream` function is necessary for proper abort handling in UI message streams. It ensures that the stream is properly consumed even when aborted, preventing potential memory leaks or hanging connections.
-## AI SDK RSC
-<Note type="warning">
-  The AI SDK RSC does not currently support stopping streams.
-</Note>

package/node_modules/ai/docs/06-advanced/03-backpressure.mdx DELETED Viewed

@@ -1,173 +0,0 @@
----
-title: Backpressure
-description: How to handle backpressure and cancellation when working with the AI SDK
----
-# Stream Back-pressure and Cancellation
-This page focuses on understanding back-pressure and cancellation when working with streams. You do not need to know this information to use the AI SDK, but for those interested, it offers a deeper dive on why and how the SDK optimally streams responses.
-In the following sections, we'll explore back-pressure and cancellation in the context of a simple example program. We'll discuss the issues that can arise from an eager approach and demonstrate how a lazy approach can resolve them.
-## Back-pressure and Cancellation with Streams
-Let's begin by setting up a simple example program:
-```jsx
-// A generator that will yield positive integers
-async function* integers() {
-  let i = 1;
-  while (true) {
-    console.log(`yielding ${i}`);
-    yield i++;
-    await sleep(100);
-  }
-}
-function sleep(ms) {
-  return new Promise(resolve => setTimeout(resolve, ms));
-}
-// Wraps a generator into a ReadableStream
-function createStream(iterator) {
-  return new ReadableStream({
-    async start(controller) {
-      for await (const v of iterator) {
-        controller.enqueue(v);
-      }
-      controller.close();
-    },
-  });
-}
-// Collect data from stream
-async function run() {
-  // Set up a stream of integers
-  const stream = createStream(integers());
-  // Read values from our stream
-  const reader = stream.getReader();
-  for (let i = 0; i < 10_000; i++) {
-    // we know our stream is infinite, so there's no need to check `done`.
-    const { value } = await reader.read();
-    console.log(`read ${value}`);
-    await sleep(1_000);
-  }
-}
-run();
-```
-In this example, we create an async-generator that yields positive integers, a `ReadableStream` that wraps our integer generator, and a reader which will read values out of our stream. Notice, too, that our integer generator logs out `"yielding ${i}"`, and our reader logs out `"read ${value}"`. Both take an arbitrary amount of time to process data, represented with a 100ms sleep in our generator, and a 1sec sleep in our reader.
-## Back-pressure
-If you were to run this program, you'd notice something funny. We'll see roughly 10 "yield" logs for every "read" log. This might seem obvious, the generator can push values 10x faster than the reader can pull them out. But it represents a problem, our `stream` has to maintain an ever expanding queue of items that have been pushed in but not pulled out.
-The problem stems from the way we wrap our generator into a stream. Notice the use of `for await (…)` inside our `start` handler. This is an **eager** for-loop, and it is constantly running to get the next value from our generator to be enqueued in our stream. This means our stream does not respect back-pressure, the signal from the consumer to the producer that more values aren't needed _yet_. We've essentially spawned a thread that will perpetually push more data into the stream, one that runs as fast as possible to push new data immediately. Worse, there's no way to signal to this thread to stop running when we don't need additional data.
-To fix this, `ReadableStream` allows a `pull` handler. `pull` is called every time the consumer attempts to read more data from our stream (if there's no data already queued internally). But it's not enough to just move the `for await(…)` into `pull`, we also need to convert from an eager enqueuing to a **lazy** one. By making these 2 changes, we'll be able to react to the consumer. If they need more data, we can easily produce it, and if they don't, then we don't need to spend any time doing unnecessary work.
-```jsx
-function createStream(iterator) {
-  return new ReadableStream({
-    async pull(controller) {
-      const { value, done } = await iterator.next();
-      if (done) {
-        controller.close();
-      } else {
-        controller.enqueue(value);
-      }
-    },
-  });
-}
-```
-Our `createStream` is a little more verbose now, but the new code is important. First, we need to manually call our `iterator.next()` method. This returns a `Promise` for an object with the type signature `{ done: boolean, value: T }`. If `done` is `true`, then we know that our iterator won't yield any more values and we must `close` the stream (this allows the consumer to know that the stream is also finished producing values). Else, we need to `enqueue` our newly produced value.
-When we run this program, we see that our "yield" and "read" logs are now paired. We're no longer yielding 10x integers for every read! And, our stream now only needs to maintain 1 item in its internal buffer. We've essentially given control to the consumer, so that it's responsible for producing new values as it needs it. Neato!
-## Cancellation
-Let's go back to our initial eager example, with 1 small edit. Now instead of reading 10,000 integers, we're only going to read 3:
-```jsx
-// A generator that will yield positive integers
-async function* integers() {
-  let i = 1;
-  while (true) {
-    console.log(`yielding ${i}`);
-    yield i++;
-    await sleep(100);
-  }
-}
-function sleep(ms) {
-  return new Promise(resolve => setTimeout(resolve, ms));
-}
-// Wraps a generator into a ReadableStream
-function createStream(iterator) {
-  return new ReadableStream({
-    async start(controller) {
-      for await (const v of iterator) {
-        controller.enqueue(v);
-      }
-      controller.close();
-    },
-  });
-}
-// Collect data from stream
-async function run() {
-  // Set up a stream that of integers
-  const stream = createStream(integers());
-  // Read values from our stream
-  const reader = stream.getReader();
-  // We're only reading 3 items this time:
-  for (let i = 0; i < 3; i++) {
-    // we know our stream is infinite, so there's no need to check `done`.
-    const { value } = await reader.read();
-    console.log(`read ${value}`);
-    await sleep(1000);
-  }
-}
-run();
-```
-We're back to yielding 10x the number of values read. But notice now, after we've read 3 values, we're continuing to yield new values. We know that our reader will never read another value, but our stream doesn't! The eager `for await (…)` will continue forever, loudly enqueuing new values into our stream's buffer and increasing our memory usage until it consumes all available program memory.
-The fix to this is exactly the same: use `pull` and manual iteration. By producing values _**lazily**_, we tie the lifetime of our integer generator to the lifetime of the reader. Once the reads stop, the yields will stop too:
-```jsx
-// Wraps a generator into a ReadableStream
-function createStream(iterator) {
-  return new ReadableStream({
-    async pull(controller) {
-      const { value, done } = await iterator.next();
-      if (done) {
-        controller.close();
-      } else {
-        controller.enqueue(value);
-      }
-    },
-  });
-}
-```
-Since the solution is the same as implementing back-pressure, it shows that they're just 2 facets of the same problem: Pushing values into a stream should be done **lazily**, and doing it eagerly results in expected problems.
-## Tying Stream Laziness to AI Responses
-Now let's imagine you're integrating AIBot service into your product. Users will be able to prompt "count from 1 to infinity", the browser will fetch your AI API endpoint, and your servers connect to AIBot to get a response. But "infinity" is, well, infinite. The response will never end!
-After a few seconds, the user gets bored and navigates away. Or maybe you're doing local development and a hot-module reload refreshes your page. The browser will have ended its connection to the API endpoint, but will your server end its connection with AIBot?
-If you used the eager `for await (...)` approach, then the connection is still running and your server is asking for more and more data from AIBot. Our server spawned a "thread" and there's no signal when we can end the eager pulls. Eventually, the server is going to run out of memory (remember, there's no active fetch connection to read the buffering responses and free them).
-{/* When we started writing the streaming code for the AI SDK, we confirm aborting a fetch will end a streamed response from Next.js */}
-With the lazy approach, this is taken care of for you. Because the stream will only request new data from AIBot when the consumer requests it, navigating away from the page naturally frees all resources. The fetch connection aborts and the server can clean up the response. The `ReadableStream` tied to that response can now be garbage collected. When that happens, the connection it holds to AIBot can then be freed.

package/node_modules/ai/docs/06-advanced/04-caching.mdx DELETED Viewed

@@ -1,169 +0,0 @@
----
-title: Caching
-description: How to handle caching when working with the AI SDK
----
-# Caching Responses
-Depending on the type of application you're building, you may want to cache the responses you receive from your AI provider, at least temporarily.
-## Using Language Model Middleware (Recommended)
-The recommended approach to caching responses is using [language model middleware](/docs/ai-sdk-core/middleware)
-and the [`simulateReadableStream`](/docs/reference/ai-sdk-core/simulate-readable-stream) function.
-Language model middleware is a way to enhance the behavior of language models by intercepting and modifying the calls to the language model.
-Let's see how you can use language model middleware to cache responses.
-```ts filename="ai/middleware.ts"
-import { Redis } from '@upstash/redis';
-import {
-  type LanguageModelV3,
-  type LanguageModelV3Middleware,
-  type LanguageModelV3StreamPart,
-  simulateReadableStream,
-} from 'ai';
-const redis = new Redis({
-  url: process.env.KV_URL,
-  token: process.env.KV_TOKEN,
-});
-export const cacheMiddleware: LanguageModelV3Middleware = {
-  wrapGenerate: async ({ doGenerate, params }) => {
-    const cacheKey = JSON.stringify(params);
-    const cached = (await redis.get(cacheKey)) as Awaited<
-      ReturnType<LanguageModelV3['doGenerate']>
-    > | null;
-    if (cached !== null) {
-      return {
-        ...cached,
-        response: {
-          ...cached.response,
-          timestamp: cached?.response?.timestamp
-            ? new Date(cached?.response?.timestamp)
-            : undefined,
-        },
-      };
-    }
-    const result = await doGenerate();
-    redis.set(cacheKey, result);
-    return result;
-  },
-  wrapStream: async ({ doStream, params }) => {
-    const cacheKey = JSON.stringify(params);
-    // Check if the result is in the cache
-    const cached = await redis.get(cacheKey);
-    // If cached, return a simulated ReadableStream that yields the cached result
-    if (cached !== null) {
-      // Format the timestamps in the cached response
-      const formattedChunks = (cached as LanguageModelV3StreamPart[]).map(p => {
-        if (p.type === 'response-metadata' && p.timestamp) {
-          return { ...p, timestamp: new Date(p.timestamp) };
-        } else return p;
-      });
-      return {
-        stream: simulateReadableStream({
-          initialDelayInMs: 0,
-          chunkDelayInMs: 10,
-          chunks: formattedChunks,
-        }),
-      };
-    }
-    // If not cached, proceed with streaming
-    const { stream, ...rest } = await doStream();
-    const fullResponse: LanguageModelV3StreamPart[] = [];
-    const transformStream = new TransformStream<
-      LanguageModelV3StreamPart,
-      LanguageModelV3StreamPart
-    >({
-      transform(chunk, controller) {
-        fullResponse.push(chunk);
-        controller.enqueue(chunk);
-      },
-      flush() {
-        // Store the full response in the cache after streaming is complete
-        redis.set(cacheKey, fullResponse);
-      },
-    });
-    return {
-      stream: stream.pipeThrough(transformStream),
-      ...rest,
-    };
-  },
-};
-```
-<Note>
-  This example uses `@upstash/redis` to store and retrieve the assistant's
-  responses but you can use any KV storage provider you would like.
-</Note>
-`LanguageModelV3Middleware` has two methods: `wrapGenerate` and `wrapStream`. `wrapGenerate` is called when using [`generateText`](/docs/reference/ai-sdk-core/generate-text), while `wrapStream` is called when using [`streamText`](/docs/reference/ai-sdk-core/stream-text).
-For `wrapGenerate`, you can cache the response directly. Instead, for `wrapStream`, you cache an array of the stream parts, which can then be used with [`simulateReadableStream`](/docs/ai-sdk-core/testing#simulate-data-stream-protocol-responses) function to create a simulated `ReadableStream` that returns the cached response. In this way, the cached response is returned chunk-by-chunk as if it were being generated by the model. You can control the initial delay and delay between chunks by adjusting the `initialDelayInMs` and `chunkDelayInMs` parameters of `simulateReadableStream`.
-You can see a full example of caching with Redis in a Next.js application in our [Caching Middleware Recipe](/cookbook/next/caching-middleware).
-## Using Lifecycle Callbacks
-Alternatively, each AI SDK Core function has special lifecycle callbacks you can use. The one of interest is likely `onFinish`, which is called when the generation is complete. This is where you can cache the full response.
-Here's an example of how you can implement caching using Vercel KV and Next.js to cache the OpenAI response for 1 hour:
-This example uses [Upstash Redis](https://upstash.com/docs/redis/overall/getstarted) and Next.js to cache the response for 1 hour.
-```tsx filename="app/api/chat/route.ts"
-import { convertToModelMessages, streamText, UIMessage } from 'ai';
-__PROVIDER_IMPORT__;
-import { Redis } from '@upstash/redis';
-// Allow streaming responses up to 30 seconds
-export const maxDuration = 30;
-const redis = new Redis({
-  url: process.env.KV_URL,
-  token: process.env.KV_TOKEN,
-});
-export async function POST(req: Request) {
-  const { messages }: { messages: UIMessage[] } = await req.json();
-  // come up with a key based on the request:
-  const key = JSON.stringify(messages);
-  // Check if we have a cached response
-  const cached = (await redis.get(key)) as string | null;
-  if (cached != null) {
-    return new Response(cached, {
-      status: 200,
-      headers: { 'Content-Type': 'text/plain' },
-    });
-  }
-  // Call the language model:
-  const result = streamText({
-    model: __MODEL__,
-    messages: await convertToModelMessages(messages),
-    async onFinish({ text }) {
-      // Cache the response text:
-      await redis.set(key, text);
-      await redis.expire(key, 60 * 60);
-    },
-  });
-  // Respond with the stream
-  return result.toUIMessageStreamResponse();
-}
-```