@olostep/ai-sdk 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +102 -2
- package/dist/index.d.ts +3758 -0
- package/dist/index.js +196 -0
- package/package.json +63 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Olostep
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,103 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @olostep/ai-sdk
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Live web search, scraping and crawling API for AI. Get structured, LLM-ready data from any site. Scale to 1M requests with 10 parallel batches at low cost.**
|
|
4
|
+
|
|
5
|
+
[Olostep](https://www.olostep.com) is web data infrastructure for AI agents and enterprise AI workflows. Its APIs provide live web search, scraping and crawling to extract, clean and structure data from the web. This package gives agents built with the [Vercel AI SDK](https://ai-sdk.dev) ready-made Olostep tools.
|
|
6
|
+
|
|
7
|
+
## Why Olostep
|
|
8
|
+
|
|
9
|
+
- Web data infrastructure for developers and enterprises
|
|
10
|
+
- Scales to hundreds of millions to billions of requests per customer per month
|
|
11
|
+
- The most performant, reliable and cost-effective API on the market
|
|
12
|
+
- Used by the fastest-growing AI companies
|
|
13
|
+
|
|
14
|
+
From extracting data from a handful of URLs to crawling hundreds of millions of pages, Olostep gives developers and AI companies reliable, scalable and highly cost-effective access to clean, structured web data.
|
|
15
|
+
|
|
16
|
+
## Install
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
npm install @olostep/ai-sdk ai zod
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Get an API key at [olostep.com/auth](https://olostep.com/auth) and set it as `OLOSTEP_API_KEY`.
|
|
23
|
+
|
|
24
|
+
## Usage
|
|
25
|
+
|
|
26
|
+
```ts
|
|
27
|
+
import { generateText, isStepCount } from 'ai';
|
|
28
|
+
import { olostepTools } from '@olostep/ai-sdk';
|
|
29
|
+
|
|
30
|
+
const { text } = await generateText({
|
|
31
|
+
model: 'openai/gpt-5-mini',
|
|
32
|
+
tools: olostepTools(),
|
|
33
|
+
stopWhen: isStepCount(5),
|
|
34
|
+
prompt: 'Read the Next.js 15 release notes and list the breaking changes, with sources.',
|
|
35
|
+
});
|
|
36
|
+
|
|
37
|
+
console.log(text);
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Or pick individual tools:
|
|
41
|
+
|
|
42
|
+
```ts
|
|
43
|
+
import { olostepScrape, olostepSearch } from '@olostep/ai-sdk';
|
|
44
|
+
|
|
45
|
+
const tools = {
|
|
46
|
+
scrape: olostepScrape(),
|
|
47
|
+
search: olostepSearch(),
|
|
48
|
+
};
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## Tools
|
|
52
|
+
|
|
53
|
+
| Tool | What it does |
|
|
54
|
+
|---|---|
|
|
55
|
+
| `olostepScrape` | Scrape a URL and return clean markdown (JavaScript rendering included) |
|
|
56
|
+
| `olostepSearch` | Search the live web and return titles, URLs and descriptions |
|
|
57
|
+
| `olostepAnswer` | Answer a question from the live web, with source URLs |
|
|
58
|
+
| `olostepMap` | List the URLs on a website, optionally ranked by a query |
|
|
59
|
+
| `olostepCrawl` | Crawl a site and return the markdown of each page, 20 at a time by default |
|
|
60
|
+
| `olostepBatch` | Scrape up to 100k URLs in one batch and return the markdown of each page, 20 at a time by default |
|
|
61
|
+
| `olostepResults` | Get the remaining pages of a crawl or batch |
|
|
62
|
+
| `olostepTools` | All of the above, keyed by name |
|
|
63
|
+
|
|
64
|
+
## Options
|
|
65
|
+
|
|
66
|
+
Every tool accepts the same options:
|
|
67
|
+
|
|
68
|
+
```ts
|
|
69
|
+
olostepTools({
|
|
70
|
+
apiKey: 'olostep_...', // defaults to process.env.OLOSTEP_API_KEY
|
|
71
|
+
maxCharacters: 20_000, // max page content returned to the model, per page
|
|
72
|
+
timeoutSeconds: 300, // how long crawl and batch tools wait for the job to finish
|
|
73
|
+
pagesPerCall: 20, // pages returned per call, up to 100
|
|
74
|
+
});
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Large crawls and batches
|
|
78
|
+
|
|
79
|
+
Page content goes to the model, so large jobs return their pages a set at a time. If a job is still running when the wait ends, the tool returns its `id` with `status` set to `in_progress`. If more pages are available, the result includes `nextCursor`. Pass the `id` and `nextCursor` to `olostepResults` to get the next pages, until you have every page.
|
|
80
|
+
|
|
81
|
+
New accounts are limited to 100 URLs per batch: contact info@olostep.com to lift this.
|
|
82
|
+
|
|
83
|
+
## Examples
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
# Call each tool directly (no LLM needed)
|
|
87
|
+
OLOSTEP_API_KEY=... npx tsx examples/tools.ts
|
|
88
|
+
|
|
89
|
+
# Run an agent through the Vercel AI Gateway
|
|
90
|
+
OLOSTEP_API_KEY=... AI_GATEWAY_API_KEY=... npx tsx examples/agent.ts
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## About Olostep
|
|
94
|
+
|
|
95
|
+
[Olostep](https://www.olostep.com) is web data infrastructure for AI agents and enterprise AI workflows. It helps businesses power AI agents, knowledge bases and data-intensive workflows without the high costs of traditional web data infrastructure, and it is trusted by some of the fastest-growing businesses in the world.
|
|
96
|
+
|
|
97
|
+
- Website: https://www.olostep.com
|
|
98
|
+
- Documentation: https://docs.olostep.com
|
|
99
|
+
- Get an API key: https://olostep.com/auth
|
|
100
|
+
|
|
101
|
+
## License
|
|
102
|
+
|
|
103
|
+
MIT
|