together-ai 0.4.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
|
|
17
17
|
## Usage
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
Create an account at [together.ai](https://www.together.ai/) and add the API key in. Then simply run the code snippet below with your preferred AI model and inputs to get back a reply.
|
|
20
20
|
|
|
21
21
|
```js
|
|
22
22
|
import Together from 'together-ai';
|
|
@@ -33,6 +33,24 @@ const result = await together.inference(model, {
|
|
|
33
33
|
});
|
|
34
34
|
```
|
|
35
35
|
|
|
36
|
+
### Streaming with LLMs
|
|
37
|
+
|
|
38
|
+
If you want to stream, simply specify `stream-tokens: true`.
|
|
39
|
+
|
|
40
|
+
```js
|
|
41
|
+
const result = await together.inference('togethercomputer/llama-2-70b-chat', {
|
|
42
|
+
prompt: 'Tell me about the history of the United States',
|
|
43
|
+
max_tokens: 1000,
|
|
44
|
+
stream_tokens: true,
|
|
45
|
+
});
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### Example Chat App
|
|
49
|
+
|
|
50
|
+
You can see an example of this library being used in a chat app here: https://simple-ai-chat.vercel.app.
|
|
51
|
+
|
|
52
|
+
The code for the example is also available. This includes code on how to stream the results of the LLM directly to the frontend: https://github.com/Nutlope/chat
|
|
53
|
+
|
|
36
54
|
## Popular Supported Models
|
|
37
55
|
|
|
38
56
|
This is a list of popular models that are supported.
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "together-ai",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.5.1",
|
|
4
|
+
"description": "Node.js SDK for Together.ai.",
|
|
5
5
|
"author": "Hassan El Mghari (@nutlope)",
|
|
6
6
|
"repository": "Nutlope/together-js",
|
|
7
7
|
"license": "MIT",
|