together-ai 0.5.0 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -7
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -25,11 +25,11 @@ const together = new Together({
|
|
|
25
25
|
auth: process.env.TOGETHER_API_KEY,
|
|
26
26
|
});
|
|
27
27
|
|
|
28
|
-
const model = 'mistralai/
|
|
28
|
+
const model = 'mistralai/Mixtral-8x7B-Instruct-v0.1';
|
|
29
29
|
|
|
30
30
|
const result = await together.inference(model, {
|
|
31
|
-
prompt: 'Suggest some fun family activities
|
|
32
|
-
max_tokens:
|
|
31
|
+
prompt: 'Suggest some fun winter family activities',
|
|
32
|
+
max_tokens: 700,
|
|
33
33
|
});
|
|
34
34
|
```
|
|
35
35
|
|
|
@@ -38,19 +38,36 @@ const result = await together.inference(model, {
|
|
|
38
38
|
If you want to stream, simply specify `stream-tokens: true`.
|
|
39
39
|
|
|
40
40
|
```js
|
|
41
|
-
const
|
|
42
|
-
|
|
43
|
-
const result = await together.inference(model, {
|
|
41
|
+
const result = await together.inference('togethercomputer/llama-2-70b-chat', {
|
|
44
42
|
prompt: 'Tell me about the history of the United States',
|
|
45
43
|
max_tokens: 1000,
|
|
46
44
|
stream_tokens: true,
|
|
47
45
|
});
|
|
48
46
|
```
|
|
49
47
|
|
|
48
|
+
### Next.js Chat App with streaming
|
|
49
|
+
|
|
50
|
+
You can see an example of this library being used in a Next.js chat app here: https://simple-ai-chat.vercel.app.
|
|
51
|
+
|
|
52
|
+
The code for the example is also available, including code on how to stream the results of the LLM directly to the frontend: https://github.com/Nutlope/chat.
|
|
53
|
+
|
|
54
|
+
### Filtering responses with Llama Guard
|
|
55
|
+
|
|
56
|
+
You can now use Llama Guard, an LLM-based input-output safeguard model, with models on the Together.ai platform. To do this, simply add `"safety_model": "Meta-Llama/Llama-Guard-7b"`.
|
|
57
|
+
|
|
58
|
+
```js
|
|
59
|
+
const result = await together.inference('togethercomputer/llama-2-13b-chat', {
|
|
60
|
+
prompt: 'Tell me about San Francisco',
|
|
61
|
+
max_tokens: 1000,
|
|
62
|
+
safety_model: 'Meta-Llama/Llama-Guard-7b',
|
|
63
|
+
});
|
|
64
|
+
```
|
|
65
|
+
|
|
50
66
|
## Popular Supported Models
|
|
51
67
|
|
|
52
|
-
This is a list of popular models that are supported.
|
|
68
|
+
This is a non-exhaustive list of popular models that are supported.
|
|
53
69
|
|
|
70
|
+
- Mixtral Instruct v0.1 (`mistralai/Mixtral-8x7B-Instruct-v0.1`)
|
|
54
71
|
- Mistral-7B (`mistralai/Mistral-7B-Instruct-v0.1`)
|
|
55
72
|
- Llama-2 70B (`togethercomputer/llama-2-70b-chat`)
|
|
56
73
|
- Llama-2 13B (`togethercomputer/llama-2-13b-chat`)
|