together 1.5.35__py3-none-any.whl → 2.0.0a7__py3-none-any.whl

together-2.0.0a7.dist-info/METADATA

@@ -0,0 +1,730 @@
+Metadata-Version: 2.3
+Name: together
+Version: 2.0.0a7
+Summary: The official Python library for the together API
+Project-URL: Homepage, https://github.com/togethercomputer/together-py
+Project-URL: Repository, https://github.com/togethercomputer/together-py
+Author-email: Together <dev-feedback@TogetherAI.com>
+License: Apache-2.0
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: OS Independent
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Requires-Dist: anyio<5,>=3.5.0
+Requires-Dist: click>=8.1.7
+Requires-Dist: distro<2,>=1.7.0
+Requires-Dist: filelock>=3.13.1
+Requires-Dist: httpx<1,>=0.23.0
+Requires-Dist: pillow>=10.4.0
+Requires-Dist: pydantic<3,>=1.9.0
+Requires-Dist: rich>=13.7.1
+Requires-Dist: sniffio
+Requires-Dist: tabulate>=0.9.0
+Requires-Dist: tqdm>=4.67.1
+Requires-Dist: types-tabulate>=0.9.0.20240106
+Requires-Dist: types-tqdm>=4.67.0.20250516
+Requires-Dist: typing-extensions<5,>=4.10
+Provides-Extra: aiohttp
+Requires-Dist: aiohttp; extra == 'aiohttp'
+Requires-Dist: httpx-aiohttp>=0.1.9; extra == 'aiohttp'
+Provides-Extra: pyarrow
+Requires-Dist: pyarrow-stubs>=10.0.1.7; extra == 'pyarrow'
+Requires-Dist: pyarrow>=16.1.0; extra == 'pyarrow'
+Description-Content-Type: text/markdown
+
+# Together Python API library
+
+<!-- prettier-ignore -->
+[![PyPI version](https://img.shields.io/pypi/v/together.svg?label=pypi%20(stable))](https://pypi.org/project/together/)
+
+The Together Python library provides convenient access to the Together REST API from any Python 3.9+
+application. The library includes type definitions for all request params and response fields,
+and offers both synchronous and asynchronous clients powered by [httpx](https://github.com/encode/httpx).
+
+It is generated with [Stainless](https://www.stainless.com/).
+
+## Documentation
+
+The REST API documentation can be found on [docs.together.ai](https://docs.together.ai/). The full API of this library can be found in [api.md](https://github.com/togethercomputer/together-py/tree/main/api.md).
+
+## Installation
+
+```sh
+# install from the production repo
+pip install git+ssh://git@github.com/togethercomputer/together-py.git
+```
+
+> [!NOTE]
+> Once this package is [published to PyPI](https://www.stainless.com/docs/guides/publish), this will become: `pip install --pre together`
+
+## Usage
+
+The full API of this library can be found in [api.md](https://github.com/togethercomputer/together-py/tree/main/api.md).
+
+```python
+import os
+from together import Together
+
+client = Together(
+    api_key=os.environ.get("TOGETHER_API_KEY"),  # This is the default and can be omitted
+)
+
+chat_completion = client.chat.completions.create(
+    messages=[
+        {
+            "role": "user",
+            "content": "Say this is a test!",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+)
+print(chat_completion.choices)
+```
+
+While you can provide an `api_key` keyword argument,
+we recommend using [python-dotenv](https://pypi.org/project/python-dotenv/)
+to add `TOGETHER_API_KEY="My API Key"` to your `.env` file
+so that your API Key is not stored in source control.
+
+## Async usage
+
+Simply import `AsyncTogether` instead of `Together` and use `await` with each API call:
+
+```python
+import os
+import asyncio
+from together import AsyncTogether
+
+client = AsyncTogether(
+    api_key=os.environ.get("TOGETHER_API_KEY"),  # This is the default and can be omitted
+)
+
+
+async def main() -> None:
+    chat_completion = await client.chat.completions.create(
+        messages=[
+            {
+                "role": "user",
+                "content": "Say this is a test!",
+            }
+        ],
+        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+    )
+    print(chat_completion.choices)
+
+
+asyncio.run(main())
+```
+
+Functionality between the synchronous and asynchronous clients is otherwise identical.
+
+### With aiohttp
+
+By default, the async client uses `httpx` for HTTP requests. However, for improved concurrency performance you may also use `aiohttp` as the HTTP backend.
+
+You can enable this by installing `aiohttp`:
+
+```sh
+# install from the production repo
+pip install 'together[aiohttp] @ git+ssh://git@github.com/togethercomputer/together-py.git'
+```
+
+Then you can enable it by instantiating the client with `http_client=DefaultAioHttpClient()`:
+
+```python
+import asyncio
+from together import DefaultAioHttpClient
+from together import AsyncTogether
+
+
+async def main() -> None:
+    async with AsyncTogether(
+        api_key="My API Key",
+        http_client=DefaultAioHttpClient(),
+    ) as client:
+        chat_completion = await client.chat.completions.create(
+            messages=[
+                {
+                    "role": "user",
+                    "content": "Say this is a test!",
+                }
+            ],
+            model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+        )
+        print(chat_completion.choices)
+
+
+asyncio.run(main())
+```
+
+## Streaming responses
+
+We provide support for streaming responses using Server Side Events (SSE).
+
+```python
+from together import Together
+
+client = Together()
+
+stream = client.chat.completions.create(
+    messages=[
+        {
+            "role": "user",
+            "content": "Say this is a test",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+    stream=True,
+)
+for chat_completion in stream:
+    print(chat_completion.choices)
+```
+
+The async client uses the exact same interface.
+
+```python
+from together import AsyncTogether
+
+client = AsyncTogether()
+
+stream = await client.chat.completions.create(
+    messages=[
+        {
+            "role": "user",
+            "content": "Say this is a test",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+    stream=True,
+)
+async for chat_completion in stream:
+    print(chat_completion.choices)
+```
+
+## Using types
+
+Nested request parameters are [TypedDicts](https://docs.python.org/3/library/typing.html#typing.TypedDict). Responses are [Pydantic models](https://docs.pydantic.dev) which also provide helper methods for things like:
+
+- Serializing back into JSON, `model.to_json()`
+- Converting to a dictionary, `model.to_dict()`
+
+Typed requests and responses provide autocomplete and documentation within your editor. If you would like to see type errors in VS Code to help catch bugs earlier, set `python.analysis.typeCheckingMode` to `basic`.
+
+## Nested params
+
+Nested parameters are dictionaries, typed using `TypedDict`, for example:
+
+```python
+from together import Together
+
+client = Together()
+
+chat_completion = client.chat.completions.create(
+    messages=[
+        {
+            "content": "content",
+            "role": "system",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+    response_format={},
+)
+print(chat_completion.response_format)
+```
+
+## File uploads
+
+Request parameters that correspond to file uploads can be passed as `bytes`, or a [`PathLike`](https://docs.python.org/3/library/os.html#os.PathLike) instance or a tuple of `(filename, contents, media type)`.
+
+```python
+from pathlib import Path
+from together import Together
+
+client = Together()
+
+client.files.upload(
+    file=Path("/path/to/file"),
+    purpose="fine-tune",
+)
+```
+
+The async client uses the exact same interface. If you pass a [`PathLike`](https://docs.python.org/3/library/os.html#os.PathLike) instance, the file contents will be read asynchronously automatically.
+
+## Handling errors
+
+When the library is unable to connect to the API (for example, due to network connection problems or a timeout), a subclass of `together.APIConnectionError` is raised.
+
+When the API returns a non-success status code (that is, 4xx or 5xx
+response), a subclass of `together.APIStatusError` is raised, containing `status_code` and `response` properties.
+
+All errors inherit from `together.APIError`.
+
+```python
+import together
+from together import Together
+
+client = Together()
+
+try:
+    client.chat.completions.create(
+        messages=[
+            {
+                "role": "user",
+                "content": "Say this is a test",
+            }
+        ],
+        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+    )
+except together.APIConnectionError as e:
+    print("The server could not be reached")
+    print(e.__cause__)  # an underlying Exception, likely raised within httpx.
+except together.RateLimitError as e:
+    print("A 429 status code was received; we should back off a bit.")
+except together.APIStatusError as e:
+    print("Another non-200-range status code was received")
+    print(e.status_code)
+    print(e.response)
+```
+
+Error codes are as follows:
+
+| Status Code | Error Type                 |
+| ----------- | -------------------------- |
+| 400         | `BadRequestError`          |
+| 401         | `AuthenticationError`      |
+| 403         | `PermissionDeniedError`    |
+| 404         | `NotFoundError`            |
+| 422         | `UnprocessableEntityError` |
+| 429         | `RateLimitError`           |
+| >=500       | `InternalServerError`      |
+| N/A         | `APIConnectionError`       |
+
+### Retries
+
+Certain errors are automatically retried 2 times by default, with a short exponential backoff.
+Connection errors (for example, due to a network connectivity problem), 408 Request Timeout, 409 Conflict,
+429 Rate Limit, and >=500 Internal errors are all retried by default.
+
+You can use the `max_retries` option to configure or disable retry settings:
+
+```python
+from together import Together
+
+# Configure the default for all requests:
+client = Together(
+    # default is 2
+    max_retries=0,
+)
+
+# Or, configure per-request:
+client.with_options(max_retries=5).chat.completions.create(
+    messages=[
+        {
+            "role": "user",
+            "content": "Say this is a test",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+)
+```
+
+### Timeouts
+
+By default requests time out after 1 minute. You can configure this with a `timeout` option,
+which accepts a float or an [`httpx.Timeout`](https://www.python-httpx.org/advanced/timeouts/#fine-tuning-the-configuration) object:
+
+```python
+from together import Together
+
+# Configure the default for all requests:
+client = Together(
+    # 20 seconds (default is 1 minute)
+    timeout=20.0,
+)
+
+# More granular control:
+client = Together(
+    timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=2.0),
+)
+
+# Override per-request:
+client.with_options(timeout=5.0).chat.completions.create(
+    messages=[
+        {
+            "role": "user",
+            "content": "Say this is a test",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+)
+```
+
+On timeout, an `APITimeoutError` is thrown.
+
+Note that requests that time out are [retried twice by default](https://github.com/togethercomputer/together-py/tree/main/#retries).
+
+## Advanced
+
+### Logging
+
+We use the standard library [`logging`](https://docs.python.org/3/library/logging.html) module.
+
+You can enable logging by setting the environment variable `TOGETHER_LOG` to `info`.
+
+```shell
+$ export TOGETHER_LOG=info
+```
+
+Or to `debug` for more verbose logging.
+
+### How to tell whether `None` means `null` or missing
+
+In an API response, a field may be explicitly `null`, or missing entirely; in either case, its value is `None` in this library. You can differentiate the two cases with `.model_fields_set`:
+
+```py
+if response.my_field is None:
+  if 'my_field' not in response.model_fields_set:
+    print('Got json like {}, without a "my_field" key present at all.')
+  else:
+    print('Got json like {"my_field": null}.')
+```
+
+### Accessing raw response data (e.g. headers)
+
+The "raw" Response object can be accessed by prefixing `.with_raw_response.` to any HTTP method call, e.g.,
+
+```py
+from together import Together
+
+client = Together()
+response = client.chat.completions.with_raw_response.create(
+    messages=[{
+        "role": "user",
+        "content": "Say this is a test",
+    }],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+)
+print(response.headers.get('X-My-Header'))
+
+completion = response.parse()  # get the object that `chat.completions.create()` would have returned
+print(completion.choices)
+```
+
+These methods return an [`APIResponse`](https://github.com/togethercomputer/together-py/tree/main/src/together/_response.py) object.
+
+The async client returns an [`AsyncAPIResponse`](https://github.com/togethercomputer/together-py/tree/main/src/together/_response.py) with the same structure, the only difference being `await`able methods for reading the response content.
+
+#### `.with_streaming_response`
+
+The above interface eagerly reads the full response body when you make the request, which may not always be what you want.
+
+To stream the response body, use `.with_streaming_response` instead, which requires a context manager and only reads the response body once you call `.read()`, `.text()`, `.json()`, `.iter_bytes()`, `.iter_text()`, `.iter_lines()` or `.parse()`. In the async client, these are async methods.
+
+```python
+with client.chat.completions.with_streaming_response.create(
+    messages=[
+        {
+            "role": "user",
+            "content": "Say this is a test",
+        }
+    ],
+    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+) as response:
+    print(response.headers.get("X-My-Header"))
+
+    for line in response.iter_lines():
+        print(line)
+```
+
+The context manager is required so that the response will reliably be closed.
+
+### Making custom/undocumented requests
+
+This library is typed for convenient access to the documented API.
+
+If you need to access undocumented endpoints, params, or response properties, the library can still be used.
+
+#### Undocumented endpoints
+
+To make requests to undocumented endpoints, you can make requests using `client.get`, `client.post`, and other
+http verbs. Options on the client will be respected (such as retries) when making this request.
+
+```py
+import httpx
+
+response = client.post(
+    "/foo",
+    cast_to=httpx.Response,
+    body={"my_param": True},
+)
+
+print(response.headers.get("x-foo"))
+```
+
+#### Undocumented request params
+
+If you want to explicitly send an extra param, you can do so with the `extra_query`, `extra_body`, and `extra_headers` request
+options.
+
+#### Undocumented response properties
+
+To access undocumented response properties, you can access the extra fields like `response.unknown_prop`. You
+can also get all the extra fields on the Pydantic model as a dict with
+[`response.model_extra`](https://docs.pydantic.dev/latest/api/base_model/#pydantic.BaseModel.model_extra).
+
+### Configuring the HTTP client
+
+You can directly override the [httpx client](https://www.python-httpx.org/api/#client) to customize it for your use case, including:
+
+- Support for [proxies](https://www.python-httpx.org/advanced/proxies/)
+- Custom [transports](https://www.python-httpx.org/advanced/transports/)
+- Additional [advanced](https://www.python-httpx.org/advanced/clients/) functionality
+
+```python
+import httpx
+from together import Together, DefaultHttpxClient
+
+client = Together(
+    # Or use the `TOGETHER_BASE_URL` env var
+    base_url="http://my.test.server.example.com:8083",
+    http_client=DefaultHttpxClient(
+        proxy="http://my.test.proxy.example.com",
+        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
+    ),
+)
+```
+
+You can also customize the client on a per-request basis by using `with_options()`:
+
+```python
+client.with_options(http_client=DefaultHttpxClient(...))
+```
+
+### Managing HTTP resources
+
+By default the library closes underlying HTTP connections whenever the client is [garbage collected](https://docs.python.org/3/reference/datamodel.html#object.__del__). You can manually close the client using the `.close()` method if desired, or with a context manager that closes when exiting.
+
+```py
+from together import Together
+
+with Together() as client:
+  # make requests here
+  ...
+
+# HTTP client is now closed
+```
+
+## Usage – CLI
+
+### Chat Completions
+
+```bash
+together chat.completions \
+  --message "system" "You are a helpful assistant named Together" \
+  --message "user" "What is your name?" \
+  --model mistralai/Mixtral-8x7B-Instruct-v0.1
+```
+
+The Chat Completions CLI enables streaming tokens to stdout by default. To disable streaming, use `--no-stream`.
+
+### Completions
+
+```bash
+together completions \
+  "Large language models are " \
+  --model mistralai/Mixtral-8x7B-v0.1 \
+  --max-tokens 512 \
+  --stop "."
+```
+
+The Completions CLI enables streaming tokens to stdout by default. To disable streaming, use `--no-stream`.
+
+### Image Generations
+
+```bash
+together images generate \
+  "space robots" \
+  --model stabilityai/stable-diffusion-xl-base-1.0 \
+  --n 4
+```
+
+The image is opened in the default image viewer by default. To disable this, use `--no-show`.
+
+### Files
+
+```bash
+# Help
+together files --help
+
+# Check file
+together files check example.jsonl
+
+# Upload file
+together files upload example.jsonl
+
+# List files
+together files list
+
+# Retrieve file metadata
+together files retrieve file-6f50f9d1-5b95-416c-9040-0799b2b4b894
+
+# Retrieve file content
+together files retrieve-content file-6f50f9d1-5b95-416c-9040-0799b2b4b894
+
+# Delete remote file
+together files delete file-6f50f9d1-5b95-416c-9040-0799b2b4b894
+```
+
+### Fine-tuning
+
+```bash
+# Help
+together fine-tuning --help
+
+# Create fine-tune job
+together fine-tuning create \
+  --model togethercomputer/llama-2-7b-chat \
+  --training-file file-711d8724-b3e3-4ae2-b516-94841958117d
+
+# List fine-tune jobs
+together fine-tuning list
+
+# Retrieve fine-tune job details
+together fine-tuning retrieve ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# List fine-tune job events
+together fine-tuning list-events ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# Cancel running job
+together fine-tuning cancel ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# Download fine-tuned model weights
+together fine-tuning download ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+```
+
+### Models
+
+```bash
+# Help
+together models --help
+
+# List models
+together models list
+```
+
+
+## Versioning
+
+This package generally follows [SemVer](https://semver.org/spec/v2.0.0.html) conventions, though certain backwards-incompatible changes may be released as minor versions:
+
+1. Changes that only affect static types, without breaking runtime behavior.
+2. Changes to library internals which are technically public but not intended or documented for external use. _(Please open a GitHub issue to let us know if you are relying on such internals.)_
+3. Changes that we do not expect to impact the vast majority of users in practice.
+
+We take backwards-compatibility seriously and work hard to ensure you can rely on a smooth upgrade experience.
+
+We are keen for your feedback; please open an [issue](https://www.github.com/togethercomputer/together-py/issues) with questions, bugs, or suggestions.
+
+### Determining the installed version
+
+If you've upgraded to the latest version but aren't seeing any new features you were expecting then your python environment is likely still using an older version.
+
+You can determine the version that is being used at runtime with:
+
+```py
+import together
+print(together.__version__)
+```
+
+## Requirements
+
+Python 3.9 or higher.
+
+## Usage – CLI
+
+### Files
+
+```bash
+# Help
+together files --help
+
+# Check file
+together files check example.jsonl
+
+# Upload file
+together files upload example.jsonl
+
+# List files
+together files list
+
+# Retrieve file metadata
+together files retrieve file-6f50f9d1-5b95-416c-9040-0799b2b4b894
+
+# Retrieve file content
+together files retrieve-content file-6f50f9d1-5b95-416c-9040-0799b2b4b894
+
+# Delete remote file
+together files delete file-6f50f9d1-5b95-416c-9040-0799b2b4b894
+```
+
+### Fine-tuning
+
+```bash
+# Help
+together fine-tuning --help
+
+# Create fine-tune job
+together fine-tuning create \
+  --model togethercomputer/llama-2-7b-chat \
+  --training-file file-711d8724-b3e3-4ae2-b516-94841958117d
+
+# List fine-tune jobs
+together fine-tuning list
+
+# Retrieve fine-tune job details
+together fine-tuning retrieve ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# List fine-tune job events
+together fine-tuning list-events ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# List fine-tune checkpoints
+together fine-tuning list-checkpoints ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# Cancel running job
+together fine-tuning cancel ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# Download fine-tuned model weights
+together fine-tuning download ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+
+# Delete fine-tuned model weights
+together fine-tuning delete ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
+```
+
+### Models
+
+```bash
+# Help
+together models --help
+
+# List models
+together models list
+
+# Upload a model
+together models upload --model-name my-org/my-model --model-source s3-or-hugging-face
+```
+
+## Contributing
+
+See [the contributing documentation](https://github.com/togethercomputer/together-py/tree/main/./CONTRIBUTING.md).