cached-openai 0.0.1__tar.gz

cached_openai-0.0.1/.gitignore

@@ -0,0 +1,165 @@
+# Keys
+openai_key.txt
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

cached_openai-0.0.1/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2025 Daniel Guetta
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

cached_openai-0.0.1/PKG-INFO

@@ -0,0 +1,115 @@
+Metadata-Version: 2.4
+Name: cached_openai
+Version: 0.0.1
+Summary: A version of the OpenAI package that allows the caching of responses
+Project-URL: Homepage, https://github.com/danguetta/cached_openai
+Project-URL: Issues, https://github.com/danguetta/cached_openai/issues
+Author-email: Daniel Guetta <daniel@guetta.com>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.8
+Requires-Dist: openai>=1.66.2
+Requires-Dist: requests>=2.32.2
+Requires-Dist: tqdm>=4.67.1
+Description-Content-Type: text/markdown
+
+# cached_openai
+
+`cached_openai` is a simple Python library that mimics the `OpenAI` python library, but can draw from a cache instead of sending the request to Open AI.
+
+When it is run in **dev mode**, it caches responses to all requests made from OpenAI, and returns the cached value if the request is made again. When it is run in **production mode**, no new cache entries are created, but if any requests stored in the cache are made again, they are returned therefrom.
+
+It is able to cache several different responses for a single request, to mimic the way the OpenAI API returns different responses when it receives the same query twice. It is also able to handle requests that return images or sound clips.
+
+The user experience with `cached_open` is identical to that with the original `openai` package. Consider, for example, the following piece of code using the "regular" `openai` api:
+```
+import openai
+client = openai.OpenAI(api_key=...)
+response = client.chat.completions.create(...)
+```
+
+In `cached_openai`, the **only** change required would be to change the first line to `import cached_openai as openai`. That's it - everything else works identically. If the query is cached, the cached version will be returned. If not, the `OpenAI` API will be queries as usual.
+
+The package can be configured to return the cached entry immediately, *or* to replicate the delay that would occur if the request were to be made from OpenAI directly.
+
+The package also works with the async version of the OpenAI API - just use `openai.AsyncOpenAI(...)`.
+
+## Why `cached_openai`?
+
+I designed this package for teaching purposes - when I teach classes that use the OpenAI API, I often provide code to students which they run on their machines before modifying it. This isn't ideal for two reasons
+  1. Every student has to pay to run the API requests in the code - there can be hundreds or thousands of these requests, and the costs can add up.
+  2. Every student is likely to get different answers out of the API - it makes it difficult to teach the class when everyone is looking at a different answer.
+
+Instead, I can now distributed `cached_openai` with a cache I prepared - students can then run the entire code for free, and get the same answers as I did. They can then modify the code to run their own version, and query OpenAI as usual.
+
+# User manual
+
+This user manual is divided into two parts - preparing a cache, and distributing it.
+
+## Preparing a cache
+
+To prepare a cache, you need to run `cached_openai` in **dev mode**. To do this, set an environment variable called `CACHED_OPENAI_DEV_MODE` to any value before you load the package. I also recommended you set an environment variable called `CACHED_OPENAI_VERBOSE` to any value to print debugging messages as the package runs. The easiest way too do this is in your code, before you import the package
+
+```
+import os
+os.environ['CACHED_OPENAI_VERBOSE'] = 'True'
+os.environ['CACHED_OPENAI_DEV_MODE'] = 'True'
+import cached_openai
+```
+
+You should then use `cached_openai` exactly as you would `OpenAI` - every request you make will be cached. In dev mode, the package creates three files in your working directory
+  - `openai_cache_temp.bin` is a temporary cache file - it is updated every time a call is made to the OpenAI API to store the results of that call. It can be updated very quickly (so as not to slow down the call), but stores data in an inneficient format.
+  - `openai_cache.cache` is the main cache file which contains all cached requests in an efficient format (a dictionary). Every time the package is loaded, the data in `openai_cache_temp.bin` is integrated into this permanent cache file, and the temporary file is deleted.
+  - `openai_cache_used.txt` stores all the keys of requests that have been made or saved since the file was last created; we'll explain the purpose of this file later.
+
+### Determining delays
+
+The package can be configured to return the cached entry immediately, *or* to replicate the delay that would occur if the request were to be made from OpenAI directly.
+
+By default, the package will be configured to return the result immediately. If you would, instead, like it to replicate the delay that was initially observed when the request was made, simply set an environment variable called `CACHED_OPENAI_DELAY_RESPONSES` to any value before you load the package.
+
+### Repeated requests
+
+When you use `cached_openai` to run a request that has already been run, a new request will not be made to OpenAI - the *cached* result is returned instead, for free.
+
+This does mean, however, that that result will be the same every time. What if you want to store several different responses to a single request? All you need to do is provide a `seed` parameter when you run that same request - if the function has been run with that identical seed before, that result is returned, but if not, the query is run again against the OpenAI API. If the underlying OpenAI API function has a seed parameter (eg: `chat.completions.create`), the seed is passed to that function, and if not (eg: `audio.speech.create`), it is stripped before it is sent.
+
+When the request is next called *without* a seed, it will return each of the seeded responses in succession, which will make it look like it is returning different results every time, just like the original API. Here's an example that might clarify this, using chat completions:
+
+  - **Input**: please give me a random number. **Output**: 42
+    - *This is the first time the request has been made, so it is sent to the OpenAI API.*\
+  - **Input**: please give me a random number (`seed=1`). **Output**: 25
+    - *This is the first time the request has been made with this seed, so it is sent to the OpenAI API.*
+  - **Input**: please give me a random number (`seed=2`). **Output**: 126
+    - *This is the first time the request has been made with this seed, so it is sent to the OpenAI API.*
+  - **Input**: please give me a random number (`seed=2`). **Output**: 126
+    - *This request has been made with this seed before, the cached result is returned*
+  - **Input**: please give me a random number. **Output**: 42
+    - *This request has been made before and is being called again for the first time; return the first saved response.*
+  - **Input**: please give me a random number. **Output**: 25
+    - *This request has been made before and is being called again for the second time; return the second saved response.*
+  - **Input**: please give me a random number. **Output**: 126
+    - *This request has been made before and is being called again for the third time; return the third saved response.*
+  - **Input**: please give me a random number. **Output**: 42
+    - *This request has been made before and is being called again for the four time; there is no fourth response stored, so loop back to the first stored answer.*
+
+### Images
+
+Image requests that return image URLs require specific handling, because OpenAI does not keep these image URLs active forever - they are deleted a few hours after the request is made. Thus, simply storing the original response with the original URL would return a broken link when the cached response was returned.
+
+To solve this problem, `cached_openai` downloads the image from the URL before returning the result, and stores it in the cache. When the same request is later called and retrieved from the cache, the image is extracted into an `image` folder in the working directory, and the URLs in the response are replaced with local URLs pointing to those extracted images.
+
+### Audio
+
+Audio requests suffer from a similar problem - OpenAI returns a file stream which then needs to be downloaded. `cached_openai` handles this seamlessly by downloading the file at request time, saving it in the cache, and simulating a file stream when the request is called again and needs to be retrieved from the cache.
+
+## Distributing the cache
+
+Once you have created the cache for your class, there are three different ways to distribute it to students
+  - Get students to install `cached_openai` directly from pypi, and distribute a cache file that they can put in their working directory containing the cache data; you can also publish this file to a URL and `cached_openai` will ask them for a URL from which to download (and potentially decompress) the file.
+  - Create your own package on pypi that forks `cached_openai` and includes your own cache file in it (note that this only works if your cache file is small enough to be uploaded to pypi).
+  - Create a self-contained `.py` file that contains not only the `cached_openai` code, but also the cache itself, embedded in the file. You can then simply distribute this `.py` file as a "batteries included" package.
+
+In all three cases, the first step is to call the `materialize` function, which will package the cache for you; see the function docstring for details.

cached_openai-0.0.1/README.md

@@ -0,0 +1,98 @@
+# cached_openai
+
+`cached_openai` is a simple Python library that mimics the `OpenAI` python library, but can draw from a cache instead of sending the request to Open AI.
+
+When it is run in **dev mode**, it caches responses to all requests made from OpenAI, and returns the cached value if the request is made again. When it is run in **production mode**, no new cache entries are created, but if any requests stored in the cache are made again, they are returned therefrom.
+
+It is able to cache several different responses for a single request, to mimic the way the OpenAI API returns different responses when it receives the same query twice. It is also able to handle requests that return images or sound clips.
+
+The user experience with `cached_open` is identical to that with the original `openai` package. Consider, for example, the following piece of code using the "regular" `openai` api:
+```
+import openai
+client = openai.OpenAI(api_key=...)
+response = client.chat.completions.create(...)
+```
+
+In `cached_openai`, the **only** change required would be to change the first line to `import cached_openai as openai`. That's it - everything else works identically. If the query is cached, the cached version will be returned. If not, the `OpenAI` API will be queries as usual.
+
+The package can be configured to return the cached entry immediately, *or* to replicate the delay that would occur if the request were to be made from OpenAI directly.
+
+The package also works with the async version of the OpenAI API - just use `openai.AsyncOpenAI(...)`.
+
+## Why `cached_openai`?
+
+I designed this package for teaching purposes - when I teach classes that use the OpenAI API, I often provide code to students which they run on their machines before modifying it. This isn't ideal for two reasons
+  1. Every student has to pay to run the API requests in the code - there can be hundreds or thousands of these requests, and the costs can add up.
+  2. Every student is likely to get different answers out of the API - it makes it difficult to teach the class when everyone is looking at a different answer.
+
+Instead, I can now distributed `cached_openai` with a cache I prepared - students can then run the entire code for free, and get the same answers as I did. They can then modify the code to run their own version, and query OpenAI as usual.
+
+# User manual
+
+This user manual is divided into two parts - preparing a cache, and distributing it.
+
+## Preparing a cache
+
+To prepare a cache, you need to run `cached_openai` in **dev mode**. To do this, set an environment variable called `CACHED_OPENAI_DEV_MODE` to any value before you load the package. I also recommended you set an environment variable called `CACHED_OPENAI_VERBOSE` to any value to print debugging messages as the package runs. The easiest way too do this is in your code, before you import the package
+
+```
+import os
+os.environ['CACHED_OPENAI_VERBOSE'] = 'True'
+os.environ['CACHED_OPENAI_DEV_MODE'] = 'True'
+import cached_openai
+```
+
+You should then use `cached_openai` exactly as you would `OpenAI` - every request you make will be cached. In dev mode, the package creates three files in your working directory
+  - `openai_cache_temp.bin` is a temporary cache file - it is updated every time a call is made to the OpenAI API to store the results of that call. It can be updated very quickly (so as not to slow down the call), but stores data in an inneficient format.
+  - `openai_cache.cache` is the main cache file which contains all cached requests in an efficient format (a dictionary). Every time the package is loaded, the data in `openai_cache_temp.bin` is integrated into this permanent cache file, and the temporary file is deleted.
+  - `openai_cache_used.txt` stores all the keys of requests that have been made or saved since the file was last created; we'll explain the purpose of this file later.
+
+### Determining delays
+
+The package can be configured to return the cached entry immediately, *or* to replicate the delay that would occur if the request were to be made from OpenAI directly.
+
+By default, the package will be configured to return the result immediately. If you would, instead, like it to replicate the delay that was initially observed when the request was made, simply set an environment variable called `CACHED_OPENAI_DELAY_RESPONSES` to any value before you load the package.
+
+### Repeated requests
+
+When you use `cached_openai` to run a request that has already been run, a new request will not be made to OpenAI - the *cached* result is returned instead, for free.
+
+This does mean, however, that that result will be the same every time. What if you want to store several different responses to a single request? All you need to do is provide a `seed` parameter when you run that same request - if the function has been run with that identical seed before, that result is returned, but if not, the query is run again against the OpenAI API. If the underlying OpenAI API function has a seed parameter (eg: `chat.completions.create`), the seed is passed to that function, and if not (eg: `audio.speech.create`), it is stripped before it is sent.
+
+When the request is next called *without* a seed, it will return each of the seeded responses in succession, which will make it look like it is returning different results every time, just like the original API. Here's an example that might clarify this, using chat completions:
+
+  - **Input**: please give me a random number. **Output**: 42
+    - *This is the first time the request has been made, so it is sent to the OpenAI API.*\
+  - **Input**: please give me a random number (`seed=1`). **Output**: 25
+    - *This is the first time the request has been made with this seed, so it is sent to the OpenAI API.*
+  - **Input**: please give me a random number (`seed=2`). **Output**: 126
+    - *This is the first time the request has been made with this seed, so it is sent to the OpenAI API.*
+  - **Input**: please give me a random number (`seed=2`). **Output**: 126
+    - *This request has been made with this seed before, the cached result is returned*
+  - **Input**: please give me a random number. **Output**: 42
+    - *This request has been made before and is being called again for the first time; return the first saved response.*
+  - **Input**: please give me a random number. **Output**: 25
+    - *This request has been made before and is being called again for the second time; return the second saved response.*
+  - **Input**: please give me a random number. **Output**: 126
+    - *This request has been made before and is being called again for the third time; return the third saved response.*
+  - **Input**: please give me a random number. **Output**: 42
+    - *This request has been made before and is being called again for the four time; there is no fourth response stored, so loop back to the first stored answer.*
+
+### Images
+
+Image requests that return image URLs require specific handling, because OpenAI does not keep these image URLs active forever - they are deleted a few hours after the request is made. Thus, simply storing the original response with the original URL would return a broken link when the cached response was returned.
+
+To solve this problem, `cached_openai` downloads the image from the URL before returning the result, and stores it in the cache. When the same request is later called and retrieved from the cache, the image is extracted into an `image` folder in the working directory, and the URLs in the response are replaced with local URLs pointing to those extracted images.
+
+### Audio
+
+Audio requests suffer from a similar problem - OpenAI returns a file stream which then needs to be downloaded. `cached_openai` handles this seamlessly by downloading the file at request time, saving it in the cache, and simulating a file stream when the request is called again and needs to be retrieved from the cache.
+
+## Distributing the cache
+
+Once you have created the cache for your class, there are three different ways to distribute it to students
+  - Get students to install `cached_openai` directly from pypi, and distribute a cache file that they can put in their working directory containing the cache data; you can also publish this file to a URL and `cached_openai` will ask them for a URL from which to download (and potentially decompress) the file.
+  - Create your own package on pypi that forks `cached_openai` and includes your own cache file in it (note that this only works if your cache file is small enough to be uploaded to pypi).
+  - Create a self-contained `.py` file that contains not only the `cached_openai` code, but also the cache itself, embedded in the file. You can then simply distribute this `.py` file as a "batteries included" package.
+
+In all three cases, the first step is to call the `materialize` function, which will package the cache for you; see the function docstring for details.

cached_openai-0.0.1/publishing_notes.md

@@ -0,0 +1,6 @@
+# To publish a new version of the pacakge
+
+Ensure build is installed
+
+```pip install --upgrade build```
+

cached_openai-0.0.1/pyproject.toml

@@ -0,0 +1,28 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "cached_openai"
+version = "0.0.1"
+dependencies = [
+    "requests>=2.32.2",
+    "tqdm>=4.67.1",
+    "openai>=1.66.2",
+]
+authors = [
+  { name="Daniel Guetta", email="daniel@guetta.com" },
+]
+description = "A version of the OpenAI package that allows the caching of responses"
+readme = "README.md"
+requires-python = ">=3.8"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: OS Independent",
+]
+license = "MIT"
+license-files = ["LICEN[CS]E*"]
+
+[project.urls]
+Homepage = "https://github.com/danguetta/cached_openai"
+Issues = "https://github.com/danguetta/cached_openai/issues"

cached_openai-0.0.1/readme.md

@@ -0,0 +1,98 @@
+# cached_openai
+
+`cached_openai` is a simple Python library that mimics the `OpenAI` python library, but can draw from a cache instead of sending the request to Open AI.
+
+When it is run in **dev mode**, it caches responses to all requests made from OpenAI, and returns the cached value if the request is made again. When it is run in **production mode**, no new cache entries are created, but if any requests stored in the cache are made again, they are returned therefrom.
+
+It is able to cache several different responses for a single request, to mimic the way the OpenAI API returns different responses when it receives the same query twice. It is also able to handle requests that return images or sound clips.
+
+The user experience with `cached_open` is identical to that with the original `openai` package. Consider, for example, the following piece of code using the "regular" `openai` api:
+```
+import openai
+client = openai.OpenAI(api_key=...)
+response = client.chat.completions.create(...)
+```
+
+In `cached_openai`, the **only** change required would be to change the first line to `import cached_openai as openai`. That's it - everything else works identically. If the query is cached, the cached version will be returned. If not, the `OpenAI` API will be queries as usual.
+
+The package can be configured to return the cached entry immediately, *or* to replicate the delay that would occur if the request were to be made from OpenAI directly.
+
+The package also works with the async version of the OpenAI API - just use `openai.AsyncOpenAI(...)`.
+
+## Why `cached_openai`?
+
+I designed this package for teaching purposes - when I teach classes that use the OpenAI API, I often provide code to students which they run on their machines before modifying it. This isn't ideal for two reasons
+  1. Every student has to pay to run the API requests in the code - there can be hundreds or thousands of these requests, and the costs can add up.
+  2. Every student is likely to get different answers out of the API - it makes it difficult to teach the class when everyone is looking at a different answer.
+
+Instead, I can now distributed `cached_openai` with a cache I prepared - students can then run the entire code for free, and get the same answers as I did. They can then modify the code to run their own version, and query OpenAI as usual.
+
+# User manual
+
+This user manual is divided into two parts - preparing a cache, and distributing it.
+
+## Preparing a cache
+
+To prepare a cache, you need to run `cached_openai` in **dev mode**. To do this, set an environment variable called `CACHED_OPENAI_DEV_MODE` to any value before you load the package. I also recommended you set an environment variable called `CACHED_OPENAI_VERBOSE` to any value to print debugging messages as the package runs. The easiest way too do this is in your code, before you import the package
+
+```
+import os
+os.environ['CACHED_OPENAI_VERBOSE'] = 'True'
+os.environ['CACHED_OPENAI_DEV_MODE'] = 'True'
+import cached_openai
+```
+
+You should then use `cached_openai` exactly as you would `OpenAI` - every request you make will be cached. In dev mode, the package creates three files in your working directory
+  - `openai_cache_temp.bin` is a temporary cache file - it is updated every time a call is made to the OpenAI API to store the results of that call. It can be updated very quickly (so as not to slow down the call), but stores data in an inneficient format.
+  - `openai_cache.cache` is the main cache file which contains all cached requests in an efficient format (a dictionary). Every time the package is loaded, the data in `openai_cache_temp.bin` is integrated into this permanent cache file, and the temporary file is deleted.
+  - `openai_cache_used.txt` stores all the keys of requests that have been made or saved since the file was last created; we'll explain the purpose of this file later.
+
+### Determining delays
+
+The package can be configured to return the cached entry immediately, *or* to replicate the delay that would occur if the request were to be made from OpenAI directly.
+
+By default, the package will be configured to return the result immediately. If you would, instead, like it to replicate the delay that was initially observed when the request was made, simply set an environment variable called `CACHED_OPENAI_DELAY_RESPONSES` to any value before you load the package.
+
+### Repeated requests
+
+When you use `cached_openai` to run a request that has already been run, a new request will not be made to OpenAI - the *cached* result is returned instead, for free.
+
+This does mean, however, that that result will be the same every time. What if you want to store several different responses to a single request? All you need to do is provide a `seed` parameter when you run that same request - if the function has been run with that identical seed before, that result is returned, but if not, the query is run again against the OpenAI API. If the underlying OpenAI API function has a seed parameter (eg: `chat.completions.create`), the seed is passed to that function, and if not (eg: `audio.speech.create`), it is stripped before it is sent.
+
+When the request is next called *without* a seed, it will return each of the seeded responses in succession, which will make it look like it is returning different results every time, just like the original API. Here's an example that might clarify this, using chat completions:
+
+  - **Input**: please give me a random number. **Output**: 42
+    - *This is the first time the request has been made, so it is sent to the OpenAI API.*\
+  - **Input**: please give me a random number (`seed=1`). **Output**: 25
+    - *This is the first time the request has been made with this seed, so it is sent to the OpenAI API.*
+  - **Input**: please give me a random number (`seed=2`). **Output**: 126
+    - *This is the first time the request has been made with this seed, so it is sent to the OpenAI API.*
+  - **Input**: please give me a random number (`seed=2`). **Output**: 126
+    - *This request has been made with this seed before, the cached result is returned*
+  - **Input**: please give me a random number. **Output**: 42
+    - *This request has been made before and is being called again for the first time; return the first saved response.*
+  - **Input**: please give me a random number. **Output**: 25
+    - *This request has been made before and is being called again for the second time; return the second saved response.*
+  - **Input**: please give me a random number. **Output**: 126
+    - *This request has been made before and is being called again for the third time; return the third saved response.*
+  - **Input**: please give me a random number. **Output**: 42
+    - *This request has been made before and is being called again for the four time; there is no fourth response stored, so loop back to the first stored answer.*
+
+### Images
+
+Image requests that return image URLs require specific handling, because OpenAI does not keep these image URLs active forever - they are deleted a few hours after the request is made. Thus, simply storing the original response with the original URL would return a broken link when the cached response was returned.
+
+To solve this problem, `cached_openai` downloads the image from the URL before returning the result, and stores it in the cache. When the same request is later called and retrieved from the cache, the image is extracted into an `image` folder in the working directory, and the URLs in the response are replaced with local URLs pointing to those extracted images.
+
+### Audio
+
+Audio requests suffer from a similar problem - OpenAI returns a file stream which then needs to be downloaded. `cached_openai` handles this seamlessly by downloading the file at request time, saving it in the cache, and simulating a file stream when the request is called again and needs to be retrieved from the cache.
+
+## Distributing the cache
+
+Once you have created the cache for your class, there are three different ways to distribute it to students
+  - Get students to install `cached_openai` directly from pypi, and distribute a cache file that they can put in their working directory containing the cache data; you can also publish this file to a URL and `cached_openai` will ask them for a URL from which to download (and potentially decompress) the file.
+  - Create your own package on pypi that forks `cached_openai` and includes your own cache file in it (note that this only works if your cache file is small enough to be uploaded to pypi).
+  - Create a self-contained `.py` file that contains not only the `cached_openai` code, but also the cache itself, embedded in the file. You can then simply distribute this `.py` file as a "batteries included" package.
+
+In all three cases, the first step is to call the `materialize` function, which will package the cache for you; see the function docstring for details.

cached_openai-0.0.1/src/cached_openai/__init__.py

@@ -0,0 +1,10 @@
+# Make the main entrypoints into the package available at the top level of the package
+from .main import OpenAI, AsyncOpenAI, DEV_MODE
+
+# Make the materialization functions available at the top level of the package, if
+# we're in dev mode
+if DEV_MODE:
+    from .main import materialize
+
+# Remove DEV_MODE from the namespace
+del DEV_MODE