ruby-openai 4.3.2 → 5.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +5 -0
- data/CHANGELOG.md +19 -1
- data/Gemfile.lock +1 -1
- data/README.md +27 -4
- data/lib/openai/audio.rb +15 -0
- data/lib/openai/client.rb +33 -23
- data/lib/openai/files.rb +7 -8
- data/lib/openai/finetunes.rb +8 -9
- data/lib/openai/http.rb +10 -10
- data/lib/openai/images.rb +5 -6
- data/lib/openai/models.rb +4 -5
- data/lib/openai/version.rb +1 -1
- data/lib/openai.rb +13 -0
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6ea6e5d9149ffa94f53c0952491e827a5a082830cb5a4ecbdafe4e4d2523f54e
|
4
|
+
data.tar.gz: 1e8072b9fce1c48612b0120e1df4d6f45e422daec13a41e4b985f60d6cc07f6a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 00b71588418d3c33fb2511147e9a500755cf864c1d4cd7c420599b7b7af7d10bd4bb0b1490ce5399efbf38ce2527461ad40c21f20685b6aba40db275d7c9c633
|
7
|
+
data.tar.gz: e2574855121d6ed5126aa809b32feab815b1bd8f668c9eff1d3f6c9e9a25ed83cbb45c19e1900695b814d79b708a9cedfaf911ea5112ba2ff6eadcc76332f980
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -5,7 +5,25 @@ All notable changes to this project will be documented in this file.
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
7
7
|
|
8
|
-
## [
|
8
|
+
## [5.1.0] - 2023-08-20
|
9
|
+
|
10
|
+
### Added
|
11
|
+
|
12
|
+
- Added rough_token_count to estimate tokens in a string according to OpenAI's "rules of thumb". Thank you to [@jamiemccarthy](https://github.com/jamiemccarthy) for the idea and implementation!
|
13
|
+
|
14
|
+
## [5.0.0] - 2023-08-14
|
15
|
+
|
16
|
+
### Added
|
17
|
+
|
18
|
+
- Support multi-tenant use of the gem! Each client now holds its own config, so you can create unlimited clients in the same project, for example to Azure and OpenAI, or for different headers, access keys, etc.
|
19
|
+
- [BREAKING-ish] This change should only break your usage of ruby-openai if you are directly calling class methods like `OpenAI::Client.get` for some reason, as they are now instance methods. Normal usage of the gem should be unaffected, just you can make new clients and they'll keep their own config if you want, overriding the global config.
|
20
|
+
- Huge thanks to [@petergoldstein](https://github.com/petergoldstein) for his original work on this, [@cthulhu](https://github.com/cthulhu) for testing and many others for reviews and suggestions.
|
21
|
+
|
22
|
+
### Changed
|
23
|
+
|
24
|
+
- [BREAKING] Move audio related method to Audio model from Client model. You will need to update your code to handle this change, changing `client.translate` to `client.audio.translate` and `client.transcribe` to `client.audio.transcribe`.
|
25
|
+
|
26
|
+
## [4.3.2] - 2023-08-14
|
9
27
|
|
10
28
|
### Fixed
|
11
29
|
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -24,13 +24,17 @@ gem "ruby-openai"
|
|
24
24
|
|
25
25
|
And then execute:
|
26
26
|
|
27
|
+
```bash
|
27
28
|
$ bundle install
|
29
|
+
```
|
28
30
|
|
29
31
|
### Gem install
|
30
32
|
|
31
33
|
Or install with:
|
32
34
|
|
35
|
+
```bash
|
33
36
|
$ gem install ruby-openai
|
37
|
+
```
|
34
38
|
|
35
39
|
and require with:
|
36
40
|
|
@@ -68,6 +72,12 @@ Then you can create a client like this:
|
|
68
72
|
client = OpenAI::Client.new
|
69
73
|
```
|
70
74
|
|
75
|
+
You can still override the config defaults when making new clients; any options not included will fall back to any global config set with OpenAI.configure. e.g. in this example the organization_id, request_timeout, etc. will fallback to any set globally using OpenAI.configure, with only the access_token overridden:
|
76
|
+
|
77
|
+
```ruby
|
78
|
+
client = OpenAI::Client.new(access_token: "access_token_goes_here")
|
79
|
+
```
|
80
|
+
|
71
81
|
#### Custom timeout or base URI
|
72
82
|
|
73
83
|
The default timeout for any request using this library is 120 seconds. You can change that by passing a number of seconds to the `request_timeout` when initializing the client. You can also change the base URI used for all requests, eg. to use observability tools like [Helicone](https://docs.helicone.ai/quickstart/integrate-in-one-line-of-code), and add arbitrary other headers e.g. for [openai-caching-proxy-worker](https://github.com/6/openai-caching-proxy-worker):
|
@@ -80,7 +90,8 @@ client = OpenAI::Client.new(
|
|
80
90
|
extra_headers: {
|
81
91
|
"X-Proxy-TTL" => "43200", # For https://github.com/6/openai-caching-proxy-worker#specifying-a-cache-ttl
|
82
92
|
"X-Proxy-Refresh": "true", # For https://github.com/6/openai-caching-proxy-worker#refreshing-the-cache
|
83
|
-
"Helicone-Auth": "Bearer HELICONE_API_KEY" # For https://docs.helicone.ai/getting-started/integration-method/openai-proxy
|
93
|
+
"Helicone-Auth": "Bearer HELICONE_API_KEY", # For https://docs.helicone.ai/getting-started/integration-method/openai-proxy
|
94
|
+
"helicone-stream-force-format" => "true", # Use this with Helicone otherwise streaming drops chunks # https://github.com/alexrudall/ruby-openai/issues/251
|
84
95
|
}
|
85
96
|
)
|
86
97
|
```
|
@@ -116,6 +127,18 @@ To use the [Azure OpenAI Service](https://learn.microsoft.com/en-us/azure/cognit
|
|
116
127
|
|
117
128
|
where `AZURE_OPENAI_URI` is e.g. `https://custom-domain.openai.azure.com/openai/deployments/gpt-35-turbo`
|
118
129
|
|
130
|
+
### Counting Tokens
|
131
|
+
|
132
|
+
OpenAI parses prompt text into [tokens](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them), which are words or portions of words. (These tokens are unrelated to your API access_token.) Counting tokens can help you estimate your [costs](https://openai.com/pricing). It can also help you ensure your prompt text size is within the max-token limits of your model's context window, and choose an appropriate [`max_tokens`](https://platform.openai.com/docs/api-reference/chat/create#chat/create-max_tokens) completion parameter so your response will fit as well.
|
133
|
+
|
134
|
+
To estimate the token-count of your text:
|
135
|
+
|
136
|
+
```ruby
|
137
|
+
OpenAI.rough_token_count("Your text")
|
138
|
+
```
|
139
|
+
|
140
|
+
If you need a more accurate count, try [tiktoken_ruby](https://github.com/IAPark/tiktoken_ruby).
|
141
|
+
|
119
142
|
### Models
|
120
143
|
|
121
144
|
There are different models that can be used to generate text. For a full list and to retrieve information about a single model:
|
@@ -174,7 +197,7 @@ client.chat(
|
|
174
197
|
# => "Anna is a young woman in her mid-twenties, with wavy chestnut hair that falls to her shoulders..."
|
175
198
|
```
|
176
199
|
|
177
|
-
Note: the API docs state that token usage is included in the streamed chat chunk objects, but this doesn't currently appear to be the case.
|
200
|
+
Note: the API docs state that token usage is included in the streamed chat chunk objects, but this doesn't currently appear to be the case. To count tokens while streaming, try `OpenAI.rough_token_count` or [tiktoken_ruby](https://github.com/IAPark/tiktoken_ruby).
|
178
201
|
|
179
202
|
### Functions
|
180
203
|
|
@@ -411,7 +434,7 @@ Whisper is a speech to text model that can be used to generate text based on aud
|
|
411
434
|
The translations API takes as input the audio file in any of the supported languages and transcribes the audio into English.
|
412
435
|
|
413
436
|
```ruby
|
414
|
-
response = client.translate(
|
437
|
+
response = client.audio.translate(
|
415
438
|
parameters: {
|
416
439
|
model: "whisper-1",
|
417
440
|
file: File.open("path_to_file", "rb"),
|
@@ -425,7 +448,7 @@ puts response["text"]
|
|
425
448
|
The transcriptions API takes as input the audio file you want to transcribe and returns the text in the desired output file format.
|
426
449
|
|
427
450
|
```ruby
|
428
|
-
response = client.transcribe(
|
451
|
+
response = client.audio.transcribe(
|
429
452
|
parameters: {
|
430
453
|
model: "whisper-1",
|
431
454
|
file: File.open("path_to_file", "rb"),
|
data/lib/openai/audio.rb
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
module OpenAI
|
2
|
+
class Audio
|
3
|
+
def initialize(client:)
|
4
|
+
@client = client
|
5
|
+
end
|
6
|
+
|
7
|
+
def transcribe(parameters: {})
|
8
|
+
@client.multipart_post(path: "/audio/transcriptions", parameters: parameters)
|
9
|
+
end
|
10
|
+
|
11
|
+
def translate(parameters: {})
|
12
|
+
@client.multipart_post(path: "/audio/translations", parameters: parameters)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
data/lib/openai/client.rb
CHANGED
@@ -1,58 +1,68 @@
|
|
1
1
|
module OpenAI
|
2
2
|
class Client
|
3
|
-
|
3
|
+
include OpenAI::HTTP
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
5
|
+
CONFIG_KEYS = %i[
|
6
|
+
api_type
|
7
|
+
api_version
|
8
|
+
access_token
|
9
|
+
organization_id
|
10
|
+
uri_base
|
11
|
+
request_timeout
|
12
|
+
extra_headers
|
13
|
+
].freeze
|
14
|
+
attr_reader *CONFIG_KEYS
|
15
|
+
|
16
|
+
def initialize(config = {})
|
17
|
+
CONFIG_KEYS.each do |key|
|
18
|
+
# Set instance variables like api_type & access_token. Fall back to global config
|
19
|
+
# if not present.
|
20
|
+
instance_variable_set("@#{key}", config[key] || OpenAI.configuration.send(key))
|
21
|
+
end
|
12
22
|
end
|
13
23
|
|
14
24
|
def chat(parameters: {})
|
15
|
-
|
25
|
+
json_post(path: "/chat/completions", parameters: parameters)
|
16
26
|
end
|
17
27
|
|
18
28
|
def completions(parameters: {})
|
19
|
-
|
29
|
+
json_post(path: "/completions", parameters: parameters)
|
20
30
|
end
|
21
31
|
|
22
32
|
def edits(parameters: {})
|
23
|
-
|
33
|
+
json_post(path: "/edits", parameters: parameters)
|
24
34
|
end
|
25
35
|
|
26
36
|
def embeddings(parameters: {})
|
27
|
-
|
37
|
+
json_post(path: "/embeddings", parameters: parameters)
|
38
|
+
end
|
39
|
+
|
40
|
+
def audio
|
41
|
+
@audio ||= OpenAI::Audio.new(client: self)
|
28
42
|
end
|
29
43
|
|
30
44
|
def files
|
31
|
-
@files ||= OpenAI::Files.new
|
45
|
+
@files ||= OpenAI::Files.new(client: self)
|
32
46
|
end
|
33
47
|
|
34
48
|
def finetunes
|
35
|
-
@finetunes ||= OpenAI::Finetunes.new
|
49
|
+
@finetunes ||= OpenAI::Finetunes.new(client: self)
|
36
50
|
end
|
37
51
|
|
38
52
|
def images
|
39
|
-
@images ||= OpenAI::Images.new
|
53
|
+
@images ||= OpenAI::Images.new(client: self)
|
40
54
|
end
|
41
55
|
|
42
56
|
def models
|
43
|
-
@models ||= OpenAI::Models.new
|
57
|
+
@models ||= OpenAI::Models.new(client: self)
|
44
58
|
end
|
45
59
|
|
46
60
|
def moderations(parameters: {})
|
47
|
-
|
48
|
-
end
|
49
|
-
|
50
|
-
def transcribe(parameters: {})
|
51
|
-
OpenAI::Client.multipart_post(path: "/audio/transcriptions", parameters: parameters)
|
61
|
+
json_post(path: "/moderations", parameters: parameters)
|
52
62
|
end
|
53
63
|
|
54
|
-
def
|
55
|
-
|
64
|
+
def azure?
|
65
|
+
@api_type&.to_sym == :azure
|
56
66
|
end
|
57
67
|
end
|
58
68
|
end
|
data/lib/openai/files.rb
CHANGED
@@ -1,33 +1,32 @@
|
|
1
1
|
module OpenAI
|
2
2
|
class Files
|
3
|
-
def initialize(
|
4
|
-
|
5
|
-
OpenAI.configuration.organization_id = organization_id if organization_id
|
3
|
+
def initialize(client:)
|
4
|
+
@client = client
|
6
5
|
end
|
7
6
|
|
8
7
|
def list
|
9
|
-
|
8
|
+
@client.get(path: "/files")
|
10
9
|
end
|
11
10
|
|
12
11
|
def upload(parameters: {})
|
13
12
|
validate(file: parameters[:file])
|
14
13
|
|
15
|
-
|
14
|
+
@client.multipart_post(
|
16
15
|
path: "/files",
|
17
16
|
parameters: parameters.merge(file: File.open(parameters[:file]))
|
18
17
|
)
|
19
18
|
end
|
20
19
|
|
21
20
|
def retrieve(id:)
|
22
|
-
|
21
|
+
@client.get(path: "/files/#{id}")
|
23
22
|
end
|
24
23
|
|
25
24
|
def content(id:)
|
26
|
-
|
25
|
+
@client.get(path: "/files/#{id}/content")
|
27
26
|
end
|
28
27
|
|
29
28
|
def delete(id:)
|
30
|
-
|
29
|
+
@client.delete(path: "/files/#{id}")
|
31
30
|
end
|
32
31
|
|
33
32
|
private
|
data/lib/openai/finetunes.rb
CHANGED
@@ -1,28 +1,27 @@
|
|
1
1
|
module OpenAI
|
2
2
|
class Finetunes
|
3
|
-
def initialize(
|
4
|
-
|
5
|
-
OpenAI.configuration.organization_id = organization_id if organization_id
|
3
|
+
def initialize(client:)
|
4
|
+
@client = client
|
6
5
|
end
|
7
6
|
|
8
7
|
def list
|
9
|
-
|
8
|
+
@client.get(path: "/fine-tunes")
|
10
9
|
end
|
11
10
|
|
12
11
|
def create(parameters: {})
|
13
|
-
|
12
|
+
@client.json_post(path: "/fine-tunes", parameters: parameters)
|
14
13
|
end
|
15
14
|
|
16
15
|
def retrieve(id:)
|
17
|
-
|
16
|
+
@client.get(path: "/fine-tunes/#{id}")
|
18
17
|
end
|
19
18
|
|
20
19
|
def cancel(id:)
|
21
|
-
|
20
|
+
@client.multipart_post(path: "/fine-tunes/#{id}/cancel")
|
22
21
|
end
|
23
22
|
|
24
23
|
def events(id:)
|
25
|
-
|
24
|
+
@client.get(path: "/fine-tunes/#{id}/events")
|
26
25
|
end
|
27
26
|
|
28
27
|
def delete(fine_tuned_model:)
|
@@ -30,7 +29,7 @@ module OpenAI
|
|
30
29
|
raise ArgumentError, "Please give a fine_tuned_model name, not a fine-tune ID"
|
31
30
|
end
|
32
31
|
|
33
|
-
|
32
|
+
@client.delete(path: "/models/#{fine_tuned_model}")
|
34
33
|
end
|
35
34
|
end
|
36
35
|
end
|
data/lib/openai/http.rb
CHANGED
@@ -64,40 +64,40 @@ module OpenAI
|
|
64
64
|
|
65
65
|
def conn(multipart: false)
|
66
66
|
Faraday.new do |f|
|
67
|
-
f.options[:timeout] =
|
67
|
+
f.options[:timeout] = @request_timeout
|
68
68
|
f.request(:multipart) if multipart
|
69
69
|
end
|
70
70
|
end
|
71
71
|
|
72
72
|
def uri(path:)
|
73
|
-
if
|
74
|
-
base = File.join(
|
75
|
-
"#{base}?api-version=#{
|
73
|
+
if azure?
|
74
|
+
base = File.join(@uri_base, path)
|
75
|
+
"#{base}?api-version=#{@api_version}"
|
76
76
|
else
|
77
|
-
File.join(
|
77
|
+
File.join(@uri_base, @api_version, path)
|
78
78
|
end
|
79
79
|
end
|
80
80
|
|
81
81
|
def headers
|
82
|
-
if
|
82
|
+
if azure?
|
83
83
|
azure_headers
|
84
84
|
else
|
85
85
|
openai_headers
|
86
|
-
end.merge(
|
86
|
+
end.merge(@extra_headers || {})
|
87
87
|
end
|
88
88
|
|
89
89
|
def openai_headers
|
90
90
|
{
|
91
91
|
"Content-Type" => "application/json",
|
92
|
-
"Authorization" => "Bearer #{
|
93
|
-
"OpenAI-Organization" =>
|
92
|
+
"Authorization" => "Bearer #{@access_token}",
|
93
|
+
"OpenAI-Organization" => @organization_id
|
94
94
|
}
|
95
95
|
end
|
96
96
|
|
97
97
|
def azure_headers
|
98
98
|
{
|
99
99
|
"Content-Type" => "application/json",
|
100
|
-
"api-key" =>
|
100
|
+
"api-key" => @access_token
|
101
101
|
}
|
102
102
|
end
|
103
103
|
|
data/lib/openai/images.rb
CHANGED
@@ -1,20 +1,19 @@
|
|
1
1
|
module OpenAI
|
2
2
|
class Images
|
3
|
-
def initialize(
|
4
|
-
|
5
|
-
OpenAI.configuration.organization_id = organization_id if organization_id
|
3
|
+
def initialize(client: nil)
|
4
|
+
@client = client
|
6
5
|
end
|
7
6
|
|
8
7
|
def generate(parameters: {})
|
9
|
-
|
8
|
+
@client.json_post(path: "/images/generations", parameters: parameters)
|
10
9
|
end
|
11
10
|
|
12
11
|
def edit(parameters: {})
|
13
|
-
|
12
|
+
@client.multipart_post(path: "/images/edits", parameters: open_files(parameters))
|
14
13
|
end
|
15
14
|
|
16
15
|
def variations(parameters: {})
|
17
|
-
|
16
|
+
@client.multipart_post(path: "/images/variations", parameters: open_files(parameters))
|
18
17
|
end
|
19
18
|
|
20
19
|
private
|
data/lib/openai/models.rb
CHANGED
@@ -1,16 +1,15 @@
|
|
1
1
|
module OpenAI
|
2
2
|
class Models
|
3
|
-
def initialize(
|
4
|
-
|
5
|
-
OpenAI.configuration.organization_id = organization_id if organization_id
|
3
|
+
def initialize(client:)
|
4
|
+
@client = client
|
6
5
|
end
|
7
6
|
|
8
7
|
def list
|
9
|
-
|
8
|
+
@client.get(path: "/models")
|
10
9
|
end
|
11
10
|
|
12
11
|
def retrieve(id:)
|
13
|
-
|
12
|
+
@client.get(path: "/models/#{id}")
|
14
13
|
end
|
15
14
|
end
|
16
15
|
end
|
data/lib/openai/version.rb
CHANGED
data/lib/openai.rb
CHANGED
@@ -7,6 +7,7 @@ require_relative "openai/files"
|
|
7
7
|
require_relative "openai/finetunes"
|
8
8
|
require_relative "openai/images"
|
9
9
|
require_relative "openai/models"
|
10
|
+
require_relative "openai/audio"
|
10
11
|
require_relative "openai/version"
|
11
12
|
|
12
13
|
module OpenAI
|
@@ -51,4 +52,16 @@ module OpenAI
|
|
51
52
|
def self.configure
|
52
53
|
yield(configuration)
|
53
54
|
end
|
55
|
+
|
56
|
+
# Estimate the number of tokens in a string, using the rules of thumb from OpenAI:
|
57
|
+
# https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them
|
58
|
+
def self.rough_token_count(content = "")
|
59
|
+
raise ArgumentError, "rough_token_count requires a string" unless content.is_a? String
|
60
|
+
return 0 if content.empty?
|
61
|
+
|
62
|
+
count_by_chars = content.size / 4.0
|
63
|
+
count_by_words = content.split.size * 4.0 / 3
|
64
|
+
estimate = ((count_by_chars + count_by_words) / 2.0).round
|
65
|
+
[1, estimate].max
|
66
|
+
end
|
54
67
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: ruby-openai
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 5.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Alex
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-08-
|
11
|
+
date: 2023-08-20 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: faraday
|
@@ -63,6 +63,7 @@ files:
|
|
63
63
|
- bin/console
|
64
64
|
- bin/setup
|
65
65
|
- lib/openai.rb
|
66
|
+
- lib/openai/audio.rb
|
66
67
|
- lib/openai/client.rb
|
67
68
|
- lib/openai/compatibility.rb
|
68
69
|
- lib/openai/files.rb
|