ruby-openai-datacircle 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +105 -0
- data/lib/openai/datacircle/calls.rb +147 -0
- data/lib/openai/datacircle/client.rb +76 -0
- data/lib/openai/datacircle/tools.rb +172 -0
- data/lib/openai/datacircle/version.rb +7 -0
- data/lib/openai/datacircle.rb +23 -0
- data/lib/ruby-openai-datacircle.rb +4 -0
- metadata +75 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 7cf9f7af7f7b0b58cd54b06c3d8665c96385bb5fe4d0f6d584616cd09e8fa719
|
|
4
|
+
data.tar.gz: ab0d7dc2afcfb425442cd3dedc10f38f6fc22a5e3ac0d5eba05037a585e87287
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: b20a9ddfb03c3c8fd96b71a3cb45e5433d6b0c08d86b69b6eb3d0e7499c5d2c9646f3425bedb34adb309c8367ac05a7ae1c2f8949bfc625eaee7303a723a7755
|
|
7
|
+
data.tar.gz: 67b55d7432bf1769aa027408e1b34625b6298911e942c877c786cc6d0685206d4ce0b7ed2ce99f0974df8d4a992528a78e0c3a6d462b643eed6108ef035924a1
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Datacircle
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Datacircle tools for ruby-openai
|
|
2
|
+
|
|
3
|
+
Find the best GTM APIs. We benchmark all the GTM APIs and we put the results on our website.
|
|
4
|
+
|
|
5
|
+
Our website: https://datacircle.dev
|
|
6
|
+
|
|
7
|
+
This gem gives [ruby-openai](https://github.com/alexrudall/ruby-openai) four function tools that call Datacircle's open API, https://api.datacircle.dev/v1. You need no key and no account.
|
|
8
|
+
|
|
9
|
+
| Tool | What the tool gives | Call |
|
|
10
|
+
| --- | --- | --- |
|
|
11
|
+
| `datacircle_list_apis` | Every GTM API on Datacircle, with the API's slug, number of plans, number of providers and best plan | `GET /apis` |
|
|
12
|
+
| `datacircle_best_plans` | One API's plans, sorted by score out of 100, best first. Each plan comes with a rank and a price per 1,000 calls | `GET /apis/{slug}/best` |
|
|
13
|
+
| `datacircle_search_plans` | Plans that match a provider or plan name, on all APIs or on one API. You can sort them by score, provider or price | `GET /plans` |
|
|
14
|
+
| `datacircle_provider` | The APIs a provider sells, and the provider's rank on each API | `GET /providers/{slug}` |
|
|
15
|
+
|
|
16
|
+
## Install
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
bundle add ruby-openai-datacircle
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
The gem needs ruby-openai 6.0.0 or later and Ruby 3.2 or later. The Responses API needs ruby-openai 8.0.0 or later.
|
|
23
|
+
|
|
24
|
+
## Chat
|
|
25
|
+
|
|
26
|
+
`tools` returns the four Datacircle tools to pass to `client.chat`. `call` runs one tool call from the model's message. `call` returns the tool message with the answer.
|
|
27
|
+
|
|
28
|
+
```ruby
|
|
29
|
+
require "openai"
|
|
30
|
+
require "openai/datacircle"
|
|
31
|
+
|
|
32
|
+
client = OpenAI::Client.new(access_token: ENV.fetch("OPENAI_API_KEY"))
|
|
33
|
+
datacircle = OpenAI::Datacircle::Tools.new
|
|
34
|
+
messages = [{role: "user", content: "Who are the 3 best providers for the email-finder API?"}]
|
|
35
|
+
|
|
36
|
+
loop do
|
|
37
|
+
response = client.chat(parameters: {model: "gpt-5", messages: messages, tools: datacircle.tools})
|
|
38
|
+
message = response.dig("choices", 0, "message")
|
|
39
|
+
messages << message
|
|
40
|
+
break puts(message["content"]) unless message["tool_calls"]
|
|
41
|
+
|
|
42
|
+
message["tool_calls"].each do |tool_call|
|
|
43
|
+
messages << datacircle.call(tool_call)
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Your own tools can go in the same request: `tools: [*datacircle.tools, your_tool]`. `call` returns `nil` when the model calls one of your own tools. You run that call yourself: `datacircle.call(tool_call) || your_call(tool_call)`. `handles?(name)` returns true when `call` runs the tool named `name`. Check `handles?` before you pass a tool call to `call`. With a streamed reply, join the pieces of each tool call by their `index`. Then pass the whole tool call to `call`.
|
|
49
|
+
|
|
50
|
+
## Responses API
|
|
51
|
+
|
|
52
|
+
`response_tools` returns the four Datacircle tools to pass to `client.responses.create`. `outputs` runs a response's function calls and returns a `function_call_output` item for each one.
|
|
53
|
+
|
|
54
|
+
```ruby
|
|
55
|
+
require "openai"
|
|
56
|
+
require "openai/datacircle"
|
|
57
|
+
|
|
58
|
+
client = OpenAI::Client.new(access_token: ENV.fetch("OPENAI_API_KEY"))
|
|
59
|
+
datacircle = OpenAI::Datacircle::Tools.new
|
|
60
|
+
input = [{role: "user", content: "Who are the 3 best providers for the email-finder API?"}]
|
|
61
|
+
|
|
62
|
+
loop do
|
|
63
|
+
response = client.responses.create(parameters: {model: "gpt-5", input: input, tools: datacircle.response_tools})
|
|
64
|
+
outputs = datacircle.outputs(response)
|
|
65
|
+
if outputs.empty?
|
|
66
|
+
puts response["output"].select { _1["type"] == "message" }.flat_map { _1["content"] }.map { _1["text"] }.join
|
|
67
|
+
break
|
|
68
|
+
end
|
|
69
|
+
input += response["output"] + outputs
|
|
70
|
+
end
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
You can send OpenAI the outputs alone, with `previous_response_id: response["id"]`, in place of the whole input. `outputs` skips calls to functions outside the four Datacircle tools. You run those calls yourself.
|
|
74
|
+
|
|
75
|
+
## Call a tool yourself
|
|
76
|
+
|
|
77
|
+
You can call a tool without a model and read the JSON answer:
|
|
78
|
+
|
|
79
|
+
```ruby
|
|
80
|
+
OpenAI::Datacircle::Tools.new.run("datacircle_best_plans", '{"slug": "email-finder", "limit": 3}')
|
|
81
|
+
# => "{\"api\":\"Employee email\",\"slug\":\"email-finder\",...,\"plans\":[{\"api\":\"Employee email\",\"rank\":1,...}]}"
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
`OpenAI::Datacircle::Calls.new` makes the same calls and gives you a Hash. The methods of `Calls` are `list_apis`, `best_plans`, `search_plans` and `provider`.
|
|
85
|
+
|
|
86
|
+
## What the model reads
|
|
87
|
+
|
|
88
|
+
Each tool gives the model a JSON object as text. Each plan comes with a rank, provider, plan, score, price per 1,000 (`perThousand`, in dollars) and a page on datacircle.dev. That page lists every field. A tool also takes an API's name or a provider's name in place of a slug.
|
|
89
|
+
|
|
90
|
+
`call` and `outputs` give the model `{"error": "..."}` when the API gives no data, for example for an unknown slug. The error holds the API's own words and what to send instead. The model can then call the tool again. A network error reaches the model as `{"error": "..."}` too. `run` raises API errors and network errors as `OpenAI::Datacircle::Error`.
|
|
91
|
+
|
|
92
|
+
`OpenAI::Datacircle::Tools.new(base_url: ...)` sets a new address for the tools, for example a proxy of yours. Leave `base_url` out and the tools call https://api.datacircle.dev/v1.
|
|
93
|
+
|
|
94
|
+
## Network
|
|
95
|
+
|
|
96
|
+
Each tool call makes one HTTPS GET request to https://api.datacircle.dev/v1. A tool makes two requests when the model gives a name instead of a slug. A tool also makes two when the model gives a provider instead of an API. A request holds the tool's own arguments and nothing else: no key and nothing from your chats. The API takes up to 60 calls a minute from one IP address. A tool waits once, for a minute at most, when the API asks for a wait. The gem's one dependency is `ruby-openai`. The gem stores nothing.
|
|
97
|
+
|
|
98
|
+
## Links
|
|
99
|
+
|
|
100
|
+
- Datacircle: https://datacircle.dev
|
|
101
|
+
- The API: https://datacircle.dev/api
|
|
102
|
+
- The API's spec: https://api.datacircle.dev/v1/openapi.json
|
|
103
|
+
- What each key means: https://datacircle.dev/fields
|
|
104
|
+
|
|
105
|
+
MIT license.
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module OpenAI
|
|
4
|
+
module Datacircle
|
|
5
|
+
# A plan holds about 90 keys (its terms of use, its aggregators...): these choose a plan, the rest is on its page.
|
|
6
|
+
PLAN_KEYS = %w[api rank provider plan score perThousand notFoundPerThousand billing pay currency units unit
|
|
7
|
+
strengths latencyMs page pricingPage docs].freeze
|
|
8
|
+
|
|
9
|
+
# Where a provider's plans are, for a model that sent a provider's name to an API's tool.
|
|
10
|
+
FOR_PROVIDER = "A provider's plans: datacircle_search_plans with its name as query, like bright data, " \
|
|
11
|
+
'or datacircle_provider.'
|
|
12
|
+
|
|
13
|
+
PRICE = "perThousand is a plan's price of 1,000 items in dollars, pay the plan's whole price in its currency."
|
|
14
|
+
|
|
15
|
+
# The four calls of https://api.datacircle.dev/v1 behind the tools, each answering a Hash: {"error" => ...} for an
|
|
16
|
+
# input the API can't take, and Error raised when the API gives no data. Tools#call gives both to the model as
|
|
17
|
+
# {"error": ...}.
|
|
18
|
+
class Calls
|
|
19
|
+
def initialize(base_url: Client::BASE)
|
|
20
|
+
@client = Client.new(base_url: base_url)
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# GET /apis, the APIs whose name or slug holds the most of the query's words first.
|
|
24
|
+
def list_apis(query: nil, limit: nil)
|
|
25
|
+
apis = @client.get('/apis')
|
|
26
|
+
words = query.to_s.downcase.tr('-', ' ').split
|
|
27
|
+
held = apis.each_with_index.filter_map do |api, index|
|
|
28
|
+
name = "#{api['api']} #{api['slug'].to_s.tr('-', ' ')}".downcase
|
|
29
|
+
matched = words.count { |word| name.include?(word) }
|
|
30
|
+
[-matched, index, api] if matched.positive? || words.empty?
|
|
31
|
+
end.sort_by { |matched, index, _| [matched, index] }
|
|
32
|
+
note = nil
|
|
33
|
+
if words.any? && (held.empty? || -held.first.first < words.size)
|
|
34
|
+
# A provider's name sent as an API's, like Bright Data, or a question: where its plans are, and the sheet's
|
|
35
|
+
# order when no API holds a word of it.
|
|
36
|
+
note = "No API's name holds every word of #{query.to_s.strip.inspect}. #{FOR_PROVIDER}"
|
|
37
|
+
if held.empty?
|
|
38
|
+
held = apis.each_with_index.map { |api, index| [0, index, api] }
|
|
39
|
+
note += " These are the sheet's APIs in its order."
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
found = { 'total' => held.size,
|
|
43
|
+
'apis' => held.first(number(limit, 1, 200) || 20).map { |_, _, api| brief(api, %w[api slug plans providers best bestPage sheetPage]) } }
|
|
44
|
+
note ? { 'note' => note }.merge(found) : found
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# GET /apis/{slug}/best
|
|
48
|
+
def best_plans(slug: nil, limit: nil)
|
|
49
|
+
api = slugify(slug)
|
|
50
|
+
return { 'error' => "Give an API's slug, like email-finder: datacircle_list_apis gives every slug. #{FOR_PROVIDER}" } if api.empty?
|
|
51
|
+
|
|
52
|
+
begin
|
|
53
|
+
found = @client.get("/apis/#{api}/best", limit: number(limit, 1, 50))
|
|
54
|
+
rescue Error => e
|
|
55
|
+
raise unless e.status == 404
|
|
56
|
+
|
|
57
|
+
# An API's name sent as its slug, like Employee email: the API of that name.
|
|
58
|
+
named = @client.get('/apis').find { |entry| slugify(entry['api']) == api }
|
|
59
|
+
# A provider's name sent as the API's slug: the API's words, and where a provider's plans are.
|
|
60
|
+
return { 'error' => "#{e.message}. datacircle_list_apis gives every API's slug. #{FOR_PROVIDER}" } unless named
|
|
61
|
+
|
|
62
|
+
found = @client.get("/apis/#{named['slug']}/best", limit: number(limit, 1, 50))
|
|
63
|
+
end
|
|
64
|
+
found.merge('plans' => Array(found['plans']).map { |plan| brief(plan) })
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# GET /plans
|
|
68
|
+
def search_plans(query: nil, api: nil, sort: nil, limit: nil, offset: nil)
|
|
69
|
+
notes = []
|
|
70
|
+
begin
|
|
71
|
+
found = search(api, query, sort, limit, offset)
|
|
72
|
+
rescue Error => e
|
|
73
|
+
raise unless e.status == 400 && e.message.include?('api must be') && query.to_s.strip.empty?
|
|
74
|
+
|
|
75
|
+
# A provider's name sent as the API: the plans whose provider or plan name holds it.
|
|
76
|
+
api, query = nil, api
|
|
77
|
+
found = search(api, query, sort, limit, offset)
|
|
78
|
+
notes << "No API is named #{query.to_s.strip.inspect}: these are the plans whose provider or plan name holds it."
|
|
79
|
+
end
|
|
80
|
+
if found['total'].to_i.zero? && query.to_s.include?('-')
|
|
81
|
+
# A slug sent as a name, like bright-data, which no name holds: its words, like bright data. A name with a
|
|
82
|
+
# hyphen, like A-Leads, is found as it is first.
|
|
83
|
+
words = query.to_s.tr('-', ' ').split.join(' ')
|
|
84
|
+
found = search(api, words, sort, limit, offset)
|
|
85
|
+
notes << "No provider's or plan's name holds #{query.to_s.strip.inspect}: these are the plans whose name holds #{words.inspect}."
|
|
86
|
+
end
|
|
87
|
+
found = found.merge('plans' => Array(found['plans']).map { |plan| brief(plan) })
|
|
88
|
+
notes.empty? ? found : { 'note' => notes.join(' ') }.merge(found)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# GET /providers/{slug}
|
|
92
|
+
def provider(slug: nil)
|
|
93
|
+
name = slugify(slug)
|
|
94
|
+
return { 'error' => "Give a provider's slug or name, like bright-data or Bright Data" } if name.empty?
|
|
95
|
+
|
|
96
|
+
begin
|
|
97
|
+
found = @client.get("/providers/#{name}")
|
|
98
|
+
rescue Error => e
|
|
99
|
+
raise unless e.status == 404
|
|
100
|
+
|
|
101
|
+
return { 'error' => "#{e.message.split(':').first}. A provider's plans by its name: datacircle_search_plans with " \
|
|
102
|
+
'words of its name as query, like bright data.' }
|
|
103
|
+
end
|
|
104
|
+
apis = Array(found['apis']).map do |entry|
|
|
105
|
+
plans = Array(entry['plans'])
|
|
106
|
+
brief(entry, %w[api slug rank bestPage]).merge('plans' => plans.size, 'best' => best(plans)).compact
|
|
107
|
+
end
|
|
108
|
+
# Its best rank first, the APIs where it has none last, in the API's order.
|
|
109
|
+
apis = apis.each_with_index.sort_by { |entry, index| [entry['rank'].is_a?(Numeric) ? entry['rank'] : Float::INFINITY, index] }.map(&:first)
|
|
110
|
+
found.merge('apis' => apis, 'total' => apis.size, 'ranked' => apis.count { |entry| entry['rank'].is_a?(Numeric) })
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
private
|
|
114
|
+
|
|
115
|
+
def search(api, query, sort, limit, offset)
|
|
116
|
+
@client.get('/plans', api: api.to_s.strip, query: query.to_s.strip,
|
|
117
|
+
sort: (sort.to_s if %w[score provider price].include?(sort.to_s)),
|
|
118
|
+
limit: number(limit, 1, 50), offset: number(offset, 0, 1_000_000))
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
def brief(item, keys = PLAN_KEYS)
|
|
122
|
+
keys.each_with_object({}) { |key, kept| kept[key] = item[key] unless item[key].nil? }
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
# A number the model sent, within the API's range, or nil when it sent none.
|
|
126
|
+
def number(value, low, high)
|
|
127
|
+
return nil if value.nil? || value.to_s.strip.empty?
|
|
128
|
+
return value.to_i.clamp(low, high) if value.is_a?(Numeric)
|
|
129
|
+
|
|
130
|
+
Integer(value.to_s.strip, 10).clamp(low, high)
|
|
131
|
+
rescue ArgumentError
|
|
132
|
+
nil
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
# The API's rule for a slug: the name in lowercase letters and digits, the rest a hyphen.
|
|
136
|
+
def slugify(value)
|
|
137
|
+
value.to_s.strip.downcase.unicode_normalize(:nfkd).gsub(/\p{Mn}/, '').gsub(/[^a-z0-9]+/, '-').gsub(/\A-|-\z/, '')
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
# A provider's best plan on an API: its lowest rank, else its first.
|
|
141
|
+
def best(plans)
|
|
142
|
+
top = plans.select { |plan| plan['rank'].is_a?(Numeric) }.min_by { |plan| plan['rank'] } || plans.first
|
|
143
|
+
top && brief(top, %w[plan score perThousand page])
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
end
|
|
147
|
+
end
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'net/http'
|
|
5
|
+
require 'uri'
|
|
6
|
+
|
|
7
|
+
module OpenAI
|
|
8
|
+
module Datacircle
|
|
9
|
+
# What Datacircle's API answered when it gave no data (its HTTP status in #status), or why it could not be reached.
|
|
10
|
+
class Error < StandardError
|
|
11
|
+
attr_reader :status
|
|
12
|
+
|
|
13
|
+
def initialize(message = nil, status: nil)
|
|
14
|
+
super(message)
|
|
15
|
+
@status = status
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# One HTTPS GET to Datacircle's open API, with no key: its answer as a Hash or an Array.
|
|
20
|
+
class Client
|
|
21
|
+
BASE = 'https://api.datacircle.dev/v1'
|
|
22
|
+
|
|
23
|
+
def initialize(base_url: BASE)
|
|
24
|
+
@base = base_url.to_s.chomp('/')
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def get(path, params = {})
|
|
28
|
+
uri = URI("#{@base}#{path}")
|
|
29
|
+
query = params.reject { |_, value| value.nil? || value.to_s.strip.empty? }
|
|
30
|
+
uri.query = URI.encode_www_form(query) unless query.empty?
|
|
31
|
+
2.times do |attempt|
|
|
32
|
+
response = request(uri)
|
|
33
|
+
return parse(response) if response.is_a?(Net::HTTPSuccess)
|
|
34
|
+
|
|
35
|
+
if response.code == '429' && attempt.zero?
|
|
36
|
+
sleep(wait(response))
|
|
37
|
+
next
|
|
38
|
+
end
|
|
39
|
+
raise Error.new(words(response), status: response.code.to_i)
|
|
40
|
+
end
|
|
41
|
+
raise Error.new("Datacircle's API answered 429 twice: wait a minute and call again", status: 429)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
private
|
|
45
|
+
|
|
46
|
+
def request(uri)
|
|
47
|
+
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https', open_timeout: 15, read_timeout: 30) do |http|
|
|
48
|
+
http.get(uri.request_uri, 'Accept' => 'application/json', 'User-Agent' => "ruby-openai-datacircle/#{VERSION}")
|
|
49
|
+
end
|
|
50
|
+
rescue SocketError, SystemCallError, Timeout::Error, OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e
|
|
51
|
+
raise Error, "Datacircle's API could not be reached: #{e.message}"
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def parse(response)
|
|
55
|
+
JSON.parse(response.body.to_s)
|
|
56
|
+
rescue JSON::ParserError
|
|
57
|
+
raise Error.new("Datacircle's API answered #{response.code} with no JSON", status: response.code.to_i)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
# The API asks to wait in Retry-After: its seconds, a minute at most.
|
|
61
|
+
def wait(response)
|
|
62
|
+
Integer(response['Retry-After'] || 30).clamp(1, 60)
|
|
63
|
+
rescue ArgumentError
|
|
64
|
+
30
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# The API's own words: its error of a 4xx, else its status.
|
|
68
|
+
def words(response)
|
|
69
|
+
parsed = JSON.parse(response.body.to_s)
|
|
70
|
+
parsed.is_a?(Hash) && parsed['error'] ? parsed['error'].to_s : "Datacircle's API answered #{response.code}"
|
|
71
|
+
rescue JSON::ParserError
|
|
72
|
+
"Datacircle's API answered #{response.code}"
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
end
|
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module OpenAI
|
|
4
|
+
module Datacircle
|
|
5
|
+
LIST_APIS = 'datacircle_list_apis'
|
|
6
|
+
BEST_PLANS = 'datacircle_best_plans'
|
|
7
|
+
SEARCH_PLANS = 'datacircle_search_plans'
|
|
8
|
+
PROVIDER = 'datacircle_provider'
|
|
9
|
+
|
|
10
|
+
# The four tools' names, in the order Tools#tools gives them.
|
|
11
|
+
NAMES = [LIST_APIS, BEST_PLANS, SEARCH_PLANS, PROVIDER].freeze
|
|
12
|
+
|
|
13
|
+
# The four tools for OpenAI::Client#chat and client.responses.create, and the run of the calls the model makes.
|
|
14
|
+
#
|
|
15
|
+
# datacircle = OpenAI::Datacircle::Tools.new
|
|
16
|
+
# client.chat(parameters: { model: ..., messages: messages, tools: datacircle.tools })
|
|
17
|
+
# message["tool_calls"].each { |tool_call| messages << datacircle.call(tool_call) }
|
|
18
|
+
#
|
|
19
|
+
# It holds no state: one can serve many threads.
|
|
20
|
+
class Tools
|
|
21
|
+
ABOUT = {
|
|
22
|
+
LIST_APIS => "Lists the GTM APIs on Datacircle's sheet (https://datacircle.dev), with each API's slug, how many " \
|
|
23
|
+
"plans and providers it has and its best plan. Call it first to find an API's slug.",
|
|
24
|
+
BEST_PLANS => "Gives an API's plans best first by their score out of 100 on Datacircle's benchmark " \
|
|
25
|
+
'(https://datacircle.dev), each with its rank, provider, plan, $ per 1,000 calls and its page. ' \
|
|
26
|
+
"#{PRICE}",
|
|
27
|
+
SEARCH_PLANS => "Searches every plan on Datacircle's sheet (https://datacircle.dev), or one API's, by provider " \
|
|
28
|
+
'or plan name, sorted by score, provider or price. Gives how many were found and up to limit ' \
|
|
29
|
+
"of them. #{PRICE}",
|
|
30
|
+
PROVIDER => "Gives each API one provider sells and its rank on each on Datacircle's benchmark " \
|
|
31
|
+
'(https://datacircle.dev), with its best plan there, its best rank first. total is how many APIs ' \
|
|
32
|
+
'it sells, ranked on how many of them it has a rank.'
|
|
33
|
+
}.freeze
|
|
34
|
+
|
|
35
|
+
PARAMETERS = {
|
|
36
|
+
LIST_APIS => {
|
|
37
|
+
'query' => { 'type' => 'string', 'description' => "Words of the API's name or slug, like email or linkedin " \
|
|
38
|
+
'profile: the APIs that hold the most of them come ' \
|
|
39
|
+
"first. The sheet's order when left out." },
|
|
40
|
+
'limit' => { 'type' => 'integer', 'minimum' => 1, 'maximum' => 200,
|
|
41
|
+
'description' => 'How many APIs to give, 1 to 200. 20 when left out: the sheet has about 150.' }
|
|
42
|
+
},
|
|
43
|
+
BEST_PLANS => {
|
|
44
|
+
'slug' => { 'type' => 'string', 'description' => "The API's slug, like email-finder or linkedin-profile, as " \
|
|
45
|
+
'datacircle_list_apis gives it.' },
|
|
46
|
+
'limit' => { 'type' => 'integer', 'minimum' => 1, 'maximum' => 50,
|
|
47
|
+
'description' => 'How many plans to give, 1 to 50. 10 when left out.' }
|
|
48
|
+
},
|
|
49
|
+
SEARCH_PLANS => {
|
|
50
|
+
'query' => { 'type' => 'string', 'description' => "Words that must all appear in the provider's or plan's " \
|
|
51
|
+
'name, like bright data.' },
|
|
52
|
+
'api' => { 'type' => 'string', 'description' => "Only this API's plans: its name or slug, like email-finder. " \
|
|
53
|
+
"A provider's name goes in query, never here. Every API " \
|
|
54
|
+
'when left out.' },
|
|
55
|
+
'sort' => { 'type' => 'string', 'enum' => %w[score provider price],
|
|
56
|
+
'description' => 'score (best first, the default), provider (A to Z) or price ($ per 1,000, ' \
|
|
57
|
+
'lowest first).' },
|
|
58
|
+
'limit' => { 'type' => 'integer', 'minimum' => 1, 'maximum' => 50,
|
|
59
|
+
'description' => 'How many plans to give, 1 to 50. 10 when left out.' },
|
|
60
|
+
'offset' => { 'type' => 'integer', 'minimum' => 0,
|
|
61
|
+
'description' => 'How many found plans to skip first. 0 when left out.' }
|
|
62
|
+
},
|
|
63
|
+
PROVIDER => {
|
|
64
|
+
'slug' => { 'type' => 'string', 'description' => "The provider's slug, like bright-data, or its name, like " \
|
|
65
|
+
'Bright Data: the name is made a slug.' }
|
|
66
|
+
}
|
|
67
|
+
}.freeze
|
|
68
|
+
|
|
69
|
+
REQUIRED = { BEST_PLANS => ['slug'], PROVIDER => ['slug'] }.freeze
|
|
70
|
+
|
|
71
|
+
# base_url: where the tools call, https://api.datacircle.dev/v1 when left out.
|
|
72
|
+
def initialize(base_url: nil)
|
|
73
|
+
@base_url = base_url || Client::BASE
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# The four tools for a chat request's tools:, each a function tool.
|
|
77
|
+
def tools
|
|
78
|
+
NAMES.map { |name| { 'type' => 'function', 'function' => function(name) } }
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
# The four tools for a Responses API request's tools: (client.responses.create, ruby-openai 8.0.0 and later).
|
|
82
|
+
def response_tools
|
|
83
|
+
NAMES.map { |name| { 'type' => 'function' }.merge(function(name)).merge('strict' => false) }
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
# Whether name is one of the four tools'.
|
|
87
|
+
def handles?(name)
|
|
88
|
+
NAMES.include?(name.to_s)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# The tool message that answers a tool call of a chat reply's message ("tool_calls"), to add to the messages; nil
|
|
92
|
+
# for a call of a tool that isn't one of the four, which you run yourself.
|
|
93
|
+
def call(tool_call)
|
|
94
|
+
function = value(tool_call, 'function') || {}
|
|
95
|
+
name = value(function, 'name')
|
|
96
|
+
return nil unless handles?(name)
|
|
97
|
+
|
|
98
|
+
{ 'role' => 'tool', 'tool_call_id' => value(tool_call, 'id'), 'content' => output(name, value(function, 'arguments')) }
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
# A function_call_output item for each call of the four tools in a response (or its "output"), to add to the next
|
|
102
|
+
# request's input. A call of another function gets none: you run it yourself.
|
|
103
|
+
def outputs(response)
|
|
104
|
+
items = response.is_a?(Array) ? response : Array(value(response, 'output'))
|
|
105
|
+
items.filter_map do |item|
|
|
106
|
+
next unless value(item, 'type').to_s == 'function_call' && handles?(value(item, 'name'))
|
|
107
|
+
|
|
108
|
+
{ 'type' => 'function_call_output', 'call_id' => value(item, 'call_id'),
|
|
109
|
+
'output' => output(value(item, 'name'), value(item, 'arguments')) }
|
|
110
|
+
end
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
# The answer of one tool to the model's arguments (a JSON object, as text or a Hash), as JSON text. Raises Error
|
|
114
|
+
# with words for the model when the API gives no data, the arguments can't be read or the tool is not one of the
|
|
115
|
+
# four. Keys the tool doesn't take are dropped, a number sent as text is read, one out of range is brought into it.
|
|
116
|
+
def run(name, arguments = nil)
|
|
117
|
+
raise Error, "No tool is named #{name.to_s.inspect}: the tools are #{NAMES.join(', ')}" unless handles?(name)
|
|
118
|
+
|
|
119
|
+
found = answer(name.to_s, read(arguments))
|
|
120
|
+
raise Error, found['error'] if found.keys == ['error']
|
|
121
|
+
|
|
122
|
+
JSON.generate(found)
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
# The answer the model reads: run's JSON, or {"error": ...} with words it can act on.
|
|
126
|
+
def output(name, arguments)
|
|
127
|
+
run(name, arguments)
|
|
128
|
+
rescue Error => e
|
|
129
|
+
JSON.generate('error' => e.message)
|
|
130
|
+
end
|
|
131
|
+
|
|
132
|
+
private
|
|
133
|
+
|
|
134
|
+
def function(name)
|
|
135
|
+
parameters = { 'type' => 'object', 'properties' => PARAMETERS[name] }
|
|
136
|
+
parameters['required'] = REQUIRED[name] if REQUIRED.key?(name)
|
|
137
|
+
{ 'name' => name, 'description' => ABOUT[name], 'parameters' => parameters }
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
def answer(name, given)
|
|
141
|
+
calls = Calls.new(base_url: @base_url)
|
|
142
|
+
case name
|
|
143
|
+
when LIST_APIS then calls.list_apis(**pick(given, :query, :limit))
|
|
144
|
+
when BEST_PLANS then calls.best_plans(**pick(given, :slug, :limit))
|
|
145
|
+
when SEARCH_PLANS then calls.search_plans(**pick(given, :query, :api, :sort, :limit, :offset))
|
|
146
|
+
when PROVIDER then calls.provider(**pick(given, :slug))
|
|
147
|
+
end
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
def pick(given, *keys)
|
|
151
|
+
keys.each_with_object({}) { |key, kept| kept[key] = given[key.to_s] unless given[key.to_s].nil? }
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# The model's arguments as a Hash of strings: none is {}, and text that is no JSON object is an Error.
|
|
155
|
+
def read(arguments)
|
|
156
|
+
given = arguments.is_a?(String) ? JSON.parse(arguments.strip.empty? ? '{}' : arguments) : arguments || {}
|
|
157
|
+
raise Error, "The arguments are no JSON object of this tool's parameters" unless given.is_a?(Hash)
|
|
158
|
+
|
|
159
|
+
given.to_h { |key, item| [key.to_s, item] }
|
|
160
|
+
rescue JSON::ParserError
|
|
161
|
+
raise Error, "The arguments are no JSON object of this tool's parameters"
|
|
162
|
+
end
|
|
163
|
+
|
|
164
|
+
# A key of a Hash as ruby-openai parses it (strings) or as an app writes it (symbols).
|
|
165
|
+
def value(hash, key)
|
|
166
|
+
return nil unless hash.respond_to?(:key?)
|
|
167
|
+
|
|
168
|
+
hash.key?(key) ? hash[key] : hash[key.to_sym]
|
|
169
|
+
end
|
|
170
|
+
end
|
|
171
|
+
end
|
|
172
|
+
end
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'openai'
|
|
5
|
+
require_relative 'datacircle/version'
|
|
6
|
+
require_relative 'datacircle/client'
|
|
7
|
+
require_relative 'datacircle/calls'
|
|
8
|
+
require_relative 'datacircle/tools'
|
|
9
|
+
|
|
10
|
+
module OpenAI
|
|
11
|
+
# Datacircle's open API as four function tools for ruby-openai, with no key: find the best GTM APIs. We benchmark all
|
|
12
|
+
# the GTM APIs and we put the results on our website, https://datacircle.dev
|
|
13
|
+
#
|
|
14
|
+
# datacircle = OpenAI::Datacircle::Tools.new
|
|
15
|
+
# client.chat(parameters: { model: ..., messages: messages, tools: datacircle.tools })
|
|
16
|
+
#
|
|
17
|
+
# datacircle_list_apis, datacircle_best_plans, datacircle_search_plans and datacircle_provider make one GET of
|
|
18
|
+
# https://api.datacircle.dev/v1 each. Tools#call runs a tool call of the model into its tool message, Tools#outputs a
|
|
19
|
+
# response's function calls into function_call_output items. The API's error, or why it could not be reached, goes to
|
|
20
|
+
# the model as {"error": ...}.
|
|
21
|
+
module Datacircle
|
|
22
|
+
end
|
|
23
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: ruby-openai-datacircle
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Wayne Hamadi
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: ruby-openai
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: 6.0.0
|
|
19
|
+
- - "<"
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: '9'
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
requirements:
|
|
26
|
+
- - ">="
|
|
27
|
+
- !ruby/object:Gem::Version
|
|
28
|
+
version: 6.0.0
|
|
29
|
+
- - "<"
|
|
30
|
+
- !ruby/object:Gem::Version
|
|
31
|
+
version: '9'
|
|
32
|
+
description: 'Four ruby-openai tools call Datacircle''s open API with no key. List
|
|
33
|
+
the GTM APIs and each API''s best plans. Search every plan, or see where one provider
|
|
34
|
+
ranks in our benchmarks. We benchmark all the GTM APIs and we put the results on
|
|
35
|
+
our website: https://datacircle.dev'
|
|
36
|
+
email:
|
|
37
|
+
- wayne@datacircle.dev
|
|
38
|
+
executables: []
|
|
39
|
+
extensions: []
|
|
40
|
+
extra_rdoc_files: []
|
|
41
|
+
files:
|
|
42
|
+
- LICENSE.txt
|
|
43
|
+
- README.md
|
|
44
|
+
- lib/openai/datacircle.rb
|
|
45
|
+
- lib/openai/datacircle/calls.rb
|
|
46
|
+
- lib/openai/datacircle/client.rb
|
|
47
|
+
- lib/openai/datacircle/tools.rb
|
|
48
|
+
- lib/openai/datacircle/version.rb
|
|
49
|
+
- lib/ruby-openai-datacircle.rb
|
|
50
|
+
homepage: https://datacircle.dev
|
|
51
|
+
licenses:
|
|
52
|
+
- MIT
|
|
53
|
+
metadata:
|
|
54
|
+
homepage_uri: https://datacircle.dev
|
|
55
|
+
source_code_uri: https://github.com/waynehamadi/ruby-openai-datacircle
|
|
56
|
+
documentation_uri: https://github.com/waynehamadi/ruby-openai-datacircle#readme
|
|
57
|
+
bug_tracker_uri: https://github.com/waynehamadi/ruby-openai-datacircle/issues
|
|
58
|
+
rdoc_options: []
|
|
59
|
+
require_paths:
|
|
60
|
+
- lib
|
|
61
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
62
|
+
requirements:
|
|
63
|
+
- - ">="
|
|
64
|
+
- !ruby/object:Gem::Version
|
|
65
|
+
version: 3.2.0
|
|
66
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
67
|
+
requirements:
|
|
68
|
+
- - ">="
|
|
69
|
+
- !ruby/object:Gem::Version
|
|
70
|
+
version: '0'
|
|
71
|
+
requirements: []
|
|
72
|
+
rubygems_version: 3.6.9
|
|
73
|
+
specification_version: 4
|
|
74
|
+
summary: Tools that find the best GTM APIs, for ruby-openai's chat and Responses API.
|
|
75
|
+
test_files: []
|