dspy-datacircle 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +69 -0
- data/lib/dspy/datacircle/calls.rb +146 -0
- data/lib/dspy/datacircle/client.rb +76 -0
- data/lib/dspy/datacircle/toolset.rb +103 -0
- data/lib/dspy/datacircle/version.rb +7 -0
- data/lib/dspy/datacircle.rb +30 -0
- metadata +74 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: a968d7cd5cf794d752e989f0b717f68acfb2a5434b195f66ff7e7289a6c6e2f9
|
|
4
|
+
data.tar.gz: 969b581594380f3ba9c5ad6a17e5c4b8b9f5c6f0c0ed25caf9566df05ed293c4
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 69fcd39bce7c555e7743d8f38087e9c867fa86ce78530111d9d68261fb71109b66bfd976b3ab3e0f5c68f123712a4ed4fbccf53615b80239c1bc23640dce2f57
|
|
7
|
+
data.tar.gz: 1522fed7228db49a74f0dd56b1db7acd5b5d2b9c349fa6b4094427f342b7392b17496fffd4729f10716798a22af28ad9433e82caad47a08b3c3bb3d6caf7e5c3
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Datacircle
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
# Datacircle tools for DSPy.rb
|
|
2
|
+
|
|
3
|
+
Find the best GTM APIs. We benchmark all the GTM APIs and we put the results on our website: https://datacircle.dev
|
|
4
|
+
|
|
5
|
+
This gem gives [DSPy.rb](https://github.com/vicentereig/dspy.rb) a toolset of four tools that call Datacircle's API at https://api.datacircle.dev/v1. You need no key and no account to call the API.
|
|
6
|
+
|
|
7
|
+
| Tool | What the tool gives | Call |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| `datacircle_list_apis` | Every GTM API on Datacircle, with the API's slug, number of plans, number of providers and best plan | `GET /apis` |
|
|
10
|
+
| `datacircle_best_plans` | One API's plans, sorted by score out of 100, best first. Each plan comes with a rank and a price per 1,000 calls | `GET /apis/{slug}/best` |
|
|
11
|
+
| `datacircle_search_plans` | Plans that match a provider or plan name. Search all APIs or one API. Sort by score, provider or price | `GET /plans` |
|
|
12
|
+
| `datacircle_provider` | The APIs a provider sells, and the provider's rank on each API | `GET /providers/{slug}` |
|
|
13
|
+
|
|
14
|
+
## Install
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
bundle add dspy-datacircle
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
The gem needs DSPy.rb 1.0 or later, and Ruby 3.3 or later. Add your model's DSPy.rb gem too, like `dspy-openai` or `dspy-anthropic`.
|
|
21
|
+
|
|
22
|
+
## Give the tools to a ReAct agent
|
|
23
|
+
|
|
24
|
+
```ruby
|
|
25
|
+
require "dspy"
|
|
26
|
+
require "dspy/datacircle"
|
|
27
|
+
|
|
28
|
+
class FindAnApi < DSPy::Signature
|
|
29
|
+
description "Answer a question about GTM APIs and their plans."
|
|
30
|
+
|
|
31
|
+
input { const :question, String }
|
|
32
|
+
output { const :answer, String }
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
DSPy.configure { |c| c.lm = DSPy::LM.new("openai/gpt-4o-mini", api_key: ENV["OPENAI_API_KEY"]) }
|
|
36
|
+
|
|
37
|
+
agent = DSPy::ReAct.new(FindAnApi, tools: DSPy::Datacircle.tools)
|
|
38
|
+
result = agent.call(question: "Which email finder API is best for under $5 per 1,000?")
|
|
39
|
+
puts result.answer
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
`DSPy::Datacircle.tools` is `DSPy::Datacircle::Toolset.to_tools`: the four tools, each a `DSPy::Tools::Toolset` tool. ReAct sends each tool's name, description and JSON schema to the model, runs the tool the model picks and sends the tool's answer back. To offer fewer tools, pass fewer, like `tools: DSPy::Datacircle.tools.first(2)`.
|
|
43
|
+
|
|
44
|
+
You can call a tool yourself:
|
|
45
|
+
|
|
46
|
+
```ruby
|
|
47
|
+
tool = DSPy::Datacircle.tools.find { |t| t.name == "datacircle_best_plans" }
|
|
48
|
+
tool.dynamic_call({ "slug" => "email-finder", "limit" => 3 })
|
|
49
|
+
# => {"api" => "Employee email", "slug" => "email-finder", ..., "plans" => [{"api" => "Employee email", "rank" => 1, "provider" => "Emelia", ...}]}
|
|
50
|
+
|
|
51
|
+
DSPy::Datacircle::Toolset.new.provider(slug: "Bright Data")
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Code that `DSPy::CodeAct` writes can call the toolset's methods the same way, once your program requires `dspy/datacircle`.
|
|
55
|
+
|
|
56
|
+
Each tool returns a Hash. Each plan comes with a rank, provider, plan, score, price per 1,000 (`perThousand`, in dollars) and a page on datacircle.dev. The plan's page lists every field of the plan. For an unknown slug or any other API error, the tool returns `{"error" => ...}` with the API's error message. The model reads the error and can call the tool again. A tool never raises an exception for an API error. Your agent keeps running.
|
|
57
|
+
|
|
58
|
+
## Network
|
|
59
|
+
|
|
60
|
+
Each tool call makes one HTTPS GET request to https://api.datacircle.dev/v1. A tool makes a second request when the model sends a name in place of a slug. A tool also makes a second request when the model sends a provider in place of an API. A request holds the tool's own arguments and nothing else: no key and nothing from your config or messages. The API takes up to 60 calls a minute from one IP address. A tool waits once, for a minute at most, when the API replies with a wait. The gem depends only on DSPy.rb. The gem stores nothing.
|
|
61
|
+
|
|
62
|
+
## Links
|
|
63
|
+
|
|
64
|
+
- Datacircle: https://datacircle.dev
|
|
65
|
+
- The API: https://datacircle.dev/api
|
|
66
|
+
- The API's spec: https://api.datacircle.dev/v1/openapi.json
|
|
67
|
+
- The meaning of each key: https://datacircle.dev/fields
|
|
68
|
+
|
|
69
|
+
MIT license.
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module DSPy
|
|
4
|
+
module Datacircle
|
|
5
|
+
# A plan holds about 90 keys (its terms of use, its aggregators...): these choose a plan, the rest is on its page.
|
|
6
|
+
PLAN_KEYS = %w[api rank provider plan score perThousand notFoundPerThousand billing pay currency units unit
|
|
7
|
+
strengths latencyMs page pricingPage docs].freeze
|
|
8
|
+
|
|
9
|
+
# Where a provider's plans are, for a model that sent a provider's name to an API's tool.
|
|
10
|
+
FOR_PROVIDER = "A provider's plans: datacircle_search_plans with its name as query, like bright data, " \
|
|
11
|
+
'or datacircle_provider.'
|
|
12
|
+
|
|
13
|
+
PRICE = "perThousand is a plan's price of 1,000 items in dollars, pay the plan's whole price in its currency."
|
|
14
|
+
|
|
15
|
+
# The four calls of https://api.datacircle.dev/v1 behind the tools, each answering a Hash. Each raises Error when the
|
|
16
|
+
# API gives no data: the tools answer it as {"error" => ...}.
|
|
17
|
+
class Calls
|
|
18
|
+
def initialize(base_url: Client::BASE)
|
|
19
|
+
@client = Client.new(base_url: base_url)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
# GET /apis, the APIs whose name or slug holds the most of the query's words first.
|
|
23
|
+
def list_apis(query: nil, limit: nil)
|
|
24
|
+
apis = @client.get('/apis')
|
|
25
|
+
words = query.to_s.downcase.tr('-', ' ').split
|
|
26
|
+
held = apis.each_with_index.filter_map do |api, index|
|
|
27
|
+
name = "#{api['api']} #{api['slug'].to_s.tr('-', ' ')}".downcase
|
|
28
|
+
matched = words.count { |word| name.include?(word) }
|
|
29
|
+
[-matched, index, api] if matched.positive? || words.empty?
|
|
30
|
+
end.sort_by { |matched, index, _| [matched, index] }
|
|
31
|
+
note = nil
|
|
32
|
+
if words.any? && (held.empty? || -held.first.first < words.size)
|
|
33
|
+
# A provider's name sent as an API's, like Bright Data, or a question: where its plans are, and the sheet's
|
|
34
|
+
# order when no API holds a word of it.
|
|
35
|
+
note = "No API's name holds every word of #{query.to_s.strip.inspect}. #{FOR_PROVIDER}"
|
|
36
|
+
if held.empty?
|
|
37
|
+
held = apis.each_with_index.map { |api, index| [0, index, api] }
|
|
38
|
+
note += " These are the sheet's APIs in its order."
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
found = { 'total' => held.size,
|
|
42
|
+
'apis' => held.first(number(limit, 1, 200) || 20).map { |_, _, api| brief(api, %w[api slug plans providers best bestPage sheetPage]) } }
|
|
43
|
+
note ? { 'note' => note }.merge(found) : found
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# GET /apis/{slug}/best
|
|
47
|
+
def best_plans(slug: nil, limit: nil)
|
|
48
|
+
api = slugify(slug)
|
|
49
|
+
return { 'error' => "Give an API's slug, like email-finder: datacircle_list_apis gives every slug. #{FOR_PROVIDER}" } if api.empty?
|
|
50
|
+
|
|
51
|
+
begin
|
|
52
|
+
found = @client.get("/apis/#{api}/best", limit: number(limit, 1, 50))
|
|
53
|
+
rescue Error => e
|
|
54
|
+
raise unless e.status == 404
|
|
55
|
+
|
|
56
|
+
# An API's name sent as its slug, like Employee email: the API of that name.
|
|
57
|
+
named = @client.get('/apis').find { |entry| slugify(entry['api']) == api }
|
|
58
|
+
# A provider's name sent as the API's slug: the API's words, and where a provider's plans are.
|
|
59
|
+
return { 'error' => "#{e.message}. datacircle_list_apis gives every API's slug. #{FOR_PROVIDER}" } unless named
|
|
60
|
+
|
|
61
|
+
found = @client.get("/apis/#{named['slug']}/best", limit: number(limit, 1, 50))
|
|
62
|
+
end
|
|
63
|
+
found.merge('plans' => Array(found['plans']).map { |plan| brief(plan) })
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# GET /plans
|
|
67
|
+
def search_plans(query: nil, api: nil, sort: nil, limit: nil, offset: nil)
|
|
68
|
+
notes = []
|
|
69
|
+
begin
|
|
70
|
+
found = search(api, query, sort, limit, offset)
|
|
71
|
+
rescue Error => e
|
|
72
|
+
raise unless e.status == 400 && e.message.include?('api must be') && query.to_s.strip.empty?
|
|
73
|
+
|
|
74
|
+
# A provider's name sent as the API: the plans whose provider or plan name holds it.
|
|
75
|
+
api, query = nil, api
|
|
76
|
+
found = search(api, query, sort, limit, offset)
|
|
77
|
+
notes << "No API is named #{query.to_s.strip.inspect}: these are the plans whose provider or plan name holds it."
|
|
78
|
+
end
|
|
79
|
+
if found['total'].to_i.zero? && query.to_s.include?('-')
|
|
80
|
+
# A slug sent as a name, like bright-data, which no name holds: its words, like bright data. A name with a
|
|
81
|
+
# hyphen, like A-Leads, is found as it is first.
|
|
82
|
+
words = query.to_s.tr('-', ' ').split.join(' ')
|
|
83
|
+
found = search(api, words, sort, limit, offset)
|
|
84
|
+
notes << "No provider's or plan's name holds #{query.to_s.strip.inspect}: these are the plans whose name holds #{words.inspect}."
|
|
85
|
+
end
|
|
86
|
+
found = found.merge('plans' => Array(found['plans']).map { |plan| brief(plan) })
|
|
87
|
+
notes.empty? ? found : { 'note' => notes.join(' ') }.merge(found)
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
# GET /providers/{slug}
|
|
91
|
+
def provider(slug: nil)
|
|
92
|
+
name = slugify(slug)
|
|
93
|
+
return { 'error' => "Give a provider's slug or name, like bright-data or Bright Data" } if name.empty?
|
|
94
|
+
|
|
95
|
+
begin
|
|
96
|
+
found = @client.get("/providers/#{name}")
|
|
97
|
+
rescue Error => e
|
|
98
|
+
raise unless e.status == 404
|
|
99
|
+
|
|
100
|
+
return { 'error' => "#{e.message.split(':').first}. A provider's plans by its name: datacircle_search_plans with " \
|
|
101
|
+
'words of its name as query, like bright data.' }
|
|
102
|
+
end
|
|
103
|
+
apis = Array(found['apis']).map do |entry|
|
|
104
|
+
plans = Array(entry['plans'])
|
|
105
|
+
brief(entry, %w[api slug rank bestPage]).merge('plans' => plans.size, 'best' => best(plans)).compact
|
|
106
|
+
end
|
|
107
|
+
# Its best rank first, the APIs where it has none last, in the API's order.
|
|
108
|
+
apis = apis.each_with_index.sort_by { |entry, index| [entry['rank'].is_a?(Numeric) ? entry['rank'] : Float::INFINITY, index] }.map(&:first)
|
|
109
|
+
found.merge('apis' => apis, 'total' => apis.size, 'ranked' => apis.count { |entry| entry['rank'].is_a?(Numeric) })
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
private
|
|
113
|
+
|
|
114
|
+
def search(api, query, sort, limit, offset)
|
|
115
|
+
@client.get('/plans', api: api.to_s.strip, query: query.to_s.strip,
|
|
116
|
+
sort: (sort.to_s if %w[score provider price].include?(sort.to_s)),
|
|
117
|
+
limit: number(limit, 1, 50), offset: number(offset, 0, 1_000_000))
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
def brief(item, keys = PLAN_KEYS)
|
|
121
|
+
keys.each_with_object({}) { |key, kept| kept[key] = item[key] unless item[key].nil? }
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# A number the model sent, within the API's range, or nil when it sent none.
|
|
125
|
+
def number(value, low, high)
|
|
126
|
+
return nil if value.nil? || value.to_s.strip.empty?
|
|
127
|
+
return value.to_i.clamp(low, high) if value.is_a?(Numeric)
|
|
128
|
+
|
|
129
|
+
Integer(value.to_s.strip, 10).clamp(low, high)
|
|
130
|
+
rescue ArgumentError
|
|
131
|
+
nil
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
# The API's rule for a slug: the name in lowercase letters and digits, the rest a hyphen.
|
|
135
|
+
def slugify(value)
|
|
136
|
+
value.to_s.strip.downcase.unicode_normalize(:nfkd).gsub(/\p{Mn}/, '').gsub(/[^a-z0-9]+/, '-').gsub(/\A-|-\z/, '')
|
|
137
|
+
end
|
|
138
|
+
|
|
139
|
+
# A provider's best plan on an API: its lowest rank, else its first.
|
|
140
|
+
def best(plans)
|
|
141
|
+
top = plans.select { |plan| plan['rank'].is_a?(Numeric) }.min_by { |plan| plan['rank'] } || plans.first
|
|
142
|
+
top && brief(top, %w[plan score perThousand page])
|
|
143
|
+
end
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
end
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'net/http'
|
|
5
|
+
require 'uri'
|
|
6
|
+
|
|
7
|
+
module DSPy
|
|
8
|
+
module Datacircle
|
|
9
|
+
# What Datacircle's API answered when it gave no data (its HTTP status in #status), or why it could not be reached.
|
|
10
|
+
class Error < StandardError
|
|
11
|
+
attr_reader :status
|
|
12
|
+
|
|
13
|
+
def initialize(message = nil, status: nil)
|
|
14
|
+
super(message)
|
|
15
|
+
@status = status
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# One HTTPS GET to Datacircle's open API, with no key: its answer as a Hash or an Array.
|
|
20
|
+
class Client
|
|
21
|
+
BASE = 'https://api.datacircle.dev/v1'
|
|
22
|
+
|
|
23
|
+
def initialize(base_url: BASE)
|
|
24
|
+
@base = base_url.to_s.chomp('/')
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def get(path, params = {})
|
|
28
|
+
uri = URI("#{@base}#{path}")
|
|
29
|
+
query = params.reject { |_, value| value.nil? || value.to_s.strip.empty? }
|
|
30
|
+
uri.query = URI.encode_www_form(query) unless query.empty?
|
|
31
|
+
2.times do |attempt|
|
|
32
|
+
response = request(uri)
|
|
33
|
+
return parse(response) if response.is_a?(Net::HTTPSuccess)
|
|
34
|
+
|
|
35
|
+
if response.code == '429' && attempt.zero?
|
|
36
|
+
sleep(wait(response))
|
|
37
|
+
next
|
|
38
|
+
end
|
|
39
|
+
raise Error.new(words(response), status: response.code.to_i)
|
|
40
|
+
end
|
|
41
|
+
raise Error.new("Datacircle's API answered 429 twice: wait a minute and call again", status: 429)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
private
|
|
45
|
+
|
|
46
|
+
def request(uri)
|
|
47
|
+
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https', open_timeout: 15, read_timeout: 30) do |http|
|
|
48
|
+
http.get(uri.request_uri, 'Accept' => 'application/json', 'User-Agent' => "dspy-datacircle/#{VERSION}")
|
|
49
|
+
end
|
|
50
|
+
rescue SocketError, SystemCallError, Timeout::Error, OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e
|
|
51
|
+
raise Error, "Datacircle's API could not be reached: #{e.message}"
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def parse(response)
|
|
55
|
+
JSON.parse(response.body.to_s)
|
|
56
|
+
rescue JSON::ParserError
|
|
57
|
+
raise Error.new("Datacircle's API answered #{response.code} with no JSON", status: response.code.to_i)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
# The API asks to wait in Retry-After: its seconds, a minute at most.
|
|
61
|
+
def wait(response)
|
|
62
|
+
Integer(response['Retry-After'] || 30).clamp(1, 60)
|
|
63
|
+
rescue ArgumentError
|
|
64
|
+
30
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# The API's own words: its error of a 4xx, else its status.
|
|
68
|
+
def words(response)
|
|
69
|
+
parsed = JSON.parse(response.body.to_s)
|
|
70
|
+
parsed.is_a?(Hash) && parsed['error'] ? parsed['error'].to_s : "Datacircle's API answered #{response.code}"
|
|
71
|
+
rescue JSON::ParserError
|
|
72
|
+
"Datacircle's API answered #{response.code}"
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
end
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module DSPy
|
|
4
|
+
module Datacircle
|
|
5
|
+
# Datacircle's open API as a DSPy.rb toolset: four tools, datacircle_list_apis, datacircle_best_plans,
|
|
6
|
+
# datacircle_search_plans and datacircle_provider, each one GET of https://api.datacircle.dev/v1 with no key.
|
|
7
|
+
#
|
|
8
|
+
# DSPy::ReAct.new(FindAnApi, tools: DSPy::Datacircle::Toolset.to_tools)
|
|
9
|
+
#
|
|
10
|
+
# Each answers a Hash. The API's error, or why it could not be reached, is answered as {"error" => ...} for the
|
|
11
|
+
# model to read and call again, never raised.
|
|
12
|
+
class Toolset < DSPy::Tools::Toolset
|
|
13
|
+
extend T::Sig
|
|
14
|
+
|
|
15
|
+
toolset_name 'datacircle'
|
|
16
|
+
|
|
17
|
+
tool :list_apis,
|
|
18
|
+
description: "Lists the GTM APIs on Datacircle's sheet (https://datacircle.dev), with each API's slug, how many " \
|
|
19
|
+
'plans and providers it has and its best plan. Call it first to find an API\'s slug.'
|
|
20
|
+
tool :best_plans,
|
|
21
|
+
description: "Gives an API's plans best first by their score out of 100 on Datacircle's benchmark " \
|
|
22
|
+
'(https://datacircle.dev), each with its rank, provider, plan, $ per 1,000 calls and its page. ' \
|
|
23
|
+
"#{PRICE}"
|
|
24
|
+
tool :search_plans,
|
|
25
|
+
description: "Searches every plan on Datacircle's sheet (https://datacircle.dev), or one API's, by provider or " \
|
|
26
|
+
'plan name, sorted by score, provider or price. Gives how many were found and up to limit of them. ' \
|
|
27
|
+
"#{PRICE}"
|
|
28
|
+
tool :provider,
|
|
29
|
+
description: "Gives each API one provider sells and its rank on each on Datacircle's benchmark " \
|
|
30
|
+
'(https://datacircle.dev), with its best plan there, its best rank first. total is how many APIs ' \
|
|
31
|
+
'it sells, ranked on how many of them it has a rank.'
|
|
32
|
+
|
|
33
|
+
# What the model reads of each argument, in place of DSPy.rb's "Parameter limit (optional)".
|
|
34
|
+
ARGUMENTS = {
|
|
35
|
+
list_apis: {
|
|
36
|
+
query: { description: "Words of the API's name or slug, like email or linkedin profile: the APIs that hold the " \
|
|
37
|
+
'most of them come first. The sheet\'s order when left out.' },
|
|
38
|
+
limit: { description: 'How many APIs to give, 1 to 200. 20 when left out: the sheet has about 150.' }
|
|
39
|
+
},
|
|
40
|
+
best_plans: {
|
|
41
|
+
slug: { description: "The API's slug, like email-finder or linkedin-profile, as datacircle_list_apis gives it." },
|
|
42
|
+
limit: { description: 'How many plans to give, 1 to 50. 10 when left out.' }
|
|
43
|
+
},
|
|
44
|
+
search_plans: {
|
|
45
|
+
query: { description: "Words that must all appear in the provider's or plan's name, like bright data." },
|
|
46
|
+
api: { description: "Only this API's plans: its name or slug, like email-finder. A provider's name goes in " \
|
|
47
|
+
'query, never here. Every API when left out.' },
|
|
48
|
+
sort: { enum: %w[score provider price],
|
|
49
|
+
description: 'score (best first, the default), provider (A to Z) or price ($ per 1,000, lowest first).' },
|
|
50
|
+
limit: { description: 'How many plans to give, 1 to 50. 10 when left out.' },
|
|
51
|
+
offset: { description: 'How many found plans to skip first. 0 when left out.' }
|
|
52
|
+
},
|
|
53
|
+
provider: {
|
|
54
|
+
slug: { description: "The provider's slug, like bright-data, or its name, like Bright Data: the name is made a slug." }
|
|
55
|
+
}
|
|
56
|
+
}.freeze
|
|
57
|
+
|
|
58
|
+
def self.schema_for_method(method_name)
|
|
59
|
+
schema = super
|
|
60
|
+
ARGUMENTS.fetch(method_name, {}).each do |name, words|
|
|
61
|
+
schema[:properties][name] = schema[:properties][name].merge(words) if schema[:properties].key?(name)
|
|
62
|
+
end
|
|
63
|
+
schema
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def initialize(base_url: nil)
|
|
67
|
+
super()
|
|
68
|
+
@calls = Calls.new(base_url: base_url || Datacircle.base_url)
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
sig { params(query: T.nilable(String), limit: T.nilable(Integer)).returns(T::Hash[String, T.untyped]) }
|
|
72
|
+
def list_apis(query: nil, limit: nil)
|
|
73
|
+
answer { @calls.list_apis(query: query, limit: limit) }
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
sig { params(slug: String, limit: T.nilable(Integer)).returns(T::Hash[String, T.untyped]) }
|
|
77
|
+
def best_plans(slug:, limit: nil)
|
|
78
|
+
answer { @calls.best_plans(slug: slug, limit: limit) }
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
sig do
|
|
82
|
+
params(query: T.nilable(String), api: T.nilable(String), sort: T.nilable(String), limit: T.nilable(Integer),
|
|
83
|
+
offset: T.nilable(Integer)).returns(T::Hash[String, T.untyped])
|
|
84
|
+
end
|
|
85
|
+
def search_plans(query: nil, api: nil, sort: nil, limit: nil, offset: nil)
|
|
86
|
+
answer { @calls.search_plans(query: query, api: api, sort: sort, limit: limit, offset: offset) }
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
sig { params(slug: String).returns(T::Hash[String, T.untyped]) }
|
|
90
|
+
def provider(slug:)
|
|
91
|
+
answer { @calls.provider(slug: slug) }
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
private
|
|
95
|
+
|
|
96
|
+
def answer
|
|
97
|
+
yield
|
|
98
|
+
rescue Error => e
|
|
99
|
+
{ 'error' => e.message }
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
end
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'dspy'
|
|
5
|
+
require_relative 'datacircle/version'
|
|
6
|
+
require_relative 'datacircle/client'
|
|
7
|
+
require_relative 'datacircle/calls'
|
|
8
|
+
require_relative 'datacircle/toolset'
|
|
9
|
+
|
|
10
|
+
module DSPy
|
|
11
|
+
# Datacircle's open API as four DSPy.rb tools, with no key: find the best GTM APIs. We benchmark all the GTM APIs and
|
|
12
|
+
# we put the results on our website, https://datacircle.dev
|
|
13
|
+
#
|
|
14
|
+
# DSPy::ReAct.new(FindAnApi, tools: DSPy::Datacircle.tools)
|
|
15
|
+
module Datacircle
|
|
16
|
+
class << self
|
|
17
|
+
# Where the tools call, https://api.datacircle.dev/v1 unless set.
|
|
18
|
+
attr_writer :base_url
|
|
19
|
+
|
|
20
|
+
def base_url
|
|
21
|
+
@base_url || Client::BASE
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
# The four tools, for DSPy::ReAct's tools: DSPy::Datacircle::Toolset.to_tools.
|
|
25
|
+
def tools
|
|
26
|
+
Toolset.to_tools
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: dspy-datacircle
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Wayne Hamadi
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: dspy
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '1.0'
|
|
19
|
+
- - "<"
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: '2'
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
requirements:
|
|
26
|
+
- - ">="
|
|
27
|
+
- !ruby/object:Gem::Version
|
|
28
|
+
version: '1.0'
|
|
29
|
+
- - "<"
|
|
30
|
+
- !ruby/object:Gem::Version
|
|
31
|
+
version: '2'
|
|
32
|
+
description: 'A DSPy.rb toolset of four tools on Datacircle''s open API, with no key:
|
|
33
|
+
list the GTM APIs, the best plans of one API, a search of every plan and one provider''s
|
|
34
|
+
ranks, for DSPy::ReAct. We benchmark all the GTM APIs and we put the results on
|
|
35
|
+
our website: https://datacircle.dev'
|
|
36
|
+
email:
|
|
37
|
+
- wayne@datacircle.dev
|
|
38
|
+
executables: []
|
|
39
|
+
extensions: []
|
|
40
|
+
extra_rdoc_files: []
|
|
41
|
+
files:
|
|
42
|
+
- LICENSE.txt
|
|
43
|
+
- README.md
|
|
44
|
+
- lib/dspy/datacircle.rb
|
|
45
|
+
- lib/dspy/datacircle/calls.rb
|
|
46
|
+
- lib/dspy/datacircle/client.rb
|
|
47
|
+
- lib/dspy/datacircle/toolset.rb
|
|
48
|
+
- lib/dspy/datacircle/version.rb
|
|
49
|
+
homepage: https://datacircle.dev
|
|
50
|
+
licenses:
|
|
51
|
+
- MIT
|
|
52
|
+
metadata:
|
|
53
|
+
homepage_uri: https://datacircle.dev
|
|
54
|
+
source_code_uri: https://github.com/waynehamadi/dspy-datacircle
|
|
55
|
+
documentation_uri: https://github.com/waynehamadi/dspy-datacircle#readme
|
|
56
|
+
bug_tracker_uri: https://github.com/waynehamadi/dspy-datacircle/issues
|
|
57
|
+
rdoc_options: []
|
|
58
|
+
require_paths:
|
|
59
|
+
- lib
|
|
60
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
61
|
+
requirements:
|
|
62
|
+
- - ">="
|
|
63
|
+
- !ruby/object:Gem::Version
|
|
64
|
+
version: 3.3.0
|
|
65
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
66
|
+
requirements:
|
|
67
|
+
- - ">="
|
|
68
|
+
- !ruby/object:Gem::Version
|
|
69
|
+
version: '0'
|
|
70
|
+
requirements: []
|
|
71
|
+
rubygems_version: 3.6.9
|
|
72
|
+
specification_version: 4
|
|
73
|
+
summary: Find the best GTM APIs, in DSPy.rb.
|
|
74
|
+
test_files: []
|