boxcars-datacircle 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +68 -0
- data/lib/boxcars/datacircle/client.rb +70 -0
- data/lib/boxcars/datacircle/tools.rb +265 -0
- data/lib/boxcars/datacircle/version.rb +7 -0
- data/lib/boxcars/datacircle.rb +6 -0
- metadata +72 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 33fe3fa76eb06a37e70630ab51318bb523f158d4a0cbfe4d25b28a2523f1c873
|
|
4
|
+
data.tar.gz: f1e13e625a4bc06c485eb6304784688c5242f5d3a77ea6fb9251acc1ca268d65
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: b4fc9e0472cace3ac51efe591ee0384d822e9a9fd88e5c8ae5c3ff9c953653ffa223addb5ac71176efcfcbbf9d4ae79ef0b7dd1b39872c79dbe260cb5e5a5925
|
|
7
|
+
data.tar.gz: 68dacbf06b6430caa2c4e79d15bc979b73fe15d9a7fc47fe1779e0c07a66fce37e27cb35a7b3642ec53e276303203cbe71272f391c9c1fe608d8a8a86ff0a8f4
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Datacircle
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Datacircle tools for Boxcars
|
|
2
|
+
|
|
3
|
+
Find the best GTM APIs. We benchmark all the GTM APIs and we put the results on our website: https://datacircle.dev
|
|
4
|
+
|
|
5
|
+
This gem gives [Boxcars](https://github.com/BoxcarsAI/boxcars) four tools that call Datacircle's open API, https://api.datacircle.dev/v1. You need no key and no account.
|
|
6
|
+
|
|
7
|
+
| Tool | What the tool gives | Call |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| `Boxcars::Datacircle::ListApis` (`datacircle_list_apis`) | Every GTM API on Datacircle, with the API's slug, number of plans, number of providers and best plan | `GET /apis` |
|
|
10
|
+
| `Boxcars::Datacircle::BestPlans` (`datacircle_best_plans`) | One API's plans, sorted by score out of 100, best first. Each plan comes with a rank and a price per 1,000 calls | `GET /apis/{slug}/best` |
|
|
11
|
+
| `Boxcars::Datacircle::SearchPlans` (`datacircle_search_plans`) | Plans that match a provider or plan name. Search all APIs or one API. Sort by score, provider or price | `GET /plans` |
|
|
12
|
+
| `Boxcars::Datacircle::Provider` (`datacircle_provider`) | The APIs a provider sells, and the provider's rank on each API | `GET /providers/{slug}` |
|
|
13
|
+
|
|
14
|
+
## Install
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
bundle add boxcars-datacircle
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
The gem needs Boxcars 0.10.14 or later, and Ruby 3.3 or later.
|
|
21
|
+
|
|
22
|
+
## Use the tools
|
|
23
|
+
|
|
24
|
+
`Boxcars::Datacircle.boxcars` gives the four tools. Give them to an agent:
|
|
25
|
+
|
|
26
|
+
```ruby
|
|
27
|
+
require "boxcars/datacircle"
|
|
28
|
+
|
|
29
|
+
agent = Boxcars::StationAgent.new(
|
|
30
|
+
instructions: "You help people choose a GTM API.",
|
|
31
|
+
tools: Boxcars::Datacircle.boxcars,
|
|
32
|
+
model: "gpt-4o"
|
|
33
|
+
)
|
|
34
|
+
puts agent.run("Which email finder API is best for under $5 per 1,000?")
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
`Boxcars::ToolTrain` takes them the same way, as `boxcars:`. Both make native tool calls, which Boxcars runs on `Boxcars::Openai`: add the `openai` gem to your Gemfile. With an engine like `Boxcars::Ollama`, use a text train, `Boxcars::ZeroShot` or `Boxcars::XMLZeroShot`:
|
|
38
|
+
|
|
39
|
+
```ruby
|
|
40
|
+
engine = Boxcars::Ollama.new(model: "qwen2.5:7b")
|
|
41
|
+
train = Boxcars::ZeroShot.new(boxcars: Boxcars::Datacircle.boxcars, engine: engine)
|
|
42
|
+
puts train.run("Which email finder API is best?")
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
A text train gives each tool one string: an API's slug, a provider's name or words of a name. A JSON object of the tool's parameters works too, like `{"slug": "email-finder", "limit": 3}`.
|
|
46
|
+
|
|
47
|
+
You can call a tool yourself:
|
|
48
|
+
|
|
49
|
+
```ruby
|
|
50
|
+
best = Boxcars::Datacircle::BestPlans.new
|
|
51
|
+
best.run(slug: "email-finder", limit: 3)
|
|
52
|
+
# => "{\"api\":\"Employee email\",\"slug\":\"email-finder\",...,\"plans\":[{\"api\":\"Employee email\",\"rank\":1,\"provider\":\"Emelia\",...}]}"
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Each tool answers JSON. Each plan comes with a rank, provider, plan, score, price per 1,000 (`perThousand`, in dollars) and a page on datacircle.dev. That page lists every field. When the API has no data, for example for an unknown slug, the answer is `{"error": ...}` with the API's words. The model reads the error and can call the tool again. `run_result` gives the answer as a `Boxcars::Result`, its status `:error` then.
|
|
56
|
+
|
|
57
|
+
## Network
|
|
58
|
+
|
|
59
|
+
Each tool call makes one HTTPS GET request to https://api.datacircle.dev/v1. A tool makes a second request when the model sends a name where a slug goes, or a provider where an API goes. A request holds only the tool's own arguments: no key, no config, nothing from your messages. The API takes up to 60 calls a minute from one IP address. A tool waits once, for a minute at most, when the API asks it to wait. The gem depends only on Boxcars and stores nothing.
|
|
60
|
+
|
|
61
|
+
## Links
|
|
62
|
+
|
|
63
|
+
- Datacircle: https://datacircle.dev
|
|
64
|
+
- The API: https://datacircle.dev/api
|
|
65
|
+
- The API's spec: https://api.datacircle.dev/v1/openapi.json
|
|
66
|
+
- What each key means: https://datacircle.dev/fields
|
|
67
|
+
|
|
68
|
+
MIT license.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'net/http'
|
|
5
|
+
require 'uri'
|
|
6
|
+
|
|
7
|
+
module Boxcars
|
|
8
|
+
module Datacircle
|
|
9
|
+
# What Datacircle's API answered when it gave no data (its HTTP status in #status), or why it could not be reached.
|
|
10
|
+
class Error < Boxcars::Error
|
|
11
|
+
attr_reader :status
|
|
12
|
+
|
|
13
|
+
def initialize(message = nil, status: nil)
|
|
14
|
+
super(message)
|
|
15
|
+
@status = status
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# One HTTPS GET to Datacircle's open API, with no key: its answer as a Hash or an Array.
|
|
20
|
+
class Client
|
|
21
|
+
BASE = 'https://api.datacircle.dev/v1'
|
|
22
|
+
|
|
23
|
+
def initialize(base_url: BASE)
|
|
24
|
+
@base = base_url.to_s.chomp('/')
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def get(path, params = {})
|
|
28
|
+
uri = URI("#{@base}#{path}")
|
|
29
|
+
query = params.reject { |_, value| value.nil? || value.to_s.strip.empty? }
|
|
30
|
+
uri.query = URI.encode_www_form(query) unless query.empty?
|
|
31
|
+
2.times do |attempt|
|
|
32
|
+
response = request(uri)
|
|
33
|
+
return JSON.parse(response.body) if response.is_a?(Net::HTTPSuccess)
|
|
34
|
+
|
|
35
|
+
if response.code == '429' && attempt.zero?
|
|
36
|
+
sleep(wait(response))
|
|
37
|
+
next
|
|
38
|
+
end
|
|
39
|
+
raise Error.new(words(response), status: response.code.to_i)
|
|
40
|
+
end
|
|
41
|
+
raise Error.new("Datacircle's API answered 429 twice: wait a minute and call again", status: 429)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
private
|
|
45
|
+
|
|
46
|
+
def request(uri)
|
|
47
|
+
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https', open_timeout: 15, read_timeout: 30) do |http|
|
|
48
|
+
http.get(uri.request_uri, 'Accept' => 'application/json', 'User-Agent' => "boxcars-datacircle/#{VERSION}")
|
|
49
|
+
end
|
|
50
|
+
rescue SocketError, SystemCallError, Timeout::Error, OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e
|
|
51
|
+
raise Error, "Datacircle's API could not be reached: #{e.message}"
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# The API asks to wait in Retry-After: its seconds, a minute at most.
|
|
55
|
+
def wait(response)
|
|
56
|
+
Integer(response['Retry-After'] || 30).clamp(1, 60)
|
|
57
|
+
rescue ArgumentError
|
|
58
|
+
30
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# The API's own words: its error of a 4xx, else its status.
|
|
62
|
+
def words(response)
|
|
63
|
+
parsed = JSON.parse(response.body.to_s)
|
|
64
|
+
parsed.is_a?(Hash) && parsed['error'] ? parsed['error'].to_s : "Datacircle's API answered #{response.code}"
|
|
65
|
+
rescue JSON::ParserError
|
|
66
|
+
"Datacircle's API answered #{response.code}"
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
@@ -0,0 +1,265 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Boxcars
|
|
4
|
+
# Datacircle's open API as four boxcars, with no key: find the best GTM APIs. We benchmark all the GTM APIs and we put
|
|
5
|
+
# the results on our website, https://datacircle.dev
|
|
6
|
+
#
|
|
7
|
+
# agent = Boxcars::StationAgent.new(instructions: "...", tools: Boxcars::Datacircle.boxcars, model: "gpt-4o")
|
|
8
|
+
#
|
|
9
|
+
# Each is one GET of https://api.datacircle.dev/v1 and answers a Boxcars::Result whose answer is JSON; the API's error,
|
|
10
|
+
# or why it could not be reached, is the JSON {"error": ...} with status :error, for the model to read and call again.
|
|
11
|
+
module Datacircle
|
|
12
|
+
# A plan holds about 90 keys (its terms of use, its aggregators...): these choose a plan, the rest is on its page.
|
|
13
|
+
PLAN_KEYS = %w[api rank provider plan score perThousand notFoundPerThousand billing pay currency units unit
|
|
14
|
+
strengths latencyMs page pricingPage docs].freeze
|
|
15
|
+
|
|
16
|
+
# Where a provider's plans are, for a model that sent a provider's name to an API's tool.
|
|
17
|
+
FOR_PROVIDER = "A provider's plans: datacircle_search_plans with its name as query, like bright data, " \
|
|
18
|
+
'or datacircle_provider.'
|
|
19
|
+
|
|
20
|
+
# The four boxcars, for a Train's boxcars or a StationAgent's tools. base_url: where they call,
|
|
21
|
+
# https://api.datacircle.dev/v1 when left out.
|
|
22
|
+
def self.boxcars(base_url: Client::BASE)
|
|
23
|
+
[ListApis, BestPlans, SearchPlans, Provider].map { |boxcar| boxcar.new(base_url: base_url) }
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# What the four share: their inputs, their answer and the API's rules.
|
|
27
|
+
class Tool < Boxcars::Boxcar
|
|
28
|
+
def initialize(name:, description:, parameters:, base_url: Client::BASE)
|
|
29
|
+
super(name: name, description: description, parameters: parameters)
|
|
30
|
+
@client = Client.new(base_url: base_url)
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
# A text train (Boxcars::ZeroShot) gives one string: the first parameter's value, or a JSON object of them all.
|
|
34
|
+
def input_keys
|
|
35
|
+
[parameters.keys.first]
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# Each parameter may be left out: a call that lacks what it needs answers an error the model reads.
|
|
39
|
+
def validate_inputs(inputs:)
|
|
40
|
+
inputs
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# { answer: Boxcars::Result }, its answer the call's JSON.
|
|
44
|
+
def call(inputs:)
|
|
45
|
+
found = begin
|
|
46
|
+
answer(**arguments(inputs))
|
|
47
|
+
rescue Error => e
|
|
48
|
+
{ 'error' => e.message }
|
|
49
|
+
end
|
|
50
|
+
json = JSON.generate(found)
|
|
51
|
+
{ answer: Boxcars::Result.new(status: found.key?('error') ? :error : :ok, answer: json, explanation: json) }
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
private
|
|
55
|
+
|
|
56
|
+
# The parameters given, by their names: a JSON object given as the first one's value is read as all of them.
|
|
57
|
+
def arguments(inputs)
|
|
58
|
+
given = inputs.to_h.transform_keys { |key| key.to_s.to_sym }
|
|
59
|
+
first = input_keys.first
|
|
60
|
+
text = given[first]
|
|
61
|
+
if text.is_a?(String) && text.strip.start_with?('{')
|
|
62
|
+
parsed = begin
|
|
63
|
+
JSON.parse(text)
|
|
64
|
+
rescue JSON::ParserError
|
|
65
|
+
nil
|
|
66
|
+
end
|
|
67
|
+
given = given.except(first).merge(parsed.transform_keys(&:to_sym)) if parsed.is_a?(Hash)
|
|
68
|
+
end
|
|
69
|
+
given.slice(*parameters.keys)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def brief(item, keys = PLAN_KEYS)
|
|
73
|
+
keys.each_with_object({}) { |key, kept| kept[key] = item[key] unless item[key].nil? }
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# A number the model sent, within the API's range, or nil when it sent none.
|
|
77
|
+
def number(value, low, high)
|
|
78
|
+
return nil if value.nil? || value.to_s.strip.empty?
|
|
79
|
+
return value.to_i.clamp(low, high) if value.is_a?(Numeric)
|
|
80
|
+
|
|
81
|
+
Integer(value.to_s.strip, 10).clamp(low, high)
|
|
82
|
+
rescue ArgumentError
|
|
83
|
+
nil
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
# The API's rule for a slug: the name in lowercase letters and digits, the rest a hyphen.
|
|
87
|
+
def slugify(value)
|
|
88
|
+
value.to_s.strip.downcase.unicode_normalize(:nfkd).gsub(/\p{Mn}/, '').gsub(/[^a-z0-9]+/, '-').gsub(/\A-|-\z/, '')
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# GET /apis, the APIs whose name or slug holds the most of the query's words first.
|
|
93
|
+
class ListApis < Tool
|
|
94
|
+
DESCRIPTION = "Lists the GTM APIs on Datacircle's sheet (https://datacircle.dev), with each API's slug, how many " \
|
|
95
|
+
"plans and providers it has and its best plan. Call it first to find an API's slug. Its input: words " \
|
|
96
|
+
"of the API's name, like email finder, or none for the sheet's order."
|
|
97
|
+
|
|
98
|
+
def initialize(name: 'datacircle_list_apis', description: DESCRIPTION, base_url: Client::BASE)
|
|
99
|
+
super(name: name, description: description, base_url: base_url, parameters: {
|
|
100
|
+
query: { type: :string, description: "Words of the API's name or slug, like email or linkedin profile: the " \
|
|
101
|
+
"APIs that hold the most of them come first. The sheet's order when left out." },
|
|
102
|
+
limit: { type: :integer, description: 'How many APIs to give, 1 to 200. 20 when left out: the sheet has about 150.' }
|
|
103
|
+
})
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
private
|
|
107
|
+
|
|
108
|
+
def answer(query: nil, limit: nil)
|
|
109
|
+
apis = @client.get('/apis')
|
|
110
|
+
words = query.to_s.downcase.tr('-', ' ').split
|
|
111
|
+
held = apis.each_with_index.filter_map do |api, index|
|
|
112
|
+
name = "#{api['api']} #{api['slug'].to_s.tr('-', ' ')}".downcase
|
|
113
|
+
matched = words.count { |word| name.include?(word) }
|
|
114
|
+
[-matched, index, api] if matched.positive? || words.empty?
|
|
115
|
+
end.sort_by { |matched, index, _| [matched, index] }
|
|
116
|
+
note = nil
|
|
117
|
+
if words.any? && (held.empty? || -held.first.first < words.size)
|
|
118
|
+
# A provider's name sent as an API's, like Bright Data, or a question: where its plans are, and the sheet's
|
|
119
|
+
# order when no API holds a word of it.
|
|
120
|
+
note = "No API's name holds every word of #{query.to_s.strip.inspect}. #{FOR_PROVIDER}"
|
|
121
|
+
if held.empty?
|
|
122
|
+
held = apis.each_with_index.map { |api, index| [0, index, api] }
|
|
123
|
+
note += " These are the sheet's APIs in its order."
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
found = { 'total' => held.size,
|
|
127
|
+
'apis' => held.first(number(limit, 1, 200) || 20).map { |_, _, api| brief(api, %w[api slug plans providers best bestPage sheetPage]) } }
|
|
128
|
+
note ? { 'note' => note }.merge(found) : found
|
|
129
|
+
end
|
|
130
|
+
end
|
|
131
|
+
|
|
132
|
+
# GET /apis/{slug}/best
|
|
133
|
+
class BestPlans < Tool
|
|
134
|
+
DESCRIPTION = "Gives an API's plans best first by their score out of 100 on Datacircle's benchmark " \
|
|
135
|
+
"(https://datacircle.dev), each with its rank, provider, plan, $ per 1,000 calls and its page. " \
|
|
136
|
+
"perThousand is a plan's price of 1,000 items in dollars, pay the plan's whole price in its currency. " \
|
|
137
|
+
"Its input: the API's slug, like email-finder, or a JSON object of slug and limit."
|
|
138
|
+
|
|
139
|
+
def initialize(name: 'datacircle_best_plans', description: DESCRIPTION, base_url: Client::BASE)
|
|
140
|
+
super(name: name, description: description, base_url: base_url, parameters: {
|
|
141
|
+
slug: { type: :string, required: true,
|
|
142
|
+
description: "The API's slug, like email-finder or linkedin-profile, as datacircle_list_apis gives it." },
|
|
143
|
+
limit: { type: :integer, description: 'How many plans to give, 1 to 50. 10 when left out.' }
|
|
144
|
+
})
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
private
|
|
148
|
+
|
|
149
|
+
def answer(slug: nil, limit: nil)
|
|
150
|
+
api = slugify(slug)
|
|
151
|
+
return { 'error' => "Give an API's slug, like email-finder: datacircle_list_apis gives every slug. #{FOR_PROVIDER}" } if api.empty?
|
|
152
|
+
|
|
153
|
+
begin
|
|
154
|
+
found = @client.get("/apis/#{api}/best", limit: number(limit, 1, 50))
|
|
155
|
+
rescue Error => e
|
|
156
|
+
raise unless e.status == 404
|
|
157
|
+
|
|
158
|
+
# An API's name sent as its slug, like Employee email: the API of that name.
|
|
159
|
+
named = @client.get('/apis').find { |entry| slugify(entry['api']) == api }
|
|
160
|
+
# A provider's name sent as the API's slug: the API's words, and where a provider's plans are.
|
|
161
|
+
return { 'error' => "#{e.message}. datacircle_list_apis gives every API's slug. #{FOR_PROVIDER}" } unless named
|
|
162
|
+
|
|
163
|
+
found = @client.get("/apis/#{named['slug']}/best", limit: number(limit, 1, 50))
|
|
164
|
+
end
|
|
165
|
+
found.merge('plans' => Array(found['plans']).map { |plan| brief(plan) })
|
|
166
|
+
end
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
# GET /plans
|
|
170
|
+
class SearchPlans < Tool
|
|
171
|
+
DESCRIPTION = "Searches every plan on Datacircle's sheet (https://datacircle.dev), or one API's, by provider or " \
|
|
172
|
+
'plan name, sorted by score, provider or price. Gives how many were found and up to limit of them. ' \
|
|
173
|
+
"perThousand is a plan's price of 1,000 items in dollars, pay the plan's whole price in its currency. " \
|
|
174
|
+
"Its input: words of a provider's or plan's name, like bright data, or a JSON object of api, query, " \
|
|
175
|
+
'sort, limit and offset.'
|
|
176
|
+
|
|
177
|
+
def initialize(name: 'datacircle_search_plans', description: DESCRIPTION, base_url: Client::BASE)
|
|
178
|
+
super(name: name, description: description, base_url: base_url, parameters: {
|
|
179
|
+
query: { type: :string, description: "Words that must all appear in the provider's or plan's name, like bright data." },
|
|
180
|
+
api: { type: :string, description: "Only this API's plans: its name or slug, like email-finder. A provider's " \
|
|
181
|
+
'name goes in query, never here. Every API when left out.' },
|
|
182
|
+
sort: { type: :string, enum: %w[score provider price],
|
|
183
|
+
description: 'score (best first, the default), provider (A to Z) or price ($ per 1,000, lowest first).' },
|
|
184
|
+
limit: { type: :integer, description: 'How many plans to give, 1 to 50. 10 when left out.' },
|
|
185
|
+
offset: { type: :integer, description: 'How many found plans to skip first. 0 when left out.' }
|
|
186
|
+
})
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
private
|
|
190
|
+
|
|
191
|
+
def answer(query: nil, api: nil, sort: nil, limit: nil, offset: nil)
|
|
192
|
+
notes = []
|
|
193
|
+
begin
|
|
194
|
+
found = search(api, query, sort, limit, offset)
|
|
195
|
+
rescue Error => e
|
|
196
|
+
raise unless e.status == 400 && e.message.include?('api must be') && query.to_s.strip.empty?
|
|
197
|
+
|
|
198
|
+
# A provider's name sent as the API: the plans whose provider or plan name holds it.
|
|
199
|
+
api, query = nil, api
|
|
200
|
+
found = search(api, query, sort, limit, offset)
|
|
201
|
+
notes << "No API is named #{query.to_s.strip.inspect}: these are the plans whose provider or plan name holds it."
|
|
202
|
+
end
|
|
203
|
+
if found['total'].to_i.zero? && query.to_s.include?('-')
|
|
204
|
+
# A slug sent as a name, like bright-data, which no name holds: its words, like bright data. A name with a
|
|
205
|
+
# hyphen, like A-Leads, is found as it is first.
|
|
206
|
+
words = query.to_s.tr('-', ' ').split.join(' ')
|
|
207
|
+
found = search(api, words, sort, limit, offset)
|
|
208
|
+
notes << "No provider's or plan's name holds #{query.to_s.strip.inspect}: these are the plans whose name holds #{words.inspect}."
|
|
209
|
+
end
|
|
210
|
+
found = found.merge('plans' => Array(found['plans']).map { |plan| brief(plan) })
|
|
211
|
+
notes.empty? ? found : { 'note' => notes.join(' ') }.merge(found)
|
|
212
|
+
end
|
|
213
|
+
|
|
214
|
+
def search(api, query, sort, limit, offset)
|
|
215
|
+
@client.get('/plans', api: api.to_s.strip, query: query.to_s.strip,
|
|
216
|
+
sort: (sort.to_s if %w[score provider price].include?(sort.to_s)),
|
|
217
|
+
limit: number(limit, 1, 50), offset: number(offset, 0, 1_000_000))
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
# GET /providers/{slug}
|
|
222
|
+
class Provider < Tool
|
|
223
|
+
DESCRIPTION = "Gives each API one provider sells and its rank on each on Datacircle's benchmark " \
|
|
224
|
+
"(https://datacircle.dev), with its best plan there, its best rank first. total is how many APIs it " \
|
|
225
|
+
"sells, ranked on how many of them it has a rank. Its input: the provider's slug or name, like " \
|
|
226
|
+
'bright-data or Bright Data.'
|
|
227
|
+
|
|
228
|
+
def initialize(name: 'datacircle_provider', description: DESCRIPTION, base_url: Client::BASE)
|
|
229
|
+
super(name: name, description: description, base_url: base_url, parameters: {
|
|
230
|
+
slug: { type: :string, required: true,
|
|
231
|
+
description: "The provider's slug, like bright-data, or its name, like Bright Data: the name is made a slug." }
|
|
232
|
+
})
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
private
|
|
236
|
+
|
|
237
|
+
def answer(slug: nil)
|
|
238
|
+
name = slugify(slug)
|
|
239
|
+
return { 'error' => "Give a provider's slug or name, like bright-data or Bright Data" } if name.empty?
|
|
240
|
+
|
|
241
|
+
begin
|
|
242
|
+
found = @client.get("/providers/#{name}")
|
|
243
|
+
rescue Error => e
|
|
244
|
+
raise unless e.status == 404
|
|
245
|
+
|
|
246
|
+
return { 'error' => "#{e.message.split(':').first}. A provider's plans by its name: datacircle_search_plans with " \
|
|
247
|
+
'words of its name as query, like bright data.' }
|
|
248
|
+
end
|
|
249
|
+
apis = Array(found['apis']).map do |entry|
|
|
250
|
+
plans = Array(entry['plans'])
|
|
251
|
+
brief(entry, %w[api slug rank bestPage]).merge('plans' => plans.size, 'best' => best(plans)).compact
|
|
252
|
+
end
|
|
253
|
+
# Its best rank first, the APIs where it has none last, in the API's order.
|
|
254
|
+
apis = apis.each_with_index.sort_by { |entry, index| [entry['rank'].is_a?(Numeric) ? entry['rank'] : Float::INFINITY, index] }.map(&:first)
|
|
255
|
+
found.merge('apis' => apis, 'total' => apis.size, 'ranked' => apis.count { |entry| entry['rank'].is_a?(Numeric) })
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
# A provider's best plan on an API: its lowest rank, else its first.
|
|
259
|
+
def best(plans)
|
|
260
|
+
top = plans.select { |plan| plan['rank'].is_a?(Numeric) }.min_by { |plan| plan['rank'] } || plans.first
|
|
261
|
+
top && brief(top, %w[plan score perThousand page])
|
|
262
|
+
end
|
|
263
|
+
end
|
|
264
|
+
end
|
|
265
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: boxcars-datacircle
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Wayne Hamadi
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: boxcars
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: 0.10.14
|
|
19
|
+
- - "<"
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: '2'
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
requirements:
|
|
26
|
+
- - ">="
|
|
27
|
+
- !ruby/object:Gem::Version
|
|
28
|
+
version: 0.10.14
|
|
29
|
+
- - "<"
|
|
30
|
+
- !ruby/object:Gem::Version
|
|
31
|
+
version: '2'
|
|
32
|
+
description: 'Four Boxcars tools on Datacircle''s open API, with no key: list the
|
|
33
|
+
GTM APIs, the best plans of one API, a search of every plan and one provider''s
|
|
34
|
+
ranks. We benchmark all the GTM APIs and we put the results on our website: https://datacircle.dev'
|
|
35
|
+
email:
|
|
36
|
+
- wayne@datacircle.dev
|
|
37
|
+
executables: []
|
|
38
|
+
extensions: []
|
|
39
|
+
extra_rdoc_files: []
|
|
40
|
+
files:
|
|
41
|
+
- LICENSE.txt
|
|
42
|
+
- README.md
|
|
43
|
+
- lib/boxcars/datacircle.rb
|
|
44
|
+
- lib/boxcars/datacircle/client.rb
|
|
45
|
+
- lib/boxcars/datacircle/tools.rb
|
|
46
|
+
- lib/boxcars/datacircle/version.rb
|
|
47
|
+
homepage: https://datacircle.dev
|
|
48
|
+
licenses:
|
|
49
|
+
- MIT
|
|
50
|
+
metadata:
|
|
51
|
+
homepage_uri: https://datacircle.dev
|
|
52
|
+
source_code_uri: https://github.com/waynehamadi/boxcars-datacircle
|
|
53
|
+
documentation_uri: https://github.com/waynehamadi/boxcars-datacircle#readme
|
|
54
|
+
bug_tracker_uri: https://github.com/waynehamadi/boxcars-datacircle/issues
|
|
55
|
+
rdoc_options: []
|
|
56
|
+
require_paths:
|
|
57
|
+
- lib
|
|
58
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
59
|
+
requirements:
|
|
60
|
+
- - ">="
|
|
61
|
+
- !ruby/object:Gem::Version
|
|
62
|
+
version: '3.3'
|
|
63
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
64
|
+
requirements:
|
|
65
|
+
- - ">="
|
|
66
|
+
- !ruby/object:Gem::Version
|
|
67
|
+
version: '0'
|
|
68
|
+
requirements: []
|
|
69
|
+
rubygems_version: 3.6.9
|
|
70
|
+
specification_version: 4
|
|
71
|
+
summary: Find the best GTM APIs, in Boxcars.
|
|
72
|
+
test_files: []
|