dataforseo-ruby 0.3.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: bd182bce70ec48794ea1e065277330eb1c86a25b2decd936ea7658273ba0d6d6
+  data.tar.gz: 20abf4e0a70019910f45b9df3999d922d7177370363439d6c93c3b4146aa9884
+SHA512:
+  metadata.gz: 2fbd2611d29d93cf391b1b81f460009a83d0453911199dfaeb93350d717b526934c276ee4939c11ebd2597510c1f5eeda33e1e1cf85cc610f0ef00a69029380f
+  data.tar.gz: 16e8bbf39e81b771e2c75b467b7ec6027580f806d36666a9e27e63657dc79457c4a9e893c9fb97feb39e1bbed77a77f7e634fb2114a8e1035e0fc709635d63cf

data/CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+## 0.3.1 — 2026-10-24
+
+- First RubyGems release; README installs from RubyGems and lists the organic queue and TaskNotReady.
+
+## 0.3.0 — 2026-10-22
+
+- google_organic_task_post returns task-level rejections in tasks[] instead of raising
+  (accepted tasks are already billed) and yields each batch as it is posted.
+
+## 0.2.0 — 2026-10-22
+
+- Google organic Standard queue: google_organic_task_post (accepts 20100) and
+  google_organic_task_get (advanced, GET); TaskNotReady for 40601/40602.
+- HttpTransport sends GET when the payload is nil.
+
+## 0.1.1 — 2026-09-13
+
+- Non-object JSON error bodies are classified by HTTP status instead of raising TypeError/NoMethodError.
+
+## 0.1.0 — 2026-09-13
+
+- Client for DataForSEO Live endpoints with typed errors and slicing.
+- Typed methods: google_ads_search_volume, bulk_traffic_estimation, bulk_ranks,
+  bulk_spam_score, backlinks_summary.
+- DataForSeo::Testing::FakeTransport.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Anton Kopylov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,73 @@
+# dataforseo-ruby
+
+Unofficial, dependency-free Ruby client for [DataForSEO](https://dataforseo.com) Live endpoints
+and the Google organic Standard queue.
+
+```ruby
+gem "dataforseo-ruby"
+```
+
+```ruby
+client = DataForSeo::Client.new(login: ENV["DATAFORSEO_LOGIN"], password: ENV["DATAFORSEO_PASSWORD"])
+
+response = client.google_ads_search_volume(keywords: ["reverse image search"], language_code: "en")
+response.items  # => [{"keyword" => "reverse image search", "search_volume" => 1500000, ...}]
+response.cost   # => 0.09
+```
+
+`Client.new` also takes `open_timeout:` (10s), `read_timeout:` (120s) and `batch_size:` (1000).
+
+## Typed methods
+
+| Method | Endpoint | Rows |
+|---|---|---|
+| `google_ads_search_volume(keywords:, language_code:, location_code: nil, search_partners: false)` | keywords_data/google_ads/search_volume/live | `result[]` |
+| `bulk_traffic_estimation(targets:, location_name:, language_code:)` | dataforseo_labs/google/bulk_traffic_estimation/live | `result[].items[]` |
+| `bulk_ranks(targets:, rank_scale:)` | backlinks/bulk_ranks/live | `result[].items[]` |
+| `bulk_spam_score(targets:)` | backlinks/bulk_spam_score/live | `result[].items[]` |
+| `backlinks_summary(target:, include_subdomains: true)` | backlinks/summary/live | `result[]` |
+| `google_organic_task_post(tasks:)` | serp/google/organic/task_post | `tasks[]` (ids) |
+| `google_organic_task_get(id:)` | serp/google/organic/task_get/advanced/{id} | `result[].items[]` |
+
+Array methods send 1,000 inputs per request. Pass a block to receive each slice's
+`Response` (with `#requested` holding only that slice) before the next request is
+made, so paid results can be saved as they arrive. Rows come back as the API sent
+them; nulls stay `nil` and absent targets stay absent.
+
+`client.post(path, tasks)` reaches other **Live** endpoints. Of the task-based
+endpoints (`20100` task created, results by GET) only Google organic is typed.
+
+## Errors
+
+Every transport, protocol or API failure is a `DataForSeo::Error`:
+
+- `ConnectionError` — timeouts, DNS, TLS, resets, unparseable bodies
+- `ApiError` (`#status_code`, `#status_message`, `#path`, `#task_id`, `#http_status`)
+  - `AuthenticationError` (40100, HTTP 401)
+  - `PaymentRequired` (40200, 40210, HTTP 402)
+  - `RateLimitError` (40202, 40209, HTTP 429)
+  - `TransientError` (50301, 50303, 50401, HTTP 5xx)
+  - `TaskNotReady` (40601, 40602)
+
+The client does not retry or pace. Know before you call:
+
+- Google Ads Live endpoints allow 12 requests per minute per account.
+- Google Ads keywords: at most 80 characters and 10 words (raises `ArgumentError`);
+  some symbols and emoji are rejected by the API and fail the whole request;
+  similar keywords in one request may share a combined volume.
+- One malformed target voids a `bulk_spam_score` request (0 items, 0 cost).
+- A timeout after the API accepted a request can still be charged; retrying pays again.
+
+## Testing
+
+```ruby
+require "dataforseo/testing"
+
+transport = DataForSeo::Testing::FakeTransport.new(
+  DataForSeo::Testing.ok_body(result: [{ "keyword" => "x", "search_volume" => 10 }])
+)
+client = DataForSeo::Client.new(login: "x", password: "y", transport:)
+transport.calls # => [[path, payload], ...]
+```
+
+A queued `Exception` instance is raised instead of returned.

data/lib/dataforseo/client.rb

@@ -0,0 +1,126 @@
+# frozen_string_literal: true
+
+module DataForSeo
+  module GoogleAds
+    MAX_KEYWORD_LENGTH = 80
+    MAX_KEYWORD_WORDS = 10
+
+    def self.validate!(keywords)
+      bad = keywords.find { it.length > MAX_KEYWORD_LENGTH || it.split.size > MAX_KEYWORD_WORDS }
+      raise ArgumentError, "keyword exceeds Google Ads limits (80 chars, 10 words): #{bad.inspect}" if bad
+    end
+  end
+
+  class Client
+    OPEN_TIMEOUT = 10
+    # seo's bulk_traffic_estimation took 34.6s for 98 targets; a slice holds 1,000.
+    READ_TIMEOUT = 120
+    BATCH_SIZE = 1000
+    OK = 20000
+
+    RESULT = ->(task) { Array(task["result"]) }
+    ITEMS = ->(task) { Array(task["result"]).flat_map { Array(it["items"]) } }
+
+    def initialize(login:, password:, transport: nil, open_timeout: OPEN_TIMEOUT,
+                   read_timeout: READ_TIMEOUT, batch_size: BATCH_SIZE)
+      @transport = transport || HttpTransport.new(login:, password:, open_timeout:, read_timeout:)
+      @batch_size = batch_size
+    end
+
+    # A task_post's 20100 "created" raises unless the caller accepts it; accept: nil checks no task.
+    def post(path, tasks, requested: [], extract: RESULT, accept: [OK])
+      body = @transport.call(path, tasks)
+      check!(body, path)
+      response_tasks = Array(body["tasks"])
+      if response_tasks.size != tasks.size
+        raise ApiError.new("expected #{tasks.size} task(s), got #{response_tasks.size}", path:)
+      end
+      response_tasks.each { check!(it, path, task_id: it["id"], accept:) } if accept
+
+      Response.new(path:, cost: body["cost"], tasks: response_tasks, items: response_tasks.flat_map(&extract), requested:)
+    end
+
+    def get(path, extract: RESULT)
+      body = @transport.call(path, nil)
+      check!(body, path)
+      response_tasks = Array(body["tasks"])
+      response_tasks.each { check!(it, path, task_id: it["id"]) }
+
+      Response.new(path:, cost: body["cost"], tasks: response_tasks, items: response_tasks.flat_map(&extract), requested: [])
+    end
+
+    SEARCH_VOLUME_PATH = "/v3/keywords_data/google_ads/search_volume/live"
+    TRAFFIC_PATH = "/v3/dataforseo_labs/google/bulk_traffic_estimation/live"
+    RANKS_PATH = "/v3/backlinks/bulk_ranks/live"
+    SPAM_PATH = "/v3/backlinks/bulk_spam_score/live"
+    SUMMARY_PATH = "/v3/backlinks/summary/live"
+    ORGANIC_TASK_POST_PATH = "/v3/serp/google/organic/task_post"
+    ORGANIC_TASK_GET_PATH = "/v3/serp/google/organic/task_get/advanced"
+    TASK_CREATED = 20100
+    TASK_POST_BATCH = 100
+
+    # No location means worldwide. Google Ads Live allows 12 requests/minute per
+    # account; pacing is the caller's.
+    def google_ads_search_volume(keywords:, language_code:, location_code: nil, search_partners: false, &block)
+      GoogleAds.validate!(keywords)
+      sliced(SEARCH_VOLUME_PATH, keywords, RESULT, block) do |slice|
+        task = { "keywords" => slice, "language_code" => language_code, "search_partners" => search_partners }
+        task["location_code"] = location_code if location_code
+        task
+      end
+    end
+
+    def bulk_traffic_estimation(targets:, location_name:, language_code:, &block)
+      sliced(TRAFFIC_PATH, targets, ITEMS, block) do |slice|
+        { "targets" => slice, "location_name" => location_name, "language_code" => language_code }
+      end
+    end
+
+    # rank_scale has no default on purpose.
+    def bulk_ranks(targets:, rank_scale:, &block)
+      sliced(RANKS_PATH, targets, ITEMS, block) { |slice| { "targets" => slice, "rank_scale" => rank_scale } }
+    end
+
+    # Sends no location/language: the endpoint rejects them (40501). One malformed
+    # target voids the whole slice.
+    def bulk_spam_score(targets:, &block)
+      sliced(SPAM_PATH, targets, ITEMS, block) { |slice| { "targets" => slice } }
+    end
+
+    def backlinks_summary(target:, include_subdomains: true)
+      post(SUMMARY_PATH, [{ "target" => target, "include_subdomains" => include_subdomains }],
+           requested: [target], extract: RESULT)
+    end
+
+    # Standard queue: results arrive via google_organic_task_get, usually within minutes.
+    # Tasks are billed on acceptance, so a rejected one comes back in tasks[] rather than
+    # raising, and each batch is yielded before the next is sent.
+    def google_organic_task_post(tasks:, &block)
+      responses = tasks.each_slice(TASK_POST_BATCH).map do |slice|
+        post(ORGANIC_TASK_POST_PATH, slice, accept: nil).tap { block&.call(it) }
+      end
+      Response.combine(responses, path: ORGANIC_TASK_POST_PATH, requested: [])
+    end
+
+    # Raises TaskNotReady while the task is still queued.
+    def google_organic_task_get(id:)
+      get("#{ORGANIC_TASK_GET_PATH}/#{id}", extract: ITEMS)
+    end
+
+    private
+
+    def check!(envelope, path, task_id: nil, accept: [OK])
+      return if accept.include?(envelope["status_code"])
+
+      raise DataForSeo.api_error(status_code: envelope["status_code"], status_message: envelope["status_message"],
+                                 path:, task_id:)
+    end
+
+    def sliced(path, inputs, extract, block)
+      responses = inputs.each_slice(@batch_size).map do |slice|
+        post(path, [yield(slice)], requested: slice, extract:).tap { block&.call(it) }
+      end
+      Response.combine(responses, path:, requested: inputs)
+    end
+  end
+end

data/lib/dataforseo/errors.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module DataForSeo
+  # Every transport, protocol or API failure. A caller's retry_on names this
+  # (or a subclass) and nothing else, so nothing may escape as its own class.
+  class Error < StandardError; end
+
+  class ConnectionError < Error; end
+
+  class ApiError < Error
+    attr_reader :status_code, :status_message, :path, :task_id, :http_status
+
+    def initialize(message = nil, status_code: nil, status_message: nil, path: nil, task_id: nil, http_status: nil)
+      @status_code = status_code
+      @status_message = status_message
+      @path = path
+      @task_id = task_id
+      @http_status = http_status
+      head = ["DataForSEO", status_code || (http_status && "HTTP #{http_status}"), "on #{path}"].compact.join(" ")
+      super(message || (status_message ? "#{head}: #{status_message}" : head))
+    end
+  end
+
+  class AuthenticationError < ApiError; end
+  class PaymentRequired < ApiError; end
+  class RateLimitError < ApiError; end
+  class TransientError < ApiError; end
+  class TaskNotReady < ApiError; end
+
+  # https://docs.dataforseo.com/v3/appendix/errors/
+  API_ERRORS = {
+    40100 => AuthenticationError,
+    40200 => PaymentRequired, 40210 => PaymentRequired,
+    40202 => RateLimitError, 40209 => RateLimitError,
+    50301 => TransientError, 50303 => TransientError, 50401 => TransientError,
+    40601 => TaskNotReady, 40602 => TaskNotReady
+  }.freeze
+
+  HTTP_ERRORS = { 401 => AuthenticationError, 402 => PaymentRequired, 429 => RateLimitError }.freeze
+
+  def self.api_error(status_code:, status_message:, path:, task_id: nil)
+    API_ERRORS.fetch(status_code, ApiError)
+      .new(status_code:, status_message:, path:, task_id:)
+  end
+
+  def self.http_error(http_status:, path:, status_code: nil, status_message: nil)
+    klass = HTTP_ERRORS.fetch(http_status) { http_status >= 500 ? TransientError : ApiError }
+    klass.new(http_status:, path:, status_code:, status_message:)
+  end
+end

data/lib/dataforseo/http_transport.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+require "json"
+require "net/http"
+require "openssl"
+require "timeout"
+require "zlib"
+
+module DataForSeo
+  class HttpTransport
+    HOST = "api.dataforseo.com"
+
+    # Timeout::Error covers Net::Open/Read/WriteTimeout; Net::HTTP requests gzip
+    # and doesn't rescue inflate errors itself, hence Zlib::Error.
+    NETWORK_ERRORS = [
+      Timeout::Error, SocketError, SystemCallError, OpenSSL::SSL::SSLError, IOError, EOFError,
+      Net::ProtocolError, Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError, Zlib::Error
+    ].freeze
+
+    def initialize(login:, password:, open_timeout:, read_timeout:)
+      @login = login
+      @password = password
+      @open_timeout = open_timeout
+      @read_timeout = read_timeout
+    end
+
+    def call(path, payload)
+      request = payload.nil? ? Net::HTTP::Get.new(path) : Net::HTTP::Post.new(path, "Content-Type" => "application/json")
+      request.basic_auth(@login, @password)
+      request.body = JSON.generate(payload) unless payload.nil?
+
+      response = Net::HTTP.start(HOST, 443, use_ssl: true,
+                                 open_timeout: @open_timeout, read_timeout: @read_timeout) { it.request(request) }
+      parse(path, response)
+    rescue *NETWORK_ERRORS => e
+      raise ConnectionError, "#{e.class}: #{e.message}"
+    end
+
+    private
+
+    # HTTP status first: some failures arrive as a non-2xx with an HTML body,
+    # and parsing that first would report a credentials problem as a network one.
+    def parse(path, response)
+      status = response.code.to_i
+      body = begin
+        JSON.parse(response.body.to_s)
+      rescue JSON::ParserError
+        nil
+      end
+
+      if status.between?(200, 299)
+        return body if body.is_a?(Hash)
+        raise ConnectionError, "unparseable body from #{path} (HTTP #{status})"
+      end
+
+      # A non-2xx with a status_code we classify (e.g. 401/40100) still goes to the
+      # client, so it raises the same class as the same code would on a 2xx.
+      return body if body.is_a?(Hash) && API_ERRORS.key?(body["status_code"])
+
+      status_code, status_message = body.values_at("status_code", "status_message") if body.is_a?(Hash)
+      raise DataForSeo.http_error(http_status: status, path:, status_code:, status_message:)
+    end
+  end
+end

data/lib/dataforseo/response.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module DataForSeo
+  # requested is what was sent in THIS request, so a caller persisting one
+  # slice can tell "omitted from the answer" from "not asked yet".
+  class Response
+    attr_reader :path, :cost, :tasks, :items, :requested
+
+    def initialize(path:, cost:, tasks:, items:, requested:)
+      @path = path
+      @cost = cost.to_f
+      @tasks = tasks
+      @items = items
+      @requested = requested
+    end
+
+    def self.combine(responses, path:, requested:)
+      new(path:, requested:,
+          cost: responses.sum(0.0, &:cost),
+          tasks: responses.flat_map(&:tasks),
+          items: responses.flat_map(&:items))
+    end
+  end
+end

data/lib/dataforseo/testing.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module DataForSeo
+  # require "dataforseo/testing" in a test suite; never loaded by the gem itself.
+  module Testing
+    class FakeTransport
+      attr_reader :calls
+
+      def initialize(*bodies)
+        @bodies = bodies
+        @calls = []
+      end
+
+      def call(path, payload)
+        @calls << [path, payload]
+        raise "FakeTransport called more times than it has bodies" if @bodies.empty?
+
+        body = @bodies.shift
+        raise body if body.is_a?(Exception)
+
+        body
+      end
+    end
+
+    def self.ok_body(result:, cost: 0.0)
+      { "status_code" => 20000, "status_message" => "Ok.", "cost" => cost,
+        "tasks" => [{ "id" => "fake-task", "status_code" => 20000, "status_message" => "Ok.",
+                      "cost" => cost, "result" => result }] }
+    end
+
+    def self.error_body(status_code:, status_message: "Error", on: :envelope)
+      if on == :task
+        { "status_code" => 20000, "status_message" => "Ok.", "cost" => 0.0,
+          "tasks" => [{ "id" => "fake-task", "status_code" => status_code, "status_message" => status_message }] }
+      else
+        { "status_code" => status_code, "status_message" => status_message, "tasks" => [] }
+      end
+    end
+  end
+end

data/lib/dataforseo/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module DataForSeo
+  VERSION = "0.3.1"
+end

data/lib/dataforseo-ruby.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require_relative "dataforseo"

data/lib/dataforseo.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+require_relative "dataforseo/version"
+require_relative "dataforseo/errors"
+require_relative "dataforseo/response"
+require_relative "dataforseo/http_transport"
+require_relative "dataforseo/client"

metadata

@@ -0,0 +1,55 @@
+--- !ruby/object:Gem::Specification
+name: dataforseo-ruby
+version: !ruby/object:Gem::Version
+  version: 0.3.1
+platform: ruby
+authors:
+- Anton Kopylov
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies: []
+description: 'A dependency-free Ruby client for DataForSEO Live endpoints: envelope
+  checking, typed errors, batching and a fake transport for tests.'
+email:
+- anton@tonic20.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- lib/dataforseo-ruby.rb
+- lib/dataforseo.rb
+- lib/dataforseo/client.rb
+- lib/dataforseo/errors.rb
+- lib/dataforseo/http_transport.rb
+- lib/dataforseo/response.rb
+- lib/dataforseo/testing.rb
+- lib/dataforseo/version.rb
+homepage: https://github.com/tonic20/dataforseo-ruby
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/tonic20/dataforseo-ruby
+  source_code_uri: https://github.com/tonic20/dataforseo-ruby
+  changelog_uri: https://github.com/tonic20/dataforseo-ruby/blob/main/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.4'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.9
+specification_version: 4
+summary: Unofficial Ruby client for the DataForSEO API
+test_files: []