crawlora 1.5.0.pre.sdk.2

data/docs/recipes.md

@@ -0,0 +1,87 @@
+# Crawlora Ruby SDK recipes
+
+Common patterns beyond the README. See [`operations.md`](operations.md) for the
+full list of operations.
+
+## Authentication
+
+```ruby
+# API key (most endpoints):
+Crawlora.client(api_key: "live_…")
+
+# JWT (dashboard/user endpoints). A raw token is sent as "Token <jwt>";
+# pass "Bearer <jwt>" yourself to override the scheme.
+Crawlora.client(jwt_token: "eyJ…")
+```
+
+Both fall back to environment variables: `CRAWLORA_API_KEY` and
+`CRAWLORA_BASE_URL`.
+
+## Retries and Retry-After
+
+```ruby
+Crawlora.client(
+  retries: 3,
+  retry_delay: 0.5,                 # exponential backoff with jitter
+  max_retry_delay: 10,
+  retry_statuses: [429, 503],       # override the default retryable set
+  on_retry: ->(attempt, error, delay) { warn "retry #{attempt} after #{delay}s (#{error.status})" }
+)
+```
+
+A custom predicate wins over the status set:
+
+```ruby
+Crawlora.client(retries: 2, retry_predicate: ->(status, _error) { status == 429 })
+```
+
+`Retry-After` (seconds or HTTP-date) is always honored, capped at
+`max_retry_delay`.
+
+## Hooks
+
+```ruby
+client = Crawlora.client(
+  before_request: ->(ctx) { ctx[:headers]["X-Trace-Id"] = SecureRandom.uuid },
+  after_response: ->(operation_id, status, _headers, body) {
+    body.is_a?(Hash) ? body.merge("_op" => operation_id, "_status" => status) : body
+  }
+)
+```
+
+`before_request` receives a mutable context (`:operation`, `:method`, `:url`,
+`:headers`); editing `:url`/`:headers` rewrites the outgoing request.
+`after_response` may return a replacement body (return `nil` to keep it).
+
+## Rate limiting and concurrency
+
+```ruby
+client = Crawlora.client(rate_limit: 10, max_concurrency: 4)
+
+threads = queries.map do |q|
+  Thread.new { client.bing.search(q: q) } # throttled to 10 rps / 4 in-flight
+end
+threads.each(&:join)
+```
+
+## Response modes
+
+```ruby
+client.request("youtube-transcript", { id: "abc" }, response_type: "text")   # String
+io = client.request("bing-search", { q: "x" }, response_type: "stream")       # StringIO
+io.read
+```
+
+`auto` (default) parses JSON when the response is JSON and returns text
+otherwise.
+
+## Custom transport (testing)
+
+Inject any object responding to
+`call(method:, url:, headers:, body:, timeout:)` and returning a
+`Crawlora::Response`:
+
+```ruby
+fake = ->(**) { Crawlora::Response.new(200, { "content-type" => "application/json" }, '{"data":[]}') }
+client = Crawlora.client(transport: fake)
+```

data/examples/bing_search.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+# Basic search call. Run with:
+#   CRAWLORA_API_KEY=... ruby examples/bing_search.rb
+require "crawlora"
+
+Crawlora.client do |client|
+  result = client.bing.search(q: "web scraping")
+  result["data"].each { |item| puts item["title"] || item.inspect }
+end

data/examples/paginate.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+# Iterate pages until the API returns an empty page. Run with:
+#   CRAWLORA_API_KEY=... ruby examples/paginate.rb
+require "crawlora"
+
+Crawlora.client do |client|
+  client.paginate_items("airbnb-room-reviews", { id: "YOUR_ROOM_ID" }, max_pages: 3).each do |review|
+    puts review.inspect
+  end
+end

data/examples/youtube_transcript.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+# Fetch a transcript in plain-text response mode. Run with:
+#   CRAWLORA_API_KEY=... ruby examples/youtube_transcript.rb
+require "crawlora"
+
+Crawlora.client do |client|
+  transcript = client.request("youtube-transcript", { id: "dQw4w9WgXcQ" }, response_type: "text")
+  puts transcript
+end