ghostcrawl 2.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b334d27abef5ab1cffd761032b5dbb056e1c28351bfc7fc12000d679f6b64a25
+  data.tar.gz: 357cca4fb48f3d8b0cd65b331ddafb0470f043db1f1b0218f2736f80fc355a51
+SHA512:
+  metadata.gz: b6cdb4008a92220a153933750191377e527670f327d870f6650b3e1177fc53a293f0dd494d9da8a8d2328a589ca537fc91acddf07afeb0a976902502e1096b27
+  data.tar.gz: 33426ca08edc9eb51242a7be7d9114fc1c2d46743740d321f46c5c49d2e49d04f2491c68ed109688f2645a70c455c343aeecba44f62a199366e1703bbd4b810e

data/LICENSE

@@ -0,0 +1,46 @@
+GhostCrawl Software License
+
+Copyright (c) 2026 GhostCrawl, LLC. All rights reserved.
+
+This software — including the GhostCrawl SDKs, command-line interface, example
+code, documentation, and the GhostCrawl self-host image (collectively, the
+"Software") — is the proprietary property of GhostCrawl, LLC ("GhostCrawl") and
+its licensors. It is licensed, not sold.
+
+1. LICENSE GRANT. Subject to your continuous compliance with this License and the
+   GhostCrawl Terms of Service (https://docs.ghostcrawl.io/legal/terms),
+   GhostCrawl grants you a limited, revocable, non-exclusive, non-transferable,
+   and non-sublicensable license to install and use the Software, including
+   within your own applications, solely to access and use the GhostCrawl service
+   under a valid GhostCrawl account.
+
+2. RESTRICTIONS. Except as expressly permitted in Section 1, you may not, in
+   whole or in part: (a) copy the Software except as strictly necessary to use it
+   as permitted; (b) modify, adapt, translate, or create derivative works of the
+   Software; (c) distribute, publish, sublicense, sell, resell, rent, lease,
+   lend, or otherwise transfer the Software, or make it available to any third
+   party, including as a hosted or managed service; (d) reverse engineer,
+   decompile, disassemble, or otherwise attempt to derive the source code,
+   structure, or internal design of any compiled component of the Software;
+   (e) circumvent, disable, or tamper with any license, authentication,
+   usage-metering, watermarking, or security mechanism; or (f) remove, alter, or
+   obscure any proprietary, copyright, or attribution notice.
+
+3. OWNERSHIP. GhostCrawl and its licensors retain all right, title, and interest
+   in and to the Software and all intellectual property rights therein. No rights
+   are granted to you except as expressly set out in this License. All rights
+   reserved.
+
+4. TERMINATION. This License terminates automatically, without notice, if you
+   breach any of its terms or cease to maintain a valid GhostCrawl account where
+   one is required. Upon termination you must cease all use of the Software and
+   destroy all copies in your possession or control.
+
+5. NO WARRANTY; LIMITATION OF LIABILITY. THE SOFTWARE IS PROVIDED "AS IS" WITHOUT
+   WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. TO THE MAXIMUM EXTENT PERMITTED BY
+   LAW, GHOSTCRAWL AND ITS LICENSORS WILL NOT BE LIABLE FOR ANY CLAIM, DAMAGES, OR
+   OTHER LIABILITY ARISING FROM OR IN CONNECTION WITH THE SOFTWARE OR ITS USE. The
+   GhostCrawl Terms of Service and Acceptable Use Policy
+   (https://docs.ghostcrawl.io/legal/terms) are incorporated by reference.
+
+For licensing inquiries, contact legal@ghostcrawl.io.

data/README.md

@@ -0,0 +1,127 @@
+# ghostcrawl — Ruby SDK
+
+The official Ruby client for the [GhostCrawl](https://ghostcrawl.io) API.
+Collect web data at scale — scrape, crawl, search, and extract structured data.
+
+## Install
+
+```bash
+gem install ghostcrawl
+```
+
+Or in your `Gemfile`:
+
+```ruby
+gem "ghostcrawl", "~> 2.1"
+```
+
+Requires Ruby 3.0+. No runtime dependencies — uses stdlib `net/http` only.
+
+## Quickstart
+
+```ruby
+require "ghostcrawl"
+
+# Token from constructor or GHOSTCRAWL_API_KEY env var
+client = Ghostcrawl::Client.new(token: "gck_live_YOUR_KEY")
+
+# Scrape a URL
+result = client.scrape(url: "https://example.com", format: "markdown")
+puts result["content"]
+
+# Start a crawl run
+run = client.crawl_runs.start(url: "https://example.com", max_depth: 2, max_pages: 50)
+puts "run_id: #{run["run_id"]}"
+
+# Web search
+results = client.search(query: "latest AI research", engine: "google", limit: 10)
+puts results["results"].length
+```
+
+## Authentication
+
+```ruby
+# Option 1: pass token directly
+client = Ghostcrawl::Client.new(token: "gck_live_YOUR_KEY")
+
+# Option 2: set environment variable (recommended for production)
+# export GHOSTCRAWL_API_KEY=gck_live_YOUR_KEY
+client = Ghostcrawl::Client.new
+```
+
+Every request sends `Authorization: Bearer <token>`. This is the only auth scheme the API accepts.
+
+## Extract structured data
+
+```ruby
+require "ghostcrawl"
+
+client = Ghostcrawl::Client.new(token: "gck_live_YOUR_KEY")
+
+data = client.extract(
+  url: "https://example.com/product",
+  schema: {
+    type: "object",
+    properties: {
+      name:        { type: "string" },
+      price:       { type: "number" },
+      description: { type: "string" }
+    }
+  }
+)
+puts "#{data["name"]} — $#{data["price"]}"
+```
+
+## Error handling
+
+```ruby
+require "ghostcrawl"
+
+client = Ghostcrawl::Client.new(token: "gck_live_YOUR_KEY")
+
+begin
+  result = client.scrape(url: "https://example.com")
+rescue Ghostcrawl::AuthenticationError
+  puts "Invalid API key — check your token"
+rescue Ghostcrawl::PaymentRequiredError
+  puts "Usage limit reached — upgrade your plan"
+rescue Ghostcrawl::RateLimitError
+  puts "Rate limit reached — retry after a short delay"
+rescue Ghostcrawl::APIError => e
+  puts "Server error: #{e.status_code}"
+rescue Ghostcrawl::GhostcrawlError => e
+  puts "API error: #{e.message} (#{e.status_code})"
+end
+```
+
+## Self-hosted
+
+```ruby
+client = Ghostcrawl::Client.new(
+  token: "gck_live_YOUR_KEY",
+  base_url: "http://localhost:8080"
+)
+```
+
+## All resources
+
+| Resource | Method / accessor | Key operations |
+|----------|------------------|----------------|
+| Scraping | `client.scrape(url:)` | Render and return page content |
+| Web search | `client.search(query:)` | Search Google, Bing, DuckDuckGo |
+| Data extraction | `client.extract(url:, schema:)` | Structured JSON from any page |
+| Deep crawl | `client.crawl(url:)` | Crawl a site depth-first |
+| URL map | `client.map(url:)` | Discover all reachable URLs |
+| Account | `client.me` | Get account info and usage |
+| Crawl runs | `client.crawl_runs` | start, list, get\_run, cancel |
+| Sessions | `client.sessions` | create, list, extend, release |
+| Profiles | `client.profiles` | list, get\_profile, create, update, delete |
+| Webhooks | `client.webhooks` | list, get\_webhook, create, delete, rotate\_secret |
+| Schedules | `client.schedules` | list, get\_schedule, create, delete |
+| Datasets | `client.datasets` | list, get\_dataset, create, delete, rows, append |
+| Recordings | `client.recordings` | list, get\_recording, delete |
+| Key-Value Store | `client.kv` | get, set, delete |
+
+## License
+
+Proprietary — GhostCrawl Software License. See [LICENSE](LICENSE).

data/_generated/ghostcrawl_client.rb

@@ -0,0 +1,27 @@
+require 'microsoft_kiota_abstractions'
+require 'microsoft_kiota_serialization_json'
+require_relative './ghostcrawl'
+require_relative './v1/v1_request_builder'
+
+module Ghostcrawl
+    ## 
+    # The main entry point of the SDK, exposes the configuration and the fluent API.
+    class GhostcrawlClient < MicrosoftKiotaAbstractions::BaseRequestBuilder
+        
+        ## 
+        # The v1 property
+        def v1()
+            return Ghostcrawl::V1::V1RequestBuilder.new(@path_parameters, @request_adapter)
+        end
+        ## 
+        ## Instantiates a new GhostcrawlClient and sets the default values.
+        ## @param request_adapter The request adapter to use to execute the requests.
+        ## @return a void
+        ## 
+        def initialize(request_adapter)
+            super(Hash.new, request_adapter, "{+baseurl}")
+            MicrosoftKiotaAbstractions::ApiClientBuilder.register_default_serializer(MicrosoftKiotaSerializationJson::JsonSerializationWriterFactory)
+            MicrosoftKiotaAbstractions::ApiClientBuilder.register_default_deserializer(MicrosoftKiotaSerializationJson::JsonParseNodeFactory)
+        end
+    end
+end