enconvert 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 803e8f4b068717fa3813eb4266afb426ac0d70da63378556a69f231a52c02bb0
-  data.tar.gz: 261b1c69bd73b44f0be655a70820549108f784bafd2e9f0c21f74f0e76cf265a
+  metadata.gz: 7245afabefb24e84a42769dc04a0dfe1693db85f6a98576dcd1d68a6c3fe4722
+  data.tar.gz: 5a5aecb734881a6e44680d749efc8813894bc9e75acf26871c6992a35fdaa9ef
 SHA512:
-  metadata.gz: 8a94b8bfdd0429848d27857da9e74a585ac553589843ba5f183b0f53847b26e2fe33a08b42cd46837dedf74fa61668776f6e619713a1d70440fbb84eb5703d8e
-  data.tar.gz: db4826110029dbe06a0ccd1bcd769d944bb286c56f61644a569c93a54fe9199bd221b0987a800cb6e53b79d2b0703e5a629255a4278e3d1a67b9e6fc753c0e1b
+  metadata.gz: '008838d207976db15e7bdf12df5fde2b3820ac20141421ed2d880d80d9808891005fc7f5e7e85af8f1fd9861bfd59ca28c82c0f3f1df7bf9c0394b6835c617fa'
+  data.tar.gz: c220ba5c98cc41e920aff61f59310ab3c846f831bdf679bf6ba182150780b120906b22b2af5ea147932a8ecda9a6e404a74a7999127649616415742abce58064

data/README.md

@@ -1,308 +1,316 @@
-# Enconvert Ruby SDK
-
-Honest eyes for your AI agent — the Ruby SDK for [Enconvert](https://enconvert.com). Ruby 3.0+.
-
-Read any web page or file into clean Markdown, JSON, or screenshots, and get a `render_quality` score (0.0–1.0) on **every** read — so a blocked, challenge, or empty-SPA page comes back flagged with a low score and warnings, never mistaken for real content. Perceive, discover, look up, distill, ingest, and watch the web; convert 40+ file and document formats through the same key.
-
-> Wiring an agent (Claude, Cursor, Windsurf, n8n, …)? The [MCP server](https://enconvert.com/mcp) is the native path — `npx @enconvert/mcp setup`. This SDK is the programmatic REST path for everything else.
-
-## Install
-
-```bash
-gem install enconvert
-```
-
-Or add to your Gemfile:
-
-```ruby
-gem "enconvert"
-```
-
-## Quick Start
-
-```ruby
-require "enconvert"
-
-client = Enconvert::Client.new(api_key: "sk_...")
-
-# Read a page the way your agent should — with a quality score attached.
-op = client.v2.perceive("https://example.com", outputs: %w[markdown structured])
-puts op.outputs["markdown"].url, op.render_quality # e.g. 0.93
-```
-
----
-
-# V2 — agent-ready data (`client.v2`)
-
-The V2 namespace turns web pages into agent-ready data: render, search, extract, ingest, and monitor. All V2 endpoints require a **private API key** and are plan-gated — a disabled feature or exhausted monthly quota raises `Enconvert::QuotaError` (HTTP 402).
-
-Every render carries `render_quality` (0.0–1.0). A low score means the page didn't render cleanly (challenge page, cookie wall, empty shell); the content is still returned, flagged, so a bad read never quietly enters your agent's context.
-
-### Perceive — render a URL into artifacts
-
-```ruby
-op = client.v2.perceive(
-  "https://example.com",
-  outputs: %w[markdown screenshot structured],
-  extract: %w[tables metadata]
-)
-puts op.render_quality          # honesty score, 0.0–1.0
-puts op.outputs["markdown"].url # 15-min signed URL
-puts op.structured
-
-# Re-sign artifact URLs later:
-again = client.v2.get_perceive_operation(op.operation_id)
-
-# Batch (<=1000 URLs; small batches run inline, larger return "queued" — poll):
-batch = client.v2.perceive_batch(
-  ["https://a.com", "https://b.com"],
-  outputs: ["markdown"],
-  output_mode: "zip"
-)
-done = client.v2.get_perceive_batch(batch.job_id)
-```
-
-### Discover — enumerate a site's URLs (no rendering)
-
-```ruby
-found = client.v2.discover(
-  "https://example.com",
-  mode: "hybrid",              # "sitemap" | "crawl" | "hybrid"
-  max_urls: 200,
-  exclude_patterns: ["/tag/"]
-)
-puts found.total, found.urls
-```
-
-### Lookup — web search with optional auto-perceive
-
-```ruby
-search = client.v2.lookup(
-  "best static site generators",
-  category: "web",             # web | news | images | scholar | patents | maps
-  num_results: 10,
-  perceive_top: 3               # auto-render top 3 results (uses perceive quota)
-)
-search.results.each { |hit| puts hit.title, hit.url, hit.perceive&.render_quality }
-```
-
-### Distill — schema-driven structured extraction
-
-```ruby
-extraction = client.v2.distill(
-  urls: ["https://example.com/pricing"],
-  schema: { plans: "list of plan names with monthly prices" },
-  css_schema: {                 # optional free CSS pass before the LLM tier
-    base_selector: ".plan-card",
-    fields: [
-      { name: "name", type: "text", selector: "h3" },
-      { name: "price", type: "text", selector: ".price" }
-    ]
-  }
-)
-puts extraction.results.first.data, extraction.results.first.extraction_tier
-
-# Or discover-then-distill:
-client.v2.distill(
-  discover_from: { url: "https://example.com", mode: "sitemap", max_pages: 10 },
-  schema: { title: "page title", summary: "one-line summary" }
-)
-```
-
-### Ingest — site or files to RAG-ready JSONL (always async)
-
-Turn a whole site — or a set of uploaded documents — into chunked, RAG-ready JSONL through one pipeline.
-
-```ruby
-# From a site:
-job = client.v2.ingest(
-  mode: "sitemap",
-  url: "https://docs.example.com",
-  max_pages: 100,
-  chunk: { max_words: 512, sentence_overlap: 1 },
-  webhook_url: "https://my.app/hooks/enconvert"
-)
-
-# Or from uploaded files (PDF, DOCX, PPTX, XLSX, CSV, HTML, EPUB, TXT/MD, legacy/ODF office):
-file_job = client.v2.ingest_files(
-  ["handbook.pdf", "notes.docx"],
-  chunk: { max_words: 512, sentence_overlap: 1 }
-)
-
-status = client.v2.get_ingest_job(job.job_id)          # poll
-puts status.output_url if status.status == "completed" # JSONL
-
-client.v2.list_ingest_jobs(limit: 20)
-client.v2.cancel_ingest_job(job.job_id)                 # idempotent
-
-# Webhook signing (HMAC):
-secret = client.v2.get_webhook_secret
-puts secret.secret, secret.signature_header
-client.v2.rotate_webhook_secret                          # invalidates old secret
-client.v2.retry_ingest_webhook(job.job_id)                # re-deliver
-```
-
-### Watch — recurring change monitoring
-
-```ruby
-watcher = client.v2.create_watcher(
-  "https://example.com/pricing",
-  frequency_minutes: 60,        # hourly floor
-  diff_mode: "auto",            # auto | text | structured | tables | metadata
-  webhook_url: "https://my.app/hooks/changes",
-  notify_email: true
-)
-
-client.v2.list_watchers
-client.v2.get_watcher(watcher.watcher_id)
-client.v2.get_watcher_snapshots(watcher.watcher_id, limit: 10)
-client.v2.update_watcher(watcher.watcher_id, status: "paused")
-client.v2.update_watcher(watcher.watcher_id, webhook_url: "") # clears webhook
-client.v2.delete_watcher(watcher.watcher_id)                  # soft-delete, idempotent
-```
-
-### V2 error handling
-
-```ruby
-begin
-  client.v2.ingest(mode: "sitemap", url: "https://example.com")
-rescue Enconvert::QuotaError
-  warn "Upgrade plan or wait for quota reset"
-end
-```
-
----
-
-# File conversion
-
-The same key also converts 40+ formats. Two "anything → X" endpoints auto-detect the input; the format-specific endpoints below give you a validated, typed path.
-
-### Anything to Markdown / PDF
-
-```ruby
-# Any document → clean Markdown (a RAG-ingestion building block):
-client.convert_to_markdown("report.docx", save_to: "report.md")
-# PDF, DOCX, PPTX, XLSX, CSV, HTML, EPUB, TXT/MD, and legacy/ODF office. (Images not supported.)
-
-# Almost anything → PDF:
-client.convert_to_pdf("slides.pptx", save_to: "slides.pdf")
-# office/ODF/Pages/Numbers/RTF/CSV, HTML, Markdown, text, images, SVG, EPUB, or a PDF passthrough.
-# Only pdf_options[:grayscale] is honored on this endpoint:
-client.convert_to_pdf("scan.pdf", pdf_options: { grayscale: true }, save_to: "gray.pdf")
-```
-
-### Image conversion
-
-```ruby
-result = client.convert_image("photo.heic", output_format: "webp", save_to: "photo.webp")
-```
-
-Any pair among `jpeg`, `png`, `svg`, `heic`, `webp` — plus PDF rasterization (`pdf` → `jpeg`). Unsupported pairs raise before any request is made:
-
-```ruby
-Enconvert.valid_outputs_for("json")  # => ["csv", "toml", "xml", "yaml"]
-Enconvert.valid_outputs_for("pdf")   # => ["jpeg"]
-```
-
-### Document & data conversion
-
-```ruby
-client.convert_document("report.docx", save_to: "report.pdf")
-client.convert_document("data.json", output_format: "yaml", save_to: "data.yaml")
-client.convert_document("notes.md", output_format: "html", save_to: "notes.html")
-```
-
-Supported inputs: `doc`/`docx`, `xls`/`xlsx`, `ppt`/`pptx`, `odt`, `ods`, `odp`, `ots`, `pages`, `numbers`, `html`, `markdown`, `csv`, `json`, `xml`, `yaml`, `toml`. (EPUB → use `convert_to_pdf` / `convert_to_markdown`.)
-
-The SDK validates every `{input}-to-{output}` pair against the conversions the API actually implements and raises immediately — with the list of valid outputs for that input — instead of sending a doomed request.
-
-### Supported conversions
-
-| Input | Outputs |
-|-------|---------|
-| json | csv, toml, xml, yaml |
-| xml | csv, json |
-| yaml | json |
-| csv | json, xml |
-| toml | json |
-| markdown | html, pdf |
-| html | pdf |
-| doc, excel, ppt, odt, ods, odp, ots, pages, numbers | pdf |
-| jpeg, png, svg, heic, webp | each other (all 20 pairs) |
-| pdf | jpeg |
-
-### URL to PDF / Screenshot / Markdown
-
-```ruby
-client.convert_url_to_pdf("https://example.com", save_to: "page.pdf")
-client.convert_url_to_screenshot("https://example.com", viewport_width: 1440, save_to: "shot.png")
-client.convert_url_to_markdown("https://example.com/article", save_to: "article.md")
-```
-
-### Website to PDF / Screenshot (whole-site batch)
-
-Discover every page of a website (sitemap, or full crawl on higher plans), convert each in the background, and receive a single ZIP. Requires a private API key with crawl access.
-
-```ruby
-batch = client.convert_website_to_pdf("https://example.com", crawl_mode: "sitemap")
-status = client.wait_for_batch(batch.batch_id, save_to: "site.zip")
-puts "#{status.completed} of #{status.total} pages converted"
-```
-
-`convert_website_to_screenshot` works the same way and produces a ZIP of PNGs.
-
-### PDF options & authenticated pages
-
-```ruby
-client.convert_url_to_pdf(
-  "https://internal.example.com/report",
-  pdf_options: { page_size: "A4", orientation: "landscape", margins: { top: 10, bottom: 10 } },
-  auth: { username: "user", password: "pass" },     # or cookies / headers, plan-gated
-  save_to: "report.pdf"
-)
-```
-
-Do not combine `auth` with an `Authorization` header — the API rejects the conflict.
-
-### Job status (async polling)
-
-```ruby
-status = client.get_job_status("job_abc123")
-puts status.presigned_url if status.status == "success"
-```
-
----
-
-## Error Handling
-
-```ruby
-begin
-  client.v2.perceive("https://example.com")
-rescue Enconvert::AuthenticationError
-  warn "Invalid API key"
-rescue Enconvert::QuotaError
-  warn "Plan feature off or quota exhausted"
-rescue Enconvert::RateLimitError
-  warn "Too many requests — slow down"
-rescue Enconvert::APIError => e
-  warn "API error [#{e.status_code}]: #{e.message}"
-end
-```
-
-## Configuration
-
-```ruby
-client = Enconvert::Client.new(
-  api_key: "sk_...",
-  timeout: 300,     # seconds, default
-  base_url: "https://api.enconvert.com" # override, default shown
-)
-```
-
-## Get an API Key
-
-Sign up at [enconvert.com](https://enconvert.com). Free tier: 100 ops/month, no credit card.
-
-## License
-
-MIT
+# Enconvert Ruby SDK
+
+Honest eyes for your AI agent — the Ruby SDK for [Enconvert](https://enconvert.com). Ruby 3.0+.
+
+Read any web page or file into clean Markdown, JSON, or screenshots, and get a `render_quality` score (0.0–1.0) on **every** read — so a blocked, challenge, or empty-SPA page comes back flagged with a low score and warnings, never mistaken for real content. Perceive, discover, look up, distill, ingest, and watch the web; convert 40+ file and document formats through the same key.
+
+> Wiring an agent (Claude, Cursor, Windsurf, n8n, …)? The [MCP server](https://enconvert.com/mcp) is the native path — `npx @enconvert/mcp setup`. This SDK is the programmatic REST path for everything else.
+
+## Install
+
+```bash
+gem install enconvert
+```
+
+Or add to your Gemfile:
+
+```ruby
+gem "enconvert"
+```
+
+## Quick Start
+
+```ruby
+require "enconvert"
+
+client = Enconvert::Client.new(api_key: "sk_...")
+
+# Read a page the way your agent should — with a quality score attached.
+op = client.v2.perceive("https://example.com", outputs: %w[markdown structured])
+puts op.outputs["markdown"].url, op.render_quality # e.g. 0.93
+```
+
+---
+
+# V2 — agent-ready data (`client.v2`)
+
+The V2 namespace turns web pages into agent-ready data: render, search, extract, ingest, and monitor. All V2 endpoints require a **private API key** and are plan-gated — a disabled feature or exhausted monthly quota raises `Enconvert::QuotaError` (HTTP 402).
+
+Every render carries `render_quality` (0.0–1.0). A low score means the page didn't render cleanly (challenge page, cookie wall, empty shell); the content is still returned, flagged, so a bad read never quietly enters your agent's context.
+
+### Perceive — render a URL into artifacts
+
+```ruby
+op = client.v2.perceive(
+  "https://example.com",
+  outputs: %w[markdown screenshot structured],
+  extract: %w[tables metadata]
+)
+puts op.render_quality          # honesty score, 0.0–1.0
+puts op.outputs["markdown"].url # 15-min signed URL
+puts op.structured
+
+# Re-sign artifact URLs later:
+again = client.v2.get_perceive_operation(op.operation_id)
+
+# Batch (<=1000 URLs; small batches run inline, larger return "queued" — poll):
+batch = client.v2.perceive_batch(
+  ["https://a.com", "https://b.com"],
+  outputs: ["markdown"],
+  output_mode: "zip"
+)
+done = client.v2.get_perceive_batch(batch.job_id)
+
+# Direct download: the response body IS the artifact bytes (no signed URL).
+# Requires exactly one artifact-producing output; metadata rides in headers.
+direct = client.v2.perceive_direct("https://example.com", outputs: ["markdown"])
+puts direct.filename, direct.content_type, direct.content.bytesize
+
+# Re-download a stored artifact from an earlier operation:
+raw = client.v2.download_perceive_artifact(op.operation_id, output: "markdown")
+```
+
+### Discover — enumerate a site's URLs (no rendering)
+
+```ruby
+found = client.v2.discover(
+  "https://example.com",
+  mode: "hybrid",              # "sitemap" | "crawl" | "hybrid"
+  max_urls: 200,
+  exclude_patterns: ["/tag/"]
+)
+puts found.total, found.urls
+```
+
+### Lookup — web search with optional auto-perceive
+
+```ruby
+search = client.v2.lookup(
+  "best static site generators",
+  category: "web",             # web | news | images | scholar | patents | maps
+  num_results: 10,
+  perceive_top: 3               # auto-render top 3 results (uses perceive quota)
+)
+search.results.each { |hit| puts hit.title, hit.url, hit.perceive&.render_quality }
+```
+
+### Distill — schema-driven structured extraction
+
+```ruby
+extraction = client.v2.distill(
+  urls: ["https://example.com/pricing"],
+  schema: { plans: "list of plan names with monthly prices" },
+  css_schema: {                 # optional free CSS pass before the LLM tier
+    base_selector: ".plan-card",
+    fields: [
+      { name: "name", type: "text", selector: "h3" },
+      { name: "price", type: "text", selector: ".price" }
+    ]
+  }
+)
+puts extraction.results.first.data, extraction.results.first.extraction_tier
+
+# Or discover-then-distill:
+client.v2.distill(
+  discover_from: { url: "https://example.com", mode: "sitemap", max_pages: 10 },
+  schema: { title: "page title", summary: "one-line summary" }
+)
+```
+
+### Ingest — site or files to RAG-ready JSONL (always async)
+
+Turn a whole site — or a set of uploaded documents — into chunked, RAG-ready JSONL through one pipeline.
+
+```ruby
+# From a site:
+job = client.v2.ingest(
+  mode: "sitemap",
+  url: "https://docs.example.com",
+  max_pages: 100,
+  chunk: { max_words: 512, sentence_overlap: 1 },
+  webhook_url: "https://my.app/hooks/enconvert"
+)
+
+# Or from uploaded files (PDF, DOCX, PPTX, XLSX, CSV, HTML, EPUB, TXT/MD, legacy/ODF office):
+file_job = client.v2.ingest_files(
+  ["handbook.pdf", "notes.docx"],
+  chunk: { max_words: 512, sentence_overlap: 1 }
+)
+
+status = client.v2.get_ingest_job(job.job_id)          # poll
+puts status.output_url if status.status == "completed" # JSONL
+
+client.v2.list_ingest_jobs(limit: 20)
+client.v2.cancel_ingest_job(job.job_id)                 # idempotent
+
+# Webhook signing (HMAC):
+secret = client.v2.get_webhook_secret
+puts secret.secret, secret.signature_header
+client.v2.rotate_webhook_secret                          # invalidates old secret
+client.v2.retry_ingest_webhook(job.job_id)                # re-deliver
+```
+
+### Watch — recurring change monitoring
+
+```ruby
+watcher = client.v2.create_watcher(
+  "https://example.com/pricing",
+  frequency_minutes: 60,        # hourly floor
+  diff_mode: "auto",            # auto | text | structured | tables | metadata
+  webhook_url: "https://my.app/hooks/changes",
+  notify_email: true
+)
+
+client.v2.list_watchers
+client.v2.get_watcher(watcher.watcher_id)
+client.v2.get_watcher_snapshots(watcher.watcher_id, limit: 10)
+client.v2.update_watcher(watcher.watcher_id, status: "paused")
+client.v2.update_watcher(watcher.watcher_id, webhook_url: "") # clears webhook
+client.v2.delete_watcher(watcher.watcher_id)                  # soft-delete, idempotent
+```
+
+### V2 error handling
+
+```ruby
+begin
+  client.v2.ingest(mode: "sitemap", url: "https://example.com")
+rescue Enconvert::QuotaError
+  warn "Upgrade plan or wait for quota reset"
+end
+```
+
+---
+
+# File conversion
+
+The same key also converts 40+ formats. Two "anything → X" endpoints auto-detect the input; the format-specific endpoints below give you a validated, typed path.
+
+### Anything to Markdown / PDF
+
+```ruby
+# Any document → clean Markdown (a RAG-ingestion building block):
+client.convert_to_markdown("report.docx", save_to: "report.md")
+# PDF, DOCX, PPTX, XLSX, CSV, HTML, EPUB, TXT/MD, and legacy/ODF office. (Images not supported.)
+
+# Almost anything → PDF:
+client.convert_to_pdf("slides.pptx", save_to: "slides.pdf")
+# office/ODF/Pages/Numbers/RTF/CSV, HTML, Markdown, text, images, SVG, EPUB, or a PDF passthrough.
+# Only pdf_options[:grayscale] is honored on this endpoint:
+client.convert_to_pdf("scan.pdf", pdf_options: { grayscale: true }, save_to: "gray.pdf")
+```
+
+### Image conversion
+
+```ruby
+result = client.convert_image("photo.heic", output_format: "webp", save_to: "photo.webp")
+```
+
+Any pair among `jpeg`, `png`, `svg`, `heic`, `webp` — plus PDF rasterization (`pdf` → `jpeg`). Unsupported pairs raise before any request is made:
+
+```ruby
+Enconvert.valid_outputs_for("json")  # => ["csv", "toml", "xml", "yaml"]
+Enconvert.valid_outputs_for("pdf")   # => ["jpeg"]
+```
+
+### Document & data conversion
+
+```ruby
+client.convert_document("report.docx", save_to: "report.pdf")
+client.convert_document("data.json", output_format: "yaml", save_to: "data.yaml")
+client.convert_document("notes.md", output_format: "html", save_to: "notes.html")
+```
+
+Supported inputs: `doc`/`docx`, `xls`/`xlsx`, `ppt`/`pptx`, `odt`, `ods`, `odp`, `ots`, `pages`, `numbers`, `html`, `markdown`, `csv`, `json`, `xml`, `yaml`, `toml`. (EPUB → use `convert_to_pdf` / `convert_to_markdown`.)
+
+The SDK validates every `{input}-to-{output}` pair against the conversions the API actually implements and raises immediately — with the list of valid outputs for that input — instead of sending a doomed request.
+
+### Supported conversions
+
+| Input | Outputs |
+|-------|---------|
+| json | csv, toml, xml, yaml |
+| xml | csv, json |
+| yaml | json |
+| csv | json, xml |
+| toml | json |
+| markdown | html, pdf |
+| html | pdf |
+| doc, excel, ppt, odt, ods, odp, ots, pages, numbers | pdf |
+| jpeg, png, svg, heic, webp | each other (all 20 pairs) |
+| pdf | jpeg |
+
+### URL to PDF / Screenshot / Markdown
+
+```ruby
+client.convert_url_to_pdf("https://example.com", save_to: "page.pdf")
+client.convert_url_to_screenshot("https://example.com", viewport_width: 1440, save_to: "shot.png")
+client.convert_url_to_markdown("https://example.com/article", save_to: "article.md")
+```
+
+### Website to PDF / Screenshot (whole-site batch)
+
+Discover every page of a website (sitemap, or full crawl on higher plans), convert each in the background, and receive a single ZIP. Requires a private API key with crawl access.
+
+```ruby
+batch = client.convert_website_to_pdf("https://example.com", crawl_mode: "sitemap")
+status = client.wait_for_batch(batch.batch_id, save_to: "site.zip")
+puts "#{status.completed} of #{status.total} pages converted"
+```
+
+`convert_website_to_screenshot` works the same way and produces a ZIP of PNGs.
+
+### PDF options & authenticated pages
+
+```ruby
+client.convert_url_to_pdf(
+  "https://internal.example.com/report",
+  pdf_options: { page_size: "A4", orientation: "landscape", margins: { top: 10, bottom: 10 } },
+  auth: { username: "user", password: "pass" },     # or cookies / headers, plan-gated
+  save_to: "report.pdf"
+)
+```
+
+Do not combine `auth` with an `Authorization` header — the API rejects the conflict.
+
+### Job status (async polling)
+
+```ruby
+status = client.get_job_status("job_abc123")
+puts status.presigned_url if status.status == "success"
+```
+
+---
+
+## Error Handling
+
+```ruby
+begin
+  client.v2.perceive("https://example.com")
+rescue Enconvert::AuthenticationError
+  warn "Invalid API key"
+rescue Enconvert::QuotaError
+  warn "Plan feature off or quota exhausted"
+rescue Enconvert::RateLimitError
+  warn "Too many requests — slow down"
+rescue Enconvert::APIError => e
+  warn "API error [#{e.status_code}]: #{e.message}"
+end
+```
+
+## Configuration
+
+```ruby
+client = Enconvert::Client.new(
+  api_key: "sk_...",
+  timeout: 300,     # seconds, default
+  base_url: "https://api.enconvert.com" # override, default shown
+)
+```
+
+## Get an API Key
+
+Sign up at [enconvert.com](https://enconvert.com). Free tier: 100 ops/month, no credit card.
+
+## License
+
+MIT