exaonruby 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +94 -0
- data/README.md +72 -0
- data/exaonruby.gemspec +8 -3
- data/lib/exa/types.rb +204 -0
- data/lib/exa/utils/sse_client.rb +279 -0
- data/lib/exa/version.rb +1 -1
- data/lib/exa.rb +9 -0
- metadata +7 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 120800cd9da222f8a7cd47f1e608d4208a71e4a5b4d0a035b8ebfd5ffd17c115
|
|
4
|
+
data.tar.gz: 559bae5acadd43bc85854c2e62fde0852a40d683b2328fd0eea27f99aeef9470
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 788f13f5bc62a00227dd41b3803bb9d2bb88e5dc5f160ac6d684b6b3df487ebd66d050dc9f2da56355b7f4752ba10d1551ace7c8d49a1c1be6863b333c10b070
|
|
7
|
+
data.tar.gz: 90cdc099f08b5815329ed77551d40004ace0e869138888679a268c73c92dedb96539864f1f5b8480d67fe6b26a9baf489c47e212d60cfc09d9b5939d221023fd
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [1.1.0] - 2025-12-18
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- **SSE Streaming** - Real-time streaming for Answer and Research APIs
|
|
13
|
+
- `Exa::Utils::SSEClient.stream_answer` - Stream answer tokens as they're generated
|
|
14
|
+
- `Exa::Utils::SSEClient.stream_research` - Stream research progress and output
|
|
15
|
+
- Automatic reconnection and error handling
|
|
16
|
+
- Ping/heartbeat support
|
|
17
|
+
|
|
18
|
+
- **Sorbet Type Definitions** - Optional static type checking
|
|
19
|
+
- `Exa::Types` module with T::Struct definitions for all API types
|
|
20
|
+
- Optional dependency on `sorbet-runtime`
|
|
21
|
+
- Full type coverage for Search, Answer, Research, Websets, Monitors, Imports, Webhooks, Events
|
|
22
|
+
|
|
23
|
+
## [1.0.0] - 2025-12-18
|
|
24
|
+
|
|
25
|
+
### Added
|
|
26
|
+
|
|
27
|
+
- **Search API**
|
|
28
|
+
- Neural, auto, fast, and deep search types
|
|
29
|
+
- Content extraction with text, highlights, and summaries
|
|
30
|
+
- Domain and date filtering
|
|
31
|
+
- Category filtering (company, person, research paper, etc.)
|
|
32
|
+
|
|
33
|
+
- **Contents API**
|
|
34
|
+
- Fetch full page contents from URLs
|
|
35
|
+
- Livecrawl support (never, fallback, preferred, always)
|
|
36
|
+
- Subpage extraction
|
|
37
|
+
|
|
38
|
+
- **Find Similar API**
|
|
39
|
+
- Discover semantically similar pages
|
|
40
|
+
- Exclude source URL options
|
|
41
|
+
|
|
42
|
+
- **Answer API**
|
|
43
|
+
- LLM-powered question answering
|
|
44
|
+
- Citations with source URLs
|
|
45
|
+
- Search options integration
|
|
46
|
+
|
|
47
|
+
- **Research API**
|
|
48
|
+
- Async research task creation
|
|
49
|
+
- Multiple models (exa-research-fast, exa-research, exa-research-pro)
|
|
50
|
+
- Structured output schema support
|
|
51
|
+
- Task polling and cancellation
|
|
52
|
+
|
|
53
|
+
- **Websets API**
|
|
54
|
+
- Full CRUD operations for Websets
|
|
55
|
+
- Item management (list, get, delete)
|
|
56
|
+
- Search operations (create, get, cancel)
|
|
57
|
+
- Enrichment operations (create, list, get, delete)
|
|
58
|
+
|
|
59
|
+
- **Monitors API**
|
|
60
|
+
- Create automated search/refresh schedules
|
|
61
|
+
- Cron expression support
|
|
62
|
+
- Monitor run tracking
|
|
63
|
+
|
|
64
|
+
- **Imports API**
|
|
65
|
+
- CSV upload with presigned URLs
|
|
66
|
+
- Entity type configuration
|
|
67
|
+
- Import status tracking
|
|
68
|
+
|
|
69
|
+
- **Webhooks API**
|
|
70
|
+
- Create webhook subscriptions
|
|
71
|
+
- Event type filtering
|
|
72
|
+
- Webhook attempt tracking
|
|
73
|
+
|
|
74
|
+
- **Events API**
|
|
75
|
+
- List and filter events
|
|
76
|
+
- Event type helpers
|
|
77
|
+
|
|
78
|
+
- **CLI**
|
|
79
|
+
- Beautiful colorful command-line interface
|
|
80
|
+
- Search, answer, similar, research commands
|
|
81
|
+
- Websets management subcommands
|
|
82
|
+
- JSON output option
|
|
83
|
+
|
|
84
|
+
- **n8n/Zapier Integration**
|
|
85
|
+
- Webhook signature verification
|
|
86
|
+
- HMAC-SHA256 with timing attack prevention
|
|
87
|
+
- Timestamp validation for replay attack prevention
|
|
88
|
+
- Framework-agnostic header parsing
|
|
89
|
+
|
|
90
|
+
- **Core Features**
|
|
91
|
+
- Automatic retry with exponential backoff
|
|
92
|
+
- Comprehensive error hierarchy
|
|
93
|
+
- Rate limit handling with retry_after
|
|
94
|
+
- YARD documentation on all public methods
|
data/README.md
CHANGED
|
@@ -13,6 +13,8 @@ A production-ready Ruby gem wrapper for the [Exa.ai](https://exa.ai) API, provid
|
|
|
13
13
|
- **Monitors**: Automated scheduled searches and content refresh
|
|
14
14
|
- **Imports**: Upload CSV data into Websets
|
|
15
15
|
- **Webhooks & Events**: Real-time notifications for Websets activity
|
|
16
|
+
- **SSE Streaming**: Real-time token streaming for Answer and Research APIs
|
|
17
|
+
- **Sorbet Types**: Optional T::Struct type definitions for static type checking
|
|
16
18
|
- **Beautiful CLI**: Colorful command-line interface
|
|
17
19
|
- **n8n/Zapier Integration**: Webhook signature verification utilities
|
|
18
20
|
- **Automatic Retries**: Built-in retry logic for transient failures
|
|
@@ -593,12 +595,82 @@ exa search "AI news" --json
|
|
|
593
595
|
exa version
|
|
594
596
|
```
|
|
595
597
|
|
|
598
|
+
## SSE Streaming
|
|
599
|
+
|
|
600
|
+
Stream tokens in real-time for Answer and Research APIs:
|
|
601
|
+
|
|
602
|
+
```ruby
|
|
603
|
+
# Stream an answer with real-time token output
|
|
604
|
+
Exa::Utils::SSEClient.stream_answer(
|
|
605
|
+
api_key: ENV["EXA_API_KEY"],
|
|
606
|
+
query: "What is quantum computing?"
|
|
607
|
+
) do |event|
|
|
608
|
+
case event[:type]
|
|
609
|
+
when :token
|
|
610
|
+
print event[:data] # Print each token as it arrives
|
|
611
|
+
when :citation
|
|
612
|
+
puts "\nSource: #{event[:data][:url]}"
|
|
613
|
+
when :done
|
|
614
|
+
puts "\n\nComplete!"
|
|
615
|
+
when :error
|
|
616
|
+
puts "Error: #{event[:data]}"
|
|
617
|
+
end
|
|
618
|
+
end
|
|
619
|
+
|
|
620
|
+
# Stream research progress
|
|
621
|
+
Exa::Utils::SSEClient.stream_research(
|
|
622
|
+
api_key: ENV["EXA_API_KEY"],
|
|
623
|
+
instructions: "Research latest AI developments"
|
|
624
|
+
) do |event|
|
|
625
|
+
case event[:type]
|
|
626
|
+
when :progress
|
|
627
|
+
puts "Progress: #{event[:data][:percent]}%"
|
|
628
|
+
when :output
|
|
629
|
+
puts event[:data]
|
|
630
|
+
end
|
|
631
|
+
end
|
|
632
|
+
|
|
633
|
+
# Instance-based streaming
|
|
634
|
+
streamer = Exa::Utils::SSEClient.new(api_key: ENV["EXA_API_KEY"])
|
|
635
|
+
streamer.answer("What is GPT-4?") { |e| print e[:data] if e[:type] == :token }
|
|
636
|
+
```
|
|
637
|
+
|
|
638
|
+
## Sorbet Type Definitions
|
|
639
|
+
|
|
640
|
+
Optional static type checking with Sorbet:
|
|
641
|
+
|
|
642
|
+
```ruby
|
|
643
|
+
# Install sorbet-runtime for type definitions
|
|
644
|
+
# gem install sorbet-runtime
|
|
645
|
+
|
|
646
|
+
require 'exa'
|
|
647
|
+
|
|
648
|
+
# Types are available when sorbet-runtime is installed
|
|
649
|
+
params = Exa::Types::SearchParams.new(
|
|
650
|
+
query: "AI research",
|
|
651
|
+
type: "neural",
|
|
652
|
+
num_results: 10,
|
|
653
|
+
text: true
|
|
654
|
+
)
|
|
655
|
+
|
|
656
|
+
# Type definitions for all API responses
|
|
657
|
+
# Exa::Types::SearchResultData
|
|
658
|
+
# Exa::Types::AnswerResponseData
|
|
659
|
+
# Exa::Types::ResearchTaskData
|
|
660
|
+
# Exa::Types::WebsetData
|
|
661
|
+
# Exa::Types::MonitorData
|
|
662
|
+
# Exa::Types::ImportData
|
|
663
|
+
# Exa::Types::WebhookData
|
|
664
|
+
# Exa::Types::EventData
|
|
665
|
+
```
|
|
666
|
+
|
|
596
667
|
## Requirements
|
|
597
668
|
|
|
598
669
|
- Ruby >= 3.1
|
|
599
670
|
- faraday >= 2.0
|
|
600
671
|
- faraday-retry >= 2.0
|
|
601
672
|
- thor >= 1.0
|
|
673
|
+
- sorbet-runtime >= 0.5 (optional, for type definitions)
|
|
602
674
|
|
|
603
675
|
## Development
|
|
604
676
|
|
data/exaonruby.gemspec
CHANGED
|
@@ -8,10 +8,11 @@ Gem::Specification.new do |spec|
|
|
|
8
8
|
spec.authors = ["tigel-agm"]
|
|
9
9
|
spec.email = []
|
|
10
10
|
|
|
11
|
-
spec.summary = "Complete Ruby client for the Exa.ai API with
|
|
11
|
+
spec.summary = "Complete Ruby client for the Exa.ai API with CLI and SSE streaming"
|
|
12
12
|
spec.description = "A production-ready Ruby gem wrapper for the Exa.ai Search and Websets APIs. " \
|
|
13
13
|
"Features neural search, LLM-powered answers, async research tasks, " \
|
|
14
|
-
"Websets management (monitors, imports, webhooks),
|
|
14
|
+
"Websets management (monitors, imports, webhooks), SSE streaming, " \
|
|
15
|
+
"Sorbet type definitions, and a beautiful CLI. " \
|
|
15
16
|
"Includes n8n/Zapier webhook signature verification utilities."
|
|
16
17
|
spec.homepage = "https://github.com/tigel-agm/exaonruby"
|
|
17
18
|
spec.license = "MIT"
|
|
@@ -24,14 +25,18 @@ Gem::Specification.new do |spec|
|
|
|
24
25
|
spec.metadata["rubygems_mfa_required"] = "true"
|
|
25
26
|
|
|
26
27
|
# Include all lib files explicitly since we may not have git
|
|
27
|
-
spec.files = Dir.glob("{lib,exe}/**/*") + %w[LICENSE.txt README.md]
|
|
28
|
+
spec.files = Dir.glob("{lib,exe}/**/*") + %w[LICENSE.txt README.md CHANGELOG.md]
|
|
28
29
|
spec.files += Dir.glob("*.gemspec")
|
|
29
30
|
|
|
30
31
|
spec.bindir = "exe"
|
|
31
32
|
spec.executables = ["exa"]
|
|
32
33
|
spec.require_paths = ["lib"]
|
|
33
34
|
|
|
35
|
+
# Core dependencies
|
|
34
36
|
spec.add_dependency "faraday", ">= 2.0", "< 3.0"
|
|
35
37
|
spec.add_dependency "faraday-retry", ">= 2.0", "< 3.0"
|
|
36
38
|
spec.add_dependency "thor", ">= 1.0", "< 3.0"
|
|
39
|
+
|
|
40
|
+
# Optional: Sorbet types (install sorbet-runtime for type checking)
|
|
41
|
+
# gem install sorbet-runtime
|
|
37
42
|
end
|
data/lib/exa/types.rb
ADDED
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# typed: strict
|
|
4
|
+
|
|
5
|
+
require "sorbet-runtime"
|
|
6
|
+
|
|
7
|
+
module Exa
|
|
8
|
+
module Types
|
|
9
|
+
extend T::Sig
|
|
10
|
+
|
|
11
|
+
# Typed request structures for Search API
|
|
12
|
+
class SearchParams < T::Struct
|
|
13
|
+
const :query, String
|
|
14
|
+
const :type, T.nilable(String)
|
|
15
|
+
const :num_results, T.nilable(Integer)
|
|
16
|
+
const :include_domains, T.nilable(T::Array[String])
|
|
17
|
+
const :exclude_domains, T.nilable(T::Array[String])
|
|
18
|
+
const :start_crawl_date, T.nilable(String)
|
|
19
|
+
const :end_crawl_date, T.nilable(String)
|
|
20
|
+
const :start_published_date, T.nilable(String)
|
|
21
|
+
const :end_published_date, T.nilable(String)
|
|
22
|
+
const :include_text, T.nilable(T::Array[String])
|
|
23
|
+
const :exclude_text, T.nilable(T::Array[String])
|
|
24
|
+
const :category, T.nilable(String)
|
|
25
|
+
const :country, T.nilable(String)
|
|
26
|
+
const :text, T.nilable(T::Boolean)
|
|
27
|
+
const :highlights, T.nilable(T::Boolean)
|
|
28
|
+
const :summary, T.nilable(T::Boolean)
|
|
29
|
+
const :livecrawl, T.nilable(String)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
class ContentsParams < T::Struct
|
|
33
|
+
const :ids, T::Array[String]
|
|
34
|
+
const :text, T.nilable(T::Boolean)
|
|
35
|
+
const :highlights, T.nilable(T::Boolean)
|
|
36
|
+
const :summary, T.nilable(T::Boolean)
|
|
37
|
+
const :livecrawl, T.nilable(String)
|
|
38
|
+
const :subpages, T.nilable(Integer)
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
class AnswerParams < T::Struct
|
|
42
|
+
const :query, String
|
|
43
|
+
const :text, T.nilable(T::Boolean)
|
|
44
|
+
const :stream, T.nilable(T::Boolean)
|
|
45
|
+
const :num_results, T.nilable(Integer)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
class ResearchParams < T::Struct
|
|
49
|
+
const :instructions, String
|
|
50
|
+
const :model, T.nilable(String)
|
|
51
|
+
const :output_schema, T.nilable(T::Hash[Symbol, T.untyped])
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# Typed response structures
|
|
55
|
+
class SearchResultData < T::Struct
|
|
56
|
+
const :id, String
|
|
57
|
+
const :url, String
|
|
58
|
+
const :title, T.nilable(String)
|
|
59
|
+
const :score, T.nilable(Float)
|
|
60
|
+
const :published_date, T.nilable(String)
|
|
61
|
+
const :author, T.nilable(String)
|
|
62
|
+
const :text, T.nilable(String)
|
|
63
|
+
const :highlights, T.nilable(T::Array[String])
|
|
64
|
+
const :summary, T.nilable(String)
|
|
65
|
+
const :image, T.nilable(String)
|
|
66
|
+
const :favicon, T.nilable(String)
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
class CostBreakdown < T::Struct
|
|
70
|
+
const :neural_search, T.nilable(Float)
|
|
71
|
+
const :deep_search, T.nilable(Float)
|
|
72
|
+
const :content_text, T.nilable(Float)
|
|
73
|
+
const :content_highlight, T.nilable(Float)
|
|
74
|
+
const :content_summary, T.nilable(Float)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
class CostData < T::Struct
|
|
78
|
+
const :total, Float
|
|
79
|
+
const :search, T.nilable(Float)
|
|
80
|
+
const :contents, T.nilable(Float)
|
|
81
|
+
const :breakdown, T.nilable(CostBreakdown)
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
class SearchResponseData < T::Struct
|
|
85
|
+
const :request_id, T.nilable(String)
|
|
86
|
+
const :results, T::Array[SearchResultData]
|
|
87
|
+
const :cost_dollars, T.nilable(CostData)
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
class AnswerResponseData < T::Struct
|
|
91
|
+
const :answer, String
|
|
92
|
+
const :citations, T::Array[SearchResultData]
|
|
93
|
+
const :cost_dollars, T.nilable(CostData)
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
class ResearchTaskData < T::Struct
|
|
97
|
+
const :research_id, String
|
|
98
|
+
const :model, String
|
|
99
|
+
const :instructions, String
|
|
100
|
+
const :status, String
|
|
101
|
+
const :created_at, T.nilable(Integer)
|
|
102
|
+
const :completed_at, T.nilable(Integer)
|
|
103
|
+
const :output, T.nilable(T.any(String, T::Hash[Symbol, T.untyped]))
|
|
104
|
+
const :error, T.nilable(String)
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
# Websets types
|
|
108
|
+
class WebsetData < T::Struct
|
|
109
|
+
const :id, String
|
|
110
|
+
const :object, String
|
|
111
|
+
const :status, String
|
|
112
|
+
const :external_id, T.nilable(String)
|
|
113
|
+
const :title, T.nilable(String)
|
|
114
|
+
const :metadata, T.nilable(T::Hash[Symbol, T.untyped])
|
|
115
|
+
const :created_at, T.nilable(String)
|
|
116
|
+
const :updated_at, T.nilable(String)
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
class WebsetItemData < T::Struct
|
|
120
|
+
const :id, String
|
|
121
|
+
const :object, String
|
|
122
|
+
const :webset_id, String
|
|
123
|
+
const :url, String
|
|
124
|
+
const :type, T.nilable(String)
|
|
125
|
+
const :status, T.nilable(String)
|
|
126
|
+
const :created_at, T.nilable(String)
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
class MonitorData < T::Struct
|
|
130
|
+
const :id, String
|
|
131
|
+
const :object, String
|
|
132
|
+
const :status, String
|
|
133
|
+
const :webset_id, String
|
|
134
|
+
const :cron, T.nilable(String)
|
|
135
|
+
const :timezone, T.nilable(String)
|
|
136
|
+
const :next_run_at, T.nilable(String)
|
|
137
|
+
const :created_at, T.nilable(String)
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
class ImportData < T::Struct
|
|
141
|
+
const :id, String
|
|
142
|
+
const :object, String
|
|
143
|
+
const :status, String
|
|
144
|
+
const :format, String
|
|
145
|
+
const :title, T.nilable(String)
|
|
146
|
+
const :count, T.nilable(Integer)
|
|
147
|
+
const :upload_url, T.nilable(String)
|
|
148
|
+
const :upload_valid_until, T.nilable(String)
|
|
149
|
+
const :created_at, T.nilable(String)
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
class WebhookData < T::Struct
|
|
153
|
+
const :id, String
|
|
154
|
+
const :object, String
|
|
155
|
+
const :status, String
|
|
156
|
+
const :url, String
|
|
157
|
+
const :events, T::Array[String]
|
|
158
|
+
const :secret, T.nilable(String)
|
|
159
|
+
const :created_at, T.nilable(String)
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
class EventData < T::Struct
|
|
163
|
+
const :id, String
|
|
164
|
+
const :object, String
|
|
165
|
+
const :type, String
|
|
166
|
+
const :data, T::Hash[Symbol, T.untyped]
|
|
167
|
+
const :created_at, T.nilable(String)
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
# Pagination types
|
|
171
|
+
class PaginatedResponseData < T::Struct
|
|
172
|
+
const :data, T::Array[T.untyped]
|
|
173
|
+
const :has_more, T.nilable(T::Boolean)
|
|
174
|
+
const :next_cursor, T.nilable(String)
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
# Type aliases for common patterns
|
|
178
|
+
SearchType = T.type_alias { T.any(String, Symbol) }
|
|
179
|
+
LivecrawlOption = T.type_alias { T.any(String, Symbol) }
|
|
180
|
+
DateInput = T.type_alias { T.any(String, Time, Date) }
|
|
181
|
+
|
|
182
|
+
# Valid enum values
|
|
183
|
+
SEARCH_TYPES = T.let(%w[auto neural fast deep].freeze, T::Array[String])
|
|
184
|
+
LIVECRAWL_OPTIONS = T.let(%w[never fallback preferred always].freeze, T::Array[String])
|
|
185
|
+
CATEGORIES = T.let(%w[
|
|
186
|
+
company person research_paper news pdf github tweet
|
|
187
|
+
personal_site financial_report
|
|
188
|
+
].freeze, T::Array[String])
|
|
189
|
+
RESEARCH_MODELS = T.let(%w[exa-research-fast exa-research exa-research-pro].freeze, T::Array[String])
|
|
190
|
+
WEBSET_STATUSES = T.let(%w[idle pending running paused].freeze, T::Array[String])
|
|
191
|
+
IMPORT_STATUSES = T.let(%w[pending processing completed failed].freeze, T::Array[String])
|
|
192
|
+
MONITOR_STATUSES = T.let(%w[enabled disabled].freeze, T::Array[String])
|
|
193
|
+
|
|
194
|
+
WEBHOOK_EVENTS = T.let(%w[
|
|
195
|
+
webset.created webset.deleted webset.paused webset.idle
|
|
196
|
+
webset.search.created webset.search.canceled webset.search.completed webset.search.updated
|
|
197
|
+
import.created import.completed
|
|
198
|
+
webset.item.created webset.item.enriched
|
|
199
|
+
monitor.created monitor.updated monitor.deleted
|
|
200
|
+
monitor.run.created monitor.run.completed
|
|
201
|
+
webset.export.created webset.export.completed
|
|
202
|
+
].freeze, T::Array[String])
|
|
203
|
+
end
|
|
204
|
+
end
|
|
@@ -0,0 +1,279 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# typed: strict
|
|
4
|
+
|
|
5
|
+
require "json"
|
|
6
|
+
require "net/http"
|
|
7
|
+
require "uri"
|
|
8
|
+
|
|
9
|
+
module Exa
|
|
10
|
+
module Utils
|
|
11
|
+
# Server-Sent Events (SSE) streaming client for Exa.ai API
|
|
12
|
+
#
|
|
13
|
+
# Provides real-time streaming for Answer and Research endpoints that support
|
|
14
|
+
# the stream=true parameter. Tokens are yielded as they're generated.
|
|
15
|
+
#
|
|
16
|
+
# @example Stream an answer
|
|
17
|
+
# Exa::Utils::SSEClient.stream_answer(
|
|
18
|
+
# api_key: ENV["EXA_API_KEY"],
|
|
19
|
+
# query: "What is quantum computing?"
|
|
20
|
+
# ) do |event|
|
|
21
|
+
# case event[:type]
|
|
22
|
+
# when :token
|
|
23
|
+
# print event[:data] # Print each token as it arrives
|
|
24
|
+
# when :citation
|
|
25
|
+
# puts "\nSource: #{event[:data][:url]}"
|
|
26
|
+
# when :done
|
|
27
|
+
# puts "\n\nComplete!"
|
|
28
|
+
# when :error
|
|
29
|
+
# puts "Error: #{event[:data]}"
|
|
30
|
+
# end
|
|
31
|
+
# end
|
|
32
|
+
#
|
|
33
|
+
# @example Stream research progress
|
|
34
|
+
# Exa::Utils::SSEClient.stream_research(
|
|
35
|
+
# api_key: ENV["EXA_API_KEY"],
|
|
36
|
+
# instructions: "Research latest AI developments"
|
|
37
|
+
# ) do |event|
|
|
38
|
+
# case event[:type]
|
|
39
|
+
# when :progress
|
|
40
|
+
# puts "Progress: #{event[:data][:percent]}%"
|
|
41
|
+
# when :output
|
|
42
|
+
# puts event[:data]
|
|
43
|
+
# end
|
|
44
|
+
# end
|
|
45
|
+
class SSEClient
|
|
46
|
+
DEFAULT_BASE_URL = "https://api.exa.ai"
|
|
47
|
+
|
|
48
|
+
# Event types that can be yielded
|
|
49
|
+
EVENT_TYPES = %i[token citation progress output done error ping].freeze
|
|
50
|
+
|
|
51
|
+
class << self
|
|
52
|
+
# Stream an answer with real-time token output
|
|
53
|
+
#
|
|
54
|
+
# @param api_key [String] Exa API key
|
|
55
|
+
# @param query [String] Question to answer
|
|
56
|
+
# @param base_url [String] API base URL
|
|
57
|
+
# @param options [Hash] Additional options (text, num_results, etc.)
|
|
58
|
+
#
|
|
59
|
+
# @yield [Hash] Event hash with :type and :data keys
|
|
60
|
+
# @yieldparam event [Hash] Event data
|
|
61
|
+
# @yieldparam event[:type] [Symbol] One of :token, :citation, :done, :error
|
|
62
|
+
# @yieldparam event[:data] [String, Hash] Event payload
|
|
63
|
+
#
|
|
64
|
+
# @return [void]
|
|
65
|
+
def stream_answer(api_key:, query:, base_url: DEFAULT_BASE_URL, **options, &block)
|
|
66
|
+
raise ArgumentError, "Block required for streaming" unless block_given?
|
|
67
|
+
raise InvalidRequestError, "query is required" if query.nil? || query.empty?
|
|
68
|
+
|
|
69
|
+
body = { query: query, stream: true }
|
|
70
|
+
body.merge!(options)
|
|
71
|
+
|
|
72
|
+
stream_request(
|
|
73
|
+
api_key: api_key,
|
|
74
|
+
url: "#{base_url}/answer",
|
|
75
|
+
body: body,
|
|
76
|
+
&block
|
|
77
|
+
)
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# Stream research task output in real-time
|
|
81
|
+
#
|
|
82
|
+
# @param api_key [String] Exa API key
|
|
83
|
+
# @param instructions [String] Research instructions
|
|
84
|
+
# @param model [String] Model to use
|
|
85
|
+
# @param base_url [String] API base URL
|
|
86
|
+
# @param options [Hash] Additional options
|
|
87
|
+
#
|
|
88
|
+
# @yield [Hash] Event hash with :type and :data keys
|
|
89
|
+
# @yieldparam event [Hash] Event data
|
|
90
|
+
#
|
|
91
|
+
# @return [void]
|
|
92
|
+
def stream_research(api_key:, instructions:, model: "exa-research", base_url: DEFAULT_BASE_URL, **options, &block)
|
|
93
|
+
raise ArgumentError, "Block required for streaming" unless block_given?
|
|
94
|
+
raise InvalidRequestError, "instructions required" if instructions.nil? || instructions.empty?
|
|
95
|
+
|
|
96
|
+
body = {
|
|
97
|
+
instructions: instructions,
|
|
98
|
+
model: model,
|
|
99
|
+
stream: true
|
|
100
|
+
}
|
|
101
|
+
body.merge!(options)
|
|
102
|
+
|
|
103
|
+
stream_request(
|
|
104
|
+
api_key: api_key,
|
|
105
|
+
url: "#{base_url}/research/v1",
|
|
106
|
+
body: body,
|
|
107
|
+
&block
|
|
108
|
+
)
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
private
|
|
112
|
+
|
|
113
|
+
# Perform SSE streaming request
|
|
114
|
+
#
|
|
115
|
+
# @param api_key [String] API key
|
|
116
|
+
# @param url [String] Full endpoint URL
|
|
117
|
+
# @param body [Hash] Request body
|
|
118
|
+
#
|
|
119
|
+
# @yield [Hash] Parsed SSE events
|
|
120
|
+
def stream_request(api_key:, url:, body:)
|
|
121
|
+
uri = URI.parse(url)
|
|
122
|
+
|
|
123
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
|
124
|
+
http.use_ssl = uri.scheme == "https"
|
|
125
|
+
http.read_timeout = 300 # 5 minutes for long streams
|
|
126
|
+
http.open_timeout = 30
|
|
127
|
+
|
|
128
|
+
request = Net::HTTP::Post.new(uri.request_uri)
|
|
129
|
+
request["Content-Type"] = "application/json"
|
|
130
|
+
request["Accept"] = "text/event-stream"
|
|
131
|
+
request["Cache-Control"] = "no-cache"
|
|
132
|
+
request["x-api-key"] = api_key
|
|
133
|
+
request.body = JSON.generate(body)
|
|
134
|
+
|
|
135
|
+
buffer = String.new
|
|
136
|
+
|
|
137
|
+
http.request(request) do |response|
|
|
138
|
+
unless response.is_a?(Net::HTTPSuccess)
|
|
139
|
+
yield({ type: :error, data: "HTTP #{response.code}: #{response.message}" })
|
|
140
|
+
return
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
response.read_body do |chunk|
|
|
144
|
+
buffer << chunk
|
|
145
|
+
events = parse_sse_buffer(buffer)
|
|
146
|
+
|
|
147
|
+
events.each do |event|
|
|
148
|
+
yield(event)
|
|
149
|
+
end
|
|
150
|
+
end
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
yield({ type: :done, data: nil })
|
|
154
|
+
rescue Net::OpenTimeout, Net::ReadTimeout => e
|
|
155
|
+
yield({ type: :error, data: "Timeout: #{e.message}" })
|
|
156
|
+
rescue IOError, Errno::ECONNRESET => e
|
|
157
|
+
yield({ type: :error, data: "Connection error: #{e.message}" })
|
|
158
|
+
rescue StandardError => e
|
|
159
|
+
yield({ type: :error, data: "Error: #{e.message}" })
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
# Parse SSE buffer and extract complete events
|
|
163
|
+
#
|
|
164
|
+
# @param buffer [String] Buffer to parse (modified in place)
|
|
165
|
+
# @return [Array<Hash>] Parsed events
|
|
166
|
+
def parse_sse_buffer(buffer)
|
|
167
|
+
events = []
|
|
168
|
+
event_data = {}
|
|
169
|
+
|
|
170
|
+
# Split on double newlines (event boundaries)
|
|
171
|
+
while (idx = buffer.index("\n\n"))
|
|
172
|
+
raw_event = buffer.slice!(0, idx + 2)
|
|
173
|
+
|
|
174
|
+
raw_event.each_line do |line|
|
|
175
|
+
line = line.strip
|
|
176
|
+
next if line.empty?
|
|
177
|
+
|
|
178
|
+
if line.start_with?("event:")
|
|
179
|
+
event_data[:event] = line[6..].strip
|
|
180
|
+
elsif line.start_with?("data:")
|
|
181
|
+
data_content = line[5..].strip
|
|
182
|
+
event_data[:data] = data_content
|
|
183
|
+
elsif line.start_with?("id:")
|
|
184
|
+
event_data[:id] = line[3..].strip
|
|
185
|
+
elsif line.start_with?("retry:")
|
|
186
|
+
event_data[:retry] = line[6..].strip.to_i
|
|
187
|
+
end
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
if event_data.any?
|
|
191
|
+
parsed = parse_event(event_data)
|
|
192
|
+
events << parsed if parsed
|
|
193
|
+
event_data = {}
|
|
194
|
+
end
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
events
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
# Parse a single SSE event into our format
|
|
201
|
+
#
|
|
202
|
+
# @param event_data [Hash] Raw event data
|
|
203
|
+
# @return [Hash, nil] Parsed event or nil
|
|
204
|
+
def parse_event(event_data)
|
|
205
|
+
event_type = event_data[:event]&.to_sym || :message
|
|
206
|
+
raw_data = event_data[:data]
|
|
207
|
+
|
|
208
|
+
return nil unless raw_data
|
|
209
|
+
|
|
210
|
+
# Try to parse as JSON
|
|
211
|
+
data = begin
|
|
212
|
+
JSON.parse(raw_data, symbolize_names: true)
|
|
213
|
+
rescue JSON::ParserError
|
|
214
|
+
raw_data
|
|
215
|
+
end
|
|
216
|
+
|
|
217
|
+
case event_type
|
|
218
|
+
when :message, :token, :delta
|
|
219
|
+
# Token/delta events contain partial answer text
|
|
220
|
+
if data.is_a?(Hash)
|
|
221
|
+
text = data[:delta] || data[:content] || data[:text] || data[:token]
|
|
222
|
+
{ type: :token, data: text } if text
|
|
223
|
+
else
|
|
224
|
+
{ type: :token, data: data }
|
|
225
|
+
end
|
|
226
|
+
when :citation, :source
|
|
227
|
+
{ type: :citation, data: data }
|
|
228
|
+
when :progress
|
|
229
|
+
{ type: :progress, data: data }
|
|
230
|
+
when :output, :result
|
|
231
|
+
{ type: :output, data: data }
|
|
232
|
+
when :done, :complete, :end
|
|
233
|
+
{ type: :done, data: data }
|
|
234
|
+
when :error
|
|
235
|
+
{ type: :error, data: data }
|
|
236
|
+
when :ping, :heartbeat
|
|
237
|
+
{ type: :ping, data: nil }
|
|
238
|
+
else
|
|
239
|
+
# Return raw data for unknown event types
|
|
240
|
+
{ type: event_type, data: data }
|
|
241
|
+
end
|
|
242
|
+
end
|
|
243
|
+
end
|
|
244
|
+
|
|
245
|
+
# Instance-based streaming for more control
|
|
246
|
+
#
|
|
247
|
+
# @param api_key [String] Exa API key
|
|
248
|
+
# @param base_url [String] API base URL
|
|
249
|
+
def initialize(api_key:, base_url: DEFAULT_BASE_URL)
|
|
250
|
+
@api_key = api_key
|
|
251
|
+
@base_url = base_url
|
|
252
|
+
end
|
|
253
|
+
|
|
254
|
+
# Stream an answer
|
|
255
|
+
# @see SSEClient.stream_answer
|
|
256
|
+
def answer(query, **options, &block)
|
|
257
|
+
self.class.stream_answer(
|
|
258
|
+
api_key: @api_key,
|
|
259
|
+
query: query,
|
|
260
|
+
base_url: @base_url,
|
|
261
|
+
**options,
|
|
262
|
+
&block
|
|
263
|
+
)
|
|
264
|
+
end
|
|
265
|
+
|
|
266
|
+
# Stream research
|
|
267
|
+
# @see SSEClient.stream_research
|
|
268
|
+
def research(instructions, **options, &block)
|
|
269
|
+
self.class.stream_research(
|
|
270
|
+
api_key: @api_key,
|
|
271
|
+
instructions: instructions,
|
|
272
|
+
base_url: @base_url,
|
|
273
|
+
**options,
|
|
274
|
+
&block
|
|
275
|
+
)
|
|
276
|
+
end
|
|
277
|
+
end
|
|
278
|
+
end
|
|
279
|
+
end
|
data/lib/exa/version.rb
CHANGED
data/lib/exa.rb
CHANGED
|
@@ -8,6 +8,15 @@ require_relative "exa/configuration"
|
|
|
8
8
|
|
|
9
9
|
require_relative "exa/utils/parameter_converter"
|
|
10
10
|
require_relative "exa/utils/webhook_handler"
|
|
11
|
+
require_relative "exa/utils/sse_client"
|
|
12
|
+
|
|
13
|
+
# Optional: Sorbet types (only loaded if sorbet-runtime is available)
|
|
14
|
+
begin
|
|
15
|
+
require "sorbet-runtime"
|
|
16
|
+
require_relative "exa/types"
|
|
17
|
+
rescue LoadError
|
|
18
|
+
# sorbet-runtime not installed, types module not available
|
|
19
|
+
end
|
|
11
20
|
|
|
12
21
|
require_relative "exa/resources/base"
|
|
13
22
|
require_relative "exa/resources/search_result"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: exaonruby
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.1.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- tigel-agm
|
|
@@ -71,14 +71,15 @@ dependencies:
|
|
|
71
71
|
version: '3.0'
|
|
72
72
|
description: A production-ready Ruby gem wrapper for the Exa.ai Search and Websets
|
|
73
73
|
APIs. Features neural search, LLM-powered answers, async research tasks, Websets
|
|
74
|
-
management (monitors, imports, webhooks),
|
|
75
|
-
webhook signature verification utilities.
|
|
74
|
+
management (monitors, imports, webhooks), SSE streaming, Sorbet type definitions,
|
|
75
|
+
and a beautiful CLI. Includes n8n/Zapier webhook signature verification utilities.
|
|
76
76
|
email: []
|
|
77
77
|
executables:
|
|
78
78
|
- exa
|
|
79
79
|
extensions: []
|
|
80
80
|
extra_rdoc_files: []
|
|
81
81
|
files:
|
|
82
|
+
- CHANGELOG.md
|
|
82
83
|
- LICENSE.txt
|
|
83
84
|
- README.md
|
|
84
85
|
- exaonruby.gemspec
|
|
@@ -114,7 +115,9 @@ files:
|
|
|
114
115
|
- lib/exa/resources/webhook.rb
|
|
115
116
|
- lib/exa/resources/webset.rb
|
|
116
117
|
- lib/exa/resources/webset_item.rb
|
|
118
|
+
- lib/exa/types.rb
|
|
117
119
|
- lib/exa/utils/parameter_converter.rb
|
|
120
|
+
- lib/exa/utils/sse_client.rb
|
|
118
121
|
- lib/exa/utils/webhook_handler.rb
|
|
119
122
|
- lib/exa/version.rb
|
|
120
123
|
homepage: https://github.com/tigel-agm/exaonruby
|
|
@@ -142,5 +145,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
142
145
|
requirements: []
|
|
143
146
|
rubygems_version: 4.0.2
|
|
144
147
|
specification_version: 4
|
|
145
|
-
summary: Complete Ruby client for the Exa.ai API with
|
|
148
|
+
summary: Complete Ruby client for the Exa.ai API with CLI and SSE streaming
|
|
146
149
|
test_files: []
|