webctx 0.2.4 → 0.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CONTRIBUTORS.md CHANGED
@@ -97,7 +97,7 @@ gh workflow run npm-publish-retry.yml --repo amxv/webctx --ref main -f version=0
97
97
  - `cmd/webctx/main.go`: CLI entrypoint
98
98
  - `internal/app/`: CLI parsing, search, ranking, scrape, env loading, and Firecrawl queue logic
99
99
  - `pkg/retrieval/`: small public adapter to the shared read-link / site-map engine
100
- - `pkg/origo/`: authenticated, stateless MCP server with exactly two tools
100
+ - `pkg/origo/`: authenticated, stateless MCP server with five tools (including structured-data discovery, inspection and execution)
101
101
  - `api/mcp.go`: Vercel Go Function entrypoint for Origo
102
102
  - `vercel.mjs`: project-specific Vercel settings for docs and Origo
103
103
  - `internal/buildinfo/`: build-time version plumbing for `--version`
package/README.md CHANGED
@@ -2,12 +2,15 @@
2
2
 
3
3
  Web context for agents, from the terminal.
4
4
 
5
- `webctx` gives you three small commands:
5
+ `webctx` provides six commands: three for web documents and three for structured data.
6
6
 
7
7
  ```bash
8
8
  webctx search "agent web research"
9
9
  webctx read-link <url>
10
10
  webctx map-site <url>
11
+ webctx research "podcast conversations about AI agents"
12
+ webctx inspect particle/podcasts/episodes/search
13
+ webctx execute particle/podcasts/episodes/search --inputs '{"semantic_search":"AI agents","limit":2}'
11
14
  ```
12
15
 
13
16
  The output is plain text or markdown, so it is easy to hand to ChatGPT, Codex, Claude Code, a shell script, or another tool. Search now retrieves relevant source excerpts automatically instead of returning only URLs. For a specific URL, add an optional question to explore related documentation and surface exact code and API details.
@@ -19,11 +22,25 @@ webctx read-link https://docs.example.com/api --question "How do authentication
19
22
 
20
23
  ## Origo: hosted MCP access
21
24
 
22
- **Origo** is the MCP transport over the same WebCTX retrieval engine. Connect an MCP client to `https://api.origo.ashray.xyz/mcp?key=<your-private-key>` to access exactly two tools: `read_link` and `map_site`. Origo does not expose `search`; the CLI continues to provide all three commands. `read_link` accepts an optional `question` to retrieve focused source context while keeping ordinary URL-only reads intact.
25
+ **Origo** is the MCP transport over the same WebCTX retrieval engine. Connect an MCP client to `https://api.origo.ashray.xyz/mcp?key=<your-private-key>` to access five tools: `read_link`, `map_site`, `research`, `inspect`, and `execute`. Origo deliberately omits federated web `search` but exposes the complete progressive data-research workflow. `read_link` accepts an optional `question` and keeps URL-only behavior unchanged.
23
26
 
24
27
  Both surfaces share native GitHub/Markdown readers and Firecrawl fallback behavior. Firecrawl's built-in cache is limited to **30 minutes**. Difficult pages automatically escalate through Firecrawl's proxy fallback and, if necessary, a temporary Browser Sandbox session. There is no Redis or additional cache database.
25
28
 
26
- Source-aware reading also prefers raw JSON, YAML, and OpenAPI files when the URL identifies one; question-focused reads can follow up to two relevant pages on the same host, including links found in `llms.txt`, and inspect embedded structured JSON. Firecrawl Alexandria tool matching and search discovery are automatic when supported, but **no paid Alexandria provider tools are executed**. The original source URLs remain explicit so an agent can verify facts.
29
+ Source-aware reading also prefers raw JSON, YAML, and OpenAPI files when the URL identifies one; question-focused reads can follow relevant references, including links found in `llms.txt`, and inspect embedded structured JSON. The new `research`, `inspect`, and `execute` commands give agents direct control over Firecrawl Alexandria's catalogue and paid data operations without learning provider-specific transport syntax. The original source URLs remain explicit so an agent can verify facts.
30
+
31
+ ## Structured data: discover → inspect → execute
32
+
33
+ `research` discovers datasets and operations by meaning, website, source, category, or group. `inspect` retrieves exact parameter types, required fields, response shapes, pricing, and copy-ready `execute` arguments. `execute` validates the inputs and credit estimate before calling one or more data operations, and preserves their source-native JSON and pagination.
34
+
35
+ ```bash
36
+ webctx research "podcast conversations about AI agents" --limit 5
37
+ webctx research --view sources --limit 5
38
+ webctx research --mode catalogue --view operations --sources particle --limit 10
39
+ webctx inspect particle/podcasts/episodes/search
40
+ webctx execute particle/podcasts/episodes/search --inputs '{"semantic_search":"AI agents","limit":2}'
41
+ ```
42
+
43
+ Browse [the structured-data guide](https://webctx.ashray.xyz/docs/data-sources) for filters, paging, batches, full upstream responses, validation, and credit handling. Discovery/inspection is free; `execute` uses paid Alexandria credits. The hosted Origo server enforces a 200-credit limit per request, with no user-facing budget controls. Estimated and actual credits are shown in responses. Third-party provider terms require separate human acceptance; Origo never accepts them automatically.
27
44
 
28
45
  Origo deploys independently through the `origo-api` Vercel project on pushes to `main`. CLI releases and npm publishing are unchanged. See the [Origo MCP guide](https://webctx.ashray.xyz/docs/origo) for architecture, setup, and security notes.
29
46
 
@@ -78,7 +95,7 @@ Recognized native GitHub auth, private/not-found, and rate-limit failures stay a
78
95
 
79
96
  ## Search
80
97
 
81
- Normal search asks Brave, Tavily, and Exa, removes duplicate URLs, automatically reads three promising sources, follows up to two relevant links, and returns excerpts plus the ranked list. Alexandria tool discovery is included when available, without paid execution.
98
+ Normal search asks Brave, Tavily, and Exa, removes duplicate URLs, automatically reads promising sources, follows relevant links, and returns excerpts plus the ranked list. Alexandria discovery is included when available; use `execute` when you need provider-native data.
82
99
 
83
100
  ```bash
84
101
  webctx search "next.js server components"
@@ -115,7 +132,7 @@ GH_TOKEN=...
115
132
  ```
116
133
 
117
134
  - `BRAVE_API_KEY`, `TAVILY_API_KEY`, and `EXA_API_KEY` power normal `search`.
118
- - `FIRECRAWL_API_KEY` powers `map-site` and page-crawl fallbacks.
135
+ - `FIRECRAWL_API_KEY` powers `map-site`, page-crawl fallbacks, and Alexandria discovery/inspection/paid execution.
119
136
  - `GH_TOKEN` or `GITHUB_TOKEN` is optional. It increases GitHub capacity and unlocks reads such as blame, Discussions, some Actions job logs, private resources the token can access, and richer PR review state.
120
137
 
121
138
  Environment variables, `.env.local`, and macOS Keychain are supported. `GH_TOKEN` takes precedence over `GITHUB_TOKEN`.
@@ -129,6 +146,7 @@ Start with the guides in `src/content/docs`:
129
146
  - **Use webctx with agents** — practical research, repo, PR, and CI workflows
130
147
  - **Search the web** — federated search and filtering
131
148
  - **Map a site** — discover pages before reading them
149
+ - **Research structured data** — browse data sources, inspect exact contracts, and execute paid queries with bounded credits
132
150
  - **Credentials** — keys and optional GitHub auth
133
151
  - **How URL reading works** — native/direct fast paths before Firecrawl
134
152
  - **How search ranking works** — URL normalization, position scoring, and provider agreement
@@ -1,8 +1,11 @@
1
1
  package app
2
2
 
3
3
  import (
4
+ "encoding/json"
4
5
  "fmt"
5
6
  "io"
7
+ "os"
8
+ "strconv"
6
9
  "strings"
7
10
 
8
11
  "github.com/amxv/webctx/internal/buildinfo"
@@ -74,6 +77,86 @@ func Run(args []string, stdout, stderr io.Writer) int {
74
77
  }
75
78
  _, _ = fmt.Fprintln(stdout, text)
76
79
  return 0
80
+ case "research":
81
+ research := ResearchDataInput{
82
+ Query: strings.Join(positional, " "),
83
+ Mode: flags["mode"],
84
+ URLs: splitCSV(flags["urls"]),
85
+ Sources: splitCSV(flags["sources"]),
86
+ Categories: splitCSV(flags["categories"]),
87
+ Groups: splitCSV(flags["groups"]),
88
+ Operations: splitCSV(flags["operations"]),
89
+ View: flags["view"],
90
+ Include: splitCSV(flags["include"]),
91
+ Raw: flags["raw"] == "true",
92
+ }
93
+ if q := flags["query"]; q != "" {
94
+ research.Query = q
95
+ }
96
+ if flag := flags["limit"]; flag != "" {
97
+ n, e := strconv.Atoi(flag)
98
+ if e != nil {
99
+ return cliDataError(stderr, dataError("invalid_limit", "--limit requires an integer."))
100
+ }
101
+ research.Limit = n
102
+ }
103
+ if flag := flags["offset"]; flag != "" {
104
+ n, e := strconv.Atoi(flag)
105
+ if e != nil {
106
+ return cliDataError(stderr, dataError("invalid_offset", "--offset requires an integer."))
107
+ }
108
+ research.Offset = n
109
+ }
110
+ result, e := ResearchData(research)
111
+ if e != nil {
112
+ return cliDataError(stderr, e)
113
+ }
114
+ writeDataJSON(stdout, result)
115
+ return 0
116
+ case "inspect":
117
+ if strings.TrimSpace(input) == "" {
118
+ return cliDataError(stderr, dataError("missing_id", "Usage: webctx inspect <operation-or-source-id> [--raw]"))
119
+ }
120
+ result, e := InspectData(InspectDataInput{ID: input, Include: splitCSV(flags["include"]), Raw: flags["raw"] == "true"})
121
+ if e != nil {
122
+ return cliDataError(stderr, e)
123
+ }
124
+ writeDataJSON(stdout, result)
125
+ return 0
126
+ case "execute":
127
+ request := ExecuteDataInput{Raw: flags["raw"] == "true", RequestID: flags["request-id"]}
128
+ if _, supplied := flags["max-credits"]; supplied {
129
+ return cliDataError(stderr, dataError("unsupported_flag", "Credit limits are managed by the server; remove --max-credits."))
130
+ }
131
+ if jsonCalls := flags["calls"]; jsonCalls != "" {
132
+ data, e := cliJSONString(jsonCalls)
133
+ if e != nil {
134
+ return cliDataError(stderr, e)
135
+ }
136
+ if e := json.Unmarshal([]byte(data), &request.Calls); e != nil {
137
+ return cliDataError(stderr, dataError("invalid_calls", "--calls must be a JSON array of {id,inputs} objects."))
138
+ }
139
+ } else if strings.TrimSpace(input) != "" {
140
+ var inputs map[string]any
141
+ if source := flags["inputs"]; source != "" {
142
+ data, e := cliJSONString(source)
143
+ if e != nil {
144
+ return cliDataError(stderr, e)
145
+ }
146
+ if e := json.Unmarshal([]byte(data), &inputs); e != nil {
147
+ return cliDataError(stderr, dataError("invalid_inputs", "--inputs must be a JSON object."))
148
+ }
149
+ }
150
+ request.Calls = []DataCall{{ID: input, Inputs: inputs}}
151
+ } else {
152
+ return cliDataError(stderr, dataError("missing_calls", "Usage: webctx execute <operation-id> --inputs JSON or --calls JSON."))
153
+ }
154
+ result, e := ExecuteData(request)
155
+ if e != nil {
156
+ return cliDataError(stderr, e)
157
+ }
158
+ writeDataJSON(stdout, result)
159
+ return 0
77
160
  default:
78
161
  _, _ = fmt.Fprintln(stderr, "Unknown tool:", tool)
79
162
  _, _ = fmt.Fprintln(stdout, usageText())
@@ -88,11 +171,19 @@ Usage:
88
171
  webctx search <query> [--exclude domain1,domain2] [--keyword phrase]
89
172
  webctx read-link <url> [--question 'what to find']
90
173
  webctx map-site <url>
174
+ webctx research [question] [--view sources|groups|operations] [--sources IDs] [--include inputs,output,examples] [--limit N] [--offset N]
175
+ webctx inspect <operation-or-source-id> [--raw]
176
+ webctx execute <operation-id> --inputs '{"field":"value"}' [--request-id ID]
177
+ webctx execute --calls '[{"id":"source/operation","inputs":{}}]'
91
178
 
92
179
  Examples:
93
180
  webctx search "next.js server components"
94
181
  webctx search "react hooks" --exclude youtube.com,vimeo.com
95
182
  webctx search "drizzle orm" --keyword "migration guide"
183
+ webctx research "podcast conversations about AI agents"
184
+ webctx research --view sources --limit 5
185
+ webctx inspect particle/podcasts/episodes/search
186
+ webctx execute particle/podcasts/episodes/search --inputs '{"semantic_search":"AI agents","limit":2}'
96
187
  webctx read-link https://docs.example.com/guide
97
188
  webctx read-link https://docs.example.com/api --question "How do auth and pagination work?"
98
189
  webctx map-site https://example.com`, version)
@@ -102,6 +193,16 @@ func parseArgs(args []string) (map[string]string, []string) {
102
193
  flags := map[string]string{}
103
194
  positional := make([]string, 0, len(args))
104
195
  for i := 0; i < len(args); i++ {
196
+ if args[i] == "--raw" {
197
+ flags["raw"] = "true"
198
+ continue
199
+ }
200
+ if strings.HasPrefix(args[i], "--") {
201
+ if k, v, ok := strings.Cut(strings.TrimPrefix(args[i], "--"), "="); ok {
202
+ flags[k] = v
203
+ continue
204
+ }
205
+ }
105
206
  if strings.HasPrefix(args[i], "--") && i+1 < len(args) {
106
207
  flags[strings.TrimPrefix(args[i], "--")] = args[i+1]
107
208
  i++
@@ -112,6 +213,38 @@ func parseArgs(args []string) (map[string]string, []string) {
112
213
  return flags, positional
113
214
  }
114
215
 
216
+ func cliJSONString(source string) (string, error) {
217
+ if !strings.HasPrefix(source, "@") {
218
+ return source, nil
219
+ }
220
+ path := strings.TrimPrefix(source, "@")
221
+ if path == "" {
222
+ return "", dataError("invalid_file", "The @file argument requires a path.")
223
+ }
224
+ b, err := os.ReadFile(path)
225
+ if err != nil {
226
+ return "", dataError("invalid_file", "Unable to read JSON input file: "+err.Error())
227
+ }
228
+ if len(b) > 1<<20 {
229
+ return "", dataError("invalid_file", "JSON input file exceeds 1 MiB.")
230
+ }
231
+ return string(b), nil
232
+ }
233
+
234
+ func writeDataJSON(out io.Writer, value any) {
235
+ bytes, err := json.MarshalIndent(value, "", " ")
236
+ if err != nil {
237
+ _, _ = fmt.Fprintln(out, "{}")
238
+ return
239
+ }
240
+ _, _ = fmt.Fprintln(out, string(bytes))
241
+ }
242
+
243
+ func cliDataError(stderr io.Writer, err error) int {
244
+ writeDataJSON(stderr, dataRawOrError(err))
245
+ return 1
246
+ }
247
+
115
248
  func splitCSV(v string) []string {
116
249
  if strings.TrimSpace(v) == "" {
117
250
  return nil
@@ -0,0 +1,214 @@
1
+ package app
2
+
3
+ import (
4
+ "context"
5
+ "crypto/rand"
6
+ "encoding/hex"
7
+ "encoding/json"
8
+ "errors"
9
+ "fmt"
10
+ "io"
11
+ "net/http"
12
+ "net/url"
13
+ "os"
14
+ "regexp"
15
+ "strconv"
16
+ "strings"
17
+ "time"
18
+ )
19
+
20
+ const dataAPIBase = "https://api.firecrawl.dev/v2"
21
+
22
+ // The server, not a client-provided tool parameter, controls this preflight ceiling.
23
+ // A production override is configured on the Origo Vercel project.
24
+ const defaultDataCreditBudget = 200
25
+ const maxDataResponseBytes int64 = 12 << 20
26
+
27
+ type DataError struct {
28
+ Code string `json:"code"`
29
+ Message string `json:"message"`
30
+ RequiresAction map[string]any `json:"requires_action,omitempty"`
31
+ Details any `json:"details,omitempty"`
32
+ }
33
+
34
+ func (e *DataError) Error() string { return e.Code + ": " + e.Message }
35
+ func dataError(code, message string) error {
36
+ return &DataError{Code: code, Message: message}
37
+ }
38
+
39
+ func dataAPIKey() (string, error) {
40
+ key := strings.TrimSpace(os.Getenv("FIRECRAWL_API_KEY"))
41
+ if key == "" {
42
+ return "", dataError("missing_credentials", "Set FIRECRAWL_API_KEY to use data source discovery and queries.")
43
+ }
44
+ return key, nil
45
+ }
46
+
47
+ func dataCreditCap() int {
48
+ value, err := strconv.Atoi(strings.TrimSpace(os.Getenv("WEBCTX_ALEXANDRIA_MAX_CREDITS")))
49
+ if err != nil || value < 1 || value > defaultDataCreditBudget {
50
+ return defaultDataCreditBudget
51
+ }
52
+ return value
53
+ }
54
+
55
+ var safeRequestID = regexp.MustCompile("^[A-Za-z0-9_-]{8,128}$")
56
+
57
+ func newRequestID() (string, error) {
58
+ var bytes [16]byte
59
+ if _, err := rand.Read(bytes[:]); err != nil {
60
+ return "", err
61
+ }
62
+ return "webctx-" + hex.EncodeToString(bytes[:]), nil
63
+ }
64
+
65
+ func doDataAPI(ctx context.Context, endpoint string, payload any, requestID string) (map[string]any, error) {
66
+ key, err := dataAPIKey()
67
+ if err != nil {
68
+ return nil, err
69
+ }
70
+ body, err := json.Marshal(payload)
71
+ if err != nil {
72
+ return nil, dataError("invalid_request", "Could not encode request as JSON: "+err.Error())
73
+ }
74
+ req, err := http.NewRequestWithContext(ctx, http.MethodPost, dataAPIBase+endpoint, strings.NewReader(string(body)))
75
+ if err != nil {
76
+ return nil, err
77
+ }
78
+ req.Header.Set("Authorization", "Bearer "+key)
79
+ req.Header.Set("Content-Type", "application/json")
80
+ if requestID != "" {
81
+ req.Header.Set("x-request-id", requestID)
82
+ }
83
+ res, err := http.DefaultClient.Do(req)
84
+ if err != nil {
85
+ return nil, &DataError{Code: "network_error", Message: "The provider request did not complete. For a paid call, retry only with the same request_id to prevent duplicate charges."}
86
+ }
87
+ defer res.Body.Close()
88
+ raw, err := io.ReadAll(io.LimitReader(res.Body, maxDataResponseBytes+1))
89
+ if err != nil {
90
+ return nil, dataError("read_error", "Could not read provider response.")
91
+ }
92
+ if int64(len(raw)) > maxDataResponseBytes {
93
+ return nil, dataError("response_too_large", "Provider response exceeded 12 MiB; use a smaller limit or pagination.")
94
+ }
95
+ var decoded map[string]any
96
+ if json.Unmarshal(raw, &decoded) != nil {
97
+ return nil, dataError("invalid_upstream_response", fmt.Sprintf("Provider returned non-JSON HTTP %d.", res.StatusCode))
98
+ }
99
+ if res.StatusCode < 200 || res.StatusCode >= 300 || decoded["success"] == false {
100
+ code := stringValue(decoded["code"])
101
+ if code == "" {
102
+ code = fmt.Sprintf("http_%d", res.StatusCode)
103
+ }
104
+ message := stringValue(decoded["error"])
105
+ if message == "" {
106
+ message = stringValue(decoded["message"])
107
+ }
108
+ if message == "" {
109
+ message = "The provider rejected the request."
110
+ }
111
+ if len(message) > 800 {
112
+ message = message[:800]
113
+ }
114
+ out := &DataError{Code: code, Message: message}
115
+ if action, ok := decoded["requiresAction"].(map[string]any); ok {
116
+ if target := stringValue(action["url"]); target != "" {
117
+ if u, e := url.Parse(target); e == nil && (u.Scheme == "https" || u.Scheme == "http") && u.Host != "" {
118
+ out.RequiresAction = map[string]any{"url": target}
119
+ }
120
+ }
121
+ }
122
+ return nil, out
123
+ }
124
+ return decoded, nil
125
+ }
126
+
127
+ func nativeDataCall(ctx context.Context, provider, capability string, options map[string]any, requestID string) (map[string]any, error) {
128
+ request := map[string]any{"alexandria": map[string]any{
129
+ "provider": provider, "capability": capability, "options": options,
130
+ }}
131
+ return doDataAPI(ctx, "/scrape", request, requestID)
132
+ }
133
+
134
+ func extractNativeData(response map[string]any) (map[string]any, error) {
135
+ data, ok := response["data"].(map[string]any)
136
+ if !ok {
137
+ return nil, dataError("invalid_upstream_response", "The response has no data object.")
138
+ }
139
+ items, ok := data["alexandria"].([]any)
140
+ if !ok || len(items) == 0 {
141
+ return nil, dataError("invalid_upstream_response", "The response contains no Alexandria results.")
142
+ }
143
+ item, ok := items[0].(map[string]any)
144
+ if !ok {
145
+ return nil, dataError("invalid_upstream_response", "The source returned an invalid result.")
146
+ }
147
+ return item, nil
148
+ }
149
+
150
+ func freeDataBrowse(ctx context.Context, options map[string]any) (map[string]any, map[string]any, error) {
151
+ response, err := nativeDataCall(ctx, "firecrawl", "find-tools", options, "")
152
+ if err != nil {
153
+ return nil, nil, err
154
+ }
155
+ item, err := extractNativeData(response)
156
+ if err != nil {
157
+ return nil, nil, err
158
+ }
159
+ details, ok := item["data"].(map[string]any)
160
+ if !ok {
161
+ return nil, nil, dataError("invalid_upstream_response", "The catalogue returned no browsable entries.")
162
+ }
163
+ return details, response, nil
164
+ }
165
+
166
+ func defaultDataContext() (context.Context, context.CancelFunc) {
167
+ return context.WithTimeout(context.Background(), 45*time.Second)
168
+ }
169
+
170
+ func dataRawOrError(err error) map[string]any {
171
+ var typed *DataError
172
+ if errors.As(err, &typed) {
173
+ return map[string]any{"error": typed}
174
+ }
175
+ return map[string]any{"error": map[string]any{"code": "internal_error", "message": err.Error()}}
176
+ }
177
+
178
+ // DataErrorResult preserves actionable failures in both CLI and MCP JSON.
179
+ func DataErrorResult(err error) map[string]any { return dataRawOrError(err) }
180
+
181
+ func listField(value any) []any {
182
+ switch v := value.(type) {
183
+ case []any:
184
+ return v
185
+ case []string:
186
+ out := make([]any, 0, len(v))
187
+ for _, s := range v {
188
+ out = append(out, s)
189
+ }
190
+ return out
191
+ default:
192
+ return nil
193
+ }
194
+ }
195
+
196
+ func stringField(value any) string { s, _ := value.(string); return s }
197
+ func mapField(value any) map[string]any { m, _ := value.(map[string]any); return m }
198
+ func numberField(value any) (float64, bool) {
199
+ switch n := value.(type) {
200
+ case float64:
201
+ return n, true
202
+ case float32:
203
+ return float64(n), true
204
+ case int:
205
+ return float64(n), true
206
+ case int64:
207
+ return float64(n), true
208
+ case json.Number:
209
+ f, e := n.Float64()
210
+ return f, e == nil
211
+ default:
212
+ return 0, false
213
+ }
214
+ }
@@ -0,0 +1,112 @@
1
+ package app
2
+
3
+ import (
4
+ "regexp"
5
+ "strings"
6
+ )
7
+
8
+ var backtickedIdentifier = regexp.MustCompile("`([A-Za-z][A-Za-z0-9_/-]{1,30})`")
9
+
10
+ // Upstream examples can legitimately be absent. Offer a clearly marked,
11
+ // locally validated sample input without claiming it was published by the
12
+ // provider or that it was executed.
13
+ func contractExamples(contract map[string]any) any {
14
+ if published := contract["examples"]; published != nil {
15
+ if len(listField(published)) > 0 {
16
+ return published
17
+ }
18
+ if len(mapField(published)) > 0 {
19
+ return published
20
+ }
21
+ }
22
+ input, needed := nativeInputTemplate(contract)
23
+ fieldInfo := map[string]map[string]any{}
24
+ for _, v := range listField(contract["options"]) {
25
+ if field := mapField(v); field != nil {
26
+ fieldInfo[stringField(field["name"])] = field
27
+ }
28
+ }
29
+ for _, name := range needed {
30
+ field := fieldInfo[name]
31
+ if value, ok := sampleFieldValue(name, field); ok {
32
+ input[name] = value
33
+ }
34
+ }
35
+ // An illustrative example should not accidentally fetch huge datasets
36
+ // when copied verbatim. Users may adjust the inputs freely.
37
+ for _, name := range []string{"limit", "per_page", "page_size", "max_results", "count"} {
38
+ if n, ok := numberField(input[name]); ok && n > 5 {
39
+ input[name] = 5
40
+ }
41
+ }
42
+ if _, _, err := parseOperationID(stringField(contract["id"])); err != nil {
43
+ return nil
44
+ }
45
+ validation := validateDataInputs(contract, input)
46
+ example := map[string]any{
47
+ "kind": "generated_from_inspected_schema",
48
+ "note": "Illustrative Origo-generated inputs; not a published provider example or an executed query.",
49
+ "inputs": input,
50
+ "valid_input_shape": validation == nil,
51
+ }
52
+ if validation != nil {
53
+ example["note"] = "Illustrative input template; fill required fields and check source-specific constraints before execution."
54
+ example["needs_input"] = needed
55
+ } else {
56
+ example["next"] = map[string]any{"tool": "execute", "arguments": map[string]any{
57
+ "calls": []any{map[string]any{"id": contract["id"], "inputs": input}},
58
+ }}
59
+ }
60
+ return []any{example}
61
+ }
62
+
63
+ func sampleFieldValue(name string, field map[string]any) (any, bool) {
64
+ if field == nil {
65
+ return nil, false
66
+ }
67
+ if values := listField(field["oneOf"]); len(values) > 0 {
68
+ for _, v := range values {
69
+ if v != nil && v != "" {
70
+ return v, true
71
+ }
72
+ }
73
+ }
74
+ if sample, ok := field["example"]; ok && sample != nil {
75
+ return sample, true
76
+ }
77
+ if samples := listField(field["examples"]); len(samples) > 0 && samples[0] != nil {
78
+ return samples[0], true
79
+ }
80
+ if v, ok := field["default"]; ok && v != nil {
81
+ return v, true
82
+ }
83
+ info := stringField(field["about"]) + " " + stringField(field["description"])
84
+ lower := strings.ToLower(name + " " + info)
85
+ switch {
86
+ case strings.Contains(name, "series_id") && strings.Contains(info, "CPIAUCSL"):
87
+ return "CPIAUCSL", true
88
+ case strings.Contains(name, "series_id") && strings.Contains(info, "UNRATE"):
89
+ return "UNRATE", true
90
+ case strings.Contains(name, "series_id") && strings.Contains(info, "GDP"):
91
+ return "GDP", true
92
+ case strings.Contains(lower, "semantic_search"):
93
+ return "AI agents", true
94
+ case name == "query" || name == "search" || name == "keyword_search":
95
+ return "example topic", true
96
+ case name == "url" || strings.HasSuffix(name, "_url"):
97
+ return "https://example.com", true
98
+ }
99
+ // Backticked descriptions sometimes contain sample codes or IDs.
100
+ if strings.Contains(lower, "id") || strings.Contains(lower, "code") {
101
+ for _, match := range backtickedIdentifier.FindAllStringSubmatch(info, 10) {
102
+ candidate := match[1]
103
+ if strings.Contains(candidate, "/") || strings.Contains(candidate, "_") {
104
+ continue
105
+ }
106
+ if len(candidate) >= 3 && !strings.EqualFold(candidate, "required") {
107
+ return candidate, true
108
+ }
109
+ }
110
+ }
111
+ return nil, false
112
+ }