reqon-dsl 0.2.0 → 0.4.0

package/docusaurus/docs/intro.md

@@ -1,90 +0,0 @@
----
-sidebar_position: 1
-slug: /
----
-
-# Introduction to Reqon
-
-Reqon is a **declarative DSL framework for building fetch, map, validate pipelines**. It's designed to make API data synchronization simple, reliable, and maintainable.
-
-## What is Reqon?
-
-Reqon allows you to define data pipelines using a clean, readable syntax. Instead of writing imperative code to fetch data, transform it, validate it, and store it, you declare *what* you want to happen, and Reqon handles *how* it happens.
-
-```vague
-mission SyncCustomers {
-  source API { auth: bearer, base: "https://api.example.com" }
-  store customers: file("customers")
-
-  action FetchCustomers {
-    get "/customers" { paginate: offset(page, 100) }
-    for customer in response.data {
-      map customer -> Customer {
-        id: .id,
-        name: .name,
-        email: .email
-      }
-      store customer -> customers { key: .id }
-    }
-  }
-
-  run FetchCustomers
-}
-```
-
-## Why Reqon?
-
-### Declarative by Design
-
-Traditional data pipelines require writing boilerplate code for HTTP requests, pagination, retries, error handling, and more. Reqon abstracts these concerns into a clean DSL, letting you focus on your business logic.
-
-### Built-in Best Practices
-
-- **Automatic pagination** - Handle offset, page number, or cursor-based pagination
-- **Retry with backoff** - Exponential, linear, or constant backoff strategies
-- **Rate limiting** - Respects API rate limits automatically
-- **Circuit breaker** - Prevents cascading failures
-- **Incremental sync** - Only fetch changes since last run
-
-### Extensible Architecture
-
-Reqon is built on [Vague](https://github.com/mcclowes/vague), a general-purpose expression DSL. This means you get powerful expression syntax for transformations and validations, while Reqon adds the execution semantics for data pipelines.
-
-## Architecture
-
-```
-┌─────────────────────────────────────────────┐
-│                  Reqon                      │
-│  ┌─────────────────────────────────────┐    │
-│  │  Missions, Actions, Pipelines       │    │
-│  │  HTTP, Stores, Scheduling           │    │
-│  └─────────────────────────────────────┘    │
-│                    │                        │
-│  ┌─────────────────▼─────────────────┐      │
-│  │              Vague                 │     │
-│  │  Lexer, Parser, Expressions        │     │
-│  │  Match, Schema, Types              │     │
-│  └───────────────────────────────────┘      │
-└─────────────────────────────────────────────┘
-```
-
-- **Vague** provides the DSL layer (lexer, parser, expression syntax)
-- **Reqon** provides the runtime/framework with execution semantics
-
-For expression syntax, pattern matching, and schema definitions, refer to the [Vague documentation](https://github.com/mcclowes/vague).
-
-## Key Concepts
-
-| Concept | Description |
-|---------|-------------|
-| **Mission** | A complete data pipeline with sources, stores, and actions |
-| **Source** | An API endpoint with authentication configuration |
-| **Store** | A storage backend (memory, file, SQL, NoSQL) |
-| **Action** | A sequence of steps that process data |
-| **Step** | A single operation (fetch, map, validate, store, etc.) |
-
-## Next Steps
-
-- [Getting Started](./getting-started) - Install Reqon and run your first mission
-- [Core Concepts](./category/core-concepts) - Learn about missions, actions, and more
-- [Examples](./examples) - See real-world examples

package/docusaurus/docs/openapi/loading-specs.md

@@ -1,305 +0,0 @@
----
-sidebar_position: 2
----
-
-# Loading OpenAPI Specs
-
-Reqon can load OpenAPI specifications from files or URLs.
-
-## Local Files
-
-### YAML Format
-
-```vague
-source API from "./openapi.yaml" {
-  auth: bearer
-}
-```
-
-### JSON Format
-
-```vague
-source API from "./openapi.json" {
-  auth: bearer
-}
-```
-
-### Relative Paths
-
-```vague
-// Relative to mission file
-source API from "./specs/api.yaml"
-
-// Absolute path
-source API from "/home/user/specs/api.yaml"
-```
-
-## Remote URLs
-
-### Public Specs
-
-```vague
-source Petstore from "https://petstore3.swagger.io/api/v3/openapi.json" {
-  auth: none
-}
-```
-
-### Authenticated Specs
-
-```vague
-source PrivateAPI from "https://api.company.com/openapi.json" {
-  auth: bearer,
-  specAuth: {
-    type: "bearer",
-    token: env("SPEC_TOKEN")
-  }
-}
-```
-
-## Spec Caching
-
-Reqon caches resolved specs:
-
-```vague
-// First run: downloads and caches
-source API from "https://api.example.com/openapi.json"
-
-// Subsequent runs: uses cache
-```
-
-### Cache Location
-
-```
-.vague-data/
-└── oas-cache/
-    └── api.example.com-openapi.json
-```
-
-### Force Refresh
-
-```bash
-# Clear cache
-rm -rf .vague-data/oas-cache/
-
-# Or use --no-cache flag
-reqon mission.vague --no-oas-cache
-```
-
-## Spec Structure
-
-### Minimum Required
-
-```yaml
-openapi: 3.0.0
-info:
-  title: My API
-  version: 1.0.0
-servers:
-  - url: https://api.example.com
-paths:
-  /items:
-    get:
-      operationId: listItems
-      responses:
-        '200':
-          description: Success
-```
-
-### Full Example
-
-```yaml
-openapi: 3.0.0
-info:
-  title: E-commerce API
-  version: 2.0.0
-  description: API for managing products and orders
-
-servers:
-  - url: https://api.example.com/v2
-    description: Production
-  - url: https://staging.api.example.com/v2
-    description: Staging
-
-paths:
-  /products:
-    get:
-      operationId: listProducts
-      summary: List all products
-      parameters:
-        - name: limit
-          in: query
-          schema:
-            type: integer
-            default: 20
-        - name: offset
-          in: query
-          schema:
-            type: integer
-            default: 0
-      responses:
-        '200':
-          description: Product list
-          content:
-            application/json:
-              schema:
-                type: array
-                items:
-                  $ref: '#/components/schemas/Product'
-
-  /products/{id}:
-    get:
-      operationId: getProduct
-      parameters:
-        - name: id
-          in: path
-          required: true
-          schema:
-            type: string
-      responses:
-        '200':
-          description: Product details
-          content:
-            application/json:
-              schema:
-                $ref: '#/components/schemas/Product'
-
-components:
-  schemas:
-    Product:
-      type: object
-      properties:
-        id:
-          type: string
-        name:
-          type: string
-        price:
-          type: number
-      required:
-        - id
-        - name
-```
-
-## Server Selection
-
-### Default Server
-
-Uses first server in spec:
-
-```yaml
-servers:
-  - url: https://api.example.com  # Used by default
-  - url: https://staging.example.com
-```
-
-### Override Base URL
-
-```vague
-source API from "./spec.yaml" {
-  auth: bearer,
-  base: "https://custom.example.com"  # Overrides spec
-}
-```
-
-### Environment-Based
-
-```vague
-source API from "./spec.yaml" {
-  auth: bearer,
-  base: env("API_BASE_URL")
-}
-```
-
-## Reference Resolution
-
-Reqon resolves `$ref` references:
-
-```yaml
-# References within same file
-$ref: '#/components/schemas/Product'
-
-# External file references
-$ref: './schemas/product.yaml'
-```
-
-## Validation
-
-### On Load
-
-Reqon validates the spec structure:
-
-```
-[Reqon] Loading OpenAPI spec from ./api.yaml
-[Reqon] Spec valid: 15 operations found
-```
-
-### Invalid Specs
-
-```
-[Reqon] Error: Invalid OpenAPI spec
-  - Missing required field: info.version
-  - Invalid path: /items must start with /
-```
-
-## Multiple Specs
-
-### Per-Source
-
-```vague
-mission MultiAPI {
-  source Products from "./products.yaml" { auth: bearer }
-  source Orders from "./orders.yaml" { auth: bearer }
-  source Users from "./users.yaml" { auth: oauth2 }
-
-  action SyncAll {
-    call Products.listProducts
-    call Orders.listOrders
-    call Users.listUsers
-  }
-}
-```
-
-### Shared Components
-
-If specs share schemas, use a single bundled spec:
-
-```vague
-source API from "./bundled-api.yaml" { auth: bearer }
-```
-
-## Troubleshooting
-
-### "Spec not found"
-
-Check file path:
-
-```bash
-ls -la ./openapi.yaml
-```
-
-### "Invalid spec"
-
-Validate externally:
-
-```bash
-npx swagger-cli validate ./openapi.yaml
-```
-
-### "Operation not found"
-
-Check operation ID matches exactly:
-
-```yaml
-paths:
-  /items:
-    get:
-      operationId: listItems  # Must match call API.listItems
-```
-
-### Network Issues
-
-For remote specs:
-
-```bash
-# Test connectivity
-curl -I https://api.example.com/openapi.json
-```

package/docusaurus/docs/openapi/operation-calls.md

@@ -1,314 +0,0 @@
----
-sidebar_position: 3
----
-
-# Operation Calls
-
-Call OpenAPI operations using the `call` syntax with operation IDs.
-
-## Basic Syntax
-
-```vague
-call SourceName.operationId
-call SourceName.operationId { options }
-```
-
-## Simple Calls
-
-### GET Operations
-
-```vague
-// OpenAPI: GET /pets with operationId: listPets
-call Petstore.listPets
-
-// OpenAPI: GET /pets/{petId} with operationId: getPetById
-call Petstore.getPetById { params: { petId: "123" } }
-```
-
-### POST Operations
-
-```vague
-// OpenAPI: POST /pets with operationId: addPet
-call Petstore.addPet {
-  body: {
-    name: "Fluffy",
-    tag: "cat"
-  }
-}
-```
-
-### PUT/PATCH Operations
-
-```vague
-// PUT - full replacement
-call API.updateItem {
-  params: { id: "123" },
-  body: { name: "New Name", status: "active" }
-}
-
-// PATCH - partial update
-call API.patchItem {
-  params: { id: "123" },
-  body: { status: "inactive" }
-}
-```
-
-### DELETE Operations
-
-```vague
-call API.deleteItem {
-  params: { id: "123" }
-}
-```
-
-## Parameters
-
-### Path Parameters
-
-For `/pets/{petId}`:
-
-```vague
-call Petstore.getPetById {
-  params: { petId: "123" }
-}
-// Generates: GET /pets/123
-```
-
-### Query Parameters
-
-For `/pets?limit=10&status=available`:
-
-```vague
-call Petstore.listPets {
-  params: {
-    limit: 10,
-    status: "available"
-  }
-}
-```
-
-### Combined Parameters
-
-```vague
-call API.listUserOrders {
-  params: {
-    userId: "123",      // Path: /users/{userId}/orders
-    status: "pending",  // Query: ?status=pending
-    limit: 50           // Query: ?limit=50
-  }
-}
-// Generates: GET /users/123/orders?status=pending&limit=50
-```
-
-## Request Body
-
-### Simple Body
-
-```vague
-call API.createItem {
-  body: {
-    name: "Test Item",
-    price: 29.99
-  }
-}
-```
-
-### Dynamic Body
-
-```vague
-for item in items {
-  call API.createItem {
-    body: {
-      name: item.name,
-      price: item.price,
-      metadata: {
-        source: "sync",
-        timestamp: now()
-      }
-    }
-  }
-}
-```
-
-### From Variable
-
-```vague
-map data -> Payload {
-  name: .name,
-  status: "active"
-}
-
-call API.createItem { body: data }
-```
-
-## Headers
-
-### Custom Headers
-
-```vague
-call API.listItems {
-  headers: {
-    "X-Request-ID": uuid(),
-    "Accept-Language": "en-US"
-  }
-}
-```
-
-### Tenant Headers
-
-```vague
-call Xero.listInvoices {
-  headers: {
-    "Xero-Tenant-Id": env("XERO_TENANT_ID")
-  }
-}
-```
-
-## Pagination with Operations
-
-```vague
-call API.listItems {
-  params: { limit: 100 },
-  paginate: offset(offset, 100),
-  until: length(response.items) == 0
-}
-```
-
-### Cursor Pagination
-
-```vague
-call API.listItems {
-  paginate: cursor(cursor, 100, "meta.nextCursor"),
-  until: response.meta.nextCursor == null
-}
-```
-
-## Combining Options
-
-```vague
-call API.searchItems {
-  params: {
-    query: "test",
-    limit: 50
-  },
-  headers: {
-    "X-Custom": "value"
-  },
-  paginate: cursor(after, 50, "pageInfo.endCursor"),
-  until: response.pageInfo.hasNextPage == false,
-  retry: {
-    maxAttempts: 3,
-    backoff: exponential
-  }
-}
-```
-
-## Response Handling
-
-```vague
-action FetchWithHandling {
-  call API.getItem { params: { id: itemId } }
-
-  match response {
-    { data: item } -> store item -> items { key: .id },
-    { error: e } -> abort e,
-    _ -> abort "Unexpected response"
-  }
-}
-```
-
-## Operation Chaining
-
-```vague
-action CreateAndFetch {
-  // Create
-  call API.createItem {
-    body: { name: "New Item" }
-  }
-
-  // response.id from creation
-  call API.getItem {
-    params: { id: response.id }
-  }
-
-  store response -> items { key: .id }
-}
-```
-
-## Error Handling
-
-```vague
-call API.riskyOperation { params: { id: "123" } }
-
-match response {
-  { code: 400 } -> abort "Invalid request",
-  { code: 401 } -> jump RefreshToken then retry,
-  { code: 404 } -> skip,
-  { code: 429 } -> retry { delay: 60000 },
-  { code: 500 } -> retry { maxAttempts: 3 },
-  _ -> continue
-}
-```
-
-## Dynamic Operation Calls
-
-### Based on Condition
-
-```vague
-action SmartSync {
-  call API.checkItem { params: { id: item.id } }
-
-  match response {
-    { exists: false } -> {
-      call API.createItem { body: item }
-    },
-    { exists: true, needsUpdate: true } -> {
-      call API.updateItem { params: { id: item.id }, body: item }
-    },
-    _ -> continue
-  }
-}
-```
-
-## Best Practices
-
-### Match Operation IDs
-
-```yaml
-# In OpenAPI spec
-operationId: listUsers  # camelCase recommended
-```
-
-```vague
-call API.listUsers  # Match exactly
-```
-
-### Use Descriptive Operations
-
-```yaml
-# Good
-operationId: createInvoice
-operationId: getInvoiceById
-operationId: listInvoicesByCustomer
-
-# Avoid
-operationId: post1
-operationId: get2
-```
-
-### Handle All Response Codes
-
-```vague
-call API.operation
-
-match response {
-  { code: 200 } -> continue,
-  { code: 201 } -> continue,
-  { code: 204 } -> continue,
-  { code: 400 } -> abort "Bad request",
-  { code: 401 } -> jump RefreshAuth then retry,
-  { code: 404 } -> skip,
-  { code: 500 } -> retry { maxAttempts: 3 },
-  _ -> abort "Unexpected response"
-}
-```