reqon-dsl 0.2.0

package/docusaurus/docs/core-concepts/schemas.md

@@ -0,0 +1,353 @@
+---
+sidebar_position: 5
+---
+
+# Schemas
+
+**Schemas** define data shapes for validation and pattern matching. They're used to validate responses, route data based on structure, and document expected data formats.
+
+## Basic Syntax
+
+```vague
+schema SchemaName {
+  field: type,
+  optionalField: type?,
+  nestedField: {
+    subField: type
+  }
+}
+```
+
+## Field Types
+
+| Type | Description | Example |
+|------|-------------|---------|
+| `string` | Text value | `"hello"` |
+| `number` | Numeric value | `42`, `3.14` |
+| `boolean` | True or false | `true`, `false` |
+| `date` | Date/datetime | `"2024-01-20"` |
+| `array` | Array of values | `[1, 2, 3]` |
+| `object` | Nested object | `{ a: 1 }` |
+| `any` | Any type | anything |
+| `null` | Null value | `null` |
+
+## Optional Fields
+
+Use `?` suffix for optional fields:
+
+```vague
+schema User {
+  id: string,
+  name: string,
+  email: string?,
+  phone: string?
+}
+```
+
+## Typed Arrays
+
+Specify array element types:
+
+```vague
+schema UserList {
+  users: array<User>,
+  total: number
+}
+
+schema Order {
+  id: string,
+  items: array<{
+    productId: string,
+    quantity: number,
+    price: number
+  }>
+}
+```
+
+## Nested Schemas
+
+Define complex nested structures:
+
+```vague
+schema Invoice {
+  id: string,
+  customer: {
+    id: string,
+    name: string,
+    address: {
+      street: string,
+      city: string,
+      country: string
+    }
+  },
+  lineItems: array<{
+    description: string,
+    amount: number
+  }>,
+  total: number
+}
+```
+
+## Schema References
+
+Reference other schemas:
+
+```vague
+schema Address {
+  street: string,
+  city: string,
+  postalCode: string,
+  country: string
+}
+
+schema Customer {
+  id: string,
+  name: string,
+  billingAddress: Address,
+  shippingAddress: Address?
+}
+```
+
+## Using Schemas for Validation
+
+Validate data against schemas:
+
+```vague
+action ValidateResponse {
+  get "/users"
+
+  for user in response.users {
+    validate user {
+      assume .id is string,
+      assume .name is string,
+      assume .email is string
+    }
+    store user -> validUsers { key: .id }
+  }
+}
+```
+
+## Using Schemas for Pattern Matching
+
+Route data based on schema matches:
+
+```vague
+schema SuccessResponse {
+  data: any,
+  status: string
+}
+
+schema ErrorResponse {
+  error: string,
+  code: number
+}
+
+schema RateLimitResponse {
+  error: string,
+  retryAfter: number
+}
+
+action HandleResponse {
+  get "/data"
+
+  match response {
+    SuccessResponse -> store response.data -> data { key: .id },
+    RateLimitResponse -> retry { delay: response.retryAfter * 1000 },
+    ErrorResponse -> abort response.error,
+    _ -> abort "Unknown response format"
+  }
+}
+```
+
+## Schema Matching Rules
+
+Schemas match when:
+1. All required fields are present
+2. Field types match
+3. Optional fields, if present, match their types
+
+```vague
+schema StrictUser {
+  id: string,     // Required
+  name: string,   // Required
+  email: string?  // Optional
+}
+
+// Matches: { id: "1", name: "John" }
+// Matches: { id: "1", name: "John", email: "john@example.com" }
+// Does NOT match: { id: 1, name: "John" }  // id is number, not string
+// Does NOT match: { id: "1" }  // missing name
+```
+
+## Type Checking with `is`
+
+Use `is` for inline type checking:
+
+```vague
+validate response {
+  assume .items is array,
+  assume .count is number,
+  assume .status is string
+}
+```
+
+## Combining Schemas
+
+Use schemas in complex match patterns:
+
+```vague
+schema PaginatedResponse {
+  data: array,
+  meta: {
+    page: number,
+    totalPages: number,
+    hasNext: boolean
+  }
+}
+
+schema EmptyResponse {
+  data: array,
+  meta: {
+    total: number
+  }
+}
+
+action FetchPaginated {
+  get "/items" { paginate: page(page, 100) }
+
+  match response {
+    PaginatedResponse where response.meta.hasNext == true -> continue,
+    PaginatedResponse -> store response.data -> items { key: .id },
+    EmptyResponse -> skip,
+    _ -> abort "Unexpected response"
+  }
+}
+```
+
+## Schema Inheritance (via Vague)
+
+Extend schemas using Vague's composition:
+
+```vague
+schema BaseEntity {
+  id: string,
+  createdAt: date,
+  updatedAt: date
+}
+
+schema User {
+  ...BaseEntity,
+  name: string,
+  email: string
+}
+
+schema Order {
+  ...BaseEntity,
+  customerId: string,
+  total: number
+}
+```
+
+For advanced schema features, see the [Vague documentation](https://github.com/mcclowes/vague).
+
+## Best Practices
+
+### Define Schemas for API Responses
+
+```vague
+mission APISync {
+  schema UserResponse {
+    users: array<User>,
+    pagination: {
+      page: number,
+      total: number
+    }
+  }
+
+  schema User {
+    id: string,
+    name: string,
+    email: string
+  }
+}
+```
+
+### Use Schemas for Error Handling
+
+```vague
+schema APIError {
+  error: {
+    message: string,
+    code: string
+  }
+}
+
+schema AuthError {
+  error: {
+    message: string,
+    code: string
+  },
+  code: number  // HTTP status code
+}
+
+action Fetch {
+  get "/data"
+
+  match response {
+    AuthError where .code == 401 -> jump RefreshToken then retry,
+    APIError -> abort response.error.message,
+    _ -> continue
+  }
+}
+```
+
+### Document Expected Formats
+
+Schemas serve as documentation:
+
+```vague
+// XeroInvoice represents an invoice from Xero API
+schema XeroInvoice {
+  InvoiceID: string,
+  InvoiceNumber: string,
+  Type: string,  // ACCREC or ACCPAY
+  Contact: {
+    ContactID: string,
+    Name: string
+  },
+  LineItems: array<{
+    Description: string,
+    Quantity: number,
+    UnitAmount: number,
+    LineAmount: number
+  }>,
+  Total: number,
+  Status: string  // DRAFT, SUBMITTED, AUTHORISED, PAID
+}
+```
+
+### Keep Schemas Close to Usage
+
+Define schemas in the same mission where they're used:
+
+```vague
+mission XeroSync {
+  // Schema definitions at the top
+  schema XeroInvoice { /* ... */ }
+  schema XeroContact { /* ... */ }
+
+  // Then sources, stores, actions...
+}
+```
+
+Or use multi-file missions to organize:
+
+```
+missions/xero-sync/
+├── mission.vague
+├── schemas/
+│   ├── invoice.vague
+│   └── contact.vague
+└── actions/
+    └── fetch.vague
+```

package/docusaurus/docs/core-concepts/sources.md

@@ -0,0 +1,339 @@
+---
+sidebar_position: 3
+---
+
+# Sources
+
+A **Source** defines an API endpoint that your mission connects to. Sources configure authentication, base URLs, rate limiting, and other HTTP client options.
+
+## Basic Syntax
+
+```vague
+source SourceName {
+  auth: authType,
+  base: "https://api.example.com"
+}
+```
+
+## Authentication Types
+
+| Type | Description |
+|------|-------------|
+| `none` | No authentication |
+| `bearer` | Bearer token in Authorization header |
+| `basic` | HTTP Basic authentication |
+| `api_key` | API key in header or query |
+| `oauth2` | OAuth 2.0 with token refresh |
+
+### No Authentication
+
+```vague
+source PublicAPI {
+  auth: none,
+  base: "https://jsonplaceholder.typicode.com"
+}
+```
+
+### Bearer Token
+
+```vague
+source GitHub {
+  auth: bearer,
+  base: "https://api.github.com"
+}
+```
+
+Credentials are provided via CLI or config:
+
+```json
+{
+  "GitHub": {
+    "type": "bearer",
+    "token": "ghp_xxxxxxxxxxxx"
+  }
+}
+```
+
+### API Key
+
+```vague
+source StripeAPI {
+  auth: api_key,
+  base: "https://api.stripe.com/v1"
+}
+```
+
+```json
+{
+  "StripeAPI": {
+    "type": "api_key",
+    "key": "sk_live_xxxx",
+    "header": "Authorization",
+    "prefix": "Bearer"
+  }
+}
+```
+
+Or in query parameter:
+
+```json
+{
+  "StripeAPI": {
+    "type": "api_key",
+    "key": "sk_live_xxxx",
+    "query": "api_key"
+  }
+}
+```
+
+### Basic Authentication
+
+```vague
+source LegacyAPI {
+  auth: basic,
+  base: "https://legacy.example.com"
+}
+```
+
+```json
+{
+  "LegacyAPI": {
+    "type": "basic",
+    "username": "user",
+    "password": "pass"
+  }
+}
+```
+
+### OAuth 2.0
+
+```vague
+source Xero {
+  auth: oauth2,
+  base: "https://api.xero.com/api.xro/2.0"
+}
+```
+
+```json
+{
+  "Xero": {
+    "type": "oauth2",
+    "clientId": "your-client-id",
+    "clientSecret": "your-client-secret",
+    "accessToken": "current-access-token",
+    "refreshToken": "current-refresh-token",
+    "tokenUrl": "https://identity.xero.com/connect/token",
+    "scopes": ["accounting.transactions.read"]
+  }
+}
+```
+
+Reqon automatically refreshes tokens when they expire.
+
+## OpenAPI Spec Sources
+
+Load source configuration from an OpenAPI specification:
+
+```vague
+source Petstore from "./petstore.yaml" {
+  auth: bearer,
+  validateResponses: true
+}
+```
+
+Benefits:
+- Base URL extracted from spec
+- Operations available via `call` syntax
+- Response validation against schemas
+
+See [OpenAPI Integration](../category/openapi-integration) for details.
+
+## Source Options
+
+### Custom Headers
+
+```vague
+source CustomAPI {
+  auth: bearer,
+  base: "https://api.example.com",
+  headers: {
+    "X-Custom-Header": "value",
+    "Accept": "application/json"
+  }
+}
+```
+
+### Rate Limiting
+
+```vague
+source RateLimitedAPI {
+  auth: bearer,
+  base: "https://api.example.com",
+  rateLimit: {
+    requestsPerMinute: 60,
+    strategy: "pause"
+  }
+}
+```
+
+Strategies:
+- `pause` - Wait when limit reached
+- `throttle` - Slow down requests
+- `fail` - Throw error when limit reached
+
+### Circuit Breaker
+
+Prevent cascading failures:
+
+```vague
+source UnreliableAPI {
+  auth: bearer,
+  base: "https://flaky-api.example.com",
+  circuitBreaker: {
+    failureThreshold: 5,
+    resetTimeout: 30000,
+    successThreshold: 2
+  }
+}
+```
+
+See [Circuit Breaker](../http/circuit-breaker) for details.
+
+### Timeout
+
+```vague
+source SlowAPI {
+  auth: bearer,
+  base: "https://slow-api.example.com",
+  timeout: 60000
+}
+```
+
+## Using Sources
+
+Sources are automatically selected when making requests:
+
+```vague
+mission MultiSource {
+  source Primary { auth: bearer, base: "https://primary.example.com" }
+  source Secondary { auth: bearer, base: "https://secondary.example.com" }
+
+  action FetchFromPrimary {
+    // Uses first source by default
+    get "/data"
+  }
+
+  action FetchFromSecondary {
+    // Explicitly use secondary source
+    get Secondary "/data"
+  }
+}
+```
+
+### Default Source
+
+The first defined source is the default:
+
+```vague
+mission Example {
+  source API { auth: bearer, base: "https://api.example.com" }
+
+  action Fetch {
+    get "/users"  // Uses API source
+  }
+}
+```
+
+### Named Source Reference
+
+Prefix requests with source name:
+
+```vague
+action FetchFromMultiple {
+  get Primary "/users"
+  get Secondary "/users"
+}
+```
+
+## Source Variables
+
+Use environment variables in source definitions:
+
+```vague
+source API {
+  auth: bearer,
+  base: env("API_BASE_URL")
+}
+```
+
+## Multiple Environments
+
+Pattern for handling different environments:
+
+```vague
+mission Sync {
+  source API {
+    auth: bearer,
+    base: match env("ENVIRONMENT") {
+      "production" => "https://api.example.com",
+      "staging" => "https://staging.api.example.com",
+      _ => "http://localhost:3000"
+    }
+  }
+}
+```
+
+## Best Practices
+
+### Use Descriptive Names
+
+```vague
+// Good
+source XeroAccounting { }
+source QuickBooksOnline { }
+source StripePayments { }
+
+// Avoid
+source API1 { }
+source Source { }
+```
+
+### Configure Appropriate Timeouts
+
+```vague
+// For fast APIs
+source FastAPI {
+  timeout: 5000  // 5 seconds
+}
+
+// For slow/bulk APIs
+source BulkExportAPI {
+  timeout: 300000  // 5 minutes
+}
+```
+
+### Always Use Rate Limiting for Production
+
+```vague
+source ProductionAPI {
+  auth: bearer,
+  base: "https://api.example.com",
+  rateLimit: {
+    requestsPerMinute: 100,
+    strategy: "pause"
+  }
+}
+```
+
+### Enable Circuit Breakers for Unreliable Sources
+
+```vague
+source ThirdPartyAPI {
+  auth: bearer,
+  base: "https://third-party.example.com",
+  circuitBreaker: {
+    failureThreshold: 5,
+    resetTimeout: 30000
+  }
+}
+```