@chaprola/mcp-server 1.6.4 → 1.8.0

package/references/cookbook.md

@@ -19,56 +19,67 @@ POST /run {userid, project, name: "REPORT", primary_file: "STAFF", record: 1}
 |-------|---------|--------------------------|
 | R1–R20 | HULDRA elements (parameters) | No — HULDRA overwrites these |
 | R21–R40 | HULDRA objectives (error metrics) | No — HULDRA reads these |
-| R41–R50 | Scratch space | **Yes — always use R41–R50 for DEFINE VARIABLE** |
+| R41–R99 | Scratch space | **Yes — always use R41–R99 for DEFINE VARIABLE** |
 
-For non-HULDRA programs, R1–R40 are technically available but using R41–R50 is a good habit.
+For non-HULDRA programs, R1–R40 are technically available but using R41–R99 is a good habit.
 
-## PRINT: Output from U Buffer
+## PRINT: Preferred Output Methods
 
-```
-PRINT 0   — output the ENTIRE U buffer contents, then clear it
-PRINT N   — output exactly N characters from U buffer (no clear)
+**Concatenation (preferred):**
+```chaprola
+PRINT P.name + " — " + P.department + " — $" + R41
+PRINT "Total: " + R42
+PRINT P.last_name                  // single field, auto-trimmed
+PRINT "Hello from Chaprola!"      // literal string
 ```
 
-Use `PRINT N` when you've placed data at specific positions and want clean output without trailing garbage. Use `PRINT 0` for quick output of everything.
+- String literals are copied as-is.
+- P./S./U./X. fields are auto-trimmed (trailing spaces removed).
+- R-variables print as integers when no fractional part, otherwise as floats.
+- Concatenation auto-flushes the line.
 
+**U buffer output (for fixed-width columnar reports only):**
 ```chaprola
-MOVE "Hello" U.1 5
-PRINT 5    // Outputs "Hello" — exactly 5 chars, no trailing spaces
+CLEAR U
+MOVE P.name U.1 20
+PUT sal INTO U.22 10 D 2
+PRINT 0                            // output entire U buffer, then clear
 ```
 
 ## Hello World (no data file)
 
 ```chaprola
-MOVE "Hello from Chaprola!" U.1 20
-PRINT 0
+PRINT "Hello from Chaprola!"
 END
 ```
 
 ## Loop Through All Records
 
+Always start programs with `OPEN PRIMARY` to declare the data file. This makes the program self-documenting and eliminates the need for `primary_format` on compile.
+
 ```chaprola
-DEFINE VARIABLE rec R1
+OPEN PRIMARY "STAFF" 0
+DEFINE VARIABLE rec R41
 LET rec = 1
 100 SEEK rec
     IF EOF GOTO 900
-    MOVE BLANKS U.1 40
-    MOVE P.name U.1 8
-    MOVE P.salary U.12 6
-    PRINT 0
+    PRINT P.name + " — " + P.salary
     LET rec = rec + 1
     GOTO 100
 900 END
 ```
 
+Compile without `primary_format` — the compiler reads the format from `OPEN PRIMARY`:
+```bash
+POST /compile {userid, project, name: "REPORT", source: "OPEN PRIMARY \"STAFF\" 0\n..."}
+```
+
 ## Filtered Report
 
 ```chaprola
 GET sal FROM P.salary
-IF sal LT 80000 GOTO 200    // skip low earners
-MOVE P.name U.1 8
-PUT sal INTO U.12 10 D 0    // D=dollar format
-PRINT 0
+IF sal LT 80000 GOTO 200           // skip low earners
+PRINT P.name + " — " + R41
 200 LET rec = rec + 1
 ```
 
@@ -76,10 +87,10 @@ PRINT 0
 
 ```chaprola
 OPEN "DEPARTMENTS" 0
-FIND match FROM S.dept_code 3 USING P.dept_code
-IF match EQ 0 GOTO 200      // no match
-READ match                   // load matched secondary record
-MOVE S.dept_name U.12 15    // now accessible
+FIND match FROM S.dept_code USING P.dept_code
+IF match EQ 0 GOTO 200             // no match
+READ match                          // load matched secondary record
+PRINT P.name + " — " + S.dept_name
 ```
 
 Compile with both formats so the compiler resolves fields from both files:
@@ -105,14 +116,47 @@ IF EQUAL U.200 U.180 12 GOTO 200   // match — jump to handler
 ## Read-Modify-Write (UPDATE)
 
 ```chaprola
-READ match                   // load record
-GET bal FROM S.balance       // read current value
-LET bal = bal + amt          // modify
-PUT bal INTO S.balance 8 F 0 // write back to S memory
-WRITE match                  // flush to disk
-CLOSE                        // flush all at end
+READ match                          // load record
+GET bal FROM S.balance              // read current value
+LET bal = bal + amt                 // modify
+PUT bal INTO S.balance F 0          // write back to S memory (length auto-filled)
+WRITE match                         // flush to disk
+CLOSE                               // flush all at end
+```
+
+## Date Arithmetic
+
+```chaprola
+GET DATE R41 FROM X.primary_modified   // when was file last changed?
+GET DATE R42 FROM X.utc_time           // what time is it now?
+LET R43 = R42 - R41                    // difference in seconds
+LET R43 = R43 / 86400                  // convert to days
+IF R43 GT 30 PRINT "WARNING: file is over 30 days old" ;
+```
+
+## Get Current User
+
+```chaprola
+PRINT "Logged in as: " + X.username
 ```
 
+## System Text Properties (X.)
+
+Access system metadata by property name — no numeric positions needed:
+
+| Property | Description |
+|----------|-------------|
+| `X.year` | Year (four digits) |
+| `X.julian` | Julian date (1–366) |
+| `X.hour` | Hour (military time) |
+| `X.minute` | Minute (0–59) |
+| `X.username` | Authenticated user |
+| `X.record_num` | Primary file record number |
+| `X.utc_time` | UTC datetime (ISO 8601) |
+| `X.elapsed` | Elapsed execution time |
+| `X.primary_modified` | Primary file Last-Modified |
+| `X.secondary_modified` | Secondary file Last-Modified |
+
 ## Async for Large Datasets
 
 ```bash
@@ -140,9 +184,7 @@ SEEK 1
     GOTO 300
 200 GET cardlvl FROM P.level
     IF cardlvl NE lvl GOTO 300           // filter by level param
-    MOVE P.kanji U.1 4
-    MOVE P.reading U.6 10
-    PRINT 0
+    PRINT P.kanji + " — " + P.reading
 300 LET rec = rec + 1
     SEEK rec
     GOTO 100
@@ -208,7 +250,7 @@ POST /query {
 | `I` | Integer (right-justified) | `  1234` |
 | `E` | Scientific notation | `1.23E+03` |
 
-Syntax: `PUT R1 INTO U.30 10 D 2` (R-var, location, width, format, decimals)
+Syntax: `PUT R41 INTO P.salary D 2` (R-var, field name, format, decimals — length auto-filled)
 
 ## Common Field Widths
 
@@ -227,19 +269,19 @@ Use these when sizing MOVE lengths and U buffer positions.
 
 | Prefix | Description |
 |--------|-------------|
-| `P` | Primary data file (current record) |
-| `S` | Secondary data file (current record) |
+| `P` | Primary data file — use field names: `P.salary`, `P.name` |
+| `S` | Secondary data file — use field names: `S.dept`, `S.emp_id` |
 | `U` | User buffer (scratch for output) |
-| `X` | System text (date, time, filenames) |
+| `X` | System text — use property names: `X.username`, `X.utc_time` |
 
 ## Math Intrinsics
 
 ```chaprola
-LET R2 = EXP R1      // e^R1
-LET R2 = LOG R1      // ln(R1)
-LET R2 = SQRT R1     // √R1
-LET R2 = ABS R1      // |R1|
-LET R3 = POW R1 R2   // R1^R2
+LET R42 = EXP R41      // e^R41
+LET R42 = LOG R41      // ln(R41)
+LET R42 = SQRT R41     // √R41
+LET R42 = ABS R41      // |R41|
+LET R43 = POW R41 R42  // R41^R42
 ```
 
 ## Import-Download: URL → Dataset (Parquet, Excel, CSV, JSON)
@@ -281,7 +323,7 @@ HULDRA finds the best parameter values for a mathematical model by minimizing th
 |-------|---------|-------------|
 | R1–R20 | **Elements** (parameters to optimize) | HULDRA sets these before each VM run |
 | R21–R40 | **Objectives** (error metrics) | Your program computes and stores these |
-| R41–R50 | **Scratch space** | Your program uses these for temp variables |
+| R41–R99 | **Scratch space** | Your program uses these for temp variables |
 
 ### Complete Example: Fit a Linear Model
 
@@ -521,6 +563,106 @@ LET PRED = PRED + R2
 
 All patterns follow the same loop structure: SEEK records, GET fields, compute PRED, accumulate `(PRED - OBS)^2` in SSR, store SSR in R21 at the end.
 
+## Parameterized Report Endpoint
+
+Combine QUERY with PARAM to build a dynamic JSON API from a published program. QUERY output is a .QR file (read-only). R20 = matched record count. Missing PARAMs are silently replaced with blank (string) or 0.0 (numeric) — check param warnings in the response for diagnostics.
+
+```chaprola
+// STAFF_BY_DEPT.CS — call via /report?publisher=admin&program=STAFF_BY_DEPT&dept=Engineering
+QUERY STAFF FIELDS name, salary, title INTO dept_staff WHERE department EQ PARAM.dept ORDER BY salary DESC
+OPEN SECONDARY dept_staff
+DEFINE name = S.name
+DEFINE salary = S.salary
+DEFINE title = S.title
+PRINT "["
+READ SECONDARY
+IF FINI GOTO done
+100 PRINT TRIM "{\"name\":\"" + name + "\",\"title\":\"" + title + "\",\"salary\":" + salary + "}"
+    READ SECONDARY
+    IF FINI GOTO done
+    PRINT ","
+    GOTO 100
+done.
+PRINT "]"
+STOP
+```
+
+Publish with: `POST /publish {userid, project, name: "STAFF_BY_DEPT", primary_file: "STAFF", acl: "authenticated"}`
+Call with: `POST /report?publisher=admin&program=STAFF_BY_DEPT&dept=Engineering`
+
+## Cross-File Filtering (IN/NOT IN)
+
+Use QUERY with NOT IN to find records in one file that don't appear in another. This is the flashcard review pattern — find unreviewed cards by excluding already-reviewed ones. One IN/NOT IN per QUERY.
+
+If the IN-file doesn't exist (e.g., new user with no progress), NOT IN treats it as empty — all records pass. This is correct: "kanji not in (nothing)" = all kanji.
+
+```chaprola
+// Step 1: Get the list of kanji the user has already reviewed
+QUERY progress INTO reviewed WHERE username EQ PARAM.username
+
+// Step 2: Filter flashcards to only those NOT in the reviewed set
+// If progress doesn't exist (new user), all flashcards are returned
+QUERY flashcards INTO new_cards WHERE kanji NOT IN reviewed.kanji
+
+// Step 3: Loop through unreviewed cards
+OPEN SECONDARY new_cards
+READ SECONDARY
+IF FINI GOTO empty
+100 PRINT S.kanji + " — " + S.reading + " — " + S.meaning
+    READ SECONDARY
+    IF FINI GOTO done
+    GOTO 100
+empty.
+PRINT "All cards reviewed!"
+done.
+STOP
+```
+
+## Public Apps: Use /report, Not /query
+
+Site keys require BAA signing. For public-facing web apps, use published reports (`/report`) instead of `/query` for all read operations. `/report` is public — no auth or BAA needed.
+
+**Pattern:** Move data logic into Chaprola programs (QUERY, TABULATE), publish them, and call `/report` from the frontend. Reserve site keys for write operations (`/insert-record`) only.
+
+```javascript
+// GOOD: public report — no auth needed, works for anyone
+const url = `${API}/report?userid=myapp&project=data&name=RESULTS&poll_id=${id}`;
+const response = await fetch(url);
+
+// BAD: /query with site key — fails if BAA not signed (403 Forbidden)
+const response = await fetch(`${API}/query`, {
+  headers: { 'Authorization': `Bearer ${SITE_KEY}` },
+  body: JSON.stringify({ userid: 'myapp', project: 'data', file: 'votes', where: [...] })
+});
+```
+
+**Why this is better:** The program runs server-side with full access. The frontend gets clean output. No API keys exposed for reads. QUERY + TABULATE in a program replaces client-side pivot logic.
+
+## Chart Data with TABULATE
+
+Use TABULATE to produce CSV output suitable for charting. This example cross-tabulates mortality data by cause and year.
+
+```chaprola
+// Generate a pivot table of death counts by cause and year
+TABULATE mortality SUM deaths FOR cause VS year WHERE year GE "2020" INTO trends
+
+// Output as CSV — ready for any charting library
+PRINT TABULATE trends AS CSV
+```
+
+Output:
+```
+cause,2020,2021,2022,2023,total
+Heart disease,690882,693021,699659,702000,2785562
+Cancer,598932,602350,608371,611000,2420653
+...
+```
+
+For web apps, use JSON output instead:
+```chaprola
+PRINT TABULATE trends AS JSON
+```
+
 ### Agent Workflow Summary
 
 1. **Inspect** — Call `/format` to see what fields exist

package/references/gotchas.md

@@ -11,21 +11,60 @@ LET temp = qty + bonus
 LET result = price * temp
 ```
 
+### No built-in functions
+There are NO functions with parentheses. No STR(), INT(), ABS(), LEN(), TRIM(), SUBSTR(), CONCAT(), FORMAT(), TOSTRING(), etc.
+- To convert number to text for output: `PRINT "Total: " + R41`
+- To write number to a field: `PUT R41 INTO P.salary D 2`
+- To convert text to number: `GET R41 FROM P.salary`
+- To clear a field: `MOVE BLANKS P.notes` or `CLEAR U`
+
+### Use field names, not numeric positions
+Always use `P.salary`, `S.dept`, `X.username` — not `P.63 10`, `S.30 4`, `X.28 8`. The compiler auto-fills positions and lengths from the format file.
+```chaprola
+// PREFERRED: field names — readable, resilient to format changes
+GET R41 FROM P.salary
+PRINT P.name + " — " + P.department
+MOVE X.username U.1
+
+// AVOID: numeric positions — fragile, unreadable
+GET R41 FROM P.63 10
+```
+
+### Use PRINT concatenation, not MOVE + PRINT 0
+```chaprola
+// PREFERRED: direct concatenation
+PRINT P.name + " earns $" + R41
+
+// AVOID: old MOVE-to-buffer pattern
+MOVE BLANKS U.1 80
+MOVE P.name U.1 20
+PUT R41 INTO U.22 10 D 2
+PRINT 0
+```
+
+### Use CLEAR, not MOVE BLANKS for full regions
+```chaprola
+CLEAR U                             // clear entire user buffer
+CLEAR P                             // clear entire primary region
+CLEAR S                             // clear entire secondary region
+MOVE BLANKS P.notes                 // clear a single field (length auto-filled)
+```
+
 ### IF EQUAL compares a literal to a location
-Cannot compare two memory locations. Copy to U buffer first.
+Cannot compare two memory locations directly. Copy to U buffer first.
 ```chaprola
 MOVE P.txn_type U.76 6
 IF EQUAL "CREDIT" U.76 GOTO 200
 ```
 
-### MOVE length must match field width
-`MOVE P.name U.1 20` copies 20 chars starting at the field — if `name` is 8 chars wide, the extra 12 bleed into adjacent fields. Always match the format file width.
+### MOVE literal auto-pads to field width
+`MOVE "Jones" P.name` auto-fills the rest of the field with blanks. No need to clear first.
 
 ### DEFINE VARIABLE names must not collide with field names
-If the format has a `balance` field, don't `DEFINE VARIABLE balance R3`. Use `bal` instead. The compiler confuses the alias with the field name.
+If the format has a `balance` field, don't `DEFINE VARIABLE balance R41`. Use `bal` instead. The compiler confuses the alias with the field name.
 
 ### R-variables are floating point
-All R1–R50 are 64-bit floats. `7 / 2 = 3.5`. Use PUT with `I` format to display as integer.
+All R1–R99 are 64-bit floats. `7 / 2 = 3.5`. Use PUT with `I` format to display as integer.
 
 ### Statement numbers are labels, not line numbers
 Only number lines that are GOTO/CALL targets. Don't number every line.
@@ -36,6 +75,13 @@ Always check `IF match EQ 0` after FIND before calling READ.
 ### PRINT 0 clears the U buffer
 After PRINT 0, the buffer is empty. No need to manually clear between prints unless reusing specific positions.
 
+### GET DATE / PUT DATE — no FOR keyword needed with property names
+```chaprola
+GET DATE R41 FROM X.utc_time           // correct — length auto-filled
+GET DATE R42 FROM X.primary_modified   // correct — length auto-filled
+PUT DATE R41 INTO U.1 20              // U buffer needs explicit length
+```
+
 ## Import
 
 ### Field widths come from the longest value
@@ -74,8 +120,8 @@ Always CLOSE before END if you wrote to the secondary file. Unflushed writes are
 
 ## HULDRA Optimization
 
-### Use R41–R50 for scratch variables, not R1–R20
-R1–R20 are reserved for HULDRA elements. R21–R40 are reserved for objectives. Your VALUE program's DEFINE VARIABLE declarations must use R41–R50 only.
+### Use R41–R99 for scratch variables, not R1–R20
+R1–R20 are reserved for HULDRA elements. R21–R40 are reserved for objectives. Your VALUE program's DEFINE VARIABLE declarations must use R41–R99 only.
 ```chaprola
 // WRONG: DEFINE VARIABLE counter R1  (HULDRA will overwrite this)
 // RIGHT: DEFINE VARIABLE counter R41

package/references/ref-apps.md

@@ -0,0 +1,195 @@
+# Building Apps on Chaprola — Architecture Reference
+
+## Overview
+
+Chaprola is a backend for frontend apps. Your React, Vue, Svelte, or Laravel app calls `api.chaprola.org` directly from the browser. No proxy server, no middleware, no infrastructure to manage.
+
+## Two Architectures
+
+### 1. Single-Owner App (simplest)
+
+One Chaprola account owns all the data. The app uses a **site key** locked to its domain. All users see the same data.
+
+**Use cases:** dashboards, public reports, internal tools, data viewers, portfolio sites.
+
+```
+Browser → React App → api.chaprola.org (site key in Authorization header)
+```
+
+**Setup:**
+1. Register a Chaprola account: `POST /register`
+2. Sign the BAA if handling health data: `POST /sign-baa`
+3. Create a site key locked to your domain:
+   ```json
+   POST /create-site-key
+   {"userid": "myapp", "origin": "https://myapp.example.com", "label": "production"}
+   ```
+   Response: `{"site_key": "site_a1b2c3..."}`
+4. Use the site key in your frontend:
+   ```javascript
+   const resp = await fetch('https://api.chaprola.org/query', {
+     method: 'POST',
+     headers: {
+       'Authorization': 'Bearer site_a1b2c3...',
+       'Content-Type': 'application/json'
+     },
+     body: JSON.stringify({ userid: 'myapp', project: 'main', file: 'products', where: [] })
+   })
+   ```
+
+**BAA requirement:** All data endpoints (including `/query`) require BAA signing. For public-facing apps that haven't signed a BAA, use published reports (`/report`) for reads instead of `/query`. Move filtering and aggregation into Chaprola programs (QUERY + TABULATE commands), publish them, and call `/report` from the frontend. Reserve the site key for write-only operations like `/insert-record`.
+
+**Security model:** The site key is checked against the `Origin` HTTP header, which browsers set automatically. This prevents other websites from using your key (CORS-level protection). However, Origin headers are trivially spoofable from non-browser clients (curl, Postman, scripts). Anyone who extracts the site key from your JavaScript has full access to the account's data. **Use this pattern only for public or semi-public data** — dashboards, product catalogs, published reports. For private data, use the multi-user pattern (each user authenticates individually) or the enterprise proxy pattern.
+
+### 2. Multi-User App (each user has their own account)
+
+Each app user registers their own Chaprola account. The app stores their API key in the browser session. Each user has their own data silo.
+
+**Use cases:** SaaS apps, multi-tenant platforms, apps where users own their data.
+
+```
+Browser → React App → api.chaprola.org (user's own API key)
+```
+
+**Setup:**
+1. Your app's registration form calls Chaprola directly:
+   ```javascript
+   // User signs up
+   const resp = await fetch('https://api.chaprola.org/register', {
+     method: 'POST',
+     headers: { 'Content-Type': 'application/json' },
+     body: JSON.stringify({ username: 'alice', passcode: 'their-secure-passcode' })
+   })
+   const { api_key } = await resp.json()
+   sessionStorage.setItem('chaprola_key', api_key)
+   ```
+
+2. Your app's login form:
+   ```javascript
+   const resp = await fetch('https://api.chaprola.org/login', {
+     method: 'POST',
+     headers: { 'Content-Type': 'application/json' },
+     body: JSON.stringify({ username: 'alice', passcode: 'their-secure-passcode' })
+   })
+   const { api_key } = await resp.json()
+   sessionStorage.setItem('chaprola_key', api_key)
+   ```
+
+3. All subsequent API calls use the user's key:
+   ```javascript
+   const key = sessionStorage.getItem('chaprola_key')
+   const resp = await fetch('https://api.chaprola.org/query', {
+     method: 'POST',
+     headers: {
+       'Authorization': `Bearer ${key}`,
+       'Content-Type': 'application/json'
+     },
+     body: JSON.stringify({ userid: 'alice', project: 'mydata', file: 'tasks', where: [] })
+   })
+   ```
+
+**Security model:** Each user authenticates individually. User A cannot access User B's data (userid enforcement). No shared secrets in the frontend. API keys are per-user, stored in the browser session.
+
+**Team data sharing:** If users need to share a project, the project owner creates an `access.json` file listing writers. See the auth reference for details.
+
+## Which Architecture Should I Use?
+
+| Question | Single-Owner | Multi-User |
+|----------|:---:|:---:|
+| One person/org owns all the data? | Yes | |
+| Users create accounts and own their data? | | Yes |
+| Public dashboard or report viewer? | Yes | |
+| SaaS with multiple tenants? | | Yes |
+| Internal tool for a small team? | Yes | |
+| App where privacy between users matters? | | Yes |
+| Data is sensitive or private? | No — use Multi-User or Enterprise | Yes |
+
+**Rule of thumb:** If you'd be uncomfortable with someone viewing all the data in that Chaprola account, don't use the single-owner pattern. The site key will be in your JavaScript source — assume it's public.
+
+## Enterprise Customers
+
+If your enterprise requires that API keys never touch the browser:
+
+Run your own backend proxy. Your React/Laravel frontend authenticates users through your own auth system (OAuth, SSO, SAML). Your backend holds the Chaprola API key server-side and proxies requests.
+
+```
+User → Enterprise App → Enterprise Backend (holds API key) → api.chaprola.org
+```
+
+This is the same pattern used with Stripe, Twilio, and other APIs. Chaprola does not provide a managed proxy — your infrastructure team owns this layer.
+
+**Most apps do not need this.** The site key + per-user auth model handles the vast majority of use cases without any backend infrastructure.
+
+## Site Keys Reference
+
+Site keys are API keys locked to a specific browser origin. They prevent your key from being used on other websites.
+
+```json
+// Create
+POST /create-site-key
+{"userid": "myapp", "origin": "https://myapp.example.com", "label": "production"}
+// Response: {"site_key": "site_...", "origin": "https://myapp.example.com"}
+
+// List
+POST /list-site-keys
+{"userid": "myapp"}
+
+// Delete
+POST /delete-site-key
+{"userid": "myapp", "site_key": "site_..."}
+```
+
+**Key facts:**
+- Format: `site_` prefix + 64 hex chars
+- Locked to one origin (exact match on the `Origin` HTTP header)
+- Never expire — persist until explicitly deleted
+- Not affected by `/login` (login rotates the main API key, not site keys)
+- Same permissions as the main API key for that account
+- Create multiple site keys for different environments (dev, staging, production)
+
+## Deploying Your App
+
+For static React/Vue/Svelte apps, Chaprola can host them:
+
+```json
+POST /app/deploy
+{"userid": "myapp", "project": "main", "app_name": "myapp"}
+```
+
+Your app is served at `https://chaprola.org/apps/{userid}/{app_name}/`. Or deploy anywhere (Vercel, Netlify, S3) and use a site key locked to that domain.
+
+## Common Patterns
+
+### Import data, then query it
+```javascript
+// Import
+await fetch('https://api.chaprola.org/import', {
+  method: 'POST',
+  headers: { 'Authorization': `Bearer ${key}`, 'Content-Type': 'application/json' },
+  body: JSON.stringify({ userid: 'myapp', project: 'main', name: 'products', data: [...] })
+})
+
+// Query
+const resp = await fetch('https://api.chaprola.org/query', {
+  method: 'POST',
+  headers: { 'Authorization': `Bearer ${key}`, 'Content-Type': 'application/json' },
+  body: JSON.stringify({ userid: 'myapp', project: 'main', file: 'products', where: [{ field: 'category', op: 'eq', value: 'electronics' }] })
+})
+const { records } = await resp.json()
+```
+
+### Insert a record
+```javascript
+await fetch('https://api.chaprola.org/insert-record', {
+  method: 'POST',
+  headers: { 'Authorization': `Bearer ${key}`, 'Content-Type': 'application/json' },
+  body: JSON.stringify({ userid: 'myapp', project: 'main', file: 'tasks', record: { title: 'New task', status: 'open', due: '2026-04-15' } })
+})
+```
+
+### Run a published report
+```javascript
+// No auth needed for public reports
+const resp = await fetch('https://api.chaprola.org/report?userid=myapp&project=main&name=DASHBOARD&format=html')
+const html = await resp.text()
+```

package/references/ref-gotchas.md

@@ -2,13 +2,18 @@
 
 ## Language
 - **No parentheses in LET.** `LET result = price * qty` only. For `price * (qty + bonus)`: use `LET temp = qty + bonus` then `LET result = price * temp`.
+- **No built-in functions.** No STR(), INT(), ABS(), LEN(), TRIM(), SUBSTR(), etc. Use PUT/GET/MOVE/PRINT concatenation instead.
+- **Use field names, not numeric positions.** `P.salary` not `P.63 10`. `X.username` not `X.15 10`. The compiler auto-fills positions and lengths.
+- **Use PRINT concatenation for output.** `PRINT P.name + " — " + R41` instead of MOVE-to-U-buffer + PRINT 0.
+- **Use CLEAR for full regions.** `CLEAR U` instead of `MOVE BLANKS U.1 65536`. Also `CLEAR P`, `CLEAR S`.
+- **MOVE literal auto-pads.** `MOVE "Jones" P.name` fills the rest of the field with blanks. No need to clear first.
 - **IF EQUAL compares literal to location.** To compare two locations, copy both to U buffer first.
-- **MOVE length must match field width.** If `name` is 8 chars wide, `MOVE P.name U.1 20` bleeds into adjacent fields.
 - **DEFINE VARIABLE names must not collide with field names.** If format has `balance`, don't `DEFINE VARIABLE balance R41`.
 - **R-variables are 64-bit floats.** `7 / 2 = 3.5`. Use PUT with `I` format for integer display.
 - **FIND returns 0 on no match.** Always check `IF match EQ 0` before READ.
 - **PRINT 0 clears the U buffer.** No need to manually clear between prints.
 - **Statement numbers are labels, not line numbers.** Only number GOTO/CALL targets.
+- **GET DATE / PUT DATE need no FOR keyword** with property names: `GET DATE R41 FROM X.utc_time`
 
 ## API
 - **NEVER use `/import` to change field widths.** `/import` REPLACES existing data. Use `/alter` to widen/narrow fields while preserving data. See `chaprola://ref/schema`.

package/references/ref-huldra.md

@@ -8,7 +8,7 @@ HULDRA finds optimal parameter values for a mathematical model by minimizing the
 |-------|---------|-------------|
 | R1–R20 | Elements (parameters to optimize) | HULDRA sets before each run |
 | R21–R40 | Objectives (error metrics) | Your program computes these |
-| R41–R50 | Scratch space | Your program's temp variables |
+| R41–R99 | Scratch space | Your program's temp variables |
 
 ## POST /optimize
 ```json

package/references/ref-import.md

@@ -44,3 +44,40 @@ Optional `format: "fhir"` for FHIR JSON reconstruction.
 ## POST /download
 `{userid, project, file, type}` → `{download_url, expires_in, size_bytes}`
 Type: `data`, `format`, `source`, `proc`, `output`.
+
+## sort_columns
+
+Reorder fields and physically sort data at import time, creating a self-indexing (clustered) data file.
+
+```json
+POST /import {
+  "userid": "...", "project": "...", "name": "STAFF",
+  "sort_columns": ["username", "kanji"],
+  "data": [...]
+}
+```
+
+- Reorders fields so sort columns come first in the format file
+- Sorts data by those columns at import time
+- Marks `KEY:1`, `KEY:2` in `.F` metadata
+- Enables binary search on the clustered key during QUERY
+
+## split_by
+
+Split a dataset into per-group data files at import time. One `.DA` file per distinct value of the split field, with a shared `.F` format file.
+
+```json
+POST /import {
+  "userid": "...", "project": "...", "name": "orders",
+  "split_by": "region",
+  "data": [...]
+}
+```
+
+- Creates one `.DA` per distinct value of the split field
+- Shared `.F` format file
+- Response includes `files_created` and `groups` object
+
+## 5GB File Size Limit
+
+Maximum 5GB per data file. Returns 413 error if exceeded. Use `split_by` for larger datasets.