@chrismo/superkit 1.0.0

package/docs/recipes/character.spq

@@ -0,0 +1,57 @@
+-- sk_include string.spq
+
+op skdoc_seq: (
+  cast(
+    {name:"sk_seq",
+     type:"op",
+     desc:"Generates a sequence 0..n-1 (like generate_series).",
+     args:[{name:"n",desc:"Number of elements to generate"}],
+     examples:[{i:"sk_seq(3)",o:"0, 1, 2"}] }, <skdoc>)
+)
+
+op sk_seq(n): (
+  split(sk_pad_left('', '0', n), '')
+  | unnest this
+  | count
+  | values count - 1
+)
+
+fn skdoc_hex_digits(): (
+  cast(
+    {name:"sk_hex_digits",
+     type:"func",
+     desc:"Returns the 16 hex digit characters as an array.",
+     args:[],
+     examples:[] }, <skdoc>)
+)
+
+fn sk_hex_digits(): (
+  split("0123456789abcdef", "")
+)
+
+fn skdoc_chr(): (
+  cast(
+    {name:"sk_chr",
+     type:"func",
+     desc:"Converts an integer (0-127) to its ASCII character.",
+     args:[{name:"code",desc:"ASCII code (0-127)"}],
+     examples:[{i:"sk_chr(65)",o:"'A'"}] }, <skdoc>)
+)
+
+fn sk_chr(code): (
+  let d = sk_hex_digits() |
+  hex(f'{d[code/16]}{d[code%16]}')::string
+)
+
+fn skdoc_alpha(): (
+  cast(
+    {name:"sk_alpha",
+     type:"func",
+     desc:"Converts 1-26 to A-Z.",
+     args:[{name:"n",desc:"Number 1-26"}],
+     examples:[{i:"sk_alpha(3)",o:"'C'"}] }, <skdoc>)
+)
+
+fn sk_alpha(n): (
+  sk_chr(64 + n)
+)

package/docs/recipes/escape.md

@@ -0,0 +1,159 @@
+---
+title: Escape
+layout: default
+nav_order: 7
+parent: Recipes
+---
+
+# Escape Recipes
+
+Source: `escape.spq`
+
+Functions for safely escaping values for CSV, TSV, and shell contexts, plus
+patterns for safe text ingestion from shell pipelines.
+
+---
+
+## sk_csv_field
+
+Escapes a string for use as a CSV field per RFC 4180. Wraps in double quotes
+and doubles internal quotes when the value contains commas, quotes, or
+newlines. Plain values pass through unchanged.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `s` | The string to escape |
+
+```supersql
+sk_csv_field('plain')
+-- => 'plain'
+
+sk_csv_field('hello, world')
+-- => quoted and wrapped
+```
+
+**Implementation:**
+
+```supersql
+fn sk_csv_field(s): (
+  grep("[,\"\n]", s)
+    ? f"\"{replace(s, "\"", "\"\"")}\""
+    : s
+)
+```
+
+---
+
+## sk_csv_row
+
+Builds a CSV row from an array of values. Each element is cast to string and
+escaped with sk_csv_field, then joined with commas.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `arr` | Array of values to format as a CSV row |
+
+**Implementation:**
+
+```supersql
+fn sk_csv_row(arr): (
+  join([unnest arr | values sk_csv_field(cast(this, <string>))], ",")
+)
+```
+
+---
+
+## sk_shell_quote
+
+Wraps a string in POSIX shell single quotes. Internal single quotes are escaped
+so the result is safe for shell interpolation. Protects against injection of `$`,
+backticks, and other shell metacharacters.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `s` | The string to quote |
+
+```supersql
+sk_shell_quote('hello world')
+-- => single-quoted string
+
+sk_shell_quote('has $var')
+-- => single-quoted, $ not expanded
+```
+
+**Implementation:**
+
+```supersql
+fn sk_shell_quote(s): (
+  f"'{replace(s, "'", "'\\''")}'"
+)
+```
+
+---
+
+## sk_tsv_field
+
+Escapes a value for use in a TSV field. Casts to string, then replaces literal
+tab and newline characters with their backslash-escaped forms.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `s` | The value to escape |
+
+**Implementation:**
+
+```supersql
+fn sk_tsv_field(s): (
+  replace(replace(cast(s, <string>), "\t", "\\t"), "\n", "\\n")
+)
+```
+
+---
+
+## Shell Patterns for Safe Text Ingestion
+
+The key insight: never interpolate untrusted text into a SuperQL string literal.
+Pipe raw text through `super` with `-i line` and let the serializer handle
+escaping. These patterns work from any language that can spawn a subprocess.
+
+### safe_text_to_record
+
+Pipe raw text into super to build a record without string interpolation.
+
+```bash
+echo "$text" | super -s -i line -c "values {body: this}" -
+```
+
+### safe_text_to_string
+
+Pipe raw text through super to get a properly escaped SUP string literal.
+
+```bash
+echo "$text" | super -s -i line -c "values this" -
+```
+
+### safe_multiline_to_record
+
+Collapse multiline text into a single record field.
+
+```bash
+echo "$text" | super -s -i line \
+  -c 'aggregate s:=collect(this) | values {body: join(s, "\n")}' -
+```
+
+### safe_append_to_sup_file
+
+Append a timestamped record with raw text to a `.sup` file.
+
+```bash
+echo "$text" | super -s -i line \
+  -c "values {ts: now(), body: this}" - >> data.sup
+```

package/docs/recipes/escape.spq

@@ -0,0 +1,102 @@
+fn skdoc_csv_field(): (
+  cast(
+    {name:"sk_csv_field",
+     type:"func",
+     desc:"Escapes a string for use as a CSV field per RFC 4180. Wraps in double quotes and doubles internal quotes when the value contains commas, quotes, or newlines. Plain values pass through unchanged.",
+     args:[{name:"s",desc:"The string to escape"}],
+     examples:[{i:"sk_csv_field('plain')",o:"'plain'"}
+               {i:"sk_csv_field('hello, world')",o:"quoted and wrapped"}
+               {i:"sk_csv_field('has newline')",o:"quoted and wrapped"}] }, <skdoc>)
+)
+
+fn sk_csv_field(s): (
+  grep("[,\"\n]", s)
+    ? f"\"{replace(s, "\"", "\"\"")}\""
+    : s
+)
+
+fn skdoc_csv_row(): (
+  cast(
+    {name:"sk_csv_row",
+     type:"func",
+     desc:"Builds a CSV row from an array of values. Each element is cast to string and escaped with sk_csv_field, then joined with commas.",
+     args:[{name:"arr",desc:"Array of values to format as a CSV row"}],
+     examples:[{i:"sk_csv_row(arr) where arr has commas",o:"fields with commas get quoted"}] }, <skdoc>)
+)
+
+fn sk_csv_row(arr): (
+  join([unnest arr | values sk_csv_field(cast(this, <string>))], ",")
+)
+
+fn skdoc_shell_quote(): (
+  cast(
+    {name:"sk_shell_quote",
+     type:"func",
+     desc:"Wraps a string in POSIX shell single quotes. Internal single quotes are escaped so the result is safe for shell interpolation. Protects against injection of $, backticks, and other shell metacharacters.",
+     args:[{name:"s",desc:"The string to quote"}],
+     examples:[{i:"sk_shell_quote('hello world')",o:"single-quoted string"}
+               {i:"sk_shell_quote('has $var')",o:"single-quoted, $ not expanded"}] }, <skdoc>)
+)
+
+fn sk_shell_quote(s): (
+  f"'{replace(s, "'", "'\\'')}'"
+)
+
+fn skdoc_tsv_field(): (
+  cast(
+    {name:"sk_tsv_field",
+     type:"func",
+     desc:"Escapes a value for use in a TSV field. Casts to string, then replaces literal tab and newline characters with their backslash-escaped forms.",
+     args:[{name:"s",desc:"The value to escape"}],
+     examples:[{i:"sk_tsv_field('col1\\tcol2')",o:"tabs replaced with literal \\t"}
+               {i:"sk_tsv_field('line1\\nline2')",o:"newlines replaced with literal \\n"}] }, <skdoc>)
+)
+
+fn sk_tsv_field(s): (
+  replace(replace(cast(s, <string>), "\t", "\\t"), "\n", "\\n")
+)
+
+-- Shell patterns for safe text ingestion
+-- The key insight: never interpolate untrusted text into a SuperQL string literal.
+-- Pipe raw text through super with -i line and let the serializer handle escaping.
+-- These patterns work from any language that can spawn a subprocess.
+
+fn skdoc_safe_text_to_record(): (
+  cast(
+    {name:"safe_text_to_record",
+     type:"shell",
+     desc:"Pipe raw text into super to build a record without string interpolation. The text never passes through shell expansion or manual escaping. Works from any language via subprocess.",
+     args:[],
+     snippet:'echo "$text" | super -s -i line -c "values {body: this}" -',
+     examples:[{i:"text with quotes and $pecial chars",o:"safely embedded in a .sup record"}] }, <skdoc>)
+)
+
+fn skdoc_safe_text_to_string(): (
+  cast(
+    {name:"safe_text_to_string",
+     type:"shell",
+     desc:"Pipe raw text through super to get a properly escaped SUP string literal. Useful when you need the escaped value for embedding in other SUP data.",
+     args:[],
+     snippet:'echo "$text" | super -s -i line -c "values this" -',
+     examples:[{i:"She said hello and backslash",o:"properly escaped SUP string"}] }, <skdoc>)
+)
+
+fn skdoc_safe_multiline_to_record(): (
+  cast(
+    {name:"safe_multiline_to_record",
+     type:"shell",
+     desc:"Collapse multiline text into a single record field. Each line is collected then joined with newline characters. The text is never interpolated through the shell.",
+     args:[],
+     snippet:'echo "$text" | super -s -i line -c "aggregate s:=collect(this) | values {body: join(s, \"\\n\")}" -',
+     examples:[{i:"multi-line text with special chars",o:"single record with embedded newlines"}] }, <skdoc>)
+)
+
+fn skdoc_safe_append_to_sup_file(): (
+  cast(
+    {name:"safe_append_to_sup_file",
+     type:"shell",
+     desc:"Append a timestamped record with raw text to a .sup file. Common pattern for scripts that log or accumulate structured data from unstructured input.",
+     args:[],
+     snippet:'echo "$text" | super -s -i line -c "values {ts: now(), body: this}" - >> data.sup',
+     examples:[{i:"arbitrary user input",o:"appended as timestamped record to .sup file"}] }, <skdoc>)
+)

package/docs/recipes/format.md

@@ -0,0 +1,51 @@
+---
+title: Format
+layout: default
+nav_order: 5
+parent: Recipes
+---
+
+# Format Recipes
+
+Source: `format.spq`
+
+---
+
+## sk_format_bytes
+
+Returns the size in bytes in human readable format.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `value` | Must be castable to uint64 |
+
+```supersql
+sk_format_bytes(1048576)
+-- => '1 MB'
+
+sk_format_bytes(0)
+-- => '0 B'
+```
+
+Supports units up to EB (exabytes). The full unit list: B, KB, MB, GB, TB, PB, EB.
+
+**Implementation:**
+
+```supersql
+const sk_bytes_units=["B", "KB", "MB", "GB", "TB", "PB", "EB"]
+const sk_bytes_divisor=1024
+
+fn _sk_bytes_unit_index(value): (
+  floor(log(value) / log(sk_bytes_divisor))::uint64
+)
+
+fn _sk_format_nonzero_bytes(value): (
+  f"{(value / pow(sk_bytes_divisor, _sk_bytes_unit_index(value)))::uint64} {sk_bytes_units[_sk_bytes_unit_index(value)]}"
+)
+
+fn sk_format_bytes(value): (
+  (value == 0) ? "0 B" : _sk_format_nonzero_bytes(value)
+)
+```

package/docs/recipes/format.spq

@@ -0,0 +1,24 @@
+-- cannot get above EB with uint64 - sorry "ZB" & "YB" and beyond...
+const sk_bytes_units=["B", "KB", "MB", "GB", "TB", "PB", "EB"]
+const sk_bytes_divisor=1024
+
+fn _sk_bytes_unit_index(value): (
+  floor(log(value) / log(sk_bytes_divisor))::uint64
+)
+
+fn _sk_format_nonzero_bytes(value): (
+  f"{(value / pow(sk_bytes_divisor, _sk_bytes_unit_index(value)))::uint64} {sk_bytes_units[_sk_bytes_unit_index(value)]}"
+)
+
+fn skdoc_format_bytes(): (
+  cast(
+    {name:"sk_format_bytes",
+     type:"func",
+     desc:"Returns the size in bytes in human readable format.",
+     args:[{name:"value",desc:"Must be castable to uint64"}],
+     examples:[{i:"sk_format_bytes(1048576)",o:"'1 MB'"}] }, <skdoc>)
+)
+
+fn sk_format_bytes(value): (
+  (value == 0) ? "0 B" : _sk_format_nonzero_bytes(value)
+)

package/docs/recipes/index.md

@@ -0,0 +1,23 @@
+---
+title: Recipes
+layout: default
+has_children: true
+nav_order: 5
+---
+
+# Recipes
+
+Reusable SuperSQL functions and operators for common tasks. Include them in your
+queries with `from` or `-I`:
+
+```supersql
+from 'string.spq' | values sk_capitalize('hello')
+```
+
+```bash
+super -I string.spq -c "values sk_capitalize('hello')"
+```
+
+Recipe source files are available in the
+[superkit](https://github.com/chrismo/superkit/tree/main/docs/recipes)
+repository.

package/docs/recipes/integer.md

@@ -0,0 +1,101 @@
+---
+title: Integer
+layout: default
+nav_order: 3
+parent: Recipes
+---
+
+# Integer Recipes
+
+Source: `integer.spq`
+
+---
+
+## sk_clamp
+
+Clamps a value between min and max.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `i` | The value to clamp. |
+| `min` | The minimum value. |
+| `max` | The maximum value. |
+
+```supersql
+sk_clamp(1, 2, 3)
+-- => 2
+
+sk_clamp(4, 2, 3)
+-- => 3
+
+sk_clamp(2, 2, 3)
+-- => 2
+```
+
+**Implementation:**
+
+```supersql
+fn sk_clamp(i, min, max): (
+  i < min ? min : i > max ? max : i
+)
+```
+
+---
+
+## sk_min
+
+Returns the minimum of two values.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `a` | The first value. |
+| `b` | The second value. |
+
+```supersql
+sk_min(1, 2)
+-- => 1
+
+sk_min(3, 2)
+-- => 2
+```
+
+**Implementation:**
+
+```supersql
+fn sk_min(a, b): (
+  a < b ? a : b
+)
+```
+
+---
+
+## sk_max
+
+Returns the maximum of two values.
+
+**Type:** function
+
+| Argument | Description |
+|----------|-------------|
+| `a` | The first value. |
+| `b` | The second value. |
+
+```supersql
+sk_max(1, 2)
+-- => 2
+
+sk_max(3, 2)
+-- => 3
+```
+
+**Implementation:**
+
+```supersql
+fn sk_max(a, b): (
+  a > b ? a : b
+)
+```

package/docs/recipes/integer.spq

@@ -0,0 +1,53 @@
+-- TODO: this doesn't cause any problems yet because we're not recursively
+-- scanning sk_include files for _their_ sk_includes.
+
+-- sk_include string.spq -- test circular dependencies
+
+op skdoc_clamp: (
+  cast(
+    {name:"sk_clamp",
+     type:"func",
+     desc:"Clamps a value between min and max.",
+     args:[{name:"i",desc:"The value to clamp."}
+           {name:"min",desc:"The minimum value."}
+           {name:"max",desc:"The maximum value."}],
+     examples:[{i:"sk_clamp(1, 2, 3)",o:"2"}
+               {i:"sk_clamp(4, 2, 3)",o:"3"}
+               {i:"sk_clamp(2, 2, 3)",o:"2"}]}, <skdoc>)
+)
+
+fn sk_clamp(i, min, max): (
+    i < min ? min : i > max ? max : i
+)
+
+fn skdoc_min(): (
+  cast(
+    {name:"sk_min",
+     type:"func",
+     desc:"Returns the minimum of two values.",
+     args:[{name:"a",desc:"The first value."}
+           {name:"b",desc:"The second value."}],
+     examples:[{i:"sk_min(1, 2)",o:"1"}
+               {i:"sk_min(3, 2)",o:"2"}
+               {i:"sk_min(2, 2)",o:"2"}]}, <skdoc>)
+)
+
+fn sk_min(a, b): (
+  a < b ? a : b
+)
+
+fn skdoc_max(): (
+  cast(
+    {name:"sk_max",
+     type:"func",
+     desc:"Returns the maximum of two values.",
+     args:[{name:"a",desc:"The first value."}
+           {name:"b",desc:"The second value."}],
+     examples:[{i:"sk_max(1, 2)",o:"2"}
+               {i:"sk_max(3, 2)",o:"3"}
+               {i:"sk_max(2, 2)",o:"2"}]}, <skdoc>)
+)
+
+fn sk_max(a, b): (
+  a > b ? a : b
+)

package/docs/recipes/records.md

@@ -0,0 +1,84 @@
+---
+title: Records
+layout: default
+nav_order: 6
+parent: Recipes
+---
+
+# Record Recipes
+
+Source: `records.spq` (includes `array.spq`)
+
+---
+
+## sk_keys
+
+Returns the keys of the top-level fields in a record. This does not go deep
+into nested records.
+
+**Type:** operator
+
+```supersql
+{a:1,b:{c:333}} | sk_keys
+-- => ['a','b']
+
+{x:10,y:20,z:30} | sk_keys
+-- => ['x','y','z']
+```
+
+**Implementation:**
+
+```supersql
+op sk_keys: (
+  fields(this) | unnest this | values this[0] | uniq | collect(this)
+)
+```
+
+---
+
+## sk_merge_records
+
+Merges an array of records into a single record by combining the fields. If
+there are duplicate keys, the last one wins.
+
+**Type:** operator
+
+```supersql
+[{a:1},{b:{c:333}}] | sk_merge_records
+-- => {a:1,b:{c:333}}
+```
+
+**Implementation:**
+
+```supersql
+op sk_merge_records: (
+  this::string
+  | replace(this, "},{",",")
+  | parse_sup(this[1:-1])
+)
+```
+
+---
+
+## sk_add_ids
+
+Prepends an incrementing id field to each record. Always returns an array.
+
+**Type:** operator
+
+```supersql
+[{a:3},{b:4}] | sk_add_ids
+-- => [{id:1,a:3},{id:2,b:4}]
+```
+
+**Implementation:**
+
+```supersql
+op sk_add_ids: (
+  sk_in_array(this)
+  | unnest this
+  | count
+  | values {id:count,...that}
+  | collect(this)
+)
+```