graphique 2.0.2__tar.gz → 2.1__tar.gz

{graphique-2.0.2 → graphique-2.1}/PKG-INFO

@@ -1,14 +1,14 @@
 Metadata-Version: 2.4
 Name: graphique
-Version: 2.0.2
+Version: 2.1
 Summary: GraphQL service for python dataframes and parquet datasets.
-Author-email: Aric Coady <aric.coady@gmail.com>
+Author: A. Coady
 License-Expression: Apache-2.0
 Project-URL: Homepage, https://github.com/coady/graphique
 Project-URL: Documentation, https://coady.github.io/graphique
 Project-URL: Changelog, https://github.com/coady/graphique/blob/main/CHANGELOG.md
 Project-URL: Issues, https://github.com/coady/graphique/issues
-Keywords: graphql,arrow,parquet
+Keywords: graphql,ibis,dataframes,arrow,parquet
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Science/Research
@@ -26,7 +26,7 @@ Classifier: Typing :: Typed
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
-Requires-Dist: pyarrow>=20
+Requires-Dist: pyarrow>=23
 Requires-Dist: strawberry-graphql[asgi,cli]>=0.288
 Requires-Dist: ibis-framework>=10.7
 Requires-Dist: pyarrow-hotfix
@@ -36,6 +36,7 @@ Requires-Dist: uvicorn[standard]; extra == "server"
 Requires-Dist: duckdb; extra == "server"
 Provides-Extra: cli
 Requires-Dist: tqdm; extra == "cli"
+Requires-Dist: typer; extra == "cli"
 Dynamic: license-file
 
 [![image](https://img.shields.io/pypi/v/graphique.svg)](https://pypi.org/project/graphique/)
@@ -74,25 +75,27 @@ outputs the graphql schema.
 The example app uses [Starlette's config](https://www.starlette.io/config/): in environment variables or a `.env` file.
 
 * PARQUET_PATH: path to the parquet directory or file
-* FEDERATED = '': field name to extend type `Query` with a federated `Table`
-* METRICS = False: include timings from apollo tracing extension
+* NAME = '': GraphQL field on `Query`; defaults to root type
 * COLUMNS = None: list of names, or mapping of aliases, of columns to select
-* FILTERS = None: json `filter` query for which rows to read at startup
 
 Configuration options exist to provide a convenient no-code solution, but are subject to change in the future. Using a custom app is recommended for production usage.
 
 ### App
 For more options create a custom [ASGI](https://asgi.readthedocs.io/en/latest/index.html) app. Call graphique's `GraphQL` on an ibis [Table](https://ibis-project.org/reference/expression-tables) or arrow [Dataset](https://arrow.apache.org/docs/python/api/dataset.html).
-Supply a mapping of names to datasets for multiple roots, and to enable federation.
+Use a `Query` type with dataset attributes for multiple roots, and to enable federation.
 
 ```python
 import ibis
-from graphique import GraphQL
+from graphique import GraphQL, typed
 
-source = ibis.read_*(...)  # or ibis.connect(...).table(...) or pyarrow.dataset.dataset(...)
+source = ibis.read_*(...)  # or `ibis.connect(...).table(...)` or `pyarrow.dataset.dataset(...)`
 # apply initial projections or filters to `source`
 app = GraphQL(source)  # Table is root query type
-app = GraphQL.federated({<name>: source, ...}, keys={<name>: [], ...})  # Tables on federated fields
+
+# multiple named fields, with optional federation keys
+class Query:
+    name = source  # or `typed(source, name, keys=...)`
+app = GraphQL(Query)
 ```
 
 Start like any ASGI app.
@@ -111,7 +114,8 @@ uvicorn <module>:app
 #### selection
 * `slice`: contiguous selection of rows
 * `filter`: select rows by predicates
-* `join`: join tables by key columns
+* `join`, `asofJoin`, `crossJoin`: join tables by key columns
+* `difference`, `intersect`, `union`: set operations on tables
 * `take`: rows by index
 * `dropNull`: remove rows with nulls
 
@@ -121,6 +125,7 @@ uvicorn <module>:app
 * `column`: access a column of any type by name
 * `row`: provides a field for each scalar of a single row
 * `cast`: cast column types
+* `unpack`: project struct fields
 * `fillNull`: fill null values
 
 #### aggregation
@@ -128,20 +133,26 @@ uvicorn <module>:app
 * `distinct`: group with all columns
 * `runs`: provisionally group by adjacency
 * `unnest`: unnest an array column
-* `count`: number of rows
+* `count`, `any`: number of rows
 
 #### ordering
 * `order`: sort table by given columns
-* options `limit` and `dense`: select rows with smallest or largest values
+* `first`: provisionally sort and filter by rank
+
+#### reflection
+* `type`: type of data source
+* `schema`: field names and types
+* `optional`: nullable for errors
+* `toSql`: compiles SQL query
 
 ### Performance
-Performance is dependent on the [ibis backend](https://ibis-project.org/backends/duckdb), which defaults to [duckdb](https://duckdb.org/). There are no internal Python loops. Scalars do not become Python types until serialized.
+Performance is dependent on the [ibis backend](https://ibis-project.org/backends/duckdb), which defaults to [duckdb](https://duckdb.org/). There are no internal Python loops. Scalars do not become Python types until serialized. Table fields are lazily evaluated up until scalars are reached, and automatically cached as needed for multiple fields.
 
-[PyArrow](https://arrow.apache.org/docs/python/) is also used for partitioned dataset optimizations, and for any feature which ibis does not support. Table fields are lazily evaluated up until scalars are reached, and automatically cached as needed for multiple fields.
+[PyArrow](https://arrow.apache.org/docs/python/) is also used for partitioned dataset optimizations. `python -m graphique.partition` is a command-line script provided in `graphique[cli]`, for out-of-core partitioning.
 
 ## Installation
 ```console
-pip install graphique[server]
+pip install graphique[server,cli]
 ```
 
 ## Dependencies

{graphique-2.0.2 → graphique-2.1}/README.md

@@ -34,25 +34,27 @@ outputs the graphql schema.
 The example app uses [Starlette's config](https://www.starlette.io/config/): in environment variables or a `.env` file.
 
 * PARQUET_PATH: path to the parquet directory or file
-* FEDERATED = '': field name to extend type `Query` with a federated `Table`
-* METRICS = False: include timings from apollo tracing extension
+* NAME = '': GraphQL field on `Query`; defaults to root type
 * COLUMNS = None: list of names, or mapping of aliases, of columns to select
-* FILTERS = None: json `filter` query for which rows to read at startup
 
 Configuration options exist to provide a convenient no-code solution, but are subject to change in the future. Using a custom app is recommended for production usage.
 
 ### App
 For more options create a custom [ASGI](https://asgi.readthedocs.io/en/latest/index.html) app. Call graphique's `GraphQL` on an ibis [Table](https://ibis-project.org/reference/expression-tables) or arrow [Dataset](https://arrow.apache.org/docs/python/api/dataset.html).
-Supply a mapping of names to datasets for multiple roots, and to enable federation.
+Use a `Query` type with dataset attributes for multiple roots, and to enable federation.
 
 ```python
 import ibis
-from graphique import GraphQL
+from graphique import GraphQL, typed
 
-source = ibis.read_*(...)  # or ibis.connect(...).table(...) or pyarrow.dataset.dataset(...)
+source = ibis.read_*(...)  # or `ibis.connect(...).table(...)` or `pyarrow.dataset.dataset(...)`
 # apply initial projections or filters to `source`
 app = GraphQL(source)  # Table is root query type
-app = GraphQL.federated({<name>: source, ...}, keys={<name>: [], ...})  # Tables on federated fields
+
+# multiple named fields, with optional federation keys
+class Query:
+    name = source  # or `typed(source, name, keys=...)`
+app = GraphQL(Query)
 ```
 
 Start like any ASGI app.
@@ -71,7 +73,8 @@ uvicorn <module>:app
 #### selection
 * `slice`: contiguous selection of rows
 * `filter`: select rows by predicates
-* `join`: join tables by key columns
+* `join`, `asofJoin`, `crossJoin`: join tables by key columns
+* `difference`, `intersect`, `union`: set operations on tables
 * `take`: rows by index
 * `dropNull`: remove rows with nulls
 
@@ -81,6 +84,7 @@ uvicorn <module>:app
 * `column`: access a column of any type by name
 * `row`: provides a field for each scalar of a single row
 * `cast`: cast column types
+* `unpack`: project struct fields
 * `fillNull`: fill null values
 
 #### aggregation
@@ -88,20 +92,26 @@ uvicorn <module>:app
 * `distinct`: group with all columns
 * `runs`: provisionally group by adjacency
 * `unnest`: unnest an array column
-* `count`: number of rows
+* `count`, `any`: number of rows
 
 #### ordering
 * `order`: sort table by given columns
-* options `limit` and `dense`: select rows with smallest or largest values
+* `first`: provisionally sort and filter by rank
+
+#### reflection
+* `type`: type of data source
+* `schema`: field names and types
+* `optional`: nullable for errors
+* `toSql`: compiles SQL query
 
 ### Performance
-Performance is dependent on the [ibis backend](https://ibis-project.org/backends/duckdb), which defaults to [duckdb](https://duckdb.org/). There are no internal Python loops. Scalars do not become Python types until serialized.
+Performance is dependent on the [ibis backend](https://ibis-project.org/backends/duckdb), which defaults to [duckdb](https://duckdb.org/). There are no internal Python loops. Scalars do not become Python types until serialized. Table fields are lazily evaluated up until scalars are reached, and automatically cached as needed for multiple fields.
 
-[PyArrow](https://arrow.apache.org/docs/python/) is also used for partitioned dataset optimizations, and for any feature which ibis does not support. Table fields are lazily evaluated up until scalars are reached, and automatically cached as needed for multiple fields.
+[PyArrow](https://arrow.apache.org/docs/python/) is also used for partitioned dataset optimizations. `python -m graphique.partition` is a command-line script provided in `graphique[cli]`, for out-of-core partitioning.
 
 ## Installation
 ```console
-pip install graphique[server]
+pip install graphique[server,cli]
 ```
 
 ## Dependencies

graphique-2.1/graphique/__init__.py

@@ -0,0 +1,2 @@
+from .interface import Dataset
+from .middleware import GraphQL, MetricsExtension, implement, typed

graphique-2.1/graphique/core.py

@@ -0,0 +1,100 @@
+"""
+Core arrow utilities.
+"""
+
+import bisect
+import itertools
+
+import ibis
+import pyarrow as pa
+import pyarrow.dataset as ds
+
+
+class links:
+    ref = "https://ibis-project.org/reference"
+    types = f"[data type]({ref}/datatypes)"
+    schema = f"[table schema]({ref}/schemas#ibis.expr.schema.Schema)"
+
+
+def getitems(obj, *keys):
+    """Nested `getitem`."""
+    for key in keys:
+        obj = obj[key]
+    return obj
+
+
+def order_key(name: str):
+    """Parse sort order."""
+    return (ibis.desc if name.startswith("-") else ibis.asc)(ibis._[name.lstrip("-")])
+
+
+def rank_over(
+    table: ibis.Table,
+    by: list[str],
+    over: list[str],
+    index: ibis.Column,
+    rank: int = 1,
+) -> ibis.Table:
+    """Filter rows by rank within each grouping window."""
+    order_by = list(map(order_key, by))
+    table = table.mutate(_=index.over(group_by=over, order_by=order_by))
+    return table.filter(table["_"] < rank).drop("_").order_by(*order_by)
+
+
+class Parquet(ds.Dataset):
+    """Partitioned parquet dataset."""
+
+    def schema(self) -> pa.Schema:
+        """partition schema"""
+        return self.partitioning.schema if hasattr(self, "partitioning") else pa.schema([])
+
+    def keys(self, *names) -> list:
+        """Return prefix of matching partition keys."""
+        keys = set(Parquet.schema(self).names)
+        return list(itertools.takewhile(lambda name: name.lstrip("-") in keys, names))
+
+    def fragments(self, counts: str = "", path: str = "__path__") -> ibis.Table:
+        """Return partition fragments as a table."""
+        parts = []
+        for frag in self.get_fragments():
+            parts.append(ds.get_partition_keys(frag.partition_expression))
+            parts[-1][path] = frag.path
+            if counts:
+                parts[-1][counts] = frag.count_rows()
+        return ibis.memtable(pa.Table.from_pylist(parts))
+
+    def filter(self, expr: ds.Expression | None) -> ds.Dataset | None:
+        """Attempt to apply filter to partition keys."""
+        try:  # raises ValueError if filter references non-partition keys
+            ds.dataset([], schema=self.partitioning.schema).scanner(filter=expr)
+        except (AttributeError, ValueError):
+            return None
+        if expr is None:
+            return self
+        paths = [frag.path for frag in self.get_fragments(expr)]
+        return ds.dataset(paths, partitioning=self.partitioning)
+
+    def to_table(self, name: str | None = None) -> ibis.Table:
+        """Return ibis `Table` from filtered dataset."""
+        paths = [frag.path for frag in self.get_fragments()]
+        return ibis.read_parquet(paths, table_name=name)
+
+    def order(self, *names: str, limit: int | None = None) -> ds.Dataset:
+        """Return ordered partitions of the dataset."""
+        table = Parquet.fragments(self, counts="" if limit is None else "_")
+        table = table.order_by(*map(order_key, names)).cache()
+        if limit is not None:
+            limit = bisect.bisect_left(table["_"].cumsum().to_list(), limit) + 1
+        return ds.dataset(table[:limit]["__path__"].to_list(), partitioning=self.partitioning)
+
+    def first(self, *names: str, rank: int = 1, dense: bool = False) -> ds.Dataset:
+        """Return ordered partitions up to max rank (dense or sparse)."""
+        keys = {name.strip("-"): order_key(name) for name in names}
+        if dense or rank == 1:
+            table = Parquet.fragments(self).aggregate(_=ibis._["__path__"].collect(), by=list(keys))
+            paths = table.order_by(*keys.values())[:rank]["_"].unnest()
+        else:
+            table = Parquet.fragments(self, counts="_").order_by(*keys.values()).cache()
+            limit = bisect.bisect_left(table["_"].cumsum().to_list(), rank) + 1
+            paths = table.semi_join(table[:limit].select(*keys).distinct(), list(keys))["__path__"]
+        return ds.dataset(paths.to_list(), partitioning=self.partitioning)

{graphique-2.0.2 → graphique-2.1}/graphique/inputs.py

@@ -12,7 +12,7 @@ from datetime import date, datetime, time
 from decimal import Decimal
 from typing import Generic, TypeVar
 
-import ibis
+import ibis.expr.types
 import pyarrow as pa
 import pyarrow.compute as pc
 import pyarrow.dataset as ds
@@ -22,7 +22,6 @@ from strawberry.annotation import StrawberryAnnotation
 from strawberry.scalars import JSON
 from strawberry.schema_directive import Location
 from strawberry.types.arguments import StrawberryArgument
-from strawberry.types.field import StrawberryField
 
 from .core import getitems, links, order_key
 from .scalars import Duration
@@ -44,14 +43,15 @@ class optional:
     """
 
 
-@use_doc(strawberry.schema_directive, locations=[Location.OBJECT, Location.FIELD_DEFINITION])
+@use_doc(
+    strawberry.schema_directive,
+    locations=[Location.ARGUMENT_DEFINITION, Location.FIELD_DEFINITION, Location.OBJECT],
+)
 class provisional:
     """Provisional feature; subject to change in the future."""
 
 
-def default_field(
-    default=UNSET, func: Callable | None = None, nullable: bool = False, **kwargs
-) -> StrawberryField:
+def default_field(default=UNSET, func: Callable | None = None, nullable: bool = False, **kwargs):
     """Use dataclass `default_factory` for `UNSET` or mutables."""
     if func is not None:
         kwargs["description"] = inspect.getdoc(func).splitlines()[0]
@@ -83,10 +83,12 @@ class Filter(Generic[T]):
     @classmethod
     def resolve_args(cls, types: dict) -> Iterable[StrawberryArgument]:
         """Generate dynamically resolved arguments for filter field."""
-        for name in types:
-            annotation = StrawberryAnnotation(cls[types[name]])  # type: ignore
-            if types[name] not in (list, dict):
-                yield StrawberryArgument(name, name, annotation, default={})
+        for name, scalar in types.items():
+            if getattr(scalar, "__origin__", None) is list:
+                annotation = ArrayFilter[scalar.__args__[0]]
+            else:
+                annotation = cls[scalar]  # type: ignore
+            yield StrawberryArgument(name, name, StrawberryAnnotation(annotation), default={})
         annotation = StrawberryAnnotation(Expression | None)
         yield StrawberryArgument("where", None, annotation, default=None)
 
@@ -98,9 +100,11 @@ class Filter(Generic[T]):
             for op, value in query:
                 match value, op:
                     case list(), "eq":
-                        yield field.isin(value)
+                        yield field == value[0] if len(value) == 1 else field.isin(value)
                     case list(), "ne":
-                        yield field.notin(value)
+                        yield field != value[0] if len(value) == 1 else field.notin(value)
+                    case _, "contains":
+                        yield field.contains(value)
                     case _:
                         yield getattr(operator, op)(field, value)
 
@@ -114,28 +118,37 @@ class Filter(Generic[T]):
                 if isinstance(value, list):
                     expr = pc.is_in(field, pa.array(value))
                     exprs.append(pc.invert(expr) if op == "ne" else expr)
+                elif op == "contains":  # placeholder, will not be in partition schema
+                    exprs.append(field == value)
                 else:
                     exprs.append(getattr(operator, op)(field, value))
         return functools.reduce(operator.and_, exprs or [None])
 
 
+@strawberry.input(description="predicates for arrays")
+class ArrayFilter(Generic[T]):
+    contains: T | None = default_field(description="array contains element")
+
+    __iter__ = Filter.__iter__
+
+
 @strawberry.input(description="name and optional alias for aggregation")
 class Aggregate:
     name: str = strawberry.field(description="column name")
     alias: str = strawberry.field(default="", description="output column name")
     where: Expression | None = None
 
-    def to_ibis(self, func: str, **options) -> tuple:
+    def to_ibis(self, func: str, *args, **options) -> tuple:
         options["where"] = self.where and self.where.to_ibis()
-        return (self.alias or self.name), getattr(ibis._[self.name], func)(**options)
+        return (self.alias or self.name), getattr(ibis._[self.name], func)(*args, **options)
 
 
 @strawberry.input
 class UniqueAggregate(Aggregate):
     approx: bool = False
 
-    def to_ibis(self, func: str, **options) -> tuple:
-        return super().to_ibis("approx_" + func if self.approx else func, **options)
+    def to_ibis(self, func: str, *args) -> tuple:
+        return super().to_ibis("approx_" + func if self.approx else func, *args)
 
 
 @strawberry.input
@@ -155,7 +168,7 @@ class OrderAggregate(Aggregate):
 class VarAggregate(Aggregate):
     how: str = "sample"
 
-    def to_ibis(self, func: str, **_) -> tuple:
+    def to_ibis(self, func: str) -> tuple:
         return super().to_ibis(func, how=self.how)
 
 
@@ -163,8 +176,8 @@ class VarAggregate(Aggregate):
 class QuantileAggregate(UniqueAggregate):
     q: float = 0.5
 
-    def to_ibis(self, func: str, **_) -> tuple:
-        return super().to_ibis(func, quantile=self.q)
+    def to_ibis(self, func: str, *_) -> tuple:
+        return super().to_ibis(func, self.q)
 
 
 @strawberry.input
@@ -175,17 +188,40 @@ class CollectAggregate(OrderAggregate):
         return super().to_ibis(func, distinct=self.distinct)
 
 
+@strawberry.input
+class ConcatAggregate(Aggregate):
+    sep: str = ","
+    order_by: list[str] = default_field([])
+
+    def to_ibis(self, func: str) -> tuple:
+        order_by = list(map(order_key, self.order_by)) or None
+        return super().to_ibis("group_" + func, sep=self.sep, order_by=order_by)
+
+
+@strawberry.input
+class ArgAggregate(Aggregate):
+    key: str
+
+    def to_ibis(self, func: str) -> tuple:
+        return super().to_ibis(func, key=self.key)
+
+
 @strawberry.input(description=f"aggregation [expressions]({links.ref}/expression-generic)")
 class Aggregates:
     all: list[Aggregate] = default_field([], func=ibis.expr.types.BooleanColumn.all)
     any: list[Aggregate] = default_field([], func=ibis.expr.types.BooleanColumn.any)
+    argmax: list[ArgAggregate] = default_field([], func=ibis.Column.argmax)
+    argmin: list[ArgAggregate] = default_field([], func=ibis.Column.argmin)
     collect: list[CollectAggregate] = default_field([], func=ibis.Column.collect)
+    concat: list[ConcatAggregate] = default_field([], func=ibis.Column.group_concat)
     count: list[Aggregate] = default_field([], func=ibis.Column.count)
     first: list[OrderAggregate] = default_field([], func=ibis.Column.first)
+    kurtosis: list[VarAggregate] = default_field([], func=ibis.expr.types.NumericColumn.kurtosis)
     last: list[OrderAggregate] = default_field([], func=ibis.Column.last)
     max: list[Aggregate] = default_field([], func=ibis.Column.max)
     mean: list[Aggregate] = default_field([], func=ibis.expr.types.NumericColumn.mean)
     min: list[Aggregate] = default_field([], func=ibis.Column.min)
+    mode: list[Aggregate] = default_field([], func=ibis.Column.mode)
     nunique: list[UniqueAggregate] = default_field([], func=ibis.Column.nunique)
     quantile: list[QuantileAggregate] = default_field([], func=ibis.Column.quantile)
     std: list[VarAggregate] = default_field([], func=ibis.expr.types.NumericColumn.std)
@@ -198,7 +234,7 @@ class Aggregates:
                 yield agg.to_ibis(name)
 
 
-@strawberry.input(description="typed scalars")
+@strawberry.input(description="typed scalars", one_of=True)
 class Scalars:
     base64: strawberry.scalars.Base64 | None = default_field(description="binary scalar")
     date: date | None = default_field(description="date scalar")
@@ -218,7 +254,9 @@ class Expression:
     name: list[str] = default_field([], description="column name(s)")
     value: JSON | None = default_field(description="JSON scalar", nullable=True)
     scalar: Scalars | None = default_field(description="typed scalar")
-    row_number: None = default_field(func=ibis.row_number)
+    row_number: None = default_field(
+        func=ibis.row_number, deprecation_reason="use `window.rowNumber`"
+    )
 
     eq: list[Expression] = default_field([], description="==")
     ne: list[Expression] = default_field([], description="!=")
@@ -240,13 +278,21 @@ class Expression:
     truediv: list[Expression] = default_field([], name="div", description="/")
 
     coalesce: list[Expression] = default_field([], func=ibis.Column.coalesce)
-    cume_dist: Expression | None = default_field(func=ibis.Column.cume_dist)
+    cume_dist: Expression | None = default_field(
+        func=ibis.Column.cume_dist, deprecation_reason="use `window.cumeDist`"
+    )
     cummax: Expression | None = default_field(func=ibis.Column.cummax)
     cummin: Expression | None = default_field(func=ibis.Column.cummin)
-    dense_rank: Expression | None = default_field(func=ibis.Column.dense_rank)
+    dense_rank: Expression | None = default_field(
+        func=ibis.Column.dense_rank, deprecation_reason="use `window.denseRank`"
+    )
     ifelse: list[Expression] = default_field([], func=ibis.expr.types.BooleanColumn.ifelse)
-    percent_rank: Expression | None = default_field(func=ibis.Column.percent_rank)
-    rank: Expression | None = default_field(func=ibis.Column.rank)
+    percent_rank: Expression | None = default_field(
+        func=ibis.Column.percent_rank, deprecation_reason="use `window.percentRank`"
+    )
+    rank: Expression | None = default_field(
+        func=ibis.Column.rank, deprecation_reason="use `window.rank`"
+    )
 
     array: Arrays | None = default_field(description="array value functions")
     numeric: Numeric | None = default_field(description="numeric functions")
@@ -283,10 +329,10 @@ class Expression:
                 yield from field
 
     def to_ibis(self) -> ibis.Deferred | None:
-        fields = list(self) or [None]
-        if len(fields) == 1:
-            return fields[0]
-        raise ValueError(f"conflicting inputs: {', '.join(map(str, fields))}")
+        fields = list(self)
+        if len(fields) > 1:
+            raise ValueError("conflicting inputs: " + ", ".join(map(str, fields)))
+        return next(iter(fields), None)  # type: ignore
 
 
 @strawberry.input(description="an `Expression` with an optional alias")
@@ -304,6 +350,7 @@ class Projection(Expression):
 class Arrays:
     alls: Expression | None = default_field(func=ibis.expr.types.ArrayValue.alls)
     anys: Expression | None = default_field(func=ibis.expr.types.ArrayValue.anys)
+    flatten: Expression | None = default_field(func=ibis.expr.types.ArrayValue.flatten)
     length: Expression | None = default_field(func=ibis.expr.types.ArrayValue.length)
     maxs: Expression | None = default_field(func=ibis.expr.types.ArrayValue.maxs)
     means: Expression | None = default_field(func=ibis.expr.types.ArrayValue.means)
@@ -313,13 +360,25 @@ class Arrays:
     sums: Expression | None = default_field(func=ibis.expr.types.ArrayValue.sums)
     unique: Expression | None = default_field(func=ibis.expr.types.ArrayValue.unique)
 
+    concat: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.concat)
+    contains: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.contains)
     index: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.index)
+    intersect: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.intersect)
+    remove: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.remove)
+    union: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.union)
+    zip: list[Expression] = default_field([], func=ibis.expr.types.ArrayValue.zip)
 
     slice: Expression | None = default_field(description="array slice")
     value: Expression | None = default_field(description="value at offset")
     offset: int = 0
     limit: int | None = None
 
+    join: Expression | None = default_field(func=ibis.expr.types.ArrayValue.join)
+    sep: str = ""
+
+    repeat: Expression | None = default_field(func=ibis.expr.types.ArrayValue.repeat)
+    n: int = 1
+
     def __iter__(self) -> Iterator[ibis.Deferred]:
         for name, (expr, *args) in Expression.items(self):
             match name:
@@ -327,6 +386,10 @@ class Arrays:
                     yield expr[self.offset :][: self.limit]
                 case "value":
                     yield expr[self.offset]
+                case "join":
+                    yield expr.join(self.sep)
+                case "repeat":
+                    yield expr.repeat(self.n)
                 case _:
                     yield getattr(expr, name)(*args)
 
@@ -448,6 +511,9 @@ class Temporal:
     description=f"window [expressions]({links.ref}/expression-tables.html#ibis.window)"
 )
 class Window:
+    over: list[str] = default_field([], description="partition column names")
+    by: list[str] = default_field([], description="order column names within each partition")
+
     lag: Expression | None = default_field(func=ibis.Column.lag)
     lead: Expression | None = default_field(func=ibis.Column.lead)
 
@@ -459,15 +525,53 @@ class Window:
     ge: Expression | None = default_field(description=r"pairwise \>=")
     sub: Expression | None = default_field(description="pairwise -")
 
-    offset: int = 1
-    default: JSON | None = default_field(None, description="default JSON scalar")
+    count: Expression | None = default_field(func=ibis.Column.count)
+    sum: Expression | None = default_field(func=ibis.expr.types.NumericColumn.sum)
+    mean: Expression | None = default_field(func=ibis.expr.types.NumericColumn.mean)
+    min: Expression | None = default_field(func=ibis.Column.min)
+    max: Expression | None = default_field(func=ibis.Column.max)
+
+    row_number: None = default_field(func=ibis.row_number)
+    rank: None = default_field(func=ibis.rank)
+    dense_rank: None = default_field(func=ibis.dense_rank)
+    percent_rank: None = default_field(func=ibis.percent_rank)
+    cume_dist: None = default_field(func=ibis.cume_dist)
+    ntile: int = default_field(0, func=ibis.ntile)
+
+    preceding: int | None = strawberry.field(
+        default=None, description="number of preceding rows in the window"
+    )
+    following: int | None = strawberry.field(
+        default=None, description="number of following rows in the window"
+    )
+    range: bool = strawberry.field(
+        default=False, description="aggregate rows based upon differences in the order by values"
+    )
+
+    offset: int = strawberry.field(default=1, description="offset for lag and lead")
+    default: JSON | None = strawberry.field(default=None, description="default JSON scalar")
     scalar: Scalars | None = default_field(description="default typed scalar")
 
-    def __iter__(self) -> Iterator[ibis.Deferred]:
+    def __iter__(self) -> Iterator[ibis.Expr]:
         (default,) = self.scalar or [self.default]
+        orderings = list(map(order_key, self.by))
+        win = cumwin = None
+        if self.over or self.by:
+            frame = ibis.range_window if self.range else ibis.window
+            win = frame(self.preceding, self.following, group_by=self.over, order_by=orderings)
+        if self.by and self.preceding is self.following is None:
+            cumwin = ibis.cumulative_window(group_by=self.over, order_by=orderings)
+        for name in ("row_number", "rank", "dense_rank", "percent_rank", "cume_dist"):
+            if getattr(self, name) is not UNSET:
+                yield getattr(ibis, name)().over(win)
+        if self.ntile:
+            yield ibis.ntile(self.ntile).over(win).cast("int32")
         for name, (expr,) in Expression.items(self):
             match name:
+                case "count" | "sum" | "mean" | "min" | "max":
+                    yield getattr(expr, name)().over(cumwin or win)
                 case "lag" | "lead":
-                    yield getattr(expr, name)(self.offset, default)
+                    yield getattr(expr, name)(self.offset, default).over(win)
                 case _:
-                    yield getattr(operator, name)(expr, expr.lag(self.offset)).fill_null(default)
+                    lag = expr.lag(self.offset).over(win)
+                    yield getattr(operator, name)(expr, lag).fill_null(default)