@harness-engineering/cli 1.23.2 → 1.24.1

package/dist/agents/skills/gemini-cli/api-pagination-cursor/SKILL.md

@@ -0,0 +1,159 @@
+# Cursor-Based Pagination
+
+> CURSOR-BASED PAGINATION REPLACES NUMERIC OFFSETS WITH OPAQUE POSITION TOKENS — EACH CURSOR ENCODES EXACTLY WHERE THE CLIENT LEFT OFF, ELIMINATING PAGE DRIFT WHEN ROWS ARE INSERTED OR DELETED BETWEEN REQUESTS AND ENABLING CONSISTENT TRAVERSAL OF LIVE DATASETS.
+
+## When to Use
+
+- Designing pagination for a feed, activity log, or timeline where rows are continuously inserted
+- Implementing `next_page_token` or `after` cursor parameters on list endpoints
+- Migrating an offset-paginated endpoint that exhibits page drift under concurrent writes
+- Building mobile clients that scroll through infinite feeds and cannot afford gaps or duplicates
+- Implementing the Relay connection specification for a GraphQL-backed REST API
+- Exposing large result sets (>10,000 rows) where deep offset scans degrade database performance
+- Designing bidirectional traversal (forward and backward) through an ordered collection
+- Reviewing a PR that returns raw database primary keys as pagination cursors
+
+## Instructions
+
+### Key Concepts
+
+1. **Opaque Cursor** — A cursor is a position token that is intentionally opaque to the client. The server encodes whatever internal position data it needs (a timestamp, a sort key, a composite of multiple columns) and base64-encodes the result so clients cannot parse or construct cursors manually. Opacity allows the server to change the internal encoding without a breaking API change.
+
+   ```
+   Internal:  { "id": 8821, "created_at": "2024-03-15T10:22:00Z" }
+   Encoded:   eyJpZCI6ODgyMSwiY3JlYXRlZF9hdCI6IjIwMjQtMDMtMTVUMTA6MjI6MDBaIn0=
+   ```
+
+2. **Forward Pagination** — The client passes an `after` cursor to retrieve the page that follows a known position. The server decodes the cursor, constructs a `WHERE` clause that selects rows strictly after the encoded position, and applies `LIMIT n`. The response includes a `next_cursor` when more results exist and omits it (or returns null) when the collection is exhausted.
+
+3. **Backward Pagination** — The client passes a `before` cursor to traverse in reverse. The server selects rows strictly before the encoded position with the sort direction reversed, then re-reverses the result set before returning it so the response order matches the canonical collection order. Not all APIs implement backward pagination — document the capability explicitly.
+
+4. **Cursor Stability Guarantees** — A cursor must remain valid for a reasonable window (typically 24–72 hours). If the sort key column is mutable (e.g., `updated_at`), cursor-encoded values may no longer identify the same logical position after an update. Prefer immutable sort keys (e.g., monotonic `id` or `created_at`) for cursor stability. Document the expiry window in the API reference.
+
+5. **Page Size Negotiation** — Accept a `limit` (or `per_page`) parameter with a capped maximum. Return the actual page size used in the response alongside the cursor, even when the client specifies `limit`. This allows clients to detect when the server reduced their requested page size.
+
+6. **has_more Flag** — Rather than computing total count (expensive), return a boolean `has_more` (or `has_next_page` / `has_previous_page` in Relay). To determine `has_more`, request `LIMIT n+1` rows from the database and return only `n` — if the extra row exists, `has_more = true`.
+
+### Worked Example
+
+GitHub's REST API uses cursor-based pagination via `Link` headers and an opaque `cursor` parameter on the GraphQL API. The REST List Issues endpoint demonstrates the pattern:
+
+**Request page 1:**
+
+```http
+GET /repos/octocat/hello-world/issues?per_page=2&state=open
+Authorization: Bearer ghp_...
+```
+
+```http
+HTTP/1.1 200 OK
+Link: <https://api.github.com/repos/octocat/hello-world/issues?per_page=2&state=open&after=Y3Vyc29yOnYyOpHOAAFBUQ%3D%3D>; rel="next"
+Content-Type: application/json
+
+[
+  { "id": 100, "number": 42, "title": "Fix build" },
+  { "id": 99,  "number": 41, "title": "Add tests"  }
+]
+```
+
+**Request the next page using the cursor from the `Link` header:**
+
+```http
+GET /repos/octocat/hello-world/issues?per_page=2&state=open&after=Y3Vyc29yOnYyOpHOAAFBUQ%3D%3D
+Authorization: Bearer ghp_...
+```
+
+```http
+HTTP/1.1 200 OK
+Link: <https://api.github.com/repos/octocat/hello-world/issues?per_page=2&state=open&after=Y3Vyc29yOnYyOpHOAAFBUQ%3D%3D>; rel="prev",
+      <https://api.github.com/repos/octocat/hello-world/issues?per_page=2&state=open&after=Y3Vyc29yOnYyOpHOAAFBUM%3D%3D>; rel="next"
+
+[
+  { "id": 98, "number": 40, "title": "Update docs" },
+  { "id": 97, "number": 39, "title": "Refactor auth" }
+]
+```
+
+The GitHub GraphQL API uses the Relay connection spec with `edges`, `node`, `pageInfo.endCursor`, `pageInfo.hasNextPage` — a canonical reference implementation of cursor pagination at scale.
+
+### Anti-Patterns
+
+1. **Exposing raw database IDs as cursors.** Returning `"cursor": "8821"` leaks internal implementation details, lets clients construct arbitrary cursors that bypass intended access controls, and makes it impossible to change the internal ID scheme. Always base64-encode an opaque payload. Even if the payload is just an integer today, opacity preserves the option to change it.
+
+2. **Using mutable columns as the sole cursor key.** If the cursor encodes `updated_at` and rows are frequently updated, a client mid-pagination may re-see rows it already received or skip rows whose `updated_at` was bumped between pages. Use an immutable monotonic column (`id`, `created_at`) as the primary cursor key. If the client must sort by a mutable field, use a composite cursor of `(mutable_field, id)` so `id` breaks ties and the position remains unique and stable.
+
+3. **Returning a total count alongside every cursor page.** `SELECT COUNT(*)` on large tables acquires a table scan or index scan that is often more expensive than the page query itself. Cursor pagination's primary advantage over offset is avoiding this scan. If a UI needs a total count, compute it asynchronously and cache it, or switch to an approximate count (`pg_class.reltuples` in PostgreSQL). Never block the paginated response on a live COUNT.
+
+4. **Accepting client-constructed cursors without validation.** If a client can construct an arbitrary cursor value, it may bypass row-level security, access soft-deleted records, or enumerate internal IDs. Always decode, validate schema, and verify the encoded values fall within the caller's access scope before using cursor data in a query.
+
+## Details
+
+### Relay Connection Specification
+
+The [Relay cursor connection specification](https://relay.dev/graphql/connections.htm) defines a canonical shape that many REST and GraphQL APIs follow:
+
+```json
+{
+  "data": {
+    "edges": [
+      { "cursor": "Y3Vyc29yMQ==", "node": { "id": "1", "name": "Alice" } },
+      { "cursor": "Y3Vyc29yMg==", "node": { "id": "2", "name": "Bob" } }
+    ],
+    "pageInfo": {
+      "startCursor": "Y3Vyc29yMQ==",
+      "endCursor": "Y3Vyc29yMg==",
+      "hasNextPage": true,
+      "hasPreviousPage": false
+    }
+  }
+}
+```
+
+Adopting this shape for REST responses reduces integration friction for teams already using Relay on the frontend and documents a well-understood contract.
+
+### Cursor Encoding in SQL
+
+For a table sorted by `created_at DESC, id DESC`, the forward-pagination query given a decoded cursor `{created_at: "2024-03-15T10:22:00Z", id: 8821}` is:
+
+```sql
+SELECT * FROM events
+WHERE (created_at, id) < ('2024-03-15T10:22:00Z', 8821)
+ORDER BY created_at DESC, id DESC
+LIMIT 26  -- request 26 to detect has_next_page; return 25
+```
+
+This row-value comparison is supported by PostgreSQL, MySQL 8+, and SQLite and uses composite indexes efficiently without an OFFSET scan.
+
+### Real-World Case Study: Slack
+
+Slack's Web API uses cursor pagination (`next_cursor` in `response_metadata`) for all list endpoints including `conversations.list`, `users.list`, and `channels.history`. Before migrating from offset to cursor pagination, Slack reported that deep-offset queries on the `channels.history` endpoint (messages at offset 50,000+) caused p99 response times exceeding 8 seconds on large workspaces. After the migration to cursor-based pagination using composite `(ts, channel_id)` cursors, p99 latency for equivalent page fetches dropped to under 80ms — a 100x improvement — because each page query became a bounded index seek rather than a full table scan.
+
+## Source
+
+- [Relay Cursor Connections Specification](https://relay.dev/graphql/connections.htm)
+- [GitHub REST API Pagination](https://docs.github.com/en/rest/using-the-rest-api/using-pagination-in-the-rest-api)
+- [Slack API — Pagination](https://api.slack.com/docs/pagination)
+- [RFC 5988 — Web Linking (Link header)](https://www.rfc-editor.org/rfc/rfc5988)
+- [Use The Index, Luke — No Offset](https://use-the-index-luke.com/no-offset/)
+
+## Process
+
+1. Choose the sort key for the cursor: prefer immutable columns (`id`, `created_at`). If the client sorts by a mutable field, design a composite cursor of `(sort_field, id)`.
+2. Implement cursor encoding: serialize the position payload as JSON, base64url-encode it, and validate the schema on decode before trusting any field.
+3. Write the page query using a row-value comparison (`WHERE (col_a, col_b) < (val_a, val_b)`) with `LIMIT n+1` to detect `has_more` without a COUNT query.
+4. Return the last row's encoded cursor as `next_cursor` (omit the field or return null when `has_more` is false). Include `has_more` as an explicit boolean.
+5. Document cursor lifetime (e.g., "cursors are valid for 24 hours") and the behavior when an expired cursor is submitted (return `400 Bad Request` with error code `cursor_expired`).
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** api-pagination-offset, api-pagination-keyset, graphql-pagination-patterns, api-filtering-sorting
+
+## Success Criteria
+
+- Cursors are opaque to clients: base64-encoded payloads that clients cannot parse or construct manually.
+- Page queries use row-value comparisons or equivalent index seeks — no OFFSET clause on the paginated query.
+- `has_more` is determined by fetching `n+1` rows, not by COUNT(\*).
+- Expired or malformed cursors return `400 Bad Request` with a machine-readable error code, not a 500.
+- Cursor lifetime is documented in the API reference with explicit expiry and renewal behavior.

package/dist/agents/skills/gemini-cli/api-pagination-cursor/skill.yaml

@@ -0,0 +1,40 @@
+name: api-pagination-cursor
+version: '1.0.0'
+description: Cursor-based pagination with opaque tokens, base64 encoding, forward/backward traversal, and cursor stability guarantees
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - api-pagination-offset
+  - api-pagination-keyset
+  - graphql-pagination-patterns
+  - api-filtering-sorting
+stack_signals:
+  - rest
+  - api
+  - pagination
+keywords:
+  - cursor-pagination
+  - opaque-cursor
+  - base64
+  - forward-pagination
+  - backward-pagination
+  - page-token
+  - seek
+metadata:
+  author: community
+  upstream: relay.dev/graphql/connections.htm
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/gemini-cli/api-pagination-keyset/SKILL.md

@@ -0,0 +1,167 @@
+# Keyset Pagination
+
+> KEYSET PAGINATION NAVIGATES LARGE RESULT SETS BY REMEMBERING THE LAST SEEN ROW'S SORT KEY RATHER THAN COUNTING SKIPPED ROWS — EACH PAGE QUERY BECOMES AN EFFICIENT INDEX SEEK THAT PERFORMS IDENTICALLY WHETHER YOU ARE ON PAGE 1 OR PAGE 1,000,000, MAKING IT THE ONLY PAGINATION STRATEGY THAT SCALES RELIABLY BEYOND TEN MILLION ROWS.
+
+## When to Use
+
+- Paginating tables with 10M+ rows where deep offset scans cause unacceptable query latency
+- Designing an export or ETL endpoint that iterates all rows of a large table without holding a database cursor open
+- Building an audit log or event stream where strict sequential consistency matters more than random access
+- Replacing an offset pagination endpoint that has hit Elasticsearch's or PostgreSQL's deep-offset performance wall
+- Implementing cursor pagination and choosing what data to encode inside the cursor token
+- Designing a list endpoint for a resource sorted by a composite key (e.g., `tenant_id + created_at + id`)
+- Reviewing a PR that uses OFFSET on a table expected to exceed one million rows
+- Evaluating performance characteristics of keyset vs. cursor vs. offset for a specific access pattern
+
+## Instructions
+
+### Key Concepts
+
+1. **The Seek Method** — Keyset pagination uses a `WHERE` predicate on the sort key columns rather than `OFFSET`. Given the last seen row with sort values `(col_a = v_a, col_b = v_b)`, the next page query is `WHERE (col_a, col_b) > (v_a, v_b) ORDER BY col_a, col_b LIMIT n`. The database resolves this with a single index seek to `(v_a, v_b)` and reads forward — no rows are scanned and discarded. Query cost is constant regardless of depth.
+
+2. **Composite Key Design** — The sort key must uniquely identify a row to guarantee no rows are skipped or duplicated across page boundaries. A single non-unique column (e.g., `status`) is insufficient. A composite key of `(sort_column, id)` — where `id` is the unique primary key — always produces a unique, stable sort position even when `sort_column` has duplicate values.
+
+   ```sql
+   -- Wrong: status is not unique; rows with identical status straddle pages arbitrarily
+   WHERE status > 'active' ORDER BY status
+
+   -- Correct: (status, id) is unique and stable
+   WHERE (status, id) > ('active', 8821) ORDER BY status, id
+   ```
+
+3. **Sort Order Stability** — The sort order applied in the `WHERE` clause must exactly match the `ORDER BY` clause. Mixing ascending and descending directions across columns requires adjusting the row-value comparison accordingly:
+
+   ```sql
+   -- All DESC: reverse the inequality
+   WHERE (created_at, id) < ('2024-03-15T10:22:00Z', 8821)
+   ORDER BY created_at DESC, id DESC
+   ```
+
+   An index on `(created_at DESC, id DESC)` resolves this with a single seek.
+
+4. **Index Requirements** — Keyset pagination is only fast when a composite index exists on exactly the columns used in the `WHERE` and `ORDER BY` clauses, in the same order and direction. Missing or misaligned indexes cause full table scans that are worse than offset pagination. Verify the query plan shows an index seek (`Index Scan` or `Index Only Scan` in PostgreSQL, not `Seq Scan`).
+
+5. **No Random Access** — Keyset pagination cannot jump to page N without traversing pages 1 through N-1. There is no equivalent of `?page=50`. Callers must follow the `next_key` token sequentially. This makes keyset pagination incompatible with numbered page controls but ideal for sequential iteration and infinite scroll.
+
+6. **Null Handling** — Null values in sort key columns require explicit handling. In SQL, `NULL` sorts last in ascending order and first in descending order by default (database-dependent). If sort key columns are nullable, add a `IS NOT NULL` constraint in the `WHERE` clause or encode null handling explicitly in the cursor.
+
+### Worked Example
+
+Stripe's Events API exposes a high-volume append-only log of all events for an account. Stripe uses keyset pagination internally, surfacing it via `starting_after` and `ending_before` parameters that accept resource IDs (opaque to callers but encoded from the primary key + timestamp):
+
+**Request first page of events:**
+
+```http
+GET /v1/events?limit=3&type=payment_intent.succeeded
+Authorization: Bearer sk_example_...
+```
+
+```http
+HTTP/1.1 200 OK
+Content-Type: application/json
+
+{
+  "object": "list",
+  "data": [
+    { "id": "evt_001", "type": "payment_intent.succeeded", "created": 1710500400 },
+    { "id": "evt_002", "type": "payment_intent.succeeded", "created": 1710500200 },
+    { "id": "evt_003", "type": "payment_intent.succeeded", "created": 1710500100 }
+  ],
+  "has_more": true,
+  "url": "/v1/events"
+}
+```
+
+**Request next page using `starting_after` with the last seen ID:**
+
+```http
+GET /v1/events?limit=3&type=payment_intent.succeeded&starting_after=evt_003
+Authorization: Bearer sk_example_...
+```
+
+```http
+HTTP/1.1 200 OK
+
+{
+  "object": "list",
+  "data": [
+    { "id": "evt_004", "type": "payment_intent.succeeded", "created": 1710500000 },
+    { "id": "evt_005", "type": "payment_intent.succeeded", "created": 1710499900 },
+    { "id": "evt_006", "type": "payment_intent.succeeded", "created": 1710499800 }
+  ],
+  "has_more": true
+}
+```
+
+The underlying query is equivalent to:
+
+```sql
+SELECT * FROM events
+WHERE type = 'payment_intent.succeeded'
+  AND (created, id) < (1710500100, 'evt_003')
+ORDER BY created DESC, id DESC
+LIMIT 4  -- fetch 4 to determine has_more
+```
+
+### Anti-Patterns
+
+1. **Using a non-unique column as the sole keyset column.** If two rows share the same value in the sort column, the seek predicate `WHERE col > value` may skip one of them or include both depending on which side of the page boundary they fall. Rows in the tie zone shift unpredictably as pages are requested. Always add the primary key as a tiebreaker to make every sort position unique.
+
+2. **Applying keyset pagination without a matching composite index.** The entire performance benefit of keyset pagination depends on an index seek. If the index does not exist or is ordered differently than the query, the database falls back to a sequential scan. Always run `EXPLAIN` before deploying a keyset-paginated endpoint and confirm the plan shows an index seek. A missing index on a 50M-row table can make page 2 slower than offset page 1.
+
+3. **Trying to support random page access with keyset pagination.** Adding a `?page=N` shortcut to a keyset endpoint requires executing N-1 seeks to find the start of page N, which is equivalent to the offset scan keyset is designed to avoid. If random access by page number is genuinely required, use offset pagination for those endpoints and keyset for the sequential-export endpoints. Do not hybridize.
+
+4. **Returning the raw sort key in the API response without encoding it.** Returning `"next_after_created": "2024-03-15T10:22:00Z"` leaks the internal sort schema and prevents future changes to the sort key design. Wrap the keyset values in a base64-encoded opaque cursor token, just as with cursor pagination, so the encoding can evolve without a breaking API change.
+
+## Details
+
+### Row-Value Comparison Support
+
+| Database       | Row-value comparison       | Composite index seek | Notes                                     |
+| -------------- | -------------------------- | -------------------- | ----------------------------------------- |
+| PostgreSQL 14+ | yes                        | yes                  | Full support; index-only scan possible    |
+| MySQL 8.0+     | yes                        | yes                  | Supported; verify index direction matches |
+| SQLite 3.37+   | yes                        | yes                  | Full support                              |
+| SQL Server     | limited                    | partial              | Use equivalent AND/OR expansion           |
+| DynamoDB       | yes (via LastEvaluatedKey) | yes (sort key)       | Native keyset via `ExclusiveStartKey`     |
+
+For SQL Server, expand row-value comparison manually:
+
+```sql
+WHERE (created_at < @v_created)
+   OR (created_at = @v_created AND id < @v_id)
+```
+
+### Real-World Case Study: Shopify Admin API
+
+Shopify's Admin REST API historically used offset pagination on high-volume endpoints like `GET /admin/api/2024-01/orders.json`. As merchant stores scaled to millions of orders, deep offset queries (`?page=500&limit=250`) caused timeout errors on the database tier. Shopify migrated these endpoints to keyset pagination using `page_info` cursor tokens (base64-encoded keyset values) in 2020. After migration, Shopify reported that 95th-percentile query latency for paginated order list requests dropped from 3.2 seconds (at deep offsets) to under 40ms at any page depth — an 80x improvement at the tail. The offset-based `page` parameter was deprecated and removed in API version 2021-04.
+
+## Source
+
+- [Use The Index, Luke — The Seek Method (No Offset)](https://use-the-index-luke.com/no-offset/)
+- [Shopify API — Paginate results with the REST Admin API](https://shopify.dev/docs/api/usage/pagination-rest)
+- [Stripe API — Pagination](https://stripe.com/docs/api/pagination)
+- [PostgreSQL — Row Constructors](https://www.postgresql.org/docs/current/sql-expressions.html#SQL-SYNTAX-ROW-CONSTRUCTORS)
+- [Markus Winand — We need tool support for keyset pagination](https://use-the-index-luke.com/blog/2013-07/pagination-done-the-postgresql-way)
+
+## Process
+
+1. Identify the sort columns: choose an immutable or append-only column (e.g., `created_at`) as the primary sort key and add the unique primary key as a tiebreaker to form a composite sort key.
+2. Verify a composite index exists on `(sort_col ASC/DESC, id ASC/DESC)` in the exact direction matching the query. Run `EXPLAIN` to confirm an index seek, not a sequential scan.
+3. Implement the seek predicate as a row-value comparison: `WHERE (sort_col, id) > (last_sort_val, last_id)` for ascending, `< ` for descending.
+4. Encode the last row's sort key values as a base64-encoded opaque cursor token in the response. Accept this token as the `after` parameter on subsequent requests. Validate the token on decode.
+5. Fetch `LIMIT n+1` rows and return only `n`; set `has_more = true` if the extra row exists. Do not run a separate COUNT query.
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** api-pagination-cursor, api-pagination-offset, db-btree-index, db-covering-index
+
+## Success Criteria
+
+- Every keyset-paginated endpoint has a composite index on its sort key columns; query plans show index seeks, not sequential scans.
+- Sort keys are composite — primary sort column plus primary key — ensuring uniqueness at every page boundary.
+- Keyset values returned to clients are wrapped in opaque base64-encoded cursor tokens, not exposed as raw column values.
+- `has_more` is determined by fetching `n+1` rows; no COUNT(\*) query runs per page request.
+- Deep-page latency (page 10,000+) is within 10% of first-page latency under production load.

package/dist/agents/skills/gemini-cli/api-pagination-keyset/skill.yaml

@@ -0,0 +1,39 @@
+name: api-pagination-keyset
+version: '1.0.0'
+description: Keyset (seek) pagination with composite key design, sort order stability, and consistent performance at 10M+ rows
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - api-pagination-cursor
+  - api-pagination-offset
+  - db-btree-index
+  - db-covering-index
+stack_signals:
+  - rest
+  - api
+  - pagination
+keywords:
+  - keyset-pagination
+  - seek-method
+  - after-key
+  - stable-sort
+  - composite-key
+  - deep-pagination
+metadata:
+  author: community
+  upstream: use-the-index-luke.com/no-offset/
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/gemini-cli/api-pagination-offset/SKILL.md

@@ -0,0 +1,135 @@
+# Offset/Limit Pagination
+
+> OFFSET/LIMIT PAGINATION IS THE SIMPLEST PAGINATION MODEL BUT CARRIES HIDDEN COSTS — COUNT(\*) QUERIES ARE EXPENSIVE ON LARGE TABLES, CONCURRENT INSERTS AND DELETES CAUSE PAGE DRIFT THAT DUPLICATES OR SKIPS ROWS, AND HIGH OFFSETS FORCE FULL INDEX SCANS THAT MAKE DEEP PAGES ORDERS OF MAGNITUDE SLOWER THAN PAGE ONE.
+
+## When to Use
+
+- Building an admin UI with numbered page controls where users jump directly to page N
+- Designing a read-heavy endpoint on a table with infrequent writes where page drift is acceptable
+- Implementing search results pagination where the total hit count must be displayed
+- Prototyping a list endpoint before write patterns are well understood
+- Reviewing a PR that adds offset pagination to a high-write feed or activity log
+- Evaluating whether an existing offset endpoint should migrate to cursor or keyset pagination
+- Designing export endpoints where the client iterates all rows in order and write concurrency is low
+- Supporting a legacy frontend that stores and navigates by page number in the URL
+
+## Instructions
+
+### Key Concepts
+
+1. **Offset and Limit** — `LIMIT n` restricts the result set to `n` rows. `OFFSET k` skips the first `k` rows before applying the limit. Together they select a window: `OFFSET 20 LIMIT 10` returns rows 21–30. The API surface maps directly: `?page=3&per_page=10` translates to `OFFSET (3-1)*10 = 20`.
+
+   ```sql
+   SELECT * FROM articles
+   ORDER BY created_at DESC
+   LIMIT 10 OFFSET 20;
+   ```
+
+2. **Total Count and Pagination Math** — Offset pagination almost always requires a total count so the client can render page numbers and a "last page" link. This requires a separate `SELECT COUNT(*) FROM articles WHERE ...` query using the same filters. On tables with millions of rows, this query can take hundreds of milliseconds and scales poorly. Cache the count aggressively or compute it asynchronously.
+
+3. **Page Drift** — If a row is inserted at the top of a sorted result set between page 1 and page 2 requests, every subsequent page shifts by one row. Page 2 now starts where page 1 ended — the first row of page 2 duplicates the last row of page 1. Conversely, a deletion causes a row to be skipped. Page drift is unavoidable in offset pagination under concurrent writes and must be documented as a known limitation.
+
+4. **Deep Offset Performance** — Database engines execute `OFFSET k` by fetching and discarding `k` rows before returning the result window. At `OFFSET 100000`, the engine scans and discards 100,000 rows on every page request regardless of indexes. This makes page 10,000 exponentially more expensive than page 1. Impose a maximum offset (e.g., `OFFSET <= 10000`) and redirect callers to cursor pagination for deeper traversal.
+
+5. **UI Implications** — Numbered page controls, "Go to page N" inputs, and total-result counts are natural UI patterns for offset pagination. Infinite scroll and "load more" patterns are a poor fit — they mask page drift and encourage deep offsets. For infinite scroll, prefer cursor pagination and omit total counts.
+
+### Worked Example
+
+The Twilio REST API uses offset pagination on its resource list endpoints, providing `page`, `page_size`, `first_page_uri`, `next_page_uri`, and `previous_page_uri` in every response:
+
+**Request page 1:**
+
+```http
+GET /2010-04-01/Accounts/AC123/Messages.json?PageSize=2
+Authorization: Basic base64(ACsid:auth_token)
+```
+
+```http
+HTTP/1.1 200 OK
+Content-Type: application/json
+
+{
+  "messages": [
+    { "sid": "MM001", "body": "Hello world",  "date_created": "2024-03-15" },
+    { "sid": "MM002", "body": "How are you?", "date_created": "2024-03-14" }
+  ],
+  "page": 0,
+  "page_size": 2,
+  "first_page_uri": "/2010-04-01/Accounts/AC123/Messages.json?Page=0&PageSize=2",
+  "next_page_uri": "/2010-04-01/Accounts/AC123/Messages.json?Page=1&PageSize=2&PageToken=PASMxxx",
+  "previous_page_uri": null,
+  "uri": "/2010-04-01/Accounts/AC123/Messages.json?Page=0&PageSize=2"
+}
+```
+
+**Request page 2:**
+
+```http
+GET /2010-04-01/Accounts/AC123/Messages.json?Page=1&PageSize=2&PageToken=PASMxxx
+Authorization: Basic base64(ACsid:auth_token)
+```
+
+Note that Twilio augments the page number with a `PageToken` to mitigate drift — a hybrid approach that preserves numbered navigation while reducing the worst-case drift to a single page boundary.
+
+### Anti-Patterns
+
+1. **Running COUNT(\*) on every page request without caching.** A live `SELECT COUNT(*) FROM large_table WHERE ...` on every API call is frequently the slowest query in the request cycle. For tables with 1M+ rows and complex filter predicates, this query can exceed 500ms. Cache the count with a short TTL (30–60 seconds), compute it in the background, or use approximate counts from database statistics. Document that totals may be slightly stale.
+
+2. **Allowing unbounded offsets.** An endpoint that accepts `?page=999999` will attempt `OFFSET 9999990` on the database, which scans and discards nearly 10 million rows. This is trivially exploitable as a denial-of-service vector and degrades performance for all other requests. Cap the maximum page number (e.g., 500 pages) and return `400 Bad Request` with a suggestion to use cursor pagination for deeper access.
+
+3. **Using offset pagination for high-write collections without documenting page drift.** An activity feed, notification inbox, or audit log with frequent inserts will exhibit visible page drift on every client that holds a page number across network requests. Clients scrolling quickly will see duplicate entries or miss events entirely. Either switch to cursor pagination, document the limitation prominently, or add a `as_of` timestamp parameter that locks the result set to a snapshot.
+
+4. **Returning a zero-indexed page number without documenting it.** Some APIs use `page=0` for the first page (zero-indexed); others use `page=1` (one-indexed). Mixing conventions causes off-by-one errors that are invisible until a client skips the first page in production. Pick one convention, document it clearly, and validate that `page=0` on a one-indexed API returns `400 Bad Request` rather than silently returning no results.
+
+## Details
+
+### Performance Degradation at Scale
+
+| Rows in table | OFFSET 0 | OFFSET 10,000 | OFFSET 100,000 | OFFSET 1,000,000 |
+| ------------- | -------- | ------------- | -------------- | ---------------- |
+| 1M rows       | ~1ms     | ~12ms         | ~120ms         | N/A (hard cap)   |
+| 10M rows      | ~1ms     | ~15ms         | ~150ms         | ~1,500ms         |
+| 100M rows     | ~2ms     | ~20ms         | ~200ms         | ~2,000ms+        |
+
+These are approximate query times for a single-column ORDER BY on an indexed column in PostgreSQL. Actual times vary by hardware and query complexity. The takeaway is linear degradation: deep pages are not a corner case — they are a predictable scaling cliff.
+
+### Hybrid Approaches
+
+When the UI genuinely requires page numbers but the dataset is live, consider:
+
+- **Keyset-anchored offset:** Capture the `id` of the first row on page 1 as a snapshot anchor. Subsequent page requests use `WHERE id <= anchor_id` to limit the scan to the snapshot window, then apply OFFSET within that window.
+- **Cursor with page number display:** Use cursor pagination internally but map cursor tokens to display page numbers in the client, accepting that "Go to page N" is not supported.
+
+### Real-World Case Study: Elasticsearch / OpenSearch
+
+The Elasticsearch `from`/`size` API (direct equivalent of OFFSET/LIMIT) enforces a hard cap of `index.max_result_window = 10000` by default. Queries with `from + size > 10000` return a `400` error. This cap exists because Elasticsearch must fetch and rank `from + size` documents across all shards before discarding the first `from` results — at large offsets, coordinator memory usage becomes a cluster stability risk. Teams that hit this limit are redirected to the `search_after` API (equivalent to keyset pagination), which is Elasticsearch's recommended pattern for deep pagination of large result sets.
+
+## Source
+
+- [MDN — HTTP Range Requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests)
+- [PostgreSQL — LIMIT and OFFSET](https://www.postgresql.org/docs/current/queries-limit.html)
+- [Twilio REST API Pagination](https://www.twilio.com/docs/usage/api/paging)
+- [Elasticsearch — Paginate search results](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html)
+- [Use The Index, Luke — No Offset](https://use-the-index-luke.com/no-offset/)
+
+## Process
+
+1. Determine the access pattern: if the UI displays page numbers and total count and the dataset has low write frequency, offset pagination is appropriate. If the dataset is a feed or log with high write frequency, prefer cursor or keyset pagination.
+2. Choose a page-numbering convention (zero-indexed or one-indexed), document it in the API reference, and validate that out-of-range page numbers return `400 Bad Request`.
+3. Implement the COUNT query for total results. Cache the result with a TTL appropriate to the write frequency of the underlying data.
+4. Impose a maximum offset cap (e.g., `page * per_page <= 10000`) and return `400 Bad Request` with an error body that references cursor pagination as the alternative.
+5. Document page drift as a known limitation in the API reference, especially for collections that receive concurrent writes.
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** api-pagination-cursor, api-pagination-keyset, api-filtering-sorting
+
+## Success Criteria
+
+- Every offset endpoint documents its maximum page depth and returns `400 Bad Request` for requests that exceed it.
+- COUNT queries are cached or computed asynchronously; no live COUNT(\*) runs on every page request for tables with 100k+ rows.
+- Page-numbering convention (zero- or one-indexed) is documented and validated in request handling.
+- Page drift is documented as a known limitation on any endpoint where the underlying collection receives concurrent writes.
+- API documentation explicitly recommends cursor or keyset pagination for use cases that require deep traversal or drift-free consistency.

package/dist/agents/skills/gemini-cli/api-pagination-offset/skill.yaml

@@ -0,0 +1,39 @@
+name: api-pagination-offset
+version: '1.0.0'
+description: Offset/limit pagination including COUNT(*) costs, page drift on inserts/deletes, max offset limits, and UI implications
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - api-pagination-cursor
+  - api-pagination-keyset
+  - api-filtering-sorting
+stack_signals:
+  - rest
+  - api
+  - pagination
+keywords:
+  - offset-pagination
+  - limit
+  - offset
+  - page
+  - page-size
+  - total-count
+  - page-drift
+metadata:
+  author: community
+  upstream: developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
+state:
+  persistent: false
+  files: []
+depends_on: []