Designs correct cursor pagination and incremental delta sync against a mutating dataset - cursor contracts, updated_at watermarks, delete propagation, checkpointing, and idempotent reprocessing. Use when someone says "my sync is skipping rows", "should I use cursor or offset pagination", "design the pagination contract for this list endpoint", "keep a local copy of this API in sync", or has an offset/page=N loop against changing data. Do NOT use for generating a typed API client or SDK from a spec - use api-client-generator instead; do NOT use for on-device offline-first sync with conflict resolution between a mobile client and server - use mobile-offline-sync instead; this skill owns server-to-server paging and one-way replication correctness.
Click to play with sound.
---
name: Pagination and Sync Engineer
description: Designs correct cursor pagination and incremental delta sync against a mutating dataset - cursor contracts, updated_at watermarks, delete propagation, checkpointing, and idempotent reprocessing. Use when someone says "my sync is skipping rows", "should I use cursor or offset pagination", "design the pagination contract for this list endpoint", "keep a local copy of this API in sync", or has an offset/page=N loop against changing data. Do NOT use for generating a typed API client or SDK from a spec - use api-client-generator instead; do NOT use for on-device offline-first sync with conflict resolution between a mobile client and server - use mobile-offline-sync instead; this skill owns server-to-server paging and one-way replication correctness.
---
# Pagination and Sync Engineer
Make paging and delta sync correct against a dataset that mutates mid-walk: never skip, never double-count, and survive interruption. A naive page loop drops and duplicates rows the instant the underlying data changes, and the corruption is silent - you find out weeks later when a customer asks why a record is missing.
## Operating procedure
Steps 1-2 fix the read contract; 3-4 fix the sync semantics; 5-6 make it survivable. Order matters: a watermark on top of broken ordering is still broken.
### Step 1: Gather inputs
- Dataset size and write rate. A small, bounded, rarely-changing dataset (under ~10k rows, changes daily or less) can be fetched in one full pull - skip the machinery. Label estimates as estimates.
- Does the source expose a cursor, an updated_at filter, soft deletes or a tombstone/changes feed? Each missing capability forces a workaround below.
- Consistency target: how stale may the local copy be?
### Step 2: Choose cursor over offset - the decision rule
Offset (LIMIT/OFFSET, page=N) is correct only when the dataset is immutable for the duration of the walk AND small enough that deep offsets stay fast. Everything else gets keyset/cursor pagination, because an insert or delete before your offset shifts every later row, skipping or repeating records, and OFFSET N scans N rows server-side.
Cursor contract rules when you own the API:
- Encode the last item's full sort key (e.g. base64 of `{"updated_at": "...", "id": "..."}`) and treat it as opaque to clients - document that parsing it voids the contract.