@motionstudies/data 0.1.0-alpha.6 → 0.1.0-alpha.8

package/README.md

@@ -5,7 +5,7 @@ Shared packages for the Motion Studies transport instrument. The source workspac
 - `@motionstudies/core`: transport contracts, indexing, interpolation and visual theme contracts; no browser or Node dependencies.
 - `@motionstudies/three`: `NationalNetworkScene`, `HubPulseScene`, `StationFlowScene`, camera framing and label-mode contracts. React, React Three Fiber and Three.js are peers; rendering internals are not public subpaths.
 - `@motionstudies/web`: picker, button tooltips, theme application, mounting, progressive loaders, observed operations and recording. Import `tokens.css` and `mobile-picker.css` for isolated widgets. `shell.css` is an optional full-page study shell scoped to `.motion-study`; `mountMotionStudy` applies that class. Fonts and edition layouts belong to consumers.
-- `@motionstudies/data`: Node-only GTFS readers, network chunking, merging and station ranking. ZIP reading requires `unzip` on the host. Source selection, provenance overrides and compilation commands belong to each edition.
+- `@motionstudies/data`: Node-only GTFS readers, ADS-B heatmap compilation, air endpoint enrichment, network chunking, merging and station ranking. ZIP reading requires `unzip` on the host. Source selection, provenance overrides and compilation commands belong to each edition.
 
 ```tsx
 import { MobilePicker } from '@motionstudies/web/components/MobilePicker'
@@ -76,6 +76,86 @@ Pass `labels` for edition translations, `loading`, or a localized `error` and `o
 
 For a custom rail or transport board, import `SplitFlapBoard` from `@motionstudies/web/components/SplitFlapBoard` and `@motionstudies/web/split-flap-board.css`. Supply `columns` (`key`, `label`, `characters`) and `rows` (`id`, `cells`, optional `tone`). Cell text longer than its flap count is visually ellipsized, with the full value retained for assistive technology and hover. `onSelectRow`, `selectedRowId` and `selectionColumn` optionally make one cell per row selectable.
 
+For bus stops and local transport, `DotMatrixBoard` accepts the same rows, selection callbacks, loading state and empty/loading messages. It uses an amber 5 × 7 LED alphabet and its own scoped stylesheet; the flip-board stylesheet is not required. Consumers can switch components without remapping their data.
+
+```tsx
+import { DotMatrixBoard } from '@motionstudies/web/components/DotMatrixBoard'
+import '@motionstudies/web/dot-matrix-board.css'
+
+<DotMatrixBoard
+  label="Bus departures"
+  columns={[
+    { key: 'route', label: 'Route', characters: 4, minCharacters: 4 },
+    { key: 'destination', label: 'Destination', characters: 24 },
+    { key: 'time', label: 'Due', characters: 6, minCharacters: 6, align: 'right' },
+  ]}
+  rows={[{ id: 'bus-71', cells: { route: '71', destination: 'City Centre', time: '2 min' } }]}
+  lineCount="auto"
+  style={{ height: 360 }}
+/>
+```
+
+`lineCount` defaults to six display slots and accepts 1–30 (values outside that range are clamped; non-finite values use six). Fixed counts keep all slots and scale their contents to the available height, so dense boards need taller containers to remain readable. `"auto"` observes the actual container and fits 1–30 rows at a target `minRowHeight` of 34px. The default board height is 320px; use `style`, `className`, or `height: '100%'` inside a parent with a defined height. Unused slots stay blank; rows beyond the visible slots are omitted, without pagination or changes to consumer selection.
+
+For dot-matrix columns, `characters` is a width weight, while `minCharacters` reserves space before the remaining width is distributed. At very narrow widths all columns scale down. Long cell values are visually ellipsized, with their full original text available to assistive technology and on hover. Characters outside the bitmap alphabet (including accented names and non-Latin scripts) use SVG text as a visual fallback. `--matrix-ink` and `--matrix-unlit` customize the LEDs. Updates are immediate, with no flashing or scrolling animation. The **Bus boards** lab specimen exercises both presentations, height/width resizing, row counts, selection, long names, loading and empty states using synthetic timetable data.
+
+`variant="uk-rail"` gives the matrix a square black enclosure, mixed-case amber lettering, matrix column headings and horizontal display bands. Optional `heading` and `headingColumnSpan` replace the visual labels across the leading columns while preserving the individual accessible column headers. `footerLabel` and `clockLabel` add a matrix footer; the consumer owns clock formatting and updates. No live clock or pagination is inferred. `DotMatrixRow.note` adds a detail line and an accessible description on the selection button. Details consume one display slot and stay with their departure: if only one slot remains, the next departure with a detail waits until there is room for both. A one-line board still shows its first departure, retaining its full note for assistive technology.
+
+## Rail, bus and airport hero cards
+
+Hero cards have separate transport-specific APIs and visual identities, with shared display components underneath:
+
+| Card | Identity | Departure fields | Default display |
+| --- | --- | --- | --- |
+| `RailStationHeroCard` | Station name, optional code and locality | Scheduled time, destination, platform, expected time/status, via/service note | UK rail matrix |
+| `BusStopHeroCard` | Stop name, optional stop code and locality | Route, destination, due estimate, optional via | Bus dot matrix |
+| `AirportHeroCard` | IATA code, airport name and city | Flight, time, destination/origin, gate, remarks, direction tabs | Split flap |
+
+Import the new cards from `@motionstudies/web/components/RailStationHeroCard` or `@motionstudies/web/components/BusStopHeroCard`, plus `@motionstudies/web/transport-hero-cards.css` (which includes both board styles). Airport imports and study-window behavior remain as documented above.
+
+```tsx
+<RailStationHeroCard
+  station={{ name: 'Bristol Temple Meads', code: 'BRI', locality: 'Bristol' }}
+  departures={[{
+    id: 'train-1', time: '17:15', destination: 'Portsmouth Harbour',
+    platform: '9', expected: '17:22', via: 'Eastleigh',
+  }]}
+  lineCount="auto"
+  boardHeight={400}
+  clockLabel="16:49:26"
+  footerLabel="Study timetable"
+  note="Synthetic timetable. Example times, not a live service."
+/>
+
+<BusStopHeroCard
+  stop={{ name: 'Anchor Road', code: 'A1', locality: 'Bristol' }}
+  departures={[{ id: 'bus-1', route: '71', destination: 'City Centre', due: '2 min' }]}
+  lineCount={6}
+  note="Synthetic timetable. Example estimates, not a live service."
+/>
+```
+
+Rail and bus consumers supply already ordered and formatted departures, including their own filtering, timezones and freshness. Missing times, platforms and statuses stay unknown rather than becoming “On time”. Both cards accept `presentation` (`'uk-rail'`, `'dot-matrix'`, `'split-flap'`), `lineCount`, `boardHeight` in pixels, `minRowHeight`, `loading`, `error`, `onRetry`, localized `labels`, and controlled `onSelectDeparture`/`selectedDepartureId`. A required `note` explains the source. The matrix fits its container; the split-flap alternative contains scrolling and places any service details in an Information column. Fixed line counts represent physical matrix lines; in split-flap mode they limit departure rows. The **Transport heroes** lab compares all three cards, switches the new cards' presentations, and exercises details, row fitting, updates, long names and failure states.
+
+The rail card additionally accepts `presentation="sbb"`: a blue-and-white typographic departure board with service badges, scheduled time, destination/via information and prominent track numbers. This layout follows the information hierarchy in [SBB's general display guide](https://www.sbb.ch/en/travel-information/stations/services-station/station-customer-information/general-display-board.html). It uses ordinary text, including accented and non-Latin names. `RailDeparture.service` supplies a train label such as `IC 1`; optional `serviceCategory` (`'intercity'`, `'international'`, `'regional'`, `'suburban'`) selects the badge treatment. `platformSector` supplies a separate sector label when known. Expected times or disruption messages appear below the scheduled time; absence of a message does not manufacture an “On time” assertion.
+
+For this layout, `lineCount` counts departures with their inline detail, and `"auto"` fits rows using a default target height of 64px. Dense fixed counts reduce type size; long values remain in the accessible text and hover titles. Labels stay consumer-owned, including the added `service` column label. The lab's **SBB departure board** option selects a synthetic Zürich HB example with German, French, Italian and English labels. Both new board styles are included in `transport-hero-cards.css`.
+
+At compact widths the SBB layout stacks the service badge under the time, preserving destination space and the separate track column. Rail and bus hero padding follows the card width rather than the viewport. The lab includes a 240–980px width slider and Compact/Mobile/Panel/Wide presets, plus a 180–640px board-height control. The size regression suite covers seven card widths, four board heights, fixed and automatic line counts, long destinations, selection during updates, mobile viewports and loading/error states in Chromium and WebKit. Prefer `lineCount="auto"` for small panels; high fixed line counts deliberately trade text size for density.
+
+```tsx
+<RailStationHeroCard
+  presentation="sbb"
+  station={{ name: 'Zürich HB' }}
+  labels={{ station: 'Bahnhof', departures: 'Abfahrt', service: 'Zug', time: 'Zeit', destination: 'Nach', platform: 'Gleis' }}
+  departures={[{ id: 'example-1', service: 'IC 1', serviceCategory: 'intercity',
+    time: '09:02', destination: 'Genève-Aéroport', via: 'Bern · Lausanne', platform: '32', platformSector: 'ABCD' }]}
+  lineCount="auto"
+  boardHeight={430}
+  note="Synthetic timetable · Example data."
+/>
+```
+
 The shared board also accepts `loading`, a localized `loadingMessage`, and `loadingRows` (default five). While loading, its decorative rows cycle through staggered letters and digits; they are hidden from assistive technology and cannot be selected. A single status message announces loading. When data arrives, characters flip through a short sequence and settle into their actual values; later changes animate only the changed characters. These CSS animations have no JavaScript timers and stop looping when loading ends or the board is removed. Reduced-motion users get static blank loading flaps and immediate final text. `AirportHeroCard` uses this shared loading treatment automatically. Use **Reload board** in the Airports lab to preview the complete loading-to-ready transition.
 
 Empty messages also appear on the flaps, in the widest column (the destination/origin column in airport cards), with the other columns blank. Longer localized messages wrap across display rows instead of being truncated. They settle with the same animation as flight details, and one hidden status announces the complete message to assistive technology. This also applies when the study clock moves into a window with no movements.
@@ -83,3 +163,19 @@ Empty messages also appear on the flaps, in the widest column (the destination/o
 Rail and other transport consumers can share the same time filtering through `movementBoardWindow(study, horizon)` and `movementsForBoard(entries, window, maxRows)` from `@motionstudies/core/domain/movement-board`. Format the returned numeric times when mapping them into `SplitFlapBoard` cells. The lab's rail board follows the same study clock and horizon as its airport card.
 
 `@motionstudies/data/air-endpoints` provides offline `enrichAirEndpoints` for existing air manifests, chunks and opening snapshots. Supply cached same-date global ADSB.lol heatmaps, an OurAirports CSV and the service date's local UTC offset. It associates only unambiguous low-altitude endpoints near a reference airport; cruise-only traces and uncertain routes stay unknown. Optional `AirEndpoint` origin/destination fields carry airport identity, observed boundary time and `observed-endpoint` evidence. `airportBoardMovements` maps full manifest entries to board rows without confusing playback chunk boundaries with flight endpoints. Input hashes and source/licence attribution are recorded in fixture metadata. These fields describe inferred observations, never flight schedules, gates or live status.
+
+## Optional live airport feed
+
+`AirportBoard` from `@motionstudies/web/components/AirportBoard` adds Study/Now controls around an existing `AirportHeroCard` configuration. Pass `studyCard` with the usual card props and `live={{ baseUrl, edition, airport }}` for the shared service. Import `airport-hero-card.css`. `labels` localizes the wrapper's control and availability messages. The lower-level `useAirportFeed` hook and core `domain/live-airport` contract are also public exports.
+
+Live timestamps are Unix seconds and use the airport's timezone for display, independently of recorded service time. The wrapper does not pass live flight IDs to the recorded scene's selection callback. The recorded card remains mounted while hidden; the edition still owns playback and can pause its study when appropriate. Only Now mode fetches flight boards; checking capabilities does not query the paid provider. Stale results carry their retrieval time and disappear when expired. Source data never falls back to synthetic or recorded flights under a live label.
+
+See [service architecture and operations](../docs/LIVE-AIRPORTS.md). The Worker is deployed separately; npm publication and edition adoption remain explicit release steps.
+
+## Shared recorded air compilation
+
+`@motionstudies/data/adsb-heatmap` consolidates the offline heatmap pipeline previously copied between editions. `ingestAdsbHeatmaps` reads cached gzip slices, decodes observations, filters transport-scale tracks, splits flights, and writes either an opening snapshot or an indexed day with overlapping chunks. Source hashes, chunk hashes and ODbL attribution accompany the output. It makes no network requests.
+
+The same `decodeAdsbHeatmap` now powers `enrichAirEndpoints`; endpoint inference retains full coordinate precision, while playback compilation retains the existing five-decimal coordinates. `transportAirTracks` and `chunkAirSnapshot` are also available for consumers that assemble their own pipeline. All functions have public TypeScript declarations and work in the packed Node package.
+
+Editions supply geographic bounds, service date, explicit UTC offset, optional timezone, input files and output paths. Flight IDs and chunk overlap retain the existing contracts. The default splits known callsign changes and gaps over 30 minutes. Set `splitTracks: false` only when reproducing a legacy opening snapshot with one ID per aircraft. See [adoption and compatibility](../docs/AIR-DATA.md).

package/adsb-heatmap.d.mts

@@ -0,0 +1,56 @@
+import type { AirSnapshot, AirTrack } from '@motionstudies/core/domain/air'
+import type { AirDayManifest, AirDayChunk, AirDayChunkDescriptor } from '@motionstudies/core/domain/air-day'
+
+export interface HeatmapWindow {
+  serviceDate: string
+  /** Explicit offset for this recording; the caller owns DST and source date selection. */
+  utcOffsetHours: number
+  windowStart: number
+  windowEnd: number
+  bounds?: readonly [west: number, south: number, east: number, north: number]
+  /** Numeric 24-bit ICAO addresses; omit to decode all aircraft. */
+  addresses?: ReadonlySet<number>
+}
+export type HeatmapSample = [time: number, longitude: number, latitude: number, altitudeFeet: number, speedKnots: number, callsign: string | undefined]
+export interface HeatmapRecord {
+  id: string
+  callsign: string
+  samples: readonly HeatmapSample[]
+}
+export interface HeatmapSource {
+  name: string
+  sha256: string
+  sourceUrl: string
+}
+export type HeatmapMetadata = AirSnapshot['metadata'] & {
+  timezone?: string
+  utcOffsetHours: number
+  sourceFiles: readonly HeatmapSource[]
+}
+export type HeatmapSnapshot = Omit<AirSnapshot, 'metadata'> & { readonly metadata: HeatmapMetadata }
+export type AirChunkIntegrity = AirDayChunkDescriptor & { readonly bytes: number; readonly sha256: string }
+export type ChunkedAirManifest = Omit<AirDayManifest, 'chunks'> & { readonly chunks: readonly AirChunkIntegrity[] }
+export type HeatmapManifest = Omit<ChunkedAirManifest, 'metadata'> & { readonly metadata: HeatmapMetadata }
+
+/** Uncompressed binary input; returns the number of accepted observations. Does not round coordinates. An absent callsign is undefined; an explicitly cleared one is empty. */
+export function decodeAdsbHeatmap(bytes: Uint8Array, options: HeatmapWindow, onSample: (address: number, sample: HeatmapSample) => void): number
+/** Does not mutate the input records. Legacy unsplit snapshots can opt out of flight segmentation. */
+export function transportAirTracks(records: Iterable<HeatmapRecord>, options?: { splitTracks?: boolean }): AirTrack[]
+export function chunkAirSnapshot(snapshot: AirSnapshot, options: {
+  chunkSeconds: number
+  stem: string
+  overlapBefore?: number
+  overlapAfter?: number
+}): { manifest: ChunkedAirManifest; chunks: { descriptor: AirChunkIntegrity; payload: AirDayChunk }[] }
+export interface HeatmapIngestOptions extends HeatmapWindow {
+  inputs: readonly string[]
+  output: string
+  bounds: readonly [west: number, south: number, east: number, north: number]
+  timezone?: string
+  chunkHours?: number
+  /** Defaults true; use false only to preserve a legacy single-track-per-aircraft snapshot. */
+  splitTracks?: boolean
+}
+export function ingestAdsbHeatmaps(options: HeatmapIngestOptions & { chunkHours: number }): Promise<HeatmapManifest>
+export function ingestAdsbHeatmaps(options: HeatmapIngestOptions & { chunkHours?: undefined }): Promise<HeatmapSnapshot>
+export function ingestAdsbHeatmaps(options: HeatmapIngestOptions): Promise<HeatmapSnapshot | HeatmapManifest>

package/adsb-heatmap.mjs

@@ -0,0 +1,223 @@
+import { createHash } from 'node:crypto'
+import { readFile, writeFile } from 'node:fs/promises'
+import { basename, dirname, join } from 'node:path'
+import { gunzipSync } from 'node:zlib'
+
+const MAGIC = 0x0e7f7c9d
+const CALLSIGN_MARKER = 0x40000000
+const NON_ICAO_MARKER = 0x01000000
+
+function serviceDayStart(options) {
+  const { serviceDate, utcOffsetHours, windowStart, windowEnd } = options
+  const midnight = Date.parse(`${serviceDate}T00:00:00Z`)
+  if (!/^\d{4}-\d{2}-\d{2}$/.test(serviceDate) || !Number.isFinite(midnight) || new Date(midnight).toISOString().slice(0, 10) !== serviceDate) {
+    throw new Error('A valid serviceDate (YYYY-MM-DD) is required')
+  }
+  if (!Number.isFinite(utcOffsetHours) || Math.abs(utcOffsetHours) > 14) throw new Error('A valid local UTC offset is required')
+  if (!Number.isFinite(windowStart) || !Number.isFinite(windowEnd) || windowStart >= windowEnd) throw new Error('A finite, increasing study window is required')
+  if (options.bounds) {
+    const [west, south, east, north] = options.bounds
+    if (options.bounds.length !== 4 || !options.bounds.every(Number.isFinite) || west < -180 || east > 180 || south < -90 || north > 90 || west >= east || south >= north) {
+      throw new Error('bounds must be [west, south, east, north] in geographic degrees')
+    }
+  }
+  return midnight - utcOffsetHours * 3600000
+}
+
+/** Decode an uncompressed readsb heatmap slice. Times remain service-local seconds, including after midnight. */
+export function decodeAdsbHeatmap(bytes, options, onSample) {
+  const dayStart = serviceDayStart(options)
+  const buffer = Buffer.isBuffer(bytes) ? bytes : Buffer.from(bytes)
+  if (buffer.length % 16 !== 0) throw new Error('Truncated heatmap record')
+  const [west, south, east, north] = options.bounds ?? [-180, -90, 180, 90]
+  const callsigns = new Map()
+  let time
+  let count = 0
+  for (let offset = 0; offset < buffer.length; offset += 16) {
+    const address = buffer.readUInt32LE(offset), encodedLatitude = buffer.readInt32LE(offset + 4)
+    if (address === MAGIC) {
+      const milliseconds = Number((BigInt(encodedLatitude) << 32n) | BigInt(buffer.readUInt32LE(offset + 8)))
+      time = Math.round((milliseconds - dayStart) / 1000)
+      continue
+    }
+    const id = address & 0xffffff
+    if ((address & NON_ICAO_MARKER) !== 0 || (options.addresses && !options.addresses.has(id))) continue
+    // Negative latitude also sets this bit in two's complement; it is still a position.
+    if (encodedLatitude >= 0 && (encodedLatitude & CALLSIGN_MARKER) !== 0) {
+      callsigns.set(id, buffer.subarray(offset + 8, offset + 16).toString('ascii').replaceAll('\0', '').trim().toUpperCase())
+      continue
+    }
+    if (time === undefined || time < options.windowStart || time > options.windowEnd) continue
+    const latitude = encodedLatitude / 1e6, longitude = buffer.readInt32LE(offset + 8) / 1e6
+    if (longitude < west || longitude > east || latitude < south || latitude > north) continue
+    const altitude = buffer.readInt16LE(offset + 12), speed = buffer.readInt16LE(offset + 14)
+    if (altitude < 0 || speed < 0) continue
+    onSample(id, [time, longitude, latitude, altitude * 25, speed / 10, callsigns.get(id)])
+    count++
+  }
+  return count
+}
+
+function round(value, digits) {
+  const scale = 10 ** digits
+  return Math.round(value * scale) / scale
+}
+
+function transportScaleTrack(record) {
+  const samples = [...record.samples]
+    .sort((first, second) => first[0] - second[0])
+    .filter((sample, index, all) => index === 0 || sample[0] !== all[index - 1][0])
+    .map((sample) => sample.slice(0, 5))
+  if (samples.length < 4) return undefined
+  const maximumSpeed = samples.reduce((maximum, sample) => Math.max(maximum, sample[4]), 0)
+  const maximumAltitude = samples.reduce((maximum, sample) => Math.max(maximum, sample[3]), 0)
+  const airlineCallsign = /^[A-Z]{2,4}\d[A-Z0-9]*$/.test(record.callsign)
+  if (maximumSpeed < 120 || maximumAltitude < 1_500) return undefined
+  if (!airlineCallsign && (maximumSpeed < 250 || maximumAltitude < 10_000)) return undefined
+  return {
+    id: record.id,
+    callsign: record.callsign || record.id.toUpperCase(),
+    start: samples[0][0],
+    end: samples.at(-1)[0],
+    samples,
+  }
+}
+
+function transportScaleTracks(record) {
+  const ordered = [...record.samples]
+    .sort((first, second) => first[0] - second[0])
+    .filter((sample, index, all) => index === 0 || sample[0] !== all[index - 1][0])
+  const segments = []
+  let current = []
+  let currentCallsign = ''
+
+  for (const sample of ordered) {
+    const callsign = sample[5]
+    const previous = current.at(-1)
+    const callsignChanged = callsign && currentCallsign && callsign !== currentCallsign
+    const longGap = previous && sample[0] - previous[0] > 30 * 60
+    if (current.length && (callsignChanged || longGap)) {
+      segments.push({ callsign: currentCallsign, samples: current })
+      current = []
+      currentCallsign = ''
+    }
+    if (callsign) currentCallsign = callsign
+    current.push(sample)
+  }
+  if (current.length) segments.push({ callsign: currentCallsign, samples: current })
+
+  return segments
+    .map((segment) => {
+      const base = transportScaleTrack({
+        id: record.id,
+        callsign: segment.callsign || record.callsign,
+        samples: segment.samples,
+      })
+      if (!base) return undefined
+      return {
+        ...base,
+        id: `${record.id}-${base.start}`,
+        icaoAddress: record.id,
+      }
+    })
+    .filter(Boolean)
+}
+
+
+/** Stable ordering and IDs; split on a known callsign change or an observation gap over 30 minutes. */
+export function transportAirTracks(records, { splitTracks = true } = {}) {
+  return [...records].flatMap(record => splitTracks
+    ? transportScaleTracks(record)
+    : [transportScaleTrack(record)].filter(Boolean))
+    .sort((first, second) => first.id.localeCompare(second.id))
+}
+
+/** Produce playback chunks with the same 180s trail and 45s interpolation overlap used by the editions. */
+export function chunkAirSnapshot(snapshot, { chunkSeconds, stem, overlapBefore = 180, overlapAfter = 45 }) {
+  if (!Number.isInteger(chunkSeconds) || chunkSeconds <= 0) throw new Error('chunkSeconds must be a positive integer')
+  if (![overlapBefore, overlapAfter].every(value => Number.isFinite(value) && value >= 0)) throw new Error('Chunk overlap must be non-negative')
+  if (!/^[a-zA-Z0-9_-]+$/.test(stem)) throw new Error('A simple chunk filename stem is required')
+  const { windowStart: start, windowEnd: end } = snapshot.metadata
+  if (!Number.isFinite(start) || !Number.isFinite(end) || start >= end) throw new Error('A finite, increasing study window is required')
+  const chunks = []
+  for (let windowStart = start; windowStart < end; windowStart += chunkSeconds) {
+    const windowEnd = Math.min(end, windowStart + chunkSeconds)
+    const id = String(Math.floor(windowStart / chunkSeconds)).padStart(2, '0')
+    const tracks = snapshot.tracks.flatMap(track => {
+      const samples = track.samples.filter(sample => sample[0] >= windowStart - overlapBefore && sample[0] <= windowEnd + overlapAfter)
+      return samples.length < 2 ? [] : [{ ...track, start: samples[0][0], end: samples.at(-1)[0], samples }]
+    })
+    const payload = { windowStart, windowEnd, tracks }
+    const serialized = `${JSON.stringify(payload)}\n`
+    chunks.push({ descriptor: {
+      id, windowStart, windowEnd, path: `${stem}-${id}.json`, trackCount: tracks.length,
+      sampleCount: tracks.reduce((sum, track) => sum + track.samples.length, 0),
+      bytes: Buffer.byteLength(serialized), sha256: createHash('sha256').update(serialized).digest('hex'),
+    }, payload })
+  }
+  const aircraft = snapshot.tracks.map(({ samples: _samples, ...track }) => ({
+    ...track, icaoAddress: track.icaoAddress ?? track.id,
+    chunkIds: chunks.filter(({ descriptor }) => track.end >= descriptor.windowStart && track.start <= descriptor.windowEnd)
+      .map(({ descriptor }) => descriptor.id),
+  }))
+  return { manifest: {
+    metadata: snapshot.metadata, bounds: snapshot.bounds, trackCount: snapshot.tracks.length,
+    sampleCount: snapshot.tracks.reduce((sum, track) => sum + track.samples.length, 0),
+    aircraft, chunks: chunks.map(chunk => chunk.descriptor),
+  }, chunks }
+}
+
+function sourceUrl(date) {
+  return `https://github.com/adsblol/globe_history_${date.slice(0, 4)}/releases/tag/v${date.replaceAll('-', '.')}-planes-readsb-prod-0`
+}
+
+/** Compile cached gzip heatmaps. The caller owns source selection, timezone and output location; no network calls. */
+export async function ingestAdsbHeatmaps(options) {
+  serviceDayStart(options)
+  if (!options.bounds) throw new Error('Study bounds are required')
+  if (!options.inputs?.length) throw new Error('At least one cached heatmap input is required')
+  if (options.chunkHours !== undefined && (!Number.isInteger(options.chunkHours) || options.chunkHours < 1)) throw new Error('chunkHours must be a positive integer')
+  if (!options.output?.endsWith('.json')) throw new Error('An output .json path is required')
+  if (options.timezone) new Intl.DateTimeFormat('en', { timeZone: options.timezone })
+  const records = new Map(), sourceFiles = []
+  // Preserve caller ordering for deterministic first-observation wins when input slices overlap.
+  for (const input of options.inputs) {
+    const raw = await readFile(input), name = basename(input)
+    const date = /^\d{4}-\d{2}-\d{2}/.exec(name)?.[0] ?? options.serviceDate
+    sourceFiles.push({ name, sha256: createHash('sha256').update(raw).digest('hex'), sourceUrl: sourceUrl(date) })
+    decodeAdsbHeatmap(gunzipSync(raw), options, (address, sample) => {
+      const id = address.toString(16).padStart(6, '0')
+      const record = records.get(id) ?? { id, callsign: '', samples: [] }
+      record.callsign = sample[5] || record.callsign
+      record.samples.push([sample[0], round(sample[1], 5), round(sample[2], 5), sample[3], round(sample[4], 1), record.callsign])
+      records.set(id, record)
+    })
+  }
+  const tracks = transportAirTracks(records.values(), { splitTracks: options.splitTracks ?? true })
+  const [minLongitude, minLatitude, maxLongitude, maxLatitude] = options.bounds
+  const snapshot = {
+    metadata: {
+      publisher: 'ADSB.lol', ...(options.timezone ? { timezone: options.timezone } : {}),
+      utcOffsetHours: options.utcOffsetHours, sourceFiles,
+      serviceDate: options.serviceDate, windowStart: options.windowStart, windowEnd: options.windowEnd,
+      sourceUrl: sourceUrl(options.serviceDate), license: 'ODbL 1.0',
+      licenseUrl: 'https://opendatacommons.org/licenses/odbl/1-0/',
+      model: 'Observed ADS-B / MLAT heatmap positions replayed at 10-second resolution',
+      note: 'Ground, non-ICAO, low-speed and low-altitude signals are excluded; altitude is rendered with explicit vertical compression.',
+      sampleIntervalSeconds: 10,
+    },
+    bounds: { minLongitude, minLatitude, maxLongitude, maxLatitude }, tracks,
+  }
+  if (options.chunkHours !== undefined) {
+    const { manifest, chunks } = chunkAirSnapshot(snapshot, {
+      chunkSeconds: options.chunkHours * 3600, stem: basename(options.output, '.json').replace(/-manifest$/, ''),
+    })
+    for (const { descriptor, payload } of chunks) {
+      await writeFile(join(dirname(options.output), descriptor.path), `${JSON.stringify(payload)}\n`)
+    }
+    await writeFile(options.output, `${JSON.stringify(manifest)}\n`)
+    return manifest
+  }
+  await writeFile(options.output, `${JSON.stringify(snapshot)}\n`)
+  return snapshot
+}

package/air-endpoints.mjs

@@ -3,6 +3,7 @@ import { dirname, join } from 'node:path'
 import { gunzipSync } from 'node:zlib'
 import { createHash } from 'node:crypto'
 import { parseCsvLine } from './gtfs.mjs'
+import { decodeAdsbHeatmap } from './adsb-heatmap.mjs'
 
 export function parseAirports(csv) {
   const [header, ...lines] = csv.trim().split(/\r?\n/)
@@ -71,36 +72,19 @@ export async function enrichAirEndpoints({ manifestPath, snapshotPaths = [], hea
   const tracks = [...manifest.aircraft, ...snapshots.flatMap(snapshot => snapshot.tracks)]
   const wanted = new Set(tracks.map(track => parseInt(track.icaoAddress ?? track.id.split('-')[0], 16)))
   const records = new Map()
-  const dayStart = Date.parse(`${date}T00:00:00Z`) - utcOffsetHours * 3600000
   const files = (await readdir(heatmapDirectory)).filter(file => file.endsWith('.bin.ttf')).sort()
   if (!files.length) throw new Error('No cached heatmap slices found')
   const sources = []
   for (const file of files) {
     const raw = await readFile(join(heatmapDirectory, file))
-    const buffer = gunzipSync(raw)
-    let time
-    const callsigns = new Map()
-    let used = false
-    for (let offset = 0; offset + 16 <= buffer.length; offset += 16) {
-      const address = buffer.readUInt32LE(offset), latitude = buffer.readInt32LE(offset + 4)
-      if (address === 0x0e7f7c9d) {
-        time = Math.round((Number((BigInt(latitude) << 32n) | BigInt(buffer.readUInt32LE(offset + 8))) - dayStart) / 1000)
-        continue
-      }
-      const id = address & 0xffffff
-      if (!wanted.has(id) || (address & 0x01000000)) continue
-      if (latitude >= 0 && (latitude & 0x40000000) !== 0) {
-        callsigns.set(id, buffer.subarray(offset + 8, offset + 16).toString('ascii').replaceAll('\0', '').trim().toUpperCase())
-        continue
-      }
-      if (time === undefined || time < 0 || time > 86400) continue
-      const altitude = buffer.readInt16LE(offset + 12), speed = buffer.readInt16LE(offset + 14)
-      if (altitude < 0 || speed < 0) continue
+    const used = decodeAdsbHeatmap(gunzipSync(raw), {
+      serviceDate: date, utcOffsetHours, windowStart: 0, windowEnd: 86400, addresses: wanted,
+    }, (id, sample) => {
       const samples = records.get(id) ?? []
-      samples.push([time, buffer.readInt32LE(offset + 8) / 1e6, latitude / 1e6, altitude * 25, speed / 10, callsigns.get(id) ?? samples.at(-1)?.[5] ?? ''])
+      sample[5] ??= samples.at(-1)?.[5] ?? ''
+      samples.push(sample)
       records.set(id, samples)
-      used = true
-    }
+    })
     if (used) sources.push({ file, sha256: createHash('sha256').update(raw).digest('hex') })
   }
   const csv = await readFile(airportCsvPath, 'utf8')

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@motionstudies/data",
-  "version": "0.1.0-alpha.6",
+  "version": "0.1.0-alpha.8",
   "private": false,
   "type": "module",
   "description": "Offline data preparation primitives for Motion Studies.",
@@ -23,6 +23,10 @@
     "node": ">=24.0.0"
   },
   "exports": {
+    "./adsb-heatmap": {
+      "types": "./adsb-heatmap.d.mts",
+      "import": "./adsb-heatmap.mjs"
+    },
     "./air-endpoints": {
       "types": "./air-endpoints.d.mts",
       "import": "./air-endpoints.mjs"
@@ -55,6 +59,6 @@
   ],
   "sideEffects": false,
   "dependencies": {
-    "@motionstudies/core": "0.1.0-alpha.6"
+    "@motionstudies/core": "0.1.0-alpha.8"
   }
 }