@aeroswarm/core 0.0.0-stage → 0.2.9

package/README.md

@@ -1,3 +1,169 @@
-# Temporary Holding Version
+# @aeroswarm/core
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Core infrastructure library shared across all AeroSwarm backend services. Provides configuration, logging, database, caching, messaging, authentication, and observability primitives — plus the canonical event types and validation schemas that define the pipeline's data contract.
+
+## Installation
+
+```bash
+pnpm add @aeroswarm/core
+```
+
+## Entry points — server vs browser
+
+The **root import is server-only**: it pulls in `postgres`, `ioredis`, `kafkajs`, `pino`, and `prom-client`, which must never reach a browser bundle. Browser-safe subsets are published as subpaths:
+
+| Import | Contents | Browser-safe |
+|--------|----------|:---:|
+| `@aeroswarm/core` | Everything (all modules below) | ❌ |
+| `@aeroswarm/core/types` | TypeScript types only | ✅ |
+| `@aeroswarm/core/utils` | Pure helpers (`toCurrentTrack`, `isRenderableTrack`, geo math, ID factories, `TOPICS`) | ✅ |
+| `@aeroswarm/core/validation` | Zod schemas | ✅ |
+
+```ts
+// ✅ frontend / shared code
+import { toCurrentTrack, isRenderableTrack } from "@aeroswarm/core/utils";
+import type { CurrentTrack, FusedTrackEvent } from "@aeroswarm/core/types";
+
+// ✅ backend services only
+import { createLogger, publishEvent, getRedis } from "@aeroswarm/core";
+```
+
+> Importing the bare root from frontend code once shipped the entire server stack (~500 kB) into the dashboard bundle and crashed it with `process is not defined`. Always use the subpaths from anything that runs in a browser.
+
+## Modules & exposed API
+
+### `types` — canonical event model
+
+`Position`, `Velocity`, `ConfidenceBudget`, `Protocol`, `Bbox`, `FusedTrackEvent`, `CurrentTrack`, `TrackStatus`, `TrackSource`, `SourceHealthEvent`, `SourceStatus`, `AlertEvent`, `AlertType`, `AlertSeverity`, `StreamEnvelope` (the WS wire format union: `snapshot | track.updated | track.removed | source.status | stream.heartbeat | stream.degraded | stream.error | alert`), `AeroConnectionStatus`, `AeroRecoverableError`.
+
+Both `FusedTrackEvent` and `AlertEvent` carry an optional `airportId` (ICAO code, e.g. `"WARR"`) for multi-airport tenancy — set by the ingesting service (`AIRPORT_ID` env) and propagated through fusion and safety-engine alerts. `CurrentTrack.airportId` mirrors it for WS filtering (`client.airportId` subscribe param).
+
+### `utils` — pure helpers (browser-safe)
+
+| Export | Purpose |
+|--------|---------|
+| `toCurrentTrack(fused, staleAfterMs?, lostAfterMs?)` | Normalize a `FusedTrackEvent` into a `CurrentTrack` with derived `status` |
+| `isRenderableTrack(t)` | Type guard: track has finite, in-range lat/lon |
+| `isInsideBbox(lat, lon, bbox)` / `haversineKm(lat1, lon1, lat2, lon2)` / `clamp(v, min, max)` | Geo/math helpers |
+| `createTrackId()` / `createAlertId()` / `createSessionId()` / `createEventUid()` / `createNanoid(size?)` / `createEventId(sourceId, icao24, ts)` | ID factories (`trk_`, `alt_`, … prefixes) |
+| `TOPICS` / `TopicName` | Kafka topic name constants (see table below) |
+| `EMERGENCY_SQUAWKS` | `Set{"7500","7600","7700"}` |
+| `STALE_AFTER_MS` / `LOST_AFTER_MS` / `MACH1_KT` | Domain constants |
+
+### `validation` — Zod schemas (browser-safe)
+
+`positionSchema`, `velocitySchema`, `currentTrackSchema`, `streamEnvelopeSchema`, `sourceHeartbeatSchema`, `wsSubscribeSchema`, `apiSuccessSchema`, `apiErrorSchema`, `apiResponseSchema` + inferred input types (`CurrentTrackInput`, `WsSubscribeInput`, `StreamEnvelopeInput`).
+
+### `config` — environment access
+
+| Export | Purpose |
+|--------|---------|
+| `getEnv(name)` / `getEnvNumber(name)` | Typed env access with defaults from `ENV_DEFAULTS` |
+| `ENV_VARS` / `EnvVarName` | Canonical env var name constants |
+| `KAFKA_GROUPS` / `KafkaGroupId` | Consumer group id constants |
+| `SERVICE_PORTS`, `LIFECYCLE_STATES`, `PROTOCOL_TYPES` | Misc constant registries |
+
+### `logger`
+
+`createLogger(name)` → namespaced Pino logger (respects `LOG_LEVEL`); `logger` — the root instance.
+
+### `cache` — Redis client, pub/sub, and domain caches
+
+| Export | Purpose |
+|--------|---------|
+| `getRedis()` / `disconnectRedis()` | Singleton ioredis client (Sentinel-aware via `REDIS_SENTINEL_URLS`) |
+| `PUB_SUB_CHANNELS` | `{ ALERT: "aero.alert" }` — the only channel with a live subscriber (client-service's real-time alert banner); FUSED/HEALTH moved to Kafka-only after a double-processing bug (see federation docs 006) |
+| `pubAlert(event)` | Fire-and-forget pub/sub publish (failures logged, rate-limited) |
+| `subChannels(channels, handler)` | Subscribe to pub/sub channels; returns unsubscribe fn |
+| `setTrack` / `getTrack` / `deleteTrack` | Per-track cache (60 s TTL) |
+| `setSourceHealth` / `getSourceHealth` | Source health cache (120 s TTL) |
+| `cacheSourceToken` / `validateSourceToken(token, fallbackFn?)` / `revokeSourceToken` | Source token auth cache |
+| `checkRateLimit(key, limit, windowS)` | Sliding-window rate limiter |
+| `registerWsClient` / `unregisterWsClient` / `getWsClientCount` | WS client registry |
+| `setSnapshot` / `getSnapshot` | Short-TTL (5 s) response snapshot cache |
+| `getSystemConfig` / `setSystemConfig` | Runtime-tunable system config KV |
+
+### `messaging` — Kafka producer/consumer
+
+| Export | Purpose |
+|--------|---------|
+| `publishEvent(topic, value, key?)` | JSON+gzip publish with backpressure cap and auto-reconnect; delivery failures are logged (rate-limited), never thrown |
+| `createConsumer({ groupId, topics, handler, fromBeginning?, … })` | Connected consumer; returns `{ disconnect, pause, resume }` |
+| `getProducer()` / `getKafka()` / `disconnectProducer()` | Lower-level access |
+| `ensureTopics()` | Create all registry topics if missing |
+| `TOPIC_REGISTRY` / `getTopicInfo` / `isValidTopic` / `validateTopics` | Topic metadata & validation |
+| `partitionKeys` | Key builders: `icao(…)`, `track(…)`, `source(…)`, `target(…)`, `alert(…)` |
+| `getConsumerGroupHealth(groupId)` / `isConsumerGroupHealthy(groupId)` | Consumer group introspection |
+
+### `metrics` — Prometheus
+
+`registry` plus pre-registered instruments: `wsClients`, `trackUpdateRate`.
+
+### `health`
+
+`buildHealthResponse(checks)` + `HealthStatus` — each service wires its own dependency checks (DB/Kafka/Redis) through this shared contract; see each service's `src/health/index.ts`.
+
+## Kafka topics (`TOPICS`)
+
+| Constant | Topic |
+|----------|-------|
+| `TOPICS.FUSED` | `aero.fused.track.v1` |
+| `TOPICS.TRUST` | `aero.trust.assessment.v1` |
+| `TOPICS.ALERT` | `aero.alert.v1` |
+| `TOPICS.ALERT_PRIORITY` | `aero.alert.priority.v1` |
+| `TOPICS.HEALTH` | `aero.source.health.v1` |
+| `TOPICS.LIFECYCLE` | `aero.track.lifecycle.v1` |
+| `TOPICS.DEADLETTER` | `aero.deadletter.v1` |
+
+Topic names are the public contract between services (and external consumers) — breaking changes require a new `.vN` topic, never an in-place change.
+
+## Usage examples
+
+```ts
+import {
+  createLogger, publishEvent, createConsumer, TOPICS, KAFKA_GROUPS,
+  pubFused, subChannels, PUB_SUB_CHANNELS, getEnv, ENV_VARS,
+} from "@aeroswarm/core";
+import type { FusedTrackEvent } from "@aeroswarm/core";
+
+const log = createLogger("my-service");
+
+// Publish a fused track to both transports (Kafka = durable, Redis = low-latency fan-out)
+function emit(event: FusedTrackEvent) {
+  pubFused(event);
+  publishEvent(TOPICS.FUSED, event, event.trackId).catch(() => null);
+}
+
+// Consume
+const consumer = await createConsumer({
+  groupId: KAFKA_GROUPS.REALTIME,
+  topics: [TOPICS.FUSED, TOPICS.HEALTH],
+  async handler({ topic, message }) {
+    if (!message.value) return;
+    log.info({ topic }, "event received");
+  },
+});
+// … consumer.disconnect() on shutdown
+```
+
+## Environment variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `DATABASE_URL` | `postgres://aero:aero@localhost:5432/aeroswarm` | PostgreSQL connection string |
+| `DATABASE_POOL_MAX` | `10` | Max pool connections |
+| `REDIS_URL` | `redis://localhost:6379` | Redis URL (or use `REDIS_SENTINEL_URLS` + `REDIS_SENTINEL_MASTER`) |
+| `REDIS_KEY_PREFIX` | `aero:` | Key namespace |
+| `KAFKA_BROKERS` | `localhost:19092` | Comma-separated broker list |
+| `KAFKA_CLIENT_ID` | `aeroswarm` | Kafka client id |
+| `KAFKA_ALLOW_CUSTOM_TOPICS` | unset | Allow publishing to topics outside the registry |
+| `LOG_LEVEL` | `info` | Pino level |
+| `NODE_ENV` | `development` | Runtime environment |
+
+## Building & testing
+
+```bash
+pnpm run build     # tsup → dist/ (root + /utils /validation /types subpath bundles)
+pnpm run typecheck
+pnpm test
+```

package/dist/chunk-TAFKYKUB.js

@@ -0,0 +1,128 @@
+// src/validation/index.ts
+import { z } from "zod";
+var positionSchema = z.object({
+  lat: z.number().min(-90).max(90),
+  lon: z.number().min(-180).max(180),
+  alt: z.number().optional(),
+  altSource: z.enum(["baro", "gnss", "mlat"]).optional(),
+  onGround: z.boolean().optional()
+});
+var velocitySchema = z.object({
+  groundSpeed: z.number().nonnegative().optional(),
+  heading: z.number().min(-1).max(361).optional(),
+  verticalRate: z.number().optional()
+});
+var currentTrackSchema = z.object({
+  trackId: z.string(),
+  source: z.enum(["ADSB", "ASTERIX_CAT010", "ASTERIX_CAT011", "ASTERIX_CAT020", "ASTERIX_CAT021", "ASTERIX_CAT048", "ASTERIX_CAT062", "FUSED"]),
+  icao24: z.string().nullable(),
+  callsign: z.string().nullable(),
+  squawk: z.string().nullable(),
+  targetType: z.enum(["aircraft", "vehicle", "unknown"]).optional(),
+  vehicleFleet: z.number().optional(),
+  airportId: z.string().optional(),
+  lat: z.number().min(-90).max(90).nullable(),
+  lon: z.number().min(-180).max(180).nullable(),
+  altitudeFt: z.number().nullable(),
+  groundSpeedKt: z.number().nullable(),
+  headingDeg: z.number().min(-1).max(361).nullable(),
+  verticalRateFpm: z.number().nullable(),
+  confidence: z.number().min(0).max(1),
+  lastSeenAt: z.number(),
+  status: z.enum(["active", "stale", "lost"])
+});
+var streamEnvelopeSchema = z.discriminatedUnion("type", [
+  z.object({
+    type: z.literal("snapshot"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({ tracks: z.array(currentTrackSchema) })
+  }),
+  z.object({
+    type: z.literal("track.updated"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({ track: currentTrackSchema })
+  }),
+  z.object({
+    type: z.literal("track.removed"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({ trackId: z.string(), reason: z.string() })
+  }),
+  z.object({
+    type: z.literal("source.status"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({ sourceId: z.string(), status: z.enum(["healthy", "degraded", "offline"]) })
+  }),
+  z.object({
+    type: z.literal("stream.heartbeat"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({}).strict()
+  }),
+  z.object({
+    type: z.literal("stream.degraded"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({ reason: z.string(), sourceId: z.string().optional() })
+  }),
+  z.object({
+    type: z.literal("stream.error"),
+    seq: z.number(),
+    serverTime: z.number(),
+    payload: z.object({ code: z.string(), message: z.string().optional(), recoverable: z.boolean() })
+  })
+]);
+var sourceHeartbeatSchema = z.object({
+  sourceId: z.string(),
+  timestamp: z.number(),
+  packetsReceived: z.number().int(),
+  packetsSent: z.number().int(),
+  bufferFillRatio: z.number().min(0).max(1),
+  latencyMs: z.number().nonnegative(),
+  protocol: z.enum(["beast", "sbs", "asterix-cat021", "asterix-cat048"]),
+  receiverVersion: z.string().optional()
+});
+var wsSubscribeSchema = z.object({
+  type: z.literal("subscribe"),
+  bbox: z.tuple([z.number(), z.number(), z.number(), z.number()]).optional(),
+  source: z.enum(["fused", "normalized"]).default("fused"),
+  minConfidence: z.number().min(0).max(1).optional(),
+  maxTracks: z.number().int().min(1).max(1e4).optional(),
+  mode: z.enum(["delta", "full"]).default("delta"),
+  maxUpdateHz: z.number().min(1).max(60).default(20),
+  encoding: z.enum(["json", "msgpack"]).default("json"),
+  airportId: z.string().min(2).max(8).optional()
+});
+var wsUpdateFilterSchema = z.object({
+  type: z.literal("update-filter"),
+  bbox: z.tuple([z.number(), z.number(), z.number(), z.number()]).optional(),
+  minConfidence: z.number().min(0).max(1).optional(),
+  maxUpdateHz: z.number().min(1).max(60).optional(),
+  maxTracks: z.number().int().min(1).max(1e4).nullable().optional()
+});
+var apiSuccessSchema = (dataSchema) => z.object({ ok: z.literal(true), data: dataSchema });
+var apiErrorSchema = z.object({
+  ok: z.literal(false),
+  error: z.object({
+    code: z.string(),
+    message: z.string(),
+    recoverable: z.boolean()
+  })
+});
+var apiResponseSchema = (dataSchema) => z.union([apiSuccessSchema(dataSchema), apiErrorSchema]);
+
+export {
+  positionSchema,
+  velocitySchema,
+  currentTrackSchema,
+  streamEnvelopeSchema,
+  sourceHeartbeatSchema,
+  wsSubscribeSchema,
+  wsUpdateFilterSchema,
+  apiSuccessSchema,
+  apiErrorSchema,
+  apiResponseSchema
+};

package/dist/chunk-ZBO7NUV3.js

@@ -0,0 +1,137 @@
+// src/utils/index.ts
+import { createHash, randomBytes } from "crypto";
+function createEventId(sourceId, icao24, ts) {
+  return createHash("sha256").update(`${sourceId}:${icao24}:${ts}`).digest("hex").slice(0, 16);
+}
+function createNanoid(size = 12) {
+  return randomBytes(size).toString("base64url").slice(0, size);
+}
+function createTrackId() {
+  return `trk_${createNanoid()}`;
+}
+function createSessionId() {
+  return `session_${createNanoid()}`;
+}
+function createEventUid() {
+  return `evt_${createNanoid(16)}`;
+}
+function isInsideBbox(lat, lon, bbox) {
+  const [minLon, minLat, maxLon, maxLat] = bbox;
+  return lon >= minLon && lon <= maxLon && lat >= minLat && lat <= maxLat;
+}
+function haversineKm(lat1, lon1, lat2, lon2) {
+  const R = 6371;
+  const dLat = (lat2 - lat1) * Math.PI / 180;
+  const dLon = (lon2 - lon1) * Math.PI / 180;
+  const a = Math.sin(dLat / 2) ** 2 + Math.cos(lat1 * Math.PI / 180) * Math.cos(lat2 * Math.PI / 180) * Math.sin(dLon / 2) ** 2;
+  return R * 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a));
+}
+function clamp(val, min, max) {
+  return Math.min(Math.max(val, min), max);
+}
+function distanceToPolylineM(lat, lon, points) {
+  if (points.length < 2) return null;
+  const kY = 111320;
+  const kX = 111320 * Math.cos(lat * Math.PI / 180);
+  let best = null;
+  let cumLen = 0;
+  let totalLen = 0;
+  const segLens = [];
+  for (let i = 0; i < points.length - 1; i++) {
+    const dx = (points[i + 1][1] - points[i][1]) * kX;
+    const dy = (points[i + 1][0] - points[i][0]) * kY;
+    const len = Math.hypot(dx, dy);
+    segLens.push(len);
+    totalLen += len;
+  }
+  if (totalLen === 0) return null;
+  for (let i = 0; i < points.length - 1; i++) {
+    const [aLat, aLon] = points[i];
+    const [bLat, bLon] = points[i + 1];
+    const px = (lon - aLon) * kX;
+    const py = (lat - aLat) * kY;
+    const bx = (bLon - aLon) * kX;
+    const by = (bLat - aLat) * kY;
+    const L2 = bx * bx + by * by;
+    const t = L2 === 0 ? 0 : clamp((px * bx + py * by) / L2, 0, 1);
+    const distM = Math.hypot(px - t * bx, py - t * by);
+    if (!best || distM < best.distM) {
+      best = {
+        distM,
+        segmentIndex: i,
+        progress: (cumLen + t * segLens[i]) / totalLen
+      };
+    }
+    cumLen += segLens[i];
+  }
+  return best;
+}
+function isRenderableTrack(t) {
+  return typeof t.lat === "number" && typeof t.lon === "number" && Number.isFinite(t.lat) && Number.isFinite(t.lon) && t.lat >= -90 && t.lat <= 90 && t.lon >= -180 && t.lon <= 180;
+}
+function toCurrentTrack(fused, staleAfterMs = 15e3, lostAfterMs = 9e4) {
+  const age = Date.now() - fused.updatedAt;
+  const status = age > lostAfterMs ? "lost" : age > staleAfterMs ? "stale" : "active";
+  const src = fused.sources.length > 1 ? "FUSED" : protocolToSource(fused.protocol);
+  const rawHeading = fused.velocity?.heading;
+  const headingDeg = rawHeading != null ? (rawHeading % 360 + 360) % 360 : null;
+  return {
+    trackId: fused.trackId,
+    source: src,
+    icao24: fused.identity.icao24 ?? null,
+    callsign: fused.identity.callsign ?? null,
+    squawk: fused.identity.squawk ?? null,
+    ...fused.identity.targetType !== void 0 && { targetType: fused.identity.targetType },
+    ...fused.identity.vehicleFleet !== void 0 && { vehicleFleet: fused.identity.vehicleFleet },
+    ...fused.airportId !== void 0 && { airportId: fused.airportId },
+    lat: fused.position?.lat ?? null,
+    lon: fused.position?.lon ?? null,
+    altitudeFt: fused.position?.alt ?? null,
+    groundSpeedKt: fused.velocity?.groundSpeed ?? null,
+    headingDeg,
+    verticalRateFpm: fused.velocity?.verticalRate ?? null,
+    confidence: fused.confidence.overall,
+    lastSeenAt: fused.updatedAt,
+    status
+  };
+}
+function protocolToSource(protocol) {
+  if (protocol === "asterix-cat010") return "ASTERIX_CAT010";
+  if (protocol === "asterix-cat011") return "ASTERIX_CAT011";
+  if (protocol === "asterix-cat020") return "ASTERIX_CAT020";
+  if (protocol === "asterix-cat021") return "ASTERIX_CAT021";
+  if (protocol === "asterix-cat048") return "ASTERIX_CAT048";
+  if (protocol === "asterix-cat062") return "ASTERIX_CAT062";
+  return "ADSB";
+}
+var EMERGENCY_SQUAWKS = /* @__PURE__ */ new Set(["7500", "7600", "7700"]);
+var MACH1_KT = 661.47;
+var TOPICS = {
+  FUSED: "aero.fused.track.v1",
+  TRUST: "aero.trust.assessment.v1",
+  HEALTH: "aero.source.health.v1",
+  LIFECYCLE: "aero.track.lifecycle.v1",
+  DEADLETTER: "aero.deadletter.v1",
+  MOVEMENT: "aero.movement.v1"
+};
+var STALE_AFTER_MS = 15e3;
+var LOST_AFTER_MS = 9e4;
+
+export {
+  createEventId,
+  createNanoid,
+  createTrackId,
+  createSessionId,
+  createEventUid,
+  isInsideBbox,
+  haversineKm,
+  clamp,
+  distanceToPolylineM,
+  isRenderableTrack,
+  toCurrentTrack,
+  EMERGENCY_SQUAWKS,
+  MACH1_KT,
+  TOPICS,
+  STALE_AFTER_MS,
+  LOST_AFTER_MS
+};