gtfs-sqljs 0.1.2 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,9 +1,14 @@
1
1
  <div align="center">
2
2
  <img src="logo.svg" alt="gtfs-sqljs logo" width="200" height="200">
3
3
  <h1>gtfs-sqljs</h1>
4
- <p>A TypeScript library for loading GTFS (General Transit Feed Specification) data into a sql.js SQLite database for querying in both browser and Node.js environments.</p>
4
+
5
+ [![npm version](https://img.shields.io/npm/v/gtfs-sqljs)](https://www.npmjs.com/package/gtfs-sqljs)
6
+
7
+ <p>A TypeScript library for loading <a href="https://gtfs.org/documentation/schedule/reference/">GTFS</a> (General Transit Feed Specification) data into a <a href="https://sql.js.org/">sql.js</a> SQLite database for querying in both browser and Node.js environments.</p>
5
8
  </div>
6
9
 
10
+ > **[Live Demo](https://sysdevrun.github.io/gtfs-sqljs-demo/)** — A fully static demo website with GTFS and GTFS-RT data running in a Web Worker, with no backend.
11
+
7
12
  ## Author
8
13
 
9
14
  **Théophile Helleboid / SysDevRun**
@@ -11,49 +16,38 @@
11
16
  - Email: contact@sys-dev-run.fr
12
17
  - Website: https://www.sys-dev-run.fr/
13
18
 
14
- ## Documentation & Demo
19
+ This project is greatly inspired by [node-gtfs](https://github.com/BlinkTagInc/node-gtfs), also MIT licensed. The main difference is that gtfs-sqljs aims to run on both browser and Node.js environments.
15
20
 
16
- 📚 **[View Documentation and Interactive Demo](https://sysdevrun.github.io/gtfs-sqljs/)**
21
+ ## Documentation & Demo
17
22
 
18
- Try the live demo to explore GTFS data, view routes with colors, and see trip schedules in action!
23
+ - [Documentation and Interactive Demo](https://sysdevrun.github.io/gtfs-sqljs/)
24
+ - [Usage Guide](https://sysdevrun.github.io/gtfs-sqljs/docs/documents/Usage_Guide.html) — detailed examples for all features
25
+ - [API Reference](https://sysdevrun.github.io/gtfs-sqljs/docs/) — full TypeDoc-generated API docs
19
26
 
20
27
  ## Features
21
28
 
22
29
  ### GTFS Static Data
23
- - Load GTFS data from ZIP files (URL or local path)
24
- - **High-performance loading** with optimized bulk inserts
25
- - **Progress tracking** - Real-time progress callbacks (0-100%)
26
- - Skip importing specific files (e.g., shapes.txt) to reduce memory usage
27
- - Load existing SQLite databases
28
- - Export databases to ArrayBuffer for persistence
29
- - Flexible filter-based query API - combine multiple filters easily
30
- - Agency query support with agency-based filtering
31
- - Full TypeScript support with comprehensive types
32
- - ✅ Works in both browser and Node.js
33
- - Efficient querying with indexed SQLite database
34
- - Proper handling of GTFS required/optional fields
35
- - Active service detection based on calendar/calendar_dates
36
- - Optimized bulk loading with transactions and batch inserts
37
-
38
- ### GTFS Realtime Support
39
- - ✅ Load GTFS-RT data from protobuf feeds (URLs or local files)
40
- - ✅ Support for Alerts, Trip Updates, and Vehicle Positions
41
- - ✅ Automatic staleness filtering (configurable threshold)
42
- - ✅ Active alert period checking
43
- - ✅ Merge realtime data with static schedules
44
- - ✅ Filter alerts and vehicle positions by route, stop, or trip
45
- - ✅ Store RT data in SQLite for consistent querying
46
- - ✅ Include RT data in database exports
47
- - ✅ Full support for both GTFS-RT `time` and `delay` fields in stop time updates
30
+ - Load GTFS data from ZIP files (URL or local path)
31
+ - **High-performance loading** with optimized bulk inserts
32
+ - **Progress tracking** - Real-time progress callbacks (0-100%)
33
+ - Skip importing specific files (e.g., shapes.txt) to reduce memory usage
34
+ - Load existing SQLite databases
35
+ - Export databases to ArrayBuffer for persistence
36
+ - Flexible filter-based query API - combine multiple filters easily
37
+ - Full TypeScript support with comprehensive types
38
+ - Works in both browser and Node.js
39
+
40
+ ### [GTFS Realtime](https://gtfs.org/documentation/realtime/reference/) Support
41
+ - Load GTFS-RT data from protobuf feeds (URLs or local files)
42
+ - Support for Alerts, Trip Updates, and Vehicle Positions
43
+ - Automatic staleness filtering (configurable threshold)
44
+ - Merge realtime data with static schedules
48
45
 
49
46
  ### Smart Caching
50
- - **Optional caching** - Copy cache implementations from `examples/cache/`
51
- - **Platform-specific stores** - IndexedDBCacheStore (browser) or FileSystemCacheStore (Node.js)
52
- - **Smart invalidation** - Based on file checksum, size, version, and library version
53
- - **Automatic expiration** - Configurable (default: 7 days)
54
- - ✅ **Cache management API** - Get stats, clean expired entries, clear cache
55
- - ✅ **Custom cache stores** - Implement your own (Redis, S3, etc.)
56
- - ✅ **Dramatic speed improvement** - Subsequent loads in <1 second
47
+ - **Optional caching** - Copy cache implementations from `examples/cache/`
48
+ - **Platform-specific stores** - IndexedDBCacheStore (browser) or FileSystemCacheStore (Node.js)
49
+ - **Smart invalidation** - Based on file checksum, size, version, and library version
50
+ - **Dramatic speed improvement** - Subsequent loads in <1 second
57
51
 
58
52
  ## Installation
59
53
 
@@ -67,1013 +61,40 @@ You also need to install sql.js as a peer dependency:
67
61
  npm install sql.js
68
62
  ```
69
63
 
70
- ## Loading the sql.js WASM File
71
-
72
- sql.js requires a WASM file to be loaded. There are several ways to handle this:
73
-
74
- ### Node.js
75
-
76
- In Node.js, sql.js will automatically locate the WASM file from the installed package:
77
-
78
- ```typescript
79
- import { GtfsSqlJs } from 'gtfs-sqljs';
80
-
81
- // The WASM file is loaded automatically
82
- const gtfs = await GtfsSqlJs.fromZip('path/to/gtfs.zip');
83
- ```
84
-
85
- ### Browser with CDN
86
-
87
- You can use a CDN to serve the WASM file:
64
+ ## Quick Start
88
65
 
89
66
  ```typescript
90
- import initSqlJs from 'sql.js';
91
67
  import { GtfsSqlJs } from 'gtfs-sqljs';
92
68
 
93
- // Initialize sql.js with CDN WASM file
94
- const SQL = await initSqlJs({
95
- locateFile: (filename) => `https://sql.js.org/dist/${filename}`
96
- });
97
-
98
- // Pass the SQL instance to GtfsSqlJs
99
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', { SQL });
100
- ```
101
-
102
- ### Browser with Bundler (Webpack, Vite, etc.)
103
-
104
- If you're using a bundler, you need to configure it to handle the WASM file:
105
-
106
- #### Vite
107
-
108
- ```typescript
109
- import initSqlJs from 'sql.js';
110
- import { GtfsSqlJs } from 'gtfs-sqljs';
111
- import sqlWasmUrl from 'sql.js/dist/sql-wasm.wasm?url';
112
-
113
- const SQL = await initSqlJs({
114
- locateFile: () => sqlWasmUrl
115
- });
116
-
117
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', { SQL });
118
- ```
119
-
120
- #### Webpack
121
-
122
- ```typescript
123
- import initSqlJs from 'sql.js';
124
- import { GtfsSqlJs } from 'gtfs-sqljs';
125
-
126
- const SQL = await initSqlJs({
127
- locateFile: (filename) => `/path/to/public/${filename}`
128
- });
129
-
130
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', { SQL });
131
- ```
132
-
133
- Make sure to copy `sql-wasm.wasm` from `node_modules/sql.js/dist/` to your public directory.
134
-
135
- ## Usage
136
-
137
- ### Creating an Instance
138
-
139
- #### From a GTFS ZIP file
140
-
141
- ```typescript
142
- import { GtfsSqlJs } from 'gtfs-sqljs';
143
-
144
- // From URL
69
+ // Load GTFS data from a ZIP file
145
70
  const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip');
146
71
 
147
- // From local file (Node.js)
148
- const gtfs = await GtfsSqlJs.fromZip('./path/to/gtfs.zip');
149
-
150
- // Skip importing specific files to reduce memory usage
151
- // Tables will be created but data won't be imported
152
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', {
153
- skipFiles: ['shapes.txt', 'frequencies.txt']
154
- });
155
- ```
156
-
157
- #### From an existing SQLite database
158
-
159
- ```typescript
160
- import { GtfsSqlJs } from 'gtfs-sqljs';
161
-
162
- // Load from ArrayBuffer
163
- const dbBuffer = await fetch('https://example.com/gtfs.db').then(r => r.arrayBuffer());
164
- const gtfs = await GtfsSqlJs.fromDatabase(dbBuffer);
165
- ```
166
-
167
- ### Progress Tracking
168
-
169
- Track loading progress with a callback function - perfect for displaying progress bars or updating UI:
170
-
171
- ```typescript
172
- import { GtfsSqlJs, type ProgressInfo } from 'gtfs-sqljs';
173
-
174
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', {
175
- onProgress: (progress: ProgressInfo) => {
176
- console.log(`${progress.percentComplete}% - ${progress.message}`);
177
-
178
- // Progress information available:
179
- console.log('Phase:', progress.phase); // Current phase
180
- console.log('File:', progress.currentFile); // Current file being processed
181
- console.log('Files:', progress.filesCompleted, '/', progress.totalFiles);
182
- console.log('Rows:', progress.rowsProcessed, '/', progress.totalRows);
183
- }
184
- });
185
- ```
186
-
187
- #### Progress Phases
188
-
189
- The loading process goes through these phases:
190
-
191
- 1. **`checking_cache`** - Checking if cached database exists (0%)
192
- 2. **`loading_from_cache`** - Loading from cache (if found, jumps to 100%)
193
- 3. **`downloading`** - Downloading GTFS ZIP file (1-30%)
194
- 4. **`extracting`** - Extracting GTFS ZIP file (35%)
195
- 5. **`creating_schema`** - Creating database tables (40%)
196
- 6. **`inserting_data`** - Importing data from CSV files (40-75%)
197
- 7. **`creating_indexes`** - Building database indexes (75-85%)
198
- 8. **`analyzing`** - Optimizing query performance (85-90%)
199
- 9. **`loading_realtime`** - Loading realtime data from feeds (90-95%) *(if configured)*
200
- 10. **`saving_cache`** - Saving to cache (95-98%)
201
- 11. **`complete`** - Load complete (100%)
202
-
203
- **Note:** When a cached database is found, phases 3-10 are skipped, and loading completes in <1 second.
204
-
205
- **Note:** The `loading_realtime` phase only occurs if `realtimeFeedUrls` are configured during initialization.
206
-
207
- #### Web Worker Example
208
-
209
- The progress callback is especially useful for web workers:
210
-
211
- ```typescript
212
- // In your web worker
213
- import { GtfsSqlJs } from 'gtfs-sqljs';
214
-
215
- self.onmessage = async (event) => {
216
- if (event.data.type === 'load') {
217
- const gtfs = await GtfsSqlJs.fromZip(event.data.url, {
218
- onProgress: (progress) => {
219
- // Send progress updates to main thread
220
- self.postMessage({
221
- type: 'progress',
222
- data: progress
223
- });
224
- }
225
- });
226
-
227
- self.postMessage({ type: 'complete' });
228
- }
229
- };
230
- ```
231
-
232
- ```typescript
233
- // In your main thread
234
- const worker = new Worker('gtfs-worker.js');
235
-
236
- worker.onmessage = (event) => {
237
- if (event.data.type === 'progress') {
238
- const progress = event.data.data;
239
- updateProgressBar(progress.percentComplete);
240
- updateStatusText(progress.message);
241
- }
242
- };
243
-
244
- worker.postMessage({ type: 'load', url: 'https://example.com/gtfs.zip' });
245
- ```
246
-
247
- #### ProgressInfo Type
248
-
249
- ```typescript
250
- interface ProgressInfo {
251
- phase: 'checking_cache' | 'loading_from_cache' | 'downloading' | 'extracting' |
252
- 'creating_schema' | 'inserting_data' | 'creating_indexes' | 'analyzing' |
253
- 'loading_realtime' | 'saving_cache' | 'complete';
254
- currentFile: string | null; // e.g., "stop_times.txt"
255
- filesCompleted: number; // Files processed so far
256
- totalFiles: number; // Total number of files
257
- rowsProcessed: number; // CSV rows imported so far
258
- totalRows: number; // Total CSV rows to import
259
- bytesDownloaded?: number; // Bytes downloaded (during 'downloading' phase)
260
- totalBytes?: number; // Total bytes to download (during 'downloading' phase)
261
- percentComplete: number; // 0-100
262
- message: string; // Human-readable status message
263
- }
264
- ```
265
-
266
- ### Querying Data
267
-
268
- The library provides two ways to query GTFS data:
269
- 1. **Flexible filter-based methods** (recommended) - Pass an object with optional filters
270
- 2. **Convenience methods** - Direct methods for common use cases
271
-
272
- #### Flexible Filter-Based Queries (Recommended)
273
-
274
- The new flexible API allows you to pass multiple optional filters in a single method call:
275
-
276
- ```typescript
277
- // Get stops - combine any filters
278
- const stops = gtfs.getStops({
279
- name: 'Station', // Search by name
280
- limit: 10 // Limit results
281
- });
282
-
283
- // Get routes - with or without filters
284
- const allRoutes = gtfs.getRoutes();
285
- const agencyRoutes = gtfs.getRoutes({ agencyId: 'AGENCY_1' });
286
-
287
- // Get trips - combine multiple filters
288
- const trips = gtfs.getTrips({
289
- routeId: 'ROUTE_1', // Filter by route
290
- date: '20240115', // Filter by date (gets active services)
291
- directionId: 0, // Filter by direction
292
- limit: 50 // Limit results
293
- });
294
-
295
- // Get stop times - flexible filtering
296
- const stopTimes = gtfs.getStopTimes({
297
- stopId: 'STOP_123', // At a specific stop
298
- routeId: 'ROUTE_1', // For a specific route
299
- date: '20240115', // On a specific date
300
- directionId: 0 // In a specific direction
301
- });
302
- ```
303
-
304
- **Available Filter Options:**
305
-
306
- - `getStops(filters?)`:
307
- - `stopId`: string - Filter by stop ID
308
- - `stopCode`: string - Filter by stop code
309
- - `name`: string - Search by stop name (partial match)
310
- - `tripId`: string - Get stops for a trip
311
- - `limit`: number - Limit results
312
-
313
- - `getRoutes(filters?)`:
314
- - `routeId`: string - Filter by route ID
315
- - `agencyId`: string - Filter by agency
316
- - `limit`: number - Limit results
317
-
318
- - `getTrips(filters?)`:
319
- - `tripId`: string - Filter by trip ID
320
- - `routeId`: string - Filter by route
321
- - `date`: string - Filter by date (YYYYMMDD format)
322
- - `directionId`: number - Filter by direction
323
- - `limit`: number - Limit results
324
-
325
- - `getStopTimes(filters?)`:
326
- - `tripId`: string - Filter by trip
327
- - `stopId`: string - Filter by stop
328
- - `routeId`: string - Filter by route
329
- - `date`: string - Filter by date (YYYYMMDD format)
330
- - `directionId`: number - Filter by direction
331
- - `limit`: number - Limit results
332
-
333
- - `getShapes(filters?)`:
334
- - `shapeId`: string | string[] - Filter by shape ID
335
- - `routeId`: string | string[] - Filter by route (via trips table)
336
- - `tripId`: string | string[] - Filter by trip
337
- - `limit`: number - Limit results
338
-
339
- - `getShapesToGeojson(filters?, precision?)`:
340
- - Same filters as `getShapes`
341
- - `precision`: number - Decimal places for coordinates (default: 6)
342
-
343
- #### Get Stop Information
344
-
345
- ```typescript
346
- // Get stop by ID
347
- const stops = gtfs.getStops({ stopId: 'STOP_123' });
348
- const stop = stops.length > 0 ? stops[0] : null;
349
- console.log(stop?.stop_name);
350
-
351
- // Get stop by code (using filters)
352
- const stops = gtfs.getStops({ stopCode: 'ABC' });
353
- const stop = stops[0];
354
-
355
- // Search stops by name (using filters)
356
- const stops = gtfs.getStops({ name: 'Main Street' });
357
-
358
- // Get all stops (using filters with no parameters)
359
- const allStops = gtfs.getStops();
360
-
361
- // Get stops with limit
362
- const stops = gtfs.getStops({ limit: 10 });
363
-
364
- // Get stops for a specific trip
365
- const stops = gtfs.getStops({ tripId: 'TRIP_123' });
366
- ```
367
-
368
- #### Get Route Information
369
-
370
- ```typescript
371
- // Get route by ID
372
- const routes = gtfs.getRoutes({ routeId: 'ROUTE_1' });
373
- const route = routes.length > 0 ? routes[0] : null;
374
-
375
- // Get all routes (using filters with no parameters)
72
+ // Query routes
376
73
  const routes = gtfs.getRoutes();
377
74
 
378
- // Get routes by agency (using filters)
379
- const agencyRoutes = gtfs.getRoutes({ agencyId: 'AGENCY_1' });
380
-
381
- // Get routes with limit
382
- const routes = gtfs.getRoutes({ limit: 10 });
383
- ```
384
-
385
- #### Get Agency Information
386
-
387
- ```typescript
388
- // Get agency by ID
389
- const agencies = gtfs.getAgencies({ agencyId: 'AGENCY_1' });
390
- const agency = agencies.length > 0 ? agencies[0] : null;
391
-
392
- // Get all agencies
393
- const allAgencies = gtfs.getAgencies();
394
-
395
- // Get agencies with limit
396
- const agencies = gtfs.getAgencies({ limit: 5 });
397
- ```
398
-
399
- #### Get Calendar Information
400
-
401
- ```typescript
402
- // Get active services for a date (YYYYMMDD format)
403
- const serviceIds = gtfs.getActiveServiceIds('20240115');
75
+ // Query stops with filters
76
+ const stops = gtfs.getStops({ name: 'Central Station' });
404
77
 
405
- // Get calendar by service ID
406
- const calendar = gtfs.getCalendarByServiceId('WEEKDAY');
407
-
408
- // Get calendar date exceptions
409
- const exceptions = gtfs.getCalendarDates('WEEKDAY');
410
- ```
411
-
412
- #### Get Trip Information
413
-
414
- ```typescript
415
- // Get trip by ID
416
- const trips = gtfs.getTrips({ tripId: 'TRIP_123' });
417
- const trip = trips.length > 0 ? trips[0] : null;
418
-
419
- // Get trips by route (using filters)
420
- const trips = gtfs.getTrips({ routeId: 'ROUTE_1' });
421
-
422
- // Get trips by route and date (using filters)
423
- const trips = gtfs.getTrips({ routeId: 'ROUTE_1', date: '20240115' });
424
-
425
- // Get trips by route, date, and direction (using filters)
78
+ // Get trips for a route on a specific date
426
79
  const trips = gtfs.getTrips({
427
80
  routeId: 'ROUTE_1',
428
81
  date: '20240115',
429
82
  directionId: 0
430
83
  });
431
84
 
432
- // Get all trips for a date
433
- const trips = gtfs.getTrips({ date: '20240115' });
434
-
435
- // Get trips by agency
436
- const trips = gtfs.getTrips({ agencyId: 'AGENCY_1' });
437
- ```
438
-
439
- #### Get Stop Time Information
440
-
441
- ```typescript
442
- // Get stop times for a trip (ordered by stop_sequence)
443
- const stopTimes = gtfs.getStopTimes({ tripId: 'TRIP_123' });
444
-
445
- // Get stop times for a stop (using filters)
446
- const stopTimes = gtfs.getStopTimes({ stopId: 'STOP_123' });
447
-
448
- // Get stop times for a stop and route (using filters)
449
- const stopTimes = gtfs.getStopTimes({
450
- stopId: 'STOP_123',
451
- routeId: 'ROUTE_1'
452
- });
453
-
454
- // Get stop times for a stop, route, and date (using filters)
455
- const stopTimes = gtfs.getStopTimes({
456
- stopId: 'STOP_123',
457
- routeId: 'ROUTE_1',
458
- date: '20240115'
459
- });
460
-
461
- // Get stop times with direction filter (using filters)
462
- const stopTimes = gtfs.getStopTimes({
463
- stopId: 'STOP_123',
464
- routeId: 'ROUTE_1',
465
- date: '20240115',
466
- directionId: 0
467
- });
468
-
469
- // Get stop times by agency
470
- const stopTimes = gtfs.getStopTimes({
471
- agencyId: 'AGENCY_1',
472
- date: '20240115'
473
- });
474
- ```
475
-
476
- #### Building Ordered Stop Lists for Multiple Trips
477
-
478
- When displaying timetables for routes where different trips may stop at different stops (e.g., express vs local service, or trips with varying start/end points), use `buildOrderedStopList()` to build an optimal ordered list of all unique stops:
479
-
480
- ```typescript
481
- // Get all trips for a route in one direction
482
- const trips = gtfs.getTrips({
483
- routeId: 'ROUTE_1',
484
- directionId: 0,
485
- date: '20240115'
486
- });
487
-
488
- // Build ordered list of all stops served by these trips
489
- const tripIds = trips.map(t => t.trip_id);
490
- const orderedStops = gtfs.buildOrderedStopList(tripIds);
491
-
492
- // Now display a timetable with all possible stops
493
- console.log('Route stops:');
494
- orderedStops.forEach(stop => {
495
- console.log(`- ${stop.stop_name}`);
496
- });
497
-
498
- // For each trip, you can now show which stops it serves
499
- for (const trip of trips) {
500
- const tripStopTimes = gtfs.getStopTimes({ tripId: trip.trip_id });
501
- console.log(`\nTrip ${trip.trip_headsign}:`);
502
-
503
- // Show all stops, marking which ones this trip serves
504
- orderedStops.forEach(stop => {
505
- const stopTime = tripStopTimes.find(st => st.stop_id === stop.stop_id);
506
- if (stopTime) {
507
- console.log(` ${stopTime.arrival_time} - ${stop.stop_name}`);
508
- } else {
509
- console.log(` --- (not served) - ${stop.stop_name}`);
510
- }
511
- });
512
- }
513
- ```
514
-
515
- **Use Cases:**
516
- - **Express vs Local Service** - Some trips skip stops that others serve
517
- - **Different Start/End Points** - Short-turn trips or extended service trips
518
- - **Peak vs Off-Peak Service** - Different stop coverage based on time of day
519
- - **Route Variations** - Multiple branches or patterns on the same route
520
-
521
- **How it works:**
522
- The method intelligently merges stop sequences from all provided trips:
523
- 1. Fetches stop times for all trips
524
- 2. Processes each trip's stops in sequence order
525
- 3. When encountering a new stop, finds the best insertion position by analyzing stops before and after it
526
- 4. Returns full Stop objects in the determined order
527
-
528
- **Example - Real-world scenario:**
529
- ```typescript
530
- // You have a bus route with:
531
- // - Local trips: A → B → C → D → E → F
532
- // - Express trips: A → C → E → F (skips B and D)
533
- // - Short trips: B → C → D (doesn't go to end of line)
534
-
535
- const allTrips = gtfs.getTrips({ routeId: 'BUS_42', directionId: 0 });
536
- const tripIds = allTrips.map(t => t.trip_id);
537
- const stops = gtfs.buildOrderedStopList(tripIds);
538
-
539
- // Result: [A, B, C, D, E, F] - all stops in correct order
540
- // Now you can create a timetable showing all stops with departure times
541
- ```
542
-
543
- #### Get Shape Information
544
-
545
- Shapes define the path a vehicle takes along a route. Use `getShapes()` to get raw shape point data and `getShapesToGeojson()` to get shapes as GeoJSON for mapping.
546
-
547
- ```typescript
548
- // Get all shape points for a specific shape
549
- const shapePoints = gtfs.getShapes({ shapeId: 'SHAPE_1' });
550
- console.log(`Shape has ${shapePoints.length} points`);
551
-
552
- // Get shapes for a specific route
553
- const routeShapes = gtfs.getShapes({ routeId: 'ROUTE_1' });
554
-
555
- // Get shapes for multiple trips
556
- const tripShapes = gtfs.getShapes({ tripId: ['TRIP_1', 'TRIP_2'] });
557
-
558
- // Each shape point contains:
559
- // - shape_id: string
560
- // - shape_pt_lat: number
561
- // - shape_pt_lon: number
562
- // - shape_pt_sequence: number
563
- // - shape_dist_traveled?: number (optional)
564
- ```
565
-
566
- #### Get Shapes as GeoJSON
567
-
568
- Convert shapes to GeoJSON format for use with mapping libraries (Leaflet, Mapbox, etc.):
85
+ // Get stop times for a trip
86
+ const stopTimes = gtfs.getStopTimes({ tripId: trips[0].trip_id });
569
87
 
570
- ```typescript
571
- // Get all shapes as GeoJSON FeatureCollection
572
- const geojson = gtfs.getShapesToGeojson();
573
-
574
- // Get shapes for a specific route
575
- const routeGeojson = gtfs.getShapesToGeojson({ routeId: 'ROUTE_1' });
576
-
577
- // Customize coordinate precision (default: 6 decimals = ~10cm)
578
- const lowPrecision = gtfs.getShapesToGeojson({ routeId: 'ROUTE_1' }, 4); // ~11m precision
579
-
580
- // GeoJSON structure:
581
- // {
582
- // type: 'FeatureCollection',
583
- // features: [{
584
- // type: 'Feature',
585
- // properties: {
586
- // shape_id: 'SHAPE_1',
587
- // route_id: 'ROUTE_1',
588
- // route_short_name: '1',
589
- // route_long_name: 'Main Street',
590
- // route_type: 3,
591
- // route_color: 'FF0000',
592
- // route_text_color: 'FFFFFF',
593
- // agency_id: 'AGENCY_1'
594
- // },
595
- // geometry: {
596
- // type: 'LineString',
597
- // coordinates: [[-122.123456, 37.123456], [-122.234567, 37.234567], ...]
598
- // }
599
- // }]
600
- // }
601
-
602
- // Use with Leaflet
603
- const geoJsonLayer = L.geoJSON(geojson, {
604
- style: (feature) => ({
605
- color: `#${feature.properties.route_color || '000000'}`,
606
- weight: 3
607
- })
608
- }).addTo(map);
609
- ```
610
-
611
- **Precision values:**
612
- - `6` decimals: ~10cm precision (default)
613
- - `5` decimals: ~1m precision
614
- - `4` decimals: ~11m precision
615
- - `3` decimals: ~111m precision
616
-
617
- ### GTFS Realtime Support
618
-
619
- This library supports GTFS Realtime data (alerts, trip updates, and vehicle positions) with automatic merging into static schedule data.
620
-
621
- #### Loading Realtime Data
622
-
623
- ```typescript
624
- // Configure RT feed URLs - data will be fetched automatically after GTFS load
625
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', {
626
- realtimeFeedUrls: [
627
- 'https://example.com/gtfs-rt/alerts',
628
- 'https://example.com/gtfs-rt/trip-updates',
629
- 'https://example.com/gtfs-rt/vehicle-positions'
630
- ],
631
- stalenessThreshold: 120 // seconds (default: 120)
632
- });
633
- // RT data is already loaded and ready to use!
634
-
635
- // Or manually fetch RT data later (uses configured URLs or pass custom URLs)
636
- await gtfs.fetchRealtimeData();
637
-
638
- // Or fetch from specific URLs
639
- await gtfs.fetchRealtimeData([
640
- 'https://example.com/gtfs-rt/combined-feed'
641
- ]);
642
-
643
- // Support local files in Node.js
644
- await gtfs.fetchRealtimeData(['./path/to/feed.pb']);
645
-
646
- // Update configuration
647
- gtfs.setRealtimeFeedUrls(['https://example.com/new-feed']);
648
- gtfs.setStalenessThreshold(60); // 60 seconds
649
-
650
- // Check when realtime data was last fetched
651
- const lastFetch = gtfs.getLastRealtimeFetchTimestamp();
652
- if (lastFetch) {
653
- const ageSeconds = Math.floor(Date.now() / 1000) - lastFetch;
654
- console.log(`RT data is ${ageSeconds} seconds old`);
655
- } else {
656
- console.log('No RT data has been fetched yet');
657
- }
658
- ```
659
-
660
- #### Querying Alerts
661
-
662
- ```typescript
663
- // Get all active alerts
664
- const activeAlerts = gtfs.getAlerts({ activeOnly: true });
665
-
666
- // Filter alerts by route
667
- const routeAlerts = gtfs.getAlerts({
668
- routeId: 'ROUTE_1',
669
- activeOnly: true
670
- });
671
-
672
- // Filter alerts by stop
673
- const stopAlerts = gtfs.getAlerts({
674
- stopId: 'STOP_123',
675
- activeOnly: true
676
- });
677
-
678
- // Filter alerts by trip
679
- const tripAlerts = gtfs.getAlerts({
680
- tripId: 'TRIP_456'
681
- });
682
-
683
- // Get alert by ID
684
- const alerts = gtfs.getAlerts({ alertId: 'alert:12345' });
685
- const alert = alerts.length > 0 ? alerts[0] : null;
686
-
687
- // Alert structure
688
- console.log(alert.header_text); // TranslatedString
689
- console.log(alert.description_text); // TranslatedString
690
- console.log(alert.cause); // AlertCause enum
691
- console.log(alert.effect); // AlertEffect enum
692
- console.log(alert.active_period); // TimeRange[]
693
- console.log(alert.informed_entity); // EntitySelector[]
694
- ```
695
-
696
- #### Querying Vehicle Positions
697
-
698
- ```typescript
699
- // Get all vehicle positions
700
- const vehicles = gtfs.getVehiclePositions();
701
-
702
- // Filter by route
703
- const routeVehicles = gtfs.getVehiclePositions({
704
- routeId: 'ROUTE_1'
705
- });
706
-
707
- // Filter by trip
708
- const tripVehicles = gtfs.getVehiclePositions({
709
- tripId: 'TRIP_123'
710
- });
711
- const vehicle = tripVehicles.length > 0 ? tripVehicles[0] : null;
712
-
713
- // Vehicle structure
714
- console.log(vehicle.position); // { latitude, longitude, bearing, speed }
715
- console.log(vehicle.current_stop_sequence);
716
- console.log(vehicle.current_status); // VehicleStopStatus enum
717
- console.log(vehicle.timestamp);
718
- ```
719
-
720
- #### Merging Realtime with Static Data
721
-
722
- The library automatically merges realtime data with static schedules when requested:
723
-
724
- ```typescript
725
- // Get trips with realtime data
726
- const tripsWithRT = gtfs.getTrips({
727
- routeId: 'ROUTE_1',
728
- date: '20240115',
729
- includeRealtime: true // Include RT data
730
- });
731
-
732
- for (const trip of tripsWithRT) {
733
- if (trip.realtime?.vehicle_position) {
734
- console.log('Vehicle location:', trip.realtime.vehicle_position.position);
735
- }
736
- if (trip.realtime?.trip_update) {
737
- console.log('Trip delay:', trip.realtime.trip_update.delay, 'seconds');
738
- }
739
- }
740
-
741
- // Get stop times with realtime delays
742
- const stopTimesWithRT = gtfs.getStopTimes({
743
- tripId: 'TRIP_123',
744
- includeRealtime: true // Include RT data
745
- });
746
-
747
- for (const st of stopTimesWithRT) {
748
- console.log(`Stop: ${st.stop_id}`);
749
- console.log(`Scheduled: ${st.arrival_time}`);
750
- if (st.realtime?.arrival_delay) {
751
- console.log(`Delay: ${st.realtime.arrival_delay} seconds`);
752
- }
753
- }
754
- ```
755
-
756
- #### Clearing Realtime Data
757
-
758
- ```typescript
759
- // Clear all realtime data
760
- gtfs.clearRealtimeData();
761
-
762
- // Then fetch fresh data
763
- await gtfs.fetchRealtimeData();
764
- ```
765
-
766
- #### GTFS-RT Enums
767
-
768
- The library exports all GTFS-RT enums for type checking:
769
-
770
- ```typescript
771
- import {
772
- AlertCause,
773
- AlertEffect,
774
- ScheduleRelationship,
775
- VehicleStopStatus,
776
- CongestionLevel,
777
- OccupancyStatus
778
- } from 'gtfs-sqljs';
779
-
780
- // Use enums for filtering or comparison
781
- if (alert.cause === AlertCause.ACCIDENT) {
782
- console.log('Alert is due to an accident');
783
- }
784
- ```
785
-
786
- ### Smart Caching
787
-
788
- The library supports optional caching of processed GTFS databases to dramatically speed up subsequent loads. The first load processes the GTFS zip file (~5-10 seconds), but subsequent loads use the cached database (<1 second).
789
-
790
- #### Setting Up Caching
791
-
792
- Cache store implementations are available in `examples/cache/`. Copy the appropriate implementation to your project:
793
-
794
- **Browser - IndexedDB:**
795
- ```typescript
796
- // Copy examples/cache/IndexedDBCacheStore.ts to your project
797
- import { GtfsSqlJs } from 'gtfs-sqljs';
798
- import { IndexedDBCacheStore } from './IndexedDBCacheStore';
799
-
800
- const cache = new IndexedDBCacheStore();
801
-
802
- // First load: processes GTFS zip file and caches the result
803
- const gtfs = await GtfsSqlJs.fromZip('gtfs.zip', { cache });
804
-
805
- // Second load: uses cached database (much faster!)
806
- const gtfs2 = await GtfsSqlJs.fromZip('gtfs.zip', { cache });
807
- ```
808
-
809
- **Node.js - FileSystem:**
810
- ```typescript
811
- // Copy examples/cache/FileSystemCacheStore.ts to your project
812
- import { GtfsSqlJs } from 'gtfs-sqljs';
813
- import { FileSystemCacheStore } from './FileSystemCacheStore';
814
-
815
- const cache = new FileSystemCacheStore({ dir: './.cache/gtfs' });
816
-
817
- const gtfs = await GtfsSqlJs.fromZip('gtfs.zip', { cache });
818
- ```
819
-
820
- **Note:** `FileSystemCacheStore` uses Node.js built-in modules (`fs`, `path`, `os`) and is **NOT compatible** with browser or React Native environments.
821
-
822
- #### Cache Invalidation
823
-
824
- The cache is automatically invalidated when any of these change:
825
- - **File checksum** (SHA-256) - Different GTFS data
826
- - **File size** - Quick check before computing checksum
827
- - **Library version** - Schema or processing logic updated
828
- - **Data version** - User-specified version (see below)
829
- - **Skipped files** - Different `skipFiles` options
830
-
831
- #### Data Versioning
832
-
833
- Use `cacheVersion` to control cache invalidation:
834
-
835
- ```typescript
836
- // Load with version 1.0
837
- const gtfs = await GtfsSqlJs.fromZip('gtfs.zip', {
838
- cacheVersion: '1.0'
839
- });
840
-
841
- // Load with version 2.0 - will reprocess and create new cache
842
- const gtfs2 = await GtfsSqlJs.fromZip('gtfs.zip', {
843
- cacheVersion: '2.0'
844
- });
845
- ```
846
-
847
- **When to increment version:**
848
- - GTFS data is updated but filename stays the same
849
- - You want to force cache refresh
850
- - Testing different processing configurations
851
-
852
- #### Cache Store Options
853
-
854
- **IndexedDBCacheStore options:**
855
- ```typescript
856
- import { IndexedDBCacheStore } from './IndexedDBCacheStore';
857
-
858
- const cache = new IndexedDBCacheStore({
859
- dbName: 'my-app-gtfs-cache' // Custom database name
860
- });
861
- ```
862
-
863
- **FileSystemCacheStore options:**
864
- ```typescript
865
- import { FileSystemCacheStore } from './FileSystemCacheStore';
866
-
867
- const cache = new FileSystemCacheStore({
868
- dir: './my-cache-dir' // Custom cache directory
869
- });
870
- ```
871
-
872
- #### Cache Management
873
-
874
- Cache management methods require passing the cache store instance:
875
-
876
- **Get cache statistics:**
877
- ```typescript
878
- import { IndexedDBCacheStore } from './IndexedDBCacheStore';
879
-
880
- const cache = new IndexedDBCacheStore();
881
- const stats = await GtfsSqlJs.getCacheStats(cache);
882
-
883
- console.log(`Total entries: ${stats.totalEntries}`);
884
- console.log(`Active entries: ${stats.activeEntries}`);
885
- console.log(`Expired entries: ${stats.expiredEntries}`);
886
- console.log(`Total size: ${stats.totalSizeMB} MB`);
887
- ```
888
-
889
- **List cache entries:**
890
- ```typescript
891
- const entries = await GtfsSqlJs.listCache(cache);
892
-
893
- entries.forEach(entry => {
894
- console.log(`Key: ${entry.key}`);
895
- console.log(`Source: ${entry.metadata.source}`);
896
- console.log(`Size: ${(entry.metadata.size / 1024 / 1024).toFixed(2)} MB`);
897
- console.log(`Age: ${((Date.now() - entry.metadata.timestamp) / 1000 / 60 / 60).toFixed(1)} hours`);
898
- });
899
- ```
900
-
901
- **Clean expired entries:**
902
- ```typescript
903
- // Remove entries older than 7 days (default)
904
- const deletedCount = await GtfsSqlJs.cleanExpiredCache(cache);
905
- console.log(`Deleted ${deletedCount} expired entries`);
906
-
907
- // Custom expiration time (3 days)
908
- const threeDays = 3 * 24 * 60 * 60 * 1000;
909
- await GtfsSqlJs.cleanExpiredCache(cache, threeDays);
910
- ```
911
-
912
- **Clear all cache:**
913
- ```typescript
914
- await GtfsSqlJs.clearCache(cache);
915
- ```
916
-
917
- #### Without Caching
918
-
919
- By default, caching is disabled. Simply omit the `cache` option:
920
-
921
- ```typescript
922
- const gtfs = await GtfsSqlJs.fromZip('gtfs.zip');
923
- // No caching - GTFS is processed fresh each time
924
- ```
925
-
926
- #### Custom Cache Expiration
927
-
928
- Change the default expiration time (default: 7 days):
929
-
930
- ```typescript
931
- const gtfs = await GtfsSqlJs.fromZip('gtfs.zip', {
932
- cacheExpirationMs: 3 * 24 * 60 * 60 * 1000 // 3 days
933
- });
934
- ```
935
-
936
- #### Custom Cache Store Implementation
937
-
938
- Implement your own cache store (e.g., Redis, S3):
939
-
940
- ```typescript
941
- import type { CacheStore, CacheMetadata } from 'gtfs-sqljs';
942
-
943
- class RedisCacheStore implements CacheStore {
944
- async get(key: string): Promise<ArrayBuffer | null> {
945
- // Implement Redis get
946
- }
947
-
948
- async set(key: string, data: ArrayBuffer, metadata: CacheMetadata): Promise<void> {
949
- // Implement Redis set
950
- }
951
-
952
- async has(key: string): Promise<boolean> {
953
- // Implement Redis exists check
954
- }
955
-
956
- async delete(key: string): Promise<void> {
957
- // Implement Redis delete
958
- }
959
-
960
- async clear(): Promise<void> {
961
- // Implement Redis clear
962
- }
963
-
964
- async list(): Promise<CacheEntry[]> {
965
- // Optional: Implement list
966
- }
967
- }
968
-
969
- const cache = new RedisCacheStore();
970
- const gtfs = await GtfsSqlJs.fromZip('gtfs.zip', { cache });
971
- ```
972
-
973
- ### Export Database
974
-
975
- ```typescript
976
- // Export to ArrayBuffer for storage (includes RT data)
977
- const buffer = gtfs.export();
978
-
979
- // Save to file (Node.js)
980
- import fs from 'fs';
981
- fs.writeFileSync('gtfs.db', Buffer.from(buffer));
982
-
983
- // Store in IndexedDB (Browser)
984
- // ... use IndexedDB API to store the ArrayBuffer
985
- ```
986
-
987
- ### Advanced Usage
988
-
989
- #### Direct Database Access
990
-
991
- For advanced queries not covered by the API:
992
-
993
- ```typescript
994
- const db = gtfs.getDatabase();
995
-
996
- const stmt = db.prepare('SELECT * FROM stops WHERE stop_lat > ? AND stop_lon < ?');
997
- stmt.bind([40.7, -74.0]);
998
-
999
- while (stmt.step()) {
1000
- const row = stmt.getAsObject();
1001
- console.log(row);
1002
- }
1003
-
1004
- stmt.free();
1005
- ```
1006
-
1007
- #### Close Database
1008
-
1009
- ```typescript
1010
- // Close the database when done
88
+ // Clean up
1011
89
  gtfs.close();
1012
90
  ```
1013
91
 
1014
- ## Complete Example
1015
-
1016
- ```typescript
1017
- import { GtfsSqlJs } from 'gtfs-sqljs';
1018
-
1019
- async function example() {
1020
- // Load GTFS data (skip shapes.txt to reduce memory usage)
1021
- const gtfs = await GtfsSqlJs.fromZip('https://example.com/gtfs.zip', {
1022
- skipFiles: ['shapes.txt']
1023
- });
1024
-
1025
- // Find a stop using flexible filters
1026
- const stops = gtfs.getStops({ name: 'Central Station' });
1027
- const stop = stops[0];
1028
- console.log(`Found stop: ${stop.stop_name}`);
1029
-
1030
- // Find routes serving this stop (via stop_times and trips)
1031
- const allStopTimes = gtfs.getStopTimes({ stopId: stop.stop_id });
1032
- const routeIds = new Set(
1033
- allStopTimes.map(st => {
1034
- const trips = gtfs.getTrips({ tripId: st.trip_id });
1035
- return trips.length > 0 ? trips[0].route_id : null;
1036
- })
1037
- );
1038
-
1039
- // Get route details
1040
- for (const routeId of routeIds) {
1041
- if (!routeId) continue;
1042
- const routes = gtfs.getRoutes({ routeId });
1043
- const route = routes.length > 0 ? routes[0] : null;
1044
- console.log(`Route: ${route?.route_short_name} - ${route?.route_long_name}`);
1045
- }
1046
-
1047
- // Get trips for a specific route on a date using flexible filters
1048
- const today = '20240115'; // YYYYMMDD format
1049
- const trips = gtfs.getTrips({
1050
- routeId: Array.from(routeIds)[0]!,
1051
- date: today
1052
- });
1053
- console.log(`Found ${trips.length} trips for today`);
1054
-
1055
- // Get stop times for a specific trip
1056
- const stopTimes = gtfs.getStopTimes({ tripId: trips[0].trip_id });
1057
- console.log('Trip schedule:');
1058
- for (const st of stopTimes) {
1059
- const stops = gtfs.getStops({ stopId: st.stop_id });
1060
- const stop = stops.length > 0 ? stops[0] : null;
1061
- console.log(` ${st.arrival_time} - ${stop?.stop_name}`);
1062
- }
1063
-
1064
- // Export database for later use
1065
- const buffer = gtfs.export();
1066
- // ... save buffer to file or storage
1067
-
1068
- // Clean up
1069
- gtfs.close();
1070
- }
1071
-
1072
- example();
1073
- ```
92
+ For detailed usage examples, see the [Usage Guide](https://sysdevrun.github.io/gtfs-sqljs/docs/documents/Usage_Guide.html).
1074
93
 
1075
94
  ## API Reference
1076
95
 
96
+ Full API documentation: [API Reference](https://sysdevrun.github.io/gtfs-sqljs/docs/)
97
+
1077
98
  ### Static Methods
1078
99
 
1079
100
  - `GtfsSqlJs.fromZip(zipPath, options?)` - Create instance from GTFS ZIP file
@@ -1095,7 +116,7 @@ All methods support flexible filtering with both single values and arrays:
1095
116
 
1096
117
  #### Calendar Methods
1097
118
  - `getActiveServiceIds(date)` - Get active service IDs for a date (YYYYMMDD format)
1098
- - `getCalendarByServiceId(serviceId)` - Get calendar by service_id
119
+ - `getCalendars(filters?)` - Get calendars (filters: serviceId, limit)
1099
120
  - `getCalendarDates(serviceId)` - Get calendar date exceptions for a service
1100
121
  - `getCalendarDatesForDate(date)` - Get calendar exceptions for a specific date
1101
122
 
@@ -1142,25 +163,6 @@ import type {
1142
163
  // Progress tracking types
1143
164
  ProgressInfo, ProgressCallback
1144
165
  } from 'gtfs-sqljs';
1145
-
1146
- const stop: Stop = gtfs.getStopById('STOP_123')!;
1147
-
1148
- // Use filter types for better type safety
1149
- const filters: TripFilters = {
1150
- routeId: 'ROUTE_1',
1151
- directionId: 0,
1152
- includeRealtime: true
1153
- };
1154
- const trips = gtfs.getTrips(filters);
1155
-
1156
- // RT types
1157
- const alerts: Alert[] = gtfs.getAlerts({ activeOnly: true });
1158
- const vehicles: VehiclePosition[] = gtfs.getVehiclePositions();
1159
-
1160
- // Progress callback with types
1161
- const handleProgress: ProgressCallback = (progress) => {
1162
- console.log(`${progress.percentComplete}% - ${progress.message}`);
1163
- };
1164
166
  ```
1165
167
 
1166
168
  ## GTFS Specification