fit-file-parser 4.0.1 → 4.1.0

package/CHANGELOG.md

@@ -1,5 +1,53 @@
 # Change Log
 
+## Unreleased
+
+## 4.1.0
+
+### Added
+
+- Generate all 124 standard messages, 1,406 fields, and 200 profile types from
+  the exactly pinned Garmin FIT SDK 21.208.0 profile.
+- Retain every recognized message in file order under the typed
+  `ParsedFit.messages` index without changing existing list, cascade, or
+  singleton outputs.
+- Decode Garmin strength-training `set` messages in list and cascade modes.
+- Add reproducible generated-profile and privacy-safe external corpus audits,
+  and enforce profile freshness and coverage in CI.
+
+### Fixed
+
+- Decode fields from their wire base types, including compatible developer
+  enum/uint8/byte definitions and correctly sized numeric arrays.
+- Reconstruct compressed timestamps and keep timestamp state isolated between
+  parser instances.
+- Accept omitted header CRCs, validate file CRCs across the complete FIT header
+  and data section, and report strict header and file CRC failures explicitly.
+- Reject structurally unsafe FIT inputs consistently in callback and Promise
+  APIs while retaining force-mode recovery for CRC corruption.
+- Correct Garmin profile mappings for OHR settings, monitoring HR, sleep,
+  time-in-zone, altitude offsets, and lap/segment flow and grit summaries.
+- Correct Celsius-to-Kelvin conversion, add `celsius` as the canonical
+  temperature unit, and retain `°C` as a supported alias.
+
+### Compatibility and documentation
+
+- Preserve compatible legacy field names, scales, value shapes, parser
+  signatures, output modes, and date behavior while adding canonical profile
+  names.
+- Add regression coverage for temperature, pressure, validation, compressed
+  timestamps, generated profile messages, repeated messages, and MTB
+  flow/grit data.
+- Refresh the README with current runtime, API, units, output modes, developer
+  fields, encoder behavior, and repository commands.
+
+## 4.0.2
+
+- Preserve record alignment when developer-field descriptions are missing or
+  appear after their message definitions.
+- Decode subsequent developer-field values once their descriptions become
+  available, in both strict and force modes.
+
 ## 4.0.0
 
 ### FIT decoder performance

package/README.md

@@ -1,181 +1,252 @@
 # fit-file-parser
 
-> Parse your .FIT files easily, directly from JS.
-> Written in Typescript
+[![CI](https://github.com/jimmykane/fit-parser/actions/workflows/ci.yml/badge.svg)](https://github.com/jimmykane/fit-parser/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/fit-file-parser.svg)](https://www.npmjs.com/package/fit-file-parser)
+[![license](https://img.shields.io/npm/l/fit-file-parser.svg)](./LICENSE)
 
-## Install
+Parse and encode FIT files in JavaScript and TypeScript. The parser supports
+files produced by Garmin, Polar, Suunto, and other FIT-compatible devices,
+including developer-defined data.
 
-```
-$ npm install fit-file-parser --save
-```
+## Features
+
+- Parse Node.js `Buffer` and standard `ArrayBuffer` inputs.
+- Choose flat lists, nested activity data, or both output shapes.
+- Convert speed, length, temperature, and pressure fields to preferred units.
+- Decode developer fields while preserving record alignment when descriptions
+  arrive after their definitions.
+- Encode profile-agnostic FIT messages with validated field definitions and
+  CRCs.
+- Use ESM or CommonJS with bundled TypeScript declarations.
+
+## Requirements
 
-## How to use
+- Node.js 20 or newer
 
-See in [examples](./examples) folder:
+## Installation
 
-### using callbacks
+```sh
+npm install fit-file-parser
+```
+
+## Quick start
+
+The Promise API is the simplest way to parse a file:
 
 ```javascript
-import fs from 'node:fs/promises'
+import { readFile } from 'node:fs/promises'
 import FitParser from 'fit-file-parser'
 
-fs.readFile('./example.fit', (err, content) => {
-  // Create a FitParser instance (options argument is optional)
-  if (err) {
-    console.error(err)
-  }
-  const fitParser = new FitParser({
-    force: true,
-    speedUnit: 'km/h',
-    lengthUnit: 'km',
-    temperatureUnit: 'kelvin',
-    pressureUnit: 'bar', // accept bar, cbar and psi (default is bar)
-    elapsedRecordField: true,
-    mode: 'cascade',
-  })
+const content = await readFile('./activity.fit')
+const parser = new FitParser({
+  mode: 'list',
+  speedUnit: 'km/h',
+  lengthUnit: 'km',
+})
 
-  // Parse your file
-  fitParser.parse(content, (error, data) => {
-    // Handle result of parse method
-    if (error) {
-      console.error(error)
-    }
-    else {
-      console.log(JSON.stringify(data))
-    }
-  })
+const data = await parser.parseAsync(content)
+
+console.log({
+  sessions: data.sessions?.length ?? 0,
+  laps: data.laps?.length ?? 0,
+  records: data.records?.length ?? 0,
 })
 ```
 
-### using async/await
+### Callback API
 
 ```javascript
-import fs from 'node:fs/promises'
+import { readFile } from 'node:fs'
 import FitParser from 'fit-file-parser'
 
-const buffer = await fs.readFile('./example.fit')
-const fitObject = await fitParser.parseAsync(buffer)
+readFile('./activity.fit', (readError, content) => {
+  if (readError) {
+    console.error(readError)
+    return
+  }
+
+  const parser = new FitParser()
+  parser.parse(content, (parseError, data) => {
+    if (parseError) {
+      console.error(parseError)
+      return
+    }
+
+    console.log(data)
+  })
+})
 ```
 
-## Encoding
+Parser errors are strings. `parseAsync()` rejects with the same value that the
+callback API receives as its first argument.
 
-`FitEncoder` writes FIT headers, message definitions, data messages, and CRCs.
-It is profile-agnostic: callers provide profile field identifiers and values in
-their raw FIT representation (including any scale or offset). Scalar 64-bit
-values use `bigint`; strings and numeric arrays use exact-size raw
-`Uint8Array` values.
+## Parser options
 
-```javascript
-import { FitBaseType, FitEncoder } from 'fit-file-parser'
+All options are optional.
 
-const encoder = new FitEncoder()
-encoder.writeMessage(0, [
-  { number: 0, size: 1, baseType: FitBaseType.Enum, value: 6 }, // FileId.type = course
-  { number: 4, size: 4, baseType: FitBaseType.Uint32, value: FitEncoder.toFitTimestamp(new Date()) },
-])
+| Option               | Values                                  | Default   | Behavior                                                                                                                                       |
+| -------------------- | --------------------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| `mode`               | `list`, `cascade`, `both`               | `list`    | Controls whether primary activity collections are returned as root lists, nested data, or both.                                                |
+| `force`              | `true`, `false`                         | `true`    | Skips header and file CRC validation and enables supported best-effort field recovery. Structural header and data bounds are always validated. |
+| `speedUnit`          | `m/s`, `km/h`, `mph`                    | `m/s`     | Converts speed-related fields.                                                                                                                 |
+| `lengthUnit`         | `m`, `km`, `mi`                         | `m`       | Converts distance, altitude, and other length-related fields.                                                                                  |
+| `temperatureUnit`    | `celsius`, `°C`, `kelvin`, `fahrenheit` | `celsius` | Converts temperature fields. `°C` remains available as a legacy alias.                                                                         |
+| `pressureUnit`       | `bar`, `cbar`, `psi`                    | `bar`     | Converts pressure and tank-pressure fields.                                                                                                    |
+| `elapsedRecordField` | `true`, `false`                         | `false`   | Adds `elapsed_time` and `timer_time`, in seconds, to records.                                                                                  |
 
-const fitBytes = encoder.close()
-```
+`force: true` does not make arbitrary bytes a valid FIT file. Inputs that are
+too short, have an invalid header size or signature, or declare data beyond the
+available bytes are rejected in both modes.
 
-Use a distinct local message number (the optional third `writeMessage`
-argument) for each recurring message shape to avoid redundant definitions.
-The encoder validates field definitions and numeric ranges before writing, so
-an exception never leaves a partial message in the output.
+## Output modes
 
-## Development
+The mode controls where sessions, laps, records, and related activity
+collections are exposed.
 
-To build the project, run:
+| Mode      | Root lists | Nested under `activity` | Default |
+| --------- | ---------- | ----------------------- | ------- |
+| `list`    | Yes        | No                      | Yes     |
+| `cascade` | No         | Yes                     | No      |
+| `both`    | Yes        | Yes                     | No      |
 
-```
-npm run build
-```
+In cascade output, sessions contain their laps and laps contain their records
+and lengths. Other parsed FIT message collections remain available where the
+parser exposes them.
 
-To run tests, run:
+Every recognized message is also retained in file order in `data.messages`.
+This additive index is useful for message kinds that historically exposed
+only the last value at the root:
 
-```
-npm test
+```javascript
+const workoutSteps = data.messages?.workout_step ?? []
+const diveSummaries = data.messages?.dive_summary ?? []
 ```
 
-To rebuild the typescript types (as they are autogenerated from the `FIT` object) run:
+Existing root lists, cascade nesting, and last-message root properties remain
+unchanged.
 
-```npm run codegen
+## Inputs
 
-```
+Both parser methods accept:
 
-> this will update the file `src/fit_types.ts` as it is only running with node (with type stripping), regenerating them requires node>=245
+- Node.js `Buffer`
+- `ArrayBuffer`
 
-To run the codegen in watch mode during local development, run:
+```javascript
+const parsed = await new FitParser().parseAsync(arrayBuffer)
+```
 
-```npm run dev
+## Developer fields
 
-```
+FIT producers may define custom fields outside the standard profile. The
+parser consumes every developer field's declared byte size so later messages
+stay aligned. If a field description is not available yet, that value is
+omitted. Subsequent values are decoded by name once the description appears.
 
-To build the local examples, run:
+## Encoding
 
-```
-npm run examples
-```
+`FitEncoder` writes FIT headers, message definitions, data messages, and CRCs.
+It is profile-agnostic: callers provide profile message and field numbers,
+base types, sizes, and values in their raw FIT representation. Applying FIT
+scales and offsets is the caller's responsibility.
 
-To lint and format the code, run:
+```javascript
+import { FitBaseType, FitEncoder } from 'fit-file-parser'
 
-```
-npm run lint
+const encoder = new FitEncoder()
+encoder.writeMessage(0, [
+  {
+    number: 0,
+    size: 1,
+    baseType: FitBaseType.Enum,
+    value: 6,
+  },
+  {
+    number: 4,
+    size: 4,
+    baseType: FitBaseType.Uint32,
+    value: FitEncoder.toFitTimestamp(new Date()),
+  },
+])
+
+const fitBytes = encoder.close()
 ```
 
-To run the typechecker, run:
+`writeMessage(globalMessageNumber, fields, localMessageNumber?)` accepts local
+message numbers from 0 through 15. Definitions are emitted automatically and
+reused until the shape assigned to that local number changes. `close()` returns
+a `Uint8Array`.
 
-```
-npm run type-check
-```
+The encoder also provides:
 
-## API Documentation
+- `FitEncoder.string(value)` for null-terminated UTF-8 field bytes.
+- `FitEncoder.toFitTimestamp(value)` for FIT timestamps.
+- `FitEncoder.calculateCRC(bytes)` for FIT-compatible CRC calculation.
 
-### new FitParser(Object _options_)
+Scalar 64-bit values use `bigint`. Strings, numeric arrays, and other
+variable-length values use exact-size `Uint8Array` values. Invalid field
+definitions or numeric ranges throw before a partial message is written.
 
-Needed to create a new instance. _options_ is optional, and is used to customize the returned object.
+## TypeScript and module formats
 
-Allowed properties :
+The package includes TypeScript declarations and exports
+`FitParserOptions`, `FitEncoderField`, and `FitEncoderOptions`.
 
-- `mode`: String
-  - `cascade`: Returned object is organized as a tree, eg. each lap contains a `records` fields, that is an array of its records (**default**)
-  - `list`: Returned object is organized as lists of sessions, laps, records, etc..., without parent-child relation
-  - `both`: A mix of the two other modes, eg. `records` are available inside the root field as well as inside each laps
-- `lengthUnit`: String
-  - `m`: Lengths are in meters (**default**)
-  - `km`: Lengths are in kilometers
-  - `mi`: Lengths are in miles
-- `temperatureUnit`: String
-  - `celsius`:Temperatures are in °C (**default**)
-  - `kelvin`: Temperatures are in °K
-  - `fahrenheit`: Temperatures are in °F
-- `speedUnit`: String
-  - `m/s`: Speeds are in meters per seconds (**default**)
-  - `km/h`: Speeds are in kilometers per hour
-  - `mph`: Speeds are in miles per hour
-- `force`: Boolean
-  - `true`: Continues even if they are errors (**default for now**)
-  - `false`: Stops if an error occurs
-- `elapsedRecordField`: Boolean
-  - `true`: Includes `elapsed_time`, containing the elapsed time in seconds since the first record, and `timer_time`, containing the time shown on the device, inside each `record` field
-  - `false` (**default**)
+ESM:
 
-### fitParser.parse(Buffer _file_, Function _callback_)
+```javascript
+import FitParser, { FitBaseType, FitEncoder } from 'fit-file-parser'
+```
 
-_callback_ receives two arguments, the first as a error String, and the second as Object, result of parsing.
+CommonJS:
 
-### fitParser.parseAsync(Buffer _file_)
+```javascript
+const {
+  default: FitParser,
+  FitBaseType,
+  FitEncoder,
+} = require('fit-file-parser')
+```
 
-returns a Promise that resolves to the result of parsing.
+## Development
 
-## Contributors
+Run commands from the repository root.
+
+| Command                            | Purpose                                            |
+| ---------------------------------- | -------------------------------------------------- |
+| `npm ci`                           | Install locked dependencies.                       |
+| `npm run build`                    | Build ESM and CommonJS output.                     |
+| `npm test -- --run`                | Run the complete test suite once.                  |
+| `npm test -- --run test/<file>.ts` | Run a focused test file.                           |
+| `npm run codegen`                  | Regenerate the Garmin profile and public types.    |
+| `npm run codegen:check`            | Verify both generated files are current.           |
+| `npm run profile:audit`            | Audit SDK profile coverage and private overlays.   |
+| `npm run corpus:check -- <path>`   | Validate an external FIT corpus without file data. |
+| `npm run lint`                     | Check lint and formatting rules.                   |
+| `npm run fmt`                      | Apply the configured formatting rules.             |
+| `npm run type-check`               | Run TypeScript without emitting files.             |
+| `npm run examples`                 | Build and regenerate checked-in example outputs.   |
+| `npm run check`                    | Run profile audit, lint, types, tests, and builds. |
+
+Do not edit `src/garmin_profile.generated.ts` or `src/fit_types.ts` manually.
+Update the pinned SDK, compatibility overrides, or a generator, then run
+`npm run codegen`.
+
+Repository-specific automation guidance is tracked in
+[`.agent/README.md`](./.agent/README.md). More examples are available in the
+[`examples`](./examples) directory, and release notes are in the
+[`CHANGELOG`](./CHANGELOG.md).
 
-All started thanks to [Pierre Jacquier](https://github.com/pierremtb)
+## Contributors
 
-Big thanks to [Mikael Lofjärd](https://github.com/mlofjard) for [his early prototype](https://github.com/mlofjard/jsonfit).
-See [CONTRIBUTORS](./CONTRIBUTORS.md).
+This project started from work by
+[Pierre Jacquier](https://github.com/pierremtb). Thanks to
+[Mikael Lofjärd](https://github.com/mlofjard) for
+[his early prototype](https://github.com/mlofjard/jsonfit), and to everyone in
+[`CONTRIBUTORS.md`](./CONTRIBUTORS.md).
 
 ## License
 
-MIT license; see [LICENSE](./LICENSE).
+MIT; see [`LICENSE`](./LICENSE).
 
-(c) 2019 Dimitrios Kanellopoulos
+Copyright 2019-present Dimitrios Kanellopoulos.

package/dist/binary.d.ts

@@ -7,10 +7,22 @@ export interface MessageTypeDefinition {
     globalMessageNumber: number;
     numberOfFields: number;
     fieldDefs: FieldDefinition[];
+    developerFieldDefs?: DeveloperFieldDefinition[];
     rawData?: any[];
 }
+export interface DeveloperFieldDefinition {
+    developerDataIndex: number;
+    fieldDefinitionNumber: number;
+    size: number;
+    resolvedFieldDef?: FieldDefinition;
+    resolvedFrom?: unknown;
+}
+export interface DecoderState {
+    lastTimestamp?: number;
+    monitoringTimestamp?: number;
+}
 export declare function addEndian(littleEndian: boolean, bytes: number[]): number;
-export declare function readRecord(blob: Uint8Array, messageTypes: MessageTypeDefinition[], developerFields: any[], startIndex: number, options: FitParserOptions, startDate: number | undefined, pausedTime: number, dataView?: DataView): {
+export declare function readRecord(blob: Uint8Array, messageTypes: MessageTypeDefinition[], developerFields: any[], startIndex: number, options: FitParserOptions, startDate: number | undefined, pausedTime: number, dataView?: DataView, decoderState?: DecoderState): {
     messageType: MesgNum | '';
     nextIndex: number;
     message?: any;