fit-file-parser 4.0.2 → 5.0.1

package/CHANGELOG.md

@@ -1,5 +1,96 @@
 # Change Log
 
+## Unreleased
+
+## 5.0.1
+
+### Fixed
+
+- Run CI and version-driven npm publishing from the repository's `main`
+  branch, with a guarded manual recovery path for a missed release.
+
+## 5.0.0
+
+### Changed
+
+- Generate the complete standard message, field, and type surface from the
+  pinned Garmin FIT SDK without handwritten standard-profile overrides.
+- Preserve SDK alphanumeric tokens when generating public `snake_case` names,
+  including `n2`, `po2`, `time128`, and Garmin product identifiers.
+- Restrict non-SDK support to a collision-checked allowlist of Garmin fields
+  and private messages observed in the external FIT corpus.
+- Emit only parsed FIT values: Garmin product names are no longer inferred,
+  and record elapsed/timer values are generated only when
+  `elapsedRecordField: true` is requested.
+
+### Breaking
+
+- Remove compatibility aliases such as `resting_calories`,
+  `recovery_advisor`, `repeat_dive_time`, and deprecated time-in-zone names.
+- Remove guessed stress fields, handwritten enum/type additions, placeholder
+  zero-mask names, and the misspelled `hip_swing_excercise_name` type.
+- Use the pinned SDK's `cadence_zone_high_bondary` spelling and generated
+  alphanumeric field/type spellings instead of manual corrections.
+- Correct generated declarations to expose runtime `Date` values, numeric FIT
+  `bool` fields and unknown enum IDs, decoded mask objects, nullable array
+  entries, and optional profile fields.
+- Keep undated cascade rows in file order and avoid duplicating them when a lap
+  or session boundary timestamp is absent.
+
+## 4.1.1
+
+### Fixed
+
+- Apply Garmin FIT profile scale and offset to primitive numeric fields, while
+  retaining semicircle conversion only for fields declared in semicircles.
+- Apply FIT developer-field offsets with the parser's equivalent signed form,
+  including scaled signed developer values.
+- Decode native diving ascent-rate fields as meters per second instead of
+  geographic coordinates.
+- Decode native dive depth and bottom-time fields with their Garmin FIT SDK
+  scale instead of exposing their encoded integer representation.
+- Keep the pinned Garmin SDK authoritative for standard field types, arrays,
+  scales, offsets, and units while retaining compatible public field names.
+
+## 4.1.0
+
+### Added
+
+- Generate all 124 standard messages, 1,406 fields, and 200 profile types from
+  the exactly pinned Garmin FIT SDK 21.208.0 profile.
+- Retain every recognized message in file order under the typed
+  `ParsedFit.messages` index without changing existing list, cascade, or
+  singleton outputs.
+- Decode Garmin strength-training `set` messages in list and cascade modes.
+- Add reproducible generated-profile and privacy-safe external corpus audits,
+  and enforce profile freshness and coverage in CI.
+
+### Fixed
+
+- Decode fields from their wire base types, including compatible developer
+  enum/uint8/byte definitions and correctly sized numeric arrays.
+- Reconstruct compressed timestamps and keep timestamp state isolated between
+  parser instances.
+- Accept omitted header CRCs, validate file CRCs across the complete FIT header
+  and data section, and report strict header and file CRC failures explicitly.
+- Reject structurally unsafe FIT inputs consistently in callback and Promise
+  APIs while retaining force-mode recovery for CRC corruption.
+- Correct Garmin profile mappings for OHR settings, monitoring HR, sleep,
+  time-in-zone, altitude offsets, and lap/segment flow and grit summaries.
+- Correct Celsius-to-Kelvin conversion, add `celsius` as the canonical
+  temperature unit, and retain `°C` as a supported alias.
+
+### Compatibility and documentation
+
+- Preserve compatible legacy field names, scales, value shapes, parser
+  signatures, output modes, and date behavior while adding canonical profile
+  names.
+- Add regression coverage for temperature, pressure, validation, compressed
+  timestamps, generated profile messages, repeated messages, and MTB
+  flow/grit data.
+- Refresh the README with current runtime, API, units, output modes, developer
+  fields, encoder behavior, and repository commands.
+
 ## 4.0.2
 
 - Preserve record alignment when developer-field descriptions are missing or

package/README.md

@@ -1,181 +1,337 @@
 # fit-file-parser
 
-> Parse your .FIT files easily, directly from JS.
-> Written in Typescript
+[![CI](https://github.com/jimmykane/fit-parser/actions/workflows/ci.yml/badge.svg)](https://github.com/jimmykane/fit-parser/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/fit-file-parser.svg)](https://www.npmjs.com/package/fit-file-parser)
+[![license](https://img.shields.io/npm/l/fit-file-parser.svg)](./LICENSE)
 
-## Install
+Parse and encode FIT files in JavaScript and TypeScript. The parser supports
+files produced by Garmin, Polar, Suunto, and other FIT-compatible devices,
+including developer-defined data.
 
+## Features
+
+- Parse Node.js `Buffer` and standard `ArrayBuffer` inputs.
+- Choose flat lists, nested activity data, or both output shapes.
+- Convert speed, length, temperature, and pressure fields to preferred units.
+- Decode developer fields while preserving record alignment when descriptions
+  arrive after their definitions.
+- Encode profile-agnostic FIT messages with validated field definitions and
+  CRCs.
+- Use ESM or CommonJS with bundled TypeScript declarations.
+
+## Requirements
+
+- Node.js 20 or newer
+
+## Installation
+
+```sh
+npm install fit-file-parser
 ```
-$ npm install fit-file-parser --save
-```
 
-## How to use
+## Migrating from 4.x to 5.0
+
+Version 5.0 is a breaking release because parsed output now follows the pinned
+Garmin FIT SDK profile without compatibility aliases or guessed fields. Parser
+construction, module imports, output modes, and parser options are unchanged.
+
+Update field access, destructuring, persisted schemas, and snapshots to use the
+SDK-backed names. Common migrations include:
+
+| 4.x name                       | 5.0 name                     |
+| ------------------------------ | ---------------------------- |
+| `speed_1s`                     | `speed1s`                    |
+| `start_n_2`, `end_n_2`         | `start_n2`, `end_n2`         |
+| `o_2_toxicity`                 | `o2_toxicity`                |
+| `avg_spo_2`, `reading_spo_2`   | `avg_spo2`, `reading_spo2`   |
+| `po_2`                         | `po2`                        |
+| `cycle_length_16`              | `cycle_length16`             |
+| `map_3_sample_mean`            | `map3_sample_mean`           |
+| `time_256`                     | `time256`                    |
+| `last_night_5_min_high`        | `last_night5_min_high`       |
+| `average_7_day_deviation`      | `average7_day_deviation`     |
+| `spo_2_data`, `hsa_spo_2_data` | `spo2_data`, `hsa_spo2_data` |
+| `repeat_dive_time`             | `repeat_dive_interval`       |
+| `cadence_zone_high_boundary`   | `cadence_zone_high_bondary`  |
+| `HipSwingExcerciseName`        | `HipSwingExerciseName`       |
+
+The same alphanumeric-token rule applies to enum strings, for example
+`camera_orientation_90` becomes `camera_orientation90`, `po_2_warn` becomes
+`po2_warn`, and `power_3s` becomes `power3s`. The generated TypeScript
+declarations are the exhaustive name and value reference for the pinned SDK.
+
+The handwritten `resting_calories` and guessed `recovery_advisor` fields have
+no 5.0 replacement. Other behavior to account for during migration:
+
+- `product_name` is emitted only when it exists in the FIT input. It is no
+  longer inferred from `manufacturer` and `product`.
+- Record `elapsed_time` and `timer_time` require
+  `elapsedRecordField: true`.
+- FIT timestamps are typed as `Date`, FIT `bool` values remain numeric, mask
+  fields are `{ value, ...flags }` objects, unknown enum values remain numbers,
+  and array entries may be `null` when the FIT invalid sentinel is retained.
+- Every profile field is optional because individual FIT message definitions
+  determine which fields are present.
+- When upgrading from 4.1.0 or earlier, remove application-side scale or offset
+  corrections for parsed numeric values, including dive depth, bottom time,
+  ascent rate, and developer fields. The parser now applies the SDK metadata.
+
+## Quick start
+
+The Promise API is the simplest way to parse a file:
+
+```javascript
+import { readFile } from 'node:fs/promises'
+import FitParser from 'fit-file-parser'
+
+const content = await readFile('./activity.fit')
+const parser = new FitParser({
+  mode: 'list',
+  speedUnit: 'km/h',
+  lengthUnit: 'km',
+})
+
+const data = await parser.parseAsync(content)
 
-See in [examples](./examples) folder:
+console.log({
+  sessions: data.sessions?.length ?? 0,
+  laps: data.laps?.length ?? 0,
+  records: data.records?.length ?? 0,
+})
+```
 
-### using callbacks
+### Callback API
 
 ```javascript
-import fs from 'node:fs/promises'
+import { readFile } from 'node:fs'
 import FitParser from 'fit-file-parser'
 
-fs.readFile('./example.fit', (err, content) => {
-  // Create a FitParser instance (options argument is optional)
-  if (err) {
-    console.error(err)
+readFile('./activity.fit', (readError, content) => {
+  if (readError) {
+    console.error(readError)
+    return
   }
-  const fitParser = new FitParser({
-    force: true,
-    speedUnit: 'km/h',
-    lengthUnit: 'km',
-    temperatureUnit: 'kelvin',
-    pressureUnit: 'bar', // accept bar, cbar and psi (default is bar)
-    elapsedRecordField: true,
-    mode: 'cascade',
-  })
 
-  // Parse your file
-  fitParser.parse(content, (error, data) => {
-    // Handle result of parse method
-    if (error) {
-      console.error(error)
-    }
-    else {
-      console.log(JSON.stringify(data))
+  const parser = new FitParser()
+  parser.parse(content, (parseError, data) => {
+    if (parseError) {
+      console.error(parseError)
+      return
     }
+
+    console.log(data)
   })
 })
 ```
 
-### using async/await
+Parser errors are strings. `parseAsync()` rejects with the same value that the
+callback API receives as its first argument.
 
-```javascript
-import fs from 'node:fs/promises'
-import FitParser from 'fit-file-parser'
+## Parser options
 
-const buffer = await fs.readFile('./example.fit')
-const fitObject = await fitParser.parseAsync(buffer)
-```
+All options are optional.
 
-## Encoding
+| Option               | Values                                  | Default   | Behavior                                                                                                                                       |
+| -------------------- | --------------------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| `mode`               | `list`, `cascade`, `both`               | `list`    | Controls whether primary activity collections are returned as root lists, nested data, or both.                                                |
+| `force`              | `true`, `false`                         | `true`    | Skips header and file CRC validation and enables supported best-effort field recovery. Structural header and data bounds are always validated. |
+| `speedUnit`          | `m/s`, `km/h`, `mph`                    | `m/s`     | Converts speed-related fields.                                                                                                                 |
+| `lengthUnit`         | `m`, `km`, `mi`                         | `m`       | Converts distance, altitude, and other length-related fields.                                                                                  |
+| `temperatureUnit`    | `celsius`, `°C`, `kelvin`, `fahrenheit` | `celsius` | Converts temperature fields. `°C` remains available as a legacy alias.                                                                         |
+| `pressureUnit`       | `bar`, `cbar`, `psi`                    | `bar`     | Converts pressure and tank-pressure fields.                                                                                                    |
+| `elapsedRecordField` | `true`, `false`                         | `false`   | Adds `elapsed_time` and `timer_time`, in seconds, to records.                                                                                  |
 
-`FitEncoder` writes FIT headers, message definitions, data messages, and CRCs.
-It is profile-agnostic: callers provide profile field identifiers and values in
-their raw FIT representation (including any scale or offset). Scalar 64-bit
-values use `bigint`; strings and numeric arrays use exact-size raw
-`Uint8Array` values.
+`force: true` does not make arbitrary bytes a valid FIT file. Inputs that are
+too short, have an invalid header size or signature, or declare data beyond the
+available bytes are rejected in both modes.
 
-```javascript
-import { FitBaseType, FitEncoder } from 'fit-file-parser'
+## Output modes
 
-const encoder = new FitEncoder()
-encoder.writeMessage(0, [
-  { number: 0, size: 1, baseType: FitBaseType.Enum, value: 6 }, // FileId.type = course
-  { number: 4, size: 4, baseType: FitBaseType.Uint32, value: FitEncoder.toFitTimestamp(new Date()) },
-])
+The mode controls where sessions, laps, records, and related activity
+collections are exposed.
 
-const fitBytes = encoder.close()
+| Mode      | Root lists | Nested under `activity` | Default |
+| --------- | ---------- | ----------------------- | ------- |
+| `list`    | Yes        | No                      | Yes     |
+| `cascade` | No         | Yes                     | No      |
+| `both`    | Yes        | Yes                     | No      |
+
+In cascade output, sessions contain their laps and laps contain their records
+and lengths. Other parsed FIT message collections remain available where the
+parser exposes them.
+
+Every recognized message is also retained in file order in `data.messages`.
+This additive index is useful for message kinds that historically exposed
+only the last value at the root:
+
+```javascript
+const workoutSteps = data.messages?.workout_step ?? []
+const diveSummaries = data.messages?.dive_summary ?? []
 ```
 
-Use a distinct local message number (the optional third `writeMessage`
-argument) for each recurring message shape to avoid redundant definitions.
-The encoder validates field definitions and numeric ranges before writing, so
-an exception never leaves a partial message in the output.
+Existing root lists, cascade nesting, and last-message root properties remain
+unchanged.
 
-## Development
+## Profile-backed output
 
-To build the project, run:
+Standard message names, field names, enum values, wire types, scales, offsets,
+arrays, and units come from the exactly pinned Garmin FIT SDK profile. Public
+names use the parser's generated `snake_case` form while preserving SDK
+alphanumeric tokens such as `n2`, `po2`, and `time128`. The parser does not add
+compatibility aliases for alternate field spellings.
 
-```
-npm run build
-```
+The small vendor extension table contains only Garmin fields and private
+messages observed in the external FIT corpus. Extensions cannot replace a
+standard SDK field or type value; the profile audit rejects collisions and
+unregistered additions.
 
-To run tests, run:
+Only values present in the FIT input are emitted. In particular,
+`product_name` is not inferred from `manufacturer` and `product`, and record
+`elapsed_time` and `timer_time` are added only when `elapsedRecordField: true`
+is requested. Parsed FIT timestamps are `Date` objects, FIT `bool` fields keep
+their numeric wire values, mask fields decode to `{ value, ...flags }` objects,
+unknown enum IDs remain numbers, and invalid entries retained inside FIT arrays
+are `null`. All profile fields are optional because each FIT message definition
+chooses which fields are present.
 
-```
-npm test
-```
+## Inputs
 
-To rebuild the typescript types (as they are autogenerated from the `FIT` object) run:
+Both parser methods accept:
 
-```npm run codegen
+- Node.js `Buffer`
+- `ArrayBuffer`
 
+```javascript
+const parsed = await new FitParser().parseAsync(arrayBuffer)
 ```
 
-> this will update the file `src/fit_types.ts` as it is only running with node (with type stripping), regenerating them requires node>=245
+## Developer fields
 
-To run the codegen in watch mode during local development, run:
+FIT producers may define custom fields outside the standard profile. The
+parser consumes every developer field's declared byte size so later messages
+stay aligned. If a field description is not available yet, that value is
+omitted. Subsequent values are decoded by name once the description appears.
 
-```npm run dev
+## Encoding
 
-```
+`FitEncoder` writes FIT headers, message definitions, data messages, and CRCs.
+It is profile-agnostic: callers provide profile message and field numbers,
+base types, sizes, and values in their raw FIT representation. Applying FIT
+scales and offsets is the caller's responsibility.
 
-To build the local examples, run:
+```javascript
+import { FitBaseType, FitEncoder } from 'fit-file-parser'
 
-```
-npm run examples
+const encoder = new FitEncoder()
+encoder.writeMessage(0, [
+  {
+    number: 0,
+    size: 1,
+    baseType: FitBaseType.Enum,
+    value: 6,
+  },
+  {
+    number: 4,
+    size: 4,
+    baseType: FitBaseType.Uint32,
+    value: FitEncoder.toFitTimestamp(new Date()),
+  },
+])
+
+const fitBytes = encoder.close()
 ```
 
-To lint and format the code, run:
+`writeMessage(globalMessageNumber, fields, localMessageNumber?)` accepts local
+message numbers from 0 through 15. Definitions are emitted automatically and
+reused until the shape assigned to that local number changes. `close()` returns
+a `Uint8Array`.
 
-```
-npm run lint
-```
+The encoder also provides:
 
-To run the typechecker, run:
+- `FitEncoder.string(value)` for null-terminated UTF-8 field bytes.
+- `FitEncoder.toFitTimestamp(value)` for FIT timestamps.
+- `FitEncoder.calculateCRC(bytes)` for FIT-compatible CRC calculation.
 
-```
-npm run type-check
-```
+Scalar 64-bit values use `bigint`. Strings, numeric arrays, and other
+variable-length values use exact-size `Uint8Array` values. Invalid field
+definitions or numeric ranges throw before a partial message is written.
 
-## API Documentation
+## TypeScript and module formats
 
-### new FitParser(Object _options_)
+The package includes TypeScript declarations and exports
+`FitParserOptions`, `FitEncoderField`, and `FitEncoderOptions`.
 
-Needed to create a new instance. _options_ is optional, and is used to customize the returned object.
+ESM:
 
-Allowed properties :
+```javascript
+import FitParser, { FitBaseType, FitEncoder } from 'fit-file-parser'
+```
 
-- `mode`: String
-  - `cascade`: Returned object is organized as a tree, eg. each lap contains a `records` fields, that is an array of its records (**default**)
-  - `list`: Returned object is organized as lists of sessions, laps, records, etc..., without parent-child relation
-  - `both`: A mix of the two other modes, eg. `records` are available inside the root field as well as inside each laps
-- `lengthUnit`: String
-  - `m`: Lengths are in meters (**default**)
-  - `km`: Lengths are in kilometers
-  - `mi`: Lengths are in miles
-- `temperatureUnit`: String
-  - `celsius`:Temperatures are in °C (**default**)
-  - `kelvin`: Temperatures are in °K
-  - `fahrenheit`: Temperatures are in °F
-- `speedUnit`: String
-  - `m/s`: Speeds are in meters per seconds (**default**)
-  - `km/h`: Speeds are in kilometers per hour
-  - `mph`: Speeds are in miles per hour
-- `force`: Boolean
-  - `true`: Continues even if they are errors (**default for now**)
-  - `false`: Stops if an error occurs
-- `elapsedRecordField`: Boolean
-  - `true`: Includes `elapsed_time`, containing the elapsed time in seconds since the first record, and `timer_time`, containing the time shown on the device, inside each `record` field
-  - `false` (**default**)
+CommonJS:
 
-### fitParser.parse(Buffer _file_, Function _callback_)
+```javascript
+const {
+  default: FitParser,
+  FitBaseType,
+  FitEncoder,
+} = require('fit-file-parser')
+```
 
-_callback_ receives two arguments, the first as a error String, and the second as Object, result of parsing.
+## Development
 
-### fitParser.parseAsync(Buffer _file_)
+Run commands from the repository root.
+
+| Command                            | Purpose                                            |
+| ---------------------------------- | -------------------------------------------------- |
+| `npm ci`                           | Install locked dependencies.                       |
+| `npm run build`                    | Build ESM and CommonJS output.                     |
+| `npm test -- --run`                | Run the complete test suite once.                  |
+| `npm test -- --run test/<file>.ts` | Run a focused test file.                           |
+| `npm run codegen`                  | Regenerate the Garmin profile and public types.    |
+| `npm run codegen:check`            | Verify both generated files are current.           |
+| `npm run profile:audit`            | Audit SDK profile coverage and private overlays.   |
+| `npm run corpus:check -- <path>`   | Validate an external FIT corpus without file data. |
+| `npm run lint`                     | Check lint and formatting rules.                   |
+| `npm run fmt`                      | Apply the configured formatting rules.             |
+| `npm run type-check`               | Run TypeScript without emitting files.             |
+| `npm run examples`                 | Build and regenerate checked-in example outputs.   |
+| `npm run check`                    | Run profile audit, lint, types, tests, and builds. |
+
+### External FIT corpus (optional)
+
+The external corpus is not part of this repository or the npm package. For the
+standard contributor layout, clone it alongside this checkout, then run the
+aggregate-only validation command:
+
+```sh
+git clone https://github.com/ThomasKuehne/FIT-test-files.git ../FIT-test-files
+npm run corpus:check -- ../FIT-test-files --allow-force-recovery
+```
 
-returns a Promise that resolves to the result of parsing.
+The command accepts any corpus path; the sibling location is only a convenient
+convention. The corpus contains a known header-CRC failure that is expected to
+recover only in force mode. It reports aggregate counts and never prints file
+names or parsed activity data.
 
-## Contributors
+Do not edit `src/garmin_profile.generated.ts` or `src/fit_types.ts` manually.
+Update the pinned SDK, audited vendor extensions, or a generator, then run
+`npm run codegen`.
+
+Repository-specific automation guidance is tracked in
+[`.agent/README.md`](./.agent/README.md). More examples are available in the
+[`examples`](./examples) directory, and release notes are in the
+[`CHANGELOG`](./CHANGELOG.md).
 
-All started thanks to [Pierre Jacquier](https://github.com/pierremtb)
+## Contributors
 
-Big thanks to [Mikael Lofjärd](https://github.com/mlofjard) for [his early prototype](https://github.com/mlofjard/jsonfit).
-See [CONTRIBUTORS](./CONTRIBUTORS.md).
+This project started from work by
+[Pierre Jacquier](https://github.com/pierremtb). Thanks to
+[Mikael Lofjärd](https://github.com/mlofjard) for
+[his early prototype](https://github.com/mlofjard/jsonfit), and to everyone in
+[`CONTRIBUTORS.md`](./CONTRIBUTORS.md).
 
 ## License
 
-MIT license; see [LICENSE](./LICENSE).
+MIT; see [`LICENSE`](./LICENSE).
 
-(c) 2019 Dimitrios Kanellopoulos
+Copyright 2019-present Dimitrios Kanellopoulos.

package/dist/binary.d.ts

@@ -1,6 +1,5 @@
 import type { FitParserOptions } from './fit-parser.js';
-import type { FieldDefinition } from './fit.js';
-import type { MesgNum } from './fit_types.js';
+import type { FieldDefinition, MessageName } from './fit.js';
 import { Buffer } from 'buffer';
 export interface MessageTypeDefinition {
     littleEndian: boolean;
@@ -17,9 +16,13 @@ export interface DeveloperFieldDefinition {
     resolvedFieldDef?: FieldDefinition;
     resolvedFrom?: unknown;
 }
+export interface DecoderState {
+    lastTimestamp?: number;
+    monitoringTimestamp?: number;
+}
 export declare function addEndian(littleEndian: boolean, bytes: number[]): number;
-export declare function readRecord(blob: Uint8Array, messageTypes: MessageTypeDefinition[], developerFields: any[], startIndex: number, options: FitParserOptions, startDate: number | undefined, pausedTime: number, dataView?: DataView): {
-    messageType: MesgNum | '';
+export declare function readRecord(blob: Uint8Array, messageTypes: MessageTypeDefinition[], developerFields: any[], startIndex: number, options: FitParserOptions, startDate: number | undefined, pausedTime: number, dataView?: DataView, decoderState?: DecoderState): {
+    messageType: MessageName | 'definition' | '';
     nextIndex: number;
     message?: any;
 };