@node-projects/speedscope 2.1.0 → 2.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,174 @@
1
+ # Import formats and background processing
2
+
3
+ The [README](../README.md#supported-file-formats) lists supported sources.
4
+ This guide describes additional fork-specific behavior and its limits.
5
+
6
+ ## Perf clock weights
7
+
8
+ Explicit `perf script` periods now determine aggregate weights. CPU/task-clock
9
+ periods are nanoseconds of sampled CPU time; off-CPU periods remain separate
10
+ sleep-time profiles, and cycles remain raw counts. Missing periods retain
11
+ explicitly labeled inference or sample counts. Timestamp gaps are not claimed
12
+ as measured execution. See [perf semantics and the public regression](PERF_IMPORT.md).
13
+
14
+ ## OpenTelemetry Profiles
15
+
16
+ Current OTLP Profiles JSON imports sampled stacks through a shared dictionary,
17
+ preserving metric weights, inline chains and thread/resource separation.
18
+ Timestamps do not become durations, and sample periods do not multiply values.
19
+ The importer currently parses the JSON in memory and creates aggregate profiles;
20
+ binary protobuf and older development layouts remain unsupported. See the
21
+ [pinned schema, limits and exact supported scope](OTEL_PROFILES.md).
22
+
23
+ ## Classic Android method traces
24
+
25
+ Open the classic concatenated key/data format emitted by Android method tracing:
26
+ a text key section followed by a little-endian `SLOW` binary section. Versions
27
+ 1, 2 and 3 are supported, including version-3 record padding. The file can be
28
+ named `.trace` or renamed; detection uses the format signature.
29
+
30
+ The importer preserves method signatures, thread identities and available Java
31
+ source locations. Clock metadata is required. `thread-cpu` and `wall` clocks
32
+ remain distinct; version-3 `dual` input creates separate labeled CPU and wall
33
+ profiles for each thread. Version-1 `global` means wall time. All values use
34
+ microseconds. CPU clocks are local to their thread; wall offsets remain relative
35
+ to the trace origin, so leading unobserved time can appear as idle weight.
36
+ The absolute date in the binary header is not added to those offsets.
37
+
38
+ Initial method exits reconstruct calls already active at trace start. Calls
39
+ still open at EOF close at that thread's last observed event. These are clipped
40
+ observation windows and do not invent work before or after the recording.
41
+
42
+ Fixed records are scanned without retaining a second parsed-record array.
43
+ Temporary state includes key dictionaries and per-thread stacks, alongside the
44
+ output builders. Key text is decoded line by line with a 16 MiB section limit
45
+ and 1 MiB line limit. The input buffer and final profiles still occupy memory.
46
+
47
+ Raw binary-only/streaming traces, newer ART packet formats, Perfetto protobuf,
48
+ Android systrace HTML and wrapped timestamp recovery are unsupported. Unknown
49
+ clocks, partial records, invalid method actions and decreasing per-thread clocks
50
+ produce explicit errors. See the
51
+ [synthetic fixture, generation script and AOSP references](../sample/profiles/android-method-trace/README.md)
52
+ for the exact supported layouts and clock policy.
53
+
54
+ ## Java thread-dump archives
55
+
56
+ Traditional HotSpot `jstack` / `Thread.print` text can be opened directly or as
57
+ an archive of captures from one JVM. Observed stacks contribute raw samples;
58
+ they do not measure elapsed time. The ZIP importer checks declared sizes and
59
+ actual expansion, feeds bounded UTF-8 lines into the parser and supports
60
+ cancellation without returning partial results.
61
+
62
+ See [text snapshot semantics](JAVA_THREAD_DUMPS.md) and the
63
+ [ZIP format and resource policy](JAVA_THREAD_DUMP_ZIP.md). Archive support is
64
+ limited to classic ZIP32 STORE/DEFLATE; it is not a generic archive extractor.
65
+
66
+ ## pprof metric and source identity
67
+
68
+ pprof's `default_sample_type` selects a sample metric by its string-table
69
+ identity. Protobuf integer objects are compared by value. If the preferred
70
+ metric is absent or cannot be found, the existing last-metric fallback applies.
71
+ For example, a recording can prefer allocation counts even when a later sample
72
+ column measures bytes; the displayed unit follows the chosen metric.
73
+
74
+ Function IDs and location IDs retain their complete unsigned 64-bit identity.
75
+ They are not truncated to their low 32 bits or rounded through a JavaScript
76
+ number, so IDs that share a low word remain distinct. String-table positions
77
+ must fit safe nonnegative integers. Source lines must fit safe positive integers.
78
+
79
+ Each location retains its own source line, falling back to the function's
80
+ definition line when needed. One location no longer overwrites the shared
81
+ function metadata used by another location. The current importer retains the
82
+ outermost entry of an inline-location chain; expanding all inlined functions
83
+ is outside this change.
84
+
85
+ Negative Go heap delta weights now import as separate negative-magnitude and
86
+ positive-contribution profiles. Compare positive minus negative in absolute mode
87
+ to inspect net changes; see [signed pprof semantics and limits](PPROF_IMPORT.md).
88
+ This does not add arbitrary-precision weights, a metric chooser or full inline
89
+ chains. See the [pprof schema used by the importer](../src/import/profile.proto).
90
+
91
+ ## Background import for large recordings
92
+
93
+ ### Streaming perf and folded stacks
94
+
95
+ UTF-8 folded stacks and `perf script` text now stream plain or gzip input into
96
+ the same incremental parsers used by ordinary imports. Supported suffixes are
97
+ `.folded`, `.collapsed`, `.stacks`, `.collapsedstack.txt`, `.perf`, `.perf.txt`
98
+ and `.linux-perf.txt`, optionally followed by `.gz`. Ordinary `.txt` files can
99
+ be detected as either format. Other names retain generic format detection.
100
+
101
+ The reader does not retain the complete expanded byte buffer, decoded text or
102
+ an array of all lines. It retains one bounded line and, for perf, one bounded
103
+ event. Input limits are 1 GiB expanded bytes, 1 Mi UTF-16 code units per line,
104
+ 4 Mi code units per perf event, 65,536 stack frames and two million samples.
105
+ A limit failure stops import; it does not retry through an unbounded reader.
106
+ Malformed weights and existing perf ordering checks remain enforced.
107
+
108
+ This path also runs for small compressed inputs and when workers are unavailable.
109
+ It yields during parsing, exposes progress and supports cancellation. Gzip needs
110
+ native `DecompressionStream`; unavailable native decompression, UTF-16 input and
111
+ other recognized formats retain the generic fallback. Final trees, sample lists,
112
+ symbol processing and worker snapshots still scale with the recording.
113
+
114
+ ### Worker routing and fallback
115
+
116
+ Inputs of at least **1 MiB** use a Web Worker when available, regardless of
117
+ filename extension. This includes perf, folded stacks, pprof, JFR, Android
118
+ method traces, supported ZIP archives and renamed captures. Format detection,
119
+ generic parsing, demangling and profile snapshot preparation happen away from
120
+ the main UI thread. Cancelling the task terminates its worker.
121
+
122
+ Recognized gzip perf/folded/text suffixes prefer a worker at any compressed
123
+ size: a tiny compressed file can expand into a large recording. Small plain
124
+ text stays on the main thread. The bounded text reader yields for cancellation
125
+ when a worker is blocked or unavailable; native gzip streaming requires
126
+ `DecompressionStream`. Unsupported encodings and unavailable streaming APIs use
127
+ the existing generic fallback, which can materialize the full input.
128
+
129
+ The worker first attempts the duration-trace and supported text streaming paths,
130
+ then uses the generic importer when those formats do not apply. Generic fallback
131
+ formats still read/materialize their input and retain their final model; routing them
132
+ through a worker does not make them streaming or out-of-core. Snapshot transfer,
133
+ geometry and final view construction still have memory costs proportional to
134
+ the recording. [Large-trace guidance](LARGE_TRACE_PERFORMANCE.md) describes the
135
+ duration-trace limits and measurements separately.
136
+
137
+ If workers are unavailable or blocked by a host's CSP, the app falls back to
138
+ the main-thread importer. DOM-dependent formats can also fall back after worker
139
+ detection. A transferred input buffer is returned before this fallback, so the
140
+ main thread does not attempt to parse an empty detached buffer. Synchronous
141
+ generic fallback parsing can block the UI until it returns; the streaming text
142
+ and asynchronous ZIP importers yield and check cancellation on the main thread.
143
+
144
+ Oversized supported ZIP files receive a four-byte signature preflight before
145
+ their complete body is read, both in the main importer and worker path. Other
146
+ formats retain their existing input/resource policies; the ZIP compressed-size
147
+ limit is not a blanket cap on every supported recording.
148
+
149
+ ### Text-stream measurements
150
+
151
+ Three fresh Node 24.19.0 ARM64 processes per mode imported a synthetic file
152
+ containing one million copies of `root;worker;leaf 1`: 19,000,000 plain bytes
153
+ or 46,140 gzip bytes. All modes produced one profile with total weight 1,000,000.
154
+ The final model is deliberately tiny, which isolates temporary input costs;
155
+ these figures do not predict memory for a million distinct stacks.
156
+
157
+ | Mode | Plain median time / peak RSS | Gzip median time / peak RSS |
158
+ | --- | ---: | ---: |
159
+ | Generic materializing importer | 466 ms / 110 MiB | 853 ms / 300 MiB |
160
+ | Streaming with worker-style scheduling | 573 ms / 86 MiB | 629 ms / 66 MiB |
161
+ | Streaming with main-thread cooperative yields | 993 ms / 85 MiB | 1,092 ms / 66 MiB |
162
+
163
+ Peak RSS columns are medians of each fresh process's peak, not retained model
164
+ heap. Worker-style measurements skip unnecessary timer yields, as the worker
165
+ can be terminated externally. They time the parsing function over a Node file
166
+ stream, excluding actual worker startup, demangling, snapshot transfer and view
167
+ construction. The cooperative path permits UI cancellation when no worker is
168
+ available; Windows timer overhead makes it slower in this test. Plain-text
169
+ streaming trades some throughput for lower memory even without those yields.
170
+
171
+ Reproduce with `node scripts/benchmark-profile.mjs <path> import`,
172
+ `text-stream-import` (worker-style scheduling), or `text-stream-main-import`
173
+ (cooperative scheduling). Do not compare these folded-stack figures with PLC
174
+ duration-trace timings as if they used the same input or parser.
@@ -3,7 +3,9 @@
3
3
  Import traditional HotSpot text output from `jstack PID` or
4
4
  `jcmd PID Thread.print` by opening a `.jstack` file or a text file containing
5
5
  the `Full thread dump …` header. Concatenate captures from the same JVM to
6
- compare observed stacks across repeated snapshots. Extract ZIP archives first.
6
+ compare observed stacks across repeated snapshots. You can also open a ZIP of
7
+ traditional text captures from the same JVM; see [archive limits and supported
8
+ ZIP formats](JAVA_THREAD_DUMP_ZIP.md).
7
9
 
8
10
  Each observed Java stack contributes one raw sample. The importer creates an
9
11
  aggregate profile per thread name and Java thread ID, falling back to `tid`
@@ -23,8 +25,11 @@ The parser iterates lines and retains the current stack and current snapshot's
23
25
  thread identities, rather than collecting every line or observation. The text
24
26
  input and resulting profile model still occupy memory; this format does not use
25
27
  the duration-trace streaming importer. Prefer captures from one JVM per file.
26
- ZIP input, structured virtual-thread JSON, native address stacks and lock/state
27
- visualization are outside the supported format.
28
+ The ZIP importer instead feeds bounded UTF-8 lines into the same parser and
29
+ checks both declared and actual decompressed sizes. Structured virtual-thread
30
+ JSON, native address stacks and lock/state visualization remain outside the
31
+ supported format. Unsupported ZIP variants can be extracted before importing
32
+ their text captures.
28
33
 
29
34
  The checked-in [synthetic example](../sample/profiles/java-thread-dump/synthetic.jstack)
30
35
  contains repeated snapshots and source locations. Format references are
@@ -0,0 +1,65 @@
1
+ # Java thread-dump ZIP archives
2
+
3
+ Open a ZIP containing traditional HotSpot `jstack` / `jcmd Thread.print` text
4
+ captures from **one JVM**. The same thread name and Java thread ID identify a
5
+ thread across snapshots; renamed threads remain separate. Each observed Java
6
+ stack contributes one raw sample, not measured elapsed time. Missing threads
7
+ and entries without Java stacks contribute no samples.
8
+
9
+ Entries are processed in deterministic, case-sensitive filename order, not ZIP
10
+ insertion order or file modification time. For example, `1.jstack`, `10.jstack`,
11
+ then `2.jstack` is the lexical order. The result is aggregate, so this ordering
12
+ does not claim a timeline or attach elapsed durations to snapshot intervals.
13
+ Every `.jstack` entry must contain its own `Full thread dump` header near the
14
+ start. Each entry boundary closes the previous snapshot.
15
+
16
+ The importer recognizes `.jstack`, `.txt`, `.log`, `.dump` and `.tdump` entries
17
+ with a `Full thread dump` header within the first 64 KiB. Other regular files
18
+ and ordinary notes are ignored as profile input, but still undergo bounded
19
+ decompression and checksum validation. Directories are skipped. Text is strict
20
+ UTF-8; UTF-16 dumps should be converted or imported as separate text files.
21
+
22
+ ## Resource policy
23
+
24
+ | Resource | Maximum |
25
+ | --- | ---: |
26
+ | Compressed archive | 32 MiB |
27
+ | Archive entries, including directories | 512 |
28
+ | Decompressed bytes per entry | 16 MiB |
29
+ | Total decompressed bytes, including ignored files | 64 MiB |
30
+ | UTF-8 filename bytes | 512 |
31
+ | Path depth | 8 components |
32
+ | Decoded line length | 65,536 UTF-16 code units |
33
+
34
+ Declared sizes are checked before decompression. Actual expanded bytes are
35
+ checked on every output chunk before decoding or parsing, so lying ZIP headers
36
+ cannot bypass the limits. The inflater emits chunks of at most 16 KiB; a limit
37
+ violation can allocate that one output chunk before it is rejected. It never
38
+ accumulates the whole expanded entry. The line decoder retains at most one
39
+ bounded line, while the parser retains the current stack, current snapshot's
40
+ thread identities and the resulting profile builders. The compressed input and
41
+ final profile model still occupy memory; this is not an out-of-core viewer.
42
+ The importer yields during work and supports cancellation without returning a
43
+ partially imported group.
44
+
45
+ ## Supported ZIP subset
46
+
47
+ Single-disk ZIP32 archives using **STORE** or **DEFLATE** are supported, including
48
+ ordinary data descriptors. Local/central headers, entry extents, decompressed
49
+ sizes and CRC32 checksums must agree. Paths are validated without extracting
50
+ anything to disk. Absolute/traversal paths, duplicate or case-colliding names,
51
+ file/directory conflicts, symlinks, special files and non-normalized Unicode
52
+ paths are rejected. Names must be ASCII or explicitly UTF-8.
53
+
54
+ Encrypted archives, ZIP64, split archives, self-extracting prefixes, archive
55
+ padding, alternate Unicode-path extra fields and other compression methods are
56
+ unsupported. Extract an unsupported archive yourself and open its text captures
57
+ individually or concatenate them into one `.jstack` file.
58
+
59
+ The checked-in [synthetic archive](../sample/profiles/java-thread-dump/synthetic.zip)
60
+ contains two short snapshots of the same thread. This is an independently
61
+ implemented ZIP32 reader based on
62
+ [PKWARE's ZIP specification](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT),
63
+ using the existing
64
+ [pako incremental inflater](https://github.com/nodeca/pako/blob/1.0.6/lib/inflate.js).
65
+ It does not use JSZip's whole-entry `async('string')` allocation path.
@@ -1,5 +1,77 @@
1
1
  # Large trace performance follow-up
2
2
 
3
+ ## Long-timeline GPU precision (2026-11-21)
4
+
5
+ The renderer previously uploaded absolute time coordinates as float32. Near
6
+ 6,000 seconds in a microsecond recording, endpoints 100 microseconds apart could
7
+ round to the same GPU value, making a call disappear at high zoom. A synthetic
8
+ [late-call recording](../sample/profiles/speedscope/long-late-calls.speedscope.json)
9
+ reproduces that failure, including early and late rectangles in one wide batch.
10
+
11
+ Rectangles are now clipped against the current atlas/source tile using CPU
12
+ doubles, then normalized before upload. The shader transforms unit coordinates
13
+ to the destination. Profile times, range-tree bounds, atlas keys, hit testing,
14
+ frame colors and measured durations retain their existing identities and values.
15
+ Each batch keeps one buffer and one full source-rectangle cache key; a shared
16
+ staging allocation is capped at 1,200,000 bytes for 10,000 rectangles. Switching
17
+ an uncached tile can require another bounded CPU normalization/upload; cached
18
+ tiles and unchanged source rectangles reuse their data. Final profile and GPU
19
+ buffer storage still scale with the recording.
20
+
21
+ Five regressions check real uploaded float32 coordinates and the renderer's
22
+ uniform transform, adjacent late calls, negative/spanning clipping, source
23
+ origin/extent changes, shared staging ownership, buffer lifecycle and limits.
24
+ Browser QA showed both 100-microsecond calls near 6,000 seconds, including after
25
+ overview reset/refocus. The private PLC imported four profiles and rendered
26
+ Time Order, Left Heavy and Sandwich's inverted callers/callees correctly during
27
+ pan/zoom checks, with no browser errors.
28
+
29
+ A warmed Node smoke using a preallocated-copy mock upload and source rectangles
30
+ covering almost all 10,000 rectangles measured about 0.19 ms per repack and
31
+ 2.6 ms for 100,000 unchanged-source requests, with one buffer allocation. These are local CPU
32
+ measurements, not browser/GPU frame-rate claims. This addresses the demonstrated
33
+ float32 failure associated with [#467](https://github.com/jlfwong/speedscope/issues/467);
34
+ the original upstream capture remains unavailable for direct comparison. These
35
+ buffer/matrix tests also extend [#125](https://github.com/jlfwong/speedscope/issues/125),
36
+ but do not replace a browser/GPU pixel-regression harness.
37
+
38
+ Final local verification passed all 418 Jest tests in 72 suites with 307
39
+ snapshots on checksum-verified Node 22.23.5 ARM64 using a clean Jest cache.
40
+ The five Node tests, type checking, lint, schema and backlog checks passed
41
+ separately on the system Node runtime. Two full Jest runs on local Node 24.19.0
42
+ ARM64 exited with native access violations without an assertion failure, so a
43
+ green local Node 24 full-suite result is not claimed. GitHub's Windows/Linux
44
+ Node 22/24 matrix validates the branch independently before merge.
45
+
46
+ ## Fixed-height Sandwich list geometry (2026-11-21)
47
+
48
+ Sandwich already rendered a viewport window. It now calculates that window and
49
+ initial scroll offset directly from row count/height, removing the size object
50
+ per frame and full-list geometry scans. A single cached selected-frame/index
51
+ replaces the frame-index Map: repeated, adjacent and wrap-around navigation are
52
+ constant time; unrelated new selections scan the existing sorted array once.
53
+ Missing results are cached. Element resizing updates the viewport and filtering
54
+ clamps stale scroll offsets, even without a browser-generated scroll event.
55
+
56
+ Tests cover 100,000-row selection/navigation, billion-row arithmetic without a
57
+ row allocation, visible-row bounds, filter shrink, resize and cleanup. The
58
+ billion-row case tests arithmetic only: browser scroll-height limits still apply.
59
+ The sorted row array, model, sorting and graph/color geometry remain proportional
60
+ to the recording. These changes reduce auxiliary storage and repeated work;
61
+ they do not establish a fully bounded-memory viewer.
62
+
63
+ Combined validation passed 410 Jest tests in 71 suites (307 snapshots), five
64
+ Node tests, type checking, lint, schema/backlog checks and both builds. The
65
+ private PLC gzip still exported four profiles, 17,510 frames and 1,039,656 call
66
+ events with the established SHA-256
67
+ `522de77afbfc4dd94f1f30c3a9551979d12db506305ffef999db9c910ae8eddb`.
68
+ The local Node streaming smoke took about 4.3 seconds to import; this is not a
69
+ browser performance guarantee or a claim of faster importing in this batch.
70
+ In browser QA the first PLC profile's 16,231-frame table rendered 40 rows at the
71
+ scroll end, clamped to zero when filtered to one row, and updated to 49 rendered
72
+ rows after a viewport resize. A generated local function link restored its
73
+ selection after reopening the same PLC recording in another tab.
74
+
3
75
  Baseline: `eb93425`, reviewed 2026-11-18. These changes continue the open-issue
4
76
  review, prioritizing large profiles and the export request in issue #89.
5
77
 
@@ -0,0 +1,117 @@
1
+ # OpenTelemetry Profiles JSON
2
+
3
+ The importer accepts the lower-camel-case JSON representation of current
4
+ `opentelemetry.proto.profiles.v1development.ProfilesData` and
5
+ `collector.profiles.v1development.ExportProfilesServiceRequest`. Both have
6
+ `resourceProfiles` and one shared `dictionary`. This is **sampled stack data**,
7
+ not OpenTelemetry distributed traces (`resourceSpans`).
8
+
9
+ The supported schema is pinned to OpenTelemetry Proto commit
10
+ [`d31f7acf9d461fd0601077de516ac1eb7051422f`](https://github.com/open-telemetry/opentelemetry-proto/blob/d31f7acf9d461fd0601077de516ac1eb7051422f/opentelemetry/proto/profiles/v1development/profiles.proto),
11
+ OTLP Profiles **development version 1**. This development schema is unstable.
12
+ Older per-profile dictionaries, `ProfileContainer`, singular `sample`, repeated
13
+ `sampleType`, and future incompatible schemas are unsupported.
14
+
15
+ Development version normally belongs to OTLP transport metadata, not JSON.
16
+ The transport specification defaults absent metadata to version 1. A standalone
17
+ file can optionally advertise `developmentVersion: 1`; other advertised values
18
+ are rejected. This annotation is an importer extension, not a standard schema
19
+ field. Unversioned files must match the documented current dictionary layout;
20
+ the importer cannot identify future incompatible changes that retain that shape.
21
+
22
+ ## Dictionary and stacks
23
+
24
+ All seven tables must be present: `stringTable`, `mappingTable`, `locationTable`,
25
+ `functionTable`, `linkTable`, `attributeTable`, and `stackTable`. The official
26
+ schema requires a default-value entry at index zero even in unused tables.
27
+ Default fields inside these entries can be omitted. Link-zero trace/span IDs
28
+ may also be all-zero hexadecimal IDs. References are nonnegative in-range
29
+ int32 indices; zero represents an unset reference.
30
+
31
+ `stackTable[].locationIndices` and inline `locationTable[].lines` are leaf-first;
32
+ the viewer reverses both to retain the full caller/inline chain. Function names
33
+ fall back to system names. File, line and column identify symbolized frames.
34
+ Unsymbolized locations retain their binary filename and exact uint64 hexadecimal
35
+ address. Equal frame values merge independently of dictionary index.
36
+
37
+ Profile/sample `attributeIndices` resolve through `attributeTable`.
38
+ Resource/scope attributes accept direct keys or `keyStrindex`, and values accept
39
+ direct strings, `stringValueStrindex`, exact int64 integers and booleans. Complex
40
+ attributes are ignored for grouping. Duplicate attribute keys are rejected.
41
+ Sample `thread.id`/`thread.name` override profile attributes and create separate
42
+ thread profiles. Integer thread IDs remain exact decimal strings. Input profile
43
+ objects, resources and scopes remain separate even when names or IDs match;
44
+ labels include the service (or executable), scope, metric, profile ordinal and
45
+ thread. Generic attributes, scope versions, profile IDs and span links are not
46
+ preserved in viewer/export metadata. Links never become stack frames.
47
+
48
+ ## Values, units and time
49
+
50
+ Each sample needs `values`, `timestampsUnixNano`, or both. Both populated arrays
51
+ must have equal length. Values sum directly; timestamp-only observations each
52
+ contribute **one in the declared sample unit**, exactly as specified by OTLP.
53
+ `period` and `periodType` are informational and never multiply weights.
54
+
55
+ Supported units are `nanoseconds`/`ns`, `microseconds`/`us`/`µs`/`μs`,
56
+ `milliseconds`/`ms`, `seconds`/`s`, `bytes`/`By`, and raw counts
57
+ (`count`, `1`, `{count}`, or empty unit). Unknown units are rejected. Values are
58
+ not rescaled when selecting the formatter. Negative differential weights and
59
+ weights/totals exceeding JavaScript's exact integer range are rejected.
60
+
61
+ Profiles are imported as **aggregate**, with measured stack weights. Epoch
62
+ timestamps do not become a duration axis, and gaps do not generate idle stacks.
63
+ Timestamps and addresses use exact uint64 parsing; decimal strings are required
64
+ for values beyond JavaScript's safe integer range. Timestamp validation uses
65
+ the exact epoch. A nonzero profile duration requires samples within the declared
66
+ half-open time range; zero-duration instant profiles skip this range check.
67
+ Overflowing time ranges are rejected. Timestamps and period metadata are not
68
+ retained in exports.
69
+
70
+ ## Limits and remaining support
71
+
72
+ The JSON document is currently parsed in memory; this is not a streaming JSON or
73
+ protobuf importer. Import processing avoids an expanded observation/record array
74
+ and does not retain flattened copies of every dictionary stack. Memory consists
75
+ of the parsed JSON, output profiles, per-thread builder state and one stack.
76
+ Main-thread processing yields every 4,096 counted operations and checks
77
+ cancellation. The dedicated import worker explicitly disables these timer
78
+ yields, because cancelling the worker terminates its execution externally.
79
+ The worker keeps the same limits and weight checks. Main-thread fallback keeps
80
+ cooperative yielding, including when no cancellation signal was supplied; the
81
+ optimization never infers worker execution from the presence of a signal.
82
+
83
+ Limits are one million dictionary entries total; one million elements per input
84
+ array; 100,000 expanded frames per stack; 4,096 attributes per set; 4,096
85
+ output profiles; 4,096 characters per grouping attribute value; and ten million
86
+ counted hierarchy/sample/observation/location/inline operations. Limits fail
87
+ explicitly rather than silently dropping samples. The parsed JSON and output
88
+ itself still consume memory proportional to their size.
89
+
90
+ Binary OTLP protobuf, old development layouts, original payload decoding,
91
+ automatic network collection, and chronological OTLP visualization remain
92
+ unsupported. Binary protobuf has no unique identifying magic; the importer does
93
+ not guess it ahead of existing pprof detection. The synthetic fixture and its
94
+ provenance are in `sample/profiles/otel-profiles`.
95
+
96
+ ## Worker import performance check
97
+
98
+ On Windows on 2026-11-21, two fresh Node processes per case imported generated
99
+ current-schema JSON based on `synthetic.json`: its samples were replaced with
100
+ 100,000 or 1,000,000 distinct `{stackIndex: 1, values: [String(i % 3 + 1)]}`
101
+ objects. This exercises ordinary JSON detection and parsing, not a pre-parsed
102
+ importer call. The benchmark invoked `processTraceInWorker` directly, including
103
+ snapshot construction; it excludes browser worker startup and message transfer.
104
+ The before case restored timer yielding in the worker options only.
105
+
106
+ | Samples | JSON size | Main thread | Worker before | Worker without timer yields |
107
+ | --------- | --------- | ------------- | ------------- | --------------------------- |
108
+ | 100,000 | 3.05 MiB | 1.82–1.91 s | 1.94 s | 0.159–0.170 s |
109
+ | 1,000,000 | 30.52 MiB | 19.00–19.46 s | 19.51–19.75 s | 1.396–1.399 s |
110
+
111
+ The worker removes 122 and 1,220 timer waits respectively. Totals remained
112
+ 199,999 and 1,999,999 nanoseconds. Main-thread yielding remains enabled.
113
+ This is a timing optimization, not a new memory reduction: process high-water
114
+ RSS for the million-sample runs was 359–382 MiB before and 392–402 MiB after,
115
+ including fixture generation and all Node runtime allocations. Post-import
116
+ retained heap was comparable. There is no additional observation array or stack
117
+ cache; whole-document JSON parsing remains the dominant memory limitation.
@@ -0,0 +1,66 @@
1
+ # Linux perf imports and weight semantics
2
+
3
+ Import text from `perf script`. Include `comm`, thread/process identity, `time`,
4
+ `event`, `period`, `ip`, `sym` and `dso` fields when available. Regular callchains
5
+ and leaf-only `--hide-call-graph` output are supported. Keeping `period` is
6
+ important: timestamps alone do not measure a sampled function's CPU consumption.
7
+
8
+ ## Periods determine the weights
9
+
10
+ | Recorded event | Period present | Period absent |
11
+ | --- | --- | --- |
12
+ | `cpu-clock`, `task-clock` | Aggregate sampled CPU time, in nanoseconds | Legacy inferred timestamp timeline, explicitly labeled |
13
+ | `offcpu-time` | Aggregate sampled off-CPU time, in nanoseconds | Sample count, explicitly labeled |
14
+ | `cycles`, including hybrid CPU PMUs | Aggregate raw cycle periods | Legacy inferred timestamp timeline, explicitly labeled |
15
+ | Other hardware/software counters | Aggregate raw event periods | Sample count, explicitly labeled |
16
+
17
+ Event modifiers such as `:u` retain these rules. Hybrid `cpu_core/cycles/u` and
18
+ `cpu_atom/cycles/u` samples are combined under the normalized `cycles:u` event.
19
+ Different events, processes and threads remain separate. If some records have
20
+ periods while others omit them, their profiles remain separate too, so inferred
21
+ seconds, nanosecond periods and counts are never added together.
22
+
23
+ Period-weighted profiles use Left Heavy, Sandwich or Call Tree. Time Order is
24
+ disabled because accumulating sampled weights does not establish exact wall-time
25
+ intervals. Missing-period clock/cycle profiles retain the older timestamp
26
+ interpolation and are labeled **inferred timeline, period missing**. That view
27
+ assigns gaps between timestamps to adjacent stacks; do not interpret it as
28
+ measured CPU time or proof that a thread was running throughout those gaps.
29
+
30
+ Periods must be finite, non-negative integers within JavaScript's exact range
31
+ (`0` through `9007199254740991`). Larger individual periods fail with an
32
+ actionable error rather than being silently rounded. Totals still use the
33
+ project's ordinary floating-point weight model. Zero periods contribute zero
34
+ weight; they do not trigger timestamp inference or become a one-unit sample.
35
+
36
+ ## Why gaps are not CPU time
37
+
38
+ The public reproduction in [upstream #393](https://github.com/jlfwong/speedscope/issues/393#issuecomment-1200547074)
39
+ contains 13 samples for `DiscoveryClient` thread `2614953`, each with a
40
+ `cpu-clock` period of `20408163` nanoseconds. Its sampled CPU weight is therefore
41
+ `265306119` ns, about **0.265 seconds**, while its timestamps span about
42
+ **30.231 seconds**. The importer now uses the periods instead of attributing
43
+ those gaps to functions. A single clock sample also retains its real period
44
+ instead of becoming an arbitrary one-unit sample.
45
+
46
+ The checked-in [regression fixture](../sample/profiles/linux-perf/idle-clock-period.linux-perf.txt)
47
+ uses those timestamp/period facts with authored synthetic stacks. The source
48
+ archive was [profile.out.gz](https://github.com/jlfwong/speedscope/files/9230041/profile.out.gz),
49
+ 22,149 bytes, SHA-256
50
+ `40b9123046dc918018f40030426851d15622077843a8cd2923371750a6fa5efa`.
51
+
52
+ This fixes sampled aggregate attribution. Exact idle-gap placement and aligned
53
+ multithread timelines remain separate work: clock periods do not say exactly
54
+ when each fragment of CPU execution occurred within a timestamp gap. An
55
+ `offcpu-time` profile is kept separate and never added to CPU execution weights.
56
+ Scheduler-switch events and their associations need their own tested timeline
57
+ mapping; the importer does not invent those intervals.
58
+
59
+ ## Format references
60
+
61
+ Linux's [software-clock sampling implementation](https://github.com/torvalds/linux/blob/master/kernel/events/core.c)
62
+ initializes clock periods in nanoseconds and arms timers with
63
+ `ns_to_ktime(period)`. The kernel's [perf record documentation](https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/perf-record.txt)
64
+ specifies that `--off-cpu` produces `offcpu-time` samples whose periods are sleep
65
+ time in nanoseconds. Other event periods remain counts in that event's own
66
+ units; cycle counts are not converted to seconds.
@@ -0,0 +1,50 @@
1
+ # pprof and signed heap deltas
2
+
3
+ Import binary protobuf pprof, including its usual gzip wrapper. The selected
4
+ metric is `default_sample_type`, or the last sample type when no usable default
5
+ is present. Positive-only recordings retain the ordinary single aggregate
6
+ profile. Time Order is unavailable: protobuf samples do not establish call order.
7
+
8
+ Go's `/debug/pprof/heap?seconds=N` can return a delta recording containing
9
+ negative values. Those values now import as two explicitly named profiles:
10
+
11
+ - **Negative contributions (magnitude; subtract)** contains the absolute values
12
+ of negative samples.
13
+ - **Positive contributions** contains positive samples.
14
+
15
+ These are signed contributions, not historical before/after heap snapshots.
16
+ Choose **Profile actions → Compare profiles…**, with negative contributions as
17
+ Baseline and positive contributions as Candidate, in **Absolute weights** mode.
18
+ Candidate minus baseline gives the net signed change for each call path. The
19
+ same-path contributions remain separate until comparison; cancellation to zero
20
+ does not discard either side. The graph's union widths represent gross
21
+ contribution magnitudes, not elapsed time or live-heap size. Normalized mode
22
+ instead compares relative composition and should not be read as net byte change.
23
+
24
+ For example, samples of -9, +12 and -2 bytes on one stack become 11 negative and
25
+ 12 positive bytes, yielding +1 net byte. All-negative recordings have an empty
26
+ positive side and remain comparable. Zero-valued samples have no width. Export
27
+ preserves both magnitude profiles in ordinary speedscope JSON; their names
28
+ describe the projection. Export does not recreate the original signed protobuf.
29
+
30
+ Selected sample values and the total on each side must fit exact JavaScript
31
+ integers (`Number.MAX_SAFE_INTEGER`). Larger values fail explicitly rather than
32
+ rounding or dropping negative values. This is not arbitrary-precision int64
33
+ support. Existing pprof limitations remain: only the selected metric is imported,
34
+ only the outermost inline location frame is retained, labels and mapping/address
35
+ metadata are not fully represented, and parsed input plus output trees scale
36
+ with capture size. Generic gzip decompression is not bounded by the text-stream
37
+ limits. This change does not make pprof streaming or out of core.
38
+
39
+ A local Go 1.24.1 `net/http/pprof` heap-delta check allocated then released 16 MiB
40
+ between collections. It imported 16,777,600 negative and 78,112 positive bytes,
41
+ giving -16,699,488 bytes net; browser comparison displayed -15.93 MiB. Allocation
42
+ overhead varies between runs. The ignored capture is an authored regression
43
+ observation, not the unavailable original upstream recording. Deterministic
44
+ checked-in tests cover both signs, same-path cancellation, all-negative values,
45
+ unit preservation, exact-range failures and export round trips.
46
+
47
+ - [Go HTTP pprof parameters](https://pkg.go.dev/net/http/pprof#hdr-Parameters)
48
+ define `seconds=N` delta recording behavior.
49
+ - [pprof protobuf schema](https://github.com/google/pprof/blob/main/proto/profile.proto)
50
+ defines sample type selection and signed int64 sample values.