profiler-md 0.8.1 → 0.9.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/api.md CHANGED
@@ -46,6 +46,20 @@ console.log(
46
46
  )
47
47
  ```
48
48
 
49
+ The same object takes a `name` for the data's source. The message of a failure
50
+ the input causes begins with it, so a diff states which of its two inputs
51
+ failed:
52
+
53
+ ```js
54
+ // Error: base.pprof: pprof: invalid protobuf encoding: …
55
+ console.log(
56
+ await diffProfilesAsync(
57
+ { data: await openAsBlob(`base.pprof`), name: `base.pprof` },
58
+ { data: await openAsBlob(`current.pprof`), name: `current.pprof` },
59
+ ),
60
+ )
61
+ ```
62
+
49
63
  ## Diffing
50
64
 
51
65
  Diff two profiles or two heap snapshots:
@@ -100,16 +114,15 @@ const options = {
100
114
  // break down every category.
101
115
  minCategoryShare: 0.05,
102
116
  // Make paths relative to a custom base URL or directory, or pass `auto` to
103
- // infer the profiled files' common ancestor directory.
117
+ // infer one from the locations of the program's own code.
104
118
  baseURL: `/path/to/project`,
105
- matchEntry: (entry, context) => {
106
- if (entry.location?.url?.pathname.includes(`/bundle.`)) {
107
- // Match bundled entries when diffing by name only, ignoring
108
- // content-hashed filenames.
109
- return { name: entry.name }
110
- }
111
- return defaultMatchEntry(entry, context)
112
- },
119
+ // Receive the conversion's diagnostics: at `warn`, an option or input that
120
+ // had no effect; at `info`, each decision made on your behalf; and at
121
+ // `debug`, the reasoning behind each decision. Only defined methods are
122
+ // called, so `console` qualifies. Set `logLevel`, which defaults to `none`,
123
+ // to receive any message.
124
+ logger: console,
125
+ logLevel: `info`,
113
126
  categorizeFunctions: (entries, context) => {
114
127
  const categories = defaultCategorizeFunctions(entries, context)
115
128
  return entries.map((entry, index) =>
@@ -127,12 +140,39 @@ const options = {
127
140
  console.log(await profileToMdAsync(await openAsBlob(`example.pprof`), options))
128
141
 
129
142
  // The same options apply to diffs. `matchEntry` takes effect only here, where
130
- // it controls which entries count as the same across the two sides.
143
+ // it controls which entries count as the same across the two sides, and a
144
+ // conversion of one profile warns that it was ignored.
131
145
  console.log(
132
146
  await diffProfilesAsync(
133
147
  await openAsBlob(`base.pprof`),
134
148
  await openAsBlob(`current.pprof`),
135
- options,
149
+ {
150
+ ...options,
151
+ matchEntry: (entry, context) => {
152
+ if (entry.location?.url?.pathname.includes(`/bundle.`)) {
153
+ // Match bundled entries when diffing by name only, ignoring
154
+ // content-hashed filenames.
155
+ return { name: entry.name }
156
+ }
157
+ return defaultMatchEntry(entry, context)
158
+ },
159
+ },
136
160
  ),
137
161
  )
138
162
  ```
163
+
164
+ ## Compressed inputs
165
+
166
+ The API decompresses an input before converting it, so a profile converts as its
167
+ profiler wrote it: a gzipped `.pb.gz` pprof profile, an LZ4 memray capture, or a
168
+ profile a user compressed.
169
+
170
+ The available codecs depend on the runtime:
171
+
172
+ | Runtime | Sync API | Async API |
173
+ | ----------- | ----------------- | --------------------------------------------------- |
174
+ | Node.js | gzip, brotli, LZ4 | gzip, brotli, LZ4 |
175
+ | Other (web) | LZ4 | gzip, LZ4, and brotli where the runtime supports it |
176
+
177
+ When you pass a format, pass a brotli input as a `Uint8Array`, an array of
178
+ chunks, or a `Blob`. A generator or `ReadableStream` of brotli fails to convert.
@@ -1,10 +1,12 @@
1
1
  # GHC eventlog
2
2
 
3
- The binary event stream
3
+ The GHC eventlog is the binary event stream
4
4
  [GHC](https://downloads.haskell.org/ghc/latest/docs/users_guide/eventlog-formats.html)'s
5
- runtime writes for a program run with `+RTS -l`, recording what the runtime did
6
- and when: garbage collections and thread scheduling. A program compiled with
7
- `-prof` and run with `+RTS -p` adds one cost-centre sample per tick.
5
+ runtime writes for a program run with `+RTS -l`, recording the event classes
6
+ `-l` selects, such as garbage collections and thread scheduling. A program
7
+ compiled with `-prof` and run with `+RTS -p` adds one cost-centre sample per
8
+ tick. `+RTS -l-au` records those samples without the garbage collection and
9
+ scheduling events.
8
10
 
9
11
  A log opens with a header declaring every event type the runtime can write, each
10
12
  with its payload size. The events follow, in the order they happened: the
@@ -18,6 +18,6 @@ the built-in `IDLE` cost centre. The runtime counts bytes and entries exactly
18
18
  and independently, so the allocation profile reports both as measured quantities
19
19
  and states no rate.
20
20
 
21
- It supports multiple metrics per profile and a single report per file.
21
+ It supports multiple metrics per profile and multiple profiles per file.
22
22
 
23
23
  Files use the `.prof` extension, named after the program as `<program>.prof`.
@@ -15,7 +15,6 @@ heap holds rather than where allocation happens. Sizes are the bytes the dump
15
15
  records for each object. The dump omits object headers, so a class's instances
16
16
  measure smaller here than the memory they occupy.
17
17
 
18
- Android's ART writes a dialect of the format with records of its own, which this
19
- tool doesn't read.
18
+ It supports a single heap snapshot per file.
20
19
 
21
20
  Files use the `.hprof` extension.
@@ -6,7 +6,7 @@ binary profiling format, written by the JDK's built-in
6
6
  [async-profiler](https://github.com/async-profiler/async-profiler).
7
7
 
8
8
  A recording splits into self-contained chunks of LEB128 varint-encoded events.
9
- One recording can mix event kinds, so it converts to one profile per kind:
9
+ One recording can mix event types, so it converts to one profile per type:
10
10
  CPU/wall sampling measured by sample count, heap and native memory allocation
11
11
  sampling measured by allocated bytes, live-object sampling measured by retained
12
12
  bytes, and lock contention measured by blocked time.
@@ -2,17 +2,18 @@
2
2
 
3
3
  The
4
4
  [JSC heap snapshot](https://webkit.org/blog/6425/memory-debugging-with-web-inspector)
5
- is a JSON format produced by JavaScriptCore, the JavaScript engine of Safari and
5
+ is a JSON format written by JavaScriptCore, the JavaScript engine of Safari and
6
6
  WebKit, that captures the full live object graph at a point in time.
7
7
 
8
- A snapshot is a flat integer array of node records (an id, a self size, a class
9
- name, and flags marking VM-internal instances) and a flat integer array of edge
10
- records (the two nodes it connects, a type, and a name), alongside the string
11
- tables the class and edge names index into.
8
+ A heap snapshot is a flat integer array of node records (an id, a self size, a
9
+ class name, and flags marking VM-internal instances) and a flat integer array of
10
+ edge records (the two nodes it connects, a type, and a name), alongside the
11
+ string tables the class and edge names index into.
12
12
 
13
- Snapshots show what objects exist in memory, unlike heap profiles, which sample
14
- allocations. That makes them useful for diagnosing memory leaks rather than
15
- allocation hotspots.
13
+ It converts to a heap snapshot, which shows what the heap holds rather than
14
+ where allocation happens.
15
+
16
+ It supports a single heap snapshot per file.
16
17
 
17
18
  Files use the `.json` extension and contain `"type": "Inspector"` and a
18
19
  `"version"`: `3` in current WebKit and in Bun's `generateHeapSnapshot("jsc")`,
@@ -1,7 +1,7 @@
1
1
  # memray
2
2
 
3
- The capture file [memray](https://github.com/bloomberg/memray), Bloomberg's
4
- Python memory profiler, writes for its own reporters to read.
3
+ A memray capture is the file that [memray](https://github.com/bloomberg/memray),
4
+ Bloomberg's Python memory profiler, writes for its own reporters to read.
5
5
 
6
6
  A capture is a binary stream of records describing a program's allocations: the
7
7
  code objects its functions belong to, each thread's frame pushes and pops, and
@@ -10,18 +10,13 @@ capture written with `--aggregate` states each stack's peak and leaked totals
10
10
  instead of every allocation. `memray run` compresses either one with LZ4, which
11
11
  this tool decompresses itself.
12
12
 
13
- A capture contains no totals, so both measures come from replaying the stream:
14
- peak memory from the allocations live at the moment total memory was highest,
15
- and leaked memory from those still live at the end. `memray flamegraph` and
16
- `memray flamegraph --leaks` report those measures. The measures count different
17
- allocations, so a capture converts to a profile each. Because memray records
18
- every allocation, the output states bytes per allocation rather than a sampling
19
- rate.
20
-
21
- This tool skips native frames, recorded by `memray run --native`, because
22
- resolving them means symbolizing instruction pointers against the binaries the
23
- traced process ran. A capture recorded with them still converts, showing its
24
- Python frames.
13
+ A capture written without `--aggregate` contains no totals, so both measures
14
+ come from replaying the stream: peak memory from the allocations live at the
15
+ moment total memory was highest, and leaked memory from those still live at the
16
+ end. `memray flamegraph` and `memray flamegraph --leaks` report those measures.
17
+ The measures count different allocations, so a capture converts to a profile
18
+ each. Because memray records every allocation, the output states its rate as
19
+ bytes per allocation rather than per sample.
25
20
 
26
21
  It supports a single byte metric per profile and multiple profiles per file.
27
22
 
@@ -1,7 +1,8 @@
1
1
  # perf.data
2
2
 
3
- The binary file [Linux `perf`](https://perfwiki.github.io/main/) writes: the
4
- on-disk form of the kernel's own `perf_event` records, specified by the kernel's
3
+ `perf.data` is the binary file [Linux `perf`](https://perfwiki.github.io/main/)
4
+ writes. It is the on-disk form of the kernel's `perf_event` records, specified
5
+ by the kernel's
5
6
  [`perf.data` file format documentation](https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/perf.data-file-format.txt)
6
7
  and
7
8
  [`perf_event.h`](https://github.com/torvalds/linux/blob/master/include/uapi/linux/perf_event.h).
@@ -15,9 +16,9 @@ a call chain, a thread, and how many event units the sample stands for, plus the
15
16
  memory mappings each process made), and feature sections hold the metadata
16
17
  `perf` collected about the machine.
17
18
 
18
- Each recorded event becomes its own profile, measured in the units its period
19
- counts: nanoseconds for the `cpu-clock` and `task-clock` software events, and
20
- the counted event otherwise (cycles, instructions, page faults).
19
+ Each recorded event converts to its own profile, measured in the units its
20
+ period counts: nanoseconds for the `cpu-clock` and `task-clock` software events,
21
+ and the counted event otherwise (cycles, instructions, page faults).
21
22
 
22
23
  It supports a single metric per profile and multiple profiles per file.
23
24
 
@@ -26,13 +27,14 @@ Files use the `perf.data` name and extension, optionally gzipped as
26
27
 
27
28
  ## Addresses instead of function names
28
29
 
29
- A `perf.data` file holds no symbol tables. `perf report` reads the recorded
30
- program and its shared libraries off the machine that recorded them, so a file
31
- alone resolves an address no further than the mapped file it fell in. Every
32
- frame is therefore one address, named by its offset into that file and located
33
- at the file's path, the same form `perf script` prints for a frame it cannot
34
- symbolize. An address in memory a program obtained without a file (an `//anon`
35
- mapping) is located at that name and categorized as JIT code.
30
+ `perf` records no symbol tables for the programs it samples, because
31
+ `perf report` reads the recorded program and its shared libraries off the
32
+ machine that recorded them. This tool reads no symbol tables, so it resolves an
33
+ address no further than the mapped file it fell in. Every frame is one address,
34
+ named by its offset into that file and located at the file's path, the same form
35
+ `perf script` prints for a frame it cannot symbolize. An address in memory a
36
+ program obtained without a file (an `//anon` mapping) is located at that name
37
+ and categorized as JIT code.
36
38
 
37
39
  Resolve an offset against the same build with `addr2line`:
38
40
 
@@ -40,13 +42,18 @@ Resolve an offset against the same build with `addr2line`:
40
42
  addr2line -fCe ./program 0x5f800
41
43
  ```
42
44
 
45
+ A caller's offset is its return address in a `perf` recording but inside the
46
+ call instruction in a simpleperf one, so subtract 1 from a `perf` caller's
47
+ offset to resolve the line of the call.
48
+
43
49
  The offset is file-relative rather than the address the code ran at, so it is
44
50
  the same across runs of the same build, and a diff of two recordings pairs the
45
- addresses that moved. A kernel frame's offset is from the start of the kernel's
46
- text (`_text`), which the kernel randomizes on every boot, so it too pairs
47
- across boots. Resolve one against the matching `vmlinux` by adding the address
48
- of `_text` from its `System.map`, or from `/proc/kallsyms` on the recording
49
- machine.
51
+ addresses that moved. A kernel frame's offset is from the symbol that follows
52
+ `[kernel.kallsyms]` in the name of the recording's kernel mapping, `_text` or
53
+ `_stext`. `perf script --show-mmap-events` prints that mapping. The kernel
54
+ randomizes that symbol's address on every boot, so the offset too pairs across
55
+ boots. Resolve one against the matching `vmlinux` by adding the symbol's address
56
+ from its `System.map`, or from `/proc/kallsyms` on the recording machine.
50
57
 
51
58
  For function names in the output, symbolize while converting instead: pipe
52
59
  `perf script` through FlameGraph's `stackcollapse-perf.pl` and pass the
@@ -55,20 +62,3 @@ collapsed stacks, which this tool also reads.
55
62
  ```sh
56
63
  perf script -i perf.data | stackcollapse-perf.pl > profile.collapsed
57
64
  ```
58
-
59
- ## Recordings this tool reads
60
-
61
- Record with `perf record -g`, which `profiler-md --help c` and
62
- `profiler-md --help zig` cover. This tool reads only the call chains the kernel
63
- walked, so it rejects a file recorded with `--call-graph dwarf`. That option
64
- leaves the chain holding one user-space frame and copies the rest of the stack
65
- into every sample for `perf report` to unwind. This tool also leaves
66
- `--call-graph lbr` branch histories unread. simpleperf's `record -g` unwinds
67
- that copy while recording and writes the frames into the chain, so this tool
68
- reads its files. On arm64, `perf report` adds one caller below the innermost
69
- frame by unwinding the sampled link register, so a stack there can name a caller
70
- this tool leaves out.
71
-
72
- It also rejects, with the reason, a file recorded with `perf record -z`
73
- (compressed), streamed to a pipe (`perf record -o -`), or written on a machine
74
- of the opposite byte order.
@@ -5,12 +5,14 @@
5
5
  format originally from Google. It is the de facto standard cross-language
6
6
  profiling format.
7
7
 
8
- Locations support inlined call frames, meaning a single instruction address can
9
- correspond to multiple logical stack frames.
8
+ A profile lists its sample types and its samples. A sample type is a metric's
9
+ name and unit, and a sample is a call stack of location IDs with one value per
10
+ sample type. A location is an address that maps to one or more source lines, so
11
+ one address stands for several frames where code was inlined.
10
12
 
11
13
  It supports arbitrary metric units, multiple metrics per profile, and multiple
12
14
  profiles per file. A file containing several counts (a heap profile's
13
- `alloc_objects` and `inuse_objects`) becomes one profile per count, each
15
+ `alloc_objects` and `inuse_objects`) converts to one profile per count, each
14
16
  reporting the metrics measured over it.
15
17
 
16
- Files are commonly gzip-compressed (extension `.pb.gz`).
18
+ Files are commonly gzip-compressed, with the `.pb.gz` extension.
@@ -6,8 +6,8 @@ Speedscope is a JSON profiling format used by
6
6
  Each profile within a file has a type: `sampled` (call stacks at points in time)
7
7
  or `evented` (frame open/close timestamps).
8
8
 
9
- It supports one metric per profile, measured in one of six units (nanoseconds,
10
- microseconds, milliseconds, seconds, bytes, or none), and multiple profiles per
11
- file.
9
+ It supports a single metric per profile, measured in one of six units
10
+ (nanoseconds, microseconds, milliseconds, seconds, bytes, or none), and multiple
11
+ profiles per file.
12
12
 
13
13
  Files use the `.speedscope.json` extension.
@@ -1,6 +1,6 @@
1
1
  # systing profile export
2
2
 
3
- The line-oriented profile export written by
3
+ The systing profile export is a line-oriented format written by
4
4
  [systing](https://github.com/josefbacik/systing), a Linux eBPF system profiler,
5
5
  specified by its
6
6
  [`docs/PROFILE_EXPORT_FORMAT.md`](https://github.com/josefbacik/systing/blob/main/docs/PROFILE_EXPORT_FORMAT.md).
@@ -0,0 +1,20 @@
1
+ # Trace Event Format
2
+
3
+ The
4
+ [Trace Event Format](https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview)
5
+ is the JSON trace format of Chromium's trace viewer. Chrome and Node.js write
6
+ it, as do tools that trace their own work, such as Clang and the TypeScript
7
+ compiler.
8
+
9
+ A file is an array of events, or an object whose `traceEvents` property is that
10
+ array. Each event has a name, a phase, a timestamp in microseconds, and the
11
+ process and thread it was recorded on.
12
+
13
+ A file converts to a trace. Duration and complete events become spans nested per
14
+ thread, async events become async spans, instant events become instants, and
15
+ each series of a counter event becomes a gauge. An event's string arguments
16
+ become its attributes.
17
+
18
+ It supports a single trace per file.
19
+
20
+ Files use the `.json` extension, except that Node.js writes `node_trace.1.log`.
@@ -1,6 +1,6 @@
1
1
  # V8 CPU profile
2
2
 
3
- The V8 CPU profile is a JSON format produced by the
3
+ The V8 CPU profile is a JSON format written by the
4
4
  [V8 JavaScript engine's sampling profiler](https://v8.dev/docs/profile).
5
5
 
6
6
  It stores a call tree (nodes) and the sequence of samples taken from it, each
@@ -1,15 +1,15 @@
1
1
  # V8 heap profile
2
2
 
3
- The V8 heap profile is a JSON format produced by
4
- [V8's sampling heap profiler](https://v8.dev/docs/profile). Node.js and Chrome
5
- DevTools generate it.
3
+ The V8 heap profile is a JSON format written by
4
+ [V8's sampling heap profiler](https://v8.dev/docs/profile), which Node.js and
5
+ Chrome DevTools run.
6
6
 
7
7
  It stores an allocation call tree whose nodes each state the bytes allocated at
8
8
  that call site, and a sequence of samples referencing those nodes.
9
9
 
10
- Heap profiles sample allocations, unlike heap snapshots, which capture all live
11
- objects at a point in time. That makes them useful for finding allocation
12
- hotspots rather than diagnosing what is live in memory.
10
+ Its samples are allocations, so it converts to a call stack profile of where the
11
+ program allocated. By default it contains only the samples whose objects were
12
+ still alive when it was written.
13
13
 
14
14
  It supports a single byte metric per profile and a single profile per file.
15
15
 
@@ -1,15 +1,16 @@
1
1
  # V8 heap snapshot
2
2
 
3
- The V8 heap snapshot is a JSON format produced by the V8 JavaScript engine's
4
- heap profiler that captures the full live object graph at a point in time.
3
+ The V8 heap snapshot is a JSON format written by the V8 JavaScript engine's heap
4
+ profiler that captures the full live object graph at a point in time.
5
5
 
6
- A snapshot is a `meta` object declaring the layout of the records that follow,
7
- flat integer arrays of node and edge records, a strings table the node and edge
8
- names index into, and a flat array of the source locations of the functions and
9
- objects the nodes stand for.
6
+ A heap snapshot is a `meta` object declaring the layout of the records that
7
+ follow, flat integer arrays of node and edge records, a strings table the node
8
+ and edge names index into, and a flat array of the source locations of the
9
+ functions and objects the nodes stand for.
10
10
 
11
- Snapshots show what objects exist in memory, unlike heap profiles, which sample
12
- allocations. That makes them useful for diagnosing memory leaks rather than
13
- allocation hotspots.
11
+ It converts to a heap snapshot, which shows what the heap holds rather than
12
+ where allocation happens.
13
+
14
+ It supports a single heap snapshot per file.
14
15
 
15
16
  Files use the `.heapsnapshot` extension.
@@ -1,14 +1,13 @@
1
1
  # WebKit timeline recording
2
2
 
3
3
  The [WebKit timeline recording](https://webkit.org/web-inspector/timelines-tab)
4
- is a JSON format captured by Safari's Web Inspector. It combines a CPU profile
4
+ is a JSON format written by Safari's Web Inspector. It combines a CPU profile
5
5
  from sampling, a browser activity breakdown (layout, rendering, script
6
6
  execution, network requests), and optionally in-use memory readings and heap
7
7
  allocation snapshots when those instruments are enabled.
8
8
 
9
9
  The recording holds the sampled call stacks alongside the duration each sample
10
- spans, so it converts to the time profile they record. This tool leaves the
11
- activity breakdown and the memory readings unread.
10
+ spans, so it converts to the time profile they record.
12
11
 
13
12
  It supports a single second metric per profile and a single profile per file.
14
13