hearald 0.0.4 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +133 -0
- package/README.md +120 -46
- package/dist/analytics.html +4 -4
- package/dist/index.css +2 -2
- package/dist/index.js +2 -2
- package/package.json +19 -5
- package/src/client.js +27 -7
- package/src/index.js +57 -11
- package/src/public/css/analytics.css +2 -2
- package/src/public/js/App.js +2 -2
- package/src/server/analytics.js +225 -72
- package/src/server/asyncHandler.js +6 -0
- package/src/server/auth.js +89 -0
- package/src/server/configuration.js +53 -0
- package/src/server/logger.js +14 -14
- package/src/server/server.js +99 -11
- package/src/server/serverEvents.js +154 -43
- package/src/server/validate.js +56 -0
- package/src/shared/utils.js +32 -16
- package/.eslintrc +0 -14
- package/TODO.md +0 -34
- package/rollup.config.js +0 -59
- package/src/server/serverTrackError.js +0 -47
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.1.0] - 2026-10-07
|
|
11
|
+
|
|
12
|
+
### Security
|
|
13
|
+
|
|
14
|
+
- **Breaking:** the analytics dashboard now needs an `auth` middleware (`analytics: { auth }`), or
|
|
15
|
+
`allowUnauthenticated: true` for local development. Before, anyone could read the stats.
|
|
16
|
+
- Added `basicAuth({ verify })`: HTTP Basic auth with a lockout after repeated wrong guesses.
|
|
17
|
+
- The data route now rejects dates that are not real calendar dates (`2026-02-31` used to be accepted
|
|
18
|
+
and moved to March) and time windows over 366 days (the date used to become part of a file path).
|
|
19
|
+
- `basicAuth` counted a wrong guess only after `verify` finished, so any number of parallel guesses got
|
|
20
|
+
past the lockout. Attempts are now counted before `verify` runs. It also accepts the `Basic` scheme
|
|
21
|
+
in any letter case, answers 500 (instead of crashing an Express 4 app) when `verify` throws, and
|
|
22
|
+
forgets old addresses (`maxTrackedIps`, default 10000). Its lockout needs Express's `trust proxy`
|
|
23
|
+
set correctly, see the README.
|
|
24
|
+
- A dashboard request for a huge window could use all the server's memory. Windows whose event files
|
|
25
|
+
add up to more than `maxWindowBytes` (default 64 MB) get 413 `{ "error": "window too large" }`, stats
|
|
26
|
+
are computed one request at a time, and the raw events are only kept for computed histograms.
|
|
27
|
+
|
|
28
|
+
### Added
|
|
29
|
+
|
|
30
|
+
- `eventEndpoint: { schema }`: allowed events and parameters for the public event endpoint. Events
|
|
31
|
+
not in the schema and parameters not listed are dropped, so the endpoint cannot store arbitrary text.
|
|
32
|
+
- `eventsDir`: where event logs and cached summaries are kept (default `events`).
|
|
33
|
+
- `purgeOldEvents({ days })`: deletes logs and cached summaries older than a retention period. It counts
|
|
34
|
+
days with the configured `nowFn` and in UTC; a file dated exactly `days` days ago is kept.
|
|
35
|
+
- `idleCloseMs` (default 60000): how long an event file stays open after its last write.
|
|
36
|
+
- `maxWindowBytes` (default 64 MB): the most event data one dashboard request may read.
|
|
37
|
+
- The test suite runs on Express 5 and Express 4 (`npm test` runs both; `HEARALD_EXPRESS=4` or `5` picks one).
|
|
38
|
+
- `analytics` is only created when `statDefinitions` is given.
|
|
39
|
+
|
|
40
|
+
### Changed
|
|
41
|
+
|
|
42
|
+
- `express` is now a peer dependency (>=4) instead of a dependency, so the host app's Express is used.
|
|
43
|
+
|
|
44
|
+
### Fixed
|
|
45
|
+
|
|
46
|
+
- The dashboard router could not be built on Express 5 (`Unexpected ? at index 17`: its optional
|
|
47
|
+
`:date?` route parameter is not allowed there), so `hearald({ analytics })` threw at startup. The
|
|
48
|
+
data route is now registered as two routes. Express 4 behaves as before.
|
|
49
|
+
- A failed attempt to open an event file (disk full, read-only file system, wrong permissions) was
|
|
50
|
+
remembered, so every later write failed too, and 60 seconds later closing it was an unhandled
|
|
51
|
+
rejection that ended the process. A failed open is now forgotten (the next write tries again), and
|
|
52
|
+
idle timers cannot reject or keep a script alive. The timer closes the file it was set for, even if
|
|
53
|
+
the events directory changed meanwhile.
|
|
54
|
+
- The error middleware could crash the process: an error that `JSON.stringify` cannot handle (a
|
|
55
|
+
circular object) or a failing write became an unhandled rejection. It now always passes the error
|
|
56
|
+
on to your own handler.
|
|
57
|
+
- On Express 4, a failing read in the dashboard routes (an unreadable cache file, a missing
|
|
58
|
+
`dist/analytics.html`) crashed the process; the error now goes to your error handler.
|
|
59
|
+
- The README mounted the event route at `/events` while the route inside is `/e`, so events went to
|
|
60
|
+
`/events/e` and the browser client's sends quietly got a 404. The README is fixed, and the client now
|
|
61
|
+
treats a non-2xx answer as a failure: `onError` is called (it still never throws).
|
|
62
|
+
- Cached summaries are named with a hash of the metric definitions and a format version
|
|
63
|
+
(`summary:<date>:<days>:<hash>.json`), so editing a definition no longer serves numbers computed with
|
|
64
|
+
the old one. Summaries cached before this change (including those written by 0.0.5, which used the
|
|
65
|
+
old `set_of_users` meaning and the old day buckets) are no longer read; they can be deleted
|
|
66
|
+
(`purgeOldEvents` still removes them by age).
|
|
67
|
+
- Histograms (raw and computed) mishandled a value named `constructor`, `toString` or `__proto__`, and
|
|
68
|
+
left out the values `0`, `false` and `""`.
|
|
69
|
+
- A truncated last line (after a crash) glued itself to the next event and lost both. A new line is
|
|
70
|
+
now started when a file is opened that does not end with one.
|
|
71
|
+
- `purgeOldEvents` ignored `nowFn` and used the server's local time zone.
|
|
72
|
+
- `dispose()` of an old client no longer silences the client that replaced it.
|
|
73
|
+
- `trackServerError` and the error middleware crashed with a ReferenceError (`ERROR_EVENT` was never
|
|
74
|
+
imported).
|
|
75
|
+
- The log level setting did nothing: the logger was built from the string `"error"` instead of the
|
|
76
|
+
configuration. Levels now work (`debug`, `info`, `warn`, `error`); the default is `warn`.
|
|
77
|
+
- Days with no events no longer log a warning for every missing file.
|
|
78
|
+
- A computed metric with an unknown formula made `generateStats` loop forever; it now throws.
|
|
79
|
+
- `nowAsPstDate` returned a Date with a forced hour and the UTC date, so day buckets were not Pacific
|
|
80
|
+
days. It now holds the real Pacific wall-clock time, with daylight saving. `defaultEndDate` uses
|
|
81
|
+
UTC date maths and follows the configured `nowFn` instead of the real clock.
|
|
82
|
+
- Reading a window of days used local-time date maths, which could skip or repeat a day around
|
|
83
|
+
daylight saving changes on servers not set to UTC.
|
|
84
|
+
- Stats for a window ending today were cached, so the dashboard showed stale numbers all day.
|
|
85
|
+
- The client's `trackError` threw when the error event had no `.error` object, and calling
|
|
86
|
+
`hearald()` twice sent every event twice (the new `dispose()` and replacement fix both).
|
|
87
|
+
- Without a schema, the event endpoint stored events with no name or with a non-object `p`.
|
|
88
|
+
- A `set_of_users` metric that names an `event` now counts only users who sent that event (it used to
|
|
89
|
+
count every user, whatever the event). A `filter` on event parameters already worked.
|
|
90
|
+
- Events with no parameters or no user no longer break metric computation.
|
|
91
|
+
- Open event files are tracked by full path, so changing `eventsDir` cannot reuse a stale file handle.
|
|
92
|
+
|
|
93
|
+
## [0.0.5] - 2025-10-05
|
|
94
|
+
|
|
95
|
+
### Added
|
|
96
|
+
|
|
97
|
+
- Export `errorMiddlware`, `trackServerEvent`, `trackServerError` as part of default `hearald` in `src/index.js'
|
|
98
|
+
|
|
99
|
+
### Fixed
|
|
100
|
+
|
|
101
|
+
- Bugfix to restore intra-day timestamp granularity of events.
|
|
102
|
+
- Percent formatted numbers now use proper formatter
|
|
103
|
+
|
|
104
|
+
### Changed
|
|
105
|
+
|
|
106
|
+
- All server-side configuration parameters are defined in `src/server/configuration.js`
|
|
107
|
+
|
|
108
|
+
## [0.0.4] - 2025-09-21
|
|
109
|
+
|
|
110
|
+
### Changed
|
|
111
|
+
|
|
112
|
+
- Use package.json `exports`
|
|
113
|
+
|
|
114
|
+
## [0.0.3] - 2025-09-19
|
|
115
|
+
|
|
116
|
+
### Added
|
|
117
|
+
|
|
118
|
+
- Initial release of Hearald
|
|
119
|
+
|
|
120
|
+
[Unreleased]: https://github.com/pariser/hearald/compare/v0.1.0...HEAD
|
|
121
|
+
[0.1.0]: https://github.com/pariser/hearald/compare/v0.0.5...v0.1.0
|
|
122
|
+
[0.0.5]: https://github.com/pariser/hearald/compare/v0.0.4...v0.0.5
|
|
123
|
+
[0.0.4]: https://github.com/pariser/hearald/compare/v0.0.3...v0.0.4
|
|
124
|
+
[0.0.3]: https://github.com/pariser/hearald/releases/tag/v0.0.3
|
|
125
|
+
|
|
126
|
+
<!--
|
|
127
|
+
`Added` for new features.
|
|
128
|
+
`Changed` for changes in existing functionality.
|
|
129
|
+
`Deprecated` for soon-to-be removed features.
|
|
130
|
+
`Removed` for now removed features.
|
|
131
|
+
`Fixed` for any bug fixes.
|
|
132
|
+
`Security` in case of vulnerabilities.
|
|
133
|
+
-->
|
package/README.md
CHANGED
|
@@ -1,71 +1,145 @@
|
|
|
1
1
|
# hearald
|
|
2
2
|
|
|
3
|
-
A single server drop-in file system based metrics and analytics library
|
|
3
|
+
A single server drop-in file system based metrics and analytics library: your apps send events, they
|
|
4
|
+
are appended to one log file per day, and a small dashboard turns them into counts, unique users and
|
|
5
|
+
histograms. No database.
|
|
4
6
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
## Usage
|
|
7
|
+
```bash
|
|
8
|
+
npm test # unit and integration tests, run against Express 5 and then Express 4
|
|
9
|
+
npm run build # rebuilds dist/ (the dashboard page)
|
|
10
|
+
```
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
## Server
|
|
13
13
|
|
|
14
14
|
```js
|
|
15
|
-
import
|
|
16
|
-
|
|
15
|
+
import express from "express";
|
|
16
|
+
import hearald, { basicAuth } from "hearald";
|
|
17
|
+
|
|
18
|
+
const app = express();
|
|
19
|
+
app.use(express.json());
|
|
20
|
+
|
|
21
|
+
const h = hearald({
|
|
22
|
+
eventsDir: "/var/lib/myapp/events", // default "events", relative to the working directory
|
|
23
|
+
// idleCloseMs: 60000, // how long an event file stays open after its last write
|
|
24
|
+
// maxWindowBytes: 64 * 1024 * 1024, // dashboard windows with more event data than this get a 413
|
|
25
|
+
getUserId: (req) => req.user?.id, // used for server-side errors
|
|
26
|
+
// Strongly recommended for any endpoint open to the internet: only these events and parameters
|
|
27
|
+
// are stored, everything else is dropped (the caller still gets a 204).
|
|
28
|
+
eventEndpoint: {
|
|
29
|
+
url: "/events", // where the route listens, inside wherever you mount eventMiddleware (default "/e")
|
|
30
|
+
schema: {
|
|
31
|
+
app_open: { platform: { type: "string", enum: ["web", "ios"], required: true } },
|
|
32
|
+
purchase: { plan: { type: "string", maxLength: 32 }, cents: { type: "number", integer: true, min: 0 } },
|
|
33
|
+
},
|
|
34
|
+
// parseBody: (req) => ({ e, u, p }), for example to take the user from a session, not the body
|
|
35
|
+
},
|
|
36
|
+
analytics: {
|
|
37
|
+
statDefinitions, // see below
|
|
38
|
+
auth: basicAuth({ verify: (user, pass) => user === "admin" && passwordMatches(pass) }),
|
|
39
|
+
// allowUnauthenticated: true, // local development only
|
|
40
|
+
},
|
|
41
|
+
});
|
|
42
|
+
|
|
43
|
+
app.use(h.eventMiddleware); // POST /events { e, u, p }: the route's own `url`, so mount it at the root
|
|
44
|
+
app.use("/admin/analytics", h.analyticsMiddleware); // the dashboard
|
|
45
|
+
app.use(h.errorMiddlware); // records errors thrown by your routes, then passes them on
|
|
46
|
+
|
|
47
|
+
await h.trackServerEvent({ event: "job_done", userId: "system", params: { count: 3 } });
|
|
48
|
+
await h.trackServerError({ error: new Error("boom"), userId: "u1", extraParams: { route: "/save" } });
|
|
49
|
+
await h.purgeOldEvents({ days: 90 }); // run nightly to keep a retention period
|
|
50
|
+
await h.shutDown(); // closes open files on exit
|
|
17
51
|
```
|
|
18
52
|
|
|
19
|
-
|
|
53
|
+
The dashboard has no default password: `analytics` throws unless you pass `auth`. `basicAuth` answers 401
|
|
54
|
+
with a challenge and locks an IP out (429) after 10 wrong guesses in 15 minutes. Store a hash of the
|
|
55
|
+
password and compare in constant time.
|
|
20
56
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
57
|
+
`url` is the full path of the route (`/events` above), and `eventMiddleware` is a router that serves it, so
|
|
58
|
+
mount it at the root. If you mount it at `app.use("/api", h.eventMiddleware)`, events arrive at
|
|
59
|
+
`/api/events`, and the browser client's `endpoint` must say so.
|
|
60
|
+
|
|
61
|
+
`basicAuth` counts wrong guesses per `req.ip`. That is the visitor's address only if your app sets
|
|
62
|
+
Express's `trust proxy` correctly: behind a reverse proxy without it every visitor shares the proxy's
|
|
63
|
+
address, and with it set too loosely a caller can forge `X-Forwarded-For` to avoid the lockout. If
|
|
64
|
+
`verify` throws, the request gets a 500 and is not counted as a wrong guess.
|
|
65
|
+
|
|
66
|
+
**The event endpoint is public and has no rate limit or size cap of its own.** Anyone who can reach it
|
|
67
|
+
can send events, so fake user ids can inflate "unique users" and a flood can fill the disk. Use `schema`
|
|
68
|
+
(it bounds string lengths and the set of event names, not how many events arrive), keep Express's
|
|
69
|
+
body limit small (`express.json({ limit: "10kb" })`), and put a rate limit (for example
|
|
70
|
+
`express-rate-limit`) in front of it, or in your own route, as part of your hosting setup. Run `purgeOldEvents` too.
|
|
29
71
|
|
|
30
|
-
|
|
72
|
+
Only `express` (>= 4) is a peer dependency; use your app's own. hearald is tested on Express 4 and 5.
|
|
73
|
+
|
|
74
|
+
## Browser
|
|
31
75
|
|
|
32
76
|
```js
|
|
33
|
-
import
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
77
|
+
import hearaldClient from "hearald/client";
|
|
78
|
+
|
|
79
|
+
const h = hearaldClient({
|
|
80
|
+
endpoint: "/events",
|
|
81
|
+
getUserId: () => currentUserId, // used when an event does not name a user
|
|
82
|
+
onError: (err, eventName) => {}, // a failed send never throws; a 404 or 500 from the server counts as failed
|
|
83
|
+
});
|
|
84
|
+
|
|
85
|
+
h.trackEvent({ eventName: "app_open", payload: { platform: "web" } });
|
|
86
|
+
h.trackVisit({ userId: "user-123" }); // adds user agent, referrer, URL and screen size
|
|
87
|
+
window.addEventListener("error", (error) => h.trackError({ error })); // ErrorEvent or Error
|
|
88
|
+
h.dispose(); // stop sending (calling hearaldClient again also replaces the previous one; an old
|
|
89
|
+
// instance's dispose() only stops that instance)
|
|
38
90
|
```
|
|
39
91
|
|
|
40
|
-
|
|
92
|
+
## Stat definitions
|
|
41
93
|
|
|
42
94
|
```js
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
95
|
+
const statDefinitions = {
|
|
96
|
+
raw_metrics: {
|
|
97
|
+
// count events
|
|
98
|
+
"opens": { aggregator: "count", event: "app_open" },
|
|
99
|
+
// the set of users who sent an event, optionally only with certain parameter values
|
|
100
|
+
"users.web": { aggregator: "set_of_users", event: "app_open", filter: { platform: "web" } },
|
|
101
|
+
// [value, count] pairs for one parameter, most common first
|
|
102
|
+
"errors.by_code": { aggregator: "histogram", event: "sync_error", field: "code" },
|
|
103
|
+
// read the visits file instead of the main log
|
|
104
|
+
"visits": { aggregator: "count", event: "visit", source_file_suffix: "visits" },
|
|
105
|
+
},
|
|
106
|
+
computed_metrics: {
|
|
107
|
+
"users.web_count": { formula: "set_count", metrics: ["users.web"] },
|
|
108
|
+
"users.new": { formula: "set_difference", metrics: ["users.week", "users.before"] }, // in the first, not the others
|
|
109
|
+
"conversion": { formula: "percent", metrics: ["buyers", "users"] }, // first / second * 100
|
|
110
|
+
"opens.per_day": { formula: "rate_time_window", metrics: ["opens"] },
|
|
111
|
+
},
|
|
112
|
+
ui: { sections: [{ title: "People", fields: [{ key: "users.web_count", label: "Web users", type: "number" }] }] },
|
|
113
|
+
};
|
|
50
114
|
```
|
|
51
115
|
|
|
52
|
-
|
|
116
|
+
Names with dots become nested objects in the result. `hidden: true` leaves a metric out of the output
|
|
117
|
+
(useful for sets that only feed a computed metric). A computed metric that names a formula that does
|
|
118
|
+
not exist is an error; one whose inputs are missing is simply left out.
|
|
53
119
|
|
|
54
|
-
|
|
55
|
-
Example:
|
|
120
|
+
## Dashboard routes
|
|
56
121
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
122
|
+
Under wherever you mounted `analyticsMiddleware`: `/` (the page), `/layout` (your `ui`), and
|
|
123
|
+
`/data/:days/:endDate?` (for example `/data/7/2026-11-01`: the 7 days ending that day; `endDate`
|
|
124
|
+
defaults to yesterday). `days` is 1 to 366 and `endDate` must be a real calendar date (`2026-02-31` is not),
|
|
125
|
+
otherwise 400. A window whose event files add up to more than `maxWindowBytes` (64 MB by default) is
|
|
126
|
+
refused with 413 `{ "error": "window too large" }`; stats are computed one request at a time, because a
|
|
127
|
+
window is held in memory while it is counted. Windows that end today or later are computed on every
|
|
128
|
+
request; finished windows are cached in `eventsDir` as `summary:<end date>:<days>:<hash>.json`, where the
|
|
129
|
+
hash covers your metric definitions, so changing a definition recomputes instead of serving old numbers.
|
|
130
|
+
Cached summaries are safe to delete at any time.
|
|
61
131
|
|
|
62
|
-
|
|
132
|
+
## Days and time zones
|
|
63
133
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
134
|
+
Events are bucketed into days with `nowFn` (default: the current Pacific wall-clock time, with
|
|
135
|
+
daylight saving) and `isoFn` (the UTC date of what `nowFn` returns). Pass `nowFn: () => new Date()` to
|
|
136
|
+
bucket by UTC days instead. Nothing depends on the server's own time zone.
|
|
137
|
+
|
|
138
|
+
With the default `nowFn`, the `t` stored in each event is also the Pacific wall-clock time, written with a
|
|
139
|
+
`Z` as if it were UTC (so noon UTC is stored as 05:00 or 04:00, and two instants an hour apart at the
|
|
140
|
+
November clock change get the same `t`). It is right for choosing the day, wrong as an exact instant. If
|
|
141
|
+
you need real instants in the logs, pass `nowFn: () => new Date()` and accept UTC days.
|
|
68
142
|
|
|
69
143
|
## Roadmap
|
|
70
144
|
|
|
71
|
-
See TODO.md
|
|
145
|
+
See TODO.md.
|
package/dist/analytics.html
CHANGED
|
@@ -134,7 +134,7 @@ th {
|
|
|
134
134
|
@media screen and (max-width: 640px) {
|
|
135
135
|
body {
|
|
136
136
|
font-size: 12px;
|
|
137
|
-
font-feature-settings:
|
|
137
|
+
font-feature-settings: "lnum";
|
|
138
138
|
}
|
|
139
139
|
}
|
|
140
140
|
|
|
@@ -264,7 +264,7 @@ th {
|
|
|
264
264
|
word-break: keep-all;
|
|
265
265
|
}
|
|
266
266
|
.bar-chart__bar__label:after {
|
|
267
|
-
content:
|
|
267
|
+
content: "";
|
|
268
268
|
width: 12px;
|
|
269
269
|
height: 12px;
|
|
270
270
|
background: var(--color-label);
|
|
@@ -915,8 +915,8 @@ th {
|
|
|
915
915
|
type="${f.type}"
|
|
916
916
|
value=${deepGet(stats.metrics, f.key)}
|
|
917
917
|
span=${f.span || null}
|
|
918
|
-
|
|
919
|
-
|
|
918
|
+
percent=${f.type === "percent"}
|
|
919
|
+
histogram=${f.type === "histogram"}
|
|
920
920
|
/>
|
|
921
921
|
`;
|
|
922
922
|
})}
|
package/dist/index.css
CHANGED
|
@@ -122,7 +122,7 @@ th {
|
|
|
122
122
|
@media screen and (max-width: 640px) {
|
|
123
123
|
body {
|
|
124
124
|
font-size: 12px;
|
|
125
|
-
font-feature-settings:
|
|
125
|
+
font-feature-settings: "lnum";
|
|
126
126
|
}
|
|
127
127
|
}
|
|
128
128
|
|
|
@@ -252,7 +252,7 @@ th {
|
|
|
252
252
|
word-break: keep-all;
|
|
253
253
|
}
|
|
254
254
|
.bar-chart__bar__label:after {
|
|
255
|
-
content:
|
|
255
|
+
content: "";
|
|
256
256
|
width: 12px;
|
|
257
257
|
height: 12px;
|
|
258
258
|
background: var(--color-label);
|
package/dist/index.js
CHANGED
|
@@ -517,8 +517,8 @@
|
|
|
517
517
|
type="${f.type}"
|
|
518
518
|
value=${deepGet(stats.metrics, f.key)}
|
|
519
519
|
span=${f.span || null}
|
|
520
|
-
|
|
521
|
-
|
|
520
|
+
percent=${f.type === "percent"}
|
|
521
|
+
histogram=${f.type === "histogram"}
|
|
522
522
|
/>
|
|
523
523
|
`;
|
|
524
524
|
})}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "hearald",
|
|
3
|
-
"version": "0.0
|
|
3
|
+
"version": "0.1.0",
|
|
4
4
|
"description": "A single server drop-in file system based metrics and analytics library.",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"exports": {
|
|
@@ -9,7 +9,10 @@
|
|
|
9
9
|
},
|
|
10
10
|
"type": "module",
|
|
11
11
|
"scripts": {
|
|
12
|
-
"build": "rollup --config"
|
|
12
|
+
"build": "rollup --config",
|
|
13
|
+
"test": "npm run test:express5 && npm run test:express4",
|
|
14
|
+
"test:express5": "HEARALD_EXPRESS=5 node --import ./test-support/register.js --test",
|
|
15
|
+
"test:express4": "HEARALD_EXPRESS=4 node --import ./test-support/register.js --test"
|
|
13
16
|
},
|
|
14
17
|
"keywords": [
|
|
15
18
|
"herald",
|
|
@@ -21,8 +24,7 @@
|
|
|
21
24
|
"license": "MIT",
|
|
22
25
|
"dependencies": {
|
|
23
26
|
"bussin-js": "^1.0.1",
|
|
24
|
-
"chalk": "^4.1.0"
|
|
25
|
-
"express": "^4.17.1"
|
|
27
|
+
"chalk": "^4.1.0"
|
|
26
28
|
},
|
|
27
29
|
"devDependencies": {
|
|
28
30
|
"@rollup/plugin-commonjs": "^15.1.0",
|
|
@@ -35,6 +37,8 @@
|
|
|
35
37
|
"eslint-plugin-jsx-a11y": "^6.6.1",
|
|
36
38
|
"eslint-plugin-react": "^7.31.10",
|
|
37
39
|
"eslint-plugin-react-hooks": "^4.6.0",
|
|
40
|
+
"express": "^5.2.1",
|
|
41
|
+
"express4": "npm:express@^4.21.2",
|
|
38
42
|
"htm": "^3.0.3",
|
|
39
43
|
"preact": "^10.3.3",
|
|
40
44
|
"prettier": "2.2.1",
|
|
@@ -45,5 +49,15 @@
|
|
|
45
49
|
"rollup-plugin-postcss": "^4.0.2",
|
|
46
50
|
"rollup-plugin-terser": "^7.0.2",
|
|
47
51
|
"style-inject": "^0.3.0"
|
|
48
|
-
}
|
|
52
|
+
},
|
|
53
|
+
"peerDependencies": {
|
|
54
|
+
"express": ">=4"
|
|
55
|
+
},
|
|
56
|
+
"files": [
|
|
57
|
+
"src",
|
|
58
|
+
"dist",
|
|
59
|
+
"README.md",
|
|
60
|
+
"CHANGELOG.md",
|
|
61
|
+
"LICENSE"
|
|
62
|
+
]
|
|
49
63
|
}
|
package/src/client.js
CHANGED
|
@@ -20,8 +20,10 @@ const trackError = ({
|
|
|
20
20
|
userId = null,
|
|
21
21
|
payload: extraPayload = {},
|
|
22
22
|
} = {}) => {
|
|
23
|
-
|
|
24
|
-
|
|
23
|
+
// `error` is normally the ErrorEvent from window.onerror, whose `.error` can be null (for example
|
|
24
|
+
// for errors from other origins), or a plain Error.
|
|
25
|
+
const errorEvent = error || {};
|
|
26
|
+
const errorObject = errorEvent.error || (error instanceof Error ? error : {});
|
|
25
27
|
|
|
26
28
|
let source;
|
|
27
29
|
if (errorObject.stack) {
|
|
@@ -35,7 +37,7 @@ const trackError = ({
|
|
|
35
37
|
eventName: "error",
|
|
36
38
|
userId,
|
|
37
39
|
payload: {
|
|
38
|
-
message: errorObject.message,
|
|
40
|
+
message: errorObject.message || errorEvent.message || String(error),
|
|
39
41
|
source,
|
|
40
42
|
file_name: errorEvent.filename || errorObject.fileName,
|
|
41
43
|
line_number: errorEvent.lineno || errorObject.lineNumber,
|
|
@@ -73,30 +75,48 @@ function emitEventToServer({
|
|
|
73
75
|
});
|
|
74
76
|
}
|
|
75
77
|
|
|
78
|
+
let activeListener = null;
|
|
79
|
+
|
|
76
80
|
export default function hearald({
|
|
77
81
|
endpoint = "/e",
|
|
78
|
-
fetchImpl = fetch,
|
|
82
|
+
fetchImpl = (...args) => globalThis.fetch(...args),
|
|
79
83
|
getUserId = () => null,
|
|
80
84
|
onError = () => {} /* err, eventName, userId, payload */,
|
|
81
85
|
} = {}) {
|
|
82
|
-
|
|
86
|
+
// The event bus is shared, so a second call (a hot reload, say) replaces the first listener
|
|
87
|
+
// instead of sending every event twice.
|
|
88
|
+
if (activeListener) eventBus.off("event", activeListener);
|
|
89
|
+
const listener = async (event) => {
|
|
83
90
|
const { eventName, userId: emittedUserId, payload } = event;
|
|
84
91
|
const userId = emittedUserId || getUserId() || null;
|
|
85
92
|
|
|
86
93
|
try {
|
|
87
|
-
await emitEventToServer({
|
|
94
|
+
const response = await emitEventToServer({
|
|
88
95
|
endpoint,
|
|
89
96
|
eventName,
|
|
90
97
|
userId,
|
|
91
98
|
payload,
|
|
92
99
|
fetchImpl,
|
|
93
100
|
});
|
|
101
|
+
// fetch only rejects on network failure; a 404 or 500 arrives as an ordinary response
|
|
102
|
+
if (response && response.ok === false) {
|
|
103
|
+
throw new Error(
|
|
104
|
+
`hearald: the server answered ${response.status} for "${eventName}"`
|
|
105
|
+
);
|
|
106
|
+
}
|
|
94
107
|
} catch (err) {
|
|
95
108
|
onError(err, eventName, userId, payload);
|
|
96
109
|
}
|
|
97
|
-
}
|
|
110
|
+
};
|
|
111
|
+
activeListener = listener;
|
|
112
|
+
eventBus.on("event", listener);
|
|
98
113
|
|
|
99
114
|
return {
|
|
115
|
+
// Removes only this instance's listener: an older instance's dispose() must not silence a newer one.
|
|
116
|
+
dispose() {
|
|
117
|
+
eventBus.off("event", listener);
|
|
118
|
+
if (activeListener === listener) activeListener = null;
|
|
119
|
+
},
|
|
100
120
|
eventBus,
|
|
101
121
|
trackEvent,
|
|
102
122
|
trackError,
|
package/src/index.js
CHANGED
|
@@ -1,25 +1,71 @@
|
|
|
1
|
-
import
|
|
2
|
-
|
|
1
|
+
import hearaldConfiguration from "./server/configuration.js";
|
|
2
|
+
|
|
3
|
+
import {
|
|
4
|
+
eventEndpointMiddleware,
|
|
5
|
+
trackServerEvent,
|
|
6
|
+
trackServerError,
|
|
7
|
+
errorTrackingMiddleware,
|
|
8
|
+
shutDown,
|
|
9
|
+
} from "./server/server.js";
|
|
3
10
|
import { createAnalyticsRouter } from "./server/analytics.js";
|
|
11
|
+
import { purgeOldEvents } from "./server/serverEvents.js";
|
|
12
|
+
import { basicAuth } from "./server/auth.js";
|
|
13
|
+
|
|
14
|
+
export { basicAuth, purgeOldEvents };
|
|
4
15
|
|
|
5
16
|
export default function hearald({
|
|
17
|
+
/* singleton configuration options */
|
|
6
18
|
logLevel,
|
|
7
|
-
|
|
8
|
-
|
|
19
|
+
nowFn /* () => nowAsDateString */,
|
|
20
|
+
getUserId /* (req) => userId */,
|
|
21
|
+
eventsDir /* where event logs are kept; default "events" */,
|
|
22
|
+
idleCloseMs /* how long an event file stays open after its last write; default 60000 */,
|
|
23
|
+
maxWindowBytes /* the most event-log bytes one dashboard request may read; default 64 MB */,
|
|
24
|
+
|
|
25
|
+
eventEndpoint: { url = "/e", parseBody, schema } = {},
|
|
26
|
+
analytics: {
|
|
27
|
+
enabled: analyticsEnabled = true,
|
|
28
|
+
statDefinitions,
|
|
29
|
+
auth,
|
|
30
|
+
allowUnauthenticated,
|
|
31
|
+
} = {},
|
|
9
32
|
} = {}) {
|
|
10
33
|
if (logLevel) {
|
|
11
|
-
|
|
34
|
+
hearaldConfiguration.setLogLevel(logLevel);
|
|
35
|
+
}
|
|
36
|
+
if (nowFn) {
|
|
37
|
+
hearaldConfiguration.setNowFn(nowFn);
|
|
38
|
+
}
|
|
39
|
+
if (getUserId) {
|
|
40
|
+
hearaldConfiguration.setGetUserId(getUserId);
|
|
12
41
|
}
|
|
42
|
+
if (eventsDir) {
|
|
43
|
+
hearaldConfiguration.setEventsDir(eventsDir);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
if (idleCloseMs !== undefined) {
|
|
47
|
+
hearaldConfiguration.setIdleCloseMs(idleCloseMs);
|
|
48
|
+
}
|
|
49
|
+
if (maxWindowBytes !== undefined) {
|
|
50
|
+
hearaldConfiguration.setMaxWindowBytes(maxWindowBytes);
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
const eventMiddleware = eventEndpointMiddleware({ url, parseBody, schema });
|
|
13
54
|
|
|
14
|
-
const
|
|
55
|
+
const analyticsMiddleware =
|
|
56
|
+
analyticsEnabled && statDefinitions
|
|
57
|
+
? createAnalyticsRouter({ statDefinitions, auth, allowUnauthenticated })
|
|
58
|
+
: null;
|
|
15
59
|
|
|
16
|
-
const
|
|
17
|
-
? createAnalyticsRouter({ statDefinitions })
|
|
18
|
-
: null;
|
|
60
|
+
const errorMiddlware = errorTrackingMiddleware({});
|
|
19
61
|
|
|
20
62
|
return {
|
|
21
|
-
|
|
22
|
-
|
|
63
|
+
errorMiddlware /* app.use(errorMiddlware) */,
|
|
64
|
+
eventMiddleware /* app.use(eventMiddleware) */,
|
|
65
|
+
analyticsMiddleware /* app.use('/analytics', analyticsMiddleware) */,
|
|
66
|
+
trackServerEvent /* ({ event, time = null, userId, params = {} }) => Promise } */,
|
|
67
|
+
trackServerError /* ({ error, userId = null, extraParams = {} }) => Promise } */,
|
|
68
|
+
purgeOldEvents /* ({ days }) => Promise<number of files removed> */,
|
|
23
69
|
shutDown,
|
|
24
70
|
};
|
|
25
71
|
}
|
|
@@ -122,7 +122,7 @@ th {
|
|
|
122
122
|
@media screen and (max-width: 640px) {
|
|
123
123
|
body {
|
|
124
124
|
font-size: 12px;
|
|
125
|
-
font-feature-settings:
|
|
125
|
+
font-feature-settings: "lnum";
|
|
126
126
|
}
|
|
127
127
|
}
|
|
128
128
|
|
|
@@ -252,7 +252,7 @@ th {
|
|
|
252
252
|
word-break: keep-all;
|
|
253
253
|
}
|
|
254
254
|
.bar-chart__bar__label:after {
|
|
255
|
-
content:
|
|
255
|
+
content: "";
|
|
256
256
|
width: 12px;
|
|
257
257
|
height: 12px;
|
|
258
258
|
background: var(--color-label);
|
package/src/public/js/App.js
CHANGED
|
@@ -303,8 +303,8 @@ class App extends Component {
|
|
|
303
303
|
type="${f.type}"
|
|
304
304
|
value=${deepGet(stats.metrics, f.key)}
|
|
305
305
|
span=${f.span || null}
|
|
306
|
-
|
|
307
|
-
|
|
306
|
+
percent=${f.type === "percent"}
|
|
307
|
+
histogram=${f.type === "histogram"}
|
|
308
308
|
/>
|
|
309
309
|
`;
|
|
310
310
|
})}
|