hearald 0.0.5 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +101 -3
- package/README.md +120 -46
- package/dist/analytics.html +6 -6
- package/dist/index.js +6 -6
- package/package.json +21 -5
- package/src/client.js +27 -7
- package/src/index.js +30 -6
- package/src/public/js/App.js +5 -5
- package/src/public/js/util.js +1 -1
- package/src/server/analytics.js +221 -64
- package/src/server/asyncHandler.js +6 -0
- package/src/server/auth.js +89 -0
- package/src/server/configuration.js +21 -1
- package/src/server/logger.js +12 -10
- package/src/server/server.js +47 -9
- package/src/server/serverEvents.js +146 -38
- package/src/server/validate.js +56 -0
- package/src/shared/utils.js +32 -14
- package/.eslintrc +0 -14
- package/.prettierignore +0 -9
- package/TODO.md +0 -34
- package/rollup.config.js +0 -59
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,101 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.1.1] - 2026-10-09
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- The dashboard threw errors the first time it loaded with no data (`Cannot read properties of
|
|
15
|
+
undefined (reading 'map')` and `(reading 'users')`). It now shows zeros and empty menus when there
|
|
16
|
+
are no events, no metrics for the chosen chart context, or no `ui` layout.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- Tests that render the dashboard in a fake browser (happy-dom) against a real hearald router.
|
|
21
|
+
|
|
22
|
+
## [0.1.0] - 2026-10-07
|
|
23
|
+
|
|
24
|
+
### Security
|
|
25
|
+
|
|
26
|
+
- **Breaking:** the analytics dashboard now needs an `auth` middleware (`analytics: { auth }`), or
|
|
27
|
+
`allowUnauthenticated: true` for local development. Before, anyone could read the stats.
|
|
28
|
+
- Added `basicAuth({ verify })`: HTTP Basic auth with a lockout after repeated wrong guesses.
|
|
29
|
+
- The data route now rejects dates that are not real calendar dates (`2026-02-31` used to be accepted
|
|
30
|
+
and moved to March) and time windows over 366 days (the date used to become part of a file path).
|
|
31
|
+
- `basicAuth` counted a wrong guess only after `verify` finished, so any number of parallel guesses got
|
|
32
|
+
past the lockout. Attempts are now counted before `verify` runs. It also accepts the `Basic` scheme
|
|
33
|
+
in any letter case, answers 500 (instead of crashing an Express 4 app) when `verify` throws, and
|
|
34
|
+
forgets old addresses (`maxTrackedIps`, default 10000). Its lockout needs Express's `trust proxy`
|
|
35
|
+
set correctly, see the README.
|
|
36
|
+
- A dashboard request for a huge window could use all the server's memory. Windows whose event files
|
|
37
|
+
add up to more than `maxWindowBytes` (default 64 MB) get 413 `{ "error": "window too large" }`, stats
|
|
38
|
+
are computed one request at a time, and the raw events are only kept for computed histograms.
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
|
|
42
|
+
- `eventEndpoint: { schema }`: allowed events and parameters for the public event endpoint. Events
|
|
43
|
+
not in the schema and parameters not listed are dropped, so the endpoint cannot store arbitrary text.
|
|
44
|
+
- `eventsDir`: where event logs and cached summaries are kept (default `events`).
|
|
45
|
+
- `purgeOldEvents({ days })`: deletes logs and cached summaries older than a retention period. It counts
|
|
46
|
+
days with the configured `nowFn` and in UTC; a file dated exactly `days` days ago is kept.
|
|
47
|
+
- `idleCloseMs` (default 60000): how long an event file stays open after its last write.
|
|
48
|
+
- `maxWindowBytes` (default 64 MB): the most event data one dashboard request may read.
|
|
49
|
+
- The test suite runs on Express 5 and Express 4 (`npm test` runs both; `HEARALD_EXPRESS=4` or `5` picks one).
|
|
50
|
+
- `analytics` is only created when `statDefinitions` is given.
|
|
51
|
+
|
|
52
|
+
### Changed
|
|
53
|
+
|
|
54
|
+
- `express` is now a peer dependency (>=4) instead of a dependency, so the host app's Express is used.
|
|
55
|
+
|
|
56
|
+
### Fixed
|
|
57
|
+
|
|
58
|
+
- The dashboard router could not be built on Express 5 (`Unexpected ? at index 17`: its optional
|
|
59
|
+
`:date?` route parameter is not allowed there), so `hearald({ analytics })` threw at startup. The
|
|
60
|
+
data route is now registered as two routes. Express 4 behaves as before.
|
|
61
|
+
- A failed attempt to open an event file (disk full, read-only file system, wrong permissions) was
|
|
62
|
+
remembered, so every later write failed too, and 60 seconds later closing it was an unhandled
|
|
63
|
+
rejection that ended the process. A failed open is now forgotten (the next write tries again), and
|
|
64
|
+
idle timers cannot reject or keep a script alive. The timer closes the file it was set for, even if
|
|
65
|
+
the events directory changed meanwhile.
|
|
66
|
+
- The error middleware could crash the process: an error that `JSON.stringify` cannot handle (a
|
|
67
|
+
circular object) or a failing write became an unhandled rejection. It now always passes the error
|
|
68
|
+
on to your own handler.
|
|
69
|
+
- On Express 4, a failing read in the dashboard routes (an unreadable cache file, a missing
|
|
70
|
+
`dist/analytics.html`) crashed the process; the error now goes to your error handler.
|
|
71
|
+
- The README mounted the event route at `/events` while the route inside is `/e`, so events went to
|
|
72
|
+
`/events/e` and the browser client's sends quietly got a 404. The README is fixed, and the client now
|
|
73
|
+
treats a non-2xx answer as a failure: `onError` is called (it still never throws).
|
|
74
|
+
- Cached summaries are named with a hash of the metric definitions and a format version
|
|
75
|
+
(`summary:<date>:<days>:<hash>.json`), so editing a definition no longer serves numbers computed with
|
|
76
|
+
the old one. Summaries cached before this change (including those written by 0.0.5, which used the
|
|
77
|
+
old `set_of_users` meaning and the old day buckets) are no longer read; they can be deleted
|
|
78
|
+
(`purgeOldEvents` still removes them by age).
|
|
79
|
+
- Histograms (raw and computed) mishandled a value named `constructor`, `toString` or `__proto__`, and
|
|
80
|
+
left out the values `0`, `false` and `""`.
|
|
81
|
+
- A truncated last line (after a crash) glued itself to the next event and lost both. A new line is
|
|
82
|
+
now started when a file is opened that does not end with one.
|
|
83
|
+
- `purgeOldEvents` ignored `nowFn` and used the server's local time zone.
|
|
84
|
+
- `dispose()` of an old client no longer silences the client that replaced it.
|
|
85
|
+
- `trackServerError` and the error middleware crashed with a ReferenceError (`ERROR_EVENT` was never
|
|
86
|
+
imported).
|
|
87
|
+
- The log level setting did nothing: the logger was built from the string `"error"` instead of the
|
|
88
|
+
configuration. Levels now work (`debug`, `info`, `warn`, `error`); the default is `warn`.
|
|
89
|
+
- Days with no events no longer log a warning for every missing file.
|
|
90
|
+
- A computed metric with an unknown formula made `generateStats` loop forever; it now throws.
|
|
91
|
+
- `nowAsPstDate` returned a Date with a forced hour and the UTC date, so day buckets were not Pacific
|
|
92
|
+
days. It now holds the real Pacific wall-clock time, with daylight saving. `defaultEndDate` uses
|
|
93
|
+
UTC date maths and follows the configured `nowFn` instead of the real clock.
|
|
94
|
+
- Reading a window of days used local-time date maths, which could skip or repeat a day around
|
|
95
|
+
daylight saving changes on servers not set to UTC.
|
|
96
|
+
- Stats for a window ending today were cached, so the dashboard showed stale numbers all day.
|
|
97
|
+
- The client's `trackError` threw when the error event had no `.error` object, and calling
|
|
98
|
+
`hearald()` twice sent every event twice (the new `dispose()` and replacement fix both).
|
|
99
|
+
- Without a schema, the event endpoint stored events with no name or with a non-object `p`.
|
|
100
|
+
- A `set_of_users` metric that names an `event` now counts only users who sent that event (it used to
|
|
101
|
+
count every user, whatever the event). A `filter` on event parameters already worked.
|
|
102
|
+
- Events with no parameters or no user no longer break metric computation.
|
|
103
|
+
- Open event files are tracked by full path, so changing `eventsDir` cannot reuse a stale file handle.
|
|
104
|
+
|
|
10
105
|
## [0.0.5] - 2025-10-05
|
|
11
106
|
|
|
12
107
|
### Added
|
|
@@ -34,9 +129,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
34
129
|
|
|
35
130
|
- Initial release of Hearald
|
|
36
131
|
|
|
37
|
-
[
|
|
38
|
-
[0.
|
|
39
|
-
[0.0
|
|
132
|
+
[Unreleased]: https://github.com/pariser/hearald/compare/v0.1.1...HEAD
|
|
133
|
+
[0.1.1]: https://github.com/pariser/hearald/compare/v0.1.0...v0.1.1
|
|
134
|
+
[0.1.0]: https://github.com/pariser/hearald/compare/v0.0.5...v0.1.0
|
|
135
|
+
[0.0.5]: https://github.com/pariser/hearald/compare/v0.0.4...v0.0.5
|
|
136
|
+
[0.0.4]: https://github.com/pariser/hearald/compare/v0.0.3...v0.0.4
|
|
137
|
+
[0.0.3]: https://github.com/pariser/hearald/releases/tag/v0.0.3
|
|
40
138
|
|
|
41
139
|
<!--
|
|
42
140
|
`Added` for new features.
|
package/README.md
CHANGED
|
@@ -1,71 +1,145 @@
|
|
|
1
1
|
# hearald
|
|
2
2
|
|
|
3
|
-
A single server drop-in file system based metrics and analytics library
|
|
3
|
+
A single server drop-in file system based metrics and analytics library: your apps send events, they
|
|
4
|
+
are appended to one log file per day, and a small dashboard turns them into counts, unique users and
|
|
5
|
+
histograms. No database.
|
|
4
6
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
## Usage
|
|
7
|
+
```bash
|
|
8
|
+
npm test # unit and integration tests, run against Express 5 and then Express 4
|
|
9
|
+
npm run build # rebuilds dist/ (the dashboard page)
|
|
10
|
+
```
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
## Server
|
|
13
13
|
|
|
14
14
|
```js
|
|
15
|
-
import
|
|
16
|
-
|
|
15
|
+
import express from "express";
|
|
16
|
+
import hearald, { basicAuth } from "hearald";
|
|
17
|
+
|
|
18
|
+
const app = express();
|
|
19
|
+
app.use(express.json());
|
|
20
|
+
|
|
21
|
+
const h = hearald({
|
|
22
|
+
eventsDir: "/var/lib/myapp/events", // default "events", relative to the working directory
|
|
23
|
+
// idleCloseMs: 60000, // how long an event file stays open after its last write
|
|
24
|
+
// maxWindowBytes: 64 * 1024 * 1024, // dashboard windows with more event data than this get a 413
|
|
25
|
+
getUserId: (req) => req.user?.id, // used for server-side errors
|
|
26
|
+
// Strongly recommended for any endpoint open to the internet: only these events and parameters
|
|
27
|
+
// are stored, everything else is dropped (the caller still gets a 204).
|
|
28
|
+
eventEndpoint: {
|
|
29
|
+
url: "/events", // where the route listens, inside wherever you mount eventMiddleware (default "/e")
|
|
30
|
+
schema: {
|
|
31
|
+
app_open: { platform: { type: "string", enum: ["web", "ios"], required: true } },
|
|
32
|
+
purchase: { plan: { type: "string", maxLength: 32 }, cents: { type: "number", integer: true, min: 0 } },
|
|
33
|
+
},
|
|
34
|
+
// parseBody: (req) => ({ e, u, p }), for example to take the user from a session, not the body
|
|
35
|
+
},
|
|
36
|
+
analytics: {
|
|
37
|
+
statDefinitions, // see below
|
|
38
|
+
auth: basicAuth({ verify: (user, pass) => user === "admin" && passwordMatches(pass) }),
|
|
39
|
+
// allowUnauthenticated: true, // local development only
|
|
40
|
+
},
|
|
41
|
+
});
|
|
42
|
+
|
|
43
|
+
app.use(h.eventMiddleware); // POST /events { e, u, p }: the route's own `url`, so mount it at the root
|
|
44
|
+
app.use("/admin/analytics", h.analyticsMiddleware); // the dashboard
|
|
45
|
+
app.use(h.errorMiddlware); // records errors thrown by your routes, then passes them on
|
|
46
|
+
|
|
47
|
+
await h.trackServerEvent({ event: "job_done", userId: "system", params: { count: 3 } });
|
|
48
|
+
await h.trackServerError({ error: new Error("boom"), userId: "u1", extraParams: { route: "/save" } });
|
|
49
|
+
await h.purgeOldEvents({ days: 90 }); // run nightly to keep a retention period
|
|
50
|
+
await h.shutDown(); // closes open files on exit
|
|
17
51
|
```
|
|
18
52
|
|
|
19
|
-
|
|
53
|
+
The dashboard has no default password: `analytics` throws unless you pass `auth`. `basicAuth` answers 401
|
|
54
|
+
with a challenge and locks an IP out (429) after 10 wrong guesses in 15 minutes. Store a hash of the
|
|
55
|
+
password and compare in constant time.
|
|
20
56
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
57
|
+
`url` is the full path of the route (`/events` above), and `eventMiddleware` is a router that serves it, so
|
|
58
|
+
mount it at the root. If you mount it at `app.use("/api", h.eventMiddleware)`, events arrive at
|
|
59
|
+
`/api/events`, and the browser client's `endpoint` must say so.
|
|
60
|
+
|
|
61
|
+
`basicAuth` counts wrong guesses per `req.ip`. That is the visitor's address only if your app sets
|
|
62
|
+
Express's `trust proxy` correctly: behind a reverse proxy without it every visitor shares the proxy's
|
|
63
|
+
address, and with it set too loosely a caller can forge `X-Forwarded-For` to avoid the lockout. If
|
|
64
|
+
`verify` throws, the request gets a 500 and is not counted as a wrong guess.
|
|
65
|
+
|
|
66
|
+
**The event endpoint is public and has no rate limit or size cap of its own.** Anyone who can reach it
|
|
67
|
+
can send events, so fake user ids can inflate "unique users" and a flood can fill the disk. Use `schema`
|
|
68
|
+
(it bounds string lengths and the set of event names, not how many events arrive), keep Express's
|
|
69
|
+
body limit small (`express.json({ limit: "10kb" })`), and put a rate limit (for example
|
|
70
|
+
`express-rate-limit`) in front of it, or in your own route, as part of your hosting setup. Run `purgeOldEvents` too.
|
|
29
71
|
|
|
30
|
-
|
|
72
|
+
Only `express` (>= 4) is a peer dependency; use your app's own. hearald is tested on Express 4 and 5.
|
|
73
|
+
|
|
74
|
+
## Browser
|
|
31
75
|
|
|
32
76
|
```js
|
|
33
|
-
import
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
77
|
+
import hearaldClient from "hearald/client";
|
|
78
|
+
|
|
79
|
+
const h = hearaldClient({
|
|
80
|
+
endpoint: "/events",
|
|
81
|
+
getUserId: () => currentUserId, // used when an event does not name a user
|
|
82
|
+
onError: (err, eventName) => {}, // a failed send never throws; a 404 or 500 from the server counts as failed
|
|
83
|
+
});
|
|
84
|
+
|
|
85
|
+
h.trackEvent({ eventName: "app_open", payload: { platform: "web" } });
|
|
86
|
+
h.trackVisit({ userId: "user-123" }); // adds user agent, referrer, URL and screen size
|
|
87
|
+
window.addEventListener("error", (error) => h.trackError({ error })); // ErrorEvent or Error
|
|
88
|
+
h.dispose(); // stop sending (calling hearaldClient again also replaces the previous one; an old
|
|
89
|
+
// instance's dispose() only stops that instance)
|
|
38
90
|
```
|
|
39
91
|
|
|
40
|
-
|
|
92
|
+
## Stat definitions
|
|
41
93
|
|
|
42
94
|
```js
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
95
|
+
const statDefinitions = {
|
|
96
|
+
raw_metrics: {
|
|
97
|
+
// count events
|
|
98
|
+
"opens": { aggregator: "count", event: "app_open" },
|
|
99
|
+
// the set of users who sent an event, optionally only with certain parameter values
|
|
100
|
+
"users.web": { aggregator: "set_of_users", event: "app_open", filter: { platform: "web" } },
|
|
101
|
+
// [value, count] pairs for one parameter, most common first
|
|
102
|
+
"errors.by_code": { aggregator: "histogram", event: "sync_error", field: "code" },
|
|
103
|
+
// read the visits file instead of the main log
|
|
104
|
+
"visits": { aggregator: "count", event: "visit", source_file_suffix: "visits" },
|
|
105
|
+
},
|
|
106
|
+
computed_metrics: {
|
|
107
|
+
"users.web_count": { formula: "set_count", metrics: ["users.web"] },
|
|
108
|
+
"users.new": { formula: "set_difference", metrics: ["users.week", "users.before"] }, // in the first, not the others
|
|
109
|
+
"conversion": { formula: "percent", metrics: ["buyers", "users"] }, // first / second * 100
|
|
110
|
+
"opens.per_day": { formula: "rate_time_window", metrics: ["opens"] },
|
|
111
|
+
},
|
|
112
|
+
ui: { sections: [{ title: "People", fields: [{ key: "users.web_count", label: "Web users", type: "number" }] }] },
|
|
113
|
+
};
|
|
50
114
|
```
|
|
51
115
|
|
|
52
|
-
|
|
116
|
+
Names with dots become nested objects in the result. `hidden: true` leaves a metric out of the output
|
|
117
|
+
(useful for sets that only feed a computed metric). A computed metric that names a formula that does
|
|
118
|
+
not exist is an error; one whose inputs are missing is simply left out.
|
|
53
119
|
|
|
54
|
-
|
|
55
|
-
Example:
|
|
120
|
+
## Dashboard routes
|
|
56
121
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
122
|
+
Under wherever you mounted `analyticsMiddleware`: `/` (the page), `/layout` (your `ui`), and
|
|
123
|
+
`/data/:days/:endDate?` (for example `/data/7/2026-11-01`: the 7 days ending that day; `endDate`
|
|
124
|
+
defaults to yesterday). `days` is 1 to 366 and `endDate` must be a real calendar date (`2026-02-31` is not),
|
|
125
|
+
otherwise 400. A window whose event files add up to more than `maxWindowBytes` (64 MB by default) is
|
|
126
|
+
refused with 413 `{ "error": "window too large" }`; stats are computed one request at a time, because a
|
|
127
|
+
window is held in memory while it is counted. Windows that end today or later are computed on every
|
|
128
|
+
request; finished windows are cached in `eventsDir` as `summary:<end date>:<days>:<hash>.json`, where the
|
|
129
|
+
hash covers your metric definitions, so changing a definition recomputes instead of serving old numbers.
|
|
130
|
+
Cached summaries are safe to delete at any time.
|
|
61
131
|
|
|
62
|
-
|
|
132
|
+
## Days and time zones
|
|
63
133
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
134
|
+
Events are bucketed into days with `nowFn` (default: the current Pacific wall-clock time, with
|
|
135
|
+
daylight saving) and `isoFn` (the UTC date of what `nowFn` returns). Pass `nowFn: () => new Date()` to
|
|
136
|
+
bucket by UTC days instead. Nothing depends on the server's own time zone.
|
|
137
|
+
|
|
138
|
+
With the default `nowFn`, the `t` stored in each event is also the Pacific wall-clock time, written with a
|
|
139
|
+
`Z` as if it were UTC (so noon UTC is stored as 05:00 or 04:00, and two instants an hour apart at the
|
|
140
|
+
November clock change get the same `t`). It is right for choosing the day, wrong as an exact instant. If
|
|
141
|
+
you need real instants in the logs, pass `nowFn: () => new Date()` and accept UTC days.
|
|
68
142
|
|
|
69
143
|
## Roadmap
|
|
70
144
|
|
|
71
|
-
See TODO.md
|
|
145
|
+
See TODO.md.
|
package/dist/analytics.html
CHANGED
|
@@ -461,7 +461,7 @@ th {
|
|
|
461
461
|
onLoadProgress(i, n);
|
|
462
462
|
|
|
463
463
|
const json = await getDataFn({ n: 1, d: iso(day) });
|
|
464
|
-
const v = json
|
|
464
|
+
const v = json?.metrics?.[context]?.[field] || 0;
|
|
465
465
|
|
|
466
466
|
data.unshift({ date: iso(day), value: v });
|
|
467
467
|
max = Math.max(max, v);
|
|
@@ -687,14 +687,14 @@ th {
|
|
|
687
687
|
fetchStats: getJson,
|
|
688
688
|
});
|
|
689
689
|
|
|
690
|
-
let UI =
|
|
690
|
+
let UI = {};
|
|
691
691
|
|
|
692
692
|
async function fetchUI() {
|
|
693
693
|
try {
|
|
694
694
|
const res = await fetch(window.location.href + "/layout");
|
|
695
|
-
UI = await res.json();
|
|
695
|
+
UI = (await res.json()) || {};
|
|
696
696
|
} catch (e) {
|
|
697
|
-
UI =
|
|
697
|
+
UI = {};
|
|
698
698
|
}
|
|
699
699
|
}
|
|
700
700
|
|
|
@@ -830,7 +830,7 @@ th {
|
|
|
830
830
|
name="context"
|
|
831
831
|
onchange=${(e) => this.changeChartContext(e)}
|
|
832
832
|
>
|
|
833
|
-
${UI.chartContexts.map(
|
|
833
|
+
${(UI.chartContexts || []).map(
|
|
834
834
|
({ id, menuName }) =>
|
|
835
835
|
m`
|
|
836
836
|
<option
|
|
@@ -899,7 +899,7 @@ th {
|
|
|
899
899
|
`}
|
|
900
900
|
</div>
|
|
901
901
|
</section>
|
|
902
|
-
${UI.sections.map((statsSection) => {
|
|
902
|
+
${(UI.sections || []).map((statsSection) => {
|
|
903
903
|
return m`
|
|
904
904
|
<section class="analytics analytics--${statsSection.id}">
|
|
905
905
|
${statsSection.menuName !== "" &&
|
package/dist/index.js
CHANGED
|
@@ -63,7 +63,7 @@
|
|
|
63
63
|
onLoadProgress(i, n);
|
|
64
64
|
|
|
65
65
|
const json = await getDataFn({ n: 1, d: iso(day) });
|
|
66
|
-
const v = json
|
|
66
|
+
const v = json?.metrics?.[context]?.[field] || 0;
|
|
67
67
|
|
|
68
68
|
data.unshift({ date: iso(day), value: v });
|
|
69
69
|
max = Math.max(max, v);
|
|
@@ -289,14 +289,14 @@
|
|
|
289
289
|
fetchStats: getJson,
|
|
290
290
|
});
|
|
291
291
|
|
|
292
|
-
let UI =
|
|
292
|
+
let UI = {};
|
|
293
293
|
|
|
294
294
|
async function fetchUI() {
|
|
295
295
|
try {
|
|
296
296
|
const res = await fetch(window.location.href + "/layout");
|
|
297
|
-
UI = await res.json();
|
|
297
|
+
UI = (await res.json()) || {};
|
|
298
298
|
} catch (e) {
|
|
299
|
-
UI =
|
|
299
|
+
UI = {};
|
|
300
300
|
}
|
|
301
301
|
}
|
|
302
302
|
|
|
@@ -432,7 +432,7 @@
|
|
|
432
432
|
name="context"
|
|
433
433
|
onchange=${(e) => this.changeChartContext(e)}
|
|
434
434
|
>
|
|
435
|
-
${UI.chartContexts.map(
|
|
435
|
+
${(UI.chartContexts || []).map(
|
|
436
436
|
({ id, menuName }) =>
|
|
437
437
|
m`
|
|
438
438
|
<option
|
|
@@ -501,7 +501,7 @@
|
|
|
501
501
|
`}
|
|
502
502
|
</div>
|
|
503
503
|
</section>
|
|
504
|
-
${UI.sections.map((statsSection) => {
|
|
504
|
+
${(UI.sections || []).map((statsSection) => {
|
|
505
505
|
return m`
|
|
506
506
|
<section class="analytics analytics--${statsSection.id}">
|
|
507
507
|
${statsSection.menuName !== "" &&
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "hearald",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.1.1",
|
|
4
4
|
"description": "A single server drop-in file system based metrics and analytics library.",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"exports": {
|
|
@@ -9,7 +9,10 @@
|
|
|
9
9
|
},
|
|
10
10
|
"type": "module",
|
|
11
11
|
"scripts": {
|
|
12
|
-
"build": "rollup --config"
|
|
12
|
+
"build": "rollup --config",
|
|
13
|
+
"test": "npm run test:express5 && npm run test:express4",
|
|
14
|
+
"test:express5": "HEARALD_EXPRESS=5 node --import ./test-support/register.js --test",
|
|
15
|
+
"test:express4": "HEARALD_EXPRESS=4 node --import ./test-support/register.js --test"
|
|
13
16
|
},
|
|
14
17
|
"keywords": [
|
|
15
18
|
"herald",
|
|
@@ -21,10 +24,10 @@
|
|
|
21
24
|
"license": "MIT",
|
|
22
25
|
"dependencies": {
|
|
23
26
|
"bussin-js": "^1.0.1",
|
|
24
|
-
"chalk": "^4.1.0"
|
|
25
|
-
"express": "^4.17.1"
|
|
27
|
+
"chalk": "^4.1.0"
|
|
26
28
|
},
|
|
27
29
|
"devDependencies": {
|
|
30
|
+
"@happy-dom/global-registrator": "^20.14.6",
|
|
28
31
|
"@rollup/plugin-commonjs": "^15.1.0",
|
|
29
32
|
"@rollup/plugin-json": "^4.1.0",
|
|
30
33
|
"@rollup/plugin-node-resolve": "^13.3.0",
|
|
@@ -35,6 +38,9 @@
|
|
|
35
38
|
"eslint-plugin-jsx-a11y": "^6.6.1",
|
|
36
39
|
"eslint-plugin-react": "^7.31.10",
|
|
37
40
|
"eslint-plugin-react-hooks": "^4.6.0",
|
|
41
|
+
"express": "^5.2.1",
|
|
42
|
+
"express4": "npm:express@^4.21.2",
|
|
43
|
+
"happy-dom": "^20.14.6",
|
|
38
44
|
"htm": "^3.0.3",
|
|
39
45
|
"preact": "^10.3.3",
|
|
40
46
|
"prettier": "2.2.1",
|
|
@@ -45,5 +51,15 @@
|
|
|
45
51
|
"rollup-plugin-postcss": "^4.0.2",
|
|
46
52
|
"rollup-plugin-terser": "^7.0.2",
|
|
47
53
|
"style-inject": "^0.3.0"
|
|
48
|
-
}
|
|
54
|
+
},
|
|
55
|
+
"peerDependencies": {
|
|
56
|
+
"express": ">=4"
|
|
57
|
+
},
|
|
58
|
+
"files": [
|
|
59
|
+
"src",
|
|
60
|
+
"dist",
|
|
61
|
+
"README.md",
|
|
62
|
+
"CHANGELOG.md",
|
|
63
|
+
"LICENSE"
|
|
64
|
+
]
|
|
49
65
|
}
|
package/src/client.js
CHANGED
|
@@ -20,8 +20,10 @@ const trackError = ({
|
|
|
20
20
|
userId = null,
|
|
21
21
|
payload: extraPayload = {},
|
|
22
22
|
} = {}) => {
|
|
23
|
-
|
|
24
|
-
|
|
23
|
+
// `error` is normally the ErrorEvent from window.onerror, whose `.error` can be null (for example
|
|
24
|
+
// for errors from other origins), or a plain Error.
|
|
25
|
+
const errorEvent = error || {};
|
|
26
|
+
const errorObject = errorEvent.error || (error instanceof Error ? error : {});
|
|
25
27
|
|
|
26
28
|
let source;
|
|
27
29
|
if (errorObject.stack) {
|
|
@@ -35,7 +37,7 @@ const trackError = ({
|
|
|
35
37
|
eventName: "error",
|
|
36
38
|
userId,
|
|
37
39
|
payload: {
|
|
38
|
-
message: errorObject.message,
|
|
40
|
+
message: errorObject.message || errorEvent.message || String(error),
|
|
39
41
|
source,
|
|
40
42
|
file_name: errorEvent.filename || errorObject.fileName,
|
|
41
43
|
line_number: errorEvent.lineno || errorObject.lineNumber,
|
|
@@ -73,30 +75,48 @@ function emitEventToServer({
|
|
|
73
75
|
});
|
|
74
76
|
}
|
|
75
77
|
|
|
78
|
+
let activeListener = null;
|
|
79
|
+
|
|
76
80
|
export default function hearald({
|
|
77
81
|
endpoint = "/e",
|
|
78
|
-
fetchImpl = fetch,
|
|
82
|
+
fetchImpl = (...args) => globalThis.fetch(...args),
|
|
79
83
|
getUserId = () => null,
|
|
80
84
|
onError = () => {} /* err, eventName, userId, payload */,
|
|
81
85
|
} = {}) {
|
|
82
|
-
|
|
86
|
+
// The event bus is shared, so a second call (a hot reload, say) replaces the first listener
|
|
87
|
+
// instead of sending every event twice.
|
|
88
|
+
if (activeListener) eventBus.off("event", activeListener);
|
|
89
|
+
const listener = async (event) => {
|
|
83
90
|
const { eventName, userId: emittedUserId, payload } = event;
|
|
84
91
|
const userId = emittedUserId || getUserId() || null;
|
|
85
92
|
|
|
86
93
|
try {
|
|
87
|
-
await emitEventToServer({
|
|
94
|
+
const response = await emitEventToServer({
|
|
88
95
|
endpoint,
|
|
89
96
|
eventName,
|
|
90
97
|
userId,
|
|
91
98
|
payload,
|
|
92
99
|
fetchImpl,
|
|
93
100
|
});
|
|
101
|
+
// fetch only rejects on network failure; a 404 or 500 arrives as an ordinary response
|
|
102
|
+
if (response && response.ok === false) {
|
|
103
|
+
throw new Error(
|
|
104
|
+
`hearald: the server answered ${response.status} for "${eventName}"`
|
|
105
|
+
);
|
|
106
|
+
}
|
|
94
107
|
} catch (err) {
|
|
95
108
|
onError(err, eventName, userId, payload);
|
|
96
109
|
}
|
|
97
|
-
}
|
|
110
|
+
};
|
|
111
|
+
activeListener = listener;
|
|
112
|
+
eventBus.on("event", listener);
|
|
98
113
|
|
|
99
114
|
return {
|
|
115
|
+
// Removes only this instance's listener: an older instance's dispose() must not silence a newer one.
|
|
116
|
+
dispose() {
|
|
117
|
+
eventBus.off("event", listener);
|
|
118
|
+
if (activeListener === listener) activeListener = null;
|
|
119
|
+
},
|
|
100
120
|
eventBus,
|
|
101
121
|
trackEvent,
|
|
102
122
|
trackError,
|
package/src/index.js
CHANGED
|
@@ -8,15 +8,27 @@ import {
|
|
|
8
8
|
shutDown,
|
|
9
9
|
} from "./server/server.js";
|
|
10
10
|
import { createAnalyticsRouter } from "./server/analytics.js";
|
|
11
|
+
import { purgeOldEvents } from "./server/serverEvents.js";
|
|
12
|
+
import { basicAuth } from "./server/auth.js";
|
|
13
|
+
|
|
14
|
+
export { basicAuth, purgeOldEvents };
|
|
11
15
|
|
|
12
16
|
export default function hearald({
|
|
13
17
|
/* singleton configuration options */
|
|
14
18
|
logLevel,
|
|
15
19
|
nowFn /* () => nowAsDateString */,
|
|
16
20
|
getUserId /* (req) => userId */,
|
|
21
|
+
eventsDir /* where event logs are kept; default "events" */,
|
|
22
|
+
idleCloseMs /* how long an event file stays open after its last write; default 60000 */,
|
|
23
|
+
maxWindowBytes /* the most event-log bytes one dashboard request may read; default 64 MB */,
|
|
17
24
|
|
|
18
|
-
eventEndpoint: { url = "/e", parseBody } = {},
|
|
19
|
-
analytics: {
|
|
25
|
+
eventEndpoint: { url = "/e", parseBody, schema } = {},
|
|
26
|
+
analytics: {
|
|
27
|
+
enabled: analyticsEnabled = true,
|
|
28
|
+
statDefinitions,
|
|
29
|
+
auth,
|
|
30
|
+
allowUnauthenticated,
|
|
31
|
+
} = {},
|
|
20
32
|
} = {}) {
|
|
21
33
|
if (logLevel) {
|
|
22
34
|
hearaldConfiguration.setLogLevel(logLevel);
|
|
@@ -27,12 +39,23 @@ export default function hearald({
|
|
|
27
39
|
if (getUserId) {
|
|
28
40
|
hearaldConfiguration.setGetUserId(getUserId);
|
|
29
41
|
}
|
|
42
|
+
if (eventsDir) {
|
|
43
|
+
hearaldConfiguration.setEventsDir(eventsDir);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
if (idleCloseMs !== undefined) {
|
|
47
|
+
hearaldConfiguration.setIdleCloseMs(idleCloseMs);
|
|
48
|
+
}
|
|
49
|
+
if (maxWindowBytes !== undefined) {
|
|
50
|
+
hearaldConfiguration.setMaxWindowBytes(maxWindowBytes);
|
|
51
|
+
}
|
|
30
52
|
|
|
31
|
-
const eventMiddleware = eventEndpointMiddleware({ url, parseBody });
|
|
53
|
+
const eventMiddleware = eventEndpointMiddleware({ url, parseBody, schema });
|
|
32
54
|
|
|
33
|
-
const analyticsMiddleware =
|
|
34
|
-
|
|
35
|
-
|
|
55
|
+
const analyticsMiddleware =
|
|
56
|
+
analyticsEnabled && statDefinitions
|
|
57
|
+
? createAnalyticsRouter({ statDefinitions, auth, allowUnauthenticated })
|
|
58
|
+
: null;
|
|
36
59
|
|
|
37
60
|
const errorMiddlware = errorTrackingMiddleware({});
|
|
38
61
|
|
|
@@ -42,6 +65,7 @@ export default function hearald({
|
|
|
42
65
|
analyticsMiddleware /* app.use('/analytics', analyticsMiddleware) */,
|
|
43
66
|
trackServerEvent /* ({ event, time = null, userId, params = {} }) => Promise } */,
|
|
44
67
|
trackServerError /* ({ error, userId = null, extraParams = {} }) => Promise } */,
|
|
68
|
+
purgeOldEvents /* ({ days }) => Promise<number of files removed> */,
|
|
45
69
|
shutDown,
|
|
46
70
|
};
|
|
47
71
|
}
|
package/src/public/js/App.js
CHANGED
|
@@ -75,14 +75,14 @@ const getDataFn = ({ n, d, q }) =>
|
|
|
75
75
|
fetchStats: getJson,
|
|
76
76
|
});
|
|
77
77
|
|
|
78
|
-
let UI =
|
|
78
|
+
let UI = {};
|
|
79
79
|
|
|
80
80
|
async function fetchUI() {
|
|
81
81
|
try {
|
|
82
82
|
const res = await fetch(window.location.href + "/layout");
|
|
83
|
-
UI = await res.json();
|
|
83
|
+
UI = (await res.json()) || {};
|
|
84
84
|
} catch (e) {
|
|
85
|
-
UI =
|
|
85
|
+
UI = {};
|
|
86
86
|
}
|
|
87
87
|
}
|
|
88
88
|
|
|
@@ -218,7 +218,7 @@ class App extends Component {
|
|
|
218
218
|
name="context"
|
|
219
219
|
onchange=${(e) => this.changeChartContext(e)}
|
|
220
220
|
>
|
|
221
|
-
${UI.chartContexts.map(
|
|
221
|
+
${(UI.chartContexts || []).map(
|
|
222
222
|
({ id, menuName }) =>
|
|
223
223
|
html`
|
|
224
224
|
<option
|
|
@@ -287,7 +287,7 @@ class App extends Component {
|
|
|
287
287
|
`}
|
|
288
288
|
</div>
|
|
289
289
|
</section>
|
|
290
|
-
${UI.sections.map((statsSection) => {
|
|
290
|
+
${(UI.sections || []).map((statsSection) => {
|
|
291
291
|
return html`
|
|
292
292
|
<section class="analytics analytics--${statsSection.id}">
|
|
293
293
|
${statsSection.menuName !== "" &&
|
package/src/public/js/util.js
CHANGED
|
@@ -48,7 +48,7 @@ export async function fetchChartData({
|
|
|
48
48
|
onLoadProgress(i, n);
|
|
49
49
|
|
|
50
50
|
const json = await getDataFn({ n: 1, d: iso(day) });
|
|
51
|
-
const v = json
|
|
51
|
+
const v = json?.metrics?.[context]?.[field] || 0;
|
|
52
52
|
|
|
53
53
|
data.unshift({ date: iso(day), value: v });
|
|
54
54
|
max = Math.max(max, v);
|