@walkeros/server-destination-aws 4.1.0-next-1778668930820 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +223 -0
- package/README.md +15 -195
- package/dist/dev.js +1 -1
- package/dist/dev.js.map +1 -1
- package/dist/dev.mjs +1 -1
- package/dist/dev.mjs.map +1 -1
- package/dist/examples/index.js +11 -47
- package/dist/examples/index.mjs +11 -47
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +1 -1
- package/dist/index.mjs.map +1 -1
- package/dist/schemas.js +1 -1
- package/dist/schemas.js.map +1 -1
- package/dist/schemas.mjs +1 -1
- package/dist/schemas.mjs.map +1 -1
- package/dist/walkerOS.json +5 -3
- package/package.json +5 -3
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
# @walkeros/server-destination-aws
|
|
2
|
+
|
|
3
|
+
## 4.1.0
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- Updated dependencies [e155ff8]
|
|
8
|
+
- Updated dependencies [e800974]
|
|
9
|
+
- Updated dependencies [e155ff8]
|
|
10
|
+
- Updated dependencies [1a8f2d7]
|
|
11
|
+
- Updated dependencies [1a8f2d7]
|
|
12
|
+
- Updated dependencies [b276173]
|
|
13
|
+
- Updated dependencies [dd9f5ad]
|
|
14
|
+
- Updated dependencies [c60ef35]
|
|
15
|
+
- Updated dependencies [adeebea]
|
|
16
|
+
- Updated dependencies [13aaeaa]
|
|
17
|
+
- Updated dependencies [e800974]
|
|
18
|
+
- Updated dependencies [adeebea]
|
|
19
|
+
- Updated dependencies [e800974]
|
|
20
|
+
- Updated dependencies [e800974]
|
|
21
|
+
- Updated dependencies [058f7ed]
|
|
22
|
+
- Updated dependencies [28a8ac2]
|
|
23
|
+
- Updated dependencies [fd6076e]
|
|
24
|
+
- @walkeros/core@4.1.0
|
|
25
|
+
- @walkeros/server-core@4.1.0
|
|
26
|
+
|
|
27
|
+
## 4.0.2
|
|
28
|
+
|
|
29
|
+
### Patch Changes
|
|
30
|
+
|
|
31
|
+
- @walkeros/server-core@4.0.2
|
|
32
|
+
|
|
33
|
+
## 4.0.1
|
|
34
|
+
|
|
35
|
+
### Patch Changes
|
|
36
|
+
|
|
37
|
+
- @walkeros/server-core@4.0.1
|
|
38
|
+
|
|
39
|
+
## 4.0.0
|
|
40
|
+
|
|
41
|
+
### Major Changes
|
|
42
|
+
|
|
43
|
+
- 93ea9c4: Event model v4: breaking changes to the `Event`, `Source`, and
|
|
44
|
+
`Entity` shapes.
|
|
45
|
+
- `event.id` is now a W3C span_id (16 lowercase hex chars), generated by the
|
|
46
|
+
collector. Reference: W3C Trace Context (W3C Recommendation, January 2020).
|
|
47
|
+
- `event.version`, `event.group`, `event.count` are removed.
|
|
48
|
+
- `source.type` is now the source kind (e.g. `browser`, `gtag`, `mcp`, `cli`).
|
|
49
|
+
New `source.platform` holds the runtime (`web` | `server` | `app` | ...).
|
|
50
|
+
- `source.id` and `source.previous_id` are removed.
|
|
51
|
+
- Browser source now sets `source.url` and `source.referrer`.
|
|
52
|
+
- MCP source sets `source.tool` per emission. CLI source sets
|
|
53
|
+
`source.command`.
|
|
54
|
+
- `Entity.nested` and `Entity.context` are now optional. Root `event.nested`
|
|
55
|
+
and `event.context` remain required.
|
|
56
|
+
- Each source self-registers via TypeScript module augmentation of `SourceMap`
|
|
57
|
+
in `@walkeros/core`.
|
|
58
|
+
- App-side coordination (`/workspaces/developer/app`) is a follow-up plan, not
|
|
59
|
+
part of this release. Telemetry from v4 CLI/MCP will not validate against
|
|
60
|
+
the existing app schema until that follow-up ships.
|
|
61
|
+
- `Mapping.Rule.skip` is renamed to `Mapping.Rule.silent`. Customer flow.json
|
|
62
|
+
configs using `skip: true` in mapping rules must rename to `silent: true`.
|
|
63
|
+
Hard cut: no legacy alias, the field is gone.
|
|
64
|
+
|
|
65
|
+
### Patch Changes
|
|
66
|
+
|
|
67
|
+
- @walkeros/server-core@4.0.0
|
|
68
|
+
|
|
69
|
+
## 3.4.2
|
|
70
|
+
|
|
71
|
+
### Patch Changes
|
|
72
|
+
|
|
73
|
+
- @walkeros/server-core@3.4.2
|
|
74
|
+
|
|
75
|
+
## 3.4.1
|
|
76
|
+
|
|
77
|
+
### Patch Changes
|
|
78
|
+
|
|
79
|
+
- @walkeros/server-core@3.4.1
|
|
80
|
+
|
|
81
|
+
## 3.4.0
|
|
82
|
+
|
|
83
|
+
### Minor Changes
|
|
84
|
+
|
|
85
|
+
- 724f97e: Migrate every step example in every walkerOS package to the
|
|
86
|
+
standardized `[callable, ...args][]` shape introduced in `@walkeros/core`.
|
|
87
|
+
Every step example's `out` is now an array of effect tuples whose first
|
|
88
|
+
element is the callable's public SDK name (`'gtag'`, `'analytics.track'`,
|
|
89
|
+
`'fbq'`, `'dataLayer.push'`, `'sendServer'`, `'fetch'`, `'trackClient.track'`,
|
|
90
|
+
`'amplitude.track'`, `'fs.writeFile'`, `'producer.send'`, `'client.xadd'`,
|
|
91
|
+
`'client.send'`, `'dataset.table.insert'`, etc.). Source examples use `'elb'`
|
|
92
|
+
as the callable; transformer examples use the reserved `'return'` keyword;
|
|
93
|
+
store examples use store-operation callables (`'get'`, `'set'`). Tests capture
|
|
94
|
+
real calls on each component's spy and assert against `example.out` directly —
|
|
95
|
+
the hardcoded `PACKAGE_CALLS` registry in the app is no longer consulted
|
|
96
|
+
(emptied; plan #3 removes it structurally).
|
|
97
|
+
|
|
98
|
+
### Patch Changes
|
|
99
|
+
|
|
100
|
+
- @walkeros/server-core@3.4.0
|
|
101
|
+
|
|
102
|
+
## 3.3.1
|
|
103
|
+
|
|
104
|
+
### Patch Changes
|
|
105
|
+
|
|
106
|
+
- @walkeros/server-core@3.3.1
|
|
107
|
+
|
|
108
|
+
## 3.3.0
|
|
109
|
+
|
|
110
|
+
### Patch Changes
|
|
111
|
+
|
|
112
|
+
- @walkeros/server-core@3.3.0
|
|
113
|
+
|
|
114
|
+
## 3.2.0
|
|
115
|
+
|
|
116
|
+
### Patch Changes
|
|
117
|
+
|
|
118
|
+
- @walkeros/server-core@3.2.0
|
|
119
|
+
|
|
120
|
+
## 3.1.1
|
|
121
|
+
|
|
122
|
+
### Patch Changes
|
|
123
|
+
|
|
124
|
+
- @walkeros/server-core@3.1.1
|
|
125
|
+
|
|
126
|
+
## 3.1.0
|
|
127
|
+
|
|
128
|
+
### Patch Changes
|
|
129
|
+
|
|
130
|
+
- @walkeros/server-core@3.1.0
|
|
131
|
+
|
|
132
|
+
## 3.0.2
|
|
133
|
+
|
|
134
|
+
### Patch Changes
|
|
135
|
+
|
|
136
|
+
- @walkeros/server-core@3.0.2
|
|
137
|
+
|
|
138
|
+
## 3.0.1
|
|
139
|
+
|
|
140
|
+
### Patch Changes
|
|
141
|
+
|
|
142
|
+
- 86c81d1: Add default export to server-transformer-fingerprint and exports
|
|
143
|
+
metadata to multi-service packages for bundler named import support
|
|
144
|
+
- @walkeros/server-core@3.0.1
|
|
145
|
+
|
|
146
|
+
## 3.0.0
|
|
147
|
+
|
|
148
|
+
### Patch Changes
|
|
149
|
+
|
|
150
|
+
- 499e27a: Add sideEffects declarations to all packages for bundler tree-shaking
|
|
151
|
+
support.
|
|
152
|
+
- Updated dependencies [499e27a]
|
|
153
|
+
- @walkeros/server-core@3.0.0
|
|
154
|
+
|
|
155
|
+
## 2.1.1
|
|
156
|
+
|
|
157
|
+
### Patch Changes
|
|
158
|
+
|
|
159
|
+
- @walkeros/server-core@2.1.1
|
|
160
|
+
|
|
161
|
+
## 2.1.0
|
|
162
|
+
|
|
163
|
+
### Minor Changes
|
|
164
|
+
|
|
165
|
+
- 97df0b2: Step examples: upgrade all packages to blueprint pattern with inline
|
|
166
|
+
mapping, no intermediate variables, no `all` export
|
|
167
|
+
|
|
168
|
+
### Patch Changes
|
|
169
|
+
|
|
170
|
+
- @walkeros/server-core@2.1.0
|
|
171
|
+
|
|
172
|
+
## 2.0.1
|
|
173
|
+
|
|
174
|
+
## 1.0.6
|
|
175
|
+
|
|
176
|
+
### Patch Changes
|
|
177
|
+
|
|
178
|
+
- @walkeros/server-core@2.0.0
|
|
179
|
+
|
|
180
|
+
## 1.0.5
|
|
181
|
+
|
|
182
|
+
### Patch Changes
|
|
183
|
+
|
|
184
|
+
- @walkeros/server-core@1.0.5
|
|
185
|
+
|
|
186
|
+
## 1.0.4
|
|
187
|
+
|
|
188
|
+
### Patch Changes
|
|
189
|
+
|
|
190
|
+
- @walkeros/server-core@1.0.4
|
|
191
|
+
|
|
192
|
+
## 1.0.3
|
|
193
|
+
|
|
194
|
+
### Patch Changes
|
|
195
|
+
|
|
196
|
+
- @walkeros/server-core@1.0.3
|
|
197
|
+
|
|
198
|
+
## 1.0.2
|
|
199
|
+
|
|
200
|
+
### Patch Changes
|
|
201
|
+
|
|
202
|
+
- 6778ab2: Add default exports for simpler CLI flow.json configuration
|
|
203
|
+
- @walkeros/server-core@1.0.2
|
|
204
|
+
|
|
205
|
+
## 1.0.1
|
|
206
|
+
|
|
207
|
+
### Patch Changes
|
|
208
|
+
|
|
209
|
+
- @walkeros/server-core@1.0.1
|
|
210
|
+
|
|
211
|
+
## 1.0.0
|
|
212
|
+
|
|
213
|
+
### Major Changes
|
|
214
|
+
|
|
215
|
+
- 67c9e1d: Hello World! walkerOS v1.0.0
|
|
216
|
+
|
|
217
|
+
Open-source event data collection. Collect event data for digital analytics in
|
|
218
|
+
a unified and privacy-centric way.
|
|
219
|
+
|
|
220
|
+
### Patch Changes
|
|
221
|
+
|
|
222
|
+
- Updated dependencies [67c9e1d]
|
|
223
|
+
- @walkeros/server-core@1.0.0
|
package/README.md
CHANGED
|
@@ -4,221 +4,41 @@
|
|
|
4
4
|
</a>
|
|
5
5
|
</p>
|
|
6
6
|
|
|
7
|
-
#
|
|
7
|
+
# @walkeros/server-destination-aws
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
•
|
|
11
|
-
[NPM Package](https://www.npmjs.com/package/@walkeros/server-destination-aws)
|
|
12
|
-
|
|
13
|
-
walkerOS follows a **source → collector → destination** architecture. This AWS
|
|
14
|
-
destination receives processed events from the walkerOS collector and forwards
|
|
15
|
-
them to AWS services. Two sub-destinations ship in the package:
|
|
9
|
+
Stream events to Amazon Kinesis Data Firehose for real-time analytics.
|
|
16
10
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
and declared subscriptions.
|
|
11
|
+
[Documentation](https://www.walkeros.io/docs/destinations/server/aws) •
|
|
12
|
+
[NPM Package](https://www.npmjs.com/package/@walkeros/server-destination-aws)
|
|
13
|
+
•
|
|
14
|
+
[Source Code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/aws)
|
|
22
15
|
|
|
23
16
|
## Installation
|
|
24
17
|
|
|
25
|
-
```
|
|
18
|
+
```bash
|
|
26
19
|
npm install @walkeros/server-destination-aws
|
|
27
20
|
```
|
|
28
21
|
|
|
29
|
-
## Quick
|
|
30
|
-
|
|
31
|
-
Configure in your Flow JSON:
|
|
22
|
+
## Quick start
|
|
32
23
|
|
|
33
24
|
```json
|
|
34
25
|
{
|
|
35
|
-
"version":
|
|
26
|
+
"version": 4,
|
|
36
27
|
"flows": {
|
|
37
28
|
"default": {
|
|
38
|
-
"
|
|
29
|
+
"config": { "platform": "server" },
|
|
39
30
|
"destinations": {
|
|
40
|
-
"
|
|
41
|
-
"package": "@walkeros/server-destination-aws",
|
|
42
|
-
"config": {
|
|
43
|
-
"settings": {
|
|
44
|
-
"firehose": {
|
|
45
|
-
"streamName": "your-firehose-stream-name",
|
|
46
|
-
"region": "eu-central-1"
|
|
47
|
-
}
|
|
48
|
-
}
|
|
49
|
-
}
|
|
50
|
-
}
|
|
31
|
+
"aws": { "package": "@walkeros/server-destination-aws", "config": {} }
|
|
51
32
|
}
|
|
52
33
|
}
|
|
53
34
|
}
|
|
54
35
|
}
|
|
55
36
|
```
|
|
56
37
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
```typescript
|
|
60
|
-
import { startFlow } from '@walkeros/collector';
|
|
61
|
-
import { destinationFirehose } from '@walkeros/server-destination-aws';
|
|
62
|
-
|
|
63
|
-
const { elb } = await startFlow({
|
|
64
|
-
destinations: [
|
|
65
|
-
{
|
|
66
|
-
destination: destinationFirehose,
|
|
67
|
-
config: {
|
|
68
|
-
settings: {
|
|
69
|
-
firehose: {
|
|
70
|
-
streamName: 'your-firehose-stream-name',
|
|
71
|
-
region: 'eu-central-1',
|
|
72
|
-
config: {
|
|
73
|
-
credentials: {
|
|
74
|
-
accessKeyId: 'your-access-key-id',
|
|
75
|
-
secretAccessKey: 'your-secret-access-key',
|
|
76
|
-
},
|
|
77
|
-
},
|
|
78
|
-
},
|
|
79
|
-
},
|
|
80
|
-
},
|
|
81
|
-
},
|
|
82
|
-
],
|
|
83
|
-
});
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
## Configuration
|
|
87
|
-
|
|
88
|
-
| Name | Type | Description | Required | Example |
|
|
89
|
-
| ---------- | ---------------- | ----------------------------------- | -------- | ------------------------------------------------------ |
|
|
90
|
-
| `firehose` | `FirehoseConfig` | AWS Firehose configuration settings | No | `{ streamName: 'walker-events', region: 'us-east-1' }` |
|
|
91
|
-
|
|
92
|
-
### Firehose Configuration
|
|
93
|
-
|
|
94
|
-
The `firehose` object has the following properties:
|
|
95
|
-
|
|
96
|
-
| Name | Type | Description | Required | Example |
|
|
97
|
-
| ------------ | ---------------------- | ------------------------------------------------- | -------- | --------------------------------- |
|
|
98
|
-
| `streamName` | `string` | Name of the Kinesis Data Firehose delivery stream | Yes | `'walker-events'` |
|
|
99
|
-
| `client` | `FirehoseClient` | Pre-configured AWS Firehose client instance | No | `new FirehoseClient(config)` |
|
|
100
|
-
| `region` | `string` | AWS region for the Firehose service | No | `'us-east-1'` |
|
|
101
|
-
| `config` | `FirehoseClientConfig` | AWS SDK client configuration options | No | `{ credentials: awsCredentials }` |
|
|
102
|
-
|
|
103
|
-
## SNS
|
|
104
|
-
|
|
105
|
-
Publish events to AWS SNS topics with idempotent topic provisioning, FIFO
|
|
106
|
-
ordering, per-event message attributes, and declared subscription fan-out.
|
|
107
|
-
Import the named export:
|
|
108
|
-
|
|
109
|
-
```typescript
|
|
110
|
-
import { destinationSNS } from '@walkeros/server-destination-aws';
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
### Required settings
|
|
114
|
-
|
|
115
|
-
| Name | Type | Required | Description |
|
|
116
|
-
| ----------- | -------- | -------- | -------------------------------------------------------------- |
|
|
117
|
-
| `topicName` | `string` | Yes | Topic name (without `.fifo` suffix unless `fifoTopic: true`). |
|
|
118
|
-
| `region` | `string` | No | AWS region. Default `eu-central-1`. |
|
|
119
|
-
| `topicArn` | `string` | No | Pre-resolved ARN. When set, init() skips its CreateTopic call. |
|
|
120
|
-
|
|
121
|
-
### Setup command
|
|
122
|
-
|
|
123
|
-
Provision the topic, attributes, tags, and declared subscriptions in one shot:
|
|
124
|
-
|
|
125
|
-
```bash
|
|
126
|
-
walkeros setup destination.sns -c flow.json
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
Setup is **authoritative-apply**: declared state is written to declared
|
|
130
|
-
resources via a single idempotent `CreateTopic` call (plus one `Subscribe` call
|
|
131
|
-
per declared subscription). Non-declared subscriptions and tags on the topic are
|
|
132
|
-
left untouched, never listed, never logged. Operators may freely manage
|
|
133
|
-
subscriptions or tags outside walkerOS without interference.
|
|
134
|
-
|
|
135
|
-
The result is JSON-stringified to stdout:
|
|
136
|
-
|
|
137
|
-
```json
|
|
138
|
-
{
|
|
139
|
-
"topicArn": "arn:aws:sns:eu-central-1:000000000000:walkeros-events",
|
|
140
|
-
"topicCreated": true,
|
|
141
|
-
"tagsApplied": 2,
|
|
142
|
-
"subscriptionsCreated": 1
|
|
143
|
-
}
|
|
144
|
-
```
|
|
145
|
-
|
|
146
|
-
### Example flow.json
|
|
147
|
-
|
|
148
|
-
```jsonc
|
|
149
|
-
{
|
|
150
|
-
"destinations": {
|
|
151
|
-
"sns": {
|
|
152
|
-
"package": "@walkeros/server-destination-aws",
|
|
153
|
-
"code": "destinationSNS",
|
|
154
|
-
"config": {
|
|
155
|
-
"settings": {
|
|
156
|
-
"topicName": "walkeros-events",
|
|
157
|
-
"region": "eu-central-1",
|
|
158
|
-
},
|
|
159
|
-
"setup": {
|
|
160
|
-
"region": "eu-central-1",
|
|
161
|
-
"displayName": "walkerOS Events",
|
|
162
|
-
"tags": { "env": "prod", "team": "data" },
|
|
163
|
-
"subscriptions": [
|
|
164
|
-
{
|
|
165
|
-
"protocol": "sqs",
|
|
166
|
-
"endpoint": "arn:aws:sqs:eu-central-1:000000000000:walkeros-q",
|
|
167
|
-
},
|
|
168
|
-
],
|
|
169
|
-
},
|
|
170
|
-
},
|
|
171
|
-
},
|
|
172
|
-
},
|
|
173
|
-
}
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
### FIFO topics
|
|
177
|
-
|
|
178
|
-
Set `setup.fifoTopic: true`. The destination auto-appends `.fifo` to the topic
|
|
179
|
-
name when missing and applies `FifoTopic`/`ContentBasedDeduplication` attributes
|
|
180
|
-
at creation. Per-event ordering is driven by `messageGroupId` and (optionally)
|
|
181
|
-
`messageDeduplicationId` mapped from event paths or static values:
|
|
182
|
-
|
|
183
|
-
```jsonc
|
|
184
|
-
"mapping": {
|
|
185
|
-
"order": {
|
|
186
|
-
"complete": {
|
|
187
|
-
"settings": {
|
|
188
|
-
"messageGroupId": "user.id",
|
|
189
|
-
"messageDeduplicationId": "id"
|
|
190
|
-
}
|
|
191
|
-
}
|
|
192
|
-
}
|
|
193
|
-
}
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
### IAM permissions
|
|
197
|
-
|
|
198
|
-
Setup role:
|
|
199
|
-
|
|
200
|
-
- `sns:CreateTopic`
|
|
201
|
-
- `sns:GetTopicAttributes` (existence probe)
|
|
202
|
-
- `sns:Subscribe`
|
|
203
|
-
- `sts:GetCallerIdentity` (account-ID resolution)
|
|
204
|
-
|
|
205
|
-
Runtime push role:
|
|
206
|
-
|
|
207
|
-
- `sns:Publish`
|
|
208
|
-
- `sns:CreateTopic` (init's idempotent ARN capture). Drop this if you
|
|
209
|
-
pre-populate `settings.topicArn` from setup output.
|
|
210
|
-
|
|
211
|
-
Recommended: separate roles for setup and runtime. Setup runs from a provisioner
|
|
212
|
-
identity; runtime push uses a least-privileged role.
|
|
213
|
-
|
|
214
|
-
## Type Definitions
|
|
215
|
-
|
|
216
|
-
See [src/types/](./src/types/) for TypeScript interfaces.
|
|
217
|
-
|
|
218
|
-
## Related
|
|
38
|
+
## Documentation
|
|
219
39
|
|
|
220
|
-
|
|
221
|
-
|
|
40
|
+
Full configuration, mapping, and examples live in the docs:
|
|
41
|
+
**https://www.walkeros.io/docs/destinations/server/aws**
|
|
222
42
|
|
|
223
43
|
## Contribute
|
|
224
44
|
|
|
@@ -229,4 +49,4 @@ Feel free to contribute by submitting an
|
|
|
229
49
|
|
|
230
50
|
## License
|
|
231
51
|
|
|
232
|
-
|
|
52
|
+
MIT
|