@grafana-experiments/sdk 0.0.0 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/README.md +339 -0
- package/RELEASING.md +36 -0
- package/dist/chunk-W7444I4G.js +792 -0
- package/dist/grafana.cjs +852 -0
- package/dist/grafana.d.cts +22 -0
- package/dist/grafana.d.ts +22 -0
- package/dist/grafana.js +49 -0
- package/dist/index.cjs +846 -0
- package/dist/index.d.cts +35 -0
- package/dist/index.d.ts +35 -0
- package/dist/index.js +26 -0
- package/dist/react.cjs +63 -0
- package/dist/react.d.cts +14 -0
- package/dist/react.d.ts +14 -0
- package/dist/react.js +36 -0
- package/dist/types-M9Kqiiqw.d.cts +171 -0
- package/dist/types-M9Kqiiqw.d.ts +171 -0
- package/package.json +73 -1
package/LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "{}"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright {yyyy} {name of copyright owner}
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
package/README.md
ADDED
|
@@ -0,0 +1,339 @@
|
|
|
1
|
+
# grafana-experiments
|
|
2
|
+
|
|
3
|
+
A browser TypeScript SDK for explicitly activating OpenFeature experiments and reporting analytics to Grafana Faro and, optionally, an existing analytics reporter. It does not create flags, configure allocation, register Odin resources, or perform statistical analysis.
|
|
4
|
+
|
|
5
|
+
## Quick start
|
|
6
|
+
|
|
7
|
+
```ts
|
|
8
|
+
import { createExperiments } from '@grafana-experiments/sdk';
|
|
9
|
+
|
|
10
|
+
const experiments = createExperiments({
|
|
11
|
+
scope: 'my-application',
|
|
12
|
+
client: openFeatureClient,
|
|
13
|
+
faro: { instance: faro },
|
|
14
|
+
reportAnalytics: existingReporter, // Optional: omit for Faro-only reporting.
|
|
15
|
+
getSessionId: () => analyticsSessionId,
|
|
16
|
+
contextKey: tenantContextKey,
|
|
17
|
+
isEnabled: () => analyticsConsent,
|
|
18
|
+
events: { guide_started: 'user-action' },
|
|
19
|
+
});
|
|
20
|
+
|
|
21
|
+
await experiments.ready;
|
|
22
|
+
|
|
23
|
+
const banner = experiments.defineExperiment({
|
|
24
|
+
id: 'learning-banner-v1',
|
|
25
|
+
flagKey: 'my-team.learning-banner',
|
|
26
|
+
flag: { type: 'boolean' },
|
|
27
|
+
});
|
|
28
|
+
|
|
29
|
+
// Invoke only at the intended entry point, such as opening the panel.
|
|
30
|
+
const state = await banner.activate();
|
|
31
|
+
const showNewBanner = state.status === 'active' && state.variant === 'treatment';
|
|
32
|
+
|
|
33
|
+
const reportAnalytics = experiments.reportAnalytics;
|
|
34
|
+
reportAnalytics('guide_started', { guideId });
|
|
35
|
+
reportAnalytics('feedback', { helpful: true });
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
The original reporter receives the same name and properties plus the reserved `experiments` array. Each array entry identifies one activated experiment, its variant, session, flag and exposure. Before activation the array is empty. Supplying an existing `experiments` property does not override SDK attribution. Objects passed to the original reporter are not redacted; Faro receives a separate copy. Reporters may return promises; rejection is handled independently of Faro reporting.
|
|
39
|
+
|
|
40
|
+
`scope` must be stable and distinct for each application. Use a new experiment ID for a new experiment run or a changed definition. `contextKey` must be a stable, non-secret identity/tenant discriminator; it is included in the session-storage key. Never use credentials or email addresses as context keys.
|
|
41
|
+
|
|
42
|
+
## Faro initialization and actions
|
|
43
|
+
|
|
44
|
+
Supply exactly one of `faro: { instance }` and `faro: { config }`. The latter accepts Faro's `BrowserConfig`, including collector, instrumentations, sampling and `beforeSend`. SDK-created instances are isolated, default to no global exposure, and include `UserActionInstrumentation` even when a custom instrumentation list is supplied. Call `await experiments.ready` before rendering instrumented surfaces. Reports made during initialization still reach the original reporter immediately; their Faro mirror waits for initialization.
|
|
45
|
+
|
|
46
|
+
An injected instance remains host-owned. Include Faro's `UserActionInstrumentation` when initializing it (Faro's default instrumentation list already includes it). If it is missing, action calls fall back to custom events with `action_fallback: 'instrumentation-unavailable'`; the SDK does not modify an injected instance's instrumentation list.
|
|
47
|
+
|
|
48
|
+
Select the representation through `events` or override it per call:
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
reportAnalytics(
|
|
52
|
+
'guide_started',
|
|
53
|
+
{ guideId },
|
|
54
|
+
{
|
|
55
|
+
faro: { type: 'user-action' },
|
|
56
|
+
},
|
|
57
|
+
);
|
|
58
|
+
reportAnalytics(
|
|
59
|
+
'feedback',
|
|
60
|
+
{ helpful: true },
|
|
61
|
+
{
|
|
62
|
+
faro: { type: 'custom-event' },
|
|
63
|
+
},
|
|
64
|
+
);
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
- Unconfigured events default to custom events. Identical consecutive calls remain countable.
|
|
68
|
+
- A user-action call starts a real Faro action; it does not also emit a custom-event mirror.
|
|
69
|
+
- Starting another action ends a still-active SDK-owned action with `end_reason: 'superseded'`. This marks the observation boundary, not successful completion of the underlying operation.
|
|
70
|
+
- A host-owned action is preserved. The call becomes a custom event with `action_fallback: 'host-action-active'`, and Faro correlates it with the active action according to its normal lifecycle.
|
|
71
|
+
- Faro ends actions naturally after browser activity. If no qualifying activity occurs, Faro may cancel the action. RudderStack call counts therefore need not equal completed Faro action counts.
|
|
72
|
+
- Explicit completion uses Faro's internal lifecycle interface, isolated in `src/faro.ts`. Compatibility is restricted to Faro 2.12.x and tested against 2.12.0 and 2.12.1.
|
|
73
|
+
|
|
74
|
+
`dispose()` stops reporting, cancels pending activations, removes SDK listeners, and ends only a still-active SDK-owned action when reporting is enabled. If consent is disabled, both disposal and context reset cancel that action without emitting it; host-owned actions remain untouched. For an SDK-created Faro it also removes instrumentations and transports. It never pauses or disposes an injected Faro or an injected OpenFeature client. Disposing does not erase session deduplication state.
|
|
75
|
+
|
|
76
|
+
## Activation and sessions
|
|
77
|
+
|
|
78
|
+
Boolean flags map `false` to control and `true` to treatment. A failed evaluation is never counted as control. Object-valued flags require a validator:
|
|
79
|
+
|
|
80
|
+
```ts
|
|
81
|
+
type BannerConfig = {
|
|
82
|
+
variant: 'control' | 'treatment' | 'excluded';
|
|
83
|
+
title: string;
|
|
84
|
+
};
|
|
85
|
+
|
|
86
|
+
const banner = experiments.defineExperiment<BannerConfig>({
|
|
87
|
+
id: 'configured-banner-v1',
|
|
88
|
+
flagKey: 'my-team.configured-banner',
|
|
89
|
+
flag: {
|
|
90
|
+
type: 'object',
|
|
91
|
+
validate: (value): value is BannerConfig =>
|
|
92
|
+
typeof value === 'object' && value !== null && 'title' in value && typeof value.title === 'string',
|
|
93
|
+
},
|
|
94
|
+
});
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
The SDK separately validates the variant. Excluded values emit no exposure. Malformed values, unavailable providers and missing flags produce `status: 'unavailable'`; keep the existing experience in that state. Readiness waits at most 10 seconds by default (`readinessTimeoutMs`). Subsequent explicit activation can retry; mounted React surfaces also retry after provider readiness/configuration recovery.
|
|
98
|
+
|
|
99
|
+
Each successful activation produces `experiment_viewed` in both destinations with:
|
|
100
|
+
|
|
101
|
+
| Field | Meaning |
|
|
102
|
+
| ------------------ | -------------------------------------------------------- |
|
|
103
|
+
| `experiment_id` | Stable experiment run identifier |
|
|
104
|
+
| `experiment_group` | Stable audience label; defaults to `rollout` |
|
|
105
|
+
| `flag_key` | Existing OpenFeature flag |
|
|
106
|
+
| `variant` | Control or treatment |
|
|
107
|
+
| `session_id` | Host analytics session, falling back to Faro's session |
|
|
108
|
+
| `exposure_id` | Opaque random UUID reused for the same cached assignment |
|
|
109
|
+
|
|
110
|
+
Exposure is always a Faro custom event, independent of action completion. The SDK attempts reporting once; this is not a collector delivery acknowledgement. Analytics and Faro retain their existing sampling and filtering policies.
|
|
111
|
+
|
|
112
|
+
The storage key serializes application scope, context, session, experiment ID, flag key and flag type. The exposure ID itself contains none of those values. Within that cache, an unchanged variant/revision/allocation/population reuses its UUID; a changed assignment gets another UUID.
|
|
113
|
+
|
|
114
|
+
## Connect a deployed flag to Odin
|
|
115
|
+
|
|
116
|
+
For an illustrative deployed boolean GOFF flag named `my-team.learning-banner`, use this explicit mapping:
|
|
117
|
+
|
|
118
|
+
| System | Field | Example |
|
|
119
|
+
| -------------------------------- | --------------------------- | ------------------------- |
|
|
120
|
+
| GOFF / OpenFeature | Deployed flag key | `my-team.learning-banner` |
|
|
121
|
+
| SDK definition | `flagKey` | `my-team.learning-banner` |
|
|
122
|
+
| SDK definition and Faro exposure | `id` / `experiment_id` | `learning-banner-v1` |
|
|
123
|
+
| Odin Experiment | `metadata.name` | `learning-banner-v1` |
|
|
124
|
+
| Odin Experiment spec | `grafanaFeatureToggle.name` | `my-team.learning-banner` |
|
|
125
|
+
|
|
126
|
+
The quick-start definition above uses these SDK values. Create the Odin resource separately with that name and flag reference, then configure its data source and measurement manifest. For this boolean example, false is control and true is treatment. `defineExperiment()` only registers an application-local handle; it does not create or validate the deployed flag, provision an Odin resource, or check names across systems. Odin's flag reference is documentation, not an assignment authority. Verify the emitted `experiment_id` and `flag_key` in the destination against the resource and deployed flag before collecting a run.
|
|
127
|
+
|
|
128
|
+
### Verify assignment before declaring randomization
|
|
129
|
+
|
|
130
|
+
`population: 'randomized'` and `allocationUnit` are host assertions, not SDK verification of GOFF. Before enabling randomized inference in Odin:
|
|
131
|
+
|
|
132
|
+
1. Record the deployed flag revision, eligibility rule, randomized percentage split, and the exact evaluation-context field GOFF uses for assignment. A targeting rule or a local override alone is not evidence of randomization.
|
|
133
|
+
2. Trace that field through the deployed provider's evaluation request. For a stack-assigned flag, every session on a stack must use the same authoritative stack identity, and the SDK's `allocationUnit.id` must identify that same entity. Do not substitute a browser session ID, or assume an organization ID means stack assignment.
|
|
134
|
+
3. Evaluate known eligible entities through the actual provider. Check that repeated evaluations for the same entity agree across sessions, that independent entities can receive both variants, and that ineligible entities are excluded. Retain the rule/context evidence: an observed 50/50 split alone does not prove randomization.
|
|
135
|
+
4. Inspect stored exposures and outcomes for matching experiment/revision, variant, allocation identity and manifest. Verify collection completeness independently of assignment: sampling, filtering, consent, failed delivery and missing exposure rows can invalidate the measurement population.
|
|
136
|
+
5. Only then set the SDK population declaration and Odin's randomized-population declaration. Set Odin's expected allocation to the deployed eligible-population split; change the experiment run/revision when the assignment contract changes. Until verification is complete, omit the randomization declaration and use descriptive results.
|
|
137
|
+
|
|
138
|
+
These checks are an adoption contract; neither the SDK nor Odin introspects GOFF to enforce them. A sample-ratio check can detect some collection or assignment failures but cannot establish that the configured assignment rule was randomized.
|
|
139
|
+
|
|
140
|
+
## Standards
|
|
141
|
+
|
|
142
|
+
The SDK follows the published standards where they exist and defines only what they leave out.
|
|
143
|
+
|
|
144
|
+
- **OpenFeature** is the only assignment authority. The SDK reads variants through the evaluation API, so any provider works.
|
|
145
|
+
- **OpenTelemetry feature-flag semantic conventions** (release candidate). Each exposure also carries the attributes of the evaluation that produced it, mapped as [OpenFeature Appendix D](https://openfeature.dev/specification/appendix-d/) recommends:
|
|
146
|
+
|
|
147
|
+
| Attribute | Source |
|
|
148
|
+
| ----------------------------- | ------------------------------------------------------------------------------ |
|
|
149
|
+
| `feature_flag.key` | Flag key |
|
|
150
|
+
| `feature_flag.result.variant` | Provider's variant |
|
|
151
|
+
| `feature_flag.result.value` | Evaluated value, only when the provider has no variant |
|
|
152
|
+
| `feature_flag.result.reason` | OpenFeature reason in lower snake case (`TARGETING_MATCH` → `targeting_match`) |
|
|
153
|
+
| `feature_flag.provider.name` | Provider metadata |
|
|
154
|
+
| `feature_flag.context.id` | Flag metadata `contextId` |
|
|
155
|
+
| `feature_flag.set.id` | Flag metadata `flagSetId` |
|
|
156
|
+
| `feature_flag.version` | Flag metadata `version` |
|
|
157
|
+
|
|
158
|
+
- The SDK does not emit OpenTelemetry's `feature_flag.evaluation` event, which is recorded on every evaluation. An exposure is reported once per session at the entry point. Add OpenFeature's OpenTelemetry hook if you also want every evaluation.
|
|
159
|
+
- `feature_flag.context.id` falls back to the targeting key in the convention. The SDK omits that fallback so identity does not reach telemetry unless the provider supplies `contextId`. Set `recordFlagValue: false` when flag values are sensitive or large.
|
|
160
|
+
- **OpenFeature Tracking API**. Supply `openFeatureTracking` to also send outcome events to `client.track()` for providers that attribute goals themselves. It is off by default because it shares business data with the flag provider. Return the tracking details to send, or `null` to skip:
|
|
161
|
+
|
|
162
|
+
```ts
|
|
163
|
+
openFeatureTracking: (name, properties) =>
|
|
164
|
+
name === 'purchase_completed' ? { value: Number(properties.amount), currency: 'GBP' } : null,
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
To explicitly share the same attribution used by Faro and customer analytics, use the third callback argument and exported helper:
|
|
168
|
+
|
|
169
|
+
```ts
|
|
170
|
+
import { toOpenFeatureTrackingDetails } from '@grafana-experiments/sdk';
|
|
171
|
+
|
|
172
|
+
// Within createExperiments options:
|
|
173
|
+
openFeatureTracking: (name, properties, context) =>
|
|
174
|
+
name === 'purchase_completed'
|
|
175
|
+
? { value: Number(properties.amount), ...toOpenFeatureTrackingDetails(context) }
|
|
176
|
+
: null,
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
`MeasurementContext` contains the event ID, captured assignments, schema versions and optional journey ID. The helper returns an independent OpenFeature-compatible object, omitting undefined fields. Existing two-argument callbacks remain unchanged: no attribution is added to their return value. Select only the fields your provider should receive. Faro redaction does not redact provider Tracking or customer analytics; each destination has its own policy and receives independent copies.
|
|
180
|
+
|
|
181
|
+
The Web SDK supplies its **current evaluation context** to the provider's `track()` method. Our explicit tracking details retain the journey's original exposure snapshot; they neither change targeting nor guarantee that the provider uses those fields for its own attribution. Identity changes invalidate old journeys. Unsupported Tracking is a no-op, and a Tracking failure does not prevent Faro reporting. An attempted call is not a storage receipt.
|
|
182
|
+
|
|
183
|
+
### Optional OpenTelemetry evaluation telemetry
|
|
184
|
+
|
|
185
|
+
Use the official `@openfeature/open-telemetry-hooks` package with an existing, host-configured OTel logs SDK and exporter. This is optional; the experiments package installs no OTel pipeline or additional queue.
|
|
186
|
+
|
|
187
|
+
```ts
|
|
188
|
+
import { EventHook } from '@openfeature/open-telemetry-hooks';
|
|
189
|
+
|
|
190
|
+
// Install once on the host's OpenFeature client after configuring OTel logs.
|
|
191
|
+
// Do not also install this hook globally.
|
|
192
|
+
client.addHooks(
|
|
193
|
+
new EventHook({
|
|
194
|
+
excludeAttributes: ['feature_flag.context.id', 'feature_flag.result.value'],
|
|
195
|
+
}),
|
|
196
|
+
);
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
This example suppresses targeting identity and raw flag values. Review other provider metadata for sensitive data before enabling export. The host must enforce consent in its OTel collection/export pipeline, including queued records when consent changes. The SDK's `isEnabled` gate controls SDK reporting, not independently installed OpenFeature hooks or the host's OTel exporter.
|
|
200
|
+
|
|
201
|
+
Three distinct signals are involved:
|
|
202
|
+
|
|
203
|
+
| Signal | Meaning | Delivery |
|
|
204
|
+
| ------------------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------- |
|
|
205
|
+
| `feature_flag.evaluation` | A flag was evaluated, even if repeatedly | Official OTel hook through the host's OTel logs pipeline |
|
|
206
|
+
| `experiment_viewed` | An eligible experience was explicitly activated | Faro and the configured exposure analytics destination |
|
|
207
|
+
| Business outcome | An operation occurred, with captured exposure/journey attribution | Faro, optional customer analytics and opt-in OpenFeature Tracking |
|
|
208
|
+
|
|
209
|
+
OTel feature-flag evaluation attributes are currently release candidate within an in-development convention. Reusing their names on a Faro exposure does **not** export that exposure through OTel. Evaluation counts must not become the exposure denominator. Native Faro actions remain distinct from explicit business outcomes; automatic errors and measurements keep origin-aware Faro enrichment.
|
|
210
|
+
|
|
211
|
+
References: [OpenFeature Tracking specification](https://openfeature.dev/specification/sections/tracking/), [official OTel hooks](https://github.com/open-feature/js-sdk-contrib/tree/main/libs/hooks/open-telemetry), and [OTel evaluation conventions](https://opentelemetry.io/docs/specs/semconv/feature-flags/feature-flags-events/).
|
|
212
|
+
|
|
213
|
+
Neither standard defines an experiment, an exposure, attribution of outcomes to an exposure, or the allocation unit. Those remain this SDK's contract (`experiment_id`, `exposure_id`, `experiments`, `allocation_unit_*`), and the existing field names are unchanged.
|
|
214
|
+
|
|
215
|
+
Assignments remain stable until the session or identity/tenant context changes. Session storage preserves assignments and deduplication across reloads in the same tab. Blocked storage falls back to in-memory deduplication. Tab duplication, concurrent SDK instances and other devices can still produce duplicate exposures; use `exposure_id` for downstream deduplication. The browser is not an authoritative source of assignment persistence.
|
|
216
|
+
|
|
217
|
+
Faro session changes are observed automatically. Call `experiments.refreshSession()` when a host session ID or consent changes. Call `experiments.resetContext(newContextKey)` when switching identity/tenant, before reporting or activating there. Calling `resetContext` with the current key explicitly clears the current registered experiments' stored assignments. Neither operation changes the OpenFeature provider's context: the host must update that as well.
|
|
218
|
+
|
|
219
|
+
Without either session ID, activation remains unavailable. Exposure per session does not imply assignment per session: Grafana continues to allocate by its server-defined stack/org context.
|
|
220
|
+
|
|
221
|
+
## React
|
|
222
|
+
|
|
223
|
+
```tsx
|
|
224
|
+
import { ExperimentsProvider, useExperiment } from '@grafana-experiments/sdk/react';
|
|
225
|
+
|
|
226
|
+
// Define the SDK and experiment once in application setup, not during render.
|
|
227
|
+
function Panel({ eligible }: { eligible: boolean }) {
|
|
228
|
+
const state = useExperiment(banner, eligible);
|
|
229
|
+
return state.status === 'active' && state.variant === 'treatment' ? (
|
|
230
|
+
<NewExperience />
|
|
231
|
+
) : (
|
|
232
|
+
<ExistingExperience />
|
|
233
|
+
);
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
<ExperimentsProvider experiments={experiments}>
|
|
237
|
+
<Panel eligible={panelIsOpen} />
|
|
238
|
+
</ExperimentsProvider>;
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
The activation hook works with a stable experiment handle, with or without the context provider. `useExperiments()` accesses the SDK from that provider. Activation happens in an effect; abandoned renders and unmounted pending activations produce no exposure. Visibility is the application's responsibility: mount does not necessarily mean visible. Changing eligibility does not undo an already recorded exposure. The provider does not own or dispose the SDK; the application's bootstrap lifecycle does.
|
|
242
|
+
|
|
243
|
+
## Grafana adapter
|
|
244
|
+
|
|
245
|
+
```ts
|
|
246
|
+
import { createGrafanaExperiments } from '@grafana-experiments/sdk/grafana';
|
|
247
|
+
|
|
248
|
+
const experiments = createGrafanaExperiments({
|
|
249
|
+
scope: 'my-plugin',
|
|
250
|
+
domain: 'my-plugin-experiments',
|
|
251
|
+
faro: { instance: faro },
|
|
252
|
+
getSessionId: () => analyticsSessionId,
|
|
253
|
+
});
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Requires `@grafana/runtime >=13.2.2 <14`. The adapter composes Grafana's local-override and OFREP proxy providers in its own OpenFeature domain. It refuses to replace an existing named provider and does not alter Grafana's default provider or targeting context. Dispose releases only the adapter's provider registration.
|
|
257
|
+
|
|
258
|
+
Ordinary events use `reportInteraction`. Exposure uses `reportExperimentView(id, group, variant)` exactly once, with no duplicate interaction event. Grafana's exposure API accepts only those three fields; Grafana supplies its own analytics identity/session context. The full SDK exposure record is available in Faro and subsequent enriched outcome events. It is not injected into unsupported Grafana exposure fields.
|
|
259
|
+
|
|
260
|
+
## Consent, filtering and failures
|
|
261
|
+
|
|
262
|
+
`isEnabled` gates SDK activation and reporting. Call `refreshSession()` when it changes; SDK-created Faro instances are paused on revocation and resumed only if the SDK paused them. The host remains responsible for pausing an injected Faro and its independently captured automatic telemetry. The SDK does not override the host's sampling or `beforeSend` filters.
|
|
263
|
+
|
|
264
|
+
Use `transformFaroProperties(name, properties)` to redact the Faro copy, or return `null` to suppress it. Faro values are converted to strings; object values are JSON encoded. Uncloneable/unserializable properties are omitted from the Faro copy without dropping other attributes. Original-reporter and Faro failures are isolated and can be observed through `onError`. No transport retries or durable queues are added.
|
|
265
|
+
|
|
266
|
+
## Development
|
|
267
|
+
|
|
268
|
+
```sh
|
|
269
|
+
npm ci --legacy-peer-deps
|
|
270
|
+
npm run check
|
|
271
|
+
npm run test:mutation
|
|
272
|
+
npm pack --dry-run
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
The standalone package lives outside Odin's plugin build. Development uses `--legacy-peer-deps` because Grafana runtime is an optional host integration and its full UI peer dependency graph is unnecessary for adapter-boundary tests. Real OpenFeature clients and Faro recording transports exercise the core; only Grafana's host boundary is substituted. A/A and Pathfinder-style examples are in `examples/`. No package is published or production application migrated by these commands.
|
|
276
|
+
|
|
277
|
+
## Version 2 measurement contract
|
|
278
|
+
|
|
279
|
+
OpenFeature remains the only assignment authority. Define `metadata` for declared allocation identity and population, or `mapAssignment(details)` to map provider results explicitly. The SDK never infers that a browser session represents a stack or organization. Missing allocation metadata is valid descriptive telemetry; Odin withholds allocation-level inference.
|
|
280
|
+
|
|
281
|
+
```ts
|
|
282
|
+
const experiment = sdk.defineExperiment({
|
|
283
|
+
id: 'checkout',
|
|
284
|
+
flagKey: 'checkout-recommendations',
|
|
285
|
+
flag: { type: 'boolean' },
|
|
286
|
+
metadata: {
|
|
287
|
+
revision: 'recommendations-v2',
|
|
288
|
+
allocationUnit: { type: 'stack', id: stackId },
|
|
289
|
+
population: 'randomized',
|
|
290
|
+
},
|
|
291
|
+
});
|
|
292
|
+
await experiment.activate({ entryId: 'checkout-entry-1' });
|
|
293
|
+
const journey = sdk.beginJourney();
|
|
294
|
+
await addProduct();
|
|
295
|
+
journey.reportAnalytics('product_added', { product: 'trail-pack' });
|
|
296
|
+
journey.reportAnalytics('purchase_completed', { amount: 48, currency: 'GBP' });
|
|
297
|
+
journey.end();
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
Use one stable `entryId` for repeated activation of a committed entry; use a new ID for a new entry. React's `useExperiment(experiment, eligible, entryId)` follows the same rule. Provider updates do not change an active journey. Re-entry evaluates again: an unchanged assignment reuses its session exposure, while variant/revision/allocation changes receive their own exposure. Omitting `entryId` preserves the original session-stable API for existing integrations.
|
|
301
|
+
|
|
302
|
+
Journey handles snapshot active assignments and are invalidated on session/identity reset or `end()`. Consent is checked when events are sent. `getTelemetryContext()` supplies captured `experiments` and `journey_id` strings for explicit Faro error/measurement calls. Ended or invalidated handles return no experiment assignments. Do not use journey handles to bypass the host's Faro consent policy.
|
|
303
|
+
|
|
304
|
+
Each logical call receives a unique `event_id`; repeated calls remain distinct, while transport retries reuse the same payload. Exposure IDs are opaque random identifiers. Assignment arrays retain every active experiment. The v2 storage namespace deliberately does not reuse v1 cache entries; existing v1 warehouse records remain readable as legacy session records without invented allocation metadata.
|
|
305
|
+
|
|
306
|
+
### Typed events and manifests
|
|
307
|
+
|
|
308
|
+
Pass `eventDefinitions` into `createExperiments`. Each definition has a positive integer `version` and `properties` mapping to `{type: 'string' | 'number' | 'boolean', required?: boolean}`. Use `defineEvent` and `createTypedReporter` to preserve TypeScript inference. `getEventManifest()` returns a JSON-serializable versioned manifest to import into Odin. Event names not in the manifest retain the original permissive reporting behavior.
|
|
309
|
+
|
|
310
|
+
Malformed declared measurements still attempt the existing customer reporter but emit an `experiment_diagnostic` to Faro rather than a valid business outcome. Event manifests, including schema versions, must match Odin's analysis configuration. Exporting a manifest does not provision flags, mutate Odin, or transmit credentials.
|
|
311
|
+
|
|
312
|
+
### Diagnostics and Faro enrichment
|
|
313
|
+
|
|
314
|
+
`getDiagnostics()` returns a bounded snapshot (latest 200 entries); `subscribeDiagnostics(listener)` observes evaluation, activation, suppression, transitions, validation and reporting attempts/failures. Observer failures cannot break application reporting. Diagnostic messages are local developer information and should not be copied into public telemetry without redaction. A reporting attempt is never represented as confirmed storage.
|
|
315
|
+
|
|
316
|
+
Use `enrichFaro: true` with SDK-owned initialization options. For an existing Faro instance, install `createExperimentEnricher({getAssignments: () => sdk?.getActiveAssignments() ?? [], isEnabled, beforeSend: existingHook})` when the host initializes Faro. The host hook runs last so its filtering/redaction remains authoritative. The SDK does not replace another integration's hooks or sampling configuration.
|
|
317
|
+
|
|
318
|
+
Measurement emission time does not prove when measured activity happened. Measurements require `context.experiment_started_at` (epoch milliseconds), or an explicit journey snapshot, to receive attribution. Pre-exposure and unscoped Web Vitals remain unattributed. The SDK never installs a second replay recorder. Native Faro user-action completion is not a business conversion.
|
|
319
|
+
|
|
320
|
+
### Storage ownership
|
|
321
|
+
|
|
322
|
+
The SDK emits the standard Faro envelope with custom attributes. Storage provisioning and database ownership remain outside the SDK.
|
|
323
|
+
|
|
324
|
+
### Delivery boundaries
|
|
325
|
+
|
|
326
|
+
An `activated` diagnostic records an exposure decision. `analytics-attempted` and
|
|
327
|
+
`faro-attempted` record calls to those destinations, not acknowledgements of storage.
|
|
328
|
+
`faro-filtered` describes the SDK's redaction transform; filtering inside a host
|
|
329
|
+
Faro hook or transport may not be observable by this SDK. Observable synchronous
|
|
330
|
+
and asynchronous reporter failures produce `reporting-error` diagnostics.
|
|
331
|
+
|
|
332
|
+
Faro's transport owns batching and retries. Retry attempts retain the original event
|
|
333
|
+
ID; separate legitimate reporting calls receive different IDs. The SDK does not
|
|
334
|
+
replace caller-owned transports or add a second delivery queue. Transport retries
|
|
335
|
+
can be exhausted, and closing a browser can lose buffered telemetry. Session-persisted
|
|
336
|
+
exposure deduplication is not a delivery receipt: reloading does not replay an exposure
|
|
337
|
+
that has already been activated. Verify ingestion in the destination when completeness
|
|
338
|
+
matters. Sampling and filtering remain controlled by the host; a complete-collection
|
|
339
|
+
declaration in Odin is an operational assertion, not a guarantee from this SDK.
|
package/RELEASING.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Publishing @grafana-experiments/sdk
|
|
2
|
+
|
|
3
|
+
Releases follow grafana-coda-app's manual GitHub Actions trusted-publishing workflow. The local package retains its `grafana-experiments` name for existing consumers and the demo. Only the release artifact is named `@grafana-experiments/sdk`; its README imports are adjusted accordingly. No npm token is stored in GitHub or Vault.
|
|
4
|
+
|
|
5
|
+
## One-time setup
|
|
6
|
+
|
|
7
|
+
1. Ensure `@grafana-experiments/sdk` exists in the npm organization. If it does not, an organization owner must bootstrap it with an authenticated manual publish before configuring its trusted publisher. Use a scratch `0.0.0` placeholder and deprecate it; reserve the real version for the verified workflow artifact. This PR does not bootstrap or publish it.
|
|
8
|
+
2. In the package's npm settings, configure GitHub trusted publishing with organization `grafana`, repository `grafana-experiments-app`, workflow filename `publish-npm.yml`, environment `Release npm package`, and permission to run `npm publish`.
|
|
9
|
+
3. Configure the GitHub environment `Release npm package` to allow only `main`, with the desired reviewer protection. The workflow also refuses dispatches from other branches.
|
|
10
|
+
|
|
11
|
+
The source repository is internal. Like Coda, the workflow explicitly disables npm provenance because private/internal source repositories do not support it. OIDC authentication still applies. See [npm trusted publishing](https://docs.npmjs.com/trusted-publishers/).
|
|
12
|
+
|
|
13
|
+
## Release
|
|
14
|
+
|
|
15
|
+
1. Merge this workflow into main. For subsequent releases, bump the SDK's package.json and package-lock.json version in a reviewed PR. SDK versions are independent of plugin versions.
|
|
16
|
+
2. Dispatch **publish-npm** on `main`. It installs with lifecycle scripts disabled, checks/tests/builds explicitly, creates a scoped tarball, and verifies every ESM/CJS/type export plus README and LICENSE is present.
|
|
17
|
+
3. The workflow checks the exact version, not the latest dist-tag. A version already published from the same commit skips publishing and can repair a missing tag. A version owned by another commit fails; registry errors other than 404 fail closed.
|
|
18
|
+
4. Successful publication creates `experiments-sdk-v<VERSION>`, avoiding the plugin's `v*` release trigger. Existing tags must point to the same commit. No automatic publish on main push is enabled.
|
|
19
|
+
5. Verify `npm view @grafana-experiments/sdk@<VERSION> version gitHead` and install the package in a consumer.
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
npm install @grafana-experiments/sdk @openfeature/web-sdk @grafana/faro-web-sdk
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Public entry points are `@grafana-experiments/sdk`, `@grafana-experiments/sdk/react` and `@grafana-experiments/sdk/grafana`.
|
|
26
|
+
|
|
27
|
+
For a local artifact check without publishing:
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
cd packages/grafana-experiments
|
|
31
|
+
npm ci --ignore-scripts --legacy-peer-deps
|
|
32
|
+
npm run check
|
|
33
|
+
node scripts/pack-release.mjs /tmp
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
An npm PUT E404 usually indicates a missing package or mismatched trusted publisher: check repository, filename and environment exactly. Do not add a long-lived npm token as a fallback. No release tag is created on a failed publish.
|