@learncard/network-brain-service 4.0.3 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +14 -0
- package/CHANGELOG.md +76 -0
- package/config/config.dev.json +5 -0
- package/config/config.example.json +5 -0
- package/config/config.production.json +5 -0
- package/config/config.scouts.dev.json +5 -0
- package/config/config.scouts.production.json +5 -0
- package/dist/accesslayer/campaign/audience.d.ts +7 -0
- package/dist/accesslayer/campaign/audience.d.ts.map +1 -0
- package/dist/accesslayer/campaign/create.d.ts +3 -0
- package/dist/accesslayer/campaign/create.d.ts.map +1 -0
- package/dist/accesslayer/campaign/delete.d.ts +2 -0
- package/dist/accesslayer/campaign/delete.d.ts.map +1 -0
- package/dist/accesslayer/campaign/read.d.ts +6 -0
- package/dist/accesslayer/campaign/read.d.ts.map +1 -0
- package/dist/accesslayer/campaign/shared.d.ts +16 -0
- package/dist/accesslayer/campaign/shared.d.ts.map +1 -0
- package/dist/accesslayer/campaign/state.d.ts +4 -0
- package/dist/accesslayer/campaign/state.d.ts.map +1 -0
- package/dist/accesslayer/campaign/update.d.ts +3 -0
- package/dist/accesslayer/campaign/update.d.ts.map +1 -0
- package/dist/accesslayer/consentflowcontract/relationships/create.d.ts.map +1 -1
- package/dist/accesslayer/consentflowcontract/relationships/update.d.ts.map +1 -1
- package/dist/accesslayer/credential-share-previews.d.ts +5 -0
- package/dist/accesslayer/credential-share-previews.d.ts.map +1 -0
- package/dist/accesslayer/inbox-batch/schema.d.ts +4 -0
- package/dist/accesslayer/inbox-batch/schema.d.ts.map +1 -0
- package/dist/accesslayer/inbox-batch/schema.test.d.ts +2 -0
- package/dist/accesslayer/inbox-batch/schema.test.d.ts.map +1 -0
- package/dist/accesslayer/inbox-batch/store.d.ts +1 -1
- package/dist/accesslayer/inbox-batch/store.d.ts.map +1 -1
- package/dist/accesslayer/profile/delete.d.ts.map +1 -1
- package/dist/accesslayer/skill/read.d.ts +12 -2
- package/dist/accesslayer/skill/read.d.ts.map +1 -1
- package/dist/accesslayer/skill/update.d.ts +11 -1
- package/dist/accesslayer/skill/update.d.ts.map +1 -1
- package/dist/app.d.ts +4 -0
- package/dist/app.d.ts.map +1 -1
- package/dist/cache/app-store.caches.d.ts +2 -1
- package/dist/cache/app-store.caches.d.ts.map +1 -1
- package/dist/config/environment.d.ts +268 -178
- package/dist/config/environment.d.ts.map +1 -1
- package/dist/config/skillEmbedding.d.ts +70 -0
- package/dist/config/skillEmbedding.d.ts.map +1 -0
- package/dist/config/skillEmbedding.neo4j.test.d.ts +2 -0
- package/dist/config/skillEmbedding.neo4j.test.d.ts.map +1 -0
- package/dist/config/stageConfig.d.ts +20 -0
- package/dist/config/stageConfig.d.ts.map +1 -1
- package/dist/constants/campaigns.d.ts +3 -0
- package/dist/constants/campaigns.d.ts.map +1 -0
- package/dist/didkit_wasm_bg.wasm +0 -0
- package/dist/helpers/bedrock-embedding.helpers.d.ts +40 -0
- package/dist/helpers/bedrock-embedding.helpers.d.ts.map +1 -0
- package/dist/helpers/bedrock-embedding.helpers.test.d.ts +2 -0
- package/dist/helpers/bedrock-embedding.helpers.test.d.ts.map +1 -0
- package/dist/helpers/bedrock-embedding.smoke.test.d.ts +2 -0
- package/dist/helpers/bedrock-embedding.smoke.test.d.ts.map +1 -0
- package/dist/helpers/boost.helpers.d.ts +1 -0
- package/dist/helpers/boost.helpers.d.ts.map +1 -1
- package/dist/helpers/skill-embedding.helpers.d.ts +65 -7
- package/dist/helpers/skill-embedding.helpers.d.ts.map +1 -1
- package/dist/helpers/skill-embedding.helpers.test.d.ts +2 -0
- package/dist/helpers/skill-embedding.helpers.test.d.ts.map +1 -0
- package/dist/index.js +2 -2
- package/dist/models/Campaign.d.ts +10 -0
- package/dist/models/Campaign.d.ts.map +1 -0
- package/dist/models/CredentialSharePreview.d.ts +14 -0
- package/dist/models/CredentialSharePreview.d.ts.map +1 -0
- package/dist/models/Integration.d.ts +1 -1
- package/dist/models/Profile.d.ts.map +1 -1
- package/dist/models/campaign-constraints.d.ts +2 -0
- package/dist/models/campaign-constraints.d.ts.map +1 -0
- package/dist/models/index.d.ts +2 -0
- package/dist/models/index.d.ts.map +1 -1
- package/dist/publish-metafile.json +1 -1
- package/dist/routes/app-store.d.ts +1 -1
- package/dist/routes/boosts.d.ts +24 -4
- package/dist/routes/boosts.d.ts.map +1 -1
- package/dist/routes/campaigns.d.ts +1411 -0
- package/dist/routes/campaigns.d.ts.map +1 -0
- package/dist/routes/contracts.d.ts +19 -6
- package/dist/routes/contracts.d.ts.map +1 -1
- package/dist/routes/credential-share-previews.d.ts +39 -0
- package/dist/routes/credential-share-previews.d.ts.map +1 -0
- package/dist/routes/credentials.d.ts +4 -1
- package/dist/routes/credentials.d.ts.map +1 -1
- package/dist/routes/inbox.d.ts +5 -5
- package/dist/routes/index.d.ts +4 -0
- package/dist/routes/index.d.ts.map +1 -1
- package/dist/routes/integrations.d.ts +3 -3
- package/dist/routes/profile-manager.d.ts +4 -0
- package/dist/routes/profile-manager.d.ts.map +1 -1
- package/dist/routes/profiles.d.ts +44 -0
- package/dist/routes/profiles.d.ts.map +1 -1
- package/dist/routes/skill-frameworks.d.ts +1 -0
- package/dist/routes/skill-frameworks.d.ts.map +1 -1
- package/dist/routes/skills-semantic-search.test.d.ts +2 -0
- package/dist/routes/skills-semantic-search.test.d.ts.map +1 -0
- package/dist/routes/skills.d.ts.map +1 -1
- package/dist/routes/test.d.ts +3 -1
- package/dist/routes/test.d.ts.map +1 -1
- package/dist/routes/workflows.d.ts.map +1 -1
- package/dist/seed/seedSkillFrameworks.d.ts +1 -0
- package/dist/seed/seedSkillFrameworks.d.ts.map +1 -1
- package/dist/seed/seedSkillFrameworks.embeddings.test.d.ts +2 -0
- package/dist/seed/seedSkillFrameworks.embeddings.test.d.ts.map +1 -0
- package/dist/types/app-manifest.d.ts +2 -2
- package/dist/types/campaign.d.ts +19 -0
- package/dist/types/campaign.d.ts.map +1 -0
- package/dist/types/consentflowcontract.d.ts +1 -1
- package/dist/types/integration.d.ts +3 -3
- package/dist/types/profile.d.ts +9 -0
- package/dist/types/profile.d.ts.map +1 -1
- package/dist/types/skill.d.ts.map +1 -1
- package/esbuildPlugins.cjs +9 -6
- package/package.json +20 -17
- package/project.json +14 -0
- package/scripts/skill-embeddings-backfill.ts +161 -0
- package/serverless-local.yml +4 -0
- package/serverless.function-env.cjs +3 -0
- package/serverless.local.yml +4 -0
- package/serverless.yml +12 -1
- package/src/accesslayer/campaign/README.md +174 -0
- package/src/accesslayer/campaign/audience.ts +152 -0
- package/src/accesslayer/campaign/create.ts +47 -0
- package/src/accesslayer/campaign/delete.ts +18 -0
- package/src/accesslayer/campaign/read.ts +125 -0
- package/src/accesslayer/campaign/shared.ts +92 -0
- package/src/accesslayer/campaign/state.ts +80 -0
- package/src/accesslayer/campaign/update.ts +50 -0
- package/src/accesslayer/consentflowcontract/relationships/create.ts +3 -1
- package/src/accesslayer/consentflowcontract/relationships/update.ts +5 -1
- package/src/accesslayer/credential-share-previews.ts +76 -0
- package/src/accesslayer/inbox-batch/schema.test.ts +91 -0
- package/src/accesslayer/inbox-batch/schema.ts +54 -0
- package/src/accesslayer/inbox-batch/store.ts +2 -38
- package/src/accesslayer/profile/delete.ts +6 -0
- package/src/accesslayer/skill/read.ts +56 -16
- package/src/accesslayer/skill/update.ts +43 -1
- package/src/app.ts +9 -0
- package/src/config/environment.test.ts +208 -0
- package/src/config/environment.ts +166 -11
- package/src/config/skillEmbedding.neo4j.test.ts +138 -0
- package/src/config/skillEmbedding.ts +202 -0
- package/src/constants/campaigns.ts +9 -0
- package/src/helpers/bedrock-embedding.helpers.test.ts +208 -0
- package/src/helpers/bedrock-embedding.helpers.ts +243 -0
- package/src/helpers/bedrock-embedding.smoke.test.ts +53 -0
- package/src/helpers/boost-storage.helpers.test.ts +155 -1
- package/src/helpers/boost.helpers.ts +52 -10
- package/src/helpers/skill-embedding.helpers.test.ts +1044 -0
- package/src/helpers/skill-embedding.helpers.ts +611 -123
- package/src/models/Campaign.ts +32 -0
- package/src/models/CredentialSharePreview.ts +30 -0
- package/src/models/Profile.ts +1 -0
- package/src/models/campaign-constraints.ts +22 -0
- package/src/models/index.ts +29 -1
- package/src/routes/campaigns.ts +179 -0
- package/src/routes/credential-share-previews.ts +74 -0
- package/src/routes/profiles.ts +7 -0
- package/src/routes/skills-semantic-search.test.ts +168 -0
- package/src/routes/skills.ts +42 -6
- package/src/routes/workflows.ts +8 -1
- package/src/seed/seedSkillFrameworks.embeddings.test.ts +82 -0
- package/src/seed/seedSkillFrameworks.test.ts +6 -0
- package/src/seed/seedSkillFrameworks.ts +83 -18
- package/src/types/campaign.ts +17 -0
- package/src/types/skill.ts +6 -0
- package/test/campaigns.spec.ts +789 -0
- package/test/consentflow.spec.ts +101 -0
- package/test/credential-share-previews.spec.ts +97 -0
- package/test/inbox-batch.spec.ts +189 -1
- package/test/verification/campaigns.ts +49 -0
- package/vitest.campaigns.config.ts +21 -0
- package/vitest.config.ts +11 -0
- package/vitest.integration.config.ts +1 -0
package/esbuildPlugins.cjs
CHANGED
|
@@ -2,6 +2,7 @@ const path = require('node:path');
|
|
|
2
2
|
const fs = require('node:fs');
|
|
3
3
|
|
|
4
4
|
const { sentryEsbuildPlugin } = require('@sentry/esbuild-plugin');
|
|
5
|
+
const { withSentryRateLimitRetry } = require('../../../scripts/sentry-rate-limit-retry.cjs');
|
|
5
6
|
|
|
6
7
|
const forceCjsSodiumWrapperPlugin = {
|
|
7
8
|
name: 'force-cjs-sodium-wrapper',
|
|
@@ -40,10 +41,12 @@ const copyDidkitWasmPlugin = {
|
|
|
40
41
|
module.exports = [
|
|
41
42
|
forceCjsSodiumWrapperPlugin,
|
|
42
43
|
copyDidkitWasmPlugin,
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
44
|
+
withSentryRateLimitRetry(
|
|
45
|
+
sentryEsbuildPlugin({
|
|
46
|
+
telemetry: process.env.SENTRY_BUILD_TELEMETRY !== 'false',
|
|
47
|
+
authToken: process.env.SENTRY_AUTH_TOKEN,
|
|
48
|
+
org: process.env.SENTRY_ORG,
|
|
49
|
+
project: process.env.SENTRY_PROJECT,
|
|
50
|
+
})
|
|
51
|
+
),
|
|
49
52
|
];
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@learncard/network-brain-service",
|
|
3
|
-
"version": "4.0
|
|
3
|
+
"version": "4.1.0",
|
|
4
4
|
"description": "",
|
|
5
5
|
"engines": {
|
|
6
6
|
"node": ">=24.12 <25"
|
|
@@ -15,6 +15,7 @@
|
|
|
15
15
|
"test:watch": "vitest --config vitest.config.ts",
|
|
16
16
|
"test:coverage": "vitest run --coverage --config vitest.config.ts",
|
|
17
17
|
"test:integration": "vitest run --config vitest.integration.config.ts",
|
|
18
|
+
"test:campaigns": "vitest run --config vitest.campaigns.config.ts",
|
|
18
19
|
"test:inbox:e2e": "vitest run --config vitest.inbox-e2e.config.ts",
|
|
19
20
|
"test:profile-search": "vitest run --config vitest.profile-search-neo4j.config.ts",
|
|
20
21
|
"test:integration:watch": "vitest --config vitest.integration.config.ts",
|
|
@@ -23,6 +24,7 @@
|
|
|
23
24
|
"seed:dev-app": "bun scripts/seed-dev-app.ts",
|
|
24
25
|
"seed:demo-persona": "bun --conditions=development scripts/seed-demo-persona.ts",
|
|
25
26
|
"skill-frameworks": "bun scripts/skill-frameworks.ts",
|
|
27
|
+
"skill-embeddings:backfill": "bun --conditions=development scripts/skill-embeddings-backfill.ts",
|
|
26
28
|
"backfill:profile-search": "bun scripts/backfill-profile-search.ts",
|
|
27
29
|
"build": "shx rm -rf dist && node bundle.mjs && tsc --p tsconfig.build.json && tsc --showConfig --p tsconfig.build.json > tsconfig.temp.json && tsc-alias --replacer ./tsc-alias-replacer.cjs -p tsconfig.temp.json && shx rm tsconfig.temp.json",
|
|
28
30
|
"typecheck": "tsc --noEmit -p tsconfig.build.json && tsc --noEmit -p tsconfig.entrypoints.json",
|
|
@@ -61,27 +63,28 @@
|
|
|
61
63
|
"testcontainers": "^9.1.1"
|
|
62
64
|
},
|
|
63
65
|
"dependencies": {
|
|
66
|
+
"@aws-sdk/client-bedrock-runtime": "^3.1139.0",
|
|
64
67
|
"@aws-sdk/client-sqs": "^3.379.1",
|
|
65
68
|
"@digitalcredentials/issuer-registry-client": "^4.1.0",
|
|
66
69
|
"@fastify/cors": "^11.2.0",
|
|
67
70
|
"@google/genai": "^1.27.0",
|
|
68
|
-
"@learncard/credential-library": "2.1.
|
|
69
|
-
"@learncard/core": "9.4.
|
|
70
|
-
"@learncard/crypto-plugin": "1.1.
|
|
71
|
-
"@learncard/did-web-plugin": "1.1.
|
|
72
|
-
"@learncard/didkey-plugin": "1.1.
|
|
73
|
-
"@learncard/didkit-plugin": "1.10.
|
|
74
|
-
"@learncard/didkit-plugin-node": "0.3.
|
|
75
|
-
"@learncard/dynamic-loader-plugin": "1.1.
|
|
76
|
-
"@learncard/encryption-plugin": "1.1.
|
|
77
|
-
"@learncard/expiration-plugin": "1.2.
|
|
71
|
+
"@learncard/credential-library": "2.1.4",
|
|
72
|
+
"@learncard/core": "9.4.41",
|
|
73
|
+
"@learncard/crypto-plugin": "1.1.41",
|
|
74
|
+
"@learncard/did-web-plugin": "1.1.41",
|
|
75
|
+
"@learncard/didkey-plugin": "1.1.41",
|
|
76
|
+
"@learncard/didkit-plugin": "1.10.6",
|
|
77
|
+
"@learncard/didkit-plugin-node": "0.3.6",
|
|
78
|
+
"@learncard/dynamic-loader-plugin": "1.1.41",
|
|
79
|
+
"@learncard/encryption-plugin": "1.1.41",
|
|
80
|
+
"@learncard/expiration-plugin": "1.2.41",
|
|
78
81
|
"@learncard/email-templates": "1.2.1",
|
|
79
|
-
"@learncard/helpers": "1.
|
|
80
|
-
"@learncard/learn-card-plugin": "1.2.
|
|
82
|
+
"@learncard/helpers": "1.7.1",
|
|
83
|
+
"@learncard/learn-card-plugin": "1.2.41",
|
|
81
84
|
"@learncard/partner-connect-core": "0.2.0",
|
|
82
|
-
"@learncard/types": "5.
|
|
83
|
-
"@learncard/vc-plugin": "1.6.
|
|
84
|
-
"@learncard/vc-templates-plugin": "1.1.
|
|
85
|
+
"@learncard/types": "5.24.0",
|
|
86
|
+
"@learncard/vc-plugin": "1.6.7",
|
|
87
|
+
"@learncard/vc-templates-plugin": "1.1.41",
|
|
85
88
|
"@sentry/serverless": "7.61.0",
|
|
86
89
|
"@trpc/server": "11.8.0",
|
|
87
90
|
"@types/lodash": "^4.14.191",
|
|
@@ -114,7 +117,7 @@
|
|
|
114
117
|
"undici": "^6",
|
|
115
118
|
"tweetnacl": "^1.0.3",
|
|
116
119
|
"uuid": "^14.0.0",
|
|
117
|
-
"zod": "4.
|
|
120
|
+
"zod": "4.6.6",
|
|
118
121
|
"zod-openapi": "^5.0.1"
|
|
119
122
|
},
|
|
120
123
|
"repository": {
|
package/project.json
CHANGED
|
@@ -127,6 +127,20 @@
|
|
|
127
127
|
"script": "test:integration"
|
|
128
128
|
}
|
|
129
129
|
},
|
|
130
|
+
"test:campaigns": {
|
|
131
|
+
"executor": "nx:run-script",
|
|
132
|
+
"cache": false,
|
|
133
|
+
"dependsOn": ["^build"],
|
|
134
|
+
"inputs": [
|
|
135
|
+
"source",
|
|
136
|
+
"tests",
|
|
137
|
+
"{projectRoot}/vitest.campaigns.config.ts",
|
|
138
|
+
"{projectRoot}/test-setup.ts"
|
|
139
|
+
],
|
|
140
|
+
"options": {
|
|
141
|
+
"script": "test:campaigns"
|
|
142
|
+
}
|
|
143
|
+
},
|
|
130
144
|
"test:boost-recipients": {
|
|
131
145
|
"executor": "nx:run-script",
|
|
132
146
|
"cache": false,
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
#!/usr/bin/env bun
|
|
2
|
+
/**
|
|
3
|
+
* LC-2206 operator backfill: (re-)embed every Skill with the configured Nova 2 generation.
|
|
4
|
+
*
|
|
5
|
+
* bun run skill-embeddings:backfill # dry run: counts only, no Bedrock calls
|
|
6
|
+
* bun run skill-embeddings:backfill --apply --target <neo4j-host> [--max N] [--page-size N] [--cursor <skillId>]
|
|
7
|
+
*
|
|
8
|
+
* - Dry run is the default and never calls Bedrock or writes to Neo4j.
|
|
9
|
+
* - --apply requires SKILL_EMBEDDING_BEDROCK_ENABLED=true and --target equal to the host in
|
|
10
|
+
* NEO4J_URI, so a copied shell cannot silently write to the wrong database.
|
|
11
|
+
* - Takes the same Redis lease as the startup backfill (one writer per generation).
|
|
12
|
+
* - Idempotent and resumable: skills whose stored key matches sha256(identity + text) are
|
|
13
|
+
* skipped; on early stop it prints the cursor to resume from.
|
|
14
|
+
*
|
|
15
|
+
* Writes only the Nova property; the legacy Google `s.embedding` and the read flag
|
|
16
|
+
* (SKILL_EMBEDDING_BEDROCK_READ_ENABLED) are untouched, so it is safe while search reads legacy.
|
|
17
|
+
*
|
|
18
|
+
* Uses the normal AWS credential chain (SigV4). Paid: each embedded skill is one InvokeModel.
|
|
19
|
+
*/
|
|
20
|
+
import { environment } from '@environment';
|
|
21
|
+
import {
|
|
22
|
+
acquireSkillEmbeddingBackfillLock,
|
|
23
|
+
buildSkillEmbeddingText,
|
|
24
|
+
getSkillEmbeddingGeneration,
|
|
25
|
+
isSkillEmbeddingEnabled,
|
|
26
|
+
runSkillEmbeddingBackfill,
|
|
27
|
+
skillNeedsEmbedding,
|
|
28
|
+
} from '@helpers/skill-embedding.helpers';
|
|
29
|
+
import { getSkillEmbeddingCandidates } from '@accesslayer/skill/read';
|
|
30
|
+
import { neogma } from '@instance';
|
|
31
|
+
|
|
32
|
+
// Public list price snapshot checked by AI Passport on 2026-10-13 (us-east-1, on-demand text).
|
|
33
|
+
const NOVA2_USD_PER_MILLION_INPUT_TOKENS = 0.135;
|
|
34
|
+
// Planning assumption only; Nova responses carry no token usage.
|
|
35
|
+
const ASSUMED_CHARS_PER_TOKEN = 4;
|
|
36
|
+
|
|
37
|
+
type Options = {
|
|
38
|
+
apply: boolean;
|
|
39
|
+
target?: string;
|
|
40
|
+
max?: number;
|
|
41
|
+
pageSize: number;
|
|
42
|
+
cursor: string | null;
|
|
43
|
+
};
|
|
44
|
+
|
|
45
|
+
const parseArgs = (argv: string[]): Options => {
|
|
46
|
+
const options: Options = { apply: false, pageSize: 100, cursor: null };
|
|
47
|
+
|
|
48
|
+
for (let i = 0; i < argv.length; i += 1) {
|
|
49
|
+
const arg = argv[i];
|
|
50
|
+
const next = () => {
|
|
51
|
+
const value = argv[i + 1];
|
|
52
|
+
if (!value) throw new Error(`${arg} needs a value`);
|
|
53
|
+
i += 1;
|
|
54
|
+
return value;
|
|
55
|
+
};
|
|
56
|
+
|
|
57
|
+
if (arg === '--apply') options.apply = true;
|
|
58
|
+
else if (arg === '--target') options.target = next();
|
|
59
|
+
else if (arg === '--max') options.max = Number(next());
|
|
60
|
+
else if (arg === '--page-size') options.pageSize = Number(next());
|
|
61
|
+
else if (arg === '--cursor') options.cursor = next();
|
|
62
|
+
else throw new Error(`Unknown argument ${arg}`);
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
return options;
|
|
66
|
+
};
|
|
67
|
+
|
|
68
|
+
const neo4jHost = (): string => {
|
|
69
|
+
try {
|
|
70
|
+
return new URL(environment.NEO4J_URI).hostname;
|
|
71
|
+
} catch {
|
|
72
|
+
return environment.NEO4J_URI;
|
|
73
|
+
}
|
|
74
|
+
};
|
|
75
|
+
|
|
76
|
+
const estimate = async (pageSize: number) => {
|
|
77
|
+
const generation = getSkillEmbeddingGeneration();
|
|
78
|
+
let cursor: string | null = null;
|
|
79
|
+
let total = 0;
|
|
80
|
+
let needed = 0;
|
|
81
|
+
let chars = 0;
|
|
82
|
+
|
|
83
|
+
while (true) {
|
|
84
|
+
const page = await getSkillEmbeddingCandidates(generation, pageSize, cursor);
|
|
85
|
+
for (const candidate of page.records) {
|
|
86
|
+
total += 1;
|
|
87
|
+
if (skillNeedsEmbedding(candidate, generation)) {
|
|
88
|
+
needed += 1;
|
|
89
|
+
chars += buildSkillEmbeddingText(candidate).length;
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
if (!page.cursor) break;
|
|
93
|
+
cursor = page.cursor;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
const tokens = Math.ceil(chars / ASSUMED_CHARS_PER_TOKEN);
|
|
97
|
+
return {
|
|
98
|
+
generation: generation.identity,
|
|
99
|
+
vectorProperty: generation.vectorProperty,
|
|
100
|
+
indexName: generation.indexName,
|
|
101
|
+
totalSkills: total,
|
|
102
|
+
needEmbedding: needed,
|
|
103
|
+
textChars: chars,
|
|
104
|
+
estimatedTokens: tokens,
|
|
105
|
+
estimatedUsd: Number(
|
|
106
|
+
((tokens / 1_000_000) * NOVA2_USD_PER_MILLION_INPUT_TOKENS).toFixed(4)
|
|
107
|
+
),
|
|
108
|
+
invokeModelCalls: needed,
|
|
109
|
+
};
|
|
110
|
+
};
|
|
111
|
+
|
|
112
|
+
const main = async () => {
|
|
113
|
+
const options = parseArgs(process.argv.slice(2));
|
|
114
|
+
|
|
115
|
+
console.info(`Neo4j host: ${neo4jHost()}`);
|
|
116
|
+
console.info('Dry-run estimate:', await estimate(options.pageSize));
|
|
117
|
+
|
|
118
|
+
if (!options.apply) {
|
|
119
|
+
console.info('Dry run only. Re-run with --apply --target <neo4j-host> to embed.');
|
|
120
|
+
return;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
if (!isSkillEmbeddingEnabled()) throw new Error('SKILL_EMBEDDING_BEDROCK_ENABLED must be true');
|
|
124
|
+
if (options.target !== neo4jHost()) {
|
|
125
|
+
throw new Error('--target must equal the NEO4J_URI host; refusing to write');
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
const lock = await acquireSkillEmbeddingBackfillLock();
|
|
129
|
+
if (!lock) throw new Error('Another skill embedding backfill holds the lock');
|
|
130
|
+
|
|
131
|
+
try {
|
|
132
|
+
const stats = await runSkillEmbeddingBackfill(
|
|
133
|
+
{
|
|
134
|
+
pageSize: options.pageSize,
|
|
135
|
+
maxEmbeddings: options.max,
|
|
136
|
+
cursor: options.cursor,
|
|
137
|
+
},
|
|
138
|
+
{ keepAlive: lock.keepAlive }
|
|
139
|
+
);
|
|
140
|
+
|
|
141
|
+
console.info('Backfill result:', stats);
|
|
142
|
+
if (stats.stoppedReason) {
|
|
143
|
+
console.info(
|
|
144
|
+
`Stopped early (${stats.stoppedReason}). Resume with --cursor ${stats.cursor ?? ''}`
|
|
145
|
+
);
|
|
146
|
+
process.exitCode = 2;
|
|
147
|
+
}
|
|
148
|
+
} finally {
|
|
149
|
+
await lock.release();
|
|
150
|
+
}
|
|
151
|
+
};
|
|
152
|
+
|
|
153
|
+
main()
|
|
154
|
+
.catch(error => {
|
|
155
|
+
console.error(error instanceof Error ? error.message : 'Backfill failed');
|
|
156
|
+
process.exitCode = 1;
|
|
157
|
+
})
|
|
158
|
+
.finally(async () => {
|
|
159
|
+
await neogma.driver.close();
|
|
160
|
+
process.exit();
|
|
161
|
+
});
|
package/serverless-local.yml
CHANGED
|
@@ -62,6 +62,10 @@ provider:
|
|
|
62
62
|
REDIS_PORT:
|
|
63
63
|
'Fn::GetAtt': [ElasticCacheCluster, RedisEndpoint.Port]
|
|
64
64
|
NOTIFICATIONS_QUEUE_URL: ${construct:notifications-queue.queueUrl}
|
|
65
|
+
SKILL_EMBEDDING_BEDROCK_ENABLED: ${env:SKILL_EMBEDDING_BEDROCK_ENABLED, "false"}
|
|
66
|
+
SKILL_EMBEDDING_BEDROCK_MODEL_ID: ${env:SKILL_EMBEDDING_BEDROCK_MODEL_ID, "amazon.nova-2-multimodal-embeddings-v1:0"}
|
|
67
|
+
SKILL_EMBEDDING_DIMENSION: ${env:SKILL_EMBEDDING_DIMENSION, "1024"}
|
|
68
|
+
SKILL_EMBEDDING_BEDROCK_READ_ENABLED: ${env:SKILL_EMBEDDING_BEDROCK_READ_ENABLED, "false"}
|
|
65
69
|
SKILL_EMBEDDING_GOOGLE_API_KEY: ${env:SKILL_EMBEDDING_GOOGLE_API_KEY, ""}
|
|
66
70
|
SKILL_EMBEDDING_BATCH_SIZE: ${env:SKILL_EMBEDDING_BATCH_SIZE, "25"}
|
|
67
71
|
SKILL_EMBEDDING_BACKFILL_ON_STARTUP: ${env:SKILL_EMBEDDING_BACKFILL_ON_STARTUP, "false"}
|
|
@@ -14,6 +14,8 @@ const PROVIDER_SECRETS = [
|
|
|
14
14
|
'MESSAGEBIRD_AUTH_TOKEN',
|
|
15
15
|
'SENTRY_DSN',
|
|
16
16
|
'POSTHOG_API_KEY',
|
|
17
|
+
// LC-2206: legacy Google skill-search read and legacy vector writes; remove after the
|
|
18
|
+
// Nova read flag has been on for the rollback window.
|
|
17
19
|
'SKILL_EMBEDDING_GOOGLE_API_KEY',
|
|
18
20
|
'SKILLS_PROVIDER_API_KEY',
|
|
19
21
|
'TWILIO_ACCOUNT_SID',
|
|
@@ -24,6 +26,7 @@ const PROVIDER_SECRETS = [
|
|
|
24
26
|
'CREDENTIAL_REFRESH_DIGEST_SECRET',
|
|
25
27
|
'SHARE_LINK_REQUEST_HASH_SECRET',
|
|
26
28
|
'APP_STORE_ADMIN_PROFILE_IDS',
|
|
29
|
+
'CAMPAIGN_ADMIN_PROFILE_IDS',
|
|
27
30
|
'LOGIN_PROVIDER_DID',
|
|
28
31
|
'NOTIFICATIONS_SERVICE_WEBHOOK_URL',
|
|
29
32
|
];
|
package/serverless.local.yml
CHANGED
|
@@ -53,6 +53,10 @@ provider:
|
|
|
53
53
|
NEO4J_URI: ${env:NEO4J_URI}
|
|
54
54
|
NEO4J_USERNAME: ${env:NEO4J_USERNAME}
|
|
55
55
|
NEO4J_PASSWORD: ${env:NEO4J_PASSWORD}
|
|
56
|
+
SKILL_EMBEDDING_BEDROCK_ENABLED: ${env:SKILL_EMBEDDING_BEDROCK_ENABLED, "false"}
|
|
57
|
+
SKILL_EMBEDDING_BEDROCK_MODEL_ID: ${env:SKILL_EMBEDDING_BEDROCK_MODEL_ID, "amazon.nova-2-multimodal-embeddings-v1:0"}
|
|
58
|
+
SKILL_EMBEDDING_DIMENSION: ${env:SKILL_EMBEDDING_DIMENSION, "1024"}
|
|
59
|
+
SKILL_EMBEDDING_BEDROCK_READ_ENABLED: ${env:SKILL_EMBEDDING_BEDROCK_READ_ENABLED, "false"}
|
|
56
60
|
SKILL_EMBEDDING_GOOGLE_API_KEY: ${env:SKILL_EMBEDDING_GOOGLE_API_KEY, ""}
|
|
57
61
|
SKILL_EMBEDDING_GOOGLE_MODEL: ${env:SKILL_EMBEDDING_GOOGLE_MODEL, "gemini-embedding-001"}
|
|
58
62
|
SKILL_EMBEDDING_BATCH_SIZE: ${env:SKILL_EMBEDDING_BATCH_SIZE, "25"}
|
package/serverless.yml
CHANGED
|
@@ -141,7 +141,8 @@ provider:
|
|
|
141
141
|
stage: ${opt:stage, "dev"}
|
|
142
142
|
region: ${opt:region, "us-east-1"}
|
|
143
143
|
# Every function shares the default execution role. Grant it read access to the stage's
|
|
144
|
-
# runtime-secrets bundle only; the id itself is passed through RUNTIME_SECRETS_ID.
|
|
144
|
+
# runtime-secrets bundle only; the id itself is passed through RUNTIME_SECRETS_ID. It may
|
|
145
|
+
# also invoke the Nova 2 embeddings model used for skill matching (LC-2206).
|
|
145
146
|
iam:
|
|
146
147
|
role:
|
|
147
148
|
statements:
|
|
@@ -149,6 +150,16 @@ provider:
|
|
|
149
150
|
Action: secretsmanager:GetSecretValue
|
|
150
151
|
Resource:
|
|
151
152
|
Fn::Sub: arn:${AWS::Partition}:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:brain-service/${sls:stage}/runtime-secrets-*
|
|
153
|
+
# LC-2206: skill-matching embeddings. InvokeModel on the one reviewed Nova 2
|
|
154
|
+
# embeddings foundation model, in-Region us-east-1 only (the model has no geo or
|
|
155
|
+
# global inference profile). SigV4 with this role; no Bedrock API keys
|
|
156
|
+
# (bedrock:CallWithBearerToken) are granted. Keep in sync with
|
|
157
|
+
# DEPLOYED_SKILL_EMBEDDING_ALLOWLIST in src/config/skillEmbedding.ts.
|
|
158
|
+
- Sid: SkillEmbeddingNova2Invoke
|
|
159
|
+
Effect: Allow
|
|
160
|
+
Action: bedrock:InvokeModel
|
|
161
|
+
Resource:
|
|
162
|
+
Fn::Sub: arn:${AWS::Partition}:bedrock:us-east-1::foundation-model/amazon.nova-2-multimodal-embeddings-v1:0
|
|
152
163
|
# Bundle mode emits only infra (Redis/queue CF intrinsics, the bundle pointer and the
|
|
153
164
|
# deploy commit); fallback mode retains the existing GitHub credential secrets. Non-secret
|
|
154
165
|
# per-stage tuning, allowlists and OIDC/share-link config are applied from checked-in stage
|
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
# Campaigns: setup, audiences and engagement (LC-2265)
|
|
2
|
+
|
|
3
|
+
This slice stores campaign definitions, server-selected audiences and shared engagement
|
|
4
|
+
state. It does not deliver notifications or render messages. LC-2266 owns dispatch and
|
|
5
|
+
`sendTestToMe`; LC-2267 owns app integration. It does not require the in-app messages V2
|
|
6
|
+
branches: `message` uses the shared `InAppMessage` schema available in this checkout.
|
|
7
|
+
|
|
8
|
+
## Administration and lifecycle
|
|
9
|
+
|
|
10
|
+
Configure `CAMPAIGN_ADMIN_PROFILE_IDS` as a private, comma-separated profile allowlist.
|
|
11
|
+
It is empty by default. Every management operation requires both an allowlisted profile
|
|
12
|
+
and the `campaigns:admin` scope (or `*:*`). The authenticated `isAdmin` discovery query
|
|
13
|
+
returns a boolean, like App Store administration; it exposes no campaigns or audiences.
|
|
14
|
+
User queries need `campaigns:read`, and enrollment/events need `campaigns:write`.
|
|
15
|
+
SDK API tokens also need `profiles:read` for the plugin's profile check.
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
// `client` is an authenticated brain-client; campaignAdmin is allowlisted.
|
|
19
|
+
const campaign = await client.campaigns.create.mutate({
|
|
20
|
+
id: 'transcript-pilot-v1',
|
|
21
|
+
name: 'Transcript pilot',
|
|
22
|
+
channels: ['inApp'],
|
|
23
|
+
message: { title: 'Explore your next step', frequency: 'once' },
|
|
24
|
+
});
|
|
25
|
+
await client.campaigns.setStatus.mutate({ campaignId: campaign.id, status: 'live' });
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
IDs are immutable, lowercase `[a-z0-9-]{3,64}`, unique, and always replace `message.id`.
|
|
29
|
+
Creation defaults to `draft`, `excludeMinors: true`, `selfEnroll: false`. Supported statuses
|
|
30
|
+
are `draft`, `live`, `paused`, `archived`; archived campaigns cannot be edited or have
|
|
31
|
+
audience members added. Only drafts and archived campaigns can be deleted: archive a live or paused
|
|
32
|
+
campaign first. Live and paused campaigns cannot transition back to draft. Admin deletion removes
|
|
33
|
+
the campaign and its relationships.
|
|
34
|
+
Use a new ID for a new campaign; deleting/reusing an ID cannot clear clients' local seen state.
|
|
35
|
+
|
|
36
|
+
`channels` must contain `inApp`, `notify`, or both without duplicates. `notify` requires
|
|
37
|
+
`notification: { title, body }`, but does not send anything in this slice. Dates are ISO
|
|
38
|
+
timestamps normalized to UTC; `startsAt` is inclusive and `endsAt` is exclusive. If both
|
|
39
|
+
exist, the end must follow the start. Partial updates preserve omitted fields; `null`
|
|
40
|
+
clears optional notification/start/end fields. Updates replace `message` as a whole.
|
|
41
|
+
|
|
42
|
+
## Audience sources
|
|
43
|
+
|
|
44
|
+
Preview is read-only. Adding an audience merges distinct profiles and preserves the first
|
|
45
|
+
membership's `addedAt` and `source`, including `selfEnrolled`. Both calls return
|
|
46
|
+
`{ count }`: the number of eligible matching profiles, **not** the number of new edges.
|
|
47
|
+
Overlapping or repeated additions therefore return their matched count without duplicating
|
|
48
|
+
membership. These are snapshots, not subscriptions to later credential claims/revocations.
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
import type { CampaignAudienceSource } from '@learncard/types';
|
|
52
|
+
|
|
53
|
+
const campaignId = 'transcript-pilot-v1';
|
|
54
|
+
const sources: CampaignAudienceSource[] = [
|
|
55
|
+
{ type: 'dids', dids: ['did:key:example'] },
|
|
56
|
+
{ type: 'emails', emails: ['learner@example.org'] },
|
|
57
|
+
{
|
|
58
|
+
type: 'credentialsSentBy',
|
|
59
|
+
senderProfileId: 'university-transcripts',
|
|
60
|
+
metadata: { kind: 'transcript' },
|
|
61
|
+
since: '2026-01-01T00:00:00Z',
|
|
62
|
+
includePending: false,
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
type: 'credentialsSentBy',
|
|
66
|
+
listingId: 'university-transcript-app',
|
|
67
|
+
metadata: { kind: 'transcript' },
|
|
68
|
+
includePending: false,
|
|
69
|
+
},
|
|
70
|
+
{ type: 'inboxClaims', issuerDid: 'did:key:issuer', since: '2026-01-01T00:00:00Z' },
|
|
71
|
+
{
|
|
72
|
+
type: 'boostRecipients',
|
|
73
|
+
boostUri: 'lc:network:network.example.org/trpc:boost:example',
|
|
74
|
+
includePending: false,
|
|
75
|
+
},
|
|
76
|
+
];
|
|
77
|
+
|
|
78
|
+
for (const source of sources) {
|
|
79
|
+
const preview = await client.campaigns.previewAudience.query({ campaignId, source });
|
|
80
|
+
// Review preview.count before adding. Counts can change if source data changes.
|
|
81
|
+
await client.campaigns.addAudience.mutate({ campaignId, source });
|
|
82
|
+
}
|
|
83
|
+
const page = await client.campaigns.pageAudience.query({ campaignId, limit: 100 });
|
|
84
|
+
// Next request: { campaignId, limit: 100, cursor: page.cursor }, while page.hasMore.
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
- DID/email lists accept 1–1000 entries per call. DIDs match stored DIDs or this network's
|
|
88
|
+
canonically encoded `did:web:<domain>:users:<profileId>` alias. Emails are trimmed/lowercased
|
|
89
|
+
and only verified contact methods qualify; the profile's display email is not used.
|
|
90
|
+
- `credentialsSentBy` requires exactly one sender profile or listing. All metadata entries
|
|
91
|
+
must match exactly on `CREDENTIAL_SENT.metadata.*`; `since` filters the send date. Metadata is
|
|
92
|
+
limited to 20 keys, with values of at most 2048 characters.
|
|
93
|
+
Keys are validated and values are bound parameters, never interpolated into Cypher.
|
|
94
|
+
- `inboxClaims` matches the stored issuer DID and filters the **claim** timestamp.
|
|
95
|
+
- Boost URIs must identify a boost on this network. Credential sources exclude revoked
|
|
96
|
+
sends/receipts even with `includePending: true`. Unaccepted sends and pending receipts
|
|
97
|
+
only qualify when that option is true. Recipient matching uses each send's `to` profile.
|
|
98
|
+
- Paging uses ascending profile IDs and reapplies the campaign's current minor policy.
|
|
99
|
+
`getStats().audience` counts stored membership, which may exceed currently eligible paging
|
|
100
|
+
results if the profile or campaign age policy changes.
|
|
101
|
+
|
|
102
|
+
Issuers should use a **dedicated sending profile or listing** for transcripts and attach
|
|
103
|
+
`metadata: { kind: 'transcript' }` when calling `sendCredential`. The server cannot inspect
|
|
104
|
+
encrypted credential contents; a metadata label is not proof of CLR type or ownership.
|
|
105
|
+
|
|
106
|
+
## Two-layer targeting and self-enrollment
|
|
107
|
+
|
|
108
|
+
The server narrows the audience from visible metadata. In the later app-integration slice,
|
|
109
|
+
the device must still confirm a rule such as issuer X **and** `vcTypes: ['ClrCredential']`
|
|
110
|
+
on the same credential before displaying the campaign. This backend does not import the
|
|
111
|
+
V2 targeting engine, evaluate those rules, or send decrypted credentials to the server.
|
|
112
|
+
|
|
113
|
+
`getMyCampaigns` returns live, in-window, enabled `inApp` campaigns for audience members,
|
|
114
|
+
plus eligible campaigns with `selfEnroll: true`, joined to the caller's engagement state.
|
|
115
|
+
Malformed legacy campaign definitions are logged and skipped per record so one bad campaign cannot
|
|
116
|
+
break discovery for every profile.
|
|
117
|
+
Discoverability is not enrollment: `recordEvent` is a no-op until membership exists.
|
|
118
|
+
After a local match, the client can call `enroll({ campaignId })`. Enrollment requires a
|
|
119
|
+
live, in-window, enabled `inApp` campaign with self-enrollment enabled and is idempotent.
|
|
120
|
+
Only profiles with a known adult birthday can self-enroll, even when the campaign has
|
|
121
|
+
`excludeMinors: false`.
|
|
122
|
+
|
|
123
|
+
**Privacy disclosure:** enrollment tells the server only that this profile matched a
|
|
124
|
+
campaign. That membership can itself reveal sensitive information. Authors must write
|
|
125
|
+
rules that reveal no more than needed. No credential content or targeting evidence is
|
|
126
|
+
accepted by the enrollment/event endpoints, and campaign routes disable Sentry input capture.
|
|
127
|
+
|
|
128
|
+
**Product/legal sign-off is still required for the age policy.** The implemented policy
|
|
129
|
+
excludes missing, empty, and non-date-format birthdays from every audience, including when
|
|
130
|
+
`excludeMinors: false`. It also excludes `type: 'child'`, guardian-managed human profiles, and
|
|
131
|
+
known under-18 birthdays when `excludeMinors` is enabled. Managed service/organization profiles
|
|
132
|
+
remain age-exempt unless explicitly typed as a child. Age uses UTC calendar dates; a Feb 29 birthday
|
|
133
|
+
reaches the cutoff on March 1 in a non-leap year. No policy approval or production rollout is implied
|
|
134
|
+
by this PR.
|
|
135
|
+
|
|
136
|
+
## Engagement and SDK
|
|
137
|
+
|
|
138
|
+
`recordEvent({ campaignId, event })` accepts `shown`, `clicked`, `dismissed`, `converted`.
|
|
139
|
+
It returns `false` without profile-lock or engagement writes for missing campaigns, nonmembers,
|
|
140
|
+
age-ineligible profiles, and campaigns that are not currently live and in-window.
|
|
141
|
+
`shown` sets `firstShownAt` once, updates `lastShownAt`, and increments `shownCount`.
|
|
142
|
+
Other events store their latest timestamp. Per-profile transactional locking prevents
|
|
143
|
+
duplicate state edges and lost increments under concurrent devices. Repeated requests
|
|
144
|
+
are separate events, not transport-idempotent retries.
|
|
145
|
+
|
|
146
|
+
Stats count distinct profiles, not impressions; `enrolled` counts memberships originally
|
|
147
|
+
created by self-enrollment. `notified` is reserved for LC-2266 and is not client-writable.
|
|
148
|
+
All user/discovery calls share a 60-call/profile/minute rate limit; admin operations do not.
|
|
149
|
+
|
|
150
|
+
The network plugin exposes `learnCard.invoke.getMyCampaigns()`,
|
|
151
|
+
`recordCampaignEvent(campaignId, event)`, `enrollInCampaign(campaignId)`, and the admin
|
|
152
|
+
methods `campaignsIsAdmin`, `campaignsCreate`, `campaignsUpdate`, `campaignsGet`,
|
|
153
|
+
`campaignsList`, `campaignsDelete`, `campaignsSetStatus`, `campaignsPreviewAudience`,
|
|
154
|
+
`campaignsAddAudience`, `campaignsPageAudience`, `campaignsGetStats`. The brain-client
|
|
155
|
+
derives these routes from `AppRouter`; there is no separate hand-written client schema.
|
|
156
|
+
|
|
157
|
+
## Verification
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
cd services/learn-card-network/brain-service
|
|
161
|
+
bun run test:campaigns
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
This starts a disposable Neo4j testcontainer, uses synthetic accounts/configuration, and
|
|
165
|
+
does not read the developer's database. Docker is required. The suite covers all source
|
|
166
|
+
types, permissions, injection rejection, minor rules, paging, lifecycle, enrollment and
|
|
167
|
+
concurrent shared state. Schema tests live in `@learncard/types`; SDK forwarding tests
|
|
168
|
+
live in `@learncard/network-plugin`. Hosted CI runs the Neo4j suite separately from the
|
|
169
|
+
service's default unit suite. No app UI, notification delivery or device QA is claimed.
|
|
170
|
+
|
|
171
|
+
The hosted Linux verification workflow also runs `test/verification/campaigns.ts` via
|
|
172
|
+
`lc verify script` after the full catalog: real signed SDK calls prove non-admin denial
|
|
173
|
+
and user isolation over HTTP. The default stack grants no admin access, so positive
|
|
174
|
+
administration and graph mutations are covered by the dedicated Neo4j suite instead.
|