@kensio/yulin 1.20.15 → 1.20.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -0
- package/dist/sdk/module/sim-sdk-module-client-interceptor.d.ts +5 -0
- package/dist/sdk/module/sim-sdk-module-client-interceptor.js +14 -0
- package/dist/sdk/module/sim-sdk-static-client-factory.d.ts +26 -0
- package/dist/sdk/module/sim-sdk-static-client-factory.js +59 -0
- package/dist/sdk/send-patch.d.ts +8 -0
- package/dist/sdk/send-patch.js +11 -0
- package/dist/service/aws/caller/sim-aws-caller-resolver.d.ts +28 -6
- package/dist/service/aws/caller/sim-aws-caller-resolver.js +18 -11
- package/dist/service/aws/caller/sim-aws-caller.d.ts +9 -0
- package/dist/service/aws/caller/sim-aws-resolved-caller.d.ts +19 -0
- package/dist/service/aws/caller/sim-aws-resolved-caller.js +26 -0
- package/dist/service/aws/factory/sim-aws-account-region-service-builder.d.ts +6 -0
- package/dist/service/aws/factory/sim-aws-account-region-service-builder.js +3 -0
- package/dist/service/aws/factory/sim-aws-account-service-cache.d.ts +6 -0
- package/dist/service/aws/factory/sim-aws-account-service-cache.js +5 -0
- package/dist/service/aws/factory/sim-aws-service-factory.d.ts +5 -0
- package/dist/service/aws/factory/sim-aws-service-factory.js +6 -0
- package/dist/service/aws/sim-aws-properties.d.ts +15 -0
- package/dist/service/aws/sim-aws.js +2 -1
- package/dist/service/iam/authorize/context/sim-iam-auth-z-caller-context-builder.d.ts +30 -7
- package/dist/service/iam/authorize/context/sim-iam-auth-z-caller-context-builder.js +20 -11
- package/dist/service/iam/authorize/context/sim-iam-auth-z-context-builder.d.ts +2 -1
- package/dist/service/iam/authorize/context/sim-iam-auth-z-context-builder.js +5 -1
- package/dist/service/iam/authorize/match/sim-iam-policy-principal-matcher.d.ts +10 -0
- package/dist/service/iam/authorize/match/sim-iam-policy-principal-matcher.js +15 -3
- package/dist/service/iam/authorize/sim-iam-account-auth-z.d.ts +11 -4
- package/dist/service/iam/authorize/sim-iam-account-auth-z.js +5 -3
- package/dist/service/iam/authorize/sim-iam-authorizer.d.ts +2 -1
- package/dist/service/iam/authorize/sim-iam-authorizer.js +1 -0
- package/dist/service/iam/role/sim-iam-role-with-policy.factory.d.ts +9 -0
- package/dist/service/iam/role/sim-iam-role-with-policy.factory.js +4 -2
- package/dist/service/iam/sim-iam-account-parts.d.ts +8 -0
- package/dist/service/iam/sim-iam-account-parts.js +1 -0
- package/dist/service/sts/auth-z/assume-role-auth-z-coordinator.d.ts +9 -2
- package/dist/service/sts/auth-z/assume-role-auth-z-coordinator.js +2 -2
- package/dist/service/sts/auth-z/assume-role-source-account-auth-z.d.ts +10 -4
- package/dist/service/sts/auth-z/assume-role-source-account-auth-z.js +8 -7
- package/dist/service/sts/auth-z/assume-role-target-auth-z.d.ts +2 -2
- package/dist/service/sts/auth-z/assume-role-trust-policy-authorizer.d.ts +6 -2
- package/dist/service/sts/auth-z/assume-role-trust-policy-authorizer.js +14 -11
- package/dist/service/sts/command/assume-role/assume-role.handler.d.ts +5 -1
- package/dist/service/sts/command/assume-role/assume-role.handler.js +5 -2
- package/dist/service/sts/command/get-caller-identity/get-caller-identity.handler.d.ts +5 -1
- package/dist/service/sts/command/get-caller-identity/get-caller-identity.handler.js +4 -1
- package/dist/service/sts/service-role/sim-service-role.js +5 -1
- package/dist/service/sts/sim-sts.d.ts +9 -1
- package/dist/service/sts/sim-sts.js +4 -0
- package/docs/README.md +54 -0
- package/docs/ai-skill/README.md +69 -0
- package/docs/cli/README.md +284 -0
- package/docs/factories/README.md +140 -0
- package/docs/lint/README.md +176 -0
- package/docs/non-aws-dependencies/README.md +261 -0
- package/docs/sdk/README.md +272 -0
- package/docs/serve/README.md +1126 -0
- package/docs/services/acm/README.md +823 -0
- package/docs/services/apigateway/README.md +1919 -0
- package/docs/services/apigatewayv2/README.md +2678 -0
- package/docs/services/athena/README.md +1044 -0
- package/docs/services/bedrock/README.md +350 -0
- package/docs/services/cloudformation/README.md +3440 -0
- package/docs/services/cloudfront/README.md +2874 -0
- package/docs/services/cloudwatch/README.md +420 -0
- package/docs/services/cognito/README.md +4923 -0
- package/docs/services/dynamodb/README.md +3784 -0
- package/docs/services/ecr/README.md +302 -0
- package/docs/services/ecs/README.md +2010 -0
- package/docs/services/elbv2/README.md +1897 -0
- package/docs/services/eventbridge/README.md +1110 -0
- package/docs/services/firehose/README.md +805 -0
- package/docs/services/glue/README.md +501 -0
- package/docs/services/iam/README.md +1283 -0
- package/docs/services/kinesis/README.md +454 -0
- package/docs/services/kms/README.md +659 -0
- package/docs/services/lambda/README.md +3645 -0
- package/docs/services/logs/README.md +868 -0
- package/docs/services/organizations/README.md +574 -0
- package/docs/services/personalize/README.md +835 -0
- package/docs/services/rekognition/README.md +1070 -0
- package/docs/services/route53/README.md +1783 -0
- package/docs/services/s3/README.md +3024 -0
- package/docs/services/scheduler/README.md +806 -0
- package/docs/services/secretsmanager/README.md +667 -0
- package/docs/services/ses/README.md +1011 -0
- package/docs/services/sns/README.md +1904 -0
- package/docs/services/sqs/README.md +1185 -0
- package/docs/services/ssm/README.md +1072 -0
- package/docs/services/stepfunctions/README.md +1332 -0
- package/docs/services/sts/README.md +245 -0
- package/docs/services/wafv2/README.md +1378 -0
- package/docs/terraform/README.md +248 -0
- package/docs/time/README.md +284 -0
- package/llms.txt +59 -0
- package/package.json +7 -4
|
@@ -0,0 +1,1070 @@
|
|
|
1
|
+
# Simulated Rekognition
|
|
2
|
+
|
|
3
|
+
Simulated Rekognition answers detection calls from results declared against images. A test can say
|
|
4
|
+
which image fails moderation or holds a cat, with no image analysis happening. The bytes are never
|
|
5
|
+
looked at.
|
|
6
|
+
|
|
7
|
+
Rekognition-specific types are imported from the `@kensio/yulin/rekognition` subpath.
|
|
8
|
+
|
|
9
|
+
## Moderating an image
|
|
10
|
+
|
|
11
|
+
`DetectModerationLabels` takes an image as bytes or as an S3 object. Every image is clean until a
|
|
12
|
+
rule says otherwise.
|
|
13
|
+
|
|
14
|
+
```typescript sim-rekognition-detect-moderation-labels
|
|
15
|
+
/**
|
|
16
|
+
* Declaring a moderation result for one S3 object and detecting it.
|
|
17
|
+
*/
|
|
18
|
+
|
|
19
|
+
import { DetectModerationLabelsCommand } from "@aws-sdk/client-rekognition";
|
|
20
|
+
import { CreateBucketCommand, PutObjectCommand } from "@aws-sdk/client-s3";
|
|
21
|
+
|
|
22
|
+
import { SimAws } from "@kensio/yulin";
|
|
23
|
+
|
|
24
|
+
const simAws = new SimAws();
|
|
25
|
+
|
|
26
|
+
await simAws.s3().createBucket(new CreateBucketCommand({ Bucket: "uploads" }));
|
|
27
|
+
await simAws.s3().putObject(
|
|
28
|
+
new PutObjectCommand({
|
|
29
|
+
Bucket: "uploads",
|
|
30
|
+
Key: "incoming/photo.png",
|
|
31
|
+
Body: Buffer.from(
|
|
32
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
33
|
+
"base64",
|
|
34
|
+
),
|
|
35
|
+
}),
|
|
36
|
+
);
|
|
37
|
+
|
|
38
|
+
// The object is declared to fail moderation.
|
|
39
|
+
simAws
|
|
40
|
+
.rekognition()
|
|
41
|
+
.moderation()
|
|
42
|
+
.onName("incoming/photo.png", { labels: ["Weapons"] });
|
|
43
|
+
|
|
44
|
+
const detected = await simAws.rekognition().detectModerationLabels(
|
|
45
|
+
new DetectModerationLabelsCommand({
|
|
46
|
+
Image: { S3Object: { Bucket: "uploads", Name: "incoming/photo.png" } },
|
|
47
|
+
}),
|
|
48
|
+
);
|
|
49
|
+
|
|
50
|
+
console.log(detected.ModerationLabels.map((label) => label.Name));
|
|
51
|
+
// [ "Violence", "Weapons" ]
|
|
52
|
+
console.log(detected.ModerationModelVersion); // "7.0"
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The image is read through simulated S3 as the caller making the detection. The caller needs
|
|
56
|
+
`s3:GetObject` for it as well as `rekognition:DetectModerationLabels`.
|
|
57
|
+
|
|
58
|
+
Image bytes go in as `Image.Bytes` instead, which needs no Bucket:
|
|
59
|
+
|
|
60
|
+
```typescript
|
|
61
|
+
const detected = await simAws
|
|
62
|
+
.rekognition()
|
|
63
|
+
.detectModerationLabels(
|
|
64
|
+
new DetectModerationLabelsCommand({ Image: { Bytes: imageBytes } }),
|
|
65
|
+
);
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Detecting labels in an image
|
|
69
|
+
|
|
70
|
+
`DetectLabels` answers with the objects, scenes and concepts an image is declared to hold. Each
|
|
71
|
+
label carries the parents, aliases, categories and instances it was declared with, and no more. A
|
|
72
|
+
label is reported as written.
|
|
73
|
+
|
|
74
|
+
```typescript sim-rekognition-detect-labels
|
|
75
|
+
/**
|
|
76
|
+
* Declaring the labels for one S3 object and detecting them.
|
|
77
|
+
*/
|
|
78
|
+
|
|
79
|
+
import { DetectLabelsCommand } from "@aws-sdk/client-rekognition";
|
|
80
|
+
import { CreateBucketCommand, PutObjectCommand } from "@aws-sdk/client-s3";
|
|
81
|
+
|
|
82
|
+
import { SimAws } from "@kensio/yulin";
|
|
83
|
+
|
|
84
|
+
const simAws = new SimAws();
|
|
85
|
+
|
|
86
|
+
await simAws.s3().createBucket(new CreateBucketCommand({ Bucket: "uploads" }));
|
|
87
|
+
await simAws.s3().putObject(
|
|
88
|
+
new PutObjectCommand({
|
|
89
|
+
Bucket: "uploads",
|
|
90
|
+
Key: "incoming/cat.png",
|
|
91
|
+
Body: Buffer.from(
|
|
92
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
93
|
+
"base64",
|
|
94
|
+
),
|
|
95
|
+
}),
|
|
96
|
+
);
|
|
97
|
+
|
|
98
|
+
simAws
|
|
99
|
+
.rekognition()
|
|
100
|
+
.labels()
|
|
101
|
+
.onName("incoming/cat.png", {
|
|
102
|
+
labels: [
|
|
103
|
+
{
|
|
104
|
+
name: "Cat",
|
|
105
|
+
confidence: 98.2,
|
|
106
|
+
parents: ["Animal", "Pet", "Feline"],
|
|
107
|
+
aliases: ["Kitten"],
|
|
108
|
+
categories: ["Animals and Pets"],
|
|
109
|
+
// A bounding box is in ratios of the image size, as AWS reports it.
|
|
110
|
+
instances: [
|
|
111
|
+
{ boundingBox: { left: 0.36, top: 0.09, width: 0.26, height: 0.85 } },
|
|
112
|
+
],
|
|
113
|
+
},
|
|
114
|
+
{ name: "Grass", confidence: 71.4 },
|
|
115
|
+
],
|
|
116
|
+
});
|
|
117
|
+
|
|
118
|
+
const detected = await simAws.rekognition().detectLabels(
|
|
119
|
+
new DetectLabelsCommand({
|
|
120
|
+
Image: { S3Object: { Bucket: "uploads", Name: "incoming/cat.png" } },
|
|
121
|
+
MaxLabels: 10,
|
|
122
|
+
}),
|
|
123
|
+
);
|
|
124
|
+
|
|
125
|
+
console.log(detected.Labels.map((label) => label.Name)); // [ "Cat", "Grass" ]
|
|
126
|
+
console.log(detected.Labels[0]?.Parents);
|
|
127
|
+
// [ { Name: "Animal" }, { Name: "Pet" }, { Name: "Feline" } ]
|
|
128
|
+
console.log(detected.LabelModelVersion); // "3.0"
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
Labels come back in descending order of confidence, which is the order real Rekognition reports them
|
|
132
|
+
in. A declared instance with no confidence of its own takes its label's.
|
|
133
|
+
|
|
134
|
+
An image no rule matches gets the built-in default result. That is the one `Mobile Phone` label from
|
|
135
|
+
the example response in the AWS `DetectLabels` documentation, with the parent, alias, category and
|
|
136
|
+
bounding box AWS documents it with. It is a real Rekognition response, though which labels an
|
|
137
|
+
unconfigured image gets is a simulator convention rather than what AWS would return for it.
|
|
138
|
+
|
|
139
|
+
A label name fills in nothing of its own. Declaring `Cat` with no parents reports `Cat` with no
|
|
140
|
+
parents, and declaring a `Pizza` nobody has heard of reports `Pizza`. Yulin ships no general label
|
|
141
|
+
ontology to check a name against or to expand one from.
|
|
142
|
+
|
|
143
|
+
## Detecting faces in an image
|
|
144
|
+
|
|
145
|
+
`DetectFaces` answers with the faces an image is declared to hold. A face says where it is and what
|
|
146
|
+
it looks like, and the response carries the attributes the request asked for.
|
|
147
|
+
|
|
148
|
+
```typescript sim-rekognition-detect-faces
|
|
149
|
+
/**
|
|
150
|
+
* Declaring the faces in one S3 object and detecting them.
|
|
151
|
+
*/
|
|
152
|
+
|
|
153
|
+
import { DetectFacesCommand } from "@aws-sdk/client-rekognition";
|
|
154
|
+
import { CreateBucketCommand, PutObjectCommand } from "@aws-sdk/client-s3";
|
|
155
|
+
|
|
156
|
+
import { SimAws } from "@kensio/yulin";
|
|
157
|
+
|
|
158
|
+
const simAws = new SimAws();
|
|
159
|
+
|
|
160
|
+
await simAws.s3().createBucket(new CreateBucketCommand({ Bucket: "uploads" }));
|
|
161
|
+
await simAws.s3().putObject(
|
|
162
|
+
new PutObjectCommand({
|
|
163
|
+
Bucket: "uploads",
|
|
164
|
+
Key: "incoming/selfie.png",
|
|
165
|
+
Body: Buffer.from(
|
|
166
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
167
|
+
"base64",
|
|
168
|
+
),
|
|
169
|
+
}),
|
|
170
|
+
);
|
|
171
|
+
|
|
172
|
+
simAws
|
|
173
|
+
.rekognition()
|
|
174
|
+
.faces()
|
|
175
|
+
.onName("incoming/selfie.png", {
|
|
176
|
+
faces: [
|
|
177
|
+
{
|
|
178
|
+
// A bounding box is in ratios of the image size, as AWS reports it.
|
|
179
|
+
boundingBox: { left: 0.3, top: 0.2, width: 0.3, height: 0.4 },
|
|
180
|
+
confidence: 99.4,
|
|
181
|
+
ageRange: { low: 18, high: 26 },
|
|
182
|
+
gender: "Female",
|
|
183
|
+
smile: true,
|
|
184
|
+
sunglasses: { value: false, confidence: 99.9 },
|
|
185
|
+
emotions: ["CALM"],
|
|
186
|
+
},
|
|
187
|
+
],
|
|
188
|
+
});
|
|
189
|
+
|
|
190
|
+
const detected = await simAws.rekognition().detectFaces(
|
|
191
|
+
new DetectFacesCommand({
|
|
192
|
+
Image: { S3Object: { Bucket: "uploads", Name: "incoming/selfie.png" } },
|
|
193
|
+
Attributes: ["ALL"],
|
|
194
|
+
}),
|
|
195
|
+
);
|
|
196
|
+
|
|
197
|
+
console.log(detected.FaceDetails.length); // 1
|
|
198
|
+
console.log(detected.FaceDetails[0]?.AgeRange); // { Low: 18, High: 26 }
|
|
199
|
+
console.log(detected.FaceDetails[0]?.Smile);
|
|
200
|
+
// { Value: true, Confidence: 99.4000015258789 }
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
Faces come back in the order they were declared. An attribute with no confidence of its own takes
|
|
204
|
+
the face's, and a face detected at 99.4 is reported as smiling at 99.4. A face declared with no
|
|
205
|
+
confidence at all is detected at the built-in one.
|
|
206
|
+
|
|
207
|
+
An image with nobody in it is `{ faces: [] }`. Two built-in results cover the counting a test
|
|
208
|
+
usually does:
|
|
209
|
+
|
|
210
|
+
```typescript
|
|
211
|
+
import {
|
|
212
|
+
simRekognitionNoFaces,
|
|
213
|
+
simRekognitionSeveralFaces,
|
|
214
|
+
} from "@kensio/yulin/rekognition";
|
|
215
|
+
|
|
216
|
+
const faces = simAws.rekognition().faces();
|
|
217
|
+
|
|
218
|
+
faces.onName("incoming/landscape.png", simRekognitionNoFaces);
|
|
219
|
+
faces.onName("incoming/crowd.png", simRekognitionSeveralFaces);
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
An image no rule matches gets the built-in default result. That is the one face from the example
|
|
223
|
+
response in the AWS `DetectFaces` documentation, with the attributes and all thirty landmarks AWS
|
|
224
|
+
documents it with. It is a real Rekognition response, though which face an unconfigured image gets is
|
|
225
|
+
a simulator convention rather than what AWS would return for it.
|
|
226
|
+
|
|
227
|
+
## Choosing the facial attributes
|
|
228
|
+
|
|
229
|
+
`BoundingBox`, `Confidence`, `Pose`, `Quality` and `Landmarks` come back whatever a request asked
|
|
230
|
+
for, being the default subset AWS always returns. `ALL` adds the rest, and naming one attribute adds
|
|
231
|
+
that one, so `["FACE_OCCLUDED"]` is the default subset with face occlusion on top.
|
|
232
|
+
`["ALL", "DEFAULT"]` is the union the two describe together.
|
|
233
|
+
|
|
234
|
+
Landmarks follow AWS too. Five come back unless `ALL` was asked for, and every declared landmark
|
|
235
|
+
when it was.
|
|
236
|
+
|
|
237
|
+
```typescript sim-rekognition-face-attributes
|
|
238
|
+
/**
|
|
239
|
+
* One face detected twice, with the default attributes and with ALL.
|
|
240
|
+
*/
|
|
241
|
+
|
|
242
|
+
import { DetectFacesCommand } from "@aws-sdk/client-rekognition";
|
|
243
|
+
|
|
244
|
+
import { SimAws } from "@kensio/yulin";
|
|
245
|
+
|
|
246
|
+
const simAws = new SimAws();
|
|
247
|
+
const imageBytes = Buffer.from(
|
|
248
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
249
|
+
"base64",
|
|
250
|
+
);
|
|
251
|
+
|
|
252
|
+
simAws
|
|
253
|
+
.rekognition()
|
|
254
|
+
.faces()
|
|
255
|
+
.byDefault({
|
|
256
|
+
faces: [
|
|
257
|
+
{
|
|
258
|
+
boundingBox: { left: 0.3, top: 0.2, width: 0.3, height: 0.4 },
|
|
259
|
+
confidence: 99.4,
|
|
260
|
+
landmarks: {
|
|
261
|
+
eyeLeft: { x: 0.35, y: 0.3 },
|
|
262
|
+
eyeRight: { x: 0.5, y: 0.3 },
|
|
263
|
+
chinBottom: { x: 0.43, y: 0.62 },
|
|
264
|
+
},
|
|
265
|
+
smile: true,
|
|
266
|
+
},
|
|
267
|
+
],
|
|
268
|
+
});
|
|
269
|
+
|
|
270
|
+
const byDefault = await simAws
|
|
271
|
+
.rekognition()
|
|
272
|
+
.detectFaces(new DetectFacesCommand({ Image: { Bytes: imageBytes } }));
|
|
273
|
+
|
|
274
|
+
console.log(Object.keys(byDefault.FaceDetails[0] ?? {}));
|
|
275
|
+
// [ "BoundingBox", "Confidence", "Landmarks" ]
|
|
276
|
+
console.log(
|
|
277
|
+
byDefault.FaceDetails[0]?.Landmarks?.map((landmark) => landmark.Type),
|
|
278
|
+
);
|
|
279
|
+
// [ "eyeLeft", "eyeRight" ]
|
|
280
|
+
|
|
281
|
+
const everything = await simAws.rekognition().detectFaces(
|
|
282
|
+
new DetectFacesCommand({
|
|
283
|
+
Image: { Bytes: imageBytes },
|
|
284
|
+
Attributes: ["ALL"],
|
|
285
|
+
}),
|
|
286
|
+
);
|
|
287
|
+
|
|
288
|
+
console.log(everything.FaceDetails[0]?.Smile?.Value); // true
|
|
289
|
+
console.log(
|
|
290
|
+
everything.FaceDetails[0]?.Landmarks?.map((landmark) => landmark.Type),
|
|
291
|
+
);
|
|
292
|
+
// [ "eyeLeft", "eyeRight", "chinBottom" ]
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
An undeclared attribute is left out of the response, in place of coming back empty. A face declared
|
|
296
|
+
with a bounding box and no more comes back as a bounding box and a confidence, however many
|
|
297
|
+
attributes the request asked for.
|
|
298
|
+
|
|
299
|
+
A declaration is checked where it is written. A bounding box or a landmark outside the image is
|
|
300
|
+
refused, as is an age range that ends before it begins, an emotion Rekognition never reports, and a
|
|
301
|
+
pair of landmarks that runs the wrong way across the face, such as an `eyeLeft` to the right of
|
|
302
|
+
`eyeRight`. So is a result declaring more than a hundred faces, the most real Rekognition detects in
|
|
303
|
+
one image. A landmark may sit outside the bounding box, because a real Rekognition face box routinely
|
|
304
|
+
excludes the chin.
|
|
305
|
+
|
|
306
|
+
## Declaring results
|
|
307
|
+
|
|
308
|
+
Results are declared per operation. `moderation()` holds the rules `DetectModerationLabels` answers
|
|
309
|
+
from, `labels()` holds the rules `DetectLabels` answers from, `faces()` holds the rules
|
|
310
|
+
`DetectFaces` answers from, and `faceMatches()` holds the rules `SearchFacesByImage` answers from.
|
|
311
|
+
All four take the same three kinds of rule, being an exact S3 object name, an exact content hash, or
|
|
312
|
+
anything at all.
|
|
313
|
+
|
|
314
|
+
```typescript sim-rekognition-moderation-rules
|
|
315
|
+
/**
|
|
316
|
+
* The three kinds of rule, and which one wins.
|
|
317
|
+
*/
|
|
318
|
+
|
|
319
|
+
import { SimAws } from "@kensio/yulin";
|
|
320
|
+
import { simRekognitionImageHash } from "@kensio/yulin/rekognition";
|
|
321
|
+
|
|
322
|
+
const simAws = new SimAws();
|
|
323
|
+
const moderation = simAws.rekognition().moderation();
|
|
324
|
+
|
|
325
|
+
// Everything not matched by another rule.
|
|
326
|
+
moderation.byDefault({ labels: [] });
|
|
327
|
+
|
|
328
|
+
// One S3 object, by the Name a request gives Rekognition.
|
|
329
|
+
moderation.onName("incoming/photo.png", { labels: ["Weapons"] });
|
|
330
|
+
|
|
331
|
+
// One image, by the hash of its bytes, for a system that generates its own
|
|
332
|
+
// object keys. These bytes would usually come from a fixture file, read with
|
|
333
|
+
// readFileSync, and the hash is of the exact bytes the test uploads.
|
|
334
|
+
const fixture = Buffer.from(
|
|
335
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQs7kDAAGyATf/cv8XAAAAAElFTkSuQmCC",
|
|
336
|
+
"base64",
|
|
337
|
+
);
|
|
338
|
+
moderation.onHash(simRekognitionImageHash(fixture), {
|
|
339
|
+
labels: [{ name: "Weapon Violence", confidence: 88.4 }],
|
|
340
|
+
});
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
A hash rule wins, then a name rule, then the default. Matching is exact, with no pattern syntax.
|
|
344
|
+
Which rule applies never depends on how specific a pattern looks.
|
|
345
|
+
|
|
346
|
+
A name is the `Name` in the request, the S3 object key. It is matched on its own, with the Bucket
|
|
347
|
+
left out, so a rule for a key applies to that key in whichever Bucket the request names. An image
|
|
348
|
+
passed as `Image.Bytes` has no name at all, and consults hash rules and then the default.
|
|
349
|
+
|
|
350
|
+
The hash is the sha256 digest of the image bytes as they were received, as lowercase hex.
|
|
351
|
+
`simRekognitionImageHash` produces it from a fixture. Re-encoding an image between uploading it and
|
|
352
|
+
detecting on it changes the digest, so hash the exact bytes the test puts through the system.
|
|
353
|
+
|
|
354
|
+
A label can be declared as a name on its own, or as a name with what is to be reported alongside it.
|
|
355
|
+
A moderation label declared as a name reports at a confidence of `96.68000030517578`, and a
|
|
356
|
+
detection label at `97.53010559082031`.
|
|
357
|
+
|
|
358
|
+
`faceMatches()` declares people where the other three declare labels. A match names one indexed
|
|
359
|
+
face, by the `ExternalImageId` it was indexed under or by the `FaceId` `IndexFaces` answered with,
|
|
360
|
+
and says how alike the search reports it as.
|
|
361
|
+
|
|
362
|
+
```typescript sim-rekognition-face-match-rules
|
|
363
|
+
/**
|
|
364
|
+
* The two ways a rule names the face a search finds.
|
|
365
|
+
*/
|
|
366
|
+
|
|
367
|
+
import {
|
|
368
|
+
CreateCollectionCommand,
|
|
369
|
+
IndexFacesCommand,
|
|
370
|
+
} from "@aws-sdk/client-rekognition";
|
|
371
|
+
import { SimAws } from "@kensio/yulin";
|
|
372
|
+
import { simRekognitionSampleImages } from "@kensio/yulin/rekognition";
|
|
373
|
+
|
|
374
|
+
const simAws = new SimAws();
|
|
375
|
+
const simRekognition = simAws.rekognition();
|
|
376
|
+
const faceMatches = simRekognition.faceMatches();
|
|
377
|
+
|
|
378
|
+
// Every image starts here, finding nobody.
|
|
379
|
+
faceMatches.byDefault({ matches: [] });
|
|
380
|
+
|
|
381
|
+
// By the external image id the indexing request gave the face. A test can
|
|
382
|
+
// write this before anything is indexed.
|
|
383
|
+
faceMatches.onName("door/visitor.jpg", {
|
|
384
|
+
matches: [{ externalImageId: "ada", similarity: 98.5 }],
|
|
385
|
+
});
|
|
386
|
+
|
|
387
|
+
// By the id IndexFaces answered with, for an application that keeps it.
|
|
388
|
+
await simRekognition.createCollection(
|
|
389
|
+
new CreateCollectionCommand({ CollectionId: "staff" }),
|
|
390
|
+
);
|
|
391
|
+
|
|
392
|
+
const indexed = await simRekognition.indexFaces(
|
|
393
|
+
new IndexFacesCommand({
|
|
394
|
+
CollectionId: "staff",
|
|
395
|
+
Image: { Bytes: simRekognitionSampleImages.oneFace() },
|
|
396
|
+
}),
|
|
397
|
+
);
|
|
398
|
+
|
|
399
|
+
faceMatches.onName("door/courier.jpg", {
|
|
400
|
+
matches: indexed.FaceRecords.map((record) => ({
|
|
401
|
+
faceId: record.Face.FaceId,
|
|
402
|
+
})),
|
|
403
|
+
});
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
An `externalImageId` rule can be written before anything is indexed. That suits a test whose own
|
|
407
|
+
code registers the face. A `faceId` rule is written after the indexing that issued the id, and names
|
|
408
|
+
one face exactly. Where the same external image id covers several faces, each one comes back as its
|
|
409
|
+
own match. A match that states no similarity reports at `99.97222137451172`, the similarity in the
|
|
410
|
+
AWS `SearchFacesByImage` example response. Declaring both kinds of id on one match, or neither, is
|
|
411
|
+
refused where the rule is written.
|
|
412
|
+
|
|
413
|
+
## Sample images
|
|
414
|
+
|
|
415
|
+
Simulated Rekognition ships with five images whose hashes are already declared. A test uploads one
|
|
416
|
+
through its own code and gets a known answer without registering anything. That is what makes an
|
|
417
|
+
application generating its own object keys testable, since the test never has to know the key.
|
|
418
|
+
|
|
419
|
+
| Image | Format | Detected as |
|
|
420
|
+
| -------------------------------------------------- | ------ | ------------------------------------------------- |
|
|
421
|
+
| `simRekognitionSampleImages.passesModeration()` | PNG | no moderation labels |
|
|
422
|
+
| `simRekognitionSampleImages.flaggedByModeration()` | JPEG | `Violence`, `Graphic Violence`, `Weapon Violence` |
|
|
423
|
+
| `simRekognitionSampleImages.noFaces()` | PNG | no faces |
|
|
424
|
+
| `simRekognitionSampleImages.oneFace()` | JPEG | one face, the built-in default face |
|
|
425
|
+
| `simRekognitionSampleImages.severalFaces()` | PNG | three faces |
|
|
426
|
+
|
|
427
|
+
```typescript sim-rekognition-sample-images
|
|
428
|
+
/**
|
|
429
|
+
* A sample image uploaded under a key the application invented.
|
|
430
|
+
*/
|
|
431
|
+
|
|
432
|
+
import { randomUUID } from "node:crypto";
|
|
433
|
+
|
|
434
|
+
import { DetectModerationLabelsCommand } from "@aws-sdk/client-rekognition";
|
|
435
|
+
import { CreateBucketCommand, PutObjectCommand } from "@aws-sdk/client-s3";
|
|
436
|
+
|
|
437
|
+
import { SimAws } from "@kensio/yulin";
|
|
438
|
+
import { simRekognitionSampleImages } from "@kensio/yulin/rekognition";
|
|
439
|
+
|
|
440
|
+
const simAws = new SimAws();
|
|
441
|
+
await simAws.s3().createBucket(new CreateBucketCommand({ Bucket: "uploads" }));
|
|
442
|
+
|
|
443
|
+
const key = `incoming/${randomUUID()}.jpg`;
|
|
444
|
+
|
|
445
|
+
await simAws.s3().putObject(
|
|
446
|
+
new PutObjectCommand({
|
|
447
|
+
Bucket: "uploads",
|
|
448
|
+
Key: key,
|
|
449
|
+
Body: simRekognitionSampleImages.flaggedByModeration(),
|
|
450
|
+
}),
|
|
451
|
+
);
|
|
452
|
+
|
|
453
|
+
const detected = await simAws.rekognition().detectModerationLabels(
|
|
454
|
+
new DetectModerationLabelsCommand({
|
|
455
|
+
Image: { S3Object: { Bucket: "uploads", Name: key } },
|
|
456
|
+
}),
|
|
457
|
+
);
|
|
458
|
+
|
|
459
|
+
console.log(detected.ModerationLabels.map((label) => label.Name));
|
|
460
|
+
// [ "Violence", "Graphic Violence", "Weapon Violence" ]
|
|
461
|
+
```
|
|
462
|
+
|
|
463
|
+
Each image is declared for the one operation it is named for. The moderation images say nothing
|
|
464
|
+
about faces and the face images say nothing about moderation. Those detections answer from their own
|
|
465
|
+
rules as they would for any other image.
|
|
466
|
+
|
|
467
|
+
The built-in rules are ordinary hash rules registered when the service is made, and declaring a rule
|
|
468
|
+
for the same image replaces it. The precedence matters here. A hash rule beats a name rule, so a
|
|
469
|
+
sample image is overridden by hash, and never by the key it was uploaded under.
|
|
470
|
+
|
|
471
|
+
```typescript
|
|
472
|
+
const sample = simRekognitionSampleImages.flaggedByModeration();
|
|
473
|
+
|
|
474
|
+
simAws
|
|
475
|
+
.rekognition()
|
|
476
|
+
.moderation()
|
|
477
|
+
.onHash(simRekognitionImageHash(sample), { labels: [] });
|
|
478
|
+
```
|
|
479
|
+
|
|
480
|
+
The images are real 16 by 16 PNG and JPEG files, 1,909 bytes in total. The format check reads their
|
|
481
|
+
magic bytes as it does for any other image. What they are pictures of decides nothing, since no
|
|
482
|
+
image is looked at.
|
|
483
|
+
|
|
484
|
+
## Moderation labels come back with their parents
|
|
485
|
+
|
|
486
|
+
A declared label expands to its whole chain in the version 7.0 moderation taxonomy. Handler code
|
|
487
|
+
that filters on the top-level category sees what it would see on AWS. Each label carries the
|
|
488
|
+
`ParentName` and `TaxonomyLevel` real Rekognition reports.
|
|
489
|
+
|
|
490
|
+
```typescript sim-rekognition-taxonomy-chain
|
|
491
|
+
/**
|
|
492
|
+
* A third level label arrives with the two labels above it.
|
|
493
|
+
*/
|
|
494
|
+
|
|
495
|
+
import { DetectModerationLabelsCommand } from "@aws-sdk/client-rekognition";
|
|
496
|
+
|
|
497
|
+
import { SimAws } from "@kensio/yulin";
|
|
498
|
+
|
|
499
|
+
const simAws = new SimAws();
|
|
500
|
+
const imageBytes = Buffer.from(
|
|
501
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
502
|
+
"base64",
|
|
503
|
+
);
|
|
504
|
+
|
|
505
|
+
simAws
|
|
506
|
+
.rekognition()
|
|
507
|
+
.moderation()
|
|
508
|
+
.byDefault({ labels: [{ name: "Drinking", confidence: 92 }] });
|
|
509
|
+
|
|
510
|
+
const detected = await simAws
|
|
511
|
+
.rekognition()
|
|
512
|
+
.detectModerationLabels(
|
|
513
|
+
new DetectModerationLabelsCommand({ Image: { Bytes: imageBytes } }),
|
|
514
|
+
);
|
|
515
|
+
|
|
516
|
+
console.log(detected.ModerationLabels);
|
|
517
|
+
// [
|
|
518
|
+
// { Name: "Alcohol", ParentName: "", TaxonomyLevel: 1, Confidence: 92 },
|
|
519
|
+
// { Name: "Alcohol Use", ParentName: "Alcohol", TaxonomyLevel: 2, ... },
|
|
520
|
+
// { Name: "Drinking", ParentName: "Alcohol Use", TaxonomyLevel: 3, ... },
|
|
521
|
+
// ]
|
|
522
|
+
```
|
|
523
|
+
|
|
524
|
+
Every label in one chain shares that chain's confidence, and `MinConfidence` filters whole chains. A
|
|
525
|
+
surviving label always names a parent the response carries. A label two chains share is reported
|
|
526
|
+
once, at the higher of the two confidences.
|
|
527
|
+
|
|
528
|
+
A label outside the taxonomy is refused where it is declared, ahead of detection time. That includes
|
|
529
|
+
a version 6.1 name that version 7.0 dropped, such as `Drug Products`, which became `Products` under
|
|
530
|
+
`Drugs & Tobacco`. Some names survived the move with a different place in the taxonomy. `Drinking`
|
|
531
|
+
is still a label, and it now sits under `Alcohol Use` rather than directly under `Alcohol`.
|
|
532
|
+
|
|
533
|
+
## Filtering by confidence
|
|
534
|
+
|
|
535
|
+
`MinConfidence` compares inclusively and defaults to what the operation defaults to on AWS, being 50
|
|
536
|
+
for `DetectModerationLabels` and 55 for `DetectLabels`. An explicit `0` asks for every label, and is
|
|
537
|
+
never read as unset.
|
|
538
|
+
|
|
539
|
+
```typescript sim-rekognition-min-confidence
|
|
540
|
+
/**
|
|
541
|
+
* Two labels declared with different confidences, filtered by the request.
|
|
542
|
+
*/
|
|
543
|
+
|
|
544
|
+
import { DetectModerationLabelsCommand } from "@aws-sdk/client-rekognition";
|
|
545
|
+
|
|
546
|
+
import { SimAws } from "@kensio/yulin";
|
|
547
|
+
|
|
548
|
+
const simAws = new SimAws();
|
|
549
|
+
const imageBytes = Buffer.from(
|
|
550
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
551
|
+
"base64",
|
|
552
|
+
);
|
|
553
|
+
|
|
554
|
+
simAws
|
|
555
|
+
.rekognition()
|
|
556
|
+
.moderation()
|
|
557
|
+
.byDefault({
|
|
558
|
+
labels: [
|
|
559
|
+
{ name: "Weapons", confidence: 96 },
|
|
560
|
+
{ name: "Gambling", confidence: 41 },
|
|
561
|
+
],
|
|
562
|
+
});
|
|
563
|
+
|
|
564
|
+
const strict = await simAws.rekognition().detectModerationLabels(
|
|
565
|
+
new DetectModerationLabelsCommand({
|
|
566
|
+
Image: { Bytes: imageBytes },
|
|
567
|
+
MinConfidence: 80,
|
|
568
|
+
}),
|
|
569
|
+
);
|
|
570
|
+
|
|
571
|
+
console.log(strict.ModerationLabels.map((label) => label.Name));
|
|
572
|
+
// [ "Violence", "Weapons" ]
|
|
573
|
+
```
|
|
574
|
+
|
|
575
|
+
Confidences are float32 values, as real Rekognition confidences are, and a declared `99.4` comes
|
|
576
|
+
back as `99.4000015258789`.
|
|
577
|
+
|
|
578
|
+
`DetectLabels` also takes a `MaxLabels`, which applies after the confidence filter and keeps the most
|
|
579
|
+
confident labels of the ones that survived it. An explicit `0` asks for no labels, and is never read
|
|
580
|
+
as unset.
|
|
581
|
+
|
|
582
|
+
```typescript sim-rekognition-max-labels
|
|
583
|
+
/**
|
|
584
|
+
* Three labels, narrowed by confidence and then by how many were asked for.
|
|
585
|
+
*/
|
|
586
|
+
|
|
587
|
+
import { DetectLabelsCommand } from "@aws-sdk/client-rekognition";
|
|
588
|
+
|
|
589
|
+
import { SimAws } from "@kensio/yulin";
|
|
590
|
+
|
|
591
|
+
const simAws = new SimAws();
|
|
592
|
+
const imageBytes = Buffer.from(
|
|
593
|
+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGO4I2IDAAL8AS3VzMq8AAAAAElFTkSuQmCC",
|
|
594
|
+
"base64",
|
|
595
|
+
);
|
|
596
|
+
|
|
597
|
+
simAws
|
|
598
|
+
.rekognition()
|
|
599
|
+
.labels()
|
|
600
|
+
.byDefault({
|
|
601
|
+
labels: [
|
|
602
|
+
{ name: "Cat", confidence: 98.2 },
|
|
603
|
+
{ name: "Grass", confidence: 88 },
|
|
604
|
+
{ name: "Fence", confidence: 62 },
|
|
605
|
+
],
|
|
606
|
+
});
|
|
607
|
+
|
|
608
|
+
const detected = await simAws.rekognition().detectLabels(
|
|
609
|
+
new DetectLabelsCommand({
|
|
610
|
+
Image: { Bytes: imageBytes },
|
|
611
|
+
MinConfidence: 80,
|
|
612
|
+
MaxLabels: 2,
|
|
613
|
+
}),
|
|
614
|
+
);
|
|
615
|
+
|
|
616
|
+
console.log(detected.Labels.map((label) => label.Name)); // [ "Cat", "Grass" ]
|
|
617
|
+
```
|
|
618
|
+
|
|
619
|
+
## Moderating an upload
|
|
620
|
+
|
|
621
|
+
An upload can moderate itself. A Bucket notification invokes a function, and the function moderates
|
|
622
|
+
the object the event names. The function calls Rekognition in the Account and Region it runs in, so
|
|
623
|
+
the rules a test registers on `simAws.rekognition()` are the ones it finds.
|
|
624
|
+
|
|
625
|
+
This is the flow the sample images exist for. The object goes in under a key the application
|
|
626
|
+
generated, and the sample image's own hash rule decides the result, leaving the test with no key to
|
|
627
|
+
name.
|
|
628
|
+
|
|
629
|
+
```typescript sim-rekognition-upload-pipeline
|
|
630
|
+
/**
|
|
631
|
+
* An upload moderated by the Lambda function its Bucket notifies.
|
|
632
|
+
*/
|
|
633
|
+
|
|
634
|
+
import { randomUUID } from "node:crypto";
|
|
635
|
+
|
|
636
|
+
import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
|
|
637
|
+
import {
|
|
638
|
+
AddPermissionCommand,
|
|
639
|
+
CreateFunctionCommand,
|
|
640
|
+
} from "@aws-sdk/client-lambda";
|
|
641
|
+
import {
|
|
642
|
+
CreateBucketCommand,
|
|
643
|
+
PutBucketNotificationConfigurationCommand,
|
|
644
|
+
PutObjectCommand,
|
|
645
|
+
} from "@aws-sdk/client-s3";
|
|
646
|
+
|
|
647
|
+
import { SimAws } from "@kensio/yulin";
|
|
648
|
+
import { makeLambdaCodeZip } from "@kensio/yulin/lambda";
|
|
649
|
+
import { simRekognitionSampleImages } from "@kensio/yulin/rekognition";
|
|
650
|
+
|
|
651
|
+
const simAws = new SimAws();
|
|
652
|
+
const moderatorArn = `arn:aws:lambda:${simAws.defaultRegionName}:${simAws.defaultAccountId}:function:moderator`;
|
|
653
|
+
|
|
654
|
+
const role = await simAws.iam().createRole(
|
|
655
|
+
new CreateRoleCommand({
|
|
656
|
+
RoleName: "ModeratorRole",
|
|
657
|
+
AssumeRolePolicyDocument: JSON.stringify({
|
|
658
|
+
Version: "2012-10-17",
|
|
659
|
+
Statement: {
|
|
660
|
+
Effect: "Allow",
|
|
661
|
+
Principal: { Service: "lambda.amazonaws.com" },
|
|
662
|
+
Action: "sts:AssumeRole",
|
|
663
|
+
},
|
|
664
|
+
}),
|
|
665
|
+
}),
|
|
666
|
+
);
|
|
667
|
+
|
|
668
|
+
await simAws.iam().putRolePolicy(
|
|
669
|
+
new PutRolePolicyCommand({
|
|
670
|
+
RoleName: "ModeratorRole",
|
|
671
|
+
PolicyName: "ModeratePolicy",
|
|
672
|
+
PolicyDocument: JSON.stringify({
|
|
673
|
+
Version: "2012-10-17",
|
|
674
|
+
Statement: [
|
|
675
|
+
// A detection has no resource to name, so this one has to be `*`.
|
|
676
|
+
{
|
|
677
|
+
Effect: "Allow",
|
|
678
|
+
Action: "rekognition:DetectModerationLabels",
|
|
679
|
+
Resource: "*",
|
|
680
|
+
},
|
|
681
|
+
// Reading the image does, so this one names the Bucket.
|
|
682
|
+
{
|
|
683
|
+
Effect: "Allow",
|
|
684
|
+
Action: "s3:GetObject",
|
|
685
|
+
Resource: "arn:aws:s3:::uploads/*",
|
|
686
|
+
},
|
|
687
|
+
],
|
|
688
|
+
}),
|
|
689
|
+
}),
|
|
690
|
+
);
|
|
691
|
+
|
|
692
|
+
await simAws.s3().createBucket(new CreateBucketCommand({ Bucket: "uploads" }));
|
|
693
|
+
|
|
694
|
+
await simAws.lambda().createFunction(
|
|
695
|
+
new CreateFunctionCommand({
|
|
696
|
+
FunctionName: "moderator",
|
|
697
|
+
Role: role.Role.Arn,
|
|
698
|
+
Handler: "index.handler",
|
|
699
|
+
Code: {
|
|
700
|
+
ZipFile: makeLambdaCodeZip({
|
|
701
|
+
"index.js": `
|
|
702
|
+
const {
|
|
703
|
+
RekognitionClient,
|
|
704
|
+
DetectModerationLabelsCommand,
|
|
705
|
+
} = require("@aws-sdk/client-rekognition");
|
|
706
|
+
|
|
707
|
+
exports.handler = async (event) => {
|
|
708
|
+
const record = event.Records[0].s3;
|
|
709
|
+
const detected = await new RekognitionClient({}).send(
|
|
710
|
+
new DetectModerationLabelsCommand({
|
|
711
|
+
Image: {
|
|
712
|
+
S3Object: { Bucket: record.bucket.name, Name: record.object.key },
|
|
713
|
+
},
|
|
714
|
+
}),
|
|
715
|
+
);
|
|
716
|
+
|
|
717
|
+
console.log(record.object.key, detected.ModerationLabels.length);
|
|
718
|
+
|
|
719
|
+
return detected.ModerationLabels.length === 0 ? "clean" : "flagged";
|
|
720
|
+
};
|
|
721
|
+
`,
|
|
722
|
+
}),
|
|
723
|
+
},
|
|
724
|
+
}),
|
|
725
|
+
);
|
|
726
|
+
|
|
727
|
+
await simAws.lambda().addPermission(
|
|
728
|
+
new AddPermissionCommand({
|
|
729
|
+
FunctionName: "moderator",
|
|
730
|
+
StatementId: "AllowS3",
|
|
731
|
+
Action: "lambda:InvokeFunction",
|
|
732
|
+
Principal: "s3.amazonaws.com",
|
|
733
|
+
SourceArn: "arn:aws:s3:::uploads",
|
|
734
|
+
SourceAccount: simAws.defaultAccountId,
|
|
735
|
+
}),
|
|
736
|
+
);
|
|
737
|
+
|
|
738
|
+
await simAws.s3().putBucketNotificationConfiguration(
|
|
739
|
+
new PutBucketNotificationConfigurationCommand({
|
|
740
|
+
Bucket: "uploads",
|
|
741
|
+
NotificationConfiguration: {
|
|
742
|
+
LambdaFunctionConfigurations: [
|
|
743
|
+
{
|
|
744
|
+
Id: "moderate-uploads",
|
|
745
|
+
Events: ["s3:ObjectCreated:*"],
|
|
746
|
+
LambdaFunctionArn: moderatorArn,
|
|
747
|
+
Filter: {
|
|
748
|
+
Key: { FilterRules: [{ Name: "prefix", Value: "incoming/" }] },
|
|
749
|
+
},
|
|
750
|
+
},
|
|
751
|
+
],
|
|
752
|
+
},
|
|
753
|
+
}),
|
|
754
|
+
);
|
|
755
|
+
|
|
756
|
+
// The sample image is already declared as failing moderation, so the key it
|
|
757
|
+
// goes in under is the application's business rather than the test's.
|
|
758
|
+
await simAws.s3().putObject(
|
|
759
|
+
new PutObjectCommand({
|
|
760
|
+
Bucket: "uploads",
|
|
761
|
+
Key: `incoming/${randomUUID()}.jpg`,
|
|
762
|
+
Body: simRekognitionSampleImages.flaggedByModeration(),
|
|
763
|
+
}),
|
|
764
|
+
);
|
|
765
|
+
|
|
766
|
+
// Delivery and the detection it triggers both happen in the background.
|
|
767
|
+
await simAws.backgroundTasksComplete();
|
|
768
|
+
```
|
|
769
|
+
|
|
770
|
+
A handler that writes a moderated copy back into the Bucket that triggered it will notify itself for
|
|
771
|
+
ever. Filter the notification configuration by prefix or suffix, as this one does.
|
|
772
|
+
|
|
773
|
+
## Face collections
|
|
774
|
+
|
|
775
|
+
A collection is what lets an application recognise the same person twice, where a detection answers what is in one image.
|
|
776
|
+
|
|
777
|
+
```typescript sim-rekognition-collections
|
|
778
|
+
/**
|
|
779
|
+
* Creating, listing and removing a Rekognition face collection.
|
|
780
|
+
*/
|
|
781
|
+
|
|
782
|
+
import {
|
|
783
|
+
CreateCollectionCommand,
|
|
784
|
+
DeleteCollectionCommand,
|
|
785
|
+
ListCollectionsCommand,
|
|
786
|
+
} from "@aws-sdk/client-rekognition";
|
|
787
|
+
import { SimAws } from "@kensio/yulin";
|
|
788
|
+
|
|
789
|
+
const simAws = new SimAws();
|
|
790
|
+
const simRekognition = simAws.rekognition();
|
|
791
|
+
|
|
792
|
+
const created = await simRekognition.createCollection(
|
|
793
|
+
new CreateCollectionCommand({ CollectionId: "staff" }),
|
|
794
|
+
);
|
|
795
|
+
|
|
796
|
+
console.log(created.CollectionArn);
|
|
797
|
+
// arn:aws:rekognition:us-east-1:888888888888:collection/staff
|
|
798
|
+
|
|
799
|
+
const listed = await simRekognition.listCollections(
|
|
800
|
+
new ListCollectionsCommand({}),
|
|
801
|
+
);
|
|
802
|
+
|
|
803
|
+
console.log(listed.CollectionIds); // ["staff"]
|
|
804
|
+
console.log(listed.FaceModelVersions); // ["7.0"]
|
|
805
|
+
|
|
806
|
+
await simRekognition.deleteCollection(
|
|
807
|
+
new DeleteCollectionCommand({ CollectionId: "staff" }),
|
|
808
|
+
);
|
|
809
|
+
```
|
|
810
|
+
|
|
811
|
+
A collection belongs to one Account and Region, as it does on AWS, so a listing in another Region misses it. Creating one under a name already held raises `ResourceAlreadyExistsException`, and removing one that was never created raises `ResourceNotFoundException`.
|
|
812
|
+
|
|
813
|
+
Every collection reports face model version 7.0. Real Rekognition stamps a collection with the version in force when it was created, and that version moves as AWS retrains. Nothing here recognises a face, so one fixed version is stated rather than a moving one invented.
|
|
814
|
+
|
|
815
|
+
## Indexing faces and finding them again
|
|
816
|
+
|
|
817
|
+
`IndexFaces` puts the faces an image holds into a collection. Which faces an image holds is what the
|
|
818
|
+
`faces()` rules declare, the same rules `DetectFaces` answers from. An image with one declared face
|
|
819
|
+
indexes one face, at the bounding box and the confidence that rule gave it, and an image no rule
|
|
820
|
+
matches indexes the built-in default face.
|
|
821
|
+
|
|
822
|
+
`SearchFacesByImage` answers from the `faceMatches()` rules. They say which indexed faces one image
|
|
823
|
+
finds, and an image no rule matches finds nobody.
|
|
824
|
+
|
|
825
|
+
```typescript sim-rekognition-face-indexing
|
|
826
|
+
/**
|
|
827
|
+
* Indexing a face into a collection and recognising the same person later.
|
|
828
|
+
*/
|
|
829
|
+
|
|
830
|
+
import {
|
|
831
|
+
CreateCollectionCommand,
|
|
832
|
+
DeleteFacesCommand,
|
|
833
|
+
IndexFacesCommand,
|
|
834
|
+
ListFacesCommand,
|
|
835
|
+
SearchFacesByImageCommand,
|
|
836
|
+
} from "@aws-sdk/client-rekognition";
|
|
837
|
+
import { CreateBucketCommand, PutObjectCommand } from "@aws-sdk/client-s3";
|
|
838
|
+
import { SimAws } from "@kensio/yulin";
|
|
839
|
+
import { simRekognitionSampleImages } from "@kensio/yulin/rekognition";
|
|
840
|
+
|
|
841
|
+
const simAws = new SimAws();
|
|
842
|
+
const simRekognition = simAws.rekognition();
|
|
843
|
+
|
|
844
|
+
await simAws.s3().createBucket(new CreateBucketCommand({ Bucket: "photos" }));
|
|
845
|
+
await simAws.s3().putObject(
|
|
846
|
+
new PutObjectCommand({
|
|
847
|
+
Bucket: "photos",
|
|
848
|
+
Key: "staff/ada.jpg",
|
|
849
|
+
Body: simRekognitionSampleImages.oneFace(),
|
|
850
|
+
}),
|
|
851
|
+
);
|
|
852
|
+
await simAws.s3().putObject(
|
|
853
|
+
new PutObjectCommand({
|
|
854
|
+
Bucket: "photos",
|
|
855
|
+
Key: "door/visitor.jpg",
|
|
856
|
+
Body: simRekognitionSampleImages.oneFace(),
|
|
857
|
+
}),
|
|
858
|
+
);
|
|
859
|
+
|
|
860
|
+
await simRekognition.createCollection(
|
|
861
|
+
new CreateCollectionCommand({ CollectionId: "staff" }),
|
|
862
|
+
);
|
|
863
|
+
|
|
864
|
+
const indexed = await simRekognition.indexFaces(
|
|
865
|
+
new IndexFacesCommand({
|
|
866
|
+
CollectionId: "staff",
|
|
867
|
+
Image: { S3Object: { Bucket: "photos", Name: "staff/ada.jpg" } },
|
|
868
|
+
ExternalImageId: "ada",
|
|
869
|
+
}),
|
|
870
|
+
);
|
|
871
|
+
|
|
872
|
+
console.log(indexed.FaceRecords.map((record) => record.Face.ExternalImageId));
|
|
873
|
+
// [ "ada" ]
|
|
874
|
+
|
|
875
|
+
const listed = await simRekognition.listFaces(
|
|
876
|
+
new ListFacesCommand({ CollectionId: "staff" }),
|
|
877
|
+
);
|
|
878
|
+
|
|
879
|
+
console.log(listed.Faces.length); // 1
|
|
880
|
+
|
|
881
|
+
// The visitor at the door is declared to be that member of staff.
|
|
882
|
+
simRekognition
|
|
883
|
+
.faceMatches()
|
|
884
|
+
.onName("door/visitor.jpg", { matches: [{ externalImageId: "ada" }] });
|
|
885
|
+
|
|
886
|
+
const found = await simRekognition.searchFacesByImage(
|
|
887
|
+
new SearchFacesByImageCommand({
|
|
888
|
+
CollectionId: "staff",
|
|
889
|
+
Image: { S3Object: { Bucket: "photos", Name: "door/visitor.jpg" } },
|
|
890
|
+
}),
|
|
891
|
+
);
|
|
892
|
+
|
|
893
|
+
console.log(found.FaceMatches.map((match) => match.Face.ExternalImageId));
|
|
894
|
+
// [ "ada" ]
|
|
895
|
+
|
|
896
|
+
const deleted = await simRekognition.deleteFaces(
|
|
897
|
+
new DeleteFacesCommand({
|
|
898
|
+
CollectionId: "staff",
|
|
899
|
+
FaceIds: listed.Faces.map((face) => face.FaceId),
|
|
900
|
+
}),
|
|
901
|
+
);
|
|
902
|
+
|
|
903
|
+
console.log(deleted.DeletedFaces.length); // 1
|
|
904
|
+
```
|
|
905
|
+
|
|
906
|
+
Each indexed face gets a `FaceId` of its own, and every face from one call shares an `ImageId`. Both
|
|
907
|
+
are uuids, as they are on AWS. An application that stores a `FaceId` and looks it up later works
|
|
908
|
+
here the way it works there.
|
|
909
|
+
|
|
910
|
+
A declared match reaches the faces the searched collection holds. `DeleteFaces` removes one and the
|
|
911
|
+
same rule then finds nobody. One rule covers both sides of a deletion. `DeletedFaces` reports the
|
|
912
|
+
ids that were there, and an id the collection never held comes back in
|
|
913
|
+
`UnsuccessfulFaceDeletions` as `FACE_NOT_FOUND`.
|
|
914
|
+
|
|
915
|
+
`FaceMatchThreshold` filters on the similarity the rule stated and defaults to 80, as it does on
|
|
916
|
+
AWS. `MaxFaces` caps how many matches come back, most alike first. A search with an image the
|
|
917
|
+
`faces()` rules give no face raises `InvalidParameterException`, as real Rekognition does when there
|
|
918
|
+
is no face to search with.
|
|
919
|
+
|
|
920
|
+
`ListFaces` reports the faces one collection holds, in the order they were indexed, and narrows to
|
|
921
|
+
the ids a request names. `MaxResults` pages the listing and the `NextToken` in the response reaches
|
|
922
|
+
the next page. A listing that asks for no page size comes back whole.
|
|
923
|
+
|
|
924
|
+
## Permissions and errors
|
|
925
|
+
|
|
926
|
+
Each detection is authorized as its own action against `*`, one of
|
|
927
|
+
`rekognition:DetectModerationLabels`, `rekognition:DetectLabels` and `rekognition:DetectFaces`. Real
|
|
928
|
+
Rekognition gives the detection operations no resource-level permissions, and a policy naming an ARN
|
|
929
|
+
reaches nothing, here as on AWS.
|
|
930
|
+
|
|
931
|
+
A collection is the other kind. It has an ARN, so `rekognition:CreateCollection`,
|
|
932
|
+
`rekognition:DeleteCollection`, `rekognition:IndexFaces`, `rekognition:ListFaces`,
|
|
933
|
+
`rekognition:SearchFacesByImage` and `rekognition:DeleteFaces` authorize against that collection's
|
|
934
|
+
ARN, and a policy naming one collection reaches only that collection. `rekognition:ListCollections`
|
|
935
|
+
reads them all, so it authorizes against `*`. Each face operation is authorized before the
|
|
936
|
+
collection is looked up. A caller with no permission for a collection never learns whether it is
|
|
937
|
+
there. A denial throws `AccessDeniedException` with a 400 status, which is
|
|
938
|
+
what real Rekognition answers with, where several other services use 403.
|
|
939
|
+
|
|
940
|
+
The caller is authorized for the detection before the image is read. A caller without the Rekognition
|
|
941
|
+
permission is told about that, and never about an S3 object.
|
|
942
|
+
|
|
943
|
+
Every S3 problem becomes `InvalidS3ObjectException`, as it does on real Rekognition, whether the
|
|
944
|
+
Bucket is missing, the object is missing, or the caller may not read it. The underlying simulator
|
|
945
|
+
error is kept as the error's `cause`, leaving a missing `s3:GetObject` grant diagnosable:
|
|
946
|
+
|
|
947
|
+
```typescript
|
|
948
|
+
try {
|
|
949
|
+
await simAws.rekognition().detectModerationLabels(command);
|
|
950
|
+
} catch (error) {
|
|
951
|
+
console.log(error.name); // "InvalidS3ObjectException"
|
|
952
|
+
console.log(error.cause); // the sim IAM access denial
|
|
953
|
+
}
|
|
954
|
+
```
|
|
955
|
+
|
|
956
|
+
Bytes that are neither a PNG nor a JPEG are refused with `InvalidImageFormatException`. The format
|
|
957
|
+
comes from the leading bytes of the image. A test that stores a placeholder string in a Bucket and
|
|
958
|
+
moderates it gets that error.
|
|
959
|
+
|
|
960
|
+
## Accounts and Regions
|
|
961
|
+
|
|
962
|
+
Rekognition is scoped to an Account and a Region, and so are the rules registered against it. A
|
|
963
|
+
detection made in one Region is answered by the rules registered in that Region.
|
|
964
|
+
|
|
965
|
+
```typescript
|
|
966
|
+
simAws.account("111111111111").region("eu-west-2").rekognition();
|
|
967
|
+
```
|
|
968
|
+
|
|
969
|
+
An image is read from a Bucket in another Account when that Bucket's policy allows the caller, as
|
|
970
|
+
real Rekognition reads across Accounts. A Bucket in another Region is refused, as real Rekognition
|
|
971
|
+
reads only Buckets in its own Region.
|
|
972
|
+
|
|
973
|
+
## Available functionality
|
|
974
|
+
|
|
975
|
+
Simulated Rekognition currently supports:
|
|
976
|
+
|
|
977
|
+
- `DetectModerationLabelsCommand`, `DetectLabelsCommand` and `DetectFacesCommand`, for an image
|
|
978
|
+
supplied as `Image.Bytes` or as `Image.S3Object`
|
|
979
|
+
- Results declared by exact S3 object name, by exact image content hash, or as a default, with the
|
|
980
|
+
hash rule winning, then the name rule, then the default
|
|
981
|
+
- The complete version 7.0 content moderation taxonomy, with a declared label expanding to its
|
|
982
|
+
parents and carrying `ParentName` and `TaxonomyLevel`
|
|
983
|
+
- Detected labels carrying declared `Parents`, `Aliases`, `Categories` and `Instances`, ordered by
|
|
984
|
+
descending confidence
|
|
985
|
+
- `MinConfidence` filtering, defaulting to 50 for moderation and 55 for label detection, and
|
|
986
|
+
`MaxLabels` after it
|
|
987
|
+
- Detected faces carrying the declared bounding box, confidence, pose, quality, landmarks, age
|
|
988
|
+
range, gender, emotions, eye direction and the eight yes or no attributes
|
|
989
|
+
- `Attributes` handling for face detection, with the default subset always returned, `ALL` adding
|
|
990
|
+
the rest, and five landmarks reported unless `ALL` was asked for
|
|
991
|
+
- `simRekognitionNoFaces` and `simRekognitionSeveralFaces`, for a test that counts faces
|
|
992
|
+
- Five built-in sample images, real PNG and JPEG files with their hashes already declared, for a
|
|
993
|
+
clean and a flagged moderation result and for zero, one and three faces
|
|
994
|
+
- `simRekognitionImageHash`, for hashing a fixture to declare a rule against
|
|
995
|
+
- PNG and JPEG format detection from the image bytes
|
|
996
|
+
- IAM authorization on `rekognition:DetectModerationLabels`, `rekognition:DetectLabels` and
|
|
997
|
+
`rekognition:DetectFaces`, with the image read from S3 as the caller
|
|
998
|
+
- `CreateCollectionCommand`, `ListCollectionsCommand` and `DeleteCollectionCommand`, scoped to one
|
|
999
|
+
Account and Region
|
|
1000
|
+
- `IndexFacesCommand`, `ListFacesCommand`, `SearchFacesByImageCommand` and `DeleteFacesCommand`,
|
|
1001
|
+
with the faces put in a collection taken from the `faces()` rules for the image they came from
|
|
1002
|
+
- Face searches declared through `faceMatches()`, by the external image id a face was indexed under
|
|
1003
|
+
or by the face id `IndexFaces` answered with, filtered by `FaceMatchThreshold` and capped by
|
|
1004
|
+
`MaxFaces`
|
|
1005
|
+
- `ListFaces` narrowing to named face ids, and paging on `MaxResults` and `NextToken`
|
|
1006
|
+
- IAM authorization on `rekognition:CreateCollection`, `rekognition:DeleteCollection`,
|
|
1007
|
+
`rekognition:IndexFaces`, `rekognition:ListFaces`, `rekognition:SearchFacesByImage` and
|
|
1008
|
+
`rekognition:DeleteFaces` against the collection's own ARN, and on `rekognition:ListCollections`
|
|
1009
|
+
against `*`
|
|
1010
|
+
- SDK interception of `RekognitionClient`, including from inside a simulated Lambda function
|
|
1011
|
+
|
|
1012
|
+
## Limitations
|
|
1013
|
+
|
|
1014
|
+
- `DetectText`, `CompareFaces` and the video operations are left out. An intercepted client sending
|
|
1015
|
+
one of those Commands is refused by name.
|
|
1016
|
+
- `SearchFaces`, which searches by face id, and the `SearchUsers` and user association operations are
|
|
1017
|
+
left out. An intercepted client sending one of those Commands is refused by name.
|
|
1018
|
+
- `QualityFilter` is refused on `IndexFaces` and `SearchFacesByImage`. Real Rekognition uses it to
|
|
1019
|
+
drop faces it judges too blurry or too small. Nothing here judges an image. A filter set on the
|
|
1020
|
+
request would drop faces on AWS and keep them here.
|
|
1021
|
+
- A `MaxFaces` on `IndexFaces` takes the faces in the order they were declared, and the rest come
|
|
1022
|
+
back in `UnindexedFaces` as `EXCEEDS_MAX_FACES`. Real Rekognition indexes the largest.
|
|
1023
|
+
- A search reports the first face declared for the image as the one it searched with. Real
|
|
1024
|
+
Rekognition uses the largest, and nothing here measures a face.
|
|
1025
|
+
- `UserId` is refused on `ListFaces`, and `UnsuccessfulFaceDeletions` never reports
|
|
1026
|
+
`ASSOCIATED_TO_AN_EXISTING_USER`. A face is never associated with a user here. A listing narrowed
|
|
1027
|
+
to one would answer with the whole collection.
|
|
1028
|
+
- A `ListFaces` page with no `MaxResults` holds the whole collection. Real Rekognition pages at a
|
|
1029
|
+
thousand faces. The two differ only for a collection larger than that.
|
|
1030
|
+
- A face detection reports the emotions that were declared and no others. Real `DetectFaces` returns
|
|
1031
|
+
all eight emotion types every time, with the ones it failed to see at a low confidence. Declare
|
|
1032
|
+
the emotions the code under test reads.
|
|
1033
|
+
- `OrientationCorrection` is left off a `DetectFaces` response, because AWS documents its value as
|
|
1034
|
+
always null.
|
|
1035
|
+
- A declared bounding box has to sit inside the image. Real Rekognition can report one that spills
|
|
1036
|
+
over, for a face at the image edge that is only partly visible. The check is kept because it
|
|
1037
|
+
catches a box written in pixels.
|
|
1038
|
+
- Landmark pairs that run across the face, such as `eyeLeft` and `eyeRight`, have to be declared in
|
|
1039
|
+
the order Rekognition reports them in. A face rolled past upright is the one case where that
|
|
1040
|
+
ordering breaks down on AWS, and it cannot be declared here.
|
|
1041
|
+
- Yulin ships no general label ontology, because AWS's is thousands of entries with no published
|
|
1042
|
+
enumerable table.
|
|
1043
|
+
- A declared label's `Parents` appear on that label alone. Real `DetectLabels` also returns each
|
|
1044
|
+
ancestor as a label in its own right, which needs the ontology above. Declare the ancestors as
|
|
1045
|
+
labels too when the code under test reads them that way.
|
|
1046
|
+
- A declared label name goes unchecked, for the same reason. Refusing a real AWS label because a
|
|
1047
|
+
Yulin list was missing it would be failing closed against Yulin's own gaps.
|
|
1048
|
+
- `DetectLabels` `Settings` filters and `IMAGE_PROPERTIES` are refused outright. Applying no filters
|
|
1049
|
+
would answer with labels the caller asked to have left out, and image quality and dominant colours
|
|
1050
|
+
would have to be invented by a simulation that looks at no images.
|
|
1051
|
+
- A custom moderation adapter named with `ProjectVersion`, and a human review loop named with
|
|
1052
|
+
`HumanLoopConfig`, are both refused outright. Answering from the built-in model would make an
|
|
1053
|
+
adapter look applied here and be applied in production.
|
|
1054
|
+
- `ContentTypes` is always empty. Real Rekognition puts `Animated` or `Illustrated` there for
|
|
1055
|
+
content it identifies as such, which needs an image to look at.
|
|
1056
|
+
- The image is read no further than its first few bytes. A PNG of a kitten declared as `Violence`
|
|
1057
|
+
comes back as `Violence`, and an image no rule matches gets the built-in `Mobile Phone` result.
|
|
1058
|
+
- The sample images are 16 by 16 pictures of coloured shapes, small enough to ship in the package.
|
|
1059
|
+
They are drawings rather than photographs of the things they are named for, since nothing decodes
|
|
1060
|
+
them.
|
|
1061
|
+
- The format comes from the image bytes and never from a stored content type. Simulated S3 keeps a
|
|
1062
|
+
`ContentType` given to `PutObject` as a metadata key, and has none at all when the uploader left
|
|
1063
|
+
it out. Trusting it would make the same bytes detectable or not depending on how they were
|
|
1064
|
+
uploaded.
|
|
1065
|
+
- A `Version` on an `Image.S3Object` is refused, because simulated S3 has no object versions and
|
|
1066
|
+
would have answered with the current one.
|
|
1067
|
+
- There are no CloudFormation resource types for Rekognition, and Rekognition is not served over
|
|
1068
|
+
`serveSimAws`.
|
|
1069
|
+
- The moderation taxonomy is the published version 7.0 label list. A label from version 6.1 is
|
|
1070
|
+
refused, since real Rekognition stopped returning one.
|