@bitmagic/cli 0.1.66-dev.5 → 0.1.66-dev.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -1
- package/assets/character-skin-authoring/SKILL.md +43 -0
- package/assets/character-skin-authoring/references/exporters.md +134 -0
- package/assets/character-skin-authoring/references/eyes.md +215 -0
- package/assets/character-skin-authoring/references/materials.md +115 -0
- package/assets/character-skin-authoring/references/playground.md +78 -0
- package/assets/character-skin-authoring/references/profiles.md +97 -0
- package/assets/character-skin-authoring/references/projection.md +190 -0
- package/assets/character-skin-authoring/references/regional-detail.md +72 -0
- package/assets/character-skin-authoring/scripts/assemble_character.py +205 -0
- package/assets/character-skin-authoring/scripts/authored_detail.py +47 -0
- package/assets/character-skin-authoring/scripts/authored_surface_patch.py +83 -0
- package/assets/character-skin-authoring/scripts/brow_groom.py +62 -0
- package/assets/character-skin-authoring/scripts/build_face_detail.py +107 -0
- package/assets/character-skin-authoring/scripts/build_surface_candidate.py +28 -0
- package/assets/character-skin-authoring/scripts/canthus_wall.py +57 -0
- package/assets/character-skin-authoring/scripts/character_build.py +54 -0
- package/assets/character-skin-authoring/scripts/eye_assets.py +333 -0
- package/assets/character-skin-authoring/scripts/eye_opening.py +114 -0
- package/assets/character-skin-authoring/scripts/eye_texture.py +35 -0
- package/assets/character-skin-authoring/scripts/eyelid_detail.py +34 -0
- package/assets/character-skin-authoring/scripts/eyelid_morph.py +171 -0
- package/assets/character-skin-authoring/scripts/eyelid_patch.py +173 -0
- package/assets/character-skin-authoring/scripts/face_atlas.py +51 -0
- package/assets/character-skin-authoring/scripts/geometry_repair.py +112 -0
- package/assets/character-skin-authoring/scripts/glb_arrays.py +13 -0
- package/assets/character-skin-authoring/scripts/glb_output.py +21 -0
- package/assets/character-skin-authoring/scripts/inspect_skin_point.py +66 -0
- package/assets/character-skin-authoring/scripts/normal_patch_height.py +38 -0
- package/assets/character-skin-authoring/scripts/prepare_face_reference.py +23 -0
- package/assets/character-skin-authoring/scripts/regional_atlas.py +51 -0
- package/assets/character-skin-authoring/scripts/select_preview.py +90 -0
- package/assets/character-skin-authoring/scripts/skin_views.py +267 -0
- package/assets/character-skin-authoring/scripts/surface_pbr_maps.py +46 -0
- package/assets/character-skin-authoring/scripts/texture_repair.py +98 -0
- package/assets/character-skin-authoring/scripts/vertex_detail_support.py +80 -0
- package/assets/character-skin-authoring/scripts/vision_blend_materials.py +74 -0
- package/assets/character-skin-authoring/scripts/vision_materials.py +276 -0
- package/assets/character-skin-authoring/scripts/workspace_paths.py +18 -0
- package/assets/metahuman/src/work/metahuman/MetaHumanActor.ts +580 -0
- package/assets/metahuman/src/work/metahuman/MetaHumanLabHooks.ts +122 -0
- package/assets/metahuman/src/work/metahuman/MetaHumanMaterials.ts +500 -0
- package/assets/metahuman/src/work/metahuman/MetaHumanSpeech.ts +242 -0
- package/assets/metahuman/src/work/metahuman/StudioEnvironment.ts +142 -0
- package/assets/metahuman/tools/lipsync/make_line.py +181 -0
- package/assets/metahuman/tools/metahuman/README.md +21 -0
- package/assets/metahuman/tools/metahuman/e2e/eval.mjs +34 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/arm-pose-check.js +32 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/bone-displacement.js +26 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/clip-shoulder.js +41 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/grafted-bones.js +11 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/helper-follow-descent.js +38 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/helper-follow-sweep.js +31 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/helper-joints-inspect.js +10 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/outfit-bad-vertices.js +26 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/outfit-body-material.js +7 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/outfit-own-skeleton.js +8 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/shirt-bone-ownership.js +25 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/shirt-edge-stretch.js +26 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/shirt-long-edges.js +23 -0
- package/assets/metahuman/tools/metahuman/e2e/probes/walk-sampler.js +105 -0
- package/assets/metahuman/tools/metahuman/e2e/shot.mjs +125 -0
- package/assets/metahuman/tools/metahuman/e2e/walk-probe.mjs +43 -0
- package/assets/metahuman/tools/metahuman/merge-skeletons.mjs +69 -0
- package/assets/metahuman/tools/metahuman/package.json +13 -0
- package/assets/metahuman/tools/metahuman/prepare.sh +25 -0
- package/assets/metahuman/tools/metahuman/reorder-body-first.mjs +20 -0
- package/assets/metahuman/tools/metahuman/smooth-outfit-weights.mjs +48 -0
- package/assets/metahuman/tools/metahuman/strip-draco.mjs +32 -0
- package/assets/metahuman/tools/metahuman/transfer-skin-weights.mjs +90 -0
- package/dist/commands/generate.d.ts +49 -0
- package/dist/commands/generate.js +168 -1
- package/dist/commands/generate.js.map +1 -1
- package/dist/generate/audio-file.d.ts +6 -0
- package/dist/generate/audio-file.js +42 -0
- package/dist/generate/audio-file.js.map +1 -0
- package/dist/generate/music.d.ts +4 -3
- package/dist/generate/music.js +6 -23
- package/dist/generate/music.js.map +1 -1
- package/dist/generate/stream.d.ts +13 -0
- package/dist/generate/stream.js +14 -1
- package/dist/generate/stream.js.map +1 -1
- package/dist/scaffold/asset-pipeline-skill-files.js +2 -0
- package/dist/scaffold/asset-pipeline-skill-files.js.map +1 -1
- package/dist/scaffold/character-skin-skill-files.d.ts +2 -0
- package/dist/scaffold/character-skin-skill-files.js +49 -0
- package/dist/scaffold/character-skin-skill-files.js.map +1 -0
- package/dist/scaffold/metahuman-skill-files.d.ts +21 -0
- package/dist/scaffold/metahuman-skill-files.js +306 -0
- package/dist/scaffold/metahuman-skill-files.js.map +1 -0
- package/dist/scaffold/project-files.js +4 -4
- package/dist/scaffold/project-files.js.map +1 -1
- package/dist/scaffold/project.js +4 -0
- package/dist/scaffold/project.js.map +1 -1
- package/dist/scaffold/skill-files.js +3 -1
- package/dist/scaffold/skill-files.js.map +1 -1
- package/dist/tools/bundles.js +8 -0
- package/dist/tools/bundles.js.map +1 -1
- package/package.json +4 -4
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# Applying the reviewed maps in the character-forger playground
|
|
2
|
+
|
|
3
|
+
The maintained helpers are in `highquality/src/` at the character-forger repository root.
|
|
4
|
+
They are not standalone engine binaries. Use the project's Python with NumPy, Pillow and
|
|
5
|
+
SciPy, and Blender for projection. Keep this adapter's single mesh/primitive limitation
|
|
6
|
+
explicit; do not silently flatten a rigged or instanced scene to make it pass.
|
|
7
|
+
|
|
8
|
+
1. Prepare and annotate exact views; dense-bake with `--surface-data --triangle-labels`.
|
|
9
|
+
2. Author `highquality/materials/faces/ID.json` from this character's face view. The target
|
|
10
|
+
bounds describe normalized body X/Z in Blender world space. Pixel dimensions belong to
|
|
11
|
+
their respective source and target views. Reusing donor correspondence is valid; reusing
|
|
12
|
+
another character's target landmarks is not. Hair surrounding a face is an enclosure,
|
|
13
|
+
not one filled exclusion polygon covering the face.
|
|
14
|
+
3. Run `python highquality/src/build_face_detail.py --id ID`. It registers band-limited
|
|
15
|
+
licensed scan relief and emits attribution. The amplitude is an authoring estimate,
|
|
16
|
+
not measured displacement. Empty coverage is an error requiring a profile correction.
|
|
17
|
+
4. Run `python highquality/src/vision_materials.py --id ID --masks MASK_REVISION --evidence EVIDENCE`.
|
|
18
|
+
The adapter writes `characters/ID/high/v1/materials/v2/character.glb` and a report. It
|
|
19
|
+
preserves source geometry/buffer bytes, nodes, existing skin data and animation data.
|
|
20
|
+
This does not add a missing body rig. Inputs are currently the immutable HY source.
|
|
21
|
+
5. Rebuild the viewer with `node hero/viewer/build.mjs`. Run
|
|
22
|
+
`CHARACTER_ID=ID node highquality/tests/vision_skin_smoke.mjs` on both renderers, then
|
|
23
|
+
inspect the screenshots. Tests check loading and camera stability, not anatomical truth.
|
|
24
|
+
|
|
25
|
+
The detail atlas uses source UVs and includes all explicitly observed skin. Face detail is
|
|
26
|
+
registered from a licensed scan; other follicular skin currently uses reduced scan relief.
|
|
27
|
+
Palms/soles use authored nonfollicular ridges. This is not a library of captured regional
|
|
28
|
+
hand/foot exemplars. Source geometry retains its existing larger creases. The report keeps
|
|
29
|
+
unresolved labels and texture defects visible.
|
|
30
|
+
|
|
31
|
+
Engine contract: explicitly tagged skin/hair materials opt into `createCharacterSurfaceMaterial`.
|
|
32
|
+
RGBA detail stores signed height around .5, roughness, chromatic residual and coverage.
|
|
33
|
+
`_FACE_UV` supplies coordinates, `_FACE_WEIGHT` coverage, `_SCATTER` the fallback transmission
|
|
34
|
+
attribute. Atlas data is linear; albedo is sRGB. The helper excludes defective HY normals on
|
|
35
|
+
labelled skin, keeps native materials elsewhere, and performs geometry-corresponding skin
|
|
36
|
+
UV seam colour equalization. It cannot remove arbitrary baked lighting or interior texture
|
|
37
|
+
stitches merely by correcting normal maps.
|
|
38
|
+
|
|
39
|
+
Material reports remain `shaded_review_required` until actual exposed regions and rendered
|
|
40
|
+
maps have been reviewed. Do not convert low conflict counts into a whole-character quality
|
|
41
|
+
claim. The b3658 scalp albedo stitch, ambiguous nails and b3843 palm colour spill are examples
|
|
42
|
+
of asset defects that require separate correction.
|
|
43
|
+
|
|
44
|
+
## Per-pixel material boundaries
|
|
45
|
+
|
|
46
|
+
After the v2 evidence build, `python highquality/src/vision_blend_materials.py --id ID`
|
|
47
|
+
creates an opt-in v3 revision. `--all-surfaces` creates v4 with hair/lip/nail PBR maps
|
|
48
|
+
as well. Both preserve geometry. They currently require one shared source material/UV
|
|
49
|
+
layout; unsupported native specular or fiber maps fail rather than being overwritten.
|
|
50
|
+
|
|
51
|
+
`skinSurfaceBlend: true` makes the engine use atlas alpha for skin response, including
|
|
52
|
+
diffusion and thin-skin transmission. Zero coverage retains the native response. Coverage
|
|
53
|
+
must have UV gutters: copy nearest occupied texels into unused atlas space, never into
|
|
54
|
+
unobserved surface regions. This distinction fixes false seams without inventing anatomy.
|
|
55
|
+
At skin/hair/lip boundaries, known material weights must sum to one. Independently fading
|
|
56
|
+
both sides to zero exposes the source material as a bright halo.
|
|
57
|
+
|
|
58
|
+
V4 expresses fiber and lip response through standard glTF ORM/specular/anisotropy/sheen
|
|
59
|
+
maps. The current fiber direction is provisional; author a comb-flow field before calling
|
|
60
|
+
hair finished. Its skin inspector shows actual skin coverage (white), not an entire mixed
|
|
61
|
+
primitive falsely labelled as skin. `viewerReady` only selects a tested local preview;
|
|
62
|
+
it is not the region-level semantic or final character quality approval.
|
|
63
|
+
|
|
64
|
+
## Runtime adoption
|
|
65
|
+
|
|
66
|
+
The engine now provides `RealisticCharacter.create(root, options)` for the loaded GLTF.
|
|
67
|
+
Use the engine's `createGltfLoader()` and resolve embedded atlas indices through its parser.
|
|
68
|
+
The adapter validates UV attributes, retains native untagged materials, preserves coverage
|
|
69
|
+
when detail is disabled, and exposes independent eye controls. It commits material changes
|
|
70
|
+
only after texture resolution succeeds. Scene postprocessing and the body mixer remain
|
|
71
|
+
caller-owned. See the GDK engine document `agent-docs/character-surfaces.md` for ownership
|
|
72
|
+
and pass ordering; do not copy the lab's former material-loading loops into each game.
|
|
73
|
+
|
|
74
|
+
For a rigged playground derivative, use `highquality/src/assemble_character.py` after
|
|
75
|
+
shared Blender binding. It converts refined geometry to canonical bind coordinates and
|
|
76
|
+
keeps canonical inverse-bind matrices; exercise pose reset and body motion, not only a
|
|
77
|
+
rest-position comparison. Unrigged material/eye previews can coexist with a rigged body
|
|
78
|
+
variant. Determine normalization from the loaded asset, not a catalog-wide rigging flag.
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# Face and eye profile authoring
|
|
2
|
+
|
|
3
|
+
These schemas describe authored inputs to the installed exporters. Values that locate
|
|
4
|
+
anatomy must come from images of this character. Keep the physical dimensions and source
|
|
5
|
+
hashes beside the profile; never copy another character's target coordinates.
|
|
6
|
+
|
|
7
|
+
## Facial detail registration
|
|
8
|
+
|
|
9
|
+
Write `highquality/materials/faces/ID.json` with:
|
|
10
|
+
|
|
11
|
+
- `version: 2`, `id`, `sourceSha256`, `evidenceImage` (workspace-relative), and
|
|
12
|
+
`evidenceImageSha256` identifying the exact source and reviewed frontal image.
|
|
13
|
+
- `annotationImageSize`: the square target image's actual pixel dimension.
|
|
14
|
+
- `bounds: [minX, minZ, maxX, maxZ]`: the target image rectangle in normalized body
|
|
15
|
+
coordinates. The current adapter requires a frontal view with direction `[0,1,0]`
|
|
16
|
+
and up `[0,0,1]`; align an arbitrarily oriented source explicitly before preparing
|
|
17
|
+
evidence. Normalized X is `(worldX - boundsCenterX) / bodyHeight`; normalized Z is
|
|
18
|
+
`(worldZ - boundsCenterZ) / bodyHeight + .5`. For a frontal evidence camera, its
|
|
19
|
+
`targetOffset` and `span` give this rectangle directly:
|
|
20
|
+
`[offsetX-span/2, .5+offsetZ-span/2, offsetX+span/2, .5+offsetZ+span/2]`.
|
|
21
|
+
- `sourceBounds` and `sourceAnnotationImageSize`: the donor registration view's
|
|
22
|
+
rectangle and pixel size. Generate and inspect it with the helper below.
|
|
23
|
+
- `landmarkNames`, `sourcePixels`, `targetPixels`: equal-length ordered arrays of
|
|
24
|
+
corresponding points. Use a well-spread non-collinear set spanning forehead,
|
|
25
|
+
temples, cheek/jaw/chin boundary, eye corners, nose and mouth. These define the
|
|
26
|
+
correspondence and its supported hull; points outside the hull receive no face detail.
|
|
27
|
+
- `regions: {"eyes": [...], "brows": [...], "lips": [...]}`: each list contains polygons
|
|
28
|
+
of target image pixels. These exclude those regions from ordinary facial detail.
|
|
29
|
+
Include facial-hair exclusions in the exclusion polygons too. Empty lists are valid
|
|
30
|
+
only when that exclusion is actually absent in the selected patch.
|
|
31
|
+
- `atlasSize`: 2048 or 4096. `independentFacialDetail: true` retains the full facial
|
|
32
|
+
resolution in the packed runtime atlas.
|
|
33
|
+
- `detailHeightRangeM`: full signed encoding range in metres (zero is .5).
|
|
34
|
+
`detailAmplitudeM`: authored donor-band calibration, not measured scan amplitude.
|
|
35
|
+
Compare the resulting relief at face-sized framing under side light; do not compensate
|
|
36
|
+
for missing texel density by increasing amplitude.
|
|
37
|
+
- `diffusionProfile`: three positive authored channel falloff ratios, such as the
|
|
38
|
+
renderer's human-skin default `[1,.55,.3]`. Fantasy skin ratios are appearance choices.
|
|
39
|
+
|
|
40
|
+
Generate the fixed licensed donor's reference image without a playground checkout:
|
|
41
|
+
|
|
42
|
+
```sh
|
|
43
|
+
python scripts/prepare_face_reference.py --workspace /path/to/assets \
|
|
44
|
+
--out /path/to/assets/evidence/donor --size 900
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Use `donor-front.png` and `registration-view.json` to author `sourcePixels`. Pixel origin
|
|
48
|
+
is top-left for both source and target. The helper uses the same donor projection and
|
|
49
|
+
UV convention as the atlas builder; choosing points from a differently framed photo
|
|
50
|
+
will misregister the relief. It renders the existing licensed files and downloads nothing.
|
|
51
|
+
|
|
52
|
+
## Optical eye fit
|
|
53
|
+
|
|
54
|
+
An eye profile has `id`, `sourceSha256`, `evidence` (absolute prepared-view folder),
|
|
55
|
+
`skinRevision: "v4"`, and exactly two entries in `eyes`. Each entry requires:
|
|
56
|
+
|
|
57
|
+
- `name`, `anatomicalSide: "left" | "right"`, and `view`, matching a manifest view.
|
|
58
|
+
- `irisCenterPx: [x,y]`, `irisRadiusPx`, and `openingPx`: an ordered convex aperture
|
|
59
|
+
polygon in that exact view. Inspect both obliques before choosing the final depth.
|
|
60
|
+
- `upperContourIndices` and `lowerContourIndices`: ordered indices along the corresponding
|
|
61
|
+
aperture edges, including their corners. These guide contacts, lashes and closure.
|
|
62
|
+
- `globeRadiusM`, `corneaBulgeM`, `frontOffsetM`, `pupilRatio`, `irisColorLinear` (RGB).
|
|
63
|
+
These are designed optical dimensions and pigment. The iris must fit inside the globe;
|
|
64
|
+
confirm against the character's physical height and side views.
|
|
65
|
+
|
|
66
|
+
Use `openingAuthority: "reviewed-contours"` only after explicitly inspecting the aperture
|
|
67
|
+
when the source has no separately classified ocular primitive. `conformOpenLids: true`
|
|
68
|
+
pushes intersecting source lid tissue outside the fitted globe; inspect its effect.
|
|
69
|
+
Optional `lashes` uses `upperCount`, `lowerCount`, `upperLengthM`, `lowerLengthM`,
|
|
70
|
+
`radiusM` and an authoring note. Reconstructed bands (`outerLidPx`, `lidSkinSamplesPx`),
|
|
71
|
+
corner lining (`canthusWalls`), medial tissue (`caruncle`) and brow strands (`browGroom`)
|
|
72
|
+
are described in [eyes.md](eyes.md). Start with the source-preserving fit; optional
|
|
73
|
+
reconstruction is not automatically an improvement, as the rejected elf trial showed.
|
|
74
|
+
|
|
75
|
+
The current blink is a fitted geometric morph, not a biomechanical eyelid simulation.
|
|
76
|
+
Preserve source folds unless an independently reviewed correction improves the result.
|
|
77
|
+
Simple smoothing of the closed support band was rejected because it created visible
|
|
78
|
+
rings. Treat any future smoothing as a new fit experiment requiring matched neutral
|
|
79
|
+
and closed-eye review.
|
|
80
|
+
|
|
81
|
+
### Several independent source-color repairs
|
|
82
|
+
|
|
83
|
+
`materials/repair-profile.json` may retain the single-operation form or contain
|
|
84
|
+
`{"repairs": [operation, ...]}`. Each operation has its own `mask`, `purpose`,
|
|
85
|
+
optional `cleanSwatches`, `blendWidthM` and optional `evidence` workspace path.
|
|
86
|
+
The latter permits fresh, color-faithful swatch views without invalidating a
|
|
87
|
+
separate anatomical bake whose camera/geometry evidence is unchanged. Source,
|
|
88
|
+
image and correspondence hashes are checked. Every donor must still land on
|
|
89
|
+
independently classified skin. Operations run in order and report separately;
|
|
90
|
+
use disjoint selections unless the overlap is explicitly intentional.
|
|
91
|
+
|
|
92
|
+
Baked illumination in a generated albedo can make a correctly lit surface dark
|
|
93
|
+
or plastic. Inspect the unlit runtime albedo and color-faithful evidence first.
|
|
94
|
+
A swatch-based correction is an art-directed reconstruction, not measured
|
|
95
|
+
reflectance recovery. Do not globally brighten skin or infer defects from RGB.
|
|
96
|
+
Review the selected region, donor color, transitions and the final relit result;
|
|
97
|
+
retain the original and the rejected candidates.
|
|
@@ -0,0 +1,190 @@
|
|
|
1
|
+
# Projection contract
|
|
2
|
+
|
|
3
|
+
The portable helper produces evidence and UV labels. It does not fit a detail exemplar,
|
|
4
|
+
change a GLB, or claim a sparse projected label map is a production-ready skin mask.
|
|
5
|
+
Use its coverage report to drive further agent inspection.
|
|
6
|
+
|
|
7
|
+
Run with a local Python containing NumPy and Pillow. Blender supplies its own NumPy.
|
|
8
|
+
The lab uses `lowpoly/.venv/bin/python`; elsewhere choose the project's environment.
|
|
9
|
+
|
|
10
|
+
```sh
|
|
11
|
+
python <skill>/scripts/skin_views.py prepare --source character.glb --views views.json --output evidence-v1 --size 1024 --blender /path/to/blender
|
|
12
|
+
python <skill>/scripts/skin_views.py bake --evidence evidence-v1 --annotations annotations.json --output masks-v1 --resolution 4096 --dense --blender /path/to/blender
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Omit `--blender` when available on PATH. `prepare` uses Blender's imported mesh and
|
|
16
|
+
world transform, BVH visibility, interpolated UVs and unlit base-color image. It currently
|
|
17
|
+
supports directly linked Principled base-color textures and the active UV set. Complex
|
|
18
|
+
material graphs must first receive a faithful albedo bake; the helper fails instead of
|
|
19
|
+
guessing. Animated models are inspected in the imported pose. Validate orientation and
|
|
20
|
+
texture appearance against the original before annotating.
|
|
21
|
+
|
|
22
|
+
Evidence PNGs preserve sRGB byte textures as encoded values. Blender's non-float
|
|
23
|
+
`Image.pixels` exposes those byte values normalized to 0–1; applying an additional
|
|
24
|
+
linear-to-sRGB transfer would brighten the evidence incorrectly. Constant Principled
|
|
25
|
+
base colors are linear and receive exactly one transfer. Float images are rejected
|
|
26
|
+
until a faithful sRGB albedo bake is supplied. New manifests explicitly record
|
|
27
|
+
`evidenceColorSpace` and `byteImageTransfer`. The playground's known-color Blender
|
|
28
|
+
regression verifies both byte-texture and constant-color paths within one byte value.
|
|
29
|
+
|
|
30
|
+
Older evidence made before this correction may be too bright. Preserve those immutable
|
|
31
|
+
revisions and annotations, prepare a new color-correct revision, and visually review it.
|
|
32
|
+
Identical camera/UV correspondence can justify transferring polygon coordinates, but
|
|
33
|
+
never claim a new visual review merely because the geometry stayed unchanged. This
|
|
34
|
+
helper correction does not alter the runtime GLB texture color pipeline.
|
|
35
|
+
|
|
36
|
+
`views.json` is a list. `direction` points **from camera into the scene** in Blender world
|
|
37
|
+
coordinates, `up` defaults to +Z. `span` is square orthographic width divided by model
|
|
38
|
+
height. `targetOffset` is relative to bounding-box center, also in model-height units.
|
|
39
|
+
Directions are geometric, not assumptions about front/back or left/right anatomy.
|
|
40
|
+
|
|
41
|
+
```json
|
|
42
|
+
[
|
|
43
|
+
{"name":"minus-y","direction":[0,1,0],"span":1.45},
|
|
44
|
+
{"name":"plus-y","direction":[0,-1,0],"span":1.45}
|
|
45
|
+
]
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
After the survey, choose offsets and tighter spans by inspecting the actual character.
|
|
49
|
+
For a top view, supply an up vector perpendicular to its direction. Each view yields a
|
|
50
|
+
PNG and NPZ containing material index, UV and viewing-angle confidence for every visible
|
|
51
|
+
pixel. Manifest records source hash, cameras, dimensions, material names and world bounds.
|
|
52
|
+
Rasterized sample coordinates refer to pixel centers; polygon coordinates are image
|
|
53
|
+
pixels from top-left, x right and y down. Use precisely the named image dimensions.
|
|
54
|
+
|
|
55
|
+
```json
|
|
56
|
+
{
|
|
57
|
+
"sourceSha256":"copy the manifest hash",
|
|
58
|
+
"views":{
|
|
59
|
+
"minus-y":[
|
|
60
|
+
{"label":"skin","polygon":[[100,100],[140,100],[140,160],[100,160]]},
|
|
61
|
+
{"label":"eyes","polygon":[[110,110],[120,110],[120,120],[110,120]]}
|
|
62
|
+
]
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
These coordinates are format examples, not annotations for any character. Later polygons
|
|
68
|
+
overwrite earlier ones within a view. Across views, contradictory labels become unknown
|
|
69
|
+
and appear in the conflict map. Grazing samples are excluded deterministically. UVs outside
|
|
70
|
+
0–1 remain unclassified rather than being wrapped into a different region.
|
|
71
|
+
|
|
72
|
+
Outputs: per-material label, observed and conflict PNGs; per-view annotation overlays,
|
|
73
|
+
reprojected atlas overlays and raw reprojected labels; archived annotations; and a JSON report. Label IDs: unknown 0, skin 1, palm 2, sole 3, lips 4, eyes 5, hair 6, nails 7,
|
|
74
|
+
non_skin 8, dorsal_hand 9. Material ordering belongs to this evidence manifest, not a promised GLB index.
|
|
75
|
+
UV labels use top-left image rows (Blender UV V is inverted during baking).
|
|
76
|
+
|
|
77
|
+
The bake does not interpolate unknown texels or decide by majority vote: conflicting
|
|
78
|
+
observations require review. Low-resolution survey rays are insufficient to cover a 4K UV
|
|
79
|
+
atlas; use region close-ups and appropriate resolution for final maps. Do not count unknown
|
|
80
|
+
background texels as missed skin without examining the UV occupancy of the actual model.
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
## Dense bake and review (tested on three additional characters)
|
|
84
|
+
|
|
85
|
+
Use `--dense` for reviewed deliverables. The default sparse mode remains useful as a small
|
|
86
|
+
projection diagnostic. Dense mode rasterizes the source UV triangles, finds the 3D point
|
|
87
|
+
for each occupied texel, projects it into annotated cameras and ray-tests visibility against
|
|
88
|
+
the exact mesh. It does not interpolate semantic labels across unseen geometry. Distinct
|
|
89
|
+
surfaces sharing UV texels are reported as overlaps and remain unknown. Increasing atlas
|
|
90
|
+
resolution cannot add semantic accuracy beyond the annotation images.
|
|
91
|
+
|
|
92
|
+
Prepare hashes the PNG/NPZ evidence. Bake checks these hashes and the source mesh hash;
|
|
93
|
+
editing an image after annotation invalidates that evidence. Legacy manifests without
|
|
94
|
+
image hashes remain readable; prepare fresh evidence for new work. Preserve the original
|
|
95
|
+
manifest with its camera definitions. Evidence must not be resized or relit for annotation.
|
|
96
|
+
|
|
97
|
+
Review BOTH `<view>-overlay.png` (your polygons) and `<view>-reprojected.png` (the actual
|
|
98
|
+
baked atlas). All prepared views are reprojected, including views without annotations.
|
|
99
|
+
Palette: green skin, mint dorsal hand, cyan palm, blue eyes, purple hair, pink lips, pale
|
|
100
|
+
yellow nails, gray non-skin. Red means unknown; orange means conflict. Unknown clothing
|
|
101
|
+
is not missing skin. The raw `*-reprojected-labels.png` images retain categorical IDs.
|
|
102
|
+
|
|
103
|
+
The report distinguishes total visible pixels from annotated visible pixels. Use
|
|
104
|
+
`unresolvedAnnotatedPixels` and `disagreeingAnnotatedPixels` to detect projection losses;
|
|
105
|
+
these are NOT whole-character skin coverage or semantic accuracy scores. Low-resolution
|
|
106
|
+
atlases lose samples near small islands and seams even in dense mode. Match the source
|
|
107
|
+
atlas resolution, inspect the result, and keep unresolved seams visible. Do not hide them
|
|
108
|
+
with unrestricted dilation. A synthetic Blender regression exercises dense coverage,
|
|
109
|
+
occluded back surfaces and overlapping UVs.
|
|
110
|
+
|
|
111
|
+
Trials found source albedo spill on soldier b3843's palms, pale hair/ear ambiguity on b5588,
|
|
112
|
+
and missing side scalp coverage on b3658. These require different responses: record a
|
|
113
|
+
source-texture defect separately; correct hair/skin boundaries from multiple views; add
|
|
114
|
+
views/annotations for exposed surfaces. Indistinct nails require closer views. Do not
|
|
115
|
+
claim a material or skin-detail improvement from a semantic mask alone.
|
|
116
|
+
|
|
117
|
+
## Runtime packaging evidence
|
|
118
|
+
|
|
119
|
+
For the full-UV material adapter, add `--surface-data --triangle-labels` to the dense bake.
|
|
120
|
+
|
|
121
|
+
To investigate a seam or conflict, use `scripts/inspect_skin_point.py --evidence <folder>
|
|
122
|
+
--masks <dense-bake> --view <name> --x <pixel-x> --y <pixel-y>`. Coordinates belong to the
|
|
123
|
+
original evidence image. The read-only helper reports the UV texel and every visible
|
|
124
|
+
contributing view, its source pixel, authored label and geometric facing. It checks the
|
|
125
|
+
source, annotations and archived camera contract; it does not resolve the labels.
|
|
126
|
+
|
|
127
|
+
Add `--height-m <known-character-height>` to report surface distances to adjacent
|
|
128
|
+
occupied atlas texels in metres. Inspect individual neighbours: crossing a UV discontinuity
|
|
129
|
+
can produce a large distance. This diagnoses texture density, not anatomy or recovered
|
|
130
|
+
detail. If a face uses half-millimetre texels, increasing bump strength cannot recover
|
|
131
|
+
sub-texel pores. Preserve the registered detail in its own atlas instead of increasing
|
|
132
|
+
coverage or exaggerating relief to hide lost resolution.
|
|
133
|
+
|
|
134
|
+
Inspect the reported image pixels. A one-pixel garment-boundary mistake in a grazing
|
|
135
|
+
view can project into a much wider strip on the model. If that source boundary is
|
|
136
|
+
ambiguous and another view clearly shows the anatomy, explicitly mark the narrow
|
|
137
|
+
uncertain region in the ambiguous view as unknown and retain the reviewed clear view.
|
|
138
|
+
Do not automatically select a label by facing score or shrink all boundaries globally.
|
|
139
|
+
The first writes occupied texel positions, normals and geometric visibility bits in
|
|
140
|
+
Blender world space. The second classifies actual triangle centers with the same camera
|
|
141
|
+
annotations and exact visibility tests. Packaging matches those centers bijectively to the
|
|
142
|
+
GLB scene transform and fails if they differ; never assume Blender and GLB vertex ordering
|
|
143
|
+
match. This avoids assigning a whole triangle by a sparsely sampled UV center.
|
|
144
|
+
|
|
145
|
+
`triangles.npz` contains centers, labels and conflicts. `material-N-surface.npz` contains
|
|
146
|
+
position, normal, occupied, viewBits and visibilityBits. `viewBits` records views supporting
|
|
147
|
+
an annotation; `visibilityBits` independently records geometric visibility, including
|
|
148
|
+
unannotated views and unlabelled visible texels. Use the latter for projecting registered
|
|
149
|
+
detail, never an annotation's polygon boundary. Neither field assigns an anatomical label.
|
|
150
|
+
Older bakes without `visibilityBits` must be regenerated, not silently substituted.
|
|
151
|
+
Before consuming visibility bits, compare the archived `dense-request.json` source,
|
|
152
|
+
ordered cameras, image size, material ordering and world bounds against the supplied
|
|
153
|
+
manifest. A source hash alone cannot detect reordered camera bits. A color-only evidence
|
|
154
|
+
revision may preserve that geometric contract; changed cameras require a new bake.
|
|
155
|
+
These are build intermediates, not game textures.
|
|
156
|
+
The current visibility bit field supports up to 32 views in one revision. Blender workers
|
|
157
|
+
return a nonzero exit code on Python errors. Keep output revisions separate when changing
|
|
158
|
+
annotations; do not overwrite evidence to bypass a hash check.
|
|
159
|
+
|
|
160
|
+
Generic `skin` agrees with an observed `palm`, `sole` or `dorsal_hand` label. Two distinct
|
|
161
|
+
specific labels still conflict. This hierarchy combines semantic observations; it does not
|
|
162
|
+
recognize anatomy. A hard material boundary magnifies small polygon gaps: overlap enclosing
|
|
163
|
+
skin and its explicit hair/lip exclusions within the same view instead of leaving a narrow
|
|
164
|
+
unlabelled strip between them. Never fill an unseen nail or a contradictory observation.
|
|
165
|
+
|
|
166
|
+
### Thin unresolved folds
|
|
167
|
+
|
|
168
|
+
A closer view in the same direction does not necessarily resolve visibility gaps. The
|
|
169
|
+
b3658 rear-neck test retained a thin unobserved strip in a shallow fold until an oblique
|
|
170
|
+
view showed the recessed surface. Inspect source, coverage and shaded images separately.
|
|
171
|
+
Add that new view and annotate actual anatomy; never increase ray tolerance or fill unknown
|
|
172
|
+
regions merely to remove a line. Sample-level coverage improvements do not prove that all
|
|
173
|
+
residual albedo or filtering seams have disappeared. Keep those review results distinct.
|
|
174
|
+
|
|
175
|
+
|
|
176
|
+
### Source-color repair across UV islands
|
|
177
|
+
|
|
178
|
+
A harmonic image fill assumes the selected patch has clean boundaries. UV seams can split one anatomical patch into many islands whose boundaries remain contaminated. If a reviewed color repair remains unchanged, inspect the repaired albedo reprojected onto the exact mesh before extending semantic labels. For such defects, author clean color swatches on the same visible anatomical surface and interpolate in surface space, keeping the repair selection separate from anatomy. Check every donor against source hashes and independently reviewed skin. Record the swatch coordinates and inspect the final render. This is texture reconstruction, not segmentation or recovered source detail.
|
|
179
|
+
|
|
180
|
+
Designed features such as a nail plate absent from the source must be explicitly recorded as authored additions. They must not silently override contradictory anatomical observations; reconcile the views or keep the design unaccepted.
|
|
181
|
+
|
|
182
|
+
## Occluding silhouettes need precise annotations
|
|
183
|
+
|
|
184
|
+
A bottom-foot view may show the shin or clothing immediately beyond the heel and
|
|
185
|
+
between toes. A polygon around the apparent foot outline must exclude those visible
|
|
186
|
+
surfaces too; they are not necessarily empty background. In the b3662 trial, a few pixels
|
|
187
|
+
of excess sole annotation produced a curved sole-detail strip on the shin. The ray test
|
|
188
|
+
correctly saw that shin surface—the semantic polygon was wrong. Trace a bad rendered
|
|
189
|
+
point with `scripts/inspect_skin_point.py`, inspect the contributing source pixel, correct
|
|
190
|
+
that specific polygon, and reproject. Do not move a shader cutoff to conceal the artifact.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Regional detail helpers
|
|
2
|
+
|
|
3
|
+
The shipped `scripts/authored_detail.py` and `scripts/authored_surface_patch.py` are
|
|
4
|
+
Blender-independent NumPy/Pillow/SciPy libraries. They evaluate already-authored curves
|
|
5
|
+
and roughness patches. They do not recognize anatomy, load a rig, or export materials.
|
|
6
|
+
Keep profiles and licensed images in the character workspace, outside the refreshed skill.
|
|
7
|
+
|
|
8
|
+
Pass the exact evidence manifest and physical character height in metres. Surface points
|
|
9
|
+
must use the evidence's Blender world axes, centered on `worldBounds` and scaled by
|
|
10
|
+
physical height divided by the source bounds' Z extent. Normals use those same axes.
|
|
11
|
+
`visibility` is the dense bake's **geometric `visibilityBits`**, not annotation `viewBits`.
|
|
12
|
+
The label array is the reviewed semantic map. Both helpers leave other regions unchanged.
|
|
13
|
+
|
|
14
|
+
`AuthoredDetail(profile_path, manifest, height_m).evaluate(points, normals, labels,
|
|
15
|
+
visibility)` returns signed relief in metres. The profile contains `sourceSha256`,
|
|
16
|
+
`evidenceHashes`, `provenance`, and `curves`. Each curve specifies `view`, `region`,
|
|
17
|
+
`pointsPx`, `widthM`, and `depthM`. Regions supported here are `skin`, `palm`, `sole`,
|
|
18
|
+
and `dorsal_hand`. Coordinates belong to the exact source-view image. Record the curves
|
|
19
|
+
as designed enhancements, not measured anatomy.
|
|
20
|
+
|
|
21
|
+
`AuthoredSurfacePatch(profile_path, manifest, height_m, workspace_root).evaluate(points,
|
|
22
|
+
normals, labels, visibility, base_roughness)` returns blended roughness. The profile has
|
|
23
|
+
`sourceSha256`, `evidenceHashes`, and `patches`. Each patch contains:
|
|
24
|
+
|
|
25
|
+
- `view`, `region`, ordered `targetPixels` around the selected polygon.
|
|
26
|
+
- `roughnessTexture`, `textureSha256`, `sourceAnnotationImageSize`, and corresponding
|
|
27
|
+
`sourcePixels` from the independently inspected donor image.
|
|
28
|
+
- `featherM` and `roughnessRange` as an explicit authored response range.
|
|
29
|
+
- `provenance`: `title`, `url`, `licenseFile`, and `licenseSha256`.
|
|
30
|
+
|
|
31
|
+
Texture and license paths are relative to the supplied workspace root. The helper checks
|
|
32
|
+
the current evidence image, texture and license hashes. Retain its `report` and append
|
|
33
|
+
all `attributions` to the exported asset's notices. Exemplar roughness is not necessarily
|
|
34
|
+
measured: record its origin accurately. This helper transfers neither color nor normals.
|
|
35
|
+
Normal-map RGB is not height data; integrating normals requires a declared tangent
|
|
36
|
+
convention and surface metric, followed by independent relief review.
|
|
37
|
+
|
|
38
|
+
An optional `height` record supplies an already prepared signed-height NumPy array:
|
|
39
|
+
`path`, `sha256`, `boundsPx` in the donor annotation coordinates, explicit `scale`,
|
|
40
|
+
and `provenance`. Values are metres, not RGB channels. The separate
|
|
41
|
+
`relief(points, normals, labels, visibility)` method returns the feathered height
|
|
42
|
+
contribution, leaving roughness evaluation unchanged. Preserve the method, source
|
|
43
|
+
normal convention, assumed metric and any artistic amplitude change in provenance.
|
|
44
|
+
Use known or explicitly designed height; this helper does not integrate normal maps.
|
|
45
|
+
|
|
46
|
+
For a square, locally planar exemplar with a declared isotropic metric,
|
|
47
|
+
`scripts/normal_patch_height.py` exposes `normal_patch_height(rgb, width_m, normal_y)`.
|
|
48
|
+
RGB is the raw linear normal-map data in [0,1]; `normal_y` must explicitly be `up` or
|
|
49
|
+
`down`. It uses [Frankot/Chellappa discrete slope integration](https://doi.org/10.1109/34.3909),
|
|
50
|
+
a reflected periodic extension, and physically specified band limiting. Save its output
|
|
51
|
+
as a signed-metre NumPy array and inspect the reconstructed folds before transfer. It
|
|
52
|
+
does not recover an unknown donor metric, recognize a skin region, or make artist-authored
|
|
53
|
+
normal maps into measured scans. The playground's Hafnia palm test explicitly assumes an
|
|
54
|
+
80mm planar crop and records that assumption; it is not a universal scale for hand assets.
|
|
55
|
+
|
|
56
|
+
`sampleResolution` can increase field sampling independently of the annotation image.
|
|
57
|
+
Physical feather width stays unchanged. When the project's exporter supports regional
|
|
58
|
+
atlas packing, `independentDetail: true` exposes the patch through `independent_planes()`.
|
|
59
|
+
The exporter must then exclude it from the coarse body bake (`include_independent=False`)
|
|
60
|
+
and register its texture coordinates separately. Do not silently omit it in an exporter
|
|
61
|
+
that lacks this path. Overlapping independent bindings require a deliberately combined
|
|
62
|
+
patch; there is no automatic regional winner.
|
|
63
|
+
|
|
64
|
+
Inspect the transferred patch at region-sized framing under side and specular lighting.
|
|
65
|
+
A small central-palm experiment is not acceptance of the whole hand, fingers or nails.
|
|
66
|
+
If the full-body atlas cannot resolve the patch, address texel density before amplifying it.
|
|
67
|
+
|
|
68
|
+
For independent atlas slots, keep a transparent margin around the authored patch inside
|
|
69
|
+
the evidence view. The playground packer rejects edge-touching patches rather than
|
|
70
|
+
clamping visible relief into a seam. Widen/reframe the evidence and review the annotation
|
|
71
|
+
again. Packing a patch at higher resolution is not a visual acceptance: inspect its relief
|
|
72
|
+
at an ordinary region-sized view and with side light before claiming improved skin.
|