expression-lab-classroom 1.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,10 @@
1
+ .firecrawl/
2
+ .research/
3
+ .venv/
4
+ __pycache__/
5
+ .pytest_cache/
6
+ .ruff_cache/
7
+ .coverage
8
+ htmlcov/
9
+ dist/
10
+ .build/
@@ -0,0 +1,80 @@
1
+ Metadata-Version: 2.5
2
+ Name: expression-lab-classroom
3
+ Version: 1.3.0
4
+ Summary: 离线 CPU 人脸点位、AU 与相对位移教学实验室
5
+ License-File: licenses/LibreFace-USC.txt
6
+ License-File: licenses/MediaPipe-Apache-2.0.txt
7
+ Requires-Python: <3.13,>=3.12
8
+ Requires-Dist: jax==0.7.1
9
+ Requires-Dist: jaxlib==0.7.1
10
+ Requires-Dist: mediapipe==0.10.21
11
+ Requires-Dist: numpy==1.26.4
12
+ Requires-Dist: onnxruntime==1.23.2
13
+ Requires-Dist: opencv-contrib-python==4.11.0.86
14
+ Requires-Dist: pyside6-essentials==6.8.3
15
+ Description-Content-Type: text/markdown
16
+
17
+ # 人脸表情实验室 · Expression Lab Classroom
18
+
19
+ 面向高中课堂的 Python 3.12 人脸点位、AU 动作特征与表情规则可视化工具。
20
+
21
+ ## 安装
22
+
23
+ Windows 10/11 x64(Python 3.12):
24
+
25
+ ```shell
26
+ py -3.12 -m pip install expression-lab-classroom
27
+ ```
28
+
29
+ 中国大陆可使用国内 PyPI 镜像,镜像同步新版本可能有延迟:
30
+
31
+ ```shell
32
+ py -3.12 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple expression-lab-classroom
33
+ ```
34
+
35
+ Mac Apple Silicon、macOS 14+ 使用 `python3.12 -m pip` 安装;建议使用虚拟环境。本项目wheel不含平台二进制,底层依赖由pip按操作系统选择。目前支持范围为上述两个平台。
36
+
37
+ ## 学生代码(9行)
38
+
39
+ ```python
40
+ from expression_lab import FaceCamera
41
+ camera = FaceCamera()
42
+ for f in camera:
43
+ expression = "自然"
44
+ if f[1] >= 7 and f[2] >= 5:
45
+ expression = "微笑"
46
+ elif f[3] >= 6:
47
+ expression = "张嘴"
48
+ camera.show(expression)
49
+ ```
50
+
51
+ 保存为 `学生主程序.py`,执行 `py -3.12 学生主程序.py` 自动打开原生桌面窗口。若Windows的 `py -3` 默认指向Python3.12,也可用 `py -3 学生主程序.py`。Mac使用 `python3.12 学生主程序.py`。不需要浏览器。
52
+
53
+ 启动后点击“采集基准”,保持自然表情约1–2秒,再做动作。点击AU行可切换代表点与曲线。`f[1]`、`f[2]`、`f[3]`分别读取AU12、AU6、AU26各自的f1;索引不是特征名称,每个AU的特征统一称为f1。完整索引映射:1→AU12、2→6、3→26、4→4、5→1、6→2、7→5、8→9、9→15、10→17、11→20、12→25。
54
+
55
+ ## 模型从国内BOS下载,之后离线运行
56
+
57
+ 代码包不包含模型权重。首次运行缺少模型时,在后台从百度BOS下载约47MB模型,界面显示进度。每个文件使用安装包内固定的SHA256核验,完成后原子写入本机用户缓存。已有完整缓存时不访问网络。失败后重新打开程序,只下载缺少或损坏的文件。
58
+
59
+ 缓存目录:Windows `%LOCALAPPDATA%/expression-lab/models/libreface-v1`;Mac `~/Library/Caches/expression-lab/models/libreface-v1`。教师可设置 `EXPRESSION_LAB_CACHE_DIR` 指定缓存根目录,底下仍保留 `models/libreface-v1`。BOS访问不需要学生的账户或密钥。
60
+
61
+ 教师可在课前联网执行以下命令,再断网使用:
62
+
63
+ ```shell
64
+ py -3.12 -m expression_lab.download_models
65
+ py -3.12 -m expression_lab.download_models --check
66
+ ```
67
+
68
+ `--check`只检查本地文件和SHA256,不联网。也可把已经准备好的 `models/libreface-v1` 目录复制到离线电脑的缓存根目录。显式 `FaceCamera(model_dir="本地目录")` 或 `--model-dir` 始终只使用指定目录,不自动下载。完整离线ZIP保持原有本地模型读取方式。
69
+
70
+ 无摄像头可执行 `py -3.12 -m expression_lab --sample`,使用随包的虚构成年人人脸样例。
71
+
72
+ ## 特征含义与模型
73
+
74
+ MediaPipe离线提供478个人脸点;LibreFace ONNX模型离线提供12个AU参考输出。默认f1采用人脸几何位移估计:按照Lien1998对齐尺度,计算相对自然基准的方向约束位移d,用 `η=d/√(d²+k²)` 换算为0–20整数。AU6、17为课堂代理参数,AU25、26共用唇间距代理;数值不是FACS官方人工A–E评级,也不表示对内心情绪的判断。
75
+
76
+ 参考:[Lien1998论文](https://www.cs.cmu.edu/~jjlien/Thesis/RiTr9831.pdf)、[LibreFace](https://github.com/ihp-lab/LibreFace)、[MediaPipe](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker)。
77
+
78
+ ## 许可证
79
+
80
+ 随包保留各上游许可证。LibreFace模型及相关实现的USC授权允许教育、研究和非营利用途;商业用途需要另行取得许可。模型来源和SHA256保留在包内清单中。
@@ -0,0 +1,64 @@
1
+ # 人脸表情实验室 · Expression Lab Classroom
2
+
3
+ 面向高中课堂的 Python 3.12 人脸点位、AU 动作特征与表情规则可视化工具。
4
+
5
+ ## 安装
6
+
7
+ Windows 10/11 x64(Python 3.12):
8
+
9
+ ```shell
10
+ py -3.12 -m pip install expression-lab-classroom
11
+ ```
12
+
13
+ 中国大陆可使用国内 PyPI 镜像,镜像同步新版本可能有延迟:
14
+
15
+ ```shell
16
+ py -3.12 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple expression-lab-classroom
17
+ ```
18
+
19
+ Mac Apple Silicon、macOS 14+ 使用 `python3.12 -m pip` 安装;建议使用虚拟环境。本项目wheel不含平台二进制,底层依赖由pip按操作系统选择。目前支持范围为上述两个平台。
20
+
21
+ ## 学生代码(9行)
22
+
23
+ ```python
24
+ from expression_lab import FaceCamera
25
+ camera = FaceCamera()
26
+ for f in camera:
27
+ expression = "自然"
28
+ if f[1] >= 7 and f[2] >= 5:
29
+ expression = "微笑"
30
+ elif f[3] >= 6:
31
+ expression = "张嘴"
32
+ camera.show(expression)
33
+ ```
34
+
35
+ 保存为 `学生主程序.py`,执行 `py -3.12 学生主程序.py` 自动打开原生桌面窗口。若Windows的 `py -3` 默认指向Python3.12,也可用 `py -3 学生主程序.py`。Mac使用 `python3.12 学生主程序.py`。不需要浏览器。
36
+
37
+ 启动后点击“采集基准”,保持自然表情约1–2秒,再做动作。点击AU行可切换代表点与曲线。`f[1]`、`f[2]`、`f[3]`分别读取AU12、AU6、AU26各自的f1;索引不是特征名称,每个AU的特征统一称为f1。完整索引映射:1→AU12、2→6、3→26、4→4、5→1、6→2、7→5、8→9、9→15、10→17、11→20、12→25。
38
+
39
+ ## 模型从国内BOS下载,之后离线运行
40
+
41
+ 代码包不包含模型权重。首次运行缺少模型时,在后台从百度BOS下载约47MB模型,界面显示进度。每个文件使用安装包内固定的SHA256核验,完成后原子写入本机用户缓存。已有完整缓存时不访问网络。失败后重新打开程序,只下载缺少或损坏的文件。
42
+
43
+ 缓存目录:Windows `%LOCALAPPDATA%/expression-lab/models/libreface-v1`;Mac `~/Library/Caches/expression-lab/models/libreface-v1`。教师可设置 `EXPRESSION_LAB_CACHE_DIR` 指定缓存根目录,底下仍保留 `models/libreface-v1`。BOS访问不需要学生的账户或密钥。
44
+
45
+ 教师可在课前联网执行以下命令,再断网使用:
46
+
47
+ ```shell
48
+ py -3.12 -m expression_lab.download_models
49
+ py -3.12 -m expression_lab.download_models --check
50
+ ```
51
+
52
+ `--check`只检查本地文件和SHA256,不联网。也可把已经准备好的 `models/libreface-v1` 目录复制到离线电脑的缓存根目录。显式 `FaceCamera(model_dir="本地目录")` 或 `--model-dir` 始终只使用指定目录,不自动下载。完整离线ZIP保持原有本地模型读取方式。
53
+
54
+ 无摄像头可执行 `py -3.12 -m expression_lab --sample`,使用随包的虚构成年人人脸样例。
55
+
56
+ ## 特征含义与模型
57
+
58
+ MediaPipe离线提供478个人脸点;LibreFace ONNX模型离线提供12个AU参考输出。默认f1采用人脸几何位移估计:按照Lien1998对齐尺度,计算相对自然基准的方向约束位移d,用 `η=d/√(d²+k²)` 换算为0–20整数。AU6、17为课堂代理参数,AU25、26共用唇间距代理;数值不是FACS官方人工A–E评级,也不表示对内心情绪的判断。
59
+
60
+ 参考:[Lien1998论文](https://www.cs.cmu.edu/~jjlien/Thesis/RiTr9831.pdf)、[LibreFace](https://github.com/ihp-lab/LibreFace)、[MediaPipe](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker)。
61
+
62
+ ## 许可证
63
+
64
+ 随包保留各上游许可证。LibreFace模型及相关实现的USC授权允许教育、研究和非营利用途;商业用途需要另行取得许可。模型来源和SHA256保留在包内清单中。
@@ -0,0 +1,13 @@
1
+ """高中人脸表情实验室:本地 CPU 模型与桌面可视化。"""
2
+
3
+ __version__ = "1.3.0"
4
+
5
+
6
+ def __getattr__(name):
7
+ # 几何计算和模型自检不需要初始化 Qt 或打开摄像头。
8
+ if name == "FaceCamera":
9
+ from .bootstrap import ensure_runtime
10
+ ensure_runtime()
11
+ from .api import FaceCamera
12
+ return FaceCamera
13
+ raise AttributeError(name)
@@ -0,0 +1,17 @@
1
+ from . import FaceCamera
2
+
3
+
4
+ def main():
5
+ camera = FaceCamera()
6
+ camera.window.result.setToolTip('AU12 的 f1 ≥ 7 且 AU6 的 f1 ≥ 5 → 微笑;AU26 的 f1 ≥ 6 → 张嘴;其余 → 自然')
7
+ for f in camera:
8
+ expression = '自然'
9
+ if f[1] >= 7 and f[2] >= 5:
10
+ expression = '微笑'
11
+ elif f[3] >= 6:
12
+ expression = '张嘴'
13
+ camera.show(expression)
14
+
15
+
16
+ if __name__ == '__main__':
17
+ main()
@@ -0,0 +1,192 @@
1
+ """学生使用的极简接口;迭代器内部处理事件、模型和资源释放。"""
2
+
3
+ import argparse
4
+ import ast
5
+ import queue
6
+ import sys
7
+ import time
8
+ from pathlib import Path
9
+
10
+ from PySide6.QtCore import QEventLoop, QTimer
11
+ from PySide6.QtGui import QFont
12
+ from PySide6.QtWidgets import QApplication
13
+
14
+ from .cameras import CameraDevice, CameraMonitor
15
+ from .core import feature_values
16
+ from .resources import sample_image
17
+ from .ui import LabWindow
18
+ from .worker import InferenceWorker
19
+
20
+
21
+ def rule_description(path):
22
+ try:
23
+ tree = ast.parse(Path(path).read_text(encoding='utf-8-sig'))
24
+ descriptions = []
25
+ for node in ast.walk(tree):
26
+ if not isinstance(node, ast.If):
27
+ continue
28
+ for statement in node.body:
29
+ if (isinstance(statement, ast.Assign) and isinstance(statement.value, ast.Constant)
30
+ and any(isinstance(target, ast.Name) and target.id == 'expression'
31
+ for target in statement.targets)):
32
+ condition = ast.unparse(node.test).replace(' and ', ' 且 ').replace(' or ', ' 或 ')
33
+ descriptions.append(f'{condition} → {statement.value.value}')
34
+ if descriptions:
35
+ return ';'.join(descriptions) + ';其余 → 默认值'
36
+ return '按学生主程序中的条件判断'
37
+ except (OSError, SyntaxError):
38
+ return '按学生主程序中的条件判断'
39
+
40
+
41
+ def parse_options(args):
42
+ parser = argparse.ArgumentParser(description='离线人脸 AU 表情实验室(Python 3.12)')
43
+ source = parser.add_mutually_exclusive_group()
44
+ source.add_argument('--camera', type=int, default=0, help='摄像头序号,默认 0')
45
+ source.add_argument('--image', '--video', dest='source', help='改用本地图片或循环视频')
46
+ source.add_argument('--sample', action='store_true', help='使用随包离线人脸图片')
47
+ parser.add_argument('--model-dir', type=Path, help='本地模型目录')
48
+ parser.add_argument('--fps', type=int, default=15, choices=range(1, 31), metavar='1–30', help='推理帧率上限,默认 15')
49
+ return parser.parse_args(args)
50
+
51
+
52
+ class FaceCamera:
53
+ def __init__(self, camera=0, *, source=None, model_dir=None, fps=15, args=None, values='features'):
54
+ if values not in ('features', 'au'):
55
+ raise ValueError('values 应为 features(0–20整数分)或 au(原始0–1强度)')
56
+ self._values = values
57
+ options = parse_options(sys.argv[1:] if args is None else args)
58
+ if source is None:
59
+ source = sample_image() if options.sample else options.source
60
+ if source is None:
61
+ source = options.camera if options.camera != 0 else camera
62
+ self._model_dir = model_dir if model_dir is not None else options.model_dir
63
+ self._fps = options.fps if options.fps != 15 else fps
64
+ self._app = QApplication.instance() or QApplication([])
65
+ self._app.setFont(QFont('Microsoft YaHei' if sys.platform == 'win32' else 'PingFang SC', 10))
66
+ self._app.setQuitOnLastWindowClosed(False)
67
+ self._loop = QEventLoop()
68
+ self._timer = QTimer()
69
+ self._timer.setInterval(16)
70
+ self._timer.timeout.connect(self._loop.quit)
71
+ self._timer.timeout.connect(self._poll_cameras)
72
+ self.window = LabWindow()
73
+ self.window.result.setToolTip(rule_description(Path(sys.argv[0])))
74
+ self._closed = False
75
+ self.current = None
76
+ self.worker = None
77
+ self._retired_workers = []
78
+ self._camera_monitor = CameraMonitor()
79
+ self._camera_monitor.start()
80
+ self.window.closed.connect(self.close)
81
+ self.window.calibrate_requested.connect(lambda: self.worker.commands.put('calibrate'))
82
+ self.window.reset_requested.connect(lambda: self.worker.commands.put('reset'))
83
+ self.window.pause_changed.connect(self._pause)
84
+ self.window.source_requested.connect(self._start_worker)
85
+ self.window.show()
86
+ self._start_worker(source)
87
+ self._timer.start()
88
+
89
+ @property
90
+ def is_closed(self):
91
+ return self._closed
92
+
93
+ def _pause(self, paused):
94
+ if paused:
95
+ self.worker.paused.set()
96
+ else:
97
+ # 丢弃暂停前尚未显示的结果,继续后等新一帧。
98
+ try:
99
+ self.worker.results.get_nowait()
100
+ except queue.Empty:
101
+ pass
102
+ self.worker.paused.clear()
103
+
104
+ def _start_worker(self, source):
105
+ if self._closed:
106
+ return
107
+ if self.worker is not None:
108
+ self.worker.stop.set()
109
+ self._retired_workers.append(self.worker)
110
+ # 摄像头驱动可能迟迟不返回。主线程只发取消信号,绝不 join/read/release。
111
+ self._retired_workers = [worker for worker in self._retired_workers if worker.is_alive()]
112
+ self._last_result_at = time.monotonic()
113
+ self._has_result = False
114
+ self._stale_notified = False
115
+ self.current = None
116
+ self.window.last_observation = None
117
+ self.window.chart.history.clear()
118
+ self.window.pause_button.setChecked(False)
119
+ self.window.canvas.observation = None
120
+ self.window.canvas.placeholder = '正在加载本地 CPU 模型…'
121
+ self.window.canvas.mirror = isinstance(source, (int, CameraDevice))
122
+ self.window.set_source(source)
123
+ self.window.show_error('正在加载本地 CPU 模型…')
124
+ self.worker = InferenceWorker(source, model_dir=self._model_dir, fps=self._fps)
125
+ self.worker.start()
126
+
127
+ def _poll_cameras(self):
128
+ try:
129
+ devices, error = self._camera_monitor.results.get_nowait()
130
+ except queue.Empty:
131
+ return
132
+ self.window.set_cameras(devices, error)
133
+
134
+ def __iter__(self):
135
+ try:
136
+ while not self._closed:
137
+ observation = self._wait_for_face()
138
+ if observation is None:
139
+ return
140
+ self.current = observation
141
+ yield feature_values(observation.geometric_au) if self._values == 'features' else observation.au.copy()
142
+ finally:
143
+ self.close()
144
+
145
+ def _wait_for_face(self):
146
+ while not self._closed:
147
+ if not self.window.paused:
148
+ try:
149
+ result = self.worker.results.get_nowait()
150
+ except queue.Empty:
151
+ result = None
152
+ if isinstance(result, dict):
153
+ self._stale_notified = True
154
+ self.window.show_error(result.get('progress', result.get('error', '正在准备模型…')))
155
+ elif result is not None:
156
+ self._last_result_at = time.monotonic()
157
+ self._has_result = True
158
+ self._stale_notified = False
159
+ if result.points is not None and (self._values == 'au' or result.geometric_au):
160
+ return result
161
+ self.window.present(result, None)
162
+ elif not self._stale_notified:
163
+ timeout = 2 if self._has_result else 8
164
+ if time.monotonic() - self._last_result_at > timeout:
165
+ self.current = None
166
+ self.window.show_error('等待摄像头画面,请切换来源。')
167
+ self._stale_notified = True
168
+ # Qt 事件循环等待下一次定时器或关窗事件,不使用占满 CPU 的忙循环。
169
+ self._loop.exec()
170
+ return None
171
+
172
+ def show(self, expression):
173
+ if not self._closed and self.current is not None:
174
+ self.window.present(self.current, str(expression))
175
+
176
+ def close(self):
177
+ if self._closed:
178
+ return
179
+ self._closed = True
180
+ self._timer.stop()
181
+ self._camera_monitor.stop.set()
182
+ self._loop.quit()
183
+ for worker in [self.worker, *self._retired_workers]:
184
+ if worker is not None:
185
+ worker.stop.set()
186
+ self.window.hide()
187
+
188
+ def __enter__(self):
189
+ return self
190
+
191
+ def __exit__(self, *_):
192
+ self.close()
@@ -0,0 +1,166 @@
1
+ """本地 MediaPipe + LibreFace ONNX CPU 推理;初始化时准备模型缓存。"""
2
+
3
+ import hashlib
4
+ import json
5
+ import time
6
+ from dataclasses import dataclass, field
7
+ from pathlib import Path
8
+
9
+ import cv2
10
+ import numpy as np
11
+
12
+ from .core import decode_intensities, face_coordinates
13
+ from .download_models import prepare_models
14
+ from .resources import model_directory
15
+
16
+ PROJECT_ROOT = Path(__file__).resolve().parents[1]
17
+ REQUIRED_MODELS = ('face_landmarker.task', 'LibreFace_AU_Encoder.onnx', 'LibreFace_AU_Intensity.onnx')
18
+ EYE_LEFT = (33, 7, 163, 144, 145, 153, 154, 155, 133, 246, 161, 160, 159, 158, 157, 173)
19
+ EYE_RIGHT = (263, 249, 390, 373, 374, 380, 381, 382, 362, 466, 388, 387, 386, 385, 384, 398)
20
+ MOUTH = (61, 146, 91, 181, 84, 17, 314, 405, 321, 375, 291, 185, 40, 39, 37, 0, 267,
21
+ 269, 270, 409, 78, 95, 88, 178, 87, 14, 317, 402, 318, 324, 308, 191, 80, 81,
22
+ 82, 13, 312, 311, 310, 415)
23
+
24
+
25
+ class ModelFiles:
26
+ def __init__(self, directory=None):
27
+ self.directory = Path(directory) if directory is not None else model_directory()
28
+
29
+ def __getitem__(self, name):
30
+ return self.directory / name
31
+
32
+ def validate(self, verify_hashes=False):
33
+ for name in REQUIRED_MODELS:
34
+ path = self[name]
35
+ if not path.is_file() or path.stat().st_size < 10_000:
36
+ raise FileNotFoundError(
37
+ f'缺少或损坏本地模型:{path}。pip 版请运行 expression-lab-models,'
38
+ '离线包请恢复 models 文件夹。')
39
+ if verify_hashes:
40
+ manifest = json.loads((self.directory / 'manifest.json').read_text(encoding='utf-8'))
41
+ for name in REQUIRED_MODELS:
42
+ if hashlib.sha256(self[name].read_bytes()).hexdigest() != manifest['files'][name]['sha256']:
43
+ raise ValueError(f'本地模型校验失败:{name}。请从完整离线包恢复。')
44
+
45
+
46
+ def aligned_face(frame_bgr, points):
47
+ """LibreFace/FFHQ 的眼睛—嘴部四边形对齐,256 图中心裁成 224。
48
+
49
+ 保持上游的轴与尺度;超出画面的部分用反射填充(不做上游的边缘模糊)。
50
+ """
51
+ left = points[list(EYE_LEFT)].mean(axis=0)
52
+ right = points[list(EYE_RIGHT)].mean(axis=0)
53
+ eye_average = (left + right) / 2
54
+ eye_to_eye = right - left
55
+ lips = points[list(MOUTH)]
56
+ mouth_average = (lips[np.argmin(lips[:, 0])] + lips[np.argmax(lips[:, 0])]) / 2
57
+ eye_to_mouth = mouth_average - eye_average
58
+ x_axis = eye_to_eye - np.array([-eye_to_mouth[1], eye_to_mouth[0]])
59
+ length = np.linalg.norm(x_axis)
60
+ if length < 1e-6:
61
+ raise ValueError('人脸对齐失败,请正对摄像头')
62
+ x_axis = x_axis / length * max(np.linalg.norm(eye_to_eye) * 2, np.linalg.norm(eye_to_mouth) * 1.8)
63
+ y_axis = np.array([-x_axis[1], x_axis[0]])
64
+ center = eye_average + eye_to_mouth * 0.1
65
+ source = np.float32([center - x_axis - y_axis, center - x_axis + y_axis, center + x_axis - y_axis])
66
+ destination = np.float32([[0, 0], [0, 255], [255, 0]])
67
+ transform = cv2.getAffineTransform(source, destination)
68
+ aligned = cv2.warpAffine(frame_bgr, transform, (256, 256), flags=cv2.INTER_LINEAR,
69
+ borderMode=cv2.BORDER_REFLECT_101)
70
+ return np.ascontiguousarray(aligned[16:240, 16:240])
71
+
72
+
73
+ def image_tensor(aligned_bgr):
74
+ rgb = cv2.cvtColor(aligned_bgr, cv2.COLOR_BGR2RGB).astype(np.float32) / 255
75
+ normalized = (rgb - np.array([0.485, 0.456, 0.406], np.float32)) / np.array(
76
+ [0.229, 0.224, 0.225], np.float32)
77
+ return np.ascontiguousarray(normalized.transpose(2, 0, 1)[None])
78
+
79
+
80
+ @dataclass
81
+ class Observation:
82
+ frame: np.ndarray
83
+ timestamp: float
84
+ points: np.ndarray | None = None
85
+ coordinates: np.ndarray | None = None
86
+ au: dict = field(default_factory=dict)
87
+ inference_ms: float = 0
88
+ status: str = '请正对摄像头'
89
+ baseline: np.ndarray | None = None
90
+ motion: dict = field(default_factory=dict)
91
+ geometric_au: dict = field(default_factory=dict)
92
+ geometry: dict = field(default_factory=dict)
93
+ standard_baseline: np.ndarray | None = None
94
+ calibration_count: int = 0
95
+ collecting: bool = False
96
+ fps: float = 0
97
+
98
+
99
+ class FaceEngine:
100
+ def __init__(self, model_dir=None, threads=2):
101
+ # 先准备本地资源;pip 版首次缺少缓存时由 BOS 下载,显式目录始终只读本地。
102
+ files = ModelFiles(prepare_models() if model_dir is None else model_dir)
103
+ files.validate(verify_hashes=True)
104
+ import mediapipe as mp
105
+ import onnxruntime as ort
106
+
107
+ self._mp = mp
108
+ self._last_timestamp = -1
109
+ options = ort.SessionOptions()
110
+ options.intra_op_num_threads = threads
111
+ options.inter_op_num_threads = 1
112
+ options.add_session_config_entry('session.intra_op.allow_spinning', '0')
113
+ self._encoder = ort.InferenceSession(str(files['LibreFace_AU_Encoder.onnx']), options,
114
+ providers=['CPUExecutionProvider'])
115
+ self._intensity = ort.InferenceSession(str(files['LibreFace_AU_Intensity.onnx']), options,
116
+ providers=['CPUExecutionProvider'])
117
+ self._encoder_input = self._encoder.get_inputs()[0].name
118
+ self._intensity_input = self._intensity.get_inputs()[0].name
119
+ landmarker_options = mp.tasks.vision.FaceLandmarkerOptions(
120
+ # 从本地读成 bytes,也支持 Windows 中文目录,避开原生文件名编码差异。
121
+ base_options=mp.tasks.BaseOptions(model_asset_buffer=files['face_landmarker.task'].read_bytes(),
122
+ delegate=mp.tasks.BaseOptions.Delegate.CPU),
123
+ running_mode=mp.tasks.vision.RunningMode.VIDEO,
124
+ num_faces=1,
125
+ min_face_detection_confidence=0.5,
126
+ min_face_presence_confidence=0.5,
127
+ min_tracking_confidence=0.5,
128
+ )
129
+ self._landmarker = mp.tasks.vision.FaceLandmarker.create_from_options(landmarker_options)
130
+
131
+ def process(self, frame_bgr, timestamp=None):
132
+ if frame_bgr is None or frame_bgr.ndim != 3 or frame_bgr.shape[2] != 3:
133
+ raise ValueError('需要有效的 BGR 摄像头画面')
134
+ started = time.perf_counter()
135
+ now = time.monotonic() if timestamp is None else timestamp
136
+ timestamp_ms = max(self._last_timestamp + 1, int(now * 1000))
137
+ self._last_timestamp = timestamp_ms
138
+ rgb = np.ascontiguousarray(cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB))
139
+ result = self._landmarker.detect_for_video(
140
+ self._mp.Image(image_format=self._mp.ImageFormat.SRGB, data=rgb), timestamp_ms)
141
+ observation = Observation(frame=frame_bgr, timestamp=now)
142
+ if result.face_landmarks:
143
+ h, w = frame_bgr.shape[:2]
144
+ points = np.array([(point.x * w, point.y * h) for point in result.face_landmarks[0]])
145
+ tensor = image_tensor(aligned_face(frame_bgr, points))
146
+ features = self._encoder.run(None, {self._encoder_input: tensor})[0].reshape(1, -1)
147
+ output = self._intensity.run(None, {self._intensity_input: features})[0]
148
+ observation.points = points
149
+ observation.coordinates = face_coordinates(points)
150
+ observation.au = decode_intensities(output)
151
+ observation.status = '已识别人脸'
152
+ else:
153
+ observation.status = '未检测到人脸 · 请正对摄像头'
154
+ observation.inference_ms = (time.perf_counter() - started) * 1000
155
+ return observation
156
+
157
+ def close(self):
158
+ if self._landmarker is not None:
159
+ self._landmarker.close()
160
+ self._landmarker = None
161
+
162
+ def __enter__(self):
163
+ return self
164
+
165
+ def __exit__(self, *_):
166
+ self.close()
@@ -0,0 +1,36 @@
1
+ """只使用标准库,让 py -3 自动转入随项目准备的 Python 3.12 环境。"""
2
+
3
+ import importlib.util
4
+ import subprocess
5
+ import sys
6
+ from pathlib import Path
7
+
8
+
9
+ def ensure_runtime(root=None):
10
+ root = Path(root) if root is not None else Path(__file__).resolve().parents[1]
11
+ environments = (
12
+ (root / '.venv', 'Scripts/python.exe' if sys.platform == 'win32' else 'bin/python'),
13
+ (root / 'runtime/macos_arm64', 'bin/python3.12'),
14
+ )
15
+ for environment, relative_executable in environments:
16
+ if Path(sys.prefix).resolve() == environment.resolve():
17
+ return
18
+ executable = environment / relative_executable
19
+ if executable.is_file():
20
+ arguments = sys.argv
21
+ specification = getattr(sys.modules.get('__main__'), '__spec__', None)
22
+ if specification is not None:
23
+ arguments = ['-m', specification.name, *sys.argv[1:]]
24
+ try:
25
+ status = subprocess.call([str(executable), *arguments])
26
+ except KeyboardInterrupt:
27
+ status = 130
28
+ raise SystemExit(status)
29
+ dependencies = ('numpy', 'cv2', 'mediapipe', 'onnxruntime', 'PySide6')
30
+ if sys.version_info[:2] == (3, 12) and all(importlib.util.find_spec(name) for name in dependencies):
31
+ return
32
+ raise SystemExit(
33
+ '请先双击 安装依赖_Windows.cmd,然后执行 py -3 学生主程序.py。\n'
34
+ 'pip 安装版:请在 Python 3.12 中用 python -m pip install 安装完整 wheel 及依赖。\n'
35
+ 'Mac 请解压完整的 Apple 芯片离线包,双击 启动实验_macOS.command。'
36
+ )