expression-lab-classroom 1.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- expression_lab_classroom-1.3.0/.gitignore +10 -0
- expression_lab_classroom-1.3.0/PKG-INFO +80 -0
- expression_lab_classroom-1.3.0/PYPI_README.md +64 -0
- expression_lab_classroom-1.3.0/expression_lab/__init__.py +13 -0
- expression_lab_classroom-1.3.0/expression_lab/__main__.py +17 -0
- expression_lab_classroom-1.3.0/expression_lab/api.py +192 -0
- expression_lab_classroom-1.3.0/expression_lab/backend.py +166 -0
- expression_lab_classroom-1.3.0/expression_lab/bootstrap.py +36 -0
- expression_lab_classroom-1.3.0/expression_lab/cameras.py +172 -0
- expression_lab_classroom-1.3.0/expression_lab/core.py +259 -0
- expression_lab_classroom-1.3.0/expression_lab/download_models.py +132 -0
- expression_lab_classroom-1.3.0/expression_lab/resources.py +40 -0
- expression_lab_classroom-1.3.0/expression_lab/ui.py +528 -0
- expression_lab_classroom-1.3.0/expression_lab/worker.py +206 -0
- expression_lab_classroom-1.3.0/licenses/LibreFace-USC.txt +14 -0
- expression_lab_classroom-1.3.0/licenses/MediaPipe-Apache-2.0.txt +218 -0
- expression_lab_classroom-1.3.0/licenses/README.md +10 -0
- expression_lab_classroom-1.3.0/models/manifest.json +37 -0
- expression_lab_classroom-1.3.0/pyproject.toml +51 -0
- expression_lab_classroom-1.3.0/validation/images/01_neutral.png +0 -0
- expression_lab_classroom-1.3.0/validation/images/provenance.json +23 -0
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: expression-lab-classroom
|
|
3
|
+
Version: 1.3.0
|
|
4
|
+
Summary: 离线 CPU 人脸点位、AU 与相对位移教学实验室
|
|
5
|
+
License-File: licenses/LibreFace-USC.txt
|
|
6
|
+
License-File: licenses/MediaPipe-Apache-2.0.txt
|
|
7
|
+
Requires-Python: <3.13,>=3.12
|
|
8
|
+
Requires-Dist: jax==0.7.1
|
|
9
|
+
Requires-Dist: jaxlib==0.7.1
|
|
10
|
+
Requires-Dist: mediapipe==0.10.21
|
|
11
|
+
Requires-Dist: numpy==1.26.4
|
|
12
|
+
Requires-Dist: onnxruntime==1.23.2
|
|
13
|
+
Requires-Dist: opencv-contrib-python==4.11.0.86
|
|
14
|
+
Requires-Dist: pyside6-essentials==6.8.3
|
|
15
|
+
Description-Content-Type: text/markdown
|
|
16
|
+
|
|
17
|
+
# 人脸表情实验室 · Expression Lab Classroom
|
|
18
|
+
|
|
19
|
+
面向高中课堂的 Python 3.12 人脸点位、AU 动作特征与表情规则可视化工具。
|
|
20
|
+
|
|
21
|
+
## 安装
|
|
22
|
+
|
|
23
|
+
Windows 10/11 x64(Python 3.12):
|
|
24
|
+
|
|
25
|
+
```shell
|
|
26
|
+
py -3.12 -m pip install expression-lab-classroom
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
中国大陆可使用国内 PyPI 镜像,镜像同步新版本可能有延迟:
|
|
30
|
+
|
|
31
|
+
```shell
|
|
32
|
+
py -3.12 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple expression-lab-classroom
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Mac Apple Silicon、macOS 14+ 使用 `python3.12 -m pip` 安装;建议使用虚拟环境。本项目wheel不含平台二进制,底层依赖由pip按操作系统选择。目前支持范围为上述两个平台。
|
|
36
|
+
|
|
37
|
+
## 学生代码(9行)
|
|
38
|
+
|
|
39
|
+
```python
|
|
40
|
+
from expression_lab import FaceCamera
|
|
41
|
+
camera = FaceCamera()
|
|
42
|
+
for f in camera:
|
|
43
|
+
expression = "自然"
|
|
44
|
+
if f[1] >= 7 and f[2] >= 5:
|
|
45
|
+
expression = "微笑"
|
|
46
|
+
elif f[3] >= 6:
|
|
47
|
+
expression = "张嘴"
|
|
48
|
+
camera.show(expression)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
保存为 `学生主程序.py`,执行 `py -3.12 学生主程序.py` 自动打开原生桌面窗口。若Windows的 `py -3` 默认指向Python3.12,也可用 `py -3 学生主程序.py`。Mac使用 `python3.12 学生主程序.py`。不需要浏览器。
|
|
52
|
+
|
|
53
|
+
启动后点击“采集基准”,保持自然表情约1–2秒,再做动作。点击AU行可切换代表点与曲线。`f[1]`、`f[2]`、`f[3]`分别读取AU12、AU6、AU26各自的f1;索引不是特征名称,每个AU的特征统一称为f1。完整索引映射:1→AU12、2→6、3→26、4→4、5→1、6→2、7→5、8→9、9→15、10→17、11→20、12→25。
|
|
54
|
+
|
|
55
|
+
## 模型从国内BOS下载,之后离线运行
|
|
56
|
+
|
|
57
|
+
代码包不包含模型权重。首次运行缺少模型时,在后台从百度BOS下载约47MB模型,界面显示进度。每个文件使用安装包内固定的SHA256核验,完成后原子写入本机用户缓存。已有完整缓存时不访问网络。失败后重新打开程序,只下载缺少或损坏的文件。
|
|
58
|
+
|
|
59
|
+
缓存目录:Windows `%LOCALAPPDATA%/expression-lab/models/libreface-v1`;Mac `~/Library/Caches/expression-lab/models/libreface-v1`。教师可设置 `EXPRESSION_LAB_CACHE_DIR` 指定缓存根目录,底下仍保留 `models/libreface-v1`。BOS访问不需要学生的账户或密钥。
|
|
60
|
+
|
|
61
|
+
教师可在课前联网执行以下命令,再断网使用:
|
|
62
|
+
|
|
63
|
+
```shell
|
|
64
|
+
py -3.12 -m expression_lab.download_models
|
|
65
|
+
py -3.12 -m expression_lab.download_models --check
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
`--check`只检查本地文件和SHA256,不联网。也可把已经准备好的 `models/libreface-v1` 目录复制到离线电脑的缓存根目录。显式 `FaceCamera(model_dir="本地目录")` 或 `--model-dir` 始终只使用指定目录,不自动下载。完整离线ZIP保持原有本地模型读取方式。
|
|
69
|
+
|
|
70
|
+
无摄像头可执行 `py -3.12 -m expression_lab --sample`,使用随包的虚构成年人人脸样例。
|
|
71
|
+
|
|
72
|
+
## 特征含义与模型
|
|
73
|
+
|
|
74
|
+
MediaPipe离线提供478个人脸点;LibreFace ONNX模型离线提供12个AU参考输出。默认f1采用人脸几何位移估计:按照Lien1998对齐尺度,计算相对自然基准的方向约束位移d,用 `η=d/√(d²+k²)` 换算为0–20整数。AU6、17为课堂代理参数,AU25、26共用唇间距代理;数值不是FACS官方人工A–E评级,也不表示对内心情绪的判断。
|
|
75
|
+
|
|
76
|
+
参考:[Lien1998论文](https://www.cs.cmu.edu/~jjlien/Thesis/RiTr9831.pdf)、[LibreFace](https://github.com/ihp-lab/LibreFace)、[MediaPipe](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker)。
|
|
77
|
+
|
|
78
|
+
## 许可证
|
|
79
|
+
|
|
80
|
+
随包保留各上游许可证。LibreFace模型及相关实现的USC授权允许教育、研究和非营利用途;商业用途需要另行取得许可。模型来源和SHA256保留在包内清单中。
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# 人脸表情实验室 · Expression Lab Classroom
|
|
2
|
+
|
|
3
|
+
面向高中课堂的 Python 3.12 人脸点位、AU 动作特征与表情规则可视化工具。
|
|
4
|
+
|
|
5
|
+
## 安装
|
|
6
|
+
|
|
7
|
+
Windows 10/11 x64(Python 3.12):
|
|
8
|
+
|
|
9
|
+
```shell
|
|
10
|
+
py -3.12 -m pip install expression-lab-classroom
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
中国大陆可使用国内 PyPI 镜像,镜像同步新版本可能有延迟:
|
|
14
|
+
|
|
15
|
+
```shell
|
|
16
|
+
py -3.12 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple expression-lab-classroom
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Mac Apple Silicon、macOS 14+ 使用 `python3.12 -m pip` 安装;建议使用虚拟环境。本项目wheel不含平台二进制,底层依赖由pip按操作系统选择。目前支持范围为上述两个平台。
|
|
20
|
+
|
|
21
|
+
## 学生代码(9行)
|
|
22
|
+
|
|
23
|
+
```python
|
|
24
|
+
from expression_lab import FaceCamera
|
|
25
|
+
camera = FaceCamera()
|
|
26
|
+
for f in camera:
|
|
27
|
+
expression = "自然"
|
|
28
|
+
if f[1] >= 7 and f[2] >= 5:
|
|
29
|
+
expression = "微笑"
|
|
30
|
+
elif f[3] >= 6:
|
|
31
|
+
expression = "张嘴"
|
|
32
|
+
camera.show(expression)
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
保存为 `学生主程序.py`,执行 `py -3.12 学生主程序.py` 自动打开原生桌面窗口。若Windows的 `py -3` 默认指向Python3.12,也可用 `py -3 学生主程序.py`。Mac使用 `python3.12 学生主程序.py`。不需要浏览器。
|
|
36
|
+
|
|
37
|
+
启动后点击“采集基准”,保持自然表情约1–2秒,再做动作。点击AU行可切换代表点与曲线。`f[1]`、`f[2]`、`f[3]`分别读取AU12、AU6、AU26各自的f1;索引不是特征名称,每个AU的特征统一称为f1。完整索引映射:1→AU12、2→6、3→26、4→4、5→1、6→2、7→5、8→9、9→15、10→17、11→20、12→25。
|
|
38
|
+
|
|
39
|
+
## 模型从国内BOS下载,之后离线运行
|
|
40
|
+
|
|
41
|
+
代码包不包含模型权重。首次运行缺少模型时,在后台从百度BOS下载约47MB模型,界面显示进度。每个文件使用安装包内固定的SHA256核验,完成后原子写入本机用户缓存。已有完整缓存时不访问网络。失败后重新打开程序,只下载缺少或损坏的文件。
|
|
42
|
+
|
|
43
|
+
缓存目录:Windows `%LOCALAPPDATA%/expression-lab/models/libreface-v1`;Mac `~/Library/Caches/expression-lab/models/libreface-v1`。教师可设置 `EXPRESSION_LAB_CACHE_DIR` 指定缓存根目录,底下仍保留 `models/libreface-v1`。BOS访问不需要学生的账户或密钥。
|
|
44
|
+
|
|
45
|
+
教师可在课前联网执行以下命令,再断网使用:
|
|
46
|
+
|
|
47
|
+
```shell
|
|
48
|
+
py -3.12 -m expression_lab.download_models
|
|
49
|
+
py -3.12 -m expression_lab.download_models --check
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
`--check`只检查本地文件和SHA256,不联网。也可把已经准备好的 `models/libreface-v1` 目录复制到离线电脑的缓存根目录。显式 `FaceCamera(model_dir="本地目录")` 或 `--model-dir` 始终只使用指定目录,不自动下载。完整离线ZIP保持原有本地模型读取方式。
|
|
53
|
+
|
|
54
|
+
无摄像头可执行 `py -3.12 -m expression_lab --sample`,使用随包的虚构成年人人脸样例。
|
|
55
|
+
|
|
56
|
+
## 特征含义与模型
|
|
57
|
+
|
|
58
|
+
MediaPipe离线提供478个人脸点;LibreFace ONNX模型离线提供12个AU参考输出。默认f1采用人脸几何位移估计:按照Lien1998对齐尺度,计算相对自然基准的方向约束位移d,用 `η=d/√(d²+k²)` 换算为0–20整数。AU6、17为课堂代理参数,AU25、26共用唇间距代理;数值不是FACS官方人工A–E评级,也不表示对内心情绪的判断。
|
|
59
|
+
|
|
60
|
+
参考:[Lien1998论文](https://www.cs.cmu.edu/~jjlien/Thesis/RiTr9831.pdf)、[LibreFace](https://github.com/ihp-lab/LibreFace)、[MediaPipe](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker)。
|
|
61
|
+
|
|
62
|
+
## 许可证
|
|
63
|
+
|
|
64
|
+
随包保留各上游许可证。LibreFace模型及相关实现的USC授权允许教育、研究和非营利用途;商业用途需要另行取得许可。模型来源和SHA256保留在包内清单中。
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
"""高中人脸表情实验室:本地 CPU 模型与桌面可视化。"""
|
|
2
|
+
|
|
3
|
+
__version__ = "1.3.0"
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
def __getattr__(name):
|
|
7
|
+
# 几何计算和模型自检不需要初始化 Qt 或打开摄像头。
|
|
8
|
+
if name == "FaceCamera":
|
|
9
|
+
from .bootstrap import ensure_runtime
|
|
10
|
+
ensure_runtime()
|
|
11
|
+
from .api import FaceCamera
|
|
12
|
+
return FaceCamera
|
|
13
|
+
raise AttributeError(name)
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
from . import FaceCamera
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
def main():
|
|
5
|
+
camera = FaceCamera()
|
|
6
|
+
camera.window.result.setToolTip('AU12 的 f1 ≥ 7 且 AU6 的 f1 ≥ 5 → 微笑;AU26 的 f1 ≥ 6 → 张嘴;其余 → 自然')
|
|
7
|
+
for f in camera:
|
|
8
|
+
expression = '自然'
|
|
9
|
+
if f[1] >= 7 and f[2] >= 5:
|
|
10
|
+
expression = '微笑'
|
|
11
|
+
elif f[3] >= 6:
|
|
12
|
+
expression = '张嘴'
|
|
13
|
+
camera.show(expression)
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
if __name__ == '__main__':
|
|
17
|
+
main()
|
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
"""学生使用的极简接口;迭代器内部处理事件、模型和资源释放。"""
|
|
2
|
+
|
|
3
|
+
import argparse
|
|
4
|
+
import ast
|
|
5
|
+
import queue
|
|
6
|
+
import sys
|
|
7
|
+
import time
|
|
8
|
+
from pathlib import Path
|
|
9
|
+
|
|
10
|
+
from PySide6.QtCore import QEventLoop, QTimer
|
|
11
|
+
from PySide6.QtGui import QFont
|
|
12
|
+
from PySide6.QtWidgets import QApplication
|
|
13
|
+
|
|
14
|
+
from .cameras import CameraDevice, CameraMonitor
|
|
15
|
+
from .core import feature_values
|
|
16
|
+
from .resources import sample_image
|
|
17
|
+
from .ui import LabWindow
|
|
18
|
+
from .worker import InferenceWorker
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
def rule_description(path):
|
|
22
|
+
try:
|
|
23
|
+
tree = ast.parse(Path(path).read_text(encoding='utf-8-sig'))
|
|
24
|
+
descriptions = []
|
|
25
|
+
for node in ast.walk(tree):
|
|
26
|
+
if not isinstance(node, ast.If):
|
|
27
|
+
continue
|
|
28
|
+
for statement in node.body:
|
|
29
|
+
if (isinstance(statement, ast.Assign) and isinstance(statement.value, ast.Constant)
|
|
30
|
+
and any(isinstance(target, ast.Name) and target.id == 'expression'
|
|
31
|
+
for target in statement.targets)):
|
|
32
|
+
condition = ast.unparse(node.test).replace(' and ', ' 且 ').replace(' or ', ' 或 ')
|
|
33
|
+
descriptions.append(f'{condition} → {statement.value.value}')
|
|
34
|
+
if descriptions:
|
|
35
|
+
return ';'.join(descriptions) + ';其余 → 默认值'
|
|
36
|
+
return '按学生主程序中的条件判断'
|
|
37
|
+
except (OSError, SyntaxError):
|
|
38
|
+
return '按学生主程序中的条件判断'
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def parse_options(args):
|
|
42
|
+
parser = argparse.ArgumentParser(description='离线人脸 AU 表情实验室(Python 3.12)')
|
|
43
|
+
source = parser.add_mutually_exclusive_group()
|
|
44
|
+
source.add_argument('--camera', type=int, default=0, help='摄像头序号,默认 0')
|
|
45
|
+
source.add_argument('--image', '--video', dest='source', help='改用本地图片或循环视频')
|
|
46
|
+
source.add_argument('--sample', action='store_true', help='使用随包离线人脸图片')
|
|
47
|
+
parser.add_argument('--model-dir', type=Path, help='本地模型目录')
|
|
48
|
+
parser.add_argument('--fps', type=int, default=15, choices=range(1, 31), metavar='1–30', help='推理帧率上限,默认 15')
|
|
49
|
+
return parser.parse_args(args)
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
class FaceCamera:
|
|
53
|
+
def __init__(self, camera=0, *, source=None, model_dir=None, fps=15, args=None, values='features'):
|
|
54
|
+
if values not in ('features', 'au'):
|
|
55
|
+
raise ValueError('values 应为 features(0–20整数分)或 au(原始0–1强度)')
|
|
56
|
+
self._values = values
|
|
57
|
+
options = parse_options(sys.argv[1:] if args is None else args)
|
|
58
|
+
if source is None:
|
|
59
|
+
source = sample_image() if options.sample else options.source
|
|
60
|
+
if source is None:
|
|
61
|
+
source = options.camera if options.camera != 0 else camera
|
|
62
|
+
self._model_dir = model_dir if model_dir is not None else options.model_dir
|
|
63
|
+
self._fps = options.fps if options.fps != 15 else fps
|
|
64
|
+
self._app = QApplication.instance() or QApplication([])
|
|
65
|
+
self._app.setFont(QFont('Microsoft YaHei' if sys.platform == 'win32' else 'PingFang SC', 10))
|
|
66
|
+
self._app.setQuitOnLastWindowClosed(False)
|
|
67
|
+
self._loop = QEventLoop()
|
|
68
|
+
self._timer = QTimer()
|
|
69
|
+
self._timer.setInterval(16)
|
|
70
|
+
self._timer.timeout.connect(self._loop.quit)
|
|
71
|
+
self._timer.timeout.connect(self._poll_cameras)
|
|
72
|
+
self.window = LabWindow()
|
|
73
|
+
self.window.result.setToolTip(rule_description(Path(sys.argv[0])))
|
|
74
|
+
self._closed = False
|
|
75
|
+
self.current = None
|
|
76
|
+
self.worker = None
|
|
77
|
+
self._retired_workers = []
|
|
78
|
+
self._camera_monitor = CameraMonitor()
|
|
79
|
+
self._camera_monitor.start()
|
|
80
|
+
self.window.closed.connect(self.close)
|
|
81
|
+
self.window.calibrate_requested.connect(lambda: self.worker.commands.put('calibrate'))
|
|
82
|
+
self.window.reset_requested.connect(lambda: self.worker.commands.put('reset'))
|
|
83
|
+
self.window.pause_changed.connect(self._pause)
|
|
84
|
+
self.window.source_requested.connect(self._start_worker)
|
|
85
|
+
self.window.show()
|
|
86
|
+
self._start_worker(source)
|
|
87
|
+
self._timer.start()
|
|
88
|
+
|
|
89
|
+
@property
|
|
90
|
+
def is_closed(self):
|
|
91
|
+
return self._closed
|
|
92
|
+
|
|
93
|
+
def _pause(self, paused):
|
|
94
|
+
if paused:
|
|
95
|
+
self.worker.paused.set()
|
|
96
|
+
else:
|
|
97
|
+
# 丢弃暂停前尚未显示的结果,继续后等新一帧。
|
|
98
|
+
try:
|
|
99
|
+
self.worker.results.get_nowait()
|
|
100
|
+
except queue.Empty:
|
|
101
|
+
pass
|
|
102
|
+
self.worker.paused.clear()
|
|
103
|
+
|
|
104
|
+
def _start_worker(self, source):
|
|
105
|
+
if self._closed:
|
|
106
|
+
return
|
|
107
|
+
if self.worker is not None:
|
|
108
|
+
self.worker.stop.set()
|
|
109
|
+
self._retired_workers.append(self.worker)
|
|
110
|
+
# 摄像头驱动可能迟迟不返回。主线程只发取消信号,绝不 join/read/release。
|
|
111
|
+
self._retired_workers = [worker for worker in self._retired_workers if worker.is_alive()]
|
|
112
|
+
self._last_result_at = time.monotonic()
|
|
113
|
+
self._has_result = False
|
|
114
|
+
self._stale_notified = False
|
|
115
|
+
self.current = None
|
|
116
|
+
self.window.last_observation = None
|
|
117
|
+
self.window.chart.history.clear()
|
|
118
|
+
self.window.pause_button.setChecked(False)
|
|
119
|
+
self.window.canvas.observation = None
|
|
120
|
+
self.window.canvas.placeholder = '正在加载本地 CPU 模型…'
|
|
121
|
+
self.window.canvas.mirror = isinstance(source, (int, CameraDevice))
|
|
122
|
+
self.window.set_source(source)
|
|
123
|
+
self.window.show_error('正在加载本地 CPU 模型…')
|
|
124
|
+
self.worker = InferenceWorker(source, model_dir=self._model_dir, fps=self._fps)
|
|
125
|
+
self.worker.start()
|
|
126
|
+
|
|
127
|
+
def _poll_cameras(self):
|
|
128
|
+
try:
|
|
129
|
+
devices, error = self._camera_monitor.results.get_nowait()
|
|
130
|
+
except queue.Empty:
|
|
131
|
+
return
|
|
132
|
+
self.window.set_cameras(devices, error)
|
|
133
|
+
|
|
134
|
+
def __iter__(self):
|
|
135
|
+
try:
|
|
136
|
+
while not self._closed:
|
|
137
|
+
observation = self._wait_for_face()
|
|
138
|
+
if observation is None:
|
|
139
|
+
return
|
|
140
|
+
self.current = observation
|
|
141
|
+
yield feature_values(observation.geometric_au) if self._values == 'features' else observation.au.copy()
|
|
142
|
+
finally:
|
|
143
|
+
self.close()
|
|
144
|
+
|
|
145
|
+
def _wait_for_face(self):
|
|
146
|
+
while not self._closed:
|
|
147
|
+
if not self.window.paused:
|
|
148
|
+
try:
|
|
149
|
+
result = self.worker.results.get_nowait()
|
|
150
|
+
except queue.Empty:
|
|
151
|
+
result = None
|
|
152
|
+
if isinstance(result, dict):
|
|
153
|
+
self._stale_notified = True
|
|
154
|
+
self.window.show_error(result.get('progress', result.get('error', '正在准备模型…')))
|
|
155
|
+
elif result is not None:
|
|
156
|
+
self._last_result_at = time.monotonic()
|
|
157
|
+
self._has_result = True
|
|
158
|
+
self._stale_notified = False
|
|
159
|
+
if result.points is not None and (self._values == 'au' or result.geometric_au):
|
|
160
|
+
return result
|
|
161
|
+
self.window.present(result, None)
|
|
162
|
+
elif not self._stale_notified:
|
|
163
|
+
timeout = 2 if self._has_result else 8
|
|
164
|
+
if time.monotonic() - self._last_result_at > timeout:
|
|
165
|
+
self.current = None
|
|
166
|
+
self.window.show_error('等待摄像头画面,请切换来源。')
|
|
167
|
+
self._stale_notified = True
|
|
168
|
+
# Qt 事件循环等待下一次定时器或关窗事件,不使用占满 CPU 的忙循环。
|
|
169
|
+
self._loop.exec()
|
|
170
|
+
return None
|
|
171
|
+
|
|
172
|
+
def show(self, expression):
|
|
173
|
+
if not self._closed and self.current is not None:
|
|
174
|
+
self.window.present(self.current, str(expression))
|
|
175
|
+
|
|
176
|
+
def close(self):
|
|
177
|
+
if self._closed:
|
|
178
|
+
return
|
|
179
|
+
self._closed = True
|
|
180
|
+
self._timer.stop()
|
|
181
|
+
self._camera_monitor.stop.set()
|
|
182
|
+
self._loop.quit()
|
|
183
|
+
for worker in [self.worker, *self._retired_workers]:
|
|
184
|
+
if worker is not None:
|
|
185
|
+
worker.stop.set()
|
|
186
|
+
self.window.hide()
|
|
187
|
+
|
|
188
|
+
def __enter__(self):
|
|
189
|
+
return self
|
|
190
|
+
|
|
191
|
+
def __exit__(self, *_):
|
|
192
|
+
self.close()
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
"""本地 MediaPipe + LibreFace ONNX CPU 推理;初始化时准备模型缓存。"""
|
|
2
|
+
|
|
3
|
+
import hashlib
|
|
4
|
+
import json
|
|
5
|
+
import time
|
|
6
|
+
from dataclasses import dataclass, field
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
|
|
9
|
+
import cv2
|
|
10
|
+
import numpy as np
|
|
11
|
+
|
|
12
|
+
from .core import decode_intensities, face_coordinates
|
|
13
|
+
from .download_models import prepare_models
|
|
14
|
+
from .resources import model_directory
|
|
15
|
+
|
|
16
|
+
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
|
17
|
+
REQUIRED_MODELS = ('face_landmarker.task', 'LibreFace_AU_Encoder.onnx', 'LibreFace_AU_Intensity.onnx')
|
|
18
|
+
EYE_LEFT = (33, 7, 163, 144, 145, 153, 154, 155, 133, 246, 161, 160, 159, 158, 157, 173)
|
|
19
|
+
EYE_RIGHT = (263, 249, 390, 373, 374, 380, 381, 382, 362, 466, 388, 387, 386, 385, 384, 398)
|
|
20
|
+
MOUTH = (61, 146, 91, 181, 84, 17, 314, 405, 321, 375, 291, 185, 40, 39, 37, 0, 267,
|
|
21
|
+
269, 270, 409, 78, 95, 88, 178, 87, 14, 317, 402, 318, 324, 308, 191, 80, 81,
|
|
22
|
+
82, 13, 312, 311, 310, 415)
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
class ModelFiles:
|
|
26
|
+
def __init__(self, directory=None):
|
|
27
|
+
self.directory = Path(directory) if directory is not None else model_directory()
|
|
28
|
+
|
|
29
|
+
def __getitem__(self, name):
|
|
30
|
+
return self.directory / name
|
|
31
|
+
|
|
32
|
+
def validate(self, verify_hashes=False):
|
|
33
|
+
for name in REQUIRED_MODELS:
|
|
34
|
+
path = self[name]
|
|
35
|
+
if not path.is_file() or path.stat().st_size < 10_000:
|
|
36
|
+
raise FileNotFoundError(
|
|
37
|
+
f'缺少或损坏本地模型:{path}。pip 版请运行 expression-lab-models,'
|
|
38
|
+
'离线包请恢复 models 文件夹。')
|
|
39
|
+
if verify_hashes:
|
|
40
|
+
manifest = json.loads((self.directory / 'manifest.json').read_text(encoding='utf-8'))
|
|
41
|
+
for name in REQUIRED_MODELS:
|
|
42
|
+
if hashlib.sha256(self[name].read_bytes()).hexdigest() != manifest['files'][name]['sha256']:
|
|
43
|
+
raise ValueError(f'本地模型校验失败:{name}。请从完整离线包恢复。')
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
def aligned_face(frame_bgr, points):
|
|
47
|
+
"""LibreFace/FFHQ 的眼睛—嘴部四边形对齐,256 图中心裁成 224。
|
|
48
|
+
|
|
49
|
+
保持上游的轴与尺度;超出画面的部分用反射填充(不做上游的边缘模糊)。
|
|
50
|
+
"""
|
|
51
|
+
left = points[list(EYE_LEFT)].mean(axis=0)
|
|
52
|
+
right = points[list(EYE_RIGHT)].mean(axis=0)
|
|
53
|
+
eye_average = (left + right) / 2
|
|
54
|
+
eye_to_eye = right - left
|
|
55
|
+
lips = points[list(MOUTH)]
|
|
56
|
+
mouth_average = (lips[np.argmin(lips[:, 0])] + lips[np.argmax(lips[:, 0])]) / 2
|
|
57
|
+
eye_to_mouth = mouth_average - eye_average
|
|
58
|
+
x_axis = eye_to_eye - np.array([-eye_to_mouth[1], eye_to_mouth[0]])
|
|
59
|
+
length = np.linalg.norm(x_axis)
|
|
60
|
+
if length < 1e-6:
|
|
61
|
+
raise ValueError('人脸对齐失败,请正对摄像头')
|
|
62
|
+
x_axis = x_axis / length * max(np.linalg.norm(eye_to_eye) * 2, np.linalg.norm(eye_to_mouth) * 1.8)
|
|
63
|
+
y_axis = np.array([-x_axis[1], x_axis[0]])
|
|
64
|
+
center = eye_average + eye_to_mouth * 0.1
|
|
65
|
+
source = np.float32([center - x_axis - y_axis, center - x_axis + y_axis, center + x_axis - y_axis])
|
|
66
|
+
destination = np.float32([[0, 0], [0, 255], [255, 0]])
|
|
67
|
+
transform = cv2.getAffineTransform(source, destination)
|
|
68
|
+
aligned = cv2.warpAffine(frame_bgr, transform, (256, 256), flags=cv2.INTER_LINEAR,
|
|
69
|
+
borderMode=cv2.BORDER_REFLECT_101)
|
|
70
|
+
return np.ascontiguousarray(aligned[16:240, 16:240])
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
def image_tensor(aligned_bgr):
|
|
74
|
+
rgb = cv2.cvtColor(aligned_bgr, cv2.COLOR_BGR2RGB).astype(np.float32) / 255
|
|
75
|
+
normalized = (rgb - np.array([0.485, 0.456, 0.406], np.float32)) / np.array(
|
|
76
|
+
[0.229, 0.224, 0.225], np.float32)
|
|
77
|
+
return np.ascontiguousarray(normalized.transpose(2, 0, 1)[None])
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
@dataclass
|
|
81
|
+
class Observation:
|
|
82
|
+
frame: np.ndarray
|
|
83
|
+
timestamp: float
|
|
84
|
+
points: np.ndarray | None = None
|
|
85
|
+
coordinates: np.ndarray | None = None
|
|
86
|
+
au: dict = field(default_factory=dict)
|
|
87
|
+
inference_ms: float = 0
|
|
88
|
+
status: str = '请正对摄像头'
|
|
89
|
+
baseline: np.ndarray | None = None
|
|
90
|
+
motion: dict = field(default_factory=dict)
|
|
91
|
+
geometric_au: dict = field(default_factory=dict)
|
|
92
|
+
geometry: dict = field(default_factory=dict)
|
|
93
|
+
standard_baseline: np.ndarray | None = None
|
|
94
|
+
calibration_count: int = 0
|
|
95
|
+
collecting: bool = False
|
|
96
|
+
fps: float = 0
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
class FaceEngine:
|
|
100
|
+
def __init__(self, model_dir=None, threads=2):
|
|
101
|
+
# 先准备本地资源;pip 版首次缺少缓存时由 BOS 下载,显式目录始终只读本地。
|
|
102
|
+
files = ModelFiles(prepare_models() if model_dir is None else model_dir)
|
|
103
|
+
files.validate(verify_hashes=True)
|
|
104
|
+
import mediapipe as mp
|
|
105
|
+
import onnxruntime as ort
|
|
106
|
+
|
|
107
|
+
self._mp = mp
|
|
108
|
+
self._last_timestamp = -1
|
|
109
|
+
options = ort.SessionOptions()
|
|
110
|
+
options.intra_op_num_threads = threads
|
|
111
|
+
options.inter_op_num_threads = 1
|
|
112
|
+
options.add_session_config_entry('session.intra_op.allow_spinning', '0')
|
|
113
|
+
self._encoder = ort.InferenceSession(str(files['LibreFace_AU_Encoder.onnx']), options,
|
|
114
|
+
providers=['CPUExecutionProvider'])
|
|
115
|
+
self._intensity = ort.InferenceSession(str(files['LibreFace_AU_Intensity.onnx']), options,
|
|
116
|
+
providers=['CPUExecutionProvider'])
|
|
117
|
+
self._encoder_input = self._encoder.get_inputs()[0].name
|
|
118
|
+
self._intensity_input = self._intensity.get_inputs()[0].name
|
|
119
|
+
landmarker_options = mp.tasks.vision.FaceLandmarkerOptions(
|
|
120
|
+
# 从本地读成 bytes,也支持 Windows 中文目录,避开原生文件名编码差异。
|
|
121
|
+
base_options=mp.tasks.BaseOptions(model_asset_buffer=files['face_landmarker.task'].read_bytes(),
|
|
122
|
+
delegate=mp.tasks.BaseOptions.Delegate.CPU),
|
|
123
|
+
running_mode=mp.tasks.vision.RunningMode.VIDEO,
|
|
124
|
+
num_faces=1,
|
|
125
|
+
min_face_detection_confidence=0.5,
|
|
126
|
+
min_face_presence_confidence=0.5,
|
|
127
|
+
min_tracking_confidence=0.5,
|
|
128
|
+
)
|
|
129
|
+
self._landmarker = mp.tasks.vision.FaceLandmarker.create_from_options(landmarker_options)
|
|
130
|
+
|
|
131
|
+
def process(self, frame_bgr, timestamp=None):
|
|
132
|
+
if frame_bgr is None or frame_bgr.ndim != 3 or frame_bgr.shape[2] != 3:
|
|
133
|
+
raise ValueError('需要有效的 BGR 摄像头画面')
|
|
134
|
+
started = time.perf_counter()
|
|
135
|
+
now = time.monotonic() if timestamp is None else timestamp
|
|
136
|
+
timestamp_ms = max(self._last_timestamp + 1, int(now * 1000))
|
|
137
|
+
self._last_timestamp = timestamp_ms
|
|
138
|
+
rgb = np.ascontiguousarray(cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB))
|
|
139
|
+
result = self._landmarker.detect_for_video(
|
|
140
|
+
self._mp.Image(image_format=self._mp.ImageFormat.SRGB, data=rgb), timestamp_ms)
|
|
141
|
+
observation = Observation(frame=frame_bgr, timestamp=now)
|
|
142
|
+
if result.face_landmarks:
|
|
143
|
+
h, w = frame_bgr.shape[:2]
|
|
144
|
+
points = np.array([(point.x * w, point.y * h) for point in result.face_landmarks[0]])
|
|
145
|
+
tensor = image_tensor(aligned_face(frame_bgr, points))
|
|
146
|
+
features = self._encoder.run(None, {self._encoder_input: tensor})[0].reshape(1, -1)
|
|
147
|
+
output = self._intensity.run(None, {self._intensity_input: features})[0]
|
|
148
|
+
observation.points = points
|
|
149
|
+
observation.coordinates = face_coordinates(points)
|
|
150
|
+
observation.au = decode_intensities(output)
|
|
151
|
+
observation.status = '已识别人脸'
|
|
152
|
+
else:
|
|
153
|
+
observation.status = '未检测到人脸 · 请正对摄像头'
|
|
154
|
+
observation.inference_ms = (time.perf_counter() - started) * 1000
|
|
155
|
+
return observation
|
|
156
|
+
|
|
157
|
+
def close(self):
|
|
158
|
+
if self._landmarker is not None:
|
|
159
|
+
self._landmarker.close()
|
|
160
|
+
self._landmarker = None
|
|
161
|
+
|
|
162
|
+
def __enter__(self):
|
|
163
|
+
return self
|
|
164
|
+
|
|
165
|
+
def __exit__(self, *_):
|
|
166
|
+
self.close()
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
"""只使用标准库,让 py -3 自动转入随项目准备的 Python 3.12 环境。"""
|
|
2
|
+
|
|
3
|
+
import importlib.util
|
|
4
|
+
import subprocess
|
|
5
|
+
import sys
|
|
6
|
+
from pathlib import Path
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
def ensure_runtime(root=None):
|
|
10
|
+
root = Path(root) if root is not None else Path(__file__).resolve().parents[1]
|
|
11
|
+
environments = (
|
|
12
|
+
(root / '.venv', 'Scripts/python.exe' if sys.platform == 'win32' else 'bin/python'),
|
|
13
|
+
(root / 'runtime/macos_arm64', 'bin/python3.12'),
|
|
14
|
+
)
|
|
15
|
+
for environment, relative_executable in environments:
|
|
16
|
+
if Path(sys.prefix).resolve() == environment.resolve():
|
|
17
|
+
return
|
|
18
|
+
executable = environment / relative_executable
|
|
19
|
+
if executable.is_file():
|
|
20
|
+
arguments = sys.argv
|
|
21
|
+
specification = getattr(sys.modules.get('__main__'), '__spec__', None)
|
|
22
|
+
if specification is not None:
|
|
23
|
+
arguments = ['-m', specification.name, *sys.argv[1:]]
|
|
24
|
+
try:
|
|
25
|
+
status = subprocess.call([str(executable), *arguments])
|
|
26
|
+
except KeyboardInterrupt:
|
|
27
|
+
status = 130
|
|
28
|
+
raise SystemExit(status)
|
|
29
|
+
dependencies = ('numpy', 'cv2', 'mediapipe', 'onnxruntime', 'PySide6')
|
|
30
|
+
if sys.version_info[:2] == (3, 12) and all(importlib.util.find_spec(name) for name in dependencies):
|
|
31
|
+
return
|
|
32
|
+
raise SystemExit(
|
|
33
|
+
'请先双击 安装依赖_Windows.cmd,然后执行 py -3 学生主程序.py。\n'
|
|
34
|
+
'pip 安装版:请在 Python 3.12 中用 python -m pip install 安装完整 wheel 及依赖。\n'
|
|
35
|
+
'Mac 请解压完整的 Apple 芯片离线包,双击 启动实验_macOS.command。'
|
|
36
|
+
)
|