feat(pybot×actpred): R2 姿态反馈方案一——MPU6050 独立 IMU token 全链路
- pybot:recorder 逐帧落盘 imu.npy(T×8,roll,pitch,ax,ay,az,gx,gy,gz,与帧/动作同调用对齐);run_loop 对声明 imu_dim 的 predictor 以 predict(..., imu=) 调用(鸭子类型,旧两参调用向后兼容) - actpred:新增 --imu-dim;model 序列 [img×F, act×H, (IMU), PRED](imu_dim>0 时 F+H+2,IMU token 位于 PRED 前);dataset/train 读 imu.npy(缺失按零向量兜底);predict/ONNX/Hailo 三后端与 export/gen_calibration/dfc_compile/_npu_test 部署链适配 imu 输入 - 姿态按物理量程固定缩放归一化(imu_norm.py,训练/推理共用,零状态) - 测试:actpred 128 passed / pybot loop+recorder 28 passed
This commit is contained in:
1 parent
3e773debc9
commit
a2ba797410
22 files changed
+1236
-133
No files matched your search
@@ -137,14 +137,24 @@ def build_deploy_set(args):
|
||||
size=jitter.shape).astype(np.float32)
|
||||
pred = np.broadcast_to(v, (args.samples, 1, 1, d)).astype(np.float32) + jitter
|
||||
|
||||
inputs = [*frames, history, pred]
|
||||
if cfg.imu_dim > 0:
|
||||
# R2 方案一:imu 输入为归一化姿态(roll,pitch/180,ax..gz/32768)。
|
||||
# 校准覆盖典型工作范围:归一化值域 ±0.5(≈±90°/±16384)均匀采样
|
||||
imu = rng.uniform(-0.5, 0.5,
|
||||
size=(args.samples, 1, 1, cfg.imu_dim)).astype(np.float32)
|
||||
inputs.append(imu)
|
||||
|
||||
np.savez_compressed(
|
||||
args.output,
|
||||
**{f"actpred_deploy/input_layer{i + 1}": arr
|
||||
for i, arr in enumerate([*frames, history, pred])})
|
||||
for i, arr in enumerate(inputs)})
|
||||
print(f"部署校准集已保存: {args.output}")
|
||||
print(f" frame0~2: {frames[0].shape}(分辨率 {w}x{h},与 checkpoint 一致)")
|
||||
print(f" action_history: {history.shape} pred_token: {pred.shape}"
|
||||
f"(学习值 ±{args.pred_jitter:.0%} 抖动)")
|
||||
if cfg.imu_dim > 0:
|
||||
print(f" imu: {inputs[-1].shape}(归一化值域 ±0.5 均匀采样)")
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
Reference in new issue
Block a user