data_juicer.ops.mapper.video_hand_motion_smooth_mapper module#

class data_juicer.ops.mapper.video_hand_motion_smooth_mapper.VideoHandMotionSmoothMapper(*args, **kwargs)[source]#

Bases: Mapper

Apply smoothing to world-space hand motions and remove outliers.

Reads hand action results (states, actions, joints_world) produced by VideoHandActionComputeMapper and applies:

  1. Extreme outlier replacement โ€” frames whose instantaneous wrist speed exceeds median + outlier_velocity_threshold * MAD are replaced by linear interpolation from neighbors (not deleted).

  2. Savitzky-Golay smoothing โ€” positions are smoothed with a Savitzky-Golay filter that preserves motion peaks while removing high-frequency jitter.

  3. Quaternion smoothing โ€” orientations are smoothed in quaternion space to avoid gimbal lock and discontinuities.

  4. Action recomputation โ€” 7-DoF actions are re-derived from the smoothed states so they stay consistent.

Reference (paper ยง3.1):

โ€œwe apply spline smoothing to the world-space hand motions and remove outliersโ€

__init__(hand_action_field: str = 'hand_action_tags', savgol_window: int = 11, savgol_polyorder: int = 3, outlier_velocity_threshold: float = 5.0, min_frames_for_smoothing: int = 5, smooth_joints: bool = True, *args, **kwargs)[source]#

Initialization method.

Parameters:

  • hand_action_field โ€“ Meta field storing hand action results (output of VideoHandActionComputeMapper).

  • savgol_window โ€“ Window length for Savitzky-Golay filter. Must be odd. Larger = smoother but may lose fast motions.

  • savgol_polyorder โ€“ Polynomial order for Savitzky-Golay filter. Must be less than savgol_window.

  • outlier_velocity_threshold โ€“ Frames whose wrist speed exceeds median + threshold * MAD are replaced by interpolation. Higher = more conservative (fewer replacements).

  • min_frames_for_smoothing โ€“ Minimum number of valid frames required to apply smoothing.

  • smooth_joints โ€“ Whether to also smooth joints_world (21-joint MANO skeleton in world space).

process_single(sample=None, rank=None)[source]#

For sample level, sample โ€“> sample

Parameters:

sample โ€“ sample to process

Returns:

processed sample