Problem of WildHandsDataset Intrinsics

[Boey-li]

Hello, thanks for the great work!

I have one issue on the intrinsics input for the WildHandsDataset. I have collected data using the realsense RGB-D camera and input the camera intrinsics to the dataset as self.K. Then, I initialize intrx = self.K.copy() and follow previous setting to get aug intrix:

intrx = self.K.copy()
intrx = data_utils.get_aug_intrix(
    intrx,
    fixed_focal_length,
    args.img_res,
    True,
    center_original[0],
    center_original[1],
    augm_dict["sc"] * scale_original,
)

I also input the camera focal length besides the intrinsics.

However, with the true camera parameters, the outputs seem deviate the hand position:

Could you please explain the potential reasons for it? And is there any setting we need to change with the true camera parameters? Thanks!

[ap229997]

I have seen something similar before, where the issue was due to incorrect transformation of camera intrinsic before and after cropping. Can you specify your intrinsic before and after the get_aug_intrix()?

[Boey-li]

Thanks for your quick reply!

Before cropping:

array([[     912.16,           0,       649.5],
       [          0,      912.42,      373.88],
       [          0,           0,           1]])

After get_aug_intrix():

array([[     243.24,           0,       173.2],
       [          0,      243.31,      99.702],
       [          0,           0,           1]])

[ap229997]

Sorry for the late reply. The demo code converts the input image to a square image (check this) before passing it to the dataloader where it is then resized to the input size required by the model. When the input image is converted to the square image, the intrinsic will change as well (check get_aug_intrix() on how to transform intrinsic for any desired image resolution). From the intrinsic you provided, it seems like the image from the realsense camera is not square, is this correct? In that case, the intrinsic need to be changed accordingly before passing to self.K.

[ap229997]

Check here.