Our team is developing an AR application using the hand mesh data from XR8.HandController. We have referred to the following two examples from your website:
The XR8.HandController.pipelineModule provides extensive data related to hands and fingers; however, I have noticed a significant discrepancy between this data and actual hands. This can be easily observed in your samples, and I have also extracted some additional images from the preview.
Using this data, we cannot create an AR application that ensures realistic hand representation, such as adding rings or watches to the hand as in your examples.
I understand that creating hand mesh data identical to actual hands is impossible, but we still require higher data accuracy compared to the samples you provide. If you have any solutions or advice, please let me know.
Thank you for reaching out to us with your concerns regarding the hand mesh data accuracy from the XR8.HandController.pipelineModule. We appreciate your detailed feedback and the examples you’ve referred to from our website.
We understand that achieving a realistic representation of hands is crucial for your AR application, especially for adding elements like rings or watches. While it is challenging to create hand mesh data identical to actual hands, there are several steps you can take to improve the accuracy of the tracking data:
Ensure Proper Wrist Visibility: Make sure the wrist is within the camera’s field of view. Proper wrist visibility significantly enhances the accuracy of hand tracking.
Avoid Side Views: Side views, such as a “karate chop” position, are less effective for tracking. For optimal results, position the hand in a way that the palm or back of the hand is more visible to the camera.
Test on Mobile Devices: The tracking system tends to perform better on mobile devices compared to desktop cameras due to better camera angles and hardware capabilities. If possible, test and use the application on smartphones for improved accuracy.
Thank you for your prompt response. We have implemented your advice to improve our system in tracking hand data.
Additionally, we have conducted several experiments to enhance the quality of the hand mesh data. However, we are encountering some performance issues during the hand data tracking process as follows:
We notice a delay between the real hand and the hand mesh data because 8thwall needs time to convert the camera image into hand mesh data. However, we would like this process to be faster. Please let us know if you have any solutions (e.g., reducing FPS, lowering resolution…).
We are also using the Mediapipe library to obtain finger information and combine it with the hand mesh data from 8thwall to reduce discrepancies with the real hand. Using both libraries simultaneously significantly slows down the processing. If you have any integration solutions that do not heavily impact system performance, please let us know.
