Is it possible to combine Sky Effects with Hand Tracking in a single 8th Wall scene?

🧑‍💻 Technical Support
1

Hi 8th Wall team,

I’m currently developing a WebAR gameplay where a 3D model is generated and flies in the sky using the Sky Effects template. Ideally, the user can tap the flying model and then grasp it with their hand, does it need to switching to Hand Tracking mode and attaching the model to the user’s palm?

Is there any officially supported way to implement this kind of gameplay — combining Sky Effects with Hand Tracking in a seamless flow?

Any suggestions or workflow from the 8th Wall team would be really appreciated!

Thanks.

2

Since you can only have Sky or Hand active at a time you’d have to restart the XR8 engine to switch the mode, you wouldn’t be able to do them both at the same time.

3

Thanks for your reply regarding Sky Effects and Hand Tracking mode conflict — that clarified a lot!

Now I’m considering switching to a Camera-Based solution instead. I found this Studio template really inspiring:

:link: Roaming Entities – uforodeo

I’d like to ask:
Can I activate Roaming Entities together with Hand Tracking mode at the same time?
In other words, is it technically possible to make a 3D model randomly float through the scene (as in the roaming_entities demo), and allow the user to grab it with their hand via XR hand tracking?

4

Hmm that’s a pretty neat idea. I’d probably do it by attaching some sort of physics collider to the hand and the objects. Keep in mind that Studio doesn’t support Hand Tracking yet so you’ll need to build this out in Editor.

Here’s a sample project that uses physics.

5

Thanks for your suggestion, I will try to use physics collider instead.

6

Dear Bulter,

I am also want to know about how to deploy this gameplay mechanism on Meta Quest 3.when I try to run the same project on a Meta Quest 3 browser (using passthrough WebXR), I encounter the following issues:

  1. Only part of the AR UI (e.g. overlay images, status text) appears on screen;
  2. The text displayed on screen is very small and blurry, almost unreadable;
  3. The overall UI layout feels off or misaligned compared to the mobile version.

Here’s a short video showing the gameplay on phone vs. Meta Quest 3:

Phone:

Quest:

Could you please advise:

  • How can I adapt my UI elementsto be readable on Quest 3?
  • Do I need to modify CSS / canvas size / device pixel ratio when running on headset?
  • Is there an official guideline or best practice for 8th Wall WebAR UI compatibility with Meta Quest 3?

Thanks so much for your help!

7

Meta Quest 3, and Headsets in general aren’t supported yet.

8

Thanks for your reply!
If I plan to develop a version of the app that works on Meta Quest 3 in the future, what would be the recommended approach? Should I switch to 8th Wall Studio version, I guess it may support Quest 3 better.
I’d love to know if there’s any roadmap or workaround for Quest 3 support. Thanks a lot!

9

Ah I realize now you’re talking about Editor this whole time. I should say Studio doesn’t support Headsets however Editor should support headsets. So Studio would not be the right approach but Editor would.

10

Thank you for your previous clarification about Editor supporting headsets.

I just tested all three of my custom WebAR gameplay modes using 8th Wall Editor on Meta Quest 3, and here are the results:

1. Gameplay 1 (Model with red dot interaction)

I can upload a photo to the backend, receive the MeshyAI-generated model, and display it in the AR view.

Using the Quest controller, I’m able to scale and rotate the model.

2. Gameplay 2 (Hand rotation to reveal)

The AR UI does not appear at all.

I can still upload a photo and receive the model from MeshyAI, but I’m unable to interact with it using either the controller or hand tracking. The model doesn’t respond.

3. Gameplay 3 (Flying model + aim-to-catch)

The AR UI can loads, and I can upload a photo and receive the model.

The model flies around the sky.

However, when I try to aim at the model, the “:bullseye: Targeting…” text is hard to appear, and the model does not stop flying even when it’s in the center of the view.

11

Hi George,

Thanks for your reply!

Just to confirm — if we want to switch from Sky Effects (using xrweb) to Hand Tracking (using xrhand), is it technically feasible to:

  1. Run the Sky mode first,
  2. Then restart the XR8 engine,
  3. And finally activate the Hand Tracking mode within the same page session?

Would the engine restart flow allow this kind of switch, or would it require a full page reload?

Really appreciate your insights!

12

You can switch it at runtime it doesn’t need a page refresh, the camera feed will cut out for a short time but you can cover it up. :slight_smile:

13

Dear Butler, thanks for your help !

After testing, I found that the system can only switch the camera three times: from world effect to face effect, then from face effect back to world effect, and finally from world effect to face effect again — at which point the screen turns black. The terminal shows a “Script Error.” I’ve also uploaded a test video to Google Drive. Please take a look — thank you!

Terminal
Terminal1569Ă—358 30.9 KB

14

Can you land your changes and share the project with the support workspace so we can take a look?

15

Dear Butler, I tested the demo project you provided — “Swap Camera (World Effects + Face Effects)” by Ian Curtis.

During test this project, I encountered the previously mentioned issue: the system can only switch the camera three times — from world effect to face effect, then from face effect back to world effect, and finally from world effect to face effect again, at which point the screen turns black. The terminal displays a “Script Error.”

16

Dear Bulter, I’m currently developing two WebAR projects using the 8th Wall platform and A-Frame. Each project supports two gameplay modes and allows the user to switch between them via a button. However, I’m encountering two technical issues, I already published the project:

Gameplay Overview:

  • Gameplay 1:
    The 3D model is placed on a horizontal ground plane. Players must tilt their phones to find and tap a hidden “Mark” on the model’s surface to reveal the story text.

  • Gameplay 2:
    The 3D model is anchored on the user’s palm. Players tilt their hand to change the model’s opacity; when the model becomes fully transparent, the story text inside is revealed.

  • Gameplay 3:
    The 3D model flies around in the sky. Players must aim steadily at the model for ~5 seconds to “unlock” the memory and display the story text.

Project 1: Switching between Gameplay 1 and Gameplay 3

Issue:
When switching from Gameplay 1 to Gameplay 3 and uploading a new image (which triggers a MeshyAI 3D model), two models appear:

  • One remains at the original grounded position (from Gameplay 1)

  • Another flies around in the sky (Gameplay 3 behavior)

Suspected Cause:
It seems that the previously loaded components (or entities) are not being fully removed or cleaned up, even after calling removeAttribute() and clearing innerHTML.

Project Link: Niko_GameSwap_1

Project 2: Switching between Gameplay 1 and Gameplay 2

Issue:
Gameplay 1 loads and runs correctly. But when switching to Gameplay 2, the photo upload triggers a network error (while it works when Gameplay 2 is loaded directly).

Suspected Cause:
Likely due to JavaScript component registration conflicts, or issues with shared input elements and asynchronous timing between gameplay modes.

Project Link: App Not Available

Technical Context:

  • Each gameplay registers its own A-Frame component: meshy-markpoint, meshy-revealhand, and meshy-flyaround.

  • Switching is handled via a shared swap-gameplay.js, where we use .setAttribute() and .removeAttribute() to attach/detach gameplay components.

  • The DOM structure includes shared <a-entity id="orbEntity"> for Gameplay 1 & 2, and <a-entity id="skyContainer"> for Gameplay 3.

  • Photo uploads are triggered from shared input elements, using base64 image transfer to a Flask backend via Ngrok, which in turn calls OpenAI and MeshyAI APIs to generate story text and 3D models.

Questions:

  1. What’s the recommended way to fully clear a component and its internal effects (e.g. child entities, animation state, event listeners) before switching modes in A-Frame?

  2. Is there a best practice to reload or reinitialize A-Frame components dynamically ?

  3. What’s the proper way to manage shared UI elements like <input> and <button> across multiple gameplay scripts, especially when switching logic is involved?

  4. Do you have an official example or template that demonstrates clean component switching in 8th Wall WebAR projects?

Thank you very much for your guidance. I really appreciate any advice you can provide!

17

Can you create a new thread with this issue and share the project with the support workspace?

18

Sure, I will create a new thread. Thanks for your advice.