In our experience we are trying to have 2 different scenes or screen
1 - simply display AR object with features like touch scale and such
2- The another scene which we would trigger via a button from the other scene that will start hand tracking to place objects on the hands.
The tricky part is the pipeline switching which we currently do with scene switching but maybe there are better ways to do this??
Currently, that’s the best way to approach it, as toggling features requires an engine restart, with the exception of image targets.
Hey @GeorgeButler,
Would you kindly advice if the following is the right way of doing it?
We are successfully switching between the 2 scenes, however as soon as we enter the hand tracking scene everything lags like hell and eventually the phone completely freezes and crash so I feel like we are not managing this properly.
import './index.css'
//
// app.js is the main entry point for your 8th Wall app. Code here will execute after head.html
// is loaded, and before body.html is loaded.
import './index.css'
import {initScenePipelineModule} from './eighti-scene-init'
import * as camerafeedHtml from './camerafeed.html'
import {handScenePipelineModule} from './hand-scene'
let watchName = ' '
let firstTime = true
const runHandPipeline = () => {
XR8.stop()
XR8.clearCameraPipelineModules()
// Add a canvas to the document for our xr scene.
document.body.insertAdjacentHTML('beforeend', `
<canvas id="camerafeed" width="${window.innerWidth}" height="${window.innerHeight}"></canvas>
`)
document.body.insertAdjacentHTML('beforeend', `
<div id="backfromAR" class=""></div>
`)
XR8.HandController.configure({
coordinates: {mirroredDisplay: false},
})
const moduleHand = [ // Add camera pipeline modules.
// Existing pipeline modules.
XR8.GlTextureRenderer.pipelineModule(), // Draws the camera feed.
XR8.Threejs.pipelineModule(), // Syncs threejs renderer to camera properties.
XR8.HandController.pipelineModule(), // Loads 8th Wall Face Engine
XR8.CanvasScreenshot.pipelineModule(), // Required for photo capture
window.LandingPage.pipelineModule(), // Detects unsupported browsers and gives hints.
XRExtras.FullWindowCanvas.pipelineModule(), // Modifies the canvas to fill the window.
XRExtras.Loading.pipelineModule(), // Manages the loading screen on startup.
XRExtras.RuntimeError.pipelineModule(), // Shows an error image on runtime error.
// Custom pipeline modules
handScenePipelineModule(watchName),
window.HandCoachingOverlay.pipelineModule(),
]
const moduleFirstTime = [ // Add camera pipeline modules.
// Existing pipeline modules.
XR8.GlTextureRenderer.pipelineModule(), // Draws the camera feed.
XR8.Threejs.pipelineModule(), // Syncs threejs renderer to camera properties.
XR8.HandController.pipelineModule(), // Loads 8th Wall Face Engine
XR8.CanvasScreenshot.pipelineModule(), // Required for photo capture
window.LandingPage.pipelineModule(), // Detects unsupported browsers and gives hints.
XRExtras.FullWindowCanvas.pipelineModule(), // Modifies the canvas to fill the window.
XRExtras.Loading.pipelineModule(), // Manages the loading screen on startup.
XRExtras.RuntimeError.pipelineModule(), // Shows an error image on runtime error.
// Custom pipeline modules
handScenePipelineModule(watchName),
]
XR8.addCameraPipelineModules(firstTime ? moduleFirstTime : moduleHand)
// Open the camera and start running the camera run loop.
XR8.run({
canvas: document.getElementById('camerafeed'),
cameraConfig: {direction: XR8.XrConfig.camera().BACK},
allowedDevices: XR8.XrConfig.device().ANY,
})
if (firstTime) {
MenuScreen()
firstTime = false
document.getElementById('backfromAR').style.display = 'none'
} else {
document.getElementById('backfromAR').style.display = 'block'
document.getElementById('backfromAR').addEventListener('click', () => {
console.log('Back from AR button clicked!')
// Add the function you want to call here, e.g., NormalScene();
NormalScene()
})
}
}
const MainScreen = () => {
// Assuming you have a WebGLRenderer already set up for Three.js
const overlay = document.createElement('div')
overlay.className = 'overlay-screen'
overlay.id = 'overlayID'
document.body.appendChild(overlay)
const button = document.createElement('button')
button.className = 'overlay-button'
button.innerHTML = 'Get Started'
button.id = 'buttonStartedID'
overlay.appendChild(button)
button.addEventListener('click', () => {
console.log('AR Button clicked! Changing scene...')
NormalScene()
})
}
const onxrloaded = () => {
XR8.stop()
XR8.clearCameraPipelineModules()
document.body.insertAdjacentHTML('beforeend', `
<button id="tryButton" class="">Try It</button>
`)
document.body.insertAdjacentHTML('beforeend', `
<button id="watch1Button">1</button>
`)
document.body.insertAdjacentHTML('beforeend', `
<button id="watch2Button">2</button>
`)
document.body.insertAdjacentHTML('beforeend', `
<button id="watch3Button">3</button>
`)
document.body.insertAdjacentHTML('beforeend', `
<button id="watch4Button">4</button>
`)
document.getElementById('tryButton').addEventListener('click', () => {
console.log('AR Button clicked! Changing scene...')
ARScene()
})
document.getElementById('watch1Button').addEventListener('click', () => {
document.getElementById('tryButton').style.display = 'block'
watchName = 'one'
})
document.getElementById('watch2Button').addEventListener('click', () => {
document.getElementById('tryButton').style.display = 'block'
watchName = 'two'
})
document.getElementById('watch3Button').addEventListener('click', () => {
document.getElementById('tryButton').style.display = 'block'
watchName = 'three'
})
document.getElementById('watch4Button').addEventListener('click', () => {
document.getElementById('tryButton').style.display = 'block'
watchName = 'four'
})
// Add a button dynamically
XR8.addCameraPipelineModules([ // Add camera pipeline modules.
// Existing pipeline modules.
XR8.GlTextureRenderer.pipelineModule(), // Draws the camera feed.
XR8.Threejs.pipelineModule(), // Creates a ThreeJS AR Scene.
XR8.XrController.pipelineModule(), // Enables SLAM tracking.
window.LandingPage.pipelineModule(), // Detects unsupported browsers and gives hints.
XRExtras.FullWindowCanvas.pipelineModule(), // Modifies the canvas to fill the window.
XRExtras.Loading.pipelineModule(), // Manages the loading screen on startup.
XRExtras.RuntimeError.pipelineModule(), // Shows an error image on runtime error.
// Custom pipeline modules.
initScenePipelineModule(), // Sets up the threejs camera and scene content.
])
// Add a canvas to the document for our xr scene.
document.body.insertAdjacentHTML('beforeend', camerafeedHtml)
const canvas = document.getElementById('camerafeed')
// Open the camera and start running the camera run loop.
XR8.run({canvas})
}
const stopHandPipeline = () => {
// Stop XR8 and clean up
// XR8.removeCameraPipelineModules()
// Remove the canvas and button from the DOM
const canvas = document.getElementById('camerafeed')
const button = document.getElementById('changeSceneButton')
const backbutton = document.getElementById('backButton')
const trybutton = document.getElementById('tryButton')
const button1 = document.getElementById('watch1Button')
const backfromARbutton = document.getElementById('backfromAR')
const button2 = document.getElementById('watch2Button')
const button3 = document.getElementById('watch3Button')
const button4 = document.getElementById('watch4Button')
const overlayID = document.getElementById('overlayID')
const buttonStartedID = document.getElementById('buttonStartedID')
if (backfromARbutton) {
backfromARbutton.remove()
}
if (trybutton) {
trybutton.remove()
}
if (overlayID) {
overlayID.remove()
}
if (buttonStartedID) {
buttonStartedID.remove()
}
if (button1) {
button4.remove()
button3.remove()
button2.remove()
button1.remove()
}
if (backbutton) {
backbutton.remove()
} if (canvas) {
document.body.removeChild(canvas)
}
if (button) button.remove()
}
const ARScene = () => {
stopHandPipeline()
window.XR8 ? runHandPipeline() : window.addEventListener('xrloaded', runHandPipeline)
}
const NormalScene = () => {
stopHandPipeline()
window.XR8 ? onxrloaded() : window.addEventListener('xrloaded', onxrloaded)
}
ARScene()
Do you have any custom assets being loaded? Like a GLB or GLTF? If so could you share or DM a link?
After reviewing the model you sent through DM, I’m fairly certain the lag is caused by the initial load.
165,048 vertices (This is quite high—ideally, you want it around 10k or lower. Consider using a lower-quality model and baking the detail from the high-poly version in your modeling software.)
6 textures at 2048x2048 (It’s recommended to use one 2048x2048 texture for the entire model’s albedo and smaller textures for normal, specular, etc maps.)
To address this, developers typically load the model and hide it from view until it’s needed. In your case, I suggest showing the model when hand tracking is enabled and hiding it otherwise to avoid stuttering. You can hide it by positioning it behind the camera (about 100 units) and then bringing it into view when needed.
For reference you can use https://gltf.report to check your model 
Thanks @GeorgeButler, for looking into this.
Yer I had a feeling our models were too high and will have to rework them.
About the last point you mentioned, not sure I follow because we build the model and set visible to false and only show it when hand is tracking so it shouldn’t affect impact performance? Or in 8th wall its a different case perhaps?
We pretty much followed the template hand tracking project
const buildWatch1 = () => {
loader.load(require('./assets/xxxxxx.glb'), (watchObj) => {
watch1 = watchObj.scene
watch1.scale.set(1.1, 1.1, 1.1)
watch1.position.set(0, -0.013, 0)
watch1.rotation.set(0, 1.5708, 0)
wristAttachment.add(watch1)
hand.add(wristAttachment)
addRealtimeReflections(watch1)
watch1.visible = false
watchEntity1 = watch1
})
}
buildWatch1()
// in updateWatch method
if (selectedName === 'one') {
watch1.visible = true
}
I believe setting the model to invisible causes it to get released from memory? In this case this is not what we want. @Evan might have more specific insight into this.