After loading the AR mode, I want to show a placeholder that searches for floor, and after the user click on the floor won’t show my model on that place with its real size.
I use self-hosting with three.js
Hi Arash, Welcome to the forums!
Check out this sample project: three.js: Absolute Scale Template | 8th Wall | 8th Wall
Hi @GeorgeButler
I already checked this template.
1-I tested that with my phone, sometimes it shows a big model that is not in the floor and sometimes it show better size of the model.
2-I do not now how to hide the CoachingOverlay manually
3- I want to create something like Amazon AR mode
@Evan Please answer my question
1- can you please share a video of the issue?
2- can you share some more context as to why you are looking to hide the coaching overlay manually? you should be able to disable it using the disablePrompt parameter:
CoachingOverlay.configure({
disablePrompt: true,
})
Hi @Evan
This is my code and a video about what I want and what I created.
How can I create something like Amazon AR and load my Model in it is real size with 8wall and Three.js
video:Creating a 3D Model Viewer Project | Loom
And this is my code
page.tex
'use client';
import { Canvas } from '@react-three/fiber';
import { Suspense, useEffect, useRef, useState } from 'react';
import { initScenePipelineModule } from './scene-init';
import { Center, Gltf, OrbitControls } from '@react-three/drei';
import { GLTFExporter } from 'three/examples/jsm/exporters/GLTFExporter';
import D3Loader from '@root/app/components/custom/3d-loader';
import { QRCodeSVG } from 'qrcode.react';
import { DeploymentStage } from '@3d/utilities';
import configService from '@root/config/config.service';
declare const XR8: any;
declare const XRExtras: any;
declare const CoachingOverlay: any;
export default function ARWith8thWall() {
const [showArMode, setShowArMode] = useState(false);
const sceneRef = useRef();
const [isSupported, setIsSupported] = useState(false);
useEffect(() => {
// 8th Wall typically requires WebGL and device permissions
const hasWebGL = () => {
try {
const canvas = document.createElement('canvas');
return !!(window.WebGLRenderingContext && canvas.getContext('webgl'));
} catch (e) {
return false;
}
};
const checkSupport = async () => {
// Additional check for permissions or specific APIs can go here
const webGLSupported = hasWebGL();
const userAgent = navigator.userAgent;
const isMobile = /iPhone|iPad|Android/i.test(userAgent);
// Example: Add additional conditions for 8th Wall compatibility if required
setIsSupported(webGLSupported && isMobile);
};
checkSupport();
}, []);
const handleExport = () => {
setShowArMode(true);
// if (sceneRef.current) {
// exportGLTF(sceneRef.current); // Export the referenced object
// }
};
useEffect(() => {
if (showArMode) {
const onxrloaded = () => {
console.log('starting XRscene');
XR8.XrController.configure({ scale: 'absolute' });
CoachingOverlay.configure({
animationColor: '#0000FF',
promptText: 'Tab to place Model',
});
XR8.addCameraPipelineModules([
// Add camera pipeline modules.
// Existing pipeline modules.
XR8.GlTextureRenderer.pipelineModule(), // Draws the camera feed.
XR8.Threejs.pipelineModule(), // Creates a ThreeJS AR Scene.
XR8.XrController.pipelineModule(), // Enables SLAM tracking.
XRExtras.FullWindowCanvas.pipelineModule(), // Modifies the canvas to fill the window.
XRExtras.Loading.pipelineModule(), // Manages the loading screen on startup.
XRExtras.RuntimeError.pipelineModule(), // Shows an error image on runtime error.
CoachingOverlay.pipelineModule(), // Show the absolute scale coaching overlay.
// Custom pipeline modules.
initScenePipelineModule(sceneRef.current), // Sets up the threejs camera and scene content.
]);
// Add a canvas to the document for our xr scene.
const canvas = document.getElementById('camerafeed');
// Open the camera and start running the camera run loop.
XR8.run({
canvas,
});
};
window.XR8 ? onxrloaded() : window.addEventListener('xrloaded', onxrloaded);
} else {
if (window.XR8) {
XR8.stop();
}
}
}, [showArMode]);
return (
<>
{configService.getStage() === DeploymentStage.dev && (
<head>
<script src="//cdn.8thwall.com/web/xrextras/xrextras.js"></script>
<script src="https://cdn.8thwall.com/web/coaching-overlay/coaching-overlay.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/three.js/r134/three.min.js"></script>
<script src="https://cdn.8thwall.com/@8thwall/surface.min.js"></script>
<script
async
src={
'https://apps.8thwall.com/xrweb?appKey=xxx'
}></script>
</head>
)}
<div className={'!h-full !w-full ' + (showArMode ? 'hidden' : 'block')}>
<Canvas
shadows
gl={{
preserveDrawingBuffer: true,
}}>
<color attach="background" args={['#f0f0f0']} />
<ambientLight intensity={2.9} />
<OrbitControls />
<Suspense fallback={<D3Loader />}>
{/* <group name="models" ref={sceneRef}>
<Model />
</group> */}
<Center top={true} position={[0, 0, 0]} ref={sceneRef}>
<Gltf src="/test-model.gltf" castShadow receiveShadow />
</Center>
</Suspense>
</Canvas>
<button
style={{
position: 'absolute',
top: '10px',
left: '10px',
zIndex: 10,
padding: '10px 20px',
background: 'orange',
color: 'white',
border: 'none',
borderRadius: '5px',
cursor: 'pointer',
}}
onClick={handleExport}>
AR Mode
</button>
</div>
{showArMode && (
<>
<div>
<button
style={{
position: 'absolute',
top: '10px',
left: '10px',
zIndex: 10,
padding: '10px 20px',
background: 'orange',
color: 'white',
border: 'none',
borderRadius: '5px',
cursor: 'pointer',
}}
onClick={() => {
setShowArMode(false);
}}>
X
</button>
</div>
{isSupported ? (
<div className="over absolute" id="loadingText">
<span id="promptText">
<svg width="100%">
<text
className="svgStroke !text-white"
x="50%"
y="50%"
dominant-baseline="middle"
text-anchor="middle">
Loading...
</text>
<text
className="svgText !text-white"
x="50%"
y="50%"
dominant-baseline="middle"
text-anchor="middle">
Loading...
</text>
</svg>
</span>
</div>
) : (
<div className="p-20 flex flex-col items-center justify-center">
<h1>Not Supported</h1>
<p>Please scan the QR code below with your mobile</p>
<QRCodeSVG value={window.location.href} />
</div>
)}
</>
)}
<canvas id="camerafeed"></canvas>
</>
);
}
Sene-init.ts
import Controls from './transform-controle';
declare const XR8: any;
declare const THREE: any;
declare const CoachingOverlay: any;
export const initScenePipelineModule = (model) => {
const podium = model;
let plane;
const raycaster = new THREE.Raycaster();
// Populates a cube into an XR scene and sets the initial camera position.
const initXrScene = ({ scene, camera, renderer }) => {
const prompt = document.getElementById('promptText').getElementsByTagName('text');
// Enable shadows in the rednerer.
renderer.shadowMap.enabled = true;
const ambientLight = new THREE.AmbientLight(0xffffff, 1.7); // Color: white, Intensity: 2.9
scene.add(ambientLight);
// Add a plane that can receive shadows.
const planeGeometry = new THREE.PlaneGeometry(2000, 2000);
planeGeometry.rotateX(-Math.PI / 2);
const planeMaterial = new THREE.ShadowMaterial();
planeMaterial.opacity = 0.67;
plane = new THREE.Mesh(planeGeometry, planeMaterial);
plane.receiveShadow = false;
scene.add(plane);
//create cube
const geometry = new THREE.BoxGeometry(1, 1, 1);
const material = new THREE.MeshNormalMaterial();
const cube = new THREE.Mesh(geometry, material);
// podium = cube;
podium.visible = false;
scene.add(model);
// Hide the canvas until models are loaded
const canvas = document.getElementById('camerafeed');
canvas.style.visibility = 'hidden';
setTimeout(() => {
// Show the canvas and update the prompt
canvas.style.visibility = 'visible';
for (let i = 0; i < prompt.length; i++) {
prompt[i].textContent = '';
prompt[i].style.display = 'block';
prompt[i].style.color = 'white';
}
const { body } = document;
body.addEventListener(
'click',
() => {
for (let i = 0; i < prompt.length; i++) {
prompt[i].style.display = 'none';
}
podium.visible = true;
plane.receiveShadow = true;
CoachingOverlay.configure({
disablePrompt: true,
});
},
{ once: true },
);
const controls = new Controls(renderer.domElement, podium);
}, 500);
// Set the initial camera position relative to the scene we just laid out. This must be at a
// height greater than y=0.
camera.position.set(0, 3, 0);
};
// Return a camera pipeline module that adds scene elements on start.
return {
// Camera pipeline modules need a name. It can be whatever you want but must be unique within
// your app.
name: 'eight-podium',
// onStart is called once when the camera feed begins. In this case, we need to wait for the
// XR8.Threejs scene to be ready before we can access it to add content. It was created in
// XR8.Threejs.pipelineModule()'s onStart method.
onStart: ({ canvas }) => {
const { scene, camera, renderer } = XR8.Threejs.xrScene(); // Get the 3js scene from XR8.Threejs
initXrScene({ scene, camera, renderer }); // Add objects set the starting camera position.
// prevent scroll/pinch gestures on canvas
canvas.addEventListener('touchmove', (event) => {
event.preventDefault();
});
// Sync the xr controller's 6DoF position and camera paremeters with our scene.
XR8.XrController.updateCameraProjectionMatrix({
origin: camera.position,
facing: camera.quaternion,
});
// Recenter content on 'select'
canvas.addEventListener(
'select',
({ position }) => {
// update the picking ray with the camera and tap position
raycaster.setFromCamera(position, camera);
// raycast against the "surface" object
const intersects = raycaster.intersectObject(plane);
if (intersects.length === 1 && intersects[0].object === plane) {
const { x } = intersects[0].point;
const { z } = intersects[0].point;
podium.position.set(x, 0.0, z);
}
},
true,
);
},
};
};
transform-controle.ts
import * as THREE from 'three';
type TouchPoint = {
x: number;
y: number;
};
const getDistance = (pt1: TouchPoint, pt2: TouchPoint): number => {
const a = pt1.x - pt2.x;
const b = pt1.y - pt2.y;
return Math.sqrt(a * a + b * b);
};
const getTouchCount = (evt: TouchEvent): number =>
evt.changedTouches ? evt.changedTouches.length : 1;
export default class Controls {
private moved: boolean;
private isScaling: boolean;
private domElement: HTMLElement;
private object: THREE.Object3D;
private touches: Record<number, TouchPoint>;
constructor(domElement: HTMLElement, object: THREE.Object3D) {
this.moved = false;
this.isScaling = false; // New state to track scaling
this.domElement = domElement;
this.object = object;
this.touches = {};
domElement.addEventListener('touchstart', this._onTouchDown.bind(this));
domElement.addEventListener('touchmove', this._onTouchMove.bind(this));
domElement.addEventListener('touchend', this._onTouchUp.bind(this));
domElement.addEventListener('touchcancel', this._onTouchUp.bind(this));
domElement.addEventListener('touchleave', this._onTouchUp.bind(this));
}
_getPointer(touch: Touch | PointerEvent): TouchPoint {
const rect = this.domElement.getBoundingClientRect();
return {
x: ((touch.clientX - rect.left) / rect.width) * 2 - 1,
y: -((touch.clientY - rect.top) / rect.height) * 2 + 1,
};
}
_getTouches(evt: TouchEvent): Record<number, TouchPoint> {
const touches: Record<number, TouchPoint> = {};
const nTouches = getTouchCount(evt);
for (let i = 0; i < nTouches; i++) {
const touch = evt.changedTouches ? evt.changedTouches[i] : evt;
//@ts-ignore
touches[i] = this._getPointer(touch);
}
return touches;
}
_onTouchDown(evt: TouchEvent): void {
this.touches = this._getTouches(evt);
this.isScaling = Object.keys(this.touches).length === 2; // Set scaling state
}
_onTouchMove(evt: TouchEvent): void {
const nTouches = getTouchCount(evt);
const touches = this._getTouches(evt);
if (
nTouches === 1 &&
touches[0] !== undefined &&
this.touches[0] !== undefined &&
!this.isScaling
) {
// 1 finger rotate
this.moved = true;
const delta = touches[0].x - this.touches[0].x;
this.object.rotateY(delta * 2);
} else if (nTouches === 2) {
// touchstart won't fire with multiple fingers.
if (Object.keys(this.touches).length < 2) this.touches = touches;
// 2 finger scale
this.moved = true;
const prevDistance = getDistance(this.touches[0], this.touches[1]);
const currDistance = getDistance(touches[0], touches[1]);
const scale = currDistance / prevDistance;
const currentScale = this.object.scale.x;
const newScale = scale * currentScale;
// this.object.scale.set(newScale, newScale, newScale);
}
this.touches = touches;
}
_onTouchUp(evt: TouchEvent): void {
if (this.moved === false) {
const selectEvent = new Event('select');
// @ts-ignore
selectEvent.position = this.touches[0];
this.domElement.dispatchEvent(selectEvent);
}
this.touches = {};
this.moved = false;
this.isScaling = false; // Reset scaling state
}
}
@Evan and @GeorgeButler Please answer my questions
I would not recommend manually hiding the coaching overlay. The coaching overlay is automatically displayed when scale has not been determined and is automatically hidden when scale has been determined.
You need to move your device forward and backwards as demonstrated in the coaching overlay. It doesn’t look like you’re moving your phone as guided in the video, so scale is likely not being determined accurately. Also, this environment is less than ideal for AR tracking. You should avoid using AR on repeating textures and/or reflective material - these features can negatively impact AR tracking.
I would recommend leaving the coaching overlay as-is; the user should move their device forwards and backwards to establish scale, and once determined it will disappear and emit the coaching-overlay.hide event on the scene element. You can use this event to then show some UI that says “tap to place” and enable tap place functionality. For that, I would recommend doing something similar to our Place Model sample project.
Hi @Evan
Thanks for the information about the Coaching overlay.
My main question was about putting a model in the world in its real size, the Place Model sample project. do not solve my problem.
Absolute Scale Problems:
1- Do not show the real size of the model.
2-After adding the model to the world, by moving mobile the model gets bigger or smaller, sometimes the model moves up from the floor, and sometimes the model disappears.
When I change the Scale from Absolute to Responsive, none of those problems happen.
Do you see the same behavior in our sample projects?
Hi @Evan
1- I can not see in your A-Frame Project, but I see in your Three.js Sample project(three.js: Absolute Scale Template | 8th Wall | 8th Wall)
2- I do not work with A-Frame, I work with Three.js
@Evan @GeorgeButler Please answer my question
Are you manually hiding the coaching overlay and/or did you remove the hide event listener/function? I’ve just tested the three.js sample project and it’s working as expected on iOS and Android. Again, this may have to do with the way you are moving your phone at the beginning of the experience (i.e. not following the movements as shown in the coaching overlay) and a poor tracking environment as seen in the video shared earlier).
Hi @Evan
I removed the coaching overlay, I just used the Absolute Scale, and that problems happened.
The three.js sample project also has the same issues.
@GeorgeButler @Evan
I tested the three.js sample template in iPhone 13 pro, it just showed that something went wrong
message.
What should I do?
1-Do you want to fix the problem of Absolute?
2-Or you want to show me another way to fix my problem