I would be very happy if the xrimagefound position were more stable.
Currently, the scale and position change frequently (the scale increases when the position is far away from the camera and decreases when the position is close to the camera), and although it looks stable, it causes problems when trying to simulate physics.
It would be great if the scale is fixed and the only thing that changes is the position.
If adding “Specify Size” to the Image Target setting, like the AR Foundation would improve this, I would like to see such a setting.
2 Likes
Thank you for the request! Are you able to provide more details on your use-case and the specific issue you’re facing with the physics simulation? Have you found a workaround for the time being?
Thank you Evan for your reply!
I would like to implement something like this in 8thwall.
I want to use a physical object (kinematic in ammo.js) that always follows the same position as the Image target and interferes with other physical objects (dynamic in ammo.js).
Hi @Ken_Mitsui can you share what you are currently experiencing? Have you created a kinematic attached via the image target? Can you set debug: true and share a screen recording?
Thanks Ian for your reply!
And I’m so sorry that I probably misunderstood.
The reason was that I was not able to setLocalScaling of the value of “detail.scale” to the rigidBody of the kinematic attached to the image target.
(Because the logic to reflect the scale was not included in [Physics simmulation in Three.js].)
detail.position was stable throughout.
The following code works now.
There are many things that need to be fixed, though, such as the slippage problem…
I should have put this in the technical support category. Sorry about that.
export const initScenePipelineModule = () => {
const purple = 0xAD50FF
let card
let cardGenerated = false
let rigidBodyGenerated = false
const clock = new THREE.Clock()
let deltaTime
let shapeIndex = 0
const mouseCoords = new THREE.Vector2()
const raycaster = new THREE.Raycaster()
const ballMaterial = new THREE.MeshPhongMaterial({color: 0x202020})
let clickRequest = false
let btTrans
let anchorRigidBody
let collisionShape
let positionBtVector3
let rotationBtQuaternion
let scaleBtVector3
// for test
let detail_
// Populates a card into an XR scene and sets the initial camera position.
const initXrScene = ({scene, camera, renderer}) => {
// Enable shadows in the rednerer.
renderer.shadowMap.enabled = true
// Add some light to the scene.
const directionalLight = new THREE.DirectionalLight(0xffffff, 0.5)
directionalLight.position.set(5, 10, 7)
directionalLight.castShadow = true
scene.add(directionalLight)
// Add a plane that can receive shadows.
const planeGeometry = new THREE.PlaneGeometry(2000, 2000)
planeGeometry.rotateX(-Math.PI / 2)
const planeMaterial = new THREE.ShadowMaterial()
planeMaterial.opacity = 0.67
const plane = new THREE.Mesh(planeGeometry, planeMaterial)
plane.receiveShadow = true
scene.add(plane)
// Set the initial camera position relative to the scene we just laid out. This must be at a
// height greater than y=0.
camera.position.set(0, 2, 2)
}
// Places content over image target
const showTarget = ({detail}) => {
const {scene, camera, renderer} = XR8.Threejs.xrScene()
// When the image target named 'model-target' is detected, show 3D model.
// This string must match the name of the image target uploaded to 8th Wall.
if (!cardGenerated) {
const material = new THREE.MeshBasicMaterial({
transparent: true,
opacity: 0.5,
color: 0xAD50FF,
})
// card = new THREE.Mesh(new THREE.BoxGeometry(0.91 * detail.scale, 0.55 * detail.scale, 0.01 * detail.scale), material)
const adjustScale = 1.32
card = new THREE.Mesh(new THREE.BoxGeometry(0.91 * adjustScale, 0.55 * adjustScale, 0.01), material)
card.position.copy(detail.position)
card.quaternion.copy(detail.rotation)
card.scale.set(detail.scale, detail.scale, detail.scale)
card.userData.ammo = {
mass: 0,
margin: 0.01,
isKinematic: true,
shapeType: 'mesh',
}
card.addEventListener('generated', (e) => {
setTimeout(() => {
card.visible = true
// eslint-disable-next-line new-cap, no-undef
btTrans = new window.Ammo.btTransform()
anchorRigidBody = card.userData.physicsBody
anchorRigidBody.getMotionState().getWorldTransform(btTrans)
collisionShape = anchorRigidBody.getCollisionShape()
// eslint-disable-next-line new-cap, no-undef
positionBtVector3 = new window.Ammo.btVector3()
// eslint-disable-next-line new-cap, no-undef
rotationBtQuaternion = new window.Ammo.btQuaternion()
// eslint-disable-next-line new-cap, no-undef
scaleBtVector3 = new window.Ammo.btVector3()
rigidBodyGenerated = true
}, 1000)
})
cardGenerated = true
scene.add(card)
}
if (rigidBodyGenerated) {
detail_ = detail
// eslint-disable-next-line new-cap, no-undef
positionBtVector3.setValue(detail.position.x, detail.position.y, detail.position.z)
btTrans.setOrigin(positionBtVector3)
// eslint-disable-next-line new-cap, no-undef
rotationBtQuaternion.setValue(detail.rotation.x, detail.rotation.y, detail.rotation.z, detail.rotation.w)
btTrans.setRotation(rotationBtQuaternion)
anchorRigidBody.getMotionState().setWorldTransform(btTrans)
// eslint-disable-next-line new-cap, no-undef
scaleBtVector3.setValue(detail.scale, detail.scale, detail.scale)
collisionShape.setLocalScaling(scaleBtVector3)
card.scale.set(detail.scale, detail.scale, detail.scale)
}
}
// Hides the image frame when the target is no longer detected.
const hideTarget = ({detail}) => {
if (detail.name === 'logo') {
// card.visible = false
}
}
const processClick = () => {
if (clickRequest) {
const {scene, camera, renderer} = XR8.Threejs.xrScene()
// 見ている方向に飛ぶBallを追加
raycaster.setFromCamera(mouseCoords, camera)
const direction = raycaster.ray.direction.clone()
let shapeGeometry
let ammoShape
switch (shapeIndex % 4) {
case 0:
shapeGeometry = new THREE.SphereGeometry(0.02, 18, 16)
ammoShape = 'sphere'
break
case 1:
shapeGeometry = new THREE.BoxGeometry(0.02, 0.02, 0.02)
ammoShape = 'box'
break
case 2:
shapeGeometry = new THREE.ConeGeometry(0.02, 0.04, 16)
ammoShape = 'cone'
break
case 3:
shapeGeometry = new THREE.CylinderGeometry(0.02, 0.02, 0.04, 16)
ammoShape = 'cylinder'
break
default:
shapeGeometry = new THREE.SphereGeometry(0.02, 18, 16)
break
}
const ball = new THREE.Mesh(shapeGeometry, ballMaterial)
ball.castShadow = true
ball.receiveShadow = true
ball.position.copy(raycaster.ray.origin)
ball.position.addScaledVector(direction, 0.1)
ball.quaternion.set(0, 0, 0, 1)
ball.userData.ammo = {
mass: 0.1,
margin: 0.01,
shapeType: ammoShape,
rollingFriction: 0.01,
restitution: 0.1,
linearVelocity: direction.multiplyScalar(1),
isSoft: false,
}
scene.add(ball)
ball.addEventListener('collide', (e) => {
// console.log('ball is collide!')
})
shapeIndex++
clickRequest = false
}
}
// Return a camera pipeline module that adds scene elements on start.
return {
// Camera pipeline modules need a name. It can be whatever you want but must be unique within
// your app.
name: 'threejsinitscene',
// onStart is called once when the camera feed begins. In this case, we need to wait for the
// XR8.Threejs scene to be ready before we can access it to add content. It was created in
// XR8.Threejs.pipelineModule()'s onStart method.
onStart: ({canvas}) => {
const {scene, camera, renderer} = XR8.Threejs.xrScene() // Get the 3js scene from XR8.Threejs
initXrScene({scene, camera, renderer}) // Add objects set the starting camera position.
// prevent scroll/pinch gestures on canvas
canvas.addEventListener('touchmove', (event) => {
event.preventDefault()
})
// Sync the xr controller's 6DoF position and camera paremeters with our scene.
XR8.XrController.updateCameraProjectionMatrix(
{origin: camera.position, facing: camera.quaternion}
)
window.addEventListener(
'touchstart', (e) => {
e.preventDefault()
if (!clickRequest) {
mouseCoords.set(
(e.touches[0].clientX / window.innerWidth) * 2 - 1,
-(e.touches[0].clientY / window.innerHeight) * 2 + 1
)
clickRequest = true
}
}, false
)
},
onUpdate: ({processCpuResult}) => {
const {scene, camera, renderer} = XR8.Threejs.xrScene() // Get the 3js scene from XR8.Threejs
deltaTime = clock.getDelta()
processClick()
},
listeners: [
{event: 'reality.imagefound', process: showTarget},
{event: 'reality.imageupdated', process: showTarget},
{event: 'reality.imagelost', process: hideTarget},
],
}
}
Sorry for the back and forth statements.
The card physics simulation itself now works by applying xrimagefound’s scale (and setLocalScaling in ammo.js) to the card object that is currently being moved to fit the image.
However, the newly generated physics object (ball) has no scale applied, so the scale relationship is broken.
Is the only way to apply scale to everything related to the card object (that I want to belong to the same physics simulation space)?
Also, although the position is constant, the spacing between positions is also scaled by the application of scale.
If I need a position to be “5 cm above the card,” do I have to apply scale to that position as well?
After all, it would be nice to have a fixed scale.
And please let me know if there is a workaround.
Hello @Ken_Mitsui
Have you referenced our Image Target to SLAM example project here:
This may be helpful as this project drops physics based objects and a constant scale size.
This looks great, but the scale of the palm tree and palms is not consistent.
In my implementation, the card is the tree and the ball is the palm.
When I adjust the card to the detail.scale, it looks like the size of the ball relative to the card changes.
I’ve tried many things since then and it worked, so I’ll share it with you.
targetPosition.set(detail.position.x, detail.position.y, detail.position.z)
stablePosition = targetPosition.sub(camera.position)
stablePosition.multiplyScalar(1 / detail.scale)
stablePosition.add(camera.position)
By using stablePosition created in this way, we can set the position without changing the scale of the target and the apparent size will not change.
This method seems to be a good choice for continuous physics simulation.
However, it is much more stable to use the normal position and scale for each.
I think this is a niche need like mine, so if anyone else has the same need, please give it a try.
4 Likes
Thank you for sharing your solution!
This topic was automatically closed 4 days after the last reply. New replies are no longer allowed.