Issues with image tracking orientation/ scaling

[asky]

Hi,

We’re developing an image target based app that’s designed to be viewed at an angle, and have experienced frequent tracking loss when aiming the phone at aggressive angles. We’re trying to avoid having a user manually place/ scale the objects in the scene, and want the image target to automatically trigger our experience.

To avoid the tracking loss, we switched over to using SLAM tracking once an image target is first detected which has worked fairly well. However, sometimes the initial image target orientation and/ or scale are incorrect (screenshots attached). Are there ways we can prevent this from happening and/ or guarantee that when we initialize our scene it’ll have the correct rotation/ scaling?

This is how we’re currently initializing our scene once the image target is set:

...
    this.scene.addEventListener(AppEvent.XR_IMAGE_TARGET_FOUND, this.handleImageTargetFound)
    this.sceneAdded = false
  },
  handleImageTargetFound(e) {
    const {detail} = e
    if (this.sceneAdded == false) {
      const selectionScene = document.createElement('a-entity')
      selectionScene.setAttribute('geometry', 'primitive:plane; width: .75; height: 1.7')
      selectionScene.setAttribute('material', 'src: #ticket-image; transparent: true;')

      const {object3D} = selectionScene
      object3D.position.copy(detail.position)
      object3D.position.y = 0
      object3D.position.z += 0.8
      object3D.quaternion.copy(detail.rotation)
      object3D.scale.set(detail.scale, detail.scale, detail.scale)
      this.el.sceneEl.appendChild(selectionScene)
      this.sceneAdded = true
    }
  },

[asky]

Also, in case it’s helpful, we’ve experienced theses rotation/ scaling issues on a variety of image targets, containing both custom graphics and photographs.

[Ian]

One suggestion would be increasing the tracking points in your image target.

Please reference this youtube tutoiral

Quick Tips: Best Image Targets For AR Tracking

[asky]

Thanks for your response.

We’ve tried with some of the default image targets provided in the 8th wall examples. We’ve experienced fewer rotation issues, however the scaling still seems to be wrong/ inconsistent fairly frequently.

Are there other solves for this?

[TonyT]

@asky Looks like you are only responding to the xrimagefound (I assume that is what AppEvent.XR_IMAGE_TARGET_FOUND is doing)

I would recommend also listening on the xrimageupdated event too - so that if the initial position/rotation/scale is off, it can be corrected after the fact.

Basically in your event handlers:

• found: create element and add to scene. set position/rotation/scale
• updated: only set position/rotation/scale

You could probably just use the same handleImageTargetFound function you already have, but add a bit more logic so that the element is only created the first time.

[asky]

@TonyT We tried adding the “updated” event listener and it fixed the issue-- thanks!

[asky]

@TonyT Sorry, actually after adding the updated event it seems like position and rotation of the model we’re adding to the scene jumps around quite a bit now. It seemed more stable when just relying on world tracking after setting the initial values of the image target.

Below is my revised code-- should i be updating the model position/ rotation/ scale every frame (like I am now) or only on the first update event fired immediately after a found event is fired?

    this.scene.addEventListener(AppEvent.XR_IMAGE_TARGET_FOUND, this.handleImageTargetFound)
    this.scene.addEventListener(AppEvent.XR_IMAGE_TARGET_UPDATED, this.handleImageTargetFound)
  },
  handleImageTargetFound(e) {
    const {detail} = e
    let selectionScene = document.getElementById('selectionScene')

    if (!selectionScene) {
      selectionScene = document.createElement('a-entity')
      selectionScene.setAttribute('id', 'selectionScene')
      selectionScene.setAttribute('geometry', 'primitive:plane; width: .75; height: 1.7')
      selectionScene.setAttribute('material', 'src: #ticket-image; transparent: true;')
      this.el.sceneEl.appendChild(selectionScene)
    }
    const {object3D} = selectionScene
    object3D.position.copy(detail.position)
    object3D.position.y = 0
    object3D.quaternion.copy(detail.rotation)
    object3D.scale.set(detail.scale, detail.scale, detail.scale)
  },

[TonyT]

It looks like you are doing something very similar to what our xrextras-named-image-target component is doing - where both found and updated callbacks set position/rotation/scale each time.

Check out the code here:

8thwall/web/blob/master/xrextras/src/aframe/components/target-components.ts#L368-L397

The main difference I see is that you are hardcoding y=0 vs the location of the image target itself. Is you image target going to be laying on the ground, or what is the use case here? Also, what is the initial position of your camera in the scene? Did you upload multiple image targets to the project that are similar?

Are you able to share a link to your experience?

[asky]

@TonyT Yes, previously we were just using the xrextras-named-image-target component and having the image target control the experience. However, we were running into some image tracking issues and believe it was partly becuase experience encourages users to aim their phone at the image target at aggressive angles and zoom into the scene, sometimes obscuring the image target.

Our image target is supposed to be laying flat on the ground and the experience is designed to appear directly above it. We have the camera set to:

  <a-camera
    ready-loader
    position="0 4 10"
    raycaster="objects: .cantap">
  </a-camera>

As a workaround, we’re trying to have the image target just initialize the 3d scene’s position/rotation/scale and use SLAM tracking for the experience after that (so the image target replaces the drag position/ pinch scale UX typically used to setup SLAM experiences.) Once the image target found, we place the new scene outside of the xrextras-named-image-target component.

I’m messaging you a link to our project now, thanks.