Issues with the gesture function

When accessing any of the following links on an iPhone:

The capture function performs well. However, when attempting to drag or scale the skulls using your finger:

  • The skulls intermittently appear and disappear.
  • The eyes of the skulls may change color unintentionally when going off-screen and then back into view.
  • There appears to be a discrepancy in the shadows once a skull is moved.

Could the code governing button colors or highlights be conflicting with the gesture function? Your assistance with this matter is greatly appreciated.

Thank you, Ang

this is my app.js

import ‘./main.css’

import {buttonBehaviorComponent} from ‘./button-behavior’
AFRAME.registerComponent(‘button-behavior’, buttonBehaviorComponent)

import {buttonBehaviorComponent2} from ‘./button-behavior2’
AFRAME.registerComponent(‘button-behavior2’, buttonBehaviorComponent2)

AFRAME.registerComponent(‘custom-capture-btn’, {
init() {
const btn = document.getElementById(‘recorder-button’)
btn.innerHTML = <img id="icon" src=${require('./assets/camera.png')}>
},
})

import ‘./main.css’

import {gestureSelectorComponent} from ‘./gesture-selector’
AFRAME.registerComponent(‘gesture-selector’, gestureSelectorComponent)

import {highlightComponent} from ‘./highlight’
AFRAME.registerComponent(‘highlight’, highlightComponent)

import ‘./index.css’

import {splashImageComponent} from ‘./splash-image’
AFRAME.registerComponent(‘splash-image’, splashImageComponent)

The skulls are directed to do the same thing here in the body

a-entity
gltf-model=“#skull01”
class=“cantap”
xrextras-hold-drag
gesture-selector=“riseHeight: 0.4; addHoldDrag: true; addTwoFingerRotate: true; addPinchScale: true; addHighlight: true”
shadow
animation-mixer=“clip: *”
animation__spin="
property: rotation;
dur: 3000;
from: 0 0 0;
to: 0 -360 0;
loop: true;
easing: linear;"
reflections=“type: realtime”
scale=“3 3 3”
position=“-3 1 -2”
rotation=“0 25 0”
a-entity

gesture selector js 


const gestureSelectorComponent = {
selectedModel: null,

schema: {
addHoldDrag: {default: true},
riseHeight: {default: 0.7},
addTwoFingerRotate: {default: false},
addOneFingerRotate: {default: false},
addPinchScale: {default: false},
addHighlight: {default: false},
},

init() {
const self = this

// Check if addHoldDrag is true and then set the attribute
if (this.data.addHoldDrag) {
  this.el.setAttribute('xrextras-hold-drag', 'riseHeight', this.data.riseHeight)
}

this.el.addEventListener('mousedown', (event) => {
  // If another model is clicked, first remove gestures from the previously selected model
  if (self.constructor.selectedModel) {
    self.removeGestures(self.constructor.selectedModel)
  }

  // Add gestures to the current clicked model
  self.addGestures(event.currentTarget)
  self.constructor.selectedModel = event.currentTarget
})

},

removeGestures(entity) {
entity.removeAttribute(‘xrextras-two-finger-rotate’)
entity.removeAttribute(‘xrextras-pinch-scale’)
entity.removeAttribute(‘xrextras-one-finger-rotate’)
entity.setAttribute(‘highlight’, ‘enabled’, false)
},

addGestures(entity) {
const {data} = this
if (data.addTwoFingerRotate) {
entity.setAttribute(‘xrextras-two-finger-rotate’, ‘’)
}
if (data.addPinchScale) {
entity.setAttribute(‘xrextras-pinch-scale’, ‘’)
}
if (data.addHighlight) {
entity.setAttribute(‘highlight’, ‘enabled’, true)
}
if (data.addOneFingerRotate) {
entity.setAttribute(‘xrextras-one-finger-rotate’, ‘’)
}
},
}

export {gestureSelectorComponent}

Without seeing a video (which would be great if you could share one) here are a couple things that come to mind.

  1. Models hiding
    This could be related to the culling aspect fo the 3D model, I woudl reference this thread:
  1. Can you share a video/photo of this to better understand what you mean?

  2. You may want to reference a shadow map thread

OK I will try these things
 I used the same code as my careBears project - Veni Vidi PolyBear - Painting By Angie Jones for "Care Bears Forever" at Corey Helford Gallery | Angie Jones | 8th Wall

And I have repeated models in there and the careBears don’t pop off and on and the shadow stays on the floor so hopefully these things will fix it all.

I have no idea what this means “then add it to the entity that is clipping” so I may be way in over my head and have to delete the functionality of moving the skulls around from this project

You can see the pop here - Sign up to view this shared file

There is not cutting in the movie, the skulls just pop over somewhere else all of a sudden. is that not happening to you when you click on the QR code? Then, once they pop the eyes change color and the shadow is not on the ground.

I republished this one to only have one skull and it does it everytime

I give up. If I dump the gesture code then the skulls are fine. Bummed I cannot figure this out. I even put three different skulls in there to assess how they are moving and where they are positioned.

I put the culling code in app.js but I do not think that is the issue because the skulls go through each other with no problem as did the careBears.

This is the body

<!-- Copyright (c) 2022 8th Wall, Inc. -->
<!-- body.html is optional; elements will be added to your html body after app.js is loaded. -->

<div id="splashimage">
  <div id="start" style="display: none">
    <h2>Start AR</h2>
  </div>
</div>

<button id="btn">buy it!</button>
<button id="btn2">website</button>

<!-- "xrweb" component has been removed from the a-scene, as we'll programatically add this later -->
<!-- when the user clicks the "Start AR" button -->

<a-scene
  splash-image
  xrextras-gesture-detector
  landing-page
  xrextras-loading
  xrextras-runtime-error
  custom-capture-btn
  xrextras-tap-recenter
  renderer="colorManagement: true"
  xrweb="allowedDevices: any"
  button-behavior
  button-behavior2
  no-cull>
  <a-assets>
    <a-asset-item id="skull01" src="assets/whiteskull.glb"></a-asset-item>
    <a-asset-item id="skull02" src="assets/hotpinkskull.glb"></a-asset-item>
    <a-asset-item id="skull03" src="assets/blueskull.glb"></a-asset-item>
    <a-asset-item id="skull04" src="assets/goldskull.glb"></a-asset-item>
    <a-asset-item id="skull05" src="assets/whiteskull.glb"></a-asset-item>
    <a-asset-item id="skull06" src="assets/whiteskull.glb"></a-asset-item>
    <a-asset-item id="skull07" src="assets/whiteskull.glb"></a-asset-item>
    <a-asset-item id="skull08" src="assets/whiteskull.glb"></a-asset-item>
    <a-asset-item id="skull09" src="assets/whiteskull.glb"></a-asset-item>
    <a-asset-item id="skull10" src="assets/whiteskull.glb"></a-asset-item>
  </a-assets>

  <xrextras-capture-button capture-mode="photo"></xrextras-capture-button>
  <xrextras-capture-config request-mic="false"></xrextras-capture-config>
  <xrextras-capture-preview></xrextras-capture-preview>

  <!-- The raycaster will emit mouse events on scene objects specified with the cantap class -->
  <a-camera
    id="camera"
    position="0 4 4"
    raycaster="objects: .cantap"
    cursor="fuse: false; rayOrigin: mouse;">
  </a-camera>

  <a-entity
    light="
      type: directional;
      intensity: 0.8;
      castShadow: true;
      shadowMapHeight:2048;
      shadowMapWidth:2048;
      shadowCameraTop: 20;
      shadowCameraBottom: -20;
      shadowCameraRight: 20;
      shadowCameraLeft: -20;
      target: #camera"
    xrextras-attach="target: camera; offset: 8 15 4"
    position="1 4.3 2.5"
    shadow>
  </a-entity>

  <a-light type="ambient" intensity="0.7"></a-light>

  <!-- models in scene -->
  <a-entity
    gltf-model="#skull01"
    class="cantap"
    xrextras-hold-drag
    gesture-selector="riseHeight: 0.3; addHoldDrag: true; addTwoFingerRotate: true; addPinchScale: true; addHighlight: true"
    shadow
    animation-mixer="clip: *"
    animation__spin="
        property: rotation;
        dur: 3000;
        from: 0 0 0;
        to: 0 -360 0;
        loop: true;
        easing: linear;"
    reflections="type: realtime"
    scale="4 4 4"
    position="-3 1 -2"
    rotation="0 25 0">
  </a-entity>

  <a-entity
    gltf-model="#skull02"
    class="cantap"
    xrextras-hold-drag
    gesture-selector="riseHeight: 0.3; addHoldDrag: true; addTwoFingerRotate: true; addPinchScale: true; addHighlight: true"
    shadow
    position="0 1 -6"
    animation-mixer="clip: *"
    animation__spin="
        property: rotation;
        dur: 4000;
        from: 0 0 0;
        to: 0 -360 0;
        loop: true;
        easing: linear;"
    reflections="type: realtime"
    scale="4 4 4">
  </a-entity>

  <a-entity
    gltf-model="#skull03"
    class="cantap"
    xrextras-hold-drag
    gesture-selector="riseHeight: 0.3; addHoldDrag: true; addTwoFingerRotate: true; addPinchScale: true; addHighlight: true"
    shadow
    position="6 1 0"
    animation-mixer="clip: *"
    animation__spin="
        property: rotation;
        dur: 2000;
        from: 0 0 0;
        to: 0 -360 0;
        loop: true;
        easing: linear;"
    reflections="type: realtime"
    scale="4 4 4">
  </a-entity>

  <!-- ground plane -->
  <a-plane
    id="ground"
    class="cantap"
    rotation="-90 0 0"
    width="1000"
    height="1000"
    material="shader: shadow"
    shadow>
  </a-plane>
</a-scene>

This is app.js

// Copyright (c) 2022 8th Wall, Inc.
//
// app.js is the main entry point for your 8th Wall web app. Code here will execute after head.html
// is loaded, and before body.html is loaded.

import './main.css'

import {buttonBehaviorComponent} from './button-behavior'
AFRAME.registerComponent('button-behavior', buttonBehaviorComponent)

import {buttonBehaviorComponent2} from './button-behavior2'
AFRAME.registerComponent('button-behavior2', buttonBehaviorComponent2)

AFRAME.registerComponent('custom-capture-btn', {
  init() {
    const btn = document.getElementById('recorder-button')
    btn.innerHTML = `<img id="icon" src=${require('./assets/camera.png')}>`
  },
})

import './main.css'

import {gestureSelectorComponent} from './gesture-selector'
AFRAME.registerComponent('gesture-selector', gestureSelectorComponent)

import {highlightComponent} from './highlight'
AFRAME.registerComponent('highlight', highlightComponent)

import './index.css'

import {splashImageComponent} from './splash-image'
AFRAME.registerComponent('splash-image', splashImageComponent)


 AFRAME.registerComponent('no-cull', { init() { this.el.addEventListener('model-loaded', () => { this.el.object3D.traverse(obj => obj.frustumCulled = false) }) }, })


This is index.css

.hidden {
  display: none !important;
}

#splashimage {
  z-index: 998;
  height: 100%;
  background-image: url('./assets/StixAndJones_Star.gif');
  position: relative;
  background-position: center;
  background-repeat: no-repeat;
  background-size: cover;
}

#start {
  /*z-index: 999;*/
  width: 50vw;
  height: auto;
  position: absolute;
  bottom: 5vh;
  left: 25vw;
  background-color: #ad50ff;
  border: 0.2em solid white;
  text-align: center;
  color: #fff;
  border-radius: 100px;
}

h2 {
  font-size: 1.3em;
  text-align: center;
  font-family: monospace;
}

:xr-overlay {
  /* the following styling is identical to :fullscreen */
  padding-left: 45px !important;
  padding-right: 45px !important;
}


This is main.css

a-frame: Custom Capture Button This project demonstrates how to customize the media recorder capture
  button. .recorder-container {
  all: unset;
  position: absolute;
  bottom: 18%;
  right: 30%;
  transform: translateX(50%);
  width: 200px;
  height: 60px;
  z-index: 1;
}

.progress-container {
  display: none;
}

#recorder-button {
  width: 100%;
  display: flex;
  background-color: rgb(255, 0, 255);
  align-items: center;
  justify-content: center;
  border-radius: 10px;
  font-family: 'Nunito', sans-serif;
  font-size: 2rem;
  color: rgb(225, 0, 255);
}

#icon {
  padding-right: 0px;
}

#btn {
  all: unset;
  position: absolute;
  bottom: 5vh;
  left: 25vw;
  transform: translateX(-50%);
  z-index: 1;
  background: rgb(251, 255, 0);
  color: rgb(0, 0, 0);
  font-family: 'Nunito', sans-serif;
  font-size: 0.9rem;
  border-radius: 15px;
  font-weight: bold;
  padding: 10px;
  cursor: pointer;
}

#btn:active {
  background-color: rgb(88, 86, 214);
}

#btn2 {
  all: unset;
  position: absolute;
  bottom: 5%;
  right: 3%;
  transform: translateX(-50%);
  z-index: 1;
  background: rgb(0, 174, 255);
  color: rgb(0, 0, 0);
  font-family: 'Nunito', sans-serif;
  font-size: 0.9rem;
  border-radius: 15px;
  font-weight: bold;
  padding: 10px;
  cursor: pointer;
}

#btn2:active {
  background-color: rgb(88, 86, 214);
}


The pops that happen on my phone happen on the simulator as well. The skulls just pop over somewhere else. The shadow becomes disengaged from the floor after the pops.

Our basic gestur selector can’t work with mulitple instances of models in the scene as it listens to clicks on the entire screen. If you want to move multiple objects you need to reference our sample project here:

You can see it reset and pop here
 are you not getting that when you go to the project and then move a skull around?

> Movie of skull pop and reset after scaling it

Yep that is what I referenced. I will check the code again


I rebuilt the scene using a clone of the AFrame Gesture selector

This is what I used on the careBears

I thought maybe the eyes changing color was connected to the realtime reflections, but I recorded a video with and without the module and you can see the eyes turn a glowing white.

I think the highlight for the move and drag function is causing the eyes to change color after it has been moved?

Maybe this clash is connected to the popping the skulls around?

> No Reflections

> Reflections

>Several Skull Movies Showing pops and eyes changing colors

Hey support and Ian :slightly_smiling_face:

I really wanted to announce this collection today on Linkedin but I cannot until I get to the bottom of this glitch. :melting_face: if I have to remove the gesture I will but let me know if there is anything else I can provide to help you help me debug this?

Here is a link to the announcement for the skulls on my website where you can click between each if need be.

I will reinstating the white skull back to where it was before I started messing with it today to debug.

I see that you have

xrextras-tap-recenter

In your a-frame scene I would suggest removing this as this re-centers the entire scene which is what is causing the jumping issue.

In the gesture selector component we are setting a highlight color to the object as shown in the examples project

If you don’t want the hgihlight to happen on selection of the object you can remove or comment out line 49 in the component script.

Along with seting highlight to false in the component

gesture-selector="addHoldDrag: true; addTwoFingerRotate: true; addPinchScale: true; addHighlight: false"

OMG Ian I know I said this before but I love you!

I have no idea where that “recenter” bit came from but that fixed it! :star_struck:

I like the highlight because it lets the user know they grabbed the object, but


I did not collapse the “eye” into the skull as a single object like I did for the careBears project and that might be why the highlight lingers after you let go of it? :thinking:

I am dumping the highlight for now


Anyways cheers to you! It is fixed!!! I am jumping up and down so happy
THANK YOU!

3 Likes