# Controlling Rigged Hand Model

**URL:** <https://forum.8thwall.com/t/controlling-rigged-hand-model/550>\
**Category:** Technical Support\
**Tags:** hand-tracking\
**Created:** [March 2, 2024, 10:56am UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550 "2024-03-02T10:56:40Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lester\_Ng](https://avatars.discourse-cdn.com/v4/letter/l/67e7ee/32.png) [@Lester\_Ng](https://forum.8thwall.com/u/Lester_Ng)\
**Post date:** [March 2, 2024, 10:56am UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/1 "2024-03-02T10:56:40Z")

</div>

Hi, Does anyone know how to export the bones in Blender to glb, cause the bone data is not showing in [https://gltf-viewer.donmccurdy.com/](https://gltf-viewer.donmccurdy.com/). If I successfully export the model with bones, how can I use it in 8thwalls hand tracking so it can track the fingers? I read about three.js SkinnedMesh but I don’t know where to start, I’m not familiar with three.js. Please help.

 ![Screenshot 2024-03-02 184722](https://us1.discourse-cdn.com/flex019/uploads/x8thwall/original/1X/1023a5615fd7508a65fc5a4b23b2fad5146095d1.png)

---

<div class="post-metadata">

**Author:** ![Ian](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.8thwall.com/ian/32/430_2.png) [@Ian](https://forum.8thwall.com/u/Ian)\
**Post date:** [March 5, 2024, 7:50pm UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/2 "2024-03-05T19:50:43Z")

</div>

Hello @Lester_Ng

I would suggest reaching out to the Blender forums or community for specifics on correctly exporting a 3D model with bones.

> **[Community — blender.org](https://www.blender.org/community/)**
>
> Anywhere you are, there's a community for you. Find independent Blender groups all around the globe.

For 8th Wall you will need to use Three.js SkinnedMesh which is a component in Three.js that’s used for models with bones. It allows the mesh to deform based on the bone animations.

More information can be found here:

[https://threejs.org/docs/#api/en/objects/SkinnedMesh](https://threejs.org/docs/#api/en/objects/SkinnedMesh)

It also may be benificial to start with a simple model to test and verify if this works first. You can find some basic hand models for A-frame here:|

> **[A-Frame – Make WebVR](https://aframe.io/docs/1.5.0/components/hand-tracking-controls.html)**
>
> A web framework for building virtual reality experiences. Make WebVR with HTML and Entity-Component. Works on Vive, Rift, desktop, mobile platforms.

---

<div class="post-metadata">

**Author:** ![Lester\_Ng](https://avatars.discourse-cdn.com/v4/letter/l/67e7ee/32.png) [@Lester\_Ng](https://forum.8thwall.com/u/Lester_Ng)\
**Post date:** [March 19, 2024, 7:47am UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/3 "2024-03-19T07:47:22Z")

</div>

Thank you for the suggestion, but I do have a problem. My model is not showing, I tried loading it in the three.js editor, and [https://gltf-viewer.donmccurdy.com/](https://gltf-viewer.donmccurdy.com/), but my model is showing and in the three.js editor, I see the bones and can control it but not in 8thwall’s editor. Here is my threejs-scene-init.js script, I don’t know if I’m doing it right.

```auto
// Build a pipeline module that initializes and updates the three.js scene based on handcontroller
// events.
const initScenePipelineModule = () => {
  let canvas_
  let modelGeometry_
  let handMesh_ = null
  const handKind = 2
  let scene
  let camera
  let renderer

  const buildHand = (modelGeometry) => {
    const hand = new THREE.Object3D() // This container will hold SkinnedMesh
    hand.visible = true

    // Load the gauntlet model
    const loader = new THREE.GLTFLoader()
    loader.load(require('./assets/Models/ironman_rigged-fixed.glb'), (gltf) => {
      const gauntletMesh = gltf.scene.children.find(child => child.isSkinnedMesh)
      if (gauntletMesh) {
        hand.add(gauntletMesh)
        handMesh_ = gauntletMesh
        handMesh_.visible = true // Initially set to false, will be controlled by show/hide
        handMesh_.scale.set(1, 1, 1)
      }
    }, undefined, (error) => {
      console.error('An error happened:', error)
    })

    // Update geometry on each frame with data from the hand controller.
    const show = (event) => {
      const {attachmentPoints} = event.detail

      // Check if handMesh_ and handMesh_.skeleton are defined
      if (!handMesh_ || !handMesh_.skeleton) return

      handMesh_.visible = true

      // Iterate over each attachment point and update the corresponding bone
      Object.entries(attachmentPoints).forEach(([key, attachment]) => {
        const bone = handMesh_.skeleton.getBoneByName(key)
        if (bone) {
          const localPosition = new THREE.Vector3().fromArray(attachment.position)
          bone.position.copy(localPosition)

          if (attachment.rotation) {
            const quaternion = new THREE.Quaternion().setFromEuler(new THREE.Euler(
              attachment.rotation.x,
              attachment.rotation.y,
              attachment.rotation.z,
              'XYZ'
            ))
            bone.quaternion.copy(quaternion)
          }
        }
      })

      handMesh_.skeleton.update()
    }

    // Hide all objects.
    const hide = () => {
      if (handMesh_) {
        handMesh_.visible = true
      }
    }

    return {
      object3d: hand,
      show,
      hide,
    }
  }

  const init = ({canvas, detail}) => {
    canvas_ = canvas_ || canvas
    modelGeometry_ = modelGeometry_ || detail

    if (!(canvas_ && modelGeometry_)) {
      return
    }

    const xrScene = XR8.Threejs.xrScene()
    scene = xrScene.scene
    camera = xrScene.camera
    renderer = xrScene.renderer

    THREE.WebGLRenderer.sortObjects = false

    renderer.setSize(window.innerWidth, window.innerHeight)
    renderer.setClearColor(0x000000)

    // add lights.
    const targetObject = new THREE.Object3D()
    targetObject.position.set(0, 0, -1)
    scene.add(targetObject)

    const directionalLight = new THREE.DirectionalLight(0xffffff, 0.8)
    directionalLight.castShadow = true
    directionalLight.position.set(0, 0.25, 0)
    directionalLight.target = targetObject
    scene.add(directionalLight)

    const bounceLight = new THREE.HemisphereLight(0xffffff, 0xffffff, 0.5)
    scene.add(bounceLight)

    // add hand mesh to the scene
    handMesh_ = buildHand(modelGeometry_)
    scene.add(handMesh_.object3d)

    // prevent scroll/pinch gestures on canvas.
    canvas_.addEventListener('touchmove', event => event.preventDefault())

    camera.position.set(0, 0, 5)

    const light = new THREE.PointLight(0xffffff, 1, 100)
    light.position.set(0, 0, 10)
    scene.add(light)

    function animate() {
      requestAnimationFrame(animate)
      renderer.render(scene, camera)
    }
    animate()

    // Add a debugging mesh
    const geometry = new THREE.BoxGeometry(1, 1, 1)
    const material = new THREE.MeshBasicMaterial({color: 0x00ff00})
    const cube = new THREE.Mesh(geometry, material)
    cube.position.set(0, 0, 0)
    scene.add(cube)
  }

  const onDetach = () => {
    canvas_ = null
    modelGeometry_ = null
  }

  const show = (event) => {
    handMesh_.show(event)
  }
  const hide = (event) => {
    handMesh_.hide()
  }

  return {
    name: 'threejsinitscene',
    onAttach: init,
    onDetach,
    listeners: [
      {event: 'handcontroller.handloading', process: init},
      {event: 'handcontroller.handfound', process: show},
      {event: 'handcontroller.handupdated', process: show},
      {event: 'handcontroller.handlost', process: hide},
    ],
  }
}

export {initScenePipelineModule}

```

---

<div class="post-metadata">

**Author:** ![Ian](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.8thwall.com/ian/32/430_2.png) [@Ian](https://forum.8thwall.com/u/Ian)\
**Post date:** [March 19, 2024, 6:06pm UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/4 "2024-03-19T18:06:53Z")

</div>

You can reference this project

> **[Real-Time Hand Physics | Ian Curtis | 8th Wall](https://www.8thwall.com/ian/hand-real-time-physics/code/)**
>
> Real-time physics using ammo.js and hand tracking

Your hand model may be too large or too small and or not in the camera view. I would suggest adding a a-sphere or a-box and getting to see that first. Then placing your model in that location to better understand how the scene is set up.

---

<div class="post-metadata">

**Author:** ![Lester\_Ng](https://avatars.discourse-cdn.com/v4/letter/l/67e7ee/32.png) [@Lester\_Ng](https://forum.8thwall.com/u/Lester_Ng)\
**Post date:** [April 18, 2024, 12:47pm UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/5 "2024-04-18T12:47:55Z")

</div>

I managed to control the bones by moving my finger. I tried it only for the index finger for now, but my problem now is that the finger’s model is not straight. Is this a model problem? but when I tried to manually control using a script, I managed to rotate it properly, I only have these bones in my model\*\*(wrist, palm, indexBaseJoint, indexMidJoint, indexUpper, indexNail)\*\* Do I need all the joints? Maybe I need to normalize the rotation that I’m getting from 8thwall’s data?

this is one of the log data I got in 8thwalls hand transform rotation

```auto
Applied rotation to base: {isQuaternion: true, _x: -0.6269103288650513, _y: -0.25340738892555237, _z: 0.5236185789108276, _w: -0.5182583332061768, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
8:33:09 PM
Applied rotation to mid: {isQuaternion: true, _x: -0.2377292960882187, _y: -0.21839633584022522, _z: 0.6775899529457092, _w: -0.6608023643493652, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
8:33:09 PM
Applied rotation to upper: {isQuaternion: true, _x: -0.15797626972198486, _y: -0.22711029648780823, _z: 0.7262154221534729, _w: -0.6293453574180603, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
8:33:09 PM
Applied rotation to nail: {isQuaternion: true, _x: 0.40335023403167725, _y: -0.6941572427749634, _z: -0.22852498292922974, _w: -0.5506638884544373, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}

```

Here’s a video on what’s happening:

[![](https://us1.discourse-cdn.com/flex019/uploads/x8thwall/original/1X/7e5ceedf0421a285622fa41e086a8ca2de36c26c.jpeg "Gauntlet 8thwall") ](https://www.youtube.com/watch?v=pr4bHRXoBLk)

---

<div class="post-metadata">

**Author:** ![Ian](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.8thwall.com/ian/32/430_2.png) [@Ian](https://forum.8thwall.com/u/Ian)\
**Post date:** [April 18, 2024, 8:32pm UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/6 "2024-04-18T20:32:18Z")

</div>

Were you able to reach out to the A-frame slack or three.js for more information?

@joshmao do you have any insight here?

---

<div class="post-metadata">

**Author:** ![joshmao](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.8thwall.com/joshmao/32/660_2.png) [@joshmao](https://forum.8thwall.com/u/joshmao)\
**Post date:** [April 18, 2024, 8:52pm UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/7 "2024-04-18T20:52:42Z")

</div>

Yeah the video is strange…

I’m not super familiar with skinned meshes either but you could try only using the Joints? (BaseJoint, MidJoint, TopJoint) or only using the bones (Lower, Upper, Nail).

---

<div class="post-metadata">

**Author:** ![Lester\_Ng](https://avatars.discourse-cdn.com/v4/letter/l/67e7ee/32.png) [@Lester\_Ng](https://forum.8thwall.com/u/Lester_Ng)\
**Post date:** [April 19, 2024, 5:28am UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/8 "2024-04-19T05:28:40Z")

</div>

I didn’t reach out to A-Frame Slack or Three.js. I’ll try to reach out to them.

---

<div class="post-metadata">

**Author:** ![Lester\_Ng](https://avatars.discourse-cdn.com/v4/letter/l/67e7ee/32.png) [@Lester\_Ng](https://forum.8thwall.com/u/Lester_Ng)\
**Post date:** [April 19, 2024, 5:31am UTC](https://forum.8thwall.com/t/controlling-rigged-hand-model/550/9 "2024-04-19T05:31:21Z")

</div>

I see, thanks for the input. I’ll try those bones and see if that fixes it. What I did was simply copying the hand transform rotation in the hand pipeline to the skinned mesh bones of my model.