Hi, Does anyone know how to export the bones in Blender to glb, cause the bone data is not showing in https://gltf-viewer.donmccurdy.com/. If I successfully export the model with bones, how can I use it in 8thwalls hand tracking so it can track the fingers? I read about three.js SkinnedMesh but I don’t know where to start, I’m not familiar with three.js. Please help.
Hello @Lester_Ng
I would suggest reaching out to the Blender forums or community for specifics on correctly exporting a 3D model with bones.
For 8th Wall you will need to use Three.js SkinnedMesh which is a component in Three.js that’s used for models with bones. It allows the mesh to deform based on the bone animations.
More information can be found here:
https://threejs.org/docs/#api/en/objects/SkinnedMesh
It also may be benificial to start with a simple model to test and verify if this works first. You can find some basic hand models for A-frame here:|
Thank you for the suggestion, but I do have a problem. My model is not showing, I tried loading it in the three.js editor, and https://gltf-viewer.donmccurdy.com/, but my model is showing and in the three.js editor, I see the bones and can control it but not in 8thwall’s editor. Here is my threejs-scene-init.js script, I don’t know if I’m doing it right.
// Build a pipeline module that initializes and updates the three.js scene based on handcontroller
// events.
const initScenePipelineModule = () => {
let canvas_
let modelGeometry_
let handMesh_ = null
const handKind = 2
let scene
let camera
let renderer
const buildHand = (modelGeometry) => {
const hand = new THREE.Object3D() // This container will hold SkinnedMesh
hand.visible = true
// Load the gauntlet model
const loader = new THREE.GLTFLoader()
loader.load(require('./assets/Models/ironman_rigged-fixed.glb'), (gltf) => {
const gauntletMesh = gltf.scene.children.find(child => child.isSkinnedMesh)
if (gauntletMesh) {
hand.add(gauntletMesh)
handMesh_ = gauntletMesh
handMesh_.visible = true // Initially set to false, will be controlled by show/hide
handMesh_.scale.set(1, 1, 1)
}
}, undefined, (error) => {
console.error('An error happened:', error)
})
// Update geometry on each frame with data from the hand controller.
const show = (event) => {
const {attachmentPoints} = event.detail
// Check if handMesh_ and handMesh_.skeleton are defined
if (!handMesh_ || !handMesh_.skeleton) return
handMesh_.visible = true
// Iterate over each attachment point and update the corresponding bone
Object.entries(attachmentPoints).forEach(([key, attachment]) => {
const bone = handMesh_.skeleton.getBoneByName(key)
if (bone) {
const localPosition = new THREE.Vector3().fromArray(attachment.position)
bone.position.copy(localPosition)
if (attachment.rotation) {
const quaternion = new THREE.Quaternion().setFromEuler(new THREE.Euler(
attachment.rotation.x,
attachment.rotation.y,
attachment.rotation.z,
'XYZ'
))
bone.quaternion.copy(quaternion)
}
}
})
handMesh_.skeleton.update()
}
// Hide all objects.
const hide = () => {
if (handMesh_) {
handMesh_.visible = true
}
}
return {
object3d: hand,
show,
hide,
}
}
const init = ({canvas, detail}) => {
canvas_ = canvas_ || canvas
modelGeometry_ = modelGeometry_ || detail
if (!(canvas_ && modelGeometry_)) {
return
}
const xrScene = XR8.Threejs.xrScene()
scene = xrScene.scene
camera = xrScene.camera
renderer = xrScene.renderer
THREE.WebGLRenderer.sortObjects = false
renderer.setSize(window.innerWidth, window.innerHeight)
renderer.setClearColor(0x000000)
// add lights.
const targetObject = new THREE.Object3D()
targetObject.position.set(0, 0, -1)
scene.add(targetObject)
const directionalLight = new THREE.DirectionalLight(0xffffff, 0.8)
directionalLight.castShadow = true
directionalLight.position.set(0, 0.25, 0)
directionalLight.target = targetObject
scene.add(directionalLight)
const bounceLight = new THREE.HemisphereLight(0xffffff, 0xffffff, 0.5)
scene.add(bounceLight)
// add hand mesh to the scene
handMesh_ = buildHand(modelGeometry_)
scene.add(handMesh_.object3d)
// prevent scroll/pinch gestures on canvas.
canvas_.addEventListener('touchmove', event => event.preventDefault())
camera.position.set(0, 0, 5)
const light = new THREE.PointLight(0xffffff, 1, 100)
light.position.set(0, 0, 10)
scene.add(light)
function animate() {
requestAnimationFrame(animate)
renderer.render(scene, camera)
}
animate()
// Add a debugging mesh
const geometry = new THREE.BoxGeometry(1, 1, 1)
const material = new THREE.MeshBasicMaterial({color: 0x00ff00})
const cube = new THREE.Mesh(geometry, material)
cube.position.set(0, 0, 0)
scene.add(cube)
}
const onDetach = () => {
canvas_ = null
modelGeometry_ = null
}
const show = (event) => {
handMesh_.show(event)
}
const hide = (event) => {
handMesh_.hide()
}
return {
name: 'threejsinitscene',
onAttach: init,
onDetach,
listeners: [
{event: 'handcontroller.handloading', process: init},
{event: 'handcontroller.handfound', process: show},
{event: 'handcontroller.handupdated', process: show},
{event: 'handcontroller.handlost', process: hide},
],
}
}
export {initScenePipelineModule}
You can reference this project
Your hand model may be too large or too small and or not in the camera view. I would suggest adding a a-sphere or a-box and getting to see that first. Then placing your model in that location to better understand how the scene is set up.
1 Like
I managed to control the bones by moving my finger. I tried it only for the index finger for now, but my problem now is that the finger’s model is not straight. Is this a model problem? but when I tried to manually control using a script, I managed to rotate it properly, I only have these bones in my model**(wrist, palm, indexBaseJoint, indexMidJoint, indexUpper, indexNail)** Do I need all the joints? Maybe I need to normalize the rotation that I’m getting from 8thwall’s data?
this is one of the log data I got in 8thwalls hand transform rotation
Applied rotation to base: {isQuaternion: true, _x: -0.6269103288650513, _y: -0.25340738892555237, _z: 0.5236185789108276, _w: -0.5182583332061768, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
8:33:09 PM
Applied rotation to mid: {isQuaternion: true, _x: -0.2377292960882187, _y: -0.21839633584022522, _z: 0.6775899529457092, _w: -0.6608023643493652, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
8:33:09 PM
Applied rotation to upper: {isQuaternion: true, _x: -0.15797626972198486, _y: -0.22711029648780823, _z: 0.7262154221534729, _w: -0.6293453574180603, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
8:33:09 PM
Applied rotation to nail: {isQuaternion: true, _x: 0.40335023403167725, _y: -0.6941572427749634, _z: -0.22852498292922974, _w: -0.5506638884544373, _onChangeCallback: [Function: function(){t.setFromQuaternion(n,void 0,!1)}]}
Here’s a video on what’s happening:
Were you able to reach out to the A-frame slack or three.js for more information?
@joshmao do you have any insight here?
Yeah the video is strange…
I’m not super familiar with skinned meshes either but you could try only using the Joints? (BaseJoint, MidJoint, TopJoint) or only using the bones (Lower, Upper, Nail).
I didn’t reach out to A-Frame Slack or Three.js. I’ll try to reach out to them.
I see, thanks for the input. I’ll try those bones and see if that fixes it. What I did was simply copying the hand transform rotation in the hand pipeline to the skinned mesh bones of my model.