[Image Targets]Video playback is delayed or doesn't start after image detection

What I want to do

I want the video to play automatically and immediately after an image is detected.

What’s happening now

Sometimes the video plays with a delay, and sometimes it doesn’t start at all, even though the image is detected and no errors are shown in the console.

:wrench: My setup

  • Platform: Niantic Studio (8thWall)

  • Video

    • format: .mp4
    • URL / import: import
    • size: 10.9 MB
    • Video length: 28 seconds
  • Source Code:

  • import * as ecs from '@8thwall/ecs'
    
    ecs.registerComponent({
      name: 'Movie Movement on Tracking',
      schema: {
        movieEntity: ecs.eid,
        cardFrame: ecs.eid,
        imageTargetName: ecs.string,
      },
      stateMachine: ({world, eid, schemaAttribute}) => {
        ecs.defineState('default')
          .initial()
    
          .listen(world.events.globalId, 'reality.imagefound', (e) => {
            const delay = 800
            const {movieEntity, cardFrame} = schemaAttribute.get(eid)
    
            ecs.PositionAnimation.set(world, movieEntity, {
              fromZ: 0,
              toZ: -0.5,
              duration: 800,
              loop: false,
              easeIn: true,
              easeOut: false,
              easingFunction: 'Bounce',
            })
    
            ecs.PositionAnimation.set(world, cardFrame, {
              fromZ: 0,
              toZ: -0.4999,
              duration: 800,
              loop: false,
              easeIn: true,
              easeOut: false,
              easingFunction: 'Bounce',
            })
    
            world.time.setTimeout(() => {
              ecs.PositionAnimation.set(world, movieEntity, {
                fromZ: -0.5,
                toZ: 0.5,
                duration: 1000,
                loop: false,
                easeIn: true,
                easeOut: false,
                easingFunction: 'Sinusoidal',
              })
            }, delay)
    
            world.time.setTimeout(() => {
              ecs.PositionAnimation.set(world, cardFrame, {
                fromZ: -0.5,
                toZ: 5,
                duration: 1000,
                loop: false,
                easeIn: true,
                easeOut: false,
                easingFunction: 'Sinusoidal',
              })
    
              ecs.ScaleAnimation.set(world, movieEntity, {
                fromX: 1.3,
                fromY: 1,
                fromZ: 1.3,
                toX: 2,
                toY: 2,
                toZ: 2,
                duration: 1000,
                loop: false,
                easeIn: true,
                easeOut: false,
                easingFunction: 'Sinusoidal',
              })
            }, delay)
    
            world.time.setTimeout(() => {
              ecs.VideoControls.set(world, movieEntity, {
                paused: false,
              })
            }, delay * 2)
    
            world.time.setTimeout(() => {
              ecs.PositionAnimation.set(world, movieEntity, {
                fromZ: 0.5,
                toZ: 0,
                duration: 1000,
                loop: false,
                easeIn: true,
                easeOut: false,
                easingFunction: 'Quadratic',
              })
    
              ecs.ScaleAnimation.set(world, movieEntity, {
                fromX: 2,
                fromY: 2,
                fromZ: 2,
                toX: 1.3,
                toY: 1,
                toZ: 1.3,
                duration: 1000,
                loop: false,
                easeIn: true,
                easeOut: false,
                easingFunction: 'Sinusoidal',
              })
    
              ecs.PositionAnimation.set(world, cardFrame, {
                fromZ: 5,
                toZ: 0.001,
                duration: 1000,
                loop: false,
                easeIn: true,
                easeOut: false,
                easingFunction: 'Sinusoidal',
              })
            }, 28000)
          })
    
          .listen(world.events.globalId, 'reality.imagelost', () => {
            const {movieEntity} = schemaAttribute.get(eid)
    
            ecs.VideoControls.set(world, movieEntity, {
              paused: true,
            })
          })
      },
    })
    
    
  • Image detection (reality.imagefound) is working

  • The video entity appears correctly

  • Animations on the video entity are working

:red_question_mark: Questions

  1. Do I need to preload the video or check if it’s ready before calling paused: false?

  2. Is there a way to check if the video is fully loaded or ready to play in Niantic Studio?

  3. Is ecs.VideoControls.set(...paused: false) enough to trigger playback, or should I do something else beforehand?

  4. Does the user need to tap the smartphone screen to automatically play the video?

I’m still new to Niantic Studio and ECS, so any example or best practice would be very helpful.

Hi, welcome to the forum!

I’m not sure preloading would change anything here, it’s hard to say without seeing the exact issue. Could you record and post a video?

Additionally check out the Video events on our documentation here: Asset Events | 8th Wall

Also I’ve added a Video space as a part of the Image Target sample project. I would recommend taking a look at that as well..

EDIT: I’ve just confirmed with the engineering team that videos with Audio will NOT play unless the user has interacted with the page through a screen tap, etc. If this is a problem I would suggest adding a Start Screen to your project to capture the gesture.