Niantic Studio: Audio not working properly in iPhone Safari

[Ken_Mitsui]

I am building an AR using Niantic studio.
What I want to do is to make a bouncing sound when a physics simulated ball entity collides with another entity.
I would like to know how to get the sound to sound like it does on the iPhone!

In ecs.Audio, you can’t specify the playback start position like play( delay ) in Three.js Audio, right?
So, in order to always set the playback position to 0, I create a new entity for Audio and play it at the timing I want it to sound as follows.

const audioEid = world.createEntity()

      if (Position.has(world, eid)) {
        const blockPos = Position.cursor(world, eid)
        // 2-2. set the same position for the generated entity
        ecs.Position.set(world, audioEid, {
          x: blockPos.x,
          y: blockPos.y, { x: blockPos.x, y: blockPos.y, z: blockPos.z, z: blockPos.z, z: blockPos.z, z: blockPos.z
          z: blockPos.z, }
        })
      }

      ecs.Audio.set(world, audioEid, {
        url: 'assets/ball.mp3', // specify actual sound effect file
        paused: false, // if false, playback starts
        loop: false,
        volume: 1.0, }
      })

      setTimeout(() => {
        world.deleteEntity(audioEid)
      }, 1000)

The sound is normal on the simulator, but when viewed on an iPhone safari, the sound is often not heard.

• Even when it does, the timing of the sound is abnormally slow compared to the simulator (slow loading?).
• The first time the sound is played, but not after the second time.
• There is no error in the console.
• It often does not sound at all.

I saw inquiries about ecs.Audio

How to use ecs.Audio?

Hello, I have some questions regarding ecs.Audio please. (1) I have an entity called GameOverScreen with a custom component responsible for playing game-over audio when certain conditions are met. If the game over happens quickly after starting the game, the audio plays as expected. However, if it takes some time before the game over occurs, the audio doesn’t play. I’m not sure why or if there is anything I should check which could be the source of this problem? Here’s my code: In the Game …

Niantic Studio: Audio does not work properly

Hello I’m using Studio, and I have a mechanic to control the volume up and down by making a face gesture. It works in the editor, but on iOS, it doesn’t work in Chrome or Safari—the volume always stays up. I also tested it on an Android device, and it works as expected. Here’s my logic: const onPowerUp = (event) => { ecs.Audio.set(world, sfxGame, { volume: 1 }); } const onDebuff = (event) => { ecs.Audio.set(world, sfxGame, { volume: 0.25 }); } I also have background audio set to …

And I tried to make the sound play once with the interaction as shown below, but the above problem is not solved.

      startEventDispatcher = async () => {
    const audioEid = world_.createEntity()
    ecs.Audio.set(world_, audioEid, {
      url: 'assets/button.mp3', paused: false, { startEventDispatcher = async ( => { const audioEid = world_.createEntity()
      paused: false, loop: false, { url: 'assets/button.mp3', url: 'assets/button.mp3', loop: false, loop: false
      loop: false,
      volume: 0.1, }
    })
    world_.events.dispatch(eid_, 'start') // dispatch event when clicked
  }

  // add a click event listener
  uiContent.addEventListener('click', startEventDispatcher)

How can I fix it?
Could you help me?

[GeorgeButler]

Hi Ken,

I’d be happy to help you figure out the issues around Audio playback. Could you land your changes and share your project with the support workspace?

[Ken_Mitsui]

Thank you, George!
I shared this with support.
I hope you take a look at it when you have time.

[GeorgeButler]

Going to be taking a look at this with the team, I’ll update here with details.

[Ken_Mitsui]

Thank you very much!
Best regards.

[GeorgeButler]

Hey Ken,

Looks like this might be a bug in our implementation of audio. Can you confirm if the audio works as expected on another browser like Safari?

[Ken_Mitsui]

Thanks, George!
I can only verify this on my iPhone, but it is the same in safari and chrome.

[Netherstory_Team]

Have you found the solution?
I have the similar problems on Safari and Chrome still.

[GeorgeButler]

Can you land your changes and publish a staging version of your experience so I can test it on different devices?

[Ken_Mitsui]

To be honest, I had given up on this one.
I haven’t opened it in a while, but the sound still doesn’t seem to be properly produced in iPhone.

I published the latest version here.
bench.8thwall.app/create-play-challenge/
I’m also working on being able to clone it, if that helps anyone.

[Yusuke_Mizutani]

Hello. I am also experiencing the same issue and it’s causing me a lot of trouble. The problem doesn’t occur in the simulator or on Android devices, but it happens on iPhone (iOS). I want to play a sound effect every time the screen is touched, but the sound plays with a delay of 0.x seconds and sounds strange. Is this a feature of the iPhone? Is there any way to resolve this?

[GeorgeButler]

Hi, welcome to the forum!

Can you share the audio clip file here so I can examine the waveform?

[Yusuke_Mizutani]

Thank you for your message. Regarding the audio clip file in question, its format is mp3, and it seems that uploading it on this forum is not possible. Could you please let me know how I should proceed?

[GeorgeButler]

You can upload it to Google Drive or something similar where I can download it. Anywhere that doesn’t require login would be perfect.

[Yusuke_Mizutani]

I have uploaded it to the Google Drive at the following URL, so please check it.

SE_button_A.mp3

[GeorgeButler]

Looking at the waveform, it seems like you’ll need to trim the head and tail to remove any empty space before the effect starts and after it ends.

I’ve recorded a video of me doing that using Audacity.

[Yusuke_Mizutani]

Thank you for checking. As you advised, I removed any empty parts in Audacity as shown in the attached file and tested it, but although the delay was slightly reduced, there was almost no improvement. Is there any other solution?

[Yusuke_Mizutani]

// TestButton.ts
import {PlayAudio} from './AudioController'

import * as ecs from '@8thwall/ecs'

ecs.registerComponent({
  name: 'testButton',
  add: (world, component) => {
    const {eid} = component
    world.events.addListener(eid,
      ecs.input.SCREEN_TOUCH_START,
      () => {
        PlayAudio(world, eid)
        ecs.Ui.mutate(world, eid, (cursor) => {
          cursor.background = '#fff'
          cursor.color = '#000'
        })
        world.time.setTimeout(() => {
          ecs.Ui.mutate(world, eid, (cursor) => {
            cursor.background = '#000'
            cursor.color = '#fff'
          })
        }, 50)
      })
  },

})

// AudioController.ts
// Helper functions for controlling audio components
import * as ecs from '@8thwall/ecs'  // This is how you access the ecs library

// Given an entity id, play its audio component
const PlayAudio = (world, componentEID) => {
  if (ecs.Audio.has(world, componentEID)) {  // If the supplied entity has an Audio component...
    const audio = ecs.Audio.cursor(world, componentEID)  // Get a reference to that Audio component
    audio.paused = false  // And play the assigned audio source
  }
}

// Given an entity id, pause its audio component if it is playing
const PauseAudio = (world, componentEID) => {
  if (ecs.Audio.has(world, componentEID)) {
    const audio = ecs.Audio.cursor(world, componentEID)
    audio.paused = true
  }
}

// Given an entity id with an audio component, change its audio source
const ChangeAudio = (world, componentEID, newAudio) => {
  if (ecs.Audio.has(world, componentEID)) {
    console.log(newAudio)
    const audio = ecs.Audio.cursor(world, componentEID)
    audio.url = newAudio
  }
}

export {PauseAudio, PlayAudio, ChangeAudio}

I will send the code to identify the cause. I have set custom components Audio and TestButton.ts on the button so that when it is tapped, PlayAudio() is triggered.

[GeorgeButler]

Can you share the project with the support workspace so I can try it on our test devices?

[Yusuke_Mizutani]

I have shared the project test-tap-se to the support workspace, so please check it.