Whenever audio starts, there’s a couple second delay before I can hear it. Haven’t been able to solve that so I just live with it.
That can be because of a power saving feature - basically PulseAudio puts your sound card to sleep when nothing is playing, and then there’s a bit of lag before it wakes up. In my case it was really annoying because I use the optical output, so when PulseAudio put the sound card to sleep, my receiver would also go to sleep after a bit, and resulted in quite a bit of a delay when it was time to get it come back up.
This fixed it for me (see part 4.8): https://wiki.archlinux.org/title/PulseAudio/Troubleshooting




I don’t know if it’s the best way, but I just download and install it as Linux is officially supported.