There is something I don't understand; maybe someone here can explain:
Sound travels at about 340 m/s (in a typical room). That means it travels about 3.4 metres in 10 milliseconds. Therefore another way to get a 10 millisecond problem is to stand 3.4 metres from the orchestra.
Most people sit farther than 3.4 metres from the orchestra, yet they don't complain about a lag between when the violin bow moves and the sound is heard. Why not?
(The speed of light is so fast that we can assume it's effectively infinite for the purposes of this argument.)
The audience doesn't need instant feedback and low latency, the performers do. If you listen to a classic rock album, you're hearing the sounds with a "latency" decades after they've been played. But the playing is cohesive and tight. If one of the guitarists was consistently 50ms off, you would notice it.
> Most Android apps have more than 100 ms of audio output latency, and more than 200 ms of round-trip (audio input to audio output) latency.
It's also much more than just 10ms latency. I play with digital instruments all the time, the latency can be as high as 15ms before I can tell. I don't know if an audience can perceive a 15ms latency, especially because you tend to "play early" to have notes land on time. But it's very upsetting for performing.
I haven't tried playing with music apps on Android recently, but when I did, the latency was not just long, but inconsistent, and would result in stuttering in the audio.
Probably because they're not directly interacting with it, they're just watching it, which makes the delay a lot less noticeable. The 10ms delay is more of a problem for interactive apps where you eg. touch the screen and expect it to instantly make a sound. In this kind of feedback loop, even small delays are distracting and it becomes difficult to keep a beat because the lag is perceptible. If you tried to remotely play an instrument from 3.4 meters away, you would probably notice it too.
Clapping synchronisation is actually a well studied spontaneous occurrence, that just takes O(1) periods, see this article for some analysis:
http://arxiv.org/pdf/cond-mat/0006423v1.pdf
Audiences have a natural tendency to slow down. They're mostly not musicians and I imagine their clapping as a response to the music (and the other clappers) rather than internalizing the music as a performer would.
If you start from a perfectly synchronized audience and assume they clap reactively, you'd expect them to delay each of their claps by ~1/2 speed of sound * width of the room. A good example for why on music you need a good rhythm (internal constant frequency driver) instead of acting reactively to perform well.
Yes, the audience will slow down regardless. Either the band slows down with them or they keep the same tempo and the clapping gets screwed up. It's frustrating as a performer trying to fight the audience and the only solution I know is to have the house levels sufficiently loud that the audience continually adjusts.
Lag is not an issue for the listener, it's an issue for the performer. An extra 10ms of lag when wearing headphones and playing a keyboard can make you feel completely disconnected from your instrument.
I'm a musician, 10ms latency would be fine, however as they note, most Android apps have 100ms latency, or 200ms round-trip latency. That is definitely not usable.
The funny part is that pipe organists would laugh at 100ms latency and say "cry me a river". With the pneumatics combined with the distance of the pipes from the performer, pipe organ latencies can be in the 200 to 500ms range. I asked my sister how she managed it and she said it was just a learned skill. She had to learn to completely tune out what she was hearing and play with the beat and music completely internalized.
The difference, of course, is that organists are usually not syncing up to other instruments. If there are other instruments involved, they tend to sync up with the organ.
They also mention that those tests were made in the best case scenario (disabling noise correction, etc. for the sake of speed) with the best performing mobile, and by following Google's low latency guidelines. Most Android apps don't have the first two luxuries in the general case, and (apparently) don't bother with the last one, thus the much higher latencies in practice.
I am too lazy to try to find the exact numbers and versions but back in the Android 1.x, 2.x days, the latency was in the 200 ms ballpark.
So things have improved a lot since then (even though there is still a lot of ground to cover)
Our brains perceive what we hear trailing what we see by small amounts to be completely normal. If you show people a video of an orchestra concert with the sound of a violin coming 50ms before seeing the bow move, most would immediately notice something is off.
Musicians performing together, however, is a much harder problem than just listening. Ask anyone who has ever performed in a DCI-style drum corps, they will tell you compensating for hearing someone on the other side of field 200ms or so late is incredibly difficult.
Probably because in this orchestra example the feedback is purely passive (you didn't take any action), whereas with Android the delays are after an action you took, so it's psychologically jarring (seeing the response to your action immediately, and hearing the response roughly 10ms later). Just my guess.
You are exactly right. This is why orchestras need a conductor who provides a visual signal for tempo, and marching bands have a drum major with a huge baton, while rock bands can just listen to the bass drum.
Android' problem is that it has a 100-200 ms lag. The stated 10 ms is the goal.
Also, most people don't notice the lag between the instruments of an orchestra because the instruments are close to each other. Their distance to the listener is not relevant.
tldr: Brain is inherently parallel, nothing happens in sync. In order to make sense of the outside world higher level functions are presented with artificially coordinated stimuli.
If you're way in the back in a big concert hall, you can definitely notice the lag, but it's not a big deal. The problem would be if different instruments reach your ears at different times, and since the closest seat is still a considerable distance from the audience, you're not going to be hearing sounds more than about ~20ms apart. The reverb of the hall also mushes everything together.
The problem you describe is a very real problem, however, for the musicians themselves. If you're sitting in a big orchestra and you try to index your playing off someone sitting on the other end of the orchestra, you will not be in time. That's why there's a conductor, so that the orchestra can be synchronized at the speed of light rather than of sound.
IANA neuroscientist but it seems reasonable that the brain will fix up small errors. It has to anyway, because it has its own input latency. Besides, the article is referring to discrepancies between multiple audio tracks, which don't depend on vision at all.
For a real-world example, listen to 2 TVs several meters apart and tuned to the same channel. At least with OTA or cable you can expect them to be playing ~simultaneously but the skew between the received signals is easily perceptible.
>Besides, the article is referring to discrepancies between multiple audio tracks, which don't depend on vision at all.
They do depend upon vision or touch if it is an interactive app.
>For a real-world example, listen to 2 TVs several meters apart and tuned to the same channel. At least with OTA or cable you can expect them to be playing ~simultaneously but the skew between the received signals is easily perceptible.
I believe what you're experiencing is the difference in decoding latency between different models of TV set. Several meters (3m) represents only about ~9ns (practically, low tens of ns if the cables are longer than necessary) maximum delay. It would not be directly perceptible by a person. Signal delay in a cable is ~1ns/ft.
>They do depend upon vision or touch if it is an interactive app.
Discrepancies between tracks are totally unrelated to the visuals. It's much easier to tell if two sounds are synced than a sound and a visual.
> Signal delay in a cable
I don't know what comment you read but it's not the one you replied to. Same model, synchronized visuals, easy to hear audio desync when you're closer to one.
Sound travels at about 340 m/s (in a typical room). That means it travels about 3.4 metres in 10 milliseconds. Therefore another way to get a 10 millisecond problem is to stand 3.4 metres from the orchestra.
Most people sit farther than 3.4 metres from the orchestra, yet they don't complain about a lag between when the violin bow moves and the sound is heard. Why not?
(The speed of light is so fast that we can assume it's effectively infinite for the purposes of this argument.)