I've had a similar conversation a few times. The way we do audio chat right now breaks the 'cocktail party effect', dumbing all interactions down to not much better than walkie-talkie levels. I think if you go down this road far enough you might even be able to emulate the 'hallway track' effect of in-person conferences, by blending in other conversations into the audio.