The Ogg file format is a right mess. They decided to break it into packet sized chunks for streaming, and that made it very annoying to parse or seek.