Extracting Your Own Location Information From Google - The Hard Way
api code google programming · 23 comments · 1,350 words · Viewed ~41,912 times.
Update! There's a new way to do this - read my latest post to find out how.
Two or three years ago, I was contacted by a recruiter from Google. They wanted to know if I'd be interested in working for the advertising giant.
We played the usual game of dancing around salary and schedules, when he dropped the bombshell - they were looking for people to work on Google+.
I asked the recruiter if he'd seen my G+ profile. At the time, it had this childish scrawl all over it.
We mutually decided not to pursue the opportunity any further.
It's a decision I've often thought about. I've been involved in my fair share of high profile failures, and I didn't feel I could survive another one on my CV. Yet, I now wonder if I would have been able to do anything to improve the awful situation Google+ finds itself in with developers.
Where Is Everybody?
This weekend, I decided I wanted to build a Weasley Family Clock. You remember the one from Harry Potter which lists where every member of the family currently is?
During our last holiday to San Francisco, my wife and I had made extensive use of Google Latitude. She was alone in an unfamiliar city and I was stuck at a conference in the suburbs. It was a simple, passive, and comforting way to keep an eye on each other's whereabouts.
Well, Google killed Latitude. Killed it stone dead. As part of their fanatical quest to drive people on to G+, they made that the only way to share the location data they were gathering.
So, my wife set up an account with the sole purpose of sharing her location with me. That's a +1 for their user numbers but -100 in good will.
Ok, so, let's get started!
Getting Location Data Through The Google+ API
You can't.
Oh, don't get me wrong, you can read the documentation which says it's supported, follow the tutorials which show you how to do it, read StackOverflow posts giving you advice, and watch YouTube videos showing how awesome it is - but you can't get any location data out of the API. Not your friends', and not your own.
I'm a Product Manager. I know that sometimes you have to make tough choices about which new features to include, or which bugs to prioritise.
What I can't understand is how anyone has the gall to actively solicit bug reports from highly engaged members of the community, and then ignore them all.
I can't understand how you can have documentation which is fundamentally contradicted by reality.
I can't understand how any manager at Google thinks that annoying the people who want to interact with you is a good idea.
It's like Google were a toddler, repeatedly punching kids in the face and then wondering why they don't want to play with it.
All the people I know who work at Google are smart, passionate, committed, and eager to do the right thing. But something in that company is stopping them. The very culture of the company seems to be "we're the best of the best, therefore everything we do is perfect. There's no need to fix our broken windows."
I practice a zero-tolerance form of Product Management. Every small customer facing defect is a signal to your customers that you don't care about them. Each bug is a slap in the face and deserves a swift remedy - or at the very least an apology and a timescale for a fix.
Google is sticking its head in the sand and hoping developers will ignore the broken functionality, and broken promises, which now damage its reputation.
How To Liberate Your Location Data From Google
You're going to be in for a fun ride - so strap in!
First, visit Google's Location History Page.
On there, you'll see a link to download your location history by exporting it to KML. Go ahead and click it.
Hey presto! A KML file with your location throughout the day.
The URL format is:
https://maps.google.com/locationhistory/b/0/kml?startTime=1398553200000&endTime=1398639600000
OK, so how do we get this programmatically?
Aha! Foolish grasshopper! Google doesn't provide any API to get this data. You must go through the Web interface.
Now, this is simply no good if we want real time access to our location. So, let's get creative.
If we have our Google cookies, we can retrieve our data fairly simply.
Using Chrome, install this Cookie Exporting add on.
In settings, allow it to access incognito windows.
Open an incognito window and log in to your Location History page.
Click the Cookie plugin, and you'll be rewarded with all your cookies for Google.
Copy and paste the data into a text file. I called mine "cookies.txt" because I'm an original and creative thinker.
To get your data using curl on the command line, run this.
curl -b cookies.txt "https://maps.google.com/locationhistory/b/0/kml?startTime=1398510000000&endTime=1398517922000"
curl will use the cookies in the text file, and retrieve your information.
The eagle-eyed among you will have noticed that the start and end times will need to be updated. They are simply the Unix timestamps for the start and end of the day.
Here's how I calculated them using PHP.
// Today's time is needed to get the bounding for the request
$currentTime = time();
// Beginning and end of the day
$startTime = strtotime("midnight", $currentTime);
$endTime = strtotime("tomorrow", $startTime);
// The Google Location history URL - gets the KML.
$locationURL = "https://maps.google.com/locationhistory/b/0/kml?startTime=" .
$startTime . "000&endTime=" .
$endTime . "000";
To request the file, using PHP, we need to tell curl where the cookies are.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $locationURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
$kml = curl_exec($ch);
curl_close($ch);
All I want is the last geocoded point in the file, so there's no need to interpret it as XML (your needs may vary). So I just lazily iterate over each line, looking for the data I want.
// Get the most recent location
foreach(preg_split("/((r?n)|(rn?))/", $kml) as $line)
{
if (strrpos($line, "") === 0)
{
//-1.2345678 51.7654321 0
$line = str_replace("", "", $line);
$line = str_replace("", "", $line);
$pieces = explode(" ", $line);
$long = $pieces[0];
$lat = $pieces[1];
}
}
And there you have it. Stealing cookies, scraping data, and calculating timestamps - all because Google can't get its shit together.
This should have been a simple call according to their well documented (but incorrect) API.
Seriously Google, why should I recommend or use your products and services, when you treat me so disrespectfully?
Alternatives
Some of you may be wondering why I didn't use a different product.
It's true that Glympse, OwnTracks, et al, do a pretty good job.
But that's yet another app to install, configure, update, maintain, and watch for battery life impact. I change phones regularly, and don't want the hassle of remembering another password to another service.
Others may question why I feel the need to track my wife's location all the time. It's simply a low effort way to ensure she knows when I'm stuck on the motorway, and I can see how long I have to clean the house and cook her dinner.
I'm building a dashboard to live by our front door which will give us an at-a-glance status update.
Hey, it works for us!
A Final Message To Google
For the sake of my sanity - and the mental well-being of every developer who has to interact with you - please follow these simple guidelines.
- Fix what is broken, before adding new functionality.
- Don't kill off a well used product without ensuring a robust transition plan.
- Make sure your documentation reflects reality.
- Acknowledge your bug reports - and act on them.
Thanks!
Get your Google Location History the hard way… Again!
Jörg says:
Jörg says:
Jörg says:
Julian Bond says:
Martin Votruba says:
Terence Eden says:
Jörg says:
Mark Hughes says:
Terence Eden says:
Mark Hughes says:
Terence Eden says:
Mark Hughes says:
Terence Eden says:
Mark Hughes says:
larick says:
Yom says:
wissam says: