Get your Google Location History the hard way… Again!
By @edent
· api code google location · 10 comments · 450 words · read ~9,849 times.
Last year, I wrote about how to extract Location History from Google. Once again, Google have changed their URLs to make it even harder to get one's current location out of their data-greedy hands.
It used to be the case that Latitude gave that information - but they killed it. Then they promised it in Google+ - but never delivered. Now they offer you a data-dump which they will email to you. Hardly convenient if you want a single day!
sigh
Recently Google released Timeline - an interactive way to view where you've been. It also offers the ability to download a single day's worth of location tracks as KML.
With a little bit of fiddling, we can get the direct URL. Strap in, it's going to get rough!
In order to get today's date (2015-09-02) the URL needs to be:
https://www.google.com/maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2015!2i8!3i2!2m3!1i2015!2i8!3i2
That URL format is… unique…
Here's how it breaks down:
pb=!1m8!1m3!1iYYYY!2iMM!3iDD!2m3!1iYYYY!2iMM!3iDD
Where MM is a Javascript style zero-based month. (Why is the first month "0" but the first day "1"? Because fuck you, that's why!)
One thing to note, when you get the KML file, the timestamps are in ISO 8601 format (Yay!) but are set to West Coast USA time (Boo!) 2015-09-02T12:15:37.836-07:00
Annoying, but not the end of the world. How do you get the data into your app?
You can't.
There's no authorised API access for this. You can't do the OAuth dance. The only way is via cookie-jacking.
Sign in to Google via your web browser. Open up your web inspector in Chrome or Firefox. Load the KML URL and choose "Copy as cURL" 
You'll end up with a command like:
curl 'https://www.google.com/maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2015!2i8!3i2!2m3!1i2015!2i8!3i2' -H 'Host: www.google.com' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-GB,en;q=0.5' --compressed -H 'DNT: 1' -H 'Cookie: --------------' -H 'Connection: keep-alive'
You can use that command - or steal your own cookie - for use in your app. Your cookie will likely expire after a set time - whereupon you'll have to reauthenticate.
What a faff! Come on, Google. Rather than pissing about with your logo, why not build something that works properly? I dare ya!
Extracting Your Own Location Information From Google - The Hard Way
The Day Google Deleted Me
Ameer Ellaboudy says:
Thanks for the post! This has greatly helped me in my application. However, I'm wondering if there's a way to do this without having to explicitly extracting your own cookie. If a user is signed into Google already, shouldn't there be a way to grab the data within a javascript script?
Hi Ameer, could you find out how you can do it? We need to solve same issue.
Ameer says:
No, I haven't found a way around this. It would be interesting if you found a way.
J says:
The KML URL is no longer there (on the timeline page, as far as I can tell).
Koen says:
I got a Syntax error when trying the above. My copied cURL format from the inspector looks like this:
curl "https://www.google.nl/maps/timeline/kml?authuser=1&pb=!1m8!1m3!1i2017!2i4!3i3!2m3!1i2017!2i4!3i3"
--2.0
-H "Host: http://www.google.nl"
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0"
-H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8"
-H "Accept-Language: en-US,en;q=0.5"
--compressed
-H "Cookie: NID=102= long string of code here;
CONSENT=YES+NL.nl+V8;
SID= some code here;
HSID= code here; SSID= code here;
APISID= code here;
SAPISID=code here;
OGPC= code here;
OGP= code here;
DV= code here"
-H "Connection: keep-alive"
-H "Upgrade-Insecure-Requests: 1
How can I adapt it to make it work?
Remove the --2.0 🙂
jimmy says:
I dont see the activity in the Network tab. I do the following: - browse to http://www.google.com (and login) - open the network monitor - clear the results - copy/paste the url (https://www.google.com/maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2015!2i8!3i2!2m3!1i2015!2i8!3i2) in the browser address and hit enter - nothing is added to the network monitor, but the kml file is downloaded.
Michael Ingraham says:
Terence – thanks for this information!
I am able to retrieve the curl command for a Timeline URL encoded to obtain the location data for a particular day. In it are all of the header (-H) fields including the cookie. The cookie itself has many sub-parameters as mentioned in other replies to this post.
I also found this post which has some Python code which can be used to download a KML. The code references a similar method of retrieving the cookie.
What I don’t understand is how to “save off” the cookie details. Should it be stored as plain text in a variable? Should if be store in a text file and referenced in that manner (I’m thinking of how my browser’s cookies get stored and used). I simply do not have the knowledge or experience regarding use of cookies to comprehend this “cookie topic” in any detail.
I actually am trying to create a slight variation of this technique. I have two Google accounts for which I want to look up the current location. Right now I have location history enabled and only turned on for the mobile device associated with each account. Interactively, I want to log on to one of those accounts, retrieve the latest location history (i.e., today’s KML), and then parse it to get the latest logged location for that account.
I’d love to be able to programmatically log in to a particular Google account (a simple snippet of Python code) becoming user ‘0’, fire off the Timeline KML HTTP request, collect the resulting output, and then use a KML parsing library (e.g., http://pythonhosted.org/pykml/). Worst case, I suppose I could log in to each account, save off that account’s specific cookie and then use the appropriate cookie depending on which account I’ve logged in. Ideally I’d like to pass the authentication information directly to the Timeline HTTP request but I haven’t gotten that to work properly even interactively. Ideally, ideally, I’d like to retrieve the cookie programmatically instead of having to store it (and have it go stale).
Another issue I’m having is making my login persistent from the login request to the KML request. Does anyone know how to do that in Python? I’d love some guidance.
Michael Ingraham says:
I meant to include this in my last post - https://github.com/alexattia/Maps-Location-History
I was able to perform some additional testing and figured out what portion of the saved curl was the "cookie_content". Indeed, it's obvious now. The cookie is the is that element in the command including everything in that string, including the 'cookie:' label. So, I know what to save.
That git uses this method and also has some great code to capture the daily location history and then parses it and performs varying types of analysis and visualizations. There are a couple of updates to the Google Location History data (LineStrings vs. Tracks) that required a few tweaks (very minor) to the code. I also added a bit to actually save off the coordinates of each "stationary" place logged. It was part of the date. It just wasn't being parsed.
So, I have the KML for the latest day. I'm keeping the logic to just save it to a file rather than trying to figure out how to stream the data back directly. I just call get_kml_file since I'm only interested in the here and now passing the current time year, month, and day. Then I have the great parsing code to get the latest logged Place coordinates. Then I delete the file immediately afterwards.
I'll need a bit of logic to find the most current "Point" in case the latest entry is a "moving" entry.
Google sure is doing everything they can to make life difficult.
Cheers!
Mike
P.S. Now the bit about doing this with multiple Google accounts and their associated cookies. And the bit about cookies going stale and needing to be refreshed.
P.P.S. If you have any insights, I'm open to any and all suggestions.
I just wanted to say thanks. I used details from your blog post to create an import from Google Timeline into a spreadsheet.
If anyone is interested, here's the link: https://github.com/Excel-projects/Excel-Timesheet