Tagged: api

A (Minor) Twitter Privacy Bug?

Quick Summary

Twitter's secure API hides the contents of the tweets you are reading. But it doesn't hide the images of those you converse with.

Raised as Issue 2175.

A Bit More Detail

Twitter has a secure (HTTPS) and insecure (HTTP) API.

When calling the secure API, all the content of the returned message (tweets) are encrypted. Eavesdroppers only see the cipher-text - essentially garbage.

However, within that cipher-text are links to insecure resources.

For example, a user requesting my tweets will get an object which contains a link to my avatar image.

Twitter is currently returning the insecure link:

"profile_image_url" : 
    "http://a2.twimg.com/profile_images/1283757621/Sketch_Avatar.jpg"

Twitter should be returning the secure link:

"profile_image_url" : 
    "https://si0.twimg.com/profile_images/1283757621/Sketch_Avatar.jpg"

Exploiting This Weakness

A user (Anna) will request the encrypted text of my tweets
She then requests the unencrypted image.
An eavesdropper (Eve) is listening in on the connection between Anna and Twitter.

Anna ---->Eve---->Twitter  (Secure request)
Anna <----Eve<----Twitter  (Secure response)

When Anna makes the initial request to Twitter, the malicious Eve can't see what they're talking about.

  • The request "https://example.com/twitter/edent" is itself encrypted. Eve only sees an encrypted request to example.com - not "twitter/edent
  • The response containing all the tweets is also encrypted
Anna ---->Eve---->Images  (insecure request)
Anna <----Eve<----Twitter  (insecure response)

Anna then makes the subsequent request for the twitter user's image, a malicious user can see

  • The URI of the request.
  • The content of the image.

Impact

Truth is, this has a pretty low security impact.

  • There is no way to determine a user's name based on the URI for their image. (Unless you already have both).
  • An eavesdropper has no way of knowing if the image is from the timeline, a reply, a DM, a search, a retweet, or the public timeline.
  • Images may be locally cached by the user's browser - so frequency analysis isn't reliable.
  • A malicious user could alter the image in transit.

Worst case scenario is that if a malicious man-in-the-middle knows which images relate to which Twitter users, they know the intercepted user has seen at least one tweet from that user.

Let's say Anna is communicating with Bob. Eve is trying to eavesdrop.
If Bob has never tweeted, and Eve sees repeated requests from Anna for Bob's avatar, she may reasonably surmise that they are exchanging DMs.

Overall

This is a pretty low-impact privacy risk.
It can be fixed by Twitter's API returning HTTPS URIs where possible.
In the meantime, developers can replace "http://a2.twimg.com/" with "https://si0.twimg.com".

Getting Images from a Foursquare Checkin

"Oi!" shouted Whatleydude, "Get Dabr to show images from foursquare checkins!"
"Righty-ho sir!" I said. I started coding furiously. Of course, things are never quite as simple as I first thought....

So, how do we go from http://4sq.com/fgIWov
to
James' Foursquare Checkin Image?

1 Expand the URL

Get your Bit.ly API Key.


http://api.bitly.com/v3/expand

   ?shortUrl=http://4sq.com/fgIWov
   &login=YOUR_BIT_LY_USERNAME
   &apiKey=YOUR_BIT_LY_API_KEY
   &format=txt

You can, if you prefer, get the info back in JSON or XML. See the Bit.ly API Expand Documentation.

2 The Foursquare URL

Bit.ly gives us the URL which includes a checkin ID and a signature.


http://foursquare.com/edent/checkin/4d7e5b4f9df3f04d7400c394

   ?s=afH3jJg3L9HpLqVIwiqp-YpNL5k

3 The Foursquare API Call

From this, we construct an API call to Foursquare. It looks like this


https://api.foursquare.com/v2/checkins/4d7e5b4f9df3f04d7400c394

   ?signature=afH3jJg3L9HpLqVIwiqp-YpNL5k
   &oauth_token=YOUR_FOURSQUARE_OAUTH_TOKEN

NOTE the Foursquare URL says ?s= whereas the API call requires ?signature=
That caused me more frustration than it should...

Getting your OAuth token is a little cumbersome - follow the steps at the Foursquare Developers site.

4 The JSON Response

I'll cut the cruft out of here, so you only see what you need to display images.
The response gives us the link to the full sized image - as well as several different sizes should you wish to display thumbnails.

  1. {
  2.   "meta":{
  3.      "code":200
  4.   },
  5.   "response":{
  6.      "checkin":{
  7.         },
  8.    ...
  9.    "photos":{
  10.       "count":1,
  11.       "items":[
  12.          {
  13.             "id":"4d7e5b4f9df3f04d7400c394",
  14.             "createdAt":1300184596,
  15.             "url":"http://playfoursquare.s3.amazonaws.com/pix/A.jpg",
  16.             "sizes":{
  17.                "count":4,
  18.                "items":[
  19.                   {
  20.                      "url":"http://playfoursquare.s3.amazonaws.com/pix/B.jpg",
  21.                      "width":720,"height":540
  22.                   },
  23.                   {
  24.                      "url":"http://playfoursquare.s3.amazonaws.com/derived_pix/C.jpg",
  25.                      "width":300,"height":300
  26.                   },
  27.                   {
  28.                      "url":"http://playfoursquare.s3.amazonaws.com/derived_pix/D.jpg",
  29.                      "width":100,"height":100
  30.                   },
  31.                   {
  32.                      "url":"http://playfoursquare.s3.amazonaws.com/derived_pix/E.jpg",
  33.                      "width":36,
  34.                      "height":36
  35.                   }
  36.                ]
  37.             },

5 Gotchas

Again, nothing in life is easy. Not all foursquare checkins have an image associated with them. In which case, we can do one of three things.

  1. Display no image
  2. Display the user's avatar - if it's public
  3. Display an image of the venue - or one of the icons associated with it.

6 Embed.ly

Dabr uses the awesome Embedly service. Rather than having hundreds of different regular expressions and API calls to get embedded images, Embed.ly gives us a single end point to call.
So, rather than rewrite huge chunks of Dabr's code, I've submitted this to the clever-clogs behind Embed.ly - that way, everyone can benefit.

Any questions, comments, or clarifications - please let me know in the comments box.

Share Android Apps on Twitter (or anywhere else)

I attended the Mobile Monday meeting "200,000 Apps - Where's Mine" last night.
One thing that became clear is that apps don't do a very good job of promoting themselves. One crippling problems with most app stores is that there's no (easy) way to share an app with a friend.

Here's some basic code for an Android app which will post the URL of your app to Twitter. Stick it in a button or menu item for easy sharing.

  1. String twitterUri = "http://m.twitter.com/?status=";
  2. String marketUri = Uri.encode("http://example.com/?q=app&amp;title=test");
  3. Intent shareOnTwitterIntent = new Intent(Intent.ACTION_VIEW, Uri.parse(twitterUri + marketUri));
  4. startActivity(shareOnTwitterIntent);

Some important things to note.

  1. This is set to post to the mobile version of Twitter.  Your user is on a phone - don't direct them to a site that won't work on their device.
  2. The second string is URI encoded.
  3. Consider if you want to post a "market://" link.  I would advise against it.  Twitter won't render it as a link and, even if it did, 90% of users won't be able to click on it.  Make it a link that will direct desktop users to your website, mobile users to a mobile friendly site and Android users direct to the market.

Facebook also has an API for this sort of sharing.


http://m.facebook.com/sharer.php?u=example.com&t=test

Again, it points to the mobile site and needs to be URL encoded.

Happy sharing!

Android Tutorial - Clickable Widgets

Another quick Android tutorial. I couldn't find an easy or correct method of launching a browser when you click on a homescreen widget. Well, here it is...

  1. public class clickWidget extends AppWidgetProvider
  2. {
  3. @Override
  4. public void onUpdate( Context context, AppWidgetManager appWidgetManager, int[] appWidgetIds )
  5. {
  6. RemoteViews remoteViews =
  7.    new RemoteViews( context.getPackageName(), R.layout.widget );
  8. remoteViews.setImageViewResource(R.id.ImageView01, drawableResourse);
  9.  
  10. ComponentName myWidget =
  11.    new ComponentName( context, clickWidget.class );
  12.  
  13. // Create an Intent to launch Browser
  14. Intent intent =
  15.    new Intent(
  16.       Intent.ACTION_VIEW, Uri.parse("http://example.com")
  17.    );
  18. PendingIntent pendingIntent =
  19.    PendingIntent.getActivity(context, 0, intent, 0);    
  20.  
  21. remoteViews.setOnClickPendingIntent(R.id.ImageView01, pendingIntent);
  22. appWidgetManager.updateAppWidget( myWidget, remoteViews);
  23. }

I've used this as the basis of a demo widget - "MI5 Terror Threat Level". The widget displays the UK's Threat Level on your homescreen. Clicking on it takes you to the MI5 page discussing the threat level.

The threat level is determined by parsing the RSS that the security services so helpfully provide. At the moment, the widget keeps a local copy of the graphics because the RSS feed contains references to "localhost" images.

You can download the widget by scanning in this QR code.

MI5 Widget - QR Code
MI5 Widget - QR Code

Twitter API - pagination and IDs

Looking for some Twitter API help.  Bit of a geeky post, this...

Pagination is the act of splitting data into logical  pages. Suppose I had a list of item, numbered 0 - 99.  If I want 20 items per page, it's trivial to see that pagination looks like:

p1 = 0-19
p2 = 20-40
p3 = 41-61
p4 = 62-82
p5 = 83-99

If I wanted to start at, say, page 55 - pagination would look like:

p1 = 55-75
p2 = 76-96
p3 = 97-99

Easy, right?  So why am I telling you this?

Twitter Timeline

Imagine that those items are Twitter Status ID.  Each one represents a tweet in your timeline.

Twitter will allow us to "page" back and forth through our timeline. If we say status ID 80 is the most recent post in our timeline, and we want to see 20 tweets at a time, pagination would look like this.

p1 = 80-60
p2 = 60-40
... etc.

Normally, that would be fine.

The only issue is that friends are posting all the time.  Imagine we start with tweets 80-60.  We go to page 2, but in the meantime, 5 new tweets have been made.

p1 = 80-60
p2 = 65-45

The user sees 5 tweets she has already read.  Not desirable.

If 20 tweets had been made before clicking on the "next" button, this is what happens.

p1 = 80-60
p2 = 80-60

Max_ID To The Rescue (AKA, the easy bit)

Luckily, Twitter allows us to use a max_id parameter in our API calls.  This says "Get the tweets older than this number."

http://api.twitter.com/1/statuses/home_timeline.json?max_id=123465789

So, using max_id we can ensure that the user never has to read the same tweet twice.  Instead of dumbly using pages, we call the specific tweets we want.

p1 max_id=80 = 80-60
p2 max_id=60 = 60-40

Easy!  We just use the oldest tweet on the page as the max_id parameter when we call the next page.

Looking To The Future (AKA, where it all goes horribly wrong)

So far, we've looked at stepping back in time.  Seeing older tweets.  Suppose we want to see newer tweets?

Twitter provides us with a since_id parameter for API calls.  This says "Get the tweets newer than this number."

Unfortunately, it doesn't work.  Well, it works but not the way I expected it to!

Suppose our user is deep down in her tweets, this is how I would expect since_id to work

max_id=60  = 60-40
 (So, let's show any more recent tweets)
since_id=60 = 80-60

We see the 20 tweets that occured since the since_id.  Right?  Wrong!  This is what happens?

max_id=60  = 60-40
(So, let's show any more recent tweets)
since_id=60 = 100-80

What?

An Explanation

The since_id retrieves tweets starting with the most recent.  It stops when it reaches the since_id.

I don't know the max_id that I'm looking for, so I can't call that.

I could call the most recent 200 tweets and look for the 20 I need.  That's wasteful in terms of bandwidth and processing - there's also no guarantee that the since_id will be in there.

So, I have a problem.  The "Older" link in my Twitter application will work.  The "Newer" links won't.

Any suggestions?