A few friends of mine regularly receive harassment on Twitter. Typically, there are two ways to deal with it:

1. Mute the offending user. This means their tweets to you never show up in your replies.
2. Block the offending user. The same as muting, except the offender is also prevented from seeing your Tweets while they're logged in.

There's a flaw with both of these tactics - the miscreant can still target messages at you.

For example, a journalist I know has blocked a particularly persistent griefer. This sad-sack continually mentions the journalist and repeatedly links to her articles. This means the abuse appears whenever the journalist searches for people linking to her articles and inevitably, when her followers get drawn in to replying on her behalf.

So, here's a radical solution - if you're blocked by a user, you shouldn't be able to @mention them.

As I suggested to Twitter

Alice is blocking Bob.

Bob cannot see Alice's Tweets.

Bob can still send @ replies to Alice.

Should our clients block Bob from sending a Tweet containing an @ reply to Alice?

Should the Twitter API block these messages from being sent?

Until Twitter gets around to implementing this at the API level, here's a way that app developers can add the functionality.

Friendship API

Using GET friendships/show we can determine if the logged in user is being blocked by a specific user.

In this example, I (@edent) have blocked the well known troll @monkey. The app can perform this request every time @monkey attempts to send a tweet containing "@edent".

https://api.twitter.com/1.1/friendships/show.json?
   source_screen_name=monkey
  &target_screen_name=edent

In return we get a message showing if the user is being blocked.

{
  "relationship": {
    "source": {
      …
      "screen_name": "monkey",
      …
      "blocking": false,
      "blocked_by": true,
      …
    },
    "target": {
      …
      "screen_name": "edent",
      …
    }
  }
}

The app can then either refuse to send the message or tell the user why it has been blocked.

Drawbacks

If enough popular apps start doing this, it might act as a small disincentive - letting abusers know that their messages are no longer welcome.

But until Twitter adopt this as the standard for their API, it will be possible for a harasser to simply switch to using the official apps.

The real change, of course, has to come from within. People have to learn that no means no - if someone chooses to ignore your messages, that's not an infringement of your right to free speech.

I'll be implementing this in Dabr over the next week. I'd be interested to hear from any other developers who make this change.

Dabr

@dabr

Fact of the month: @dabr is now 5 years old! Born in August 2008 when @davidcarrington was inspired by @Whatleydude.

---

In those days, Twitter had a kick-arse API and were happy for people to build exciting new services on it. I was eager to contribute and got stuck in to improving it.

Dabr was fast - much faster than the original mobile Twitter site. It had hashtag searching, reply-to-all, thumbnail previews of links, Twitpic integration, and a whole bunch of other interesting features. It was used by Chinese political dissidents to get around the Great Firewall, and some of the most popular Twitter users.

But, over the last few years, the project stagnated. Partly because Twitter didn't really release anything new with its API, partly because Dabr was complete. It did everything it was supposed to, and ran quite happily. The last significant code update was in 2013. Sure, there were a few bugs hanging around, but no one really had the motivation to do much about it.

Until now.

I thought it'd be a fun(!) Xmas project to resurrect Dabr, give the code-base a spring clean, a new lick of paint, squash some bugs, and improve the functionality. So I did.

You can log in to https://dabr.eu/ for the latest and greatest version. All the Dabr code is Open Source on GitHub - please raise bug reports there, and feel free to send pull requests.

What's New

This is a fairly comprehensive list of all the improvements.

• Image upload.
• Menus now use CSS fonts from Fontello. Looks nicer & faster to download.
• Much better pagination support.
• Twitter API via CodeBird.
• Better embedding thanks to EmbedKit.
• Layout changes & improvements.
• Cleaner HTML. Should be lighter to download.
• More options, including a new colour theme.
• Better accessibility (Thanks Andy!)
• JavaScript only used for character counter - nothing else.
• HTTP Status messages display useful information.
• ...lots more bug fixes :-)

To Do

There are still some kinks to work out. Feel free to add to the issues list or submit some code.

• Hashtag linking is still dodgy - and doesn't work in bios.
• Lists aren't available. Does anyone use them?
• Image sending in Direct Messages doesn't work.
• Conversation threading (made harder by lack of API).
• Counter doesn't recognise URLs - gives misleading character count.
• Translation text files. Currently everything is inline.
• Better 404 handling.
• General slimming down of HTML and CSS.

If you want to experience the new Dabr, point your phone, tablet, or desktop browser at https://dabr.eu/.

Dabr is designed to be simple to deploy on your own website. Visit Dabr on GitHub to find out more.

Today, I am pleased to announce that Chinese artist, political prisoner, and sower of porcelain sunflower seed, Ai Weiwei uses dabr!

On last night's Newsnight, as part of a package on Ai Weiwei's release, they showed him using Twitter. Here's a screenshot.

Click for larger image

One interesting thing, none of the user's avatars appear. That's because, at a guess, the images are retrieved from Twitter's servers which are blocked by the Great Firewall of China.

Indeed, if we zoom in on the text we can see the letters "GFW". My Mandarin is too rusty to translate the rest - can anyone else help?

So, looks like my next addition to Dabr will be to use an image proxy to ensure avatar images make it through China's firewall!

Thanks to Kristian Carter for alerting me.

Awesome!

Rather than using an external service like Embed.ly to retrieve thumbnails, all the data is embedded within Twitter Entities.

So, if you request a status using "include_entities=true", you will be able to grab the image and display the thumbnail using the following code.

function twitter_get_media($status) {
   if($status->entities->media) {

      $url = $status->entities->media[0]->media_url_https;

      $width = $status->entities->media[0]->sizes->thumb->w;
      $height = $status->entities->media[0]->sizes->thumb->h;

      $media_html = "<a href="" . $url . "" target='_blank'>";
      $media_html .=  "<img src="" . $url . ":thumb" width="" . $width .
         "" height="" . $height . "" />";
      $media_html .= "</a><br />";

      return $media_html;
   }
}

So, a tweet like this: https://twitter.com/twitter/status/76360760606986241 Will render like this (in Dabr):

Notes

This is very rough and ready proof of concept code. Beware of the following:

• This will only take the first image from the tweet.
• Only images are supported - I'm not sure how their proposed video sharing will work.
• There's no error checking.
• In the above code, the https URL is used - if you want a non-SSL link, you'll need to remove the "_https"

Enjoy!

Dabr's strength for developers is two-fold

1. Dead easy to install. Unzip the files, fill in your API key(s), upload, done.
2. It's under an incredibly permissive Open Source MIT License. Essentially anyone can do anything with the code and they don't need to ask permission, nor contribute anything back to the code base.

It's a really popular tool. Although it's hard to count how many users are on the main Dabr server - not to mention all the clones - it's obvious that many people find it an indispensable way to access Twitter.

According to the New York Times, Rafinha Bastos is the most influential person on Twitter. Guess what client he uses? So, with a kick-arse product and no silly "Intellectual Property" restrictions, we have let a thousand flowers bloom. There are clients deployed all over the world. Including on my secure server - SSL Dabr. And here's one more...

Welcome UberSocial!

The most recent flower is UberSocial. As well as having the most popular BlackBerry Twitter client, and an iPhone client - they've now beta-launched a mobile web client - based on Dabr!

I'm really excited to see Dabr spreading far and wide. The intention - for me - has never been to make pots of money from it. The adverts on SSL Dabr just about cover the hosting costs.

What I like seeing is my code being used far and wide - and helping people. I know it's corny, but seeing my code being used by Twitter users in Egypt during the uprising was overwhelming. In a tiny, almost insignificant way, I helped.

I don't think my coding is good enough to get something into a really important project - like the Linux Kernel, or the International Space Station - but seeing millions of users is personally edifying.

So, long live Open Source and long live Dabr!

So, how do we go from http://4sq.com/fgIWov to

1 Expand the URL

Get your Bit.ly API Key.

http://api.bitly.com/v3/expand
   ?shortUrl=http://4sq.com/fgIWov
   &login=YOUR_BIT_LY_USERNAME
   &apiKey=YOUR_BIT_LY_API_KEY
   &format=txt

You can, if you prefer, get the info back in JSON or XML. See the Bit.ly API Expand Documentation.

2 The Foursquare URL

Bit.ly gives us the URL which includes a checkin ID and a signature.

http://foursquare.com/edent/checkin/4d7e5b4f9df3f04d7400c394
   ?s=afH3jJg3L9HpLqVIwiqp-YpNL5k

3 The Foursquare API Call

From this, we construct an API call to Foursquare. It looks like this

https://api.foursquare.com/v2/checkins/4d7e5b4f9df3f04d7400c394
   ?signature=afH3jJg3L9HpLqVIwiqp-YpNL5k
   &oauth_token=YOUR_FOURSQUARE_OAUTH_TOKEN

NOTE the Foursquare URL says ?s= whereas the API call requires ?signature= That caused me more frustration than it should...

Getting your OAuth token is a little cumbersome - follow the steps at the Foursquare Developers site.

4 The JSON Response

I'll cut the cruft out of here, so you only see what you need to display images. The response gives us the link to the full sized image - as well as several different sizes should you wish to display thumbnails.

{
  "meta":{
     "code":200
  },
  "response":{
     "checkin":{
        },
   ...
   "photos":{
      "count":1,
      "items":[
         {
            "id":"4d7e5b4f9df3f04d7400c394",
            "createdAt":1300184596,
            "url":"http://playfoursquare.s3.amazonaws.com/pix/A.jpg",
            "sizes":{
               "count":4,
               "items":[
                  {
                     "url":"http://playfoursquare.s3.amazonaws.com/pix/B.jpg",
                     "width":720,"height":540
                  },
                  {
                     "url":"http://playfoursquare.s3.amazonaws.com/derived_pix/C.jpg",
                     "width":300,"height":300
                  },
                  {
                     "url":"http://playfoursquare.s3.amazonaws.com/derived_pix/D.jpg",
                     "width":100,"height":100
                  },
                  {
                     "url":"http://playfoursquare.s3.amazonaws.com/derived_pix/E.jpg",
                     "width":36,
                     "height":36
                  }
               ]
            },

5 Gotchas

Again, nothing in life is easy. Not all foursquare checkins have an image associated with them. In which case, we can do one of three things.

1. Display no image
2. Display the user's avatar - if it's public
3. Display an image of the venue - or one of the icons associated with it.

6 Embed.ly

Dabr uses the awesome Embedly service. Rather than having hundreds of different regular expressions and API calls to get embedded images, Embed.ly gives us a single end point to call. So, rather than rewrite huge chunks of Dabr's code, I've submitted this to the clever-clogs behind Embed.ly - that way, everyone can benefit.

Any questions, comments, or clarifications - please let me know in the comments box.

Pagination is the act of splitting data into logical  pages. Suppose I had a list of item, numbered 0 - 99.  If I want 20 items per page, it's trivial to see that pagination looks like:

p1 = 0-19
p2 = 20-40
p3 = 41-61
p4 = 62-82
p5 = 83-99

If I wanted to start at, say, page 55 - pagination would look like:

p1 = 55-75
p2 = 76-96
p3 = 97-99

Easy, right?  So why am I telling you this?

Twitter Timeline

Imagine that those items are Twitter Status ID.  Each one represents a tweet in your timeline.

Twitter will allow us to "page" back and forth through our timeline. If we say status ID 80 is the most recent post in our timeline, and we want to see 20 tweets at a time, pagination would look like this.

p1 = 80-60
p2 = 60-40
... etc.

Normally, that would be fine.

The only issue is that friends are posting all the time.  Imagine we start with tweets 80-60.  We go to page 2, but in the meantime, 5 new tweets have been made.

p1 = 80-60
p2 = 65-45

The user sees 5 tweets she has already read.  Not desirable.

If 20 tweets had been made before clicking on the "next" button, this is what happens.

p1 = 80-60
p2 = 80-60

Max_ID To The Rescue (AKA, the easy bit)

Luckily, Twitter allows us to use a max_id parameter in our API calls.  This says "Get the tweets older than this number."

http://api.twitter.com/1/statuses/home_timeline.json?max_id=123465789

So, using max_id we can ensure that the user never has to read the same tweet twice.  Instead of dumbly using pages, we call the specific tweets we want.

p1 max_id=80 = 80-60

p2 max_id=60 = 60-40

Easy!  We just use the oldest tweet on the page as the max_id parameter when we call the next page.

Looking To The Future (AKA, where it all goes horribly wrong)

So far, we've looked at stepping back in time.  Seeing older tweets.  Suppose we want to see newer tweets?

Twitter provides us with a since_id parameter for API calls.  This says "Get the tweets newer than this number."

Unfortunately, it doesn't work.  Well, it works but not the way I expected it to!

Suppose our user is deep down in her tweets, this is how I would expect since_id to work

max_id=60  = 60-40

 (So, let's show any more recent tweets)

since_id=60 = 80-60

We see the 20 tweets that occured since the since_id.  Right?  Wrong!  This is what happens?

max_id=60  = 60-40

(So, let's show any more recent tweets)

since_id=60 = 100-80

What?

An Explanation

The since_id retrieves tweets starting with the most recent.  It stops when it reaches the since_id.

I don't know the max_id that I'm looking for, so I can't call that.

I could call the most recent 200 tweets and look for the 20 I need.  That's wasteful in terms of bandwidth and processing - there's also no guarantee that the since_id will be in there.

So, I have a problem.  The "Older" link in my Twitter application will work.  The "Newer" links won't.

Any suggestions?

You can see the full code on Dabr's Google Code page.

First of all, you'll need to have enabled OAuth for your Twitter client. I use Abraham's excellent OAuth libraries for PHP.

This tutorial assumes you already have OAuth working. I'll attempt to explain what I'm doing as I go along - but the code should be pretty readable.

Start by reading the Twitpic API documentation. You will also need to register for an API key - this only takes a few seconds.

We start by setting CURL's headers to those required by Twitpic

//Has the user submitted an image and message?
if ($_POST['message'])
{
    $twitpicURL = 'http://api.twitpic.com/2/upload.json';

    //Set the initial headers
    $header = array(
                            'X-Auth-Service-Provider: https://api.twitter.com/1/account/verify_credentials.json',
                            'X-Verify-Credentials-Authorization: OAuth realm="http://api.twitter.com/"'
                        );

Next, we create the OAuth credentials that we need. Essentially, we're signing a URL request. We then pass that on to Twitpic and they verify it with Twitter. We never pass our OAUTH_CONSUMER_SECRET - so Twitpic can't impersonate us.

    //Using Abraham's OAuth library
    require_once('OAuth.php');

    // instantiating OAuth customer
    $consumer = new OAuthConsumer(OAUTH_CONSUMER_KEY, OAUTH_CONSUMER_SECRET);

    // instantiating signer
    $sha1_method = new OAuthSignatureMethod_HMAC_SHA1();

    // user's token
    list($oauth_token, $oauth_token_secret) = explode('|', $GLOBALS['user']['password']);
    $token = new OAuthConsumer($oauth_token, $oauth_token_secret);

    // Generate all the OAuth parameters needed
    $signingURL = 'https://api.twitter.com/1/account/verify_credentials.json';

    $request = OAuthRequest::from_consumer_and_token($consumer, $token, 'GET', $signingURL, array());

    $request->sign_request($sha1_method, $consumer, $token);

We add these generated credentials into the header.

    $header[1] .= ", oauth_consumer_key=\"" . $request->get_parameter('oauth_consumer_key') ."\"";
    $header[1] .= ", oauth_signature_method=\"" . $request->get_parameter('oauth_signature_method') ."\"";
    $header[1] .= ", oauth_token=\"" . $request->get_parameter('oauth_token') ."\"";
    $header[1] .= ", oauth_timestamp=\"" . $request->get_parameter('oauth_timestamp') ."\"";
    $header[1] .= ", oauth_nonce=\"" . $request->get_parameter('oauth_nonce') ."\"";
    $header[1] .= ", oauth_version=\"" . $request->get_parameter('oauth_version') ."\"";
    $header[1] .= ", oauth_signature=\"" . urlencode($request->get_parameter('oauth_signature')) ."\"";

Add everything into CURL

    //open connection
    $ch = curl_init();

    //Set paramaters
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);

    //set the url, number of POST vars, POST data
    curl_setopt($ch,CURLOPT_URL,$twitpicURL);

The data we send to Twitpic (message text, image and key) have to go via POST.

    //TwitPic requires the data to be sent as POST
    $media_data = array(
                            'media' => '@'.$_FILES['media']['tmp_name'],
                          'message' => ' ' . stripslashes($_POST['message']), //A space is needed because twitpic b0rks if first char is an @
                          'key'=>TWITPIC_API_KEY
                        );

    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch,CURLOPT_POSTFIELDS,$media_data);

All done, we now send the data to Twitpic.

    //execute post
    $result = curl_exec($ch);
    $response_info=curl_getinfo($ch);

    //close connection
    curl_close($ch);

If this has worked, Twitpic will pass back the URL of the image we posted. We then need to post the entire message to Twitter ourselves (Twitpic can't do it for us).

    if ($response_info['http_code'] == 200) //Success
    {
        //Decode the response
        $json = json_decode($result);
        $id = $json->id;
        $twitpicURL = $json->url;
        $text = $json->text;
        $message = trim($text) . " " . $twitpicURL;

This next part is specific to Dabr - your client may post things differently.

        //Send the user's message to twitter
        $request = API_URL.'statuses/update.json';

        $post_data = array('source' => 'dabr', 'status' => $message);
        $status = twitter_process($request, $post_data);

        //Back to the timeline
        twitter_refresh("twitpic/confirm/$id");
    }

If it didn't work, Twitpic will tell us why.

    else
    {
        $content = "<p>Twitpic upload failed. No idea why!</p>";
        $json = json_decode($result);
        $content .= "<br /><b>message</b> " . urlencode($_POST['message']);
        $content .= "<br /><b>json</b> " . print_r($json);
        $content .= "<br /><b>Response</b> " . print_r($response_info);
        $content .= "<br /><b>header</b> " . print_r($header);
        $content .= "<br /><b>media_data</b> " . print_r($media_data);
        $content .= "<br /><b>URL was</b> " . $twitpicURL;
        $content .= "<br /><b>File uploaded was</b> " . $_FILES['media']['tmp_name'];
    }
}

One of the advantages of the text processing is that it will recognise that www.example.com is a URL and automatically create a hyperlink. Considering that dropping the "http://" represents 5% saving on Twitter's 140 character limit for messages, this is great.

So, I was mightily surprised to get this bug report from user "schmmuck"

Dabr rendering error

How very odd... This is how it looks on m.twitter.com.

m.twitter rendering error

Twitter also use mobile.twitter.com for smartphones. Here's how that site renders the text.

mobile.twitter rendering error

Finally, let's take a look at the "canonical" rendering at Twitter.com

Twitter rendering error

The Problem(s)

The first issue is inconsistency.  Twitter ought to be using the same regex for each of its sites.  It doesn't.  This means that different developers will get divergent experiences.  This leads to confusion, which leads to fear, which, as we all know, leads to anger.... and so forth.

Secondly, and more importantly, parsing is hard.  There are so many edge cases that errors inevitably creep in.  My post about hashtags explains the problems in defining what should be recognised.

So, based on what we've seen, should Twitter recognise any of the following as URLs?

news.bbc.co.uk - no www there.

invalid.name - a silly URL, but a valid one.

खोज.com - International domains contain more than just ASCII

All the above are valid - yet they're not recognised by Twitter.

A (Simple) Solution?

There is a canonical list of TLDs which is also available as a plain text list.

Any string containing a "." followed by a valid TLD, then followed by a space or "/" should be treated as a URL.

Your thoughts?

@@ and geotag

Reply to All

The @@ symbol allows you to reply to all the people mentioned within the tweet. It only shows up on tweets which mention other users - so you should only ever see it when it can be used.

Hitting @@ on the above tweet will pre-populate the text box with "@topgold @whatleydude @dabr".

It should remove any duplicates and should also remove your username.

Geotagging

Twitter recently allowed the geotagging of tweets. This allows you to set the longitude and latitude of where you are when you wrote your message. Clicking on the green globe (it just looks like a dot at that size!) takes you to a mobile mapping service.

Google Maps Mobile

It only shows up on tweets which are geotagged - so you should only ever see it when it can be used.

How Does It Work?

You can see all the code in the SVN, but here's a quick rundown.

Geotagging is very simple.  Statuses now have a <geo> field.  If this is populated, it will contain a longitude and latitude.

We can take those values and pass them to Google Maps Mobile (or your favourite mobile mapping service).

if ($geo !== null)
{
   $latlong = $geo->coordinates;
   $lat = $latlong[0];
   $long = $latlong[1];
   $actions[] = theme('action_icon', "http://maps.google.co.uk/m?q={$lat},{$long}", 'images/map.png', 'MAP');
}

Reply to all is a little more involved. Twitter provide an excellent set of Text Libraries. These provide officially sanctioned regular expressions to help your project find URLs, hashtags and usernames.

Using the Twitter Text Extractor class we can determine how many usernames are mentioned in a tweet. Any duplicates are removed - as is the user's username.

Any questions - just ask!

This sort of knowledge is common in life - but is fatal in computing and design. Take the following tweet I received.

Mark Hawkins

@Mawkins

Replying to @edent @edent @dabr you folks aware ampersands / &s don't seem to work as part of hashtag links?

---

The complaint was that #tfm&a should be rendered as #tfm&a not #tfm&a.

Everyone knows that's how hashtags work!

On Twitter's website, find the page which discusses hashtag syntax. Find where they explain how they should be styled.

You can't.

And thus implicit knowledge is born. Dabr only looks at letters and numbers in a hashtag. It assumes that any other character is the end of the tag.

Dabr's Hashtag

Without official guidance - implicit knowledge develops.

Has Dabr Got It Wrong?

No. I don't think so. Take a look at how Twitter on the web renders hashtags...

Twitter's Web Site

...and on the mobile.

Twitter Mobile

So Where Does Render The Full Tag?

Several applications don't render tags in the same way as Twitter. Take a look at SocialScope

SocialScope Hashtags

Tweetie2

I'll upload more screenshots if I find examples of "badly behaved" hashtags.  Please let me know if you find any.

What Does Twitter Say?

Twitter has one page devoted to hashtags.  It is a support page for hashtags.  This explains to people what hashtags are.  There's no detail on valid characters, maximum length, or any of the things which might be useful for a developer or designer.

Edit 2010-02-25

David Dorward has pointed out that there is an official resource. On the Twitter Engineering blog - which isn't linked to from the developer site - there is a page discussing hashtags and how to validate them. You'll notice that they are rather circumspect on what should constitute a hashtag.

Conclusion

Standards and guidelines allow developers to create compatible applications.

Without explicit recommendations, developers will diverge as widely as possible.  Twitter - and everyone with an interest in compatibility and usability - needs to ensure that the knowledge they impart is explicit.

Letting people make it up as they go along leads to confusion.

It's one of my gripes with Linux.  If you're editing a configuration file, you are relying on yourself to sanity check your input - often without knowing what the limits are.

Take these two different examples.

In a text file, we might have:

#Maximum Widgets to fidget
maxW_to-F = 0

Whereas a GUI would show

How many Widgets do you want to fidget? 1 2 3 5

Even if you don't know the rules behind Widget fidgetting (must be a prime number lower than 7), the GUI won't let you choose a value that you can't select. The GUI doesn't prevent you setting an innapropriate value - just an illegal one. Your config file, however, could be set to any crazy value that a user type - often resulting in "unpredictable" results.

#Maximum Widgets to fidget
maxW_to-F = seventeen

It's with this in mind that I've made the following change to Dabr - the mobile Twitter client.

To Auth or Not To Auth? That Is The Question

Twitter's API has bug / peculiarity (reported to their discussion board) which causes Dabr to log a user out. Let me explain the steps

• User 1 (@private) has set her tweets to "protected".
• This means no one can see @private's tweet unless she allows them.
• @private has not allowed User 2 (@edent) to view her tweets.  She is protect from his view.
• @edent clicks to view @private's profile.
• @edent can see that @private has 42 friends, 17 followers and 3 favourites.

So far, this is the same behaviour on Twitter's website as it is through their API.  Here's the difference

Web

• @edent tries to see @private's followers and can see their names, profile pictures etc.
• @edent can also see @private's friends
• @edent cannot see @private's favourites (or even how many favourites she has)

API

• @edent tries to see @private's followers, friends or favourites
• Because @edent isn't allowed to see @private's info, the API returns 401 Authorisation Required.

This is where things get tricky. Dabr sees the 401 and concludes that the user has invalid credentials.  It then, as a security measure, clears the user's cookie and logs them out.

This may be a little harsh, but HTTP 401 essentially says that the authorisation has failed.

Fixing It

There are 3 ways that this could be fixed

1. Twitter could rationalise the API to allow access to the same content that a web user gets.
2. Twitter could return a different status code.
3. Dabr needs fixing.

So, how do we get Dabr not to log out when it receives a 401 only under these specific circumstances?

Looking at the code, we can see that Dabr simply sees the HTTP response code.  To fix it we'll need to pass extra parameters, check where the code was called from, investigate all the edge cases, add more logic to the system, futz around breaking things, etc... etc...

What a pain in the...

Or

Don't let users do things they can't do.

If a user can't see the information - why do we even let them try to see the information?  Why can't we just get rid of the link?

This is what a user currently sees:

Old Style

We've established that they can't view followers, friends and favourites.  So we can get rid of those links (but not the information).

New Style

(Incidentally, I've changed the order of the links.  I've tried to group together similar items.  Followers, friends, favourites and lists go together. Then DM. Finally, follow, block, report spam.)

Now a user cannot click through to an unwanted error message.

Or

There is another way round this.  With "Direct Messages" we could do the same thing - simply remove the link if you're not able to send that user a DM.

Instead, we've taken the approach of displaying a suitable error message.

Direct Message Warning

The advantage of this is that the user gets an explanation as to why they are unable to complete an action.

Your Thought?

Which do you prefer? Being unable to click on a link (with no explanation) or clicking on a link only to be given a warning message?

This was a problem I faced when trying to get Dabr to render the ASCII art⁰ produced by Aral Balkan's Feathers App. Feathers uses line breaks to achieve images like...

〰❀❃ Introducing ❃❀〰
My new iPhone app,
░░░▒█ Feathers █▒░░░

What's confusing to me, as a developer is the inconsistent way Twitter handles line breaks. For example...

Twitter's website ignores line breaks.

Desktop Spacing

Twitter Mobile, however, does line breaking correctly.

Android Spacing

Twitter has no style guide for rendering and - if it did - appears to render inconsistently. So, what's the correct thing to do?

In my opinion, respect the user. If a user has indicted that they want something, it is a developer's role to implement it (unless it causes severe problems for the system).

How to achieve it?

You could use all sorts of complex regular expressions to get line breaks and convert them. Thankfully, PHP has this function built in with nl2br() which will give you (X)HTML line breaks wherever a break occurs in the text.

Usually, I would be a big fan of this move - especially if it forces password anti-pattern sites like TwitPic to implement the new, secure standard.

This means that you won't be able to log in to a third party site by giving them your username and  password.  You will have to use OAuth to securely validate with the main Twitter site.

But - as ever - there's a dark side to OAuth.

Repressive Regimes

One of the joys of Twitter is that its clients are decentralised from the main site.

This means that if Twitter is blocked in your country, you can use a third party client (like Dabr) to access it.

Twitter User -> Dabr -> Twitter API -> Dabr -> User

If Dabr became sufficiently popular to be blocked by an oppressive regime, you can switch to any one of the thousands of Twitter web clients.

OAuth forces you to the main Twitter site.  While you may visit Dabr to start with, you would be redirected to Twitter to complete OAuth.  If Twitter is blocked, you can't connect.

At a stroke, Twitter has shut itself off to anyone in a repressive country.

This has been picked up by some concerned users.

A (Hacky) Way Around It

There's really only one way around this problem.  The third party web client has to act as a man-in-the-middle.  There's a patch for Dabr - developed by cnyegle - which will ask for a username and password, then proxy it to Twitter, get the OAuth token and tweet on behalf of the user.

From the user's point of view, they are still giving the (potentially untrusted) site the username and password.

Challenge Response

This could be solved by implemented a challenge / response system.

1. Alice visits the Dabr website.
2. Dabr asks Alice for her username (and only her username)
3. Dabr asks Twitter for the challenge associated with Alice's username.
4. Twitter checks that Dabr is an authorised website (i.e. has signed up for OAuth).
5. Twitter returns the response:  A secret phrase which Alice has previously chosen.
6. Dabr displays this phrase to Alice.
7. Alice knows that Twitter trusts Dabr
8. Dabr asks Twitter for the password challenge.
9. Twitter returns that it requires the 3rd, 5th and last character from Alice's password (the characters requested change randomly).
10. Dabr asks Alice for only those characters.
11. If Alice provides the correct characters, an OAuth token is granted to Dabr to tweet on behalf of Alice.

This has the advantage of proving that Dabr is trusted (by displaying Alice's pre-defined secret phrase) and mitigating the risk that Dabr is untrusted (by only revealing part of the password).

Conclusion

This is a very new area, and I've not had a chance to read through all of the proposals.  Nevertheless, it remains a fundamental problem that, if you can't access a site, you need to delegate your trust to someone else.

I'm not a security expert - so I would appreciate someone pointing out the flaws in my reasoning.

For a service which started out operating by SMS, Twitter takes a surprisingly unenlightened view of mobile. It's main mobile service - http://m.twitter.com/ - is almost completely devoid of useful features. That's one of the main impetuses behind the development of Dabr. Their latest mobile site - http://mobile.twitter.com/ - is really only suitable for the tiny minority of people who have smartphones.

So, understandably, many people use 3rd party sites like Dabr. They are now faced with a dilemma - give an untrusted site their username and password or try to use OAuth on the mobile.

A few weeks ago came the announcement that OAuth was finally ready for mobile... Was it? No. Once again a "mobile friendly" site designed with masses of JavaScript and guaranteed not to work with the majority of phones on the market.

Here's how mobile OAuth looks on a variety of popular mobile phones.

BlackBerry

BlackBerry Twitter OAuth

While this looks pretty enough, it doesn't work. The buttons aren't clickable. I've tried with and without JavaScript. No matter where I click, nothing happens.

Android

The Android's User-Agent isn't detected by Twitter as being a mobile phone. While it's true that the browser is very capable - the OAuth screen is a lot more usable when it's in mobile mode.

Android Twitter OAuth

So, it works, but it doesn't look nice.

N95

The N95 makes a good test phone because it's popular. Probably more popular than the iPhone.

N95 Twitter OAuth

N95 Twitter OAuth

It's not pretty - but at least it works.

Others

The Sharp GX-10 is my default test phone. One of the first phones with a colour HTML browser. If your site can work on this phone, it will work on any phone. There are no screenshot capabilities for this phone - but rest assured, it does not work.

The three phones I've demo'd above are very popular modern phones - AKA the minority. If they don't work well, what chance for the people using older phones?

Useless! How hard can it be? All it needs is a username field, a password field and a button. That's just about the most basic page imaginable. It should be child's play to make it work on mobile.

This was first raised in March 2009 on Twitter's issues list. It's currently the most popular bug.

So, we're stuck in a dire situation. Third-Party mobile sites get access to Twitter users' passwords because Twitter are unable or unwilling to develop a simple OAuth form. It would be fascinating to know how many of Twitter's security breaches are down to corrupt or insecure 3rd party sites which leak passwords.

I've tested this to work on the Vodafone 360 H1 - but it should work with any JIL handset.

Because it isn't certified, you may need to dial *#35767# to remove the H1's security check.

The code is very simple.  The JIL SDK allows you to call specific phone application - in this case, all I've done is invoked the browser.

<body>
   <script type="text/javascript">
      Widget.openURL("http://m.dabr.co.uk/");
   </script>
   <div>
      Launching Dabr!
   </div>
</body>

One thing to note, if you're deploying the widgets from your website, ensure you have set the MIME type to "application/widget" - as per the W3C standards - otherwise many devices won't recognise it as a valid widget.

If you're interest in developing for this platform, there's a €1,000,000 bounty up for grabs.  There's also a Dev Camp at Mobile World Congress this year.

I work for Vodafone - this is my personal blog, I don't to speak for them.  All opinions, claims and mistakes are my own.

I'm a big fan of AudioBoo - for those who don't know it,  AudioBoo allows you to record some audio on your phone and then publish it to the web.  It's like instant podcasting.  Originally only for the iPhone, AudioBoo now works on Android and there's an S60 version comeing out "soon".

I'm very proud to announce that as of r260 - Dabr now support AudioBoo!

It all started with a tweet from David Carrington (creator of Dabr) asking how one would go about linking directly to a tweeted 'boo.

So - how do I turn a tweeted 'boo like http://boo.fm/b73803 to the associated MP3 of http://audioboo.fm/boos/73803-rockbandboo.mp3?

Bring Me The API!

An API is a way for two computers to talk to each other.  The client (dabr) asks the server (AudioBoo) a question.  The server replies with some data and the client then does something with it.

AudioBoo's API is a "work in progress" - that's the polite term for "not quite finished". There are various APIs to get information about a 'boo. However, it's not particularly intuitive. For example, if I use /audio_clips/ http://api.audioboo.fm/audio_clips/b73803.mp3 I get an error. I have to drop the b to get http://api.audioboo.fm/audio_clips/73803.mp3

I asked on the AudioBoo forums if there was a better way but I didn't get a quick response - so I turned to twitter. One of the top AudioBoo bods, Mark Rock, is an active twitterer (seriously - isn't there a better word than that?)

Mark Rock

@markrock

Replying to @davidcarrington@davidcarrington @edent just updated server so this works - http://boo.fm/b1234.mp3

---

So, now I can link http://boo.fm/b73803 to http://boo.fm/b73803.mp3 - simple. Here's what it looks like in the browser.

AudioBoo

Phreadz

Phreadz is a service which allows you to post VITAL - Video Images Text Audio Links - it's a fantastic service, so when Kosso (its founder) asked me to integrate it into Dabr, I was keen to take on the challenge.

The code was quite simple and can be seen in r261. But there was one small complication - image sizes.

The thumbnail images from Phreadz aren't always suitable for mobiles. If images are too large, it will slow download speeds and potentially cost people money.

So, I used tinysrc.mobi. Tinysrc is a very simple API which will resize any image to a mobile friendly format. It detects the phone's user-agent string and creates a suitable sized image.

Here's what it looks like.

Phreadz

Conclusions

I've used 4 separate APIs here - twitter, AudioBoo, Phreadz, tinysrc - the joy of Web 2.0 is that all those disparate services can combine to create something new and magical.

API designers often need to think about how their products will be used "in the wild". It's very easy to create something that works very well for your requirements - but much harder to think about what others will need.

Getting the client was fairly simple, but could be better. Simply visiting http://seesmic.com was enough to bring up a mobile friendly page with download instructions. However, scrolling down presented this mess. You simply can't rely on users to know what make, model or firmware version they have. Use the user-agent string to do as much of the hard work as possible. If a user tries to install an incompatible version, it won't work and they'll blame you.

The download itself is a sprightly 172KB and installs very quickly - it also doesn't require a reboot. Once you've signed in, the main interface is very simple. UberTwitter, for example, presents the user with complex set-up options on the first run. Seesmic gets straight to the action. Seesmic is entirely menu driven - there are no shortcut keys. UberTwitter allows me to hit R for reply - on Seesmic, I have to delve into the menus. Luckily, they're short and have obvious names. Reading a tweet gives the usual option, hashtags and @s are hyperlinked - as are web addresses. On usability flaw, the cursor is at the top of the screen. A user has to scroll through the "Web" hyperlink before getting to the links within the tweet.

Again, because there are no keyboard shortcuts, everything has to go through the menu. Searching is problematic. Pressing enter doesn't start a search, it inserts a new line. The search button isn't the first thing you get to when you scroll down. While the results are standard, this odd error message kept popping up. I encountered this error several times.

Writing new tweet has some great usability touches - and some real clunkers. The closer a user gets to filling the 140 characters, the more red the screen turns. URLs can be automatically shortened and images can be added. This shows one of the great usability failures of Seesmic. Rather than use the BlackBerry's file select utility, it uses its own - and it's dreadful. Ugly looking, no ability to search, slow, no preview. Overall, a real let-down.

I don't know why companies often insist on creating their own versions of well established system functions. Especially when they add nothing and remove plenty.

Sending a tweet was continually problematic for me. Although it eventually relented. Seesmic also makes great use of notifies, showing you on your home screen how many unread tweets you have. Overall, Seesmic isn't a bad client. There are a few rough-around-the-edges bugs and the file selection is atrocious but other than that, it works. Power users like me will miss the shortcut keys of UberTwitter, and UT's system-wide integration (uploading photos, capturing hashtags in emails etc.).  It lacks other features such as in-line photos, ability to see followers and friends, it also has no way of marking a tweet as a favourite or seeing favourites.

One feature it does have - a first in mobile twitter clients - is the ability to view lists.  You can't add, edit or create - but you can see the lists you have created or are following.

Conclusion

Seesmic works well enough as a basic twitter client for BlackBerry - but there's nothing exciting about it.

The code is fairly simple - although I still suck at regular expressions.

if (preg_match_all('#youtube.com/watch?v=([_-dw]+)#', $tmp, $matches, PREG_PATTERN_ORDER) > 0)
 {
  foreach ($matches[1] as $match)
  {
  $images[] = theme('external_link', "http://m.youtube.com/watch?v={$match}", "<img src='http://i.ytimg.com/vi/{$match}/1.jpg' class='twitpic' />");
  }
 }

Here's how it looks.  A user writes a tweet which contains a YouTube URL

A thumbnail embedded in a tweet

The thumbnail is displayed and hyperlinked to the mobile version of YouTube

Pointing to Mobile YouTube

When clicked on, the user is taken straight to the video's page.

The Destination

There were three main issues with this code

1. How do we extract the video's ID from the URL? YouTube video IDs contain alphanumeric character and the - and _ characters. They can also have other parameters passed.  There can also be text after the URL.  The regular expression above takes care of them.
2. How do we get the thumbnail?  Previously, I had been using the image URL given in the m.YouTube page - something like http://i.ytimg.com/vi/plvE4882B7Y/default.jpg?w=80&h=60 - however those URLs now need signing. So, using the YouTube API, I took a look at the Video Information XML file - http://gdata.youtube.com/feeds/api/videos/plvE4882B7Y - it showed that all videos have a full sized thumbnail - http://i.ytimg.com/vi/plvE4882B7Y/0.jpg and a mobile friendly thumbnail - http://i.ytimg.com/vi/plvE4882B7Y/1.jpg
3. How do we link to the mobile version of YouTube?  There are two ways to do this - either allow YouTube to detect the User-Agent and redirect based on that, or simply point to the mobile version.  While a regular YouTube link looks like http://www.youtube.com/watch?v=plvE4882B7Y the mobile one simply substitutes www for m : http://m.youtube.com/watch?v=plvE4882B7Y

TODO

For phones which support playing YouTube videos inline (Android, some Nokias etc) I may actually embed the video, rather than just link to it.  I may also detect Desktop Browsers and send them to the regular version of YouTube.

So, hurrah for LongURLPlease - a service which takes those horrid little links and turns them in to full sized URLs.

Here's the basic code in PHP to use the service.

function long_url($shortURL)
{
 //Use the LongURLPlease API
 $url = "http://www.longurlplease.com/api/v1.1?q=" . $shortURL;

 //Set up CURL
 $curl_handle=curl_init();
 curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
 curl_setopt($curl_handle,CURLOPT_URL,$url);

 //Get the JSON response
 $url_json = curl_exec($curl_handle);

 //Close the connection
 curl_close($curl_handle);

 //Decode the JSON
 $url_array = json_decode($url_json,true);

 //Get the Long URL out of the array
 $url_long = $url_array["$shortURL"];

 if ($url_long == null) //The service couldn't find a long URL
 {
 return $shortURL;
 }

 return $url_long;
}

So, how should this be displayed to the user in Dabr?

There are three possibilities.

Display

<a href="short.url/foo">VeryVeryLong.Url/Bar</a>

This keeps the redirect, but it displays the full URL for the user.

Pros.

• Get to see what the content is.
• Allows the short URL service to record hits etc.

Cons.

• Won't highlight if you've already visited it.
• May cause problems if the VeryLong.Url is too long for the screen.

All Change

<a href="VeryVeryLong.Url/Bar">VeryVeryLong.Url/Bar</a> This completely replaces the short URL.

Pros.

• Hurrah! No more silly URLs.
• Will highlight if you've already visited it.
• You can seen what you're about to click on.

Cons.

• Breaks any statistics management the short URL service was performing (is this a con? Surely the receiving website should be recording such details?)
• May cause problems if the VeryLong.Url is too long for the screen.

Background

<a href="VeryVeryLong.Url/Bar">short.url/foo</a>

Pros.

• Will highlight if you've already visited it.
• You can seen what you're about to click on if you hover over the URL.
• Won't cause problems if the URL is too long for the screen.

Cons.

• Breaks any statistics management the short URL service was performing (is this a con? Surely the receiving website should be recording such details?)
• User can't immediately see what they're about to click on.

Conclusions

I prefer replacing the URL completely.  I see no value in the various short URL services.  The stats that they gather don't seem any better that those your webserver can provide.

However, long URLs can be messy - especially on small screened devices.

Long URLs on a BlackBerry

So, which do you prefer?