I decided to look at flooding data. The recent floods in the UK are a brutal remember of the realities of climate change and our poor stewardship of the nation's waterways.

The UK Government has a large collection of Flooding Data online - including some very detailed river-by-river data.

Initially, we thought it would be a great idea if every river in the country could Tweet. Telling local residents how high its water levels were and how likely it was to flood.

Unfortunately, the data isn't in brilliant shape. It's hard to find a reference guide to see what the data mean, where the measuring stations are, and which levels indicated likely flooding.

Instead, we settled on something simpler. There is a 3 Day Flooding Forecast XML file.

First, the good news! We were able to successfully parse the data and get a Twitter bot running which, once per day, publishes a three-day forecast.

UK Flood Forecast

@FloodForecast
Automated

The flood forecast for Wednesday is... pic.x.com/kml7fs4vln

---

The source-code is available on GitHub. In order to run it, you'll need 3 things.

1. Python 2.7.
2. Tweepy - needed to interact with Twitter.
3. Twitter Developer Tokens - needed to post to Twitter.

Once you've installed Python and Tweepy, and added in your OAuth keys to script, it can be run by simply issuing this command:

python 3dayflood.py

Ok, so now the bad news :-)

The data wasn't in a particularly great state. Let's go through it step-by-step.

• XML - ok, just a minor gripe. XML is just as easy to work with as JSON, but it's nice to be offered a choice.
• No valid schema - if you're going to use XML, you might as well do it properly!
• Welsh Dates. The UK Government has a statutory obligation to publish in English and Welsh - I don't have a problem with that. There is, however, no need to print dates in both languages. Dates should either be represented as ISO 8601 (2014-05-25T10:30:00+0100) or as a UNIX Timestamp (1401046863). That way, the programmer can easily determine the time and, if needed, format it in English, Welsh, French, Esperanto, and Klingon.
• Ideally, each day should be in its own object - rather than split several ways.
• The images... *sigh*... I was expecting that there would be a link to each image. Nope! It's a Base64 encoded PNG. Not terribly hard to decode, true, but not the best way. They are fairly low resolution, which is a shame.
• The summary stuff is fine:
  
  But the next lot of data are rather tangled.
• The model here is "Risk → Day → Area → Region". This seems somewhat illogical to me. Surely the user wants to see "Day → Region → Risk"? The area is fairly inconsequential - I don't care if my county is flooding, just if I am. The areas seem fairly nebulous and don't conform to any normal geo-spacial coding of which I'm aware.
• Or, perhaps, the model should be "Region → Day → Risk" - that way, rather than searching through each Risk in order to find my local area, I can get a direct forecast for my specific area and ignore everything else.
• Finally, there's no coding on the regions - they should at least have a WOEID, Lat/Long paid, or similar.

Here's a quick a dirty look at how I would have structured the data (in JSON).

{
    "date" : "2014-05-25T10:30:00+0100",
    "summary" : {
        "english" : "...",
        "cymru"   : "...."
    }
    "days" : [
        {
            "day"     : 1,
            "date"    : "2014-05-25",
            "image"   : "http://example.com/day1.png",
            "regions" : [
                {
                    "name" : "Cambridgeshire",
                    "risk" : "low",
                    "id"   : "123456"
                },
                {...},
                {...}
            ]
        },
        {...},
        {...}
    ]
    "regions" : [
        {
            "name" : "Cambridgeshire",
            "id"   : "123456",
            "days" : [
                {
                    "day"  : 1,
                    "date" : "2014-05-25",
                "risk" : "low"
                },
                {...},
                {...}
            ]
        },
        {...},
        {...}
    ]
    "risks" : [
        {
            "risk" : "low",
            "summary" : {
                "english" : "...",
                "cymru"   : "..."
            }
        },
        {...},
        {...}
    ]
}

You can get the code from GitHub. Enjoy!