In their post "How I Reversed Amazon's Kindle Web Obfuscation Because Their App Sucked" they describe the process of spoofing a web browser, downloading a bunch of JSON files, reconstructing the obfuscated SVGs used to draw individual letters, and running OCR on them to extract text.

There were a few problems with this approach.

Firstly, the downloader was hard-coded to only work with the .com site. That fix was simple - do a search and replace on amazon.com with amazon.co.uk. Easy!

But the harder problem was with the OCR. The code was designed to visually centre each extracted glyph. That gives a nice amount of whitespace around the character which makes it easier for OCR to run. The only problem is that some characters are ambiguous when centred:

When I ran the code, lots of full-stops became midpoints, commas became apostrophes, and various other characters went a bit wonky.

That made the output rather hard to read. This was compounded by the way line-breaks were treated. Modern eBooks are designed to be reflowable - no matter the size of your screen, lines should only break on a new paragraph. This had forced linebreaks at the end of every displayed line - rather than at the end of a paragraph.

So I decided to fix it.

A New Approach

I decided that OCRing an entire page would yield better results than single characters. I was (mostly) right. Here's what a typical page looks like after de-obfuscation and reconstruction:

As you can see - the typesetting is good for the body text, but skew-whiff for the title. Bold and italics are preserved. There are no links or images.

Here's how I did it.

Extract the characters

As in the original code, I took the SVG path of the character and rendered it as a monochrome PNG. Rather than centring the glyph, I used the height and width provided in the glyphs.json file. That gave me a directory full of individual letters, numbers, punctuation marks, and ligatures. These were named by fontKey (bold, italic, normal, etc).

Create a blank page

The page_data_0_4.json has a width and height of the page. I created a white PNG with the same dimensions. The individual characters could then be placed on that.

Resize the characters

In the page_data_0_4.json each run of text has a fontKey - which allows the correct glyph to be selected. There's also a fontSize parameter. Most text seems to be (the ludicrously precise) 19.800001. If a font had a different size, I temporarily scaled the glyph in proportion to 19.8.

Each glyph has an associated xPosition, along with a transform which gives X and Y offsets. That allows for indenting and other text layouts.

The characters were then pasted on to the blank page.

Once every character from that page had been extracted, resized, and placed - the page was saved as a monochrome PNG.

OCR the page

Tesseract 5 is a fast, modern, and reasonably accurate OCR engine for Linux.

Running tesseract page_0022.png output -l eng produced a .txt file with all the text extracted.

For a more useful HTML style layout, the hOCR output can be used: tesseract page_0022.png output -l eng hocr

Or, a PDF with embedded text: tesseract page_0022.png output -l eng pdf

Mistakes

OCR isn't infallible. Even with a high resolution image and a clear font, there were some errors.

• Superscript numerals for footnotes were often missing from the OCR.
• Words can run together even if they are well spaced.
• Tesseract can recognise bold and italic characters - but it outputs everything as plain text.

What's missing?

Images aren't downloaded. I took a brief look and, while there are links to them in the metadata, they're downloaded as encrypted blobs. I'm not clever enough to do anything with them.

The OCR can't pick out semantic meaning. Chapter headings and footnotes are rendered the same way as text.

Layout is flat. The image of the page might have an indent, but the outputted text won't.

What's next?

This is very far from perfect. It can give you a visually similar layout to a book you have purchased from Amazon. But it won't be reflowable.

The text will be reasonably accurate. But there will be plenty of mistakes.

You can get an HTML layout with hOCR. But it will be missing formatting and links.

Processing all the JSON files and OCRing all the images is relatively quick. But tweaking and assembling is still fairly manual.

There's nothing particularly clever about what I've done. The original code didn't come with an open source software licence, so I am unable to share my changes - but any moderately competent programmer could recreate this.

Personally, I've just stopped buying books from Amazon. I find that Kobo is often cheaper and their DRM is easy to bypass. But if you have many books trapped in Amazon - or a book is only published there - this is a barely adequate way to liberate it for your personal use.

The Worker as Futurist project asks another question: if SF is so important to the operations of capitalism in the twenty-first century, does it also have a radical potential that might lend itself to the struggles of workers and other oppressed people within, against and beyond that system? (see Jameson 2005; Roke 2020)

I'll be honest, it feels a little patronising. The project articulates its theory of change (which only slightly puts me in mind of Peter Cook's famous quote) - hoping that radical fiction might move mountains:

We at the Worker as Futurist project believe that workers can fruitfully understand, theorize, and plot forms of resistance to capitalism through creative expression, especially through the process of writing, particularly within the genre of SF. If, as we have argued, SF is now active in important ways at the very heart of capitalism, maybe that genre is also somehow the system’s achilles’ heel?

Still, it makes for a fascinating introduction to a somewhat uneven set of stories. There is some delightfully weird fiction - I especially enjoyed the idea of the University of the Phoenix using AI to write curses on corporations. In amongst that there's some fairly standard stories of worker exploitation, a few overwritten pieces, and one or two which would make great full length books.

The afterword is very clear about the reason for this book existing:

To me, it never mattered whether or not the stories written by the workers in this project would be considered “good” by the well-read literary elite. To me, it only mattered that those who took up the challenge wrote. And by writing, and by speaking their truths, they exploded the paradigm that would render their subjectivity nonexistent.

And I heartily agree with that. I'd rather read a new writer explore the problem-space rather than yet-another trilogy about romance magicians.

The book is free to download or, if you think the master's tools can dismantle the master's house, you can purchase a copy on Amazon.

Background

Amazon have decided to stop letting customers download their purchased eBooks onto their computers. That means I can't strip the DRM and read on my non-Amazon eReader.

So I guess I'm not spending money with Amazon any more. I'm moving to Kobo for three main reasons:

1. They provide standard ePubs for download.
2. ePub DRM is trivial to remove.
3. Kobo will undercut Amazon's prices!

Here's the thing. I want to buy my eBooks. It is trivial to pirate almost any modern book. But, call me crazy, I like rewarding writers with a few pennies. That said, I'm not made of money, so I want to get the best (legal) deal possible.

Kobo do a price-match with other eBook retailers. It says:

We'll award a credit to your Kobo account equal to the price difference, plus 10% of the competitor’s price.

I found a book I wanted which was £4.99 on Kobo. The Amazon Kindle price was £4.31.

4.99 - ( (4.99 - 4.31) + (4.31 * 0.1) ) = 3.88

I purchased the book, sent a request for a price match, and got this email a few hours later:

OK! So what steps can we automate, and which will have to remain manual?

Amazon Pricing API

Amazon have a Product Advertising API. You will need to register for the Amazon Affiliate Program and make some qualifying sales before you get API access.

In order to search for an ISBN and get the price back, you need to send:

{
 "Keywords": "isbn:9781473613546",
 "Resources": ["Offers.Listings.Price"],
}

Using the updated Python API for PAAPI:

from paapi5_python_sdk import DefaultApi, SearchItemsRequest, SearchItemsResource, PartnerType

def search_items():
    access_key = "ABC"
    secret_key = "123"
    partner_tag = "shkspr-21"
    host = "webservices.amazon.co.uk"
    region = "eu-west-1"

    api = DefaultApi(access_key=access_key, secret_key=secret_key, host=host, region=region)

    request = SearchItemsRequest(
        partner_tag=partner_tag,
        partner_type=PartnerType.ASSOCIATES,
        keywords="isbn:9781473613546",
        search_index="All",
        item_count=1,
        resources=["Offers.Listings.Price"]
    )

    response = api.search_items(request)

    print(response)

search_items()

(Add your own access key, secret key, and tag. You may need to change the host and region depending on where you are in the world.)

That returns something like:

{
    "search_result": {
        "items": [
            {
                "asin": "B09JLQHHXN",
                "detail_page_url": "https://www.amazon.co.uk/dp/B09JLQHHXN?tag=shkspr-21&linkCode=osi&th=1&psc=1",
                "offers": {
                    "listings": [
                        {
                            "price": {
                                "amount": 2.99,
                                "currency": "GBP",
                                "display_amount": "£2.99"
                            }
                        }
                    ]
                }
            }
        ]
    }
}

(I've truncated the above so it only shows the relevant information.)

Kobo ISBN & Price

Let's get the ISBN and Price of a book on Kobo. There's no easy API to do this. But, thankfully, Kobo embeds some Schema.org metadata.

Look at the source code for https://www.kobo.com/gb/en/ebook/venomous-lumpsucker-1

<div id="ratings-widget-details-wrapper" class="kobo-gizmo"
     data-kobo-gizmo="RatingAndReviewWidget"
     data-kobo-gizmo-config ='{&quot;googleBook&quot;:&quot;{\r\n  \&quot;@context\&quot;: \&quot;http://schema.org\&quot;,\r\n  \&quot;@type\&quot;: \&quot;Book\&quot;,\r\n  \&quot;name\&quot;: \&quot;Venomous Lumpsucker\&quot;,\r\n  \&quot;genre\&quot;: [\r\n    \&quot;Fiction \\u0026 Literature\&quot;,\r\n    \&quot;Humorous\&quot;,\r\n    \&quot;Literary\&quot;\r\n  ],\r\n  \&quot;inLanguage\&quot;: \&quot;en\&quot;,\r\n  \&quot;author\&quot;: {\r\n    \&quot;@type\&quot;: \&quot;Person\&quot;,\r\n    \&quot;name\&quot;: \&quot;Ned Beauman\&quot;\r\n  },\r\n  \&quot;workExample\&quot;: {\r\n    \&quot;@type\&quot;: \&quot;Book\&quot;,\r\n    \&quot;author\&quot;: {\r\n      \&quot;@type\&quot;: \&quot;Person\&quot;,\r\n      \&quot;name\&quot;: \&quot;Ned Beauman\&quot;\r\n    },\r\n    \&quot;isbn\&quot;: \&quot;9781473613546\&quot; …'>
</div>

Getting the data from the data-kobo-gizmo-config is a little tricky.

• Using Python Requests won't work because Kobo seem to run a JS CAPTCHA to detect scraping.
• There is a Calibre-Web Kobo plugin but it requires you to have a physical Kobo eReader in order to get an API key.
• The Rakuten API is only for the Japanese store.

So we have to use the Selenium WebDriver to scrape the data:

from selenium import webdriver
from bs4 import BeautifulSoup
import json

#   Open the web page
browser = webdriver.Firefox()
browser.get("https://www.kobo.com/gb/en/ebook/venomous-lumpsucker-1")

#   Get the source
html_source = browser.page_source

#   Soupify
soup = BeautifulSoup(html_source, 'html.parser')

#   Get the encoded JSON Schema
schema = soup.find_all(id="ratings-widget-details-wrapper")[0].get("data-kobo-gizmo-config")

#   Convert to object from JSON
parsed_data = json.loads(schema)

#   Decode the nested JSON strings
parsed_data["googleBook"] = json.loads(parsed_data["googleBook"])

#    Get ISBN and Price
price = parsed_data["googleBook"]["workExample"]["potentialAction"]["expectsAcceptanceOf"]["price"]
isbn  = parsed_data["googleBook"]["workExample"]["isbn"]
print(isbn)
print(price)

Kobo Wishlist

OK, nearly there! Given a Kobo book URl we can get the price and ISBN, then use that ISBN to get the Kindle price. But how do we get the Kobo book URl in the first place?

I'm adding all the books I want to my Kobo Wishlist.

Inside the Wishlist is a scrap of JavaScript which contains this JSON:

{
    "value": {
        "Items": [
            {
                "Title": "Venomous Lumpsucker",
                "Price": "£2.99",
                "ProductUrl": "/gb/en/ebook/venomous-lumpsucker-1",
            }
        ],
        "TotalItemCount": 11,
        "ItemCountByProductType": {
            "book": 11
        },
        "PageIndex": 1,
        "TotalNumPages": 1,
       }
}

(Simplified to make it easier to understand.)

Although there's a price, there's no ISBN, So you'll need to use the "ProductUrl" to get the ISBN and Price as above.

Sadly, unlike Amazon, there's no way to publicly share a wishlist. Getting the JSON requires logging in, so it's back to Selenium again!

This should be enough:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import time

browser = webdriver.Firefox()
browser.get("https://www.kobo.com/gb/en/account/wishlist")

#       Log in
username_box = browser.find_element(By.NAME, "LogInModel.UserName")
username_box.clear()
username_box.send_keys('you@example.com')

password_box = browser.find_element(By.NAME, "LogInModel.Password")
password_box.clear()
password_box.send_keys('p455w0rd')

password_box.send_keys(Keys.RETURN)

time.sleep(5) # Wait for load and rendering

But the Kobo presents a CAPTCHA which prevents login.

There is an unofficial API which, sadly, doesn't seem to work at the moment.

Next Steps

For now, I'm saving specific Kobo book URls into a file and then running a scrape once per day. Hopefully, the unofficial Kobo API will be working again soon.

It was quite the culture shock.

To us, it felt weird, insincere, and creepy. But, like the over-enthusiastic whooping and cheering from US sitcom audiences, it seemed to be an accepted part of the culture.

A few years later, the Disney Store in the UK decided that they needed to import this style of greater. I remember walking into a store and seeing a clearly mortified teenager reciting the same spiel as in the American stores.

It was not a success. British culture doesn't really do over-the-top, flowery, purple-prose. It reads to us as sarcasm. I'm not saying this is an attractive characteristic of the average Britisher - but it is noticeable. It causes great cognitive dissonance when American friends and colleagues give us praise - surely they are being insincere? But, no, everything is awesome to them.

With the rise of International English, most countries' linguistic foibles get absorbed into the zeitgeist. We do the needful once we have circled back and then get into a muddle over what the verb "to table" means. Mostly, it works. Mostly.

Recently, I had a chat with an Amazon UK customer service assistant. I assume that they weren't in the UK and were outsourced to a country where the weather is hotter and the standard of written English is good enough. They were incredibly helpful so I finished my conversation with a well-intentioned "have a lovely day".

In return, I got this:

Wow! My initial thought was that they were being heavily sarcastic. If someone in a UK shop had said that to me in person, I'd assume they wanted to step outside for a punch-up. Seriously, that's fighting words over here! And the implied threat to my family...?

Of course, it is none of that. They probably have a list of canned responses which they can pre-select. Responses which have been extensively focus-grouped and found to be awesome. Or maybe I wasn't talking to a human - perhaps a bot just decided to repay my compliment as though I were an American.

Anyway, if you're doing prompt-engineering on an LLM please be sure to set the sarcasm detection to the lowest common denominator!

Reactions from the Fediverse

Except for new book notifications.

I saw a pulsing yellow light on the dot. I've memorised all of the various signs and portents the accurs'd device can summon up, so I asked it (in English) what notifications it had for me. It replied, naturally, in German.

I couldn't grab an audio recording in time, and I've no idea which command invokes a repetition. But the Alexa app contains a handy history of every utterance to spew forth from its speaker.

To be clear, my IP address shows that I am in the UK. I have never bough German lager from Amazon. I've never watched Das Boot on Amazon Prime. I haven't even asked Alexa to play the soundtrack to Cabaret!

Whatever. Eine Kline Glitche um Der Matrix.

And then, a few weeks later, it happened again!

Was ist das? Ein großer Haufen Schweinescheiße!

To be even clearer - neither of the authors are German. I stopped learning German when I was 14. My liebe Frau happens to have German A-Level but she swears blind that she only uses it to speak with Hans, our flaschenspüler - and never in front of Alexa.

So what gives?

As far as I can tell, neither of these authors has a new book out. But a German translation of their work has been published. Amazon knows I follow these authors and, because Amazon are Kapitalistische Warzenschweinficker, will gently encourage me to by more of their works. Ordinarily, that might be useful. But as I don't speak the language, it's all a bit pointless.

For some reason, this only seems to happen with German translations. No idea why.

It seems that Amazon's prowess in deep-artificial-learning-neural-technobabble doesn't actually extend to anything useful - like making sure it doesn't read out garbled multilingual sentences.

What a bunch of Kartoffelköpfe!

Amazon don't offer an API for wishlists (innovative my shiny metal arse). So this uses Beautiful Soup to grab the data from the HTML. To be fair, there's also some microdata on the page, which makes things slightly easier.

The code is available on github.com/edent/Amazon-Wishlist-Pricedrop-Alert - but here's a brief walkthrough of parsing and dealing with pagination:

First up, the standard boilerplate for getting and parsing HTML

from bs4 import BeautifulSoup
import requests
import re
import json

I start by passing this function my wishlist URL https://www.amazon.co.uk/gp/registry/wishlist/1A1NYHTAZ3N6V/. Amazon only returns 10 items per page of HTML, but it includes a "pagination" URL at the end of each page. Later on, we'll recursively call this function based on the "next page" URL.

def get_wishlist(url):
    #   Load the wishlist
    response = requests.get(url)
    page_html = response.text
    #   Parse the page
    soup = BeautifulSoup(page_html, 'html.parser')
    return soup

The item name - for example "Fly Fishing by JR Hartley" - needs to be scraped out of the HTML. It looks something like this:

<a id="itemName_IS7IZUKCMTBZG" 
   class="a-link-normal" 
   title="Fly Fishing by JR Hartley"
   href="...">
   Fly Fishing by JR Hartley
</a>

Use BeautifulSoup to grab the string inside the anchor. Use a regex to match anything starting with itemName_.

def get_items(soup):
    #   Get the item names
    for match in soup.find_all('a', id=re.compile("itemName_")):
        item = match.string.strip()
        item_list.append(item)

Prices and product IDs are slightly easier to scrape. They have microdata and JSON. For example:

<li data-id="1A1NYHTAZ3N6V"
    data-itemId="I3VQB497DB6HJF"
    data-price="8.07"
    data-reposition-action-params="{"hasComparisonTable":false,"itemExternalId":"ASIN:B0962VD2G8|A1F83G8C2ARO7P","listType":"WishList","sid":"260-3142506-2992056"}"
    class="a-spacing-none g-item-sortable">...

The JSON in data-reposition-action-params parses out to:

{
  "hasComparisonTable": false,
  "itemExternalId": "ASIN:B0962VD2G8|A1F83G8C2ARO7P",
  "listType": "WishList",
  "sid": "260-3142506-2992056"
}

Beautiful soup can grab the data-price and then the Amazon ID:

def get_prices_and_ids(soup):
    #   Get the price and ID from data attributes
    for match in soup.find_all("li", class_="g-item-sortable"):
        price = match.attrs["data-price"]
        price_list.append(price)
        json_data = json.loads(match.attrs["data-reposition-action-params"])
        # Will be something like "ASIN:B095PV5G87|A1F83G8C2ARO7P"
        id = json_data["itemExternalId"].split(":")[1].split("|")[0]
        id_list.append(id)

That will get us all the names, IDs, and prices on the first page. What about subsequent pages?

At the bottom of the page is this HTML:

<input
   type="hidden"
   name="showMoreUrl"
   value="/hz/wishlist/slv/items?filter=unpurchased&paginationToken=eyJGcm9tVV...MDl9&itemsLayout=LIST&sort=default&type=wishlist&lid=1A1NYHTAZ3N6V"
   class="showMoreUrl"/>

Retrieving the URL gives another HTML page with the next set of wishlist items on it. The last page of results has <div id="endOfListMarker"> which we can use to detect when to stop getting pages.

def get_paginator(soup):
    paginator = None
    ##  Find the paginator
    if soup.find("div", {"id": "endOfListMarker"}) is None:
        #   If the end tag doesn't exist, continue
        for match in soup.find_all('input', class_="showMoreUrl"):
            paginator = "https://www.amazon.co.uk" + match.attrs["value"]
    else:
        paginator = None
    return paginator

Putting it all together looks like this:

#   Set up the lists
item_list  = []
price_list = []
id_list    = []

#   Message text
message = "Here are the recent price drops:\n"

def get_all(url):
    global counter
    counter = counter + 1
    print( "Getting page " + str(counter))
    soup = get_wishlist(url)
    get_items(soup)
    get_prices_and_ids(soup)
    paginator = get_paginator(soup)
    if paginator is not None:
        get_all(paginator)

#   Get all the items on the wishlist
#   Which page are we on?
counter = 0
get_all("https://www.amazon.co.uk/gp/registry/wishlist/1A1NYHTAZ3N6V/")

That stores all the data in some lists. Next up, put them in a Pandas DataFrame for ease of access:

#   Place into a DataFrame
all_items = zip(id_list, item_list, price_list)
new_prices = pd.DataFrame(list(all_items), columns = ["ID", "Name", "Price"])

For comparing prices, I'm just being lazy. All I care about is if the price has dropped since the last time I looked. I don't care what the highest or the lowest price is.

I save the prices as a CSV. The next time the code runs it reads that CSV into a DataFrame called old_prices

#   Read the old file
if exists("old_prices.csv"):
    old_prices = pd.read_csv("old_prices.csv")
else:
    old_prices = new_prices.copy()

Now it's a case of iterating through the new prices and comparing each item with its old price.

If the price has dropped, send me a message. If the price of a book is under a quid, I also want to know that as well.

#   Compare prices
for id in new_prices["ID"]:
    new_price = new_prices.loc[new_prices["ID"]==id, "Price"].values[0]
    name      = new_prices.loc[new_prices["ID"]==id, "Name" ].values[0]
    #   If a book has recently been added to the wishlist, it won't have an old price
    if id in old_prices.values:
        old_price = old_prices.loc[old_prices["ID"]==id, "Price"].values[0]
        #   Anything less than a quid is good knowing about.
        #   Some prices are ""-Infinity", so check the price is more than zero
        if float(new_price) < 1 and float(new_price) > 0:
            message += (name + "\n£" + str(new_price) + " was £" + str(old_price) + " https://www.amazon.co.uk/dp/"+id + "\n")
        elif float(new_price) < float(old_price) and float(new_price) > 0:
            message += (name + "\n£" + str(new_price) + " was £" + str(old_price) + " https://www.amazon.co.uk/dp/"+id + "\n")

print(message)

There's a slight wrinkle because some of the prices are -Infinity

Terence Eden is on Mastodon

@edent

Yay The Semantic Web! pic.x.com/aY20t5hGoG

---

That's usually when the item is unavailable.

Next, overwrite the old_files.csv with today's prices - we'll use these tomorrow as the old prices.

#   Save the Data
new_prices.to_csv('old_prices.csv', index=False)

Finally, how does the message get sent? It's OK to print something on the console - but I want to get the alerts via email.

This uses smtplib to send something via your own SMTP server. You'll need to add your own email server details here!

import smtplib
from email.message import EmailMessage

#   Send Email
def send_email(message):
    email_user = 'me@example.com'
    email_password = 'P455w0rd!'
    to = 'you@example.com'
    msg = EmailMessage()
    msg.set_content(message)
    msg['Subject'] = "Today's price drops"
    msg['From'] = email_user
    msg['To'] = to
    server = smtplib.SMTP_SSL('smtp.example.com', 465)
    server.ehlo()
    server.login(email_user, email_password)
    server.send_message(msg)
    server.quit()

And that's pretty much it! Stick it in a crontab and have it run once per day. Amazon can get a bit funny about screen scraping, so best not to run it too often or you'll hit an IP-based CAPTCHA.

There are a couple of minor annoyances.

• Some prices fluctuate by pennies. Perhaps you only want to know if the price is lower by more than £1, or more than 10%?
• No tracking the lowest price. Do you want to know if it's the lowest price in the last few weeks?
• Doesn't automatically buy stuff. It would be nice to just buy a book any time it drops to be lower than a quid.

The full code is available on github.com/edent/Amazon-Wishlist-Pricedrop-Alert.

Enjoy!

"Alexa... Notifications?"

"You have one notification. An item on your wishlist has dropped in price. The … is now only £…"

And that's how my wife found out what I planned to get her for her birthday!

What happened to cause this? I maintain several Amazon Wishlists® of things I want to buy. One of those is for presents I might want to buy my wife - and it is set to private. If you want to buy me a present, you can view my public wishlist. But my private ones are private to me.

If you go to edit your wishlist's privacy settings, you'll see this small disclaimer:

Going over to https://www.amazon.co.uk/alexashopping/notification there's this toggle switch.

So, there's no way to switch off notifications from private lists. You have to switch them off for everything.

In this case, the harm was minimal. I'll have to find something else as a surprise gift. But imagine if I had a "private" wishlist for something embarrassing or upsetting? I don't remember ever switching on the option for my Alexa to announce to my entire household that there is a price-drop on my weird fetish.

I've written before about anti-social app design. The tech bros working on apps often don't consider that people have families. And that they live with people that they want to keep secrets from.

According to the UK's Office of National Statistics - only 28% of households contain a single occupant. The majority of people live with other people.

What to do next

If you use private wishlists, and have an Alexa, you have a few of options.

1. Stop using Amazon's wishlists, and keep a separate list elsewhere.
2. Turn off all price drop notifications.
3. Write a ranty blog post and hope a product manage at Amazon takes notice.

Pre-empting Your Comments

Before responding to this post, please consider the following:

"Why didn't you read the disclaimer?"

I set up this wishlist long before the Alexa was invented. The disclaimer didn't exist then.

"You should have gone through every single option and made sure you were happy!"

That's unrealistic. Options should be set to preserve privacy by default. Asking the user to go through dozens of different pages of options to prevent their privacy being violated is an unreasonable burden.

"This is your fault for being tied in to Amazon's ecosystem!"

Possibly.

jEfF BeZoS Is aLwAyS LiStEnInG To yOu!!1!!!!111!

As long as he gives me cheap same-day delivery, IDGAF.

Here's "Brian" - their English, male, received pronunciation voice - reading John Betjeman's poem "Slough":

The pronunciation of all the words is incredibly lifelike. If you heard it on the radio, it might sound like a half-familiar BBC presenter. It has a calm, even tone which suits the poem splendidly.

The rhythm is also spot on. That's mostly a function of the short lines and helpful punctuation the poem contains. Much like iambic pentameter, or a limerick, the syllables lend themselves to a specific and identifiable cadence.

But the emphasis is all wrong. The poem just... ends. There's no sense of finality in the tone. You'd expect a competent reader to recognise "tinned minds" as being worthy of stressing. Polly does have some capability to mark specific words for emphasis, but it's all very manual.

There's no synthetic emotion. Do you feel the rage, desperation, sadness, hopelessness of the poem? While Polly has some SSML (Speech Synthesis Markup Language) support - the range of emotions it can express are severely limited. And, again, must be applied manually.

"I used to be an adventurer like you, but then i took an arrow in the knee!"

One of the reasons stock phrases pop up so often in video games is that it is expensive to write and record thousands of different lines of dialogue.

We're almost at a stage where a computer can procedurally generate lines for background characters to speak, and then "record" an audio version in an array of styles. No more expensive voice actors, no more memetic references for in-group homophily. Each player of a game will have a completely different dialogue experience.

But the bit that we're still missing is the automation of emphasis and emotion and comic timing and understatement and... all the things which trained actors spend years learning how to do successfully.

In 2011, the film critic Roger Ebert had surgery which eliminated his voice. He proposed the following "Ebert Test" for synthetic voices:

If the computer can successfully tell a joke, and do the timing and delivery, as well as Henny Youngman, then that’s the voice I want.

We're so close, I can taste it. The Turing Test for realistic voices is whether they can move the audience to tears with poetry.

The Amazon 4K Fire Stick is a pretty cool bit of kit. It's an Android device which can push 4K HDR video with surround sound to your TV. But, like any technology, it's buggy. Here's the bug report I've tried to send to Amazon and the BBC. They haven't answered - so I'm publishing it here as well.

BBC iPlayer supplies 4K video - but the app doesn't switch to 4K mode, it stays at 1080p.

Amazon debug

To turn on the Fire stick's debug mode - and get access to X-Ray - follow these instructions.

This lets you see the bandwidth being used, HDCP information, and audio/video information. Here's a 4K presentation via Amazon's own video player. In the top left is the HDMI output information and bandwidth stats. On the right is the video codec data.

iPlayer

Within iPlayer you can opt-in to the beta programme, this gives you access to 4K content. But let's take a look at the debug information. 4K video is being streamed at 25fps. But if we look in the top left corner, the app hasn't switched out of 1080p @ 60fps.

The end result is a juddering rescaled mess.

So, if you or a friend works for either Amazon or the BBC, please ask them to take a look, eh?

I recently bought the new 4K Amazon Fire Stick. It's a little Android dongle which plays videos. It's neat - but quite often displays weird text errors.

Take the kids' TV show House of Anubis, the Fire displays the description like this:

Looking at the source code for the description:

That's the character "private use two" (U+0092). What on earth is that doing there?

Well, in the ancient Windows-1252 encoding, 0x92 is ’ - the curly quote. It looks like someone has tried to write "in the house’s attic" on an old Windows machine, Amazon have ingested the data thinking it was Unicode, and displayed an error!

Here's "Young Sheldon":

How does the word "isn't" become "isn’t"? There's a great explanation from Justin Weiss, basically:

• Unicode ’ (U+2019) contains 3 bytes - 0xE2 0x80 0x99
• When those three bytes are converted to Windows-1252, they become â € ™

The Android version of Prime video goes a step further:

The Euro Symbol has been converted into its HTML entity €

There are hundreds of examples of video descriptions being mangled like this.

And, occasionally, you find this hot mess: What's going on?

Two countries separated by a common language

Except, of course, this doesn't happen on the American version of Amazon Prime Video.

Here's the description of Gotham S03 from Amazon USA:

Once the text gets to this side of the pond, it goes horribly wrong.

As well as the â€TM issue, there's another new snag. The hyphen is actually - hyphen-minus (U+002D). Something in Amazon's conversion process is transforming that to – en dash (U+2013).

That then gets buggily encoded to â€".

I've reported these errors to my friends at Amazon - they are in the process of correcting them.

You can buy the Amazon Fire TV Stick 4K Ultra HD for just £50.

Amazon Alexa is a fun little bit of kit. But it can be tricky getting it to work with all your smart devices. Not every company has an Alexa skill - just like not every company has an app.

Using Flask-Ask it is possible to bring Alexa smarts to a range of previously mute devices.

Alexa coding works on "intents" - the following is a simple intent. That is, you can only ask the skill one thing. No state is maintained, no multiple commands to get right, no complexity. This gets information from a single source and speaks it.

Code

The basic Python is pretty simple and can be adapted to query almost any basic JSON API. Let me walk you through it.

All the boilerplate needed to set up the skill:

import logging
from operator import itemgetter

import requests
from flask import Flask
from flask_ask import Ask, statement

app = Flask(__name__)
ask = Ask(app, '/')
logger = logging.getLogger()

@ask.launch
def launch():
    return stats()

The API that you want to call. This is a basic JSON API which doesn't require authentication.

ENDPOINT = "https://example.com/api/maslow.json"

Here's the main part of the Skill. When the intent is triggered, call the API. Get the data and format it for speech. If there was an error, tell the user.

Make sure that the name of the intent is identical to the one you set up in the Alexa Developer console.

@ask.intent("BatteryIntent")
def stats():
    r = requests.get(ENDPOINT)
    data = r.json()

    if r.status_code == 200:
        percent = data['battery/amphours']
        speech = "Your Moixa battery is at " + str(int(round(percent))) + " percent capacity right now."
    else:
        speech = "There was a problem connecting to the battery."

    logger.info('speech = {}'.format(speech))
    return statement(speech)

Short and easy.

Deploying

Again, Amazon don't make it easy to deploy Alexa skills - here's a guide to getting started using Zappa.

Remember, Alexa is not AI. You must painstakingly type in all the "utterances" that you think your users might say to activate the skill.

Problems

There is one major problem with retrofitting Alexa skills.

Lots of Internet connected devices have no ability to log in remotely - and certainly don't have the OAuth systems that Amazon demands. Alexa has no ability to directly connect to IP addresses on its own subnets.

This means most skills will need hard-coded credentials - and a way to traverse into your network.

In short, this means means they can't be shared on the Amazon store.

Is today recycling day? Or green waste day? Or humble landfill day? Should my food caddy go out? Gosh! Who can keep track? I mean, obviously I could look at my calendar. Or sign up for free SMS alert. Or use push notifications in their app. But where's the fun in that?

"Alexa..."

{
  "uid": "https://example.com/bins/",
  "updateDate": "2017-07-04T00:00:00+01:00",
  "titleText": "Bins",
  "mainText": "Today's bin collection is Green Refuse Bins and Food Caddies",
  "redirectionUrl": "https://www.oxford.gov.uk/homepage/154/find_your_bin_collection_day"
}

https://apps.cloud9technologies.com/oxford/citizenmobile/openapi/addresses?postcode=OX1%203HZ

https://apps.cloud9technologies.com/oxford/citizenmobile/openapi/wastecollections/200004675068

<?php

$wasteURL = "https://apps.cloud9technologies.com/oxford/citizenmobile/openapi/wastecollections/";
$uprn = "";

//  Get the data
$data = file_get_contents($wasteURL . $uprn);

//  Data is in JSON
$json = json_decode($data, true);

//  Strip out unneeded elements
unset($json["WasteCollectionDates"]["UPRN"]);

//  An array to hold the collection information
$collections = array ();

//  Loop through the array
//  Store the data as timestamp=>bins
foreach ($json["WasteCollectionDates"] as $key => $value) {
    $details =  $value["ContainerDescription"];
    $date = strtotime($value["CollectionDate"]);
    if ($collections[$date] != null)
    {
        //  Multiple bins can be collected each day
        $details = " and " . $details;
    }

    //  Ignore empty elements
    if ($date != null)
    {
        $collections[$date] .= $details;
    }
}

//  Date order the array
ksort($collections);
//  Place pointer at the start of array
reset($collections);

//  The array key is a UNIX timestamp
$first_date = key($collections);
$first_bins = $collections[$first_date];

//  Is the next bin collection today, tomorrow, or further in the future?
//  https://stackoverflow.com/a/25623230/1127699
$today = new DateTime(); // This object represents current date/time

$today->setTime( 0, 0, 0 ); // reset time part, to prevent partial comparison

//  Get the timezone as the bins API doesn't return it
$timezone = $today->getTimezone();

//  What is the date of the next bin collection?
$match_date = DateTime::createFromFormat("U", $first_date);
//  Set the timezone based on the local time
$match_date->setTimezone($timezone);
$match_date->setTime( 0, 0, 0 ); // reset time part, to prevent partial comparison

$diff = $today->diff( $match_date );
$diffDays = (integer)$diff->format( "%R%a" ); // Extract days count in interval

$headline = "";

switch( $diffDays ) {
    case 0:
        $headline = "Today's bin collection is " . $first_bins;
        break;
    case +1:
        $headline = "Tomorrow's bin collection is " . $first_bins;
        break;
    default:
        $headline = "The next bin collection is " . $first_bins . " on ". $match_date->format('l');
}

//  Set the correct header for JSON data
header('Content-Type: application/json');
?>{
  "uid": "https://example.com/bins/",
  "updateDate": "<?php echo $today->format(DateTime::ATOM); ?>",
  "titleText": "Bins",
  "mainText": "<?php echo $headline; ?>",
  "redirectionUrl": "https://www.oxford.gov.uk/homepage/154/find_your_bin_collection_day"
}

I've tried previously to create an interactive Alexa skill - it did not go well. This time, I thought I'd create an easier skill - a Flash Briefing. You ask Alexa for your daily news report and it reads out items that you've configured - news, weather, traffic, that sort of thing.

For a basic skill, all you need is a tiny scrap of JSON. In this case:

{
  "uid": "https://example.com/flood/index.php",
  "updateDate": "2017-07-02T09:30:00Z",
  "titleText": "UK Flood Forecast",
  "mainText": "The overall flood risk is VERY LOW for the next five days. Minor surface water flooding impacts are possible, but not expected, on Thursday.",
  "redirectionUrl": "https://flood-warning-information.service.gov.uk/"
}

If you can dynamically generate something like that, and host it on a website, you can make a skill.

Following the Amazon process is slightly tedious - especially when it gives you errors like this.

Pro Tip - tell your users why and where something has failed. They can't be expected to magically guess what the problem is.

You can beta test a skill before it goes live. The submission and review process was painless - took about 12 hours on a weekend.

Voice Interfaces For Government

(My thoughts - not my employer's)

This is an unofficial service that I've built using the Open Data produced by my friends at the environment agency.

In an ideal world, you wouldn't need to create specific skills like this. Nor would you have to create a specially formatted file for every voice assistant out there. Websites would be full of semantic markup and intelligent assistants would be easily able to work out when the council offices were open, which bus routes were delayed, and what the flood risk was.

We're not there yet.

There is a huge industry push towards voice interfaces. Is it incumbent on Government to create services like this, or is it enough to create APIs and let others build things?

There's a long-standing moratorium on the UK Government creating mobile apps - we can't waste money developing for a dozen different platforms, keeping them updated, reacting to changing requirements from store owners.

Is the same true for voice interfaces? Siri, Cortana, Alexa, whatever Google's one is called, Bixby, and a dozen more round the corner. Each needing a different configuration to get them to vocalise your news. Each needing you to sign up to yet-another developer agreement.

My job is looking at open standards. We're too early in this industry to converge on one-standard-to-rule-them all. But if Voice is The Future™, we all need to think about how we make it as easy to deploy interactive speech services as possible.

Voice feels like it could be something - but only if it is open to all.

You can install the UK's flood forecast on Amazon's Alexa.

I flatter myself as a reasonably competent techie and programmer, but fuck me AWS Lambdas and Alexa skills are a right pile of shite! Sorry if that sounds a bit harsh, but they're a pain in the arse to get anything done.

I wanted something simple. When I say "Solar Panels", call this API, then say this phrase. That's the kind of thing which should take 5 minutes in something like IFTTT. Instead, it took around two hours of following out-of-date official tutorials, and whinging on Twitter, before I got my basic service up and running.

A quick bit of preparatory searching on Alta Vista 2.0 and I'd got incredibly frustrated.

Terence Eden is on Mastodon

@edent

Does anyone have an EASY tutorial for Alexa skills?
As in, not 27 steps where step 5 just says "install node" & 26 says "see other tutorial"

---

I ended up following this "easy" 30 step guide to develop a basic skill.

It's not so bad, but it does reveal Amazon's contempt for developers. Several of the steps contained errors, it involves multiple logins, random clicks, and a bunch of copy & pasting. Dull and complex.

Terence Eden is on Mastodon

@edent

Replying to @edentBrilliant! AWS picked up the wrong address from my Amazon UK account. How?
It also wouldn't copy across my credit card details! Why?

---

Terence Eden is on Mastodon

@edent

Replying to @edentThe cloud is everywhere!
The cloud is nowhere!
The cloud loves you!
The cloud knows all!
The cloud only works in specific US locations! pic.x.com/15dibbda0v

---

A frustrating and ultimately unsatisfying experience. I ended up using StackOverflow to correct errors in my code because the documentation was so woefully lacking.

The Code

The Python is convoluted, but manageable. When it hears the trigger phrase it opens a JSON API, extracts a result, then speaks it. It's mostly scaffolding. This is based on the example code. I've removed the comments.

from __future__ import print_function
import json, requests

# --------------- Helpers that build all of the responses ----------------------
def build_speechlet_response(title, output, reprompt_text, should_end_session):
    return {
        'outputSpeech': {
            'type': 'PlainText',
            'text': output
        },
        'card': {
            'type': 'Simple',
            'title': "SessionSpeechlet - " + title,
            'content': "SessionSpeechlet - " + output
        },
        'reprompt': {
            'outputSpeech': {
                'type': 'PlainText',
                'text': reprompt_text
            }
        },
        'shouldEndSession': should_end_session
    }


def build_response(session_attributes, speechlet_response):
    return {
        'version': '1.0',
        'sessionAttributes': session_attributes,
        'response': speechlet_response
    }

# --------------- Functions that control the skill's behavior ------------------
def get_welcome_response():
    API_url = 'https://example.com/'
    response = requests.get(url=API_url)
    data = json.loads(response.text)
    watts = data['Body']['Data']['PAC']['Values']['Result']
    session_attributes = {}
    card_title = "Welcome"
    speech_output = "Your Solar Panels are generating " + str(watts) + " watts right now."
    reprompt_text = ""
    should_end_session = True
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))

def handle_session_end_request():
    card_title = "Session Ended"
    speech_output = "May your day be sunny and bright! "
    should_end_session = True
    return build_response({}, build_speechlet_response(
        card_title, speech_output, None, should_end_session))

# --------------- Events ------------------
def on_session_started(session_started_request, session):
    print("on_session_started requestId=" + session_started_request['requestId']
          + ", sessionId=" + session['sessionId'])

def on_launch(launch_request, session):
    print("on_launch requestId=" + launch_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    return get_welcome_response()

def on_intent(intent_request, session):
    print("on_intent requestId=" + intent_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    intent = intent_request['intent']
    intent_name = intent_request['intent']['name']

def on_session_ended(session_ended_request, session):
    print("on_session_ended requestId=" + session_ended_request['requestId'] +
          ", sessionId=" + session['sessionId'])

# --------------- Main handler ------------------
def lambda_handler(event, context):
    print("event.session.application.applicationId=" +
          event['session']['application']['applicationId'])
    if event['session']['new']:
        on_session_started({'requestId': event['request']['requestId']},
                           event['session'])
    if event['request']['type'] == "LaunchRequest":
        return on_launch(event['request'], event['session'])
    elif event['request']['type'] == "IntentRequest":
        return on_intent(event['request'], event['session'])
    elif event['request']['type'] == "SessionEndedRequest":
        return on_session_ended(event['request'], event['session'])

This is not AI

I kinda thought that Amazon would hear "solar panels" and work out the rest of the query using fancy neural network magic. Nothing could be further from the truth. The developer has to manually code every single possible permutation of the phrase that they expect to hear.

This isn't AI. Voice interfaces are the command line. But you don't get tab-to-complete.

Amazon allow you to test your code by typing rather than speaking. I spent a frustrating 10 minutes trying to work out why my example code didn't work. Want to know why? I was typing "favourite" rather than the American spelling. Big Data my shiny metal arse.

Why not IFTT?

So, there is an official If-This-Then-That channel for Alexa.

But like most IFTTT services, it isn't well supported by the company. It works with a few blessed services, but you can't bring in your own APIs, nor define your own responses.

It is barely tested and has all sorts of weird restrictions.

Here's a tip, gang, if your service can't cope with upper-case characters that means it isn't ready to release to the public.

The founder of SaySpring recommended their easy to use product:

Stacey Higginbotham

@gigastacey

This entire thread is killing me. Especially in light of Amazon's press about Alexa being easy as it's competitive differentiator. x.com/edent/status/8…

Terence Eden is on Mastodon

@edent

Does anyone have an EASY tutorial for Alexa skills?
As in, not 27 steps where step 5 just says "install node" & 26 says "see other tutorial"

---

---

Mark C. Webster

@markcwebster

Replying to @gigastacey@gigastacey @edent this is what we're working on at @sayspring. We're just getting started, but take a look.

---

Sadly, it's only available to US customers.

Driving me dotty

I reluctantly got a Dot because I thought it would be a nifty way to control my new Internet connected light switches.

Most Alexa skills require you to have the sort of lifestyle where you are regularly desperate to know what the weather is like at a specific airport. Or have a life which is intimately tied to the range of Amazon-only services.

Taking a look through what developers have released, it's an obvious conclusion that most developers have better things to do that spend time battling with Amazon's inadequate developer experience.

Oh, and there's the requisite "fart apps" and other high quality services;.

The future may be voice interfaces - but Amazon aren't leading the way there.

I'm going to walk you, step-by-step, through a scam that targetted me today. Along the way we'll see how to avoid falling prey to these monsters.

It starts with a text

I was sent this SMS from a number that I didn't recognise.

Let's count the mistakes!

1. In the UK, we place the currency symbol before the number.
2. British English spells "authorise" with an S rather than a Z.
3. Amazon don't send out links using the bitly shortening service.

These phishers are hoping that you're too stupid to notice the mistakes. They are also betting that you're too busy to think for a minute about whether Amazon are likely to contact you via SMS rather than email or app notification.

Let's say you're tired enough to click through, what do you find?

Stealing Beauty

At first glance, this looks identical to the regular Amazon mobile sign in page.

But! Look closely. There are three obvious mistakes.

1. No https - you should know by now that you never enter your password unless the website is protected with those magic letters.
2. The web address is not amazon.co.uk! It is some other site. Again, you should never enter a password unless you are sure that the site is legitimate.
3. Finally - what is that � character doing at the bottom of the site? It should be a © symbol. Far too sloppy for Amazon.

Surely you wouldn't fall for this? Sadly, not everyone in the world is as bright as you.

Tell me more, tell me more

So, you unwisely entered your password. The scammers are already riffling through your Amazon account with the stolen password. They're buying themselves toys using your credit card. But they ain't done with you yet!

They want everything! Once they have your home address and phone number, they can bombard you with scams. Scroll down, and it gets worse.

Your bank details! Your mother's maiden name! Are these details enough for them to fool your bank? With all the information you're giving them, they'll be able to take over your entire life!

Fool me once

Because the scammers use bitly, we can take a look at how many people are clicking through this link.

By the time Bitly had flagged the link as malicious, over 250 people had clicked on it.

Would you have fallen for it? These scammers are betting that you're not paying attention. That you'll blindly trust any link you see.

Phone companies are powerless to stop this sort of spam - SIM cards can be purchased anonymously, and computers can pump out thousands of texts before their suspicious behaviour is noticed.

Bitly knows it is a vector for phishing attacks, but can't check every link before it goes live.

The only solid defence is you! It is up to you to keep your wits up, to examine every message you receive, and every link you click. You have to stay safe all the time - the scammers only need to get lucky once.

If you want to harden your defences, you need to take action now.

1. Activate Two-Factor Authentication for Amazon - even if someone steals your password, they'll also need your phone in order to log in.
2. Use LastPass to generate strong and unique passwords for every site you use. If you use the same password on multiple sites - it is only a matter of time before one is lost, and then all are lost.

Stay safe out there!

The Process

There's no direct link to 2FA settings. So the process is slightly convoluted. Assuming you are signed in to your Amazon account, you need to

• Go to https://www.amazon.co.uk/your-account
• Click on "Login & Security Settings"
• Then "Advanced Security Settings"

You can now start to add 2FA to your account.

There are two ways you can get your 2FA code. The most secure way is by using an authenticator app like Authy or FreeOTP.

If you can't install apps - or just don't like them - you can get your code delivered to you via SMS.

Let's ignore the American number formatting (555!) - is an SMS code sensible?

• SMS works everywhere, even on the dumbest phone.
• No app needed.
• Swap your SIM to a new phone and have instant access.

That last one is the biggest weakness. It is terrifyingly easy for a scammer to ring up your phone company and get your number swapped to a new SIM. If a scammer wants the codes off your app they have to physically steal your phone and then unlock it (you do have a secure password, right?). With SMS, all they have to do is convince some hapless call centre worker that you need your number transferred.

There's also the little matter that SMS isn't encrypted - but if the security services desperately want access to your Amazon account, I'm sure they have their own means.

2FA Problems

Far from being a scrappy start-up, Amazon is now a maze of interconnected legacy systems. There are several ancient services with Amazon can't or won't update. This means they don't get 2FA support.

This is a problem which I recently encountered with PayPal. Old apps don't support new security - weakening the usefulness of security for everyone.

Of course, there's no mention of which apps don't support 2FA. Their proposed solution of sticking your 2FA code to the end of your password is... interesting. It implies that if the system doesn't recognise your password decrypted password, it will split it in two and try it again. I wonder if that leaves them open to subtle timing attacks, or any other issues?

The point of 2FA is that you use it everywhere - otherwise you're introducing a weak point in your security. Amazon will happily let you turn off 2FA on specific devices.

I can kinda see their reasoning. It is annoying to be forced into using the 2FA on your regular handset. But that's also the point. Making it slightly harder for us makes it extraordinarily hard for an attacker.

Despite these shortcomings, I urge you to switch on 2FA. Amazon holds a surprising amount of your personal data - and the consequences of your Amazon account being hacked can be dire.

There are hundreds of sites which support 2FA. You should make sure you use it wherever possible.

I've often joked that if my employers could pay me in Amazon vouchers, it would save me a lot of hassle. I'm one of those insufferable people who prefers a delivery driver dropping off a single light bulb rather than having to pop to the shops like some sort of savage.

But now the dream is over. Amazon have bundled their sub-standard video offering, a moribund music subscription, and an underwhelming photo backup service in with their excellent free delivery service. In doing so, they've jacked the price from fifty to eighty quid.

Last year I was able to convince them to give me Prime delivery at the old price. But this year? No dice.

And so now I get this passive-aggressive message whenever I visit Amazon.

Here's the thing - I don't listen to much music, I don't live in an area with same day delivery, I already have a photo backup service, and I don't use a Kindle for my reading. Even if I wanted to watch more TV (I don't) Amazon refuse to build an app for my fairly recent smart TV. The same TV I bought on Amazon...

This really shouldn't come as a surprise to me. Amazon works on a socialist model - as described by the CEO Jeff Bezos:

Levy: Speaking of pricing, I wanted to ask about your decision to include streaming video as part of Amazon Prime. Why not charge separately for that? It’s a completely different service, isn’t it?

Bezos: There are two ways to build a successful company. One is to work very, very hard to convince customers to pay high margins. The other is to work very, very hard to be able to afford to offer customers low margins. They both work. We’re firmly in the second camp. It’s difficult—you have to eliminate defects and be very efficient. But it’s also a point of view. We’d rather have a very large customer base and low margins than a smaller customer base and higher margins.
Jeff Bezos Owns the Web in More Ways Than You Think - Wired Magazine, 2011

The more people taking part, the cheaper the cost for everyone. As a political philosophy I'm all for it, but I'm not keen on it being foisted on me as part of a capitalist transaction.

I personally benefit from a well educated, healthy public even if I don't use the services myself - I'm not sure I'll see any benefit from people being able to watch re-runs of Downton Abbey!

Eighty quid for "free" delivery is just too much. I dare say I'll still shop there, but not with as much regularity.

Jeff, if you're reading, please just offer your delivery service as a separate subscription.

I know that Amazon won't mourn the loss of one customer. But I mourn the fact that the subscription economy is slowly killing the services I enjoy.

I spoke to customer services - here's the transcript (edited for clarity) :

To be clear, Amazon weren't forthcoming in whether this was available to everyone. I suggest you ask Amazon very nicely!

In my case, I haven't watched a single Amazon video. I don't even think I've logged into the service. Both Amazon and Panasonic refuse to support my TV. Even if they did, the list of shows available really isn't particularly to my tastes.

Ideally, I'd like to unbundle their useless photo service (I host my own) and their Kindle Lending Library (I have an actual Library Card which has more benefits).

Still, I can't grumble. £50 Prime is a pretty good deal. Isn't it?

Is Prime Worth It?

I loved Amazon Prime. The day we moved house, as we sat exhausted on the sofa surrounded by boxes, I pulled out my smartphone and went crazy. The next morning, even more boxes arrived. Everything from cleaning products to doormats. Over the next few months, that's just how it was. If I saw that we needed something, a couple of clicks on the mobile website and - hey presto - a box would appear the very next day.

I didn't bother to compare prices with other sites - just click and wait. Once (and this is truly tragic) I was feeling a bit peckish on a Saturday afternoon, come Sunday morning the Amazon man dropped off a kilogram bag of chilli puff-rice crackers. Mmmmmm chilli crackers.

For fifty quid a year, it was absolutely perfect. Next day delivery on just about everything - from a single lightbulb to a cross-trainer, from last minute birthday presents to cooking supplies. Click, wait, get.

Then they went and screwed it all up.

They put the price up to eighty-quid. Their rationale was they were now bundling their crappy video service with it. Fine, I guess, but not for me - as I described in detail earlier on my blog. They refuse to support any of the devices I own and, even if they did, there's nothing on there I find of value.

Last year I wangled an extension to my existing price of £49. This year I've done the same.

So, the conundrum. Is it worth paying £50 a year for "free" next day delivery?

The central lie of Amazon Prime is "One-Day Delivery at no extra cost for eligible items."

A lie? I think so. It makes it sound like Prime items aren't more expensive than their regular brethren.

That £4.99 product can be had for £4.25 - a 15% discount - although delivery isn't next day.

On a small pack of rawl plugs, you're paying £2 for Prime's next day delivery.

For more expensive items, like a PS4, there's generally not price difference. But it really does add up for smaller items. And that's kinda the joy of Prime - I can order a single rawlplug, if that's all I need, without having to faff around on the high street.

Given the extra cost of (some) Prime items - is it worth paying for the service?

It's time for some maths! YAY! MATHS! (NB, these numbers are very fuzzy.)

Let's start with the number of items I've purchased from Amazon in the last year. The US site allows users to export a spreadsheet of their orders. The UK site doesn't offer such a facility. Thanks Amazon for truly demonstrating your commitment to the World Wide Web!

According to my account, I bought around 80 items using Prime in 2014. Crikey. I may be an addict....

Anyway...

The extra cost of Prime for each item was ~50p - £2.50 - although not every item charged extra for being available on Prime.

The cost of next day delivery for non-Prime items is between £2 and £10 (!).

If you regularly pay for next day delivery, it's probably cheaper to get Prime.

If you don't need next day delivery...?

Hmmm....

Very very very roughly - Prime costs me £90. £50 for membership and £40 in extra charges for next day delivery. Over all the items I buy, it's probably a good deal... I'd pay more if I didn't have Prime...

Ultimately, what I'm paying for is knowing when delivery will take place. I work from my home in Oxford, an office in Slough, and an office in London. I don't like buying something, then waiting until I'm in the correct location to pick it up. I like placing an order on Saturday and having it arrive on Sunday. Saying that an order will take 1-3 days makes it hard to plan where I have to be in order to get my item.

I'm fundamentally lazy. And I wonder - is that worth the extra cost?

Dear Amazon

If you're reading this blog, here's my wishlist!

• Unbundle Prime delivery and video. Sure, offer them together at a discount price if you like - but good products don't need to be forced onto customers.
• Let customers export their data - in CSV or similar format. Take a look at midata.
• How about making Prime prices the same as regular items? That'd be nice.
• If an item isn't available on Prime - perhaps refund (some of?) the delivery charges.
• Stop making it hard for customers to decide whether they want to give your their custom. The couple of hours I've spent writing this blog post could have been spent browsing your website.

IMDb was one of the earliest success stories for the UK Internet industry when, in 1998, Amazon.com bought them.

Given its proud heritiage, why don't Amazon care about the British users of its site? If you visit the site on your phone, this is the advert you see.

Oh! Cool! The new Kindle tablet. Why, I just gotta get me one of those! Let's click the link...

Oh...

Yup, the advert is for the American tablet. The price is listed in US Dollars and, once I'm on the Amazon.com website, there's no obvious way to get to the .co.uk version.

This is madness! The IMDb is consistently ranked in the top 50 most visited sites worldwide. Over 60% of its visitors are from outside the USA. Why are Amazon ignoring all those customers?

Mobile advertising really isn't that hard. You find a way to match a user to a product. One of the simplest ways to do that is to determine the country the user is in. You show Indian adverts to people living in India, and Japanese adverts to people living in Japan. That's not rocket science.

If, for some strange reason, you can only afford to design a single advert - your back end services should redirect the user to the website which is most appropriate. Most of the French people clicking on your link are going to want a site which is written in their language and priced in their currency.

It's unlike Amazon to leave money on the table like this. They have a vast international audience and yet they're simply ignoring their needs.

Why?

In it, I rant against service providers trying to lock their customers into a monoculture. Companies are always looking for the edge which will make them stand out - they think that restricting what their users can do is the answer. It is not.

Openness and network effects are the biggest drivers of usage - an MP3 bought from Amazon works on an iPod bought from Apple, and an MP3 bought from iTunes works perfectly on a Kindle Fire bought from Amazon. Customers and businesses both win when open standards are embraced and lock-in is eschewed.

I started my blog post by asking for something I thought to be quite reasonable, but some high-priests of Brand Synergy took to be treason:

I want to watch Nokia movies on my Samsung hardware running Google's Android, and then back them up to DropBox.

Today I've rubbed up against the annoying side of proprietary unreasonableness. Amazon Instant Video.

For reasons best know to themselves, the geniuses at Amazon have decided to put up the cost of Prime membership by 60%. As well as getting next day delivery on thousands of items, I can now also watch Amazon Instant Video - previously known as LoveFilm.

Deep joy.

This has gone down like a lead balloon among many erstwhile Prime customers. Not everyone likes watching TV, or even has the broadband to support this.

But, hey, I thought I'd give it a go before deciding if it wasn't for me.

Panasonic

I have a fairly new Panasonic TX-L37E5B (as sold on Amazon). Barely 18 months old. It's a "Smart" TV - with access to BBC iPlayer and a host of other services.

But not LoveFilm.

For over a year Panasonic has been refusing to update their app store to include Amazon's LoveFilm. Or 40D, ITV Player, Demand 5, etc. I wonder if they've done a deal with Netflix to keep competing services off their TVs? Or if Panasonic's SDK is so hideous that Amazon simply can't deliver a competent service using them?

This is what Panasonic's ecosystem looks like. How many names there do you recognise?

I repeatedly contacted Panasonic for a comment about this, but each time I received the same bland reply.

Panasonic Service UK

@PanaService_UK

Replying to @edent@edent We are looking into the possibility of adding services but cannot make any guarantee as to whether it will be possible at the moment

---

Panasonic, it seems, are quite happy to ignore customer demand. Once they have your money, they cease to care.

I spoke to Amazon about this and asked which TVs they support:

I have checked into this, and I can see that the eligible models for the televisions are:

LG Smart TV (2012 or newer)

Samsung Smart TV (2012 or newer)

and Sony Smart TV (2010 or newer)

Ah well, I thought, I can just use Panasonic's Web Browser to access LoveFilm, eh?

You need Silverlight software version 4.0 or higher to watch this title.

It looks like rival Netflix are abandoning Silverlight. Even Microsoft, the technology's creator, sees no future in the decrepit software - it's not much of a surprise to see that it's not available in TV browsers. But, perhaps, the browser will work with Flash sites?

The web browser does not support Adobe Flashplayer, and therefore, is unable to display catch-up services such as ITV player and 4oD. There are no current plans to upgrade the browser's compatibility.

Panasonic's FAQ

Now, in fairness, Panasonic just about supports some HTML5 video.

So it's theoretically possible that Amazon could decide to use open standards for its new video service, and Panasonic TV owners would be able to watch via their browsers rather than having a dedicated app.

Android

There's no Android App for Amazon Instant Video either. Just let that sink in for a moment. Amazon produce the Android powered Kindle Fire - and yet can't be bothered to produce a video streaming app for the millions of Android devices out there.

Madness! Or, a pathetic attempt to get people to buy Amazon's Kindle Fire tablets - seemingly the only Android devices allowed to access their streaming service.

"Ecosystems" Can Go To Hell

Amazon - for reasons best known to them - have decided to bundle a physical-goods delivery system and a video-streaming platform. I'm sure an MBA somewhere can explain why this multiplay service is destined to win hearts and minds.

Because Amazon have chosen to go down a locked and proprietary route, they've missed out on the interoperability which makes the world wonderful.

I'm not a Luddite. I have a modern TV, smartphones, tablets, and games consoles - none of which Amazon have chosen to support.

Actually, my Xbox 360 will support it - after I pay £40 per year for Xbox Live Gold Membership.

Here's an idea... Amazon could use open standards, develop apps which work on the majority of available platforms, and gather millions of customers who actually want their service.

Instead, they've gone with the rent-seeking approach of strong-arming their customers into paying more for a service they cannot use and do not want.

I've no doubt some folk will cancel their Prime membership - but enough will put up with the price rise to make this quarter's numbers look just swell!

Customers get screwed and Amazon gets to artificially inflate the numbers on its moribund streaming service.

This will not do.

I do not want services which only run on specially blessed devices. No one wants that! Companies which artificially restrict access in order to prop up their business models don't deserve to survive in the modern, interconnected world.

The true meaning of "ecosystem" is a bunch of independent organisms competing and co-operating. Symbiosis - not synergy.