In their post "How I Reversed Amazon's Kindle Web Obfuscation Because Their App Sucked" they describe the process of spoofing a web browser, downloading a bunch of JSON files, reconstructing the obfuscated SVGs used to draw individual letters, and running OCR on them to extract text.

There were a few problems with this approach.

Firstly, the downloader was hard-coded to only work with the .com site. That fix was simple - do a search and replace on amazon.com with amazon.co.uk. Easy!

But the harder problem was with the OCR. The code was designed to visually centre each extracted glyph. That gives a nice amount of whitespace around the character which makes it easier for OCR to run. The only problem is that some characters are ambiguous when centred:

When I ran the code, lots of full-stops became midpoints, commas became apostrophes, and various other characters went a bit wonky.

That made the output rather hard to read. This was compounded by the way line-breaks were treated. Modern eBooks are designed to be reflowable - no matter the size of your screen, lines should only break on a new paragraph. This had forced linebreaks at the end of every displayed line - rather than at the end of a paragraph.

So I decided to fix it.

A New Approach

I decided that OCRing an entire page would yield better results than single characters. I was (mostly) right. Here's what a typical page looks like after de-obfuscation and reconstruction:

As you can see - the typesetting is good for the body text, but skew-whiff for the title. Bold and italics are preserved. There are no links or images.

Here's how I did it.

Extract the characters

As in the original code, I took the SVG path of the character and rendered it as a monochrome PNG. Rather than centring the glyph, I used the height and width provided in the glyphs.json file. That gave me a directory full of individual letters, numbers, punctuation marks, and ligatures. These were named by fontKey (bold, italic, normal, etc).

Create a blank page

The page_data_0_4.json has a width and height of the page. I created a white PNG with the same dimensions. The individual characters could then be placed on that.

Resize the characters

In the page_data_0_4.json each run of text has a fontKey - which allows the correct glyph to be selected. There's also a fontSize parameter. Most text seems to be (the ludicrously precise) 19.800001. If a font had a different size, I temporarily scaled the glyph in proportion to 19.8.

Each glyph has an associated xPosition, along with a transform which gives X and Y offsets. That allows for indenting and other text layouts.

The characters were then pasted on to the blank page.

Once every character from that page had been extracted, resized, and placed - the page was saved as a monochrome PNG.

OCR the page

Tesseract 5 is a fast, modern, and reasonably accurate OCR engine for Linux.

Running tesseract page_0022.png output -l eng produced a .txt file with all the text extracted.

For a more useful HTML style layout, the hOCR output can be used: tesseract page_0022.png output -l eng hocr

Or, a PDF with embedded text: tesseract page_0022.png output -l eng pdf

Mistakes

OCR isn't infallible. Even with a high resolution image and a clear font, there were some errors.

• Superscript numerals for footnotes were often missing from the OCR.
• Words can run together even if they are well spaced.
• Tesseract can recognise bold and italic characters - but it outputs everything as plain text.

What's missing?

Images aren't downloaded. I took a brief look and, while there are links to them in the metadata, they're downloaded as encrypted blobs. I'm not clever enough to do anything with them.

The OCR can't pick out semantic meaning. Chapter headings and footnotes are rendered the same way as text.

Layout is flat. The image of the page might have an indent, but the outputted text won't.

What's next?

This is very far from perfect. It can give you a visually similar layout to a book you have purchased from Amazon. But it won't be reflowable.

The text will be reasonably accurate. But there will be plenty of mistakes.

You can get an HTML layout with hOCR. But it will be missing formatting and links.

Processing all the JSON files and OCRing all the images is relatively quick. But tweaking and assembling is still fairly manual.

There's nothing particularly clever about what I've done. The original code didn't come with an open source software licence, so I am unable to share my changes - but any moderately competent programmer could recreate this.

Personally, I've just stopped buying books from Amazon. I find that Kobo is often cheaper and their DRM is easy to bypass. But if you have many books trapped in Amazon - or a book is only published there - this is a barely adequate way to liberate it for your personal use.

I'd purchased an eBook and was appalled by how crappy the accessibility was. If you don't know, modern ePub books are just HTML wrapped in a zip file. They have all of the accessibility advantages of the web and should be easy to read no matter if you're sighted or not.

But not this eBook! Part of the story concerned WhatsApp messages sent by the protagonist. Here's one of them:

See that smudge in the middle? The one smaller than a full stop? That's meant to be an emoji. Rather than use an actual emoji, they've stuck in a tiny pixel image which won't scale with text size.

Here is is:

Can't see it? Let me blow it up to a more readable size.

OK, that's annoying for sighted readers, but just about understandable. What about people who are using a screen reader? Luckily, ePubs can use HTML's "alt text" feature which will describe an image which can't be seen.

Curses! That's, perhaps, not too annoying for a decorative image - but later in the book there are pictures of a ransom note. Despite the plot-twisting text in the illustration, the alt text just says "image".

I found the publisher's website and scoffed at their proclaimed accessibility statement. I sent them an email which basically consisted of the above. I realised it was probably futile, but I've got to spend my privilege points somehow.

The next day, they wrote back!

I wanted to reply to let you know we are taking our commitments to accessibility in our ebooks seriously […] I will get our ebook team to check this as a matter of urgency and get back to you with an update.

Fair play. But talk is cheap, would they actually take action? A few days later, they sent a follow-up:

We have checked the file for this title again and found issues with it. We have sent this back to the conversion house to have the file corrected and expect this to be delivered by the end of the week. We would then expect this corrected version to be with Kobo by the end of the following week, i.e. 25th July.

We have also been spot-checking other files to see if there is a wider issue and where necessary will follow the same workflow to ensure fully accessible versions are available as widely as possible.

And, you know what, I think that's totally reasonable. Yeah, they should have caught it before publication - but it is a complex book and they're a small publisher. They took my complaint seriously and actually did something about it.

A week or so rolled by and they sent me this:

Just to update you that we have been back and forth with the conversion house getting this title up to scratch. There were various complicating factors which should now be resolved and an updated file has now been distributed and should be available through your chosen retailer presently. If you have any other queries please do let us know.

I logged on to my eBook provider, clicked "read" and…

It was fixed! All the images had decent alt text as well.

Rather annoyingly, the retailer didn't notify me that there was an update available. I can't blame the publisher for that though.

Still, 3 weeks from report to fix is pretty good I reckon.

When I last contacted a publisher about a mistake in their ebook, it took over 3 months to fix it. Perhaps things are slowly getting better?

Anyway, please complain about poor accessibility. Don't shout into the void of social media - write a polite but insistent email telling (not asking) people to fix their shit. Sometimes, just sometimes, it does work.

Anyway you should read The Mysterious Case of the Alperton Angels by Janice Hallett - it is brilliant and accessible.

The txtr beagle was designed to be a stripped-down and simplified eReader⁰.

As far as I can tell, it never shipped. There were a few review units sent out but I can't find any evidence of consumers getting their hands on one. Also, that £8 price was the subsidised price when purchased with a mobile contract. Their website ceased working long ago.

But it got me intrigued. Moore's law is supposed to drive down the cost of electronics. So where are all the dirt-cheap eReaders?

The cheapest Kindle for sale on Amazon UK right now is about £100. Back in 2012, it was about £70. Taking inflation into account, that price has stayed static. Brands like Kobo are also in the £100 to £150 range.

About the cheapest retail eReader is the PocketBook Lux 4 for £85 or the (terribly reviewed) Woxter Scriba for £70.

AliExpress has loads of second-hand and obsolete models at cheap-ish prices. But a surprising dearth of new eReaders.

Going wholesale, Alibaba has a range of models, some of which clock in at around £30.

But, of course, that's before shipping and tax. They won't come with any manufacturer's warranty and don't expect any software updates. Also, good luck getting accessories!

So what's stopping new eReaders being released at a cheap(er) price? I think it comes down to four main things.

Reading is a niche hobby

Around 40% of UK adults didn't read a single book last year. That survey combines reading books and listening to audiobooks. Of the 60% who do read/listen, about 14% primarily listen. Of those that read, around 60% do so on paper books.

If reading is niche, reading electronically is a tiny niche! This is somewhat of a chicken-and-egg argument. If an eReader were the same cost as a mass-market paperback, I'm sure many more paper-book readers would become converts.

The whole point of an eInk reader is that it is a distraction-free environment. Yeah, you could scroll TikTok on one, but it isn't a pleasant experience. An eReader is designed for one thing only, unlike a phone or tablet. Do enough people want to carry yet-another-bloody-device just for reading?

eInk is expensive

The company which makes eInk hold several patents on the process. They're not a patent troll; they're building a business and selling mega-hectares of the stuff. Understandably, they have an interest in keeping prices high. They don't want to cannibalise their own market.

A basic 6 inch screen with wiring costs around £20 wholesale - that's from Alibaba, so doesn't include tax and shipping. That's before you've added any electronics or a operating system.

Speaking of which…

Android is a bottleneck

The promise of the Android Open Source Project was a free Operating System for anyone to use. The reality has been a little different. Most people want to be able to use basic Android functionality - like download operating system updates or reading apps. But Google doesn't allow that for eInk devices.

As I understand it, Google requires Android devices to have colour screens and, so I've read, won't certify eInk eReaders for newer versions of Android.

So manufacturers have to source parts which have drivers for older versions of Android. Or they have to develop their own OSes.

Books are fungible

Back when Apple sold iPods, they knew that the majority of purchasers would buy MP3s direct from Apple. The perfect symbiotic relationship! But the walled-gardens cracked and now people can buy their music from anywhere.

Amazon keeps this model for its eBooks. Unless you're prepared to get technical, you can only read Amazon books on your Amazon Kindle paid for with your Amazon wallet.

Games consoles are often sold at a loss because the manufacturer knows they'll make it up in game sales and subscriptions.

A low-price manufacturer is unlikely to also run a book store and wouldn't be able to cross-subsidise their hardware with content sales.

Alternatives

Some people have tried building open source eReaders but they're either abandoned, not suitable for production, or ridiculously expensive.

Buying second hand is relatively cheap - often under £50. But eInk screens can be brittle, and older ones may have scratches or cracks which are effectively unrepairable.

How cheap is cheap?

I'd love a £8 eReader. Something I could throw in a pocket and not worry about damaging. An eReader which was the same price as a hardback book - around £20 - would be amazing.

But I don't think we'll get there soon. The monopoly on screen technologies sets a retail floor of around £30, before the rest of the hardware is taken into account. Niche hardware is viable - but only with decent OS support. Other than Kobo and Amazon, no book retailer wants to stray outside their core competency to develop and subsidise hardware.

So I guess it's buy second-hand, or wait for the patents to expire.

Here's a rough and ready Python script which will tell you when any the books on your wishlist have dropped below a certain amount.

Prerequisites

1. Install kobodl following their guide.
2. Log in with your account by running kobodl user add
3. Check that the configuration file is saved in the default location /home/YOURUSERNAME/.config/kobodl.json

Get your wishlist

The kobodl function GetWishList() takes a list of users and returns a generator. The generator contains the book's name and author. The price is a string (for example 5.99 GBP) so needs to be split at the space.

Here's a quick proof of concept:

import kobodl
wishlist = kobodl.book.actions.GetWishList( kobodl.globals.Settings().UserList.users )
for book in wishlist:
    print( book.Title + " - "  + book.Author + " " + book.Price.split()[0] )

Sort the wishlist

Using Pandas, the data can be added to a dataframe and then sorted by price:

import kobodl
import pandas as pd

#   Set up the lists
items  = []
prices = []
ids    = []

wishlist = kobodl.book.actions.GetWishList( kobodl.globals.Settings().UserList.users )

for book in wishlist:
    items.append( book.Title + " - "  + book.Author )
    prices.append( float( book.Price.split()[0] ) )
    ids.append( book.RevisionId )

#   Place into a DataFrame
all_items = zip( ids, items, prices )
book_prices = pd.DataFrame( list(all_items), columns = ["ID", "Name", "Price"])
book_prices = book_prices.reset_index()  

#   Get books cheaper than three quid
cheap_df = book_prices[ book_prices["Price"] < 3 ]

Create the Message

This will write the body text of the email. It gives you the price, book details, and a search link to buy the book.

from urllib.parse import quote_plus

#   Search Prefix
website = "https://www.kobo.com/gb/en/search?query="

#   Email Body
message = ""

for index, row in cheap_df.sort_values("Price").iterrows():
    name  = row["Name"]
    price = str(row["Price"])
    link = website + quote_plus( name )
    message += "£" + price + " - " + name + "\n" + link + "\n\n"

Send an Email

Python makes it fairly easy to send an email - assuming you have a co-operative mailhost.

import smtplib
from email.message import EmailMessage

#   Send Email
def send_email(message):
    email_user = 'you@example.com'
    email_password = 'P@55w0rd'
    to = 'destination@example.com'
    msg = EmailMessage()
    msg.set_content(message)
    msg['Subject'] = "Kobo price drops"
    msg['From'] = email_user
    msg['To'] = to
    server = smtplib.SMTP_SSL('example.com', 465)
    server.ehlo()
    server.login(email_user, email_password)
    server.send_message(msg)
    server.quit()

send_email( message )

Setting the settings

When running as a script, it is necessary to ensure the settings are correctly initialised.

from kobodl.settings import Settings

my_settings = Settings()
kobodl.Globals.Settings = my_settings

The End Result

I have a cron job which runs this every morning. It sends an email like this:

Next Steps

Some possible ideas. If you can code these, let me know!

• Save the prices so it sees if there's been a drop since yesterday.
• Compare prices to Amazon for eBook Arbitrage.
• Automatically buy any book that hits 99p.

Happy reading!

But here's the thing with the LCP DRM scheme; they do give you the key! As I've written about previously, LCP mostly relies on the user entering their password (the key) when they want to read the book. Oh, there's some deep cryptographic magic in the background but, ultimately, the key is sat on your computer waiting to be found. Of course, cryptography is Very Hard™ which make retrieving the key almost impossible - so perhaps we can use a different technique to extract the unencrypted content?

One popular LCP app is Thorium. It is an Electron Web App. That means it is a bundled browser running JavaScript. That also means it can trivially be debugged. The code is running on your own computer, it doesn't touch anyone else's machine. There's no reverse engineering. No cracking of cryptographic secrets. No circumvention of any technical control. It doesn't reveal any illegal numbers. It doesn't jailbreak anything. We simply ask the reader to give us the content we've paid for - and it agrees.

Here Be Dragons

This is a manual, error-prone, and tiresome process. This cannot be used to automatically remove DRM. I've only tested this on Linux. It must only be used on books that you have legally acquired. I am using it for research and private study.

This uses Thorium 3.1.0 AppImage.

First, extract the application:

./Thorium-3.1.0.AppImage --appimage-extract

That creates a directory called squashfs-root which contains all the app's code.

The Thorium app can be run with remote debugging enabled by using:

./squashfs-root/thorium --remote-debugging-port=9223 --remote-allow-origins=*

Within the Thorium app, open up the book you want to read.

Open up Chrome and go to http://localhost:9223/ - you will see a list of Thorium windows. Click on the link which relates to your book.

In the Thorium book window, navigate through your book. In the debug window, you should see the text and images pop up.

In the debug window's "Content" tab, you'll be able to see the images and HTML that the eBook contains.

Images

The images are the full resolution files decrypted from your ePub. They can be right-clicked and saved from the developer tools.

Files

An ePub file is just a zipped collection of files. Get a copy of your ePub and rename it to whatever.zip then extract it. You will now be able to see the names of all the files - images, css, fonts, text, etc - but their contents will be encrypted, so you can't open them.

You can, however, give their filenames to the Electron app and it will read them for you.

Images

To get a Base64 encoded version of an image, run this command in the debug console:

fetch("httpsr2://...--/xthoriumhttps/ip0.0.0.0/p/OEBPS/image/whatever.jpg") .then(response => response.arrayBuffer())
  .then(buffer => {
    let base64 = btoa(
      new Uint8Array(buffer).reduce((data, byte) => data + String.fromCharCode(byte), '')
    );
    console.log(`data:image/jpeg;base64,${base64}`);
  });

Thorium uses the httpsr2 URl scheme - you can find the exact URl by looking at the content tab.

CSS

The CSS can be read directly and printed to the console:

fetch("httpsr2://....--/xthoriumhttps/ip0.0.0.0/p/OEBPS/css/styles.css").then(response => response.text())
  .then(cssText => console.log(cssText));

However, it is much larger than the original CSS - presumably because Thorium has injected its own directives in there.

Metadata

Metadata like the NCX and the OPF can also be decrypted without problem:

fetch("httpsr2://....--/xthoriumhttps/ip0.0.0.0/p/OEBPS/content.opf").then(response => response.text())
  .then(metadata => console.log(metadata));

They have roughly the same filesize as their encrypted counterparts - so I don't think anything is missing from them.

Fonts

If a font has been used in the document, it should be available. It can be grabbed as Base64 encoded text to the console using:

fetch("httpsr2://....--/xthoriumhttps/ip0.0.0.0/p/OEBPS/font/Whatever.ttf") .then(response => response.arrayBuffer())
  .then(buffer => {
    let base64 = btoa(
      new Uint8Array(buffer).reduce((data, byte) => data + String.fromCharCode(byte), '')
    );
    console.log(`${base64}`);
  });

From there it can be copied into a new file and then decoded.

Text

The HTML of the book is also visible on the Content tab. It is not the original content from the ePub. It has a bunch of CSS and JS added to it. But, once you get to the body, you'll see something like:

<body>
    <section epub:type="chapter" role="doc-chapter">
        <h2 id="_idParaDest-7" class="ct"><a id="_idTextAnchor007"></a><span id="page75" role="doc-pagebreak" aria-label="75" epub:type="pagebreak"></span>Book Title</h2>
        <div class="_idGenObjectLayout-1">
            <figure class="Full-Cover-White">
                <img class="_idGenObjectAttribute-1" src="image/cover.jpg" alt="" />
            </figure>
        </div>
        <div id="page76" role="doc-pagebreak" aria-label="76" epub:type="pagebreak" />
        <section class="summary"><h3 class="summary"><span class="border">SUMMARY</span></h3> 
        <p class="BT-Sans-left-align---p1">Lorem ipsum etc.</p>
    </section>

Which looks like plain old ePub to me. You can use the fetch command as above, but you'll still get the verbose version of the xHTML.

Putting it all together

If you've unzipped the original ePub, you'll see the internal directory structure. It should look something like this:

├── META-INF
│   └── container.xml
├── mimetype
└── OEBPS
    ├── content.opf
    ├── images
    │   ├── cover.jpg
    │   ├── image1.jpg
    │   └── image2.png
    ├── styles
    │   └── styles.css
    ├── content
    │   ├── 001-cover.xhtml
    │   ├── 002-about.xhtml
    │   ├── 003-title.xhtml
    │   ├── 004-chapter_01.xhtml
    │   ├── 005-chapter_02.xhtml
    │   └── 006-chapter_03.xhtml
    └── toc.ncx

Add the extracted files into that exact structure. Then zip them. Rename the .zip to .epub. That's it. You now have a DRM-free copy of the book that you purchased.

BONUS! PDF Extraction

LCP 2.0 PDFs are also extractable. Again, you'll need to open your purchased PDF in Thorium with debug mode active. In the debugger, you should be able to find the URl for the decrypted PDF.

It can be fetched with:

fetch("thoriumhttps://0.0.0.0/pub/..../publication.pdf") .then(response => response.arrayBuffer())
  .then(buffer => {
    let base64 = btoa(
      new Uint8Array(buffer).reduce((data, byte) => data + String.fromCharCode(byte), '')
    );
    const blob = new Blob([buffer], { type: "application/pdf" });
    const link = document.createElement("a");
    link.href = URL.createObjectURL(blob);
    link.download = "publication.pdf"; // filename for saving
    link.click();
    URL.revokeObjectURL(link.href);
  });

Copy the output and Base64 decode it. You'll have an unencumbered PDF.

Next Steps

That's probably about as far as I am competent to take this.

But, for now, a solution exists. If I ever buy an ePub with LCP Profile 2.0 encryption, I'll be able to manually extract what I need from it - without reverse engineering the encryption scheme.

Ethics

Before I published this blog post, I publicised my findings on Mastodon. Shortly afterwards, I received a LinkedIn message from someone senior in the Readium consortium - the body which has created the LCP DRM.

They said:

Hi Terence, You've found a way to hack LCP using Thorium. Bravo!

We certainly didn't sufficiently protect the system, we are already working on that.

From your Mastodon messages, you want to post your solution on your blog. This is what triggers my message.

From a manual solution, others will create a one-click solution. As you say, LCP is a "reasonably inoffensive" protection. We managed to convince publishers (even big US publishers) to adopt a solution that is flexible for readers and appreciated by public libraries and booksellers.

Our gains are re-injected in open-source software and open standards (work on EPUB and Web Publications).

If the DRM does not succeed, harder DRMs (for users) will be tested.

I let you think about that aspect

I did indeed think about that aspect. A day later I replied, saying:

Thank you for your message.

Because Readium doesn't freely licence its DRM, it has an adverse effect on me and other readers like me.

• My eReader hardware is out of support from the manufacturer - it will never receive an update for LCP support.
• My reading software (KOReader) have publicly stated that they cannot afford the fees you charge and will not be certified by you.
• Kobo hardware cannot read LCP protected books.
• There is no guarantee that LCP compatible software will be released for future platforms.

In short, I want to read my books on my choice of hardware and software; not yours.

I believe that everyone deserves the right to read on their platform of choice without having to seek permission from a 3rd party.

The technique I have discovered is basic. It is an unsophisticated use of your app's built-in debugging functionality. I have not reverse engineered your code, nor have I decrypted your secret keys. I will not be publishing any of your intellectual property.

In the spirit of openness, I intend to publish my research this week, alongside our correspondence.

Their reply, shortly before publication, contained what I consider to be a crude attempt at emotional manipulation.

Obviously, we are on different sides of the channel on the subject of DRMs.

I agree there should be many more LCP-compliant apps and devices; one hundred is insufficient. KOReader never contacted us: I don't think they know how low the certification fee would be (pricing is visible on the EDRLab website). FBReader, another open-source reading app, supports LCP on its downloadable version. Kobo support is coming. Also, too few people know that certification is free for specialised devices (e.g. braille and audio devices from Hims or Humanware).

We were planning to now focus on new accessibility features on our open-source Thorium Reader, better access to annotations for blind users and an advanced reading mode for dyslexic people. Too bad; disturbances around LCP will force us to focus on a new round of security measures, ensuring the technology stays useful for ebook lending (stop reading after some time) and as a protection against oversharing.

You can, for sure, publish information relative to your discoveries to the extent UK laws allow. After study, we'll do our best to make the technology more robust. If your discourse represents a circumvention of this technical protection measure, we'll command a take-down as a standard procedure.

A bit of a self-own to admit that they failed to properly prioritise accessibility!

Rather than rebut all their points, I decided to keep my reply succinct.

As you have raised the possibility of legal action, I think it is best that we terminate this conversation.

I sincerely believe that this post is a legitimate attempt to educate people about the deficiencies in Readium's DRM scheme. Both readers and publishers need to be aware that their Thorium app easily allows access to unprotected content.

I will, of course, publish any further correspondence related to this issue.

Background

Amazon have decided to stop letting customers download their purchased eBooks onto their computers. That means I can't strip the DRM and read on my non-Amazon eReader.

So I guess I'm not spending money with Amazon any more. I'm moving to Kobo for three main reasons:

1. They provide standard ePubs for download.
2. ePub DRM is trivial to remove.
3. Kobo will undercut Amazon's prices!

Here's the thing. I want to buy my eBooks. It is trivial to pirate almost any modern book. But, call me crazy, I like rewarding writers with a few pennies. That said, I'm not made of money, so I want to get the best (legal) deal possible.

Kobo do a price-match with other eBook retailers. It says:

We'll award a credit to your Kobo account equal to the price difference, plus 10% of the competitor’s price.

I found a book I wanted which was £4.99 on Kobo. The Amazon Kindle price was £4.31.

4.99 - ( (4.99 - 4.31) + (4.31 * 0.1) ) = 3.88

I purchased the book, sent a request for a price match, and got this email a few hours later:

OK! So what steps can we automate, and which will have to remain manual?

Amazon Pricing API

Amazon have a Product Advertising API. You will need to register for the Amazon Affiliate Program and make some qualifying sales before you get API access.

In order to search for an ISBN and get the price back, you need to send:

{
 "Keywords": "isbn:9781473613546",
 "Resources": ["Offers.Listings.Price"],
}

Using the updated Python API for PAAPI:

from paapi5_python_sdk import DefaultApi, SearchItemsRequest, SearchItemsResource, PartnerType

def search_items():
    access_key = "ABC"
    secret_key = "123"
    partner_tag = "shkspr-21"
    host = "webservices.amazon.co.uk"
    region = "eu-west-1"

    api = DefaultApi(access_key=access_key, secret_key=secret_key, host=host, region=region)

    request = SearchItemsRequest(
        partner_tag=partner_tag,
        partner_type=PartnerType.ASSOCIATES,
        keywords="isbn:9781473613546",
        search_index="All",
        item_count=1,
        resources=["Offers.Listings.Price"]
    )

    response = api.search_items(request)

    print(response)

search_items()

(Add your own access key, secret key, and tag. You may need to change the host and region depending on where you are in the world.)

That returns something like:

{
    "search_result": {
        "items": [
            {
                "asin": "B09JLQHHXN",
                "detail_page_url": "https://www.amazon.co.uk/dp/B09JLQHHXN?tag=shkspr-21&linkCode=osi&th=1&psc=1",
                "offers": {
                    "listings": [
                        {
                            "price": {
                                "amount": 2.99,
                                "currency": "GBP",
                                "display_amount": "£2.99"
                            }
                        }
                    ]
                }
            }
        ]
    }
}

(I've truncated the above so it only shows the relevant information.)

Kobo ISBN & Price

Let's get the ISBN and Price of a book on Kobo. There's no easy API to do this. But, thankfully, Kobo embeds some Schema.org metadata.

Look at the source code for https://www.kobo.com/gb/en/ebook/venomous-lumpsucker-1

<div id="ratings-widget-details-wrapper" class="kobo-gizmo"
     data-kobo-gizmo="RatingAndReviewWidget"
     data-kobo-gizmo-config ='{&quot;googleBook&quot;:&quot;{\r\n  \&quot;@context\&quot;: \&quot;http://schema.org\&quot;,\r\n  \&quot;@type\&quot;: \&quot;Book\&quot;,\r\n  \&quot;name\&quot;: \&quot;Venomous Lumpsucker\&quot;,\r\n  \&quot;genre\&quot;: [\r\n    \&quot;Fiction \\u0026 Literature\&quot;,\r\n    \&quot;Humorous\&quot;,\r\n    \&quot;Literary\&quot;\r\n  ],\r\n  \&quot;inLanguage\&quot;: \&quot;en\&quot;,\r\n  \&quot;author\&quot;: {\r\n    \&quot;@type\&quot;: \&quot;Person\&quot;,\r\n    \&quot;name\&quot;: \&quot;Ned Beauman\&quot;\r\n  },\r\n  \&quot;workExample\&quot;: {\r\n    \&quot;@type\&quot;: \&quot;Book\&quot;,\r\n    \&quot;author\&quot;: {\r\n      \&quot;@type\&quot;: \&quot;Person\&quot;,\r\n      \&quot;name\&quot;: \&quot;Ned Beauman\&quot;\r\n    },\r\n    \&quot;isbn\&quot;: \&quot;9781473613546\&quot; …'>
</div>

Getting the data from the data-kobo-gizmo-config is a little tricky.

• Using Python Requests won't work because Kobo seem to run a JS CAPTCHA to detect scraping.
• There is a Calibre-Web Kobo plugin but it requires you to have a physical Kobo eReader in order to get an API key.
• The Rakuten API is only for the Japanese store.

So we have to use the Selenium WebDriver to scrape the data:

from selenium import webdriver
from bs4 import BeautifulSoup
import json

#   Open the web page
browser = webdriver.Firefox()
browser.get("https://www.kobo.com/gb/en/ebook/venomous-lumpsucker-1")

#   Get the source
html_source = browser.page_source

#   Soupify
soup = BeautifulSoup(html_source, 'html.parser')

#   Get the encoded JSON Schema
schema = soup.find_all(id="ratings-widget-details-wrapper")[0].get("data-kobo-gizmo-config")

#   Convert to object from JSON
parsed_data = json.loads(schema)

#   Decode the nested JSON strings
parsed_data["googleBook"] = json.loads(parsed_data["googleBook"])

#    Get ISBN and Price
price = parsed_data["googleBook"]["workExample"]["potentialAction"]["expectsAcceptanceOf"]["price"]
isbn  = parsed_data["googleBook"]["workExample"]["isbn"]
print(isbn)
print(price)

Kobo Wishlist

OK, nearly there! Given a Kobo book URl we can get the price and ISBN, then use that ISBN to get the Kindle price. But how do we get the Kobo book URl in the first place?

I'm adding all the books I want to my Kobo Wishlist.

Inside the Wishlist is a scrap of JavaScript which contains this JSON:

{
    "value": {
        "Items": [
            {
                "Title": "Venomous Lumpsucker",
                "Price": "£2.99",
                "ProductUrl": "/gb/en/ebook/venomous-lumpsucker-1",
            }
        ],
        "TotalItemCount": 11,
        "ItemCountByProductType": {
            "book": 11
        },
        "PageIndex": 1,
        "TotalNumPages": 1,
       }
}

(Simplified to make it easier to understand.)

Although there's a price, there's no ISBN, So you'll need to use the "ProductUrl" to get the ISBN and Price as above.

Sadly, unlike Amazon, there's no way to publicly share a wishlist. Getting the JSON requires logging in, so it's back to Selenium again!

This should be enough:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import time

browser = webdriver.Firefox()
browser.get("https://www.kobo.com/gb/en/account/wishlist")

#       Log in
username_box = browser.find_element(By.NAME, "LogInModel.UserName")
username_box.clear()
username_box.send_keys('you@example.com')

password_box = browser.find_element(By.NAME, "LogInModel.Password")
password_box.clear()
password_box.send_keys('p455w0rd')

password_box.send_keys(Keys.RETURN)

time.sleep(5) # Wait for load and rendering

But the Kobo presents a CAPTCHA which prevents login.

There is an unofficial API which, sadly, doesn't seem to work at the moment.

Next Steps

For now, I'm saving specific Kobo book URls into a file and then running a scrape once per day. Hopefully, the unofficial Kobo API will be working again soon.

But the other day I had someone scoff at me and say "Good luck reading when civilisation collapses! I'll still be able to read by candle-light."

This is nonsense. In the case of a Zombie Attack, am I really going to lug a few hundred kilograms of books with me? Or am I going to slip a lightweight eReader into my pocket and have ten thousand books at my disposal?

Also, where is he getting all those candles from? My eBook has a built in light which is less likely to attract Zombies and isn't a massive fire hazard.

Perhaps he's concerned that an eReader battery won't last more than a couple of weeks and then I'll be stuffed. OK, sure. But I have a solar powered portable battery. As long as the dust clouds don't blot out Earth's yellow sun, I should be able to read indefinitely.

Now, it's true that my eReader might get a damaged screen, or dropped in a puddle, or suffer any of the same calamities which can also befall paper books.

Perhaps my interlocutor's glasses will break. If so, an eBook would let him boost the font size quite comfortably.

Frankly, the only thing paper books are good for in a post-apocalyptic wasteland is burning for fuel⁴.

But - and this is a big but - even if eReaders are vastly inferior in an emergency... so fucking what?

If civilisation doesn't collapse, I'd've wasted time, energy, money, and effort maintaining a physical library. I'd be limited to how many books I can take on a beach holiday. Every time I moved house I'd be straining my back carrying all of them. The fire-risk of hundreds of books would have been for naught.

It's the same as people who don't pay into a pension because they might be hit by a bus tomorrow. Sure. But what if you're unlucky enough to reach old age?

You can either live each day like its your last - and deal with the consequences of tomorrow when they come - or you can take a reasoned approach to the inevitability of the future.

So I'm going to enjoy all the comforts of an eReader now. Come the end-of-the-world, I will regret... well, probably nothing.

I don't want to live like a prepper and have a basement full of pickled vegetables slowly fermenting, or a library of paper slowly crumbling.

I want to take full advantage of the modern world while it still exists.

I have their previous model, the Likebook Ares. After less than a year of use, I noticed screen discolouration. The eBay reseller wasn't interested in helping me with a return. Boyue ignored my repeated complaints - and I was stuck with a defective unit. So I waited until Boyue released their new version and cheekily asked if I could review it. They happily sent me this in exchange for an honest review and YouTube video.

So here is my extremely honest review!

If you're a casual eBook user - you're almost certainly better off with a Kindle. Amazon have perfected an ecosystem where you can buy a book from your eReader and have it in seconds. They may be evil, predatory capitalists, but if your Kindle breaks or goes a bit wonky they're great at fixing it or sending you a fresh one. Amazon will provide constant software updates and work hard to make sure the Kindle is easy to use. Amazon have customer service agents to hold your hands if you don't understand how to use your Kindle, or if it is faulty.

The Likebook P78 is everything a Kindle is not. And that may be a good or bad thing depending on what you want to do with it.

You get no support - other than what the community can provide on Reddit or MobileRead Forums. Boyue will not respond to your emails or Tweets. You might get a firmware update - but it will be a painful, manual process to apply it.

P78's killer feature is that it is fairly open Android (V8.1) tablet. Want to install random apps on there? Go for it! Want to replace the homescreen app? Fine. Don't like the default reading experience? Install a different reader. Need some different fonts? Copy them over via USB or Bluetooth and you're good to go. Change the keyboard, music player, and every single aspect of the device until it's unique to you.

Want to SSH into it, or copy your library using NextCloud, or play Angry Birds - it's all possible. It's a reasonably hackable device - with none of the polish of its Amazon rival.

If you're happy with that (and I am!) read on...

Unboxing

Here's a long video showing how the unboxing and initial set-up goes:

As you can see, it isn't exactly slick. But once done, you get a well-specified tablet.

Tech Specs

USB-C! This is the main reason I got the previous model. One cable to charge my laptop, phone, headphones, and eReader. Nice! Not even Kindle can do USB-C yet.

In terms of size, it is identical to the previous Ares model. It even fits into the same case.

Unlike the Ares - this doesn't have a Wacom digitiser. So you can use the touchscreen with your fingers, but not a stylus.

It also gains a MicroSD slot. I don't really get why. Most eBooks are under 1MB. Even if you want to carry your entire comic book collection around, the ~20GB of usable space on the tablet should be sufficient. But, hey, shove a spare card in their and go nuts.

Speakers on the bottom a good enough for speech. You probably won't want to use them with music - but for an audio book, they're sufficient. It also has BlueTooth if you'd rather listen on headphones.

Text to speech is build in. The default Android experience is adequate.

Cool Android Features

Cast! Yes, the device supports casting - so you can share your books to any big screen that has a ChromeCast plugged in.

But... You're limited to 720p - so text looks like shit.

Terence Eden is on Mastodon

@edent

I can cast my eReader to my TV - but the resolution is rubbish.
Anyone know how to set Android to mirror the cast destination's resolution rather than the native one? pic.x.com/vrafyium33

---

Copy books over via USB, Bluetooth, or WiFi. You're not shackled to Amazon's ecosystem.

Sideload apps. Play OGG, MP3, and FLAC. It can do everything an Android tablet can do.

Screen and lighting

Like most modern eReaders, it has a set of LEDs round the edge of the screen. You can set the brightness and the colour temperature. If you prefer a gentle orange light for reading at night and a bright white light for the beach, that's only a tap away.

The screen is gorgeous. It's a single slab of glass, so there are no shadows from the bezel. The screen isn't too shiny - so it's great for reading even in strong direct sunlight.

The resolution is 1872x1404 - that's even better than the most recent Kindle! Text rendering is smooth and beautiful. It's even better quality than printed paper.

Default Reading Experience

And this is where it all goes wrong. The in-built z-Reader is garbage.

Perhaps that's a bit harsh. It has lots of customisable options to play with, and supports horizontal layout if that's what you like.

But the text layout engine is just unsuitable for English text. Letters frequently overlap, hyphenation is all over the shop, and the fonts aren't particularly nice. Although you can install your own.

The user interface is poorly translated - with lots of Chinese characters popping up. The default dictionary is Chinese - even if you've set the interface to English.

You can follow my guide to adding an English Dictionary to Boyue Likebook eReaders.

At best, I'd class it at "barely adequate".

You can install 3rd party eReaders, like Kindle and Google Books.

Install KOReader

I use KOReader. It's a brilliant and hackable eBook reader. Lots of lovely options to fiddle with and very customisable. Once installed, it can notify you of updates and install them for you.

Customisation

There are lots of options to go through. Sadly, the UI isn't very well done - with lots of instances of overlapping elements.

Localisation

Like most cheap Chinese electronics, there is only a cursory effort to localise the interface. You're probably not going to spend a lot of time in the settings screen, but it shows how much of an afterthought usability is.

Install apps

You can side-load whatever Android apps you want. There is an in-built store:

Again, poor localisation. And what's with have store categories with no apps in there?

(Note - it produces colour screenshots, but the screen itself is greyscale only.)

Verdict

If you want an eInk Android tablet - this is pretty good buy for under £200. If you're happy with lots of customisation and fiddling around, get it. If you want a seamless reading experience, grab a Kindle.

For those of us who like diving into every setting, changing every last feature of our devices, and can overlook a badly translated and messy UI - it's a good device.

The LikeBook range is mostly sold on eBay and AliExpress. Buy with caution and in the expectation that, if something goes wrong, you'll be left in the lurch.

I thoroughly recommend you read it. But there's no official eBook copy. Why?

If you trawl the Dark Web™ (2nd page of Google) you're sure to find hundreds of samizdat copies. Some laboriously typed up by hand, others scanned and OCR'd by machines. All of them filled with imperfections, and none of them making money for the original publishers or the author's estate.

The audiobook seems to be still available. But the dead-tree copy seems relegated to the second-hand shelves.

It looks like the book is being republished next year, but with no sign of an eBook. It is infuriating.

This grumpy post isn't really about Future Shock. There are so many books which are hidden from our knowledge. I know Google used to scan in books, but got clobbered for copyright infringement. But that's not good enough. It's error-prone, lacking semantics, and locked in to one provider.

In the book "The Long Tail" we were promised that the future was "selling less of more". That is, the marginal cost of digital reproduction meant that stores would have infinite supply of an infinite variety of stock. Sure, some things would only sell a copy or two per year. But who cares? The cost of storage, reproduction, and transmission, is negligible.

Sadly, that turns out not to be the case. I'm sure there is a digital version of the book at the publisher - but the format used to create the printed book probably isn't suitable for selling to the public. An ePub copy requires someone to typeset it, validate it, check it on a variety of devices, clear the rights, design a cover, generate an ISBN, load into bookshops - and probably a thousand other things that I can't even imagine.

Non-digital-native files are expensive to convert to digital files. And with uncertain returns, it's no wonder so many critical texts are unlocked only by piracy. This is not a long-term strategy, and is of dubious legality and morality.

We need a better way to free up human knowledge.

I only had 7 days, and did quite a lot of sight-seeing, so there's only 6 entries here. Will try for a more relaxing holiday next year!

First up, "Station Eleven" by Emily St. John Mandel.

A cracking "end of the world" novel, neatly tied together via Shakespeare's King Lear. As we were staying in a villa in the middle of nowhere, the haunting descriptions of isolation really resonated with me.

It isn't markedly different from most other apocalyptic fiction novels - and the "twist" is pretty obvious - but it doesn't suffer unduly for it. It cracks along at a fair pace and is generous with its humour, despair, and flashbacks.

The only real criticism is that electricity is seen as an almost mythical element. In reality, small solar chargers and hand cranks are readily available and quite sufficient to power, say, walkie-talkies, torches, music players etc.

Still, a great yarn.

On to "The Sparrow" by Mary Doria Russell.

I've decidedly mixed feelings on this one. Without a doubt it is an epic and thoroughly disturbing tale of "First Contact". In terms of "hard" sci-fi it pretty much has it all - and the science is impeccable.

And yet, the theology… This is a tale of "Jesuits in Spaaaaaaace" so it is understandable that their morally dubious superstitions would be at the forefront of the plot. But it is laid on with such a heavy trowel that I honestly thought that it might be a Scientology-esque tale designed to convert and conscript the unwary.

Because all the characters are motivated by religious fervour, I found it rather hard to relate to them. By the end, however, I found myself wondering if this book - almost revelling in its torture-porn - was a warning against organised religion.

Sublime sci-fi, but not for the faint-of-heart.

Terence Eden is on Mastodon

@edent

Have a book reading holiday coming up.
What are your favourite female-authored Sci-fi novels?

Go!

---

Stefan Czerniawski

@pubstrat

Replying to @edent@edent The Just City, Radiant, The First Fifteen Lives of Harry August, Infidel

---

After the bleakness of "The Sparrow" it was a blessed relief to read "The First Fifteen Lives of Harry August" by Claire North. The plot is simple - Groundhog Day but for your entire life. When you die, you're immediately reborn - with memories intact - as yourself.

I didn't want the book to end! It was joyful in its mastery of time-travel story telling. Although dealing with some weighty topics, it had a delightful lightness of touch that made the whole book skip along. At times, it felt like I was reading the extended adaptation of a BBC mini-series, so clearly could I see all the characters.

It's rare to find a book seemingly without flaws - but this simply sparkles with delight and ingenuity. It is spy-thriller, cold-war noir, boys-own adventure, and desperately sad love story rolled into one.

Chris Mcr

@chrismcr101

@edent Margaret Atwood - Oryx and Crake, The Handmaid's Tale, Lauren Beukes - The Shining Girls

---

The only real dud of the recommendations was "The Shining Girls" by Lauren Beukes.

This was a real disappointment. It's sort-of sci-fi - in that the murder can travel through time - but there's no explanation of why the murders are being committed. No reason given as to why the girls are "shining". No insight into the mind of the killer nor the powers which are driving him.

Perhaps it is that I don't read many thrillers but I found the whole book unsatisfying. The protagonists will never be able to solve the mystery because no-one suspects time-travel as an MO - so it feels like the whole story is a waste of time. Ah well.

Finally, I hit a gold mine!

Mark O'Neill

@marxculture

Replying to @edent@edent Ancillary Justice, The Dispossessed, Brightness Falls From the Air

---

James Tiptree Jr is the pen-name of Alice Sheldon - an Air Force Major, CIA operative, and Doctor of Experimental Psychology. Although she wrote many short stories, James only wrote two novels - now gathered in one eBook collection.

Up the Walls of the World is, frankly, a belter! Telepaths, flying aliens, military experiments, doomsday devices! What more could you want from a sci-fi novel? The way she writes about the alien experience is perfect, never getting bogged down in exposition, never giving in to the temptation to make their world more relateable.

Bonus points for what must be a very early mention of the Internet - dialling up to connect to super-computers FTW!

The final book was "Brightness Falls From the Air." What starts off as a charming backwater sci-fi tale quickly descends (or ascends) into a Agatha Christie-style whodunnit. There's scheming, sex, drugs, intrigue, unrequited love, and murders.

Towards the end, it does rather begin to resemble a 1970s Doctor Who serial - a double-cross at every turn and a cliffhanger at the end of every chapter. A little over-wrought, perhaps, but wrapped in such a great story that it is easy to forgive.

…And that's a wrap! Now eagerly awaiting my next holiday - do let me know your recommendations.

I often wonder how much I read during the average day. A few thousand words of tweets, couple of hundred in Facebook posts, a dozen articles on blogs and news sites - and an unhealthy smattering of Reddit and other fora. All told, I am probably reading the equivalent of half a novel per day.

And yet... it doesn't feel like I'm doing proper reading. I don't get my teeth stuck into a single subject or story, it always feels like I'm flitting from one thing to the next without ever really examining it in detail. I don't know if this lack of focus has a detrimental effect on my ability to focus on a single task - but it seems unhealthy. Like mindlessly gobbling down a tube of Pringles, I often find myself reading my phone or laptop without any real comprehension of whether I'm enjoying the words I'm seeing.

So, for a week long holiday in the Bahamas, I went on a digital detox. No phone, no Internet, no news, blogs, tweets, or texts - just me and a Nook stuffed with books. Oh, I took my wife along too - well, I needed someone with whom I could discuss literature!

It was like slaking a thirst I never knew I had! I averaged around two books per day - I'd have got through more, but the early morning cocktails make it somewhat hard to concentrate on the written word. Herein you can find a brief synopsis of what I read, in the order I read them, and how enjoyable they were. I tend to alternate between fiction and non-fiction, with a bias to modern books and autobiographies.

You're Never Weird on the Internet (Almost) - Felicia Day

A gentle start - an almost perfect aeroplane read. That's not a criticism! Felicia Day's delightful clarity of style is particularly refreshing. She tells the somewhat unlikely tale of her rise to Internet Stardom - via the pain of growing up nerdy. For a celebrity autobiography, it's honest about the darker times in her life without every straying into the "Mommie Dearest" melodrama.

There are two, I felt, curious omissions. For someone who has worked with all sorts of interesting people - there are very few "behind the curtain" stories. I'm not talking about how Joss Whedon is an abusive megolomaniac, or how the actor playing Tinkerballa would only eat blue M&Ms - but just the sort of fun details about working on other hit shows. I'm aware that celeb biogs can often turn into "and then I worked with Larry, who is such a love!" - and I certainly don't begrudge her right to privacy - but it seemed like a curious omission. So too does the exclusion of how she met her partner. Again, this is her story and she gets to protect her personal life with as much vigour as she likes - but after several chapters of how she was a social outcast, it feels a little odd to just drop in her partner without even a good "and that's how we met" story.

Still, all that said, it's a cracking look at how web-scale products work, what it's like to be part of a runaway success, and the downside of meeting people AFK.

How to Be a Husband - Tim Downling

I really don't know what lead me to pick this one up - I've never read one of Tim's columns in The Guardian. Again, an excellent plane read. A light and witty look at (fairly) modern marriage is like.

I found the sequences talking about fatherhood a bit wearying, but I guess that's more of a reflection on my distaste of procreation rather than the author's talents.

It's slightly derivative of Mil Millington's Things My Girlfriend And I Have Argued About, but it was funny enough to away from the in-flight movies.

So You've Been Publicly Shamed - Jon Ronson

A fairly short work, but interesting nonetheless. It looks at the aftermath of Internet Hate Mobs - what happens when real people get caught in a scandal which ignites social media? It relies heavily on in-person interviews rather than rampant speculation. The cast of characters ranges from the sympathetic people caught up in a whirlwind, to the downright unrepentant who have heaped tragedy upon themselves.

It strikes me that there still is a need for mob justice. Our legal system cannot keep up with new social mores and often leaves people feeling inadequately served by the paltry justice it metes out. I'm not saying that it's right to launch a crusade against someone who thought they were making a private remark - but it is a natural consequence of our disenchantment with societal institutions.

One thing to note is that this book has been heavily trailed and syndicated in the media. I'd read several of the chapters before getting the book, which somewhat diminishes it.

News from the Clouds - Robert Llewellyn

The bloke who plays Kryten has written an epic sci-fi trilogy, all made possible through the Unbound Crowdfunding site for authors.

The series is designed to be a positive spin on the future. What if, instead of destroying the planet and filling it with killer robots, we make a better society? What starts off as a hippy-dippy look at the world slowly becomes darker and more engrossing.

The protagonist has a dispiriting tendency towards the obtuse, which helpfully allows the other charters a chance to engage in plot-hole-filling exposition - and to the self-critical, which makes him somewhat of an unsympathetic narrator. That said, it's a jolly good romp through the clouds, looking at the different directions in which society could develop. Well worth a read if you like your sci-fi meaningful and homespun.

Doing Good Better: Effective Altruism and a Radical New Way to Make a Difference - William MacAskill

This book asks one very important questions - how can you best make a difference to the world? It looks at the effectiveness of charitable giving and whether it is sensible to devote one's life to working for good causes.

Much like books by Malcom Gladwell, Atul Gawande, and Nassim Nicholas Taleb - this is a very short book padded out with annecdata.

The two core ideas are fantastic:

1. Check that the money you are donating is being used wisely.
2. It may be worth donating your career rather than your earnings.

That's it! The book spends a bit too much time promoting 80,000 Hours - the charity the author has set up and too little time talking about how one can accurately assess which charities are worthwhile. Despite being a UK charity, the case studies seem biased towards American projects.

This is a vital and important book - and it has made me reassess which charities I support - I just wish it had a bit more substance to it.

Armada - Ernest Cline

Oh! But this book is ridiculously campy fun! After the success of Ready Player One, it's a shorter and simpler tale - less impressive in its majesty, but just as exciting.

The plot is... well... it's hard to call it unoriginal because it relies heavily on the reader understanding the homages to great (and terrible) sci-fis of yesteryear.

I'll admit that the plot didn't move in the direction I expected which, given the paper thin characters, reflects rather poorly on me!

This is a boys-own-adventure (two whole women!) of pure escapism. Given the upcoming film adaptation of his first novel, this reads like it has half an eye on a TV-series. No, scratch that, it kinda reads like the tie-in novelization of a show that Fox cancelled prematurely!

Fast, silly, fun.

A Book for Her - Bridget Christie

What is this "feminism" nonsense all the girls are going on about, eh? Is it just bra-burning and shouting at people on Twitter?

This is by turns a hilarious and depressing book. Part comedienne's autobiography, part tale of consciousness raising, part history of modern feminism. It's a delicious ramble through the thickets of a lady-comic's travails - "ramble" being the operative word as she employs the same "anti-comedy" repetition made popular by TV's Richard Herring and Stewart Lee. It works slightly better on stage with the helpless chuckling of a crowd surrounding you, but she genuinely caused me to guffaw into my daiquiri.

If you haven't seen Bridgit Christie on stage, you should check your privilege and rectify the situation immediately!

The Shepherd's Crown (Discworld Novels) - Terry Pratchett

The book I was dreading. I saved it until halfway through the holiday. Once read, that's it; no more Discworld. Wailey! Wailey!

Really, all that can be said is that this is an exemplary Discworld novel. The complicated side-plots so reminiscent of his middle-era are kept to a minimum, the wit is just as brutal, and the world-building mythos is superbly handled. This is the most emotionally complex of his works, utterly devastating and hilarious.

It starts with an emotional sucker-punch and gradually absorbed me back into a world I've known since childhood. Spat out at the other end, I felt rather relieved that it lived up to expectations. It gladly joins the others on my shelf.

Ayoade on Ayoade - Richard Ayoade Malkovich malkovich malkovich malkovich, malkovich? Malkovich malkovich malkovich - malkovich-malkovich!

It's hard* to know how to assess** a book*** like this. It's far too clever for its own good - a frustrating and humorous look at... what? It's neither autobiography nor comic novel. An anti-novel**** which trudges the reader through a series of comic vignettes. It reminded me of some of Woody Allen's work. If you get the joke - you'll find it hilarious. Everyone else will just look on in bemusement.

Funny in places, bewildering throughout, and makes judicious use of footnotes with almost the same skill as Terry Pratchett*****.

*But not too hard

** If we can ever really assess anything

*** When all is said and done, what is a book? Merely a bagatelle of dreams in which we sometimes catch a reflection of our nightmares.

**** Or "antinovel" if you will

***** Indeed, Ayoade makes frequent reference to the Discworld in a pamphlet he published in 2004 entitled "Wither Soup"

The Martian - Andy Weir

I asked my friends for funny books to keep me entertained on holiday, Thayer recommended this one. Frankly, it all sounded a bit depressing - an astronaut is abandoned on Mars, left for dead he unexpectedly survives - but no one knows he is alive. Grim, right?

Not a bit of it! It is consistently funny, utterly tense, and a good smattering of real science and maths in there.

It also fails the Bechdel-Wallace Test in an interesting new way! While there are strong female characters all everyone talks about is this one guy :-)

I can see why it has been made into a movie - I stayed up all night, fighting jetlag, just to finish it.

Robin Ince's Bad Book Club: One man's quest to uncover the books that taste forgot

Comedian Robin Ince has a problem. Like many readers, he can't walk past a charity shop without poking his nose in for a quick rummage. Sadly, Robin has become addicted to "bad" books. Not just poorly written, but badly conceived, and of rare ineptitude.

This is a series of delightfully snarky reviews of some of the dreadful genres of books (Christian Gynaecologist Romance! Killer Crabs! Faded Celebrity Autobiographies!) It never ventures into being mean-spirited, in fact it positively revels in the joy of finding new depths of crimes against literature.

Not a long book, but good for a holiday read.

The Three-Body Problem - Cixin Liu

This won the Hugo Award and deservedly so. The translator adds several footnotes to help readers unfamiliar to Chinese history understand the context of the novel. Particularly interesting is this, from the Translator's Postscript:

The best translations into English do not, in fact, read as if they were originally written in English. The English words are arranged in such a way that the reader sees a glimpse of another culture’s patterns of thinking, hears an echo of another language’s rhythms and cadences, and feels a tremor of another people’s gestures and movements.

Ken Liu

That's one of the aspects which make this such a compelling novel. The sci-fi is hard - space, aliens, gravity, maths - but much of the "other-worldliness" comes from (human) protagonists.

There's a little too much monologuing for my liking - with side characters taking the time to pontificate on the ramifications of the plot - but it all hangs together well. The pace drives along the action without every sacrificing the reader's understanding.

The first part of a trilogy - I'm impatient for the next two episodes!

Thinking About It Only Makes It Worse: And Other Lessons from Modern Life - David Mitchell

This is little more than a rehash of David's columns for The Guardian.

They're... fine, I guess. A lot of his humour works best as a spoken rant. Parts are genuinely funny - but the righteous indignation works best as an occasional treat. Reading page after page begins to feel like hard work.

One to dip into now and again - but a bit of a slog for a pool-side read.

China in Ten Words - Yu Hua

Back to China again! Acclaimed novelist Yu Hua writes a biography of himself and modern China.

A book such as this can only thinly penetrate the rise of the Middle Kingdom to global prominence. It's a moving story of an individual trying to navigate the aftermath of Mao's Cultural Revolution and find a position for themselves.

It occasionally feels like it trivialises the oppression and brutalisation which occurred in society - and it is surprisingly uncritical of the way in which China has developed, despite acknowledging its limitations.

It is a lyrical and poetic look at China - admittedly only through the eyes of one man - which vividly illustrates why and how it became such a force in the modern world.

The Blind Giant: How to Survive in the Digital Age - Nick Harkaway

Can you steal a book you were given for free? At some Google event I was given a promotional copy after seeing the author chair a panel. I didn't want to take the hardback in my suitcase, so I "procured" a eBook copy from a dodgy torrent site. Is that moral? Is it legal? Is it the normal way of doing things in the future?

The Blind Giant covers all these points and more. It is a guidebook to the future - if you've been digitally connected since childhood, there's a good chance parts of this book are redundant. That said, it's always good to be reminded of how others see and experience the new economy.

Although published in 2013 (an age in 'net time) it is still relevant. Most importantly, it challenges us to think about the future with an eye for social norms. We shouldn't wait until we have robot/brain interfaces, we need to start shaping society and laws to cope with them. In many ways, this is happening with "driverless" vehicles - laws are being drafted for their inevitable arrival and, hopefully, those employed as drivers are looking for ways to diversify.

The book is written in a fairly academic style - and isn't best suited for an 8 hour transatlantic flight - but it does an admirable job of stimulating thoughts about what challenges netizens will face in the coming years.

In the introduction, Nick points out the folly of predictions -

More generally: it is inevitable that I will be wrong about any number of predictions. No book which tries to see the present and anticipate the future can be both interesting and consistently right. I can only hope to be wrong in interesting ways.

He was more right than he knew! The book is resplendent with links to social objects:

Throughout The Blind Giant, you will find printed hypertext links ... If you click on these links or type them into your web browser, you will find yourself at the book’s website, where you can share a fragment of the text with friends ... Thoughts on paper are fixed, but the world moves on.

Sadly, many of the links are now dead. Sic transit gloria electricus!

And I'm done! 15 books in just over a week - I feel more relaxed than ever and grateful that I'm still able to read novels when I'm not distracted by Twitter. I can't wait for my next holiday!

So, here it is, the world famous mystery that is the Voynich Manuscript now in convenient eBook format.

• PDF - suitable for Kindle, nook, Kobo, Android, iOS and for most devices (60MB)
• CBZ - suitable for comic book readers, tablets, etc. (60MB)

As the book is pure images, I decided not to convert to .mobi or .epub. Those are great formats, but offer nothing to picture books that can't be provided by PDF.

Here's how I converted the book for all the different formats.

Size

The original files were quite large - over 4MB per image - leading to a total eBook file size of 660MB. A little on the chunky side to easily distribute.

Using mogrify, it was easy to convert the images to a more manageable size.

A simple mogrify -resize 25% -quality 85 *.jpg took the total file size down to 63MB with no visible reduction in quality.

But, of course, there were a series of problems.

Firstly, some of the pages are double or triple width. They are scanned at a different size from all the other files.

In this case, the quarter size file is too small to read. Here it is after resizing:

Indeed, each image is at a subtly different size. So simply resizing by percentage doesn't work well. The size of all the images was determined using ImageMagick's "identify" command. While the largest images had a height of around 6,000 pixels, the smallest had a height of 1352. I figured that a height of 1280 would be enough to satisfy most eReaders and tablets.

mogrify -resize x1280 -quality 85 *.jpg

Once done, the total file size of all the pages was 59MB. Good enough! With the advantage of having every page the same height - if not width.

Cropping

The scans of the images contain lots of extraneous information - most notably the edges of other pages.

Manually cropping each image is possible - although tedious. It results in a slightly higher quality image - click for bigger:

Although it would be possible to automate a simple crop, each page has a unique layout. If there's further interest, I may spend some free evenings cropping the pages individually.

Page Order

As I said in my blog post of 2010:

The Yale site has all the scans available as high-res JPGs or MrSIDs - but it's a pain to download hundreds of images from the site.
So - I turned to a torrent. Don't worry! These images are hundred of years old - they are in the public domain.

(NB, if your ISP is censoring the link from The Pirate Bay, you can use this Magnet Link instead).

I'm still slightly unsure of the order of all the pages. Some have numbers printed on them, but it's not clear whether they're sequential. Indeed, it's probably impossible to know how the pages were originally ordered. So, I've left them in the same order as the Yale scan.

Converting

I didn't transform the images to greyscale because, although current eInk devices are only black and white, future devices will have full colour displays. Although using greyscale would reduce the file size by a minor amount, most modern devices should have enough internal memory to store a colour file of that size and - hopefully - have enough power to render each page without difficulty.

Kindle Formatting

I initially uploaded the PDF to Amazon's Kindle store. After it had finished converting, it ended up looking like this: All the pages had been rotated and chopped in half. Helpful.

I decided that it would be easier to use Kindle Comic Creator - it seemed to work fine in Linux using WINE.

That had the advantage of being easily able to select sections of pages as "panels". I had to whack the quality of the images down a little to fit within Kindle's 50MB upload limit. However, despite the total image size being 30MB, the resultant file was still over 60MB! It also turned out that the Kindle Comic Creator resized images so they were no taller than 1024px.

Using the Kindle Creation software imports and keeps your original files - doubling the file size! There are tools to strip the bloat but they won't produce valid files to upload to the Kindle store.

So, I had to make a dirty compromise. I scaled the images to 1,000 pixels high, and set JPG compression to 65%. The images aren't as high quality as I would like - but the file size is a svelte 49MB (despite the source being 22MB!).

After all that kerfuffle, once Amazon converted it, it was back down to 23MB! If there's interest, I'll see if there's any way I can make it higher quality in the future.

Conclusion

I really don't know why I didn't complete this a few years ago. Just another of those projects which got lost on the back burner!

• PDF - suitable for Kindle, nook, Kobo, Android, iOS and for most devices (60MB)
• CBZ - suitable for comic book readers, tablets, etc. (60MB)

Yesterday, I received this delightful email from the publishers Simon and Schuster.

NEW book by the "Prophet of Silicon Valley", Jaron Lanier, available for your review!

Dear Mr. Vogels,

I'm writing today to let you know of a new book from Simon & Schuster that I think your readers will be interested in — WHO OWNS THE FUTURE? By Jaron Lanier (available 5/7/13).

Jaron Lanier is the bestselling author of You Are Not a Gadget, the father of virtual reality, and one of the most influential thinkers of our time.

[Snip a load of marketing fluff]

I'd like to send you a copy of the book to review.

Well, let's ignore the fact that I am not "Mr Vogels". PR is a hard job and getting people's names right isn't a fundamental part of relationship building.

So, as the book sounded interesting, I asked to be emailed a copy.

I'm happy to send you a hardcover copy. If you might provide a mailing address, I'll put one in the mail today.

As regular readers know, last year I burned all my paper books and switched exclusively to ebooks. I related that fact, and mentioned that it would be much quicker than international shipping - surely a priority if you want advanced reviews.

Unfortunately, I'm not able to send ebooks as review copies at this time. Many thanks for your interest.

So, the "Prophet of Silicon Valley" wants his words stamped on to bits of dead tree, then air shipped at great expense across the Atlantic - as opposed to emailing me a few MB?

How very futuristic.

Don't Copy That Floppy!

I don't know if this is Simon and Schuster police on ebooks - or Jaron Lanier's. Given Jaron's stance on DRM and his publisher's ongoing DRM law-suit it could be either.

What I do know is this:

• Regional release dates are antithetical to the Internet. The UK edition of "Who Owns The Future" went on sale in the UK on 7th March - it goes on sale on May 7 in the US/Canada and May 22 in Australia/NZ
• 1 minute of Googling found an eBook copy, without DRM, nestling on a Russian server. Along with the usual polarity of Magnet links.

It just goes to show, even the most influential futurist, and his publishers, are still struggling with the reality of the Internet and the ongoing eBook revolution.

The book certainly sound very interesting. To quote from the preface:

The very action of reading makes you the hero of the story I am telling. Maybe you bought, or stole, a physical copy, paid to read this on your tablet, or pirated a digital copy off a share site. Whatever the prequel, here you are, living precisely the circumstances described in this book.

If you paid to read this, thank you! This book is a result of living my life as I do, which I hope provides value to you. The hope of this book is that someday we’ll all have more ways to grow wealth as a side effect of living our lives creatively and intelligently, with an eye to doing things of use to others.

If you paid to read, then there has been a one-way transaction, in which you transferred money to someone else.

If you got it for free, there has been a no-way transaction, and any value traded will be off the books, recorded not in any ledger but rather in the informal value systems of reputation, karma, or other wispy forms of barter. That doesn’t mean nothing has happened. Maybe you’ll get some positive strokes over a social network because of what you say about the book. That sort of activity might benefit us both. But it’s a kind of benefit that is unreliable and perishable.

"Who Owns The Future?" is available from all good book stores, as both a hardback or eBook. Jaron Lanier would probably prefer you buy it, but is probably equally wary of your personal details ending up in a mega data warehouse.

As requested, I recorded all the sessions I could. Here are my notes, and the audio from each session I attended.

I've based some of the information from the excellent LibraryCampLondon Wiki, including the session proposal page.

Hidden Collections

Katie Birkwood lead a fascinating session. It turns out that there are hundreds of thousands of books hidden away in libraries. They're not in the records because of a lack of decent cataloguers. Which means people can't find them. Which means they're rarely used. Which means penny pinching bureaucrats want to "dispose" of them - homely or otherwise.

One suggestion I made was to use "negative metrics". Look at your search results - what common queries receive no results? Those are the books you need to prioritise for cataloguing.

Audio

Tweets

https://twitter.com/NazlinBhimani/status/307811824257077248

Chris B.

@libchris

crowdsourcing - how many have message on catalogue that invites contributions? (COPAC prominently says if an error let us know) #libcampldn

---

Overdrive DRM is Broken (and that's ok)

I gave a demonstration on just how ridiculously easy it is to circumvent the ePub DRM system used by OverDrive - the system used for ebook lending in many libraries.

I used the metaphor of a locked box of doughnuts. I handed the box to a willing victim and told her that she may only unlock the box if she promised only to eat one doughnut and not to share any with her friends. I then gave her the key. Shortly afterwards the group had scoffed the lot! That's what makes DRM impossible; you give the lock and the key to the same person and expect them never to behave dishonourably.

I then facilitated a discussion around whether DRM was ever justified, whether libraries should boycott OverDrive, and if it was legal for a librarian to tell a patron how to circumvent artificial restrictions.

Audio

Tweets

I'm happy to say the session was generally well received.

shedsue

@shedsue

Most exciting session of the day IMO Hacking Overdrive #libcampldn NOW! With @edent

---

LibraryCamp

@LibraryCamp

Replying to @edent@edent on hols with kindle. Ebooks in libraries. Prob 1) two weeks loan - why - this is a digital file! WTF #libcampldn #ebooks

---

Lloyd Davis

@LloydDavis

Much anxious excitement in the group as @edent showed how straightforward DRM is to break on epubs #libcampldn

---

Claire Back

@calire

eBook session one of the best I’ve been to at a Library Camp. @edent and not just because of the doughnuts. #LibCampLDN

---

Facilitating the "Coming of Age" of Open Source Software in Libraries

Primarily this was a discussion about the Open Library Environment. An innovative foundation to open source many parts of the library environment.

We also discussed how to use other open source products - like FireFox on Internet connected PC.

Annoyingly, one of the participants asked for the session not to be recorded - which I have no problem with, but the guy didn't even speak during the discussion. What's that all about?

CodeClub

Greedily, I ran a second session after being asked to talk about my experiences teaching kids to code using Scratch.

I talked about the CodeClub.org.uk website and resource - telling the librarians how they could get involved.

I even showed off some of the Scratch games I had written. Hopefully it will encourage more libraries to run coding sessions for kids.

Audio

Comics

Try as I might, I just can't get in to graphic novels. I used to love the Beano as a kid, but most modern comics leave me cold.

It was, therefore, fascinating to hear about the rich tapestry of comics within libraries. How they're often used as gateways for reluctant readers. How inexperienced librarians often misfile them. And how explicit manga is not always suitable for young readers.

My resolution - to try to find at least one comic that I really enjoy. I've been recommended Maus (the story of the holocaust told from the perspective of some mice) and Kill Shakespeare (a frankly bonkers epic tale about all of Shakespeare's characters).

Audio

Overall

A really worthwhile and invigorating experience. Senate House Library is a delightful venue - although the layout of the rooms made the sessions a little noisy at times.

I enjoyed coming to an unconference which was so diverse, and had almost nothing to do with my day-to-day life. I'd recommend that every BarCamp attendee participates in an event which is totally outside of their normal life.

Once I went to university, I stopped going to the library. Even university libraries are pretty poor for computer science books - and the ability to buy cheap paperbacks online obliterated my need to visit them. It wasn't until Surrey libraries started offering digital borrowing that I even thought about my local library. Nowadays, I'm found there once a week teaching computer programming to kids.

The popular children’s-author Terry Deary says that libraries are obsolete and, in these times of austerity, we shouldn't feel bad about them closing down.

Naturally, this has outraged both traditional and progressive authors.

Neil Gaiman

@neilhimself

Selfish & stupid, shortsighted & sad. Mostly selfish. Terry Deary gets avaricious & anti-library: bit.ly/15de7Ve

---

Even regular people seem to think that libraries ought to be preserved.

Neil Gaiman

@neilhimself

Selfish & stupid, shortsighted & sad. Mostly selfish. Terry Deary gets avaricious & anti-library: bit.ly/15de7Ve

---

Metty

@_Metonymy_

Replying to @neilhimself@neilhimself what?!? Our library is like the center of our community! Tons of events happen there, and it is always packed

---

(I've picked those two statuses more-or-less at random, there are thousands of people who tweeted Neil about this.)

What it comes down to is asking "what is a library for?"

When I've expressed the opinion that high-street shopping needs to die off, someone always says to me "But old folk like going out shopping - it's their only chance to interact and chat with people."

The is a skeuomorph solution. We're keeping alive a vestigial part of society when really, we ought to be re-engineering it. We wouldn't say "electric car engines should be designed in such a way that they should be started by hand cranks and towed by horses," would we?

The solution for pensioners' social lives isn't necessarily coupled with a retail experience - we should have decent care, community centres, and services which meet people's needs.

It's exactly the same with libraries.

Last month, I wrote about the proper use of the library. It's no longer solely about borrowing books or looking up back issues of periodicals. It's about the Internet, a community meeting space, a learning environment. Being the rampant egotist that I am, I'll quote myself:

The proper use of a library is a space where people can feel safe and enjoy free access to culture.

Let me spell it out simply. Lending books is not what a library is for any more.

We need to decouple the idea of book loans from that of a library. All those people who say that their library is used for events, for poetry readings, for toodlers, for accessing the web, and for teaching kids to code - they're not talking about a building for book lending, they're talking about a community centre.

For readers and authors who are worried about people not discovering new books, or being unable to take a chance on a new author or genre, the ability for a random small town library to carry a specific book is vastly inferior to an author giving away free copies on their website.

Amazon is full of authors allowing their books to be downloaded for free (or at extremely low cost) in the hope that the reader will be sufficiently interested to buy the next book in the series.

Rethinking The Library

What would happen if we shut down all the libraries and gave everyone in the UK a Kindle?

(Aside from massive protests!)

Here's some back of the envelope calculation...

Unison have produced an excellent pamphlet about library provisions in the UK. In it, they estimate that the total expenditure on UK libraries in 2006 was £1,063,120,000.

A billion quid plus change. I assume that covers buying books, staffing, buildings, etc. The Public Lending Right in the UK means that authors get paid when their book is borrowed from the library (6.2p per borrow, to a maximum of £6,600). According to the PLR, the total cost of this was £7.6 million in 2006. That's roughly 122,580,000 library borrows per year - a little over two per person.

I'm unsure if the £1 billion figure includes the PLR's £7.6 million - but let's say it does to err on the side of caution.

A basic eReader's cost ranges from £60 for a Kobo, to £70 for a Kindle, to £80 for a Nook. Let's assume that technology gets cheaper, that eReaders are treated like books for VAT purposes (0% rather than 20%), and economies of scale means that prices drop.

It's not a stretch of the imagination to say that next year a basic e-ink ereader could cost no more than £30. Indeed - the Txtr beagle eReader costs a mere £8. Yes, eight.

The billion pound yearly library budget costs roughly £20 per person per year.

So, dissolve all the libraries. Give everyone in the UK a voucher good for £30 off a qualifying eReader (you can buy the cheap as chips version or pay extra to get the Kindle HD Super Max Plus if you want).

With the remainder of the money, continue the Public Lending Scheme but tie it to a national digital library.

You can either borrow a book - in which case the state will pay the author. You can download a public domain books - for free. Authors can sell their books for whatever price they choose - or give them away.

Who loses? Every person - not just children - gets fast access to infinite knowledge, authors get exposure and get paid, vast sums of public money are saved, and we can use the remainder of the money to digitize our archives.

The physical buildings which house books can be converted into community centres, meeting places, Internet hubs - without the need to store books and insist on silence.

Rip, Mix, Burn

I don't mean we should literally set fire to libraries - nor their book collections.

Apple launched its iMac and integrated CD writer with the slogan "Rip, Mix, Burn".

The idea is simple. Rip the music out of your CDs or vynil and convert them to digital information, mix them up to create new things, burn the new tracks into the world.

And that's exactly what is needed with libraries. Rip the analogue books to digital formats, remix the services so they're more useful to people, burn the new way of experiencing culture into society.

My bookshelves used to burst at the seams. Every individual shelf bowed violently from the over-stuffed mass of paperbacks squeezed onto it. Shakespeare rubbed up with Straczinsky. A complete set of Terry Pratchett was enviously glowered at by a patchy Enid Blyton collection. Half-read oddities nestled with well worn volumes. A copy of "Charlie and the Chocolate Factory" which taught me how to read as a child occupying the same shelf as the a Camille Paglia book purchased solely to make me look intellectual. I somehow acquired two copies of Machiavelli's "The Prince". What I want to convey to you is that I have a deep and abiding love for books. That is why I must burn them.

Every time I find an ebook copy of a book I have on my shelf, the ebook version is downloaded - redundantly backed up - and placed in my Calibre library. The physical book is burned.

Please, don't get me wrong. I'm not engaging in a Fahrenheit 451 orgy of destruction. Each physical book is carefully checked and then given to a charity shop so it can be enjoyed anew by someone else. I'm not a monster! I even gift-aid my donations.

But, for a while, it felt like I was doing something terrible. Destroying or disrespecting books is a secular sin - that's what comes of having an English teacher for a parent. I'm trying to be a hip, 21st century guy and live digitally - but I have a heap of 20th century baggage (and a bunch of monkey-brained concepts) which are hard to let go of.

Having physical stuff feels good. Ultimately though, digital stuff is better. More convenient, easier to save in case of a fire, more useful, and takes up less space. I'm doing to my books what I did to my CDs - going 100% digital.

But What About The Smell Of Books?

Throughout the web, you'll see people saying "Oh! But reading an ebook doesn't feel the same! You don't get same smell as old books! They're cold and soulless."

This is nonsense. Find an ebook copy of the book you loved as a child. After a minute, you'll be right inside Willy Wonka's Chocolate Factory and won't care whether the words are on eink or written on papyrus.

CDs are better at reproducing music than vinyl records ever were. DVDs are better than VHS. Ebooks are better than physical books. They contain the same words, the same stories and ideas, they entertain and delight in exactly the same way.

And now, I can carry my whole library with me wherever I go.

What Is A Library For?

Libraries have many ancillary functions that they've developed over the years. At their core, they are no more than a semi-convenient warehouse for books.

I asked my local library how many books they had - this was their response.

Terence Eden is on Mastodon

@edent

Replying to @SurreyLibraries@SurreyLibraries hi, do you know how many books are in the Surrey Library system? Or how many physical books a library like Woking has?

---

Surrey Libraries UK

@SurreyLibraries

Replying to @edent@edent Hi - at the end of last year we had 1,708,142 books in stock (Surrey Libraries) Woking Library had 74,805.

---

(I assume that's physical volumes rather than distinct titles as they have duplicate copies of popular books.)

How Many Books Could a Bookchuck Chuck?

Could I carry on my Kindle every single book that my local library holds?

Ebooks come in various file sizes. A typical novel weighs in at under 500KB. An illustrated book is typically under 3MB. A comic - or other work with a large quantity of images - could be over 100MB.

Judging from my collection, the average ebook file is around 1MB. Which is rather convenient for our calculations.

Woking Library has 74,805 books. At 1MB each - that's 73GB of files.

Buying a 75GB hard disk today is almost impossible. The smallest size readily available is 1TB. That's 1024GB. Or, enough to hold 1,048,576 typical ebook. Slightly shy of the total number of books held by all the libraries in the county.

Total cost for a 1TB external hard drive? Around £60. Less than the price of a couple of cheap bookshelves.

If you wanted to carry the books on a MicroSD card - suitable for a nook or other ereader - then a 32GB card will set you back £30. You could carry around 90,000 ebooks with you for less than £100.

Realistically, though, how many books can you read in your lifetime? Even on holiday with nothing to do but laze on a beach and read - I manage about one book per day.

Were I to live to 100 years of age, and read one book per day, every day. I would barely be able to fill a single 32GB MicroSD card.

Everything I've ever read could fit on one of these.

What's My Point?

I have none. Thank you for reading anyway.

Adding File Size

This shows the sizes of the eBook files. Screenshot shows a demonstration using the free "Hacking The BBC" eBook.

--- /data/web/calibre_php_server-V0.2.7/db.php  2011-11-20 10:19:00.000000000 +0000
+++ db.php  2011-11-20 10:24:16.000000000 +0000
@@ -571,7 +571,7 @@
            }

            // Formats
-           $data = $this->db->query(sprintf('SELECT format,name
+           $data = $this->db->query(sprintf('SELECT format,name,uncompressed_size
                                              FROM data WHERE book=%d', $id));
            $fmts = array();
            while ($data and $row = $data->fetchArray()) {
@@ -581,7 +581,8 @@
                $fmts[] = array
                    ('format' => $row['format'],
                     'name' => $row['name'],
-                    'URL' => "index.php/book_format/$id/$book_name");
+                    'URL' => "index.php/book_format/$id/$book_name",
+                    'size' => round( ($row['uncompressed_size'] / 1024 / 1024), 2) . "MB");
            }
            $book['formats'] = $fmts;

@@ -809,4 +810,4 @@
            }
        }
    }
-?>
 No newline at end of file
+?>
--- /data/web/calibre_php_server-V0.2.7/smarty/templates/books_section.tpl  2011-11-20 10:19:01.000000000 +0000
+++ newTemplates/templates/books_section.tpl    2011-11-20 10:12:04.000000000 +0000
@@ -15,13 +15,13 @@
             {{if $books[book].rating_url}}
{{/if}}

             


             {{section name=format loop=$books[book].formats}}

-                {{$books[book].formats[format].format}}


+                {{$books[book].formats[format].format}} {{$books[book].formats[format].size}}


             {{/section}}

             

Escaping Ampersands

Because I'm a nerd, I believe in making valid HTML. That means that & should properly be written as &

--- /data/web/calibre_php_server-V0.2.7/smarty/templates/index.tpl  2011-11-20 10:19:01.000000000 +0000
+++ /data/www/calibre/newTemplates/templates/index.tpl  2011-11-19 23:31:17.000000000 +0000
@@ -6,7 +6,7 @@
 {{include file="header.tpl" title="Home"}}

 <table><tr><td>

 <div style="padding-right: 20px; float: left; width: 200px; font-family: 'Arial Rounded MT Bold';">

-   <a href="index.php?m=titles&p=1">

+   </a><a href="index.php?m=titles&p=1">

    <span style="padding-left:10px">Titles</span></a>

    [{{$title_count}}]

 </div>
diff -rupN /data/web/calibre_php_server-V0.2.7/modules//book_base.php /data/www/calibre/modules//book_base.php
--- /data/web/calibre_php_server-V0.2.7/modules//book_base.php  2011-11-20 10:19:01.000000000 +0000
+++ /data/www/calibre/modules//book_base.php    2011-11-19 23:29:37.000000000 +0000
@@ -34,10 +34,10 @@
            // Get the books for page N
            for ($i = min($start, count($books)); $i < min($end, count($books)); $i++) {
                $res[] = $books[$i];
-               $res[count($res)-1]['cover'] = 'index.php?m=cover&id=' . $books[$i]['id'];
+               $res[count($res)-1]['cover'] = 'index.php?m=cover&id=' . $books[$i]['id'];
                if ($short_form)
                    $res[count($res)-1]['details_url'] =
-                               'index.php?m=book&id=' . $books[$i]['id'];
+                               'index.php?m=book&id=' . $books[$i]['id'];
            }
            $smarty->assign('books', $res);
            $p_n_url = '';
@@ -47,7 +47,7 @@
            if ($page > 1)
                $smarty->assign('page_back', "index.php?{$p_n_url}p=" . ($page-1));
            if ($end < count($books))
-               $smarty->assign('page_forw', "index.php?$p_n_url&p=" . ($page+1));
+               $smarty->assign('page_forw', "index.php?$p_n_url&p=" . ($page+1));
            $smarty->assign('page', $page);
            $smarty->assign('maxpage', (int)((count($books)-1)/$config['books_page_count'])+1);
            $smarty->assign('sortable_fields', $sortable_fields);
@@ -56,4 +56,4 @@
            $smarty->assign('prev_next', $p_n_array);
        }
    }
-?>
 No newline at end of file
+?>
diff -rupN /data/web/calibre_php_server-V0.2.7/modules//category.php /data/www/calibre/modules//category.php
--- /data/web/calibre_php_server-V0.2.7/modules//category.php   2011-11-20 10:19:01.000000000 +0000
+++ /data/www/calibre/modules//category.php 2011-11-19 23:26:57.000000000 +0000
@@ -34,17 +34,17 @@
                    // item has a rating. Include the URL to display it.
                    $items[] = array('icon' => get_icon_path($v->category, $db),
                            'name'      => $v->name,
-                           'href'      => "index.php?m=catval&id=$v->id&p=1&&cat=" .
-                                           urlencode($v->category) . '&v=' . urlencode($v->name),
+                           'href'      => "index.php?m=catval&id=$v->id&p=1&cat=" .
+                                           urlencode($v->category) . '&v=' . urlencode($v->name),
                            'count'     => $v->count,
-                           'rating'    => "index.php?m=rating&r=" . round($v->avg_rating, 2)
+                           'rating'    => "index.php?m=rating&r=" . round($v->avg_rating, 2)
                            );
                } else {
                    $items[] = array('icon' => get_icon_path($v->category, $db),
                            'name'      => $v->name,
-                           'href'      => "index.php?m=catval&id=$v->id&p=1&cat=" .
+                           'href'      => "index.php?m=catval&id=$v->id&p=1&cat=" .
                                            urlencode($v->category) .
-                                           '&v=' . urlencode($v->name),
+                                           '&v=' . urlencode($v->name),
                            'count'     => $v->count
                            );
                }
@@ -60,4 +60,4 @@
    }

    $mod = new DoCategory();
-?>
 No newline at end of file
+?>
diff -rupN /data/web/calibre_php_server-V0.2.7/modules//catval.php /data/www/calibre/modules//catval.php
--- /data/web/calibre_php_server-V0.2.7/modules//catval.php 2011-11-20 10:19:01.000000000 +0000
+++ /data/www/calibre/modules//catval.php   2011-11-19 23:29:01.000000000 +0000
@@ -44,7 +44,7 @@
                                        ));
            $smarty->assign('category_name', $fm = $db->fm->name_for($cat));
            $smarty->assign('category_value', $_REQUEST['v']);
-           $smarty->assign('up_url', "index.php?m=category&id=$id&cat=".urlencode($cat));
+           $smarty->assign('up_url', "index.php?m=category&id=$id&cat=".urlencode($cat));
        }

        function template() {
@@ -53,4 +53,4 @@

    }
    $mod = new DoCatval();
-?>
 No newline at end of file
+?>
diff -rupN /data/web/calibre_php_server-V0.2.7/modules//home.php /data/www/calibre/modules//home.php
--- /data/web/calibre_php_server-V0.2.7/modules//home.php   2011-11-20 10:19:01.000000000 +0000
+++ /data/www/calibre/modules//home.php 2011-11-19 23:25:53.000000000 +0000
@@ -21,7 +21,7 @@
                $m = $db->fm->metadata_for($k);
                $items[] = array(   'icon' => get_icon_path($k, $db),
                                    'name' => $m['name'],
-                                   'href' => "index.php?m=category&cat=" . urlencode($k),
+                                   'href' => "index.php?m=category&cat=" . urlencode($k),
                                    'count'=> count($v)
                                );
            }
@@ -34,4 +34,4 @@
        }
    }
    $mod = new DoHome();
-?>
 No newline at end of file
+?>
</td></tr></table>

More

Both patches should be in the official version soon. More to come!

I love Calibre, it's the perfect eBook management tool. It comes with a built in WWW server so you can easily access your library on the go. The only problem is that this really only works if you have a single machine dedicated to Calibre. For various reasons, I don't have a single machine.

I have a desktop, laptop, and server. The Calibre Library is just a database with a set of files and folders - so all three machines sync via DropBox. As long as I don't have the Calibre program open on my desktop and laptop at the same time, everything is hunky-dory.

However, having Calibre running on the server buggers everything up. So, I'm using a separate program - Calibre-PHP Content Server.

Installation was relatively simple, assuming you've already got apache and PHP installed, you'll also need GD, SQLite3, and Smarty.

This was how I installed them on Ubuntu

sudo apt-get install smarty php5-gd php5-sqlite php5-sqlite3sqlite3

Configuration is slightly confusing. Copy the config_default.php to config_local.php and edit it.

$config['library_dir'] = '/data/Dropbox/eBooks/Calibre Library';
$config['smarty_dir'] = '/data/smarty-cache';
$config['smarty'] = '/usr/share/php/smarty';

The library directory must be readable to the webserver (chmod a+r) and all the directories above it have to be searchable (chmod a+x).

The "smarty_dir" is where the cache is stored. It needs two sub-folders, smarty_cache and smarty_templates_c. Both of these directories must be writable by the webserver.

That should be it. There's help available at the official help thread.

Here are the rules:

• Everyone has to write the opening line of a novel, based on the novel's summary.
• One player has the real opening line to copy down.
• One other player (the guesser) has to guess which of the opening lines is the real one.
• If you fool the guesser, you win the point. If the guesser is correct, she gets the point.

Simple and perfect. You don't need to be well read - you just need enough of an imagination to write what you think someone else will think the opening line of a novel is.

There's a minor problem - there are only a limited number of cards and none of them are very new. There's no Harry Potter, We Need To Talk About Kevin, or American Gods.

As with any game, it can get a little bit repetitive - once you've played it a dozen times it can become a bit obvious which are the real extracts from the books.

So, what's needed is a whole set of new cards.

The rules say:

Each card gives you the title, author and plot summary of a novel or short story.

The card also has the opening and closing sentence of the book.

So, for example, the new card for John le Carré's "Our Kind of Traitor" would say (SPOILER WARNING!)

Title: Our Kind Of Traitor

Author: John le Carré

Plot: A young Oxford academic and his girlfriend take a holiday to Antigua. By chance they bump into a Russian millionaire called Dima who owns a peninsula and a diamond-encrusted gold watch. Dima propels the young lovers on a journey to the murkiest cloisters of the City of London and its unholy alliance with Britain's Intelligence Establishment.

Opening Line: "At seven o'clock of a Caribbean morning, on the island of Antigua, one Peregrine Makepiece, otherwise known as Perry, an all-round amateur athlete of distinction and until recently tutor in English literature at a distinguished Oxford college, played three sets of tennis against a muscular, stiff-backed, bald, brown-eyed Russian man of dignified bearing in his middle fifties called Dima."

Closing Line: "Was it sex? Was it drugs? Was it arms? For want of a shred of evidence it was none of them. Terror, the great catch-all these days, has also been considered, but rejected out of hand. No group has claimed responsibility."

Making It Happen

I was planning on writing a Calibre Plugin to automate the generation of cards from my fulsome library. It would be fairly easy to extract the author, summary, and cover image. However, finding the first line from a book is a slightly tricky matter - not to mention how you determine where the last line is.

So I think I'll stick to manually for the time being.

I think this could be a great way to revatilise a wonderful game. Only a few - non-technical - problems.

Would I face a massive lawsuit from the good folk at Oxford Games? Would the insanity of the UK's copyright laws stop me?

More importantly, would anyone be interested in this? If you're an Ex Libris player and would be interested in some new cards - please leave a comment on this post!

I've finally received a reply...

Dear Terence,

Some time ago you contacted us regarding the following eBook title, "One of Our Thursdays is Missing (eBook): Thursday Next Series, Book 6" (ISBN: 9781444713039). We have had our eBook supplier investigate the content issue of this eBook title in conjunction with the publishers. They have confirmed that this title has been corrected.

Our suppliers have said, "We apologize for any delay that it has taken to resolve this issue. Issues with content require corrections from the publisher and can often take weeks or months to resolve. Unfortunately the time it took to correct this issue was due to delays from the publisher and not OverDrive."

You will now be able to purchase and download the correct content for this title if you wish to do so. Please accept my apologies for the delay.

Kind regards,

In fairness, they did refund the cost of the ebook. Rather annoyingly, they revoked my access to download it again.

I realise that it must take some time to find a sober editor in a publishing company - but 12 weeks to replace one image with another really doesn't seem right.

It's sloppy errors and poor customer service like this which will kill the eBook revolution.

Or, perhaps, that's just what some publishers want...?