<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="https://shkspr.mobi/blog/wp-content/themes/edent-wordpress-theme/rss-style.xsl" type="text/xsl"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	     xmlns:dc="http://purl.org/dc/elements/1.1/"
	   xmlns:atom="http://www.w3.org/2005/Atom"
	     xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	  xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>
<channel>
	<title>json &#8211; Terence Eden’s Blog</title>
	<atom:link href="https://shkspr.mobi/blog/tag/json/feed/" rel="self" type="application/rss+xml" />
	<link>https://shkspr.mobi/blog</link>
	<description>Regular nonsense about tech and its effects 🙃</description>
	<lastBuildDate>Sat, 11 Jul 2026 12:41:40 +0000</lastBuildDate>
	<language>en-GB</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.0.2</generator>

<image>
	<url>https://shkspr.mobi/blog/wp-content/uploads/2023/07/cropped-avatar-32x32.jpeg</url>
	<title>json &#8211; Terence Eden’s Blog</title>
	<link>https://shkspr.mobi/blog</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title><![CDATA[Scattered thoughts on social geolocation]]></title>
		<link>https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/</link>
					<comments>https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#comments</comments>
				<dc:creator><![CDATA[@edent]]></dc:creator>
		<pubDate>Wed, 22 Jul 2026 11:34:12 +0000</pubDate>
				<category><![CDATA[/etc/]]></category>
		<category><![CDATA[ActivityPub]]></category>
		<category><![CDATA[BlueSky]]></category>
		<category><![CDATA[fediverse]]></category>
		<category><![CDATA[FourSquare]]></category>
		<category><![CDATA[geolocation]]></category>
		<category><![CDATA[json]]></category>
		<category><![CDATA[open standards]]></category>
		<category><![CDATA[OpenStreetMap]]></category>
		<guid isPermaLink="false">https://shkspr.mobi/blog/?p=73050</guid>

					<description><![CDATA[I want to be able to share my location with my friends on social media. Twitter (RIP) had a way to attach an optional location to a post. Facebook still lets me check in to venues. But neither ActivityPub (Mastodon) nor AT Protocol (BlueSky) allow me to do that.  I&#039;ve been banging on about this for a while so was delighted to be invited to chat with the GeoSocial Task Force on their monthly call. …]]></description>
										<content:encoded><![CDATA[<p>I want to be able to <a href="https://shkspr.mobi/blog/2026/04/why-is-it-so-hard-to-passively-stalk-my-friends-locations/">share my location with my friends</a> on social media. Twitter (RIP) had a way to attach an optional location to a post. Facebook still lets me check in to venues. But neither ActivityPub (Mastodon) nor AT Protocol (BlueSky) allow me to do that.</p>

<p>I've been <a href="https://shkspr.mobi/blog/2024/01/rebuilding-foursquare-for-activitypub-using-openstreetmap/">banging on about this for a while</a> so was delighted to be invited to chat with the <a href="https://github.com/swicg/geosocial">GeoSocial Task Force</a> on <a href="https://www.w3.org/events/meetings/ed630a3d-7581-4053-9978-75949ad42f2a/">their monthly call</a>.</p>

<p>But, before I start, please remember that these proposals are not mandatory. If you don't want to share your location, then please don't share your location. This is an optional feature which shouldn't be used by people with a strong need for privacy.</p>

<p>We were primarily discussing the nascent <a href="https://swicg.github.io/geosocial/microsyntax.html">Geographical Microsyntax proposals</a>.</p>

<p>Firstly, I <em>love</em> the idea of being able to share my location. I want my friends to know I'm in town for a gig. I want people to think I'm cool because I'm seeing some experimental theatre. I want to virtue signal that I'm giving blood.</p>

<p>Similarly, I want to be able to find social media posts by location. What's happening right now in Paris? Did anyone else hear that sonic boom in London? Which cool people are also at this gig with me?</p>

<p>But, I do think these proposals have some issues which need to be addressed.</p>

<h2 id="user-research"><a href="https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#user-research">User Research</a></h2>

<p>The primary thing missing from the proposal is any form of understanding what users want to do with a geotagged post.</p>

<p>You see a post like:</p>

<blockquote><p>Hanging out with friends. I just checked in to <u>Leicester Square</u>.</p></blockquote>

<p>What do you expect to happen when you click that link?</p>

<ul>
<li>See who is at the location?</li>
<li>Read more information about the place being discussed?</li>
<li>Find all posts at that specific address?</li>
<li>Discover posts which are <em>close</em> to there?</li>
<li>Something else?</li>
</ul>

<p>What other things do people want to do with geotagged posts?</p>

<ul>
<li>Find all posts within 5Km of me?</li>
<li>Discover users who have posted near me?</li>
<li>Get emergency alerts for the city I'm in?</li>
<li>See what's trending in a specific country?</li>
</ul>

<p>Without knowing what people might want to do with this information, it's rather hard to design a service which meets their needs.</p>

<h2 id="to-micro-or-not-to-micro"><a href="https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#to-micro-or-not-to-micro">To micro or not to micro?</a></h2>

<p>Where I disagree with the proposals is the idea of using inline text as a form of microsyntax.</p>

<p>Proposals like "I am in /London/ eating ice-cream" or "Checked in to Leicester Square L:N123456" fail for a couple of reasons.</p>

<p>Things like #hashtags and @mentions developed organically on social networks before they were eventually adopted by the platforms themselves. It is rare that a top-down diktat can be used to tell people how they <em>should</em> be formatting their posts.</p>

<p>More importantly, people are <em>crap</em> at formatting things consistently! Every event I go to has people using <code>#Event</code> alongside <code>#Event2026</code> or <code>#Event26</code> or a hundred variations.</p>

<p>Even if users could format it consistently, names are ambiguous. Is <code>🌐:Paris</code> the one in France or <a href="https://en.wikipedia.org/wiki/Paris,_Texas_(film)">Texas</a>?</p>

<p>I'm strongly of the opinion that text is for text, not syntax.</p>

<h2 id="openness-%f0%9f%86%9a-centralisation"><a href="https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#openness-%f0%9f%86%9a-centralisation">Openness 🆚 Centralisation</a></h2>

<p>Saying that you're on a specific point on the globe doesn't always provide useful information. Even with altitude, it isn't always possible to work out if your in the Starbucks or the Costa next door. Do you want to say you're in a train station, or a distinct platform, or even a specific train journey?</p>

<p>There are various services which offer unique IDs per loosely-defined place.</p>

<p>Google Maps provides a <a href="https://developers.google.com/maps/documentation/places/web-service/place-id">Place ID</a> for every thing that it knows about. There's a similar service from <a href="https://foursquare.com/products/places-api/">FourSquare</a>.  But both of those are closed and proprietary systems - they can only be updated by the company that creates them.</p>

<p>Wikipedia's Wikidata has <a href="https://www.wikidata.org/wiki/Q676755">IDs for lots of places</a>, similarly OpenStreetMap has <a href="https://wiki.openstreetmap.org/wiki/Key:place">nodes with names</a> which can be used to identify locations.</p>

<p>But all of these suffer from the same flaw; they are centralised.</p>

<p>If <a href="https://killedbygoogle.com/">Google kills its service</a> then all those Place IDs die.</p>

<p>This has happened before. Yahoo used to run the <a href="https://en.wikipedia.org/wiki/WOEID">Where On Earth IDentifier</a> service but discontinued it.</p>

<p>While OSM and Wikipedia are great projects, they're still centralised. If you're banned from OSM, you can't create a new node.</p>

<p>So, to my mind, the <em>primary</em> way of identifying a place has to use a fully open, globally agreed, and unrestricted standard.</p>

<h2 id="location-plus-plus"><a href="https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#location-plus-plus">Location Plus Plus</a></h2>

<p>The obvious choice is latitude and longitude as defined by <a href="https://www.linz.govt.nz/guidance/geodetic-system/coordinate-systems-used-new-zealand/geodetic-datums/world-geodetic-system-1984-wgs84">WGS 84</a>. It allows for arbitrary precision anywhere on the planet.</p>

<p>No one can stop you issuing lat/long just because a country is under sanctions. The standards community can't revoke your access to it. The co-ordinates are well understood by almost all software.</p>

<p>There are some alternatives. Google's <a href="https://maps.google.com/pluscodes/">Plus.Codes</a> build on top of lat/long to make something more human readable. But, crucially, human readability is <em>not</em> the issue here. This is metadata which should never be shown to a human. If a user interface wants to format a lat/long they can do so as a Plus.Code or any other format.</p>

<h2 id="geojson"><a href="https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#geojson">GeoJSON</a></h2>

<p>There are dozens of <a href="https://wiki.ietf.org/en/group/geopriv/LocationFormats">competing ways to mark up a location</a>.</p>

<p>It seems sensible to me that, given most major social media protocols use JSON, adding <a href="https://geojson.org/">GeoJSON</a> would be the simplest way to add geographic information to a post's metadata.</p>

<p>At its most basic, a single "Point" can be shared like so:</p>

<pre><code class="language-json">{
    "type": "Feature",
    "geometry": {
        "type": "Point",
        "coordinates": [-0.13005,51.5103]
    }
}
</code></pre>

<p>It can have as much or as little precision as needed.</p>

<p>If the place being checked into has a name, it can also be added - this could be used for display purposes within the message</p>

<pre><code class="language-json">{
    "type": "Feature",
    "geometry": {
        "type": "Point",
        "coordinates": [-0.13005,51.5103]
    },
    "properties": {
        "name": "Leicester Square"
    }
}
</code></pre>

<p>The <code>properties</code> can be as complex as the sender wants. For example it could add Wikidata references, OpenStreetMap IDs, telephone numbers, or some cool new thing which hasn't been invented yet.</p>

<p>Here's a maximalist example of checking in to a park:</p>

<pre><code class="language-json">{
    "type": "Feature",
    "geometry": {
        "type": "Polygon",
        "coordinates": [
        [
                [-0.1303791,51.5099976],
                [-0.1295305,51.5100444],
                [-0.1297077,51.510947 ],
                [-0.1308786,51.5105859],
                [-0.1308893,51.5105793],
                [-0.1303791,51.5099976]
            ]
        ]
    },
    "properties": {
        "name": "Leicester Square",
        "WikiData": "Q848912",
        "OSM": "4082589",
        "GooglePlaceID": "ChIJPcTFENIEdkgR94E58rgB69o",
        "PlusCode": "9C3XGV69+4X",
        "Quinfrob": "🌐89451_𰻝",
        "address": {
            "road": "Leicester Square",
            "neighbourhood": "St. James's",
            "quarter": "East Marylebone",
            "suburb": "Covent Garden",
            "city": "City of Westminster",
            "ISO3166-2-lvl8": "GB-WSM",
            "state": "England",
            "ISO3166-2-lvl4": "GB-ENG",
            "postcode": "WC2H 7NA",
            "country": "United Kingdom",
            "country_code": "gb"
        }
    }
}
</code></pre>

<h2 id="whats-next"><a href="https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/#whats-next">What's Next</a></h2>

<p>My personal opinion is that the following needs to happen:</p>

<ul>
<li>Talk to some users - find out what they might use this for.</li>
<li>Work with privacy advocates to reduce possible harms (for example, being able to redact a location from a previously shared post).</li>
<li>Discuss with developers about how they'd work with this metadata.</li>
<li>Clients should start adding GeoJSON metadata to their posts, even if it can't be displayed yet.</li>
</ul>

<p>You can <a href="https://hedgedoc.socialweb.coop/hNLDxDqoQPaFlOf4aJ4E8g?view">read the minutes of the meeting</a> and I'd encourage you to <a href="https://www.w3.org/events/meetings/ed630a3d-7581-4053-9978-75949ad42f2a/">join the next call</a>.</p>
<img src="https://shkspr.mobi/blog/wp-content/themes/edent-wordpress-theme/info/okgo.php?ID=73050&HTTP_REFERER=RSS" alt="" width="1" height="1" loading="eager">]]></content:encoded>
					
					<wfw:commentRss>https://shkspr.mobi/blog/2026/07/scattered-thoughts-on-social-geolocation/feed/</wfw:commentRss>
			<slash:comments>13</slash:comments>
		
		
			</item>
		<item>
		<title><![CDATA[HOWTO: Sort BitWarden Passwords by Date]]></title>
		<link>https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/</link>
					<comments>https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/#comments</comments>
				<dc:creator><![CDATA[@edent]]></dc:creator>
		<pubDate>Wed, 14 Feb 2024 12:34:30 +0000</pubDate>
				<category><![CDATA[/etc/]]></category>
		<category><![CDATA[bitwarden]]></category>
		<category><![CDATA[HowTo]]></category>
		<category><![CDATA[json]]></category>
		<category><![CDATA[linux]]></category>
		<guid isPermaLink="false">https://shkspr.mobi/blog/?p=49616</guid>

					<description><![CDATA[I highly recommend BitWarden as a password manager. It is free, open source, and has a great range of apps and APIs.  The one thing it doesn&#039;t have is a way to sort your accounts by creation date.  I now have over a thousand accounts that I&#039;ve added - so I wanted to prune away some of the older ones.  So, here&#039;s how to do it.  Export your vault  In the desktop version of BitWarden, go to File → E…]]></description>
										<content:encoded><![CDATA[<p>I highly recommend <a href="https://bitwarden.com">BitWarden</a> as a password manager. It is free, open source, and has a great range of apps and APIs.</p>

<p>The one thing it <a href="https://community.bitwarden.com/t/sort-items-by-date-of-modification-addition-last-use-etc/2484">doesn't have is a way to sort your accounts by creation date</a>.  I now have over a thousand accounts that I've added - so I wanted to prune away some of the older ones.</p>

<p>So, here's how to do it.</p>

<h2 id="export-your-vault"><a href="https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/#export-your-vault">Export your vault</a></h2>

<p>In the <em>desktop</em> version of BitWarden, go to File → Export Vault.  Choose the JSON format (this doesn't work for CSV) and follow the on-screen instructions.</p>

<img src="https://shkspr.mobi/blog/wp-content/uploads/2024/02/BitWarden-Export-fs8.png" alt="Screenshot of the BitWarden export page." width="793" height="500" class="aligncenter size-full wp-image-49617">

<p><strong>NOTE!</strong> This will save <em>all</em> of your passwords to disk. Do not do this on a shared machine. Delete the file as soon as you are done with it.</p>

<h2 id="sort-with-jq"><a href="https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/#sort-with-jq">Sort with JQ</a></h2>

<p>Using the <a href="https://jqlang.github.io/jq/"><code>jq</code> utility</a> it is possible to search and sort the exported JSON.</p>

<p>This command pipes your export to JQ. It selects all the <code>items</code>, then it sorts by when the item was last edited. It then displays the name of the account and the date, followed by a newline.</p>

<pre><code class="language-bash">cat bitwarden.json | jq -r '.items | sort_by(.revisionDate) | .[] | .name, .revisionDate, " " '
</code></pre>

<h2 id="purge"><a href="https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/#purge">Purge!</a></h2>

<p>You <em>could</em> <a href="https://bitwarden.com/help/cli/#delete">use the API to delete items</a> based on their ID.  Personally, I'd rather go through manually.</p>

<h2 id="whats-next"><a href="https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/#whats-next">What's next?</a></h2>

<p>It would be great if BitWarden allowed sorting by date in their UI. Even better if they could sort by usage.  Until then, I'll spend every Valentine's Day manually deleting old and unloved accounts.</p>
<img src="https://shkspr.mobi/blog/wp-content/themes/edent-wordpress-theme/info/okgo.php?ID=49616&HTTP_REFERER=RSS" alt="" width="1" height="1" loading="eager">]]></content:encoded>
					
					<wfw:commentRss>https://shkspr.mobi/blog/2024/02/howto-sort-bitwarden-passwords-by-date/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
		<item>
		<title><![CDATA[KYLI - because it is superior to JSON]]></title>
		<link>https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/</link>
					<comments>https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#comments</comments>
				<dc:creator><![CDATA[@edent]]></dc:creator>
		<pubDate>Tue, 21 Mar 2017 12:08:04 +0000</pubDate>
				<category><![CDATA[/etc/]]></category>
		<category><![CDATA[data]]></category>
		<category><![CDATA[json]]></category>
		<category><![CDATA[usability]]></category>
		<guid isPermaLink="false">https://shkspr.mobi/blog/?p=24889</guid>

					<description><![CDATA[This is a (silly) attempt to fix some of the shortcomings of JSON. Hence it is named after the goddess of music.  It uses C0 Control Characters  Here is an example:  ␜    ␁       This is a KYLI document    ␂       ␝ GroupName          ␞ data ␟ value          ␛ Comments are supported too!          They can be multilined easily. ␙   I&#039;ve used Unicode Control Pictures so you can see what&#039;s happening.…]]></description>
										<content:encoded><![CDATA[<p>This is a (silly) attempt to fix some of the shortcomings of JSON. Hence it is named after <a href="http://www.kylie.com/">the goddess of music</a>.</p>

<p>It uses <a href="https://en.wikipedia.org/wiki/C0_and_C1_control_codes"><code>C0</code> Control Characters</a></p>

<p>Here is an example:</p>

<pre><code class="language-_">␜
   ␁
      This is a KYLI document
   ␂
      ␝ GroupName
         ␞ data ␟ value
         ␛ Comments are supported too!
         They can be multilined easily.
␙
</code></pre>

<p>I've used <a href="http://graphemica.com/blocks/control-pictures">Unicode Control Pictures</a> so you can see what's happening.  In reality, ␜ is <code>&amp;#28;</code> - which on your display looks like .</p>

<h2 id="why-kyli-is-100x-better-than-crummy-json"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#why-kyli-is-100x-better-than-crummy-json">Why KYLI is 100x better than crummy JSON</a></h2>

<p>JSON is ok, I guess. But it is fragile - breaking if a comma is out of place. It is also <em>really</em> hard to use with text; you either have to escape every <code>"</code> or <code>'</code> - or remember to HTML encode (and decode) everything. It doesn't have comments.  There's no versioning.</p>

<p>KYLI attempts to improve on some of those shortcomings. Here's how it works.</p>

<h2 id="philosophy"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#philosophy">Philosophy</a></h2>

<ol>
<li>Magic characters control the flow of the document.</li>
<li>A magic character is terminated by the presence of another magic character.  This means...</li>
<li>Whitespace is a first class citizen.  It can be used in names, values, comments.</li>
<li>The only characters which <em>can't</em> be used within names and values are magic characters.

<ol>
<li>Every other Unicode character is available for names, data, values, comments.</li>
</ol></li>
<li>There are no "types" - everything is a string.

<ol>
<li>If you want to use an alternative numbering system, or use <code>⊨</code> and <code>⊭</code> for boolean, that's encouraged.</li>
</ol></li>
</ol>

<h2 id="structure"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#structure">Structure</a></h2>

<p>KYLI:</p>

<ul>
<li><strong>Will</strong> start with <code>␜</code> File Separator.</li>
<li><strong>May</strong> have a header - indicated with <code>␁</code>

<ul>
<li>A header can contain any pertinent information. For example, KYLI version or document metadata. </li>
<li>Headers are <em>unstructured</em>. They can hold any valid text (including XML or JSON). </li>
</ul></li>
<li><strong>Will</strong> indicate the start of the data using <code>␂</code> Start of Text.</li>
<li><strong>May</strong> contain data:

<ul>
<li>A datum's <em>name</em> <strong>will</strong> start with <code>␞</code> Record Separator.</li>
<li>A datum's <em>value</em> <strong>will</strong> start with <code>␟</code> Unit Separator.</li>
<li>All values are Unicode strings. There is no concept of number types or booleans.</li>
<li>Data <em>may</em> be grouped:

<ul>
<li>The group's name <strong>will</strong> start with <code>␝</code> Group Separator.</li>
<li>There is no limit to how many subgroups there may be.</li>
</ul></li>
</ul></li>
<li><strong>Will</strong> end with <code>␙</code> End of Medium.</li>
</ul>

<h2 id="examples"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#examples">Examples</a></h2>

<p>It is helpful to think of KYLI as a stream of Unicode characters.  When <em>viewing</em> a document, you may want to "pretty print" it for your convenience.</p>

<h3 id="basic"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#basic">Basic</a></h3>

<p>The minimum viable KYLI is something like:</p>

<pre><code class="language-_">␜␞age␟42␙
</code></pre>

<h3 id="multi-lingual"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#multi-lingual">Multi-lingual</a></h3>

<p>Names can be any valid Unicode string (that is, which doesn't contain control characters).</p>

<pre><code class="language-_">␜
   ␁
      Generated on 2017-03-25
   ␂
      ␝ People's names
         ␝ 英国人
            ␛ That's the Chinese word for
              "English People"
            ␝ 123456
               ␞ Name ␟ Terence Eden
            ␝ 789456
               ␞ Name ␟ John Lennon
␙
</code></pre>

<p>Which is transmitted as</p>

<pre><code class="language-_">␜␁Generated on 2017-03-25␂␝Different People␝英国人␛That's the Chinese word for
"English People"␝123456␞Name␟Terence Eden␝789456␞Name␟John Lennon␙
</code></pre>

<h2 id="multiline-data-easily"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#multiline-data-easily">Multiline data easily</a></h2>

<pre><code class="language-_">␜
   ␂
      ␞ HTML for the site ␟ &lt;!doctype html&gt;
&lt;html&gt;
   &lt;head&gt;
       &lt;title&gt;
      ...
      ␞ date ␟ 2018-01-15
␙
</code></pre>

<h2 id="why-this-is-better"><a href="https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/#why-this-is-better">Why this is better</a></h2>

<ul>
<li>It discourages editing by hand! You should be using tools to create data or config files. They can verify their validity quicker and more accurately than any human. </li>
<li>You don't need to worry about escaping "special" characters like <code>"</code>!</li>
<li>Comments!</li>
<li>No worrying about data types!</li>
<li>More compact!</li>
<li>No bloody commas tripping things up!</li>
<li>Metadata!</li>
<li>No confusion over whether something is an object, array, or string!</li>
<li>Easier to validate!</li>
</ul>

<p>I am confident that KYLI will supplant JSON <del>in the 1990s pop charts</del> as the preferred data transmission standard of the 21<sup>st</sup> Century.</p>

<p><a href="https://www.flickr.com/photos/evarinaldiphotography/6805204016/"><img src="https://shkspr.mobi/blog/wp-content/uploads/2017/03/Kylie-striking-a-pose.jpg" alt="Kylie striking a pose https://www.flickr.com/photos/evarinaldiphotography/6805204016/" width="512" height="342" class="aligncenter size-full wp-image-24962"></a></p>
<img src="https://shkspr.mobi/blog/wp-content/themes/edent-wordpress-theme/info/okgo.php?ID=24889&HTTP_REFERER=RSS" alt="" width="1" height="1" loading="eager">]]></content:encoded>
					
					<wfw:commentRss>https://shkspr.mobi/blog/2017/03/kyli-because-it-is-superior-to-json/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
		<item>
		<title><![CDATA[YQL and The Pirate Bay]]></title>
		<link>https://shkspr.mobi/blog/2012/05/yql-and-the-pirate-bay/</link>
					<comments>https://shkspr.mobi/blog/2012/05/yql-and-the-pirate-bay/#comments</comments>
				<dc:creator><![CDATA[@edent]]></dc:creator>
		<pubDate>Wed, 02 May 2012 07:17:55 +0000</pubDate>
				<category><![CDATA[/etc/]]></category>
		<category><![CDATA[politics]]></category>
		<category><![CDATA[jquery]]></category>
		<category><![CDATA[json]]></category>
		<category><![CDATA[legal]]></category>
		<category><![CDATA[piracy]]></category>
		<category><![CDATA[Star Wars]]></category>
		<category><![CDATA[the phantom menace]]></category>
		<category><![CDATA[the pirate bay]]></category>
		<category><![CDATA[yql]]></category>
		<guid isPermaLink="false">http://shkspr.mobi/blog/?p=5649</guid>

					<description><![CDATA[For reasons best known to themselves, certain sections of the entertainment industries seem to believe that bolting the stable door shutting down The Pirate Bay will stop all piracy.  It&#039;s as though they think that people won&#039;t be able to use a proxy, circumvent the Cleanfeed block, or simply use a search engine to find another torrent site.  Build Your Own Pirate Bay?  Proxying is a very simple…]]></description>
										<content:encoded><![CDATA[<p>For reasons best known to themselves, certain sections of the entertainment industries seem to believe that <del datetime="2012-05-01T08:34:33+00:00">bolting the stable door</del> <a href="http://www.guardian.co.uk/technology/2012/apr/30/british-isps-block-pirate-bay">shutting down The Pirate Bay will stop all piracy</a>.</p>

<p>It's as though they think that people won't be able to use a proxy, circumvent the <a href="http://en.wikipedia.org/wiki/Cleanfeed_%28content_blocking_system%29">Cleanfeed</a> block, or simply use a search engine to find another torrent site.</p>

<h2 id="build-your-own-pirate-bay"><a href="https://shkspr.mobi/blog/2012/05/yql-and-the-pirate-bay/#build-your-own-pirate-bay">Build Your Own Pirate Bay?</a></h2>

<p>Proxying is a very simple concept.</p>

<ul>
    <li>Alice is forbidden from speaking to Bob.</li>
    <li>Alice can speak to Eve.</li>
    <li>Eve can speak to Bob.</li>
    <li>Alice, therefore, can use Eve to communicate with Bob.</li>
</ul>

<p>So, a user who wishes to access The Pirate Bay would have to do something quite complex to use a proxy?  No, this is all there is to it:</p>

<pre>SELECT * FROM html
WHERE url="https://thepiratebay.se/search/ubuntu/0/7/0"
AND xpath='//tr'</pre>

<p>This uses YQL and xpath to extract all the information from a Pirate Bay search (in this case, for Ubuntu - which is legally distributed through Bit Torrent).</p>

<p>Simply, this asks Yahoo (an American site) to contact The Pirate Bay (a Swedish site) to deliver information to a user in Britain.</p>

<p>You can play with the results yourself <a href="https://web.archive.org/web/20120504160649/http://developer.yahoo.com/yql/console/">in the Yahoo Console</a>.</p>

<p>This returns a JSON string which can then be easily parsed (e.g. using jQuery). Simple.</p>

<h2 id="aha-but-what-about-downloading-a-torrent"><a href="https://shkspr.mobi/blog/2012/05/yql-and-the-pirate-bay/#aha-but-what-about-downloading-a-torrent">Aha! But What About Downloading A Torrent?</a></h2>

<p>In the olden days (well, last year) there was a fly in the ointment.  You had to download a .torrent file from the website. That meant that you needed a way to connect to, in this case, The Pirate Bay or find a proxy which was willing to transfer files.</p>

<p>Nowadays, people use the <a href="http://en.wikipedia.org/wiki/Magnet_URI_scheme">magnet protocol</a>.  Here's what a magnet link looks like:</p>

<pre>magnet:?xt=urn:btih:fa692da0488aee23e5eb605a87be934ad7cec106</pre>

<p>Short enough to fit into a text message and, handily, can be embedded in an HTML document with no need to download an extra file.  Paste those 60 characters into your torrent client, and it will connect to the cloud and start downloading the file you requested.</p>

<p>So, a single web request to Yahoo and a line of JavaScript code is all it takes to circumvent this blockade.</p>

<h2 id="next-move"><a href="https://shkspr.mobi/blog/2012/05/yql-and-the-pirate-bay/#next-move">Next Move</a></h2>

<p>So, do the UK courts need to order ISPs to block Yahoo as well?  Or play whack-a-mole with all the new torrent sites springing up?  Let's not forget, in 2004 the huge Bit Torrent search engine <a href="http://en.wikipedia.org/wiki/Suprnova">Suprnova</a> was sued out of existence. Just like with the Hyrda, a decapitation lead to multiple sites springing up.</p>

<p>Piracy is a problem of convenience.  A pirated copy is</p>

<ul>
    <li>Faster to download.</li>
    <li>Quicker to watch (no unskipable trailers).</li>
    <li>More convenient to transfer to different devices.</li>
    <li>Global availability (no artificial regional restrictions).</li>
    <li>Immense back-catalogue (Star Wars, for example).</li>
    <li>Cheaper.</li>
</ul>

<p>The only downsides are that they are often of dubious legality, and occasionally of poor quality.</p>

<p>The entertainment industries have to compete on <strong>all these points</strong>.  I'll admit, that they will almost certainly not be able to compete with "free" - although monthly unlimited subscriptions come close.</p>

<p>The rest are problems of their own making.  I described how <a href="https://shkspr.mobi/blog/2009/05/the-10th-aniversary-of-the-death-of-the-modern-film-industry/">I downloaded The Phantom Menace back in 1999</a>.  13 long years later and the movie industry still isn't even close to where it needs to be.</p>

<p>Amazon have done pretty well from selling raw MP3s - a simple web interface, pay a small bit of money, instant high-quality download which is DRM free.  Where's the equivalent for films? Or for TV? Or radio?</p>

<p>The pernicious restrictions around geography also must end.  I want to watch Veep just as much as the Americans do. Why do I have to wait even an hour, let alone a week?</p>

<p>Finally, Star Wars <em>still</em> isn't available to (legally) download.  If I have a hankering for Jar Jar Binks at 3AM, I have to order a DVD and wait while it is physically transported from a warehouse. That's such a 19th Century way of thinking that it hurts my brain.</p>

<p>Get all that right and maybe - just maybe - the "piracy problem" will solve itself.</p>

<p>Of course, alternatively, it may be too late.  For 13 years people have been used to downloading without paying.  That's a long period of learned behaviour.  How content providers can convince people to change the habit of a lifetime is beyond my knowledge.</p>
<img src="https://shkspr.mobi/blog/wp-content/themes/edent-wordpress-theme/info/okgo.php?ID=5649&HTTP_REFERER=RSS" alt="" width="1" height="1" loading="eager">]]></content:encoded>
					
					<wfw:commentRss>https://shkspr.mobi/blog/2012/05/yql-and-the-pirate-bay/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
	</channel>
</rss>
