So, you want to write your dissertation or thesis in Markdown. But how do you manage all your citations?

1. Install Zotero
2. Install the Better BibTex plugin
3. Restart Zotero. The BBT plugin will launch a configuration screen - use it to set your preferences
4. Install VS Code (or VS Codium)
5. Install the VS Code Zotero plugin

Now, when you want to enter a citation in Markdown, the combined plugins will generate something like this:

## Abstract

Space is really big @adamsHitchhiker1979

That @adamsHitchhiker1979 is the citation.

Formatting citations

You want to export your Markdown to something like ePub or PDF with nicely formatted citations like (Adams, 1979)

Use Pandoc!

1. Install Pandoc
2. In Zotero, use File → Export Library. Export in Better BibTex format as test.bib.

Add the following configuration section to the top of your Markdown file:

---
bibliography: test.bib
...

Run the following command:

pandoc --citeproc test.md -o test.epub

Hey presto! You now have a nicely formatted ePub with all the citations in your preferred format, and the bibliography at the end.

Format the citations better

The citations don't look the way I want them to!

1. Download your preferred Citation Style from Zotero, save it as whatever.csl
2. Add the following into your Markdown file's configuration section csl: whatever.csl
3. Important Make sure that your citations are wrapped in [ and ] otherwise Pandoc will mess things up. See the Pandoc Citations Extension for more details.

Your Markdown should look something like this:

---
bibliography: test.bib
csl: whatever.csl
link-citations: true
...

# Intro

Computers are complicated [@verletComputerExperimentsClassical1967].

Then run pandoc --citeproc test.md -o test.epub - you can change the file type to html, pdf, or anything else your system supports.

Make it look even nicer

This is the boilerplate that I use.

---
bibliography: test.bib
csl: whatever.csl
link-citations: true
reference-section-title: References
title: My brilliant title
...

# Intro

Computers are complicated [@verletComputerExperimentsClassical1967].

## Something

More info [@smithEvolutionInnovationCapability2017]

<div id="refs">
## References
</div>

## The End
Goodbye!

That gives you a title, ensures that all in-text citations link down to the references, and puts the references somewhere other than the end. Here's the HTML it produces using pandoc --citeproc test.md -o bib.html :

<h1 id="intro">Intro</h1>
<p>Computers are complicated <span class="citation" data-cites="verletComputerExperimentsClassical1967">(<a
         href="https://shkspr.mobi/blog/2022/11/zotero-citations-in-markdown-publishing-to-epub-or-pdf/#ref-verletComputerExperimentsClassical1967" role="doc-biblioref">Verlet, 1967</a>)</span>.</p>
<h2 id="something">Something</h2>
<p>More info <span class="citation" data-cites="smithEvolutionInnovationCapability2017">(<a
         href="https://shkspr.mobi/blog/2022/11/zotero-citations-in-markdown-publishing-to-epub-or-pdf/#ref-smithEvolutionInnovationCapability2017" role="doc-biblioref">Smith <em>et al.</em>, 2017</a>)</span>
</p>
<div id="refs" class="references csl-bib-body" role="doc-bibliography">
   <h2 id="references">References</h2>
   <div id="ref-smithEvolutionInnovationCapability2017" class="csl-entry" role="doc-biblioentry">
      Smith, C. <em>et al.</em> (2017) <span>‘The <span>Evolution</span> of an
         <span>Innovation Capability</span>’</span>, <em>Research-Technology
         Management</em>, 60(2), pp. 26–35. Available at: <a
         href="https://doi.org/10.1080/08956308.2017.1276388">https://doi.org/10.1080/08956308.2017.1276388</a>.
   </div>
   <div id="ref-verletComputerExperimentsClassical1967" class="csl-entry" role="doc-biblioentry">
      Verlet, L. (1967) <span>‘Computer "<span>Experiments</span>" on
         <span>Classical Fluids</span>. <span>I</span>. <span>Thermodynamical
            Properties</span> of <span>Lennard-Jones Molecules</span>’</span>,
      <em>Physical Review</em>, 159(1), pp. 98–103. Available at: <a
         href="https://doi.org/10.1103/PhysRev.159.98">https://doi.org/10.1103/PhysRev.159.98</a>.
   </div>
</div>
<h2 id="the-end">The End</h2>
<p>Goodbye!</p>

If that was useful to you - or if you think you know a better way - please drop a comment in the box.

Recently, someone let me know that they'd found one of GitHub repos in a paper they'd read. It hadn't even occurred to me to search for those!

So, shove your GitHub URl into Google Scholar - https://scholar.google.com/scholar?q=github.com/edent - and you'll see if any repos have made it into academia.

Have you spotted any of your code in interesting places? Let me know in the comments ☺

Leigh Dodds

@ldodds

Discovered that someone built a tool for making SPARQL more usable by building on my sparql-doc tool:

Paper: ceur-ws.org/Vol-2721/paper…

Terence Eden is on Mastodon

@edent

Well worth searching your GitHub URL with Google Scholar.
E.g. scholar.google.com/scholar?q=gith…
Great way to find out if your code has made it into any academic publications. pic.x.com/fpysrjhws6

---

---

Simon Willison

@simonw

Thanks to this tip I just found out Datasette gets a mention in this academic paper! "What Else Is New? Open Data Users Need to Know What’s Changed" computer.org/csdl/magazine/…

---

🕊

@sephr

Replying to @edent@edent Woah I got 8 pages of results. Completely unexpected!

---

Daniel Schildt

@autiomaa

Google Scholar search results scholar.google.com/scholar?hl=en&… found one academic reference to my list of companies that use GitHub. github.com/d2s/companies/…

Documentation can be useful in research & learning.

---

Pratyush Mittal

@faltoo

Wow! Screener.in has been cited in at least 21 papers: scholar.google.com/scholar?start=…

---

As part of a data analytics class, I'm learning about "database sharding". That is, the process of splitting data between multiple machines. But where does the term come from?

Wikipedia - the source of all truth - says:

In a database context, most recognize the term "shard" is most likely derived from either one of two sources: Computer Corporation of America's "A System for Highly Available Replicated Data"

It lists the reference as "Sarin, DeWitt & Rosenburg, Overview of SHARD: A System for Highly Available Replicated Data, Technical Report CCA-88-01, Computer Corporation of America, May 1988"

It is a heavily cited paper. But it doesn't seem to exist!

I've contacted the authors of this, and other papers, but they've not been able to supply me with a copy of the paper.

As far as I can tell, it was originally an internal company report to the Computer Corporation of America. Their new owners didn't respond to a request for archival material.

Perhaps I can't find it because the authors' names are misspelled?

This 1989 thesis from MIT spells the name as "Rosenberg" - with an e, not a u. (Thanks to Suzy Hamilton for helping me find that paper.]

But can we trust this source? Probably; it was written by one Ronni Lynne Rosenberg - and I assume she can spell her own name correctly! I've updated the Wikipedia citation.

But now I'm stumped. Everyone refers to this ur-paper, but I can't find it anywhere. I've checked all the sources I have access to. And even some of those despicable sites which share academic PDFs for free. None of them have it.

This leads me to conclude one of three possibilities:

1. It exists, but I'm too stupid to find it
2. People are citing things which they haven't read
3. I have fundamentally misunderstood how academia works

What do you reckon?

2021-06-30 UPDATE! The inimitable Dr Laura James has found a clue! The British Library holds a copy of "TECHNICAL REPORT- COMPUTER CORPORATION OF AMERICA CCA".

I've requested an interlibrary loan from my university to see if it contains this mythical document.

2021-07-13 Update! Sadly, it's a bust. I got a lovely and detailed email from the British Library. In it, they say:

I have asked my colleague in Boston Spa, in Yorkshire to check the shelves and our holdings for this catalogue record:

Title: TECHNICAL REPORT- COMPUTER CORPORATION OF AMERICA CCA.

Holdings Notes: Document Supply 8715.133000 81/05, 1981-

Shelfmark(s): Document Supply 8715.133000

Sadly it is rather misleading and the dash -, implies that we have CCA-81/05 onwards but in fact we just have that particular technical report and no others. I will ask our data quality team to correct the record for future enquirers.

The quest continues!

For example ([Smith, 1984](https://doi.org/.....))

This is possible, if you're willing to hack around with CSL - the Citation Style Language.

Here's a fragment of CSL that I've adapted from Brenton M. Wiernik's zotero-tools/apa-doi-in-text.csl

  <macro name="url-intext">
    <choose>
      <if variable="DOI">
        <text variable="DOI" prefix="https://doi.org/"/>
      </if>
      <else-if variable="URL">
        <text variable="URL"/>
      </else-if>
      <else>
        <text value=""/>
      </else>
    </choose>
  </macro>
  <citation et-al-min="4" et-al-use-first="1" disambiguate-add-year-suffix="true" disambiguate-add-names="true" disambiguate-add-givenname="true" collapse="year">
    <sort>
      <key macro="year-date"/>
    </sort>
    <layout delimiter="; ">
      <group prefix="([" suffix="]" delimiter=", ">
        <group delimiter=", ">
          <text macro="author-short"/>
          <text macro="year-date"/>
        </group>
        <group>
          <label variable="locator" form="short" suffix=" "/>
          <text variable="locator"/>
        </group>
      </group>
      <text macro="url-intext" prefix="(" suffix="))"/>
    </layout>
  </citation>

I'm a neophyte at CSL, so I'm sure there is a better way to do it. Essentially, this is creating a group, starting with ([. Inside that it places the author(s) and date. It ends the group with ] - then obtains the URl from a macro, surrounds it by ( and )), then prints it.

There are a few bugs. If there is no URl, you might end up with mismatched brackets. But it seems to work OK.

My University insists on Cite Them Right's Harvard style - so that's what I'm using.

I've released a Zotero CSL file on GitLab if you want to use it. Feedback is very welcome - because I have no idea what I'm doing!

Schema.org is a Microdata standard which allows machines to read your HTML and create semantic relations between documents.

Here's a minimum viable citation:

<blockquote itemprop="citation" itemscope itemtype="http://schema.org/ScholarlyArticle">
    <span itemprop="author" itemscope itemtype="http://schema.org/Person">
        <span itemprop="name">
            Terence Eden
        </span>
    </span> 
    <cite itemprop="headline">
        Proof that P ≠ NP
    </cite>
    (<span itemprop="datePublished">2025</span>)
    <a itemprop="url" href="https://ex.doi.org/99.9999/1234">
        https://ex.doi.org/99.9999/1234
    </a>
    <span itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
        <span itemprop="name">
            Journal of Impossible Research
        </span>
    </span>
</blockquote>

That says: This citation is a scholarly article which has a headline, date, and URl. It has an author who is a person with a name. It has a publisher which is an organisation with a name.

Here's a more full example, including ORCID, page numbers, etc:

<blockquote itemscope itemtype="http://schema.org/ScholarlyArticle">
    <span itemprop="citation">
        <span itemprop="author" itemscope itemtype="http://schema.org/Person">
            <link itemprop="url" href="http://orcid.org/0000-0003-4542-8599"/>
            <span itemprop="name">
                <span itemprop="familyName">Losavio</span>, <span itemprop="givenName">Michael M.</span>
            </span>
        </span>,
        <cite itemprop="headline">The Internet of Things and the Smart City: Legal challenges with digital forensics, privacy, and security</cite>
        (<time itemprop="datePublished" datetime="2018-04-27T03:13:15Z">2018</time>)
        Page: <span itemprop="pagination">e23</span>.
        <span itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
            <span itemprop="name">Wiley</span>
        </span>
        <span itemprop="publication">Security and Privacy</span>
        <a itemprop="url" href="https://doi.org/10.1002/spy2.23">https://doi.org/10.1002/spy2.23</a>
    </span>
</blockquote>

APIs

Sadly, CrossRef doesn't do Schema.org, so we have to use their API to get some JSON and convert it to JSON+LD.

Here's how to install their official Python library:

pip3 install --user crossref-commons

The Code

import crossref_commons.retrieval
import os
import re

# Set a friendly header
os.environ['CR_API_MAILTO'] = 'yourEmail@example.com'

reference = input("Enter a DOI: ")

# Check DOI is valid
# https://www.crossref.org/blog/dois-and-matching-regular-expressions/
regex = r"^10.\d{4,9}/[-._;()/:A-Z0-9]+$"
matches = re.search(regex, reference, re.IGNORECASE)

if matches:
    # Call the API
    data = crossref_commons.retrieval.get_publication_as_json(reference)

    # Start the citation
    citation_html = '<span itemscope itemtype="http://schema.org/ScholarlyArticle"><span itemprop="citation">'

    # Get all the authors
    authors = []
    for a in data["author"]:
        author_html = '<span itemprop="author" itemscope itemtype="http://schema.org/Person">'

        if "ORCID" in a:
            author_html += '<link itemprop="url" href="' + a["ORCID"] + '"/>'

        author_html += '<span itemprop="name"><span itemprop="familyName">' + a["family"] +'</span>, <span itemprop="givenName">' + a["given"] + '</span></span></span>'

        authors.extend({author_html})

    # Add the authors to the citation
    for author in authors:
        citation_html += author
        citation_html += " & "
    # Remove the last &
    citation_html = citation_html.rstrip(' &')

    # Title and any language information
    if "title" in data:
        headline  = data["title"][0]

        if "language" in data:
            lang = data["language"]
            citation_html += ' <q><cite itemprop="headline" lang="'+lang+'"><span itemprop="inLanguage" content="'+lang+'">'+headline+'</span></cite></q> '
        else:
            citation_html += ' <q><cite itemprop="headline">'+headline+'</cite></q> '

    # Add the date
    if "published-print" in data:
        year = data["published-print"]["date-parts"][0][0]
        citation_html += '(<time itemprop="datePublished" datetime="'+str(year)+'">' + str(year) + '</time>) '
    elif "issued" in data:
        year = data["issued"]["date-parts"][0][0]
        citation_html += '(<time itemprop="datePublished" datetime="'+str(year)+'">' + str(year) + '</time>) '
    elif "created" in data:
        datetime = data["created"]["date-time"]
        year     = data["created"]["date-parts"][0][0]
        citation_html += '(<time itemprop="datePublished" datetime="'+datetime+'">' + str(year) + '</time>) '
    elif "deposited" in data:
        datetime = data["deposited"]["date-time"]
        year     = data["deposited"]["date-parts"][0][0]
        citation_html += '(<time itemprop="datePublished" datetime="'+datetime+'">' + str(year) + '</time>) '

    # Page number information
    if "page" in data:
        citation_html += ' page: <span itemprop="pagination">' + data["page"] + '</span>. '

    # Publisher
    if "publisher" in data:
        citation_html += '<span itemprop="publisher" itemscope itemtype="http://schema.org/Organization"><span itemprop="name">'+data["publisher"]+'</span></span>. '

    # Publication
    if "container-title" in data:
        citation_html += '<span itemprop="publication">'+data["container-title"][0]+'</span>. '

    # DOI link
    if "DOI" in data:
        doi = data["DOI"]
        doi_url = "https://doi.org/" + doi
        citation_html += '<a itemprop="url" href="'+doi_url+'">'+doi_url+'</a>'

    # End the citation
    citation_html += '</span></span>'

    print(citation_html)

Opinions about citations

There are too many citation styles. And most of them suck. I used to hate seeing "Smith (1991)" as the only reference. Theoretically, a DOI is the only citation you need - but I discovered that some were missing from major resolvers. And, it is probably helpful to have some human readable information to aid discoverability.

I've tried to keep to author name(s), title, year, publisher, publication, page, DOI. That's more-or-less MLA. But, because of the microdata, a machine can understand the citation and you can convert to your preferred style.

Or, if you wish, you can adapt this code to pump out a different citation style.

Source code on GitLab.

It emerged this morning^* that the Guardian newspaper has realised that the way it writes is unsuitable for the web.

^*Source: Guardian Newspaper, 18/11/2011, page http://www.guardian.co.uk/media/mind-your-language/2011/feb/18/mind-your-language-day-date-time

By using non-specific language, I have introduced a degree of ambiguity which makes it hard for reader - both in the present day and the future - to understand the ideas I am trying to convey.

For example - the above text doesn't state which of the many Guardian newspapers is under discussion. The words "this morning" are highly subjective depending on timezone - and the date at which the article is read. Finally, the source link is separate from the text making it hard for an automatic process to understand what words relate to which website.

The above could be rewritten as...

The Guardian has realised that the way it writes is unsuitable for the web.

We can, of course, extend our markup to make it easier for both humans and robots to understand what it is we have written. Using the ideas behind the Semantic Web we can embed information about the relationships, dates, times, locations, etc, in a meaningful way.

Page Numbers

This leads me into the bizarre decision of Amazon to introduce page numbering for its Kindle eBook reader.

I hope this post will convince you that this move is philosophically wrong and - potentially - dangerous.

Why Do Paper Books Have Page Numbers?

Paper books (not "real books" as some people term them) are physically printed on individual sheets of paper. Page numbering is a limitation with non-digital media to allow semi-rapid and predictable access to pseudo-exact locations.

For example, a paper book may say

"For further information on the Infinite Improbability Drive, see Adams 1979, page 42"

It doesn't tell us where on a page unless it also introduces a paragraph number. As a paragraphs aren't usually numbered, this makes locating the exact reference hard for both machines and people.

Furthermore, different versions of the same text may have substantially different formatting - larger print, more images, smaller paper, etc - so every time a book is republished all page references have to be recalculated.

Page numbers are a hack. They are an illusion designed to give humans the impression that an imprecise reference is specific.

Why Don't eBooks Have Page Numbers?

Because they don't need them! An eBook is a digital document which may be marked up in a many ways. From the humble text document as an unbroken stream of characters - to a meta-data rich ePub full of syntactic markup.

An eBook never knows the size of the screen upon which it will be displayed. Even if it did, it wouldn't know how large its text would render, nor what font would be used, nor if the screen orientation were to change.

The idea of a page number for a digital document is an absurdity. Page numbers are only needed for an item that may be printed out in the physical world. Even then, they still suffer from the above problems.

Why Use Page Numbers In Quotes?

The idea is simple. To use the Harvard Style as an example - other styles are equally useless - we see this.

Lawrence (1966, p.124) states "we should expect..."

So, assuming we can find the exact copy of "Lawrence's" paper, printed in exactly the same font and size, on exactly the same shaped paper, and typeset identically, we can simply flip through to page 124 and scan through the page until we find the quote we're looking for.

Rubbish! Utter, unmitigated, total and utter rubbish. A long, boring and error-prone process which does little to help us find the information for which we are looking.

The Role Of Computers

We have computers for one simple reason - they perform boring jobs with speed and accuracy.

If I want to find where Alan Turing declared "It is possible to invent a single machine which can be used to compute any computable sequence" within "On Computable Numbers, with an Application to the Entscheidungsproblem" I can do one of two things.

1. I can click on a link which takes me straight to the sentence or chapter
2. I can hit "search" or "find" in my document viewer

Having documents which are correctly marked-up is the key to successful linking of data. Manually marking up a document is tiresome and problematic for humans. Where we fail, computers make up for our shortcomings with their incredible speed at searching through documents.

My Kindle is a fairly slow as far as modern computers are concerned. Yet it has no difficulty searching through millions of words in hundreds of documents to find the single sentence I'm looking for. And it all takes less time than it would for me to flick through some pages.

A Changing World

Page numbering as a system of referencing relies on such an unlikely series of events as to be worse than useless. If both reader and writer do not have identical copies of the quoted work, there is a real risk of confusion and misunderstanding.

Page numbers are an ugly hack which fail to achieve the precision they so desperately crave. Let's remove the hack and replace it with something which actually works.

People who quote a page number from an electronic book don't understand how the modern world works. Documents have evolved - so must our citation styles.