There are some discussions about what tools repository owners should have to help them. Disabling AI on repos is popular - but ignored by Microsoft. Being able to delete PRs is helpful - but still makes work for maintainers. Adding more AI to review new PRs and issues is undoubtedly popular with those who like seeing number-go-up - but of dubious use for everyone else.

I'd like to discuss something else - reputation scores.

During Hacktoberfest, developers are encouraged to contribute to repositories in order to win a t-shirt. Naturally, this leads to some very low-effort contributions. If a contribution is crap, maintainers can apply a "Spam" label to it.

Any user with two or more spammy PR/MRs will be disqualified.

This works surprisingly well as a disincentive! Since that option was added, I had far fewer low-effort contributions. When I did apply the spam label, I got a few people asking how they could improve their contribution so the label could be removed.

However, there is no easy way to see how many times a user has been labelled as a spammer. Looking at a user account, it isn't immediately obvious how trustworthy a user is. I can't see how many PRs they've sent, how many have been merged or closed as useless, nor how many bug reports were helpful or closed as irrelevant.

There are some badges, but I don't think they go far enough.

I think it could be useful if maintainers were able to set "contributor controls" on their repositories. An entirely optional way to tone down the amount of unhelpful contributions.

Here are some example restrictions (and some reasons why they may not help):

• Age of account. Only accounts older than X days, weeks, or years can contribute.
  • This disenfranchises new users who may have specifically signed up to report a bug or fix an issue.
• Restrict PRs to people who have been assigned to an issue.
  • May be a disincentive to those wishing to contribute simple fixes.
• Social labelling. Have other maintainers marked this user as a spammer?
  • Could be abused or used for bullying.
• Synthetic Reputation Score. Restrict contributions to people with a "score" above a certain level.
  • How easy will it be to boost your score? What if you get accidentally penalised?
• Escrow. Want to open a PR / Issue, put a quid in the jar. You'll forfeit it if you're out of line.
  • Not great for people with limited funds, or who face an unfavourable exchange rate. Rich arseholes won't care.

Obviously, all of these are gameable to some extent. It also incentivises the theft or sale of "high reputation" accounts. Malicious admins could threaten to sanction a legitimate account.

But apps like Telegram show me when someone has changed their name or photo (a good sign of a scammer). AirBnB & Uber attempt to provide a rating for users. My telephone warns me if an unknown caller has been marked as spam.

I don't know which controls, if any, GitHub will settle on. There is a risk that systems like this could prohibit certain people from contributing - but the alternative is maintainers drowning in a sea of slop.

I think all code-forges should adopt optional controls like this.

Sometimes when I raise an issue on GitHub, or write a comment, other users leave me Emoji reactions. Perhaps a 👍 or 🎉 if they like my contribution, but occasionally a 👎 or 😕 if they're foolish enough to think I'm wrong.

The problem is, GitHub doesn't tell me that someone has 🚀'd my wisdom. If GitHub was as good as Facebook, it would present a little 🔔 to let me know exactly how many ❤️s I have received. Instead I have to manually check every issue I've raised to see if the hive-mind judges me worthy.

You might be thinking that there's an API for finding the reaction count to a specific piece of content - and you'd be right! The only problem is that it requires you to send it a specific content ID. So pretty bloody useless unless you want to construct a mega-query of everything you've ever written.

Enter the terrifying world of GraphQL - where men fear to tread and AIs are driven mad. It is possible to squeeze the API until the pips squeak. Here's a GraphQL query, run using the gh CLI, which grabs any issue with over zero reactions, displays how many reactions it received, and who gave what sort of reaction.

gh api graphql -f query='
query {
  search(query: "author:@me reactions:>0", type: ISSUE, first: 10) {
    nodes {
      ... on Issue {
        url
        reactions(last: 50) {
          totalCount
          nodes {
            content
            user {
              login
            }
          }
        }
      }
    }
  }
}'

As you might be able to decipher, that looks for the 10 most recent issues. If you are prolific, you may want to increase that number - although it will increase the time it takes for the query to run. If you have the temerity to dare to retrieve more than 100, you'll be slapped with the dreaded EXCESSIVE_PAGINATION error.

Similarly, it only gets the most recent 50 reactions. The count will be be the total number of reactions, no matter how low you set the number.

In return, you'll get a hideous mass of JavaScript which looks like it has been vomited up by a disgruntled Cacodemon:

{
  "data": {
    "search": {
      "nodes": [
       {
          "url": "https://github.com/validator/validator/issues/1814",
          "reactions": {
            "totalCount": 9,
            "nodes": [
              {
                "content": "THUMBS_UP",
                "user": {
                  "login": "markohoza"
                }
              },
              {
                "content": "EYES",
                "user": {
                  "login": "adamwolf"
                }
              },

There is no way to get anything older. If someone liked a comment you made in 2019, you will never know!

If you hate your eyes enough to read through the search type documentation, you'll notice there is no way to search Pull Requests. This is, of course, a rotten lie. If you read different documentation you'll see that PRs are classed as a type of issue. Why? Because your sanity is not worth the cost of updating things.

Anyway, it makes our life slightly easier. We can search both Issues and PRs in one easy to chew lump of GraphQL:

gh api graphql -f query='
query {
  search(query: "author:@me reactions:>0", type: ISSUE, first: 100) {
    nodes {
      ... on Issue {
        url
        reactions(last: 100) {
          totalCount
          nodes {
            content
            user {
              login
            }
          }
        }
      }
      ... on PullRequest {
        url
        reactions(last: 100) {
          totalCount
          nodes {
            content
            user {
              login
            }
          }
        }
      }
    }
  }
}'

Again, beware gazing into the JSON lest the JSON gazes into you!

{
  "data": {
    "search": {
      "nodes": [
        {
          "url": "https://github.com/WICG/webmonetization/pull/634",
          "reactions": {
            "totalCount": 2,
            "nodes": [
              {
                "content": "CONFUSED",
                "user": {
                  "login": "tomayac"
                }
              },
              {
                "content": "HEART",
                "user": {
                  "login": "tomayac"
                }
              }
            ]
          }
        },

OK, so it should be pretty damned simple to get the number of reactions to any comments, right? RIGHT?!?!

No. Because consistency is a dirty word and GraphQL was designed in the bowels of hell as a way to keep API developers from ever obtaining a state of grace.

There's no way I could find to use reactions:>0 with a comment search query. This will get you lots of useless unreacted results. I guess you can filter them with jq or just scratch your monitor with razor blades so you don't have to see their empty laughing maws.

gh api graphql -f query='
query {
  viewer {
    issueComments(last: 10) {
      nodes {
        url
        reactions(last: 10) {
          totalCount
          nodes {
            content
            user {
              login
            }
          }
        }
      }
    }
  }
}'

And, again, JSON nested like wheels within wheels and fires within fires:

{
  "data": {
    "viewer": {
      "issueComments": {
        "nodes": [
          {
            "url": "https://github.com/home-assistant/supervisor/issues/6474#issuecomment-3740347148",
            "reactions": {
              "totalCount": 0,
              "nodes": []
            }
          },
          {
            "url": "https://github.com/edent/3D-UK-Money/issues/1#issuecomment-3757022146",
            "reactions": {
              "totalCount": 1,
              "nodes": [
                {
                  "content": "THUMBS_UP",
                  "user": {
                    "login": "MickeyF2010"
                  }
                }
              ]
            }
          },

What Have We Learned Today?

The Necronomicon was probably written in GraphQL. Any form of Daemon summoning must use nested queries and frightening syntax.

Trying to track reactions to your content will drive you mad. There's a reason this knowledge is forbidden.

Disclaimer

This post was not sponsored by GitHub. Although I did drink rather too many of their free beers at FOSDEM. Consider this post payback for that self-induced hangover.

Mostly.

A few weeks ago, I received this email from GitHub.

On the surface, this is a sensible email. They want all their members to only have strong 2FA and I still had SMS configured as a fallback method. Except, of course, I should not be a member. I should have been kicked out when I handed back my laptop and lanyard. There was still a bit of pandemic pandemonium about - but surely in the last few years someone should have audited the organisation's membership?

The JML process is critical to cybersecurity. There's no point having fancy controls if you don't revoke the permissions of people who are no longer entitled to access. On a fully integrated system this is (usually) easy - untick a box on Active Directory or whatever and *poof* the user is banned.

But with external systems the problem is harder. You now need to keep track of external usernames, synchronise them with internal names, periodically check them for updates, integrate with an API, and - in some cases - take manual action. It's clear that this particular bit of the NHS had slipped up. Looking through the private list of collaborators, there were many old accounts.

I was able to see all private collaborators:

I could see all private repositories:

I even had access to create new repositories - including special ones:

To be abundantly clear, there was no medical data on GitHub. There was no patient data available for me to view. Absolutely nothing medically sensitive was stored there. This isn't a GDPR or medical privacy issue. If I had made any changes to the code stored on there, it would never have made it to production. There were no API keys or sensitive data or passwords for me to exfiltrate. The NHS BSA is a business unit - not a medical unit.

Nevertheless, it is important that all parts of a large organisation are able to quickly and competently remove users once they have left.

Timeline

• 2025-10-17
  • Received GitHub email.
  • Visited https://www.nhs.uk/.well-known/security.txt to get details of how to raise security issues.
  • Raised the issue on HackerOne
• 2025-10-21
  • After triage, the issue was assigned directly to the BSA.
• 2025-10-31
  • I was removed from the organisation.
  • 
  • Requested permission to publish this post. No objection received.
• 2025-12-02
  • Published

Doing it manually

You can search GitHub for a URl, and sort the results with the newest first, like this:

https://github.com/search?q=%22shkspr.mobi%22&type=issues&s=created&o=desc

Using the API

GitHub has a fairly straightforward API - although it uses slightly different parameters.

https://api.github.com/search/issues?q=shkspr.mobi&sort=created&order=desc

That will return a bunch of items. Here's the 29th. I've truncated it down to only what is necessary for our purposes:

{
    "html_url": "https://github.com/swicg/activitypub-webfinger/issues/29",
    "id": 3286159033,
    "number": 29,
    "title": "Tracking support for non-ascii characters",
    "user": {
        "login": "evanp",
    },
    "created_at": "2025-08-02T17:52:46Z",
    "updated_at": "2025-08-02T18:50:27Z",
    "body": "One of the benefits of using Webfinger is that it's […]"
}

Action

I'm not very good at creating actions. But this should:

1. Search GitHub for mentions of your URl.
2. Store the results.
3. If there is a new entry - open a new issue describing it.

You will need to set your repository to private in order to not spam other repos. You will also need to go to your repo settings and give the action write permissions. You'll also need a Personal Access Token with sufficient permissions to write to your repo. I bloody hate actions. YAML? Eugh!

name: API Issue Watcher

on:
  schedule:
    - cron: '*/59 * * * *'

permissions:
  issues: write
  contents: write

jobs:
  watch-and-create:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Restore latest seen ID
        id: cache-latest
        uses: actions/cache@v4
        with:
          path: .github/latest_seen.txt
          key: latest-seen-1
          restore-keys: |
            latest-seen-

      - name: Fetch latest item from API
        id: fetch
        run: |
          curl -s 'https://api.github.com/search/issues?q=EXAMPLE.COM&s=created&order=desc' > result.json
          jq -r '.items[0].id' result.json > latest_id.txt
          jq -r '.items[0].title' result.json > latest_title.txt
          jq -r '.items[0].html_url' result.json > latest_url.txt
          jq -r '.items[0].body // ""' result.json > latest_body.txt

      - name: Compare with previous run
        id: check
        run: |
          NEW_ID=$(cat latest_id.txt)
          OLD_ID=$(cat .github/latest_seen.txt 2>/dev/null || echo "")
          echo "NEW_ID=$NEW_ID" >> $GITHUB_OUTPUT
          echo "OLD_ID=$OLD_ID" >> $GITHUB_OUTPUT
          if [ "$NEW_ID" != "$OLD_ID" ]; then
            echo "NEW_ITEM=true" >> $GITHUB_OUTPUT
          else
            echo "NEW_ITEM=false" >> $GITHUB_OUTPUT
          fi

      - name: Open new issue if new item found
        if: steps.check.outputs.NEW_ITEM == 'true'
        uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.MY_PAT }}
          script: |
            const fs = require('fs');
            const title = fs.readFileSync('latest_title.txt', 'utf8').trim();
            const url = fs.readFileSync('latest_url.txt', 'utf8').trim();
            const body = fs.readFileSync('latest_body.txt', 'utf8').trim();
            await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: `[API] ${title}`,
              body: `Found new item: [${title}](${url})\n\n${body}`
            });

      - name: Update latest seen ID
        if: steps.check.outputs.NEW_ITEM == 'true'
        run: |
          mkdir -p .github
          cp latest_id.txt .github/latest_seen.txt

      - name: Save cache
        uses: actions/cache@v4
        with:
          path: .github/latest_seen.txt
          key: latest-seen-1
          restore-keys: |
            latest-seen-

This is probably all kinds of wrong. If you know how to improve it, please let me know!

The results weren't pretty.

In the early part of the 21st Century, there was a fad for "AI". The magical, Artificial Intelligence which provided all the answers. The market had decided that AI was The Next Big Thing and tried to shove it into every product. You can probably see where this is going, right?

I don't particularly mind companies experimenting with AI. It's good to explore a problem and see if it fits a user's needs. But the current crop are just so shit it makes me wonder whether anyone tested them.

GitHub has forced its new Copilot button on to every page. The first thing I asked it was whether it could be turned off.

It pointed me to this page: https://docs.github.com/en/copilot/getting-started-with-github-copilot/disabling-github-copilot

Except - and I hate to be a pedant - that link 404s. There's nothing there. It doesn't exist. It is made up.

This AI, which I am supposed to trust with my code, doesn't even understand itself.

This isn't an AI Mirror Test. This isn't me trying to find out if the large-language model is conscious, aware, or has a soul. I'm not asking complex reasoning, or asking it to make an æsthetic judgement.

This is a basic functionality test.

Is the computer able to accurately provide information about itself?

That's it. That's all I want. My first interaction with Copilot it lied to me about itself. Why would I trust it again?

You can leave feedback for GitHub about this problem. I'm sure a human will answer you.

Lots of companies encourage their staff to blog. It's free PR! It makes them look like they're on the cutting edge of technology! It helps with recruitment!

It can also be a corporate nightmare. What if the developer says something stupid? What if it accidentally reveals something top secret? What if the CEO doesn't like it?

And so, gradually, any free-wheelin' developer blog gradually succumbs to the tender mercies of the comms team. It starts with help editing and clarifying the writing. Then it turns into massaging the tone-of-voice. Before too long you have to wait for a slot on the corporate comms grid. And then you need seven levels of sign-off just to post something anodyne like "5 reasons we switched to Rust".

*sigh*

So, maybe once or twice, I cheated.

Rather than writing a blog post called "An interesting issue configuring GLaDOS with YAML" I opened an issue on our public GitHub repo.

Oh, sure, it wasn't as "bloggy". And I still made sure not to discuss anything secret or sensitive. For some of the more contentious issues, I ran it by a few trusted colleagues first. But, for all intents and purposes, it was a blog post. Nerds got to reply in the comments, people shared it on HN, people quoted it on social media.

I've since left that sort of restrictive environment. I still see my old team occasionally publishing a useful blog post. But I keep my eyes on their GitHub issues to see the real story!

What's the moral here?

Should you bypass your employer's strict social media policy? No, absolutely not. Should you ensure that your GitHub issues are well-formatted, interesting to read, and useful to others? Yes.

Several years ago, I came across Felix Geisendörfer's Pull Request Hack. The premise is simple - if people are making decent Pull Requests to your project then you should give them commit access.

It sounds mad, I know. But it has worked really well in my case. I launched Super Tiny Icons six years ago. It was surprisingly popular and lots of people seemed to enjoy using them - and a few even contributed.

So I gave them Commit access. That means they are able to push directly to my codebase. They can also approve other people's Pull Requests.

Like I say, totally mad and irresponsible.

But people are, by and large, lovely.

Over the years, I've added about 20 random people as collaborators. Some have been more active than others. A few have refused to take on the responsibility. But several are still there - saving me dozens of hours, and helping even more people contribute.

What's the worst that could happen?

OK. Lots of things. The Pull Request Hack probably isn't suitable if you're running big projects. And it is almost certainly a stupid idea if you write code which is actively used by multiple downstream projects. And if your stuff has even the slightest chance of compromising security then you're better off sticking to trusted members.

But if you're just farting around with some scrappy code and you want a quick way to build an ad-hoc team? I don't want to go full "Cathedral and Bazaar", but it is pretty fucking magic.

And it gives me a modicum of hope that some of my work might continue after I'm gone. Sorry, that's a bit morbid.

So a huge thank you to everyone who has contributed.

Recently, someone let me know that they'd found one of GitHub repos in a paper they'd read. It hadn't even occurred to me to search for those!

So, shove your GitHub URl into Google Scholar - https://scholar.google.com/scholar?q=github.com/edent - and you'll see if any repos have made it into academia.

Have you spotted any of your code in interesting places? Let me know in the comments ☺

Leigh Dodds

@ldodds

Discovered that someone built a tool for making SPARQL more usable by building on my sparql-doc tool:

Paper: ceur-ws.org/Vol-2721/paper…

Terence Eden is on Mastodon

@edent

Well worth searching your GitHub URL with Google Scholar.
E.g. scholar.google.com/scholar?q=gith…
Great way to find out if your code has made it into any academic publications. pic.x.com/fpysrjhws6

---

---

Simon Willison

@simonw

Thanks to this tip I just found out Datasette gets a mention in this academic paper! "What Else Is New? Open Data Users Need to Know What’s Changed" computer.org/csdl/magazine/…

---

🕊

@sephr

Replying to @edent@edent Woah I got 8 pages of results. Completely unexpected!

---

Daniel Schildt

@autiomaa

Google Scholar search results scholar.google.com/scholar?hl=en&… found one academic reference to my list of companies that use GitHub. github.com/d2s/companies/…

Documentation can be useful in research & learning.

---

Pratyush Mittal

@faltoo

Wow! Screener.in has been cited in at least 21 papers: scholar.google.com/scholar?start=…

---

Over the last 20 years, open source software has undergone a significant shift—from providing an optimistic model for public collaboration to undergoing constant maintenance by the often unseen solo operators who write and publish the code that millions of users rely on every day. In Working in Public, Nadia Eghbal takes an inside look at modern open source software development, its evolution over the last two decades, and its ramifications for an internet reorienting itself around individual creators. By delineating the structure of open source projects, she explores, for the first time, the maintenance costs of production that software incurs for its developers. Drawing on hundreds of developer interviews and analyses of platforms like Twitter, Facebook, Instagram, Twitch, and YouTube, Eghbal argues that examining who produces things on the internet, and not just what they produce, helps us understand the value of online content today.

This is an excellent book. If you're an experienced open source developer, it'll bring clarity to a bunch of concepts you're familiar with but couldn't name. If you're managing teams who work with - or produce - open source, it'll give you a greater understanding of the scene and its limitations.

Given the current furore with Log4J developers basically being volunteers, the book presents several ways that critical open source can be funded.

But, for me, the best thing about it is the way that it reflects our culture at this moment in time. Sure, it covers the history of F/OSS and it looks to the future - but it is mostly about what working on Open Source is like right now.

If you're interested in software as a cultural movement - and its intersection with commerce and government - this is the book for you.

Edent's 7th Law (My blog; my rules!) states "the best way to get an open source project to fix an issue is to send a slightly wrong Pull Request."

Let me explain...

Two years ago, I noticed an annoying bug in the markdown parser of WordPress's JetPack plugin. I did what every good open sorcerer is supposed to do - I wrote out a comprehensive bug report, including screenshots, and posted it in the wrong place.

I didn't know it was the wrong place, but they kindly directed me to the right place.

A year later and no-one had taken a look at the bug. I get it. It's low priority and they have hundreds of bugs coming in. So I added a reproduceable test case.

Another year went by, and I asked for help. I am not an expert in JetPack's baroque structure and didn't fancy spending a few days trying to understand it. With a bit of guidance, I found the correct file and submitted a patch.

I knew the PR wasn't brilliant. It had no tests, only handled a limited subset of the problem, it didn't quite conform to the project style, and was done in a bit of a rush. But, crucially, it worked. I could demonstrate that this problem could be fixed - even if I hadn't fixed it in the right way.

To be clear - this wasn't a malicious PR. I just didn't have the time or energy to dedicate to writing something perfect. I knew from experience that it's easier to fix a bug in the right way if you've seen it fixed the wrong way.

And that's what happened. The talented folks in the project - seeing that it was possible - are now creating a patch which works properly.

I've done this a few times. Spotted a bug, written an almost-working fix, and let people more clever than me worry about the details.

Don't spam open source projects with crappy PRs. Take your time to do something which works. Show that the issue is solvable. Let others with more experience finesse your code into something beautiful.

So I downloaded PuTTY and never looked back. It had a simple GUI. Well… simple enough. It was impossible to type an IP address into a box designed for a port number, it could tell that username was unlikely to contain a non-ASCII symbol, and that made it slightly easier to configure a connection.

The job of the computer is to do the hard work for us. Computers shouldn't recognise that we've made a mistake - they should stop us making mistakes in the first place!

Flash forward to today. I wanted to release an update to my NPM package of SuperTinyIcons - which are hosted on GitHub.

I figured that I'd have to click a link on GitHub, do the OAuth dance, click on a few GUI elements, and hit a big save button.

Nope!

I have to create YAML text files. Manually. Because it is the Year of Our Lord 2020 and computer engineering hasn't evolved since the earliest days of UNIX.

Don't get me wrong - I love the utility and versatility of the command line. I'll happily manually edit a JSON file in nano and use sed to update a bunch of documents. I rely on diff and grep daily. But when it comes to configuring things - I am undone. I don't want to learn whitespace rules just to get something working.

Let's take a simple example. Here's a snippet of YAML from the official guide to releasing NPM packages via GitHub:

jobs:
  build:
    runs-on: ubuntu-latest

What are valid options for "runs-on"? Where would I find them? What are the differences between them? What happens if I spell something wrong?

Luckily, the GUI was invented to save us from ourselves:

--Please choose an option-- Ubuntu Latest (This is the most common option) Grarch 3.5 (Useful if you need flargon support) Plan9 (Choose this if you require Python 1) Windows 3.11 (Only works on older, 16 bit projects)

M A G I C !

This goes for almost every element of the config file. If there are a set list of choices, present them to the user. If there is only one option, hide it. If there are two options, give them a checkbox.

The thing is, GitHub gets part of the way there. If you want to setup a webhook - you get a GUI. Why does one part of GitHub offer a user-friendly way to manage configuration, and another part only provide a user-hostile environment?

To make it worse… NPM IS OWNED BY GITHUB!

I'm not saying you can't have a text file. I'm not trying to take your toys away from you. Have a sudo mode which lets user hand-stitch a config together with butterflies and a binary keyboard. Go nuts! Want to store the config in git so you can see what changes have been made? Brilliant! To be explicitly clear - store the config in a text file. But have a GUI first, and expose the text file to any masochists who want to manually replace all the tabs with spaces.

I just can't be bothered reading 22 thousands words of YAML spec, a further two thousands words of GitHub instructions, and - no doubt - a hundred different StackOverflow questions, just to send data from one computer to another.

I heard someone say that part of Amazon's success was its relentless focus on convenience. How is manually creating fragile text files convenient for me?

We have to stop this fetishization of the text file as the primary method for configuring computers.

Here are the options we considered:

• Get everyone to sign up for a secondary GitHub account to use for this project.
• Redact the history so no-one could see personal information.
• Use the same GitHub credentials for everyone.

There are problems with all the above approaches. Additional accounts are a huge administrative and cognitive burden. Scrubbing the history on publication doesn't facilitate transparency. And sharing login details? Yeuch!

So what could we do?

GitHub has a nifty feature - it allows you to commit on-behalf-of an organisation.

With a simple change to your commit message, you can show that you're working on behalf of an organisation. Handy if you want to separate your commits into personal and professional. Or if you're a member of multiple organisations.

Here's what we wanted - publish as an organisation!

I'd like to sketch out how this might work - then I'd like some criticism from you on why it's bad idea.

User Story

• As an @Organisation on GitHub
• With a team of authorised members
• I want all the public commits from the team to show as coming from the @Organisation
• So that I can preserve the privacy of the authorised members

(This could also work by committing as the team.)

Practicalities

• Add a team to GitHub
• Populate it with members
• Flick on this option:
  • Enable commit privacy. All commits will show as from @Organisation.

Any member of the team can see that this commit came from Alice and that commit came from Bob. But people outside the team would just see both of those commits coming directly from the organisation.

This enables the team to see who made which changes, but keeps it private for everyone else.

Downsides

Does this lull developers into a false sense of security? If the commit message still says "Asked to add this by @alice. Yours sincerely, @bob" then privacy is compromised.

Is this even possible with the git protocol? Teams are a part of GitHub - but I don't think they're part of the specification. Would this screw up the cryptographic signing of commits?

Part of open source is transparency. Do we want to lose that?

GitHub would be able to de-anonymise you. But you're already trusting them with parts of your infrastructure anyway.

Your thoughts?

Is this useful - or am I barking up the wrong tree?

Ryan C. Gordon

@icculus

Google gave me an award for my contributions to Linux gaming and open source, and while I’m honored to be acknowledged, I can’t accept money from a company that is actively engaging in union busting.

---

Ryan C. Gordon

@icculus

Replying to @icculusWhen I agreed to the award, I hadn’t heard about @eiais’s firing, or any of the others. It’s not much money, but I don’t think quantity is relevant.

---

Ryan C. Gordon

@icculus

Replying to @icculusAs such: I’m announcing the 2019 Icculus Microgrant. If you have a cool open source project, reply here about it and the one I like best, I’ll give $250 dollars. Do whatever you want with the money. I’ll try to pick one this week.

If anyone wants to match my donation, DM me.

---

Ryan C. Gordon

@icculus

Replying to @icculus(If I don’t get any good projects, I’ll buy 250 bucks worth of pizza for the next Google organizing meeting or something. But if you’re proud of your project, tiny or not, that should be all you need to tweet it at me. Good luck!)

---

Huge thanks to Ryan, and everyone else who chipped in.

Because I'm part of GitHub Sponsors, the donation will be matched by GitHub.

So, that's US$500 - about £375. Nice!

If you'd like to sponsor us, you can do so on GitHub or via PayPal directly. Or, we have an Amazon wishlist if you'd like to get us a physical present.

What will we spend the grant on?

This is what we said we'd use the money for:

Terence Eden is on Mastodon

@edent

Replying to @icculus@icculus @mjg59 My wife and I run openbenches.org
It's an open source / open data project which crowdsources photos of memorial benches.
$250 would allow us to move away from Google's OCR engine, and on to a faster web host.

---

No one is auditing us, so we could spend it all on a luxury holiday to Bognor Regis. But we won't!

Although... There are a lot of unmapped benches in Bognor...

The web is cheap!

Don't tell anyone who wants to give us money, but it's quite cheap to run a niche site on the web!

We're lucky to be in the free tier of our 3rd party providers:

• OpenCage provides the reverse geocoding, and is free for 2,500 calls per day.
• CloudImage is free for small projects.
• Auth0 does social login - and offers more features free for Open Source projects.
• MapBox renders beautiful OpenStreetMap tiles for free, up to 50,000 per month.
• Google Cloud Vision we're just about within their 1,000 free scans per month limit.

We've occasionally gone over those free limits, so will keep a couple of quid in reserve in case of a surge in our popularity.

Our major cost is hosting. We paid £300 for 3 years of shared hosting with Krystal.uk - use code EDENT at checkout and we both get a fiver.

That hosting deal runs out in August this year, so the bulk of the grant will go to upgrading our package to something a bit bigger. Notably, we're capped at 2GB of RAM which we seem to regularly max out.

Because we have over 130GB of photos, we'll probably stay with Krystal - unless anyone knows of a UK based hosting provider with impeccable green credentials?

In the spirit of the grant, we'll look at moving away from Google's Cloud Vision. Current contenders are

1. Amazon Rekognition
   • Although they're just as ethically dubious
2. Microsoft's Computer Vision
   • Ditto!
3. The Open Source Tesseract OCR
   • Which is sponsored by Google...

Suggestions very welcome!

Unions

Regular readers will know that I'm a proud member of Prospect - the union for ambition. As a professional knowledge-worker, I rely on my union to provide me with legal services and support me in pay negotiations. If you're in the UK and want to be part of a union for professionals - join Prospect today.

If you join Prospect, and mention my name, I'll make a £10 donation to charity.

Here's a quick guide to how it works:

Repo settings

There's now a new option in your repository settings. Tick the box and click the button. Easy!

WTF is YAML?

This is a bit of a weird one. To set up sponsorship, you have to hand-edit a YAML text file. I would have expected this to be a easy to use GUI. Or something to OAuth with payment providers. Oh well, it does the job.

You can add up to four GitHub contributors. Handy if multiple people work on your project. Limit seems a bit low though.

The file is stored in /.github/FUNDING.yml if you want to change it. If you screw up the formatting, there are no error messages and the sponsorship button won't appear.

New buttons!

At the top of your repo, there will be a new ❤️ button - next to the watch, stars, and fork buttons.

Click it, and you'll get a pop-up with the options you've set.

The GitHub sponsorship doesn't show up for me - I'm on a wait-list I think - but you can link out to:

• Patreon - the big name in sponsoring creators.
• Ko-Fi - a UK based company which uses PayPal.
• Open Collective - an open source friendly payment provider
• tidelift - another open source funder.
• Community Bridge - developed by the Linux foundation.
• Custom - I used an Amazon Wishlist - but you can also use Venmo or any other service which lets you craft payment URLs. For example Liberapay, or a PayPal link. You can't set a custom name for the custom URL.

That's a pretty good list to start. Only time will tell if it drives people to pay for open source contributors.

There's no way to add the same sponsorship details to every repo you have - so there's a lot of copy & pasting needed.

Anyway, please sponsor OpenBenches or any of the other Open Source stuff I do!

It also allows users to reply by email. The reply is then automatically posted in the issue thread. Also handy. But a little dangerous.

Lots of people have email signatures which contain personal details. When these people reply to a GitHub notification they may unwillingly share their contact details in public.

GitHub does a reasonable job of hiding this from the casual observer. When it detects the signature block, it hides it behind a button.

What happens if you click on that overflow button?

I've manually redacted the user's details.

Finding Numbers

A quick search for common email signatures like "This email and any attachments are confidential" reveals thousands of issues like this. Many have phone, fax, and postal details of the user.

Searching GitHub issues for "This email and any attachments" & "+44(0)" gives a hundred or so results - mostly from UK individuals replying to emails.

Similar searches for other formats of phone numbers reveal thousands of potential accidental leaks.

What should GitHub do?

GitHub can obviously detect when a reply is an email - and includes an icon on the issues screen to show the user.

The signature detection isn't perfect - in the above example it hasn't recognised it.

GitHub could proactively scan incoming emails and offer to redact personal info from them.

Repository owners can also edit comments to make sure that personal data hasn't accidentally leaked.

This isn't just an issue of personal responsibility. When you receive an email, it is natural to reply. You may not even realise you have a corporately mandated signature. You may not know how to remove it. There's nothing in the email that suggests replying will post your details in public.

Yes, people should take a little more care - but I think GitHub should do a better job of detecting and deleting personal information.

GitHub users have a username (mine is @edent) and have a user ID number (mine is #837136).

If you want to redirect a user ID to a username, you can use the little service I've cobbled together:

https://edent.github.io/github_id/#837136

That will take your browser to my GitHub page, using nothing but my ID.

Why?

• Some login services only give you the GitHub user's ID.
• GitHub users can change their username - but their ID stays the same.

How?

Inspired by Caius Durling's useful GitHub username to ID service.

Mine is a scrap of JavaScript which uses this undocumented endpoint to get user info.

<script type="text/javascript" charset="utf-8">
function get_github_name_for(id) {
    $.getJSON('https://api.github.com/user/' + id + "?callback=?", 
    function(json){
        var github_user_url = json["data"]["html_url"]
        window.location.replace(github_user_url)
        return false
    });
}
$(document).ready(function() {
    if (document.location.hash) {
        var id = document.location.hash.match(/^#(.*?)$/)[1]
        get_github_name_for(id)
    }
});
</script>

Feel free to copy it, or use https://edent.github.io/github_id/#837136 as a handy little webservice.

This doesn't affect the whole site - just targeted pages. It doesn't require elevated permissions, nor any special skills. This is just GitHub punching itself in the face.

Here's how it works.

• An attacker creates thousands of comments in their own repos which contain references to a specific issue or PR in an external repo.
• When that issue or PR page is loaded, GitHub tries to render every single reference from all repos.
• This often makes the page completely unavailable (Unicorn Error) and always slows down page loading and rendering time.
• Site owners are unable to remove the malicious links, leading to a either permanent degradation of page loading time, or a page which can never be loaded.

Uses

Suppose a GitHub issue is talking about a security vulnerability - a malicious actor can deny access to that page.

Similarly, a competitor can disrupt your normal GitHub workflow.

Users on slow / mobile Internet connections will have a markedly worse experience accessing pages.

Examples

A good source of examples is the Linux Kernel.

Take a look at Pull Request #12.

(Edit - this page now appears fixed and no longer times out.)

Most of the time, the page fails to load. When it does load, it renders slowly on desktop. At the bottom are hundreds of links to places which appear to refer back to this. But they don't! Instead, the comments often contain lists of numbers like:

#5 ffffffff812a12de (____fput+0x1e/0x30)
#6 ffffffff8111708d (task_work_run+0x10d/0x140)
#7 ffffffff810ea043 (do_exit+0x433/0x11f0)
#8 ffffffff810eaee4 (do_group_exit+0x84/0x130)

GitHub gets confused and thinks that those numbers refer to an Issue or Pull Request.

When it tries to render the page, it can timeout while gathering all of the comments which appear to be links.

In my experiments I found dozens of pages which repeatedly gave timeout errors.

Severity and Disclosure

This is a low impact bug.

When there are thousands of comments across dozens of repositories, gathering all references can be time consuming. Once the servers have managed to successfully render the page, it reduces the likelihood of the page being blocked again.

I reported it via HackerOne on the 13th of February. The next day GitHub responded:

In the meantime, there isn't much you can do to protect yourself. There's no way to bulk remove malicious references.

On GitHub's side, they should be truncating pages before a timeout occurs. They already do this on mobile - perhaps it should be standard everywhere?

So, here's the inspiring story of how a rag-tag band of adventurers brought it back from the dead!

Several years ago, Justin Scarpetti created a tool to extract data from an Amazon wishlist - the imaginatively named Amazon Wish Lister. It used that most vulgar of programming practices - Screen Scraping!

Yup, gobble up the HTML and attempt to parse it. Needs must in a dire situation. It worked, but it felt dirty!

Actually, scratch that. It mostly worked. It wasn't retrieving prices due to a change in the page structure and it was hard coded to the US site.

Time to fix things!

The Aim

I want to add something to my Amazon Wishlist and have it automatically appear on my Secret Santa Tumblr.

The Changes

• Add support for international Amazon stores - not just .com. This was pretty easy, all Amazon stores use the same URL structure for their wishlists and (mostly) the same HTML for their pages. Some Amazon stores don't have wishlists at all.
• Get the prices. Again, a simple fix - although it is at the mercy of Amazon changing their page structure. Such are the perils of screen scraping.
• Better XML Output. Some weirdos prefer XML to JSON. Fools! This was intended to be a precursor to RSS support before I realised there was an easier way to do it.
• Get the AISN. Due to Amazon's consistent URl structure, it was easy to grab the item's ID. Useful for the next few parts.
• Bigger images. By default, Amazon gives fairly small images as standard. A little URl manipulation and we can get much larger picture served over HTTPS. Weirdly, all images are served from the .com site.
• Amazon Affiliate Support. One of my major reasons for doing this project is that I earn a small amount of revenue from Amazon whenever someone clicks my links & then goes on to buy something.
• RSS Feed Generation. With all that groundwork laid, I was able to quickly generate a (mostly) valid RSS feed. Every time something is added to the wishlist, the RSS feed updates. Perfect!

Rather than monkeying around with the Tumblr API, I just used IFTTT. It checks the RSS and should post any new items onto the blog.

The Result

IT WORKS! Visit my FiverFun Tumblr to see it in action. Each time I find a cool stocking-filler under five quid, I add it to a specific wishlist. IFTT picks it up and sends it to Tumblr.

You can download the updated code from GitHub

Bugs

It wouldn't be an Open Source project without a few bugs!

If you can help with any of these, please chip in over at GitHub.

• Japanese text encoding is hard!. I'm not sure if it's a Shift_JS vs UTF-8 issue, or a multibyte thing - but something is screwing up the text formatting.
• India's prices are nested within another set of tags. Need a way to always get the innermost content.
• Both of the above may be related to the screenscraping library - the ancient phpQuery. It may be worth either fixing it or moving onto a new library.
• International dates are problematic. I need to find a way to easily convert them into RFC-822 format.
• Transient text encoding issues - mostly, I think, due to vendors copy & pasting from MS Word.

If you've enjoyed this post, you can buy something for me from my Amazon Wishlist :-)

It's very temping to see software as racist when, in reality, it's more likely to have a root cause of unconscious assumptions.

Take, for example, GitHub. You can host all of your software projects on there - as long as you speak English.

Wait? What?

Try adding a repository which contains, say, Chinese - and all those beautiful characters will be replaced with "-".

I asked GitHub about this, and quickly got this reply.

Unfortunately, at the moment, you can only use ASCII (i.e. Windows-1252) characters in Repo names. Most things on GitHub.com support non-ASCII but because of limitations in Git, the repo name isn't one of them. Sorry about the international-unfriendliness

Interestingly, that's not quite the case. Windows-1252 contains some characters with accents - they simply aren't recognised by GitHub.

We don't live in a homogeneous world. US English is not the global language. Even if it was, ASCII is insufficient to the task of information interchange.

ASCII was invented in 1972 - 40 years later and our brand new shiny kit is hamstrung by the needs of the telegraph industry! It's like that wonderful urban legend about the Space Shuttle being constrained by the size of a horse's arse.

Obviously, GitHub isn't racist. Either they or the originators of Git have assumed that their local dialect is sufficient for a service which aims to be universally acceptable. All the more strange given that Linus Torvalds, the creator of Git, is Finnish and - one presumes - knows about ääkköset (the "extra" letters in the Finnish alphabet).

At this stage in the maturity of the software industry, we should consider the practice of not supporting Unicode as outmoded and dangerous as assuming every year can be represented by a two digit number.

There's a world outside our narrow viewpoint and, if we want to do business with that world, we need to speak their language.

Ahem...

When I first started shkspr.mobi it was intended to be an easy way to get Shakespeare on your phone. At that time, there were no mobile formatted texts of his plays and sonnets, so I had to create them. Finding Shakespeare's works in a suitable format for conversion wasn't too hard - but it meant lots of crufty code to read text files line-by-line. Yuck.

A few years later, I stumbled across Open Source Shakespeare. The project grew out of Eric Johnson's MA thesis. It's a remarkably good idea with only one minor problem. The database it uses is Microsoft Access.

MS Access, as a database, could best be described as

deformed, crooked, old and sere, ill faced, worse bodied, shapeless everywhere, vicious, ungentle, foolish, blunt, unkind, stigmatical in making, worse in mind

(Comedy of Errors, Act IV, Scene II)

There are a few Open Source Shakespeare projects on GitHub, but they don't seem very practical.

So, naturally, I've decided to create my own version of Shakespeare's works - in MySQL :-)

This is what it looks like: You can download it from GitHub.

I've stripped out a lot of the extraneous stuff from the original version - word counts, etc. So it should be a fairly lean database which is easy to use. I'm not a database professional, so I would be grateful if you could suggest any improvements. Either using this blog's comment form or on GitHub.

There are four tables

Paragraphs

This is where the main body of text is. A typical row will look like this

• WorkID: hamlet
• ParagraphID: 639015
• ParagraphNum: 3427
• CharID: hamlet
• PlainText: Has this fellow no feeling of his business, that he sings atngrave-making?
• Act: 5
• Scene: 1

Works

This is what translates the "WorkID" into something human readable - plus some extra metadata

• WorkID: hamlet
• Title: Hamlet
• LongTitle: Tragedy of Hamlet, Prince of Denmark, The
• Date: 1600
• GenreType: Tragedy

Character

This is what translates the CharID into a human readable name and description

• charID: hamlet
• CharName: Hamlet
• Abbrev: Ham
• Works: Tragedy of Hamlet, Prince of Denmark, The
• Description: son of the former king and nephew to the present king

Chapters

This gives the setting for each Act and Scene.

• WorkID: hamlet
• ChapterID: 18893
• Act: 5
• Scene: 1
• Description: Elsinore. A churchyard.

What's Next?

The next steps for the project are fairly obvious:

1. Write some high level example code to show people how to use the database.
2. Make shkspr.mobi a showcase site which runs off the database.
3. Fix any bugs and inconsistencies that people find.

You can download the Shakespeare MySQL Database from GitHub.