Reputation Scores for GitHub Accounts

The folks at GitHub know that Open Source maintainers are drowning in a sea of low-effort contributions. Even before Microsoft forced the unwanted Copilot assistant on millions of repos, it was always a gamble whether a new contributor would be helpful or just some witless jerk. Now it feels a million times worse.

There are some discussions about what tools repository owners should have to help them. Disabling AI on repos is popular - but ignored by Microsoft. Being able to delete PRs is helpful - but still makes work for maintainers. Adding more AI to review new PRs and issues is undoubtedly popular with those who like seeing number-go-up - but of dubious use for everyone else.

I'd like to discuss something else - reputation scores.

During Hacktoberfest, developers are encouraged to contribute to repositories in order to win a t-shirt. Naturally, this leads to some very low-effort contributions. If a contribution is crap, maintainers can apply a "Spam" label to it.

Any user with two or more spammy PR/MRs will be disqualified.

This works surprisingly well as a disincentive! Since that option was added, I had far fewer low-effort contributions. When I did apply the spam label, I got a few people asking how they could improve their contribution so the label could be removed.

However, there is no easy way to see how many times a user has been labelled as a spammer. Looking at a user account, it isn't immediately obvious how trustworthy a user is. I can't see how many PRs they've sent, how many have been merged or closed as useless, nor how many bug reports were helpful or closed as irrelevant.

There are some badges, but I don't think they go far enough.

I think it could be useful if maintainers were able to set "contributor controls" on their repositories. An entirely optional way to tone down the amount of unhelpful contributions.

Here are some example restrictions (and some reasons why they may not help):

• Age of account. Only accounts older than X days, weeks, or years can contribute.
  • This disenfranchises new users who may have specifically signed up to report a bug or fix an issue.
• Restrict PRs to people who have been assigned to an issue.
  • May be a disincentive to those wishing to contribute simple fixes.
• Social labelling. Have other maintainers marked this user as a spammer?
  • Could be abused or used for bullying.
• Synthetic Reputation Score. Restrict contributions to people with a "score" above a certain level.
  • How easy will it be to boost your score? What if you get accidentally penalised?
• Escrow. Want to open a PR / Issue, put a quid in the jar. You'll forfeit it if you're out of line.
  • Not great for people with limited funds, or who face an unfavourable exchange rate. Rich arseholes won't care.

Obviously, all of these are gameable to some extent. It also incentivises the theft or sale of "high reputation" accounts. Malicious admins could threaten to sanction a legitimate account.

But apps like Telegram show me when someone has changed their name or photo (a good sign of a scammer). AirBnB & Uber attempt to provide a rating for users. My telephone warns me if an unknown caller has been marked as spam.

I don't know which controls, if any, GitHub will settle on. There is a risk that systems like this could prohibit certain people from contributing - but the alternative is maintainers drowning in a sea of slop.

I think all code-forges should adopt optional controls like this.