Can you follow your own getting started guide?
By @edent
· documentation Open Source · 10 comments · 350 words
I was trying to install a new open source project and was having a hell of a time. Nothing seemed to be working despite me following the tutorial to the letter. I was getting the most bizarre error messages and was on the verge of quitting to become a goat farmer, when I threw one last dice... I read the tutorial.
Previously I'd been copying and pasting the instructions as I went. One step said "Make sure the bauxite configuration command is set to true" but the code provided said ./configure --magic M --more-magic QxZp --bauxite false --turnip green -z
And there is was! I changed a false to a true and everything started working. Being the good netizen that I am, I sent a pull request to fix the documentation.
And then it struck me.
There's no CI/CD for documentation.
Oh, don't get me wrong. Things like OpenAPI can auto-generate documentation based on your code - but it can't write a "getting started" tutorial.
Looking back through the documentaion I'd encountered, it was clear that the tutorial had been wrong for a few years. It was a small project, so it wasn't hugely surprising that hordes of users hadn't complained. But, to me, it points to a general problem. I find this issue all the bloody time! Whether it is a big established project or a little indie gadget - I follow a tutorial only to find out it is missing a step, or assumes I have a library installed, or hasn't been updated for the latest version.
This is my plea to all developers. Spin up a fresh machine - without anything installed other than the base OS - and see if you can follow your own tutorial.
If you can't, rewrite it step-by-step until it works.
@Edent I'm a very highly experience failure in computing.Doubtless this is in part due to congenital idiocy.However spending 100s of hours not getting programs to work has, I believe, contributed somewhat to my miniscule progress over the decades.And also contributed to my hatred of computers, most website designers and hobbyist-level software that passes itself off as ready for domestic use.
The deficiency you identify is remarkable only when it's absent.
Dom says:
One approach I've played with in this space is having the documentation strongly coupled with the test suite (hence providing CI for docs): https://github.com/dontcallmedom/jsodiff/blob/main/tool/update-documentation.js https://github.com/dontcallmedom/jsodiff/blob/main/test/cli.js (which of course it itself very badly documented)
@Edent Software documentation can be the worst! I also don't like it when I visit a project page and there is no description of what it does, but there's either investor-speak or point update information. "Why do I want this module?" feels to me to be the best use of a root url.
@Edent I'm reading "The Design of Everyday Things" by Donald Norman right now, he points out that people very often think it is their fault that they don't understand how something works (spoiler alert: it's not - it's bad design or communication). So that may be a reason why the tutorial had been wrong for a few years. I work with many "non-technical" people - they simply give up in these cases.
Johannes Rexx says:
Sins I see far too often include:
Failing to indicate to first clone a repository and cd into the project directory, and then issue a bunch of commands. Failing to provide a complete list of dependencies, for example package
abc-devyarn, npm, pnpm et. al. don't work the same way, one may work, another does not Build instructions that are old and don't work on contemporary distros anymorenicopap says:
The rust programming language toolchain has a built-in testing tool that — in addition to running unit tests — scraps documentation for examples and run them, this way, examples are always up-to-date.
Now it just feels like running around naked when I don't have the safety blanket of a tool that tests my examples.
@blog Excellent suggestion, and to take it even further: make an install script for your software, and put something in your CI pipeline that verifies that it can be installed using said script.
I believe@nevonnen had an idea about executable markdown files sometime around a decade ago. That seems like it could be a useful approach.
@blog While your point is good and true, code snippets in Rust doc comments are unit tests and exercised with the rest of the unit tests. It’s not everything you ask for but it’s something.
@blog Amusingly I do follow my own setup instructions for old projects as my memory is bad. I frequently find mistakes 🙂 One thing I noticed is that the most common cause of trouble is code changes without a corresponding update to documentation.
@blog ah yea. I do this pretty normally but I'm in infra and such so spinning up a full stack from scratch one-touch is my jam. I can see how it would escape most people who just get stuff working and move on.
More comments on Mastodon.