We've built a towering pile of shite
Open Source rant · 21 comments · 1,100 words · read ~17,340 times.
This a rant, written at midnight, after battling software errors. Set your profanity filters accordingly.
I despair over the state of software engineering - specifically, stability. We seem to have lost the understanding that computers are there to do the hard work for us. And I don't think we ever believed in a user-centred approach to developer tools.
The last time I developed an Android app all by myself was… fuck… Doughnut? Kitkat? Too long ago. Last week, I found a small open source Android app that I wanted to alter. Nothing drastic, just changing a few strings and a couple of images.
The repo was 3 years old. Surely it wouldn't be that hard to get running again? Ha!
Here's what went well.
- Installing Android Studio. I remember when this was a chore - but I just installed the Flatpak, opened it, and let it update.
- Cloning the repo. Again, simple.
- Importing the project. Couple of clicks. Done.
Then it all went to hell.
Fail #1 - moving targets
Gradle sync failed: Could not find com.afollestad:bridge:3.2.5.
That's annoying. One of the libraries the app wanted to import was lost. Where was it?
It appears to be in MVN Repository with a note that:
this artifact is located at Spring Plugins repository (https://repo.spring.io/plugins-release/)
Things move. I get that. But why isn't there some form of automatic forwarding in place?
I fixed the issue and moved on.
But, as a new(ish) developer - how do I understand the complicated relationship between an app, the tools, the repositories, and all their weirdness? Where's the user-centred design in all this?
Fail 2 - automagic updates
Google's Android Studio is complicated. And it puts all the effort on the user. It kept complaining that targets were too low, and needed to be updated. It didn't actually offer to update anything itself. It didn't tell me where the error was. It didn't say "hey, we can automatically update some things. Here are the risks. Want to proceed?" It just left me in limbo.
So I found the strings and manually changed 5 numbers.
Why don't our tools do the hard work for us? An IDE isn't a screwdriver - it has knowledge about what it is doing and what problems it is encountering. It doesn't need to make changes unilaterally, but it could at least make helpful suggestions.
It reminds me of one of my earliest frustrations when I first started to learn C++ back in the 1990s. The compiler would say "Missing semicolon on line 427." To which I'd reply "If you know what is missing, and you know where it is missing, why not add it back yourself? You useless piece of crap!"
Fail 3 - change of syntax
Moving on. Deep breaths. Gradle started complaining that I couldn't use "compile" any more. Apparently it was deprecated. OK. So what's the fix? Again, neither Gradle nor Android Studio would tell me. They knew there was a problem, they probably knew what would get it working. But they stayed silent. Because fuck the users, amirite!?
The fix was simple. I manually replaced the word "compile" with the word "implementation".
That's it.
I did that a dozen times and it worked. What's the difference between the two? I don't know. I don't particularly care. I'm sure there was a very good reason for replacing one word with another. I'm sure it makes a profound difference to someone. But for me, the end result was the same. The app now compiled.
Imagine a user interface which suddenly said "you must turn things clockwise from now on!" You'd be all like "WTF? What's wrong with anticlockwise? That's the way I've done it for years." And they'd be all like "Dude! Clockwise is the way to go. It all has to change." And you'd go "Mate, can't you just install them backwards or something so I don't have to change my behaviour to accommodate your needs?" And then they'd spit at you.
Why doesn't Android Studio / Gradle / Whatever just go "Mate - you've used compile here - that won't work. Want me to automatically update the syntax? It'll probably be fine."
Don't make me think. And don't make me waste my time fixing problems that you have caused.
Fail 4 - who are our users?
I ranted on Twitter, of course, and had a series of frustrating conversations with someone from Gradle. Twitter is a crappy mechanism for reasoned discourse. So I'll try to be more lucid here.
Forums are full of people going "I can't get this project to work - Gradle is throwing weird error messages. Help me. I don't understand. Please - help!"
Gradle does not do any user research. It never sits down with independent developers, newbies, or seasoned pros. It never tests its error messages. It doesn't do developer relations. It doesn't spot bad implementations in the wild and offer helpful suggestion. It doesn't relentlessly focus on making things easier for users.
Gradle focusses on large organisations and caters almost exclusively to their needs. That's fair enough. It's their business and they can run it how they like.
The same is broadly true of Android. Its development tools are focussed on large teams of highly trained software engineers, who all intimately understand every esoteric aspect of App development.
It's great if you're a salaried employee with a large support structure. But for small and amateur developers, it's a nightmare.
I've lost count of the number of projects I've seen which start by saying "Simply install Flargnle in the normal way..." Oh, great, another technology that I need to learn in order to make a cool widget.
Fail 5 - extrapolating from limited data
I've used this project as an example. And I am being deeply unfair to Gradle, Maven, Android, and everyone else. I'm tired and frustrated.
But I'm sure you've found the same thing with other bits of software. You have to download endless libraries - and then get stuck in dependency hell because v1.1 isn't compatible with v1.2 and the original Google Code link has died and the mailing list hasn't received a message since 2015 and the author's GitHub shows they're more interested in the latest JavaScript framework and the error message points you to a USENET message and the last StackOverflow answer is "nevermind, figured it out".
The job of a computer is to do the hard and boring work for us.
Best of: What I blogged about in 2020
💯 and this is sooooo frustrating. Ultimately it saps the joy out of tinkering with software. I used to spend weeks on this sort of thing but I’m old and tired now. I realised I was spending all my time troubleshooting instead of using computers. I just want things to work!
I’ve come to accept that unless you spend 90% of your time working in any platform/toolchain 90% of your time will be spent configuring/debugging your toolchain, managing dependencies and resolving platform API changes.
All this talk of ‘dev UX’ in recent years was bullshit.
I absolutely agree with this in some cases. Most recently i experienced linter rage. Cryptic messages about whitespace and weirdly not offering to fix them, leaving me to guess is not a good developer experience.
We’ve built a towering pile of shite: shkspr.mobi/blog/2020/06/w… Comments: news.ycombinator.com/item?id=234385…
This blog post is so on-point, I recently had similar frustrations with my own Android project, to the point where I don't think I can maintain my own code anymore, thanks to the towering pile of shite it sits on.
shkspr.mobi/blog/2020/06/w…
@hhariri this is relevant because gradle is a barrier to entry into the kotlin world. I’ve become more familiar with Gradle since Kobalt has become a lonelier place, and this means I’m less able to detect its warts. JB should ask newbies shkspr.mobi/blog/2020/06/w…
What an enlightening read- I never really thought about the user needs of the developer before and the impact ... thanks for sharing
Pablo says:
Tl;dr “You kids get off my lawn! And run it all in VMs”
You can take your COBOL and JCL from 1985 and be very confident that will run on your 2020 mainframe. That’s a feature - not a bug. It’s also a language intended for use by people who were business people and not mathematicians. (There were no CS degrees in 1959). We’ve also gotten rid of the largely self-contained, low-impact, click-run tools like PowerBuilder and pre-DotNet Visual Basic. Visual Studio is still pretty much there, but the languages are now targeted at professional, full-time developers. But Microsoft owns the whole too chain and tries to make it largely click and go.
In some cases, when you pull back the curtains, most modern tool chains seem like they are stitched together like something from Etsy - each version (or even installation) is a hand-crafted, unique one-off that you’ll never be able to make again. (I’m looking at you, JavaScript). Forget about having a client call you in 5 years and ask you to add a few features to that app you built. You might go to GitHub and see the projects are there, but the last commit was four years and 11 months ago, during the 5 minutes when that was the tool everyone had to use (if they were a ‘serious’ developer.)
I came to despise Java because I felt it was a hodge-podge of layered libraries and stitched together solutions. Couple that with out of date posts on Stack Overflow and dealing with Java issues felt tedious. But now it’s looking like the responsible adult in the room. It’s like seeing your thrice-divorced, rarely employed uncle is at least sober while all your other relatives invested their retirement savings in lottery tickets, got blind-drunk, and started rolling joints. Python is kind of there, but have you ever worked through build issues on libraries that have C dependencies on Windows?
Maybe the solution is to run everything in VMs that you can back up somewhere for cold storage. That seems wasteful, but 12 TB drives are (relatively) cheap. Buy them in pairs to mirror them. Sure, that’s many terabytes of VM images but at least you could get back to a working build. Now that even low end desktops have 8 cores/16 threads, lopping off 2 cores for your dev tools, 2 cores for your test environment, and 2 cores for databases or whatever, seems reasonable. That leaves 2 cores for your machine on which you run all the tools you would have to re-license if you ran them in VMs.
ben says:
The entire computing stack from the hardware stack is broken from the conceptual paradigm of what software is, through the hardware with their leaking abstractions and CPUs hidden inside CPUs, through endless strata of buggy libraries and crap code - we build on sand. I did a PhD in software engineering in the hopes of changing something, but after long years in the trenches, I gave up. Since last year I've become a builder of wood and stone. I am infinitely happier for it.
My own rant from last year: https://gist.github.com/nileshtrivedi/7cd622d4d521986593bff81bfa1e5893
P. says:
I’m working for a large organization that makes – among many other things – embedded software. I’m involved in creating + maintaining (one of) the tool chain(s). We only have a couple thousand users (i.e. embedded devs) with somewhat foreseeable use cases, so one would think we should be able to get our dev UX right. But it is incredibly hard, because the minute you try to “do the right thing automatically” (like, in your example, placing a semicolon where it should be), a bunch of very vocal users/devs will come out an shout: “What the hell were you thinking, how dare your crappy tools touch my code? The world could have ended because of that!” … they are the same people that argue we (the organization) should just train our devs better, rather than trying to make the tools hide the complexity from the devs. And what can I say: I’d love to do that. But just as you said, sometimes devs just want that stuff works, rather than spend days or weeks to understand the details of a tool chain…
I do little bits of Android dev every few years and this is exactly my experience.
Excellent rant: “If you know what is missing, and you know where it is missing, why not add it back yourself? You useless piece of crap!” shkspr.mobi/blog/2020/06/w…
It'd be interesting to know what tools are used by the app developers inside Google. Also what tools are used by app developers inside Apple.
“Don’t make me waste my time fixing problems that you have caused.”
This reminds me of a great talk by Adam Harvey called “What PHP learned from Python” about not changing things that users have come to depend on just because they are “incorrect”. Often things become “incorrect” even though they are working fine because the original developer has read about some new dev/architecture/naming paradigm and got excited.
https://m.youtube.com/watch?v=V9ZC2CwkE0I
By far the best modern language for owning it’s prior decisions and sticking to them, is Clojure.
Fail 4 is very annoying. The focus of these tools is too much on enterprise users and too little on every level or casual dabbler. It’s pushing us on a path of over-specialization where the barrier for entry is getting higher and higher. This is not the future I was hoping for.
We’ve built a towering pile of shite – Terence Eden’s Blog shkspr.mobi/blog/2020/06/w… < This wonderful ranting blog post is a fair reflection of the state of much modern mobile software development.
Paul says:
I cannot state enough how it warms my heart to read this and others replies (only because masochists enjoy company). I have a small android project that I normally have to update once a year, and then forget for the next eleven months. Every time Android Studio updates, the project fails to build again. I then spend more time trying to get the original project to build than I actually spend on development after the build is fixed. The only thing consistent in this entire thing is that it always fails!
Piotr Jagielski says:
I sympathize with a lot of what you wrote. There's no doubt that many tools could do much more to make life easier for software engineers. I had exactly the same observation as you about the missing semicolon in C++ some years ago.
However, the paragraph about Gradle not doing any user research, never testing error messages, not offering helpful suggestions, etc is completely ungrounded and simply not true.
Could Gradle do a better job at it? Sure, and constructive feedback is always welcome. But asserting that Gradle basically doesn't care about its users is deeply unfair to people who work hard to continuously make it better.
Thanks Pitor. As you're an employee of Gradle, could you point me to some published user research? I couldn't find any. If you could let me know which newbie-friendly developer conferences Gradle are attending, I'd be happy to promote them. I did search around, but couldn't find any.
The reason why I said Gradle didn't do any research with normal users, is because that's what your principle software engineer told me.
I'm sure Gradle is fantastic. But judging from all the forum posts - and reactions to this blog post - it simply isn't very user friendly.
Piotr Jagielski says:
I think what Cedric meant is that there are some very well-known companies that successfully use Gradle. It doesn’t mean that large organizations is the sole focus of Gradle or that we don’t do any research with “normal” users.
We have worked with countless users directly to improve their build automation and shorten feedback times. These include projects of different sizes and engineers with different experience levels. There a number of ways to gather feedback from the broader community including GitHub issues, forums, feedback form on our website, etc. We also present at many conferences, although obviously not in the last few months.