A hiring test I'd like to run
By @edent
· hiring · 12 comments · 750 words · read ~15,587 times.
(For the avoidance of doubt, I have nothing to do with hiring at my employer. This test is probably deeply problematic in ways you'll angrily Tweet me about.)
I'd like to tell you a story!
I am a teenager applying for temp jobs. It's the late 1990s and the temp agency have asked me to take a Microsoft Office test. You know the sort, do some data entry, format a letter, maybe perform a mail-merge.
"No worries!" I think, as I sit in front of the wheezing 486 with a nicotine stained keyboard. "I know Word 2.0 and Word 6.0!"
The CRT monitor juddered into life, Microsoft Word swam into view. But it wasn't the MS Word I knew and loved. It was Word '97. Utterly unfamiliar to me. A letter to a client appeared on screen.
"Make the client's name BOLD" said the prompt.
"Too easy!" I thought. I selected the text and hit CTRL+B.
"BZZZT!" said the prompt, "Incorrect. You have 2 attempts left."
Whoever had programmed this test had a precise way they wanted me to complete the tasks. So, I hit the Bold icon and carried on.
I made things italic, added bullets, adjusted some indents, and then got stumped.
"Change the paper size to A4".
OK... Is that in the File menu or Edit? No! Format? I hovered my mouse over File, then chickened out and clicked Edit.
"BZZZT!"
Yeah, yeah. I burned another lifeline and eventually found the submenu they wanted me to find. This was infuriating! And so it went on. Every keyboard shortcut I knew was rejected. If I didn't immediately know the precise location of the option, I got dinged.
Never mind my 80WPM, I was being tested on how well I knew an unfamiliar bit of software. It was relentless, unfair, and annoying.
Then came the Excel test. I was doomed.
"Please create a PIVOT TABLE from this data."
GAH! In theory, I knew how to do this. But under pressure and on modern software...?
The timer in the corner - did I mention the timer? - counted down the minutes until the ordeal was over.
And then... a brainwave!
I hit F1.
The computer didn't buzz or chastise me. It pulled up the help pages! I quickly looked up Pivot Tables, read the examples, and completed the task. I did the same for VLOOKUP, calculating mortgage interest, and all the other esoteric questions.
The software gave me a 100% score. I was sure to get on the agency's books! I turned around triumphant to the invigilator.
"You failed," she sneared.
I was agog.
"You cheated!" she spat venomously. "You didn't know how to do any of those things, so you just cheated!"
I was ushered out of the office and wandered down the street in a daze until I reached the next temp agency.
So, here's the hiring test I'd like to run.
Put someone in front of a bit of software they've never seen before and ask them to complete a set of tasks.
It isn't a test of how much they know. It's a test of whether they know how to look for help.
- Are they able to read a manual?
- Can they formulate a search query?
- How do they assess whether the tutorial they found is suitable or reliable?
- What steps do they take to make sure they're finding - and learning - the right information?
I'd tell them that at the start, obviously. This isn't The Secret Rules For Getting Hired.
"I know you've never used Blender*," I'd say, "And this job doesn't require it. But we want to see how quickly and accurately you can learn to use something unfamiliar while under pressure."
* Or whatever.
Because this is what the modern world is like. Tomorrow, the UI to your computer or Word Processor or Twitter is going to radically change. An unwanted update is going to move all the icons and disrupt the order of menus. Is that going to stop you from doing your job?
None of us are born knowing how to use software. But we all need the curiosity to say "I don't know how this works - but I know how to find out."
Anyway, the second temp agency didn't bother with a computer test. They stuck me in a call centre where I was screamed at all day by people who swore blind that they had been mis-sold mobile phone contracts.
There's no moral to that part of the story. Sorry.
Bill Cumming says:
AHH. Yes the old Microsoft office test. Where a software that gives you 3 to 5 different ways to do a thing, but only one is the "Microsoft way" when it comes to the test!! ( The most annoying test I was ever trained to administer!!)
Chris says:
That reminds me of a typing test I did for a temp agency, my typing is not the fastest but I realised I could ctrl+c and ctrl+v. I waited a couple of minutes before hitting ctrl+v and got an impossible WPM and told them what I did. They were impressed and were happy to put me forward for a job entering data into spreadsheets – oh the joy!
Simon says:
Things were more laid back in the 70s. I was sitting in an outside office waiting for my fiance who was interviewing for a post when the interviewer’s door opened and I was asked “Do you know what a computer is?”. Well, I did … I’d had a job a couple of years previously inputting bubble-chamber data into a big IBM mainframe in Birmingham University and had been shown the computer, and at college I had run one FORTRAN program from a deck of punch cards, so yes, I did know what a computer was! The next week I started my new post as Trainee Computer Programmer (COBOL). And I hadn’t even gone for a job. Then I started reading IBM training manuals, Computing, Computer Weekly and Datalink. Passing that test kept me busy for the next 45 years.
Oliver says:
F1 is how I got an A* in GCSE IT.
D. T. says:
Good hiring techniques involve some form of this. I've often seen people present difficult programming challenges fully expecting the interviewee to say "I don't know how to do this, but I know how I would."
shkspr.mobi/blog/2019/12/a… A billion times, yes! Let's evolve the way we test folks we hire. Great read by @edent
Don says:
I was given exactly the kind of test you're looking to run, several years ago prior to joining my current organization.
"List the tools and technologies you've used," prompted my interviewer. I rambled off tools I'd both used in my day job and played with on the side. "So you've never used Docker?" Nope." "Ansible?" Nope. "Kafka?" Strike three, I thought. "Deploy Kafka inside of a Docker container using Ansible. Here's a laptop with no software on it!" - as I'm handed a laptop welcoming me to my new Mac.
Over the next 45 minutes, I was convinced the interviewer thought I was dense. At the end, having gotten to the point where Ansible was provisioning a non-starting Kafka inside the container, he asked me to stop, thanked me for my time, and left. I wasn't getting this job; it was the most intimidating interview I'd ever had. I started working there three weeks later.
A couple of months in, I mentioned the interview to the interviewer: "Oh, I thought you did pretty well?" Our colleague's jaw drops - apparently, he never says that about anyone.
I learned from this that the interviewer wasn't asking me to deploy Kafka inside a Docker container using Ansible - he was testing to see whether or not I could learn. I've worked with some incredibly bright folks at this org, and I'll highly vouch for the strategy.
A hiring test I'd like to run shkspr.mobi/blog/2019/12/a… (news.ycombinator.com/item?id=218130…)
Jeff says:
I do a verbal version of this people I interview that I like to call Missing All Information. Basically, I tell them to ask me whatever questions they need to solve the problem. Then I describe an issue to the person (something like “computer A can ping computer B, but B cannot ping A”) and let them at it. The overwhelming majority just start stabbing in the dark with solutions, but there are the few that ask the most important question: what is the error? When I answer that the issue and solution are painfully clear, no guessing needed.
I founded a company called Parabol (https://parabol.co), and we're in the middle of a hiring cycle now. My last job was at a Organization Strategy & Development firm called Undercurrent. All prospective employees are required to conduct a "batting practice"—or short project—on a area they know nothing about before an offer is made. It tests for two things:
Can they unpack a problem they aren't familiar with? Can they work along with teammates?
We've carried that practice into our company, except we pay folks for the time spent on their project.
It used to be that one's job required us to do the same sorts of things, day in and day out. Mastery was achievable. Now, new methods and technologies are coming up so quickly, evaluating whether or not an employee can train themselves by pulling on technology and other people is an essential skill. It's remarkable how so many hiring processes haven't caught up to this new reality.
The ill considered emphasis on mastery is why I'm so disdainful of "coding quiz" evaluations (and they people who hire based on those scores) or certifications. I don't care a prospect knows Python, what I want to know is if I can stick them in a corner and they can teach themselves Rust, or Swift, or the next great tool or system.
Would this breach your secret rules thing? Probably. paulclarke.com/photography/bl…
Kevin says:
We do your hiring test. We give dev candidates a language they've probably never seen before, some test data, and 6 problems. Internet access is allowed. In one hour they need to correctly solve 3 of the 6 problems. It's an awesome hiring test.