Coding tests are broken

How does a multiple choice quiz question or a timed coding test correctly assess a person's technical skill?

The short answer

In no way, shape or form does a multiple choice quiz question or timed coding test correctly assess anything. Really. They mean absolutely nothing. The creators of TestDome and probably others like HackerRank should go out of business. These apps are really not improving the world and are extremely misaligned with how the world works. Don’t rely on the results from these apps and I recommend you completely discard them

The long answer

Multiple choice quiz questions are hilarious. Even if you don’t know the answer, you can always take an educated guess (...maybe even a wild guess to a certain degree) and there is a chance you’d guess correctly 

A timed coding assignment that presents the person with a problem that requires an understanding of some esoteric algorithm, mathematical equation you have not read about or some library that you have not used before is actually very unreasonable and is simply testing the person’s memory and nothing else

This goes back to the exact same concept that exists in school with exams. They simply test a students memory and have not evolved for the world eams now find themselves in

Mostly everyone on this planet has instant access to all the world's information. It’s great if you are gifted with a good memory–I most certainly feel my memory is good, but why would you not use the tools of the modern world to assist you if you don’t have a good memory or just want to be completely certain?

I’m forever looking up class and function synopsis even when my IDE has code-completion and I usually click a few extra results in the search, that open in new tabs in my browser, just so I can get a brief glimpse into questions that other people have asked that are relating to the class/function at hand. What’s actually wrong with doing this? Surely this does not make me a bad web application developer or DevOps engineer?

Timed coding tests disagree. Coding tests that are simulated in a setting where you cannot do this type of activity really disagrees it seems

Even the concept of time in these simulated tests make little sense. Have you thought of how long it actually takes to get code to production and everything that’s involved in-between? Solving the actual problem and writing code is one thing, waiting for PRs to be reviewed, updating tickets, writing comprehensive documentation, merging branches in and waiting for CI/CD pipelines is another

An aside

The whole concept of “it’s how you think” raises many questions too. It’s very important to either have or try to cultivate a logical way of thinking but I feel you’ll still be caught up in these timed or incorrectly simulated coding tests. You understand logic. You solve problems in a methodical way. Now without looking this up: what type is the second parameter on the function call_implements() that’s found in the PHP standard library? There are hundreds of other functions that I could apply this question too

The way you think can improve and evolve overtime. As you read, listen, analyse problems and actually build things, you’ll improve how you think and this will shape your view of the world

Arguments for my statement:

Grit: the power of passion and perseverance | Angela Lee Duckworth

Outliers: Why Some People Succeed and Some Don't

Arguments against my statement:

Jordan Peterson | The Most Terrifying IQ Statistic

A better way to screen candidates

From what I can tell, coding assignments are there to screen out fraud which is totally understandable. There are a lot of fraudsters in industry so something has to be done to factor this in

Ask basic questions that any developer should know

I’ve been asked screening questions in multiple face-to-face interviews and I feel this is a good indicator as to whether the person is real or not

Questions I’ve been asked during an interview:

Questions I’ve asked other people during an interview:

These are all fairly simple questions and you would expect someone that has been in industry for a while to at least know the answer to 1 or 2 of the questions

The coding test must be reasonably timed and take-home

There is absolutely nothing wrong with take-home coding tests or assignments, assignments that would ask the person to build or implement something

When the assignment is take-home, the person is relaxed and they can work in a more real world setting. It’s very common practice to research how other people have built the same piece of software or looking up class/function synopsis or even asking questions on StackOverflow

You’d review the candidates work at the end of the reasonable time limit and then cast your opinion on the work (whether the software works, if it’s built well and follows good computer science principles)

The simulation must be real world

I’ve built a simulated test that is aimed at testing DevOps engineers. I simulated a shell that completely runs in a browser (I read the incoming command and return relevant output back to the user) and I have built BASH command completion in and each of the supported commands has a -h (help) option in so it should be simple enough for a candidate to figure his way around

This is what I mean by real world. You don’t have to recall command synopsis from memory and now all the candidate needs to do is find his way through the test. This is exactly what debugging will look like on the production stack and as you learn more about how the application works, you’ll be quicker to solve problems in future

Understanding how to effectively build an application, architecting the backend infrastructure in a scalable way is not something I’m covering here

Actual on-the-ground work under a probation period

They have a passion for the work, they answered at least 1 or 2 of the screening questions and the take-home test works. The next step is probably the most important of the entire hiring process: actually hire the candidate and evaluate ability to perform the job

The evaluation process is fairly simple: do they get the work done? If the answer to this question is no, you have really not lost anything. Just let the probation period expire

It’s also very important to focus on output and not input. Micromanaging is a definite no-no and always allow for remote work

Reading elsewhere