Absolutely everything you need to know in 2024 about assessing candidates skills when hiring
This guide is aimed at hiring managers, talent acquisition, recruiters and anyone else who's interested in implementing hiring skills assessments in their organisation.
The adoption of skills-based hiring has exploded in recent years, as companies finally begin to overthrow the shackles of traditional hiring approaches.
In this guide you will learn lots, including:
OK let’s go, everything you need to know about skills tests for 2024.
Skills tests, also known as skills assessments or skills quizzes, are a way to objectively evaluate someone’s skills & knowledge in a given area.
For example, if you wanted to hire a Data Analyst, a Software Engineer or a Sales Development Rep, each of those roles will have a different set of expected skills, either required skills or nice-to-have skills.
These skills will typically involve a combination of ‘soft’ skills and ‘hard’ (technical) skills. Either way, a skills test will then evaluate the candidates in these 4 areas, measuring how strong they are in each of these skills.
The result will normally come out as a simple percentage, making it easy to interpret and compare candidates. This objective, data-driven approach is one of the key reasons for the take-off of skills-based hiring over the last few years.
Note, when used internally on existing team members, skills tests tend to be called - for whatever reason - capabilities assessments.
Skills tests can be used to evaluate any skill, be it technical skills (hard skills) or soft skills.
In practice, the skills tested depend on the role’s requirements, that’s why it’s absolutely essential to think ahead of time about what the requirements are for the position - this then informs all of the hiring process, including the coverage of the skills test.
When it comes to soft vs hard skills, it should be said, soft skills are inherently more difficult to measure, because they tend to be highly subjective.
For example, measuring someone’s verbal communication skills is actually tricky. Are you measuring how well they explain technical concepts to non-technical stakeholders? Or perhaps how succinct they are? Or how easy they were to understand? Or how charismatic they were when you spoke with them? Depending on who you ask, what ‘good’ communication is will vary significantly.
Because of this subjectivity, it makes the results of soft skills assessments less predictive of job-performance than technical skills assessments, which are inherently more objective.
Technical or hard skills tend to involve things that are factual, and either are right or wrong. This makes them easier to measure and also more predictive of job-performance than soft skills tests.
Skills tests can be used at any stage of the hiring process including as a pre screening tool, screening tool, at the interview stage and as a final validation.
The main difference is normally how long and complex the skills tests will be. Pre-screening skills tests will typically be 20-40 minutes and only cover off the most fundamental basics, whereas when used further down the hiring funnel, a more in-depth skills assessment will cover a wider breadth of skills in more detail, and might be a few hours of effort for the candidate.
The best place to use skills tests is as a pre-screening tool, instead of relying on manual CV screening, which opens up your hiring process to significant bias & discrimination. It also saves a lot of time, expedites the hiring process and prevents you from missing out on the best candidates.
The science for skills tests is crystal clear. They are an indisputably better predictor of job-performance than other selection methods like interviews, reference checks, personality tests or CVs.
The most comprehensive study of selection methods for predicting job performance was by Hunter & Schmidt. They published their substantial study in The Psychological Bulletin in 1998.
They performed a meta analysis of 85 years of research into what predicts job performance. They investigated 19 different selection methods for predicting job performance, including various types of interviews, personality tests, work sample tests, IQ tests (general mental ability), reference checks, peer reviews and more.
To cut a very long story short, they found that skills tests are the best predictors of future job performance. What they refer to in the paper as ‘work sample tests’ and ‘job knowledge tests’ are ranked 1st and 3rd most valid, respectively.
The 2nd most predictive method is what they called general mental abilities (also known as IQ tests or cognitive abilities tests) tests. A special note is that they work best for junior candidates who have not yet developed skills, like graduates and interns.
Skills tests evaluate a candidate’s current skills and knowledge.
The other main types of tests are:
Other than tests, there are other ways to select candidates, including: referrals, peer ratings, experience, education background, unstructured interviews, structured interviews, job trials and reference checks. Typically of course a variety of methods are used when deciding who to hire, rather than just one.
It’s worth emphasising that some of these methods are completely invalid - they do not predict job performance, at all. For example, age, interests, years of education, and years of experience, in particular, do not predict job performance, as shown in Hunter & Schmidt’s comprehensive analysis.
Using skills tests to select candidates has a number of benefits.
Hire the right person
When you’re hiring someone, what you’re really getting is access to their skills. So without measuring that, you’re really stabbing in the dark. Anyone can put anything on a CV and can bluff their way through an interview. Hiring someone without testing their skills would be like a basketball team playing some dude who turned up and said ‘Sure I’m pretty good’. Not going to be a great outcome. The cost of a bad hire is huge, and can you guess the leading cause of a bad hire? The candidate didn’t have the right skills. That’s a true hiring own-goal!
Using skills tests helps to make the hiring process much fairer. Candidates can be objectively evaluated based on their measured skills, and this helps to minimise potential bias. It also makes it really simple to compare candidates - just look at their scores. Other methods like referrals make hiring a bit of an old boy’s club, and CV screening leads to significant discrimination.
In most jurisdictions, it’s a requirement that hiring - at least in principle - be merit-based. There are sometimes carve outs in laws to allow for overt acts of ‘positive discrimination’ to rebalance some existing inequity, but this is normally tightly controlled. So, make sure to set up your hiring process to be objective and merit-based, to stay on the right side of the law.
Get hidden gems
By giving every candidate a fair chance with an objective screening skills test, you will identify candidates you were previously incorrectly rejecting. Minimising this hiring error is crucial, especially during periods where there are talent shortages. CV screening tends to favour candidates who have packed their CVs with buzzwords, inflate their abilities, and this approach is open to significant discrimination.
Take for example, the recent story of Nimash Paranapalliyage, who was forced to change his name on his applications in order to get any call-backs. This is an utter disgrace, and should not be happening in this day and age.
When moving to a more skills-based hiring approach, like with any change, you should expect some pushback from anyone who’s change averse. It’s important to push through this initial resistance and adopt skills-based hiring properly. The most common myths you’re likely to hear are things like ‘Will candidates do this?’, ‘But aren’t tests biased?’, ‘I can just judge someone by chatting with them’. We cover these off below.
Not testing candidates’ skills is like flying blind. You can do it, but it’s certainly not recommended. By failing to properly evaluate your candidates skills you’ll be open to mis-hires, unfairness, illegal hiring and missing out on some of the best candidates who have applied to your roles.
The skills-based hiring movement of the last 5-10 years has come about from 4 big trends:
Artificial Intelligence is such an incredibly fast-changing field, it’s quite hard to commit to what impact it has had or will have on hiring in general, or skills tests specifically. On one side, there are concerns over candidates’ ability to cheat, and on the other side it opens up the opportunity for companies to evaluate candidates in more clever ways.
Watch this space!
When used instead of CVs as a screening or pre-screening tool, skills assessments solve the systemic discrimination that CV screening allows. By moving to a skills based hiring model, you can help to ensure your hiring process is merit-based, and that the appalling discrimination cannot happen any more.
Skills tests and interviews are both selection methods for deciding who to hire. They are not mutually exclusive. The comprehensive analysis of Hunter & Schmidt from 85 years of studies into predicting job performance shows that skills tests are more predictive than interviews.
Within interviews, structured interviews were found to be much more predictive than unstructured interviews.
Skills tests are an easy way to evaluate lots of things in a short period of time, making them an efficient way to understand the candidate.
Skills tests also allow you to evaluate each candidate in an apples-for-apples way, that interviews just don’t.
Finally, interviews are heavily biased towards tall, good looking and confident people.
In-person tests are a lot more challenging to administer. You’ll need an office space, a computer for the candidate and obviously then also need to do everything in person.
Online skills tests tend to be easier to administer and can be done asynchronously. With advanced cheating protection and online proctoring, you don’t lose anything by having candidates complete the tests online.
That’s a great discussion - here’s a comprehensive analysis of that decision.
The main reason to build use your own test is if a) you already have one set-up b) you rarely hire c) you’re sure the take-home project hasn’t been leaked online c) you work in a large bureaucracy where you simply don’t have any agency over which software you use and you have no budget and d) you or your team are experts and can evaluate the candidates’ skills properly.
Otherwise, we’d generally recommend using a dedicated skills testing product instead of hacking out your own.
On Alooba, we’ve assessed more than 50k candidates and surveyed their opinion on testing. It’s probably one of the most extensive datasets in the world about how candidates feel about skills tests.
Candidates generally agree that skills tests are a fair and objective way of hiring. They tend to be sick of the same BS as everyone else, and they can see how it’s better than having your CV rejected because they’re Asian, and other nonsense like that.
Candidates feel most positive about skills tests when:
Candidates are also dramatically more likely to complete tests when:
Any system will have some level of expected fraud or cheating, and recruitment is no different. A relatively small percentage of candidates will try to cheat, either by gaining some kind of unfair advantage, getting someone else to take the test on their behalf, or getting help from some AI.
This is one of the most important reasons why creating & managing your own skills tests will be challenging - without any cheating protection measures at all, you’ll be opening your hiring process up to abuse.
In order to protect the integrity of the measures, Alooba does not publish details of its advanced cheating prevention system. Learn more about these from your Alooba account manager.
Skills tests help to create a fair, objective and merit-based hiring process. For example, by replacing manual CV screening with a skills-based screening approach, you do your bit to prevent the shocking discrimination that happens with traditional screening.
It really depends on whether you create your own skills tests or use an off-the-shelf product.
We get a high flow of applicants, which leads to potentially longer lead times, causing delays in the pipelines which can lead to missing out on good candidates. Alooba supports both speed and quality. The speed to return to candidates gives us a competitive advantage. Alooba provides a higher level of confidence in the people coming through the pipeline with less time spent interviewing unqualified candidates.
Scott Crowe, Canva (Lead Recruiter - Data)