- almost 2 years ago
- In this blog
- Back to all blogs
Not All Assessments Are Created Equal
In one of our last education pieces we looked at ‘Which Assessments Should You Use’and established that a combination of cognitive ability tests, and assessments of personality and emotional intelligence are going to give you deep insight into your candidates, and the opportunity to predict their future performance.
However, not all assessments are created equal.
What Makes a Good Test or Assessment?
There are many factors that go in to making a test or questionnaire ‘good’ and as psychologists we focus heavily on the ‘validity’ and ‘reliability’ of tests. Previously we discussed ‘How Does Psychometric Assessment Work?’ which touches on this process.
However, to keep things light, what makes a ‘good’ test is its ability to consistently identify differences in individuals, for the test results to be consistent if I complete it today and complete it in many years time, and its ability to predict something. For us at Chandler Macleod we are most often focused on the ability of a test or measure to predict future job performance.
How Do We Test The Tests?
Once we believe we have a ‘good’ assessment or test that meets our strict criteria (that is it can measure individual differences, and that the results are the same if I complete it today and tomorrow and are not impacted by my mood or the weather), then we must work out whether it predicts job performance.
We do this by having several hundred, or perhaps thousands, of people complete a personality or ability assessment, and then have someone (usually their manager) rate their performance on tasks at work, or look at actual data like number of mistakes made when entering data.
Looking at this group of several hundred or thousands of people we then analyse the data for patterns. For example we might see that people who score higher on our personality dimension of ‘detail oriented’ make fewer mistakes at work when entering data.
At this point, we can say that a test predicts a real life outcome that might be valuable to your business. i.e. people who make fewer mistakes.
Do All Tests or Assessments Predict Outcomes?
Short answer, no.
Longer answer, they may differ in their ability to predict outcomes which comes down to the quality of the test and question design.
Which Assessments Don’t Predict Outcomes?
There are a number of ‘personality’ assessments which should not be used to try and predict outcomes and should not be used in the hiring process. The two most commonly known and used are the DISC Assessment and the Myers-Briggs Type Personality Indicator (MBTI).
On a repeated basis neither of these assessments have been shown to predict future job performance. They may be interesting, they may provide someone with insight into some of their personal styles, but they do not predict future job performance.
Which Assessments Predict Job Performance?
There are a number of mainstream and high quality assessments which Chandler Macleod use with our clients and can be found on our Assessment Gateway. These include SHL OPQ, Saville WAVE, Hogan Personality Inventory, 16PF, and our own personality assessment Helix.
Earlier we talked about looking for patterns and relationships. Ultimately when we measure the ability of an assessment to measure job performance we are looking for a relationship of 1.0. That is the measure perfectly predicts performance, however this is extremely difficult to do when you think about all the things that might impact your job performance (such as your boss), so we aim for as close to 1.0 as possible.
So, Which Assessments Best Predict Job Performance?
In a study completed by our friends at Saville Consulting evaluating the common mainstream assessments they reported the numbers ‘closest to 1.0’ in the table below. We replicated this methodology with our own personality assessment Helix so we can provide comparisons between our assessments and theirs.
Relationship to Job Performance
Saville Wave Professional Styles
Chandler Macleod’s Helix
Saville Wave Focus
Hogan Personality Inventory
We at Chandler Macleod People Insights are proud to have developed our assessment of Helix which outperforms many of the better-known assessments not only in the Australian market, but globally.
Chandler Macleod’s Helix Assessment of Personality
Helix measures a person’s fit for a role. That is, a person’s motivation, attitudes, preferences and mindsets required to successfully perform the role. Helix has 206 questions measuring 36 traits and takes approximately 20 minutes to complete.The tool is reliable and predicts someone’s skill or competency as later rated by their supervisor or colleague..Participants respond to statements asking their behaviours, emotions, and preferences (e.g., “I am usually full of new ideas”, and “I enjoy tasks more when I am part of a team”) on a scale from Strongly Disagree to Strongly Agree).
The assessment comprises 5 higher order factors underpinned by 36 dimensions that assess pro- and anti-social or productive and unproductive work behaviours. The 5 factors and associated 36 dimensions are detailed in the graphic below.
Helix Integrity Index
In addition to its 5 core factors and 36 dimensions, Helix contains an Integrity Index.The Integrity Index predicts the likelihood of a candidate engaging in counterproductive work behaviours (e.g., absenteeism, dishonesty, misuse of resources, and intentional rudeness) compared to others.
For example, in our research we found that 14% of people who scores low on the Integrity Index reported having blamed someone at work for an error they made, compared to only 1.40% of those people who did not have a low score on the Integrity Index.
As another example, 24% of people who scored low on the Integrity Index reported having stolen something from their employer. This compared to only 9% of people who did not score low on the Integrity Index.
While other providers offer separate integrity assessments, Helix takes an assessment internal approach to measuring integrity – that is, no extra questions are required to measure integrity, consequently, keeping the number of items applicants need to complete as small as possible.
It compares favourably to other personality assessments in several ways:
As discussed, earlier research shows that Helix is a better predictor of job performance than other tools in the Australian market
Helix clearly indicates whether a candidate is ‘faking’.Helix contains three indicators of invalid response tendencies.Like the OPQ, it contains an indicator of inconsistent responding.However, it also contains indicators of acquiescence (whether a candidate is agreeing to all items), and socially desirable responding (whether a candidate is potentially “faking” their responses in order to appear better).With the use of these three indicators, it is possible to be more certain of the interpretation of a candidate’s personality results.In contrast, the OPQ only contains an inconsistency measure which indicates whether a candidate has responded in a consistent way throughout the assessment.
Helix contains more (36 vs. 32 in the OPQ) and a broader range of traits than many other assessments.For instance, Helix measures maladaptive, or “dark” traits, such as manipulativeness and impulsivity, as well as traits not captured by other personality assessments, such as “Focus”, which contains elements of effortful and attentional control.These traits enable a more fine-grained analysis of a candidate’s potential across a variety of competencies (e.g., resilience), with a greater ability to identify traits of specific relevance to unique roles.
If you are looking for an effective measure of personality that can help you predict your employees future performance reach out to understand more on 1300 664 305 or at firstname.lastname@example.org. Please also reach out if you would like access to our source research and data behind the development of Helix.
- almost 2 years ago
- In this blog
- Back to all blogs