Personality Testing: Caveats & Provisos to Consider when using Psychometrics

• Have you taken a personality test recently and are somewhat dissatisfied with what it reveals about you?
• Does your Boss seem to trust the findings of your psychometric profile more than his experience of you when it comes to your growth and that promised promotion?
• Are you a HR leader seeking a tool which may aid you to discern and grow talent? And yet you still are not fully convinced by the ‘scientific research’ that each tool boasts of but are unable to articulate your ambivalence?
• Are you a Coach who leverages psychometrics frequently and is sometimes beset with an element of doubt with how the tool profiles your client?
• Are you a CEO who gets pushed around by your HR head and that external consultant, into blindly trusting a personality report which claims to know your employee better?
• Do you wonder when your colleague introduces himself or herself as a Type or a Label (ESTJ, high USD etc.) as if the label can explain away all her or his recent behaviors?

 

If you answer any or more than one of the questions with a tentative ‘YES’ then this blog is meant for you!

 

The Context

In 2010, Anne Murphy Paul in her book – The Cult of Personality Testing, estimated the testing industry as a US$ 400 million industry, growing at 8-10% annually. She was wrong! By June 2015, Times Magazine estimated the ‘Testing Industry’, largely Personality Assessment as a US$2 Billion industry – a clear testimony to how testing and psychometrics are gaining traction as tools to aid processes of hiring, managing, and growing talent. Recent guesstimates have already doubled that number for 2017 but one can never be sure.

In the recent years, Big data and associated analytics are promising the CEO the magical antidote to the VUCA world – the treasure of mapping predictability – predictable customers, predictable supply chains, and predictable markets et al…

The HR function has not been far behind!

Today, HR professionals, particularly those in the talent assessment and development ecology – a composite of recruiters, assessors, and coaches seem to be equally taken in or rather bewitched by the notion of predictability. And this ecology is willing to pay top dollar to any tool or test that promises trends or predictive analytics.

This blog would like to offer the reader five key aspects to think over and consider before deploying personality testing as a tool or worse still as that crystal ball to assess the human being and to predict her behaviors!

These five aspects or themes that form the structure of this blog are:

1. A critique on the inherent design assumptions and axioms of Personality Testing
2. A better understanding of the design of Assessment and Scales
3. Positioning of psychometric tests today – what anxieties and fears do these tests target?
4. How do we look at the phenomena of behavior from the group-as-a-whole perspective?
5. The Dark Side or your De-railers – What does all this mean?

 

Caveat 1

Inherent Design Assumptions and Axioms of Psychometric Testing

 

Most personality tests, buttressed by ‘scientific research’, claim to know who we are, our behaviors, our traits or types, and our de-railers!

Today, tests are being deployed in every aspect of our life and yet very little is really known about how these tests got created and deployed in the first place, nor is much offered about their creator(s). We just don’t seem to have the time nor the energy to see through the jargon of validity, reliability, factor analysis, lexical hypothesis and even understand how this knowledge came to be in the first place.

For example, did you know that creator of 16PF, William Catell was also authoring books promoting eugenics to social problems. In one of these titled, the Fight for Our National Intelligence, he warned about the trend that people with inferior ability were having larger families resulting in the steady decline of England’s average intelligence. In fact, Catell recommended that the government should incentivize intelligent people to have more offspring. However, when he moved to USA, this interest only became darker, where he stated that a group should be forced to eliminate itself for the its own good and the good of the humanity. He coined a new religion known as Beyondism that reinforced many such beliefs.

 

Now why is Catell important?

Catell was mentored by Charles Spearman – every self-respecting MBA would know Spearman invented Factor Analysis. Catell wanted to apply statistical approaches to create a science of personality and dreamed of “unraveling the structures of temperament and motivation …”

He joined the thinkers who were aligned to the method of lexical hypothesis which was based on the assumption – “all aspect of human personality that are important have already been recorded in the substance of language”. He began with 4504 words and used many statistical tools including of course factor analysis. He had the access to one of the world’s first computers known as Illiac I. Between 1940 and 1950, Catell publish more than 36 papers that led to the identification of 16 building blocks of personality.

Suffering from Physics envy, or should I say Chemistry envy, he actually constructed a table like the periodic table, claiming to have discovered the bedrock traits of personality, giving each variable a new name. For example, Autia meant imaginative, Harria meant no-nonsense, and Zeppia meant energetic. Catell created the 16 Personality Factor Questionnaire – a 187 item test.

The journey did not stop here for we enter strange times indeed …

 

The MAGICAL Number of BIG 5

Now Catell’s work on personality energized a number of researchers to work with the Lexical Hypothesis from 1949 onwards across the Atlantic. In 1949, Donald Fiske applied factor analysis and emerged with five factors. In 1959, Ernest Tipes and Raymond Christal in Texas also found 5 factors. In 1963, in Michigan, Warren Norman reported 5 factors. Again in 1981, John Digman in Hawaii came up with … 5 factors again!

Bingo, people believed that the Holy Grail was discovered as far as personality of humans was concerned!! Thus in 1981, Lewis Goldberg of the university of Oregon named the factors – the Big 5.

The Phoenix of Personality Testing

The Big 5 in many ways rescued Personality Testing from the attack of a Stanford professor Walter Mischel. Prof. Mischel had done a tremendous research that pointed out that personality tests don’t do a very good job of predicting human actions. He emerged with a correlation of 0.30 between personality tests and people’s behavior. I would like to quote Mischel when he wrote that – “Unlike rats and other lower organisms who have been psychology’s favorite subjects, humans do exceedingly complex and varied things… and researchers have a grossly oversimplified view that missed the richness and uniqueness of individual lives.”

Mischel had delivered a killer blow and it took the work of Paul Costa Jr. and Robert McCrae in 1970s and 1980s, when they resurrected the Big 5.

Many psychologists climbed onto the Big 5 bandwagon and the following assessment tools became famous, reviving the ecology of personality testing – including NEO PI-R, Workplace Big 5, Hogan (HPS). The Big 5 having influenced most instruments today used for personality testing evolved their own design assumptions which are neither known nor questioned.

For example, in all the certification programs that I have enrolled into – I have seen the following happen:

• Now most Indian managers, who deploy these tools never ask themselves that the 5 Factors could be too few to adequately describe the infinite variety of human nature.

 

• The same managers never ask why the 5 factors do not look at Faith or Sexuality – two powerful forces that define us.

 

• None of the Indian managers I know, ever ask the question that a significant part of personality assessment has been done in USA and England – the Anglo-Saxons seem to have some claim over personality assessment, and that the culture as well as language in USA and UK are different from the wide assortment of 30 plus spoken languages in India.

 

• None of the Indian managers ask themselves that the Big 5 do not assume plurality within the individual – for example would the Big 5 ever explain how a person who is generally stoic and unemotional – gets overwhelmed and almost falls apart when offered sincere empathy and love?

 

• Lastly none of the MBAs, who seem to profess a liking for statistics including Factor Analysis and MDS, ever ask why a mechanical process like Factor analysis can be deployed to understand the human psyche.

 

For the reader, who does not know about the Big Five – it refers to, and you may use the acronym OCEAN to remember:

 

1. Extroversion – the proclivity to actively reach out to Others
2. Neuroticism – the proclivity or disposition to feel negative emotions
3. Agreeableness – the proclivity or tendency to be good-natured and collaborative
4. Conscientiousness – the propensity to be goal directed and organized
5. Openness – the tendency to be imaginative and curious.

 

And more interestingly, the ecologies behind these tools are becoming so powerful, in terms of wealth, partners, and stakeholders that one does not find enough critiquing from within the ecology of the assessments and psychometrics. Have you noticed that the purveyor of an assessment tool will never critique his or her own tool, but is quite willing to criticize competing tools?

 

I would invite you to read up including Annie Murphy Paul’s work on researching other tools including MBTI, MMPI and TAT – where you may have both a sensing of the design assumptions of the designers as well as the limits of application when it comes to personality testing.

 

Personally speaking, I await the day when modern computing with Big Data will churn out a superior building block of the Human Psyche – I am sure it will emerge with the magical number – 42!!

 

(Paying ode to Douglas Adam’s The Hitchhiker’s Guide to the Galaxy)

 

 

Caveat 2

Understanding of the Assessment Approach, Measures and the Scales

 

It is fashionable to be allergic to math and statistics these days. Very few people around me display any curiosity on the assessment approach or design of measures that underlie the personality tests, and yet have clear preferences for instruments and tools.

 

If you were to look at the following instruments – MBTI, DISC, HOGAN, CPI, EUM, 16PF, Workplace Big 5, – they fall within two categories of assessment, with their own design of scales and measurement. The two categories are Normative and Ipsative and I would at least like to introduce these for the reader to be aware of their limitations and usage.

 

Category 1

NORMATIVE ASSESSMENT & SCALES

Normative scales get constructed as measures (either a Likert scale or true / false) on one statement at a time and allow for quantification of individuals’ feelings and perception on certain topics. Scores can be then combined to create scale scores and can be easily submitted to more statistical procedures.

The big advantage is that Normative Scales can be compared and measured against a Normed Population – which is represented by a bell curve; and thus, for any individual – his or her scores can be compared and contrasted on any attribute.

Normative scales offer more insights on inter-individual differences, and are more popular in Northern America whereas Ipsative scales, that are used for intra-individual assessments are getting wider use in Europe and India.

On the PLUS side, Normative Assessment based Personality Tests can be useful for RECRUITMENT or SELECTION for you can compare and contrast.

On the MINUS side, research also says that Normative Assessment also a high chance of ‘faking’ (social desirability) and most instrument designs need to be evaluated on whether this process of faking or projecting a better image while self-referencing is picked up by the instrument.

 

Category 2

IPSATIVE ASSESSMENT

Ipsative assessments are scored or scaled on forced choice making or ranking between two equally powerful options. Ipsative assessments are used explore relative proclivities or strengths of the person being assessed.

The EUM-I and EUM-O instruments for example or even the DISC instrument is structured on Ipsative assessment where forced ranking or choice making is used as a method.

Most US researchers frown upon Ipsative assessment for it does not allow an objective comparison between individuals. Ipsative assessment is also difficult for sophisticated statistical methods – though there are methods where one can do similar analysis.

On the PLUS side, Ipsative assessment is very useful for working with Self Aware people on the fronts of Team Building or Coaching contexts. On the MINUS side, these tools may not be useful for Norm comparisons and thus are not very useful in Recruitment or Selection – especially when one is contrasting and comparing a person with the Norm population.

But the biggest PLUS, IPSATIVE assessment can boast of is that social desirability or compulsion to fake scores gets eliminated.

 

Which is Better? How does one Know?

There is currently a battle royal between Saville Assessment tools and others including Hogan for the latter deploy Normative Assessment and the former is using a mix of Ipsative and Normative assessment. Each of these groups are quoting their own statistics when one looks at validity of the instruments and attacking the other.

The more research papers I read, the more concerned I get, as both Saville and Hogan are powerful lobbies and ecologies in the psychometric world, and are emerging with research that sounds impressive and looks invincible.

I guess one has to go back to the basics and ask oneself – what am I assessing and why?

 

 

CONCLUSION

The part 1 of the blog covers the first two Caveats – Design Assumptions and Assessment Approach that a manager or coach must know about while using Psychometric tests and personality assessment.

This blog has become incredibly long and the second part would look at my favorite caveats and concerns as the personality assessment business expands and grows:

 

Caveat 3:         What needs do these tests consciously and unconsciously fulfil?

Caveat 4:         Why don’t we look at Group Dynamics that influence the Individual?

Caveat 5:         What is the Dark Side and the De-railers these instruments seek to map?

 

Your comments and questions would energize me more towards completing Part 2.

Do write in your thoughts, feelings and judgments!!