Online Surveys Studies & Articles Papers Research Request

Order Your Copy
The Psychology of Sales Call Reluctance®

They promise "Career Counseling." They say they can "Predict Sales Success With 70% Accuracy," take the measure of your "Leadership Potential" and, if that's not enough, uncover liars with "Integrity Testing." They're out there, poised and ready to help you reach your full potential by way of "Assessment-based Performance Coaching." But concealed beneath these claims may be more packaging than performance, more hype than help, and a new 21st century twist on an old social issue.

These are perilous times for individuals and organizations seeking to purchase and use psychological tests. While established, reputable organizations such as Sigma Assessment Systems, the Institute for Personality & Ability Testing (IPAT) and others still play a vital role in personal and career assessment services, the landscape has changed. With greater access to high level professional services than ever before, thanks to the proliferation of the Internet, a familiar and potentially dangerous phenomenon has re-emerged: psychological hucksterism. Scientific psychological testing, never free from healthy controversy, now faces perhaps its greatest challenge - how to educate and protect consumers from the onslaught of pseudo-scientific "experts" touting questionable "tests" on the World Wide Web.

According to Professor Clive A. Fletcher of Goldsmith's College, University of London, the problem of irresponsible psychological testing practices has already been exposed in the UK. A program has been implemented by the British Psychological Society and major test publishers to help verify the competence of people purchasing and using psychological tests. The US, however, seems content to trail behind, preferring a faith-based attitude toward psychological testing.

Damage More Than Skin Deep

In one respect, fake psychological tests are like $50 counterfeit Rolex watches. Unsophisticated consumers may be happy with their purchase - for awhile. Until, that is, their "authentic 18K gold" watch stains their skin green. The fraudulent scientific veneers and dubious "advice" cranked out by charlatans and crackpots to the unsuspecting public can likewise turn to soul-staining rust. Unlike bogus watches, however, serious life and career decisions may be (mis)guided by test results. Career direction can be altered. Opportunities are defaulted. Money is lost. You may discover you've used a shovel where earth moving equipment was needed, or invited a Yugo to race the Indianapolis 500.

Howard Gershenfeld, M.D., Ph.D., an assistant professor in the Department of Psychiatry at the University of Texas Southwestern Medical Center in Dallas, warns, "There is a lot of junk science out there. Poor tests can have harmful, unintended consequences." For example, says Dr. Gershenfeld, test results can mislead individuals into developing illusory stories and prejudices about themselves. "People need to be skeptical," he says. "They need to ask, 'What's the evidence for this test?'"

The consequences of improper test selection, for companies and employees alike, can be grave. People can be traumatized by incorrect information derived from unproven tests provided by inadequately skilled or, worse yet, uncaring people. Take the case of Gary. Gary was a literary editor who worked for a large financial services company. His conduct and job performance had always been exemplary and professional. However, Gary became angry, depressed and uncertain of his future after seeing the results of a test he'd recently completed. His company's well-intentioned human resources department had purchased the latest and greatest whiz-bang psychological assessment tool for "employee development." It compared what you said about yourself to what co-workers said about you when answering the same set of psychological test questions. The procedure was the work of a former life insurance salesman turned "peak performance expert," marketed internationally by a well-known sales training outfit. Gary was psychologically trashed by what he read. Categorical, harsh and over-stated, test results characterized Gary in a way that was totally outside his experience or his experience with his co-workers. "Lacks respect of co-workers" the report read. "Potential for Success: Poor," it concluded. Gary was devastated. Undoubtedly others were, too. When his co-workers were asked about Gary, they all said the information they provided about Gary was plainly positive. They were as surprised as he was.

What was the test actually measuring, if anything? It was impossible to find out. Documentation on the tool was incomplete and sounded more like a third-rate sermon than a technical description of the underbelly of a psychological test. Statistical studies supporting its construction and use were sparse to non-existent.

Big Words, Big Promises, Big Mistake?

Mental testing is a complex, highly technical specialization within scientific psychology. According to Professor Ira H. Bernstein, co-author of Psychometric Theory, a leading testing textbook, specialists in psychological test design and construction (psychometricians) don't agree among themselves on what exactly constitutes a properly refined psychological assessment. Their heated ideological warfare in scientific publications has amplified the reigning confusion about acceptable testing practices with contentious posturing and impenetrably technical jargon. In the process, they have helped create a consumer vacuum that is only too willingly filled by charlatans, cranks and crackpots.

Notwithstanding the arcane vocabulary of psychometrics, there are important, basic guiding principles test users need to know to properly evaluate any assessment tool. Some, like test security, are simple. Who gets to see your scores? What provisions are in place for keeping test materials and scores confidential? Others are more subtle.

Consider, for example, this not-uncommon claim: "70% accurate predicting sales success." Sounds great, you say. But think again. If any test could determine with 70% accuracy who will succeed before the fact, you'd have a real conundrum on your hands. If it's true, what role does management play in shaping successful employees? Or sales training? Or salary incentives? Or sales contests? Or changing market conditions? Not much, if your test really is 70% accurate. In sales selection, as in other areas, if it sounds too good to be true, it probably is.

The fact is, no legitimate psychological test, with the possible exception of some single attribute tests like typing speed, has ever even approached predictive accuracy of 70%. Good tests, properly constructed and honestly marketed, can truthfully aspire to approximately 30% "explanatory" power on a good day, under optimum conditions. But seventy percent? Get real! When you see figures like this, you may be dealing with improper or misleading validation procedures, math errors or, most likely, an "ethically challenged" test vendor.

Surfing the Psychological Testing Ghetto

Marketing traps like this are one reason you should consider using a knowledgeable guide, someone familiar with the terrain, to help you select the tests you purchase and use. The American Psychological Association (APA) has codified very specific standards for psychological test construction and development. Most colleges and universities employ qualified faculty who know how to tell a good test from a bad one, legitimate tests from the pretenders. They can help. Give them a call.

The APA (www.apa.org) also maintains a catalog of curriculum-approved college and university psychology programs. Ideally, your expert guide should have graduated from a program on the APA's list. An alternative professional organization, the American Psychological Society (APS), also has a web site ( www.psychologicalscience.org) and may be able to provide guidance.

However, when it comes to buying and using psychological tests, even these precautions are not foolproof. In recent years the proliferation of higher level academic degrees in all disciplines has created a meritocracy of sorts, one whose members are experts by credential rather than competence. Nearly every member of the clergy now touts a doctorate. MBA's have become so plentiful they're no longer special. Everybody knows somebody who has one (or more). You can even buy a Ph.D. or a Master's degree, or both, by mail order for as little as $130.

Test the Tests

Even with competent assistance, buying psychological tests on the World Wide Web is risky business. It takes patience, prudence and research. Working with good tests, obtained from reputable sources and accompanied by competent support, can be effective and rewarding. Wandering through the Web's psychological testing ghetto alone, in the dark, can be costly and dangerous. Be smart. Be warned.

Bottom line? When evaluating tests, do your homework now so you don't have to cope later. We've provided a checklist of basic quality control standards you can use to get started. Most of these criteria can be applied either to specific tests you may be considering or to the companies who develop and market them. When you've completed the checklist, do the math and see how many red flags their practices raise. No test or testing company is perfect, but if you begin to see a pattern of shaky standards or off-kilter marketing practices, perhaps you should hang on to that company check or purchase order and keep looking.

Go Ahead. Test the Test!