I have, for many years now, wondered about the value of psychological testing. I’ve known many mental health professionals who swore by it, others who weren’t so sure, still others who avoided it like the plague. Over time, I’ve found much disagreement between my own observations and diagnoses of clients and the results of psychological testing. Assuming that I have some competence as a diagnostician and therefore am not always wrong, I must concur with the wisdom of one veteran psychologist who told me psychological testing is only about as good as the tester. (It also goes without saying that diagnosing is only as good as the diagnostician.)
Let my illustrate with an anecdote from my own life – for two reasons. First, I know the case rather well. Second, I can violate my own confidentiality all I want.
Many years ago, as a young naval intelligence officer, I was assigned to a program that was run jointly by the Defense Intelligence Agency (DIA) and the Central Intelligence Agency (CIA). To give me the requisite security clearances, the DIA relied on the results of the special background investigation (SBI) that was already available, plus additional data like recommendations from previous commanding officers, level of training in a particular field of intelligence, and fitness reports. The CIA, on the other hand, also required a full psychological assessment before they would clear me. I therefore hopped on my trusty steed and galloped off to the nearest approved psychologist to get assessed.
The assessment consisted the the obligatory review of my record and a lengthy interview, followed by several psychological tests with varying degrees of objectivity. I patiently suffered through the afternoon-long process, then went home knowing that I was on my way to the new job.
Not so fast.
Two weeks elapsed – no clearance. After a worried call from my prospective CO, I got concerned and went to my Special Security Officer (SSO) to ask what was going on. The SSO made a few calls, then informed me that the CIA wouldn’t clear me because testing revealed that I had homosexual tendencies, which made me a security risk.
I couldn’t help but glance downward, at the midriff of my stubbornly heterosexual person. “Huh?” I asked. The SSO shrugged.
I went to my current CO to ask what next. “Retest,” said he. “Let’s find out what’s up.” He promised to make some calls and sent me home, but called me the same evening. “Go back to the same shrink,” he said.
Great. More torture. But I went back manfully and sat down for yet another interview. The psychologist explained that he had reached his conclusion on the combination of some of my oral answers in the original interview as well as the testing. He said that I had told him that (a) I like opera and (b) I don’t mind doing housework. This, together with apparently unfavorable scores on a few scales of the Minnesota Multiphasic Personality Inventory (mainly the Masculinity/Femininity scale), indicated at least latent homosexual tendencies.
Knowing what I know today, I should have asked about the evidence for any association (never mind correlation or causation!) between opera and housework on the one hand and homosexual tendencies on the other. But I didn’t, which was just as well because, based on what happened next, I would have probably messed up my chances beyond repair.
At this point, the good psychologist noticed that I talked funny (i.e. with an accent). He asked where I was from, and I told him. “Ah!” he opined. “Then you’re fine.” He then admitted that, MMPI scores, Mozart and dirty dishes to the contrary, I was most likely a flaming hetero and therefore not much of a security risk. At least, the bad guys couldn’t exploit me because of my sexual proclivities. He finally explained that, while American men with my kind of profile should be suspected of harboring homosexual tendencies, European men didn’t.
Again, knowing what I know now, I might have asked what evidence there was for a Yes for American men but a No for European men. I might have also asked if it’s really true that European men like opera and housework better than American men. But this was one of the few times in my life that I managed to keep my mouth shut for the duration of the event and therefore came out the winner.
This was an formative experience for me, but I didn’t begin to think about it as such until many years later, when I entered the mental health field. There are several lessons to be drawn from this incident:
- First, no matter what the test is, or claims to be, the interpreter is still human, and it is this eminently fallible human being who will draw conclusions from the results. As this vignette illustrates, at least some of these conclusions will be based on nonsensical or at least unsupported assumptions.
- Even the most “objective” test only yields a set of numbers. Reread point No. 1.
- The so-called projective tests (e.g. the Rorschach inkblot test, draw-a-person/house/tree, the Thematic Apperception Test or TAT) have minimal empirical backing. It has been said that projective test results say more about the interpreters than the subjects.
- Even tests that are touted to be very reliable may have limited validity. (Reliability = how consistently does the test yield its results; validity = “best available approximation to the truth or falsity of a given inference, proposition or conclusion.” [Cook & Campbell, 1979])
- At the end of the day, the tester, a fallible human being, is still observing the client in the test environment, and it’s a good question how much the observation influences the interpretation of test results.
- Last but not least – the mental health professions have been extensively – unacceptably – politicized. It’s easy to use psychological tests (and diagnoses, too!) to pathologize people who, for one reason or another, are disapproved of. Regrettably, I have observed too much of this, but that’s another story.
Conclusion: caveat emptor. Never trust psychological tests without a lot of confirmatory evidence from other sources. This is particularly important when mental health practitioners do not spend enough time with clients. And never accept test results and diagnoses that suggest pathology without considering non-pathological explanations for 6he object of your concern.