Type I, II Errors in Recruiting

“Ignorance is preferable to error, and he is less remote from the truth who believes nothing than he who believes what is wrong.”—Thomas Jefferson

Which kind of error is worse: accepting an applicant you should have rejected, or rejecting one you should have accepted? If you work for a usually super-cautious and lobbyist-resistant organization like the FDA (Food and Drug Administration), the answer would appear to be simple: committing what is called a “Type I” error is worse, which, in the case of the FDA, means it’s a bigger mistake to accept a drug or a staff member you should have rejected.

Error Pressure

At least psychologically, for an administrator overseeing drug approval, the pressure to avoid “false positives” (Type I errors), viz., accepting the claim that a drug is safe and efficacious when in fact it isn’t, will be much greater than the pressure to avoid “false negatives” (Type II errors), viz., rejecting a drug that is in fact safe and efficacious.

One reason for the likely difference in pressure at the desk of an FDA administrator is that the false positives will lead to much bigger lawsuits, liability judgments and ugly press coverage than the false negatives, reflecting the fact that, in general, a “sin of commission” is viewed as more serious, more deliberate and more readily discovered than a “sin of omission”. One reason for this is that intent is, in general, more easily and commonly proved or imputed with sins of commission than with sins of omission. For that latter reason, negligent homicide (a sin of omission) customarily has less severe penalties than premeditated murder (a sin of commission).

The same is true for any embassy: in the absence of offsetting bribery or other conspiracy benefits, granting a visa request that shouldn’t be approved, e.g., from a known terrorist, is much worse—in terms of the consequences for the public (and then for the visa department).

In terms of a cautious recruiter’s or employee’s desire to “not make waves”, it would, on first reflection, seem that a Type I error (false positive) creates a much, much bigger and potentially disastrous wave than a Type II error (false negative).  For instance, just ask yourself this—who would make you angrier: someone who gave you a bad stock tip (Type I, false positive), or someone who failed to give you a good one (Type II, false negative)?

Statistical Flip-flop

On the other hand, what about the person who, with the best of intentions, mistakenly shouts “Fire!” in a crowded theater? Is that false positive (the abnormal condition of fire being falsely reported) worse than the false negative (the abnormal condition of a real fire going unreported), Type II error of notshouting “Fire!” when in fact there is one? Perhaps a well-intentioned “boy who cries wolf” even though there wasn’t one (Type I, false positive error) is a better boy than the one who was silent after thinking he spotted a wolf among the sheep (Type II, false negative).

In these two examples, it clearly seems to be the case that the Type II/false negative error (rejecting the warning of the boy when there really is a wolf) is more serious than the Type I/false positive (accepting the warning of the boy when there isn’t a wolf)—precisely the reverse of the FDA and embassy scenarios in which a Type I error is worse than a Type II error.

Null as Dull

Sometimes such reversals of the expectation that a Type I error is worse than a Type II are the result of the wording regarding what is to count as the “null hypothesis”—what is called the “state of nature”, the most likely and least interesting hypothesis, e.g., that a particular experimental drug will not succeed where all others have failed, that a visa applicant is not a terrorist, or that the applicant in your office is not“the one”.

For example, normally in recruiting, the null hypothesis would be that the applicant is notoutstanding—the hypothesis likeliest to be a priori true and the least interesting. By contrast, the “alternative hypothesis” to be tested is the much more interesting proposition that the applicant is outstanding. But, if my null hypothesis is, instead, that a particular job applicant is better than the average applicant, the “alternative hypothesis” is that heis not better than the others, i.e., is the same or worse than them.

Hence, a false positive test result in this latter instance would suggest that he is no better than the others, when in fact he is. Therefore, as a consequence of accepting this false positive result, as a recruiter, I would rejectthe applicant when I shouldn’t have, whereas in the normal case in which the null hypothesis is that the applicant is notoutstanding, a false positive would lead to accepting the applicant when I shouldn’t.

In other words, because of the wording of the null hypothesis, in cases like this one, a Type I error leads to erroneously rejecting an applicant rather than to erroneously acceptinghim, as it would in the FDA or embassy cases.

Much More Than Mere Changes in Wording

However, issues related to wording and rewording of hypotheses aside, the fact remains that there is no automatic assurance that a Type I error is more serious or disastrous than a Type II, or vice versa. Even the difference in the probabilities between the two types of errors is not a reliable measure of the seriousness of the error—apart from mathematical seriousness.

In addition to the “mathematical seriousness” measured in terms of probability, the second, often more important measure of the seriousness of the error is the real-world consequences of the mistake.

Infernos and Casinos

A simple example will make this clear: Suppose you are in a movie theater inferno and you have to run for the left exit or the right exit, when you see smoke seeping from only the left one, but don’t know which, if either, is safe, while suspecting that neither is. So, you instantly and unconsciously frame your null hypothesis: “Neither exit is safe.” But seeing some smoke on the left, you run right, accepting one of the three alternative, but unequally weighted hypotheses, namely, “the right exit is safe”, rather than “the left exit is safe” or “staying put is safe”.

As it turns out, the only reason there is no smoke on the right is that a partially collapsed ceiling has smothered it, whereas the left exit was in fact usable. When you run into that right hallway, the rest of the ceiling collapses on you. You become a victim of your accepting a false positive result, having tested the exit the way personnel testing tests for excellence and having overrated your choice.

Clearly, in this case, the consequences of the (fatal) mistake completely override the probability that your choice was wrong, especially given the evidence that suggested that the right exit was safer than the left and staying where you were.

This scenario has parallels in recruiting: You have to make a decision about Applicant X. You don’t want to accept her if it’s going to be a mistake; but then you don’t want to reject her, if that’s going to be a mistake. You think about the probability that the respective choices are mistakes. You guess that there’s an 80 percent chance she’s the right candidate. So, that means you must also estimate there’s a 20 percent chance she isn’t.

If the probability of being wrong were all that mattered, it would be a bigger mistake to reject her than to accept her. You’d play the odds. But the odds are not the only consideration. The second consideration, as noted above, is what the consequences of being wrong are going to be.

Suppose that, if you are wrong in hiring her, despite the odds favoring doing so, you will be fired. If you don’t hire her, you bury her resume and move on to the next applicant.

How to Decide?

So, in practical terms, what are you supposed to do when making such hiring recommendations or decisions?  You shouldn’t inflexibly always interpret a Type I/false positive error as worse than a Type II/false negative, or vice versa. Nor should you regard the choice between committing one or the other as a toss-up.

Instead, you have to balance the probability of making the wrong choice with the consequences of making that wrong choice. That’s exactly the way many bets are placed at the FDA or in  casinos.

The wisdom of betting at a casino table on getting a king to complete your straight depends not only on the odds of getting the king and not only on those odds plus the size of your bet, but also on whether you can afford the consequence of losing that bet or of forgoing it by folding, when winning would change your life.

What makes an understanding of the difference between Type I and Type II errors useful is that it can nudge you into considering in each specific instance which error is the costlier of the two and to evaluate the probabilities and (dis)utilities that underlie your assessment and decision, even if it must be on a case-by-case basis.

Your Self-Test and Mine

To reinforce your understanding of the differences between Type I and Type II errors and of the possible consequences of them, a final example should suffice:

1. Null hypothesis (Ho) = “There were better things to do than read this article”
2. Alternative hypothesis (H1) = “Nothing could have been better than reading this article”.
3. Type I error/false positive=accepting H1 when it is false
4. Type II error/false negative=rejecting H1 when it is true

If you now grasp the difference between Type I and Type II errors, you’ll understand that regardless of whether you accept  H1, or reject it…

… I lose.