IQ vs RQ & Research Study on Correctness vs Confidence

IQ vs RQ

Keith Stanovich, a professor of applied psychology at the University of Toronto, distinguishes between intelligence quotient (IQ) and rationality quotient (RQ).

Intelligence Quotient (IQ): Psychologists measure IQ through specific tests, including the Wechsler Adult Intelligence Scale, and it correlates highly with standardized tests such as the SAT.

IQ measures something real, and it is associated with certain outcomes. For example, thirteen-year-old children who scored in the top decile of the top percent (99.9th percentile) on the math section of the SAT were eighteen times more likely to earn a doctorate degree in math or science than children who scored in the bottom decile of the top percent (99.1st percentile).

Rational Quotient (RQ): is the ability to think rationally and, as a consequence, to make good decisions. Whereas we generally think of intelligence and rationality as going together, Stanovich’s work shows that the correlation coefficient between IQ and RQ is relatively low at .20 to .35. IQ tests are not designed to capture the thinking that leads to judicious decisions.

Stanovich laments that almost all societies are focused on intelligence when the costs of irrational behavior are so high.

But you can pick out the signatures of rational thinking if you are alert to them. According to Stanovich, they include adaptive behavioral acts, efficient behavioral regulation, sensible goal prioritization, reflectivity, and the proper treatment of evidence.

Warren Buffett, chairman and chief executive officer (CEO) of Berkshire Hathaway, equates IQ to the horsepower of an engine and RQ to the output. We all know people who are high on IQ but average or low on RQ. Their efficiency is poor. There are others without dazzling IQs but who consistently make sound decisions. They are highly efficient.

How I got here is pretty simple in my case. It’s not IQ, I’m sure you’ll be glad to hear. The big thing is rationality. I always look at IQ and talent as representing the horsepower of the motor, but that the output—the efficiency with which that motor works—depends on rationality. A lot of people start out with 400-horsepower motors but only get a hundred horsepower of output. It’s way better to have a 200-horsepower motor and get it all into output.

Research Study of Correctness vs Confidence

Here we present the results of a classic calibration test. Subjects who participated went to the website, http://confidence.success-equation.com, and saw 50 true-false questions. Exhibit 3 is a screenshot of the site.

The subjects then answered either true or false and were asked to register a probability of correctness, from 50 to 100 percent, in increments of 10 percentage points. If you have no idea whether the answer is true or false you should select an answer at random and enter “50%” as your probability of correctness. If you are certain of the answer you provide, you click “100%.”

We accessed the results of 1,985 participants, all of whom were anonymous.

Results of Study

Below is a distribution of someone’s mean correctness vs their confidence.

Note that proper calibration does not require being right all of the time (the dot in the upper right-hand corner indicates that either someone cheated or a deity took the test) but rather being close to the 45-degree line. It’s knowing what you know and knowing what you don’t know.

The second exhibit shows how many participants were overconfident, underconfident, and well-calibrated. We define well calibrated as a subjective probability within 1 percentage point of the percent correct.

Based on that criterion, 82.7 percent of the participants were overconfident (subjective confidence exceeded actual percent correct), 11.7 percent were underconfident (subjective confidence less than actual percent correct), and only 5.6 percent were well calibrated.