Performance appraisal of leaders and managers is, like the weather, a favorite topic of casual discussion. Such appraisals tend to fall into three categories:
** Weak form. This asks how conditions have changed over time, on the leader’s watch. In 1984, Ronald Reagan’s campaign slogan against Michael Dukakis was “Are you better off than four years ago?” This is an easy assessment to make, though the judgments one reaches are likely to be colored heavily by influences outside of the leader’s control. Was the leader just lucky or smart? The weak form test doesn’t parse this out.
**Semi-strong form. This asks how performance compares to some benchmark. For instance, many investors in the stock market compare their returns to the S&P 500 Index. The use of a benchmark has the advantage of taking into account influences outside of one’s control. But choosing the right benchmark can be a challenge, a fact that has sprouted whole industries of investment consultants, HR consultants, and so on. Still, a semi-strong form assessment would trump a weak form assessment.
**Strong form. This asks how performance compares to what it would have been had the leader not been present. This assumes the existence of some attractive alternative leader: diligent, intelligent, well-informed, etc. The strong-form test tries to approximate what economists call the “cost of the lost opportunity.” Comparing an actual leader against the next best alternative captures not only the general benchmarks but also the attributes of other leaders like the one whose performance you are assessing. This kind of comparison is the gold standard in much of social science. Unfortunately, in many fields, actually observing a very close comparable is impossible. Often, the best one can do is speculate.
I am reminded of these bases of appraisal by the case of Alan Greenspan. He was the Chairman of the U.S. Federal Reserve System from 1987 to 2006 and arguably one of the most powerful people in the world. Since he retired so recently, the media have offered a steady diet of appraisals of his tenure as Chairman. How did he do? Nobel Laureate Milton Friedman called Greenspan “the most effective Chairman since the inception of the Fed.” Others think that Greenspan may have been just lucky since he did not have to deal with the severe economic shocks of some of his predecessors. Still others (such as libertarians and conspiracy theorists) have been hostile. Opinion on Greenspan covers the entire range.
The tough-minded practitioner should go beyond opinion to inspect the evidence. Performance appraisals are nothing if they aren’t fact-based. Ideally, one would test the facts in a way to eliminate the possibility that the outcome is merely due to chance.
A recent article by Yale Professor Ray Fisher offers a refreshing review of the evidence that approximates a strong form test—whether you are picky about the details of this test, it affords an evidence-based comparison of Greenspan against next-best alternatives. Fisher tests the performance of all five post-World War II Chairmen of the Fed against smart robots who are trying to minimize inflation and unemployment. Fisher’s findings suggest that Greenspan topped the field.
Research like this isn’t conclusive. After all, the decisions of the Fed Chairman will echo long after his retirement. Maybe it is too early to judge. And maybe someone will invent better robots against which to compare the live Chairmen. And maybe we expect more from Chairmen than low unemployment and inflation. But an evidence-based and test-based appraisal almost certainly moves us closer to the Truth.
It would be interesting to see such appraisals applied to celebrity executives such as Steve Jobs, Bill Gates, Jack Welch, Warren Buffett, Donald Trump, Rupert Murdoch, and Oprah Winfrey. We lionize these people largely on the basis of weak-form tests. How would they compare to the economists’ best alternative?
“What do you know?” “How do you know?” These are deep questions. The search for answers spawned the Enlightenment, research universities, statistical testing, and the field of epistemology. It is tempting to take a binary view of knowledge: either you know your stuff or you don’t. The three forms of performance appraisals illustrate that conclusions about performance and mastery can range from flimsy to rock-solid. I think tough-minded practitioners take a nuanced and pragmatic view: for them, the question is how well you know what you know. The simple answer is that the stronger the test, the more confident can be the conclusion.
Posted by Robert Bruner at 07/26/2007 10:50:13 AM