The Wolf at the Door: A Parable about Ratings

By Bob Bruner-

Forty years had passed since the Wolf had had his last encounter with the Three Little Pigs. Now, he was running a debt rating service, focused on collateralized mortgage obligations (CMOs). The Three Pigs were principals in a financial institution that traded and invested in CMOs. The first pig specialized in mortgages on houses of straw; the second in houses of sticks; and the third in houses built of bricks. Letting bygones be bygones, the Three Pigs decided to seek the opinion of the Wolf regarding the credit quality of a bundle of CMOs that were offered to them for investment. The Wolf came to their door at the appointed hour. The following conversation ensued:

Wolf: In my opinion, these CMOs merit the highest rating, AAA.

1st Pig: Is this just an opinion, or is there some factual basis to support it?

Wolf: Sure, I did some research. I asked some friends, who think well of the mortgage loan originators. The originators have sure been growing rapidly. And I looked at the general default rate on CMOs. I judge these CMOs to be similar to the general pool of CMOs, and the default rate on all CMOs isn’t very high. Finally, I did look at the terms of two or three of the mortgages in each CMO—they looked fine.

2nd Pig: How do you know these CMOs aren’t AA-rated or lower? How do you tell the AAA’s apart from the others?

Wolf: You can pretty much tell just by looking at them. A top-rated CMO has a certain je ne sais quois. You know the very best quality when you see it.

3rd Pig: What, then, does your debt rating system measure? What does it predict? And how do you know it predicts accurately?

Wolf: Well, I’ve been in this business forty years and I almost never have had a bad experience with an AAA-rated CMO. My ratings measure credit quality, which predicts default risk.

On the basis of this exchange, the credulous Three Little Pigs decided to invest in the bundle of CMOs, which the Wolf had rated AAA. But starting in 2006, the default rate on the mortgages contained within the CMOs skyrocketed. The successive write-offs on these CMOs ultimately caused the collapse of the Three Little Pigs’ financial institution. The Wolf inadvertently succeeded in blowing down their house, as he had vowed years earlier.

What went wrong? The credit ratings provided by the Wolf were flawed. First, they were not objective and transparent; the Wolf’s biases and frame of mind influenced the ratings. The Wolf offered no framework of analysis. The ratings could not be reproduced by an impartial judge; they were just opinions.

Second, the Wolf did say that he had looked at two or three of the mortgages underlying the CMOs. But he could not attest that his sample was representative of the population. Every CMO contains hundreds or thousands of mortgages—simply to examine a few of the mortgages could hardly give assurance of the nature of the whole pool of mortgages.

Third, the credit rating had no proof of validity. The Wolf’s bland assertion that he “almost never” had a problem with AAA-rated CMOs is no proof of the highest credit quality. What constitutes a “problem” or a “bad experience”? If his rating is an indication of default risk, what is the probability of default associated with an AAA rating? What measures prove the validity of his claim to high credit quality?

Finally, the Wolf’s credit ratings did not differentiate credit quality to a significant degree. In statistics, a difference is significant if it could not be due to some low chance, such as one random occurrence in 100. A good system of credit ratings should meaningfully explain the difference in default risk between rating categories. But the Wolf offered nothing other than “gut feel” for explaining that difference. He could not really assert that his AAA rating was significantly different from his other possible debt ratings.

I offer this story as a way to stimulate your thinking about what might characterize a “good” rating. We’ve read a lot about bad ratings in recent months. I have former students and friends at the rating agencies and know that they are earnest people who are inconsistent with my characterization of the Wolf. It distressed me to see the rating agencies at the epicenter of the current financial crisis, for having issued AAA ratings on CMOs that subsequently tanked. A year ago, Floyd Norris at the New York Times wrote, “Rating agency downgrades do not destroy markets for corporate bonds, simply because enough information is disseminated that other analysts can reach their own conclusions. But the securitization markets collapsed when it became clear the rating agencies had been overly optimistic. When a security goes from AAA to junk within a few weeks, it does not inspire confidence in the rating process. Every financial disaster deserves a scapegoat, because someone must be blamed when bad investments are made. Such scapegoats are seldom without fault, but their venality can easily be overstated. Equity analysts and corporate crooks took the blame after the technology bubble burst. This time it could be the credit rating agencies.” Sure enough, a subsequent investigation by the SEC turned up some juicy emails that impair our confidence in credit ratings: one note said, “Let’s hope we are all wealthy and retired by the time this house of cards falters.” So, the SEC imposed new rules on the agencies earlier this month.

What is the place of B-schools in all of this? My story and the recent news coverage about ratings certainly bespeak the importance of at least four virtues, which B-schools can help to instill:

** intellectual rigor around concepts such as objectivity, representativeness, validity, and significance;

**critical thinking—the frame of mind to check assumptions, logic, and conclusions;

**integrity, the ability to test processes and outcomes against deep values held by individuals, firms, and society.

**candor, the capacity to speak truth to power;

We’ve been here before. The rash of business scandals surfaced around the turn of the decade because of the absence of these and other virtues. B-schools responded then. As best I can tell, they are responding in like fashion to the current crisis.

It seems inevitable that the ratings industry will change: more regulation; more transparency; more independence; and payment by investors rather than issuers—all of these are under active discussion today. Certainly, the rating methodologies will come under intense scrutiny in the avalanche of litigation to come. At the conclusion of the fairy tale of The Three Little Pigs, the Wolf attempts to attack the pigs by climbing down the chimney of the 3rd Little Pig; instead, he falls into a boiling cauldron in which he is cooked and thereafter eaten. This is rough justice, but no less than in the modern version, where the rating agencies land in a stew of government investigations and lawsuits that will last for years.

Posted by Robert Bruner at 12/28/2008 03:18:09 PM