Thursday, May 14, 2009

Black swans in finance, pharmaceuticals, and food safety

Back in January, Joe Nocera, had a piece in the Sunday NY Times Magazine called "Risk Management," where he attempted to explain what exactly caused the Wall Street meltdown. Nocera takes a close look at a financial tool called VaR -- Value at Risk which was developed by quants at JPMorgan in the early 1990s.

"[VaR] measure the boundaries of risk in a portfolio over short durations, assuming a "normal" market. For instance, if you have $50 million of weekly VaR, that means that over the course of the next week, there is a 99 percent chance that your portfolio won't lose more than $50 million.

VaR became extremely popular as JPMorgan made the decision to give the tool away. Over time the international Basel Committee on Banking Supervision ruled that "banks could rely on their own internal VaR calculation to set their capital requirements" and the Securities and Exchange Commission "mandated that financial firms would have to disclose their risk to investors -- and VaR became the de facto measure." VaR became the standard by which Wall Street was measured. Paradoxically, as more firms adopted VaR -- executives pushed their firms to take on even greater risk -- nobody wanted to be left out of the gold rush ("At the height of the bubble, there was so much money to be made that any firm that pulled back because it was nervous about risk would forsake huge short-term gains and lose out to less cautious rivals.")

The problem with VaR was several-fold. 1.) It was poorly built -- current risks were modeled on the stock market performance of just the last few years -- a period characterized by an expanding real estate bubble and irrational exuberance in the market. 2.) VaR is correct 99% of the time -- but what about that other 1% of the time? VaR has no ability to predict or prevent the 1% disaster scenario -- the so called "black swan" where an unanticipated risk destroys your entire company.

The fact that you are not likely to lose more than a certain amount 99 percent of the time tells you absolutely nothing about what could happen the other 1 percent of the time. You could lose $51 million instead of $50 million -- no big deal. You could also lose billions and go out of business. VaR has no way of measuring which it will be."

The 1% disaster scenario was named "the black swan" by Nassim Nicholas Taleb who wrote a book called, The Black Swan: The Impact of the Highly Improbable. 1% isn't zero -- there are 252 trading days a year -- so sooner or later a black swan is going to show up. And if you are resting easy relying on the fact that VaR has you covered (as most Wall Street firms did over the last several years) -- your company is gonna be in trouble when the inevitable black swan hits.

Starting in the spring of 2007, black swans started to appear everywhere. The housing bubble popped. Mortgage backed securities were junk. Credit default swaps (basically insurance policies that bet that a company won't go under) were worthless. AAA credit ratings were a lie. Hedge funds started to collapse. AIG -- which was underwriting much of the worldwide risk -- had to be taken over by the federal government because it too was bankrupt. 1% of the time happened -- and in an increasingly interconnected global economy -- when it does it takes the entire world economy down with it.

But here's the thing -- black swans aren't just a Wall Street phenomenon. I bet black swans are everywhere. What if many of the biggest health and safety disasters all around us are just the result of black swans in other industries. For example, what if:

99% of the time the anti-depressant works as prescribed, the other 1% of the time the kid goes on a killing spree and shoots up the school.

99% of the time the vaccine prevents the dreaded disease as promised. The other 1% of the time the kid develops autism and is locked inside his own body for the rest of his life.

99% of the time the aspartame makes our food sweet and delicious. The other 1% of the time (or even the other .1% of the time!) it causes an autoimmune disorder that destroys someone's quality of life.

99% of the time, the pollution from the factory seems harmless. The other 1% of the time it causes MS or Lou Gehrig's disease or leukemia.

99% of the time the factory farm produces some damn good bacon. The other 1% of the time it produces the swine flu.

We live in a world where 99% percent accuracy is deemed "good enough" in finance, pharmaceuticals, environmental protection, food safety, etc. And the truth is, it's not. If Boeing had a 99% success rate -- they would be out of business in less than a year. No one would ever fly because their planes would be dropping out of the sky all of the time.

1% is actually a catastrophically high failure rate. And yet we tend to measure statistical significance only to a level of 5% or 1% or .1% and act like we've got things covered. Statistical significance at 5%, 1% or even .1% is a good enough model if we are playing horseshoes. But when actual human lives are at stake (as they are in almost any decision involving medicine or health and safety) it would seem down right irresponsible not to measure statical significance to a level of at least .0001% (99.9999% certainty or rather a one in 1 million chance of the event occurring). It's like our statistical measures are built for an early more innocent age where shit just happened and we didn't know why. But these days, it seems unconscionable to walk around acting like the 1% scenario will never happen -- when we know it happens all the time.

Update #1: There's something odd about the Joe Nocera article that's still bugging me. Nocera is all busy pondering the koan of whether:
VaR and the other risk models Wall Street relies on could have helped prevent the financial crisis if only Wall Street paid better attention to them? Or did Wall Street's reliance on them help lead us into the abyss?
Look, it's an interesting question -- said differently was it the Hal 9000 or the human element that screwed things up?

But it seems to me -- that's not the question at all! The real question is: why was VaR modeled on only a couple years worth of boom-time era data (and based on the ridiculous assumption that "housing prices never go down")? Any 5th grader in the country could tell you that a reliable VaR would have to include all of the data from the present back until the 1920s -- so that one would have at least one great depression and several deep recessions built into the model as probable scenarios. And you mean to tell me that NO ONE on Wall Street -- with all of their MBAs and fancy math degrees, didn't figure that out? Nocera has this "aw gee golly shucks" approach to this matter -- as if it was just an understandable oversight. It wasn't an oversight. The math and fancy equations of VaR were the ruse -- designed to fool the public into thinking their investments were safe. But surely the quants were in on the scam. Surely the math geniuses devising the models knew they were bunk -- and just a marketing tool -- not a risk management device. And yet Joe Nocera and Alan Greenspan and Timothy Geithner and Barack Obama are all like, "oh gee golly shucks that probably wasn't the best model" rather than pursuing the obvious criminality of the entire enterprise.

No comments: