Lessening the Reliance on VaR: It's About Time

Taking a new approach to quantifying risk: it isn't just for Taleb zealots any more. And when such a move hits mainstream, you know something big is afoot. Bloomberg ran a pretty informative article today on the shift in risk management practices among investment banks, and how a firm's stated VaR (Value at Risk) did not necessarily relate to how badly they were hit in the recent credit crunch:

Goldman Sachs Group Inc., the firm with the highest nominal VaR, was the sole investment bank to report record earnings in the fourth quarter, while New York-based Merrill, which had the second-lowest nominal VaR of the five biggest U.S. securities firms, posted a $9.8 billion loss for the last three months of 2007, the biggest in its 94-year history.

I find this statistical artifact fairly humorous. It's not that VaR is worthless; it just needs to be seen for what it is. A quick snapshot of book-wide risk assuming normal markets. This is a number risk managers and business heads should have. But if, and only if, it is augmented by far more detailed analyses that take into account non-normally distributed market movements, skyrocketing correlations in gapping markets and shocks that are far beyond those witnessed in even the past 30 years. Taleb has a quote in the article that makes the point quite poignantly:

``Finance is an area that's dominated by rare events,'' said Nassim Taleb, a research professor at London Business School and former options trader. ``The tools we have in quantitative finance do not work in what I call the `Black Swan' domain.''

While I'm sure we've all had our fill of Taleb, the Black Swan, Fooled by Randomness, etc., his points are well-taken and have certainly been borne out over the past decade. I remember a week in Q2 2004 while I was running DB Advisors when we had three days - in a row - where our portfolio moved beyond the 95% confidence interval. Three days in a row of moves exceeding two standard deviations from the mean - talk about a black swan! But it happened. And it happened after one of the greatest quarters in my lifetime, when almost every trading strategy worked beautifully. And then - plop. But these things happen. Far more frequently then ordinary quantitative finance techniques would indicate.

And even when using stress tests and breaking away from the limitations of normally distributed outcomes, one needs to be very, very careful. Here are a few great quips from the Bloomberg article that drive the point home:

The other risk tool commonly used by securities firms, known as stress testing or scenario analysis, also failed to prepare the industry for the plummeting value of AAA-rated securities that had previously been deemed the most creditworthy, he said.          

``Stress tests are only as good or as predictive as the scenarios used and in many cases the scenarios that played out were much more severe than people anticipated,'' (Ed) Hida said. ``One lesson learned is that these stress tests should be broader, should consider more scenarios.''                  

(Colm) Kelleher, who became Morgan Stanley's CFO in October, explained the flaw in the firm's stress testing in a Dec. 19 interview, the day the company reported its first unprofitable quarter.           

``Our assumptions included what at the time was deemed to be a worst-case scenario,'' he said. ``History has proven that the worst-case scenario was not the worst case.''

So the banks are moving in the right direction. It's about time. And not just for their own risk management purposes, but for investor disclosure. Because don't you want to know the true level and types of risks embedded in the books of the firms whose stocks you own? I sure do. And it looks like we are getting closer to that day. Thankfully.
           

Comments

[rob davies]

the 99% confidence is 2.33 sima move correct?
so a 3 sigma move would be 99.8 confidence interval?
also where can i read about var and calculations with numbers examples in easy plain english?

[Yaser Anwar]

A great read on VaR related issues

http://www.fooledbyrandomness.com/jorion.html

[Greg Battle]

STS: You are right. It's very difficult to model how a "run for the exit/entrance" amplifies the impact of any event and how events beget events.

The idea that we can compare one firm's VaR with another is complete baloney. VaR says nothing about the cultural and operational risks involved at any firm. Have you ever seen hubris, greed or discipline as a factor in a VaR model? How about computer hacking? ;-)

All this VaR-bage is one giant pacifier for those who cannot accept uncertainty for what it is. I'd argue that two and three sigma events like those that Roger saw in '04 happen in the market every day, you just have to look to find them.

[RBH]

Years ago, around the time of the LTCM collapse, a currency trader named DeRosa made a prescient remark: "VAR is a lighthouse for the soon-to-be shipwrecked."

VAR is a snare and a deception as a risk estimator: it fails worst when one needs it most.

Roger's observation of a three-day run of moves outside a 95% confidence interval is not an anomaly. Depending on the kind of VAR estimation technique (e.g., delta-normal) and the composition of a portfolio, the number of outliers (days outside any given confidence interval) in a decent-sized moving window (say, 100 days) can vary by nearly an order of magnitude.

[Yaser Anwar]

Great article- thank you very much!

[STS]

Correlation is key. Fat tails happen because above certain thresholds (percolation of fear across tight relationship networks -- eg "worried calls from trusted buddies") independence no longer holds and the nice cancellations which drive the central limit theorem stop happening. On a good day, nobody is looking over their shoulder at what "everybody" else is doing, because there is no "everybody." There is no mob in the street/traffic jam/etc. But sometimes things go critical and "everybody" is doing the same thing (honking the horn, staring at the burning building, etc.).