Taleb and Model Fragility – NYU-Poly
June 17, 2011
I went along to spend a day in Brooklyn yesterday at NYU-Poly, now the engineering school of NYU containing the Department of Finance and Risk Engineering. The event was called the "The Post Crisis World of Finance" was sponsored by Capco.
First up was Nassim Taleb (he of Black Swan fame). His presentation was entitled "A Simple Heuristic to Assess Tail Exposure and Model Error". First time I had seen Nassim talk and like many of us he was an interesting mix of seeming nervousness and confidence whilst presenting. He started by saying that given the success and apparent accessibility to the public of his Black Swan book, he had a deficit to make up in unreadability in this presentation and his future books.
Nassim recommenced his on-going battle with proponents of Value at Risk (see earlier posts on VaR) and economists in general. He said that economics continues to be marred by the lack of any stochastic component within the models that most economists use and develop. He restated his view that economists change the world to fit their choice of model, rather than the other way round. He mentioned "The Bed of Procrustes" from Greek mythology in which a man who made his visitors fit his bed to perfection by either stretching them or cutting their limbs (good analogy but also good plug for his latest book too I guess)
He categorized the most common errors in economic models as follows:
- Linear risks/errors – these were rare but show themselves early in testing
- Missing variables – rare and usually gave rise to small effects (as an aside he mentioned that good models should not have too many variables)
- Missing 2nd order effects – very common, harder to detect and potentially very harmful
He gave a few real-life examples of 3 above such as a 10% increase in traffic on the roads could result in doubling journey times whilst a 10% reduction would deliver very little benefit. He targeted Heathrow airport in London, saying that landing there was an exercise in understanding a convex function in which you never arrive 2 hours early, but arriving 2 hours later than scheduled was relatively common.
He described the effects of convexity firstly in "English" (his words):
"Don't try to cross a river that is on average 4ft deep"
and secondly in "French" (again his words – maybe a dig at Anglo-Saxon mathematical comprehension or in praise of French mathematics/mathematicians? Probably both?):
"A convex function of an average is not the average of a convex function"
Nassim then progressed to show the fragility of VaR models and their sensitivity to estimates of volatility. He showed that a 10% estimate error in volatility could produce a massive error in VaR level calculated. His arguments here on model fragility reflected a lot of what he had proposed a while back on the conversion of debt to equity in order to reduce the fragility of the world's economy (see post).
His heuristic measure mentioned in the title was then described which is to peturb some input variable such as volatility by say 15%, 20% and 25%. If the 20% result is much worse than the average of the 15 and 25 ones then you have a fragile system and should be very wary of the results and conclusions you draw from your model. He acknowledged that this was only a heuristic but said that with complex systems/models a simple heuristic like this was both pragmatic and insightful. Overall he gave a very entertaining talk with something of practical value at the end.