Tuesday, February 24, 2009

If I had a hammer...

It's not uncommon for mathematics to be misused in the medical research community. In fact, I might even go so far as to state that mathematics and/or statistics are commonly misused in research. From my experience, this usually comes from a desire to have a "black box", where you can just plug in a patient population and get out a published paper(look ma, no thinking!). You see a fancy Whiz! Bang! mathematical technique designed to measure something about people bouncing a ball, and you say "Hey neat! Can I use that on people bouncing a ball on the stairs?" "Well... sure... but you might run into problems if..." "Great! Thanks!" And after a couple iterations you end up having chimpanzees climbing stairs underwater in another solar system and its a big mess because all the underlying assumptions of the model have been violated... but hey, it'll still get published! And thus you get these cascading errors where people really have no concept of what's really going on with their analysis, and all that's important is that "other people are getting published doing it". Hell, people even make entire careers out of having a single hammer in their toolbox and hammering everything with it. I've always felt this sort of "math abuse" was a huge problem... millions of research dollars funneled down black holes and countless man hours chasing spurious conclusions derived from improper methodology... and it seemed to me to be something mainly an issue plaguing academic research.

But then I read an interesting article by Felix Salmon in Wired called "Recipe for Disaster: The Formula That Killed Wall Street" that put my concerns into perspective. It's about some model that some dude on Wall Street made up to price opaque securities:

For five years, Li's formula, known as a Gaussian copula function, looked like an unambiguously positive breakthrough, a piece of financial technology that allowed hugely complex risks to be modeled with more ease and accuracy than ever before. With his brilliant spark of mathematical legerdemain, Li made it possible for traders to sell vast quantities of new securities, expanding financial markets to unimaginable levels.


I don't mean to post any spoilers, but let me just say up front that Wall Street dies at the end. Now, is that all Li and his formula's fault? Well no... as the article eventually makes clear. Any model has assumptions/parameters/initial conditions and what-have-you that have to be fully understood before they can be appropriately applied. Indeed, as an article from waaaay back in 2005 concluded the last time this formula's misbegotten reputation as a magic bullet was being called into question:
This wasn't really the fault of the model, which was designed mainly to help price the tranches, not to make predictions. True, the model had assumed the various credit curves would move in sync. But it also allowed for investors to adjust this assumption -- an option that some, wittingly or not, ignored.
As far as I can tell, nobody is calling into question Li's math... or saying he flubbed something or made it all up... they're upset that their misapplication and misunderstanding of his work only made them fabulous amounts of wealth for a few years... before destroying our entire economy.

Whoops? I'd have more Schadenfreude about chickens coming home to roost if we all didn't end up paying for their mistakes.