Imagine if in the last ten years before the collapse, the huge failed financial and insurance institutions had had independent research units devoted to doing basic, open, and critical research on matters of relevance to the business, ethics, and future of these institutions. Might it have made a small difference for the better regarding the fate of these institutions?
[Full disclosure:] I make my living by doing basic research.
Combinatorics and more 
But, Gil,
The “huge failed financial and insurance institutions” did have an enormous amount of research “devoted to matters of relevance to the business, ethics, and future of these institutions”.
It is true that this research was mostly not “basic, open, or critical”, as you would like, but most likely not the lack of any of these traits was the key problem. It would seem that the incentives (to the researchers themselves as well as to those that employed them) are key here.
Er, no.
I come from the theoretical CS community and ended up at a Wall St. “prestigious institution”, and I got fired for pointing out precisely these problems.
Doesn’t work.
Dear Noam,
Probably to get researchers incentives right “independent” is of importance along basic, open, skeptical etc. (I do not know what type of “research” these companies did have.)
I agree that there are several reasons for skepticism regarding my thought. Basic independence academic research did not predict the crisis and cannot explain it or even characterize it as it happens now. I cannot see, even in principle, how independent basic research units can help, say the tobacco industry. I suppose this is why I only asked about “small difference for the better”. But perhaps independent, open, wide-scoped research units in huge firms can sometimes make small difference for the better, and give a good signal to clients and investors.
It is easy to blame “Wall Street institutions”. The US politicians on
the left and on the right have been
doing this recently. It is simplistic
and wrong. It is the banks that ended
up losing money, after all. They lost
money because the average Joe walked
away from his debt. The average Joe
did not directly suffer in this, but later
on he got fired from his job…
I would suggest looking at the “basic
research” that was carried in academia.
Academic departments in the US rushed
to set up math-finance programs.
They went on to hire faculty and started
churning out Masters and PhDs.
I would argue that research in math
finance was a fad, and was low quality.
Perhaps the academia should ask itself
why it was unable to address the really
important problems.
One answer is that these problems are
difficult. Very few people in academia
do research that is relevant and has
practical consequences. It is publish or
perish in academia and the so-called
applied disciplines are often remarkably
disconnected from reality.
Typically, such research has no aesthetic value (i.e. it is not pure math),
yet it will never lead to any practical applications.
info tech has represented a unique generational disjuncture — eg, the young are more skilled and thus have more authority than the old. there was a time when senior management was in fact senior, when the elders were valuable for their cautious decision-making style. the ascendancy of ever-changing computer know-how has necessarily devalued the value of experience.
as a database prog/analyst for a large bank i was accustomed to higher pay for comparatively less education and experience; also, since my business “customers” could not comprehend what i was doing, my only supervision came from other programmers who shared many of my youthful deficiencies in judgment.
the story of the advent of “program trading” (large-scale automated arbitrage) is instructive if it’s true — after a brief test, solomon bros started running an application to jump on small price discrepancies… the thing ran like a top with the exception of its user interface, which tended to lag the data a bit. a clerk was running the thing, hitting enter whenever it beeped or blinked, whatever… but one day the confirmations weren’t showing, so she kept pressing the key trying to get a response. meanwhile markets were experiencing an historic rally. turns out, solly made a hundred times more money than they had at what had seemed sensible trades. thus sensible was redefined.