Back into the data mine.

Jason Zweig of the Wall Street Journal is the co-author of The Intelligent Investor: The Definitive Book on Value Investing. A Book of Practical Counsel, the one investment book everyone should have. Ben Graham's original version was published in 1949, the new one is co-authored with Warren Buffet, so you know this is quality merchandise. It is straightforward, gimmick-free advice, with no scent of dubious "get rich quick" formulas.

In contrast, a strong contender for the mother of all gimmicky, dubious, get-rich formulas is found in "Stupid Data Miner Tricks", chapter 6 of the Nerds on Wall Street book, which conclusively demonstrates the utterly meaningless and accidental correlations between butter production in Bangladesh and US stock market returns over a twelve year period. Throw in a couple of other bogus variables, and you get to a regression that "explains" 99% of the market returns over the period, but means nothing.  

Jason did a story in the August 8 WSJ, Data Mining Isn't a Good Bet For Stock-Market Predictions, that revisited the data mine, and included a four minute interview on the poster child for bad quant behavior.

Personal finance columnist Jason Zweig interviews
David Leinweber, author of "Nerds on Wall Street."


The Journal wanted to put in the charts showing the butter numbers along with the S&P, but they have a policy of only using charts for which they have the underlying data, and the 14 year old CD-ROM that contained the Bangladeshi dairy data had gone missing.

Berkeley student Max Dama went in search of the Bangladeshi butter series, to get the WSJ some data in time for the deadline. Alas, some things, like 20 year old butter numbers, are hard to find even on the web*.  In a display of true data miner machismo, Max shifted his sights to mine elsewhere and found a NEW AND IMPROVED highly predictive (but useless) leading indicator for the US stock market.

By data mining we've found that the number of buried treasures discovered in England and Wales is predictive of the Dow Jones Average 1 year later. 92% of the variance, i.e. correlation=.9187, of the DJIA is "explained" by the treasure discoveries.  (In compliance with the Journal's admirable standard, the data and source details are here.)

 

 
Note to World: This is why you should hire Berkeley students.
_______
* Butter in Bangladesh update. The old data is still AWOL, but the UN does cough up this rather nasty butter data. Unlike the original data, there are lots of missing values, and it shows imports and exports, instead of production. FYI, Bangladesh imports about a thousand times more butter than it exports. The investment applications of the Bangadeshi Butter Balance are left for further research.

Comments

[Fatima]

Such articles do help to liberalize and I want to thank you for that! But to understand the details, one should read a lot of literature, which is not enough in the internet. To make the search easier, you may http://www.pdfqueen.com use pdf ebooks search engine

[Oxy]

Once again: stock markets are systems with short term positive feedback - such systems simply have to oscillate on on their own.

[Buy Online Rx]

Given how bad we looked at trial, do we even want to appeal the Witt case? If we don't appeal Witt, though, have we in effect conceded the whole ball game? Will a lame duck session pull our butts out of the fire? This is really no fun.

[mens health]

So don't pick on the poor kings just because they're the only one smart enough to fool everyone for the entire length of the novel into believing they're the good guys.

[ordinateur pc portable]

one should read a lot of literature, which is not enough in the internet.

[labatterie]

This is really no fun.

[vibram five fingers]

a strong contender for the mother of all gimmicky, dubious, get-rich formulas is found in "Stupid Data Miner Tricks"

[dr dre headphones]

Once again: stock markets are systems with short term positive feedback - such systems simply have to oscillate on on their own.

[office 2007 key]

a lot of literature, which is not enough in the internet.liberalize and I want to thank you for that! But to understand the details, one should read a lot of literature, which is not enough in the internet