Phil highlights some of the mistakes you can easily make if you use regression blindly:
Regression can really lead you astray – for example, this paper, of which I'm one of the co-authors (for contributing fielding analysis, not for the regression.) The regression result is quite unintuitive and counter to what everyone else has found.
Regression lends an aura of "correctness" to a study – you can say "controlling for parental income and education, we found that the number of ice cream cones eaten as a child explains 93% of the variation in career earnings" – but it doesn't mean that you found the right answer, or even, as Phil shows with triples in baseball, that you get the correct sign on your coefficients. (Perhaps the most egregious error of this sort was in the famous "Beer Makes you Smart" study.)
