Statistical Significance

In a world of uncertainty it is nice to have the illusion that something is solid. We spend a lot of time in our introductory quantitative methods courses teaching our students about statistical techniques that were designed essentially for situations where making a decision is the object of the exercise. Reject or fail to reject the null with such and such a probability of making a reject decision when the null is in fact correct. What could be more solid than that? Unfortunately most of the time, at least in the social sciences, decision making is largely irrelevant and a judgment about statistical significance is only one of the things that should go into a proper evaluation of the evidence for or against a particular scientific proposition. Regrettably scientific institutions, like journals, often get into the hands of people who don't appear terribly thoughtful about the publication criteria they establish and enforce. Their psychological need for what they think of as clear rules often gets the better of their scientific judgment - at least that is the charitable interpretation. For why paying too much attention to significance levels isn't a great idea follow this link http://www.demogr.mpg.de/papers/working/wp-2007-037.pdf