Andrew Gelman: The problems with p-values are not just with p-values

The problems with p-values are not just with p-values:



Andrew Gelman says:

"Ultimately the problem is not with p-values but with null-hypothesis significance testing, that parody of falsificationism in which straw-man null hypothesis A is rejected and this is taken as evidence in favor of preferred alternative B (see Gelman, 2014). Whenever this sort of reasoning is being done, the problems discussed above will arise. Confidence intervals, credible intervals, Bayes factors, cross-validation: you name the method, it can and will be twisted, even if inadvertently, to create the appearance of strong evidence where none exists.

What, then, can and should be done? I agree with the ASA statement’s final paragraph, which emphasizes the importance of design, understanding, and context—and I would also add measurement to that list."

He concludes that there are two issues:  taking dataset and the statistical method as given, rather than these being part of the process of analyis; seeing statistics as a process that translates random nature into certainty.

'via Blog this'