Attacks on P-Value
Attacks on P-Value
About
While attacks on the usage of P value is nothing new recent attacks are becoming more serious. They come from more and more mainstream source with a lot of them calling to stop the usage of the significance testing altogether
These attacks are in general blaming P-Value for the naturally tentative trial and error processes of scientific discoveries and presuming that banning the P-Value would make the process cleaner and less error prone.
However tentative, the skeptical scientists still have to form unambiguous opinions, proximately to move forward in their investigations and ultimately to present results to the wider community.
For obvious reasons they have to constantly maintain the balance between false positive and false negative errors.
P-value can help in controlling and maintaining this balance and banning P-Value and other statistical tests will not help in maintaining this balance.
P-Value
In null hypothesis significance testing, the p-value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is correct. A very small p-value means that such an extreme observed outcome would be very unlikely under the null hypothesis.
Null Hypothesis Significance Testing
Null Hypothesis Significance Testing is a common statistical test to see if your research findings are statistically interesting.
The Attacks on P-Value
In recent times there have been numerous attacks in the mainstream media few of which are listed below.
Benjamin et al (2018) in Nature Human Behaviour - It was simply suggested that we stop using 0.05 cutoff and start using 0.005 instead.
Amrhein et al (2019) in Nature - They were calling for the dismantling of significance testing. According to them we should not call "statistically significant" and "not statistically significant" in general and we should "quit categorizing". They do not mind confidence intervals but are against using them as basis for claiming statistically significant results.
The journal Epidemiology - It bans the P-value altogether, but not confidence intervals. This in fact has been its practice since its founding in 1990.
The journal Basic and Applied Social Psychology - It had banned the P-value, or more generally the Null Hypothesis Significance Testing Procedure since 2015. They claim P-value and confidence interval suffer from the "inverse inference problem" and hence are invalid. However they reserve the right to make case by case judgment for Bayesian Procedures.
Where Is P-Value Unavoidable?
Kaplan-Meier survival curves - These are common plots in medical applications
Wilcoxon and non-parametric rank tests
Trend tests across an ordinal axis
Any test with more than one degree of freedom
Manhattan Plot
Conclusion
According to me banning P-value will cause a lot of issues for lot of statistical applications and is mostly motivated by statisticians from the Bayesian school of thought as theoretically the P-value violates the likelihood principle and hence they suggest Bayes factor as an alternative.
References
Amrhein, V., Greenland, S. and McShane, B. (2019). Scientists rise up against statistical significance. Nature 567(7748): 305-307.
Amrhein, V., Trafimow, D. and Greenland, S. (2019). Inferential statistics as descriptive statistics: There is no replication crisis if we don’t expect replication. The American Statistician 73: 262–270.
Benjamin. D. et al. (2018). Redefine statistical significance. Nature Human Behaviour 2: 6–10.
Efron, B. and Tibshirani, R. (1993). An introduction to the bootstrap. Boca Raton: Chapman and Hall.
Fisher, R. A. (1926). Arrangement of field experiments. Journal of Ministry of Agriculture XXXIII: 503-13.
Fraser, D.A.S. (2011). Is Bayes Posterior a quick and dirty confidence. Statistical Science 26: 299-316.
Guilfoyle, J.R., Struthers, W., van Monsjou, E. and Shoikhedbrod, A. (2019). Sorry is the hardest word to say: The role of self-control in apologizing. Basic and Applied Social Psychology 41: 72-90.
Kraemer, H.C. (2019). Is it time to ban the P-value? JAMA Psychiatry. Published online August 7, 2019.
Mack, M.J. et al. 2019. Transcatheter aortic-valve replacement with a balloon-expandable valve in low-risk patients. New England Journal of Medicine 380: 1695-1705.
Pawitan, Y. (2001). In All Likelihood. Oxford: Oxford University Press.
Schweder, T. and Hjort, N.L. (2016). Confidence, Likelihood and Probability. Cambridge: Cambridge University Press.
Ziliak, S. and McCloskey, D. (2008). The Cult of Statistical Significance. Ann Arbor: University of Michigan Press.
Pawitan, Yudi. “Defending the P-Value.” ArXiv.Org, 2020.
Subscribe
If you were forwarded BrainDump you can subscribe below.
Share
If you want to share BrainDump you can do so from below.
If you want to share this post you can do so below.
If you want to comment on this post you can do so below.