Making Statistical Methods More Useful
Some Suggestions From a Case Study
Abstract
I present a critique of the methods used in a typical article. This leads to three broad conclusions about the conventional use of statistical methods. First, results are often reported in an unnecessarily obscure manner. Second, the null hypothesis testing paradigm is deeply flawed: Estimating the size of effects and citing confidence intervals or levels is usually better. Third, there are several issues, independent of the particular statistical concepts employed, which limit the value of any statistical approach—for example, difficulties of generalizing to different contexts and the weakness of some research in terms of the size of the effects found. The first two of these are easily remedied—I illustrate some of the possibilities by reanalyzing the data from the case study article—and the third means that in some contexts, a statistical approach may not be worthwhile. My case study is a management article, but similar problems arise in other social sciences.
Originalmente publicado en SAGE Open (2013, enero-marzo), 1-11. doi:10.1177/2158244013476873. Copyright 2013: Michael Wood. Traducido al español con permiso de los titulares de los derechos de autor.
References
Bayarri, M. J., & Berger, J. O. (2004). The interplay of Bayesian and frequentist analysis. Statistical Science, 19, 58-80.
Becker, T. E. (2005). Potential problems in the statistical/control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8, 274-289.
British Medical Journal. (2011). Research. Recuperado de http://resources.bmj.com/bmj/authors/types-of-article/research
Bolstad, W. M. (2004). Introduction to Bayesian statistics (2a ed.). Hoboken, NJ: Wiley.
Cashen, L. H., & Geiger, S. W. (2004). Statistical power and the testing of null hypotheses: A review of contemporary management research and recommendations for future studies. Organizational Research Methods, 7, 151-167.
Christy, R., & Wood, M. (1999). Researching possibilities in marketing. Qualitative Market Research, 2, 189-196.
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.
Cortina, J. M., & Folger, R. G. (1998). When is it acceptable to accept a null hypothesis: No way Jose? Organizational Research Methods, 1, 334-350.
Coulson, M., Healey, M., Fidler, F., & Cumming, G. (2010). Confidence intervals permit, but do not guarantee, better inference than statistical significance testing. Frontiers in Psychology, 1, 1-9.
Diaconis, P., & Efron, B. (1983, Mayo). Computer intensive methods in statistics. Scientific American, 248, 96-108.
Gardner, M., & Altman, D. G. (1986). Confidence intervals rather than P values: Estimation rather than hypothesis testing. British Medical Journal, 292, 746-750.
Gephart, R. P. J. (2004). Editor’s note: Qualitative research and the academy of management journal. Academy of Management Journal, 47, 454-462.
Glebbeek, A. C., & Bax, E. H. (2004). Is high employee turnover really harmful? An empirical test using company records. Academy of Management Journal, 47, 277-286.
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. (1998). ICH harmonized tripartite guideline: Statistical principles for clinical trials (E9). Recuperado desde http://www.ich.org
King, G. (1986). How not to lie with statistics: Avoiding common mistakes in quantitative political science. American Journal of Political Science, 30, 666-687.
Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
Lindsay, R. M. (1995). Reconsidering the status of tests of significance: An alternative criterion of adequacy. Accounting, Organizations and Society, 20, 35-53.
Mingers, J. (2006). A critique of statistical modelling in management science from a critical realist perspective: Its role within multimethodology. Journal of the Operational Research Society, 57, 202-219.
Morrison, D. E., & Henkel, R. E. (1970). The significance test controversy. Londres: Butterworths.
Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
Shaw, J. D., Gupta, N., & Delery, J. E. (2005). Alternative conceptualizations of the relationship between voluntary turnover and organizational performance. Academy of Management Journal, 48, 50-68.
Siebert, W. S., & Zubanov, N. (2009). Searching for the optimal level of employee turnover: A study of a large UK retail organization. Academy of Management Journal, 52, 294-313.
Simon, H. A. (1996). The sciences of the artificial. Cambridge, MA: MIT Press.
Simon, J. L. (1992). Resampling: The new statistics. Arlington, VA: Resampling Stats.
Taleb, N. N. (2008). The black swan: The impact of the highly improbable. Londres: Penguin.
Vandenberg, R. J. (2002). Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods, 5, 139-158.
Wood, M. (2002). Maths should not be hard: The case for making academic knowledge more palatable. Higher Education Review, 34, 3-19.
Wood, M. (2005). Bootstrapped confidence intervals as an approach to statistical inference. Organizational Research Methods, 8, 454-470.
Wood, M. (2012a). Bootstrapping confidence levels for hypotheses about regression models. Recuperado de http://arxiv.org/abs/0912.3880v4
Wood, M. (2012b). P values, confidence intervals, or confidence levels for hypotheses? Recuperado de http://arxiv.org/abs/0912.3878v4
Wood, M., Capon, N., & Kaye, M. (1998). User-friendly statistical concepts for process monitoring. Journal of the Operational Research Society, 49, 976-985.
Wood, M., Kaye, M., & Capon, N. (1999). The use of resampling for estimating control chart limits. Journal of the Operational Research Society, 50, 651-659.
Wood, M., & Christy, R. (1999). Sampling for possibilities. Quality & Quantity, 33, 185-202.
Yin, R. K. (2003). Case study research: Design and methods (3a ed.). Thousand Oaks, CA: SAGE.
Copyright (c) 2014 Michael Wood, D.Phil.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Los artículos emplean licencias de acceso abierto distribuido bajo los términos de la licencia de Creative Commons 4.0 (https://creativecommons.org/licenses/by-nc-nd/4.0/), la cual permite su uso,
distribución y reproducción de forma libre siempre y cuando el o los autores reciban el respectivo crédito.