Will G Hopkins, Alan M Batterham Sportscience 22, 19-27, 2018 (sportsci.org/2018/mbivind.htm) Institute for Health and Sport, Victoria University, Melbourne, Australia; School of Health and Social Care, Teesside University, Middlesbrough, UK. Email. Magnitude-based inference (MBI) has again been subjected to detailed scrutiny by an establishment statistician in one of our top journals. Kristin Sainani's critique is on four fronts. First, she claims that the probabilistic statements in MBI, such as the treatment is possibly beneficial, are invalid, because these are Bayesian statements and MBI is not Bayesian. This claim is false, because MBI is Bayesian with a minimally informative prior, so the probabilities provided by MBI are objective trustworthy estimates of uncertainty in the true value. Sainani supports instead "qualitative judgments" of the lower and upper confidence limits, without realizing that the level of confidence renders such judgments quantitative, and they are in fact MBI. Secondly, she regards as "specious" the logic in MBI that there is no Type-I error when the true effect is trivial and the MBI outcome is likely substantial, because the effect is also unlikely trivial (e.g., with a probability of 0.06). But according to her logic, "specious" would also apply to failure to declare a Type-I error in null-hypothesis significance testing (NHST), when the true effect is zero and the outcome is non-significant (e.g., with a p value of 0.06). She shows that our definitions of error "wildly underestimate" Type-II error rates, but her estimates are based on the null hypothesis, which is no longer a trustworthy approach to inference. Thirdly, she highlights the high Type-I error rates for clinical MBI, yet these are comparable with those of NHST over a range of small sample sizes and trivial effect magnitudes, and they occur mostly with effects presented to the clinician or practitioner as only possibly beneficial. Finally, she claims that unclear outcomes in MBI (when the uncertainty allows for substantial positive and negative effects, or benefit and harm) should be counted as inferential errors. We reject this claim, on the grounds that an error does not occur until a decision is made about the true magnitude. We previously adopted this reasoning even-handedly with conservative NHST and showed the error rates, rates of decisive outcomes, and publication bias were generally superior in MBI. She makes several other crucial errors, including her claim that publications we cited as evidence supporting the theoretical basis of MBI "do not provide such evidence." Her recognition of several possible contributions of MBI to the debate on inference is followed immediately by its dismissal as unsound or demonstrably false, while many other valuable original contributions are simply overlooked. We conclude that her recommendation that MBI should not be used is itself based on unsound or demonstrably false assertions. Researchers can continue to use MBI in the knowledge that it represents a valuable advance on NHST, with the benefits of Bayesian probabilistic inference and without the drawback of a subjective prior. KEYWORDS: Bayesian statistics, effect, clinical importance, likelihood, null-hypothesis significance test, p value, probability, sample, smallest important difference, statistical significance.
© Copyright 2018 Sportscience. AUT University. Alle Rechte vorbehalten.