Evaluation of empirical type I error rates of F and normality tests under different variance and mean conditions in multi-treatment CRDs
DOI:
https://doi.org/10.33837/msj.v8i1.1719Keywords:
Level of Significance, ANOVA, Completely Randomized Design, Normal Distribution, Experimental ErrorsAbstract
Hypothesis tests, such as normality tests, are extensively employed in Agricultural Sciences to evaluate the normality assumption of the F test in the Analysis of Variance (ANOVA) when large sample sizes are unavailable. Nonetheless, researchers conducting these tests are exposed to the risk of committing type I or type II errors, with probabilities that are influenced by different experimental conditions. This study assesses the empirical type I error rate of hypothesis tests by considering the equality (inequality) of treatment means, the homogeneity (heterogeneity) of variances, and different numbers of repetitions per treatment. Applying Completely Randomized Designs (CRD), sub-scenarios were simulated for each experimental scenario, with 10,000 iterations performed for each sub-scenario. Response variable values and experimental residuals were generated and subjected to appropriate tests. The results demonstrate that when the assumption of homogeneity of variances is violated, both the F and normality tests (excluding the Kolmogorov-Smirnov test) exhibit higher empirical type I error rates. Additionally, for normality tests, these error rates increase with the number of repetitions. Conversely, without such violations, the error rates remain stable and closely approximate the theoretical significance level for all analyzed hypothesis tests.
References
Acutis, M., Scaglia, B., & Confalonieri, R. (2012). Perfunctory analysis of variance in agronomy, and its consequences in experimental results interpretation. European Journal of Agronomy 43, 129-135.
Anderson, T.W. & Darling, D.A. (1952). Asymptotic Theory of Certain “Goodness of Fit” Criteria Based on Stochastic Processes. The Annals of Mathematical Statistics 23, 193-212.
Arnastauskaitė, J., Ruzgas, T. & Bražėnas, M. (2021). An Exhaustive Power Comparison of Normality Tests. Mathematics 9, 788.
Barbetta, P. A., Reis, M. M., Bornia, A. C. (2004). Estatística: para cursos de engenharia e informática. (São Paulo, Atlas).
Bussab, W. O., Morettin, P. A. (2010). Estatística Básica. 6a ed. (Editora Saraiva).
Casella, G. & Berger, R. L. (2022). Statistical Inference. 2nd. ed (Duxbury/Thomson Learning).
Campos, H. (1976). Estatística Experimental Não-Paramétrica. 2a ed. (Piracicaba, Escola Superior de Agricultura “Luiz de Queiroz”, Universidade de São Paulo).
Fukunaga, E. T., Guibu, I. A., Moraes, J. C. (2018). Bases de Estatística para Profissionais de Saúde. (Memnon: CEALAG- Centro de Estudos Augusto Leopoldo Ayrosa Galvão).
Henrique, F.H., Laca-Buendía, J.P. (2010). Comportamento morfológico e agronômico de genótipos de algodoeiro no município de Uberaba-MG. FAZU em Revista 7, 32-36.
Kelter, R. (2021). Analysis of type I and II error rates of Bayesian and frequentist parametric and nonparametric two-sample hypothesis tests under preliminary assessment of normality. Computational Statistics 36, 1263–1288.
Keskin, S. (2006). Comparison of Several Univariate Normality Tests Regarding Type I Error Rate and Power of the Test in Simulation based Small Samples. Journal of Applied Science Research 2, 296–300.
Kulkarni, H. V., Patil, S. M. (2021). Uniformly implementable small sample integrated likelihood ratio test for one-way and two-way ANOVA under heteroscedasticity and normality. AStA Advances in Statistical Analysis 105, 273–305.
Mood, A. M. (1974). Introduction to the theory of statistics. 3. ed. (McGraw-Hill, Inc).
Nguyen, D., Kim, E., Wang, Y., Pham, T. V., Chen, Y.H. & Kromrey, J. D. (2019). Empirical comparison of tests for one-factor ANOVA under heterogeneity and non-normality: A Monte Carlo study. Journal of Modern Applied Statistical Methods 18.
Ogunleye, L. I., Oyejola, B. A., Obisesan, K. O. (2018). Comparison of Some Common Tests for Normality. International Journal of Probability and Statistics 7, 130–137.
Öztuna, D.; Elhan, A. H., Tüccar, E. (2006). Investigation of Four Different Normality Tests in Terms of Type 1 Error Rate and Power under Different Distributions. Turkish Journal of Medical Sciences 36, 171–176.
Piepho, H. P., & Edmondson, R. N. (2018). A tutorial on the statistical analysis of factorial experiments with qualitative and quantitative treatment factor levels. Journal of Agronomy and Crop Science 204, 429-455.
Razali, N.M. & Wah, Y.B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. J. Stat. Model Anal. 2, 21-33.
Rodrigues, J., Piedade, S. M. E; Lara, I.A.R; Henrique, F.H. (2021). Type I error in multiple comparison tests in analysis of variance. Acta Scientiarum 45.
Searle, S. R., Gruber, M. H. J. (2016). Linear Models. 2nd. ed. [S.l.] (Wiley Series in Probability and Statistics).
Shapiro, S.S. & Wilk, M.B. (1965). An Analysis of Variance Test for Normality (Complete Samples). Biometrika 52, 591-611.
Thadewald, T., Büning, H. (2007). Jarque-Bera Test and its Competitors for Testing Normality - A Power Comparison. Journal of Applied Statistics 34, 87–105.
Torman, V.B.L., Coster, R., Riboldi, J. (2012). Normalidade de variáveis: métodos de verificação e comparação de alguns testes não-paramétricos por simulação. Revista Clinical & Biomedical Research 32.
Welch, B. L. (1951). On the comparison of several mean values: An alternative approach. Biometrika 38, 330–336.
Wilcox, R. R. (1988). A new alternative to the ANOVA F and new results on James’s second‐order method. British Journal of Mathematical and Statistical Psychology 41, 109–117.
Wilcox, R. R. (1989). Adjusting for Unequal Variances When Comparing Means in One-Way and Two-Way Fixed Effects ANOVA Models. Journal of Educational and Behavioral Statistics 14, 269–278.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 The author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish in this journal agree to the following terms:
a) The Authors retain the copyright and grant the journal the right to first publication, with the work simultaneously licensed under the Creative Commons Attribution License that allows the sharing of the work with acknowledgment of authorship and initial publication in this journal.
b) Authors are authorized to assume additional contracts separately, for non-exclusive distribution of the version of the work published in this journal (eg, publishing in institutional repository or as a book chapter), with acknowledgment of authorship and initial publication in this journal.
c) Authors are allowed and encouraged to publish and distribute their work online (eg in institutional repositories or on their personal page) at any point before or during the editorial process, as this can generate productive changes, as well as increase impact and citation of the published work.
