The Impact of Preliminary Tests on Statistical Reproducibility of Location Tests

Abstract

This thesis investigates statistical reproducibility (RP) as a predictive inference problem within the framework of nonparametric predictive inference (NPI). NPI is focused on the prediction of future observations using existing data. In this thesis, statistical reproducibility is defined as the probability of the event that, if the test was repeated under the same conditions and with the same sample size, the same test outcome would be obtained. This thesis presents contributions to NPI reproducibility for location tests and preliminary tests which are preliminary statistical analyses performed before the main or location hypothesis testing to evaluate assumptions for their validity. There is an ongoing debate about whether preliminary tests are necessary to validate assumptions for location tests; some argue they are important for optimal performance while others caution against their use. This thesis aims to evaluate the RP for location tests, both with and without preliminary tests, aiming to examine the impact of preliminary tests on the RP for location tests. The potential impact of preliminary tests on RP of location tests is explored through simulation studies that compare RP of location tests with and without such preliminary tests. The findings suggest that the impact of preliminary tests on RP for location tests is small, they do not substantially lead to improved or deteriorated RP of location tests.