Reproducibility of Hypothesis Tests for 2 × 2 Contingency Tables.

Abstract

The 2 × 2 contingency table is an important data structure in statistical analysis used to examine and compare the association between two binary variables. This table arranges data into four cells, which can be analysed using various statistical methods such as chi-square test, likelihood ratio test, Fisher’s exact test, and McNemar test.

Nonparametric Predictive Inference (NPI) is a frequentist statistics method, which provides lower and upper probabilities for events involving one or more future observations. This thesis introduces NPI for 2 × 2 tables data and illustrates its use for several inference problems.

In statistics, hypothesis testing is a method of statistical inference used to draw conclusions, with the outcome being either rejecting or not rejecting the null hy- pothesis. Statistical reproducibility is the probability that, repeating a test under identical conditions and with the same sample size will lead to the same outcome. The reproducibility of an experiment’s conclusion is a critical concept in every field of research. NPI and NPI-bootstrap methods have been used to study statistical reproducibility. In this thesis, we employ these methods to assess the reproducibility of statistical hypothesis tests based on a single 2 × 2 table and on multiple 2 × 2 tables.

Furthermore, this thesis explores the reproducibility of hypothesis tests using Bayesian inference to predict future observations through the posterior predictive distribution. By introducing NPI for 2 × 2 tables and employing Bayesian approaches, this thesis advances the study of statistical reproducibility for hypothesis tests. Reproducibility is low for both the NPI and Bayesian inference methods when the test statistic is near the threshold between rejecting and not rejecting the null hypothesis. On the other hand, reproducibility increases as the test statistic moves away from this threshold.