Non-parametric test for the mean of a bounded variable.

A non-parametric one-sample test for the mean. The only assumptions it makes about the data is that they are i.i.d. and that the underlying distribution is bounded from both sides.

gaffke_test(
  x,
  mu = 0.5,
  alpha = 0.05,
  lb = 0,
  ub = 1,
  B = 10000,
  alternative = c("two.sided", "less", "greater")
)

gaffke_test_paired(
  x,
  y,
  mu = 0,
  alpha = 0.05,
  lb = 0,
  ub = 1,
  B = 10000,
  alternative = c("two.sided", "less", "greater")
)

gaffke_ci(probs, B = 10000, alpha = 0.05)

gaffke_p(
  probs,
  mu = 0.5,
  alpha = 0.05,
  B = 10000,
  alternative = c("two.sided", "less", "greater")
)

Arguments

x

a vector of observed values

mu

the mean under null hypothesis

alpha

the level of the test

lb, ub

the lower and upper bounds for x (for the paired version, it also applies to y you can provide a v ector of length 2 to give different bounds to x and y)

ub

the upper bound for x

B

number of bootstrap samples for the null distribution

alternative

the alternative for the test.

y

a second vector, the same length as x

probs

a vector of observed values, scaled to [0, 1]

Value

gaffke_test and gaffke_test_paired return an object of class htest, gaffke_p and gaffke_ci return just the p-value / CI as numeric for easier use in batch workflows.

Details

The test is based on Gaffke 2005, though a more detailed analysis and exposition can be found in Learned-Miller & Thomas 2020.

The test is expected to be valid for any bounded distribution without further assumptions. The test has been proven valid only for special cases but no counterexample is known despite some efforts in the literature to find some. The test also provides way more power (and tighter confidence intervals) than other non-parametric tests for bounded means - in fact its power converges quite quickly to that of the t-test, which is known to be optimal in the large data limit.

The paired two-sample version translates to a one-sample test with bounds computed to constrain the difference. There is no known unpaired two-sample version.

In SBC the test is useful for testing the data-averaged posterior for binary variables - the paired version is to be used when the prior is not analytically known and so the simulated value of the variable is the second sample.

gaffke_p and gaffke_ci are intended for easier use in batch workflows as they directly return the relevant numeric values, not a test object. Note that those functions expected their arguments to be scaled to [0, 1] already, while gaffke_test will work with any upper and lower bounds and do the scaling for you.

References

Gaffke, N. (2005). “Three test statistics for a nonparametric one-sided hypothesis on the mean of a nonnegative variable.” Mathematical Methods of Statistics, 14(4): 451–467.

Learned-Miller, E. and Thomas, P. S. (2020). “A New Confidence Interval for the Mean of a Bounded Random Variable.” https://arxiv.org/abs/1905.06208

Examples

set.seed(84623)
# true mean of x = 0.5
x <- rbeta(10, shape1 = 0.1, shape2 = 0.1)
gaffke_test(x)
#> 
#> 	Gaffke's test for the mean of a bounded variable (using 10000 samples)
#> 
#> data:  x
#> lower bound = 0, upper bound = 1, p-value = 0.1118
#> alternative hypothesis: true mean is not equal to 0.5
#> 95 percent confidence interval:
#>  0.05819751 0.55233984
#> sample estimates:
#>     mean 
#> 0.230763 
#> 
# Standard t-test is invalid here due to small sample size
t.test(x, mu = 0.5)
#> 
#> 	One Sample t-test
#> 
#> data:  x
#> t = -2.3145, df = 9, p-value = 0.0459
#> alternative hypothesis: true mean is not equal to 0.5
#> 95 percent confidence interval:
#>  -0.03238985  0.49391593
#> sample estimates:
#> mean of x 
#>  0.230763 
#> 

# Paired version + scaling - true difference in means is ~2.5
x_scaled <- x * 2 + 1
y_scaled <- -1 + 3 * rbeta(length(x), shape1 = 5 * (x / 3), shape2 = 5 * (1 - x / 3))
gaffke_test_paired(x_scaled, y_scaled, lb = c(1, -1), ub = c(3, 2), mu = 2)
#> 
#> 	Gaffke's paired test for the difference of means of bounded variables
#> 	(using 10000 samples)
#> 
#> data:  x_scaled minus y_scaled
#> lower bound = -1, upper bound = 4, p-value = 1
#> alternative hypothesis: true mean is not equal to 2
#> 95 percent confidence interval:
#>  1.206934 2.761599
#> sample estimates:
#> mean difference 
#>        2.192575 
#> 

# As the number of draws increases, the test converges to a t-test
x <- rbeta(100, shape1 = 0.1, shape2 = 0.1)
gaffke_test(x)
#> 
#> 	Gaffke's test for the mean of a bounded variable (using 10000 samples)
#> 
#> data:  x
#> lower bound = 0, upper bound = 1, p-value = 0.9138
#> alternative hypothesis: true mean is not equal to 0.5
#> 95 percent confidence interval:
#>  0.3984379 0.5819400
#> sample estimates:
#>      mean 
#> 0.4900182 
#> 
t.test(x, mu = 0.5)
#> 
#> 	One Sample t-test
#> 
#> data:  x
#> t = -0.21546, df = 99, p-value = 0.8299
#> alternative hypothesis: true mean is not equal to 0.5
#> 95 percent confidence interval:
#>  0.3980927 0.5819437
#> sample estimates:
#> mean of x 
#> 0.4900182 
#>