Uses ranks to compute coverage and surrounding uncertainty of posterior credible intervals.
The uncertainty is only approximate (treating coverage for each interval as a set of independent
Bernoulli trials, while in fact they are not independent), so for making claims on presence/
absence of detectable discrepancies we strongly recommend using
The uncertainty about the coverage can however be useful for guiding decisions on whether
more SBC steps should be performed (i.e. whether we can rule out that the coverage of
the given backend differs too much for our purposes from the optimal value).
empirical_coverage(stats, width, prob = 0.95, interval_type = "central")
a data.frame of rank statistics (e.g. as returned in the
a vector of values between 0 and 1 representing widths of credible intervals for which we compute coverage.
determines width of the uncertainty interval around the observed coverage
data.frame with columns
width (width of the interval as given
width_represented the closest width that can be represented by
the ranks in the input (any discrepancy needs to be judged against this rather than
estimate - observed coverage for the interval,
ci_high the uncertainty
estimate (width of the interval is given by the
Note that while coverage of central posterior intervals (with the default
type = "central")
is often of the biggest practical interest, perfect calibration of central intervals
still leaves space for substantial problems with the model (e.g. if the posterior 25% - 50% intervals
contain 50% of the true values and the posterior 50% - 75% interval never contains the true value,
the central 50% interval still has the ideal 50% coverage), so investigating central
intervals should always be accompanied by checks with
or by using
type = "leftmost", because if all leftmost credible intervals are well calibrated,
then all intervals are well calibrated.