`R/calculate.R`

`empirical_coverage.Rd`

Uses ranks to compute coverage and surrounding uncertainty of posterior credible intervals.
The uncertainty is only approximate (treating coverage for each interval as a set of independent
Bernoulli trials, while in fact they are not independent), so for making claims on presence/
absence of detectable discrepancies we strongly recommend using `plot_ecdf()`

or `plot_ecdf_diff()`

.
The uncertainty about the coverage can however be useful for guiding decisions on whether
more SBC steps should be performed (i.e. whether we can rule out that the coverage of
the given backend differs too much for our purposes from the optimal value).

`empirical_coverage(stats, width, prob = 0.95, interval_type = "central")`

- stats
a data.frame of rank statistics (e.g. as returned in the

`$stats`

component of SBC_results), at minimum should have at least`variable`

,`rank`

and`max_rank`

columns)- width
a vector of values between 0 and 1 representing widths of credible intervals for which we compute coverage.

- prob
determines width of the uncertainty interval around the observed coverage

- inteval_type
`"central"`

to show coverage of central credible intervals or`"leftmost"`

to show coverage of leftmost credible intervals (i.e. the observed CDF).

A `data.frame`

with columns `variable`

, `width`

(width of the interval as given
in the `width`

parameter), `width_represented`

the closest width that can be represented by
the ranks in the input (any discrepancy needs to be judged against this rather than `width`

),
`estimate`

- observed coverage for the interval, `ci_low`

, `ci_high`

the uncertainty
interval around `estimate`

(width of the interval is given by the `prob`

argument).

Note that while coverage of central posterior intervals (with the default `type = "central"`

)
is often of the biggest practical interest, perfect calibration of central intervals
still leaves space for substantial problems with the model (e.g. if the posterior 25% - 50% intervals
contain 50% of the true values and the posterior 50% - 75% interval never contains the true value,
the central 50% interval still has the ideal 50% coverage), so investigating central
intervals should always be accompanied by checks with `plot_ecdf()`

or `plot_ecdf_diff()`

or by using `type = "leftmost"`

, because if all leftmost credible intervals are well calibrated,
then all intervals are well calibrated.