Binary prediction calibration computation and visualisation.

Obtain estimate of binary prediction calibration and the associated uncertainty interval for further processing or plot it directly.

binary_calibration_from_bp(
  bp,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...
)

plot_binary_calibration_diff(
  x,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  prob_histogram = TRUE
)

# S3 method for SBC_results
plot_binary_calibration_diff(res, ...)

# S3 method for data.frame
plot_binary_calibration_diff(
  bp,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  prob_histogram = TRUE
)

plot_binary_calibration(
  x,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  prob_histogram = TRUE
)

# S3 method for SBC_results
plot_binary_calibration(res, ...)

# S3 method for data.frame
plot_binary_calibration(
  bp,
  type = c("reliabilitydiag", "calibrationband"),
  ...,
  prob_histogram = TRUE
)

Arguments

bp: the binary probabilities --- typically obtained with binary_probabilities_from_stats(). Can however be manually constructed, it needs to be a data.frame with columns variable, prob and simulated_value.
type: the type of calibration uncertainty bands to compute, see details.
alpha: the level associated with the confidence intervals reports
...: additional arguments passed to reliabilitydiag::reliabilitydiag() or calibrationband::calibration_bands()
prob_histogram: Whether a histogram of the observed probabilities should be overlaid with the calibration curve.
res: An SBC_results object

Value

binary_calibration_from_bp returns a data.frame with columns variable, prob, estimate, low and high, for each variable, it contains an estimate + confidence interval across a range of probabilities (in equal steps). The plot_ methods return a ggplot2

object showing either the calibration curve or the difference between the calibration curve and perfect calibration (the diagonal)

Details

When type = "reliabilitydiag", the intervals are for the null distribution assuming perfect calibration using reliabilitydiag::reliabilitydiag(). When type = "calibrationband" the intervals are around the observed calibration using calibrationband::calibration_bands() --- in our experience the calibrationband method has less sensitivity to detect miscalibration.