Binary prediction calibration computation and visualisation.

Obtain estimate of binary prediction calibration and the associated uncertainty interval for further processing or plot it directly.

binary_calibration_from_bp(
  bp,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  region.position = NULL
)

plot_binary_calibration_diff(
  x,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  region.position = NULL,
  prob_histogram = TRUE
)

# S3 method for class 'SBC_results'
plot_binary_calibration_diff(res, ...)

# S3 method for class 'data.frame'
plot_binary_calibration_diff(
  bp,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  region.position = NULL,
  prob_histogram = TRUE
)

plot_binary_calibration(
  x,
  type = c("reliabilitydiag", "calibrationband"),
  alpha = 0.05,
  ...,
  region.position = NULL,
  prob_histogram = TRUE
)

# S3 method for class 'SBC_results'
plot_binary_calibration(res, ...)

# S3 method for class 'data.frame'
plot_binary_calibration(
  bp,
  type = c("reliabilitydiag", "calibrationband"),
  ...,
  region.position = NULL,
  prob_histogram = TRUE
)

Arguments

bp: the binary probabilities — typically obtained with binary_probabilities_from_stats(). Can however be manually constructed, it needs to be a data.frame with columns variable, prob and simulated_value.
type: the type of calibration uncertainty bands to compute, see details.
alpha: the level associated with the confidence intervals reports
...: additional arguments passed to reliabilitydiag::reliabilitydiag() or calibrationband::calibration_bands()
region.position: for type ="reliabilitydiag" we may choose whether the uncertainty interval surrounds the estimate (region.position = "estimate") or the null distribution (region.position = "diagonal")
prob_histogram: Whether a histogram of the observed probabilities should be overlaid with the calibration curve.
res: An SBC_results object

Value

binary_calibration_from_bp returns a data.frame with columns variable, prob, estimate, low and high, for each variable, it contains an estimate + confidence interval across a range of probabilities (in equal steps). The plot_ methods return a ggplot2 object showing either the calibration curve or the difference between the calibration curve and perfect calibration (the diagonal)

Details

When type = "reliabilitydiag", the intervals are based on reliabilitydiag::reliabilitydiag() and depending on region.position can be centered on the null distribution of perfect calibration or on the estimated calibration. When type = "calibrationband" the intervals are around the estimated calibration using calibrationband::calibration_bands() — in our experience the calibrationband method has less sensitivity to detect miscalibration, but they require somewhat weaker assumptions.