Create tidymodels tuning metrics for concentration-index model selection

Builds a yardstick yardstick::metric_set() for use in tune::tune_grid() with ineqTrees parsnip engines. The returned metrics recover the socioeconomic rank and optional case weights from the original analysis data using the .row column supplied by tune_grid().

These metrics are intended for tidymodels workflows. The package-native tune_ci_tree() and tune_ci_forest() helpers remain the most direct way to tune on fitted CI partitions because they can inspect the fitted tree or forest objects. In tidymodels, when a node column is not supplied, validation gain uses predictions as the grouping variable, matching ci_gain().

Usage

ci_tuning_metric_set(
  data,
  rank_name,
  case_weight_name = NULL,
  type = "CIg",
  metrics = c("validation_gain", "relative_validation_gain"),
  node_name = NULL
)

Arguments

data

Original analysis data used to create the resamples.

rank_name

Name of the socioeconomic rank column in data.

case_weight_name

Optional name of the case-weight column in data.

type

One of "CI", "CIg", "CIc", or "L".

metrics

Character vector of CI metrics to include. Supported values are "validation_gain" and "relative_validation_gain".

node_name

Optional name of a node or partition column. If present in the prediction data passed to the metric, that column is used as the validation partition. Otherwise predictions are used as the partition.

Value

A yardstick metric set.

Examples

if (requireNamespace("yardstick", quietly = TRUE)) {
  toy_data <- data.frame(
    rank = c(10, 20, 30, 40),
    outcome = c(1, 0, 1, 0),
    weight = c(1, 1, 2, 2)
  )

  ci_tuning_metric_set(
    toy_data,
    rank_name = "rank",
    case_weight_name = "weight",
    type = "CI",
    metrics = c("validation_gain", "relative_validation_gain")
  )
}
#> A metric set, consisting of:
#> - `validation_gain()`, a numeric metric          | direction: maximize
#> - `relative_validation_gain()`, a numeric metric | direction: maximize