Extras - workflowsets

Introduction to tidymodels

How can we compare multiple model workflows at once?

Evaluate a workflow set

wf_set <- workflow_set(list(forested ~ .), list(tree_spec, rf_spec))
wf_set
#> # A workflow set/tibble: 2 × 4
#>   wflow_id              info             option    result    
#>   <chr>                 <list>           <list>    <list>    
#> 1 formula_decision_tree <tibble [1 × 4]> <opts[0]> <list [0]>
#> 2 formula_rand_forest   <tibble [1 × 4]> <opts[0]> <list [0]>

Evaluate a workflow set

wf_set_fit <- wf_set %>%
  workflow_map("fit_resamples", resamples = forested_folds)
wf_set_fit
#> # A workflow set/tibble: 2 × 4
#>   wflow_id              info             option    result   
#>   <chr>                 <list>           <list>    <list>   
#> 1 formula_decision_tree <tibble [1 × 4]> <opts[1]> <rsmp[+]>
#> 2 formula_rand_forest   <tibble [1 × 4]> <opts[1]> <rsmp[+]>

Evaluate a workflow set

wf_set_fit %>%
  rank_results()
#> # A tibble: 6 × 9
#>   wflow_id          .config .metric  mean std_err     n preprocessor model  rank
#>   <chr>             <chr>   <chr>   <dbl>   <dbl> <int> <chr>        <chr> <int>
#> 1 formula_rand_for… Prepro… accura… 0.753 0.00435    10 formula      rand…     1
#> 2 formula_rand_for… Prepro… brier_… 0.167 0.00316    10 formula      rand…     1
#> 3 formula_rand_for… Prepro… roc_auc 0.757 0.00992    10 formula      rand…     1
#> 4 formula_decision… Prepro… accura… 0.704 0.00653    10 formula      deci…     2
#> 5 formula_decision… Prepro… brier_… 0.214 0.00349    10 formula      deci…     2
#> 6 formula_decision… Prepro… roc_auc 0.692 0.00496    10 formula      deci…     2

The first metric of the metric set is used for ranking. Use rank_metric to change that.

Lots more available with workflow sets, like collect_metrics(), autoplot() methods, and more!

Your turn

When do you think a workflow set would be useful?

Discuss with your neighbors!

03:00