Extras - workflowsets

Introduction to tidymodels

How can we compare multiple model workflows at once?

Evaluate a workflow set

wf_set <- workflow_set(list(forested ~ .), list(tree_spec, rf_spec))
wf_set
#> # A workflow set/tibble: 2 × 4
#>   wflow_id              info             option    result    
#>   <chr>                 <list>           <list>    <list>    
#> 1 formula_decision_tree <tibble [1 × 4]> <opts[0]> <list [0]>
#> 2 formula_rand_forest   <tibble [1 × 4]> <opts[0]> <list [0]>

Evaluate a workflow set

wf_set_fit <- wf_set %>%
  workflow_map("fit_resamples", resamples = forested_folds)
wf_set_fit
#> # A workflow set/tibble: 2 × 4
#>   wflow_id              info             option    result   
#>   <chr>                 <list>           <list>    <list>   
#> 1 formula_decision_tree <tibble [1 × 4]> <opts[1]> <rsmp[+]>
#> 2 formula_rand_forest   <tibble [1 × 4]> <opts[1]> <rsmp[+]>

Evaluate a workflow set

wf_set_fit %>%
  rank_results()
#> # A tibble: 6 × 9
#>   wflow_id         .config .metric   mean std_err     n preprocessor model  rank
#>   <chr>            <chr>   <chr>    <dbl>   <dbl> <int> <chr>        <chr> <int>
#> 1 formula_rand_fo… Prepro… accura… 0.919  0.00525    10 formula      rand…     1
#> 2 formula_rand_fo… Prepro… brier_… 0.0617 0.00338    10 formula      rand…     1
#> 3 formula_rand_fo… Prepro… roc_auc 0.972  0.00310    10 formula      rand…     1
#> 4 formula_decisio… Prepro… accura… 0.894  0.00562    10 formula      deci…     2
#> 5 formula_decisio… Prepro… brier_… 0.0817 0.00434    10 formula      deci…     2
#> 6 formula_decisio… Prepro… roc_auc 0.951  0.00378    10 formula      deci…     2

The first metric of the metric set is used for ranking. Use rank_metric to change that.

Lots more available with workflow sets, like collect_metrics(), autoplot() methods, and more!

Your turn

When do you think a workflow set would be useful?

Discuss with your neighbors!

03:00