# Load any R Packages you may need
library(tidyverse)
library(parsnip)
library(tidymodels)
library(rpart.plot)
library(ggridges)
Introduction
Honey bees are an extremely important species. They are a keystone species in that they pollinate much of our food. Without honey bees flowering plants will decline which will have a negative consequence for all ecosystems. The Save The Bees movement has become increasingly popular within the last few years because beekeepers are losing a high percentage of their honeybee colonies every year. And just as worrisome is the that the wild bee populations are also in decline. Of course, there are many factors that have to do with the recent decline but there are some that may have a greater affect than others. Honey bees are at risk of becoming endangered which is why it is important to look at the causes of the decline.
The data set Bee Colonies from TidyTuesday is made up of two main tables with information taken from US honeybee colony farmers. One table, named colony, provides information on honey bee colonies in terms of number of colonies, maximum, lost, percent lost, added, renovated, and colonies lost. The data also provides another table, named stressor, that identifies colony health stressors. Using this dataset, we can ask the question What are the main causes of colony loss in the US?
Data Wrangling
First, let’s make sure to clean the data a little bit!
- We can group the months into season instead. So originally there was January-March, April-June, July-September, and October-December. I instead made months October-December and January-March be the Winter season, and April-June and July-September be the Summer season.
- I got rid of the data from the year 2021, because it only had data on half of the year, so it did not give accurate enough information to be representative on the whole year.
- I selected the columns that had to do with colony lost count and percentage, since that is the data we are exploring.
- I added a new column id for later use when joining the two tables together.
colony = colony %>%
mutate(season = case_when(
months %in% c('January-March', 'October-December') ~ 'Winter',
months %in% c('April-June', 'July-September') ~ 'Summer'
)) %>%
filter(year != 2021) %>%
mutate(id = row_number()) %>%
select(id, year, season, state, colony_n, colony_lost, colony_lost_pct)
head(colony)
## # A tibble: 6 × 7
## id year season state colony_n colony_lost colony_lost_pct
## <int> <dbl> <chr> <chr> <dbl> <dbl> <dbl>
## 1 1 2015 Winter Alabama 7000 1800 26
## 2 2 2015 Winter Arizona 35000 4600 13
## 3 3 2015 Winter Arkansas 13000 1500 11
## 4 4 2015 Winter California 1440000 255000 15
## 5 5 2015 Winter Colorado 3500 1500 12
## 6 6 2015 Winter Connecticut 3900 870 22
For the stressor table, I cleaned the data in the same way. I grouped the months into seasons instead, filtered out the year 2021 data, and created a column for id to help with joining the tables.
stressor = stressor %>%
mutate(season = case_when(
months %in% c('January-March', 'October-December') ~ 'Winter',
months %in% c('April-June', 'July-September') ~ 'Summer'
)) %>%
filter(year != 2021) %>%
mutate(id = row_number()) %>%
select(year, season, state, stressor, stress_pct)
head(stressor)
## # A tibble: 6 × 5
## year season state stressor stress_pct
## <dbl> <chr> <chr> <chr> <dbl>
## 1 2015 Winter Alabama Varroa mites 10
## 2 2015 Winter Alabama Other pests/parasites 5.4
## 3 2015 Winter Alabama Disesases NA
## 4 2015 Winter Alabama Pesticides 2.2
## 5 2015 Winter Alabama Other 9.1
## 6 2015 Winter Alabama Unknown 9.4
Preliminary Exploratory Analysis
Now that we have our data in a way that is easier to manage, we can explore it a bit more.
The graph below shows the average percentage of colonies lost in the US during the winter and summer seasons by year.
colony %>%
group_by(year, season) %>%
summarize(avg_colony_lost_pct = mean(colony_lost_pct, na.rm = TRUE)) %>%
ggplot(aes(x = year, y = avg_colony_lost_pct, fill = season)) + geom_col(position = "dodge") + labs(x = "", y = "Average Colonies Lost (%)", fill = "", title = "Average Percentage of Total Colonies Lost in the U.S.", subtitle = "From 2015-2020") + theme_minimal()

We can see that in the winter months, there is a higher average colonies lost percentage for each year (2015-2020). This may be because bee colonies have a hard time staying alive during colder months, but it’s interesting to see how the percentage on colonies lost during the winter months has increased during 2016-2018. There may be other reasons as to why the average colonies lost percentage is at its peak in 2018.
The graph below explores the stressor table. It shows the average stress percentage for each stressor from the years 2015 -2020.
na.omit(stressor) %>%
group_by(year, stressor) %>%
summarize(stress_pct_avg = mean(stress_pct)) %>%
ggplot(aes(x = year, y = stress_pct_avg, color = stressor)) + geom_point() + geom_smooth() + labs(x = "", y = "Average Stress (%)", color = "Stressor", title = "Average Stress Percentage for Each Stressor", subtitle = "From 2015 to 2020")

We can see that the stressor Varroa Mites, has the highest average stress percentage throughout the years, by a lot when compared to other stressors. This means that the Varroa mites is the stressor that has the highest percent of colonies affected by it. Again, in 2018, there is a peak in a high stress percentage for most stressors, including the Varroa mites.
Joining Tables
Now that we’ve see two causes of colony loss separately, let’s look at them together. To do this, we can merge the two tables together.
First, we can fix the stressor table to look at only the Varroa mites stress percent by state.
stressor_vm = stressor%>%
filter(stressor == "Varroa mites") %>%
arrange(year) %>%
mutate(id = row_number()) %>%
select(id, year, season, state, stressor, stress_pct)
head(stressor_vm)
## # A tibble: 6 × 6
## id year season state stressor stress_pct
## <int> <dbl> <chr> <chr> <chr> <dbl>
## 1 1 2015 Winter Alabama Varroa mites 10
## 2 2 2015 Winter Arizona Varroa mites 26.9
## 3 3 2015 Winter Arkansas Varroa mites 17.6
## 4 4 2015 Winter California Varroa mites 24.7
## 5 5 2015 Winter Colorado Varroa mites 14.6
## 6 6 2015 Winter Connecticut Varroa mites 2.5
Now that the stressor table is filtered to only have the Varroa mites stress percentage, we can join it to the colony table.
colonyJoinedStressorVM = colony %>%
inner_join(stressor_vm, by = c("id", "year", "season", "state"))
head(colonyJoinedStressorVM)
## # A tibble: 6 × 9
## id year season state colony_n colony_lost colony_lost_pct stressor
## <int> <dbl> <chr> <chr> <dbl> <dbl> <dbl> <chr>
## 1 1 2015 Winter Alabama 7000 1800 26 Varroa mi…
## 2 2 2015 Winter Arizona 35000 4600 13 Varroa mi…
## 3 3 2015 Winter Arkansas 13000 1500 11 Varroa mi…
## 4 4 2015 Winter California 1440000 255000 15 Varroa mi…
## 5 5 2015 Winter Colorado 3500 1500 12 Varroa mi…
## 6 6 2015 Winter Connecticut 3900 870 22 Varroa mi…
## # … with 1 more variable: stress_pct <dbl>
Exploratory Data Analysis
With this new data, we can explore the relationships between season and stress percentage of Valorroa mites, with colony lost percentage.
The graph below shows the relationship between the percentage of colonies affected by Varroa mites and the percentage of total colonies lost in the U.S. More specifically, it looks at the relationship when taking the season into account.
colonyJoinedStressorVM %>%
ggplot(aes(x = stress_pct, y = colony_lost_pct, color = season)) + geom_point() + geom_smooth(method = "lm", se = FALSE) + labs(x = "Colonies Affected by varroa mites (%)", y = "Total Colonies Lost (%)", color = "Season", title = "Percentage of Total Colonies Lost Based on Percentage of Colonies Affected by Varroa Mites ", subtitle = "By season")
Visually, we can see that there appears to be a general positive correlation between stress pct and colony lost percent, with winter seasons having greater colony lost percentage. Both higher stress percents of Varroa mites and the winter season together, have the highest colony lost percentage.
Let’s investigate these relationships through modeling now!
Predictive Modeling
Linear Regression Model
Target: Percent of total colonies lost (colony_lost_pct)
Feature Percent of colonies affected by Varroa mites (stress_pct) and season (season)
Let’s build a multiple regression model that predicts colony_lost_pct while considering our features of stress_pct and season !
set.seed(14)
#1. Data Splitting
colonyJoinedStressorVM_split = initial_split(colonyJoinedStressorVM, prop = 0.7)
colonyJoinedStressorVM_train = training(colonyJoinedStressorVM_split)
colonyJoinedStressorVM_test = testing(colonyJoinedStressorVM_split)
#2. Fit model
lm_fit_train =
linear_reg() %>%
fit(colony_lost_pct ~ stress_pct + season, data = colonyJoinedStressorVM_train)
lm_fit_train
## parsnip model object
##
##
## Call:
## stats::lm(formula = colony_lost_pct ~ stress_pct + season, data = data)
##
## Coefficients:
## (Intercept) stress_pct seasonWinter
## 6.1542 0.1062 4.1184
#3. Generate Predictions
lm_pred = lm_fit_train %>%
predict(new_data = colonyJoinedStressorVM_test)
#4. Prediction Metric (RMSE)
colonyJoinedStressorVM_test %>%
mutate(colony_lost_pct_Pred = lm_pred$.pred) %>%
rmse(estimate = colony_lost_pct_Pred, truth = colony_lost_pct)
## # A tibble: 1 × 3
## .metric .estimator .estimate
## <chr> <chr> <dbl>
## 1 rmse standard 6.33
We created a multiple regression model: colony_lost_pct_hat = 0.1062(stress_pct) + 4.1184(seasonWinter)
The RMSE for the multiple regression model was 6.329434. This a very low RMSE. This means that the multiple regression model fits the dataset very well! But this number should be recieved with caution because the relationship between our features (season and stress_pct) and target (colony_lost_pct) may not actually be a linear relationship.
Let’s look at the relationship again using a different model!
Decision Tree
The predictive modeling of linear regression that we used, may not be the best model. Our data may need a non-linear model.
To create a more accurate predictive model, we can use a decision tree to employ the CART (classification and regression tree) algorithm :
set.seed(14)
#1. Data Splitting
colonyJoinedStressorVM_split = initial_split(colonyJoinedStressorVM, prop = 0.7)
colonyJoinedStressorVM_train = training(colonyJoinedStressorVM_split)
colonyJoinedStressorVM_test = testing(colonyJoinedStressorVM_split)
#2. Fit model
tree_fit =
decision_tree() %>%
set_mode(mode = "regression") %>%
fit(colony_lost_pct ~ stress_pct + season,
data = colonyJoinedStressorVM_train)
rpart.plot(tree_fit$fit, roundint = FALSE)

#3. Generate Predictions
tree_pred = tree_fit %>%
predict(new_data = colonyJoinedStressorVM_test)
#4. Prediction Metric (RMSE)
colonyJoinedStressorVM_test %>%
mutate(colony_lost_pct_Pred = tree_pred$.pred) %>%
rmse(truth = colony_lost_pct, estimate = colony_lost_pct_Pred)
## # A tibble: 1 × 3
## .metric .estimator .estimate
## <chr> <chr> <dbl>
## 1 rmse standard 6.60
By viewing the CART output, we can see where the variable season is split. And if we follow along, we can see that the winter months have more higher stress percentages.
The RMSE for the CART model with multiple features was 6.597867. Meanwhile the RMSE for the multiple regression model was 6.329434.
Conclusion
While the RMSE for the multiple regression model was technically smaller, the RMSE for the CART model was not completely off. They were relatively close in size, which means that both models that we created to predict values of colony_lost_pct were generally good at doing so. Of course, these are only two ways of predicting values, so there may be other models that may do a more accurate job. But in exploring both the data visually and through modeling, we can see the effect that the winter season, and varroa mites have on the percentage of colony lost.
The dataset we explored did not contrain many different factors. There could have been outside factors that affect the percentage of colony lost in the US. Perhaps the season and stressors are not the only reason why honeybee colonies are being threatened. It is also important to note that there may be a relationship between the varroa mites and the season which may have had an effect on our ability to perform accurate and useful predictive modeling.
There are many other factors that we must keep looking at to make sure that our honeybee colonies thrive again.
LS0tCnRpdGxlOiAiU2F2ZSB0aGUgQmVlcyEiCmF1dGhvcjogIkNhcm9saW5lIE1hemFyaWVnb3MiCmRhdGU6ICIwNS8xMy8yMDIyIgpvdXRwdXQ6IAogIGh0bWxfZG9jdW1lbnQ6CiAgICB0aGVtZTogZmxhdGx5CiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgY29kZV9kb3dubG9hZDogdHJ1ZQotLS0KCmBgYHtyLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0KIyBMb2FkIGFueSBSIFBhY2thZ2VzIHlvdSBtYXkgbmVlZApsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShwYXJzbmlwKQpsaWJyYXJ5KHRpZHltb2RlbHMpCmxpYnJhcnkocnBhcnQucGxvdCkKbGlicmFyeShnZ3JpZGdlcykKCmBgYAoKYGBge3IsIGVjaG8gPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFLCBtZXNzYWdlID0gRkFMU0V9CmNvbG9ueSA8LSByZWFkcjo6cmVhZF9jc3YoJ2h0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9yZm9yZGF0YXNjaWVuY2UvdGlkeXR1ZXNkYXkvbWFzdGVyL2RhdGEvMjAyMi8yMDIyLTAxLTExL2NvbG9ueS5jc3YnKQpzdHJlc3NvciA8LSByZWFkcjo6cmVhZF9jc3YoJ2h0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9yZm9yZGF0YXNjaWVuY2UvdGlkeXR1ZXNkYXkvbWFzdGVyL2RhdGEvMjAyMi8yMDIyLTAxLTExL3N0cmVzc29yLmNzdicpCmBgYAoKIyBJbnRyb2R1Y3Rpb24KCkhvbmV5IGJlZXMgYXJlIGFuIGV4dHJlbWVseSBpbXBvcnRhbnQgc3BlY2llcy4gVGhleSBhcmUgYSBrZXlzdG9uZSBzcGVjaWVzIGluIHRoYXQgdGhleSBwb2xsaW5hdGUgbXVjaCBvZiBvdXIgZm9vZC4gV2l0aG91dCBob25leSBiZWVzIGZsb3dlcmluZyBwbGFudHMgd2lsbCBkZWNsaW5lIHdoaWNoIHdpbGwgaGF2ZSBhIG5lZ2F0aXZlIGNvbnNlcXVlbmNlIGZvciBhbGwgZWNvc3lzdGVtcy4gVGhlIFNhdmUgVGhlIEJlZXMgbW92ZW1lbnQgaGFzIGJlY29tZSBpbmNyZWFzaW5nbHkgcG9wdWxhciB3aXRoaW4gdGhlIGxhc3QgZmV3IHllYXJzIGJlY2F1c2UgYmVla2VlcGVycyBhcmUgbG9zaW5nIGEgaGlnaCBwZXJjZW50YWdlIG9mIHRoZWlyIGhvbmV5YmVlIGNvbG9uaWVzIGV2ZXJ5IHllYXIuIEFuZCBqdXN0IGFzIHdvcnJpc29tZSBpcyB0aGUgdGhhdCB0aGUgd2lsZCBiZWUgcG9wdWxhdGlvbnMgYXJlIGFsc28gaW4gZGVjbGluZS4gT2YgY291cnNlLCB0aGVyZSBhcmUgbWFueSBmYWN0b3JzIHRoYXQgaGF2ZSB0byBkbyB3aXRoIHRoZSByZWNlbnQgZGVjbGluZSBidXQgdGhlcmUgYXJlIHNvbWUgdGhhdCBtYXkgaGF2ZSBhIGdyZWF0ZXIgYWZmZWN0IHRoYW4gb3RoZXJzLiBIb25leSBiZWVzIGFyZSBhdCByaXNrIG9mIGJlY29taW5nIGVuZGFuZ2VyZWQgd2hpY2ggaXMgd2h5IGl0IGlzIGltcG9ydGFudCB0byBsb29rIGF0IHRoZSBjYXVzZXMgb2YgdGhlIGRlY2xpbmUuIAoKVGhlIGRhdGEgc2V0ICoqQmVlIENvbG9uaWVzKiogZnJvbSAqVGlkeVR1ZXNkYXkqIGlzIG1hZGUgdXAgb2YgdHdvIG1haW4gdGFibGVzIHdpdGggaW5mb3JtYXRpb24gdGFrZW4gZnJvbSBVUyBob25leWJlZSBjb2xvbnkgZmFybWVycy4gT25lIHRhYmxlLCBuYW1lZCAqY29sb255KiwgcHJvdmlkZXMgaW5mb3JtYXRpb24gb24gaG9uZXkgYmVlIGNvbG9uaWVzIGluIHRlcm1zIG9mIG51bWJlciBvZiBjb2xvbmllcywgbWF4aW11bSwgbG9zdCwgcGVyY2VudCBsb3N0LCBhZGRlZCwgcmVub3ZhdGVkLCBhbmQgY29sb25pZXMgbG9zdC4gVGhlIGRhdGEgYWxzbyBwcm92aWRlcyBhbm90aGVyIHRhYmxlLCBuYW1lZCAqc3RyZXNzb3IqLCB0aGF0IGlkZW50aWZpZXMgY29sb255IGhlYWx0aCAgc3RyZXNzb3JzLiBVc2luZyB0aGlzIGRhdGFzZXQsIHdlIGNhbiBhc2sgdGhlIHF1ZXN0aW9uICoqV2hhdCBhcmUgdGhlIG1haW4gY2F1c2VzIG9mIGNvbG9ueSBsb3NzIGluIHRoZSBVUz8qKgoKCgoKIyBEYXRhIFdyYW5nbGluZwoKRmlyc3QsIGxldCdzIG1ha2Ugc3VyZSB0byBjbGVhbiB0aGUgZGF0YSBhIGxpdHRsZSBiaXQhIAoKLSBXZSBjYW4gZ3JvdXAgdGhlIG1vbnRocyBpbnRvIHNlYXNvbiBpbnN0ZWFkLiBTbyBvcmlnaW5hbGx5IHRoZXJlIHdhcyBKYW51YXJ5LU1hcmNoLCBBcHJpbC1KdW5lLCBKdWx5LVNlcHRlbWJlciwgYW5kIE9jdG9iZXItRGVjZW1iZXIuIEkgaW5zdGVhZCBtYWRlIG1vbnRocyBPY3RvYmVyLURlY2VtYmVyIGFuZCBKYW51YXJ5LU1hcmNoIGJlIHRoZSBXaW50ZXIgc2Vhc29uLCBhbmQgQXByaWwtSnVuZSBhbmQgSnVseS1TZXB0ZW1iZXIgYmUgdGhlIFN1bW1lciBzZWFzb24uIAotIEkgZ290IHJpZCBvZiB0aGUgZGF0YSBmcm9tIHRoZSB5ZWFyIDIwMjEsIGJlY2F1c2UgaXQgb25seSBoYWQgZGF0YSBvbiBoYWxmIG9mIHRoZSB5ZWFyLCBzbyBpdCBkaWQgbm90IGdpdmUgYWNjdXJhdGUgZW5vdWdoIGluZm9ybWF0aW9uIHRvIGJlIHJlcHJlc2VudGF0aXZlIG9uIHRoZSB3aG9sZSB5ZWFyLgotIEkgc2VsZWN0ZWQgdGhlIGNvbHVtbnMgdGhhdCBoYWQgdG8gZG8gd2l0aCBjb2xvbnkgbG9zdCBjb3VudCBhbmQgcGVyY2VudGFnZSwgc2luY2UgdGhhdCBpcyB0aGUgZGF0YSB3ZSBhcmUgZXhwbG9yaW5nLgotIEkgYWRkZWQgYSBuZXcgY29sdW1uICppZCogZm9yIGxhdGVyIHVzZSB3aGVuIGpvaW5pbmcgdGhlIHR3byB0YWJsZXMgdG9nZXRoZXIuCgpgYGB7ciwgd2FybmluZyA9IEZBTFNFLCBtZXNzYWdlID0gRkFMU0V9CmNvbG9ueSA9IGNvbG9ueSAlPiUgCiAgbXV0YXRlKHNlYXNvbiA9IGNhc2Vfd2hlbigKICAgIG1vbnRocyAlaW4lIGMoJ0phbnVhcnktTWFyY2gnLCAnT2N0b2Jlci1EZWNlbWJlcicpIH4gJ1dpbnRlcicsIAogICAgbW9udGhzICVpbiUgYygnQXByaWwtSnVuZScsICdKdWx5LVNlcHRlbWJlcicpIH4gJ1N1bW1lcicKICApKSAlPiUgCiAgZmlsdGVyKHllYXIgIT0gMjAyMSkgJT4lIAogIG11dGF0ZShpZCA9IHJvd19udW1iZXIoKSkgJT4lCiAgc2VsZWN0KGlkLCB5ZWFyLCBzZWFzb24sIHN0YXRlLCBjb2xvbnlfbiwgY29sb255X2xvc3QsIGNvbG9ueV9sb3N0X3BjdCkKCmhlYWQoY29sb255KQpgYGAKCkZvciB0aGUgc3RyZXNzb3IgdGFibGUsIEkgY2xlYW5lZCB0aGUgZGF0YSBpbiB0aGUgc2FtZSB3YXkuIEkgZ3JvdXBlZCB0aGUgbW9udGhzIGludG8gc2Vhc29ucyBpbnN0ZWFkLCBmaWx0ZXJlZCBvdXQgdGhlIHllYXIgMjAyMSBkYXRhLCBhbmQgY3JlYXRlZCBhIGNvbHVtbiBmb3IgKmlkKiB0byBoZWxwIHdpdGggam9pbmluZyB0aGUgdGFibGVzLiAKCmBgYHtyLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0Kc3RyZXNzb3IgPSBzdHJlc3NvciAlPiUgCiAgbXV0YXRlKHNlYXNvbiA9IGNhc2Vfd2hlbigKICAgIG1vbnRocyAlaW4lIGMoJ0phbnVhcnktTWFyY2gnLCAnT2N0b2Jlci1EZWNlbWJlcicpIH4gJ1dpbnRlcicsIAogICAgbW9udGhzICVpbiUgYygnQXByaWwtSnVuZScsICdKdWx5LVNlcHRlbWJlcicpIH4gJ1N1bW1lcicKICApKSAlPiUgCiAgZmlsdGVyKHllYXIgIT0gMjAyMSkgJT4lIAogIG11dGF0ZShpZCA9IHJvd19udW1iZXIoKSkgJT4lIAogIHNlbGVjdCh5ZWFyLCBzZWFzb24sIHN0YXRlLCBzdHJlc3Nvciwgc3RyZXNzX3BjdCkKCmhlYWQoc3RyZXNzb3IpCmBgYAoKCgojIFByZWxpbWluYXJ5IEV4cGxvcmF0b3J5IEFuYWx5c2lzCgpOb3cgdGhhdCB3ZSBoYXZlIG91ciBkYXRhIGluIGEgd2F5IHRoYXQgaXMgZWFzaWVyIHRvIG1hbmFnZSwgd2UgY2FuIGV4cGxvcmUgaXQgYSBiaXQgbW9yZS4gCgpUaGUgZ3JhcGggYmVsb3cgc2hvd3MgdGhlIGF2ZXJhZ2UgcGVyY2VudGFnZSBvZiBjb2xvbmllcyBsb3N0IGluIHRoZSBVUyBkdXJpbmcgdGhlIHdpbnRlciBhbmQgc3VtbWVyIHNlYXNvbnMgYnkgeWVhci4gCmBgYHtyLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0KY29sb255ICU+JSAKICBncm91cF9ieSh5ZWFyLCBzZWFzb24pICU+JSAKICBzdW1tYXJpemUoYXZnX2NvbG9ueV9sb3N0X3BjdCA9IG1lYW4oY29sb255X2xvc3RfcGN0LCBuYS5ybSA9IFRSVUUpKSAlPiUgCiAgZ2dwbG90KGFlcyh4ID0geWVhciwgeSA9IGF2Z19jb2xvbnlfbG9zdF9wY3QsIGZpbGwgPSBzZWFzb24pKSArIGdlb21fY29sKHBvc2l0aW9uID0gImRvZGdlIikgKyBsYWJzKHggPSAiIiwgeSA9ICJBdmVyYWdlIENvbG9uaWVzIExvc3QgKCUpIiwgZmlsbCA9ICIiLCB0aXRsZSA9ICJBdmVyYWdlIFBlcmNlbnRhZ2Ugb2YgVG90YWwgQ29sb25pZXMgTG9zdCBpbiB0aGUgVS5TLiIsIHN1YnRpdGxlID0gIkZyb20gMjAxNS0yMDIwIikgKyB0aGVtZV9taW5pbWFsKCkKYGBgCgpXZSBjYW4gc2VlIHRoYXQgaW4gdGhlIHdpbnRlciBtb250aHMsIHRoZXJlIGlzIGEgaGlnaGVyIGF2ZXJhZ2UgY29sb25pZXMgbG9zdCBwZXJjZW50YWdlIGZvciBlYWNoIHllYXIgKDIwMTUtMjAyMCkuIFRoaXMgbWF5IGJlIGJlY2F1c2UgYmVlIGNvbG9uaWVzIGhhdmUgYSBoYXJkIHRpbWUgc3RheWluZyBhbGl2ZSBkdXJpbmcgY29sZGVyIG1vbnRocywgYnV0IGl0J3MgaW50ZXJlc3RpbmcgdG8gc2VlIGhvdyB0aGUgcGVyY2VudGFnZSBvbiBjb2xvbmllcyBsb3N0IGR1cmluZyB0aGUgd2ludGVyIG1vbnRocyBoYXMgaW5jcmVhc2VkIGR1cmluZyAyMDE2LTIwMTguIFRoZXJlIG1heSBiZSBvdGhlciByZWFzb25zIGFzIHRvIHdoeSB0aGUgYXZlcmFnZSBjb2xvbmllcyBsb3N0IHBlcmNlbnRhZ2UgaXMgYXQgaXRzIHBlYWsgaW4gMjAxOC4gCgoKCgpUaGUgZ3JhcGggYmVsb3cgZXhwbG9yZXMgdGhlIHN0cmVzc29yIHRhYmxlLiBJdCBzaG93cyB0aGUgYXZlcmFnZSBzdHJlc3MgcGVyY2VudGFnZSBmb3IgZWFjaCBzdHJlc3NvciBmcm9tIHRoZSB5ZWFycyAyMDE1IC0yMDIwLiAKYGBge3IsIHdhcm5pbmcgPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFfQpuYS5vbWl0KHN0cmVzc29yKSAlPiUgCiAgZ3JvdXBfYnkoeWVhciwgc3RyZXNzb3IpICU+JSAKICBzdW1tYXJpemUoc3RyZXNzX3BjdF9hdmcgPSBtZWFuKHN0cmVzc19wY3QpKSAlPiUgCiAgZ2dwbG90KGFlcyh4ID0geWVhciwgeSA9IHN0cmVzc19wY3RfYXZnLCBjb2xvciA9IHN0cmVzc29yKSkgKyBnZW9tX3BvaW50KCkgKyBnZW9tX3Ntb290aCgpICsgbGFicyh4ID0gIiIsIHkgPSAiQXZlcmFnZSBTdHJlc3MgKCUpIiwgY29sb3IgPSAiU3RyZXNzb3IiLCB0aXRsZSA9ICJBdmVyYWdlIFN0cmVzcyBQZXJjZW50YWdlIGZvciBFYWNoIFN0cmVzc29yIiwgc3VidGl0bGUgPSAiRnJvbSAyMDE1IHRvIDIwMjAiKQpgYGAKCldlIGNhbiBzZWUgdGhhdCB0aGUgc3RyZXNzb3IgVmFycm9hIE1pdGVzLCBoYXMgdGhlIGhpZ2hlc3QgYXZlcmFnZSBzdHJlc3MgcGVyY2VudGFnZSB0aHJvdWdob3V0IHRoZSB5ZWFycywgYnkgYSBsb3Qgd2hlbiBjb21wYXJlZCB0byBvdGhlciBzdHJlc3NvcnMuIFRoaXMgbWVhbnMgdGhhdCB0aGUgVmFycm9hIG1pdGVzIGlzIHRoZSBzdHJlc3NvciB0aGF0IGhhcyB0aGUgaGlnaGVzdCBwZXJjZW50IG9mIGNvbG9uaWVzIGFmZmVjdGVkIGJ5IGl0LiBBZ2FpbiwgaW4gMjAxOCwgdGhlcmUgaXMgYSBwZWFrIGluIGEgaGlnaCBzdHJlc3MgcGVyY2VudGFnZSBmb3IgbW9zdCBzdHJlc3NvcnMsIGluY2x1ZGluZyB0aGUgVmFycm9hIG1pdGVzLiAKCgoKIyBKb2luaW5nIFRhYmxlcyAKCk5vdyB0aGF0IHdlJ3ZlIHNlZSB0d28gY2F1c2VzIG9mIGNvbG9ueSBsb3NzIHNlcGFyYXRlbHksIGxldCdzIGxvb2sgYXQgdGhlbSB0b2dldGhlci4gVG8gZG8gdGhpcywgd2UgY2FuIG1lcmdlIHRoZSB0d28gdGFibGVzIHRvZ2V0aGVyLgoKCkZpcnN0LCB3ZSBjYW4gZml4IHRoZSBzdHJlc3NvciB0YWJsZSB0byBsb29rIGF0IG9ubHkgdGhlIFZhcnJvYSBtaXRlcyBzdHJlc3MgcGVyY2VudCBieSBzdGF0ZS4gCmBgYHtyLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0Kc3RyZXNzb3Jfdm0gPSBzdHJlc3NvciU+JSAKICBmaWx0ZXIoc3RyZXNzb3IgPT0gIlZhcnJvYSBtaXRlcyIpICU+JSAKICBhcnJhbmdlKHllYXIpICU+JSAKICBtdXRhdGUoaWQgPSByb3dfbnVtYmVyKCkpICU+JSAKICBzZWxlY3QoaWQsIHllYXIsIHNlYXNvbiwgc3RhdGUsIHN0cmVzc29yLCBzdHJlc3NfcGN0KQoKaGVhZChzdHJlc3Nvcl92bSkKYGBgCgpOb3cgdGhhdCB0aGUgc3RyZXNzb3IgdGFibGUgaXMgZmlsdGVyZWQgdG8gb25seSBoYXZlIHRoZSBWYXJyb2EgbWl0ZXMgc3RyZXNzIHBlcmNlbnRhZ2UsIHdlIGNhbiBqb2luIGl0IHRvIHRoZSBjb2xvbnkgdGFibGUuIApgYGB7ciwgd2FybmluZyA9IEZBTFNFLCBtZXNzYWdlID0gRkFMU0V9CmNvbG9ueUpvaW5lZFN0cmVzc29yVk0gPSBjb2xvbnkgJT4lIAogIGlubmVyX2pvaW4oc3RyZXNzb3Jfdm0sIGJ5ID0gYygiaWQiLCAieWVhciIsICJzZWFzb24iLCAic3RhdGUiKSkKCmhlYWQoY29sb255Sm9pbmVkU3RyZXNzb3JWTSkKYGBgCgojIEV4cGxvcmF0b3J5IERhdGEgQW5hbHlzaXMKCldpdGggdGhpcyBuZXcgZGF0YSwgd2UgY2FuIGV4cGxvcmUgdGhlIHJlbGF0aW9uc2hpcHMgYmV0d2VlbiBzZWFzb24gYW5kIHN0cmVzcyBwZXJjZW50YWdlIG9mIFZhbG9ycm9hIG1pdGVzLCB3aXRoIGNvbG9ueSBsb3N0IHBlcmNlbnRhZ2UuCgoKVGhlIGdyYXBoIGJlbG93IHNob3dzIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiB0aGUgcGVyY2VudGFnZSBvZiBjb2xvbmllcyBhZmZlY3RlZCBieSBWYXJyb2EgbWl0ZXMgYW5kIHRoZSBwZXJjZW50YWdlIG9mIHRvdGFsIGNvbG9uaWVzIGxvc3QgaW4gdGhlIFUuUy4gTW9yZSBzcGVjaWZpY2FsbHksIGl0IGxvb2tzIGF0IHRoZSByZWxhdGlvbnNoaXAgd2hlbiB0YWtpbmcgdGhlIHNlYXNvbiBpbnRvICBhY2NvdW50LiAKYGBge3IsIHdhcm5pbmcgPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFfQpjb2xvbnlKb2luZWRTdHJlc3NvclZNICU+JSAKICBnZ3Bsb3QoYWVzKHggPSBzdHJlc3NfcGN0LCB5ID0gY29sb255X2xvc3RfcGN0LCBjb2xvciA9IHNlYXNvbikpICsgZ2VvbV9wb2ludCgpICsgZ2VvbV9zbW9vdGgobWV0aG9kID0gImxtIiwgc2UgPSBGQUxTRSkgKyBsYWJzKHggPSAiQ29sb25pZXMgQWZmZWN0ZWQgYnkgdmFycm9hIG1pdGVzICglKSIsIHkgPSAiVG90YWwgQ29sb25pZXMgTG9zdCAoJSkiLCBjb2xvciA9ICJTZWFzb24iLCB0aXRsZSA9ICJQZXJjZW50YWdlIG9mIFRvdGFsIENvbG9uaWVzIExvc3QgQmFzZWQgb24gUGVyY2VudGFnZSBvZiBDb2xvbmllcyBBZmZlY3RlZCBieSBWYXJyb2EgTWl0ZXMgIiwgc3VidGl0bGUgPSAiQnkgc2Vhc29uIikKYGBgClZpc3VhbGx5LCB3ZSBjYW4gc2VlIHRoYXQgdGhlcmUgYXBwZWFycyB0byBiZSBhIGdlbmVyYWwgcG9zaXRpdmUgY29ycmVsYXRpb24gYmV0d2VlbiBzdHJlc3MgcGN0IGFuZCBjb2xvbnkgbG9zdCBwZXJjZW50LCB3aXRoIHdpbnRlciBzZWFzb25zIGhhdmluZyBncmVhdGVyIGNvbG9ueSBsb3N0IHBlcmNlbnRhZ2UuIEJvdGggaGlnaGVyIHN0cmVzcyBwZXJjZW50cyBvZiBWYXJyb2EgbWl0ZXMgYW5kIHRoZSB3aW50ZXIgc2Vhc29uIHRvZ2V0aGVyLCBoYXZlIHRoZSBoaWdoZXN0IGNvbG9ueSBsb3N0IHBlcmNlbnRhZ2UuCgoKTGV0J3MgaW52ZXN0aWdhdGUgdGhlc2UgcmVsYXRpb25zaGlwcyB0aHJvdWdoIG1vZGVsaW5nIG5vdyEKCgojIFByZWRpY3RpdmUgTW9kZWxpbmcgCgojIyBMaW5lYXIgUmVncmVzc2lvbiBNb2RlbAoKKipUYXJnZXQqKjogUGVyY2VudCBvZiB0b3RhbCBjb2xvbmllcyBsb3N0ICgqY29sb255X2xvc3RfcGN0KikKCioqRmVhdHVyZSoqIFBlcmNlbnQgb2YgY29sb25pZXMgYWZmZWN0ZWQgYnkgVmFycm9hIG1pdGVzICgqc3RyZXNzX3BjdCopIGFuZCBzZWFzb24gKCpzZWFzb24qKQoKCkxldCdzIGJ1aWxkIGEgbXVsdGlwbGUgcmVncmVzc2lvbiBtb2RlbCB0aGF0IHByZWRpY3RzICpjb2xvbnlfbG9zdF9wY3QqIHdoaWxlIGNvbnNpZGVyaW5nIG91ciBmZWF0dXJlcyBvZiAqc3RyZXNzX3BjdCogYW5kICpzZWFzb24qICEKCmBgYHtyLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0Kc2V0LnNlZWQoMTQpCiMxLiBEYXRhIFNwbGl0dGluZwpjb2xvbnlKb2luZWRTdHJlc3NvclZNX3NwbGl0ID0gaW5pdGlhbF9zcGxpdChjb2xvbnlKb2luZWRTdHJlc3NvclZNLCBwcm9wID0gMC43KQoKY29sb255Sm9pbmVkU3RyZXNzb3JWTV90cmFpbiA9IHRyYWluaW5nKGNvbG9ueUpvaW5lZFN0cmVzc29yVk1fc3BsaXQpCmNvbG9ueUpvaW5lZFN0cmVzc29yVk1fdGVzdCA9IHRlc3RpbmcoY29sb255Sm9pbmVkU3RyZXNzb3JWTV9zcGxpdCkKCiMyLiBGaXQgbW9kZWwKbG1fZml0X3RyYWluID0gCiAgbGluZWFyX3JlZygpICU+JSAKICBmaXQoY29sb255X2xvc3RfcGN0IH4gc3RyZXNzX3BjdCArIHNlYXNvbiwgZGF0YSA9IGNvbG9ueUpvaW5lZFN0cmVzc29yVk1fdHJhaW4pCmxtX2ZpdF90cmFpbgoKIzMuIEdlbmVyYXRlIFByZWRpY3Rpb25zCmxtX3ByZWQgPSBsbV9maXRfdHJhaW4gJT4lIAogIHByZWRpY3QobmV3X2RhdGEgPSBjb2xvbnlKb2luZWRTdHJlc3NvclZNX3Rlc3QpCgojNC4gUHJlZGljdGlvbiBNZXRyaWMgKFJNU0UpCmNvbG9ueUpvaW5lZFN0cmVzc29yVk1fdGVzdCAlPiUgCiAgbXV0YXRlKGNvbG9ueV9sb3N0X3BjdF9QcmVkID0gbG1fcHJlZCQucHJlZCkgJT4lIAogIHJtc2UoZXN0aW1hdGUgPSBjb2xvbnlfbG9zdF9wY3RfUHJlZCwgdHJ1dGggPSBjb2xvbnlfbG9zdF9wY3QpCmBgYAoKV2UgY3JlYXRlZCBhIG11bHRpcGxlIHJlZ3Jlc3Npb24gbW9kZWw6IGNvbG9ueV9sb3N0X3BjdF9oYXQgPSAwLjEwNjIoc3RyZXNzX3BjdCkgKyA0LjExODQoc2Vhc29uV2ludGVyKQoKVGhlIFJNU0UgZm9yIHRoZSBtdWx0aXBsZSByZWdyZXNzaW9uIG1vZGVsIHdhcyA2LjMyOTQzNC4gVGhpcyBhIHZlcnkgbG93IFJNU0UuIFRoaXMgbWVhbnMgdGhhdCB0aGUgbXVsdGlwbGUgcmVncmVzc2lvbiBtb2RlbCBmaXRzIHRoZSBkYXRhc2V0IHZlcnkgd2VsbCEgQnV0IHRoaXMgbnVtYmVyIHNob3VsZCBiZSByZWNpZXZlZCB3aXRoIGNhdXRpb24gYmVjYXVzZSB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gb3VyIGZlYXR1cmVzICgqc2Vhc29uKiBhbmQgKnN0cmVzc19wY3QqKSBhbmQgdGFyZ2V0ICgqY29sb255X2xvc3RfcGN0KikgbWF5IG5vdCBhY3R1YWxseSBiZSBhIGxpbmVhciByZWxhdGlvbnNoaXAuIAoKTGV0J3MgbG9vayBhdCB0aGUgcmVsYXRpb25zaGlwIGFnYWluIHVzaW5nIGEgZGlmZmVyZW50IG1vZGVsIQoKCgojIyBEZWNpc2lvbiBUcmVlCgpUaGUgcHJlZGljdGl2ZSBtb2RlbGluZyBvZiBsaW5lYXIgcmVncmVzc2lvbiB0aGF0IHdlIHVzZWQsIG1heSBub3QgYmUgdGhlIGJlc3QgbW9kZWwuIE91ciBkYXRhIG1heSBuZWVkIGEgbm9uLWxpbmVhciBtb2RlbC4KClRvIGNyZWF0ZSBhIG1vcmUgYWNjdXJhdGUgcHJlZGljdGl2ZSBtb2RlbCwgd2UgY2FuIHVzZSBhIGRlY2lzaW9uIHRyZWUgdG8gZW1wbG95IHRoZSBDQVJUIChjbGFzc2lmaWNhdGlvbiBhbmQgcmVncmVzc2lvbiB0cmVlKSBhbGdvcml0aG0gOiAKCmBgYHtyLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0Kc2V0LnNlZWQoMTQpIAojMS4gRGF0YSBTcGxpdHRpbmcKY29sb255Sm9pbmVkU3RyZXNzb3JWTV9zcGxpdCA9IGluaXRpYWxfc3BsaXQoY29sb255Sm9pbmVkU3RyZXNzb3JWTSwgcHJvcCA9IDAuNykKCmNvbG9ueUpvaW5lZFN0cmVzc29yVk1fdHJhaW4gPSB0cmFpbmluZyhjb2xvbnlKb2luZWRTdHJlc3NvclZNX3NwbGl0KQpjb2xvbnlKb2luZWRTdHJlc3NvclZNX3Rlc3QgPSB0ZXN0aW5nKGNvbG9ueUpvaW5lZFN0cmVzc29yVk1fc3BsaXQpCgojMi4gRml0IG1vZGVsCnRyZWVfZml0ID0gCiAgZGVjaXNpb25fdHJlZSgpICU+JQogIHNldF9tb2RlKG1vZGUgPSAicmVncmVzc2lvbiIpICU+JQogIGZpdChjb2xvbnlfbG9zdF9wY3QgfiBzdHJlc3NfcGN0ICsgc2Vhc29uLCAKICAgIGRhdGEgPSBjb2xvbnlKb2luZWRTdHJlc3NvclZNX3RyYWluKQoKcnBhcnQucGxvdCh0cmVlX2ZpdCRmaXQsIHJvdW5kaW50ID0gRkFMU0UpCgojMy4gR2VuZXJhdGUgUHJlZGljdGlvbnMKdHJlZV9wcmVkID0gdHJlZV9maXQgJT4lCiAgcHJlZGljdChuZXdfZGF0YSA9IGNvbG9ueUpvaW5lZFN0cmVzc29yVk1fdGVzdCkKCiM0LiBQcmVkaWN0aW9uIE1ldHJpYyAoUk1TRSkKY29sb255Sm9pbmVkU3RyZXNzb3JWTV90ZXN0ICU+JQogIG11dGF0ZShjb2xvbnlfbG9zdF9wY3RfUHJlZCA9IHRyZWVfcHJlZCQucHJlZCkgJT4lCiAgcm1zZSh0cnV0aCA9IGNvbG9ueV9sb3N0X3BjdCwgZXN0aW1hdGUgPSBjb2xvbnlfbG9zdF9wY3RfUHJlZCkKCmBgYAoKQnkgdmlld2luZyB0aGUgQ0FSVCBvdXRwdXQsIHdlIGNhbiBzZWUgd2hlcmUgdGhlIHZhcmlhYmxlIHNlYXNvbiBpcyBzcGxpdC4gQW5kIGlmIHdlIGZvbGxvdyBhbG9uZywgd2UgY2FuIHNlZSB0aGF0IHRoZSB3aW50ZXIgbW9udGhzIGhhdmUgbW9yZSBoaWdoZXIgc3RyZXNzIHBlcmNlbnRhZ2VzLiAKClRoZSBSTVNFIGZvciB0aGUgQ0FSVCBtb2RlbCB3aXRoIG11bHRpcGxlIGZlYXR1cmVzIHdhcyA2LjU5Nzg2Ny4gTWVhbndoaWxlIHRoZSBSTVNFIGZvciB0aGUgbXVsdGlwbGUgcmVncmVzc2lvbiBtb2RlbCB3YXMgNi4zMjk0MzQuIAoKCgoKIyBDb25jbHVzaW9uIAoKV2hpbGUgIHRoZSBSTVNFIGZvciB0aGUgbXVsdGlwbGUgcmVncmVzc2lvbiBtb2RlbCB3YXMgdGVjaG5pY2FsbHkgc21hbGxlciwgdGhlIFJNU0UgZm9yIHRoZSBDQVJUIG1vZGVsIHdhcyBub3QgY29tcGxldGVseSBvZmYuIFRoZXkgd2VyZSByZWxhdGl2ZWx5IGNsb3NlIGluIHNpemUsIHdoaWNoIG1lYW5zIHRoYXQgYm90aCBtb2RlbHMgdGhhdCB3ZSBjcmVhdGVkIHRvIHByZWRpY3QgdmFsdWVzIG9mICpjb2xvbnlfbG9zdF9wY3QqIHdlcmUgZ2VuZXJhbGx5IGdvb2QgYXQgZG9pbmcgc28uIE9mIGNvdXJzZSwgdGhlc2UgYXJlIG9ubHkgdHdvIHdheXMgb2YgcHJlZGljdGluZyB2YWx1ZXMsIHNvIHRoZXJlIG1heSBiZSBvdGhlciBtb2RlbHMgdGhhdCBtYXkgZG8gYSBtb3JlIGFjY3VyYXRlIGpvYi4gQnV0IGluIGV4cGxvcmluZyBib3RoIHRoZSBkYXRhIHZpc3VhbGx5IGFuZCB0aHJvdWdoIG1vZGVsaW5nLCB3ZSBjYW4gc2VlIHRoZSBlZmZlY3QgdGhhdCB0aGUgd2ludGVyIHNlYXNvbiwgYW5kIHZhcnJvYSBtaXRlcyBoYXZlIG9uIHRoZSBwZXJjZW50YWdlIG9mIGNvbG9ueSBsb3N0LiAKClRoZSBkYXRhc2V0IHdlIGV4cGxvcmVkIGRpZCBub3QgY29udHJhaW4gbWFueSBkaWZmZXJlbnQgZmFjdG9ycy4gVGhlcmUgY291bGQgaGF2ZSBiZWVuIG91dHNpZGUgZmFjdG9ycyB0aGF0IGFmZmVjdCB0aGUgcGVyY2VudGFnZSBvZiBjb2xvbnkgbG9zdCBpbiB0aGUgVVMuIFBlcmhhcHMgdGhlIHNlYXNvbiBhbmQgc3RyZXNzb3JzIGFyZSBub3QgdGhlIG9ubHkgcmVhc29uIHdoeSBob25leWJlZSBjb2xvbmllcyBhcmUgYmVpbmcgdGhyZWF0ZW5lZC4gSXQgaXMgYWxzbyBpbXBvcnRhbnQgdG8gbm90ZSB0aGF0IHRoZXJlIG1heSBiZSBhIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSB2YXJyb2EgbWl0ZXMgYW5kIHRoZSBzZWFzb24gd2hpY2ggbWF5IGhhdmUgaGFkIGFuIGVmZmVjdCBvbiBvdXIgYWJpbGl0eSB0byBwZXJmb3JtIGFjY3VyYXRlIGFuZCB1c2VmdWwgcHJlZGljdGl2ZSBtb2RlbGluZy4gCgpUaGVyZSBhcmUgbWFueSBvdGhlciBmYWN0b3JzIHRoYXQgd2UgbXVzdCBrZWVwIGxvb2tpbmcgYXQgdG8gbWFrZSBzdXJlIHRoYXQgb3VyIGhvbmV5YmVlIGNvbG9uaWVzIHRocml2ZSBhZ2Fpbi4gCgoK